August 2021 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Della E. McMillan, Kwasi Ampofo, Annie Dela Akanko, and Lynn Hurtak. EVALUATION Council for Scientific and Industrial Research–Savanna Agricultural Research Institute Technical and Financial Support Project Final Performance Evaluation Report COUNCIL FOR SCIENTIFIC AND INDUSTRIAL RESEARCH– SAVANNA AGRICULTURAL RESEARCH INSTITUTE TECHNICAL AND FINANCIAL PROJECT FINAL PERFORMANCE EVALUATION REPORT August 3, 2021 Contracted under 641-A18-FY14- IL#03 Prepared by: Della E. McMillan, Kwasi Ampofo, Annie Dela Akanko, and Lynn Hurtak Submitted by: USAID/Ghana Monitoring, Evaluation, and Technical Support Services II (METSS II) Project Photo credit: Farmer participation in a CSIR-SARI-sponsored varietal selection exercise for a rice variety in 2017. Photo by Wilhelm Nomu Kutah. Image used with the permission of CSIR-SARI. DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV CONTENTS EXECUTIVE SUMMARY............................................................................................................... I 1.0. EVALUATION OVERVIEW ............................................................................................ 1 1.1. Evaluation Purpose ........................................................................................................................................ 1 1.2. Evaluation Questions .................................................................................................................................... 1 2.0. PROJECT BACKGROUND .............................................................................................. 2 2.1. Mechanism Description ................................................................................................................................ 2 3.0. EVALUATION METHODS AND LIMITATIONS ......................................................... 8 3.1. Evaluation Methodology ............................................................................................................................... 8 3.2 Limitations ..................................................................................................................................................... 11 4.0. FINDINGS AND CONCLUSIONS ............................................................................... 13 4.1. EQ 1: To what extent has the CSIR-SARI Technical and Financial Support Project achieved its intended goal and objectives as defined by the results framework? ............................................... 13 4.2. EQ 2: Is there evidence of improved organizational development and a strengthened institution of SARI (increased research capacity) and has this resulted in improved welfare of smallholder farmers in Northern Ghana? ............................................................................................. 27 4.3. EQ 3: What unintended contributions, results, and/or outcomes have the project approach and activities achieved relative to transforming CSIR-SARI into a center of excellence for research and development in the savannah agro-ecological zone? ................................................. 31 4.4. EQ 4.A) To what extent has the project addressed gender issues in relation to capacity￾building in research and development, and has this had an impact on gender mainstreaming in agricultural research? 4.B) What are the lessons learned and best practices, and from which stakeholders or beneficiaries, in achieving results? ............................................................................. 34 4.5. EQ 5.A) To what extent are the project results likely to continue after the United States Government (USG) support ends? 5.B) What organizational development support and institutional strengthening assistance would CSIR-SARI need for it to improve its ability to sustainably benefit businesses and farmers in Ghana? ........................................................................ 45 ANNEXES .................................................................................................................................... 52 Annex 1. Evaluation Statement of Work............................................................................................................ 53 Annex II. Performance Indicator Tracking Table (PITT) (FY 2015–2019) ................................................ 67 Annex III. Fieldwork ................................................................................................................................................ 75 Annex IV. Data Analysis ......................................................................................................................................... 89 Annex V. Bibliography, Additional Tables, and Case Studies ..................................................................... 165 CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV ACRONYMS ADVANCE Agricultural Development and Value Chain Enhancement AEA Agriculture Extension Agent AGRA Alliance for a Green Revolution in Africa AgTILs AgTech Innovation Labs AMEP Activity Monitoring and Evaluation Plan AOR Agreement Officer’s Representative ATT Agriculture Technology Transfer BMS Breeding Management System C Core CDCS Country Development Cooperation Strategy CEO Chief Executive Officer CGIAR Consultative Group on International Agricultural Research CIMMYT International Maize and Wheat Improvement Center COE Core of Excellence COVID-19 Coronavirus Disease 2019 CRI Crops Research Institute CSIR Council for Scientific and Industrial Research DO Development Objective EG Economic Growth EGS Early Generation Seed EQ Evaluation Question ET Evaluation Team FAW Fall Army Worm FGD Focus-Group Discussion FY Fiscal Year G2G Government to Government GARNET Ghanaian Academic and Research Network GIP Gender Integration Plan GIS Geographic Information System GoG Government of Ghana Ha Hectare ICT Information and Communication Technology IFDC International Fertilizer Development Center IGF Internally Generated Funds IMC Internal Management Committee IR Intermediate Result CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV ISFM Integrated Soil Fertility Management ISU Iowa State University IT Information Technology IWAD Integrated Water Management & Agricultural Development Ghana Limited KII Key Informant Interview KMS Knowledge Management System LOA Life of Activity M&E Monitoring and Evaluation MAG Modernizing Agriculture in Ghana Mbps Megabits Per Second METSS Monitoring, Evaluation, and Technical Support Services MIS Management Information System MoFA Ministry of Food and Agriculture MOU Memorandum of Understanding MS Master of Science MT Metric ton N/A Not Applicable NARS National Agricultural Research Systems NASTAG National Seed Trade Association of Ghana NC Non-core NGO Non-Governmental Organization NREN National Research and Education Network PFJ Planting for Food and Jobs PhD Doctor of Philosophy PIL Project Implementation Letter PIRS Performance Indicator Reference Sheet PITT Performance Indicator Tracking Table PPP Public-Private Partnership PPRSD Plant Protection and Regulatory Services Directorate PVO Private Voluntary Organization RELC Research Extension Linkage Committee RFI Request for Information SANREM Sustainable Agriculture and Natural Resource Management SARI Savanna Agricultural Research Institute SEEDPAG Northern Region Seed Producer Association of Ghana SIL Soybean Innovation Lab SOW Statement of Work SP Sub-purpose CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV SSA Sub-Saharan Africa UDS University for Development Studies UIUC University of Illinois, Urbana Champaign U.S. United States USAID United States Agency for International Development USAID/Ghana United States Agency for International Development Ghana Mission USG United States Government VPN Virtual Private Network I | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV EXECUTIVE SUMMARY EVALUATION PURPOSE The purpose of this final performance evaluation is to assess the extent to which the goal, purpose, sub￾purposes, and outputs1 of the United States Agency for International Development Ghana Mission’s (USAID/Ghana) Council for Scientific and Industrial Research–Savanna Agricultural Research Institute (CSIR-SARI) Technical and Financial Support Project have been achieved, and how the implementation of the project interventions contributed to the 2013–2019 USAID/Ghana Country Development Cooperation Strategy (CDCS) Development Objective 2 (DO 2), “Sustainable and broadly shared economic growth,” and Intermediate Results (IRs) 2.1-2.4.2 The evaluation is expected to provide USAID and CSIR-SARI with data on outcomes, best practices, and lessons learned to inform future programming. PROJECT BACKGROUND In its most recent CDCS (2013–2019), USAID/Ghana committed itself to supporting, “sustainable and broadly shared economic growth,” by investing in a series of agricultural development projects in northern Ghana. The CSIR-SARI project was expected to contribute to this process by strengthening the principal agricultural research institution in northern Ghana—CSIR-SARI—to support these projects. The stated purpose of the project is to, “strengthen the capacity of SARI to deliver on its core mandate,” which is to, “provide small-scale farmers in the three regions of northern Ghana with appropriate innovations/options/ technologies to increase their food production based on a sustainable production system, which maintains and/or increases soil fertility.”3 The project was designed and executed by CSIR-SARI through a five-year government-to-government (G2G) grant from USAID/Ghana to achieve three overlapping sub-purposes and ten outputs identified in a 2012 baseline assessment funded by USAID/Ghana through Africa Lead:4  Sub-Purpose 1: Capacity development for agricultural research in northern Ghana;  Sub-Purpose 2: Agricultural technologies development and dissemination; and  Sub-Purpose 3: Market and client-oriented research approach including coordination, project management, communication, and monitoring and evaluation (M&E). The same activities were expected to contribute to the higher level goal to, “transform CSIR-SARI into a Center of Excellence for agricultural research and development in the savannah agro-ecological zones in Sub￾Saharan Africa,” by, “enabling SARI to integrate into the emerging global agricultural research system to keep abreast of rapid advances in scientific knowledge, and to improve the cost-effectiveness of technology generation by capturing spill-ins and through collaborative research efforts.”5 EVALUATION QUESTIONS, DESIGN, METHODS, AND LIMITATIONS 1 This project did not include objectives but purposes, sub-purposes, and outputs (see Figure 1 in the main body of this report). 2 USAID/Ghana. 2014. Program Description. Support to the Council for Scientific and Industrial Research-Savanna Agricultural Research Institute (CSIR-SARI) (January 14, 2015–January 14, 2020). Accra: Economic Growth (EG) Office, USAID/Ghana. Pg. 14. 3 Stated purpose of the project. Ibid. The original three regions were Northern, Upper East, and Upper West. This same area is now organized into five regions: Northern, Savanna, North East, Upper East, and Upper West. 4 John Nene-Osom Azu and Kwesi Opoku-Debrah. 2012. Assessment of the Savanna Agricultural Research Institute (SARI) Nyankpala, Northern Region, Ghana. Washington, DC: Africa Lead for USAID/Ghana (July 12, 2012). 5 USAID/Ghana 2014. Program Description. Op. cit. Pg. 13. II | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV The evaluation team (ET) used a mixed-methods approach to respond to the five evaluation questions (EQs), including: 1) a document review; 2) key informant interviews (KIIs) and focus-group discussions (FGDs) with all six stakeholder groups;6 and 3) two post-interview online questionnaires targeted to three of the six stakeholder groups that worked closely with the project (the SARI staff, key government agencies the project worked with, and academic and research partners and other donor￾funded projects) to get anonymous feedback to the key questions. The ET interviewed 107 people from six stakeholder groups in 50 FGDs and KIIs and received 45 responses to the online questionnaire. After collecting the data, the ET engaged in an intensive review of the themes emerging from the interviews for each of the five EQs. There were several limitations the ET attempted to address in the design and execution of the study. 1. The principal limitation was assembling the background documentation and a sampling frame for the FGDs and KIIs since the project did not leave behind either a stakeholder database or core project documentation file when it closed out in March 2020. To manage this limitation, the evaluation focal person collaborated with a four-person evaluation working group7 the CSIR￾SARI director designated to support the evaluators (since the project was closed and there were no more current project staff) to reconstruct a list of core and non-core individuals who benefitted from the project and identify the missing documentation. Once this sampling frame was constructed, the ET selected a representative sample of core CSIR-SARI staff and other stakeholders for interviewing. As key documents were identified, they were listed in a master bibliography and filed in Google Drive folders. 2. Secondly, the project never developed Performance Indicator Reference Sheets (PIRSs) for the custom indicators in the project M&E plan, and therefore, never reported on the custom indicators that formed the basis for judging the performance of the ten outputs. Because of this, the only consistent reporting was on the six USAID/Ghana standard indicators. To address this limitation, the ET worked with the evaluation focal person and working group to develop an updated Performance Indicator Tracking Table (PITT) that included the missing information on the custom indicators. 3. A third limitation was the logistical challenge of conducting interviews remotely due to the Coronavirus Disease 2019 (COVID-19) pandemic, which precluded in-person interviews. The ET tried to mitigate the impact of this limitation by: 1) splitting the interviewees into groups to enable effective facilitation of discussions via Zoom; 2) conducting most interviews through a combination of Zoom and local telephone (for those with limited access to internet); and 3) including a fourth team member with the necessary language, communication, and organizational skills to organize this type of interview process. 4. A fourth limitation was the difficulty of attributing any result to the project alone since most results were expected to capitalize on: 1) the technological and capacity-building achievements of a previous generation of donor-funded project executed by the Alliance for a Green Revolution in Africa (AGRA) and other donor-funded projects (a.k.a. legacy8 projects that worked with CSIR-SARI before 2015); and 2) important synergies with the ongoing donor and Government of Ghana (GoG) projects with which it overlapped. The ET managed this issue by 6 The six stakeholder groups are: 1) CSIR-SARI project staff and oversight committees; 2) government agencies the project collaborated with; 3) academic and research partners and other donor-funded projects; 4) local partners (for activities they engaged in); 5) media/communication partners (for the activities they participated in); and 6) USAID/Ghana. 7 This working group consisted of three current staff and one retired staff who worked on the project. See Annex III.E.2 for a list of these individuals and their credentials. 8 Term often used in the interviews. III | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV asking the people in the FGDs and KIIs to describe how the project activity they were involved with (or discussing) related to earlier or ongoing donor or GoG-funded initiatives. 5. A fifth limitation was the difficulty of identifying and contacting the community-based stakeholders and the Ministry of Food and Agriculture (MoFA) Agricultural Extension Agents (AEAs) that participated in and benefitted from project-sponsored trainings, field days, and adaptive and demonstration field trials since the project never developed a stakeholder database with contact information. None of the strategies the ET co-identified (with USAID/Ghana and Monitoring, Evaluation, and Technical Support Services II [METSS II]) for mitigating this constraint were fully successful.9 FINDINGS AND CONCLUSIONS EQ 1: To what extent has the CSIR-SARI Technical and Financial Support Project achieved its intended goal and objectives as defined by the results framework? Findings Sub-purpose (SP) 1: Capacity development for agricultural research in northern Ghana One of the best quantitative indicators of this increased capacity—which capitalized on the capacities developed by other USAID and non-USAID funded legacy and ongoing projects—was the project’s dramatic overachievement of its original target for, “number of technologies and management practices, under field testing or made available for transfer” (336 vs. 80, 420 percent of target) (Annex II). Although 60 percent of CSIR-SARI staff and 70 percent of partner staff interviewed in FGDs/KIIs agreed or strongly agreed the project, “increased CSIR-SARI’s capacity to support for agricultural research in northern Ghana” (Annexes IV.A.1 and A.2), there was a cross-cutting frustration among the staff—confirmed in the project’s close-out report—with the project’s sub-par performance achieving the proposed management reforms under Output 1.1. Based on this triangulation of data, the ET concluded the project only partially achieved its expectations for this SP. Output 1.1. Core scientists trained and re-tooled. Internal records and reports confirm the project trained 134 core scientists, administrators, and technicians through 36 sponsored workshops and three exchange visits to international centers of excellence in Africa, the United States (U.S.), and India. While there is clear evidence these trainings helped improve the research output of CSIR-SARI, the project never linked the trainings to the development of a more broad-based capacity development plan for the institute or the priority value chains, which was a major bone of contention with both the CSIR-SARI staff,10 in large part because there was very little understanding of: 1) how the training priorities were identified; 2) who got trained and when; and 3) why some trainings staff thought were critical were not offered. In spite of this, the ET concluded there was ample quantitative and qualitative evidence the project contributed either directly (through formal training or exchange visits) or indirectly (through improved infrastructure and on-farm trials) to the core capacity of key staff associated with the three priority value chains, which was the project’s original expected result for this output. Output 1.2. Organizational quality and efficiency increased. In contrast, the ET agrees with both the close￾out report and the FGDs/KIIs that the project was unable to achieve the types of management and financial reforms that were the chief expected results for this output. This was attributed to several factors, including the senior CSIR-SARI top management’s gross misunderstanding of the project and the USAID/Ghana’s Program Implementation Letter (PIL) and the USAID Imprest system for transferring project funds, which contributed to the project never being able to create the type of stable, independent project management model needed to increase organizational quality and efficiency 9 The challenges and multiple courses of action taken to address them were explained to USAID/Ghana in the weekly and biweekly reports the ET sent to USAID/Ghana during fieldwork. 10 Only 21–42 percent of staff agreed or strongly agreed the project training increased the capacity of the institute. IV | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV (Activity 1.2.1). They were also never able to provide intended training or capacity-building for CSIR￾SARI’s management board and internal management committee (IMC) (Activity 1.2.3), nor develop the long- and short-term staff capacity development plan (Activity 1.2.7). Output 1.3. Infrastructure built and renovated. Between 2015 and March 2020, the project completed 14 infrastructure development activities (renovations or new construction) and updated CSIR-SARI’s weak internet connection (which was not in the original proposal) by building an unexpected collaboration with the Ghanaian Academic and Research Network (GARNET), increasing the connection on the main station/center from 2 megabits per second (Mbps) to 10/Mbps in 2018, and then, finally, to 155 Mbps in 2020. Although the total infrastructure projects completed was only 50 percent of the original target of 28 due to higher construction costs and shifts in program priorities, these activities occupied the same percentage of the budget as originally envisioned (Annex V.B.1 and V.B.2). The stakeholders’ chief infrastructure criticisms were: 1) most targeted investments for the research out-stations (including improving their internet infrastructure) were never executed; and 2) there was very little attention given to the issue of training staff in basic infrastructure maintenance and upkeep. Based on this triangulation of information, the ET concluded the project, by and large, achieved its original expected results for this output. SP 2: Agricultural technologies development and dissemination Seventy-seven (77) percent of the 26 partner staff interviewed in FGDs/KIIs and all of the community￾based stakeholders either agreed or strongly agreed the “project has increased CSIR-SARI’s support to the development and dissemination of improved agricultural technologies.” Two of the best quantitative indicators of the project’s impact on accelerating CSIR-SARI’s capacity for agricultural technologies development and dissemination are (Annex II): 1. The development and release of eight new climate-smart varieties with direct support from the project, as well as the development and release of another four varieties through a combination of support from this project and other donor-funded initiatives, bringing the total to 12 (133 percent of the original target of nine); and 2. The project’s dramatic (420 percent) overachievement of its original life of activity (LOA) target for new technologies, practices, and approaches under various phases of research, development, and uptake for the priority crops (80 targeted vs. 336 achieved11), in large part because it was very successful in achieving its original goal of capitalizing on a wide variety of new technologies developed under the previous generation of donor-funded projects (Annex II). Only 55 percent of the CSIR-SARI staff interviewed agreed or strongly agreed with this statement because they argued the impact could have been greater had the project been better managed. Although the ET agrees the impact could have been greater with more effective management, it concluded the project did achieve a critical mass of the results expected under this SP. Output 2.1. SARI’s seed operations modernized. Eighty-four (84) percent of the project’s partner staff interviewed agreed with the project’s final close-out report, which stated the project’s investments under Objective 1—combined with the full execution of all ten activities under Output 2.112—helped modernize CSIR-SARI’s seed operations. One of the best quantitative indicators of this was the 11 These are mainly accessions. Each accession is counted once. The breeding programs for maize, rice, soybean, and the other mandate crops of CSIR-SARI are always working with several accessions/germplasms at any point in time. Several accessions can be planted for observation on a small piece of land (Prince Etwire, email communication, March 16, 2022). 12 Staff reported this was due to liquidation problems, which delayed the transfer of funds for several critical activities. To address this issue, staff reported they strengthened their collaboration with other ongoing donor and GoG-funded projects and, “kept going.” V | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV institute’s successful overachievement of most of its annual targets for breeder seed and foundation seed production (from 26.1 metric tons [MT] in 2015 to 102.7 MT in FY 2017 —150 percent of target; to 127.4 MT in FY 2018—182 percent of target; and 108.1 MT in FY 2019—152 percent of target). This represented 212 percent of the revised LOA targets (Annex II).13 Even though most staff agreed the project helped CSIR-SARI improve its compliance with the Ghana Fertilizer and Seed Acts, only 35-38 percent of staff who responded to this question either agreed or strongly agreed because most seed operations were still rainfed with limited mechanization, and there was very limited private-sector investment in either seed processing or cold storage (none of which were in the original proposal). Based on this triangulated feedback, the ET concluded the project achieved most of the original expected results for this output. Output 2.2. Integrated Soil Fertility Management (ISFM) practices developed and disseminated. The project’s tracking data and final close-out report confirmed it was also very successful in achieving its original target (100 percent) for the development of three ISFM protocols for the three priority crops that included, “appropriate [e.g., evidence-based] application, proper weed and pest control, proper land management and drainage, harvest and post-harvest handling strategies as well as farm management,” that are now being diffused to key government and private-sector actors (e.g., seed companies, agro-dealers) (Annex II). Despite this evidence of success, only 16-21 percent of CSIR-SARI staff agreed the project helped develop improved soil fertility practice, and only 29-35 percent agreed it increased the dissemination of these practices because the dissemination was just starting when the project ended (confirmed by the project’s PITT data, Annex II). The percentage of respondents who agreed or strongly agreed with this statement was higher for project partners (48 percent for development and 64-71 percent for dissemination). Since there was never a clear target for scale up in the project plan or work plan and clear evidence (from the project report and interviews) of potential impact, the ET concluded the project achieved its expected results for this output. Output 2.3. Capacity of technical staff built. Based on an analysis of the list of reported trainings, there were 196 technicians trained in various areas critical to CSIR-SARI’s core mandate. Although most of the technicians interviewed reported some of the trainings were quite useful, only 19 percent of the 26 CSIR-SARI staff who responded to this question in the FGDs, and 42 percent of the 31 staff who responded to the online questionnaire, agreed or strongly agreed this training achieved its expected result because: 1) some of the other trainings needed were not offered; 2) there was no systematic plan or support for scaling up the training to other staff; 3) many of the technical trainings (like the molecular breeding tools training) that were supposed to have occurred in the first and second year of the project did not occur until the final three years; and 4) some of the technicians who got the training did not have access to the equipment needed to make use of the training. Based on this information—and the fact the project never linked the training to the development of a short- or medium-term training strategy as intended in the proposal—the ET concluded the project only partially achieved the original expected results. Output 2.4. Research Extension Linkage Committee (RELC) mechanism improved. After the first year, the project discontinued its support for the nine activities it was originally expected to support for the regional and district-level RELC mechanisms because a Canadian-funded project (Modernizing Agriculture in Ghana [MAG]) was funding these. Although CSIR-SARI did not provide any direct financial support to the RELCs after the first year or include the results from the RELC operations in their annual reports: 1) MoFA’s records indicate regional and district-level RELCs continued to meet regularly (Annex II); and 2) staff interviews confirmed many of the CSIR-SARI researchers and technicians continued to attend these regional and district-level meetings where they reported on the new 13 The original LOA target in the proposal was to increase the baseline production by 100 percent. The revised target, based on the sum of the annual targets, was 367.9 MT (Annex II). VI | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV technologies that their units were developing and field testing. Fifty-seven (57) percent of GoG and partner staff who responded to the online questionnaire agreed or strongly agreed, “the project has helped SARI to improve the RELC mechanism for agricultural technologies dissemination.” For this reason, the ET concluded that even through the project cut its support to the RELCs after the first year, it partially achieved its originally expected results for this output. SP 3: Market and client-oriented research approach developed Two of the best indicators of the impact of the project on client adoption of new technologies being developed for the three priority crops include: 1) a 75-percent achievement of the original LOA target for “number of farmers and others who have applied new technologies or management practices” (Outcome Indicator 3.1. Annex II); and 2) an 86-percent achievement of the original target for, “number of hectares (ha) under improved technologies or management practices” (Outcome Indicator 3.2, Annex II). The percent achievement of the original target for farmer technology application would be even higher (170 vs 75 percent) if the tracking data sent to USAID/Ghana had not included the targets for the sixth year of the project (an additional 23,308 farmers) that was never funded. Based on this triangulation of information, and the support for strengthening the project’s commercialization unit (as scheduled under Output 3.2), the ET concluded the project only partially achieved its goals for this SP. Output 3.1. Information and Communication Technology (ICT) systems for development developed. This project’s support, combined with the support of the USAID-funded Agriculture Technology Transfer (ATT) Project, helped the institute create its first ICT and knowledge management unit, which was up and running with its first full-time staff person in the first year. Based on stakeholder feedback and project documentation, the ET agrees with the project’s close-out report that the project’s extensive investment in the construction and equipment of the institute’s first ICT unit, in combination with the mentoring and support it received from ATT, “resulted in a significant improvement of CSIR-SARI’s capacity to share information and communicate with partners.” One of the best indicators of this improved communication was the project’s ability to achieve 88-100 percent of its revised targets for radio, TV, newsprint, leaflets, newsletters, and exhibitions outlined in the 2017 ICT strategy (Annex II). Both the close-out report and most of project partners interviewed in the FGDs/KIIs considered the project’s extensive investment in the development of CSIR-SARI’s first ICT unit, “resulted in a significant improvement in CSIR-SARI’s capacity to share information and communicate with partners,” and that had been further strengthened by increasing internet connectivity at the main research center at Nyankpala. Since the chief criticism of this achievement was plans for connecting CSIR-SARI’s three field stations to the internet were not completed before the project ended (even though improvement of the ICT connectivity bandwidth issues at Nyankpala had not been one of the project’s stated goals), the ET concluded the project successfully achieved all of its original objectives, plus one very important one not envisioned in the original plan. Output 3.2. Commercialization program strengthened. Unfortunately, there was little high-level administrative support for the project’s activities designed to strengthen the institute’s capacity to commercialize some of its services (like training and land preparation, soil testing) and products (like improved seed). Although 2.70 percent of the original budget was designated for this activity, only 0.18 of the final budget was spent on it. One of the best quantitative indicators of the project’s failure to build this capacity is, “even though the technical team was able to increase early generation seed (EGS) production by more than ten-fold and uptake of EGS by 98 percent, the project was not able to account for the increased internally generated funds (IGFs) over the project phase due to some internal institutional VII | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV weaknesses.”14 Another indicator is the project’s custom indicator, which showed after a 76 percent increase in IGFs in 2016 followed by a 115 percent increase in 2017, the institute actually lost money in 2019 (negative 55 percent) (Annex II). Based on the triangulation of this data with stakeholder feedback, the ET concludes that almost none of the original results for this output were ever achieved.15 Output 3.3. M&E system developed and operationalized. Unfortunately, instead of recruiting a qualified M&E specialist to lead the activity as anticipated in the project plan and budget, the project appointed a series of researchers who were already fully deployed to serve as acting M&E leads. This high level of turnover (three leads in five years) continued until one of the junior staff economists—and a group of cohorts (none of whom were trained under this project)—returned from graduate training to become the de facto CSIR-SARI M&E lead and focal persons.16 Most staff interviewed agreed this high level of turnover had a negative impact on the project’s ability to undertake the nine activities expected to contribute to the achievement of this output and resulted in the project not having a fully functional M&E system until its final year. Thus, it is not surprising only 21 percent of CSIR-SARI staff in FGDs/KIIs and 35 percent of CSIR-SARI staff who responded to the online questionnaire agreed, “the project’s support has strengthened SARI’s M&E systems (Annex IV.A.1).” Based on this information, the ET concludes that the project’s support for improving CSIR-SARI’s M&E systems was only a partial success and, “there is still a lot of room for improvement of M&E at SARI.” Conclusions The project: 1. Achieved a critical mass of their expected targets for five of the ten (50 percent) expected outputs (Outputs 1.1, 1.3, 2.1, 2.2, and 3.1). 2. Partially achieved some of the targets, but at a lower-than-expected level, for two (29 percent) of the expected outputs (Outputs 2.3 and 3.3). 3. Eliminated the funding of one of the critical outputs designed to build capacity of the RELCs, which found funding from another source (Output 2.4). 4. Failed to achieve any of the major institutional reforms and processes or targets identified under two of the ten (20 percent) expected outputs (Outputs 1.2 and 3.2). Despite the project’s sub-par achievement on two of the expected outputs focusing on increased organizational capacity and efficiency and commercialization (Outputs 1.2 and 3.2), there is clear evidence from project reports and stakeholder feedback that the successful achievement or partial achievement of a critical mass of activities under the other eight has increased CSIR-SARI’s capacity to support its core mandate in northern Ghana. For these achievements to be sustained through a combination of attracting new donor-funded projects and commercially generated IGFs, CSIR-SARI will have to address the organizational and efficiency issues this project targeted (but did not execute) under Outputs 1.2 and 3.2 as quickly as possible with its own resources and additional mentoring it may be able to leverage from its existing base of donor-funded projects. 14 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 37 15 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pp. 36-37. 16 Dr. Prince Maxwell Etwire returned from PhD training in December 2018 and became the acting M&E lead. To date, October 2021, CSIR-SARI has not officially appointed a full-time M&E lead (email communication, Dr. Prince Etwire, November 10, 2021). Other members of the current M&E team returned from graduate studies in 2017 and 2018. VIII | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV EQ 2: Is there evidence of improved organizational development and a strengthened institution of SARI (increased research capacity) and has this resulted in improved welfare of smallholder farmers in northern Ghana? Findings EQ 2.A. Has SARI increased its capacity in key areas that affect community out-reach? Five of the best quantitative indicators of CSIR-SARI’s increased capacity in key areas that affect community outreach are: (Annex II): 1. A substantial increase (from 20 to 336 in 2018) in the “number of technologies or management practices under research, under field testing, or made available for transfer.” 2. The accelerated roll out (and approval) of 12 new climate-smart varieties of seed for the three priority crops (maize, rice, and soybean) (133 percent of target)—eight of which were produced with direct support from the project, plus another four produced and released through a combination of support from the project and other donor-funded initiatives. 3. The successful pilot testing, review, and adaptation of soil-specific protocols for priority crop ISFM (100 percent of target), currently being scaled up (post project) through MoFA and the private sector. 4. A substantial overachievement (219 percent) of the original target for seed production. 5. The progressive pilot testing and scale up of a new project-funded initiative to train MoFA AEAs in the new CSIR-SARI technologies to facilitate MoFA’s collaboration in the design and execution of the institute’s on-farm demonstration and adaptive trials and field days, which have emerged as a critical pipeline for communication about CSIR-SARI’s new technologies to both farmers and the private sector (156 percent of target for AEA training and 99 percent of target for the trials). Another indicator cited by both the project close-out report and many community and project partner stakeholders is the project’s successful co-mobilization (with MoFA) of a comprehensive public and farmers’ education program about the Fall Army Worm (FAW) control and management in 2016.17 Sixty-seven (77) percent of CSIR-SARI staff and 72 percent of partner staff who participated in FGDs/KIIs either agreed or strongly agreed the project substantially increased CSIR-SARI’s, “connections with farmers.” EQ 2.B. Is there evidence that this increased capacity has improved the welfare of smallholder farmers? Three of the best quantitative indicators of the project’s impact on smallholder farmers are its: 1) seventy-five (75) percent achievement of the original target for the number of farmers and others who reported applying the new seed and agronomic practices promoted by the project; 2) sixty-seven (67) percent achievement of the number of households benefitting from the activities; and 3) eighty-eight (88) percent achievement of the original targets for the number of hectares under improved CSIR-SARI technologies for seed and agronomic practices (Annex II). These beneficiary-based achievements are all the more remarkable given the project’s budget liquidation issues, which directly affected the total number of individuals who benefitted from the project’s short-term trainings by 70 percent (from the LOA target of 49,094 to 14,794; Annex II). 17 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID-Ghana. Pp. 17, 31. IX | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV The project’s close-out report concluded the institute’s increased production of EGS for the three target crops (from 29.7 MT in FY 2017 to 71 MT in FY 2019; Annex II), “increased the community-level demand for and utilization of certified seed from 14 percent in 2014 to 35 percent in 2019.”18 This finding was confirmed by the project’s 2016 end-line technology uptake survey that showed the rate of application of improved seed increased but was still below 40 percent for maize and soybean. Based on the results of the CSIR-SARI-managed on-station and on-farm trials, the project’s close-out report stated using the proposed package of ISFM practices and improved seed was associated with, “increased grain yields of maize, soybean, and rice by 20, 24 and 45 percent, respectively, as well as increased efficiency of fertilizer, and other agro-inputs.” Although these results were promising, the actual promotion of the new ISFM to encourage their adoption by the public and private sector (Phase II and Phase III) did not start until FY 2019 (Annex II). Even with this late roll out, 32 percent of the farmers included in the end-line survey reported using at least some of the improved cultural practices, and 23 percent reported using the recommended practices for soil-related fertility and conservation.19 Unfortunately, there is no quantitative data on if and how the increased adoption of these protocols is affecting farmer income or yields or what the most critical challenges to further scale-up are likely to be. Conclusions There is qualitative and quantitative evidence the project’s support for basic infrastructure and joint field trials with MoFA, combined with its support for building and equipping a new ICT center and co-training (with ATT) CSIR-SARI’s first ICT communications director, increased CSIR-SARI’s capacity for extension and ICT activities that promoted new seed varieties and improved production techniques for rural farmers. The community-based interviews conducted by the ET with farmers and MoFA AEAs, as well as data from ATT, confirmed the project’s internal tracking data that: 1) many new technologies (especially seed and cultivation practices) the project helped develop or hone from earlier projects are being scaled up; and 2) these new technologies have the potential to increase farmers’ yield; but 3) it is hard to track the actual impact of these technology scale-ups on farmers’ yields or income, or constraints to further scale￾up with the project’s M&E data. The same community-based interviews underscored the critical importance of CSIR-SARI strengthening its collaboration with the next generation of GoG and donor￾funded projects (including those funded by USAID/Ghana) to address some of the major challenges likely to affect a more broad-based scale up of the new technologies being developed. EQ 3: What unintended contributions, results, and/or outcomes have the project approach and activities achieved relative to transforming CSIR-SARI into a Center of Excellence for Research and Development in the Savannah Agro-ecological Zone? Unexpected Outcomes That Had a Positive Impact on CSIR-SARI’s Transformation Four unexpected contributions of the project not envisioned in the original proposal or work plan identified by most of the stakeholders and project reports include: 1. The project’s support for the state-of-the-art nutrition lab and improving the internet connectivity on the main CSIR-SARI research center. 2. The project’s spectacular over-achievement of its original goals for producing EGS in response to the increased demand for certified and foundation seed stimulated by the GoG Planting for Food and Jobs (PFJ) initiative in 2017. 18 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID-Ghana. Pg. 9. 19 Disaggregated PITT prepared in conjunction with the FY 2019 annual report. X | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV 3. The project’s support to the CSIR-SARI Entomology Unit, which helped it quickly develop technical solutions to manage FAW and provided training to the frontline MoFA staff on how to identify and control the various stages of the pest, which played a valuable role in the community-based management of the pest. 4. The strong partnership that emerged with the University of Illinois, Urbana Champaign (UIUC), which helped transform CSIR-SARI’s soybean unit from one of the weakest units in CSIR-SARI to one of the strongest by co-funding (with this project) a number of key activities which: a) strengthened the technical capacity of CSIR-SARI’s soybean researchers and technicians; b) strengthened the researchers’ connection to other international centers of excellence for soybean research in Sub-Saharan Africa, increasing their access to the improved germplasm needed for new variety development; c) helped improve their understanding of the gender issues affecting new variety adoption and scale up; d) included hosting the international soybean trials in 2020;20 and e) helped conceptualize and partially implement a cost-effective solution to CSIR-SARI’s weak internet connectivity (Annex IV.B.3). Unexpected Outcomes That Had a Negative Impact on CSIR-SARI’s Transformation Seventy-eight (78) percent of the 50 FGDs/KIIs and the close-out report identified the project’s weak management, which had a host of negative impacts on the project’s approach and activities, as the principal unexpected negative result. The same FGDs/KIIs and literature review identified a number of factors that affected CSIR-SARI’s willingness and ability to address these unexpected negative outcomes. The most frequently cited factors were:  CSIR-SARI’s weak management capacity at both the director and management board levels.  The senior administration’s weak understanding of the project proposal and USAID/Ghana’s rules and regulations for procurement outlined in the PIL at the beginning of the project.  The lack of structured, consistent management mentoring by another USAID-funded project.  Neither the IMC nor the project management board ever received the planned management or M&E training, or even a comprehensive briefing on the project, until its very end even though project records show funds to support some of the IMC meetings were disbursed.21 USAID/Ghana did not agree to fund an external mid-term as outlined in the design and the approved M&E plan, which could have provided both the project and USAID/Ghana a forum for addressing these management concerns. Conclusions The ET found evidence of two categories of unexpected outcomes and contributions that had a dramatic impact on CSIR-SARI’s transformation into a regional center of excellence: 1) new activities that strengthened the project’s achievement of its three sub-purposes and CSIR-SARI’s core mandate to its local stakeholders; and 2) CSIR-SARI’s weak management capacity, both at the director and management board levels, which contributed to CSIR-SARI’s failure to execute its original program plan for management, M&E, and commercialization. EQ 4.A) To what extent has the project addressed gender issues in relation to capacity￾building in research and development, and has this had an impact on gender 20 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID-Ghana. Pg. 24. 21 The ET confirmed this finding through an examination of a sample of IMC and management board meeting minutes, as well as KIIs with board members who served during the time of the project. XI | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV mainstreaming in agricultural research? 4.B) What are the lessons learned and best practices, and from which stakeholders or beneficiaries in achieving results? Findings EQ 4.A. Gender Even though the 2011 baseline capacity assessment—the basis for the design of this project—identified the lack of diversity in CSIR-SARI and the management board as a critical constraint,22 there was little discussion of strategies to address gender issues other than to disaggregate data collection in either the project proposal or the M&E plans or reports. The first mention of gender as a cross-cutting intermediate result or objective is in the logical framework of the 2016 revision of the project’s M&E plan. Although the USAID/Ghana PIL identified five expectations for gender mainstreaming in its text,23 the PIL did not require the project to develop a gender integration plan (GIP). Only one workplan activity in the five-year plan (Activity 3.3.8) under Output 3.3 (M&E system developed and operationalized) focused on gender, and this activity was to, “Undertake a study to examine the roles of gender in agricultural production.” The activity description says the study was expected to show how, “the role of gender in [the] agricultural production system can inform targeting of technology development and dissemination,” and to generate a database for this study in Year 1.24 Unfortunately, this study was not finalized until January 2019, two months before the project officially closed.25 The project’s impact on gender mainstreaming is not mentioned in the close-out report. Although the project did comply with USAID/Ghana’s rules and regulation for disaggregated data collection on the six standard Feed the Future indicators, none of this data was presented in the annual or close-out reports. Capacity development for agricultural research in northern Ghana. Eight (13 percent) of the 60 CSIR-SARI staff identified as core beneficiaries/participants in the project and nine (seven percent) of the 120 staff identified as non-core were women, which is slightly lower than the percentage of female CSIR-SARI staff (19 percent) and technicians (23 percent) in 2020. Only three (18 percent) of the 17 junior staff who recently completed Doctors of Philosophy (PhDs) and three (10 percent) of the 30 staff who recently completed their Masters of Science (MSCs) were women. In 2016, the Core of Excellence (COE) reorganization identified 12 leaders on nine themes, two of whom were women—one a recent PhD and one just starting her PhD. A third female PhD staffer was added to the Core of Excellence team after completing her PhD, bringing the total of core component team female members to 25 percent. Although women were encouraged to participate in all project activities, and their participation was tracked by the M&E officers, there was no discussion of formal targets for increasing the number of qualified women applying for jobs and training, and no monitoring of any progress toward achieving more equitable mainstreaming of women in the institute. Agricultural technologies development and dissemination. All 19 leaders of community-based groups, MoFA agents, and CSIR-SARI field technicians who participated in the on-farm adaptive and demonstration trials and farmer field schools emphasized the positive impact CSIR-SARI setting targets for female participation had on increasing the number and level of female beneficiaries. Unfortunately, this formal commitment to gender (which was also reported back to the ET in the MoFA and AEA interviews) is 22 John Nene-Osom Azu and Kwesi Opoku-Debrah. 2012. Assessment of SARI, Nyankpala, Northern Region, Ghana. Accra: Africa Lead for USAID/Ghana. Pg. 13. 23 These expectations were: “1) Adequate representation and participation of women in the hiring of staff; 2) Adequate representation of women in both short and long-term trainings; 3) Formulation of agricultural policy that takes into account gender-based constraints; 4) Data collection that include gender-disaggregated data; and 5) Research topics and agricultural policy recommendations that address gender-based constraints.” 24 CSIR-SARI. 2015. CSIR-SARI Technical and Financial Support Project. Workplan Outline Year One (January 14, 2015– December 31, 2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 36. 25 CSIR-SARI. Direct Support Project. 2019. Gender Roles in Agricultural Production Systems in Northern Ghana. Nyankpala: CSIR-SARI (January 2019). XII | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV hard to document with the project data, which shows women were: 1) twenty-six (26) percent of the 6,156 producers who received short-term training from the project; 2) thirteen (13) percent of government workers the project trained; and 3) only 3 percent of the 70 private-sector individuals trained by the project. EQ 4.B. Lessons Learned and Best Practices Best Practices Based on the FGDs, KIIs, and literature review, the ET identified three examples of best practice the majority of CSIR-SARI, GoG, and partner projects identified as ones they, “would like to see included in future USAID-funded projects.”  On-farm research and AEA training. Sixty-eight (68) percent of the 50 FGDs/KIIs identified the project’s strong collaboration with MoFA on the design, execution, and analysis of the demonstration and multi-site trials as an example of best practice they think should be replicated in future programs.  ICT and communication. The second most widely cited example of best practice (48 percent of the FGDs/KIIs) was the project’s strong support and collaboration on developing the institute’s first ICT program (which was also supported by ATT) and improving the institute’s access to internet (which was also supported by the Soybean Innovation Lab [SIL]).  Seed operations. The third most widely cited example of best practice (33 percent of the FGDs/KIIs) was the project’s support for the modernization of CSIR-SARI’s seed operations, which was the driving force behind its ability to satisfy the strong increase in demand generated by the PFJ initiative, although staff recognize other USAID-funded projects also contributed to this success. Lessons Learned In addition to the best practices identified in the FGDs/KIIs and online questionnaires, stakeholders identified six major lessons learned and 12 associated sub-lessons learned for future programs for different actors (e.g., USAID/Ghana and/or CSIR-SARI).26 Lesson A. Management support (USAID/Ghana and CSIR-SARI). Future capacity-building projects should have strong support for the proposed management structure and the project plan from both the top administrators of the institution and its governance board. Lesson B. Technology dissemination (USAID/Ghana and CSIR-SARI). Future donor-funded projects designed to promote agricultural development in northern Ghana should consider ways these projects can: 1) access services from CSIR-SARI through its commercialization unit if and when this unit becomes more functional; and 2) reduce the cost of credit for the critical complementary investment private-sector seed producers need to sustain and scale up their commercial seed production, like cold storage and seed processing facilities. Lesson C. ICT and communication (USAID/Ghana and CSIR-SARI). Future donor-funded projects supporting agricultural development in northern Ghana should also consider:  Capitalizing on and supporting CSIR’s emerging ICT unit’s outreach programs to strengthen their effectiveness, efficiency, and impact on promoting new higher-yielding seed varieties and ISFM practices.  Supporting (through cost-sharing and/or overhead) some of the recurrent cost of and structural weaknesses of the institute’s current internet system; and 26 Annex IV, Table 2 includes a more detailed analysis of which stakeholders groups identified which lesson learned and specific sub-lessons. XIII | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV  Collaborating with CSIR-SARI researchers on policy papers and ICT programs to: 1) lobby elected officials in Accra to better support agricultural research in general and CSIR-SARI in particular; and 2) better identify some of the most critical policy issues that affect farmers’ ability to buy seed. Lesson D. M&E (CSIR-SARI). CSIR-SARI needs to: 1) ensure future proposals for large capacity￾building projects include adequate training and a line budget for M&E and the salary of M&E specialist with experience in USAID programming and/or a part-time consultant with demonstrated experience if the new project is unable to afford a full-time position; and 2) encourage new donor-funded projects to offer grants to University of Development Studies (UDS) students to study the impact of CSIR-SARI’s research and collaboration with MoFA. Lesson E. Infrastructure (CSIR-SARI). CSIR-SARI needs to: 1) strengthen CSIR-SARI’s internal systems for designing, executing, and maintaining crucial infrastructure to encourage additional donor and GoG investment in improving its facilities; 2) develop better systems for ensuring outside donor investments in basic infrastructure development (including internet) and new or renovated labs for the field stations actually reach their intended targets; and 3) build better systems for ensuring this improved infrastructure is less costly to maintain (like promoting solar and third-party maintenance contracts with institutions like GARNET). Lesson F. Gender mainstreaming (Cross-cutting) (CSIR-SARI). CSIR-SARI needs to: 1) ensure any future donor-funded program to build its research or outreach capacity include a robust GIP; and 2) encourage new donor projects that support agricultural development in northern Ghana fund a group of activities to help build the capacity of women to develop commercial seed production and other value chain activities that capitalize on the improved technologies produced by CSIR-SARI. Conclusions EQ 4.A. Gender. Although there is both qualitative and quantitative evidence that CSIR-SARI’s commitment to increasing women’s participation in its field trials and field days may have increased their participation in and benefits from these activities, these were never part of a conscious gender strategy. Based on feedback from the FGDs/KIIs and literature review, the ET concluded: 1) there was very little consideration of gender issues in either the USAID/guidance (e.g., the initial Request for Information [RFI], PIL, or project design document, work plans, or reporting; and 2) the project’s impact on, “gender issues in relation to capacity-building in research and development,” and, “gender mainstreaming in agricultural research,” 27 was minimal. EQ 4.B. Best practice and lessons learned. Despite a certain amount of variation between partner groups, there was a great deal of overlap and consensus between the stakeholders on the three best practices and six lessons learned integrated into the evaluation’s summary recommendations and sub￾recommendations. EQ 5.A) TO WHAT EXTENT ARE THE PROJECT RESULTS LIKELY TO CONTINUE AFTER THE UNITED STATES GOVERNMENT (USG) SUPPORT ENDS? 5.B) WHAT ORGANIZATIONAL DEVELOPMENT SUPPORT AND INSTITUTIONAL STRENGTHENING ASSISTANCE WOULD CSIR-SARI NEED FOR IT TO IMPROVE ITS ABILITY TO SUSTAINABLY BENEFIT BUSINESSES AND FARMERS IN GHANA? 27 The text of EQ 4.A. XIV | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Findings Output 1.1. Core scientists trained and re-tooled. Based on the triangulation of the data in the feedback from the FGDs/KIIs and project reports, the ET agrees the results under this output will be sustained with CSIR-SARI’s current projected levels of GoG and donor support. For them to be fully sustained, CSIR￾SARI will need additional, “organizational development support and institutional strengthening assistance,” to achieve the results the project identified under Outputs 1.2 and 1.3, as well as improved advocacy (based on research and M&E results) to strengthen the GoG’s core support for agricultural research. Output 1.2. Increased organizational quality and efficiency. Forty-nine (49) percent of staff either agreed or strongly agreed the project catalyzed a number of improvements in CSIR-SARI’s operational procedures (like procurement, accounting, and reporting) that have made the institute more efficient. Another 30 percent of staff reported they only slightly agreed with this statement because many of the changes are just starting. In light of this feedback and the growing interest of the institute in attracting new donor￾funded projects (which will require these reforms), the ET concluded it is likely (but not completely assured) at least some of these new procedures will be sustained. Given the extent of CSIR-SARI’s management issues, any future donor-sponsored project wishing to strengthen CSIR-SARI’s organizational capacity in these areas should be structured as a mentorship activity given the institute’s weak internal capacity to do this on its own. Output 1.3. Infrastructure and facilities built. Based on the feedback from the FGDs/KIIs, the ET concluded that, while the project’s investment in infrastructure is sustainable in the short-run, CSIR-SARI needs to strengthen its internal infrastructure management systems as well as its commercialization in order to generate the revenue it needs for additional investment and updates. Output 2.1. Seed operations modernized. Based on project records, the ET agrees with the majority of the CSIR-SARI staff interviewed that the project’s investments in modernizing its seed production are likely to be sustained in the short-term because CSIR-SARI, “has a direct financial stake in sustaining these activities, which are its principal source of commercial revenue.” At the same time, the ET agrees with the CSIR-SARI staff and other partners who identified a number of constraints the institute needs to address in the next three years to sustain the quality, price, and demand for CSIR-SARI’s breeder seed, including: 1) the projected phase out of the GoG PFJ seed subsidies in 2024; 2) weak development of private￾sector seed processing and cold storage facilities for seed in northern Ghana; 3) weak coordination between CSIR-SARI, the Plant Protection and Regulatory Services Directorate (PPRSD), and the commercial seed producers needed to ensure seed quality; and 4) CSIR-SARI’s weak and limited access to the types of mechanization and irrigation needed to increase production. Output 2.2. ISFM practices developed and disseminated. In contrast, the ET agrees with many stakeholders (especially those in the private sector) who consider the sustainability of the project’s considerable achievements in developing and disseminating improved ISFM likely but at greater risk for two reasons: 1) this is not currently a research area that generates commercial income for the institute; and 2) the limited capacity of CSIR-SARI’s soils lab, which plays a critical role in giving farmers and AEAs information to determine what type of fertilizer is needed. Since there are a growing number of elite commercialization farmers who require these services and may be willing to pay for them, this is an area where CSIR-SARI could strengthen its commercialization unit in the ways envisioned by the project and reiterated in this report. Output 2.3. Technical staff capacity built. Since CSIR-SARI’s routine training of its technicians is assured under GoG funding, most staff were optimistic that at least some technical capacity developed under the project could be sustained, at least in the short term. However, for the institute to ensure its staff stays up-to-date on new technology trends, and to ensure an efficient and effective allocation of any GoG funds it receives, CSIR-SARI needs to strengthen its internal systems for tracking the training of its staff as was envisioned but never implemented under Activity 1.2.7 of the project. XV | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Output 3.1. ICT and Knowledge Management System (KMS) for enhanced market systems development built. Based on feedback from the FGDs and KIIs, the ET concluded the post-project sustainability of the project’s considerable achievements in building the institute’s first ICT system and improving the internet system on the main campus are likely to be sustained in the short term, but are at greater risk because none of the routine costs for operating either system (other than labor) are covered by the institute’s GoG budget. To address this issue, CSIR-SARI needs to develop more effective organizational systems for getting new and existing projects to contribute to the routine costs of the internet, website, and documentation/database library that all of them are using and benefitting from. Output 3.2. Commercialization programs strengthened. The ET agreed with the project’s close-out report that the modest achievements under this output are unlikely to be sustained and are a critical cross￾cutting problem that affects the sustainability of five of the ten other project achievements that do not rely on core government funding and/or are unlikely to attract additional donor investment (Outputs 1.2, 2.2, 2.4, 3.1, and 3.3). The ET concludes that any short-term solution to this issue will require CSIR￾SARI to restructure the internal systems it uses to manage this unit and that this is an area (like management) that could benefit from outside mentoring from other donor-funded projects.28 Output 3.3. M&E system developed and operationalized. Although the project never achieved its original goal of developing and operationalizing an M&E system for CSIR-SARI, it contributed to: 1) building the basic capacity of a new generation of CSIR-SARI economists; and 2) these economists trying to build the type of institute-wide M&E system the project was expected to develop. Since a functional M&E system affects the long-term sustainability of each and every other one of the critical outputs targeted by the project—as well as CSIR-SARI’s ability to attract additional outside donor funds and manage the IGFs from the institute’s commercial sales—this is an area CSIR-SARI needs to develop, and one that the institute’s key partners (like USAID/Ghana) might consider supporting through other projects. Conclusions In the short term (one to five years) the sustainability of the project’s results is assured for four of the ten project outputs, which are all or partially funded by GoG (like staff salaries) or by revenue earned from commercial seed sales, and/or are likely to benefit from overhead or direct financial support or mentoring (in the case of the new operational procedures) from other donor-funded projects (Outputs 1.1, 1.3, 2.1, and 2.3). Although the results of five more outputs (Outputs 1.2, 2.2, 2.4, 3.1, and 3.3) are likely to be sustained, the conditions for fully sustaining them at their current levels are not guaranteed. The principal reasons for this are: 1) they are not supported by the core GoG budget; 2) the funds generated by the CSIR￾SARI commercial unit are insufficient to support them; and 3) in most cases, they are not the types of activities CSIR-SARI’s current roster of donor-funded projects are willing or able to support. Unfortunately, due to the limited success of the project executing the activities needed to strengthen CSIR-SARI’s commercialization unit (Output 3.2), this unit is not fully functionally nor likely to be without extensive, “development support and institutional strengthening,” from CSIR-SARI and supportive donors. 28 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Close-out Report (2015–2019). Nyankpala: CSIR-SARI for USAID/Ghana. Pp. 36- 37. XVI | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV RECOMMENDATIONS (see Sections 4.1.3, 4.2.3, 4.3.3, 4.4.3, 4.5.3, and Annex V.B.6 for a detailed list of sub-recommendations). CSIR-SARI Recommendation 1. Project management systems. CSIR-SARI needs to develop stronger systems for ensuring that large donor-funded projects like this one have appropriate systems for independent project management, mentorship, start-up/launch activities, supervision, and finance, and staff management.. Recommendation 2. Infrastructure and equipment. CSIR-SARI needs to develop better systems for cost-sharing or generating user-fees to support the cost of routine maintenance and updating of the infrastructure and equipment. Recommendation 3. Commercialization. Given the critical importance of CSIR-SARI’s commercialization activities in supplementing its core GoG funding, it needs to: 1) advocate for and support private-sector investments in seed production infrastructure; 2) put the CSIR-SARI commercialization unit under the direct supervision of the CSIR-SARI’s director and management board; 3) create a line budget for the business plan developed under this project; 4) create indicators for tracking the execution of the business plan; and 5) develop and enforce a professional code that clarifies any restrictions on scientists and technicians setting up side businesses. Recommendation 4. M&E. CSIR-SARI should consider ways its current and future donor-funded projects can assist the institute with: 1) current efforts to create an agency-wide M&E system that complies with international donor standards of best practice (for PITTs, M&E budgets, M&E unit set-up, and M&E mainstreaming) and links to the institute’s new five-year strategic plan; and 2) support more robust M&E training and mentoring of the economists serving as M&E specialists and general staff. Recommendation 5. Technology dissemination and scale-up. CSIR-SARI needs to strengthen the existing and projected synergies between itself and its principal GoG collaborators and beneficiaries (such as PPRSD and MoFA), as well as other donor-funded partner projects in northern Ghana and other Savannah areas of West Africa to sustain and scale up the joint activities (like seed quality control, on-farm demonstration and adaptive trials, and ICT messaging) developed under the project. Recommendation 6. ICT. CSIR-SARI needs to identify ways other donor-funded initiatives in northern Ghana that work directly or indirectly with CSIR-SARI can continue to support and scale-up the promising ICT initiatives that this project (and the USAID-funded ATT as well as earlier USAID￾funded initiatives) helped the institute develop. It is further recommended that CSIR-SARI pool those resources to create a comprehensive ICT budget for CSIR-SARI that includes a line item for a fully trained ICT coordinator with responsibility for developing an ongoing ICT management plan that provides for ongoing improvements in internet connectivity, networking infrastructure, network administration/management, and ICT training for all CSIR-SARI staff members. Recommendation 7. Gender. CSIR-SARI needs to strengthen its internal capacity to: 1) incorporate gender and youth employment issues into future project designs; 2) mainstream the gender and youth targeting activities into its institutional culture; and 3) monitor these activities in ways that feed into needs-based strategies to encourage gender integration and the recruitment and retention of female staff. USAID/Ghana Recommendation A. Pre-conditions for large budget transfers. Make receipt of any sizable G2G grant conditional on the beneficiary institution having a robust management and steering committee structure in place by the fourth month of the project. XVII | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Recommendation B. Collaboration with other USAID/Ghana-funded initiatives. Require future G2G projects to sign a detailed memorandum of understanding (MOU) with any other USAID/Ghana-funded project that is expected to mentor it (like ATT that mentored this project) that clarifies the type of mentoring support to be provided and how the mentorship activities will be co￾monitored and reported to the donors funding the projects. Recommendation C. Mid-term evaluations. Require new G2G projects to include a detailed description of, and designated budget for, a mid-term evaluation to provide a forum to address any management, implementation, M&E, reporting, or governance issues that emerge in the first half of the project. Recommendation D. Budget transfers. Consider a more flexible system for forwarding money to G2G projects than the monthly Imprest system used under this project. Recommendation E. Fee-based training. Encourage future USAID-funded projects in northern Ghana to access fee-based training programs from CSIR-SARI for their staff. Recommendation F. Collaboration with MoFA. Encourage future USAID-funded projects in northern Ghana to identify ways to help scale up CSIR-SARI’s successful collaboration with MoFA for AEA training, field trials, and field days. Recommendation G. ICT internet connectivity. Given the cross-cutting impact of poor internet connectivity on capacity-building and outreach on this project and many other national institutions in Ghana, USAID/Ghana should encourage G2G projects to complete SIL’s new online ICT Health CheckUp assessment tool, and use the results from that assessment to work with their local internet cooperatives or projects(e.g., GARNET, SIL’s ICT Connectivity Project, and/or the computer scientists in the CSIR Electronics Unit) that can help lower the costs of initial installation and maintenance of sustainable internet services. Recommendation H. Gender issues in design. USAID/Ghana should consider strengthening the guidance it provides for GoG institutions that apply for G2G grants: This guidance should: 1) require new project designs to include gender and youth mainstreaming as a project sub-purpose/sub-IR in the results framework; and 2) identify some of the most important documents on the USAID website programs can use to help incorporate gender into the initial design of their GIPs (including issues related to staffing, budget, governance, and M&E). Recommendation I. Gender issues in the PIL. If a new G2G project plan does not include a solid plan for GIP, USAID/Ghana should require this in the PIL as it did on another G2G project executed in the same time period as this one.29 29 D.E. McMillan, L. Hurtak, A.D. Akanko, and Evans Arzi. 2020. Fisheries and Coastal Management Capacity Building Support Project Final Evaluation. Accra: METSS II Project for USAID/Ghana. Pg. 38. 1 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV 1.0. EVALUATION OVERVIEW 1.1. EVALUATION PURPOSE The purpose of the final performance evaluation is to assess the extent to which the Council for Scientific and Industrial Research-Savanna Agriculture Research Institute (CSIR-SARI) achieved the goal and objectives of the CSIR-SARI Technical and Financial Support Project and how the implementation of project interventions has contributed to achieving the United States Agency for International Development Ghana Mission’s (USAID/Ghana’s) 2013–2019 Country Development Cooperation Strategy (CDCS) Development Objective 2 (DO 2), “Sustainable and broadly shared economic growth,” and its Intermediate Results (IRs) 2.1-2.4.30 The project focused on strengthening the capacity of SARI to deliver on its mandate to provide farmers in Ghana’s Northern, Upper East, and Upper West regions with appropriate innovations/options/technologies to increase their food production based on a sustainable production system which maintains and/or increases soil fertility. The evaluation will determine how the implementation of the project activities contributed to, “Transform[ing] CSIR-SARI into a Center of Excellence for agricultural research and development in the savannah agro-ecological zones in Sub￾Saharan Africa.”31 The evaluation will also identify which project components worked well or not and why, to serve as learning to inform future design of similar projects. 1.2. EVALUATION QUESTIONS The USAID/Ghana Statement of Work (SOW) for the evaluation identified five questions the evaluation team (ET) is required to answer:32 1. To what extent has the CSIR-SARI Technical and Financial Support Project achieved its intended goal and objectives as defined by the results framework? 2. Is there evidence of improved organizational development and a strengthened institution of SARI (increased research capacity) and has this resulted in improved welfare of smallholder farmers in northern Ghana? 3. What unintended contributions, results, and/or outcomes has the project approach and activities achieved relative to transforming CSIR-SARI into a Center of Excellence for Research and Development in the Savannah Agro-ecological Zone? 4. 4.A)To what extent has the project addressed gender issues in relation to capacity-building in research and development, and has this had an impact on gender mainstreaming in agricultural research? 4.B) What are the lessons learned and best practices, and from which stakeholders or beneficiaries, in achieving results? 5. 5.A)To what extent are the project results and outcomes likely to continue after the United States Government (USG) support ends? 5.B) What organizational development support and institutional strengthening assistance would CSIR-SARI need for it to improve its ability to sustainably benefit businesses and farmers in Ghana? 30 SARI. 2015. Program Description. Support to CSIR-SARI (January 14, 2015–January 14, 2020). Accra: SARI for USAID/Ghana. Annex 1. 31 USAID/Ghana 2014. Program Description. Op. cit. Pg. 13. 32 USAID/Ghana. 2020. Statement of Work. Final Performance Evaluation of CSIR-SARI Technical and Financial Support Project. Accra: Monitoring, Evaluation, and Technical Support Services (METSS II) Project for USAID/Ghana. Pp. 4-5. 2 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV 2.0. PROJECT BACKGROUND 2.1. MECHANISM DESCRIPTION 2.1.1. CONTEXT Northern Ghana is both the most food-insecure region and the breadbasket of the country. For these reasons, the region has been a major focus of Government of Ghana (GoG), bilateral, and multilateral donor agricultural investments for the last 20 years. Given the critical importance of CSIR-SARI’s role in providing the technologies needed to fuel this investment, CSIR-SARI funded its first capacity assessment and strategic plan in 2005.33 In 2012, USAID commissioned a second institutional assessment of CSIR￾SARI to guide its support for a new generation of Economic Growth (EG) programs under its new CDCS for Ghana (2013–2019).34 The findings from this second assessment revealed although CSIR-SARI implemented relatively successful programs on crop improvement, soil fertility management, cropping systems improvement, crop protection/post-harvest, and socio-economic studies, there were several serious challenges, including:35  Satisfying the needs of stakeholder farmers, especially for small-scale and emerging commercial farmers, seed quality maintenance, and promotion of new varieties;  Stopping the increasing deterioration of the natural resource base—wind and water erosion, low soil organic matter, and poor soil structure; and  Dealing with the negative influence of climate change characterized by erratic rainfall. At the functional level, the 2012 assessment showed CSIR-SARI was mostly nascent or expanding in nine areas of organizational development—governance, management practice, human resource management, service delivery, external relations, sustainability, financial resource management, financial vulnerability, and financial viability/adequacy. 2.1.2. GOAL, OBJECTIVES, AND THEORY OF CHANGE Goal (purpose) and objectives (sub-purposes). The stated goal of the project was, “to establish SARI as a well-managed research organization that effectively develops and disseminates crop and integrated soil fertility management technologies suitable to the Guinea and Sudan savannah agro-ecological zones of Ghana.”36 The stated purpose of the project was to strengthen the capacity of CSIR-SARI to deliver on its mandate, “to provide small-scale farmers in the three regions of northern Ghana with appropriate innovations/options/technologies to increase their food production based on a sustainable production system, which maintains and/or increases soil fertility” (Figure 1). 33 Eva Osei and Augustine Opoku Antwi. 2005. CSIR-SARI. Report on Assessment of CSIR-SARI and Strategic Planning Process. August 2005. Kaneshie, Accra: Nkum Associates for CSIR-SARI. 34 John Nene-Osom Azu and Kwesi Opoku-Debrah. 2012. Assessment of the Savanna Agricultural Research Institute (SARI) Nyankpala, Northern Region, Ghana. Washington, DC: Africa Lead for USAID/Ghana (July 12, 2012) 35 Ibid. 36 USAID/Ghana. 2014. Program Description. Support to CSIR-SARI (January 14, 2015–January 14, 2020). Accra: EG Office, USAID/Ghana. Pg. 13 3 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Figure 1. CSIR-SARI Technical and Financial Support Project Results Framework37 Source: CSIR-SARI. 2014. CSIR-SARI Technical and Financial Support Project. Work Plan Outline Year One (January 14, 2015–December 31, 2020). Implementation Letter No. 641-002 Fiscal Year (FY)14-I #02. Nyankpala: CSIR-SARI. Pg. 7. 37 This results framework is the one the project used for reporting starting in the first year. Although there are minor differences between this one (i.e., shifting the placement of the Research Extension Linkage Committee [RELC] activities from Sub-purpose [SP] 1 to SP 2 and the placement of the monitoring and evaluation (M&E) support from Result 1.2 to Result 3.3), these results and sub-purposes align with the original (2014) M&E Plan and Program Description. 4 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Based on the results of the USAID-funded Africa Lead assessment in 2012, the CSIR-SARI Technical and Financial Support Project was designed to achieve three sub-purposes (SPs):38  SP 1: Capacity development for agricultural research in northern Ghana;  SP 2: Agricultural technologies development and dissemination; and  SP 3: Market and client-oriented research approach including coordination, project management, communication, and monitoring and evaluation (M&E). Theory of Change (results framework). The results framework describes the pathways by which the completion of project activities under the ten outputs were expected to contribute to the achievement of the three project sub-purposes and specific outputs (Figure 1). Based on its analysis of the results framework, the ET created the development hypothesis below to guide the evaluation. The development hypothesis states that: IF these CSIR-SARI Technical and Financial Support Project activities are successful in:  Developing human and institutional capacity for agricultural research through: - Training and re-tooling of CSIR-SARI’s core staff in key program areas (Output 1.1); - Increasing the organizational quality and efficiency of CSIR-SARI (Output 1.2); and - Building new and rehabilitating the basic infrastructure and facilities that the institute needs to support the top-quality research (Output 1.3);  Developing and disseminating agricultural technologies by: - Modernizing CSIR-SARI’s seed operations in line with the Ghana Plant and Fertilizer Act (Output 2.1); - Developing and disseminating integrated soil fertility management (ISFM) practices (Output 2.2); - Building the capacity of CSIR-SARI’s technical staff to apply modern tools/techniques in research (Output 2.3); and - Improving the Research Extension Linkage Committee (RELC) mechanism for agricultural technologies dissemination (Output 2.4); and  Developing market and client-oriented research approaches including coordination, project management, communication, and M&E by: - Developing information and communication technology (ICT) and knowledge management systems (KMSs) for enhanced market-driven research for development (Output 3.1); - Strengthening the commercialization program of the institute (Output 3.2); and - Developing and operationalizing an M&E system (Output 3.3); THEN it will transform CSIR-SARI into a Center of Excellence for agricultural research and development in the savannah agro-ecological zones of Sub-Saharan Africa. 2.1.3. TARGET AREAS AND TARGET POPULATION GROUPS The project’s principal targets were CSIR-SARI researchers at the main research center in Nyankpala and staff supporting the three outstations—the Northern Region, Upper East, and Upper West Farming Systems Research Groups. The target population for the research activities implemented by CSIR-SARI included smallholder farmers with activities such as on-farm adaptive and demonstration trials co￾managed by CSIR-SARI researchers and the Ministry of Food and Agriculture (MoFA). 38 USAID/Ghana. 2014. Program Description. Support to the CSIR-SARI (January 14, 2015–January 14, 2020). Accra: EG Office, USAID/Ghana. Pg. 14. 5 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV The institute collaborated with various stakeholders such as government agencies, academic and research institution partners, and other donor-funded projects and media/communication agencies to implement the project. 2.1.4. PROGRAM IMPLEMENTATION PLAN January 2015-March 2017: In the original implementation plan for the project, it was expected it would have: 1) an independent project implementation unit that would include an independent project manager, a full-time M&E coordinator, and a full-time accountant attached to the project by CSIR-SARI as part of its in-kind contribution; 2) a budget line to ensure the correct operation of this unit; and 3) a steering committee to serve as an interface between the project and the CSIR-SARI management board as was the norm in other large donor-funded projects at CSIR-SARI. These structures were never created and the project remained under the direct supervision and oversight of the CSIR-SARI director and management board.39 Day-to-day management was overseen by a project manager and project M&E coordinator, who both held full-time research positions at CSIR-SARI and were appointed by the CSIR￾SARI director. The project’s only paid employee, who worked full time for most of the project without other obligations to CSIR-SARI, was an administrative assistant.40 Although the project proposal anticipated the creation of a steering committee to interface between the project and the management board (as with other CSIR-SARI donor projects), it was never created. The CSIR-SARI director and his staff continued to have advisory and decision-making authority for the project, as well as the responsibility of approving project work plans and budgets and regularly monitoring activities. March 2017-March 2020: Based on an internal review, USAID/Ghana facilitated through the USAID￾funded Agriculture Technology Transfer (ATT) Project’s budgeted support for research capacity￾building, a series of follow-up meetings, and a 14-day study tour for seven staff and administrators to the United States (U.S.), the project made a series of changes CSIR-SARI and USAID/Ghana felt were justified to get greater results in the project’s remaining time (see Annex V.B.5. for a review of this process and the resulting changes). This reorganization (described in detail in the Fiscal Year [FY] 2017 annual report), “affected the implementation of some activities, as resources have [had] to be redirected for maximum efficiency and impact,”41 of the original three sub-purposes and ten outputs (Figure 1). These implementation changes included:  Developing a revised plan of action for the project that focused on the achievement of four outcomes and 13 sub-outcomes (Text Box 1).  Returning to the program implementation plan’s original model of having an independent project manager, M&E coordinator, and accountant.  Creating an independent advisory board for the project comprised of leading international experts in plant breeding and seed production.  Shifting from an implementation model focused on CSIR-SARI’s existing departments and research teams to a more focused group of CSIR-SARI staff and activities that operated as eight component teams: 39 Each CSIR institute has a management board that is appointed by the CSIR head office in consultation with each institute to oversee its general governance. The CSIR-SARI board is made up of six external members and the institute’s director. The CSIR-SARI deputy director, head of accounts, and head of administration are members in attendance. The board meets twice a year. The CSIR-SARI Internal Management Committee (IMC) is comprised of 16 members who are all heads of recognized units, sections, divisions, and unions within the institute. IMC’s mission is to assist CSIR-SARI’s director supporting the day-to￾day running of the institute, 40 Only the last project manager served full time, and then only once he retired. 41 CSIR-SARI. 2017. USAID/Ghana—CSIR-SARI Technical and Financial Support Project. Annual Report (October 2016– September 2018). Nyankpala: CSIR-SARI. Pg. 3. 6 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV - Components 1-3 focused on building the breeding programs, extension, and seed operations production for the three priority crops supported by the project (maize, rice, and soybean) as envisioned in Components 1.2, 2.1, 2.3, and 2.4 in the original project plan (Figure 1). - Component 4 focused on accelerating the project’s original commitment to scaling up CSIR-SARI’s research and dissemination of new technologies to promote integrated soil fertility (Component 2.2 in the original project plan, Figure 1). - Component 5 focused on strengthening CSIR-SARI’s commercial processes and contacts in ways that would create the types of predictable, well-managed revenue streams CSIR was asking all of its institutions to develop in order to be more self-sufficient (Component 3.2 in the original project plan, Figure 1). - Component 6 focused on building the global management capacity of CSIR-SARI and its senior researchers, including SARI’s capacity to oversee infrastructure development, renovation, and maintenance (Components 1.2 and 1.3 in the original project plan, Figure 1). - Component 7 focused on helping CSIR-SARI develop the type of institution-wide M&E system it needs for results-based programming (Component 3.3 in the original project plan, Figure 1). - Component 8 focused on building CSIR-SARI’s ICT systems (Component 3.1 in the original project plan, Figure 1). Text Box 1. The Four Outcomes and 13 Sub-outcomes Identified as Capacity-building Needs for the Revision of the Project as a Core of Excellence (COE) Model in March 2017 Outcome 1: Set up the COE 1.1. Recruitment of a small team. 1.2. Set up a management structure. 1.3. Identify and elect key national and international relevant partners. 1.4. Develop strategy and key milestones. 1.5. Establish and operationalize an information, education, and communications unit. 1.5.1. Strengthen communication processes to make COE activities more visible to the wider public, private, and civil-society actors. 1.5.2. Empower CSIR-SARI to market itself adequately through the provision of capacity-development assistance in the areas of advocacy, establishment of a public relations/media unit, and establishment of a media strategy to assure constant interaction and advertisement of CSIR-SARI’s activities. 1.5.3. Develop a strategy for promoting and marketing COE service and products, which would also open up more opportunities to serve the interest of its many stakeholders, in addition to the contracts it is able to negotiate with large external donors. Outcome 2: Mentorship, coaching, and professional development for the COE staff 2.1. Sign Memorandum of Understanding (MOU) with Iowa State University (ISU) and enter into public-private partnership agreements for effective implementation of COE activities. 2.2. Support needs-based study cum exposure visits and organize capacity-development programs to enhance capacities of COE staff. 2.3. Conduct demand-driven, socio-economical, responsible research for development aimed at strengthening CSIR-SARI’s breeding program, knowledge systems, ICT, communications, and business strategies to improve and ensure long-term social and financial sustainability of CSIR-SARI. 2.3.1. Set up a germ plasm unit seed lab facility and seed storage space to begin the process of improving quality and supply of breeder and foundation seed. -Train staff to enhance their capacity to handle the activities of the seed value chain. -Consider a service-for-fee concept to foster public-private partnerships (PPPs) by allowing private companies and commercial seed growers to produce certified seed on their premises. 7 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV -Consider establishing a one-stop shop to provide service throughout the seed value chain and promote seed entrepreneurship. Outcome 3: Develop a training office in seed research and seed business development capabilities 3.1. Identify the regional and national seed research and seed business development training needs to determine the content of the future training curriculum. 3.2. Develop pilot courses and training of trainers’ curriculum. 3.3. Set up seed research and seed business development training center. Outcome 4: Information sharing and cooperation 4.1. Set up an internet-seed tool for the COE and basic sharing of information on seed research and seed business development. 4.2. Set up a documentation center on the COE premises. 4.3. Organize annual national workshops to raise awareness on the importance of seed, seed research, and seed business development. 4.4. Organize an annual regional workshop on seed research and innovation. Source: Wilson Dogbe. 2018. Power Point Presentation on Review of Implementation of COE Project. August 2018. CSIR￾SARI Project Documentation. This new implementation model was referred to as the Core of Excellence (COE) model—to distinguish it from the original Center of Excellence implementation model (January 2015–March 2017), which focused on building the capacity of the entire institution to be a national and international Center of Excellence. In July 2017, the project—with support from ATT—organized an official one-day launch of the new COE model and signed an initial agreement with Iowa State University (ISU) to provide technical back-up, which was never fully executed (Annex IV.B.5). In January 2019, the project signed a second technical support contract with AgTech Innovation Labs (AgTILs) to provide technical backstopping during its final year (March 2019–December 2019) (Annex IV.B.5). 8 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV 3.0. EVALUATION METHODS AND LIMITATIONS 3.1. EVALUATION METHODOLOGY The ET used a mixed-methods approach to collect and analyze data to complete the performance evaluation. The ET used document review, focus group discussions (FGDs), key informant interviews (KIIs), and a post-interview online questionnaire to gather quantitative and qualitative data to support the evaluation findings and conclusions. These data collection methods are summarized below. The approved evaluation work plan: 1) contains the ET’s original methodology and anticipated limitations;42 and 2) explains the role of the four-person working group and evaluation focal person appointed by the CSIR-SARI director to assist the ET in getting information on the project since it had closed (see Annex III.E.2. for a more detailed description of their roles). Altogether, the team interviewed 107 people from six stakeholder groups in 31 FGDs and 19 KIIs, and received 45 responses to the questionnaire. 3.1.1. DOCUMENT REVIEW Project and Partner Documents. In conjunction with the pre-planning process, the CSIR-SARI staff responsible for M&E activities and the four-person working group the CSIR-SARI director tasked with backstopping the ET43 helped the ET co-develop: 1) an initial list of basic project documents and where to find them; 2) summary tables of key activities; and 3) an initial list of stakeholder groups as basis for sampling that included contact information. In the course of interviewing, the project staff and ET identified the need for other documents and summary tables from the later phase of the project (i.e., after the 2017 assessment44 and subsequent revision of the project’s implementation plan and key project partners (See Section 2.1.4 and Annex V.B.5).45 Performance Indicator Tracking Table Update. Although the project consistently reported on its progress toward the targets identified for the seven USAID EG standard outcome indicators (Annex II), it did not consistently report on the other outcome or output indicators in its original M&E plans or the two successive revisions of the plan (Annex II). Since the ET needed the quantitative data to assess the performance of the original set of project results, as well as the standard Feed the Future indicators, USAID/Ghana approved a request for the ET to develop a consolidated and updated Performance Indicator Tracking Table (PITT) using data from the CSIR-SARI Project Implementation Letter (PIL) reports and documentation. The summary PITT presented in Annex II merges two types of data: 1) the summary data the project reported to USAID on the seven Feed the Future standard indicators; and 2) the data the team reported for the other indicators (project outputs and outcomes) that were not 42 Della E. McMillan, Kwasi Ampofo, Annie Dela Akanko, and Lynn Hurtak. 2021. Work Plan: Final evaluation of CSIR-SARI Technical and Financial Support Project. Accra: The METSS II Project for USAID/Ghana (April 6, 2021). 43 This list of individuals and their qualifications for backstopping the ET are provided with permission in Annex III.E.2. 44 Dileepkuar Guntuku. 2016 (March). Assessment of the Institutional Capacity and Needs of the Savanna Agricultural Research Institute (SARI. Nyankpala, Northern Region, Ghana. Ames, Iowa: International Fertilizer Development Center [IFDC] and ISU for USAID/Ghana). (Final version with corrections dated April 25, 2016). 45 A complete list of the documents the ET and evaluation working group assembled to inform the evaluation is presented in Annex V.A. and saved in a Google Drive folder which will be turned over to both USAID/Ghana and CSIR-SARI after the evaluation is approved. Any personal information the working group does not find appropriate to include would be removed. 9 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV originally reported as part of the CSIR-SARI PIL annual reporting. Whenever possible—in the course of writing—the ET attempted to cross-reference these figures to information in the quarterly, annual, or final close-out report. 3.1.2. DATA COLLECTION INSTRUMENTS To facilitate triangulation, the FGD/KII guides and post-interview online questionnaire asked interviewees to rank a core set of comparable questions using a seven-point Likert scale.46 This triangulation helped the ET: 1) provide more in-depth explanation of the patterns of responses between the FGDs/KIIs and the online questionnaire; and 2) compare the responses of different stakeholder categories. 3.1.3. QUALITATIVE METHODS (FGD/KIIS) In an effort to reduce the complexity of note taking, data entry, and analysis, the ET tailored a core set of questions deemed necessary to answer the five Evaluation Questions (EQs) adapted to each of the six stakeholder groups. These questions were then packaged into four FGD/KII guides. The ET collaborated with the four senior staff and M&E focal person to identify (and confirm the contact information) of the initial list of stakeholders. Although the ET and the evaluation focal person made an exhaustive attempt to facilitate CSIR-SARI developing a stakeholder list for the community-based stakeholders (Stakeholder Group 4), they were unsuccessful due to a number of internal communication, logistical, and documentation constraints within CSIR-SARI (Table 1).47 Table 1. Number of FGD/KII and Post-Interview Questionnaires Targeted and Completed Stakeholder Groups # FGDs & KIIs # FGDs # KIIs # of People (with double counting) # of People (without double counting48) # of People # of People Target (e.g., Sample Frame) Total FGD KII FGD/KIIs FGD/KIIs Online Questionnaire Annex III.B Stakeholder Group 1: CSIR-SARI project staff and oversight committees 22 13 9 63 58 (34 core and 24 non￾core) 31 60 core and 120 non￾core staff (180 total) Stakeholder Group 2: Government agencies the project collaborated with 7 5 2 14 14 14 14 Stakeholder Group 3: Academic and research institution partners 3 2 1 4 4 2 7 46 The rankings were: 1) strongly disagree; 2) disagree; 3) somewhat disagree; 4) neither agree nor disagree (i.e., no opinion); 5) somewhat agree; 6) agree; and 7) strongly agree. 47 These constraints are fully documented in the informal weekly and bi-weekly updates the ET sent to USAID/Ghana and CSIR￾SARI. 48 Six people participated in more than one FGD. 10 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Stakeholder Groups # FGDs & KIIs # FGDs # KIIs # of People (with double counting) # of People (without double counting48) # of People # of People Target (e.g., Sample Frame) Total FGD KII FGD/KIIs FGD/KIIs Online Questionnaire Annex III.B and other donor￾funded projects Stakeholder Group 4: Community-based partners49 15 10 5 19 37 N/A Not available Stakeholder Group 5: Media/communication partners 2 1 1 6 5 N/A 7 Stakeholder Group 6: USAID/Ghana 1 0 1 1 1 N/A 5 Total (all categories) 50 31 19 107 100 45 31 staff; 14 partners) N/A Source: Annex III.B. Overall, the ET interviewed 100 people in Ghana in 19 KIIs and 31 FGDs (Table 1). The KII and FGDs were analyzed to determine: 1) the Likert rankings; and 2) the frequency of different themes in the qualitative explanations for why the people in the interview gave specific rankings. Comparing the number of people with the original stakeholder list in the evaluation work plan, the sample was highly representative for the core CSIR-SARI staff involved with the project and reasonably representative for the other stakeholder groups except for the community-based groups and USAID/Ghana staff (Table 1 and Annex III.C.).50 3.1.4. QUANTITATIVE DATA COLLECTION (ONLINE QUESTIONNAIRE) In an effort to provide quantitative verification of the qualitative data collected by the FGDs and KIIs, the ET sent online questionnaires to individuals who participated in the FGDs and KIIs for the three 49 This includes: (4.a) community-based associations and people working with them to execute the project activities (i.e., CSIR￾SARI staff familiar with the trials, the community-based group in the trials, and the Ministry of Food and Agriculture [MoFA] Agriculture Extension Agent [AEAs]); (4.b) associations producing seed like the Golinga Innovation Platform, the Integrated Water Management and Agricultural Development Ghana Limited (IWAD), and the Northern Region Seed Producer Association of Ghana (SEEDPAG); (4.c) private seed companies; (4.d). large-scale commercial farms; (4.e.) the National Seed Trade Association of Ghana (NASTAG); and (4.f) agro-dealers. The project did not track the number of community-based direct or indirect beneficiaries. 50 Although M&E experts have reliable annual data that 2,000 farmers participated in the project-sponsored demonstration trials, the project did not capture the names, locations, or contact persons for these groups since the chief role of the project was to train and backstop the MoFA agents to conduct the trials. Although the ET was able to identify some of these groups, it discovered the group leaders were unable to distinguish between the support they received from this project and other projects like ATT and a previous generation of Alliance for a Green Revolution in Africa (AGRA). Given these challenges, plus the logistical and financial challenges of getting either a MoFA or CSIR-SARI agent out to facilitate the interviews, USAID/Ghana concurred with the ET’s decision to interview a sample of MoFA agents and CSIR-SARI field staff who supported these community-based groups rather than the community-based groups themselves. 11 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV stakeholder groups that had the most consistent internet access. One questionnaire was sent to Stakeholder Group 1, CSIR-SARI project staff and oversight committees, and another, more abbreviated version was sent to Stakeholder Groups 2 and 3, government and project partners (Table 1). The post￾interview questionnaire was sent to 70 SARI staff; 31 (52 percent) responded. The abbreviated version was sent to 17 government and research partners (including MoFA AEAs); 14 (82 percent) responded (Table 1). 3.2 LIMITATIONS There were several limitations to this evaluation. 1. The principal limitation was assembling the background documentation and establishing a sampling frame for the FGDs and KIIs since the project did not leave behind either a stakeholder database or core project documentation file when it closed out in March 2020. To manage this limitation, the evaluation focal person worked with his colleagues from SARI to reconstruct a list of core and non-core individuals who benefitted from the project and identify the missing documentation. Once this sampling frame was constructed, the ET selected a representative sample of the core SARI staff and reasonable sample of other stakeholders for interviewing. As key documents were identified, they were listed in a master bibliography and filed in Google Drive folders. 2. The project did not have an up-to-date PITT, which should contain the full set of project indicators as well as the annual and life of activity (LOA) targets for specific indicators. Since none of the quarterly or annual reports included a PITT, there was no discussion of the project’s progress toward the achievement of specific indicators.51 To manage this limitation, the ET collaborated with the CSIR-SARI M&E focal person on the development of a retroactive PITT which was then reviewed and validated by the CSIR-SARI evaluation working group upon approval from USAID/Ghana (Annex II). 3. The logistical challenge of conducting the interviews remotely due to the Coronavirus Disease 2019 (COVID-19) pandemic precluded in-person interviews. The ET mitigated the impact of this limitation by: 1) splitting the interviewees into groups to enable effective facilitation of discussions via Zoom; 2) conducting most interviews through a combination of Zoom and local telephone (for those with limited access to internet); and 3) including a fourth team member with the necessary language communication and organizational skills to organize this type of interview process. 4. There was difficulty attributing any result to the project alone since most results were expected to capitalize on: 1) the technological and capacity-building achievements of a previous generation of donor-funded project executed by the Alliance for a Green Revolution in Africa (AGRA) and other donor-funded projects (a.k.a. legacy52 projects that worked with CSIR-SARI before 2015); and 2) important synergies with the ongoing donor and GoG projects with which it overlapped. To manage this risk, the ET asked people being interviewed in the FGDs and KIIs to describe how the project activity they were involved with (or discussing) related to earlier or ongoing donor or GoG-funded initiatives. 5. It was difficult to identify and contact the community-based stakeholders who participated in and benefitted from the project-sponsored trainings, field days, and adaptive and demonstration field 51 In general, the quarterly reports focus on describing the project’s progress toward the execution of the activities in the work plan, not toward the achievement of the targets as measured by the indicators, which is not uncommon for smaller government-to-government (G2G) projects funded by USAID. 52 Term that was often used in the interviews. 12 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV trials since the project never developed a stakeholder database with contact information. To manage this limitation, the ET received approval to interview the MoFA AEAs and SARI field staff who worked in the beneficiary communities to elicit the relevant information to aid the analysis of the data. 13 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV 4.0. FINDINGS AND CONCLUSIONS 4.1. EQ 1: TO WHAT EXTENT HAS THE CSIR-SARI TECHNICAL AND FINANCIAL SUPPORT PROJECT ACHIEVED ITS INTENDED GOAL AND OBJECTIVES AS DEFINED BY THE RESULTS FRAMEWORK? 4.1.1. FINDINGS The ET based its assessment of EQ 1 on the extent to which CSIR-SARI achieved the project’s intended purpose, three sub-purposes, and ten outputs as defined by the results framework on: 1) internal data CSIR-SARI used to track and report on the impact of these investments in its annual and close-out reports; 2) the updated PITT the ET helped create, which included both the six standard indicators the project reported on to USAID/Ghana and the custom indicators identified to assist measure outcomes at the purpose and sub-purpose level (Annex II); 3) the principal stakeholders’ perceptions about how these investments affected the expected outputs and sub-purposes the project was expected to achieve (Annexes IV. A-F); and 4) the key factors stakeholders and the project’s close-out report identified as having contributed to or detracted from the effective execution of specific outputs (Annexes IV. A-F). SP 1: Capacity development for agricultural research in northern Ghana The activities under SP 1 were expected to help CSIR-SARI achieve outcomes the 2012 baseline assessment deemed critical to the successful achievement of its mandate by:  Output 1.1. Strengthening the technical capacity of CSIR-SARI’s core scientists;  Output 1.2. Executing a three-pronged strategy53 designed to address a number of weaknesses in CSIR-SARI’s governance, management practices, human resource management, and service delivery systems; and  Output 1.3. Improving the quality and functionality of some of the most important infrastructure the institute needed to conduct top quality research. One of the best quantitative indicators of this increased capacity—which capitalized on the capacities developed by other USAID and non-USAID-funded legacy and ongoing projects—was the project’s dramatic overachievement (336 vs. 80, 420 percent of target) of its original target for, “number of technologies and management practices, under field testing or made available for transfer,” (Annex II). Although 60 percent of CSIR-SARI staff and 70 percent of partner staff interviewed in FGDs/KIIs agreed or strongly agreed the project, “increased CSIR-SARI’s capacity to provide support for agricultural research in northern Ghana” (Annexes IV.A.1 and A.2), there was a cross-cutting frustration among the staff, confirmed in the project’s close-out report, with the project’s sub-par performance achieving the proposed management reforms under Output 1.1. Based on this triangulation of data, the ET concluded the project only partially achieved its expectations for this SP (Table 2). 53 This three-pronged strategy focused on: 1) developing an independent project management unit capable of leading this type of internal management reform within SARI; 2) building the capacity of the SARI scientists, management board, and internal management committee (IMC) for performance-based project management, proposal development, and financial oversight of large donor-funded grants; and 3) facilitating SARI conducting a, “review of its first strategic plan to inform the development of a new strategic plan,” and, “institutional performance framework and monitoring plan” (CSIR-SARI. 2014. Program Description. Pp. 15-16). 14 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Table 2. Stakeholder Feedback and Project Documentation About the Extent to Which the CSIR-SARI Technical and Financial Support Project Achieved Its Intended Sub-purposes and Outputs Output by SP Staff Agree54 (FGD/KII￾Questionnaire) Partners Agree (FGD/KII￾Questionnaire) Close-out Report’s Assessment and PITT (Annex II) ET Summary Assessment Likert Rankings 6-7 6-7 N/A SP 1 60-61% 79-70% Partial Partial Core scientists trained and retooled 21-42% Not ranked Yes Yes Organizational quality increased 35-57% 57%55 No No Infrastructure built 48-55% 57%56 Yes Yes SP 2 45-66% 77% Yes Yes 2.1. Seed operations modernized 38-35% 8457 -35% Yes Yes 2.2.A. ISFM—development 21-35 48-64 Yes Yes 2.2.B. ISFM—dissemination 16-29% 71% 2.3. Taught staff to apply modern tools 19-42% n/a58 -57% Not discussed Partial 2.4. RELC mechanisms improved 32-35% n/a-57% N/A funding eliminated Yr1 N/A funding eliminated Yr1 SP 3 30-23% 65-n/a Partial Partial 3.1. ICT and KMS systems developed 44-55% Not ranked Yes Yes 3.2. Commercialization program strengthened 9-23% Not ranked No No 3.3 M&E system developed and operationalized 21-35% Not ranked Partial Partial Methodology: Rankings are based on SARI staff (Stakeholder Group 1) and partner (Stakeholder Groups 2, 3, and 6) Likert rankings for the EQs in the FGDs, KIIs, and post-interview questionnaires. The evidence from project reports and M&E (Yes/Limited/No) is based on the literature review. The final column is the team’s assessment (Yes/Likely/No) based on a comparison of the results based on different sources of information and the original expectations outlined for this activity in the project description. Source: Annexes IV.A.1, IV.A.2, IV.B.1 and IV.B.2. 54 First figure is the response from FGDs/KIIs; second figure is from online questionnaire 55 Ranked in online questionnaire, not in FGDs/KIIs. 56 Ranked in online questionnaire, not in FGDs/KIIs. 57 The FGDs/KIIs included more community-based stakeholders like AEAs and agro-dealers who were commenting on the utility of the project’s support for scaling up SARI’s demonstration and adaptive trials and farmer field days. 58 Online questionnaire only. 15 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Output 1.1. Core scientists trained and re-tooled. The internal records and reports confirm the project trained 136 core scientists, administrators, and technicians through 36 sponsored workshops and three exchange visits to international centers of excellence in Africa, the U.S., and India (Annex II).59 Between 2015 and March 2020, the project:  Supported six training and mentorships that provided, “tailored leadership and management training…to improve research administration,” for 23 research scientists and administrators (Activity 1.1.1).  Provided supplementary assistance to three staff members who had other sources of funding for their Master of Science (MS) and Doctor of Philosophy (PhD)-level training (Activity 1.1.2; Annex II).60  Supported 14 technical trainings focused on building the researchers’ and technicians’ research skills in molecular biology, experimental design, Breeding Management Systems (BMSs), foundation seed production, electronic agricultural library research, basic computer skills, statistical software and data analysis, and the design and reporting of on-farm adaptive and demonstration trials (Activity 1.1.3).  Strengthened the researchers’ connection with their community-based stakeholders—the principal objective of their mandate—by pilot testing and scaling up a new model of community￾based farmer field days and demonstration and adaptive trials that trained 6,156 producers, 80 representatives of private seed companies and the major actors in irrigation and water resource development, and a large number61 of AEAs (Activity 1.1.4). 62  Organized six “expert exchange visits” (Activity 1.1.5) for researchers and administrators to international centers of excellence and conferences in Zambia, Zimbabwe, Senegal, India, and the U.S. (Activity 1.1.6). Although the project’s revised PITT shows 136 scientists, technicians, administrators, and M&E and communication staff received various project-supported capacity-training and skills development (Annex II), only 21 percent of CSIR-SARI staff who participated in FGDs/KIIs and 46 percent of online questionnaire respondents either agreed or strongly agreed the project-supported training and retraining strengthened CSIR-SARI’s capacity to deliver on its core mandate. Another 56 percent of staff who participated in FGDs/KIIs and 32 percent who responded to the online questionnaire only somewhat agreed (Table 2; Annexes A.1 and B.1). The most frequently cited reasons for these lower rankings were: 1) not everyone at CSIR-SARI was trained (i.e., the principal recipients of the training were the people associated with the priority value chains and the project activities); 2) a smaller percentage of people working in the priority value chains were trained in the field stations than at the main station at Nyankpala; 3) the project never developed a transparent process for determining who got trained, nor did it link the training targets to CSIR- SARI’s long and short-term staff capacity development as it was supposed to do under Activity 1.2.7 in the original project plan; and 4) many 59 Since there were no summary records disaggregated by type of training, this information was compiled by the ET based on the annual reports, which were then reviewed and corrected by the former project accountants and staff for accuracy in November 2021. Since the ET was unable to get disaggregated training figures for some of the trainings, this table is not included in the annexes. 60 In the original proposal it was expected the project would fund at least five full scholarships for staff. This target was revised during the first year due to the availability of other funds. 61 Although the project has not yet been able to identify the exact number of MoFA AEAs trained, this training—and its benefits—were corroborated by the FGDs/KIIs and annual reports. 62 Disaggregated PITT standard indicators developed by the project for its FY 2019 annual report. 16 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV elements of the original training schedule and targets for training staff were shelved when the COE alignment happened in 2017. In spite of these identified deficiencies, the ET concluded there was ample quantitative and qualitative evidence the project, by and large, contributed either directly (through formal training or exchange visits) or indirectly (through improved infrastructure or on-farm trials) to improving the core capacity of the key staff associated with the three priority value chains, which was the project’s original expected result for this output. Output 1.2. Organizational quality increased. For a variety of reasons—which included the senior CSIR-SARI top management’s gross misunderstanding of the project and the PIL (which was reported on in the close-out report as well as in the FGDs/KIIs)63—the project never executed three of the seven activities the 2012 baseline CSIR-SARI assessment and the project workplan deemed critical to improving the institute’s organizational quality and efficiency, which had a negative impact on the project’s implementation.64 Although the project did manage to update its strategic plan as intended (Activity 1.2.6), the project’s liquidation problems (which were linked to the difficulty the staff had understanding USAID/Ghana’s Imprest budget transfer system) delayed the execution of the plan to the start of the third year instead of the first year. This delay reduced the utility of the strategic plan update as an organizational capacity-building tool during the project since it left little time for more effective follow-up.65 It is important to note, however, that the project did end up having an important post￾project policy impact in that it became the basis for a new five-year strategic plan approved by CSIR￾SARI in 2021.66 Therefore, it is not surprising that the close-out report and 45 percent of the staff who responded to the online questionnaire concluded the project “was not able to achieve the expected improvement in the internal coordination and project management.” 67 In an effort to identify how project activities under Output 1.2 could be reconfigured to better achieve the expected outcomes, ATT conducted a second institutional assessment in January 2017. This report concluded that since the scale of CSIR-SARI’s, “immediate management issues and deficits,” was too big for the project to tackle in the time remaining, the project should shift the focus of its activities under SP 1 to building the capacity of, “a core group of researchers in the maize, soy, and rice seed value chains, along with communications and business development (revenue generation),” as a stepping stone to, “creating 63 This was a common theme in the FGDs and KIIs with staff, and is cross-validated in the project’s close-out report (CSIR￾SARI Technical and Financial Support Project, Pg. 38). 64 The activities not executed include: 1) the creation of the type of stable, independent project management model needed to execute these sweeping administrative changes (Activity 1.2.1); 2) the failure to support the types of training and capacity￾building for CSIR-SARI’s management board and IMC needed to improve their role in oversight and quality control (Activity 1.2.3); and 3) the failure to help CSIR-SARI develop long-term and short-term staff capacity development plans, which reduced the effectiveness and impact of the project-supported training (Activity 1.2.7) (CSIR-SARI. 2015. CSIR-SARI Technical and Financial Support Project. Workplan Outline Year One (January 14, 2015–December 31, 2020). Nyankpala: CSIR-SARI. Pp. 13- 16). 65 Samuel D Braimah. 2017. The CSIR-SARI Strategic Plan: 2018–2222. Nyankpala: The USAID Direct Support Project for CSIR￾SARI (for additional outputs of this exercise see Section I.1.3.1 of Annex V.A). 66 CSIR-SARI. 2021. Final Draft CSIR-SARI Strategic Plan. 2021–2025. Nyankpala: CSIR-SARI. Note: This post-project impact accounts for 48 percent of CSIR-SARI staff who responded to the online questionnaire agreed or strongly agreed the project￾funded strategic planning process increased the organizational quality and efficiency of CSIR-SARI (Annex IV.D.1). 67 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 33. Annexes IV.A.1 and IV.D.1. 17 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV opportunities” on which to build the type of, “long-term change process within SARI,” envisioned for Output 1.2 in the original project document.68 To assist with this shift from the original Center of Excellence model to this more focused COE model, the project signed a series of contracts with ISU and AgTILs starting in July 2017 supported under the budget for Output 1.2. Unfortunately, the same budget liquidation problems69 delayed the execution of many core activities and limited the effectiveness and impact of this type of “hand-holding” (e.g., mentorship) by AgTILs and ISU until the last year of the project (Annex V.B.5). As a result, the ET agreed with the project’s close-out report that, “the project…was not able to address issues surrounding the capacity of individuals and organizational learning within SARI to facilitate the building of social capital and trust, develop knowledge, skills, and attitudes, and successfully create an organizational culture and a set of capabilities to enable the institute to set objectives, achieve results, solve problems, and create adaptive procedures to support national development.”70 Output 1.3. Infrastructure built and renovated. Between 2015 and March 2020, the project executed all seven of the activities envisioned in the original proposal. These included: 1) the construction of five new and the rehabilitation of nine outdated facilities that affected the ability of the institute to fulfill its mandate (Activities 1.3.1 and 1.3.2); 2) the purchase of basic office equipment, computers, laboratory equipment, vehicles, and motorbikes the priority programs needed to execute the proposed project activities (Activities 1.3.3-1.3.5); and 3) the construction and equipment of a new ICT unit (Activity 1.3.6).71 The project was unable to move forward with the original activity of, “procuring a modern MIS [Management Information System] to streamline administrative and management procedures” (Activity 1.3.7). It did, however: 1) successfully introduce and train staff in a new system of accounting software that increased their efficiency; and 2) address one of the most important cross-cutting infrastructure constraints not identified in the original proposal or first work plan—the institute’s weak internet connection—by building an unexpected collaboration with the Ghana Academic and Research Network (GARNET),which increased the internet connectivity of the institute’s main offices in Nyankpala. Despite these positive achievements, only 48 percent of the CSIR-SARI staff and 57 percent of the partner staff who responded to the online questionnaire either agreed or strongly agreed with the statement, “the project-built and project-improved infrastructure has benefitted agricultural research activities in northern Ghana.” Two of the most frequent reasons cited by the staff for their rankings were: 1) the actual number of investments was smaller than expected (14 investments versus the original target of 28) due to higher-than-expected costs and shifts in the administration’s priorities; and 2) most of the targeted investments in infrastructure development for the research out-stations (including improving the internet infrastructure) were never executed. Although the number of investments was smaller, 68 Dileepkumar Guntuku. 2016. Assessment of the Institutional Capacity and Needs of SARI. Ames: IFDC and ISU for USAID/Ghana (March 2016). Pg. 10. 69 The project’s close-out report describes these budget liquidation problems: “The poor understanding and adherence to USAID systems and operations as captured in the Program Implementation Letter affected drastically the burn rate. SARI, not used to the USAID advance and liquidation system, suffered many times from not getting advances as planned. On the average, advances were received about three times a year instead of the monthly. There were delays at USAID/Ghana’s level and CSIR-SARI level. This affected the decisions and activities of the project. Field activities of the project being seasonal in nature suffered most from these delays and impacted negatively on the success of some particular activities. This affected the implementation of annual plans.” (CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 38.) 70 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 11. 71 The first-year work plan envisioned the procurement of equipment and tools for the establishment/rehabilitation of a climate change, communication, and geographic information system (GIS) unit under Activity 1.3.6. Based on a shift in the institute’s priorities, these funds were reallocated to the construction and equipment of a state-of-the-art nutrition and food science lab. 18 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV these activities occupied the same percentage of the budget as was originally envisioned (Annexes V.B.2 and V.B.3). Some of the other complaints focused on the project’s insufficient investment in: 1) laboratory equipment; and 2) and staff training for basic maintenance and upkeep for the new and renovated infrastructure. Based on this triangulation of information, the ET concluded the project, by and large, achieved its original expected results for this output. SP 2: Agricultural technologies development and dissemination The activities under SP 2 were expected to help CSIR-SARI:  Output 2.1. Modernize its seed operations significantly in line with the Plant and Fertilizer Act;  Output 2.2. Scale up the pilot testing and dissemination of the improved ISFM practices for the new varieties being produced;  Output 2.3. Build the capacity of the institute’s technical staff to apply modern tools and techniques in research; and  Output 2.4. Continue to strengthen the RELC mechanisms that CSIR-SARI helped initiate as a way of more effectively connecting the MoFA extension staff with CSIR-SARI’s research. Two of the best quantitative measures of the project’s impact on accelerating CSIR-SARI’s capacity for agricultural technologies development and dissemination are (Annex II):  The development and release of eight new climate-smart varieties of seed with direct support from the project, as well as the development and release of another four varieties through a combination of support from this project and other donor-funded initiatives, bringing the total to 12 (133 percent of the original target of nine); and  The project’s dramatic (420 percent) overachievement of its original LOA target for new technologies, practices, and approaches under various phases of research, development, and uptake for the priority crops (80 targeted vs. 336 achieved) (Annex II). Seventy-seven (77) percent of the partner staff interviewed in FGDs/KIIs and all of the community-based stakeholders either agreed or strongly agreed the “project has increased CSIR-SARI’s support to the development and dissemination of improved agricultural technologies.” Although 55 percent of the CSIR-SARI staff interviewed agreed or strongly agreed with this statement, there was a general consensus that even this spectacular impact was much less than it should have been due to CSIR-SARI’s poor management of the project, which led to some of the activities critical to the achievement of the four outputs being delayed, canceled, or under-funded. Although the ET agrees this impact could have been greater with more effective management, it believes the project achieved a critical mass of the results expected under this SP (Table 2). Output 2.1: Seed operations modernized. One major achievement of the project was CSIR-SARI’s overachievement of most of its annual targets for the commercial production of both breeder and foundation seed (from 26.1 metric tons [MT] in 2015 to 102.7 MT in 2017 [150 percent of target], 127.4 MT in FY 2018 [182 percent of target] to 108.1 MT in FY 2019 [152 percent of target]). This represents 219 percent of the revised LOA target for this indicator (Annex II). Eighty-four (84) agreed or strongly agreed that the project’s investments under Objective 1—combined with the full execution of all ten expected activities under Output 2.1—helped modernize CSIR-SARI’s seed operations. Even though most staff agreed the project helped CSIR-SARI improve its compliance with the Ghana Plant and Fertilizer Act (2010), only 38 percent either agreed or strongly agreed with this statement because most of CSIR-SARI’s seed operations are still rainfed with limited mechanization, and there was very limited private-sector investment in either seed processing or cold storage (none of which were in the original proposal) (Annex V.A.1). 19 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Some of the major project-funded activities staff and project reports attributed these achievements were: 1) the collection, characterization, and evaluation of 500 diverse germplasm, thus creating the high genetic variability needed for breeding purposes; 2) staff training in the use of new laboratory equipment, statistical methodologies, and software for analyzing data; 3) exchange programs with international centers for breeders to acquaint themselves with modern breeding tools and methods; 4) the creation of SARI’s first seed research and production unit to address early generation seed (EGS) quality issues and coordinate EGS production; 5) training 4,000 farmers, seed growers, and seed companies that participated in the joint demonstration and adaptive trials; and 6) conducting 368 farmer field days during which the researchers explained the results of the trials to MoFA staff and the local communities. Eighty-four (84) percent of GoG and non-government partners who responded to the online questionnaire agreed or strongly agreed with the statement that the project increased CSIR-SARI’s capacity to support the development and dissemination of improved seed technologies (Annex IV.D.2). This was also one of the most frequently cited project impacts in the community-based interviews (Annex IV.B). Based on this triangulated feedback, the ET concluded that in spite of these deficiencies, the project achieved most of the original expected results for this output. Output 2.2: ISFM practices developed and disseminated. The project also achieved its original target (100 percent) for the development of three ISFM protocols for the three priority crops, which included, “appropriate (e.g. evidence-based) application, proper weed and pest control, proper land management and drainage, harvest, and post-harvest handling strategies as well as farm management,” being diffused to the key government and private sector actors (e.g., seed companies, agro-dealers) (Annex II). The development of the protocols, which were adjusted to the specific soils of each district, was facilitated by the project’s investment in training 468 MoFA agents (159 percent of the original target) who conducted 200 on-farm trials (21 percent of the original target of 97272) that trained 4,000 farmers, seed growers, and seed companies that participated in the joint demonstration and adaptive trials, and conducted 368 farmer field days for the new varieties that focused heavily on improved agronomic practices.73 The close-out report noted this research showed that the applications of the protocols, “increased grain yields of maize, soybean, and rice by 20 percent, 24 percent, and 45 percent, respectively, as well as increased [the] efficiency of fertilizers and other agro-inputs used by farmers.”74 In spite of this early evidence of success, only a quarter of the CSIR-SARI staff agreed or strongly agreed the project developed improved soil fertility practices (16-21 percent) or increased the dissemination of those practices (29-35 percent). The percentage of respondents who agreed or strongly agreed with this statement was higher for the project partners (48 percent for development and 64-71 percent for dissemination). The chief reason staff gave for these lower rankings was the actual promotion of the new techniques did not start until the final year of the project (FY 2019) (Annex II). Even with this limited promotion, the project’s final technology uptake study revealed 32 percent of the 18,617 farmers the project end-line survey were using the soil-related fertility and conservation practices recommended 72 This figure reports only the field trials directly co-monitored by the CSIR-SARI scientists working with MoFA agents. The actual number would probably be at least triple this number if it tracked the non-official (e.g. not directly supervised or monitored) demonstration trials trained AEAs conducted in other villages where they worked, based on feedback the ET got from the AEAs and community-based groups interviewed. 73 The project’s rapid scale up of the on-farm demonstration trials and farmer field days that started in FY 2015 is widely heralded by all CSIR-SARI staff and community-based stakeholders as an example of best practice. 74 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 30. 20 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV in the project’s ISFM protocol.75 One constraint often cited in interviews was farmers’ limited access to the soil-testing services, without which they cannot make informed investments in improved fertilizer and more intensive land management practices.76 Since there was no clear target or scale-up in the project proposal or work plan, and clear qualitative and quantitative evidence of potential impact (which was the expectation in the proposal), the ET concluded the project, by and large, achieved its expected results for this output. Output 2.3: Capacity of technical staff built. The activities under this output were designed to keep, “the staff of the institute….abreast…of new techniques/tools and modern trends in research and be able to apply (especially the new biotechnology tools) in addressing the needs of farmers and other stakeholders.” The project’s 2016 M&E plan did not, however, identify any specific targets for this training nor did the project report on it either annually or in the close-out report.77 Project records show 196 technicians were trained. This training included: 1) seven courses on accounting and procurement (32 technicians trained continuously over a five-year period); and 2) a biotechnology training component consisting of a baseline course in Accra (that six technicians attended) followed by a biotechnology tools course (that 20 technicians attended), which was postponed from FY 2015 to FY 2017. Most of the other training programs occurred in the project’s fourth and fifth year and focused on: 1) vegetable and seed production (four technicians); 2) foundation seed production (ten technicians); 3) basic computer skills (35 technicians); 4) using the BMS (35 technicians); 5) the establishment of demonstration trials (ten technicians); and 6) experimental designs and data analysis (no numbers given). Although most technicians interviewed reported some of the trainings were quite useful, only 19 percent of the CSIR staff interviewed in the FGDs/KIIs and 42 percent of the staff who responded to the online questionnaire agreed or strongly agreed the project-sponsored trainings, “built the ability of SARI’s staff to apply modern tools/techniques in research” (Table 2; Annexes IV. A.1 and D.1). The most frequent justifications for these rankings were: 1) other needed trainings were not offered; 2) there was no systematic plan or support for scaling up the training to other staff; 3) many technical trainings (like the molecular breeding tools training78) were supposed to have occurred in the first and second year of the project but did not occur until the final three years; 4) some technicians who got the training did not have access to the equipment needed to make use of it; and 5) the project never developed the type of short- and medium-term strategy for capacity-building envisioned in the project proposal and work plan. Based on this feedback and the project’s internal training records, the ET concluded the project was only partially successful in achieving expected results for this output (Table 2). Output 2.4. RELC mechanism improved. After the first year, the project discontinued its initial support for the nine activities for the regional and district-level RELC mechanisms because a Canadian￾funded project (Modernizing Agriculture in Ghana [MAG]) funded these. Although SARI did not provide any direct financial support to the RELCs after the first year: 1) MoFA records indicate the regional and district-level RELCs continued to meet regularly (Annex II); and 2) staff interviews confirmed many field station researchers and technicians continued to attend these meetings where they reported on new 75 Disaggregated PITT submitted with the FY 2016 annual report. 76 This constraint was also identified by two earlier projects: AGRA and the Sustainable Agriculture and Natural Resource Management (SANREM) Innovation Lab. Illustrative quotes from the KIIs and FGDs with project staff include: 1) “our soils lab— at the institute—needed a facelift. The face lift did not occur;” 2) “the current state of the lab is insufficient to support both research and dissemination;” 3) “the project did well on developing soil fertility practices [most of which had been developed on earlier projects] but the dissemination support was inadequate;” and 4) “we expected to see a massive improvement of techniques on the ground. In fact, however, we have not seen much change in local practices” (Annex IV.A.1). 77 The Activity Monitoring and Evaluation Plan (AMEP) PIRS for Outcome Indicator 1.2 (Standard Indicator EG 3.2-2) did not permit a disaggregated analysis of technician training since all CSIR-SAR staff were grouped with government stakeholders. 78 The technical training in molecular tools for breeding was originally scheduled for FY 2015. 21 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV technologies CSIR-SARI was developing and disseminating. It is not surprising, therefore, that 35 percent of CSIR-SARI staff and 57 percent of GoG and partner staff who responded to this question in the online questionnaire agreed or strongly agreed, “the project has helped SARI to improve the RELC mechanism for agricultural technologies dissemination,” even though the project was not (after the first year) providing any type of direct financial support (Table 2; Annex A.1). For this reason, the ET concluded that even though the project cut its support to the RELC after the first year, it partially achieved its originally expected results for this output. SP 3: Market and client-oriented research approach developed The activities under SP 3 were designed to strengthen CSIR-SARI’s relationships with the increasingly diverse array of agricultural actors identified in the original proposal (e.g., “farmers, MoFA, [other] government [institutions], other research institutions, development partners, implementation partners [non￾governmental organizations (NGOs)/private voluntary organizations (PVOs)] and, most importantly, private-sector actors in the agricultural industry,” 79) needed to scale up adoption of new technologies developed by CSIR￾SARI. The project was expected to achieve this result by:  Output 3.1. Developing the types of ICT and KMS needed for enhanced market-driven research for development;  Output 3.2. Strengthening the institution’s commercialization program; and  Output 3.3. Developing and operationalizing a robust M&E system for the institute. Two of the best indicators of the project’s impact on client adoption of the new technologies developed for the three priority crops were: 1) a 75-percent achievement of the original LOA target for: ”number of farmers and others who have applied new technologies or management practices” (Outcome Indicator 3.1. Annex II); and 2) an 86-percent achievement of the original target for “number of hectares (ha) under improved technologies or management practices” (Outcome Indicator 3.2, Annex II). Despite these positive achievements, only 30 percent of CSIR-SARI staff in FGDs/KIIs either agreed or strongly agreed “the project has helped SARI develop a more market and client-oriented research approach (Annex IV.D.2).” Sixty-five (65) percent of partner staff agreed with this statement (Annex IV.A.2). When asked to explain their rankings, CSIR-SARI staff responses focused on the project’s weak development of the commercialization unit (Output 3.2), which the institute depends on to support most of its operating costs (Annex IV. A.1). Based on the triangulation of this data, the ET concluded the project only partially achieved the expected results for this SP (Table 2). Output 3.1. ICT systems for development developed. Starting in 2015, the project supported or co￾supported (with ATT) all six activities80 identified as critical to the development of, “a public relations unit and a communication strategy that would enable it to engage with the public through the mounting of multi￾media campaigns (e.g. postings on a well-developed website, newsprint, magazines, and radio) that are integrated into a long-range strategic plan.”81 By the end of FY 2017, CSIR-SARI designated four senior staff 79 CSIR-SARI. Program Description. Support to CSIR-SARI (January 14, 2015–January 14, 2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 21. 80 The project confirmed execution through the budget figures and reports for the following activities: 1) Activity 3.1.1— establish and equip a technology information system to deliver information to stakeholders; 2) Activity 3.1.2—develop a communication strategy for the institute; 3) Activity 3.1.3—set up a public relations and communication unit; 4) Activity 3.1.4— develop communication tools to facilitate communication among the CSIR-SARI staff as well as to stakeholders; 5) Activity 3.1.5—engage consultants to develop a structure for knowledge management at SARI (including a website); and 6) Activity 3.1.6—organize seminars to sensitize staff on intellectual property rights to build scientists’ capacity to register their new technologies. 81 CSIR-SARI. 2015. Program Description. Support to the CSIR-SARI (January 14, 2015–January 14, 2020). Nyankpala: CSIR￾SARI for USAID/Ghana. Pg. 23. 22 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV for the unit, as well as three supporting staff and six national service personnel to oversee four components (public relations, knowledge management, information technology, and data management/library). The project’s support—combined with mentoring from ISU’s ATT media specialist also working as a consultant on the capacity-building grant CSIR-SARI developed with ISU—contributed to a substantial increase in the unit’s output (from 17 products in FY 2016, to 36 in FY 2017, 31 in FY 2018, and 38 in FY 2019) and the achievement of about half the original targets identified for the production of radio messages, TV broadcasts and spots, and exhibitions, and 80 percent of the original targets for leaflets and newsletters (Annex II). Although the project started helping the institute create its first website and online KMS in FY 2017, neither system was fully functional until the Soybean Innovation Lab (SIL) funded three field visits by its information technology (IT) specialist in 2018 and 2019 (Annex V.B.3). These field visits generated a proposal for redoing SARI’s IT networking infrastructure to make it fully functional (fiber and wireless), including equipping the institute’s main offices on the Nyankpala center and the remote field stations at Manga and Wa to connect to the internet via a virtual private network (VPN). This proposal was partially executed in 2019 and resulted in a dramatic increase in the internet connectivity of the ICT unit at the institute’s headquarters office, but was never extended to the field stations. Both the close-out report and most of the project partners interviewed in FGDs/KIIs said the project’s extensive investment in the development of CSIR-SARI’s first ICT unit, “resulted in a significant improvement in CSIR-SARI’s capacity to share information and communicate with partners.” The chief criticisms of CSIR-SARI staff interviewed were: 1) the persistent weakness of the connection, even at the main CSIR-SARI center; 2) field stations were never connected; and 3) high monthly cost, which is not covered by CSIR-SARI’s core GoG support Annexes IV. A.1 and D.1). Many also noted that although the project helped create CSIR-SARI’s first website and online library, neither system is user￾friendly and there is limited staff capacity for this type of high-level information systems support. Despite these identified issues, the ET concluded that the project achieved this expected result (Table 2). Output 3.2. Commercialization program strengthened. The activities under Output 3.2 were expected to help CSIR-SARI, “re-orient itself to meet the emerging requirements for engagement with the private sector” by achieving six “pathways” (or results) that the institute needed to achieve in order to “further foster private sector-driven research and technology generation to meet the needs of its core clientele.”82 There is no evidence from the close-out report or FGDs/KIIs that any of the six illustrative results the project proposal identified for this output were achieved. One of the best quantitative indicators of the project’s failure to build the capacity of its commercialization unit to sell CSIR-SARI’s services and products and manage the money generated by these sales, which was the core function of the commercialization unit, is “even though the technical team was able to increase EGS production by more than 82 Result: 3.2.1—Developing “a more comprehensive and formal institutional-level MOU to avoid the practice in which private contracts are given to scientists on a personal level [as] it is not advisable for SARI scientists to work outside the institutional arrangements;” Result 3.2.2—Establishing a “transparent fee for service mechanisms to attract the private sector to use the services of the institute in providing solutions to their research needs;” Result 3.2.3—Publicizing [the] “capabilities of SARI in providing [answers] to various research problems of the private sector through various multi-media channels;” Result 3.2.4— Permitting SARI to get “more involved in the activities and operations of the various appropriate partners to help them identify their research needs which the institute or any of its local and international partners could provide answers to;” Result 3.2.5— “SARI creating a discussion platform to encourage and engage the private sector to participate and invest in more[research and extension] activities;” and Result 3.2.6—“Developing a website for dissemination of research findings as well as soliciting feedback on its work…[as well as] information brochures and other ICT products on these technologies that agro-dealers and extension agents can use to promote the correct use of the technologies.” CSIR-SARI. 2014. Program Description. Support to CSIR-SARI (January 14, 2015–January 14, 2020. Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 22-23. 23 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV ten-fold and uptake of EGS by 98 percent, the project was not able to account for the increased internally generated funds (IGF) over the project phase due to some internal institutional weaknesses.”83 Only 9 percent of the 33 CSIR-SARI staff FGDs/KIIs and 23 percent of the 31 staff who responded to the online questionnaire agreed, “the project’s support has strengthened SARI’s commercialization systems—i.e., offering fee-based systems and goods (like improved seed) to stakeholders and others” (Annex IV.A.1). This opinion was echoed by the responses of the private-sector partners interviewed, one of whom concluded: “They have the technical capacity—and that has increased dramatically over the last five years. What they do not have are the strong systems for commercialization” (Annexes IV.A.2). Most of the CSIR-SARI staff and the project’s close-out report attributed a big part of the problem with this output to the lack of high-level support for addressing these internal institutional issues from either the SARI director’s office or the management board.84 Based on the triangulation of this data with the stakeholder’s feedback, the ET concluded that almost none of the original results for this output were ever achieved (Table 2).85 Output 3.3. M&E system developed and operationalized. The nine activities under Output 3.3 were designed to help CSIR-SARI develop an effective institute-wide, “M&E system (M&E plan, data gathering tools, PITT, special studies) to monitor the performance of indicators specified in the project document.”86 Although seven of the nine activities were executed, the institute never developed the type of “effective M&E system (M&E plan, data gathering tools, PITT, special studies) to monitor the performance of indicators specified in the program document” envisioned in the project description.87 Only 21 percent of CSIR-SARI staff in FGDs/KIIs and 35 percent of CSIR-SARI staff who responded to the online questionnaire agreed or strongly agreed that “the project’s support has strengthened SARI’s M&E systems” (Annex IV, Table 1; Annexes IV.A.1 and B.1). Most staff attributed the weak impact of the activities under this component to: 1) the senior administrators’ lack of ownership of M&E; which contributed to 2) the high rates of turnover in the staffing of this position on the project. Instead of recruiting a qualified M&E specialist to lead the activity (as anticipated in the project plan and budget) the project appointed a series of researchers who were already fully deployed to serve as acting M&E leads. This high level of turnover (three leads in five years) continued until one of the junior staff economists— and a group of cohorts (none of whom were trained under this project)—returned from graduate training to become the de facto CSIR-SARI M&E lead and focal points. Although there is a general consensus among all of the key actors interviewed (and the ET) that the project only partially achieved its expected output (i.e., the development and operationalization of a CSIR-SARI-wide M&E system), many staff reported the on-the-job training they received from 83 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 37 84 Ibid. Pg. 39. 85 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pp. 36-37. 86 Activity 3.3.1—hire consultant to establish a robust M&E system for the Institute; Activity 3.3.2—strengthen the capacity of staff on M&E system design and implementation and data base management; 3.3.3—hire consultant to evaluate CSIR-SARI’s research programs; Activity 3.3.4—undertake field monitoring visits; 3.3.5—organize and attend periodic review meetings; Activity 3.3.6—undertake a study to evaluation adoption and impact of existing technologies; Activity 3.3.7—undertake a study to examine the impact of climate change on farm productivity; Activity 3.3.8—undertake a study to examine the roles of gender in agricultural production systems; and Activity 3.3.9—establish effective commodity chains. CSIR-SARI. Program Description. Support to CSIR-SARI (January 14, 2015–January 14, 2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 15 87 CSIR-SARI. Program Description (January 14, 2015 to January 14, 2020) Nyankpala: CSIR-SARI. Pg. 15. 24 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV USAID/Ghana helped them better understand why CSIR-SARI needs more internal support for M&E.88 One of the best indicators of this longer-term post-project impact is the most recent update of the project’s five-year strategic plan (completed in early 2021) includes a detailed plan for the staffing and roll out of the type of institution-wide M&E plan this project was supposed to have produced.89 Based on this triangulation of information, the ET concluded that the project only partially achieved its original expectations for this component (Table 2). 4.1.2. CONCUSIONS In spite of the challenges, and the project’s limited capacity for managing a large donor-funded grant, the ET concluded the project (Table 2):  Achieved a critical mass of its expected targets (or results) for five of the ten expected outputs (Outputs 1.1, 1.3, 2.1, 2.2, and 3.1);  Eliminated the funding of one the output designed to build the capacity of the RELCs, which received funding from another source; (Output 2.4);  Achieved some of the targets—but at a lower-than-expected level—for two outputs (Outputs 2.3 and 3.3); and  Failed to achieve most of the expected targets for the final two outputs (Outputs 1.2 and 3.2). Four cross-cutting issues that negatively affected the project’s ability to achieve and document its progress toward the achievement of its expected targets were: 1) the project’s failure to implement the original plan for creating an independent management unit; 2) CSIR-SARI’s limited background in and capacity for the management of large donor-funded projects and USAID/Ghana’s Imprest budget transfer system; 3) the weak oversight and capacity of the CSIR-SAR’s commercialization unit; and 4) the project’s failure to implement the proposed model for the creation of a fully functional M&E system until its final months. 4.1.3. RECOMMENDATIONS Based on the ET’s analysis of from the FGDs and KIIs, project literature, and USAID/Ghana, the ET identified three clusters of recommendations for CSIR-SARI and four clusters of recommendations for USAID/Ghana. A complete list of recommendations and sub-recommendations for CSIR-SARI and USAID/Ghana is in Annex V.B.6. The recommendations pertinent to the issues raised under EQ 1 are summarized below. CSIR-SARI Recommendation 1. Project management systems.  Sub-recommendation 1.1. New donor-funded project designs. Ensure any future donor￾funded large projects (e.g., projects that represent more than 15 percent of the institute’s budget): - 1.1.1. Start-up and launch. Have a formal launch in the first few months of a project to ensure CSIR-SARI’s administrators, staff, technicians, and other key stakeholders have a good understanding of the project’s plan, work plan, and the donor’s rules and regulations for M&E, finance, and procurement. 88 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 36. 89 CSIR-SARI. 2021. Final Draft. CSIR-SARI Strategic Plan. 2021–2025. Nyankpala: CSIR-SARI. 25 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV - 1.1.2. Staffing. 1) Hire a manager with a strong management background, belonging to a reputable institute, with demonstrated experience in successful capacity-building of national agricultural research institutes to ensure appropriate mentoring; 2) use that person to monitor/coach the CSIR-SARI staff person who takes over the role once the recruited manager’s tenure expires; 3) provide project managers with full-time salaries or top-offs; 4) acknowledge the time, level of effort, and contributions of all institute staff involved in new donor-funded projects on their annual CSIR-SARI evaluations; and 5) if the CSIR-SARI staff are not eligible for top-offs (under new projects), consider what other compensations these individuals might be scheduled to receive (e.g., training, publication support, or sabbatical opportunities). - 1.1.3. Project steering committee. Have an empowered steering committee that meets regularly and includes a representative from the donor. - 1.1.4. Mid-term evaluation. Execute a mid-term evaluation to provide a participatory mechanism for CSIR-SARI and the donor to address any management, implementation, or governance issues that emerge in the first half of the LOA while there is still time to correct them.  Sub-Recommendation 1.2. Internal management, finance, and governance systems. - 1.2.1. CSIR-SARI management board. Strengthen the audit function of the CSIR-SARI management board. - 1.2.2. CSIR-SARI internal management committee (IMC). Ensure all of the managers of large donor-funded projects have a seat on the institute’s IMC. - 1.2.3. Mentoring. Require any future donor-funded project that is expected to mentor CSIR￾SARI on capacity to sign a memorandum of understanding (MOU) clarifying the type of mentoring support to be provided and how the mentorship activities will be monitored and reported to the CSIR-SARI administration, IMC, and board, as well as a representative of the project’s donor. - 1.2.4. Accounting. Continue to tighten the institute’s accounting system by using appropriate accounting software and ensuring regular audits at all levels by an independent auditor. Recommendation 2. Infrastructure and equipment. Develop better systems for cost-sharing or generating user fees to support the cost of routine maintenance and updating of infrastructure and equipment. Recommendation 3. Commercialization.  Sub-recommendation 3.1. Complementary private-sector investment. Advocate for private-sector investments in seed production infrastructure and then partner with CSIR-SARI to carry out research for them to replicate and commercialize.  Sub-recommendation 3.2. Supervision. Put the commercialization unit under the direct supervision of the CSIR-SARI director and management board.  Sub-recommendation 3.3. Commercialization unit budget line. Create a line budget within CSIR-SARI for commercialization linked toward the achievement of the unit’s business plan.  Sub-recommendation 3.4. Commercialization unit monitoring. Create a set of robust indicators that the institute’s top management and boards can use to track the unit’s progress toward the execution of its business plan.  Sub-recommendation 3.5. Code of conduct. Develop and enforce a professional code of conduct that clarifies CSIR-SARI’s support for plant breeders’ rights and any restrictions on research scientists and technicians setting up unrestricted side businesses to sell seed or other products. 26 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Recommendation 4. M&E.  Sub-recommendation 4.1. New project PITT. Require all new CSIR-SARI projects to include a standard PITT in all of their annual reports that includes both standard indicators (required by the donor) as well as custom indicators to track the achievement of their principal expected results and outputs.  Sub-recommendation 4.2. New project M&E mentoring. Require all new CSIR-SARI projects to anticipate the need for periodic support (on an annual basis during the LOA) to help the project understand the importance of M&E, setting reasonable targets, and measuring progress toward the execution of these targets in their annual report and specific donor’s rules about changing indicators and targets over the LOA.  Sub-recommendation 4.3. New project budgets. Require all new CSIR-SARI projects to have a dedicated M&E budget that must be returned to the donor if it is not spent and that has no possibility of being converted into funds that support other activities.  Sub-recommendation 4.4. M&E unit set-up. Designate one M&E coordinator and one M&E focal point for each field station, and encourage CSIR-SARI’s new and existing donor￾funded projects to help build their capacity through consistent mentoring by a professional M&E specialist (or contractors).  Sub-recommendation 4.5. M&E mainstreaming. Provide appropriate M&E baseline training to all senior and junior staff and administrators, and utilize online short courses and in￾house certification programs to help new and existing staff improve their proficiency. USAID/Ghana Management and Accounting Systems Recommendation A. Pre-conditions for large budget transfers. Make receipt of the large initial budget transfer associated with a government-to-government (G2G) capacity-building grant conditional on the beneficiary institution having a robust management and steering committee structure in place by the fourth month of the project, where a baseline assessment identified the need for major management reforms and the institution requested USAID/Ghana assist it in making these reforms to improve its organizational capacity. Recommendation B. Collaboration with other USAID/Ghana-funded initiatives. Require future G2G projects USAID/Ghana supports in northern Ghana to sign a detailed MOU with any other USAID/Ghana-funded project that is expected to mentor it (like ATT that mentored this project) that clarifies the type of mentoring support to be provided and how the mentorship activities will be reported to USAID/Ghana. Recommendation C. Mid-term evaluations. Require new G2G projects to include a detailed description of, and designated budget for, a mid-term evaluation to provide a forum to address any management, implementation, M&E, reporting, or governance issues that emerge in the first half of the project. Recommendation D. Budget transfers. Consider a more flexible system for forwarding money to G2G projects than the monthly Imprest system used under this project, or provide intensive mentoring to the beneficiary institute in these systems if it has never used them. 27 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV 4.2. EQ 2: IS THERE EVIDENCE OF IMPROVED ORGANIZATIONAL DEVELOPMENT AND A STRENGTHENED INSTITUTION OF SARI (INCREASED RESEARCH CAPACITY) AND HAS THIS RESULTED IN IMPROVED WELFARE OF SMALLHOLDER FARMERS IN NORTHERN GHANA? 4.2.1. FINDINGS The evaluators based their assessment of EQ 2 on: 1) the project’s close-out report; 2) data generated by CSIR-SARI to track the standard Feed the Future and custom indicators in the PITT (Annex II);90 and 3) feedback from the FGDs/KIIs with community and non-community-based stakeholders. EQ 2.A. Has SARI increased its capacity in key areas that affect community outreach?91 Five of the best quantitative indicators of CSIR-SARI’s increased capacity in key areas that affect community outreach are (Annex II): 1. A substantial increase (from 20 to 336 in 201892) in the “number of technologies or management practices under research, under field testing, or made available for transfer.” 2. The accelerated roll out (and approval) of 12 new climate-smart varieties of seed for the three priority crops (maize, rice, and soybean) (133 percent of target)—eight of which were produced with direct support from the project, plus another four produced and released through a combination of support from the project and other donor-funded initiatives. 3. The successful pilot testing, review, and adaptation of soil-specific protocols for priority crop ISFM (100 percent of target), currently being scaled up (post project) through MoFA and the private sector. 4. A substantial overachievement (219 percent) of the original target for seed production. 5. The progressive pilot testing and scale up of a new project-funded initiative to train MoFA AEAs in the new CSIR-SARI technologies to facilitate MoFA’s collaboration in the design and execution of the institute’s on-farm demonstration and adaptive trials and field days, which have emerged as a critical pipeline for communication about CSIR-SARI’s new technologies to both farmers and the private sector (156 percent of target for AEA training and 99 percent of target for the trials). The project’s close out report notes: 1) “more than 4,000 smallholder farmers, seed growers, and seed companies participated in the project’s field activities and research and agronomic practices;” 2) “6,000 farmers benefitted directly from the agricultural training and education including soil health management, agronomy of food crop production, and marketing;” and 3) “the number of stakeholders who sought for the institute’s services within this period (2015–March 2020) quadrupled.”93 Another qualitative indicator frequently cited by both the project close-out report and many community and project partner stakeholders is the project’s successful co-mobilization (with MoFA) of a comprehensive public and farmers’ education program about the Fall Army Worm (FAW) control and management in 2016.94 90 The project PITT included one outcome and four output indicators for SP 1 (increased organizational capacity) and two outcome and six output indicators for SP 2 (agricultural technologies developed and disseminated) (Annex II). 91 Although these sub-questions were not in the approved SOW for the evaluation, they are added to clarify the link between the text and the EQ. 92 The original target was 80 technologies; the actual achievement of the project—working in close collaboration with the other partners—was 336 (420 percent of target) (Annex II). 93 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana Pg. 32. 94 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID-Ghana. Pp. 17, 31. 28 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Sixty-seven (77) percent of CSIR-SARI staff and 72 percent of partner staff who participated in FGDs/KIIs either agreed or strongly agreed the project substantially increased CSIR-SARI’s “connections with farmers.” There is a strong consensus between the different categories of stakeholders and the project’s close-out report that this was achieved by:  Helping the institute respond to the increased demand for breeder seed galvanized by the Planting for Food and Jobs (PFJ) Initiative’s subsidies.  Supporting the core costs of scaling up its collaboration with MoFA on: 1) demonstration and adaptive on-farm trials and farmer field days; and 2) technical training for the rapid growing of small-scale and larger-scale commercial certified seed producers.  Helping CSIR-SARI create its first ICT unit and scale up its media programs that shared the benefits of the new technologies and the ISFM practices needed to realize the full return of investing in improved seed and agronomic practices.95 Strengthening the direct communication between a wide variety of community-level actors— including the MoFA field agents, agro-dealers, National Seed Trade Association of Ghana (NASTAG) members (which include the major commercial producers), and Plant Protection and Regulatory Services Directorate (PPRSD) units—and CSIR-SARI scientists in its three principal field stations. EQ 2.B. Is there evidence that this increased capacity has improved the welfare of smallholder farmers? Technology adoption. Eighty-eight (88) percent of the 25 partner staff interviewed in FGDs/KIIs,96 71 percent of CSIR-SARI staff who responded to the online questionnaire, and all four leaders associated with community-based groups who participated in the project-supported on-farm demonstrations and adaptive trials survey either agreed or strongly agreed the project—in combination with the PFJ Initiative—increased farmers’ access to improved seed and improved technologies (Annexes IV.D.1 and B). Three of the best quantitative indicators of the project’s impact on smallholder farmers are its: 1) seventy-five (75) percent achievement of the original target of the number of farmers and others who applied the new seed and agronomic practices promoted by the project; 2) sixty-seven (67) percent achievement of the number of households benefitting from the project’s activities; and 3) eighty-eight (88) percent achievement of the original target for the number of hectares under the improved CSIR￾SARI technologies for seed and agronomic practices (Annex II). These population-based achievements are all the more remarkable given the project’s budget liquidation issues, which directly affected the total number of individuals who benefitted from the project’s short-term trainings by 70 percent (from the target of 49,094 to 14,794; Annex II). In spite of these issues, the project’s close-out report concluded the institute‘s increased production of EGS for the three target crops (from 29.7 MT in FY 2017 to 71 MT in FY 2019, Annex II) was greatly helped by this project. It led to a “significant improvement in accessibility to foundation seed by certified seed producers,” which, in turn, based on data from ATT, “increased the community-level demand for and utilization of certified seed from 14 percent in 2014 to 35 percent in 2019.”97 The project’s 2016 end-line 95 This included facilitating 20 radio programs, 17 television programs, 20 news articles, 41 leaflets, 16 newsletters, and eight exhibitions (Annex II). 96 Eighty-six (86) percent of staff who responded to the post-interview questionnaire also agreed or strongly agreed. 97 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID-Ghana. Pg. 9. 29 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV technology uptake survey confirmed this information by showing the rate of improved seed adoption increased but was still below 40 percent for maize and soybean, but higher for rice. The same survey, however, showed only 10 percent of maize farmers in the sample adopted drought-tolerant maize varieties, which the project attributed to, “the misconceptions about hybrid seed in general and the high rate of complementary input associated with the use of hybrid.” The disaggregated PITT prepared in conjunction with the FY 2019 annual report showed:98  “Use of tractor for land preparation and other activities is more noticeable (above 63 percent) for all households in the sample.”  “Fertilizer use [was] low among soybean farmers but relatively high among rice and maize farmers.”  “[Fifty] 50 percent of the soybean farmers use inoculum to improve nitrogen fixation. This may explain the low use of mineral fertilizer among soybean farmers.”  “Use of organic fertilizer (manure) is extremely low (2 percent) for rice farmers but more than 27 percent for maize and soybean farmers.”  “More than 40 percent of the farmers use herbicide to control weeds.”  “About 88 percent of maize farmers practice row planting, while 42 percent practice maize legume rotation.”  Many farmers increased their use of mechanization in FY 2019: 1) Forty-six (46) percent of rice farmers were using bunds to manage water in their fields; 2) Forty-seven (47) percent of maize farmers used shellers; and 3) thirty-one (31) and 6 percent of rice and soybean farmers, respectively, interviewed reported using threshers. Yields, income, and farmer welfare. The project’s close-out report concluded using the proposed package of ISFM practices and improved seed (appropriate fertilizer application, proper weed and pest control, proper land preparation and drainage, harvest and post-harvest handling strategies, and farm management) was associated with, “increased grain yields of maize, soybean, and rice by 20, 24, and 45 percent, respectively, as well as increased the efficiency of fertilizer and other agro-inputs,” 99 for direct￾beneficiary farmers. At the same time, the project’s disaggregated 2019 PITT highlighted that these protocols were still in an early stage of rollout and adoption and required additional support to realize the wider impacts. Even with the late roll-out of new ISFM protocols, two of the project’s custom indicators (based on MoFA data) showed a modest increase in the average yields and percent of post￾harvest crops for the targeted crops in northern Ghana between 2015 and 2018 (for yield) and between 2015 and 2019 for post-harvest losses (Annex II).This includes (Annex II):  Outcome Indicator 1: A marginal increase in the average yield for all of the crops: 1) from 1.92 MT/ha at baseline to 2.26 MT/ha in 2018 (no data yet for 2019) for maize; 2) from 2.75 MT/ha at baseline to 2.96 MT/ha in 2018 for rice; and 3) from 1.65 MT/ha at baseline to 1.72 MT/ha in 2018 for soy.  Outcome Indicator 2: A marginal decrease in the percent of post-harvest losses for the targeted crops: 1) from 20 percent at baseline to 15 percent for maize in 2019; 2) from 14.4 percent at baseline to 12 percent for rice in 2019; and 3) from 15.6 percent at baseline to 12.5 percent for soy in 2019. 98 Prince Maxwell Etwire 2019. CSIR-SARI Technical and Financial Support Project Disaggregated PITT (for the USAID Standard EG indicators). 99 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID-Ghana Pg. 30. 30 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV One would assume, therefore, that if adoption rates could be increased in the future that, all other things being equal, it would increase smallholders’ income, resilience, and welfare. Sixty-eight (68) percent of project partner staff and the four community-based group leaders agreed or strongly agreed this increased access to improved seed and training they received also increased the direct beneficiaries’ income (Annexes IV.A.1. and IV.B). A few noted, “since the new varieties are more drought tolerant, this [also] increased their resilience.” Although 70 percent of CSIR-SARI staff agreed the project helped increase farmer welfare, they also noted the project’s impact on income and farmer welfare was, “a little difficult to know,” since this impact was not tracked for all groups by the project’s M&E system. The lack of tracking data with which to assess these community-level impacts was the principal reason most CSIR-SARI scientists and technicians refused to weigh in on the community-level impacts of the program (Annex IV. A.1. Questions 61-65). The same FGDs/KIIs with CSIR-SARI staff, government and non-government partners, and community￾based stakeholders identified six major challenges that limited the community-level impact of the project and would require CSIR-SARI to strengthen its collaboration with other GoG and donor-funded projects to address: 1) limited scale up of CSIR-SARI’s innovative collaboration with MoFA on the design, execution, and assessment of demonstrations and adaptive trials; 2) MoFA and PPRSD regional staff’s limited knowledge about appropriate production practices needed to get the maximum benefit from the improved seed and ISFM practices; 3) the weak capacity of many farmers (and district governments) for marketing their increased production through simple supports like requiring traders to use standardized measurement tools; 4) the difficulty seed-producing farmers had in purchasing high￾quality foundation seed for producing certified seed without traveling to the CSIR-Crops Research Institute (CRI) in Kumasi or CSIR-SARI home office in Nyankpala; 5) high cost of credit for the other technologies farmers needed to support sustainable increases in crop productivity and income (e.g., tractor-based land preparation services and fertilizer for the average farmer, and improved cold storage and processing for the commercial seed producers); and 6) whether or not farmers would continue to adopt the improved seed once the PFJ initiative no longer subsidized the cost of seed. 4.2.2. CONCLUSIONS There is both qualitative and quantitative evidence the project’s support for basic infrastructure, joint field trials with MoFA—combined with its support for building and equipping a new ICT center and co￾training (with ATT) CSIR-SARI’s first ICT coordinator—increased CSIR-SARI’s capacity for extension and ICT activities that promoted new seed varieties and improved production techniques for rural farmers. The community-based interviews conducted by the ET with farmers and MoFA AEAs confirmed the project’s internal tracking data (as well as the data from ATT) that showed this increased access to improved seed, combined with the project’s highly successful collaboration with MoFA on scaling up on￾farm demonstration and adaptive trials and community-based farmer field days, has the potential to increase farmers’ yields, income, and resilience for the direct beneficiary farmers. The community-based interviews underscored the critical importance of CSIR-SARI strengthening its collaboration with the next generation of GoG and donor-funded projects (including those funded by USAID/Ghana) to address some of the six challenges likely to affect a more broad-based scale up of newly developed technologies. 4.2.3. RECOMMENDATIONS CSIR-SARI Recommendation 5. Technology dissemination and scale-up. 31 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV  Sub-recommendation 5.1. Co-execution. Encourage new and existing donor-funded projects that collaborate with CSIR-SARI in northern Ghana to co-fund and co-execute joint technical and mentoring activities on issues like pest monitoring, soil mapping, training, on-farm trials, and extension.  Sub-recommendation 5.2. CSIR-MoFA collaboration. Identify ways other donor-funded projects can help scale up CSIR-SARI’s successful collaboration with MoFA for AEA training, field trials, and field days.  Sub-recommendation 5.3. MOUs. If projects decide to co-execute or collaborate, encourage them to formalize this collaboration through MOUs, and monitor and report on their collaboration with CSIR-SARI in order to avoid duplication and strengthen synergies.  Sub-recommendation 5.4. Partner de-briefings. Ensure the regional MoFA and PPRSD staff (as well as any donor-funded project they work with) get regular debriefings on CSIR￾SARI’s current research endeavors and receive all annual reports and work plans connected with any joint programs they support.  Sub-recommendation 5.5. Training. Pilot test the feasibility of offering a limited number of fee-based training programs for key program partners (PPRSD, MoFA, NGOs, and donor-funded projects). Recommendation 6. ICT.  Sub-recommendation 6.1. ICT repository. Develop a central online repository for all CSIR-SARI’s ICT materials over the last ten years so they can be easily scaled up to new and existing projects.  Sub-recommendation 6.2. ICT monitoring. Develop gender-sensitive assessment tools for tracking the impact of different outreach methodologies (e.g., portable video communication tools and community-based theater). USAID/Ghana Recommendation E. Fee-based training. Encourage future USAID-funded projects in northern Ghana to access fee-based training programs for their staff and/or support trainings for project partners. Recommendation F. Collaboration with MoFA. Identify ways future USAID-funded projects can help scale up CSIR-SARI’s successful collaboration with MoFA for AEA training, field trials, and field days. 4.3. EQ 3: WHAT UNINTENDED CONTRIBUTIONS, RESULTS, AND/OR OUTCOMES HAVE THE PROJECT APPROACH AND ACTIVITIES ACHIEVED RELATIVE TO TRANSFORMING CSIR-SARI INTO A CENTER OF EXCELLENCE FOR RESEARCH AND DEVELOPMENT IN THE SAVANNAH AGRO￾ECOLOGICAL ZONE? The ET based its assessment of EQ 3 on: 1) stakeholder perspectives of unexpected contributions, outcomes, and outputs—both positive and negative; 2) a comparison of the original project model and first five-year implementation plan with what was implemented based on the project’s tracking data, final budget report, and annual and close-out reports; and 3) an analysis of the stakeholder feedback and close-out report, which describes the impact of these different unexpected outcomes on the project’s results. 4.3.1. FINDINGS Unexpected Outcomes That Had a Positive Impact on CSIR-SARI’s Transformation 32 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV  The most frequently cited “unexpected positive outcome” of the project in the FGDS/KIIs (42 percent of the responses to this question) and echoed in the final close-up report100 was the profound impact of the project’s investment in certain types of infrastructure CSIR-SARI needed to be more demand driven and responsive to new needs. Two of the three most frequently cited key investments (the state-of-the-art nutrition lab and the internet improvements) were not envisioned in the original proposal.  A second unexpected outcome widely featured in the project documentation101 and in the PITT (Annex II) was the project’s spectacular over-achievement of its original goals for producing EGS in response to the increased demand for certified and foundation seed stimulated by the GoG’s 2016 PFJ initiative (26 percent of the 38 responses to this question).  A third unexpected outcome reported in both the FGDs/KIIs and FY 2016, FY 2017, and final close-out reports was how the project helped the CSIR-SARI Entomology Unit develop technical solutions to manage FAW, and provided training to the frontline MoFA staff on how to identify and control the pest’s various stages, which played a valuable role in the community￾based pest management.  A fourth widely documented positive impact was the strong partnership that emerged with the University of Illinois, Urbana Champaign (UIUC)-based SIL (which was not listed in the original list of project partners) that helped strengthen the execution and impact of this project by: 1) complementing its research and infrastructure support with additional technical support; and 2) hosting the international soybean trials in 2020.102 The same collaboration helped conceptualize a cost-effective solution to CSIR-SARI’s weak internet connectivity, which strengthened both the project and the institute as a whole (see Annex IV.B.3). Unexpected Outcomes That Had a Negative Impact on CSIR-SARI’s Transformation Seventy-eight (78) percent of the unexpected outcomes identified in the FGDs/KIIs focused on the project’s weak management and the difficulty USAID/Ghana and the senior researchers associated with the project faced in trying to fix these management issues during the project’s last three years (Annex IV.F.1). The same interviews identified a number of factors that affected the project’s willingness and ability to address these management issues:  CSIR-SARI’s weak management capacity at both the director and management board levels.103  The weak understanding of the CSIR-SARI administration and senior staff of the project proposal and USAID/Ghana’s rules and regulations for procurement outlined in the PIL during the first year of the project.  Even though this weak management capacity was identified in the 2011 baseline assessment, the project plan did not anticipate any sort of structured handholding by another USAID-funded project. 100 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pp. 21-24. 101 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pp. 9, 20-30. 102 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 24. 103 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pp. 38-40. 33 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV  Even though project records showed funds were disbursed for the budget items for the IMC meetings, neither the CSIR-SARI management board nor its IMC ever got the management and M&E training anticipated in the project plan nor received a comprehensive briefing on the project until its very end, so none of them had sufficient knowledge about what the project was about or the difficulties it was experiencing.104  USAID/Ghana did not agree to fund the type of external mid-term evaluation that was envisioned in the original design.105 January 2017–December 2018. In an effort to fix the project’s weak management start-up, USAID commissioned ATT to conduct a full management assessment.106 This assessment led to a series of unexpected changes in the program execution model deemed necessary to achieve its expected results. The first was narrowing the focus of the project to a smaller group of staff, hand-selected by the CSIR￾SARI administration into six component teams referred to as a COE for the priority value chains in order to accelerate the impacts the project could have on the research, extension, and commercial seed activities in its remaining time. The second was supporting the development of a public-private partnership (PPP) with an international seed company based in India to help the institute develop a more reliable stream of revenue to ensure the execution of its research programs. The third was co-funding, with ATT, a formal launch of the new COE for CSIR-SARI in June 2017 to familiarize all of the project stakeholders with the new model and, in conjunction with the re-launch, signing a contract with ISU to assist the project with the new model’s implementation in July 2017 (Annex V.B.5). While there is clear evidence from the project’s annual reports that this reorganization helped accelerate the effective execution of many of the original trainings and capacity-building programs in FY 2018 and FY 2019, the project was still running behind on its burn rate (i.e., its expenditure of project funds) and still had $3.1 million in project funds (62 percent of the original grant) in November 2019. January 2019–March 2020. To address this issue, the consultant working with the staff encouraged them to consider the development of a PPP with a large Indian seed company that was expected to provide CSIR-SARI with a more reliable source of commercial revenue to support its research and breeding activities. Once this decision was made, the project staff asked permission to modify the already approved FY 2019 work plan in order to have the necessary funds to contribute to the PPP, which led to many of the activities that had been approved for the last year being cut out of the budget at the last minute. In addition, the project applied for a one-year extension to complete the execution of the PPP proposal. This extension was not, however, approved because the USAID/Ghana audit report showed:107 “A lack of structure and transparency in the accountability of the commercialization unit. Some examples include: 1)CSIR- SARI could not account for the proceeds of some of the activities under the commercialization unit, the proceeds [of which] were supposed to be used as a revolving fund for the first two years of its establishment; 2) CSIR-SARI’s partnership with the Integrated Water 104 These points were raised in the interviews with IMC and management board members and verified by the ET through a review of a sample of the minutes from the two boards. 105 “Consistent with the new USAID evaluation policy, impact and performance evaluations will be periodically conducted on Feed the Future-related programs in a relatively structured analytical effort to answer specific program management questions.” (CSIR-SARI. 2014. Program Description. Support to CSIR-SARI [January 14, 2014–January 14, 2020]. Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 25). 106 Dileepkumar Guntuku. 2016 (March). Assessment of the Institutional Capacity and Needs of SARI. Ames, Iowa: IFDC and ISU (Final version with corrections dated April 25, 2016). 107 This information on the audit is based on written feedback from the project Agreement Officer’s Representative (AOR) because the ET was not given a copy of the audit report to review (November 12, 2021). 34 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Management and Agricultural Development Ghana Limited (IWAD) (a private enterprise) to produce EGS ended in indebtedness to CSIR/SARI; and 3) the Canadian MAG seed production contract with CSIR-SARI ended with the institute receiving all the funding, but was unable to produce and deliver the required seeds for two subsequent years.” Based on these results, both USAID/Ghana and some senior CSIR-SARI administrators felt CSIR SARI did not have the capacity to build and sustain the type of high-level PPP that was being proposed and recommended the project be closed as planned with an unspent balance of $1,966,163.20 (35 percent of the grant). 4.3.2. CONCLUSION The ET found evidence of two categories of unexpected outcomes and contributions that had a dramatic impact on CSIR-SARI’s transformation into the type of regional Center of Excellence that was originally envisioned. This includes:  New activities that strengthened the project’s achievement of its three sub-purposes and CSIR￾SARI’s core mandate to its local stakeholders; and  CSIR-SARI’s weak management capacity—both at the director and management board levels— contributed to: 1) its failure to execute its original program plan for management, M&E, and commercialization; and 2) the project’s weak capacity to expend the project funds at the rate anticipated by the original project description and PIL (e.g., its low burn rate). 4.3.3. RECOMMENDATIONS CSIR-SARI Recommendation 1. Project management systems. The sub-recommendations for improving CISR-SARI’s project management systems (1.1-1.2) were detailed in Section 4.1.3. USAID/Ghana Recommendations A-D. Management and accounting systems. Recommendations for future project’s management and accounting systems (A-D) were detailed in Section 4.1.3. 4.4. EQ 4.A) TO WHAT EXTENT HAS THE PROJECT ADDRESSED GENDER ISSUES IN RELATION TO CAPACITY-BUILDING IN RESEARCH AND DEVELOPMENT, AND HAS THIS HAD AN IMPACT ON GENDER MAINSTREAMING IN AGRICULTURAL RESEARCH? 4.B) WHAT ARE THE LESSONS LEARNED AND BEST PRACTICES, AND FROM WHICH STAKEHOLDERS OR BENEFICIARIES, IN ACHIEVING RESULTS? 4.4.1. FINDINGS EQ 4.A. To what extent has the project addressed gender issues in relation to capacity￾building in research and development, and has this had an impact on gender mainstreaming in agricultural research? The ET based its assessment of EQ 4.A. on: 1) the extent to which the gender issues associated with capacity-building in research and development were identified and addressed in the original project plan, work plans, annual and close-out reports, and the PITT; 2) any quantitative or qualitative evidence (from the FGDs/KIIs) the project addressed gender issues in the execution of the activities associated with the ten project components designed to achieve the three sub-purposes; and 3) any evidence from the FGDs/KIIs and project reports that these activities had an impact on gender mainstreaming in CSIR￾SARI’s agricultural research. 35 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Project Plan, PIL, Work Plan, M&E Plan, and Reports The 2011 baseline capacity assessment identified the lack of gender diversity in the CSIR-SARI staff and management board as a critical constraint the G2G project being designed should consider.108 In spite of this, the approved project plan does not include a single reference to gender, any gender strategy, or gender targets.109 There was also no mention of gender as a cross-cutting objective of the project in the project’s first logical framework, which was included in the M&E plan submitted with the proposal.110 USAID/Ghana’s PIL did, however, include the following text: “The Gender Analysis (2011) conducted for Ghana’s Country Development Cooperation Strategy (CDCS, 2013–2017) was used to inform the Activity design. Findings of the Gender Analysis pertinent to this Activity is the lack of a gender lens in formulating agricultural policy and the lack of women representation in the policy making process. To address these constraints, this Activity will ensure: (1) Adequate representation and participation of women in the hiring of staff; (2) Adequate representation of women in both short and long-term trainings; (3) Formulation of agricultural policy that takes into account gender-based constraints; (4) Data collection that include gender-disaggregated data; and (5) Research topics and agricultural policy recommendations that address gender-based constraints.” 111 Although the issue of gender mainstreaming was added to the logical framework of the project in the revised M&E plan submitted 60 days after the PIL was signed, there was no mention of gender targeting or any strategy for addressing gender other than anticipating the need for gender disaggregated reporting.112 The project’s first-year work plan did not, however, include gender as a cross-cutting sub￾purpose in either the results framework or the list of activities.113 Only one work plan activity in the five-year plan under Output 3.3 (M&E system developed and operationalized) focused on gender, Activity 3.3.8 (Undertake a study to examine the roles of gender in agricultural production). The activity description says the study was expected to show how, “The role of gender in [the] agricultural production system can inform targeting of technology development and dissemination,” and generate a database for this study in Year One.114 Unfortunately, this study was not finalized until January 2019 during the first quarter of the final year of the project, which limited its utility.115 108 John Nene-Osom Azu and Kwesi Opoku-Debrah. 2012. Assessment of SARI, Nyankpala, Northern Region, Ghana. Accra: Africa Lead for USAID/Ghana. Pg. 13. 109 CSIR-SARI. 2012. Program Description. Support to CSIR-SARI (January 14, 2015–January 14, 2020). Nyankpala: CSIR-SARI. 110 The only mention of gender is in the context of gender disaggregation. “The data management system proposed in this M&E plan takes into consideration the disaggregation of data by sex, location, income, and age of the beneficiaries where applicable. This will enable CSIR-SARI to track and report to USAID/Ghana and other stakeholders on the outcome and impact of the USAID support accruing to different categories of people in northern Ghana.” CSIR-SARI. 2013. Monitoring and Evaluation Plan. Support to Council for Scientific and Industrial Research—Savannah Agricultural Research Institute (CSIR-SARI). Nyankpala: CSIR-SARI M&E office for USAID/Ghana. Pp. 3 and 7. 111 USAID/Ghana. 2014. Project Implementation Letter NO. 641-A18-FY14-IL#03 between CSIR-SARI and USAID, Accra, Ghana. CSIR-SARI. Accra, Ghana: USAID/Ghana. Pg. 10. 112 CSIR-SARI. 2015. Monitoring and Evaluation Plan. Support to Council for Scientific and Industrial Research—Savannah Agricultural Research Institute (CSIR-SARI). Nyankpala: CSIR-SARI M&E office for USAID/Ghana. Pp. 4 and 8. 113 CSIR-SARI. 2015. CSIR-SARI Technical and Financial Support Project. Workplan Outline Year One (January 14, 2015– December 31, 2020). Nyankpala: CSIR-SARI for USAID/Ghana. 114 CSIR-SARI. 2015. CSIR-SARI Technical and Financial Support Project. Workplan Outline Year One (January 14, 2015– December 31, 2020). Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 36. 115 CSIR-SARI. Direct Support Project. 2019. Gender Roles in Agricultural Production systems in Northern Ghana. Nyankpala: CSIR-SARI (January 2019). 36 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV CSIR-SARI already had a strong institutional commitment to tracking the participation of women in its training and research programs. The M&E officers were considered the de facto gender focal persons for this project and the wider institute. It is not surprising, therefore, that not one of the project’s annual reports includes a sub-section focused on gender mainstreaming, nor is it surprising it was also not a priority in the close out report, although the project did comply with USAID/Ghana’s rules and regulation for disaggregated reporting on the six standard Feed the Future indicators. SP 1: Capacity Development for Agricultural Research in Northern Ghana Output 1.1. SARI’s core scientists trained and re-tooled in key program areas. Only 13 percent of the 60 CSIR-SARI staff who were identified as “core” participants in the project and seven percent of the 120 staff identified as “non-core” staff in 2020 were women. Although the senior CSIR-SARI management states it is committed to the recruitment and retention of women, and encourages junior female staff to get the graduate degrees they need for promotion, only three of the 17 (18 percent) junior staff who recently completed PhDs and three of the 30 (10 percent) staff who recently completed their MSs with funding from various donor-funded projects and the GoG were women. The project did very little to redress this inequity through long-term training, as the number of post￾graduate scholarships this project supported was reduced from nine to three, with only one of the three given to a woman.116 The project did, however, make a concerted effort to provide both management and other types of leadership training to the senior women who already had their graduate training or were completing it. In 2016, the COE reorganization identified 12 leaders (on nine themes), two of whom were women—one a recent PhD and one just starting her PhD. A third female staff member was added to the COE team after completing her PhD,117 bringing the total of core component team female members to 25 percent. The project’s internal tracking data shows women were:  Twenty-five (21 percent) of the 92 senior CSIR-SARI staff and technicians trained.  Seventeen (9 percent) of the 196 MoFA agents trained at the main office of CSIR-SARI in Nyankpala.  1,615 (26 percent) of the 6,156 participants in the community-based field trainings (i.e., adaptive and demonstration trials and farmer field days).118  One (17 percent) of the six staff who benefitted from the two-week, project-sponsored exchange visit for COE leaders to ISU and the UIUC in early 2017.  One (50 percent) of the two people on project-supported visits to Consultative Group on International Agricultural Research (CGIAR) research centers to get germplasm and the International Maize and Wheat Improvement Center (CIMMYT) research facility in Zimbabwe.  Seven (9 percent) of the 80 staff who participated in the project-sponsored statistics training.  Nine (38 percent) of the 24 staff who participated in the training and technical backstopping on USAID rules and regulations and procurement procedures.119 116 USAID usually targets 40 percent for females in order to mainstream gender issues into development in its development projects. 117 Obtained from the Core of Excellence Update. Nyankpala: SARI with inputs from ISU, ATT, USAID, and SARI teams. Slide 10, PowerPoint presentation. 118 Disaggregated PITT prepared as part of the FY 2019 annual report. 119 Project databases, June 15, 2021. 37 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV  Nineteen (23 percent) of the 82 technicians who benefitted from other short-term trainings.120  One (33 percent) of the three staff who were assisted with long-term training. Output 1.2. Increased organizational quality and efficiency of SARI. Although the project was expected to help SARI develop and implement a long- and short-term staff capacity development plan, the description of this plan (Activity 1.2.7, which was never completed) did not list gender equity as one of the key considerations. Neither the 2017 ATT-funded COE design, which provided the basis for this project’s work plans starting in June 2017,121 nor CSIR-SARI’s second official strategic plan, which was funded by this project (Activity 1.2.5, Output 1.2), completed in 2017 and finalized in 2019,122 nor the most recent (2021) draft revision of the 2019 five-year strategic had a section that focused on the most critical gender issues related to staff or the institute’s capacity for addressing gender in its research programs. SP 2: Agricultural Technologies Development and Dissemination Although the program plan and PIL expected the project to target women in the field trials and training programs, only 26 percent of the 6,156 producers who received short-term training from the project and 13 percent of government workers the project trained were female.123 The same project records show that by the end of FY 2018 (the last year this data was disaggregated in the PITT), only 3 percent of the 70 private-sector individuals trained were women.124 Unfortunately, it is not possible from this break out to determine how many of the AEAs trained were female since the current M&E staff were unable to locate the list of AEAs trained. When the MoFA AEAs were asked to compare the project’s commitment to gender inclusion in the trials with the commitment of earlier trials, they said that while earlier projects encouraged women to participate in the field trials, this project was especially insistent (Annex IV.A.2 and IV.B). However, this was hard to document with the project data since neither CSIR￾SARI nor MoFA developed a master data set on the field trials. SP 3: Market and Client-oriented Research Approach Developed Output 3.1. ICT. Although there is anecdotal evidence from some of the media and AEA interviews that the project’s scale up of ICT programming increased women’s access to information about new technologies in ways that increased their living standards, this was hard to document since CSIR-SARI’s first ICT strategy did not include a sub-component of activities focused on women or youth and had no mechanisms for monitoring impact. Output 3.2. Commercialization program. Although the project-supported business plan (completed in March 2019, one year before the project closed) has a two-pronged strategy of working with high￾and low-income farmers, there are only two mentions of women stakeholders in the entire document, 120 Some short courses women benefitted from included procurement and stores management, the new CSIR head office accounting software, office practice and management, and a molecular biology course in experimental designs and analysis (Project databases, June 15, 2021). 121 Dileepkumar Guntuku. 2016 (March). Assessment of the Institutional Capacity and Needs of the Savanna Agricultural Research Institute (SARI. Nyankpala, Northern Region, Ghana. Ames, Iowa: IFDC and ISU. (Final version with corrections dated April 25, 2016). 122 Samuel D. Braimah. 2017. The CSIR-SARI Strategic Plan: 2018–2022. Nyankpala: The USAID Direct Support Project for CSIR-SARI (Draft). Samuel D. Braimah. 2019. Research for Food Security and Wealth-The CSIR-SARI Strategic Plan: 2019– 2023. Nyankpala (Final Version). Although the original draft of this document included a budget line for a gender integration framework (pg. 44) this was considered an M&E study. 123 Disaggregated PITT prepared as part of the FY 2019 annual report. 124 Disaggregated PITT prepared as part of the FY 2019 annual report. 38 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV and these references focused on one small part (rice market women) of women’s involvement in the three priority value chains (rice, maize, and soybean). Output 3.3. M&E system. All three versions of the project’s M&E plans related to or supported by this project emphasized the critical importance of having sex-disaggregated data. However, the project was only able to produce gender-disaggregated data for one of the six EG indicators (EG. 3.2-1, number of individuals who have received USG-supported short-term agricultural sector productivity or food security training) for producers and people in government (through FY 2018) and people in private￾sector firms (through FY 2018).125 Only two of the M&E special studies, the baseline and adoption studies,126 included data on sex and age. Although there was no explicit mention of youth, the adoption report identified the beneficiaries by both sex and youth, which could have permitted disaggregated analysis of the data for youth. Despite this strong emphasis on sex-disaggregated data, there is very little evidence in any of the reports or work plans that this data was used to inform research except for the soybean value chain, where the gender experts associated with SIL made use of gender constraints to improve upon the research, which facilitated the rapid scale up of some of the soybean varieties by CSIR-SARI.127 Feedback from the FGDs, KIIs, and Post-interview Questionnaires CSIR-SARI staff. In general, most of the CSIR-SARI staff interviewed felt the project’s impact on gender mainstreaming was one of its weakest areas (Annex IV).  Training: Only 17 (30 percent) of the 56 CSIR-SARI staff who responded to this question in FGDs/KIIs and 10 (32 percent) of the 31 staff who responded to the post-interview questionnaire agreed or strongly agreed the project, “helped SARI track the participation of women in its training programs” (Annex IV.A.1),  Research: Only 18 (33 percent) of the 55 staff who responded to this question in FGDs/KIIs and eight (16 percent) of the 31 who responded to the post-interview questionnaire agreed or strongly agreed the project, “helped track the participation of women in their research programs” (Annex IV.A.1)  Gender equity: Only 13 (24 percent) of the 55 of FGD/KIIs participants and seven (23 percent) of the 31 who responded to the post-interview questionnaire agreed or strongly agreed the “project’s support for research and infrastructure development is helping SARI to achieve a more equitable gender balance in agricultural research in Northern Ghana” (Annex IV.A.1).  Cross-cutting impact on CSIR-SARI’s research and development programs: 125 1) CSIR-SARI. 2013. Monitoring and Evaluation Plan. Nyankpala: CSIR-SARI. (This was reviewed by METSS II; it was submitted as an annex to the project plan). 2) USAID Direct Support to SARI. 2015. Monitoring and Evaluation Plan. Nyankpala: M&E plan for CSIR-SARI. (This was reviewed by METSS II). 3) CSIR-SARI. 2016. Performance Monitoring Plan: CSIR-SARI. Nyankpala: CSIR-SARI. (June 2016) (File dated June 19, 2016) (This was not reviewed by METSS II). Two other versions of the M&E plans produced with funds from this project were not reviewed by METSS II and are not considered in this discussion of gender: a) USAID Direct Support to SARI. 2016. Activity Monitoring and Evaluation Plan (AMEP). Nyankpala: USAID-Direct Support Project for SARI. (Submitted by Iddrisu Yahaya). (September 2016). (This was reviewed by METSS II). b) Samuel D. Braimah. 2017. The CSIR-SARI Strategic Plan M&E Framework. Nyankpala: The USAID Direct Support Project for CSIR-SARI (This was not reviewed by METSS II). 126 1) CSIR-SARI Direct Support Project. M&E Team 2016. Status of Maize, Rice and Soybean Production Systems in Northern Ghana. Nyankpala: CSIR-SARI (October 1, 2016); 2) CSIR-SARI Direct Support Project. M&E Team. 2019. Adoption of Agricultural Technologies in Northern Ghana. Nyankpala: CSIR-SARI (March 1, 2019). 127 SIL. 2019. Activities and Impact Report. Ames, Iowa: SIL for USAID. SIL. 2020. Activities and Impact Report. Ames, Iowa: SIL for USAID. 39 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV  CSIR-SARI staff: Only nine (16 percent) of the 55 CSIR-SARI staff who responded to the question in FGDs/KIIs and seven (23 percent) of the 31 who responded to the post￾interview questionnaire agreed the “project addressed gender issues in SARI’s research and development projects.”  Media: Four (80 percent) of the five media specialists the project worked with either disagreed or only somewhat agreed the project, “addressed gender issues in its collaboration,” with their media outlet.  Non-community-based stakeholders: The non-community-based partners (e.g., GoG, project, and private-sector partners) were slightly more positive about the project’s support for gender mainstreaming. Specifically, 15 (60 percent) of the 25 non-CSIR-SARI stakeholder group participants in the FGDs/KIIs and eight (57 percent) of the 14 GoG and project partners who responded to the post-interview questionnaire agreed, “the project had addressed gender issues in SARI’s research and development (Annexes IV.A.2 and B.2)..”  Community-based stakeholders: In contrast, all 19 leaders of community-based groups, MoFA agents, and CSIR-SARI field technicians who participated in the on-farm adaptive and demonstration trials and farmer field schools emphasized the positive impact CSIR-SARI setting targets for female participation had on increasing the number and level of female beneficiaries. EQ 4.B. WHAT ARE THE LESSONS LEARNED AND BEST PRACTICES, AND FROM WHICH STAKEHOLDRES OR BENEFICIARIES, IN ACHIEVING RESULTS? The ET based its response to EQ 4.B. on the frequency with which different groups of stakeholders identified a project activity as an example of best practice or identified a particular lesson learned for future programs based on what did or did not work in the project (Annex IV. Tables 1 and 2). Best Practice Based on the FGDs, KIIs, and literature review, the ET identified three examples of best practice that the majority of CSIR-SARI, GoG, and partner projects identified as ones they, “would like to see included in future USAID-funded projects” 128 (Annex IV, Table 1). On-farm research and AEA training. Sixty-eight (68) percent of the 50 FGDs/KIIs identified this project’s strong collaboration with MoFA on the design, execution, and analysis of the demonstration and multi-site trials as an example of best practice they think should be replicated in future programs. While this best practice was pilot-tested and scaled up by a number of earlier USAID-funded projects,129 many of these earlier demonstration trials were managed by the staff associated with other donor￾funded projects (including other USAID-funded projects), not the CSIR-SARI researchers themselves. The unique twist this project brought to the table, cited by most stakeholders, was it facilitated the involvement of the CSIR-SARI researchers and technicians in the training of the MoFA AEAs who conducted the trials and the field days. A wide range of stakeholders noted these joint activities with MoFA facilitated CSIR-SARI: 1) having information on a wide group of geographical locations which reduced the number of years that the scientists needed to develop and release new varieties; 2 testing and releasing new varieties, developed by them and other projects, over a wide geographical area; 3) accelerating the dissemination of new varieties that had been developed with earlier support (i.e., legacy 128 Text used in the questionnaire. 129 The PITT shows CSIR-SARI was conducting 218 on-farm demonstration trials in 2015 before the current project started funding these activities (Annex II). 40 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV projects) but not yet disseminated; and 4) strengthening MoFA agents’ connections with CSIR-SARI field stations the AEAs, seed producers, and agro-dealers. ICT. The second most widely cited example of best practice (48 percent of the FGDs/KIIs) was this project’s construction and equipping of the new ICT center and its co-development (with ATT) of the ICT staff and the institute’s internet access (with SIL) (Annex IV, Table 1). In addition, the FGDs/KIIs identified five sub-examples of ICT best practices, including: 1) organizing live broadcasts with farmers using the new technologies with call-ins moderated by CSIR-SARI and/or MoFA staff; 2) inviting media to various project-sponsored events; 3) building and equipping the ICT media center; 4) collaborating with ATT on the development of CSIR-SARI’s first ICT strategy and the initial staffing of the center; 5) consolidating the information used in the project-sponsored trainings into manuals that facilitate MoFA agents scaling up the trainings they receive; and 6) hiring several University for Development Studies (UDS) journalism students to work as interns. Seed operations: The third most widely cited example of best practice (33 percent of the FGDs/KIIs agreed or strongly agreed with this statement) was this project’s support for the modernization of CSIR-SARI’s seed operations by its investment in basic infrastructure and training(Annex IV, Table 1).” The chief criticism of the project’s achievements in this area was the project’s limited impact on the use of irrigation and mechanized harvesting, seed processing, and packaging. Lessons Learned In addition to the best practices identified in the FGDs/KIIs and online questionnaires, the ET identified major lessons learned for future USAID/Ghana and CSIR-SARI projects. The target for the recommendation is identified in parentheses. Annex IV, Table 2 includes a more detailed analysis of which stakeholder groups made which recommendations, as well as a list of the major sub￾recommendations. Lesson A. Management support (USAID/Ghana and CSIR-SARI). Seventy-eight (78) percent of the 50 FGDs/KIIs and 100 percent of the non-community-based stakeholder groups identified the critical importance of future G2G projects and capacity-building projects for CSIR-SARI having a strong high￾level management support for the proposed management structure and the project plan from both the top administrators of the institution and its governance board (Annex IV. Table 1).130 While top management is key to any project, it is especially important for a G2G project because there is no implementing partner (like a consulting firm, NGO, or U.S. university) interface between USAID and the project manager. In contrast to other USAID-funded projects, a G2G project has to be implemented, “through the existing systems and structures of the GoG institution.”131 Many of the same interviews emphasized the critical importance of a highly participatory strategic planning process and launch in the first year of the project to ensure that all of the top administrators and staff understand the project plan and PIL. Unfortunately, this project’s strategic planning process did not start until 2017, and the final strategy document was not approved until just before the project closed (FY 2019). There was also no participatory start-up launch. Lesson B. Technology dissemination (USAID/Ghana and CSIR-SARI). All of the non-community￾based stakeholders emphasized that future donor-funded projects designed to promote agricultural development in northern Ghana can capitalize on CSIR-SARI’s increased capacity by: 1) Purchasing training and technology services from the institute; and 2) reducing the cost of credit for the critical 130 In response to this question in the FGDs and KIIs, the stakeholders identified ten sub-recommendation that have been incorporated into the global recommendations of this report (Annex IV. Table 1). 131 Comments on draft made by Grace Sebugah, former project AOR, September 2021. 41 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV complementary investments that private-sector seed producers need to sustain and scale up their commercial seed production, especially private-sector cold storage and seed processing facilities (Annex IV, Table 2). Lesson C. ICT and communication (USAID/Ghana and CSIR-SARI). The critical importance of strengthening the key stakeholders in northern Ghana’s access to ICT programs that promote the new technologies developed by CSIR-SARI by continuing to:  Build the capacity of MoFA, CSIR-SARI, and private-sector media outlets (newspaper, radio, social media, UDS journalism program)132 to conceptualize, execute, and monitor the efficacy of media and ICT outreach designed to promote new climate-smart technologies;  Help CSIR-SARI improve and lower the cost of its enhanced internet connectivity and provide resources it might need to improve its internet connectivity (if this is needed); and  Consider new ways CSIR-SARI researchers can use their research and M&E results to lobby on behalf of agricultural research as well as some of the most critical policy issues that affect farmers’ ability to buy seed.  Recognize that most of the ICT successes identified above occurred on the communications side of the ICT unit. Without taking away from those successes, more attention needs to be paid to the physical networking infrastructure of CSIR-SARI, on which all of SARI depends. The networking topology needs to be revised and upgraded, improvements to network management and ICT service offerings to researchers must be made, and greater attention must be given to network security and privacy. Connectivity to the remote stations at Manga and Wa should be established and a VPN established linking those researchers to Nyankpala. Consider contracting with GARNET to complete the necessary physical upgrades and explore hiring them on an ongoing basis to backstop and train local network administrators. Lesson D. M&E (CSIR-SARI). For CSIR-SARI to attract and manage large donor-funded projects, it needs to continue strengthen its internal capacity for M&E to:  Ensure future project designs include: 1) adequate training and a line budget for M&E; 2) monitoring M&E results against their original and revised targets for a particular output or outcome; 3) hiring and retaining an M&E specialist with experience in USAID programming and/or a part-time consultant with demonstrated experience if the new project is unable to afford a full-time position; and 4) M&E plans with PIRS for all custom and standard indicators required by the donor; and  Strengthen its linkages to UDS by encouraging future projects to offer grants to UDS students to study the impact of CSIR-SARI’s research and collaboration with MoFA. These grants would provide CSIR-SARI with a continuous progress and impact the assessment process that feeds back to the project and informs what is working and what is not, so that it can amend its 132 Key elements not addressed under this project that stakeholders suggested include: 1) hire an externa media consultant to monitor and support any new project’s communication strategy to ensure appropriate support and collaboration with other donor-funded projects and GoG actors (“This would make sure they were constantly involved in media engagement”); 2) set and monitor concrete targets for media and communications engagement with community-based stakeholders, especially women; 3) have new projects facilitate media outlets interested in agricultural development form a common media platform, creating a WhatsApp account so communications coordinators can communicate directly with different media actors and encourage other donor-funded projects to support the same media platform so it can be sustained; 4) strengthen the connection between UDS professors and students studying journalism and any media platform or support for agricultural development in northern Ghana; 5) support special technical trainings for journalists; 6) require new projects to support ICT and capitalize on earlier project’s achievements like the successful movie van outreach programs; and 7) anticipate the need to prepare and periodically update CSIR-SARI’s press materials. 42 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV interventions as they continue rather than wait until there is a mid-term review or end-of￾project assessment. Lesson E. Infrastructure (CSIR-SARI). CSIR-SARI needs to strengthen its internal systems for designing, building, and maintaining the critical infrastructure it needs to execute its core mandate by:  Ironing out the processes needed to approve infrastructure and equipment expenses at the start of the project so infrastructure budgets can be approved in the first year and fully executed before the project ends;  Ensuring at least some of the technicians directly involved in managing the new infrastructure are invited to and participate in the contractor debriefings (so they can provide timely feedback on the design before implementation and identify potential issues while there is still time to correct them);  Developing better systems for ensuring outside donor investments that are intended for the field stations actually reach the field stations; and  Building better systems for ensuring the improved infrastructure is maintained such as : 1) anticipating the need to train project staff and technicians in routine maintenance and repair, as well as how to use the new technology or infrastructure; 2) facilitating CSIR-SARI’s pilot-testing and scaling up its access to solar energy to reduce the recurrent cost of electricity; and 3) considering ways special contracts (with institutions like GARNET) might provide cost-effective solutions for the installation and maintenance of highly technical equipment and training for operating various internet-based functions (like the online database, network monitoring, and website management). Lesson F. Gender mainstreaming (CSIR-SARI). CSIR-SARI needs to strengthen its internal capacity to address gender and youth issues in agricultural development by:  Ensuring that any future donor-funded program to build its research or outreach capacity includes a robust gender integration plan (GIP) and appropriate staffing and budgetary support to strengthen gender, including helping CSIR-SARI.  Appointing a qualified staff person to serve as an agency-wide gender coordinator.  Encouraging new donor projects that support agricultural development in northern Ghana to fund capacity-building of female researchers and/or research in issues related to gender gaps in commercial seed production. 4.4.2. CONCLUSIONS EQ 4.A. There is evidence the project did build the capacity of a number of women researchers for leadership positions, trained female AEAs, and targeted women for inclusion in their on-farm demonstration trials and field days. There is also evidence these activities contributed to CSIR-SARI’s existing commitment to gender mainstreaming and agricultural research, and this increased inclusion of women in the research programs helped fuel the dramatic increase in soybean production during the LOA. The project did not, however, contribute to the development of a more broad-based commitment to the recruitment, training, and retention of women researchers and technicians nor to the development of a more broad-based gender strategy for the institute. While there is qualitative evidence from some of the community-based FGDs and KIIs that some of these activities increased women’s access to information about improved technologies (through the demonstration trials and field days) and the adoption of these new technologies improved women’s livelihoods, this is hard to prove with the project’s tracking data or reports since none of this was tracked. Although there is anecdotal evidence that youth benefitted from some of the new technologies and may be involved in commercial seed production, this was not monitored. 43 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Three principal reasons cited for this weak consideration of gender in the design, execution, and monitoring of the project were: 1. The project plan paid very little attention to gender and did not anticipate the need for a gender coordinator or any budget for gender activities (other than an initial baseline assessment). 2. Gender mainstreaming was not a stated output or purpose in the project’s official results framework that provided the basis for its work plans and reporting, so it was not an issue that was consistently reported on or monitored. 3. Although the PIL outlined USAID/Ghana’s expectations for the project, CSIR-SARI did not have the internal capacity, leadership, or experience to develop a GIP on its own. EQ 4.B. The stakeholders interviewed in the FGDs/KIIs and online questionnaires identified three clusters of activities—one focused on on-farm research and AEA training, one focused on ICT, and a third focused on modernizing CSIR-SARI’s seed operations—that they considered to be best practices. Despite a certain amount of variation between partner groups, there was a great deal of overlap and consensus between the stakeholders on the six principal lessons learned that have been integrated into the evaluation recommendations. 4.4.3. RECOMMENDATIONS After project management, finance, and staffing systems (79 mentions), the most frequently cited number of recommendations were for gender (40 mentions), and almost all of the recommendations focused on CSIR-SARI (Annexes IV.E. and V.B.6). CSIR-SARI Recommendation 6. ICT.  Sub-recommendation 6.2. ICT monitoring. Develop gender-sensitive assessment tools for tracking the impact of different outreach methodologies (e.g., portable video communication tools and community-based theater).  Sub-recommendation 6.3. ICT baseline. Conduct a second baseline diagnostic of CSIR￾SARI’s internet connectivity to assess the cost of: 1) extending internet service to its field stations; 2) making the system (including online library and website) more user-friendly; 3) lowering the routine operating costs; and 4) sub-contracting some basic maintenance, website, and database updates.  Sub-recommendation 6.4. ICT management. Identify ways CSIR-SARI can generate the funds it needs to support the recurrent costs of its internet access, website, and documentation/database library since these costs are not covered by the GoG’s core budget for the institute. Recommendation 7. Gender.  Sub-recommendation 7.1. New CSIR-SARI project designs. - 7.1.1. New project results frameworks. Activities with gender mainstreaming considerations should be required to include an IR or Sub-IR on gender and youth in the results framework. - 7.1.2. New project GIPs. Require any new projects with gender mainstreaming objectives to design and submit a GIP within a determined timeframe similar to the procedure for a monitoring, evaluation, and learning (MEL) plan. New projects should also include in their staffing at least one designated gender and youth point of contact. New projects’ budgets should also make room for gender and youth activities and corresponding gender-related indicator targets for all core activities. New projects should also explore ways these GIPs 44 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV can be mainstreamed by ensuring they comply with, are integrated into, and tracked and reported on as part of the institution’s strategic planning process. - 7.1.3. CSIR-SARI gender indicators. To facilitate a more integrated approach to gender, CSIR￾SARI needs to identify a limited number of disaggregated indicators by gender and region but be generic enough to feed into the gender requirements of specific donors like USAID. - 7.1.4. Staff and budgets. Require large new project designs to: 1) have a qualified staff person serve as a gender and youth focal person; and 2) ensure there is a gender budget line to support appropriate technical backstopping, studies, and training. - 7.1.5. Evidence-based strategies. Encourage new project designs to support evidence-based strategies for women by linking the monitoring data from specific project’s and CSIR-SARI’s institute-wide M&E system to the design of new and adjustment of existing CSIR-SARI and project-specific research and commercialization strategies.  Sub-recommendation 7.2. Outreach and collaboration with MoFA on dissemination. Encourage all the associations and producer groups MoFA and CSIR-SARI collaborate with on demonstration and adaptive trials to have women in leadership positions to strengthen the communication with the women farmers.  Sub-recommendation 7.3. Staffing and leadership training. - 7.3.1. Student internships. Facilitate female university students doing internships and theses connected with SARI’s technical research or collaborating with MoFA in conjunction with the GIP. - 7.3.2. Management training. Facilitate women’s access to long- and short-term management training in order to capitalize on the growing number of women working as SARI technicians in conjunction with the GIP. - 7.3.3. Staffing recruitment and retention. Set targets for hiring and retention of women staff and technicians in conjunction with the GIP. - 7.3.4. Core staff. Appoint one qualified CSIR-SARI staff person (and recognize this appointment by their official level of effort letter, which is the basis for their annual evaluation) to serve as the institute’s gender coordinator, and assign one staff person in each field station to serve as the station focal person. - 7.3.5. Inter-donor coordination and support. Pilot test the concept of a donor working group on gender to advise the GIP process and ensure appropriate coordination between the different international donors that support CSIR-SARI projects that include gender mainstreaming. USAID/Ghana Recommendations A-D. Management. Recommendations for future projects’ management and accounting systems (A-D) were detailed in Section 4.1.3. Recommendation G. ICT internet connectivity. 1) Ask new USAID/Ghana-funded G2G projects to include an indicator of internet connectivity, like the four-variable internet health indicator SIL pilot tested on this project, in future baseline capacity assessments; 2) track any strategy developed to address internet-based issues identified through one or more custom indicators in the PITT; 3) if a baseline diagnostic indicates the institution being targeted by the G2G grant suffers from weak connectivity or systems for maintaining, repairing, or keeping the system user-friendly, encourage G2G projects to include a budget to explore mentoring from local internet cooperatives and COEs (e.g., GARNET, SIL’s ICT Connectivity Project, and/or the computer scientists in the CSIR Electronics Unit) that can lower the initial costs of installation and maintenance and help G2G partners avoid costly 45 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV mistakes that are hard to fix; and 4) identify ways some of the routine costs of maintaining the system can be shared with other donors who support the institution. Recommendation H. Gender issues in design. Strengthen the guidance USAID/Ghana provides to GoG institutions that apply for G2G grants. This guidance should: 1) require new project designs to include gender and youth mainstreaming as a project sub-purpose/sub-IR in the results framework; and 2) identify some of the most important documents on the USAID websites programs can use to help incorporate gender into the initial design of their GIPs (including issues related to staffing, budget, governance, and M&E). Recommendation I. Gender issues in the PIL. If a new G2G project plan does not include a solid plan for GIP, USAID/Ghana should require this in the PIL as it did on another G2G project executed at the same time as this one. 4.5. EQ 5.A) TO WHAT EXTENT ARE THE PROJECT RESULTS LIKELY TO CONTINUE AFTER THE UNITED STATES GOVERNMENT (USG) SUPPORT ENDS? 5.B) WHAT ORGANIZATIONAL DEVELOPMENT SUPPORT AND INSTITUTIONAL STRENGTHENING ASSISTANCE WOULD CSIR-SARI NEED FOR IT TO IMPROVE ITS ABILITY TO SUSTAINABLY BENEFIT BUSINESSES AND FARMERS IN GHANA? The ET based its assessment of EQ 5 on the different stakeholders’ perspectives on: 1) whether the major project results for specific outputs are likely to be sustained in the medium- and long-term with the current and projected context of GoG and external-donor support for CSIR-SARI; 2) the chief challenges CSIR-SARI is likely to face in sustaining and scaling up these activities; and 3) CSIR-SARI’s own reflection on these issues as manifested in the project’s close-out report. 4.5.1. FINDINGS SP 1: Capacity development for agricultural research in northern Ghana Output 1.1: Core scientists trained. Although the short-term impact of the project’s support for core scientists’ training is assured because scientists’ salaries are guaranteed (in perpetuity) by GoG, the research programs are still completely dependent on outside donor support or revenue generated from seed sales to support research programs, and on outside donor-funded projects to continue training and retraining the staff and technicians they rely on to execute their research programs and produce breeder seed. Despite these shortcomings, only 56 percent of the CSIR-SARI staff agreed or strongly agreed, and 33 percent slightly agreed, this capacity was likely to be sustained (Table 3), in large part because the project had not been successful in improving, “their financial systems” and, “must learn to generate money to survive.”133 Based on this triangulation with the project reports, the ET agrees the activities under this output are likely to be sustained with CSIR-SARI’s current projected levels of GoG and donor support. However, for them to be fully sustained, CSIR-SARI will need additional, “organizational development support and institutional strengthening assistance,” to achieve the results this project identified under Outputs 1.2 and 3.2, as well as improved advocacy (based on research and M&E results) to strengthen the GoG’s core support for agricultural research. Output 1.2: Organizational quality and efficiency of CSIR-SARI increased. In spite of the project having a steep learning curve on USAID rules and regulations, most of the staff reported the project catalyzed a number of improvements in CSIR-SARI’s operational procedures (like procurement, accounting, and reporting) that have made CSIR-SARI more efficient. Forty-nine (49) percent of staff either agreed or 133 Project partner. EQ 5, Question 128. 46 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV strongly agreed that these changes would be sustained (Table 3). Another 30 percent of the staff reported they only slightly agreed with this statement because: 1) the project-supported changes were just getting started and/or had not yet been fully implemented; and 2) the top management of the institute still needed to make the necessary management reforms the project’s owned close-out report deemed, “critical for sustaining the [other] achievements of the project.” 134 In light of this feedback, the ET considers many of these positive impacts are likely to be sustained but are in need of additional support from CSIR-SARI, which is likely to be forthcoming since many of the new procedures this project put in place will both attract and be required by future donors. The ET also agrees with the close-out report that given the extent of the project’s management problem, any future donor-sponsored project that wishes to strengthen CSIR-SARI’s organizational capacity in these areas, “should be structured as a mentorship” activity.135 Output 1.3: Infrastructure built. In contrast, most of the staff (79 percent) and partners (100 percent) interviewed agreed (Levels 5-7, Table 3) that the project’s extensive investment in new and renovated infrastructure and equipment will be sustained, though at less-than-perfect levels, due to: 1) poor quality of some renovations (like the canteen); and 2) lack of funds for maintenance, which is a core management function neither GoG nor other donor-funded projects are likely to contribute to unless CSIR-SARI improves its internal management systems and overhead policies, which were not targeted by this project. Based on this feedback, which triangulates with the project reports that underscore the issue of long-term maintenance and updating of equipment, the ET concludes these investments are sustainable in the short term. However, CSIR-SARI needs to strengthen its internal infrastructure management systems and its commercialization unit in order to generate the revenue the project needs for additional investment and updates. 134 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2019). Nyankpala: CSIR-SARI for USAID/Ghana. Pp. 38-39. 135 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2019). Nyankpala: CSIR-SARI for USAID/Ghana. Pp. 38-39. 47 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Table 3. Evaluation Team Assessment of Extent to Which Results Achieved by the Project for Specific Outputs are Likely to Continue and What Types of Additional Support Will Be Needed Based on Stakeholder Feedback and Literature Review136 Output by Sub-purpose Staff Agree or Strongly Agree Staff Somewhat Agree Partners Agree or Strongly Agree Partners Somewhat Agree ET Assessment: Short-Term (1-5 Years) Likert Rankings 6-7 5 only 6-7 5 only N/A Sub-purpose 1: 79% 14% 1.1. Core scientists trained 56% 33% n/a n/a Yes 1.2. Organizational quality increased 49% 30% 57% 21% Likely 1.3. Infrastructure built 44% 35% 57% 43% Yes Sub-purpose 2: 28% 56% —For improved seed* 58% 32% —For ISFM* 37% 16% 2.1. Seed operations modernized 62% 29% Yes 2.2. ISFM disseminated 62% 28% Likely 2.3. Technicians’ capacity built 53% 30% Yes 2.4. RELC mechanisms improved 60% 1% Likely Sub-purpose 3: —Client-oriented research approaches* 56% 24% ----Market-oriented research* 35% 26% 3.1. ICT and knowledge management systems 38% 51% Likely 3.2. Commercialization 52% 37% No 3.3. M&E system 47% 39% Likely n: percent of respondents giving this answer out of the total responding to the question in FGDs/KIIs.137 Italics and asterisk (*): Questions only asked of partners. Methodology: Rankings are based on CSIR-SARI staff (Stakeholder Group 1) and partner (Stakeholder Groups 2, 3, and 6) Likert rankings for EQs in the FGDs and KIIs. The short- and medium-term assessment (Yes/Likely/No) is based on this stakeholder feedback from the FGDs and KIIs and questionnaires as well as the project records and reports (Annexes IV.A. and, IV.A.2). 136 Yes: permanent funding or follow-on funding from another project or CSIR-SARI’s commercial activities assured; Likely: strong commitment to sustaining the activity but not assured by another donor-funded project or CSIR-SARI’s commercial activities; No: not likely to be sustained without additional support from another donor-funded grant. 137 This table does not include the information from the stakeholders who responded to the post-interview questionnaire unless very different from those in FGDs/KIIs. 48 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV SP 2: Agricultural technologies development and dissemination Output 2.1: Seed operations. Most stakeholders who were interviewed agreed with the project’s close-out report that: 1) the project was, “successful to a large extent in modernizing SARI’s seed operations,”138; and 2) these achievements are likely to be sustained because CSIR-SARI, “has a direct financial stake in sustaining these activities, which are its principle source of commercial revenue” 139 (Table 3). At the same time, the respondents identified a number of constraints that could affect the quality, price, and demand for CSIR-SARI’s breeder seed, including: 1) the projected phase-out of the GoG PFJ seed subsidies in 2024; 2) weak development of private-sector seed processing and cold storage facilities for seed in northern Ghana (which is needed to complement the infrastructure that CSIR-SARI developed under this and other USAID-funded projects); 3) weak coordination between CSIR-SARI, PPRSD, and the commercial seed producers needed to ensure seed quality; and 4) CSIR-SARI’s weak and limited access to the types of mechanization and irrigation needed to increase production. Output 2.2: ISFM practices disseminated. In contrast, many stakeholders (especially in the private sector) felt CSIR-SARI’s substantial achievements in the area of ISFM were at greater risk than those related to its seed operations for two reasons (Table 3). The first is because, in spite of the huge potential impact the project-supported ISFM research and training was expected to have on farmers’ yields, this is unlikely to be a profitable income-generating activity for the institute like the commercial seed production. The second is the limited capacity of CSIR-SARI’s soil lab, which plays a critical role giving farmers and AEAs information to determine what type of fertilizer is needed. Since there are a growing number of elite commercial farmers who require these services and may be willing to pay for them, this is an area that would benefit from CSIR-SARI strengthening its commercialization unit in the ways envisioned by this project. Output 2.3: Technical capacity built. Since CSIR-SARI’s routine training of its technicians is assured under GoG funding, most staff were optimistic that at least some technical capacity developed under the project could be sustained, at least in the short term. However, for the institute to ensure its staff stays up to date on new technology trends, CSIR-SARI needs to strengthen its internal systems for tracking the training of its staff (as was envisioned but never implemented under Activity 1.2.7 of this project). Output 2.4: RELC mechanisms improved. Although most stakeholders considered the prospects for sustaining the RELC mechanism is likely but not fully assured since these activities remain highly “dependent on donor funding” (Table 3).140 Since the stakeholders do not pay for these services (and there is no internal government budget to support them), this capacity—and its impacts—is likely to continue to scale up and down depending on whether or not it has funding. SP 3: Market and client-oriented research approach developed Output 3.1: ICT and KMSs developed. Internet: While the project completed the delivery of dependable 155 Kbps ICT connectivity to the main CSIR-SARI location at Nyankpala from GARNET, the local wiring and wireless infrastructure 138 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2019). Nyankpala: CSIR-SARI for USAID/Ghana. Pg. 33. 139 “There is great commitment on the part of the leadership [to modernizing CSIR-SARI’s seed operations since] this is their core business” and “[CSIR-SARI] make[s] money out of it.” (Illustrative quotes from FGDs and KIIs with CSIR-SARI staff). 140 During the gap years there is no activity since, “even the vehicles [that support it] were funded by donors.” (Illustrative quote from FGD/KII with CSIR-SARI staff). 49 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV needed at Nyankpala for good service at all locations was never completed; nor was the connectivity ever completed for the two remote stations at Manga and Wa. Most CSIR-SARI staff who participated in the FGDs/KIIs reported this as a critical cross-cutting constraint affecting achievements under the other respective outputs. 141 Unless CSIR-SARI manages to resolve the remaining connectivity infrastructure networking issues at all locations, even the current system that provides internet access to most of the main CSIR-SARI offices at Nyankpala is at risk. ICT: Although the close-out report and most of the CSIR-SARI staff consider the project’s ICT support as one of its greatest success stories, the short-term sustainability of these activities is likely but not fully guaranteed due to CSIR-SARI no longer having a budget to support ICT activities (which are not covered by core GoG funding) or a system in place for making sure other donor-funded projects contribute to the cost of the internet (Table 3). To address this issue, CSIR-SARI needs to develop more effective organizational systems for getting new and existing projects to contribute to the routine costs of the internet, website, and documentation/database library that all of them are using and benefitting from. In undertaking this needed change, CSIR-SARI (both administrators and staff) are encouraged to look at ICT investments, if implemented skillfully and properly using state-of-the-art tools, as a means that will allow them to do their jobs more effectively and more efficiently. Output 3.2: Commercialization program strengthened. Although the project never achieved its original goal of building the capacity of the commercialization unit to oversee CSIR-SARI’s marketing and sales, its commercial sales boomed because of the sharp increase in demand from GoG’s PFJ. Going forward, however, the weak capacity of this unit is a critical cross-cutting problem that affects the sustainability of five of the ten other project achievements (the five outputs marked “likely” in Table 3). The ET agrees with the project’s close-out report that this is one of two outputs that is non-sustainable and will require a great deal of additional organizational restructuring and capacity-building to address.142 Output 3.3. M&E system developed and operationalized. Although the project never achieved its original goal of developing and operationalizing an M&E system for CSIR-SARI, it contributed to (but not in the ways identified in the proposal): 1) building the M&E capacity of a new generation of CSIR-SARI economists who recently completed their PhDs and MSCs (in FYs 2020 and 2021); and 2) trying to develop the type of CSIR-SARI-wide M&E system this project was expected to develop under this output. Although 86 percent of the staff interviewed felt this increased capacity would be sustained, only 47 percent strongly agreed or agreed, and 39 percent only slightly agreed (Table 3). Three of the most frequently cited challenges staff identified to ensuring this capacity is sustained are: 1) the lack of central administrative support for the creation of a CSIR-SARI M&E unit, including an M&E coordinator; 2) none of the current economists has any formal M&E training or technical assistance other than intermittent feedback and training they got from USAID (via Monitoring, Evaluation, and Technical Support Services II [METSS II]) during its remote and in-person supervision missions; and 3) most CSIR-SARI staff trainings anticipated in the proposal never happened so the newly tasked (i.e., not yet officially appointed) M&E focal persons have excellent PhD and MSC-level training in economics but very little formal training or mentoring in M&E. 4.5.2. CONCLUSIONS In the short term, (one to five years) the sustainability of the project’s results is assured for four of the 141 “The internet infrastructure is deficient, which greatly impedes the effective uploading and dissemination of the ICT tools,” that CSIR￾SARI developed (under this project and ATT) and continues to develop (Illustrative quote from FGD/KII with CSIR-SARI staff, Annex IV.A.1). 142 CSIR-SARI. 2020. USAID/Ghana-CSIR-SARI Technical and Financial Support Project. Close-out Report (2015–2019). Nyankpala: CSIR-SARI for USAID/Ghana. Pp. 36-37. 50 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV ten project outputs, which are all or partially funded by GoG (like staff salaries) or revenue earned from commercial seed sales, and/or are likely to benefit from overhead or direct financial support from other donor-funded projects (Outputs 1.1, 1.3, 2.1, and 2.3; Table 3). Although the results of five more outputs (Outputs 1.2, 2.2, 2.4, 3.1, and 3.3) are likely to be sustained, the conditions for fully sustaining them at their current levels are not guaranteed. The principal reasons for this are: 1) they are not supported by the core GoG budget; 2) funds generated by the CSIR-SARI commercialization unit are insufficient to support them; and 3) in most cases, they are not the types of activities CSIR-SARI’s current roster of donor-funded projects are willing or able to support. Unfortunately, due to the limited success of the project executing the activities needed to strengthen CSIR-SARI’s commercialization unit (Output 3.2), this unit is not fully functionally nor likely to be without extensive, “development support and institutional strengthening,” from CSIR’s central administration, CSIR-SARI’s top administrators, as well as supportive donors. 4.5.3. RECOMMENDATIONS CSIR-SARI (see Sections 4.1.3, 4.2.3, 4.3.3, and 4.4.3). Annex V.B.6. lists the summary recommendations adopted by the ET for each EQ. Recommendation 1. Project management systems. The sub-recommendations for strengthening the sustainability of the current project’s impact on management (sub-recommendations 1.1–1.2) were detailed in Section 4.1.3. Recommendation 2. Infrastructure and equipment. The key recommendation for infrastructure and equipment is detailed in Section 4.1.3. Recommendation 3. Commercialization. The five key sub-recommendations for increasing the long-term sustainability of the project’s achievements under commercialization (Sub-recommendations 3.1–3.5) were detailed in Section 4.1.3. Recommendation 4. M&E. Recommendations for M&E (sub-recommendations 4.1–4.5) were detailed in Section 4.1.3. Recommendation 5. Technology dissemination and scale-up. The five sub-recommendations for strengthening the sustainability of the project’s results under technology dissemination and scale up (5.1–5.5) were detailed in Section 4.2.3. Recommendation 6. ICT. The two sub-recommendations for sustaining and scaling up the project’s impact on ICT outreach were detailed in Section 4.2.3 (Sub-recommendations 6.1-6.2); the sub￾recommendations for strengthening the project’s ability impact on gender mainstreaming and on the project’s impact on improving the institute’s internet connectivity are described in Section 4.5.3 (Sub￾recommendations 6.3-6.4). Additional ICT recommendations identified as critical to strengthening the GoG’s willingness and ability to support CSIR-SARI’s core budget include:  Sub-recommendation 6.5. Advocacy. Strengthen the capacity of CSIR-SARI’s M&E and IT units to backstop its scientists in dealing with the media, publishing their own data and project M&E data, and using and the institute’s M&E data and research results for advocacy at the national level.  Sub-recommendation 6.6. Collaboration. Consider ways USAID-funded projects and innovation labs can backstop these research, M&E, and advocacy efforts through their existing budgets. 51 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV USAID/Ghana Recommendation E. Fee-based training. Encourage future USAID-funded projects in northern Ghana to access fee-based training programs from CSIR-SARI for their staff. Recommendation F. Collaboration with MoFA. Encourage future USAID-funded projects in northern Ghana to identify ways to help scale up CSIR-SARI’s successful collaboration with MoFA for AEA training, field trials, and field days. 52 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV ANNEXES 53 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV ANNEX 1. EVALUATION STATEMENT OF WORK Date: October 27, 2020 Disclaimer: This publication was produced for review by the United States Agency for International Development. STATEMEMENT OF WORK FINAL PERFORMANCE EVALUATION OF CSIR-SARI TECHNICAL AND FINANCIAL SUPPORT PROJECT 54 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV I. PURPOSE OF THE EVALUATION Audience and intended Users The audience of the evaluation report will be the USAID/Ghana Mission, specifically, the Economic Growth Office, and the implementing partner. Results of the evaluation will be used by USAID/Ghana Office of the Economic Growth to inform design of similar projects in future. The implementer of the project (CSIR-SARI) will also learn about their strengths and weaknesses and utilize the recommendations therein to improve upon the Center’s capacity to make available sustainable agricultural production systems for smallholder farmers in the project zone of influence. II. SUMMARY INFORMATION The USAID/Ghana – CSIR-SARI Technical and Financial Support Project is a five- year activity that contributes to USAID’s Ghana Economic Growth Office intermediate result of “Increased Competitiveness of Major Food Value Chains and the overall goal of fostering broad-based, sustained, and inclusive economic growth.” Table 1: Project Identification data Strategy / Project / Activity Name USAID | GHANA – CSIR-SARI Technical and Financial Support Project USAID Office Economic Growth Office Implementer Council for Scientific and Industrial Research (CSIR), Savannah Agricultural Research Institute (SARI) Cooperative Agreement/Contract # 641-A18-FY14- IL#03 Total Estimated Ceiling of the Evaluated Project/Activity (TEC) US$5,549,625.35 Life of Strategy, Project, or Activity January 14, 2015 – December 12, 2020 Active Geographic Regions Northern, Upper East and Upper West Development Objective(s) (DOs) DO2: Sustainable and Broadly Shared Economic Growth, IR2.1: Increased competitiveness of major food chains Required evaluation Yes External or internal evaluation External III. BACKGROUND A. Description of the Problem and Context In recent years, the economic and technical environment for agricultural research in Ghana has changed considerably. Several factors in regional trade and domestic demand for food have shifted, and economic incentives and improved technology is now a critical factor in the country's ability to exploit its comparative and competitive advantages. Technological needs are changing toward more knowledge and skill-intensive agriculture and more complex farming systems. Important changes in the technology for research itself, especially, the new biotechnology and informational technologies, are raising new issues in organizing national research systems, related to economies of scale, international collaboration, and public-private linkages. These increasing demands on research systems contrast with the lack of institutional innovation in research− system management and organization and the stagnation or decline in funding for agricultural research. The substantial investment in National Agricultural Research Systems (NARS) over the past two decades, although providing high pay-offs, has not yet resulted in the institutional capacity to sustain those pay-offs in a rapidly evolving technical and policy environment. Many systems are suffering a crisis of management, with top-heavy bureaucracy, centralization of decision making, and lack of incentives for the innovation process which are very critical for research. 55 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV It is in light of the important role that agricultural research can play in increasing productivity, and reducing poverty and malnutrition in the northern regions that Africa Lead143, at the instance of USAID/Ghana’s Economic Growth Office, conducted an institutional assessment of the CSIR/Savanna Agricultural Research Institute established to conduct agricultural research in the northern regions. The results of the assessment showed that the Institute faces various challenges including poor and inadequate accommodation and research infrastructure, inadequate state-of-the-art equipment, limited commercialization activities to generate funds and poor reward and incentive-packages for staff. It also found that although CSIR-SARI has been successful in a number of research and program executions in cooperation and collaboration with international agricultural research centers and in attracting funding from development partners, foreign universities and other local programs, the institute and its outstations are currently facing serious challenges relating to infrastructure, research equipment and supplies, logistics, internet connectivity, as well as, an absence of Information Communication Technology (ICT) facilities, leading to poor dissemination of key information. At program level the Africa Lead assessment revealed that although CSIR-SARI currently runs relatively successful programs on crop improvement, soil fertility management, cropping systems improvement, crop protection/post-harvest and socio-economic studies, there still are several serious challenges such as the following: i. the need to satisfy seed needs of stakeholder farmers: especially, for small-scale and emerging commercial agriculture, seed quality maintenance and promotion of new varieties; ii. the need to arrest increasing deterioration of the natural resource base: wind and water erosion, low soil organic matter and poor soil structure; and iii. the necessity for dealing with the negative influence of Climate Change: characterized by erratic rainfall. At the functional level, the assessment, employed a simplified framework to examine the “fit” among CSIR-SARI’s Mission, the needs of the constituents and stakeholders it seeks to serve and its capacities in relation to technical, organizational, financial and infrastructural support systems that would ensure the fulfillment of its mandate. The results showed that SARI is mostly nascent or expanding in the nine areas of organizational development assessed (governance, management practice, human resource management, service delivery, external relations, sustainability, financial resource management, financial vulnerability and financial viability/adequacy). With these issues in view, USAID/Ghana’s Economic Growth Office, through the FTF multi-year strategy, which concentrates on commercializing rice, maize and soya value chains, sought to support CSIR-SARI in building its research capacity to efficiently and effectively deliver on its core mandate of promoting agricultural development through development of appropriate technologies for productivity enhancement. B. Description of the Intervention To Be Evaluated and Theory of Change USAID Ghana’s Support to the Council for Scientific and Industrial Research - Savanna Agricultural Research Institute (CSIR-SARI) focus efforts on expanding its research capacity to make available a suite of productivity enhancing technologies suitable for farmers, and support other FTF projects that work directly with farmers in the FTF intervention zone. This support is aimed at enabling CSIR-SARI to integrate into the emerging global agricultural research system to keep abreast of rapid advances in scientific knowledge, and to improve the cost effectiveness of technology generation by capturing spill− ins and through collaborative research efforts 143 Africa Lead is the USAID/Ghana Feed the Future Building Capacity for African Agricultural Transformation (Africa Lead) Activity 56 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV The overall goal is to establish CSIR-SARI as a well-managed research organization that effectively develops and disseminates crop and integrated soil fertility management technologies suitable to the Guinea and Sudan savannah agro-ecological zones of Ghana. These technologies should address the constraints and needs of end users in Northern Ghana through regular consultations during the technology development process. Through results of institutional assessments and in consultation with the management and key staff, the following three activities will be pursued with the overall goal of Transforming SARI into a Centre of Excellence for agricultural research in Northern Ghana:  Activity 1: Capacity Development for Agricultural Research in Northern Ghana  Activity 2: Agricultural Technologies Development and Dissemination  Activity 3: Market and Client-Oriented Research Approach including Coordination,  Project Management, Communication and M&E The development hypothesis of the CSIR-SARI Project is: IF these CSIR-SARI activities are successful in: ● Developing the human and institutional capacity for Agricultural research ● Developing technologies and dissemination for increased agricultural productivity, Then it Will Transform CSIR-SARI into a Centre of excellence for Research and Development in the Savanna Agro￾ecological Zone. C. Project Target Areas and Target Beneficiaries The CSIR-SARI is a research institute with the mandate to develop new crop varieties that can help improve yield, reduce food insecurity and improve livelihoods of the people of Northern Ghana. SARI’s research is focused on breeding, climate smart, agronomy, protection, soil improvement and socioeconomics. The target population of the research is the Northern farm households. Specifically, most of the research work is researcher managed with few on-farm studies for technology dissemination. These are done in the Sissala West and East districts, Yendi, Damango and Wa West districts. IV. EVALUATION QUESITONS In order to guide the final evaluation, the following questions will be answered: 1. To what extent has the CSIR-SARI Technical and Financial Support Project achieved its intended goal and objectives as defined by the results framework? 2. Is there evidence of an improved organizational development and a strengthened institution of SARI (increased research capacity) and has this resulted in improved welfare of smallholder farmers in Northern Ghana? 3. What unintended contributions, results and/or outcomes has the Project approach and activities achieved relative to transforming CSIR-SARI into a Centre of excellence for Research and Development in the Savanna Agro-ecological Zone? 4. To what extent has the Project addressed gender issues in relation to capacity building in research and development and has this had an impact on gender mainstreaming in agricultural research? What are the lessons learned and best practices; and from which stakeholders or beneficiaries, in achieving results? 5. To what extent are the project results and outcomes likely to continue after the USG support? A) What organizational development support and institutional strengthening assistance would CSIR-SARI need for it to improve its ability to sustainably benefit businesses and farmers in Ghana. 57 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV V. EVALUATION DESIGN AND METHODOLOGY It is anticipated that a cross-sectional descriptive approach will be adopted for this evaluation. This approach shows snapshots across different subgroups who received services from the CSIR-SARI. It is generally expected that the Consultants will employ appropriate scientific data collection methods, preferably a mix of qualitative and quantitative data collection and analysis methods, including data triangulation, to arrive at the evaluation findings that fully respond to the evaluation questions and make recommendation thereof. The consultants should share all data collection tools with the CSIR-SARI, METSS II and USAID/Ghana for review and feedback and/or discussion, with sufficient time, before they are applied in the field. They should utilize, to the extent possible various data collection methods including but not limited to the following: A. Review of existing Documentation: Project documents that should be reviewed may include but not limited to Project Appraisal Document, baseline reports, quarterly, annual progress reports and any other assessment reports related to the CSIR-SARI project. B. Key Informant Interviews: The team is expected to hold interviews with key partners of the project especially the CSIR-SARI Technical and Financial Support Project, and other Partners including actors in the private sector who have been engaged in the project implementation. C. Survey of Beneficiaries: The team is expected to cover a sample of project beneficiaries, institutions and Private sector entities who have a stake in the CSIR – SARI project. D. Personal Observation: The evaluation team is expected to visit selected physical investments supported by the CSIR–SARI Project and document how they are functioning and benefiting the beneficiary institutions. VI. DELIVERABLES AND REPORTING REQUIREMENTS 1. Evaluation Work plan: Within two (2) weeks of the award of the contract, the lead evaluator shall complete and present a draft work plan for the evaluation to the Contracting Officer’s Representative (COR) at USAID/Ghana, Chief of Operations (COO) at METSS II and the Project Manager of CSIR-SARI Project for approval. The evaluation work plan will comprise of an evaluation design, which will include:  Detailed evaluation design matrix that links the Evaluation Questions from the SOW to data sources, methods, and the data analysis plan;  Draft questionnaires and other data collection instruments;  List of potential interviewees and sites to be visited and proposed selection criteria and/or sampling plan (must include sampling methodology and methods, including a justification of sample size and any applicable calculations) and  Limitations to the evaluation design. The evaluation work plan will also include 1. Draft schedule and logistical arrangements; 2. Members of the evaluation team, delineated by roles and responsibilities; 3. Evaluation milestones; 4. Anticipated schedule of evaluation team data collection efforts; 5. Locations and dates for piloting data collection efforts, if applicable; 6. Proposed evaluation methodology including selection criteria for comparison groups, if applicable. The data analysis plan should clearly describe the evaluation team’s approach for analyzing quantitative and qualitative data (as applicable), including proposed sample sizes, specific data 58 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV analysis tools, and any software proposed to be used, with an explanation of how/why these selections will be useful in answering the evaluation questions for this task. Gender, geographic, and role (beneficiary, implementer, government official, NGO, etc.) disaggregation must be included in the data analysis, where applicable. The consultant must receive approval of work plan before beginning fieldwork. 2. In-briefing: Within two (2) working days of arrival in Accra, the evaluation team will meet with CSIR-SARI staff, METSS II Project and Economic Growth Office for introductions and to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, methodology, and work plan, and/or to adjust the SOW, if necessary. 3. Mid-term Briefing and Interim Meetings The evaluation team is expected to hold a mid-term briefing with CSIR-SARI staff, METSS II Project and/ or USAID/Ghana Economic Growth Office on the status of the evaluation, including potential challenges and emerging opportunities. The team will also provide METSS II Project Chief of Operations with periodic briefings and feedback on the team’s findings, as agreed upon during the in-briefing. If desired or necessary, weekly briefings by phone can be arranged. 4. Final Exit Briefing: The evaluation team is expected to hold a final exit briefing to discuss the status of data collection and preliminary findings. The presentation will be scheduled as agreed upon during the in-briefing. The evaluation team is expected to hold a presentation either in person, or by virtual means to discuss the summary of findings and conclusions with CSIR- SARI staff, METSS II Project and/or Economic Growth Office. The evaluation team will consider comments and make necessary revisions. 5. Draft Evaluation Report: The draft evaluation report should be consistent with the guidance provided in Section IX, Final Report Format sub-section. The report will address each of the questions identified in Section IV of this SOW and any other issues the team considers to have a bearing on the objectives of the evaluation. Any such issues can be included in the report only after consultation with METSS II Project. Once the initial draft evaluation report is submitted, CSIR- SARI staff, METSS II Project and USAID/Ghana will take ten (10) business days to review and comment on the initial draft, after which the consolidated comments would be submitted to the evaluation team. The evaluation team will then be asked to submit a revised final draft report within five (5) business days, and again CSIR￾SARI staff, METSS II Project and USAID/Ghana will review and send comments on this final draft report within ten (10) business days of its submission. 6. Final Evaluation Report The evaluation team will be asked to take no more than three (3) business days to respond/incorporate the final comments from all contributors (CSIR- SARI staff, METSS II Project and USAID/Ghana). VII. EVALUATION TEAM COMPOSITION The evaluation team will consist of a team leader and three other experts. A representative from the USAID/Ghana Mission and METSS II Project may be delegated to work full-time with the evaluation team or to participate in selected evaluation activities. Senior Technical Consultant (Lead): 1. The consultant must be someone with extensive experience as team leader of mid-term and final assessments of USAID funded project evaluations and the concept of “participatory” assessment processes that elicit high level monitoring and evaluation. 59 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV 2. Specialist should have a postgraduate degree in Agriculture Sciences, or an applicable social science field. S/he should have at least 10 years’ senior level experience working in agricultural sector in a developing country. 3. S/he should have extensive experience in project management, institutional capacity building programs, conducting both qualitative and quantitative evaluations/ assessments and strong familiarity with agricultural finance. Excellent oral and written skills are required. 4. The Team Leader should also have experience in leading evaluation teams and preparing high quality documents. The Team Leader will take specific responsibility for assessing and analyzing the project’s progress towards achieving its targets, factors for such performance, benefits/impact of the strategies, and compare with other possible options. S/he will also suggest ways of improving the present performance, if any. 5. S/he will provide leadership for the team, finalize the evaluation design, coordinate activities, arrange periodic meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations into a high quality document. S/he will write the final report. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to the USAID/Ghana team and other major partners. 6. Experience with data collection procedures, surveys, and analysis of data using statistical analysis tools. 7. This person should also have good interpersonal and diplomatic skills and demonstrated expertise in gender in order to ensure that the USAID/Feed the Future protocols for considering gender in an assessment are met. Senior Technical Writing Consultant and Evaluation Management Consultant (Co-Lead): The senior technical writing and evaluation management consultant should be someone with at least 10 years’ experience in managing or co-managing participatory assessment exercise and working with development and food security professionals to improve their skills for systematic write up of reports and bulletins that comply with US government M&E quality standards. The specialist must have a minimum of a masters in a relevant technical areas and have good organizational and pedagogical skills as well as extensive experience in FTF and USAID guidelines and standards including 508 compliances. He/she will have the role of ensuring that all the written deliverables comply with USAID/FTF rules and regulations and USAID/Ghana expectations. He / She will play a critical role in determining a speedy and efficient process for document turnaround. Senior Agricultural Specialists. Agricultural fields (Agronomy/ Crop Science/Crop Production/Plant Science, Agricultural Extension and Rural Sociology, Agricultural Economics or any other), or an applicable social science field and have firsthand familiarity with the capacity building issues that CSIR￾SARI is focusing on. S/he should have at least 10 years’ senior level experience working in Plant breeding and agricultural sector in Ghana. This individual will insure that the most important technical capacity issues and institutional issues that were targeted by the project are measured appropriately in the interviews and that the recommendations and “lessons learned” are realistic. The work plan must provide information about evaluation team members, including their curricula vitae, and explain how they meet the requirements in the evaluation SOW. Submissions of writing samples or links to past evaluation reports and related deliverables composed by proposed team members are highly desirable. All team members must provide to USAID/Ghana a signed statement attesting to a lack of conflict of interest or describing an existing conflict of interest relative to the activity being evaluated (i.e., a conflict of interest form). Other Team Member: Qualifications and Experience A proven record of leadership in research, sound technical knowledge and relevant experience in in the agricultural sector, public policy, or an applicable social science field, or related relevant fields, program design and implementation, analysis and report writing; 1. At least five years’ experience in institutional capacity building and organizational development in developing countries 60 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV 2. Experience in conducting both qualitative and quantitative evaluations/assessments around improving capacity for service delivery and policy formulation; 3. A proven team player experience; and 4. Excellent spoken and written skills in English. VIII. EVALUATION SCHEDULE A. Period of Performance Work is to be carried out over a period of approximately Twelve (12) weeks, beginning on or about January 11, 2021 with field work and draft report completed March 15, 2021 and final report and close out concluding April 16, 2021. The below evaluation schedule is illustrative and will be updated in collaboration with USAID prior to finalization of the work plan Performance Evaluation Schedule Timing (Anticipate d submission date) Anticipated Duration / Level of Efforts (Days) Proposed Activities January 11 10, 2021 Evaluation contract awarded to Consultants January 22, 2021 5 Evaluation team completes pre-planning interviews of relevant Project, USAID/Ghana and METSS II staff. Data and information will be used to inform the development of the evaluation work plan January 29, 2021 5 Consultants prepare and submit a detailed evaluation work plan to METSS II Project February 2, 2021 1 Evaluation team (Consultants) conducts in-brief with METSS II, USAID and CSIR- SARI on evaluation work plan February 8, 2021 USAID, METSS II and CSIR- SARI review evaluation work plan and submit feedback to evaluation team February 10, 2021 1 Evaluation team integrates comments into work plan and submits final document to METSS II Project February 12, 2021 2 Evaluation team completes pre-testing of evaluation instruments/tools March 5, 2021 15 Collect data from evaluation respondents in the project target areas March 15, 2021 5 Analysis data and write the draft evaluation report March 26, 2021 USAID, METSS II and CSIR - SARI review evaluation draft report and submit feedback to evaluation team April 5, 2021 3 Evaluation team incorporates comments and prepare final evaluation report April 12, 2021 USAID, METSS II and CSIR-SARI review revised evaluation report and submits final feedback to evaluation team April 16, 2021 1 Evaluation team Leader submits formatted Final Evaluation Report IX. FINAL REPORT FORMAT The evaluation final report should include an abstract; executive summary; evaluation purpose; introduction; background of the project context and project being evaluated; the evaluation questions; the methodology; the limitations to the evaluation; key findings, conclusions, and recommendations; and lessons learned (if applicable). 61 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV The evaluation report should be easily understood and should identify key points clearly, distinctly, and succinctly. The findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. The Executive Summary should be 2-5 pages in length and present a concise and accurate statement of the most critical elements of the report. It should summarize key points (purpose and background, evaluation questions, methods, findings, conclusions, recommendations and lessons learnt). The report should provide a brief Background information, which should include country and/or sector context; specific problem or opportunity the intervention addresses; and the development hypothesis, theory of change, or simply how the intervention addresses the problem. Describe the specific strategy, project, activity, or intervention to be evaluated including (if available) award numbers, award dates, funding levels, and implementing partners. The Evaluation Purpose and Questions should state the purpose of, audience for, and anticipated use(s) of the evaluation. Clearly state the evaluation questions in this section. Explain the evaluation or survey Methodology in detail and clearly state sources of information. Disclose limitations to the evaluation, especially, those associated with the methodology (e.g. selection bias, recall bias, unobservable differences between comparator groups, etc.). NOTE: A summary of methodology can be included in the body of the report, with the full description provided as an annex. Findings, Conclusions, and Recommendations: Address all evaluation questions in this Statement of Work (SOW) or document why some evaluation questions and/or comments from METSS II Project or CSIR-SARI or USAID/Ghana have not been addressed for approval. If evaluation findings assess person-level outcomes, they should also be separately assessed for both males and females. Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. If recommendations are included, separate them from findings and conclusions. Recommendations should be supported with specific findings and should be action-oriented, practical and specific. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methods (e.g., in sampling; data availability; measurement; analysis; any potential bias such as sampling/selection, measurement, interviewer, response, etc.) and their implications for conclusions drawn from the evaluation findings. Annexes to the report must include:  Evaluation SOW (updated, not the original, if there were any modifications);  Evaluation methods;  All data collection and analysis tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides;  All sources of information or data, identified and listed;  Statements of difference regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team, if applicable;  Signed disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of or describing existing conflicts of interest; and  Summary information about evaluation team members, including qualifications, experience, and role on the team. The final version of the evaluation report will be submitted to USAID/Ghana and METSS II Project in hard copy as well as electronically. The report format should be restricted to Microsoft products and 62 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV 12- point Times New Roman font type should be used throughout the body of the report, with page margins 1” top/bottom and left/right. The main report should not exceed 45 pages, excluding the executive summary, references and annexes. The evaluation team leader shall incorporate METSS II Project, CSIR- SARI, and USAID/Ghana’s comments and submit the final report to USAID in electronic format (Microsoft Word) as well as printed and bound copies (Three copies in English) no later than six working days of the receipt of the comments. X. CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT The draft and final evaluation reports will be evaluated against the following criteria to ensure quality.  Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the project;  Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly;  The Executive Summary should present a concise and accurate statement of the most critical elements of the report;  Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with CSIR- SARI Staff, METSS II Project, USAID/Ghana;  Evaluation methodology should be explained in detail and sources of information or data properly identified;  Limitations to the evaluation should be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.);  Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions;  Conclusions should be specific, concise, and include an assessment of quality and strength of evidence to support them supported by strong quantitative and/or qualitative evidence;  If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females; and  If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific. XI. EVALUATION MANAGEMENT Logistical Support The METSS II Project will support with necessary logistical arrangements. CSIR-SARI together with METSS II will support the evaluation team identify key documents, and arrange for meetings with key stakeholders. The evaluation team is responsible for arranging other meetings as identified during the course of the evaluation and may request support from CSIR-SARI and METSS II project. In all these arrangements the USAID/Ghana CSIR-SARI COR and EG Office M&E Specialist should be notified prior to each of these meetings. The consultants should make arrangements for accommodation, vehicles needed for site visits, and meetings, and/or contact METSS II Project to make the arrangements with reasonable notice, if necessary. 63 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV However, the evaluation team members will be required to make their own payments for lodging, meals and incidental expenses. CSIR-SARI or the METSS II Project will make available their conference room or another convenient location upon request of the evaluation team as work space. The evaluation team is however, expected to use their own computers. Technical Direction Technical direction during the performance of the evaluation contract will be provided by the METSS II projects’ Chief of Operations (COO) and USAID/Ghana COR, who will be the point of contact for the Consultant’s Evaluation Team during performance. The COR is the only USG official authorized to make changes to the terms and conditions of the evaluation contract. In the event that the Consultant believes that it is required to perform activities outside the evaluation contract, they must immediately contact/inform the AOR before performing these tasks. The Consultant will not be paid any amount in excess of the evaluation budget set forth in the evaluation contract. CRITERIA EVALUATING THE TECHNICAL PROPOSAL Technical proposals will be evaluated based on the criteria outlined below Technical Evaluation Criteria Points Technical Approach 50 Personnel 50 Total Possible Evaluation Point 100 Applicants/Consultants should note that these criteria and sub-criteria serve: (a) as the standard against which the proposal will be evaluated, and (b) to identify the significant matters that Applicant should address in their proposal. The specific evaluation criteria are as follows: Technical Approach – (50 points) The technical approach should address the following sub-criteria which will be considered in the evaluation of the proposal: ● A clear strategy to address the evaluation questions, and achieve the evaluation requirements. ● Degree or extent to which the Applicant’s technical approach strategy, methodology and activities has the potential to accomplish the tasks within the implementation period. ● Clear articulation of anticipated challenges and description of how they will be addressed. ● An illustrative work plan with timeframes and durations for mobilizing staff, field work, data analysis, and reporting. Work plan should demonstrate logical linkages between task areas and timeframes that combines a strong balance of rigor and reflection. Personnel (50 points) Clear, concise and defined proposed key personnel, their role, responsibility, qualification and relevant experience. Include CVs of proposed key personnel, illustrating experience in similar assignments, and references. Refer to Section VII for the required qualification and experience of the required team members for the evaluation. Cost Proposal Evaluation Contract award will be made on a best value basis, where all non-cost (technical) factors will be significantly more important than cost. The consultants should have the structure that will allow it to 64 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV provide the best value and greatest results at the lowest cost. The price proposal of the contract period shall be evaluated in terms of reasonableness and realism to determine the appropriate cost for the work, including the firm’s understanding of the work, and their ability to perform the work. XII. LIST OF ANNEXES During the evaluation and upon request, CSIR-SARI, Economic Growth Office in USAID/Ghana and/or METSS II Project will avail to the contractor the following documents: 1. Council for Scientific and Industrial Research –Savanna Agricultural Research Institute Technical and Financial Support (CSIR- SARI) Project Description document; 2. CSIR-SARI Project M&E Plan 3. CSIR-SARI Project baseline report 4. CSIR-SARI Project Annual and Quarterly Reports –CSIR-SARI Project 5. List of districts and communities 6. List of beneficiaries and stakeholders 7. Case studies and internal evaluation documents developed by project implementers; and 8. Other special studies undertaken by the CSIR-SARI and USAID/Monitoring, Evaluation and Technical Support Services (METSS) as part of project implementation. 65 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV ANNEX Figure 1: CSIR-SARI Project Results Framework . . 66 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV List of Indicators NO. Results Area Indicator Title Level of Disaggregation IR 1: Improved Agricultural Productivity EG.3-6,7,8 Gross Margin per hectare, per animal, per cage obtained with USG assistance Crop: rice, Maize, Soy Sex: Male, Female IR 1: Improved Agricultural Productivity Sub-IR1.1: Enhanced human and institutional capacity development for increased sustainable agricultural sector productivity EG.3.2-17 Number of farmers and others who have applied new technologies or management practices with USG assistance Sex: male, female; Type of value chain actors: farmers, seeds producers, processors, traders; Technology type: Crop genetics; Cultural practices; Pest management, disease management; Soil￾related Fertility and Conservation; Irrigation; water management- non-irrigation￾based; Climate mitigation; Climate adaptation; Marketing and distribution; Post-harvest handling and storage; Value￾added Processing; Other IR 1: Improved Agricultural Productivity Sub-IR1.1: Enhanced human and institutional capacity development for increased sustainable agricultural sector productivity EG.3.2-2: Number of individuals who have received USG supported long￾term agricultural sector productivity or food security training Sex: Male, Female Duration: New, Continuing IR 1: Improved Agricultural Productivity Sub-IR1.1: Enhanced human and institutional capacity development for increased sustainable agricultural sector productivity EG.3.2-1: Number of individuals who have received USG supported short￾term agricultural sector productivity or food security training Type of individual Sex: Male, Female EG.3-1 Number of households benefiting directly from USG assistance under Feed the Future Duration: New, Continuing Location: Rural, Urban/peri￾urban. IR 1: Improved Agricultural Productivity Sub-IR1.2: Enhanced Technology Development, Dissemination, Management and Innovation EG.3.2-18: Number of hectares under improved technologies or management practices as a result of USG assistance Type of Technology Sex: Male, Female, Joint, Association-applied IR 1: Improved Agricultural Productivity Sub-IR1.2: Enhanced Technology Development, Dissemination, Management and Innovation EG.3.2-7: Number of technologies or management practices under research, under field testing, or made available for transfer as a result of USG assistance Phase of development: -Under research as a result of USG assistance; -Under field testing as a result of USG assistance; -Made available for transfer as a result of USG assistance 67 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV ANNEX II. PERFORMANCE INDICATOR TRACKING TABLE (PITT) (FY 2015–2019) 144 Indicator Title Base -line Yea r Base￾line Value 2016 Targe t 2016 Actua l 2016 % Achieve d 2017 Targe t 2017 Actual 2017 % Achieve d 2018 Targe t 2018 Actua l 2018 % Achieve d 2019 Target 2019 Actu al 2019 % Achieve d LOA Target LOA Achieved LOA % Achieve d Purpose level Indicators Outcome Indicator 1. Increase in yields of targeted crops for northern Ghana (Mt/ha)145 Maize FY1 5 1.92 n/a146 1.99 n/a n/a 2.05 n/a n/a 2.26 n/a n/a No info yet147 n/a n/a 2 n/a Rice FY1 5 2.75 n/a 2.92 n/a n/a 3.01 n/a n/a 2.96 n/a n/a Same n/a /a 2.5. n/a Soy FY1 5 1.65 n/a 1.65 n/a n/a 1.68 n/a n/a 1.72 n/a n/a Same n/a n/a 1.3 n/a Impact Indicator 2. % post-harvest losses for targeted crops148 Maize FY1 5 20 n/a 20 n/a n/a 18 n/a n/a 16 n/a n/a 15 n/a n/a n/a n/a Rice FY1 5 14.4 n/a 14.4 n/a n/a 14 n/a n/a 13 n/a n/a 12 n/a n/a n/a n/a Soy FY1 5 15.6 n/a 15.6 n/a n/a 14 n/a n/a 13 n/a n/a 12.5 n/a n/a n/a n/a Outcome Indicator 3 % increase in services provided by CSIR-SARI149 n/a n/a n/a -2150 n/a n/a 76% n/a n/a 115% n/a151 n/a -55% n/a152 n/a153 20% n/a 144 Methodology: This PITT was reconstructed retroactively by the CSIR SAR M&E Lead working in close collaboration with other senior staff who worked on the project. The first draft was prepared in April. A second revision was done in November 2011 to respond to comments from the METSS II Project. This PITT is based on six of the seven Feed the Future required indicators and the six custom indicators that were identified in the 2016 AMEP Plan that USAID/Ghana approved on September 23, 2016. The targets in this PITT are the ones identified in the AMEP unless otherwise explained in footnotes. 145 Based on data from MoFA. 146 n/a: non-applicable. 147 CSIR-SARI confirmed (November 10, 2021) that MoFA has not yet competed the analysis of its 2019 data so this data is not available yet. The data from an alternative source (FAO) shows yields of maize or the region that are 1.95, 2.88 for rice and 1.68 for soy. 148 Source: Sugri, I., Abubakari, M., Owusu, R. K., & Bidzakin, J. K.2021. Postharvest losses and mitigating technologies: evidence from Upper East Region of Ghana. Sustainable Futures, 3, 100048. doi:10.1016/j.sftr.2021.100048. 149 Definition: Calculation based on budget line for soil analysis , combine, tractor, seeds , workshop etc. (includes all IGF). 150 CSIR-SARI had a net loss in this year. 151 Negative loses were reported this year based on data from the CSIR-SARI Accounting Office 152 Negative loses were reported this year based on data from the CSIR-SARI Accounting Office 153 Although increasing IGF was one of the major goals of the project, the original proposal did not include a target for this. 68 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Indicator Title Base -line Yea r Base￾line Value 2016 Targe t 2016 Actua l 2016 % Achieve d 2017 Targe t 2017 Actual 2017 % Achieve d 2018 Targe t 2018 Actua l 2018 % Achieve d 2019 Target 2019 Actu al 2019 % Achieve d LOA Target LOA Achieved LOA % Achieve d Sub-Purpose 1: Capacity Development for agricultural research Outcome 1.1. EG.3.2-7: Number of technologies or management practices under research, under field testing, or made available for transfer as a result of USG assistance FY1 5 0 20 0 0% 20 0 0% 20 336 1,680% 20 0 0% 80 336 420% Outcome1.2.EG. 3.2-2: Number of individuals who have received USG supported long-term agricultural sector productivity or food security training FY1 5 0 3 3 100% 3 3 100% 3 2 67% 2 1 50% 9154 3 33% Output Indicator 1.1. Number of divisional, sectional, and M&E staff trained FY1 5 0 0155 n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a156 136157 (no double counting) n/a Output Indicator 1.2. Number of short, medium, and FY1 5 0 n/a 8 N/A N/A 12 N/A N/A 9 N/A N/A 8 N/A n/a 37 n/a 154 The original proposal anticipated funding graduate training for five individuals. (Pg 6). This target was increased to nine in the 2016 AMEP. 155 No annual targets were set. 156 No targets were set for this indicator in either the proposal or any of the three versions of the M&E plan. 157 Data sheets from the CSIR-SARI accounting office (Sebastian Tigbee). November 10, 2021. This is a more accurate figure than the tentative figure of 107 that was reported in the final report (Pg. 9). 69 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Indicator Title Base -line Yea r Base￾line Value 2016 Targe t 2016 Actua l 2016 % Achieve d 2017 Targe t 2017 Actual 2017 % Achieve d 2018 Targe t 2018 Actua l 2018 % Achieve d 2019 Target 2019 Actu al 2019 % Achieve d LOA Target LOA Achieved LOA % Achieve d long-term courses undertaken158 Output Indicator 1.3. Number of infrastructure rehabilitated or constructed FY1 5 0 n/a 2 n/a n/a 4 n/a n/a 7 n/a n/a 1 n/a 28 14159 50% Output Indicator 1.4. Number of new units established FY1 5 0 1 1 100% 1 1 100% 0 0 0 0 0 0 1 2 160 (the ICT and seed unit) 200% Sub-purpose 2: Agricultural technologies development and dissemination Outcome Indicator 2.1.EG.3.2-1: Number of individuals who have received USG supported short-term agricultural sector productivity or food security training FY1 5 0 10,15 4 2,356 23 11,17 7 5,879 53 13,41 2 6,006 45 14,306 553 4 49,049 161 14,794 30% Outcome Indicator 2.2. EG.3-1 Number of households benefiting FY1 5 0 11,00 0 11,49 7 105% 12,65 0 9,000 71% 7,864 6299 80% 9,043 545 6% 40,557 1627 27,341 67% 158 This figure does not include the people who participated in the M&E trainings but only those trained in the project sponsored workshops. The staff trained in M&E is reported under Indicator 3.4.A below. 159 No annua targets were given for infrastructure in the 2016 AMEP . 160 Although the nutrition unit existed before the project, it did not become functional until this project funded a state of the art nutrition lab. 161 This number from the 2016 AMEP includes targets for a sixth year (FY2020). If this number was deducted, the target would be 33,849 and the percent of achievement would be 38%. 162 This target from the 2016 AMEP includes targets for a sixty year (FY2020). If the numbers from this unfunded year are deducted, the target becomes 29, 587 and the project’s percent of achievement is 92 percent. 70 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Indicator Title Base -line Yea r Base￾line Value 2016 Targe t 2016 Actua l 2016 % Achieve d 2017 Targe t 2017 Actual 2017 % Achieve d 2018 Targe t 2018 Actua l 2018 % Achieve d 2019 Target 2019 Actu al 2019 % Achieve d LOA Target LOA Achieved LOA % Achieve d directly from USG assistance under Feed the Future Output Indicator 2.1. Number of new varieties of maize, rice, soybean and other mandate crops developed163 9 12164 133% Maize 10 0 0 0 0 n/a 7 n/a n/a 0 n/a n/a 0 n/a 3 7 239% Rice 11 0 0 0 0 n/a 0 n/a n/a 0 n/a n/a 4 n/a 3 4 133% Soybean 7 0 0 0 0 n/a 0 n/a n/a 1 n/a n/a 0 n/a 3 1 33% Output Indicator 2.2. Number of site-specific ISFM technologies developed for the three priority crops) FY1 5 0 3 draft 100% 3 3 revised based on researc h 100% 3 3 revise d based on field tests 100% 3 3 draft 100% 3 3 100% Phase I: Under research as a result of USG assistance 0 3 3 100% Phase II: Under field testing as a result of USG assistance 0 3 3 100% 3 3 100% Phase III: Made available for uptake as a result of USG assistance 0 3 3 100% Phase IV: Demonstrated 0 3 3 100% 163 Kusi et al. 2021. 164 The close-out report states that the project “contributed significantly to the release of nine high yielding climate smart varieties.” These nine varieties that were produced with direct support from this project. In fact, one of these nine varieties was not approved after the project closed. So, the actual number of varieties produced with direct support from the project and released was eight, not nine. In addition to this, another four varieties was produced and released through a combination of support from this project and other donor funded initiatives, bringing the total to twelve. 71 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Indicator Title Base -line Yea r Base￾line Value 2016 Targe t 2016 Actua l 2016 % Achieve d 2017 Targe t 2017 Actual 2017 % Achieve d 2018 Targe t 2018 Actua l 2018 % Achieve d 2019 Target 2019 Actu al 2019 % Achieve d LOA Target LOA Achieved LOA % Achieve d uptake by the public and/or private sector with USG assistance Output Indicator 2.3. Quantities of breeder and foundation seeds:165 2.3.A. Distributed (tons)166 FY1 5 145 Maize FY1 5 3.3 n/a 19.7 n/a n/a 15.2 n/a n/a 11.6 n/a n/a 2.1 n/a n/a 42 n/a Rice FY1 5 20.9 n/a n/a n/a 75 n/a n/a 97 n/a n/a 85 n/a n/a 51 n/a Soybean FY1 5 1.9 n/a 10 n/a n/a 12.5 n/a n/a 18.8 n/a n/a 21 n/a n/a 52 n/a 2.3.B. Produced (total) FY1 5 26.1 n/a167 29.7 n/a 67 102.7 153% 70 127.4 182% 71 108.1 152% 168168 367.9 219% Maize FY1 5 3.3 n/a 19.7 n/a n/a 15.2 n/a n/a 11.6 n/a n/a 2.1 n/a n/a n/a n/a Rice FY1 5 20.9 n/a n/a n/a n/a 75 n/a n/a 97 n/a n/a 85 n/a n/a n/a n/a Soybean FY1 5 1.9 n/a 10 n/a n/a 12.5 n/a n/a 18.8 n/a n/a 21 n/a n/a n/a n/a Output Indicator 2.4. Number of regional and district RELC meetings and FY1 5 48 n/a n/a n/a n/a 48 n/a n/a 48 n/a n/a 60 n/a n/a n/a n/a 165 Annual projections were not made for these only LOA targets. 167 The revised AMEP (2016) did not set any targets or 2016. 168 The revised AMEP (2016) included a target of 72 tons of see for FY 2020 the year. If this number is subtracted from the AMEP original LOA target 240) ten the LOA target to be considered in the evaluation is 168. 72 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Indicator Title Base -line Yea r Base￾line Value 2016 Targe t 2016 Actua l 2016 % Achieve d 2017 Targe t 2017 Actual 2017 % Achieve d 2018 Targe t 2018 Actua l 2018 % Achieve d 2019 Target 2019 Actu al 2019 % Achieve d LOA Target LOA Achieved LOA % Achieve d monitoring tours organized169 Output Indicator 2.5. Number of AEAs trained FY1 5 0 n/a n/a n/a 100 `144 144% 100 144 144% 100 180 180% 300170 468 156% Output Indicator 2.6. 171Number of on farm adapted trials or demonstrations conducted FY1 5 218 172 n/a 223 n/a 243 96 40% 243 79 33% 243 104 43% 729173 720 99% Sub-Purpose 3 Market and client-oriented research approach developed Outcome 3.1. EG.3.2-17 Number of farmers and others who have applied new technologies or management practices with USG assistance FY1 5 ?? 6,000 15,00 0 250% 13,20 0 12,850 97% 15,18 0 11,13 3 73% 18,216 401 2% 52,596 174 39,384 75% Outcome3.2. EG.3.2-18: Number of hectares under FY1 5 0 3,000 1,376 46% 3,300 3,235 98% 3,795 4,200 111% 4225.3 2 2532 1??? 10,095 8836.32 88% 169 Kusi et al. 2021. 170 The 2016 AMEP included a target for FY 2020. (100). If this target is deducted from the original LOA target in the 2016 in the AMEP, The target would be 300. This is not an EG indicator so this is permissible. 171 The data that CSIR-SARI is reporting for this indictor includes on-farm demonstration and adaptive trails that are funded by this project as well as other non-USAID funded demonstration trials (Dr. Prince Maxwell Etwire, November 10, 2021). The project did not keep a separate tracking of the ones that they supported since many were co￾funded. 172 This baseline figure includes demonstration and adaptive trials that are funded by other projects as we as by the USAID project. 173 The 2016 AMEP included a target or Y2020 (243). If this target is deducted from the original target in the 2016 AMEP (973), the adjusted LOA target would be 729. Since this is not an EG indicator, this adjustment was made. 174 This target which was presented in the 2016 AMEP included data for a sixth year (FY 2020) which the project never got. The target through FY 2019 was actually 23,308 which would make the rate of achievement 170 percent vs. 75 percent. 73 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Indicator Title Base -line Yea r Base￾line Value 2016 Targe t 2016 Actua l 2016 % Achieve d 2017 Targe t 2017 Actual 2017 % Achieve d 2018 Targe t 2018 Actua l 2018 % Achieve d 2019 Target 2019 Actu al 2019 % Achieve d LOA Target LOA Achieved LOA % Achieve d improved technologies or management practices as a result of USG assistance Output Indicator 3.1. Number of communication and knowledge systems developed FY1 5 0 2 1 50% 0 0 0 0 0 0 0 0 0 2 175 1 50% Output Indicator 3.2. Number of radio, TV, web based programs transmitted (Total) Radio 0 3 1 n/a 12 n/a n/a 15 n/a n/a 17 n/a 50176 47 94% TV n/a 1 n/a n/a 14 n/a n/a 12 n/a n/a 15 n/a 30 34 113% Newsprint n/a 5 n/a n/a 12 n/a n/a 9 n/a n/a 18 n/a 50 44 88% Leaflet n/a 13 n/a n/a 12 n/a n/a 12 n/a n/a 7 n/a 50 44 88% Newsletter n/a 4 n/a n/a 4 n/a n/a 5 n/a n/a 4 n/a 16 17 106% Exhibition n/a 4 n/a n/a 4 n/a n/a 5 n/a n/a 6 n/a 16 19 119% Outcome Indicator 3.2. Number of BPs developed for commercialization department FY1 5 00 0 0 0 1 0% 0% 1 0% 0% 1 1 100% 1 1 100% Output Indicator 3.3. Number of households interviewed to assess the impact of CSIR-SARI activities 0 0 500 1000 200% 500 0 0% 500 0 0% 500 500 100% 500 year annual sample One baseline and one final N/A 175 The IT unit already existed when the project was created. The goal was to create a communication unit within this. The other target was for a knowledge management system to be created. This was not created. 176 Wilhelm Kutah, November 10, 2021 based on the 2017 Communications Plan updated based on project reports November 12, 2021. 74 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Indicator Title Base -line Yea r Base￾line Value 2016 Targe t 2016 Actua l 2016 % Achieve d 2017 Targe t 2017 Actual 2017 % Achieve d 2018 Targe t 2018 Actua l 2018 % Achieve d 2019 Target 2019 Actu al 2019 % Achieve d LOA Target LOA Achieved LOA % Achieve d Output Indicator 3.4. M&E system for SARI in place and operational 3.4.A.# of staff trained in M&E 0 0 n/a 40 n/a n/a P n/a n/a P n/a n/a P n/a n/a 5 n/a 3.4.B.M&E plans developed FY1 5 1 177 1 revise d 1 178 100% 0 0 0 0 0 0 0 0 0 1 Novemb er 12, 2021.1 100% 3.4.C.SARI M&E position created and operational179 FY1 5 0 1 1 100% 1 1 100% 1 1 100% 1 3 300% n/a n/a n/a Source: EG indicators--Prince Maxwell Etwire based on project reports. March 16, 2021. Sources: Non-EG Indicators: Prince Maxwell Etwire, Nicholas Denwar, Asia M. Mohammed Goria B. Adu, Samuel O. Abrebrese, Doris K. Pouzaa, Robert K. Owusu, and Wilhelm N. Kutah. April 23 2021 (draft). Revised to fit the new PITT format (October 21- 27, 2021). 177 A draft AMEP was submitted with the proposal. 178The draft AMEP was revised. 179 1) In 2016-2017, the same person served as the M&E lead. 2) In 2018, a new person took up the role. 3) In 2019, another new person took up the role ( Prince Maxwell Etwire). In addition to this, three other members of the economics team (recently returned from graduate studies) were made M&E focal points for the three sub stations in addition to their other duties. One of the three left, bringing the total number of M&E focal persons to three in 2019. To date, none of the three M&E focal point persons have been officially appointed to these roles so that they are recognized on their annal level of effort evaluation. 75 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV ANNEX III. FIELDWORK Annex III.A. USAID/GHANA’s CSIR-SARI Technical and Financial Support Project Evaluation Work Plan Tentative Dates and Phases Activity Key Deliverables Phase I: Pre-Planning January 28 Initial meetings with METSS II COO and revision of contracts February 2 Contracts prepared and signed Contracts February 2-February 23 Stakeholder lists co-facilitated with evaluation working group Documentation co-identified with evaluation working group Documentation review by ET and pre-analysis Documentation March 5-12 Stakeholder and beneficiary list revised and verified Stakeholder list (draft) February 23-March 17 Development of first draft of work plan Draft work plan to METSS II and SARI evaluation working group and evaluation focal person for internal review March 19 Submit draft work plan to METSS and USAID/Ghana March 24 Receive comments Integrate comments and resubmit March 29 Resubmit work plan April 9 Work plan approved Phase II: Fieldwork March 29-April 9 Pilot test/revise FGD/KII guides and post interview questionnaire as well as draft summary tables for inclusion in the report (Proved by METSS) Pilot-test/revise questionnaires Collaborate with evaluation focal person in setting up the interviews and confirming contact information that is needed for he interviews Interviews set up April 9-30 FGD and KII interviews and progressive launch of the post￾interview online questionnaire Interviews Phase III: Data clean-up and analysis and additional documentation and data collection and review May 1-28 Phase IV. Write up and Team Internal Reviews June 1-July 5 Draft Report Source: CSIR-SARI Technical and Financial Support Project Final Performance Evaluation. July 5, 2021. 76 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex III.B. List of People Interviewed in the CSIR-SARI Technical and Financial Support Project FGDs and KIIs, March–April 2021 No. First and Last Name Organization Function/Role Category180 1 Gilbert Nachin CSIR Former Administrator/ Project Manager, CSIR-SARI 1 2 Christopher Angometry CSIR-SARI Auditor 1 3 Washeed Alhassan UTV Journalist 5 4 Jonathan Agawini CSIR-SARI Field Officer 4 5 Eric Quaye CSIR-SARI Deputy Director in Charge of Ghana Seed Inspection Division 2 6 Janet Chigabatia Adama CSIR-SARI Former Board Member 1 7 Wilson Dogbe CSIR-SARI Former Rice Component Lead and Acting Project Manager 1 8 Philip Nyamah CSIR-SARI Snr. Administrative Officer 1 9 Robert K. Owusu CSIR-SARI Senior Scientific Secretary (Post Harvest Specialist) 1 10 Nicholas Denwar CSIR-SARI Snr. Research Scientist 1 11 Alidu Feruza CSIR-SARI Principal Administrative Assistant 1 12 Issah Issifu CSIR-SARI Principal Library Assistant 1 13 Musah Iddi CSIR-SARI Chief Works Superintendent 1 14 Wilhelm Kutah CSIR-SARI Communication Specialist 1 15 Augustine Owusu CSIR-SARI Snr. Works Superintendent 1 16 Zakaria Sieni CSIR-SARI Chief Technical Officer 1 17 Samuel Musah CSIR-SARI Works Superintendent 1 18 Sammy Voleto CSIR-SARI Works Superintendent 1 19 Mohammed Jabiru CSIR-SARI Prin. Works Superintendent 1 20 Sebastian Tigbee CSIR-SARI Project Accountant 1 21 Paul Berko CSIR-SARI Accountant 1 22 Abdulai Baba Alhassan CSIR-SARI Accountant 1 23 Mahama A. Rufai CSIR-SARI Accountant 1 24 Alhassan Abukari CSIR-SARI Principal Stores Superintendent 1 25 Wumbei Mohammed CSIR-SARI Principal Accounting Assistant 1 26 Yakubu Mohammed CSIR-SARI Principal Accounting Assistant 1 27 Zulai Abhihiba CSIR-SARI Principal Accounting Assistant 1 28 Francis Kusi CSIR-SARI Snr. Research Scientist 1 29 Roger A.L. Kanton CSIR-SARI Deputy Director 1 30 Julius Yirzagla CSIR-SARI Research Scientist 1 31 Issah Sugri CSIR-SARI Snr. Research Scientist 1 32 Nelson .K. Abass CSIR-SARI Chief Accounting Assistant 1 33 Saaka Buah CSIR-SARI Director of SARI 1 34 Edward Martey CSIR-SARI Research Scientist 1 180 Category 1- CSIR-SARI staff Category 2- Government agencies the project collaborated with Category 3- Academic and research institution partners and other donor-funded projects Category 4- Local partners (community-based groups, associations that benefitted from training and demonstration trials, extension agents working with the three CSIR-SARI farming systems regional groups, farmers associations that participated in trainings/demonstration, associations producing seed, CSIR-SARI field staff that worked with community-based groups) 77 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV No. First and Last Name Organization Function/Role Category180 35 Samuel Oppong A. CSIR-SARI Research Scientist 1 36 Haruna Bashiru CSIR-SARI Principal Technologist 1 37 Nutsugah C. Mawuli CSIR-SARI Principal Technical Officer 1 38 Desmond Adogoba Sunday CSIR-SARI Principal Technologist 1 39 Alhassan Nuhu Jinbaani CSIR-SARI Principal Technologist 1 40 Alhassan Lansa Abdulai CSIR-SARI Principal Technologist 1 41 George Mahama CSIR-SARI Research Scientist 1 42 Godwin Opoku CSIR-SARI Chief Technical Officer 1 43 Bella Thomas CSIR-SARI Senior Technical Officer 1 44 Asieku Yahaya CSIR-SARI Research Scientist 1 45 Abdul-Karim Alhassan CSIR-SARI Chief Accounting Assistant 1 46 Asiata Ali CSIR-SARI Chief Admin. Assist. 1 47 Anslem B. Nyuor CSIR-SARI Principal Technologist 1 48 Iddrisu Yahaya CSIR-SARI Research Scientist 1 49 Abubakari Mutari CSIR-SARI Research Scientist- 1 50 Abampoka A. Paulina CSIR-SARI Snr. Technical Officer 1 51 Rosina Nyaboase Former staff of SARI Office Manager/Secretary 1 52 Samuel Braimah Delwinde Development Consult SARI Strategic Plan Consultant 1 53 Prince Etwire CSIR-SARI Research Scientist 1 54 John .K. Bidzakin CSIR-SARI Research Scientist 1 55 John Azu USAID Africa Lead Africa Lead Technical Support Services Consultant 1 56 Christopher Akai PPRSD Plant Protectionist 2 57 Japhet Asante PPRSD Seed Inspector 2 58 Paul Hixon University of Illinois Project Lead in Breeding (plant breeder) 3 59 Prof. Brian Diers University of Illinois Project Lead on the ICB 3 60 Doris K. Puozaa CSIR-SARI Research Scientist (Team leader for the seed unit) 1 61 Michael Wilson CSIR Institute for Scientific and Technological Information Computer Engineer 2 62 Gloria Boakyewaa Adu CSIR-SARI Research Scientist (Maize component lead) 1 63 Hawa Musah MoFA MoFA Regional Director- Northern Region 2 64 Dr Victor Agyeman CSIR Director General-CSIR 2 65 Musa Taylor ATT Deputy Chief of Party, ATT 3 66 Edward Appiah CSIR-SARI Marketing Officer 1 67 Mumuni Abubakari CSIR-SARI Chief. Marketing Assistant 1 68 Yamyolya B. Alhassan CSIR-SARI Principal Marketing Assistant 1 69 Sasu Yeboah MoFA MoFA Regional Director- Upper West Region 2 70 Francis Enor MoFA MoFA Regional Director- Upper East Region 2 71 Joshua Diedong MoFA Regional Agriculture Officer 2 72 Akuliya Godwin MoFA Regional Agriculture Officer 2 78 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV No. First and Last Name Organization Function/Role Category180 73 Robert Kpefatinga Issaka MoFA Regional Agriculture Officer 2 74 Asodina Francis MoFA Regional Agriculture Officer 2 75 Alhassan Zimi MoFA Regional Agriculture Officer 2 76 Timothy Zangina MoFA Regional Agriculture Officer 2 77 Abukari Abdulai Heritage Seed Company Owner 4 78 Alhaji Issahaku Mahama Tiyumtaba Farms Owner 4 79 Sumani Iddrisu Mohammed CSIR-SARI Field Staff (West Gonja) 4 80 Albert Futukpor Ghana News Agency Journalist 5 81 Arnold Asafu-Adjaye 123 FM Journalist 5 82 Samuel Sam B&FT Journalist 5 83 Fuseina Issah Savannah Radio (now GBC Accra) Journalist 5 84 Abu Sakara SAKFOS Farms Chief Executive Officer (CEO) 4 85 Alhaji Mallam Seidu Kokobila farms CEO 4 86 David Konlan IWAD Seed Business Manager 4 87 Fatawu Imoro181 Golinga Irrigation Farmers Group Secretary 4 88 Joseph Adjabui MoFA Agriculture Extension Officers (Bolga) 4 89 Sarah Ayamba MoFA Agriculture Extension Officers (Bolga) 4 90 Yao Awulsor Moses MoFA Agriculture Extension Officer 4 91 Ridwan Harun MoFA Agriculture Extension Officer 4 92 Grace Sebugah USAID Ag. Program Management Specialist 6 93 Nathan Biassey Seyan Enterprises Agro-Dealer 4 94 Abdul Aziz Nasir MoFA Agriculture Extension Officer 4 95 Roger Anyueta MoFA Agriculture Extension Officer 4 96 Saed Sumaila MoFA Agriculture Extension Officer 4 97 Saaka Adams Chief Operating Officer METSS II 3 98 Fuseini Sulemana Worebogu Tiyumtaba Womens Group Soy Farmer 4 99 Stephen Issifu Tishee Kpangmanga Farmer Association Maize Farmer 4 100 Augusta Nyamadi Clottey NASTAG CEO 4 Source: A. Dela Akanko. March 27, 2021. 181 The interview included 18 members of the farmer group (12 males and six females). 79 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex III.C. Target vs. Achievement of the Number of People Interviewed in Different Stakeholder Groups for the CSIR￾SARI Technical and Financial Support Project Final Evaluation Interviews (KII & FGD) Target Interviews (FGD & KII) Achieved People Target People (includes double counting) Achieved 5/10/2021 People (excludes double counting) Post Interview Questionnaire Target Post Interview Questionnaire Achieved 5/10/2021 Original Target Category 1: CSIR￾SARI Project Staff and Oversight Committees 15 22 60 63 57 (49 male and 8 female) 60 60 (includes CSIR-SARI staff in category 4.A.1) sent questionnaire (31 responded) (52% response rate) 60 (33%) of the 180 people identified as core (C) and non-core (NC) and about 100% of the CSIR￾SARI staff identified as core (Annex V in Work Plan) Category 2: Government Agencies the Project Collaborated with 7 7 13 14 14 (13 male and 1 female) No target 17 sent questionnaire (14 responded) (82% response rate) 13 (100% of the core informants identified for this category will be targeted by FGDs & KIIs) Category 3: Academic and Research Institution Partners and Other Donor￾Funded Projects 4 3 7 4 4 (4 male; 0 female) No target 7 (100% of the core informants identified for this category) Category 4: Local Partners (for activities they engaged in) n/a 15 n//a 19182 19 (17 male; 2 female) n/a n/a n/a 182 The figure includes 7 community level Agricultural Extension Agents who were interviewed as stakeholders from Category 4 because of their engagement at the community level. 80 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Interviews (KII & FGD) Target Interviews (FGD & KII) Achieved People Target People (includes double counting) Achieved 5/10/2021 People (excludes double counting) Post Interview Questionnaire Target Post Interview Questionnaire Achieved 5/10/2021 Original Target 4.A. Community￾Based Associations that Benefitted from Training and Demonstration Trials 4.A.1. CSIR-SARI staff familiar with trials 3 2 3 2 3 3 (as part of the CSIR-SARI staff questionnaire) 6 183 (100%) 4.A.2. Community￾based groups (New 4/23 Target revised) involved in demonstration and adaptive trials 9 (revised to 2) (new 4/23) 2 2 2 n/a n/a 9 184 (3 per region)185 4.A.3. MoFA AEAs trained under the project (New 4/23 Stakeholder Category Added) No target in work plan (new 4/23) 5 6 7 7 All AEAs sent Merged with government and partner group No target. Added. 4.B. Associations Producing Seed 4.B.1. Golinga Innovation Platform 2 1 (Golinga) 3-6 1 male facilitator and 18 n/a n/a 3 groups (Golinga, Botanga, Navrongo) (66% 183 The project conducted demonstration trials in six districts. Unfortunately, the project did not keep very accurate records on the location of these groups since the point of contact is the extension agent, so the total number of groups is not known. A representative sample of groups will be chosen from a sample identified by the extension agents and SARI staff who worked with these groups (Text Box 1). 184 The CSIR-SARI project did not work directly with community-based groups in this category, rather they worked through the extension agents. 185 This is the one community-based category for which it is not yet possible to document the total number of stakeholders. 81 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Interviews (KII & FGD) Target Interviews (FGD & KII) Achieved People Target People (includes double counting) Achieved 5/10/2021 People (excludes double counting) Post Interview Questionnaire Target Post Interview Questionnaire Achieved 5/10/2021 Original Target people (6 female; 12 male) People coverage of three groups)186 4.B.2. IWAD (soybean only) 1 1 1 1 n/a n/a 1 key contact. Members TBD 4.B.3. SEEDPAG 2 n/a 8 The President was interviewed in another group interview n/a n/a 72 members187 (11%) 4.C. Private Seed Companies 1 1 4 2 n/a n/a 100% of the four most active companies 4 people 4.D. Large-Scale Commercial Farms 1 3 2 n/a n/a 100% of the 3 most active farms 4.E. NASTAG 1 1 1 1 n/a n/a 1 (100% of the most important seed trade association the project collaborated with) 4.F. Agro-dealers (New Stakeholder Category Added 4/23) New target (4/23) 1 1 No target 1 n/a n/a 186 In conjunction with the sampling, the M&E specialist will determine if there are significant differences between the three groups. If not, the team will interview leaders from two of the three groups. 187 Sampling will classify the male and female participants by level of engagement as a basis for choosing the sample. 82 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Interviews (KII & FGD) Target Interviews (FGD & KII) Achieved People Target People (includes double counting) Achieved 5/10/2021 People (excludes double counting) Post Interview Questionnaire Target Post Interview Questionnaire Achieved 5/10/2021 Original Target Category 5: Media/Communication (for the activities they participated in) 3 32 7 5 journalists + The CSIR￾SARI media specialist =6 (5 male and 1 female) 5 (4 male and 1 female) n/a n/a 7 (100%) Category 6: USAID 3 1 5 1 1 (1 female) n/a 5 (100%) Total (all categories) 19 50 107 + 18 community￾based participants (includes double counting) 100 (87 male and 13 female) without double counting 60 to staff No target partners 31 staff 14 Partners 45 Total No target Source: CSIR-SARI Final Performance Evaluation. June. 2021. 83 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex III.D. Stakeholder List Totals from Which Sample Was Drawn, Revised Stakeholder List from Which Sample Was Drawn, Number of People Included in Interviews, and Total Number of People Interviewed for Each Stakeholder Category and Sub-category in the CSIR-SARI Technical and Financial Support Project Final Evaluation Stakeholder Category Proposed Samples for Data Collection Using KIIs Proposed Samples for Data Collection Using FGDs Proposed Samples for Data Collection Using Email Questionnaire Initial Stakeholder List Totals from Which Sample Drawn Final Stakeholder List from Which Sample Drawn (verified by consultants) # of People Included in FGDs/KIIs (including double counting) # of People included in Question￾naire # of People in Each Category Interviewed Totals (without double counting) Category 1: CSIR￾SARI Project Staff and Oversight Committees 4 11 60 181 101 63 57 57 Category 2: Government Agencies the Project Collaborated with 4 3 15 13 14 14 14 Category 3: Academic and Research Institution Partners and Other Donor￾Funded Projects 2 1 4 8 7 4 3 4 Category 4: Local Partners (for activities they engaged in) 35 25 19 19 4.A. Community￾Based Associations that Benefitted from Training and 84 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Stakeholder Category Proposed Samples for Data Collection Using KIIs Proposed Samples for Data Collection Using FGDs Proposed Samples for Data Collection Using Email Questionnaire Initial Stakeholder List Totals from Which Sample Drawn Final Stakeholder List from Which Sample Drawn (verified by consultants) # of People Included in FGDs/KIIs (including double counting) # of People included in Question￾naire # of People in Each Category Interviewed Totals (without double counting) Demonstration Trials 4.A.1. CSIR-SARI staff familiar with trials 3 2 3 2 2 4.A.2. Community-based groups (New 4/23 Target revised) involved in demonstration and adaptive trials 9 NA 2 2 4.A.3. MoFA AEAs (trained under the project (New 4/23 Stakeholder Category Added) 8 7 7 4.B. Associations Producing Seed 4.B.1. Golinga Innovation Platform 2 1 1 1 4.B.2. IWAD (soybean only) 1 1 1 1 1 4.B.3. SEEDPAG 2 2 4.C. Private Seed Companies 1 4 2 2 85 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Stakeholder Category Proposed Samples for Data Collection Using KIIs Proposed Samples for Data Collection Using FGDs Proposed Samples for Data Collection Using Email Questionnaire Initial Stakeholder List Totals from Which Sample Drawn Final Stakeholder List from Which Sample Drawn (verified by consultants) # of People Included in FGDs/KIIs (including double counting) # of People included in Question￾naire # of People in Each Category Interviewed Totals (without double counting) 4.D. Large-Scale Commercial Farms 1 3 2 2 4.E. NASTAG 1 1 1 1 4.F. Agro-dealers (New Stakeholder Category Added 4/23) N/A N/A N/A 1 1 Category 5: Media/Communic ation (for the activities they participated in) 2 1 6 6 6 5 Category 6: USAID 2 1 4 4 1 1 Total (all categories) 19 33 249 156 107 100 Source: A. Dela Akanko. May 27, 2021. 86 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex III.E.1. Background Qualifications, Roles, and Responsibilities of the CSIR-SARI Technical and Financial Support Project Final Evaluation Team Team Member Qualifications How Qualifications Contribute Role and Responsibilities Della E. McMillan -Ph.D. in Anthropology from Northwestern University and Senior Research Scientist in the Department of Anthropology at the University of Florida (Adjunct) -40 years of experience in working with multi-disciplinary agricultural research and extension -Over 30 years of experience in development policy research and familiarity with USAID M&E procedures, including serving as team leader on 19 mid-term and final USAID project evaluations -Experience with the design, execution, and assessment of university and NGO capacity￾building programs in Africa and US M&E background and familiarity with USAID evaluation norms. Facilitates the evaluation design and reporting process -Team leader -Lead writer on work plan, reports, and weekly updates -Co-lead (with Akanko) on developing stakeholder lists and retrieving and organizing critical documentation -Shared responsibility (with Akanko and Ampofo) for interviewing Kwasi Ampofo -Ph.D. in Agricultural Entomology from University of Queensland, Australia -Over 40 years of experience in the CGIAR system and agricultural NGOs in Africa -Over 40 years of experience as a project manager of USAID and other international donor-funded agricultural research and extension programs Extensive experience in technical organizational research and management of agricultural research institutions and projects. Facilitates the ET’s understanding of CSIR￾SARI’s technical and organizational issues and achievements -Co-lead (with Hurtak) on detailed documentation review -Lead on management and financial review in relation to EQs 1 and 2 -Shared responsibility for interviews and write-ups (with Akanko and McMillan) Annie Dela Akanko -MBA in Leadership and Management, York St. John University of London. Certificate in Sustainable Development, University, London. BA Integrated Development Studies, University for Development Studies (UDS) Ghana -Over 16 years of M&E experience with a strong focus on the design, organization, and conduct of FGDs, KIIs, and online surveys in a number of countries with a focus on Ghana -Strong background in capacity development and capacity assessments M&E background and familiarity with the Tamale area and local language, UDS, and SARI makes her an effective national coordinator for the evaluation -Co-lead with the team leader on pre-planning, including the elaboration of the stakeholder list and documentation base -National coordinator for the evaluation and homologue of the evaluation focal person -Lead on community-based interviews 87 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Team Member Qualifications How Qualifications Contribute Role and Responsibilities Lynn Hurtak -MS in Education from the University of Florida -13 years of experience managing and co-managing USAID evaluations and M&E processes in 11 African countries, including four years in Ghana -Experience with the design, implementation, and analysis of simple user-friendly online and email-based surveys in Africa and the US Background in M&E and Ghana will facilitate the management and logistics of project and ensure a high-level analysis of FGDs and KIIs -Deputy team leader with shared responsibility for writing up work plan, final report, and weekly reports -Lead on creation of the data instruments and data analysis system, including creation of the evaluation database -Responsible for budget management, final reporting, and documentation -Co-lead with Ampofo on detailed document review Source: McMillan, Akanko, Ampofo, and Hurtak. 2021. CSIR-SARI Technical and Financial Support Project Evaluation Work Plan. Accra, Ghana: METSS II Project. Annex III.E.2. Background Qualifications, Roles, and Responsibilities of the CSIR-SARI Technical and Financial Support Project Final Evaluation Focal Person and Evaluation Working Group CSIR-SARI Evaluation Support Team Qualifications How Qualifications Contribute Evaluation Role and Responsibilities Prince Maxwell Etwire -Ph.D. in economics -Final M&E Advisor for the project -Participated in many USAID￾funded M&E trainings As the key person responsible for compiling reports during the last stages of the project, he has a good understanding of the key documents and individuals Evaluation focal person (appointed by the CSIR￾SARI director) Robert Owusu -MSc in Post-harvest and Food Preservation Engineering -Senior Scientific Secretary -Acted as Project Manager for 11 months The four members of the evaluation working group have extensive experience with CSIR￾SARI and supported the project. As such, they provide the ET with a co￾active institutional memory and understanding of the project, its timeline, and critical documentation Evaluation working group (appointed by the SARI director). This group supports the evaluation focal person in retroactively creating a project stakeholder database and helping collect quantitative data needed to measure the major project outcomes and outputs using the updated IPTT guide. Sabastian Tigbee -Bachelor of Commerce￾University of Cape Coast Ghana. -Project Accounts Officer -6 years working with the project and over 15 years total of working with USAID-sponsored projects at SARI Wilson Dogbe -Ph.D. in Soil Science and Certificate in Farming Systems Research, ICRA, Netherlands -Principal Research Scientist, CSIR-SARI, Nyankpala with 88 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV CSIR-SARI Evaluation Support Team Qualifications How Qualifications Contribute Evaluation Role and Responsibilities 30 years of experience in rice research and development -Over 25 years of experience in the management of local and international-funded projects for CSIR -Serviced as the last Project Manager, March 2017-2019 Nicholas N. Dewar Ph.D. in Agronomy (plant breeding) -Senior Research Scientist who carried out activities under variety development for the project and served as team lead for crop improvement for SARI since 2018 -Component lead for soybean Source: McMillan, Akanko, Ampofo, and Hurtak. 2021. CSIR-SARI Technical and Financial Support Project Evaluation Work Plan. Accra, Ghana: METSS II 89 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV ANNEX IV. DATA ANALYSIS Annex IV. Table 1. Most Frequently Cited Examples of Best Practice in Achieving Results by Stakeholder Category (1-6)188 Best Practices and Illustrative Quotes 1 2, 3, 6 5 4.F 4.A￾4.E Total # FGDs & KIIs that answered question 24 18 3 1 3 SP 1 Management training and other training of the senior CSIR-SARI staff was important. 6 2 8 Support for infrastructure development (offices, equipment, and internet) and farm equipment at CSIR-SARI was critical/good. 4 2 6 Support to update CSIR-SARI strategic planning process. 5 5 The project trained staff in accounting and M&E in ways that strengthened staff capacity in accountability and reporting and is being scaled up to other projects. 8 8 SP 2 Collaborative on-farm research with MoFA: 33 The project’s strong collaboration with MoFA and support for MoFA AEA training orchestrated through the field stations for demonstration and multi-site trials is “absolutely critical to the COE mandate of both MoFA and SARI.” 189 9 12 1 1 23 MoFA AEA training is an example of best practice that needs to be continued and scaled up. This allows MoFA agents to work better with farmers. 11 11 Gender mainstreaming in field research and collaboration with MoFA: Strong record for targeting and including women and youth in: 1) on-farm trials and field days; and 2) strong encouragement of women and women’s groups to become commercial seed producers. as well as “Women…are better connected to SARI. They ae feeling more confident and having their own fields; they are improving their livelihoods.” 5 5 1 3 14 Modernization of CSIR-SARI’s seed operations: 18 Strong collaboration between the GoG Seed Certification Unit and CSIR-SARI. 4 188 n=Number of times a best practice was identified in a FGD or KII. Category 1: SARI staff and consultants; Category 2: GoG, academic and research partners; other USAID-funded projects; USAID/Ghana. Categories 2,3, and 6: USAID. Category 5: Media partners. Category 4.F. Agro-dealer; 4.A-E. Other community-based stakeholder groups. 189 Stakeholders noted this facilitated CSIR-SARI: 1) having information on a wide group of geographical locations; 2) reducing the number of years needed to complete the breeding cycles; 3) testing and releasing new varieties developed by them and other projects over a wide geographical area; 4) accelerating the dissemination of new varieties developed but not yet disseminated with support from earlier (a.k.a. legacy) projects; 5) strengthened MoFA agents’ connections with CSIR-SARI field stations; 6) CSIR-SARI researchers in the field stations were accessible (by telephone) to the AEAs, seed producers, and agro￾dealers was important: “When we call them now, they pick up the phone;” and 7) Another stated: “These demonstration trials are absolutely critical to the core mandate of both MoFA and SARI.” 90 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Best Practices and Illustrative Quotes 1 2, 3, 6 5 4.F 4.A￾4.E Total Strong collaboration with private-sector seed producers and their collaboration with the out growers (that needs to be scaled up). 3 4 CSIR-SARI’s support for the development of improved seed “is very vital to the economy of the country and wellbeing of the farmers and needs to be scaled up.” 1 6 SP 3 ICT/Communication: The construction and equipping of a new ICT center and co￾development of CSIR-SARI’s first communication including publicity and marketing and associated scale up of a wide variety of outreach initiatives (radio, manuals, etc.), some of which were identified as best practice in FGDs/KIIs.190 10 7 3 1 3 24 New project designs for CSIR-SARI: 8 Design: The initial project design being based on a comprehensive needs assessment (by Africa Lead) contributed to the quality of the design. 2 2 Collaboration: There was a powerful synergy between the ADVANCE, ATT, SIL, and the project.191 2 1 3 Component Teams: The project’s clustered leadership and crop-specific working groups (components) introduced in 2017 was a good approach. 3 3 Source: CSIR-SARI Technical and Financial Support Project, Final Performance Evaluation, FGDs and KIIs. Re￾analysis of frequencies breakdown. June 9, 2021. 190 Key examples of best practice identified include: 1) transmitting in local languages; 2) live broadcasts to farmers with call ins; 3) inviting media to various events like launches; 3) building and equipping the ICT center; 4) live broadcasts of farmers using the new technologies; 5) collaborating with ATT on the development of CSIR-SARI’s first ICT strategy and the initial staffing of the center; 6) consolidating information used in trainings into manuals which facilitates MoFA agents scaling up their training; 7) hiring several UDS journalism students to work as interns; and 8) “helping SARI improve its weak internet connectivity by accessing the GARNET system and developing SARI’s first fully functional website and online database storage,” which was mentioned here but also in many of the other EQ discussions. One AEA agent noted: “Many of the local and regional radio stations are now imitating the MoFA agent moderated call in programs the project started.” Another reported: “The project’s role plays and video programs improved unity and cohesion in the community about improved technologies.” 191 The collaboration with SIL for the soybean value chain and ICT was frequently cited as an example of best practice in the discussion section of many EQs. 91 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex IV. Table 2. Most Frequently Cited Examples of Lessons Learned in Achieving Results in the FGDs and KIIs of the Direct Support Project Final Performance Evaluation by Stakeholder Categories (1-6)192 Lessons Learned and Illustrative Quotes Group 1 Groups 2, 3, 6 Group 5 Group 4.F Groups 4.A-E Total # FGDs & KIIs 24 18 3 1 3 Gender Mainstreaming (Cross-cutting) Future projects need to: 8 —Anticipate a robust gender integration plan and appropriate staffing and budgetary support to strengthen gender mainstreaming (including helping CSIR-SARI have a recognized gender coordinator). 3 1 1 5 —Scale up a group of activities to help build the capacity of women to develop commercial seed production and other value chain activities. 2 1 3 Management (SP 1) Future projects need to: 39 —Consider predicating the initial disbursement of funds to CSIR-SARI having in place the necessary management structures and oversight structures (i.e. an independent project manager and accountant and informed steering committee to interface with the management board) . 2 8 10 —Consider hiring a manager from outside the institution to ensure appropriate mentoring. 1 1 —Ensure conditions are in place to hire (or secund) and retain consistent independent leadership for key functions (project manager, M&E coordinator, accountant, gender focal point). 3 3 —Conduct organizational assessments as a basis for the design of the project. Anticipate the need for a baseline organizational assessment (like the one done by Africa Lead for this project) as a basis for setting targets for a more “wholistic approach to capacity-building” that targets and tracks key capacities wholistic development of key capacities including exchange visits and a transparent process for identifying and monitoring who gets access to the project sponsored training and when. 3 2 4 —Avoid major changes in the approved project plan until there is a mid-term evaluation led by an external consultant because changing the plan upsets people and creates delays. 3+1 1 4+1 —Ensure the project design includes team building exercises and an in-depth participatory 2 1 4 192 n=# of FGDs/KIIs that responded to these questions; *=based on lessons learned from issues they felt were not given adequate consideration in the current project. Category 1: CSIR-SARI staff and consultants; Categories 2, 3,and 6: GoG, academic and research partners; other USAID-funded projects; USAID/Ghana. Category 6: USAID. Category 5: Media partners. Category 4.F. Agro-dealer; 4.A-E. Other community-based groups. 92 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Lessons Learned and Illustrative Quotes Group 1 Groups 2, 3, 6 Group 5 Group 4.F Groups 4.A-E Total launch so project staff and key stakeholders understand the project design as well as USAID’s rules and regulations for execution in order to ensure timely execution of the project plan and avoid having a slow burn rate. —Anticipate the need for and execution of appropriate training for the board (which was planned but never executed under this project) to ensure appropriate oversight of donor-funded projects and CSIR-SARI’s growing commercialization activities. 2 2 4 —Make sure all project documents are filed both during the project and for the period of time that donors require post project. 3 3 —Continue to streamline CSIR-SARI’s bureaucratic processes 4 4 —Continue to strengthen CSIR-SARI’s existing systems for internal audits. 1 1 Infrastructure (SP 1). Future projects need to: 9 Strengthen CSIR-SARI’s systems for designing and overseeing new construction and renovation by: —Ironing out the processes needed to approve infrastructure and equipment expenses at the start of the project so infrastructure budgets can be approved in the first year and be fully executed before the project ends. 4 4 —Ensuring at least some technicians who will be directly involved in managing the new infrastructure are invited to and participate in the contractor debriefings (so they can provide timey feedback on the design and identify potential issues while there is still time to correct them). 1 1 —Developing better systems for ensuring outside donor investments in basic infrastructure development (including internet) and new or renovated labs (like the soils labs) for the field stations, which are critical to the successful scale￾up of CSIR-SARI’s research, dissemination, and commercialization programs, actually reach the field stations. 2 2 Strengthen the prospects that outside donor￾funded investments in new infrastructure— including internet systems—are maintained by: —Anticipating the need to train project staff and technicians in routine maintenance and repair as well as how to use the new technology or infrastructure. When new infrastructure and equipment is introduced the project-sponsored training needs to not only emphasize how to use the technology but how to repair and maintain it. 1 1 93 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Lessons Learned and Illustrative Quotes Group 1 Groups 2, 3, 6 Group 5 Group 4.F Groups 4.A-E Total —Facilitating CSIR-SARI pilot-testing and scaling up its access to solar energy (to reduce the recurrent cost of electricity). 1 1 —Considering ways special contracts (with institutions like GARNET) might provide more cost-effective solutions for the installation and maintenance of highly technical equipment and various internet-based functions (like the online database, internet connectivity and website) than using staff. 2 2 Technology Dissemination (SP 2): Future projects designed to strengthen the CSIR Institutes’ capacity to disseminate its new technologies need to: 26 —Strengthen: 1) the different research teams’ capacity to build and maintain the market linkages needed to commercialize their services and products; and 2) the financial systems needed to capture and re-invest revenue from product and service sales. 3 4 7 —Build the capacity of the different units to operate more efficiency and give them the necessary bureaucratic autonomy and authority they need to generate funding for CSIR-SARI. 2 1 3 —Anticipate the need to build CSIR-SARI’s ability to coordinate with its key GoG partners at the regional and district level by sharing reports, regular debriefings, and appointing one GoG coordination point person as was done at the end of this project . 193 5 5 —Anticipate certain research and outreach areas—like ISFM practices and organic fertilizer production—need support even though they do not generate the types of revenue that the sale of breeder seed to private-sector producers does. 3 3 —Improve on CSIR-SARI’s foundation seed support including short duration climate smart technologies. 3 3 —Consider options for strengthening private￾sector support for the development of private￾sector cold and processing like linking projects to the existing programs USAID supports that provide loan guarantees to banks for agricultural lending (i.e., programs that reduce the cost of credit for critical complementary investment). 4 1 5 ICT and Communication (SP 3) Future donor-funded projects need to: 22 193 This recommendation was also broached in all of the FGD/KIIs with MoFA and PPRSD staff. Since these response were not coded for the other EQs, they are not reflected in this particular data set although the issue is mentioned in other EQ data sets. 94 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Lessons Learned and Illustrative Quotes Group 1 Groups 2, 3, 6 Group 5 Group 4.F Groups 4.A-E Total —Anticipate the critical importance of building the capacity of MoFA, CSIR-SARI and private￾sector media outlets (newspaper, radio, social media, UDS journalism program) to conceptualize, execute, and monitor the efficacy of media and ICT outreach. 4 4 3 1 12 —Ensure future baseline assessments (for CSIR￾SARI and any institution that USAID/Ghana is supporting for capacity-building) include: 1) an objective external assessment by qualified technical specialists of connectivity issues in order to have a realistic plan and budget and technical support from other USAID-funded projects and qualified national firms or cooperatives for addressing them; 2) identifying contractors (like GARNET) that can help CSIR-SARI complement its internal technical capacity for operating the internet and updating the website’s internal knowledge management system; and 3) supporting various technical options that would facilitate field stations getting access to internet (which they do not currently have). 1 3 4 —Consider ways future projects can use M&E findings to: 1) lobby the national government and elected officials to better support agricultural research in general and CSIR-SARI in particular; and 2) better identify critical policy issues that affect farmers’ ability to buy seed. 1 2 3194 6 M&E: Future projects to support capacity building at CSIR-SARI need to: 18 —Ensure future designs anticipate: 1) adequate training and budgets for M&E; 2) monitoring M&E results against expenses; and 3) hiring and retaining qualified independent leadership to manage the ME system. 11 11 —Link the disbursement of funds in the second year to the establishment and approval of a robust M&E system. 6 6 —Consider offering grants to students from UDS to study of the impact of CSIR-SARI’s research and collaboration with MoFA and RELC. 1 1 Source: CSIR-SARI Technical and Financial Support Project, Final Performance Evaluation, FGDs and KIIs. Re￾analysis of frequencies breakdown. June 9, 2021. 194 All three community-based group leaders interviewed identified the limited access their groups had to services for land preparation as a major constraint and suggested future projects that support CSIR-SARI might work with other projects to facilitate private-sector individuals and youth getting access to loans for tractor service (three of the three interviews and fertilizer supply visits (two of the three interviews). 95 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex IV.A.1. CSIR-SARI Staff (Stakeholder Group 1) FGDs/KIIs Summary Results # Questions Rank 1-7 Explain Your Ranking EQ 1 1 SP 1. This project has increased SARI’s capacity to support agricultural research in Northern Ghana. (Rank 1-7) 1-4: 4 5: 20 6: 29 7: 6 %5-7: 93% (55/59) -We were supposed to have two years for renovation and infrastructure development, but the development did not go as fast as I wanted it to go. We only have a single store for stationery, fertilizer and chemicals. - There was a poor understanding of the protocols to follow in the project implementation, both administration and technical. They were overwhelmed by the amount of money involved. - Even though there was some training there needed to be more - They were able to renovate the out stations. The project supported early generation seed development for the development of new varieties, this had an impact on their performance. -I strongly agree because the use of the funds of this project: 1) supported to the senior researchers (who were short of funds and doing contract work from private companies to make ends meet 2) This project put everyone together and they were able to harmonize their operations - Because the project has invested in infrastructure and training - The project has supported the researchers in the north. The scientists got a lot of training and a lot of support to conduct the research. The infrastructure was also supported -This job requires an academic background but also a background in project management. The first person did not have that background. After it went to Dr. Dogbe (who had the necessary technical background and experience with other projects) we were able to navigate -We have been given the capacity but it was not wholly as expected. The understanding was that we would have a chain from the beginning to the end. We did not have that full pattern of capacity building - In terms of infrastructure, they could have done better. We wanted to improve seed production. That was not done. The internet is running now but some infrastructure like the seed processing and cold storage was not competed. The three most important infrastructure needs are listed below: 1. Irrigation was built by ATT but it is not functioning. We expected that ATT would have handed over a functional unit. 2. Cold storage (was rehabilitated but still have issues) 3. Seed conditioning plant was never funded 5 Output 1.1. The project has helped SARI’s scientists to train and retool in key program areas that have or are likely to strengthen the capacity of SARI to deliver on its mandate 1-4: 8 5: 19 6: 7 7: - %5-7: 76% (26/34) - The funding had a lot of conditions attached to it and made the difficult activities - Setting up a COE was not going to help them deliver on their mandate - The funds that were given helped accelerate the training. There was some training that went on. It was not what we wanted, as much as we expected -They had a program to train certain categories of scientists. The senior management were supposed to go to management training, for example, but (1) not many people were trained, (2) the reasons that accounted for some people getting trained and others not getting trained were not clear, and (3) The original training schedule was shelved when the COE realignment happened 11 Output 1.2. The project has increased the organizational quality and efficiency of SARI 1-4: 12 5: 10 6: 12 7: - %5-7: 65% (22/34) - There was a lot of infrastructure improvement on the project but we were not able to carry out all the infrastructure projects. Due to delays in funding. - Increased quality but it could have done better. The procurement and accounting staff could have been given more training. - Too much bickering - Communication within the institution has been improved. It is quicker to get approvals. SARI has learned a lot from the USAID project about better project management. 96 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking - It has approved our efficiency by 30 to 40 percent over what we did - There are several key activities that we are still not able to do. We are still not able to do certain analyses we need to do 22 Output 1.3. The project-built and project-improved infrastructure has benefitted agricultural research activities in Northern Ghana. 1-4: 11 5: 5 6: 14 7: 1 %5-7: 65% (20/31) - The project was supposed to cover 90 percent of the infrastructure of the project, due to funding delays only some of the structures were completed. - We had planned to build the seed lab and processing center; that never materialized. If we had done those it would have impacted a lot more - I would have expected more investment in equipment - We have a new nutrition lab. We have a new communications building. It helped refurbish our cold room. The outstation had problems that were not met (internet, their institutional needs, and the outputs). -We were expecting big labs. Some things were done but some equipment is still missing. We have a chemistry lab that is supposed to take care of some analyses, but it is not operating as it should. It is there but the full benefit from this infrastructure is not what it should be. USAID gave the infrastructure. A lot of money was invested the germ plasm bank. It was functioning. The project did what it was supposed to do but they could not do it all. Maintenance is also an overlooked issue 23 SP 2. The project has increased SARI’s capacity to support the development and dissemination of improved agricultural technologies. (Rank 1-7) 1-4: 8 5: 20 6: 19 7: 4 %5-7: 84% (43/51) - The rain pattern disappointed us and the links to funding decreased…given this problem we need to increase our investment in irrigation. - Through the USAID project we had promotional leaflets, and radio talks about them. That sort of communication worked very well and had a BIG impact on adoption. - I would say 80% of the old strategic plan that was developed under the USAID project made its way into the new plan for SARI - The people working in commercialization did not get any additional training - The dissemination of quality seed is still a major problem in N. Ghana - Experienced burn rate (i.e. expenditures vs. targets for expenditure) problems. Then the project got hijacked -In terms of variety development they did well but they did not put too much emphasis on other technologies other than technologies related to soils. They used radio and tv programs to promote dissemination, especially during varietal release - The project has developed technologies. We have seen the technologies going to the farmers - Need more training and lab development in the field stations. Need to not limit to the three crops - The funding was there but access to the funding was not easy. Bureaucracy limited access to funds for research and limited impact for maize and soybean. This resulted in funds being returned. - Vehicles were provided and resources were provided for technology dissemination. This was what was bad: The dissemination of results focused only on three crops. Not all the crops at SARI supports were covered. -There were several trainings. There were lots of trainings that involved the new varieties. I saw how MoFA improved too. Some of the farmers had no idea about the rice or the other varieties. It helped them improve their crop yields - The revenue they receive from government is insufficient to support them which means they are likely to remain highly dependent on donor funding. For the GoG to increase its support, they need to better understand the direct and indirect returns to research. 24 For the rice component. (Rank 1- 7) 1-4: 18 5: 10 6: 2 7: 8 - The performance of the component teams was better than for the project as a whole. Rice was able to release some other varieties with support from other projects. 97 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking %5-7: 53% (20/38) - To date, the dissemination of the high yielding new varieties is not very wide spread -(1) the project has contributed to releasing 5 new varieties (with other project); (2) it has put on a high pedestal the foundation seed component 25 For the maize component. (Rank 1- 7) 1-4: 1 5: 12 6: 27 7: 5 %5-7: 98% (44/45) - the new varieties of maize have been disseminated more widely than the new varieties of maize because the new varieties are striga resistant and drought tolerant - Upper West is in the Sudano Sahel. The varieties were very suitable short, drought resistant, and striga resistant varieties. These new varieties were developed under earlier projects but promoted under the current project. 26 For the soybean component. (Rank 1- 7) 1-4: 2 5: 7 5.5: 1 6: 21 7: 14 %5-7: 96% (43/45) - Soy is new in Ghana. The desire for soy pushed people to come to us - because the project gave them a new working space (a whole facility) so all the team had working space 49 Output 2.1. The activities the project supported significantly modernized SARI’s seed operations in line with the Plant and Fertilizer Act. 1-4: 16 5: 5 6: 12 7: 1 %5-7: 53% (18/34) - The seed production in northern Ghana was very low. They were able to spread the tentacles. - The seed operations are now totally in line with the Ghana fertilizer and seed law. We are finding all the rules and regulations - Problem 1: The weak links in the seed VC are leading to poor seed quality. The problem we have is that a high percentage of the guys involved in the seed inspection unit are producing seed on their own farms or supervising the farms of their friends. This means that the police are policing the police. Problem 2: Production increased, but critical parts of the VC related to processing and cold storage remain weak link 50 Output 2.2. The project has strengthened SARI’s DEVELOPMENT of improved soil fertility practices. 1-4: 18 5: 9 6: 7 7: - %5-7: 47% (16/34) - The project created a soil fertility lab. Our soil lab. at the institute needed a facelift . The face lift did not occur. - The current state of the lab is insufficient to support both research and dissemination - The project did well on developing soil fertility practices (mostly other projects) but the dissemination support inadequate 1. There were a lot of soil fertility experts under the agronomic component that got support from the project. 2. We also worked with crop rotation (ground nut and cowpea). 3. Because of this project and others that came before it (like SANRAM and AGRA), the farmers have really understood that their soils are degraded - We expected to see a massive improvement of techniques on the ground. In fact, however, we have not seen much change in local practices 51 Output 2.2.The project has strengthened SARI’s DISSEMINATION of improved soil fertility practices. 1-4: 9 5: 4 6: 6 7: 1 %5-7: 55% (11/20) -They did a lot of wonderful manuals. They have done a lot of research papers. This increases the chance that new projects and other donors will have materials to work with and that they will come to SARI to ask for help. The attrition rate is low so they are still there 52 Output 2.3. The project has built the ability of SARI’s staff to apply modern tools /techniques in research. 1-4: 18 5: 3 6: 4 7: 1 %5-7: 31% (8/26) - The maize, soy bean, got training. Some people benefitted from training. Technicians from other units got training but farm management did not. - Technicians need modern equipment and they need training. Zero technicians at Wa got training - We have not, however, built the staff’s capacity very much. There were things like the statistics training that were very useful;. If you go to the field, you are still using a ruler (i.e. old technology) 98 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 53 Output 2.4. The project has helped SARI to improve the Research Extension Linkage Committee (RELC) mechanism for agricultural technologies dissemination. 1-4: 14 5: 9 6: 10 7: 1 %5-7: 59% (20/34) - When there is good communication you see an impact on the fields. - There was another project that was tasked after the 2nd year. Although SARI was heavily involved in these activities, there has not been any structured monitoring of it - We decided not to do this because it could be accused of double counting and any time there is double counting you have a problem. It would have been ideal to collaborate with them. - This was in the original work plan but it got stopped after one year because the “Modernizing Agriculture in Ghana” project that was supported by Global Canada was supporting it. It is working well when it is funded. RELC is popular with donors because it works. 54 SP 3. The project has helped SARI develop a more market and client￾oriented research approach (Rank 1-7) 1-4: 21 5: 14 6: 13 7: 2 %5-7: 58% (29/50) -The market part was to follow the development of the new seed. That is where the commercial impact would come in. Because they had not fully developed the institutional strategy for that. - Before we were doing things with no strategic plans. The research and technical staff were not working as a team. With the USAID project we got a strategic plan to follow in planning activities. We learned how to work more strategically. - The technologies are not readily available. - Most of our research is client oriented. We operate on a Farming Systems approach, we do constraints analysis and share the results with the breeders and other researchers. This information feeds into the breeding programs. SO THE VARIETIES ARE CLIENT ORIENTED. Where we are lacking is we are not getting to the industry. - We don’t have the mandate yet to deal with them. Our mandate to produce and meet the commercial/market - There is no coordination between the units that support this (it is lacking). - Although good research is going on, I have not seen an aggressive shift from research output to Market orientation 55 Output 3.1. The project’s support has strengthened SARI’s systems for information and communications technology (ICT) and knowledge management 1-4: 6 5: 14 6: 10 6.5: 1 7: 5 %5-7: 83% (30/36) - We need to marry the ICT and the commercialization together. You need to communicate and market the product to an audience - The internet back bone is deficient - The project helped improve internet access but more assistance is needed to take it to the level we require for our work. -How come the outstations did not get connected? This was not apparent to the staff. As we were making the efforts to extend to the outstations, they changed the project on us. We would have needed new funding - The project has helped. We had very few computers. Internet access helped but not very good. We were given five desk top computers and three printers. The cables were laid down - The lack of connectivity of the field stations to the main SARI office near Tamale is a very big problem. Even the bungalows— accommodations do not have internet 56 Output 3.2. The project’s support has strengthened SARI’s commercialization systems—i.e., offering fee-based services and goods (like improved seed) to stakeholders and others. 1-4: 17 5: 13 6: 3 7: - %5-7: 48% (16/33) - I think it is going to be very hard to support these programs based on the current structure for managing fee-based services and seed sales - We are at the very initial stages. Aside from seed, very little was done on this in commercialization. Aside from that, the other “cost centers” were not very developed. -No one was willing to keep count of how much money they spent. Unfortunately, the internet did not come in until the project was going to work with outgrowers. Ideally the money from the outgrowers was supposed to be put into a revolving fund. As a project manager, I never saw the cost benefit analysis. This part was a real failure -One problem with the commercialization is that the breeders are producing seed and the institute is benefitting from these sales, but this seed is being developed by different projects (not just this project) and each project has its own criteria for what happens to the funds generated by the sales. More of the critical resources that we need to provide seed 99 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking are: (1) equipment that people can rent, (2) consulting services that commercial seed growers can hire, (3) appropriate facilities, and (4) better access to solar power to offset the steep cost of electricity for the Institute. 57 Output 3.3. The project’s support has strengthened SARI’s monitoring and evaluation (M&E) systems. 1-4: 11 5: 15 5.5: 1 6: 7 7: - %5-7: 68% (23/34) -Lack of belief and trust at the management level in the concept of M&E. I was not part of the initial M&E trainings. There were two types of M&E that were supposed to be going on: (1) one for SARI (all of its activities). (2) The second was for the project. During five years we had 3 M&E guys The finance piece was also weak. There was a query from the first manager… About the M&E. This stopped the money. The finance shut down cut down on the funding - The M&E system came in too late to capture the real impact of the project -There were resources. 2) Why they dd not do it. 3) ON the top of my head these are things that accounted for that. 3.a. They did not have core M&E people they were research scientists. Even if you built their capacity during the project. They need to hire an expert to do that. 3.b.It is not an issue of mentoring. You get mentoring when you have something to do. It should not be just core M&E people for the project. You had to have people at the “top” to understand the M&E culture. The capacity building should have included everyone from the top to the bottom. This structure was not executed. It was designed but not implement EQ 2 58 The project has helped SARI strengthen its organizational capacity in order to improve farmer welfare (e.g., income, crop productivity, resilience) in Northern Ghana. (Rank 1-7) 1-4: 2 5: 15 6: 30 7: 5 %5-7: 96% (50/52) -The ICT was infrastructure was good. I think the project also helped the scientists to develop the seed they were supposed to for the farmers. We had an M&E system to coordinate with all other SARI projects so we could better tell what all the projects were doing and measure their impact. - It helped us to build our capacity for strategic planning. It has also enabled us to improve the organizational capacity. They have accepted reporting. Now the Institute is producing improved soy and maize varieties. - the development and dissemination of SARI’s innovative technologies has not been widespread - The strong points are the field days and sharing the results of the trials. - SARI has strengthened its connections with farmers. They are using the systems ad varieties 59 There is evidence of strengthened organizational capacity to improve farmer welfare. 1-4: 2 5: 7 6: 10 7: 1 %5-7: 90% (18/20) - There was some money to build the capacity of staff to help improve the quality of their demonstration - there were things that negatively and positively affected organizational capacity to improve farmer welfare: A. Positive: 1. Some of the investments like office space affected everyone. 2.This project contributed a lot to the organizational capacity of the three teams (rice, maize, soybean). 4. Some investments affected everyone (the internet, the office facilities) B. Negative: 1. Some of the experiments we started were abandoned after the realignment. 2. The RELC support stopped after one year. The constraints were identified in the first year and then there was no follow-up. 3. Vehicles were purchased for the three crops that were focused on but not for the agronomy component. 5. All the research teams were funded by different donor-funded projects supporting them and each project had its rules and regulations about what happened to the money. It was not going into one account. This is not a good system but this was encouraged. It is still happening. Some of the breeder seed was produced at Wa was not even disclosed to the 100 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking commercialization unit so this money is not being captured. SARI is not organized to run a successful seed business; it is a research organization. 6. The capacity of the commercialization unit did not go up or down. 7. We sometimes have difficulty paying for the recurrent costs of key investments that the project made like the monthly cost of the internet. 7.A. Internet: The running cost of the internet we have to support (last year). Paul Hixon paid for just one month. Every month it is 16,740. Paul gave us 3000. We still have to come up with enough money to pay for 7 months. 7.B. Electricity: We have trouble paying for electricity as well and need more helping in developing solar energy. 8. The commercialization unit’s capacity did not go up or down. The head of that unit is quite dubious. He is involved in a side business of selling seed on his own. 9. One cannot attribute any of this impact to just one project. When the farmers talk about working with SARI they are talking about all projects not just this one but projects like the earlier AGRA projects. 10. The contract and conditions for the PPP with the Indian company were not well understood by the SARI staff which made them suspicious. - How did they benefit? It reduced the need for frequent visits and the results they got from us 60 There is evidence that farmer welfare has been increased. 1-4: 2 5: 4 6: 14 7: - %5-7: 90% (18/20) - That one is a little difficult to know. Now that there have been improvements in the seed quality and supply, this will trickle down to increased productivity and profits. - Because of the poor leadership what we accomplished was small compared to other projects - capacity has increased but this is not being tracked for all groups 61 Community-based associations benefitted from trainings and demonstration trials. 1-4: 3 5: 3 6: 3 7: - %5-7: 67% (6/9) - Mostly we worked through t MOFA. One of the things that the staff appreciated was the capacity built for certified seed. They also collated the information. 62 Seed producer associations benefitted from SARI’s strengthened organizational capacity. 1-4: 3 5: 1 6: 4 7: 1 %5-7: 67% (6/9) - There is clear evidence from production that these groups are having their capacity increased - I gave this a four because it is unclear to me how many new varieties were released and how many new ones were developed. In fact, these activities happened in concert with other projects. 63 Private seed companies benefitted from SARI’s strengthened organizational capacity. 4: 3 5: 1 6: 2 7: 2 %5-7: 63% (5/8) 64 Large-scale commercials farms benefitted from SARI’s strengthened organizational capacity. 1-4: - 5: 4 6: 4 7: - %5-7: 100 (8/8) 65 NASTAG benefitted from SARI’s strengthened 1-4: 3 5: 2 6: 2 7: 1 - We interfaced with NASTAG’s members not NASTAG. 101 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking organizational capacity. %5-7: 63% (5/8) EQ 3 89 What have been some of the unexpected outcomes (both good and bad) of this project for SARI? (List) Good unexpected outcomes: -The emphasis on report writing was very good. -Renovations and the two labs established -Offices are wired. Internet has improved. - Farm level understanding of the need for improved seed is much greater than before - Radio had big impact - Commercialization was more successful 1-Onfarm trials focused on building the capacity of the local communities to adopt high yielding varieties had the unintended consequence of making the researchers more client oriented. -I started my Master’s program in 2013. I came back when the USAID project started. The time I left for school…the infrastructure has improved. ONE UNINTENDED CONSEQUENCE IS TO HELP MOTIVATE STAFF -People got training to improve - The infrastructure investments were good—like the infrastructure - It also exposed us to the critical importance of managing projects rigorously and consequences of not doing so 1. Provision of internet 2. THE high levels of government seed request under “planting for food and jobs” program stimulated a dramatic increase in the demand for high quality seeds and this in turn stimulated their work on seeds and the release of new varieties. 3. The seed processing facility that was initiated without adequate consultation 4. A dramatic increase in the demand for and use of high quality seed that was not just the result of this project but of this project in concert with other donor projects and the GoG “Planting for Food and Jobs” project 5. The project ended up partnering with big farms, some even did joint initiatives to produce seed of SARI varieties 1. support of the USAID-funded project for SARI gave to the ADVANCE project increased their linkages of these projects to new technologies for other crops as well (cowpea and sorghum and root crops) 2. With this project, SARI was able to contribute to the proper the raw materials that the private see companies (i.e. NASTAG members) needed to become viable. This benefitted rice, soy and maize but also all the other crops 3. strong collaboration between the Soybean Innovation Lab and SARI that was envisioned in the original proposal, was provide the type of technical support that SARI need to overcome it internet problems 1. SIL (Soybean Innovation Lab) support for diagnosis and identification of a solution to the internet issue was an unexpected outcome that dramatically increased the connectivity although it did not solve the problem entirely. 2. They have developed and strengthened the seed systems that have encouraged the SARI varieties to distribute. SARI is one of the 18 countries that hosted the soybean trials and this has increased SARI’s soybean germplasm 1.The nutrition/food lab is a big thing that happened to SARI. (this was expected) 2.What was not expected was the seed storage facility for the rice. This I had to fight for. 150 metric to facility. This was started by the internal generated funds…but the project ended and I convinced project. 3.Internet building capacity for research. If you are building capacity 102 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 4. Accelerated scale up of new varieties of cowpeas that were developed by one of the innovation lab. 5. The soybean lab was not in the original lab. 1.The internet 2.The soybean facility was not part of the original plan but we were able to add it. 1.The Project developed a new variety of soy and the government is scaling it up through the Planting for Food and Jobs subsidy program. It has been a s big success. 2.The Project has strengthened SARI’s relationships with MoFA to the point that MoFA has taken over the RELC mechanism that was originally led by SARI. 1. Strengthened SARI‘s procurement systems at the Institute level in ways that are improving project program management globally. 2. SARI is the only Institute in SARI that has successfully executed a Government to Government project for USAID. Most of the other big capacity building projects have been routed through the CG system. This was a pilot and was not executed perfectly but it includes important lessons learned for all CSIR. 3. Short term training improved our skills on this project and some of these trainings (in M&E, project management, procurement) and we used these skills to attract and manage other projects. 1.The strong response to Planting for Food and Jobs. 2.The ToT training for the Mofa Ag. Agents 1. I did not expect the US ambassador to pay us a visit. 2. We rehabilitated some infrastructure that was not in the original proposal - The strategic plan was much larger and more participatory approach 1. The nutrition program (was not envisioned). This was exciting. 2. Another good unexpected outcome was the training on how to identify various life cycle of the pests and how to manage the “fall army worm.” The training and the public events were supported by AGRA and by USAID. It swept through Africa. This was initiated by AGRA then USAID and FAO came on board. What we did with the stakeholders were jointly supported by this project and another project. There is a report and fliers. 3.When we visited Iowa and we were taking about it that the internet was brought. Good: 1. The project provided the farmers money for field days for transport and lunch and snack. It helps motivate farmers to attend. Encouraged attendance. 2.Our data quality improved. 3 The project strengthened SARI’s relationship with the farmers and MoFA. Bad unexpected outcomes: -Lack of communication about dollar/cedi conversions made accounting complicated. -People did not cooperate, even the Director. -No project launch to inform public and encourage collaboration. -Lack of support from the top-director -No collaboration training - There was very little personal motivation to participate. Financially people did not benefit. Normally when we traveled under CSIR and GoG conditions we got a night allowance and spent it without having to account for expenses Other projects give institutional support for the bills etc. and they give allowances. This one did not. - What we were expected to renovate was not up to standards. We need more offices. 103 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking - We need fences for the fields. It is more of a problem than building because there is encroachment on the fields. (Note this was also mentioned in Manga) -A bad one is the abrupt end of the projects. The project was over and no approval had come for a no cost of extension. If it had been extended we could have wrapped things up better. -A the start of the project, we believed it was to support the entire institution. Then in 2016 they got this “core of excellence” and a lot of people were left out. We have no idea how that happened. The focus changed and lots of people were disappointed - Commercialization unit did not get the funds that were targeted for it. - The project did not have an independent project manager - We did not reach out to the other G2G projects such as at UCC for guidance on how we could have addressed some of the problems we were facing - quality issues in seed related to third party production - The organizational capacity of the board was not improved 1. The baseline assessment and design under estimated the impact that bad connectivity would have on every aspect of capacity building 2. They did not follow the original proposal which was approved in the program implementation letter 3. The original emphasis on gender in the project document did not include mechanisms for design, implementing and overseeing the execution of a gender strategy 4. Had a midterm evaluation been conducted, then many of the issues with the role out of the M&E system and gender and even internet connectivity would have identified and corrected at mid-term. 5. The scholarships that were supposed to be funded by this project were not funded 6. The original proposal did not anticipate the creation of a gender focal point for SARI or for the field offices -The recommendations that the SIL made for using Garnett to do the rewiring was turned down because the $100,000.00 estimate was considered too high and the person with the lower bid (a contractor from India) was unable to compete the job - The slow burn rate which forced you to do last minute realignments of budgets that were not approved and to return funds that were not expended. 1. One of the major parts of the COE was the creation of a seed business development unit that would be to champion the production of seed but this was never created 2. In place of the seed business development unit, the leadership went after the development of a PPE with an Indian partner and they said if we had to agree to it 3. The other bad outcome is that we had to give money back but we had big problems that we needed to spend the money on because of our weak procurement and accounting systems for grants 4. The big problem at the beginning is that we did NOT have an independent project manager who could act without being controlled by the Director [as specified in the proposal]. 5. The last issue was the issue of information flow which resulted in key project management decisions were not being shared 6. The steering committee (that was envisioned in the proposal) did not materialize 7. There was no remuneration or consistent way of rewarding of people who were in leadership positions for their additional work 1. The original project proposal was good but many key aspects of it (like the management structure, M&E system, and governance system) were not executed. 104 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 2.Some things never happened because certain activities were cut when the project had the COE realignment . The original proposal was supposed to make the whole institute better. 3. The lack of an independent manager was a major outcome. This was a big issue. If they had given a good project manager from the start 4.The exclusion of other crops had unintended consequences in that it discouraged some of the research scientists from working on these crops 1. In contrast to the project plan, the field stations did not invest very much in infrastructure (like office space) or even training. All of these activities were dropped because of the project implementation. 2. The first three years of the project, we did not have an independent manager (as envisioned in the project proposal). 3.Many of the people who were involved in the first three years of the project were dropped in the second half after the 2016 strategy reboot 1. The project did not have an independent manager and the M&E officer as specified in the project document until the last year 2. The monthly imprests was a big problem 3. The outstations did not benefit from the investments to the degree that was expected in the project document 4. There was a strong lack of transparency across the board from the beginning in terms of budget and strategic planning.. Initially the final decisions were made by the Director 5. A huge amount of the money went into administration after the COE realignment 6. The M&E training and support was concentrated on the economists 7. When it started, no one understood the project 1.The extension was not extended as expected 1.Strategic Plan Did not happen till third and fourth year of the project. 2.When I looked at the COE I thought it came out of the reboot 1. The project had to go through a lot of changes 2. We lost some of the budget. 3. We had a plan and we changed plans right from the beginning to the end. That affected the project negatively Bad: 1.SARI staffing of the technologist positions in the field stations was inadequate 2. The original proposal and budget envisioned investments I the West Gonja field station that never happened (office renovations , storage, drying slab) 3.The small budget for Technicians’ Trave to the villages limited the technician’s ability to support all the villages in the district 4. The field station/stations never got connected to the interet as was originally envisioned 90 What have been some of the unexpected outputs (both good and bad) of the project’s support to community-based groups. (List) Good unexpected outcomes: -This project accelerated the dissemination of varieties that were already developed. - Increased farmer access to high quality seed. - Associations and vendors have increased their understanding of how to grow seed commercially. - The dissemination of improved technologies for cowpea and ground nut accelerated. - The farmers are benefiting from the haulms (of soybeans) from threshing the seed for their animals. -98% INCREASE in seed sold to companies HAS BEEN TO CREATE NEW EMPLOYMENT FOR WOMEN LABORERS hired by sari to process the seed. - women are getting more income from seed—this has been dramatically scaled up because of the scale for women’s employment in the seed value chain. The seed center was supposed to be a “spillover” from this project 105 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 1.Accelerated scale up of cowpea in upper east that were developed in other projects. 2. Original proposal did not anticipate the profitability of the outgrower system to both sari and the farmers 1. Increased production and productivity of the three crops we have focused on under this project. 2. Unprecedented increase in the number of farmers that we mentored on growing seed that have become commercial seed producers due to the steep increase in demand for improved seed Bad unexpected outcomes: - Breach of contracts by sub-contractors - The outgrower system left a bad taste in peoples ‘mouths. 91 What have been some of the unexpected outputs (both good and bad) of the project’s support to private￾sector businesses. (List) Good unexpected outcomes: - Has strengthened linkages to SARI - Private seed producers have been helped to develop at a much faster rate than originally expected due to high demand created by the GoG planting for food and jobs program -10 years ago there were no private seed companies. Private seed production has taken off in the last five years due to this project working n concert with other projects (like ATT) and the Planting for Food ad Jobs Initiative Bad unexpected outcomes: - The out growers were not paid the price they were promised. EQ 4 95 The project addressed gender issues in SARI’s research and development projects. (Rank 1-7) 1-4: 35 5: 10 5.5: 1 6: 9 7: - %5-7: 36% (20/55) - I did some gender sensitization training. I emphasized that they should try to involve women in issues. They were supposed to support farmers and they had to target women. With the USAID project I did not have the sensitization plan. - We have made a conscious effort in planting and harvesting and in the village. They are paid for work on the station and in the village. We do on-farm demonstration. This has created jobs for women. - Before this project—before this set -up—they were not very careful. After this training (on the job) the result was a conscious effort by the project -The larger question is whether we have been responsive to gender issues in our breeding programs. Gender is bigger than “sex.” Gender is about reaching out to vulnerable groups that are often left out. I do not think we have had that focus -The environmental mitigation and monitoring plan that was approved by USAID included reporting on the environmental compliance. This was linked to project monitoring and reporting. This is the model that should have been used for gender and youth to ensure proper reporting - Gender was a focus the project but there was not big emphasis on it . Nonetheless there was an effort to do it because we had been influenced by other projects - The one who was managing gender left. No one told me that we needed a strategy -SARI was on the path of mainstreaming gender in its research program. In the last 6 year we employed more women than in the last 30 years. Even though there was not a gender unit, the project added to that 99 M&E: The project helped SARI track the participation of women in its training program (Rank 1-7) 1-4: 22 5: 16 5.5: 1 6: 8 7: 9 %5-7: 61% - Most of the programs already included women - Under the Feed the Future monitoring system we were required to disaggregate the data to show gender participation. - We had some gender disaggregation from the field - Very little done although we were making conscious efforts to include women in our training (I8) 106 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking (34/56) 100 M&E: The project helped SARI track the participation of women in its research programs (Rank 1-7) 1-4: 16 5: 17 5.5: 1 6: 10 7: 8 %5-7: 69% (36/52) - We tracked the participation. On the attendances sheets we captured the male or female and the age as well for youth - It has improved. A woman scientist played a key role in the soybean program and scaling up some of the nutritious soy-based products that SARI developed to the government. The head of the maize group is a woman. The head of the seed unit is a woman. 101 LEADERSHIP: The project’s support for research and infrastructure development is helping SARI to achieve a more equitable gender balance in agricultural research in Northern Ghana. (Rank 1-7) 1-4: 23 5: 18 5.5: 1 6: 10 7: 3 %5-7: 58% (32/55) - We had scholarships. There were some ladies who were supposed to do the masters -It helped - I personally did not see within the structure (or the project) any conscious effort to promote women into decision making positions. One reason is probably because a lot of the women were just getting started. I have not seen women moving into the top positions. I am sure in the next five years this will change. They are getting better established. - SARI as an institution was under pressure from CSIR to improve the gender balance in staffing and the project respected this but did not make it a core function of its program EQ 5 108 Identify three practices from this project that you would like to see included in future USAID-funded projects. (List) First develop a list then rank the top 3 1. Strategic planning /working as one unit 2. Effective communication of results, including publicity and marketing 3. Accountability 1.The emphasis on seed production (very important) 2.The emphasis on ICT (very important) 3. More involvement of women (gender integration plan) 4. training is good—it needs to cover all categories….it needs to be based on a plan. 5..more attention to renovating workshop 6.Fencing 7.infrastrucutre budgets need to get approved more quicky (process needs to be ironed out clearly from the start so infrastructure projects can be fully executed before the project ends) 8. transfers need to happen quicker to avoid delays in activities 9. Need more farm equipment. 1.They operated faster than the rest of the system. The SARI system was very slow and bureaucratic. Future projects need to streamline SARI’s traditional bureaucratic processes (like procurement) in order to execute projects more efficiently. 2. *Future projects should continue to support SARI’s ICT unit but they need to facilitate and encourage and indeed monitor that these programs are designed, executed and evaluated in close collaboration with the commercialization unit. 1. The project design was based on a formal capacity assessment of 9 different areas that was ground truthed by USAID 2. The extensive investment in infrastructure and facilities was critical (offices, equipment, and internet). 3. The demonstration and multi-site trials enabled SARI to have information on a wide group geographical location. This enabled them to reduce the number of years needed to complete the breeding cycles and also to give them the capacity to test and release varieties over a wide geographical area 1.Collaboration with the Soybean Innovation Lab was very good.. 107 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 2.The support for developing the internet and the collaboration with the soybean innovation lab in conducting and assessment and developing solutions to the internet ab 3.Infrastructure improvements (like the soybean lab) were made. 4.Management training 1.The leadership training 2.The infrastructure 3.Equipment 1.Management training worked very well. Needs to be scaled up. 2.The outgrower system needs to be taken to scale. 3.The component team approach for research was a good approach. - The original design was very good Infrastructure was supported but not sufficiently. A lot that was promised was not done. Equipment was supported but not sufficiently. 3.The short term training was supported but not sufficiently. There was no clarity or criteria for showing the training. The list said that everyone should be trained 1.Future programs should involve the entire institution and all the staff. The relevance of their activities. 2.There should be a gender office with a gender coordinator. There should also be a gender coordinator in each of the stations. There should be gender targets. Gender integration plan should have training. 3.Capacity development. You need to have target for it and system for tracking it in the current project. 4.When new technology is introduced, the training needs to emphasize how to use the technology but also how to repair and how to maintain it. 5.SARI needs better coordination between the commercialization, the production, and the accounting unit. 6.Make sure that there is a complementary of roles with the institution and these roles are protected and policed by the SARI Director, the Internal Management board, and the Management Board 1.Strong linkages that were established and nourished between SARI staff and stakeholders. 2.The capacity built among scientists. 3. Expert visits. As part of that we had a plan for scientists to visit other sites and bring them out. Only a few of these visits were executed. These should be encouraged 1. Continue to improve the communications systems. It got a lot of boost from the USAID project. 2. We had experts coming into SARI. Need to encourage experts coming in. 3. The commercialization efforts improved but they need additional work. In the future we need to build not a quality control 1.Farmer training support was very good. 2.The time of implementing the project activities was (they did it at the appropriate times of year) 3.The flow of support (money) from the donor to the activities was good. 110 Name the principle lessons learned from what worked and did not work in this project for USAID to consider when designing similar programs in the future. (List) First develop a list of what DID WORK then rank the top 3: 1 A clear reporting system 2. Clustered leadership and crop-specific working groups 3. Development and implementing activity work plan process 1. Build the capacity for strategic planning at the institutional level. 2. Anticipate the need for investment in renovation. 3. Communications center is very important….that communication center went well. - Involving women in commercial seed production increases the efficiency and efficacy of the activity - The critical importance of strategic planning 108 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking - When you have sub-contractors you need to involve technicians in debriefings - Discipline in monitoring expenditures against results - This project emphasized the training and capacity of staff in M&E. 1.Bureuacratic procedures can have a negative impact on the efficient organization of projects. 2. Given the critical role of the commercialization units in ensuring the long-term sustainability of SARI, future projects need to: 1) build their capacity to operate efficiently and 2) to have the necessary bureaucratic autonomy and authority they need to do what they are tasked with doing to generate the funding for SARI. 1. Infrastructure is needed (they need a chemistry lab). 2. Communication aspect good and needs to be scaled up. 3. Demonstration trials and support for farmers very good 1. There was a powerful synergy between Advance, ATT and the USAID￾funded SARI project 2. Future projects to support capacity building at SARI need to link he disbursement of funds for the second year toed on the establishment and approval of r robust M&E system that includes: 1) a full time M&E coordinator; (2) M&E focal people in the field programs; (c) solid IPTT that includes the indicators needed the relevant component projects is critical to providing quality data at any time for strategic planning and performance reporting; as well as (d) a plan and budget for the conduct of an external mid-term evaluation in order to facilitate a full review of the project and any unforeseen challenges that need to be addressed. 3. A robust gender strategy should be part of the program design 4. Future programs should consider predicating the initial disbursement of funds to SARI having the management and oversight structures (i.e. a steering committee, an independent project manager, and a model for reporting the SARI board and CSIR oversight council) in the proposal in place 1. The issue of connectivity has to be addressed. 2. The issue of management and mentoring for management must be built into the project from day-one and continually monitored. 3. Having a viable M&E system should a condition of funding 1.Consistent leadership is critical to manage the M&E system 2.Consistent independent project manager is critical. 3.A project should to change curse in the middle…t crates 4.Documentation, if we are going to change focus and everything you do as a project should be documented and given for the records. 5.For best practice important to have a good teering committee. 6.The field station need to have well defined budgets to ensure that they get the resources that they need. 1. Every single achievement under this project was the product of many mothers and fathers. Going forward SARI needs to keep better records about how different activities capitalize on earlier investments 2. The critical importance of having an independent project management team (manager, and M&E officer and an accountant) 3. The critical importance of providing a solid training up front is to ensure that all of the senior staff understand USAID rules and regulations for the FTF program and what is admissible and not admissible in the budgets 4.The critical importance of having a mid-term evaluation in order to adjust things not anticipated. Had they had a mid-term all these problems could have been sorted out 5. Future projects need to be very careful to not include items in the project documents (like positions for coordinators or salary top offs) that are not part of the mainstream ongoing systems of SARI 1.Good project pan 109 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 2.Participatory process used to develop a strategic plan for SARI 3. Commitment to having a stronger focus on commercialization 4. The original proposal anticipated the need to improve infrastructure on the main campus and the field stations 1. The critical importance of establishing a plan and sticking to it. (both rank 7) THIS IS THE MOST IMPORTANT. 2. How to engage stakeholders in research outputs. (both rank 7) 3. Fee for service approach (both rank 7) 4. Critical importance of having an independent project manager from outside SARI from the start First develop a list of what DID NOT WORK then rank them: 1. Working as a team did not go well. 2. Training and support for M&E 3. Bureaucratic process was a major constraint. 1. Use per diem not reimbursement for the actual expenditure. 2. Give the money at the start for the infrastructure so that it is completed by the end of the project 3. Clear start-up meeting about what you can and can’t do with the money 4. Strengthen the existing systems for internal audits 1. Do not change projects in the future…it upsets people. 2. Having a gender integration plan –or just gender targets 1.Manager of the project should be recruited from outside the institution. 2. Should have had a steering committee. This project had lots of resources it shoud have had a steering committee to guide its implementation. 3.Future projects ed to include solar energy as electricity bills are huge. 4.Very important need to train the SARI board and improve its role in projects. 5.Need to help SARI develop better sustainability plans. 6.They need to learn to write proposals to help governance. 7. Lobbying of government to support research. (work tours to countries like Tanzania). (consider training the parliamentarians about the value of research) 8. Consider hiring students from UDS to study the impact research SARI activities for activities like the RELC mechanism. 1. Soil lab was neglected; it needs to be expanded and renovated 2. The long-term training that was anticipated needs to be supported and scaled up 1.If you do not have a gender coordinator you have difficulty in maintaining the program. 2.Broad based participation is critical to major institutional change. At the start of the project, everyone understood that the project was to involve almost everyone. At some point there was a lot of “cutout” without upsetting people. When this happened it caused a lot of conflict. 3.SARI needs to develop a more streamlined procurement process. Long delays in procurement can discourage people and reduce the impact of capacity building 1. The project did not follow either the original project plan or the strategic plan in its first five years 2.The management structure in the proposal was never put in place. 3.The SARI board did not give much oversight to the project EQ 6 117 1.1. Rate the likelihood that “core scientists trained and retooled in key program areas” will 1-4: 6 5: 17 6: 17 6.5: 1 7: 11 - some critical trainings were not completed - We are still dependent on donor funding. The IGF insufficient. We are not mandated to be commercial sales institution. SARI makes about 40% on the sale of seed. 110 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking be sustained or continued after the program ends. (Rank 1-7) %5-7: 88% (46/52) - Any time they produced their financial reports: they simply don’t have the money to sustain this. Most of the training they have comes from projects - Despite the failings of the project, they did increase capacity and it will be sustained 118 1.2. Rate the likelihood that “increased organizational quality and efficiency of SARI” will be sustained or continued after the program ends. (Rank 1-7) 1-4: 11 5: 15 5.5: 1 6: 17 7: 9 %5-7: 79% (42/53) - Strongly disagree because of the financial instability (I6) - They increased their personal capacity but as an organization they did not build the capacity - It is in SARI’s own interest to maintain the organizational quality and efficiency it has gained in order to continue to attract funding and satisfy its clientele - I doubt that this will continue. Even during the period supported by the project it was not being accepted 119 1.3. Rate the likelihood that “infrastructure and facilities of the Institute built” will be sustained or continued after the program ends. (Rank 1-7) 1-4: 11 5: 18 6: 22 7: 1 %5-7: 79% (41/52) - Most were renovations. Only one was completely new. - We have the facilities there, but the fund to maintain them (and people) are not there. We need better equipment for communications. - There are emerging problems with the canteen, the walls already have wide cracks - Yes, but there is the need for continuous sensitization for individuals to develop sense of ownership as well as maintenance culture - There are real problems with maintenance 120 2.1. Rate the likelihood that “SARI’s seed operations significantly modernized” will be sustained or continued after the program ends. (Rank 1-7) 1-4: 5 4.5: 1 5: 14 6: 22 7: 10 %5-7: 90% (47/52) - If there is new equipment for assessing the quality of the seeds, we should look at this. - The USAID funds helped us get started. We cannot continue. The core competence has been developed but we are not there. - As long as government policy emphasizes planting for food and jobs (aside for fertilizer) that will be a leverage. - This is their core business so they are motivated to try to sustain it. They make money out of it - There is a great commitment on the part of leadership given the amount of money the sector generates for the institute - They were not modernized. There is no seed lab 121 2.2. Rate the likelihood that “integrated soil fertility management practices developed and disseminated” will be sustained or continued after the program ends. (Rank 1-7) 1-4: 5 5: 14 5.5: 1 6: 23 7: 10 %5-7: 91% (48/53) - The problem will be to get modern equipment. - Our lab facilities are limited and out of date. - This is something the farmer want. - The challenge has to do with funding of outreach programs which is currently lacking -This is an example of a case where integrated soil fertility management support under this project as well as other projects (in Ghana) convinced the PFJ program to change its subsidy on fertilizer. With integrated soil fertility they have also increased their interested in organic fertilizer 122 2.3. Rate the likelihood that “capacity of technical staff to apply modern tools/techniques in research built” will be sustained or continued after the 1-4: 9 5: 15 5.5: 1 6: 19 7: 9 %5-7: 83% (44/53) -Staff turnover (retirement and change of jobs) could threaten the sustainability as new skills would have to be built. Furthermore, now tools and techniques could emerge that makes current modern tools - Very few if any techniques learned 111 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking program ends. (Rank 1-7) 123 2.4. Rate the likelihood that “RELC mechanisms for agricultural technologies dissemination improved” will be sustained or continued after the program ends. (Rank 1-7) 1-4: 13 4.5: 1 5: 7 6: 17 7: 15 %5-7: 75% (40/53) - The RELC committee mechanism is a useful set up but it needs more attention from government. The farmers’ demands. This needs to move forward -This is dependent on donor funding. When the projects come in they try to smuggle in this support. During the gap years-- no activity. Even the vehicles that were bought to support this were funded by donors. No core support 124 3.1. Rate the likelihood that “ICT and knowledge management systems for enhanced market driven research for development developed” will be sustained or continued after the program ends. (Rank 1-7) 1-4: 6 4.5: 1 5: 21 6: 5 7: 12 %5-7: 87% (39/45) - Not as fully developed as we would like to do it - When the funding was going on it went on but now that the funds have stopped we are not doing those activities any more. - This meeting has access to internet because of USAID. ( - the internet infrastructure at is deficient which is greatly impedes the effective uploading and dissemination of the ICT tools that they are developing. - There is a gradual improvement in our ICT and knowledge management systems which I strongly believe will get better with time - this needs to be monitored; we need to be able to see if it is improving or not 125 3.2. Rate the likelihood that “commercialization program at the institute strengthened” will be sustained or continued after the program ends. (Rank 1-7) 4: 5 5: 17 6: 20 7: 4 %5-7: 89% (41/46) - They have the capacity and the staff but without the project, they do not have funds to continue the commercialization at the level they had it before. - We have old farm equipment. That is a big problem. This affects seed production. The lab (soil, micro-biology) facilities are inadequate. - Planting for Food and Jobs ensures the sustainability -Per our strategic plan, we are expected to be more commercially oriented which also feeds into our drive for internal generation of funds to support research activities. In view of this, I believe the commercialization of research will be very strong - Commercialization activities of the Institute appeared but not have been at the fore during the latter stages of the project ( - Before the project, seed production was handled by the commercialization wing. No one wanted to buy the seed before this project because the quality was bad. Under this project we improved the quality of the seed. The certification process is very efficient -If you have a system where I give my breeder seed to the commercialization unit and I go to them and ask them to give money. They have been marginalized because they are not trusted; the breeders do not trust them. The problem is that you cannot police this. If you produce maize seed. Some are keeping the seed in their houses. Someone comes to the breeder to buy the seed. The issue I found out that I am dealing with is that the breeders are selling the seed for less than the recommended price. This has a negative impact on the reputation of SARI. The quality could be bad seed because people are buying from technician. If the farmer plants seed and says it is bad this is not good. The commercialization people are producing on their own. Our model for commercialization is not sustainable 126 3.3. Rate the likelihood that “M&E 1-4: 8 5: 15 - Lots of training produced good people but they left which meant that the long-term impact of the training was minimal. 112 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking system developed and operationalized” will be sustained or continued after the program ends. (Rank 1-7) 6: 14 7: 6 %5-7: 81% (35/43) - SARI’s decision not to report on any of the “custom” indicators that were identified n the 2013 and 2014 M&E pan makes it difficult to see where is SARI as an institution is at any point tie on tits path toward becoming a center of excellence. - The extent of harmonization and operationalization of the M&E system will largely depend on leadership who feels strongly about having a SARI wide M&E system - Our M&E system is the strongest in CSIR. Now we have people in place 127 What additional organizational development and institutional strengthening support might be needed? (List) 1.Labs, 2.Guest House, 3.irrigable research fields 4.basic computers for scientist and Technicians, 5.proper office furnishing with requisite facilities Source: CSIR-SARI Technical and Financial Support Project, Final Performance Evaluation, FGDs and KIIs. 113 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex IV.A.2. Stakeholder Groups 2, 3, and 6 FGDs/KIIs Summary Results # Questions Rank 1-7 Explain Your Ranking EQ 1 1 Sub-Purpose (SP) 1. This project has increased SARI’s capacity to support agricultural research in Northern Ghana. (Rank 1-7) 1-4: 1 5: 6 6: 13 7: 3 %5-7: 96 (22/23) - I still think that since SARI has 3 substations situated in 3 different regions in the north they need to undertake significant activities in those other regions as well. I would give an overall ranking of 5 - SARI trained farmers to use inoculated soybean seeds to help fix nitrogen in their soils. The visibility of the project was low. We hardly see the project - Even the seed lab needed to be renovated. The training boosted their performance; they were able to a little more than before. - The poor quality of the internet installation; the lack of a fully functional website; and lack of a user friendly knowledge management system are critical bottle necks tat affect the institution at all levels. -The program somewhere along the line took a lot of time to do a redesign and bring in the private sector. This resulted in money that was targeted for infrastructure being used for other purposes. (1) We were not able to achieve the end result fully. (2) It appeared that we were not able to bring along all the other staff. We could not get the staff on board, they were saying that we should have kept to the original design by completing the other infrastructure that was originally designed. I thought they had agreed to it. The reboot was not bad, it was the approach, it just took up resources -The have the technical capacity—and that has increased dramatically over the last five years. What they do not have are the strong systems for commercialization that they need to sustain this capacity without outside donor support - It has increased SARI’s capacity. Before the project, SARI had nothing to do with us. I assume they had issues of finance. This support helped them to respond - Only a selected umber of AEAS benefit from the training. Ideally all AEAs should benefit from the training. We are so many; in my district we have 20/21 agents. -It has gone up. The last five years they have increased the number of drought tolerant varieties. They have done training and sensitization of the farmers. They have also done demonstration trials. They have also trained the dealers on the safe use of agro-chemicals - When a project comes to a district, funds have to be advanced for implementation. If the funds are late it creates challenges. There were delays sometime which affected the timing on planting -SARI support helps them to reach out to more farmers on a monthly basis. The farmers response the demonstrations and interaction among farmers has exposed them to new varieties of soy bean that are doing extremely well - The big challenge that happened with this project was that it tried to look at everything. That has probably negatively affected their success. Thee were people who had no idea at all how to run a project; they did not even know what an indicator was 2 -In regions where the project worked in Northern Ghana 1-4: 1 5: 5 6: 9 7: 8 %5-7: 96 (22/23) - There are still bottle necks to scale up. Some of the technologies moved spontaneously to other areas - The impact was great in the 3 communities where we did demonstration trials ( - Interacting with other colleagues in other regions, SARI is making more impact on agriculture as compared to the past 3 -In regions where the project did not work in Northern Ghana 1-4: 6 5: 14 6: 2 7: - %5-7: 73 (16/22) - Once the breeder seed goes into circulation it goes to foundation and certified seed it spreads across communities. - There is a difference. The communities near the research station benefit a lot - You can see a big difference between the areas where they work and where they don’t 114 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking - SARI is more visible in northern Ghana. The farmers are always eager to get access to better quality seed. They look for SARI or MoFA seed -There are some areas where they cannot get to. That is why the ToT training is so smart. If they are not present in an area do not assume they are not receiving training. An ag officer can give that training 23 SP 2. The project has increased SARI’s capacity to support the development and dissemination of improved agricultural technologies. (Rank 1-7) 1-4: 2 5: 4 6: 8 7: 12 %5-7: 92 (24/26) - They have done a lot in foundation seed technology. In the area of seed they have done well (on supply of foundation seed). They need to work on fertilizer, irrigation. - There has been a significant increase in quantity of breeder and foundation and certified seed. -they have developed a lot of new varieties of groundnuts and soybeans as well as the cereals. What are the major crops in upper west? The number one crop is maize followed by groundnuts and soybeans and cowpea and sorghum - I have seen the impact of their demonstrations and training on the MoFA agents. Prior to that we often did not have this information. ATT helped disseminate the information on the new varieties, it is important to emphasize that this impact is because of other donor funded projects as well as this one and the PFJ initiative -SARI provided the training on the chemicals to control the fall army worms, and supported the AEAs through training on the use of the chemicals and its effect on the fall army worm from production until harvesting giving them the capacity to transfer the knowledge to the farmers. The demonstration enables them to call farmers to share their knowledge among themselves and the effect is eminent in their communities. MOFA AEAs can reach out to more farmers in a week as a result of SARI’s support 27 -Technologies for seed 1-4: - 5: 4 6: 8 7: 13 %5-7: 100 (25/25) - SARI came up with a new technology of inoculants foy soybean and other legumes to help improve nitrogen content in soils. - Even though we worked in 3 villages, we had field days. A larger number of villages would participate in the field days - This project helped set up labs which built the confidence of the SARI staff. They also trained the SARI staff on the new equipment. Those interventions improved the seed sector 28 -Complementary technologies for soil management and crop production 1-4: 3 5: 10 6: 6 7: 6 %5-7: 88 (22/25) - The scientist need to help the private seed companies better understand the composition of the soils. In terms of chemical amendments to the soil, the scientists need to be involved in - SARI does not have capacity for any of this. Integrated soil fertility management. That is an extension issues. - They came and talked to the farmers about soil. They emphasized the importance of good agronomic practices and testing the soil. Those farmers who participated in the trials saw the benefit and the SARI staff were easily accessible to the farmers to discuss them 54 SP 3. The project has helped SARI develop a more market and client￾oriented research approach (Rank 1- 7) 1-4: 2 5: 6 5.5: 1 6: 10 7: 7 %5-7: 92 (24/26) - I don’t think they have done much on helping farmers in marketing their produce, but the performance is increasing 1.They helped the farmers to produce compost fertilizer. 2.Seed growers are adequately trained. SARI has trained farmers in the proper way of producing seeds, organizing their seed business 3. SARI has strengthened the RELC. We have the annual RELC meeting at the district, regional and national level and SARI takes this information back. 4. SARI has built the capacity of the agro-dealers to do their business by building their knowledge. 5. They have trained AGRIC service providers—those with harvesters, threshers, 6.The also trained some of the processors on quality rice packaging. I am aware that this training has been cascaded through ToT to other farmers (50 people I am aware of in one specific incident). - The technology responds to the needs of the farmers - SARI has come up to the “consumer preference.” SARI has come up with soybean varieties with high oil content. These are laudable achievements. SARI is prioritizing the standard consumer’s needs We give them a high ranking on 115 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking the technical breaking programs but more work needs to be one on the policy impacts on market development. - They did not sell themselves sufficiently. Not just one seed. But you need to understand the whole v.c - Needs to improve the internet. Needs to improve the knowledge -Where the project was defective was in the dissemination of information: a) they relied on farmer field trials and SARI based trainings; b) their website was completely dysfunctional for most of the duration of the project -A vast difference between the last five years and before. Prior to 2015, there was not a VC orientation at SARI or in MoFA. Before then we just focused on growing the crops. We did not have the perspective of producing for the market. SARI has helped build this VC approach and building the capacity of the MoFA staff to produce for the market. We now have nucleus farmers as well to help with marketing -SARI tells the farmers to grow the new variety and to prepare, cook and consume in their traditional ways and then give them feedback. The relationship between the farmers and the agro-dealers and SARI is stronger -I recently went to farmers to ask them what their needs were. They were facing certain challenges. I quickly called one of the researchers. They produce tiger nuts. They brought in the varieties and I sent the varieties in to SARI. They welcomed this - One of the big problems with SARI today is that the staff are producing the seed on their own. Everyone is working on their own. People will give them a receipt but the money is not going to SARI. Every researcher has a side job. Rules and regulations are not adhered to. If the laws are strictly enforced, it will work EQ 2 58 The project has helped SARI strengthen its organizational capacity in order to improve farmer welfare (e.g. income, crop productivity, resilience) in Northern Ghana. (Rank 1-7) 1-4: 1 5: 6 6: 5 7: 13 %5-7: 96 (24/25) - This is difficult. This depends on how we define “welfare.” Is welfare because they have good access to seed? - there have been stakeholder meetings. A lot of farmer groups have been trained by SARI. SARI also established VC linkages. The VC chain committee is performing well. Even some of the MoFA staff has been trained. These people have had their capacity built to advocate for improved access to farm During the project there has been a lot of collaboration with other projects even animal service (this has further strengthened the linkage). - The big difference is that research is demand driven. Farmers are involved in every stage of the research from development to release - The reason for this ranking is because when you develop better seeds you increase yields and income. They have released a lot of high yielding varieties. They have trained farmer in good agronomic practices and increased their productivity. The radio and the television programs add to this (these should definitely continue since they do not have the capacity to cover the whole area directly) - the varieties have helped farmers better deal with climate change by promoting more drought tolerant varieties. This has increased their resilience - SARI’s capacity to improve farmer welfare has gone way up. Some of the new technologies have been important -MAJOR PROBLEMS (SUMMARIZED) : 1.The scientists themselves are selling the seed. So the money is not coming back to the commercialization unit. 2.Some scientists are producing their own seed alongside the small scale producers. Sari is developing the breeder seed and some are producing their own breeder seed. This is why we wanted to the coe. We were building a value chain of the nastag so that a point in time would come so that if people could get basic, foundation and certified seed. ATT also helped them produce some seed. We were supporting the entire value chain. So the breeder seed never got to the seed producers. So we had a serious problem of building the capacity of the seed producers 116 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking - SARI was able to get into the communities more than before. They are closer to the farmer than before. We have their numbers so we can get information on the crops. Everything they publish. We know more about their research. We also like the booklets they provided us (that focus on seed production) - the demonstrations are not being scaled up to a lot of communities. The impact is only felt in those communities and the adjacent communities 66 The project has increased farmers’ access to improved seed. (Rank 1-7) 1-4: 2 5: 1 6: 7 6.5: 1 7: 14 %5-7: 96 (24/25) - They have increased access to foundation seed. - SARI gives high quality foundation seed - There has been an appreciable increase in access to improved seed due the Planting for Food Jobs initiative. -For the five years, 3-4 of my farmers have now become seed growers that produce seed bring it back to SARI to check on it. This has helped us. This is for soy and cowpea. These farmers face certain problems. Their big problem is getting the foundation seed to multiply Problem 1: It is difficult for them to get the foundation SEED We in the upper west get foundation seed from the same office. You have to get foundation seed from either Kumasi or Nyankpala Problem 2: Pricing is an issue. We would like SARI to regulate the pricing. The price for certified seed in one community is different from others. Farmers are charging different prices - It is not the project that did this; it was the planting for food and jobs that achieved it 67 This greater access to seed has increased farmers’ yields. (Rank 1-7) 1-4: 1 5: 9 6: 7 7: 8 %5-7: 96 (24/25) - Usually increased access to seed translates into higher yields. What is important to me has it improved their income. We can still have yields but they need access to market. The missing variable is the other things: 1) good land preparation; 2) fertilizer; 3) pesticides; 4) post-harvest how our farmers are able to preserve and sell their produce. - SARI has increased farmers’ awareness. SARI improved this but they must “share this glory” with other institutions. - Now, no farmer will not invest with improved Seed. Now they are looking for hybrids. If they see no hybrids they go for open pollinated varieties. Improved seed is now the order. Fertilizer use has also increased because they see the value in yield increase. - There has been a modest increase in yields - the upscaling is not occurring in all communities. Not many farmers are aware of the increased access to improved seed. For this to occur they need to scale up demonstrations and link to radio - Optimum increase in yields as compared to what they used to get from their yields. Generally, there has been a 50-60% increase in yields 68 This greater access to seed has increased farmers’ incomes. (Rank 1- 7) 1-4: 2 5: 6 6: 10 7: 7 %5-7: 92 (23/25) - Their incomes are going up. Where there is increased yields, livelihoods are improving and income is going up -“There has been an appreciable increase in income due to the increased in area under production. The area planted to improved varieties has increased. If you are attributing only to seed. It was not just seed it was the other complementary technologies that worked with the seed. This would be a six. It was not just yield per unit labor but increased area. That was the big contribution to planting for food and jobs. It increased the area planted with improved varieties very quickly. It was almost immediate. If it had been left on its own, the increased area would have increased but not so fast. We would not have the problem with the need for seed cleaning machines.” - They make more money as they sell it. If they consume it that is different - The lack of standard measurements ---and wide variation between zones—is negatively affecting income because it affects farmers sales. This enables the aggregators to take advantage - Groundnuts, soy and maize may face the glut during marketing and buyers may buy it at a throw away rate due to poor storage facilities - We are hoping that even after the subsidy goes down they will want to continue using certified seed. This is going to be a challenge due to pricing 117 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking EQ 3 89 What have been some of the unexpected outcomes (both good and bad) of this project for SARI? Good unexpected outcomes: - increased quality seed and the collaboration with the inspections. There has been a good interaction between the seed inspection and certification staff to ensure that farmers get quality seed. - another positive impact is that this has created new opportunities for women in post-harvest processing of the seed before sale by the private sector and SARI. 1. They have many success stories that we know about Golinga that were even more successful than expected 2. SARI has been able to respond to the huge increase in demand for early generation seed and getting seed producers to produce certified seed 3. The AGRIC officers have had their capacity built to the point that they are now much better partners with SARI. 4. The new soy bean inoculant technology combined with the Agric Extension agent (AA) training in group dynamic has increased women’s participation in soy bean production in ways that are benefitting them 5. SARI did a gender training for the AGRIC agents which has increased their ability to work with women farmers - The extremely close collaboration with ATT. They listed ATT as a partner and later they became de-facto technical advisor 1. Some of the seed producers are starting to invest in irrigation so that they can grow over multiple seasons 2. Because of the irrigation, and shorter cycle seed, the area cultivated in the river basins is increasing 3. Youth are going into dry season production of pepper, onions and cabbage 1.Planting for Food and Jobs increased the demand by local farmers for certified, which helped our businesses grow. 2.This rapid increase in commercial seed production since 2017 has created new employment opportunities for women - Successful record in responded to the extremely high spike in demand created by the PFJ program 1. It has shown us the extent of challenge that farmers face in increasing their yields 2. This project pointed out that when there is capacity building you need to look at it across the board 1.This project in combination with the PFJ has catalyzed dramatic expansion of private seed producer companies in northern Ghana (both large and small) through training and technical support. 2. This project has had a major impact on seed quality and standards in northern Ghana by improving the capacity of private sector seed production and helping the private producers meet the SARI protocols and standards 3. combination of the quality of the improved seed is encouraging farmers to increase the land area under production 1.In the last five years the number of farmers using SARI foundation seed to grow certified has increased a lot. This is a very positive development. Because of planting for Food and Jobs, there is a subsidy on seed. 1 kg. of subsidized soy seed was 17 Ghana cedis; the subsidized price is 7 cedis. The price for maize in the market maize gran is 4-5 cedis. 2. Farmer income has gone up a lot. Previously, it was not like that. They used to not have good seed. 3. Market demand for the crops has gone up because the appearance of the crops /produce has gone up; packaging has improved as well. 1.The rapid increase in the demand for improved seed due to the Planting for Food and Jobs. This was a huge opportunity. 2. When we designed this project, we did not anticipate the soybean innovation lab. This provide a huge layer of support. 118 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 3. Another unexpected thing was the shift to the COE . That was not expected the “core of excellence.” The positive was that this was support and correct a situation. 4. There was widespread support across all ag actors for SARI’s leadership in identifying a PPE (public private enterprise) seed processing initiative that could have secured the future of the institute. Had this happened it would have made SARI strengthen its internal systems. This would have helped supply the rapid increase in demand that PFJ created. It was not going to kill the seed producers. They could have contributed to the PPE -The SARI staff have been very open to us.. I am very happy with my relationship with Dr. Saaka. Anything I find wanting I call him. I can call the people in Wa and they will pick up the phone and give me answers. I can even walk into their offices 1.SARI started the development of a 4 hour radio program, AGRIC today that continues and is very positive -Farmers are looking for seeds and tractor for farming and SARI developed and made these seed varieties available to farmers -SARI’s support to field development has enhanced their movement and adoption of new technologies in the demo sites -The opportunity for farmers to come together and learn from different theological background from each other is very impressive Bad unexpected outcomes: - this success has overloaded the current base of processing and cold storage. 1. We all expected the seed lab to be refurbished; it was not. 2. Some of the staff should have been trained more on seed technology. 3. The board (of which I am a member) had very little understanding of the project until the last minute when the project had big problems (note this fits with the data from the board minutes). 4. (The original project included funds to build the capacity of the board) This was never mentioned and we you never got training 5. There was confusion between the Centre for Excellence and the Core of excellence 1. Seed Yields are highly dependent on rain which affects profitability 2. The sharp increase in the demand that was created by the Planting for Food and Jobs program quicky outstripped the seed producer’s access to seed processing and storage 3. Recent shifts in the contracting mechanisms and subsidies associated with the Planting for Food and Jobs Program have affected seed sales and the speed with which seed producers get paid by the government agents that purchase seed for this program 1. Limited seed cleaning and storage capacity in northern Ghana was exposed 2. High cost of capital reduced the global impact of the project on seed quality by reducing farmer’s ability to follow sari’s recommendations that foundation seed be grown on newly cleared fields and by reducing the quality of cleaning and sorting 3. high price of capital negatively affected new field clearance which is critical to producing high quality foundation seed 4. high price of capital negatively affected private sector ability to invest in storage and processing that is needed to sustain the increased demand 5. high price of capital contributed to adulteration of lower quality foundation and certified seed 1. There is clear evidence of impact on the direct beneficiaries that participated in the activities but it is not region wide 2. The procurement problems 3. We underestimated the critical importance of having solid informed communication about the project with different stakeholders 4. The project wasted resources by trying to change course on the project in the last 18 months 119 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 5. The project was not very effective in disseminating its results because they relied primarily on demonstrations with very little use of electronic communication including their website 6. The SARI commercialization unit which was the focus of output 3.2 was underutilized and marginalized 1.The rapid increase in demand and supply has exposed the weakness in SARI’s internal management systems especially those related to commercialization. 2.The same rapid increase in commercial seed production has outstripped the capacity of the private sector to have access to seed processing equipment and cold storage. 3.The MoFA extension officers who are the frontline people dealing with the farmers need more intensive education on seed quality and the complementary production practices needed to intensify production to sustain the yield increases associated with additional land area and improved seed. 1. The rapid increase in commercial production of seeds in the past five years has created shortage of cold storage units to store the seed not sold. Facilities are owned by PPRSD. 2. The rapid increase in commercial production of seeds has stressed the capacity of the seed processing units. The facilities are owned by PPRDs and a few private businesses. -SARI handles the facilities for the breeder seed and the foundation seed. -PPRSD also has facilities as well. -These things (processing and cold storage) ensure quality and keep the price down - The community- based seed producers did not anticipate that they would have to store their seeds. They harvest and process the seed and their storage capacity was not enough - The original project plan did not anticipate the need for cooking and tasting demonstrations for the new varieties - Some of the farmers approached me about how to get the foundation seed they need to go into certified seed production. Currently, it is not obvious to the farmer how to approach SARI 1.They only worked with us for three years. 2.We expected them to scale up the demonstration trials and the field days. That did not happen 1.Unexpected problems. Exposed the insufficiencies in processing technology and cold storage . When you have a bumper crop, there is a long wait for cleaning and they may have trouble getting storage. 1.For me they did not take advantage of it though they did mention it. The first year they did some good work because ATT was helping them. In the first year they made a great mark in 2015 and 2016. After that it did not go very well. 2. The negative was that the COE ==which was built in the system as a show piece—it did not succeed. It was expected to be a show piece. This was a negative unexpected. Had hoped that this would turn SARI into a show piece. 3. The PPE did not happen 1.Some of the seeds of the new varieties are in short supply. This IS not their problem. 2.Since a high percentage of the agro-dealers are illiterate, they have not had the means to benefit as much from the project. 3.The fact that the Wa division of the national agro-dealer association are not fully functional (or meeting regularly) has limited SARI’s ability to talk to them (and train them) as a group either in English or local languages 1.Rapid unregulated production of certified seed price variance between groups. (new market constraint) 2.Lack of standard measurement has reduced the income impact of improved technologies (new market constraint) 120 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 3.Rapid increase in production (due to improved inputs) is exposing the weaknesses/insufficiencies in crop storage - Groundnut and soybean treatments is a challenge because the inoculum must be kept in cold chain so it is not readily available. - This rapid growth in demand for seed has exposed the insufficiencies in the cold storage and seed processing plants - The other major thing is that there has been huge interest and a lot of companies getting involved in commercial vegetable production -a huge problem— if not the biggest problem— is that the high cost of credit is making it hard for the private sector to invest in cold storage and seed processing plants as well as the irrigation facilities EQ 4 95 The project addressed gender issues in SARI’s research and development projects. (Rank 1- 7) 1-4: 6 5: 4 6: 8 7: 7 %5-7: 76% (19/25) - I saw nothing that specifically targeted gender issues. Normally that statement was “cross cutting” but if you ask them what they did they will find it challenging - Gender is very critical and the board should be more informed about it. - They encourage women to be part of the process. Normally they encourage 50% of participants to be women. More women are growing soybean and ground nuts than men. It is more than before - I remember that they stressed that in training in outgrowers. The women on my farms are happy -in the northern part of Ghana, the issue of Gender has been a major issue. Culturally it is also a problem. I can assure you it has focused on being inclusive. There have been conscious efforts to see that women participate in their programs. When you follow the value chain, you can see that women are involved all along the value chain. When they run workshops they look at the gender balance in the trainings. They made a big effort - When we had contacts with SARI we were encouraged to include women -If my memory serves me right, most of the participants were all women groups. Maize is important to the women. Planting is done by women; fertilization and harvesting. In our part of the world, the main economic activities were led women. That is why we encouraged the women to participate - Gender equity was big. More women than men participation. I want SARI to encourage women in agriculture - With the selection of farmers for field work, SARI advised MoFA to consider gender in their selection 96 -Staff and leadership 1-4: 11 5: 2 6: 6 7: 6 %5-7: 56 (14/25) - There has been a modest increase. They did not have women breeders. Since this project started they have one woman leading the Maize component of SARI’s breeding program. - CSIR Crops Research Institute has been consciously trying to hire women. It was led by a woman. A very talented, focused and organized person who mentored women 97 -Participation in outreach programs 1-4: 7 5: 1 6: 8 7: 9 %5-7: 72 (18/25) - In my farm, we started with 100 farm workers with 70% men, the others were women. The 30 women ended up as the most reliable work force. There was no intent to deal with women at the beginning. EQ 5 109 Identify three best practices from your collaboration with this project that should be incorporated into First develop a list then rank the top 3: -Strong collaboration between the seed certification division and the SARI -Working with private sector seed producers and their collaboration with the outgrowers. -Demonstration trials were a best practice. Farmer fields. -Continue to involve women but scale it up. 1.The Market linkages (input VC) support. 121 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking future projects. (List) 2.Empowermet of women in agriculture support want scaled up. 3.Agric processing was supported and should be scaled up. 4. AA training (agent training) needs to be continued and scaled up. 1.The emphasis on infrastructure was good. They established a lot of very good infrastructure. 2.The area of communication became very positive. Online publication of their newsletters. 3.They did some good support for working with the private sector to commercialize some of the new varieties. 4.ATT worked a lot with them (not vice versa). In fact the correct way to say this is “They responded to ATT’s leadership”; “ATT tried to walk them through the USAID hoops” 1.Building the capacity of the staff is an excellent idea. 2.The development of improved seed is very vital to the economy of the country and the wellbeing of the farmers. This needs to be scaled up. 3. Future projects need to better support the management structure that is needed to fully support these activities. 1. Consistent bimonthly technical review meetings with SARI 2. The SARI ICT models are good and need to be scaled up 1.Support for breeding needs to continue for high yielding varieties that meet international standards that are also in demand in the local markets. 2. Capacity building of medium and small- scale seed production companies. (want to scale up and include more women’s association and women owned companies). 3.Increase their production of inoculants in sufficient amounts to satisfy the existing demand for inoculated seed for soybean and groundnuts (and even cowpea) in the entire catchment areas 4. Encourage ICT programs to continue and scale up. These benefit everyone. 1.Continue support for breeder and foundation seed. 2.Continue to strengthen RELC 3. Continue to invest in infrastructure (new construction and rehabilitation of old) 4. Continue to improve the internet and the website - Wholistic approach to capacity building 1.The recognition of the private sector as a partner on their own terms is very important. 2.Stepping out of their research offices to engage with the private sector in co￾development of the new varieties on their fields. 3.The extensive training of the seed dealers and producers has been very important. This was a combination of one on one training and radio. All of this worked well together. 1.Strong collaboration with the farmers and AGRIC agents that contributed to SARI ‘s capacity and vice versa. 2.The conscious effort that was made to include women and youth in their programs. 3.There was proper planning and review at the end of each year: (1) the teams from the upper east farming systems group (based in Manga); and (2) the RELCs 1.High number of women who participated in the trials and the support they received from women leaders and staff from SARI on these trials. 2. Demonstration trials and field days are good and need to be scaled up in future USAID funded projects. 3. Future projects need to include on female crops. (soybeans; okra; sesame) 4. The MoFA agent trainings at SARI were useful and should be scaled up 1.Demonstration trials are a learning center for the farmers (1st weeding, harvesting, etc.). These demonstration trials are a very good thing. 2. Field days are very important. 3.Training of MoFA AEAS at the district level is very important. We want this scaled up 122 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 1.increased research on varietal release 2.the researchers did good presentations both internally and externally 3.There were some serious dedicated staff who wanted them to succeed 1.Community-based trials. This is very good. It encourages communication between the farmers ad SARI. 2.They are very accessible. When we call them they pick up the phone. 3. They are always developing new varieties and always doing adaptive trials on the varieties to see how the high yielding varieties can be improved. 4.I like the brochures on how to produce different types of seed. There are lots of booklets. Are you aware of any other materials. -These pamphlets are for seed producers. - For the farmers they organized trainings. 1.Radio broadcast (AGRIC today) 2.Demonstration trials 3.AEA training 4. Strong communication between AEAs and the SARI 1.Still continue with the use of the inoculant which improves soil fertility 2.Continue the development and release of new varieties due to climate change for early maturing seeds 3.Varieties developed should be high yielding -Good collaboration between the research and the private sector. If there is something that they could do it would help. Initially they did not work with the private sector. This is helping them right their wrongs ISSUES THAT WERE NOT ADDRESSED 1.Government stakeholders need more awareness of the project and the level of stakeholder engagement on TV, radio and through meetings 2.There was very little collaboration between MoFA and the project. 1. They changed from the original plan which created challenges 2. Never executed the management structures they were supposed. 3. As a result of 1 and 2 they had slow burn rate and had to give back money. 1. The main recommendation is that we want research on animals and trees crops like guava, baobab, and apples –across all areas of crops. We want to see stronger soils research. 2. One key recommendation is to strengthen research on soil. 3. We would like to see more research on vegetables. We have huge potential. We think there is a technical gap. 1.Scale up a group of activities to help build the capacity of women to develop seed production 2. Encourage all projects to have private sector participation in their steering committees and meet three times a year. 3.. More support for economic research to identify critical policy issues that affect farmer’s ability to buy seed. 4. Consider using USAID loan guarantees to encourage banks to lend money to commercial seed providers to develop cold storage and seed processing plants. 5.Need to build the capacity of the regulators to understand the technical issues related to the new varieties. If not done, this can be a problem. 1.Strengthen the researchers’ involvement in the field. It increased but not enough. 2. More flexibility between budgets. 3. Independent project manager. 4.More involvement and training of the management board 5. Loan guarantees or Lease/Option to buy to encourage the private sector invest in improved processing and storage 6. Focus on other crops. Two key areas: (1) urgently need a high yield sorghum hybrid with good brewing qualities; (2) urgent need for soybean variety with high pod clearance for efficient mechanical harvesting 1.SARI needs to strengthen its internal systems for managing the funds generated by its fee-based services including seed sales. 123 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 2.The commercial unit needs expand it capacity to engage with end users identification of their needs and responding to those needs. 3.SARI needs to strengthen its outreach to women along the value chains that they support—particularly women involved in aggregation of crops for sale (because they are aware of the markets) and women involved in agro-dealer business. 4. Future programs need to facilitate private sector actors in developing the business plans they need to access low interest loans from bank for seed processing and storage equipment 1.Oneline short courses for AEAs and for the researchers (short courses and more long-term courses also good) 2. Future trainings need to award certificates to the AEAs. 3. Future trainings need to include audiovisuals and printed materials to help with scaling up ToT trainings 1.The management was bad 2. The management board was not giving adequate support 3. There very LOW oversight initially from CSIR 1.Lower the cost of soil testing and increase farmers’ access to the kits that they need to collect soils for testing. 2.Support the development of action-oriented training videos in local languages and circulating these messages through movie vans/tricycles that can access remote areas 110 Name the principle lessons learned from what worked and did not work in this project for USAID to consider when designing similar programs in the future. (List) First develop a list of what DID WORK then rank the top 3: 1.Communication with local government stakeholders is critical to sustaining the activities. This includes ensuring that the reports are circulated to them. MOFA was engaged but they need more engagement and continuity. They need to expand the scale. 2.The project was very good. Everything done o this project should be scaled up to the orth. 3.Need to scale up their support of the market linkages component. The Golinga seed growers are an example of best practice. Some are even exporting their rice. If more support is given to this. 4. SARI needs to improve on its foundation seed support. 5. Organic fertilizer. SARI needs to produce more organic fertilizer and more available. 1. SARI plays critical role in any current or future attempt to develops cereals and legumes in the north . 2. Board oversight—and the materials needed to ensure oversight—is critical to transparent project execution. 3. Transparent exchange of information to all stakeholders (small holders, MoFA and SAR staff)—is critical to effective project execution. 4. Steering committees should be required of all SARI projects. - Agriculture thrives on research. With strong research, AGRIC can boost. We want to see more support to research to come out with more varieties. More short duration climate smart technologies. - Technology development is critical to taking us from one step to another in terms of agricultural production in northern Ghana. They have introduced us to new technologies, we want them to move us to the next level. - The project helped SARI improve its internet connection by accessing the GARNET system and develop its first fully functional website -There is no way the private sector can do well without the public sector. This type of project which seeks to strengthen both the public and private sector is very good. This is the way to go to sustain our development operations. If we should do anything at all we need to further strengthen the public and private sector working -What we learned from our collaboration with SARI has made me more conscious of the particular challenges of supporting maize and rice to the point that I am now competent to advise anyone on these two crops 124 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking - With our collaboration with SARI I am now better placed to advise farmers on the different types of varieties they should consider for different climate and soil conditions 1.Demonstraton trials are absolutely critical to the core mandate of both MOFA and SARI for two reasons: 1) it gets the message out; 2) it provides scientists feedback that they can incorporated into their research programs. 2.Capacty building of MoFA agents is absolutely critical to the core mandate of both MoFA and SARI for two reasons: 1) It improved my professional capacity to communicate to my farmers about new technologies; but it also 2) improved my capacity to communicate with SARI. This training is critical. - The SARI model for supporting MoFA agent training at the station and demonstration models is an example of best practice that needs to be scaled up and sustained - I have learned a lot about farming from SARI about production. I have learned how to grow seed. They have been very willing to support me. I pray that they continue because they improve farming activities. Without them we would miss a lot. They have a lot of information. Without them, my farming activities would not have improved. 1. They are able to inform and educate farmers on the use of inoculum but due to cold chain, input dealers cannot sell the inoculum to farmers 2. Working with the SARI officials has been helpful (esp. the Scientists) for a the MoFA extension agents First develop a list of what DID NOT WORK then rank them: - cold storage and processing: As SARI’s capacity to produce seed increased, it quickly outstripped the capacity for processing and cold storage - If SARI’s capacity to successfully create new employment….there needs to be a gender expert/coordinator for the institute who can provide leadership on these issues. - Certain partners—like the seed division –need information on what is happening in the M&E Yields; income impacts; women employment Key partners would include: Seed Producers Associations, MoFA, and the Seed Certification Unit. - Steering committee is critical - Building the financial system needed to capture AND RE-INVEST revenue from seed sales if the institutions like SARI are to continue to support these activities after the project ends. They gave the seed on loan and they never collected. 1.SARI does not have the in-house technical expertise to continue to update the aesthetics of the site or to make it more user friendly. 2.SARI never developed an institution-level knowledge management system. 3.Athough the internet access improved at the main campus, the main campus was never rewired to make it user friendly and none of the field stations got access. - What I would like to tell the US government is to be more supportive of SARI and supporting the MOFA to scale up the demonstrations - Due to cold chain, input dealers cannot sell the inoculum to farmers -Older varieties need to be re-examined by the labs to make sure that we are keeping the genetic qualities of these seeds to meet modern needs, not just focus on new variety release.. There are old varieties that we need to look at again. Farmers still like some of the old varieties that were developed by global 2000. We need to re-evaluate the older varieties to meet the modern requirements EQ 6 128 SARI’s capacity to support agricultural research in 1-4: 3 5: 11 6: 5 7: 1 - This will depend on the budget for research they receive from Government. This is part of their normal activities. It is expected that “fee for services” would generate income to complement the government budget allocation. The “plant breeders’ rights” bill is now law. If you release a variety you get a certain income from that variety. 125 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking Northern Ghana will continue once the project ends. (Rank 1-7) %5-7: 85 (17/20) - They have to improve on their financial systems management or they will not be able to sustain their research capacity. The institute must learn how to generate money to survive 129 SARI’s capacity to support the development and dissemination of improved agricultural technologies will continue once the project ends. (Rank 1-7) 1-4: 3 5: 10 6: 4 7: 1 %5-7: 83 (15/18) - We strongly disagree that SARI will be able to continue. The level of engagement was much higher. We did not even have a day to rest 130 -Technologies for seed 1-4: 2 5: 6 6: 10 7: 1 %5-7: 89 (17/19) - They were already doing this before the project. The project ramped it up. They can continue but develop new varieties and respond to market demands without donor funding - They were not able to scale up in their last two years and w/o donor funding they unable to scale up on their own - There is almost no support in Wa. They are coming for research on farm but they are not engaging us. They are staying on station. The on-farm demonstrations are useful. 131 -Complementary technologies for soil management and crop production 1-4: 9 5: 3 6: 7 7: - %5-7: 53 (10/19) - SARI less likely to continue working on this set of technologies because they are not bringing I financial revenues like the seed production technologies. - This will continue because of the RELC. The SARI staff are heavily involved in the RELCS 132 SARI will continue to develop a more market and client￾oriented research approach once the project ends. (Rank 1-7) 1-4: 9 5: 2 6: 4 7: 1 %5-7: 44 (7/16) - if they have to depend on their own resources unlikely - They will try . Part of the output is that market demand has been established. A certain amount of foundation seed demand has been created. It will grow - Now they are doing client-oriented research. The market-oriented preference aspect will require market research 133 -Client-oriented research approach 1-4: 8 5: 2 6: 7 7: 1 %5-7: 56 (10/18) - This fits with their core mandate. - RELC is a component that makes this 134 -Market-oriented research approach 1-4: 11 5: 4 6: 1 7: 1 %5-7: 35 (6/17) 135 -For the seed producer associations 1-4: 7 5: 5 6: 7 7: - %5-7: 63 (12/19) - They will sustain because it generates funds - SARI sells the foundation seed. They have a financial interest in maintaining this relationship 126 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 136 -For the SARI commercial seed￾production enterprises 1-4: 5 4.5: 1 5: 6 6: 7 7: 1 %5-7: 75 (15/20) -The structures are not in place to monitor activities. This is a research facility. They are not well adapted to handling the commercialization. They must better link to the commercialization and focus on the seeds - Before this project we were paying them to produce seed that private producers wanted - If they are able to push the engagement with the public Source: CSIR-SARI Technical and Financial Support Project, Final Performance Evaluation, FGDs and KIIs. 127 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex IV.B. FGDs/KIIs Summary Results for Community-based Associations and Private￾sector Partners (Stakeholder Group 4) # Questions Rank 1-7 Explain Your Ranking EQ 1 The USAID￾funded CSIR-SARI Capacity and Technical Support Project has made the following groups more effective: (Rank 1- 7) -They have benefitted in terms of yields and the quality of production; their livelihoods have benefitted. They now understand that seed is very very important to them. Before they did not know how to get quality seed. Now they buy seed from the seed producer groups in the villages. 29 MoFA extension staff 7 30 Small seed-producer groups 5 -they need to be trained more in order to identify obstacles. We need them to be better able to determine the quality of the seed, the current quality of the seed they deliver is not high enough. We need to be able to identify the right quality of the seed 31 Associations hosting field demonstrations and farmer field schools 7 - They see our varieties are better than their traditional varieties. The yields are much better 32 Community people who participated in farmer field schools 7 - This helped all the farmers. Everyone came to the demonstration trials and trainings. they learned from the demos that they could produce two crops in a season EQ 2 The CSIR-SARI Capacity and Technical Support Project has increased the efficiency and efficacy of the following: (Rank 1- 7) 74 MoFA extension staff 75 Small seed-producer groups 76 Associations hosting field demonstrations and farmer field schools 77 Community people who participated in farmer field schools The CSIR-SARI Capacity and Technical Support Project has increased the 128 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking profitability of the following: (Rank 1- 7) 78 Small seed-producer groups 6 79 Associations hosting field demonstrations and farmer field schools 6 80 Community people who participated in farmer field schools 5 EQ 3 92 Working with the CSIR-SARI Capacity and Technical Support Project created new opportunities and unexpected outcomes. (yes/no) Explain your answer (List new opportunities and unexpected outcomes.) -Youth: “Most of the youth find that farming is more profitable. They find they can make money. The youth are finding the new varieties—Wandata and cowpea and the soy bean —are crops they can make money with. They are now more interested in going into agriculture. They can get more production during the dry season from along the river bank.” It has improved the livelihoods of the youth. Wandata is a newly released maize variety. The maize and the cowpea are cash crops the river banks are good for producing this. This was not done before. (a) Wandata (maize) (b) cowpea (wankai); (c) soybean (2 varieties, Favor plus another). They are growing in both the dry and wet season this was not done before. -Cowpea technology: Cowpea is important. That is a cash crop. Cowpea is making more money for the farmers. -Dry season has become more profitable. Dry season production has gone up a lot. -Women are getting more empowered: They are better connected to SARI. They are able to better express themselves. Women are feeling more confident about having their own fields; they are improving their livelihood. EQ 4 The CSIR-SARI Capacity and Technical Support Project encouraged the following groups to involve women in the activities: (Rank 1-7) 102 MoFA extension staff 103 Small seed-producer groups 7 104 Associations hosting field demonstrations and farmer field schools 7 105 Community people who participated in farmer field schools 7 - Project encouraged women to participate in the activities. Covid 19 did not affect SARI’s work on-farm very much EQ 5 129 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 111 What activities worked well with this project that you would like to see in future projects? - The basic model was fabulous. It was a very very good model. It enabled us to come together to solve problems - It enabled us with technology and logistical support to engage with MoFA to reach the farmers…and to scale up our demonstration trials. We needed the infrastructure and the equipment. 112 What activities did not work well with this project that could be addressed in future projects? - One of the challenges is they did not have enough equipment. They need tractors. Some of the research equipment. Some instruments. We need a lot of services - When we do our experiments. We need fences. For the office for the demonstration trials the animals destroy them. They need vehicles EQ 6 The following groups have continued to work with CSIR-SARI after the project ended in December 2019: (Rank 1-7) 137 MoFA extension staff 5 - We continued to work with the groups. They continue to come to us and get new varieties 138 Small seed-producer groups 5 - The seed production still goes on. They come to us and pick the seed and see the farmers 139 Associations hosting field demonstrations and farmer field schools 5 - Funding is a problem 140 Community people who participated in farmer field schools 5 - Funding is a problem Source: CSIR-SARI Technical and Financial Support Project, Final Performance Evaluation, FGDs and KIIs. 130 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex IV.C. FGDs/KIIs Summary Results for Media Actors # Questions Rank 1-7 Explain Your Ranking EQ 1 14 The USAID￾funded CSIR-SARI Capacity and Technical Support Project has strengthened SARI’s capacity to collaborate with the major media outlets in Northern Ghana. (Rank 1-7) 4: 1 6: 3 7: 1 %5-7: 80 (4/5) -I worked with SARI and not the project per se. I am not that aware of what the project did. -Prior to this, SARI has not taken an interest in building the understanding of people on the technologies they generate. They have not yet trained the journalists. They have limited capacity for dealing with the press: 1) no tradition of press briefing; 2) staff are not trained in how to work with journalists; and 3) the communications officer is not fully empowered to talk with the press without consulting his bosses—the technical scientists and/or the Director 4 The CSIR-SARI project’s support for media and communications has increased SARI’s capacity to support agricultural research in Northern Ghana. (Rank 1-7) 5: 2 6: 3 7: 1 %5-7: 100 (6/6) -With the kind of stories I do, they are mainly stores to tell the funding partners what they (SARI) have been up to…but not necessarily to inform the farmer about the methods they use to achieve new technologies -I went to an event a few months after the emissions, and one farmer was talking about how much the project with SARI had increased their access to seeds. I know it did make a difference from the call-=in traffic we got. Before the project SARI had its old way of communication. This project and the productions we did with them increased SARI’s impact. This has helped the farmers 48 The CSIR-SARI project’s support for media and communications increased SARI’s capacity to support the development and dissemination of improved agricultural technologies. (Rank 1-7) 5: 1 6: 4 7: 1 %5-7: 100 (6/6) -For this impact to occur, I am not an inside guy. Our duty is to transmit and publicize what SARI does through our channels. But we could not do much beyond this for the local farmer to adopt it the technology EQ 2 70 The CSIR-SARI project’s support for media and communications has helped strengthen the local communities’ understanding of and willingness to invest in improved seed. (Rank 1-7) 4: 1 5: 1 6: 2 7: 1 %5-7: 83 (5/6) -When they did the story with us, we made an off-hand reference to the improved varieties. BUT for the farmers themselves to be able to appreciate the new varieties. (I was not on the ground with them). I have not seen any audio- visual materials or mobile platforms - Whatever they do, they are trying to better tailor the innovations to the needs of the farmers. They are doing their best. Whether the farmers understand and adopt is another issue - A lot of other projects have contributed as well. 131 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking 85 Which types of media programming supported by the CSIR-SARI project do you consider to have been most and least effective in improving farmers’ welfare? Most effective: 1.Facilitated you talking to farmers. Very effective the way they took us to the farms. These field visits were very useful; useful to talk to the communities. SARI provided limited support for costs and lunch. 2. The story was posted on face book; also re-broadcast on the weekends. 3. Explained the new variety and new technology 1.Broadcasts in local languages that the people could call an comment on. 2.Broadcasts that were based on field visits with farmers when the scientist were doing demonstration plots 1.The technical manuals that we developed early on and update periodically. 2. The radio programming in local languages (the one with radio 1,2,3) was the most successful. 3.The radio programming with Savannah radio is the 2nd most effective. 4.The radio-net program is very important. We pilot tested this in five communities. It is a pilot test we conducted in five communities from the Free University in Amsterdam. USAID did not buy into it. Since USAID dd not buy into it, we had no money for staff or for new programming. What we did was lose local workers (drivers) who spoke the local language to read updated versions of the technical manuals they had developed n 2015, one page at a time. Least effective: 1. They did not support the transportation costs to and from the site. 2. They should widen up the communication angle. Open up new ways of working with journalists. That attitude is not just for SARI it cuts across different institutions. When they call me, I make sure that the story comes. There other media outlets that they call and the media houses do not get the story out. They want to invest time in choosing media houses that they will convey their message. 3. When I file my story, my news editor decides when they will put it out. The type of follow up they do is to see if the story goes out or not. They have not followed up on what impact the story had. 4. Lack of independence of the journalists to interview the farmers. The experts sit with the farmers. This inhibited the independence of the farmers to talk openly to us. Because of the distance between the campus and the field sites, we don’t have resources to meet the farmers on follow ups. The stories would be more credible if we did. 5.None of the transmissions were in the Dagbani, one of the major local languages of the focus region 1.Scientists talking 2.Programs that only talked with male farmers (needed to talk to female farmers too) 86 The CSIR-SARI project’s support for media relations and communications has increased farmers’ access to information and training about improved soil management practices. (Rank 1- 7) 4: 4 5: 1 6: 1 %5-7: 33 (2/6) -They listed the advantages of the new varieties, that they do not degrade the soil. If you grow soybean with your maize the soybean will fix some nitrogen for the maize plant to use. Soil fertility management was discussed - We dd not target soil management per se. We tended to focus on the virtues of the new varietals and the agronomic practices that were being recommended for the new varieties—not soil management. 87 The CSIR-SARI project’s support 4: 4 5: 1 - Agronomy was pushed ; soil management was not a major focus of this. 132 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking for media relations and communications was effective in increasing farmers’ knowledge about appropriate production practices necessary to get the maximum benefit from the improved seed. (Rank 1-7) 7: 1 %5-7: 33 (2/6) 88 The CSIR-SARI project’s support for media relations and communications has improved the welfare of rural farmers in Northern Ghana. (Rank 1-7) 4: 4 6: 1 7: 1 %5-7: 33 (2/6) - We did increase access to information about these things indirectly. EQ 3 93 The CSIR-SARI project’s collaboration with media outlets has created new opportunities for rural farmers in Northern Ghana. (Rank 1-7) 6: 4 7: 1 %5-7: 100 (5/5) 94 Have these new opportunities created any unexpected community-level outcomes? (yes/no) If yes, please explain: - I presume (but cannot confirm that the farmer would benefit) - Many of the local radio stations are now imitating your programs. Radio etc. (we did a video documentary on this) EQ 4 98 The CSIR-SARI project addressed gender issues in its collaboration with your media outlet. (Rank 1-7) 3: 2 4: 1 5: 2 %5-7: 40 (2/5) - It did not talk about women. None of the farmers were women -Most of the engagement of the two people that conducted community-level visits with farmers in conjunction with the preparation of the programs they were commissioned to produce were dominated by men. Most of the SARI staff they engaged with were women. This is despite the fact that most of the beneficiaries are women. Even when we went to the village you did not even talk to women - In the interviews we tried to have women talk as well as men. It was not a target but it was indirectly embedded in it 106 Were there any examples of best practice for strengthening If yes, please explain: -The key thig we did was to try to always select a woman . We required our reporters to take to the female farmers and how they uderts. Any time we did an interview or a training 133 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking gender inclusion through the media campaigns in this project that future projects could learn from? (yes/no) 107 What additional recommendations do you have for using media to strengthen gender mainstreaming in SARI’s agricultural research in future donor-funded programs? 1. The women only help during harvest. Most of the work is done by men 2.Support of Community-based journalists—especially women—can play a huge role in facilitating medica communication with women and communities. This could go a long way have social access that an outsider would not have. The community would accept them as someone talking to them about our own issues 1.Target women in interviews. 2.Radio net from m the data we got this did not make a big difference. The women could access it at home EQ 5 115 Best Practice: Do you have any examples of media outreach from this project that should be incorporated into the design of future USAID￾funded projects that work with SARI? (yes/no) If yes, please explain: 1. Do more drama on the themes (for radio and tv) 2. Give the journalists more opportunity to conduct interviews on their own; 3. Facilitate transportation to the campus so that the journalists can stay informed; 4. Transmit in local languages 1. Live broadcasts of farmers using the new technologies with call-ins 2. Inviting and supporting media coming to various events like the launches 1.Radio program should continue also from the start local languages 2.Radio net 3 Conscious policy for gender targeting (interviewing women on all the big stories) 4.The CT center is a very good idea. Future projects need to continue to update equipment especially for web studio. This is part of the structure web. 5.The manuals good idea; need to scale them up and put them in local languages 116 Are there any lessons learned regarding what worked and didn’t work in terms of media collaboration under this project that USAID should consider for future projects? (yes/no) If yes, please explain: 1. As a media consultant, I would recommend for SARI to take up these matters, it is important to have a media consultant that follows at every step. This would make sure that they were constantly involved in media engagement. 2. Use this consultant from the beginning and even include him/her in the design of the proposal so that you can capture what is happening. You need to make sure it is done from the planning phase. It should not be just when they are launching the project and when they are closing it. This would keep you from just doing a story on cowpea every day. You need to get into changing the mindset of the farmers. 3. More engagement is needed. Especially with women. We should do programs that focus consistently on women. You can do one on farmers in general….but programs focused on women would be very important . This could even lead them to asking the chief for land. 4. Consider creating a WhatsApp account so that the communications coordinator can communicate directly about new developments. 5. It is important to find like minded media houses to ensure that the farmers are getting good information. We have other entities. All the government agencies have a common media platform. We would then develop better methods for communicating 134 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Rank 1-7 Explain Your Ranking -Even though there is a formal commitment to media support the media will not get the support they need nor will it be sustainable after the project ends, unless this support is institutionalized and carefully monitored 1.Journalist training: Train media journalists (we did this on an earlier project; there is a precedent) and then helping the trained journalists stay in touch with one another. 2.Journalist networking: In an earlier biotech project, we did an email list and what’s app to keep people posted. 3.Require new projects to support ICT in conjunction with their projects: Reactivate the idea (floated by ISU during the COE review but never formally approved) that future project proposals to SARI should include a budget line for ICT. 4.Movie van outreach: Consider re-activating the type of movie vans that SARI supported on some of its earlier USAID-funded project and letting donor projects and MoFA rent them to show some of the SARI developed videos. 5.Press briefing material: Anticipate the need for press briefing materials. EQ 6 147 The collaboration between my media outlet and SARI has continued since the project closed. (Rank 1-7) 3: 2 6: 2 7: 1 %5-7: 60 (3/5) - They do not have the means. They will talk about the lack of resources. They have not restructured their system to continue to support - This is something that cannot stop 148 Is this collaboration likely to continue? (yes/no) Why or why not? In the last year it has gone down. -No. They do not have the means -Yes. We have come to a level where we have information and they need information. Their job is to provide information to people. They need the information n order to never their rankings. No longer how it takes it cannot stops Source: CSIR-SARI Technical and Financial Support Project, Final Performance Evaluation, FGDs and KIIs. 135 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex IV.D.1. CSIR-SARI Staff Post-Interview Online Questionnaire Summary Results (/31) # Questions Scored 1-4 Scored 5 Scored 6 Scored 7 % Scored 5-7 EQ 1 1 SP 1. This project has increased SARI’s capacity to support agricultural research in Northern Ghana. 4 8 18 1 87% Output 1.1. The project has helped SARI’s scientists to train and retool in key program areas that have or are likely to strengthen the capacity of SARI to deliver on its mandate: 6 -In northern Ghana. 8 10 11 2 74% 7 -In the savannah areas of West Africa. 13 10 7 1 58% 8 -For the rice component. 11 6 10 4 65% 9 -For the maize component. 8 8 11 4 74% 10 -For the soybean component. 8 6 11 6 74% 11 Output 1.2. The project has increased the organizational quality and efficiency of SARI. 5 12 11 3 84% 12 The project enabled SARI to improve and strengthen its new model for an Independent Project Manager, who was focused 100% on achieving project objectives. (Activity 1.2.1) 13 8 6 4 58% 13 Developing public relations and communication plans (in order to strengthen SARI’s linkages to community-based groups and the private sector) helped build the organizational capacity of SARI. (Activity 1.2.2) 6 10 11 4 81% 15 Building the capacity of the CSIR-SARI Management Board increased the organizational quality and efficiency of SARI (Activity 1.2.3) 13 10 8 - 58% 16 Building the capacity of the Internal Management Committee (IMC) increased the organizational quality and efficiency of SARI. (Activity 1.2.3) 13 12 5 1 58% 17 The new project management structure—the Advisory Council, project management/steering committee and committee of component leaders—increased the organizational quality and efficiency of SARI. (Activity 1.2.4) 14 11 6 - 55% 18 Training staff in USAID financial procurement and reporting regulations increased the organizational quality and efficiency of SARI. (Activity 1.2.5) 10 8 11 2 68% 19 Updating SARI’s strategic plan for making SARI a Centre of Excellence (CoE) increased the organizational quality and efficiency of SARI (Activity 1.2.6) 7 9 11 4 77% 20 Providing support for the implementation of SARI’s long and short-term staff capacity 8 13 8 2 74% 136 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Scored 1-4 Scored 5 Scored 6 Scored 7 % Scored 5-7 development plan increased the organizational quality and efficiency of SARI (Activity 1.2.7) 21 The project enabled SARI to improve and strengthen its financial systems for managing donor-funded projects. 10 9 10 2 68% 22 Output 1.3. The project-built and project￾improved infrastructure has benefitted agricultural research activities in Northern Ghana. 6 8 10 7 81% 23 SP 2. The project has increased SARI’s capacity to support the development and dissemination of improved agricultural technologies. 4 7 16 4 87% 49 Output 2.1. The activities the project supported significantly modernized SARI’s seed operations in line with the Plant and Fertilizer Act. 7 13 10 1 77% 50 Output 2.2. The project has strengthened SARI’s DEVELOPMENT of improved soil fertility practices. 9 17 5 - 71% 51 Output 2.2.The project has strengthened SARI’s DISSEMINATION of improved soil fertility practices. 9 13 8 1 71% 52 Output 2.3. The project has built the ability of SARI’s staff to apply modern tools /techniques in research. 6 12 13 - 81% 53 Output 2.4. The project has helped SARI to improve the Research Extension Linkage Committee (RELC) mechanism for agricultural technologies dissemination. 12 8 8 3 61% 54 SP 3. The project has helped SARI develop a more market and client-oriented research approach. 13 11 7 - 58% 55 Output 3.1. The project’s support has strengthened SARI’s systems for information and communications technology (ICT) and knowledge management 4 10 12 5 87% 56 Output 3.2. The project’s support has strengthened SARI’s commercialization systems—i.e., offering fee-based services and goods (like improved seed) to stakeholders and others. 10 14 7 - 68% 57 Output 3.3. The project’s support has strengthened SARI’s monitoring and evaluation (M&E) systems. 8 12 9 2 74% EQ 2 58 The project has helped SARI strengthen its organizational capacity in order to improve farmer welfare (e.g. income, crop productivity, resilience) in Northern Ghana. 9 9 12 1 71% 66 The project has increased farmers’ access to improved seed. 3 6 17 5 90% 137 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Scored 1-4 Scored 5 Scored 6 Scored 7 % Scored 5-7 69 The project was effective in increasing farmers knowledge about appropriate production practices needed to get the maximum benefit from the improved seed. 4 8 14 5 87% 71 The project was effective in increasing farmers’ access to information and training about improved integrated soil fertility management practices. 8 11 9 3 74% EQ 3 EQ 4 95 The project addressed gender issues in SARI’s research and development projects. 15 11 5 - 52% 99 The project helped SARI track the participation of women in its training programs. 17 4 10 - 45% 100 The project helped SARI track the participation of women in its research programs. 15 8 8 - 52% 101 The project’s support for research and infrastructure development is helping CSIR￾SARI to achieve a more equitable gender balance in agricultural research in Northern Ghana. 13 11 5 2 58% EQ 5 EQ 6 117 1.1. Rate the likelihood that “core scientists trained and retooled in key program areas” will be sustained or continued after the program ends. 6 11 11 3 81% 118 1.2. Rate the likelihood that “increased organizational quality and efficiency of SARI” will be sustained or continued after the program ends. 5 9 15 2 84% 119 1.3. Rate the likelihood that “infrastructure and facilities of the Institute built” will be sustained or continued after the program ends. 3 10 13 5 90% 120 2.1. Rate the likelihood that “SARI’s seed operations significantly modernized” will be sustained or continued after the program ends. 6 7 14 4 81% 121 2.2. Rate the likelihood that “integrated soil fertility management practices developed and disseminated” will be sustained or continued after the program ends. 8 7 13 3 74% 122 2.3. Rate the likelihood that “capacity of technical staff to apply modern tools/techniques in research built” will be sustained or continued after the program ends. 4 12 13 2 87% 123 2.4. Rate the likelihood that “RELC mechanisms for agricultural technologies dissemination improved” will be sustained or continued after the program ends. 7 7 11 6 77% 124 3.1. Rate the likelihood that “ICT and knowledge management systems for enhanced 3 11 16 1 90% 138 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Scored 1-4 Scored 5 Scored 6 Scored 7 % Scored 5-7 market driven research for development developed” will be sustained or continued after the program ends. 125 3.2. Rate the likelihood that “commercialization program at the institute strengthened” will be sustained or continued after the program ends. 8 11 8 4 74% 126 3.3. Rate the likelihood that “M&E system developed and operationalized” will be sustained or continued after the program ends. 5 8 14 4 84% Source: CSIR-SARI Technical and Financial Support Project, Final Performance Evaluation, Post-Interview Online Questionnaire. 139 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex IV.D.2. Government Partners and Non-Government Donor and Project Partners Post-Interview Online Questionnaire Summary Results (/14) # Questions Scored 1-4 Scored 5 Scored 6 Scored 7 % Scored 5-7 EQ 1 1 SP 1. This project has increased SARI’s capacity to support agricultural research in Northern Ghana. 1 2 5 6 93% 11 Output 1.2. The project has increased the organizational quality and efficiency of SARI. 3 3 5 3 79% 22 Output 1.3. The project-built and project￾improved infrastructure has benefitted agricultural research activities in N Ghana. - 6 4 4 100% 23 SP 2. The project has increased SARI’s capacity to support the development and dissemination of improved agricultural technologies. - 2 3 9 100% 49 Output 2.1. The activities the project supported significantly modernized SARI’s seed operations in line with the Plant and Fertilizer Act. - 2 6 6 100% 50 Output 2.2. The project has strengthened SARI’s DEVELOPMENT of improved soil fertility practices. 2 3 4 5 86% 51 Output 2.2.The project has strengthened SARI’s DISSEMINATION of improved soil fertility practices. 2 2 4 6 86% 52 Output 2.3. The project has built the ability of SARI’s staff to apply modern tools /techniques in research. 4 2 5 3 71% 53 Output 2.4. The project has helped SARI to improve the Research Extension Linkage Committee (RELC) mechanism for agricultural technologies dissemination. 3 3 2 6 79% EQ 2 58 The project has helped SARI strengthen its organizational capacity in order to improve farmer welfare (e.g. income, crop productivity, resilience) in Northern Ghana. 2 3 3 6 86% 66 The project has increased farmers’ access to improved seed. - 2 4 8 100% 69 The project was effective in increasing farmers knowledge about appropriate production practices needed to get the maximum benefit from the improved seed. 1 1 5 7 93% 71 The project was effective in increasing farmers’ access to information and training about improved integrated soil fertility management practices. 1 4 4 5 93% EQ 4 95 The project addressed gender issues in SARI’s research and development projects. 5 1 7 1 64% Source: CSIR-SARI Technical and Financial Support Project, Final Performance Evaluation, Post-Interview Online Questionnaire. 140 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex IV.E. Summary Results of FGDs/KIIs Recommendations by Category Recommendations Frequency Collaboration Future projects need to strengthen collaboration with projects working the same area (soil mapping)/G2G projects, even adding a budget line to ensure their continuation. 2 Strengthen communication with key government stakeholders. (brief MoFA more often), including co-reporting on tech dissemination, advocating for ag research 3 Future USAID-funded GoG projects need to anticipate the need to formalize the type of mentorship from other USAID funded projects ( like ATT and the Soy-bean innovate lab provided for this project, especially for internet, ICT and marketing) that these project need to be successful and efficient. Could include co-funding and co-monitoring 5 SARI needs to strengthen its linkages with northern Niger and Niger and its research institutes in francophone areas. Possibly use mini sabbaticals and give mini-scholarships for learning French. 1 1.Encourage more MoFA training in the future but should be more practical 2. After the training give certificates. 3. There should be written modules in English to facilitate them training other groups. 4. At graduation do more publicity on the radio and TV to increase policy makers and government and private sector peoples’ awareness of this training and the technology disseminated 3 Commercialization The laboratories could provide service for a fee to farmers (including cheaper soil analysis), including demand driven research, be proactive in negotiating with end users about their varietal needs, developing better breeder seeds. Focus these efforts on field stations 11 Strengthen SARI’s ability to commercialize the seed, including marketing/branding division, storage and processing plant to sell for domestic and foreign markets; possibly funding to support the development of a lager industrial seed production company in northern Ghana (include researching factors that affect farmer’s willingness and ability to purchase seed) 9 CSIR needs to support the current GoG commitment to new law to protect plant breeders rights; support advocacy for seed certification 2 Need to strengthen linkages with the large industrial purchasers of seed, private seed producers, industry, and all private partners to disseminate seed (including a clearer legal mandate to engage with commercial businesses) 7 Set up a monitoring system to monitor progress of the institution responding to the demands of different stakeholder on commercialization (including helping build SARI’s capacity to conduct the due diligence they need when contracting with companies through signed MOUs) 2 Help SARI develop better systems for restricting research scientists setting up side businesses to sell seed (which reduces the credibility of the institute) 2 Future projects need to include a budget for the commercialization unit as well as robust indicators (developed with the involvement of the commercialization unit staff in the design of the M&E studies and reporting) to track the execution of SARI’s internal targets for building the commercialization unit and its activities and to link budgets to impact on the ground. 7 Build staff capacity to ensure the funds generated by commercial activities return to their appropriate account and to ensure a transparent system 3 Future government programs should facilitate private companies have their own facilities (including seed-storage systems) and labs that they can market, encourage private-sector seed investment including access to low-interest loans. Should be highly regulated 11 Extension Build the capacity of the community based groups toward groups/cooperatives; also build the capacity of the researcher to be more oriented toward the need of the farmer (adaptive research) (including the development of wide-scale soil testing) 4 Identify central points for processing plants and warehouses (and possibly tractors) that could be managed and maintained by groups of farmers 4 141 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Recommendations Frequency USAID should consider working with other donors and the Ministry of Finance, MoFA, and CSIR on identifying ways the cost of agricultural finance can be lowered 2 Bring in other crops based on the food security needs of the different regions (including cowpea, groundnut, and sorghum 4 anticipate strong collaboration with the highly successful ongoing RELC system that links SARI and MoFA because it works and is established (including more meetings) 3 Give partners more technical trainings (including in varieties, climate change practices, and FAW) using videos and written materials in local languages. 7 Need to anticipate the need for more structured feedback on how the technologies are performing in their local communities (include MoFA) 4 Scale up to other communities; encourage community to community visits to learn from surrounding farmers 2 SARI need to facilitate a massive scale up of MoFA demonstration trials and field days for the new varieties to have their desired impact 1 Gender All future projects (and SARI) need to have a gender integration plan/strategy with clear policy, trainings, budget, and oversight (revise strategic plan) 7 SARI needs to have a gender expert and gender focal points at all outstations. Consider a gender working group. 8 Anticipate ways to build gender mainstreaming into SARI’s ongoing programs (using an agency-wide system that will capture the disaggregated indicators by gender but also by region and be generic enough to feed into requirements of specific donors) 4 Monitor/report on gender-based outreach (include participation targets) and use results to develop need-based strategies 7 Encourage grants to women to strengthen their collaboration with the MoFA agents in scaling up new technologies 1 A strategy for youth (male and female) should be under gender in order for it to be easier to track 2 Include a gender/youth checklist (including tracking) in the design of the project 2 SARI needs to strengthen its training and technical assistance for women VC actors (and specific trainings for women farmers) in order to strengthen the women’s involvement 3 Facilitate female university students doing internships and theses that are connected to SARI’s work (or MoFA) 6 Develop long-term and short-term management training in order to capitalize on the growing number of women working as SARI technicians; scale up hiring of women (using targets) 3 Encourage all the associations and producer group it works with to have women in leadership positions to strengthen the communication with the women farmers in these groups 1 ICT Develop an online library resource/ knowledge management system for its documents, possibly using INSTI to provide routine updating of the website in order to ensure that it is user friendly and adapted to the need of the Institute’s key stakeholder groups and give trainings to groups on its use 5 The entire SARI GARNET internet system needs to be reworked/finished being installed and connected to field stations 4 Future USAID funded projects need to emphasize the issue of internet connectivity as well as the other factors that affect NARS ability to maintain and use internet-based technology in their initial needs assessment and design (possibly using the four point assessment tool developed by the ICT unit of the soybean innovation to establish and monitor targets for improved connectivity and utilization); including a communications budget and communications officer 3 Figure out how to lower internet costs (has been successfully pilot tested in other countries) 1 Strengthen the capacity of the government to understand the benefits of research in conjunction with future ICT support 1 Create an ICT steering committee that could better link the institute with INSTI as a pilot initiative that could be emulated by the other 12 CSIR institutes 1 142 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Recommendations Frequency SARI needs to require all the projects it collaborates with to contribute to the routine internet costs. Going forward they might consider having 10% overhead for administration. This needs to be included in the proposals that donor submit to SARI 3 Future programs need to consider scaling up their support for extension videos (like those developed with AGRA support) and drama to facilitate the scale up of SARI’s new technologies (including integrating quality seed with soil fertility management)—which should include an M&E component. Consider using tricycle movie vans to help disseminate videos and include brochures about new varieties in local languages. 4 Strengthen the support for the staff in producing papers for publications, policy documents, communiques 1 SARI needs to consider ways that it can use to strengthen its linkages to the MofA AEAs and farmers with smart-phones in order to speed up their response to questions 1 SARI needs to strengthen its radio program to complement the demonstration trials and field days in order to scale up the impact of its technologies 1 encourage Ag officers and SARI ICT unit to collaborate on the identification of women who could help promote new technologies that SARI is developing and disseminating both through the demonstration trials and radio programs 1 SARI and future donor funded projects that are designed to build SARI’s capacity need to build the capacity of the staff to work with the press, including training to build the capacity of researchers on how to communicate about their work 1 Infrastructure SARI can recommend that new projects pay some of the costs associated with sustaining some of this critical infrastructure, include as a budget line 5 Irrigation is needed if seed production to go up 3 Future projects need to follow the rules and regulations to curb corruption and ensure quality construction 1 Future programs need to further strengthen: SARI’s laboratory facilities by supporting the development of a GIS lab, a seed lab and a fully functioning biotech lab; Fencing to protect research areas; infrastructure for the out-stations; capacity for cold storage (using solar energy) to store its own breeding materials and support commercial seed producers, including rehabilitating existing storage facilities; and updated farm management equipment (include budget lines for each) 13 Future trainings need to emphasize maintenance and repair not just use of equipment and infrastructure 1 Management weekly bulletin should continue 1 Institutional capacity building projects could benefit from having an independent outside person as program manager during start-up and using the person to mentor a person as project manager once that person’s tenure expire. Manager should be focused full time on the project, have full time salary or receive top ups, be part of the SARI internal management committee, have signatory rights on all expenditures, and have a strong management background 13 USAID needs to strengthen their supervisory role to ensure that the project is implemented according to the original project guidelines in collaboration with the beneficiary institution 3 Future programs need to encourage collaboration between internal units (auditors and M&E; commercialization and ICT; production team, the accounts, and the commercialization unit 4 Future projects need to anticipate the sustainability plans from the start so that they are in pace and operational by the closure 2 SARI’s mandate has to change because this new set of activities that CSIR is requiring most of the Institute to adopt. The mandate implies that the technologies should only be for resource partners….needs to be expanded to the current trend of commercialization 1 143 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Recommendations Frequency Steering committee should be a requirement before the release of funds and should include all key stakeholders (including outside groups like agro-dealers) 13 Future projects need an independent management board that is kept informed of all developments (especially accounting) 8 Anticipate the need for strong feedback loops with stakeholders 1 Future projects should always include the budget for mid-term Surveys and evaluations worked in from the start 4 Future projects should include a clear launch 1 The accounting systems need to be tightened, use better software, and be audited regularly at all levels by independent auditor; consider decentralizing the research funds to facilitate timely accessing of funds for research and avoiding the bureaucratic hurdles 4 The SARI Director needs to be reviewed by the board and should have management experience 1 Encourage SARI to honor its current policy of giving finders’ fees to scientists that attract grants 1 Future projects need to identify their core project staff and time commitment so that the staff can focus and not be torn between different projects -SARI needs to clarify the key staff that need to be involved from 50-100 percent of time in the proposal. - Annual evaluations need to monitor the participation of those persons in the projects they are supposed to support. - The same work plan agreements need to spell out what incentives the staff have for working on the project (i.e. training, publication support, sabbatical opportunities). -have a highly participatory, in-depth strategic planning processing the first year in order to ensure ownership 4 -Future projects need to make receipt of funds conditional on having a project management structure in place. -Future projects might consider a more flexible system of forwarding a percentage of the money to avoid them having the problems created by the monthly imprest system - developing a formal/MOU defined management support from another USAID-funded -project (like the ATT project or the Soybean innovation lab) during the first three years 3 M&E Future projects need to anticipate the need for baseline assessment of capacity like the one that SARI did with Africa as well as systems for monitoring progress toward the achievement of the goals set in the original baseline assessment -consciously select a simple set if indictors that it can use to track the impact of its community-level outreach on community-based stakeholders including the associations, and private see companies that benefitted from their companies 3 The audit system and accounting system needs to have better software 1 Future funding to SARI should be dependent on SARI creating a SARI-wide M&E unit (not projectized). -Including more funds to enable them to do their work and survey (equipment—computers, etc.) -dedicate one economist to M&E - One on the staff needs to be appointed the M&E point person - at least one M&E focal person in each of the SARI field stations - consider ways that M&E can be better mainstreamed and owned by all of the stakeholders in SARI 11 SARI needs to improve post-harvest monitoring of its seed quality and results at the farm level 1 Encourage staff to publish articles that are based on the M&E data 1 SARI needs to identify a few simple indicators that it can use to co-monitor its support for the MoFA led RELC mechanism 1 More effort needs to be made mainstream M&E into the institution by: 1) familiarizing the scientists (as the M&E people with information on M&E principles as we the indicators that the SARI report for SARI; requiring M&E training for all staff 4 144 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Recommendations Frequency Future projects need to consider mal grants t encourage UDS professors and their students to conduct research studies that will better document the household level impacts of SARI’s new technologies Organizational Capacity A short course on project management should be required, followed up with intensive management –2 or 3 week training---so that they get leadership training. Should be offered annually. 5 Need more training and technical assistance to support accounting and procurement [software and training) future projects need to ensure that all key staff to get basic training in procurement also to understand USAID’s rules and regulations and the project implementation agreement 5 Future projects need to develop better system for organizational capacity in the core campus and outstations (Future projects need to train staff in how to use the results framework and theory of change to guide reporting and strategic planning, steering committee meeting and reports to board) 6 Future projects need to build the capacity of the scientists to advocate for policy changes at the central level that will provide better support for research 1 Support for publication and writing is important (consider mini-sabbaticals for writing and analysis that facilitate partnering young researchers) 2 Consider using an innovation lab or other international institution that has a proven record of successful project management that led to positive change in capacity building in NARS to back stop SARI in management capacity and designing and scaling up the management capacity building of staff. 1 Short term trainings that are off-site helped the staff to focus 2 Future projects should build SARI’s capacity to co-report with other projects (including district level MoFA offices on the effectiveness of its back-up to the district level RELC systems) 2 Training and capacity building should be built into program and monitored in the IPTT 2 -Future projects need a plan and targets for computer, software, and lab equipment for the farming systems groups. -Future programs need to decentralize technician training as well senior staff training. -Future M&E systems need to track training and equipment by field station not just total. Need for disaggregated M&E on training equipment. 1 Future projects need to look at the institution as a whole and give priority to investments and training that benefit the entire institution (i.e. all of the component teams -i.e. breeding, management, M&E, Accounting, and integrated soil fertility management; not just a few crops) Refresher trainings should be enhanced 2 SARI and future donor funded projects that are designed to build SARI’s capacity for technology research and dissemination should anticipate the need for building the capacity to work with journalists is critical. -The journalists themselves need to help design the training. -The journalists need to help SARI build its capacity to produce press briefing kits for journalists. -The journalists need to help SARI staff become more adept at working with journalists 1 Other USAID needs to consider a quantitative study of the impact of the USAID-funded project as well as other supported projects at SARI on local stakeholders within the catchment area. This includes the savannah zone (for the three farming systems areas). The faring systems groups do not cover the entire 5 regions of the north of Ghana Scientists who receive resources, must account for what they produce with the resources Future projects to target seed need to start year one so that all aspects by completed by the end. SARI is still lacking certain aspects Future projects need to recognize the specificity of soil management needs and practices SARI needs to ensure stable staffing for technologists based in the three field stations and try to increase the number if possible Explore mechanisms for increasing the gas allowances for the SARI field technicians at the three field stations can be increased in order to strengthen their o support the adaptive and demonstration 145 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Recommendations Frequency trials they supervise as well as SARI’s support for the MoFA managed demonstration trials and the private seed producers in their district SARI needs to strengthen its adaptive trials for seed varieties and soil fertilization practices in the field stations like Wa and Bolgatanga in order to better address climate change factors affecting these area since these issues vary widely between areas n the north Future programs should facilitate MoFA working with the local governments /administrators creating standard weight and measures for all crops Source: CSIR-SARI Technical and Financial Support Project, Final Performance Evaluation, FGDs and KIIs. 146 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex IV.F.1. Summary Results of CSIR-SARI Staff FGDs/KIIs Frequency of Non-ranked Responses Concerning Good and Bad Unexpected Outcomes # Questions Responses Frequency EQ 3 89 What have been some of the unexpected outcomes (both good and bad) of this project for SARI? (List) Good unexpected outcomes: The emphasis on report writing was very good 1 Renovations and the two labs (soybean, rice) established 7 Offices are wired. Internet has improved thanks to Soybean Innovation Lab support for diagnosis and identification of a solution 8 Farm level understanding of the need for improved seed is much greater than before & had the unintended consequence of making the researchers more client oriented. 3 Radio had big impact 1 Commercialization was more successful, in part due to Planting for Food and Jobs program 3 The project ended up partnering with big farms and other projects, some even did joint initiatives to produce seed of SARI varieties 2 They have developed and strengthened the seed systems that have encouraged the SARI varieties to distribute 2 The Project has strengthened SARI’s relationships with MoFA to the point that MoFA has taken over the RELC mechanism that was originally led by SARI 3 Short term training improved our skills on this project and some of these trainings (in M&E, project management, procurement) in ways that are improving project program management globally and we used these skills to attract and manage other projects 2 This was a pilot and was not executed perfectly but it includes important lessons learned for all CSIR in executing G2G grants 1 I did not expect the US ambassador to pay us a visit 1 The strategic plan was much larger and more participatory approach 1 The nutrition program (was not envisioned). This was exciting 1 Another good unexpected outcome was the training on how to identify various life cycle of the pests and how to manage the fall army worm 1 Our data quality improved 1 Bad unexpected outcomes: Lack of communication about dollar/cedi conversions made accounting complicated 1 People did not cooperate, even the Director 1 No project launch to inform public and encourage collaboration 2 Lack of support from the top-director 1 No collaboration training 1 There was very little personal motivation to participate. Financially people did not benefit. Other projects give institutional support for the bills etc. and they give allowances 2 What we were expected to renovate was not up to standards. We need more offices. the field stations did not invest very much in infrastructure 4 We need fences for the fields. It is more of a problem than building because there is encroachment on the fields 1 147 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Responses Frequency A bad one is the abrupt end of the projects. The project was over and no approval had come for a no cost of extension. If it had been extended we could have wrapped things up better 2 A the start of the project, we believed it was to support the entire institution. Then in 2016 they got this “core of excellence” and a lot of people were left out. We have no idea how that happened. The focus changed and lots of people were disappointed. They did not follow the original proposal which was approved in the program implementation letter 7 Commercialization unit did not get the funds that were targeted for it 2 The project did not have an independent project manager 5 We did not reach out to the other G2G projects such as at UCC for guidance on how we could have addressed some of the problems we were facing 1 quality issues in seed related to third party production 1 The organizational capacity of the board was not improved 1 The baseline assessment and design under-estimated the impact that bad connectivity would have on every aspect of capacity building 2 The original emphasis on gender in the project document did not include mechanisms for design, implementing and overseeing the execution of a gender strategy. The original proposal did not anticipate the creation of a gender focal point for SARI or for the field offices 1 Had a midterm evaluation been conducted, then many of the issues with the role out of the M&E system and gender and even internet connectivity would have identified and corrected 1 The scholarships that were supposed to be funded by this project were not funded 1 The recommendations that the SIL made for using Garnett to do the rewiring was turned down because the $100,000.00 estimate was considered too high and the person with the lower bid (a contractor from India) was unable to compete the job 1 The slow burn rate which forced you to do last minute realignments of budgets that were not approved and to return funds that were not expended 3 In place of the seed business development unit, the leadership went after the development of a PPE with an Indian partner and they said if we had to agree to it 1 the issue of information flow (transparency) which resulted in key project management decisions were not being shared 2 The original project proposal was good but many key aspects of it (like the management structure, M&E system, and governance system) were not executed 2 The exclusion of other crops had unintended consequences in that it discouraged some of the research scientists from working on these crops 1 The monthly imprests was a big problem 1 The M&E training and support was concentrated on the economists 1 Strategic Plan Did not happen till third and fourth year of the project 1 90 What have been some of the unexpected outputs (both good and bad) of the project’s support to community-based groups. (List) Good unexpected outcomes: This project accelerated the dissemination of varieties that were already developed 5 Increased farmer access to high quality seed 1 Associations and vendors have increased their understanding of how to grow seed commercially 1 148 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Responses Frequency The farmers are benefiting from the haulms (of soybeans) from threshing the seed for their animals 1 women are getting more income from seed—this has been dramatically scaled up because of the scale for women’s employment in the seed value chain. 2 Original proposal did not anticipate the profitability of the out-grower system to both sari and the farmers 2 Bad unexpected outcomes: Breach of contracts by sub-contractors 1 The outgrower system left a bad taste in peoples ‘mouths 1 91 What have been some of the unexpected outputs (both good and bad) of the project’s support to private-sector businesses. (List) Good unexpected outcomes: Has strengthened linkages to SARI 1 Private seed producers have been helped to develop at a much faster rate than originally expected due to high demand created by the GoG planting for food and jobs program 2 Bad unexpected outcomes: The out growers were not paid the price they were promised 1 EQ 5 108 Identify three practices from this project that you would like to see included in future USAID￾funded projects. (List) Strategic planning /working as one unit 2 Effective communication of results, including publicity and marketing 2 Accountability 2 The emphasis on seed production 1 The emphasis on ICT, they need to facilitate and encourage and indeed monitor that these programs are designed, executed and evaluated in close collaboration with the commercialization unit 4 More involvement of women 2 Management training is good—it needs to cover all categories….it needs to be based on a plan 6 infrastructure budgets need to get approved more quicky (process needs to be ironed out clearly from the start so infrastructure projects can be fully executed before the project ends) 3 Need more farm equipment but equipment additions were good 3 The original project design was good and based on a formal capacity assessment of 9 different areas that was ground truthed by USAID 2 The extensive investment in infrastructure and facilities was critical (offices, equipment, and internet), but not sufficient 4 The demonstration and multi-site trials enabled SARI to have information on a wide group geographical location. This enabled them to reduce the number of years needed to complete the breeding cycles and also to give them the capacity to test and release varieties over a wide geographical area 1 149 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Responses Frequency Collaboration with the Soybean Innovation Lab was very good 1 The outgrower system needs to be taken to scale 1 The component team approach for research was a good approach 1 Capacity development. You need to have target for it and system for tracking it in the current project 2 When new technology is introduced, the training needs to emphasize how to use the technology but also how to repair and how to maintain it 1 Make sure that there is a complementary of roles with the institution and these roles are protected and policed by the SARI Director, the Internal Management board, and the Management Board 1 Expert visits. As part of that we had a plan for scientists to visit other sites and bring them out. Only a few of these visits were executed. These should be encouraged 1 Farmer training support was very good 1 110 Name the principle lessons learned from what worked and did not work in this project for USAID to consider when designing similar programs in the future. (List) what DID WORK Clustered leadership and crop-specific working groups 2 Development and implementing activity work plan process 1 Build the capacity for strategic planning at the institutional level 3 Anticipate the need for investment in renovation 3 Communications center is very important….that communication center went well 2 Involving women in commercial seed production increases the efficiency and efficacy of the activity 1 When you have sub-contractors you need to involve technicians in debriefings 1 This project emphasized the training and capacity of staff in M&E. Future projects to support capacity building at SARI need to link he disbursement of funds for the second year toed on the establishment and approval of r robust M&E system. Discipline in monitoring expenditures against results 6 Given the critical role of the commercialization units in ensuring the long-term sustainability of SARI, future projects need to: 1) build their capacity to operate efficiently and 2) to have the necessary bureaucratic autonomy and authority they need to do what they are tasked with doing to generate the funding for SARI 2 Demonstration trials and support for farmers very good 2 There was a powerful synergy between Advance, ATT and the USAID-funded SARI project 1 A robust gender strategy should be part of the program design 1 Future programs should consider predicating the initial disbursement of funds to SARI having the management and oversight structures (i.e. a steering committee, an independent project manager, and a model for reporting the SARI board and CSIR oversight council) in the proposal in place 2 The issue of connectivity has to be addressed 1 The critical importance of establishing a plan and sticking to it 2 Consistent, independent leadership is critical to manage the M&E system and project 3 Documentation, if we are going to change focus and everything you do as a project should be documented and given for the records 3 150 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Responses Frequency The field station need to have well defined budgets to ensure that they get the resources that they need 1 The critical importance of having a mid-term evaluation in order to adjust things not anticipated. Had they had a mid-term all these problems could have been sorted out 1 Fee for service approach 1 what DID NOT WORK Working as a team did not go well 2 Training and support for M&E 2 Bureaucratic process was a major constraint to procurement. No steering committee 4 Use per diem not reimbursement for the actual expenditure 1 Give the money at the start for the infrastructure so that it is completed by the end of the project 1 Strengthen the existing systems for internal audits 1 Do not change projects in the future…it upsets people 2 Having a gender integration plan –or just gender targets and coordinator 2 Manager of the project should be recruited from outside the institution 1 Future projects ed to include solar energy as electricity bills are huge 1 Very important need to train the SARI board and improve its role in projects 2 Need to help SARI develop better sustainability plans 1 They need to learn to write proposals to help governance 1 Lobbying of government to support research 1 Consider hiring students from UDS to study the impact research SARI activities for activities like the RELC mechanism 1 Soil lab was neglected; it needs to be expanded and renovated 1 Source: CSIR-SARI Technical and Financial Support Project, Final Performance Evaluation, FGDs and KIIs. 151 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex IV.F.2. Summary Results of Other Stakeholder Groups FGDs/KIIs Frequency of Non-ranked Responses for Good and Bad Unexpected Outcomes # Questions Responses Frequency EQ 3 89 What have been some of the unexpected outcomes (both good and bad) of this project for SARI? Good unexpected outcomes Increased demand for and amount of quality seed thanks to PFJ, including collaboration with the inspections. 7 another positive impact is that this has created new opportunities for women in post-harvest processing of the seed before sale by the private sector and SARI 3 The AGRIC officers have had their capacity built to the point that they are now much better partners with SARI, including gender training 1 The extremely close collaboration with ATT. They listed ATT as a partner and later they became de-facto technical advisor 1 Some of the seed producers are starting to invest in irrigation so that they can grow over multiple seasons 1 Because of the irrigation, and shorter cycle seed, the area cultivated in the river basins is increasing 2 Youth are going into dry season production of pepper, onions and cabbage 1 This project pointed out that when there is capacity building you need to look at it across the board 1 catalyzed dramatic expansion of private seed producer companies in northern Ghana (both large and small) through training and technical support, improving seed quality and standards 1 Farmer income has gone up a lot. Previously, it was not like that. They used to not have good seed 1 Market demand for the crops has gone up because the appearance of the crops /produce has gone up; packaging has improved as well 1 When we designed this project, we did not anticipate the soybean innovation lab. This provide a huge layer of support 1 Another unexpected thing was the shift to the COE. That was not expected the “core of excellence.” The positive was that this was support and correct a situation 1 There was widespread support across all ag actors for SARI’s leadership in identifying a PPE (public private enterprise) seed processing initiative that could have secured the future of the institute. Had this happened it would have made SARI strengthen it internal systems 1 SARI started the development of a 4 hour radio program, AGRIC today that continues and is very positive 1 The SARI staff have been very open to us; SARI’s support to field development has enhanced their movement and adoption of new technologies in the demo sites 2 Bad unexpected outcomes We all expected the seed lab to be refurbished; it was not 1 Some of the staff should have been trained more on seed technology; commercialization unit underutilized 3 The board had very little understanding of the project until the last minute when the project had big problems. Board did not know its capacity training was part of the project 1 There was confusion between the Centre for Excellence and the Core of excellence 1 152 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Responses Frequency Seed Yields are highly dependent on rain which affects profitability 1 Recent shifts in the contracting mechanisms and subsidies associated with the Planting for Food and Jobs Program have affected seed sales and the speed with which seed producers get paid by the government agents that purchase seed for this program 1 high price of capital negatively effected new field clearance which is critical to producing high quality foundation seed 1 high price of capital negatively affected private sector ability to invest in storage and processing that is needed to sustain the increased demand 8 There is clear evidence of impact on the direct beneficiaries that participated in the activities but it is not region wide 1 The procurement problems 1 We underestimated the critical importance of having solid informed communication about the project with different stakeholders, including over website 1 The project wasted resources by trying to change course on the project in the last 18 months 1 The rapid increase in demand and supply has exposed the weakness in SARI’s internal management systems especially those related to commercialization 1 The MoFA extension officers who are the frontline people dealing with the farmers need more intensive education on seed quality and the complementary production practices needed to intensify production to sustain the yield increases associated with additional land area and improved seed 1 The original project plan did not anticipate the need for cooking and tasting demonstrations for the new varieties 1 We expected them to scale up the demonstration trials and the field days. That did not happen 1 The negative was that the COE ==which was built in the system as a show piece—it did not succeed. It was expected to be a show piece 1 The PPE did not happen 1 Some of the seeds of the new varieties are in short supply 1 Since a high percentage of the agro-dealers are illiterate, they have not had the means to benefit as much from the project 1 Rapid unregulated production of certified seed price variance between groups. (new market constraint) 1 Lack of standard measurement has reduced the income impact of improved technologies (new market constraint) 1 Groundnut and soybean treatments is a challenge because the inoculum must be kept in cold chain so it is not readily available 1 109 Identify three best practices from your collaboration with this project that should be incorporated into future projects. (List) Strong collaboration between the seed certification division and the SARI 4 Working with private sector seed producers and their collaboration with the outgrowers 4 Demonstration trials were a best practice. Farmer fields 5 Continue to involve women & youth but scale it up 5 Agric processing was supported and should be scaled up 1 AA training (agent training) needs to be continued and scaled up 6 The emphasis on infrastructure was good. They established a lot of very good infrastructure 2 153 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Responses Frequency The area of communication became very positive. Online publication of their newsletters. ICT improvement and scale up including pamphlets and radio 7 “They responded to ATT’s leadership”; “ATT tried to walk them through the USAID hoops” 1 Building the capacity of the staff is an excellent idea 2 The development of improved seed is very vital to the economy of the country and the wellbeing of the farmers. This needs to be scaled up 6 Future projects need to better support the management structure that is needed to fully support these activities 2 Increase their production of inoculants in sufficient amounts to satisfy the existing demand for inoculated seed for soybean and groundnuts (and even cowpea) in the entire catchment areas 1 Wholistic approach to capacity building 2 They are very accessible. When we call them they pick up the phone 1 Still continue with the use of the inoculant which improves soil fertility 1 ISSUES THAT WERE NOT ADDRESSED Government stakeholders need more awareness of the project and the level of stakeholder engagement on TV, radio and through meetings 1 There was very little collaboration between MoFA and the project 1 They changed from the original plan which created challenges 1 Never executed the management structures they were supposed 1 they had slow burn rate and had to give back money 1 The main recommendation is that we want research on animals and trees crops like guava, baobab, and apple –across all areas of crops and We would like to see more research on vegetables 2 One key recommendation is to strengthen research on soil 2 Scale up a group of activities to help build the capacity of women to develop seed production and other VC activities 2 More support for economic research to identify critical policy issues that affect farmer’s ability to buy seed 1 Consider using USAID loan guarantees to encourage banks to lend money to commercial seed providers to develop cold storage and seed processing plants 3 Need to build the capacity of the regulators to understand the technical issues related to the new varieties 1 More flexibility between budgets 1 Independent project manager; bad management 2 More involvement ad training of the management board 2 SARI needs to strengthen its internal systems for managing the funds generated by its fee based services including seed sales 1 The commercial unit needs expand it capacity to engage with end users identification of their needs and responding to those needs 2 One line short courses for AEAs and for the researchers, including printed materials, audiovisuals, and certificates 1 Support the development of action oriented training videos in local languages and circulating these messages through movie vans/tricycles that can access remote areas 1 110 Name the principle lessons learned from what worked and did not work in this project for USAID to consider when designing similar what DID WORK 154 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Responses Frequency programs in the future. (List) Communication with local government ( and all) stakeholders is critical to sustaining the activities. This includes ensuring that the reports are circulated to them. They need to expand the scale 2 Need to scale up their support of the market linkages component 1 SARI needs to improve on its foundation seed support, including short duration climate smart technologies 3 Organic fertilizer. SARI needs to produce more organic fertilizer and more available 1 SARI plays critical role in any current or future attempt to develops cereals and legumes in the north 1 Board oversight—and the materials needed to ensure oversight—is critical to transparent project execution 1 Steering committees should be required of all SARI projects 1 The project helped SARI improve its internet connection by accessing the GARNET system and develop its first fully functional website 1 we need to further strengthen the public and private sector working together 1 SARI collaboration allows MoFA AEAs to work better with farmers on maize, rice, and climate and soil conditions (MoFA capacity building) 5 They are able to inform and educate farmers on the use of inoculum but due to cold chain, input dealers cannot sell the inoculum to farmers 1 Demonstration trials are absolutely critical to the core mandate of both MOFA and SARI 1 what DID NOT WORK cold storage and processing: As SARI’s capacity to produce seed increased, it quickly outstripped the capacity for processing and cold storage 1 there needs to be a gender expert/coordinator for the institute 1 Steering committee including all partners is important 1 Building the financial system needed to capture and re-invest revenue from seed sales if the institutions like SARI are to continue to support these activities after the project end 1 SARI does not have the in-house technical expertise to continue to update the aesthetics of the site or to make it more user friendly 1 SARI never developed an institution-level knowledge management system 1 Although the internet access improved at the main campus, the main campus was never rewired to make it user friendly and none of the field stations got access 1 more supportive of SARI and supporting the MOFA to scale up the demonstrations 1 Due to cold chain, input dealers cannot sell the inoculum to farmers 1 Older varieties need to be re-examined by the labs to make sure that we are keeping the genetic qualities of these seeds to meet modern needs, not just focus on new variety release 1 Source: CSIR-SARI Technical and Financial Support Project, Final Performance Evaluation, FGDs and KIIs. 155 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex IV.F.3. Summary Results of Community-Based Associations and Private-sector Partners FGDs/KIIs Frequency of Non-ranked Responses to Interview Questions # Questions Responses Frequency EQ 3 92 Working with the CSIR-SARI Capacity and Technical Support Project created new opportunities and unexpected outcomes. (yes/no) Youth: Most of the youth find that farming is more profitable. They find they can make money 1 Cowpea technology: Cowpea is important. That is a cash crop. Cowpea is making more money for the farmers 1 Dry season has become more profitable. Dry season production has gone up a lot. 1 Women are getting more empowered: They are better connected to SARI. They are able to better express themselves. Women are feeling more confident about having their own fields; they are improving their livelihood 1 EQ 5 111 What activities worked well with this project that you would like to see in future projects? It enabled us with technology and logistical support to engage with MoFA to reach the farmers…and to scale up our demonstration trials. We needed the infrastructure and the equipment. 1 112 What activities did not work well with this project that could be addressed in future projects? One of the challenges is they did not have enough equipment. They need tractors. Some of the research equipment. Some instruments. We need a lot of services 1 When we do our experiments. We need fences. For the office for the demonstration trials the animals destroy them. They need vehicles 1 Source: CSIR-SARI Technical and Financial Support Project, Final Performance Evaluation, FGDs and KIIs. 156 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex IV.F.4. Summary Results of Media Actor FGDs/KIIs Frequency of Non-ranked Responses to Interview Questions # Questions Responses Frequency EQ 3 94 Have these new opportunities created any unexpected community-level outcomes? (yes/no) Many of the local radio stations are now imitating your programs 1 EQ 5 115 Best Practice: Do you have any examples of media outreach from this project that should be incorporated into the design of future USAID-funded projects that work with SARI? (yes/no) Do more drama on the themes (for radio and tv) 1 Give the journalists more opportunity to conduct interviews on their own 1 Facilitate transportation to the campus so that the journalists can stay informed 1 Transmit in local languages 2 Live broadcasts of farmers using the new technologies with cal ins 1 Inviting and supporting media coming to various events like the launches 1 Radio net 1 Conscious policy for gender targeting 1 The CT center is a very good idea. Future projects need to continue to update equipment especially for web studio 1 The manuals good idea; need to scale them up and put them in local languages 1 116 Are there any lessons learned regarding what worked and didn’t work in terms of media collaboration under this project that USAID should consider for future projects? (yes/no) it is important to have a media consultant that follows at every step. This would make sure that they were constantly involved in media engagement 2 157 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV # Questions Responses Frequency include him/her in the design of the proposal so that you can capture what is happening 1 More engagement is needed. Especially with women 1 Consider creating a WhatsApp account so that the communications coordinator can communicate directly about new developments 2 It is important to find like minded media houses to ensure that the farmers are getting good information. We have other entities. All the government agencies have a common media platform 1 Journalist training 1 Require new projects to support ICT in conjunction with their projects 1 Movie van outreach 1 Anticipate the need for press briefing materials 1 Source: CSIR-SARI Technical and Financial Support Project, Final Performance Evaluation, FGDs and KIIs. 158 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex IV.G. Major Recommendations and Sub-recommendations Identified in Interviews For Future USAID/Ghana and CSIR-SARI Programs Recommendation and Sub-Recommendation for Future USIAID/Ghana-funded Projects and CSIR-SARI USAID/ Ghana CSIR-SARI Mentions195 Gender and Youth Integration: Strengthen USAID/Ghana’s and CSIR-SARI’s capacity to: 1) incorporate gender into initial project designs, 2) mainstream the activities into their institutional culture, and 3) monitor these activities in ways that feed into needs-based strategies to encourage gender integration and the recruitment and retention of female staff. 40 Design: USAID/Ghana should consider new guidance for G2G projects: 1) require new project designs to include gender and youth mainstreaming as a project sub-purpose/sub-IR in the results framework; and 2) identify some of the most important documents on the USAID websites programs can use to help incorporate gender into the initial design of their gender integration plans (including issues related to staffing, budget, governance, and M&E) (nine FGDs/KIIs). Yes N/A (9) Mainstreaming: CSIR-SARI should consider: 1) having a gender expert and gender focal point at all outstations; 2) linking its gender integration plans to CSIR-SAR’s established strategic planning processes; 3) pilot-testing the development of a multi￾donor working group to support a more harmonized approach to gender within the institution; and 4) working with senior management at each institution to develop a set of agency￾wide indicators generic enough to comply with gender requirements of the principal international donors that support the institutions as well as systems and checklists for ensuring this information is used to develop needs-based gender strategies (17 FGDs/KIIs). Yes (17) Staff and Community-based Leader Recruitment and Retention: CSIR-SARI should consider ways new projects can strengthen the beneficiary institutions’ recruitment and retention of female staff and community-based leaders using management training, grants, and hiring targets196 (14 FGDs/KIIs). N/A Yes (14) Project Management 85 General Management Systems: Ensure USAID/Ghana’s new G2G project designs include realistic management plans, mentorship, start-up/launch activities, and supervision. Start-up and Launch: 1) Make receipt of funds from USAID/Ghana conditional on having a robust management and steering committee structure in place by the fourth month of the project (13 FGDs/KIIs). Yes N/A (18) 195 Mentions refers t the number of times a recommendation or sub-recommendation was mentioned in the 50 FGDs/KIIs. 196 Five clusters of sub-recommendations that emerged from the interviews included: 1) offer grants to qualified individual women and youth to scale up new technologies; 2): build the capacity and promotion of competent female leaders in the farmers’ associations and producer groups that participate in adaptive on-farm trials and farmer fields days that MoFA and CSIR-SARI support; 3) support female university students doing internships and theses connected with CSIR-SARI’s technical research; 4) support both long- and short-term management training for femalee staff and technicians; and 5) help CSIR-SARI better integrate gender mainstreaming (including setting targets for hiring and retention of female staff) into its new five-year strategic plan (2021-2025). 159 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Recommendation and Sub-Recommendation for Future USIAID/Ghana-funded Projects and CSIR-SARI USAID/ Ghana CSIR-SARI Mentions195 2) Ensure all new G2G projects have a formal launch in the first year so staff and key stakeholders have a good understanding of the project, its implementation plan, and USAID/Ghana’s rules and regulations for M&E, procurement, and reporting (five FGDs/KIIs). Mentorship: 1) Hire a manager with a strong management background belonging to a reputable institute with demonstrated experience in successful capacity-building of national agricultural research institutes to ensure appropriate mentoring. 2) Use that person to monitor/coach the CSIR-SARI staff person who takes over the role once the recruited manager’s tenure expires (13 FGDs/KIIs) 3) Ensure: 1) both project managers have full-time salaries or receive top-offs and 2) all projects that commit substantial funding to CSIR-SARI (e.g., more than 15 percent of the institute’s budget) have a seat on the institute’s IMC (13 FGDs/KIIs). Yes Yes (17) 4) Require any USAID-funded projects expected to mentor or collaborate with CSIR-SARI to sign a memorandum of understanding (MOU) clarifying the type of mentoring support to be provided and how the mentorship activities will be co￾monitored and reported to USAID/Ghana (four FGDs/KIIs). Yes Yes (4) Governance: To ensure transparency, USAID/Ghana and CSIR￾SARI should ensure all future projects include a steering committee that includes a representative from the donor (13 FGDs/KIIs). Yes Yes (13) USAID Oversight and Evaluation: 1) Anticipate the need for the USAID Agreement Officer’s Representative (AOR) to make frequent supervisory missions, especially during the first year (three FGDs/KIIs). 2) Require future G2G and CSIR-SARI projects include a detailed description of, and designated budget for, a mid-term evaluation to address any management, implementation, or M&E issues that emerge in the first two years (four FGDs/KIIs). Yes Yes (7) Financial Management Systems: Anticipate the need to continue to build CSIR-SARI’s capacity to apply for and manage donor￾funded grants, as well as any commercial service activities it is expected to provide from the start in order to sustain and scale-up project achievements. Management Board: Strengthen the audit function of the CSIR￾SARI management board (eight FGDs/KIIs). N/A Yes (8) Project Personnel: 1) Acknowledge the time, level of effort, and contributions of individuals involved in new and future donor-funded projects on their annual CSIR-SARI evaluations. 2) If staff positions are not eligible for top-offs, consider what other compensations these individuals might be scheduled to receive (e.g., training, publication support, or sabbatical opportunities) (four FGDs/KIIs). Yes Yes (4) Accounting Systems: N/A Yes (11) 160 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Recommendation and Sub-Recommendation for Future USIAID/Ghana-funded Projects and CSIR-SARI USAID/ Ghana CSIR-SARI Mentions195 1) Continue to support CSIR-SARI’s efforts to tighten its accounting system by using appropriate accounting software and ensuring regular audits at all levels by an independent auditor (eight FGDs/KIIs). 2) Consider having a more flexible system for forwarding money to G2G projects than the monthly Imprest197 system used under this project (three FGDs/KIIs). Yes N/A (3) . Infrastructure and Equipment 27 Infrastructure: Encourage USAID/Ghana and other donors to support CSIR-SARI’s efforts to address some of its vital infrastructure and equipment needs that were not met under this project198 (21 FGDs/KIIs). Yes N/A (21) Infrastructure Management Plans: Link future funding for infrastructure to CSIR-SARI developing better systems for cost-sharing or generating user-fees to support the cost of routine maintenance and updating of the infrastructure and equipment (six FGDs/KIIs). N/A Yes (6) Technology Dissemination and Scale Up: Strengthen the existing and projected synergy between CSIR-SARI and its principal GoG collaborators and beneficiaries (such as PPRSD and MoFA), as well as other donor-funded partner projects in northern Ghana and other Savannah areas of West Africa. 38 MOUs: Encourage new USAID-funded projects and innovation labs in northern Ghana to: 1) identify, formalize through MOUs, monitor, and report on their collaboration with CSIR￾SARI in order to avoid duplication and strengthen synergies; and 2) consider co-funding and co-executing joint technical and mentoring activities on issues like pest monitoring, soil mapping, training, and extension (seven FGDs/KIIs). Yes Yes (7) Collaboration: 1) Ensure regional MoFA and PPRSD staff get regular debriefings on CSIR-SARI’s current research endeavors and receive all annual reports and work plans connected with any joint programs they support (three FGDs/KIIs). 2) Pilot test the feasibility of offering a limited number of fee￾based training programs for key programs partners (PPRSD, MoFA, NGOs, and donor-funded projects) (seven FGDs/KIIs). Yes Yes (31) 197 USAID/Ghana used the Imprest system to advance money to the project, which was then responsible for accounting for it. The problem with system was CSIR-SARI could not account for the money on time or follow USAID/Ghana regulations, which created delays in the release of project funds. 198 Staff identified a number of pressing needs, including most of the infrastructure and equipment the project envisioned for the field stations, better access to irrigation (which is critical to additional increases in seed production), a geographic information system lab, a seed lab, a fully functioning biotech lab, a larger soils lab, a better systems for linking the main soil labs to the field stations, fencing to protect the research areas, and more capacity for cold storage and solar power. 161 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Recommendation and Sub-Recommendation for Future USIAID/Ghana-funded Projects and CSIR-SARI USAID/ Ghana CSIR-SARI Mentions195 3) Identify ways other USAID-funded projects could help scale up CSIR-SARI’s successful collaboration with MoFA for AEA training, field trials, and field days (21 FGDs/KIIs).199 ICT 27 Internet: Ensure future USAID/Ghana and CSIR-SARI projects support capacity-building to strengthen the beneficiary institution’s internet connectivity and internet-based ICT services (i.e., networking infrastructure, intranet services, website, online library resources, database security). M&E and New Project Designs: 1) Require any baseline assessment that feeds into the design of a new project to measure the baseline connectivity of the system as well as other factors that affect the institution’s ability to maintain, repair, and keep the system user-friendly. 2) Track any strategy developed to address any internet-based issues identified through one or more standard indicators in the IPTT (three FGDs/KIIs).200201 Yes Yes (3) Internet Mentorship: Explore mentoring from local internet cooperatives and centers of excellence (e.g., GARNET, SIL’s ICT Connectivity Project, and/ or the computer scientists in the CSIR Electronics Unit) that can lower the initial costs of installation and maintenance and help G2G partners avoid costly mistakes that are hard to fix (seven FGDs/KIIs). Yes Yes (7) CSIR-SARI Internet: USAID/Ghana should conduct a second baseline diagnostic to assess the cost to: 1) extend CSIR-SARI’s internet service to the field stations; 2) make the system (including online library and website) more user-friendly; 3) lower the routine operating costs; and 4) sub-contract some basic maintenance, website, and database updates (13 FGDs/KIIs).202 Yes Yes (13) 199 Stakeholder suggestions included: 1) making training more practical (action oriented); 2) giving CSIR-SARI certificates to the AEA trainees (if relevant) after the training; 3) supporting the development of written modules in English that AEAs can use to cascade the training to other groups; 4) publicizing (on radio, television, and social media) AEA graduations to increase policy makers, government, and private-sector awareness of this training and the technologies being promoted; 5) strengthening CSIR￾SARI’s follow-up communication with MoFA agents by developing a stakeholder database on the agents and groups they train; 6) encouraging community-to-community visits to learn from farmers participating in the CSIR-SARI-MoFA on-farm trials; 7) adding other crops based on the food security needs of the different regions, including cowpea, groundnut, and sorghum; 8) contributing both financially and in-kind support to the district and regional-level RELC mechanisms led by MoFA; and 9) collaborating with MoFA AEAs on getting more structured real-time co-reporting on how new technologies are performing in beneficiary communities (Suggestions1-5 discussed in eight FGDs/KIIs; Suggestion 6 in two FGDs/KIIs; Suggestion 7 in four FGDs/KIIs; Suggestion 8 in three FGDs/KIIs; and Suggestion 9 in four FGDs/KIIs). 200 In conjunction with the baseline assessment of the CSIR-SARI system, the SIL consultant pilot tested the use of a four￾indicator assessment tool in the initial diagnostic that has since been refined and scaled up in other countries where the SIL intervenes (see: https://ajfand.net/Volume20/No5/Goldsmith19385.pdf). 201 In conjunction with the baseline assessment of the CSIR-SARI system, the SIL consultant pilot tested the use of a four￾indicator assessment tool in the initial diagnostic that has since been refined and scaled up in other countries where the SIL intervenes (see: https://ajfand.net/Volume20/No5/Goldsmith19385.pdf). 202 A series of sub-recommendations for extending and lowering the operating costs of the current systems have been made by the SIL technical support unit and CSIR-SARI computer experts, including: 1) developing a fiber optic connection from Nyankpala to the main station and field stations; and 2) pulling together a consortium of donors to support a system-wide 162 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Recommendation and Sub-Recommendation for Future USIAID/Ghana-funded Projects and CSIR-SARI USAID/ Ghana CSIR-SARI Mentions195 Diversify ICT Funding and Oversight: 1) Build CSIR-SARI’s capacity to negotiate its existing and future partners contributing to the routine costs of the internet, website, and documentation/database library. 2) Organize a CSIR-SARI donor working group, whose initial focus should be on resolving the issues related to the internet.(four FGDs/KIIs).203 Yes Yes (4) ICT Outreach: Ensure future USAID/Ghana G2G and CSIR-SARI ICT initiatives are based on a critical analysis of lessons learned from earlier programs supported by CSIR-SARI and its partner projects. ICT Documentation and M&E: 1) Develop of a central online repository for all CSIR-SARI’s ICT materials over the last ten years so they can be easily scaled up to new and existing projects. 2) Develop a gender-sensitive assessment tools for tracking the impact of different outreach methodologies (e.g., portable video communication tools and community-based theater) (seven FGDs/KIIs).204 Yes Yes (7) Advocacy and Communication: 1) Strengthen the capacity of CSIR-SARI’s M&E and ICT units to backstop its scientists in dealing with the media, publishing their own data and project M&E data. 2) Strengthen CSIR-SARI’s capacity to use M&E and research results for advocacy at the national level. 3) Consider ways USAID-funded Food Security Innovation Lab could backstop these research, M&E and advocacy activities (four FGDs/KIIs). Yes Yes (4) Commercialization: Given the critical importance of CSIR-SARI’s commercialization activities in supplementing its core GoG funding, encourage CSIR-SARI to seek donor assistance in strengthening its b: 1) internal commercialization systems; and 2) marketing strategies with the existing and emerging commercial seed and other markets. 61 Demand-driven Research and Support: Encourage international donors (like USAID/Ghana) that are active in northern Ghana to build the capacity of CSIR-SARI’s field stations to better Yes Yes (11) connection for all 13 CSIR institutions which would eliminate most of the recurrent costs of the system (see Annex V.B.3 for a discussion of the SIL’s reflection on this issue). 203 This internet oversight committee could meet remotely and include the most relevant CSIR-SARI staff as well as the internet specialist associated with different projects like SIL or the technical support staff of different CGIAR groups like International Institute of Tropical Agriculture. 204 The detailed impact analysis for its ICT approaches ATT conducted—including smartphone apps effective in demonstrating new technologies to AEAs and farmers—is a good example of this type of assessment. A number of other examples of best practice in CSIR-SARI outreach were pilot-tested and scaled up under some of the earlier AGRA projects, but the current ICT director had no documentation on these earlier initiatives since he was not in the unit at the time. 204 Various stakeholders suggested this applied research is needed to identify the key factors that affect famers and the large industrial companies that purchase seeds and producers’ willingness and ability to purchase improved seed. 163 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Recommendation and Sub-Recommendation for Future USIAID/Ghana-funded Projects and CSIR-SARI USAID/ Ghana CSIR-SARI Mentions195 support demand-driven research and laboratory support (11 FGDs/KIIs). Private-sector Commercialization Infrastructure: CSIR-SARI and MoFA should consider identifying: 1) areas where there is the need for more private-sector seed infrastructure (e.g., laboratories, seed processing facilities) in northern Ghana (four FGDs/KIIs); and 2) use this information to leverage donors’ interest in offering loan guarantees to address these gaps (13 FGDs/KIIs). Yes Yes (13) Oversight: Improve internal oversight and support of CSIR￾SARI’s commercialization unit under the direct supervision of the CSIR-SARI director and management board (12 FGDs/KIIs).205 N/A Yes (12) M&E and Budgets: Future CSIR-SARI capacity-building projects should include: 1) a line budget for the commercialization unit; and 2) robust indicators management can use to track the unit’s progress (nine FGDs/KIIs).206 N/A Yes (9) Code of Conduct: Facilitate the development of a CSIR-SARI professional code of conduct that clarifies CSIR-SARI’s support for plant breeders’ rights and any restrictions on research scientists and technicians setting up unrestricted side business to sell seed or other products (four FGDs/KIIs). N/A Yes (4) M&E: Future USAID/Ghana support for CSIR-SARI should assist the institution’s current efforts to create an agency-wide M&E system and comply with standard international donor standards of best practice for M&E. 20 IPTT: Require all new USAID/Ghana G2G and CSIR-SARI projects to have: 1) an IPTT that includes well-defined indicators (with PIRS) for all major outcomes and output indicators that monitor projects’ impacts on capacity as well as outreach to MoFA, RELCs, and other donor-funded projects; and 2) a dedicated M&E budget that must be returned to the donor if it is not spent and that has no possibility of being converted into funds that support other activities (eight FGDs/KIIs). Yes Yes (8) M&E Unit Set-up. Future USAID/Ghana G2G and CSIR-SARI projects need to: 1) designate one M&E coordinator and one M&E focal point for each field station; and 2) ensure these individuals have consistent mentoring by a professional M&E specialist or contractors (11 FGDs/KIIs). Yes Yes (11) M&E Mainstreaming: Future USAID/Ghana G2G and CSIR-SARI Projects need to: 1) provide appropriate baseline training to all senior and junior staff and administrators; and 2) utilize online Yes Yes (4) 205 Various stakeholders suggested this applied research is needed to identify the key factors that affect famers and the large industrial companies that purchase seeds and producers’ willingness and ability to purchase improved seed. 206 The stakeholders emphasized this information is needed for: 1) tracking the progress of CSIR-SARI as an institution in responding to the execution of its internal targets for becoming more market and client-oriented; 2) building the core internal capacities and systems (including accounting) needed for the commercial unit to be effective; and 3) linking future budgets for athe commercial unit to its impacts on the ground. 164 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Recommendation and Sub-Recommendation for Future USIAID/Ghana-funded Projects and CSIR-SARI USAID/ Ghana CSIR-SARI Mentions195 short courses and in-house certification programs to help new and existing staff improve their proficiency (four FGDs/KIIs). Source: CSIR-SARI Technical and Financial Support Project. Final Performance Evaluation July 2021 based on interviews. 165 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV ANNEX V. BIBLIOGRAPHY, ADDITIONAL TABLES, AND CASE STUDIES Annex V.A. CSIR-SARI Technical and Financial Support Project Final Evaluation Bibliography A. Project Proposal and COE Assessments Supported by ATT and the Project A.1. Proposal and Implementation Letter USAID/Ghana. Implementation Letter (unsigned). Date on file name is November 14, 2014. Signed version in the file is January 14, 2015. USAID/Ghana. 2014. Program Description. Support to CSIR-SARI (April 2014-November 30, 2018 (Final version dated February 14, 2014). USAID/Ghana. 2014. Program Description. Support to CSIR-SARI (January 14, 2015 to January 14, 2020). Accra: EG Office, USAID/Ghana. (Final version dated November 2, 2013). A.2. COE Assessments and Strategies (April 2016-December 2019) A.2.1. COE Assessments and Proposed Project Reorganization (supported by ATT) A.2.1.1. General Capacity Building and Mentorship Michael Dockery. 2016 (April). Email from Stephen Nutsugah to Nicholas Denwar, Gloria Boakyewaa , Wilson Dogbe, and Gilbert Nachim. Subject “SARI Assessment Report—ISU” (Dr. Dileep Guntuku). Request for review.” Dileepkumar Guntuku. 2016 (March). Assessment of the Institutional Capacity and Needs of SARI. Nyankpala, Northern Region, Ghana. Ames, Iowa: IFDC and ISU. (Final version with corrections dated April 25, 2016). Robert Kwasi Owusu. 2016 (June). “Invitation to core of Excellence Planning Meeting.” Nyankpala: CSIR￾SARI and ATT. Invitation to a joint meeting with ATT on June 15, 2016 review the results of the needs assessment. CSIR-SARI. 2016 (June 15). Planning Meeting of “COE at SARI.” Program. Speakers: Stephen Nutsugah, Samson Konlan, Michael Dockery, Dileep Kumar Guntuku, David Benson, Lulu Rodriguez. Not final (draft). CSIR-SARI Direct Support Project. 2017 (estimated). Implementation Arrangements (for the COE). One page document. Dileepkumar Guntuku. 2017. COE at SARI. Status Report 2013 (PowerPoint) with inputs from ISU, ATT, USAID, and SARI teams. Letter from Stephen Nutsugh to Dr. Guntuku confirming CSIR-SARI faculty participation in COE training from March 15-28, 2017. SARI COE Records. CSIR-SARI. 2017. List of participants the Proposed SARI trainings at ISU and UIUC. Nyankpala: CSIR-SARI. CSIR-SARI. 2017. Training Program to Enhance Capacities of SARI Staff. Nyankpala: CSIR-SARI. (Note a signed note from the director confirmed the transfer of this along with the budget to ISU in February 2017). Dileepkumar Guntuku. 2017. Training Program for COE Leaders of SARI. March 15, 28, 2017. Hosted by the Seed Science Center, College of Agriculture and Life Sciences. ISU. Training Calendar. 166 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Dileepkumar Guntuku. 2017 (March). COE at SARI . Status report. With inputs from ISU and SARI COE Team Leaders. Ames: ISU. Dileepkumar Guntku. April 19 2017. Email to Nutsugah, Dogbe, Denwar, Boakwaa, Kutah, Appiah at SARI and Lin Misra at ISU and Samson Konlan at USAID). Subject: (1) Meeting with Samson Konlan during his visit on ISU; and (2) proposed course of action. A.2.1.2. Development of Training Offer in Seed Research USAID/Ghana Agricultural Technology Transfer Project (ATT). 2016. Lead: IFDC. Iowa State University (Subcontractor. Scope of Work. Proposed travel to Ghana: October 8-15, 2016. David L. Benson, Ph.D. (Iowa State University Consultant). Nyankpala: ATT Project for USAID/Ghana. David Benson. 2016 (June). SARI Core of Excellence Breeding Program Critical Needs Assessment. Nyankpala: ATT for CSIR-SARI Technical and Financial Support Project and USAID/Ghana). A.2.2. COE ISU Contract (July 2017-December 2020. Wilson Dogbe. April 2017 ad July 2017. Review of Implementation of COE project. PowerPoint. Nyankpala: CSIR-SARI Technical and Financial Support Project Archive. A.2.3. AgTILs Contracts (March 2019-December 2019) AgTILs Budget draft for review (March 2019-December 2019). Ames, Iowa: AGTILs.207 AgTILs. 2019. First Quarter Report (Marxy 2019). Activities performed by AgTILs as per the contractual agreement between CSIR-SARI and AgTILs. Ames, Iowa: AgTILs for the CSIR-SARI Technical and Financial Support Project and USAID/Ghana. AgTILs. 2019. Fourth Quarter Report (April-June 2019). Activities performed by AgTILs as per the contractual agreement between CSIR-SARI and AgTILs. Ames, Iowa: AgTILs. for the CSIR-SARI Technical and Financial Support Project and USAID/Ghana. AgTILs. 2019. Third Quarter Report (July-Sept 2019(.). Activities performed by AgTILs as per the contractual agreement between CSIR-SARI and AgTILs. Ames, Iowa: AgTILs. for the CSIR-SARI Technical and Financial Support Project and USAID/Ghana. AgTILs. 2019. Fourth Quarter Report (October-December 2019). Activities performed by AgTILs as per the contractual agreement between CSIR-SARI and AgTILs. Ames, Iowa: AgTILs. for the CSIR-SARI Technical and Financial Support Project and USAID/Ghana. A.2.4. Study-Visits of SARI Staff in Preparation for COE Reorganization Funded by the Project and USAID Debriefings AgTILs. Budget draft for review (March 2019-December 2019). Nyankpala: CSIR-SARI Technical and Financial Support Project Archives.208 207 Note: This budget for 3.5 million US dollars was 0.5 million over the $3.00 that the project had at the time of developing the budget. They anticipated making up the shortfall by leveraging funds from other projects. 208 Revised budget prepared by ISU consultants that includes support for the training visits and other activities associated with the PPE. This budget for 3.5 million US dollars was 0.5 million over the $3.000,000 that the project had at the time of developing the budget. They anticipated making up the shortfall by leveraging funds from other projects. 167 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV CSIR-SARI Technical and Financial Support Project. 2019 (April 4). Brief on a courtesy call of Ghanaian Delegation to India on his Excellency the Ghana High Commissioner to India on the 4th of April 2019. Nyankpala: CSIR-SARI Technical and Financial Support Project Archives. Wilson Dogbe. 2019 (April 9). Notes on the debriefing meeting at USAID office on the 9th of April 2019. Memo. Present at the meeting were, The Mission Director, Jenna Tachman, Grace Odoom, Grace Sebugah, Victor Agyeman, (DG of CSIR), Wilson Dogbe (COE Project Manager) and one other staff from USAID. Nyankpala: CSIR-SARI Technical and Financial Support Project Archives. Grace Sebugah. 2019 (May 7). Notes from briefing by Madam Grace (USAID-Activity Manager) after CSIR DG and the Honorable Minister of MESTI met USAID team and the Deputy Ambassador to the US. Discussion: A joint venture between CSIR -SARI and Prasad Seeds. May 7, 2019. Nyankpala: CSIR￾SARI Technical and Financial Support Project Archives. CSIR-SARI Technical and Financial Support Project. 2019 (May)209 De-briefing Paper. Seed Research and Innovation Center at the Council of Scientific and Industrial Research-Savanna Agricultural Research Institute (CSIR-SARI) Nyankpala: CSIR-SARI Technical and Financial Support Project Archives. A.3. Feasibility Study and Debriefings for the PPP A.3.1. Feasibility Studies AgTILs. 2019. Executive Summary of Techno-Feasibility Report for Establishing a Seed Research and Conditioning Center in Ghana Under Public Private Partnership Arrangement. Ames, Iowa: AgTILs for the CSIR-SARI Technical and Financial Support Project and USAID/Ghana (July). AgTILs. 2019. Executive Summary of Techno-Feasibility Report for Establishing a Seed Research and Conditioning Center in Ghana Under Public Private Partnership Arrangement, Ames, Iowa: AgTILs for the CSIR-SARI Technical and Financial Support Project and USAID/Ghana (July). A.3.2. Stakeholder Meetings to Discuss Feasibility Study CSIR-SARI Technical and Financial Support Project. 2019 (October) Recommendations from the USAID COE Component Leaders and CSIR-SARI Divisional Heads Meeting Held in the Seminar Room on October 1, 2019 on the Matter of the Partnership between CSIR-SARI and Prasad Seeds Limited. Internal memo. CSIR-SARI Technical and Financial Support Project. 2019 [November & December].210 [Seed Sector Stakeholders Meeting]211 Meeting Reports. Seed Research and Conditioning Center in Ghana Anchored on Public Partnership. Nyankpala: CSIR-SARI Technical and Financial Support Project for USAID/Ghana. Wilson Dogbe. 2019. Key Project Achievements –2015-2019. (short word document addressed to Grace Sebugah) Nyankpala: CSIR-SARI Technical and Financial Support Project for USAID/Ghana. B. Annual and Quarterly Reports B.1. Annual Reports 209 Date estimated; not on cover. 210 Date was not on cover. It appears to have been organized in early November. 211 This meeting brought together the CSIR-SARI director and one senior technologist, the project consultant who developed the feasibility study, the project manager and the USAID/Ghana AOR with 11 leaders from the major government (e.g. PPRSD), multi-lateral (e.g. AGRA) and private sector seed (e.g. NASTAG) actors in seed sector actor to review the SRCG feasibility study (16 people total attended the meeting). 168 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV CSIR-SARI Technical and Financial Support Project. 2016. Annual Report (October 2015 to September 2016). Implementation Letter No. 641-A18-FY14-IL #03. Nyankpala: CSIR-SARI Technical and Financial Support Project for USAID/Ghana. CSIR-SARI Technical and Financial Support Project. 2016. Annual Report (October 2016 to September 2017). Implementation Letter No. 641-A18-FY14-IL #03. Nyankpala: CSIR-SARI Technical and Financial Support Project for USAID/Ghana. CSIR-SARI Technical and Financial Support Project. 2016. Annual Report (October 2017 to September 2018). Implementation Letter No. 641-A18-FY14-IL #03. Nyankpala: CSIR-SARI Technical and Financial Support Project for USAID/Ghana. CSIR-SARI Technical and Financial Support Project. 2016. Annual Report (October 2018 to September 20169. Implementation Letter No. 641-A18-FY14-IL #03. Nyankpala: CSIR-SARI Technical and Financial Support Project for USAID/Ghana. CSIR-SARI Technical and Financial Support Project. 2020. Close-out Report (2015-2020). Nyankpala: CSIR Technical and Financial Support Project for USAID. B.2. Quarterly Reports B.2.1. For the Project FY 2015 (Q1, Q2) FY 2016 (Q3) FY 2017 (Q1, Q2) FY 2018 (Q3) FY 2019 (Q1, Q2, Q3; for Q4 report and FY 2020 Q1 reports, see AgTILs reports below.) B.2.2. For the AgTILs Contracts (March 2019-December 2019)212 AgTILs. 2019. First Quarter Report (Mary 2019). Activities performed by AgTILs as per the contractual agreement between CSIR-SARI and AgTILs. Ames, Iowa: AgTILs for the CSIR-SARI Technical and Financial Support project and USAID/Ghana. AgTILs. 2019. Fourth Quarter Report (April-June 2019). Activities performed by AgTILs as per the contractual agreement between CSIR-SARI and AgTILs. Ames, Iowa: AgTILs. for the CSIR-SARI Technical and Financial Support project and USAID/Ghana. AgTILs. 2019. Third Quarter Report (July-Sept 2019(.). Activities performed by AgTILs as per the contractual agreement between CSIR-SARI and AgTILs. Ames, Iowa: AgTILs. for the CSIR-SARI Technical and Financial Support project and USAID/Ghana. AgTILs. 2019. Fourth Quarter Report (October-December 2019). Activities performed by AgTILs as per the contractual agreement between CSIR-SARI and AgTILs. Ames, Iowa: AgTILs. for the CSIR-SARI Technical and Financial Support project and USAID/Ghana. C. CSIR-SARI M&E Unit C.1. Studies 212 These are repeated above as they are both reports and strategy statements. 169 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV CSIR-SARI Direct Support Project. M&E Team 2016. Status of Maize, Rice and Soybean Production Systems in Northern Ghana. Nyankpala: CSIR-SARI (October 1, 2016). CSIR-SARI Direct Support Project. M&E Team. 2019. Adoption of Agricultural Technologies in Northern Ghana. Nyankpala: CSIR-SARI (March 1, 2019). C.2. M&E Plans CSIR-SARI. 2013. Monitoring and Evaluation Plan. Nyankpala: CSIR-SARI (Note this one was reviewed by METSS II). USAID Direct Support to SARI. 2015. Monitoring and Evaluation Plan. Nyankpala: M&E plan for CSIR￾SARI (Note this one was reviewed by METSS II). CSIR-SARI. 2016. Performance Monitoring Plan: CSIR-SARI. Nyankpala: CSIR-SARI. (June 2016) (File is dated June 19, 2016) (Note this one was not reviewed by METSS II). USAID Direct Support to SARI. 2016.Activity Monitoring and Evaluation Plan (AMEP). Nyankpala: USAID-Director Support Project for SARI. (Submitted by Iddrisu Yahaya). (September 2016). (Note: This one was reviewed by METSS II). Samuel D. Braimah. 2017. The CSIR-SARI Strategic Plan M&E Framework. Nyankpala: The USAID Direct Support Project for CSIR-SARI (Note: This was not reviewed by METSS II). C.3. Feed the Future Summary Reports Prince Maxwell Etwire. 2020. Summary Reporting on LOE Achievements on the Feed the Future Standard Indicators for the USAID Direct Support Project to SARI. Nyankpala: CSIR-SARI Technical and Financial Support Project Archives. Prince Maxwell Etwire. Summary Reporting on the FTF Standard Indicators for the USAID Direct Support Project to SARI: 2016 and 2015. C.4. Trip Reports Abdul-Razak Mohammed with support from I. Yahaya, J.K. Bidzakin, N. Jinbaani, S. Adogba, E. Martey, W.N. Kutah, W. Dogbe, and P.M. Etwire. 2019. Activity 3.3.9 (in the work plan). Report on Establishing an Effective Commodity Value Chain (Innovation Platforms) in Savelugu District in Northern Region. February 708, 2019. Venue: SIMLI Center- Savelugu. Nyankpala:-CSIR-SARI Technical and Financial Support Project Archives. D. Project Governance Structures D.1. COE Advisory Committee Wilson Dogbe. 2017 (October 21, 2017). Minutes of the USAID-CSIR Direct Support COE Advisory Council Meeting. Ames, Iowa: ISU Seed Science Centre. Note: The meeting was deliberately scheduled to overlap with the World Food Prize Program . A second meeting was tentatively scheduled to take place in Nyankpala during the 3rd or 4th week of April 2017 but did not occur. (the project paid for the CSIR DG, the Director and the Project manager to attend). A second meeting organized on October 2018 in collaboration with the World Food Prize (the project paid for the CSIR DG, the Director and the Project manager to attend). 170 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV D.2. CSIR-SARI Management Committee Minutes213 No. 15. September 29, 2016. Minutes of Management Board Meeting. No. 15. Venue: SARI Seminar Room. (5 pages). No. 16. April 16, 2018. Inauguration of CSIR-Savanna agricultural Research Institute Management Board and Minutes of the 16th Board Meeting. Venue: SARI Seminar Room (13 pages). No. 17. September 3, 2018. Minutes of Management Board Meeting. No. 17. Venue: SARI Seminar Room. No.18. June 21, 2019. Minutes of Management Board Meeting. Number of Meeting:18. Venue: SARI Seminar Room. No. 19. February 20, 2020. Minutes of Management Board Meeting. Number of meeting: 19. CSIR Seminar Room. D.3. CSIR-SARI Internal Management Committee Minutes214 No. 30. May 19, 2015. Minutes of Internal Management Committee Meeting. Number 30. Venue SARI Seminar Room. (8 pages) No. 31. November 17, 2015. Minutes of Internal Management Committee Meeting. Number 31. Venue SARI Seminar Room. (10 pages) No 32. May 16, 2016. Minutes of Internal Management Committee Meeting. Number 32. Venue SARI Seminar Room. (10 pages). No. 33. August 15, 201. Minutes of Internal Management Committee Meeting. Number 33. Venue SARI Seminar Room. (8 pages). No. 34. December 5, 2016. Minutes of Internal Management Committee Meeting. Number 34. Venue SARI Seminar Room. (6 pages). No 37. August 15, 2017. Minutes of Internal Management Committee Meeting. Number 37. Venue SARI Seminar Room. (8 pages). No 40. May 22, 2018. Minutes of Internal Management Committee Meeting. Number 40. Venue SARI Seminar Room. (6 pages). No 43. May 22, 2019. Minutes of Internal Management Committee Meeting. Number 43. Venue SARI Seminar Room. (7 pages). No 45. November 19, 2019. Minutes of Internal Management Committee Meeting. Number 45. Venue SARI Seminar Room. (7 pages). E. Gender and Environmental Mitigation and Monitoring Plan Documents CSIR-SARI Technical and Financial Support Project. 2016. Environmental Mitigation and Monitoring Plan. Nyankpala: CSIR-SARI Technical and Financial Support Project. (January 30, 2016 final). 213 In conjunction with the evaluation, the ET requested and received a sample of the minutes in order to see how it was reporting on the project. 214 In conjunction with the evaluation, the ET requested and received a sample of the minutes in order to see how it was reporting on the project and other donor-funded initiatives. 171 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV CSIR-SARI Direct Support Project. 2019. Gender Roles in Agricultural Production Systems in Northern Ghana. Nyankpala: CSIR-SARI (January 2019) F. Technical Reports from Component Teams F.I. ICT215 Wilheim Kutah. 2017. Proposal SARI Communications Plan. March 25, 2017. Nyankpala: CSIR-SARI (draft) Wilhelm Kutah. 2016. SARI Research Monitor. June, 2016. Nyankpala: CSIR-SARI Wilhelm Kutah. 2016. SARI Research Monitor. September, 2016. Nyankpala: CSIR-SARI Wilhelm Kutah. 2016. SARI Research Monitor. December, 2016. Nyankpala: CSIR-SARI Wilhelm Kutah. 2017. SARI Research Monitor. March, 2017. Nyankpala: CSIR-SARI Wilhelm Kutah. 2017. SARI Research Monitor. September, 2017. Nyankpala: CSIR-SARI Wilhelm Kutah. 2017. SARI Research Monitor. December, 2017. Nyankpala: CSIR-SARI Wilhelm Kutah. 2018. SARI Research Monitor. March, 2018. Nyankpala: CSIR-SARI Wilhelm Kutah. 2018. SARI Research Monitor. July, 2018. Nyankpala: CSIR-SARI Wilhelm Kutah. 2018. SARI News Brief. March 19, 2018. Nyankpala: CSIR-SARI Wilhelm Kutah. 2018. SARI News Brief. February 20, 2018. Nyankpala: CSIR-SARI Wilhelm Kutah. 2018. Special Edition Newsletter. February, 2018. Nyankpala: CSIR-SARI Wilhelm Kutah, 2017, TLIII Newsletter. September, 2017. Nyankpala: CSIR-SARI Wilhelm Kutah, Kusi F, Attamah P, Agyare RY, Lamini S, Muktharu Z, Sugri I, Adjebeng-Danquah J, Owusu EY, Kubi-Tetteh G, Buah SSJ, Kanton RAL, Razak MA and Asante M. 2017. Cowpea: Field Establishment. September, 2017. Manga: CSIR-SARI216 Wilhelm Kutah, Kusi F, Attamah P, Agyare RY, Lamini S, Muktharu Z, Sugri I, Adjebeng-Danquah J, Owusu EY, Kubi-Tetteh G, Buah SSJ, Kanton RAL, Razak MA and Asante M. 2017. Cowpea: Harvesting and Post-Harvest Management. September, 2017. Manga: CSIR-SARI217 Wilhelm Kutah, Kusi F, Attamah P, Agyare RY, Lamini S, Muktharu Z, Sugri I, Adjebeng-Danquah J, Owusu EY, Kubi-Tetteh G, Buah SSJ, Kanton RAL, Razak MA and Asante M. 2017. Preparation for Profitable Cowpea Production. September, 2017. Manga: CSIR-SARI218 Wilhelm Kutah, Kusi F, Attamah P, Agyare RY, Lamini S, Muktharu Z, Sugri I, Adjebeng-Danquah J, Owusu EY, Kubi-Tetteh G, Buah SSJ, Kanton RAL, Razak MA and Asante M. 2017. Rougeing: Cowpea Seed Production. September, 2017. Manga: CSIR-SARI219 215 These documents have not yet been given to us for inclusion in the documentation base. These are products of the project; the project supported the capacity-building of Kutah and the printing of the documents as well. 216 Although cowpea was not a priority value chain, the project supported this activity. 217 Although cowpea was not a priority value chain, Kutah’s capacity-building was supported under this project. 218 Although cowpea was not a priority value chain, Kutah’s capacity-building was supported under this project. 219 Although cowpea was not a priority value chain, Kutah’s capacity-building was supported under this project. 172 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Wilhelm Kutah, M. Haruna, M. Abudulai, N.N. Denwar, A.M. Mohammed and A. B. Salifu. 2017. Farmers Guide to Successful Soybean Production in Ghana. 2017. Nyankpala: CSIR-SARI Wilhelm Kutah, R. Oteng-Frimpong, M.A. Rasheed, I.A. Rashid, N.N. Denwar and S.K. Nutsugah. 2017. Growing Groundnut Profitably in the Guinea Savanna of Ghana. August, 2017. Nyankpala: CSIR-SARI. 220 Wilhelm Kutah, Wilson Dogbe, Samuel Abebresse. 2016. Rice Scalable Technologies. September, 2016. Nyankpala: CSIR-SARI Wilhelm Kutah, Gloria B Adu, H Alidu, P Aboyadana, Ik Amegbor, A Mugis, E Munah. 2017. Newly Released Stress Tolerant Hybrid Maize Varieties in Ghana. May, 2017. Nyankpala: CSIR-SARI. Wilhelm Kutah, Gloria B Adu, H Alidu, IK Amegbor, Robert Owusu, SK Nutsugah, A Oppong, Manfred Bondzie Ewool, P Ribeiro, Frank D Coffie, Stella Ennin, Isaiah Baba. 2019. Hybrid Maize Seed Production: A resource and reference manual. 2019. Nyankpala: CSIR-SARI. Wilhelm Kutah, R. Oteng-Frimpong, M.A. Rasheed, A R Issah, N.N. Denwar, D K Puozaa. 2017. Improved Groundnut Varieties. May, 2017. Nyankpala: CSIR-SARI.221 Wilhelm Kutah, J Nboyine, F Kusi, G B Adu, H Alidu, M Abudulai. 2017. Fall Army Worm Outbreak in Ghana: Facts and Management Approaches. 2017. Nyankpala: CSIR-SARI. F.2. Other None received. G. Budgets and Equipment Register (Projections and Actual Expenditures) and Equipment /Infrastructure Register for the Project G.1. Budgets CSIR-SARI. 2017 (March 2017). USAID Direct Support Project. Budget Estimate—Six participants to the US (March 2017). Prepared/Verified by Sebastian Tigbee. CSIR-SARI. 2021. USAID Direct Support Project. Actual expenses for five years. Prepared/Verified by Sebastian Tigbee. CSIR-SARI. 2021. USAID Direct Support Project. Fund Accountability Statement. FY2015-FY2019. Prepared/Verified by Sebastian Tigbee. Month 2021. CSIR-SARI. 2017. SARI COE Agreement with ISU. July 1, 2017-September 2019. Nyankpala: CSIR-SARI. Draft.. CSIR-SARI. 2021. USAID Summarized Budget 2015-2019. Nyankpala: CSIR -SARI. Draft. AgTILs Budget draft for review 222(March 2019-December 2019). Note: This budget for 3.5 million US dollars was 0.5 million over the $3.00 that the project had at the time of developing the budget. They anticipated making up the shortfall by leveraging funds from other projects. G.2. Equipment Register for the Direct Support Project CSIR-SARI. USAID equipment Register, 2015-2019. H. Donor/Grant History of CSIR-SARI 220 Although cowpea was not a priority value chain, Kutah’s capacity-building was supported under this project. 221 Although cowpea was not a priority value chain, Kutah’s capacity-building was supported under this project to do it. 222 This budget is filed under the COE assessment as it was a proposal. 173 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV 2018 Inhouse accounts 2019 In house review GoG and Donor support Grants and partners In-house 2015 Inhouse 2016 In-house 2017 accounts. I. Key Project Deliverables/Reports/Consultations I.1. CSIR-SARI Business Plan CSIR-SARI Direct Support Project. 2019.Business Plan for Savanna Agricultural Research Institute. Nyankpala: The CSIR-SARI Direct Support Project for USAID. March 2019. 1.2. Nutrition Francis Kweku Amaglogh. 2019. A progress report on the SARI-COE Food and Nutrition Unit. Report #2. Nyankpala: The USAID Direct Support Project for CSIR-SARI. (Note Contract January 2019-August 2019). I.3. CSIR-SARI Strategic Plans I.3.1. 2017 Strategic Plan Samuel D. Braimah. 2017. The CSIR-SARI Strategic Plan: 2018-2022. Nyankpala: The USAID Direct Support Project for CSIR-SARI. Samuel D. Braimah. 2017. The CSIR-SARI Strategic Plan: 2018-2022 –Brochure Summary. Nyankpala : The USAID Direct Support Project for CSIR-SARI. Samuel D. Braimah. 2017. The CSIR-SARI Strategic Plan: 2018-2022. Abridged Version. Nyankpala: The USAID Direct Support Project for CSIR-SARI. Samuel D. Braimah. 2017. The CSIR-SARI Strategic Plan: Theory of Change. Nyankpala: The USAID Direct Support Project for CSIR-SARI. Samuel D. Braimah. 2017. The CSIR-SARI Strategic Plan M&E Framework. Nyankpala: The USAID Direct Support Project for CSIR-SARI. Samuel D. Braimah. 2017. The CSIR-SARI Strategic Plan. PowerPoint. The USAID Direct Support Project for CSIR-SARI. Project Archives. I.3.2. 2021 Strategic Plan (that is heavily based on the 2017 one) CSIR-SARI. 2021. Final Draft CSIR-SARI Strategic Plan. 2021-2025. Nyankpala: CSIR-SARI. 1.3.3. 2005 Strategic Plan Eva Osei and Augustine Opoku Antwi. 2005. CSIR-SARI Report on Assessment of CSIR-SARI and Strategic Planning Process. Kaneshie-Accra: Nkum Associations for CSIR-SARI (August 2005). I.4. Internet Activities (see also SIL Under Key Partners) Emmanuel Togo and Ato Yawon. 2019. CSIR-Savanna Agricultural Research Institute. Presentation based on May 23-24, 2019 Field Visit. PowerPoint Presentation. 174 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV J. Key Partners and Legacy Projects J.1. Key Partner—Africa Rice Africa Rice. 2015. Sub-contract Agreement between the Africa Rice Center (AfricaRice) and CSIR-SARI (Period of Agreement 1 June 2015-31 May 2016). USAID Seed Scaling Project. Africa Rice Center (September 6, 2015): USAID/Ghana. CSIR-SARI. 2016. USAID Rice Seed Scaling Project. Annual Report. Nyankpala: CSIR-SARI. (March 2016). CSIR-SARI. 2016. MOU between CSIR-SARI and Heritage Seed Company for Foundation Seed Production of Selected Rice Varieties and Demonstrations on Same. Nyankpala CSIR-SARI. J.2. Key Partner—Africa Lead John Nene-Osom Azu and Kwesi Opoku-Debrah. 2012. Assessment of the Savanna Agricultural Research Institute (SARI) Nyankpala , Northern Region, Ghana. Washington, DC: Africa Lead for USAID/Ghana (July 12, 2012). J.3. Key Partner—ATT USAID/Feed the Future. 2015. Fact Sheet: Feed the Future USAID Agriculture Technology Transfer Project (ATT). Washington: FTF. ATT. 2018. Feed the Future Ghana Agriculture Technology Transfer Project. Final Project Report. 2013- 2018. USAID Cooperative Agreement No. AID-641-A-13-00001. Nyankpala: International Fertilizer Development Center (IFDC) for USAID/Feed the Future. ATT. 2016. Annual Report (FY2016-2017) (1 October 2015-31 September 2016). Nyankpala: IFDC for USAID/Feed the Future. IFDC. 2013. Technical Application. Feed the Future. USAID Agricultural Technology Transfer Project. Muscle Shoals, Alabama: IFDC. (June 11, 2013). John Nene Osom Azu. 2018. Report on USAID/IFDC Agricultural Technology Transfer Project’s Lessons Learned Workshops: Sustaining the Legacy. Nyankpala: ATT Project for USAID/Feed the Future. (December 2018). J.4. Key Partner—SIL Soybean Innovation Lab. 2021. Soybean Innovation Lab (IL) and SARI Collaboration Overview (FY2015- FY2020). Champagne-Urbana, Illinois: Soybean Innovation Lab (SIL). J.5. Agricultural Value Chain Mentorship Project. Agricultural Value Chain Mentorship Project (. 2016. End-of-Project Technical Report (June 2011– January 2015). Grant No. 2011 AVCF 001. Nyankpala: AVCMP for the IFDC, Savanna Agricultural Research Institute (SARI) and the Ghana Agricultural Associations Business Information Centre through the Alliance for Green Revolution in Africa (AGRA) for the Danish International Development Agency . K. Other Relevant National and Regional Documents Consulted Ernest A. Asiedu, J. Wobil, F. Ansah-Amprofi, S. Gyan-Ansah, F.N. Darimaani, and A. Larbi. 2019 (January). The Ghana Seed Industry: Advances and Prospects. Report of the Seed Sector Support Workshop Organized by the Ministry of Food and Agriculture—Savannah Zone Agricultural Productivity Improvement Project. Nyankpala: Savannah Zone Agricultural Productivity Improvement Project (SAPIP), for the Ministry of Food and Agriculture. (January 15-17, 2019 at Nyankpala, Ghana). 175 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Republic of Ghana. 2019. Plants and Fertilizer Act (Act 803). Accra: Government of Ghana. Nutrition in Northern Ghana. L. Refereed Journal Articles Linked to Capacity-building, Infrastructure, Research and Equipment Support the Project Funded from FY 2015-present. 1. Amagloh, F.K., Amagloh F.C. and Coad, J. (2016) Maltose is an inappropriate indicator of digestibility of complementary foods containing substantial amounts of this simple sugar. International Food Research Journal 23(2), 879-884. 2. Dogbe W, Sogbedji J. M, Buah S. S. J (2015). Site-specific Nutrient Management for Lowland Rice in the Northern Savannah Zones of Ghana. Curr Agri Res. 3(2), 109-117. doi : http://dx.doi.org/10.12944/CARJ.3.2.04 3. Dogbe, W., Abebrese, S. O., Owusu, R., Inusah, B., and Danaa, A. (2016). Performance of eleven introduced improved lowland rice varieties in the northern Savanna zones of Ghana. African Journal of Agricultural Sciences. 11(5), 324-329 4. Abibou Niang, Mathias Becker, Frank Ewert, Ibnou Dieng, Thomas Gaiser, Atsuko Tanaka, Kalimuthu Senthilkumar, Jonne Rodenburg, Jean-Martial Johnson, Cyriaque Akakpo, Zacharie Segda, Henri Gbakatchetche, Famara Jaiteh, Ralph K. Bam, Wilson Dogbe, Sékou Keita, Nianankoro Kamissoko, Illiassou Maïga Mossi, Oladele S. Bakare, Madiama Cissé, Idriss Baggie, Komlan A. Ablede, Kazuki Saito (2017). Variability and determinants of yields in rice production systems of West Africa. Field Crops Research 207, 1-12 5. Buah S.S.J., Ibrahim H., Derigubah M., Kuzie M., Segtaa J.V., Bayala J., Zougmore R. and Ouedraogo M. (2017). Tillage and fertilizer effect on maize and soybean yields in the Guinea savanna zone of Ghana. Agriculture & Food Security 6:17. 6. Abebrese, S. O., Dartey, P. K. A., Akromah, R., Gracen, V. E., Offei, S. K., & Danquah, E. Y. (2018). Identification of CMS maintainers and restorers for hybrid rice development in Ghana. Journal of Crop Improvement, 1-15. 7. Abebrese, S. O., Dartey, P. K. A., Akromah, R., Gracen, V. E., Offei, S. K., & Danquah, E. Y. (2018). Genetics of anther indehiscence with exerted stigmas and its application in hybrid rice breeding. Journal of Crop Improvement, 32(4), 552-565. 8. Adu G.B., Alidu H., Amegbor I.K., Abdulai M.S., Nutsugah S.K., Obeng-Antwi K., Kanton R.A.L., Buah S.S., Kombiok J.M., Abudulai M., Etwire P.M. (2018). Performance of maize populations under different nitrogen rates in northern Ghana. Annals of Agricultural Sciences 63 (2): 145- 152. 9. Chigeza, G., Boahen, S., Gedil, M., Agoyi, E., Mushoriwa, H., Denwar, N., Gondwe, T., Tesfaye, A., Kamara, A., Chikoye, D. (2018). Genomics, genetics, and breeding of soybean: Advances in cultivar development in the African Tropics. Plant Breeding 10. Abebrese, S. O., Yeboah, A., Dogbe., W., Dartey, P.K.A.,Akromah, R., Vernon Edward Gracen, V.E., Offei, S. K. and Danquah, E. Y. (2019). Evaluation of Yield, Reaction to Diseases, and Grain Physical Attributes of Some Introduced Rice Hybrids in Ghana. International Journal of Agronomy, Volume 2019, Article ID 3926765, 8 pages https://doi.org/10.1155/2019/3926765 11. Abebrese, S. O., Martey, E., Dartey, P. K. A., Akromah, R., Edward, V., Gracen, S. K. O., & Danquah, E. Y. (2019). Farmer preferred traits and potential for adoption of hybrid rice in Ghana. Sustainable Agriculture Research, 8(3), 38-48. 12. Adu, G. B., Awuku, F. J., Amegbor, I. K., Haruna, A., Manigben, K. A. and Aboyadana, P.A. (2019). Genetic characterization and population structure of maize populations using SSR markers. Annals of Agricultural Sciences. 64 (2019): 47–54. 13. Adu, G. B., Badu-Apraku, B., Akromah, R., Garcia-Oliveira, A. L., Awuku, F. J. and Gedil, M. (2019). Genetic diversity and population structure of early-maturing tropical maize inbred lines using SNP markers. PLoS ONE 14(4): e0214810. 176 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV 14. Adu, G. B., Badu-Apraku, B., Akromah, R., Haruna, A., Amegbor, I. K. and Manigben, K. A. (2019). Grain yield and stability of early-maturing single-cross hybrids of maize across contrasting environments. Journal of Crop Improvement. DOI: 10.1080/15427528.2019.1661055. 15. Denwar, N. N., Frederick J. Awuku, Brian Diers, Francisca Addae‐Frimpomaah, Godfree Chigeza, Richard Oteng‐Frimpong, Doris K. Puozaa and Michael T. Barnor. (2019). Genetic diversity, population structure and key phenotypic traits driving variation within soyabean (Glycine max) collection in Ghana. Plant Breed. 138:577–587. https ://doi. org/10.1111/pbr.12700 DOI: 10.1080/15428052.2018.1495588 16. Nboyine, J. A., Kusi, F., Abudulai, M., Badii, B.K., Zakaria, M., Adu, G.B., Haruna, A., Seidu, A., Osei, V., Alhassan, S., Yahay, A. (2019). A new pest, Spodoptera frugiperda (J.E. Smith), in tropical Africa: Its seasonal dynamics and damage in maize fields in northern Ghana. Crop Protection 127 (2020)104960. 177 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex V.B.1. CSIR-SARI Technical and Financial Support Actual Expenses (2015–2020) vs. the Original Budget for the Sub￾Purposes, Outputs, and Major Activities in the Results Framework Activity No. Activity Description FY 2015 Budget FY 2016 Budget FY 2017 Budget FY 2018 Budget FY 2019 Budget FY 2020 Budget Actual Expenditu re Total US$ % Of Actual Budget Spent Original Budget Projecti on % Of Total Budget Projecti on Sub￾purpose 1 Capacity development for agricultural research in Northern Ghana Output1. 1 SARI’s core scientists trained and re-tooled in key program areas 0 No data. 223 325,000 5.85% Output 1.2 Increased organizational quality and efficiency of SARI 1.2.1 Compensation for hired administration and operation staff￾Contract Project Manager and Office Assistant 577.36 568.18 2,812.50 10,734 14,692.04 0.40% 700,000 12.61% 1.2.2 Consultancy on the establishment of COE for SARI by ISU and AgTILs 183,088.70 63,234 246,322.70 6.62% New N/A 1.2.6 Update Institutes strategic plan for making SARI a Center of Excellence in agricultural research and development in the 0 0.00% 50,000 0.90% 223 The figures for Output 1.1 were merged with 1.2. 178 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Activity No. Activity Description FY 2015 Budget FY 2016 Budget FY 2017 Budget FY 2018 Budget FY 2019 Budget FY 2020 Budget Actual Expenditu re Total US$ % Of Actual Budget Spent Original Budget Projecti on % Of Total Budget Projecti on Savanna zones of the WA sub-region 1.2.7 Provide support for the implementation of SARI’s long and short￾term staff capacity development plan 0 0.00 80,000 1.44% & 1.2. SUBTOTAL 577.36 568.18 185,901.20 73,968. 261,014.74 7.02% 1,155,000 20.81% Output 1.3 Infrastructure and Facilities of the Institute Built in Order to Support the Conduct of Top Quality Research 1.3.1 Rehabilitate SARI structures (Damongo, Nyankpala, Wa, Manga) 314,336. 70 28,862.6 9 305,365. 89 612,060.16 1,260,625.44 33.90% 704,625.35 12.70% 1.3.2 Construct new infrastructure for the Institute (Nyankpala, Manga, Wa, etc ) 507,921. 11 413,955. 81 921,876.92 25% 600,000 10.81% 1.3.3 Purchase vehicles and motor bikes 112,967. 66 27,906.9 8 140,874.64 3.79% 250,000 4.50% 1.3.4 Purchase equipment and tools for field work and workshop 0 0.00% 80,000 1.44% 1.3.5 Procure equipment and supplies for offices and laboratories 299,913.60 31,120.4 0 331,034.00 8.90% 500,000 9.0% 1.3.6 Procure equipment and tools for the establishment/rehabilita tion of the climate change, 0 0.00 200,000 3.6% 179 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Activity No. Activity Description FY 2015 Budget FY 2016 Budget FY 2017 Budget FY 2018 Budget FY 2019 Budget FY 2020 Budget Actual Expenditu re Total US$ % Of Actual Budget Spent Original Budget Projecti on % Of Total Budget Projecti on communication, and GIS units 1.3.7 Procure a modern MIS to streamline administrative and management procedures 0 0.00 40,000 0.72% 1.3. Subtotal 2,654,411 46.84% 2,374,625.3 5 42.77% Sub￾purpose 1 SUBTOAL 427,304. 36 564,690. 78 413,955. 81 305,365. 89 911,973.76 31,120.4 0 2,654,411.00 71.38% 3,529,625.3 5 63.60%% Sub￾purpose 2: Agricultural technologies development and dissemination Output 2.1 SARI’s seed operations significantly modernized in line with plant and fertilizer Act 2.1.1 CROPS: RICE COMPONENT 17,226.2 6 29,595.2 1 26,962.2 7 12,946.9 1 25,852.76 112,583.41 3.03% 2.1.2 CROPS: SOYBEAN COMPONENT 17,226.2 6 41,472.5 1 26,274.4 0 17,440.9 1 18,650.92 121,065.00 3.26% 2.1.3 CROPS: MAIZE COMPONENT 17,226.2 6 59,316.6 5 24,854.3 8 11,996.8 2 17,890.42 131,284.53 3.53% 2.1.4 POST-HARVEST LOSSES-Develop strategies to reduce field and post-harvest losses for developed varieties 2,682.36 2,682.36 0.07% 180 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Activity No. Activity Description FY 2015 Budget FY 2016 Budget FY 2017 Budget FY 2018 Budget FY 2019 Budget FY 2020 Budget Actual Expenditu re Total US$ % Of Actual Budget Spent Original Budget Projecti on % Of Total Budget Projecti on 2.1.5 Develop good agricultural practices to enhance and sustain productivity of the developed varieties 2.1.6 NUTRITION-Conduct characterization/quality parameters, food product development/recipe refinement, sensory and consumer acceptability evaluations for developed varieties 7,967.79 6,684.10 1,666.10 25,733.26 42,051.25 1.13% 2.1.7 CROPS PROTECTION￾ACT.2.1.9 Train seed producers/farmers/inpu t dealers in seed production and storage 12,838.4 2 6,203.18 10,544.34 29,585.94 0.80% 2.1. Subtotal 485,433.96 13.06% 1,280,000.0 0 23.06% Output 2.2. ISFM practices developed and disseminated 1,186.04 21,672.0 8 6,728.36 1,433.64 5,342.11 36,362.23 0.98% 300,000 5.4% Output 2.3. Capacity of technical staff to apply modern tools/techniques in research built 0 0.00 50,000 0.9% Output 2.4. RELC mechanism for agricultural technologies dissemination improved 9,307.81 12,094.7 8 21,402.59 0.58% 90,000 1.62% 181 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Activity No. Activity Description FY 2015 Budget FY 2016 Budget FY 2017 Budget FY 2018 Budget FY 2019 Budget FY 2020 Budget Actual Expenditu re Total US$ % Of Actual Budget Spent Original Budget Projecti on % Of Total Budget Projecti on Sub￾purpose 2 SUBTOAL 62,172.6 3 187,639. 80 91,503.5 1 57,688.2 4 144,194.60 543,198.78 14.61 1,720,000 30.99%% Sub￾purpose 3: Internal coordination, project management, communication, and M&E Output 3.1. ICT and Knowledge Management Systems for enhanced market-driven research for development developed 3.1.1. ICT and Knowledge Management Systems for enhanced market-driven research for development developed 7,717.27 2,403.00 10,120.27 0.27% No data. No data. 3.1.2 Installation of 155Mbps leased Circuit from SARI to NITA by GARNET 29,958.0 2 29,958.02 0.81% No data No data 3.1. Subtotal 40,078.29 1.08% 120,000 2.16% Output 3.2. Commercialization program of the Institute strengthened 3,016.59 3,614.32 6,630.91 0.18% 150,000 2.70% 2.1.8 SEED DEVELOPMENT￾Procurement of foundation seed 6,000.68 40,180.79 46,181.47 1.24% Output 3.3. M&E system developed and operationalized 98,878.9 6 80,716.0 2 6,507.39 27,073.44 213,175.81 5.73% 250,000 4.5% Sub￾purpose 3 SUBTOTAL 98,878.9 6 80,716.0 2 6,507.39 10,733.8 6 33,090.76 29,958.0 2 259,885.01 6.99% 520,000 9.37 182 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Activity No. Activity Description FY 2015 Budget FY 2016 Budget FY 2017 Budget FY 2018 Budget FY 2019 Budget FY 2020 Budget Actual Expenditu re Total US$ % Of Actual Budget Spent Original Budget Projecti on % Of Total Budget Projecti on TOTAL EXPENDITURE US($) 588,355. 95 833,046. 60 512,544. 07 374,356. 17 1,275,160. 32 135,046. 42 3,718,509.53 100.00 5,769,625.3 5, 100% Source: Sebastian Tigbee, Senior Accountant, CSIR-SARI, February 25, 2021 (for projected budget) and March 12, 2021 for actual expenditures based on project records. Verified by Wilson Dogbe, March 12, 2021. 183 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex V.B.2. USAID/Ghana CSIR-SARI Direct Support Budget for Five Years (FY 2015-2019) Infrastructure Targeted Versus Actual No. Description Target (EoL) Implemented (Y/N) Sub-purpose 1 Capacity development for agricultural research in Northern Ghana Output 1.1 SARI’s core scientists trained and re-tooled in key program areas 1.1.1 Provide tailored leadership and management training to divisional and sectional heads of the Institute to improve research administration Y 1.1.2 Provide long-term training to plant breeders, agronomists, soil scientists, micro￾biologists, etc. to improve research output of the Institute Partial 1.1.3 Provide short-medium-long-term technical training to scientists and technicians of the Institute in support of the Institute’s capacity development plan Y 1.1.4 Provide technical assistance to SARI on engaging key stakeholders for increased effectiveness, collaboration and partnership Y 1.1.5 Establish experts exchange program for SARI to improve management coordination and institutionalization of best research management practices 15 N 1.1.6 Provide training to Scientists and technicians in statistical software for data analysis 60 Y Output 1.2 Increased organizational quality and efficiency of SARI 1.2.1 Recruit a Program Manager on contract and provide resources for effective running of his office N 1.2.1.1 Recruit a Program Manager 1 Y 1.2.1.2 Set up a project Coordination unit 1 Y 1.2.1.3 Pay salaries of Project Manager and other project staff PM = 60 OM = 60 Act = 60 Y 1.2.1.4 Set up Core of Excellence project Management Structure (Advisory board [AB] and Project Management team [PMT]) and support their operations. AB= 4meeting/yr PMT=24meeting/yr N 1.2.1.5 Develop a CoE policy Document N 1.2.1.6 Develop institutional performance framework and monitoring plan Y 1.2.1.7 Undertake routine maintenance of equipment, vehicles and facilities Y 1.2.1.8 Procure insurance for vehicles and motorbikes Y 1.2.1.9 Facilitate domestic and international travels for CoE administrative staff (Transport, Accommodation & DSA) Y 1.2.1.10 Undertake domestic travels to meetings and field visits 20 Y 1.2.1.11 Undertake international travels 5 Y 1.2.1.12 Pay monthly voice and data subscription of project. 60 Y 1.2.1.13 Provide monthly fuel for the running of the CoE administration 26000 Y 1.2.2 Develop public relations and communication plans 1 Y 184 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV 1.2.3 Build the capacity of the CSIR-SARI Management Board and IMC to increase its efficiency in decision-making Y 1.2.3.1 Build Capacity of Management board members 6 N 1.2.3.2 Build capacity of IMC members 16 Y 1.2.4 Provide leadership and performance-based management training Y 1.2.4.1 The challenge of change for Top level Management 3 N 1.2.4.2 Project Planning and Management 4 Y 1.2.4.3 Financial Modeling 3 N 1.2.4.4 Report Writing and Corporate communication skills 2 Y 1.2.4.5 Office Management and Supervisory Skills 8 Y 1.2.4.6 Grant Proposal Writing 4 N 1.2.4.7 Procurement Management for Public Sector Organizations 3 Y 1.2.4.8 Public Financial Management 3 Y 1.2.5 Provide training on USAID’s financial and procurement regulations and reporting systems 1 Y 1.2.6 Update Institutes strategic plan for making SARI a Center of Excellence (CoE) in agricultural research and development in the Savanna zones of the WA sub￾region 1 Y 1.2.6.1 Recruit consultant to develop new strategic plan Y 1.2.6.2 Organize strategic planning workshop Y 1.2.6.3 review meeting to discuss draft strategic plan Y 1.2.6.4 Printing of strategic plan Y 1.2.6.5 Launch of new strategic plan N 1.2.6.6 Awareness creation on strategic plan Y 1.2.7 Provide support for the implementation of SARI’s long and short-term staff capacity development plan PhD 10 Msc 8 BSC & HND 5 Partial 1.2.7.1 Support to staff on PhD training 10 Y 1.2.7.2 Support to staff on MSc training 8 N 1.2.7.3 Support to staff on Bsc training N 1.2.7.4 Support to staff on HND training N 1.2.7.5 Support to staff on short term skills development training Y Output 1.3 Infrastructure and Facilities of the Institute Built in Order to Support the Conduct of Top Quality Research 1.3.1 Rehabilitate SARI structures (Damongo, Nyankpala, Wa, Manga, Jirapa, Tumu, Babile, Nakpanduri, etc) Y 1.3.1.1 Renovate Senior members bungalows 2 Y 1.3.1.2 Renovate Senior members flats 3 Y 185 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV 1.3.1.3 Renovate CSIR-SARI canteen 1 Y 1.3.1.4 Renovate senior staff Quarters 1 Y 1.3.1.5 Renovate CSIR-SARI office block 1 Y 1.3.1.6 Renovate Seed Cold room facility 1 Y 1.3.1.7 refurbish offices CoE Research Coordinating Unit 1 Y 1.3.1.8 1.3.1.9 1.3.1.10 1.3.1.11 Renovation of Bungalow @ Manga station 1 Y 1.3.1.12 Renovation of input and seed stores & platforms @ Manga station 2 Y 1.3.1.13 Fencing of station & experimental fields @ Manga 1 N 1.3.1.14 Renovate farm house and platform @ Damongo station 1 N Renovate office block at Damongo station N 1.3.1.15 Retool the soil and plant analysis lab @ Nyankpala N 1.3.1.16 Rehabilitate supervisors quarters 4 N 1.3.1.17 Rehabilitate artisan quarters 12 N 1.3.1.18 Rehabilitate labourers quarters 20 N 1.3.1.19 Rehabilitate 5-room office block at Wa 1 Y 1.3.1.20 Rehabilitate Conference room facility @ Wa 1 N 1.3.1.21 Construct a 30 Mt seed storage facility and drying platform @ Wa 1 N 1.3.2 Construct new infrastructure for the Institute (Nyankpala, Manga, Wa, Damongo etc. ) N 1.3.2.1 Construct ICT and nutrition facility at Nyankpala 1 Y 1.3.2.2 Construct office and lab facility for soybean team 1 Y 1.3.2.3 Construct a 100Mt storage facility for Rice EGS. 1 Y 1.3.2.4 Construct Seed Research and training Lab (Working Room) 1 N 1.3.2.5 Construct a Seed processing/Conditioning facility and seed storage house and a drying platform 1 Y 1.3.2.6 Construct a 30 Mt seed storage facility and drying platform @ Wa 1 1.3.2.7 Fence and landscape CSIR-SARI Conference centre 1.3.3 Purchase vehicles and motor bikes Y 1.3.3.1 Purchase vehicle for field monitoring 3 Y 1.3.3.2 Purchase motor bikes for field monitoring by techicians 10 Y 1.3.4 Purchase equipment and tools for field work and Workshop Y Equipment and tools for field expt Y Equipment and tools for general workshop N 186 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Equipment and tools for welding shop N Equipment and tools for Vulganizing shop N Equipment and tools for Electrical shop N Equipment and tools for painting shop N Equipment and tools for masonry shop N 1.3.5 Procure equipment and supplies for offices and laboratories Y 1.3.5.1 Furniture, Equipment & supplies for offices Y 1.3.5.3 Furniture, Equipment & supplies for Nutrition Lab Y 1.3.5.4 Equipment and furniture for Rice storage facility Y 1.3.5.5 Furniture, Equipment & supplies for Biotech lab N 1.3.5.6 Furniture, Equipment & supplies for seed research and training facility Y 1.3.5.7 Furniture equipment and supplies for Seed Processing centre N 1.3.6 Procure equipment and tools for the establishment/rehabilitation of the climate change, communication and GIS units N 1.3.6.1 Equipment and Furniture for climate change and GIS unit N 1.3.6.2 Equipment and furniture for communication & ICT facility Y 1.3.7 Procure a modern MIS to streamline administrative and management procedures N Sub-purpose 2 Agricultural technologies development and dissemination Output 2.1 SARI’s seed operations significantly modernized in line with plant and fertilizer Act 2.1.1 Carry out germplasm collection, characterization and evaluation Y 2.1.2 Develop new varieties through hybridization, generation advancement and field evaluation Y 2.1.3 Carry out procedures leading to variety release Y 2.1.4 Carry out breeder seed production, variety promotion and maintenance Y 2.1.5 Develop strategies to reduce field and post-harvest losses for developed varieties Y 2.1.6 Develop good agricultural practices (GAPs) to enhance and sustain productivity of the developed varieties Y 2.1.7 Conduct characterization/quality parameters, food product development/recipe refinement, sensory and consumer acceptability evaluations for developed varieties Y 2.1.8 Train seed producers/farmers/input dealers in seed production and storage Y 2.1.9 Train agro-chemical dealers on safe and efficient use of agro-chemicals Y 2.1.10 Promote community seed production and distribution scheme Y 187 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Output 2.2 Integrated soil fertility management (ISFM) practices developed and disseminated 2.2.1 Conduct initial soil analyses and site-specific fertilizer recommendation trials on rice, maize and soybeans Y 2.2.2 Conduct integrated soil fertility management demonstrations on rice, maize and soybean Y 2.2.3 Identify, select and characterize benchmark soils on rice, maize and soybean Y 2.2.4 Conduct yield-response trials on rice, maize and soybean Y 2.2.5 Develop fertilizer recommendation maps on rice, maize and soybean in northern Ghana using developed simulation models and GIS concepts Y 2.2.6 Conduct trials and demonstrations on conservation agriculture practices using FFS/FFF/FLC Y 2.2.7 Conduct soil fertility trials through use of cover crops for Carbon sequestration and nutrient cycling Y 2.2.8 Develop and produce a training manual on ISFM for training technical staff Y 2.2.9 Develop and promote sustainable soil and water conservation practices Y 2.2.10 Train technical staff and farmers on ISFM Y Output 2.3 Capacity of technical staff to apply modern tools/techniques in research built 2.3.1 Provide refresher and other training to field technicians in laboratory safety/equipment handling/data collection and management and quality assurance procedures Y 2.3.2 Provide training for administrative, accounting and audit staff Partial Output 2.4 RELC mechanism for agricultural technologies dissemination improved 2.4.1 Improve the ability of the Research, Extension and farmer Linkage Committee (RELC) to strengthen critical research, extension linkages and ensure effective development of technologies through RELC work planning Y 2.4.2 Strengthen the weak links of the RELC mechanism designed to identify, consolidate and prioritized research needs of farmers in the northern regions Y 2.4.3 Strengthen capacity of research staff to package and disseminate research findings 2.4.4 Organize zonal, district and regional planning sessions N 2.4.5 Conduct ToT for extension officers at the regional level Y 2.4.6 Conduct training of farmers at the district level Y 2.4.7 Conduct monitoring of on-farm adaptive trials and demonstrations. Y 2.4.8 Conduct RELC executive members meetings twice a year Y 2.4.9 Produce and disseminate technical leaflets and production guides Y Sub-purpose 3 Internal coordination, project management, communication and Monitoring and Evaluation 188 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Output 3.1 ICT and Knowledge Management Systems for enhanced market-driven research for development developed 3.1.1 Establish a technology information center Y 3.1.2 Develop communication strategy and plan for the Institute Y 3.1.3 Set up public relations and communications unit Y 3.1.4 Develop communication tools (newsletters/website, Campaigns, Adverts, Newsletters, Brochures, video, Radio/TV, Exhibitions) Y 3.1.5 Engage consultant to develop a structure for knowledge management at SARI N 3.1.6 Organize seminars to sensitize staff on intellectual property rights Y Output 3.2 Commercialization program of the Institute strengthened 3.2.1 Engage consultant to develop a business plan for SARI’s products and services 3.2.2 Develop fee-for-service mechanisms in collaboration with the private sector 3.2.3 Provide advocacy training assistance to improve the Institute’s performance in sourcing funds 3.2.4 Contract a consultant to provide training to staff in the area of technology commercialization Output 3.3 Monitoring and Evaluation (M&E) system developed and operationalized 3.3.1 Hire consultant to establish a robust M&E system for the Institute (M&E plan, PPITT, MIS, and reporting procedures) Y 3.3.2 Strengthen the capacity of staff on M&E system design and implementation and database management (training courses etc. related to M&E) Y 3.3.3 Hire a consultant to conduct an impact evaluation of SARI’s program Y 3.3.4 Undertake field monitoring visits Y 3.3.5 Organize and attend periodic review meetings Y 3.3.6 Undertake a study to evaluate adoption and impact of existing technologies Y 3.3.7 Undertake a study to examine the impact of climate change on farm productivity. N 3.3.8 Undertake a study to examine the role of gender in agricultural production systems Y 3.3.9 Establish effective commodity value chains (innovation platforms) Y Source: CSIR-SARI Financial Officer Sebastian Tigbee, 2021. 189 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex V.B.3. Interactions Between the ICT Connectivity Project of SIL with the Staff of SARI in Nyankpala, Ghana and the Leadership of GARNET in Accra, Ghana, FY 2015-2020 Date Activity Nov 15, 2016 Pete Goldsmith, SIL PI, was concerned about the lack of dependable ICT connectivity at SARI and how it was limiting the effectiveness of its researchers. Goldsmith invited Paul Hixson to his office to discuss the issue with Dileepkumar Guntuku, an ATT sub￾contractor from ISU. After the meeting, Goldsmith and Hixson proposed a small $25K project involving Hixson conducting a couple of exploratory visits to SARI to gain on-the￾ground information, submit assessment reports, and draw up proposal for any necessary follow-up activities. (see attached email outlining proposal – this proposal was never accepted or followed up on by ATT). March 16-17, 2017 Goldsmith arranged for Hixson to hold a full-day seminar on the changing nature of ICT for Ag Researchers globally and the need to strengthen ICT capacity at SARI, before Hixson was even part of SIL. The seminar was conducted for visiting members of the SARI COE team Dileep brought to ISU for a month-long leadership training. Attendees included Director Stephan Nutsugah, Senior Researchers Dr. Nicholas Denwar and Dr. Wilson Dogbe, and Communications Specialist Wilhelm Kutah. March 27, 2017 SIL submitted a four-year project proposal to Dileep to have Hixson and a second IT networking specialist work under Goldsmith’s direction to improve SARI’s networking infrastructure, connectivity, and ICT training. The proposal involved extensive in-country time working with local staff and commercial IT vendors. The proposal was never accepted by Dileep or discussed further. We later discovered Dileep had contracted with a young IT professional from Zimbabwe who works at International Crops Research Institute for the Semi-Arid Tropics to conduct a less expensive study of how to resolve SARI’s ICT problems. Oct, 2018 When SIL received a three-year renewal of USAID funding, SIL leadership carved out a small research budget for ICT connectivity and selected Hixson to lead that initiative. Hixson quickly brought on board Tracy Smith, current deputy CIO at UIUC for ICT innovation as a member of the research team. As soon as Hixson was on the project, he made his first field visit to SARI late in October. While there he used his first iteration of the ICT Health CheckUp tool to try to identify contributing factors for SARI’s inability to provide consistent, dependable ICT connectivity to its researchers. During that visit, Hixson conferred with Denwar, Dogbe, and Kutah. In addition, Hixson spent a lot of time with SARI IT staffers Musah Iddi and Daniel Akovuta. During the initial tests, SARI’s network was so bad that it behaved like a frozen network. On later tests, it tested at such a low bandwidth it was deemed inadequate for even casual users – and was totally inadequate for senior researchers. Subsequent interviews revealed SARI’s staff and leadership had concentrated all efforts to improve ICT connectivity by working with commercial telecom companies like Ghana Vodaphone and MTN, and they were totally unaware of the presence in Ghana of a National Research and Education Network (NREN) that could provide higher bandwidth at lower cost for members than commercial vendors. See our published report of this research which revealed SARI had major problems with all four categories of our ICT Health CheckUp (connectivity, physical infrastructure, intranet services, and IT professional staffing): https://ajfand.net/Volume20/No5/Goldsmith19385.pdf Spring 2019 Hixson worked with Dogbe, Smith, and Dale Smith at the University of Oregon’s Network Startup Resource Center to refine the ICT Health CheckUp tool to better identify critical ICT physical networking problems at National Agricultural Research Systems (NARS) institutions. Hixson also worked remotely with the folks at GARNET and encouraged them to send their chief engineer Emmanuel Togo to SARI and prepare a proposal for what it would take to totally redo SARI’s networking infrastructure and make it fully functional (fiber and wireless), including equipping the main Nyankpala campus to 190 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Date Activity connect via a virtual private network (VPN) enabled circuit with the remote research stations at Manga and Wa. GARNET gave a presentation/proposal for that work in late May 2019. The total cost of that proposed work was originally set by GARNET at $670,765 (GHC) – approximately $116,00 USD (see attached project proposal from GARNET). Hixson advised Wilson that was a reasonable proposal and recommended SARI work with GARNET to finalize it. Wilson told Hixson that the problem was GARNET’s proposal was too expensive and although SARI needed to make these improvements, they just did not have enough unobligated funds remaining in the project to cover such a cost for ICT infrastructure. August 2019 Over the summer, Hixson worked with NSRC partners at the University of Oregon who agreed to contribute some networking switches to GARNET so they could lower the cost of their proposed network redesign for SARI. At the same time, per joint discussion with all parties, it was agreed to replace the 14 Cisco switches with 14 less-expensive Ubiquiti switches. GARNET submitted a revised cost estimate for re-doing SARI’s networking infrastructure at the main Nyankpala campus. The total cost of the revised proposal was cut from a previous $670,765 (GHC) – approximately $116,00 USD --- to $258,350 (GHC) — approximately $47,842 USD. Hixson urged Wilson to accept this bid and select GARNET to design and rebuild SARI’s wired and wireless infrastructure. (see attached email) October 2019 Hixson visited SARI as part of a three-country visit to NRENs in Ghana, Malawi, and Zambia. While in Accra, Hixson met with GARNET’s CIO, Lucas Chigabatia, who confirmed GARNET’s continued interest in making SARI a member of GARNET, and thus eligible for member-based pricing of high bandwidth connectivity. In discussions with GARNET, Hixson learned GARNET was irritated that SARI had rejected GARNET’s revised network redesign proposal and hired an external IT consultant from India to do the work. GARNET, understandably, changed their position to one of still being willing to deliver connectivity to SARI, but drew the line at helping SARI with any trouble-shooting of local networking problems beyond the point of the networking handoff to SARI’s local networking infrastructure. Hixson carried that message to Nyankpala and worked with Dogbe to write up a report for the director recommending SARI move from MTN to GARNET (draft version of that document enclosed as an attachment). While at SARI, Hixson also met SV Prasant, the less expensive IT consultant from India whom SARI had meanwhile hired, based on a recommendation from Dileep, to do the work GARNET had previously submitted a bid to do. Prasant had been at SARI for a couple of months when Hixson arrived, and shortly after Hixson left, Prasant returned home to India. Prasant’s design remains only partially implemented to this day, and thus the wireless infrastructure at SARI is still problematic and the VPN connections to Manga and Wa were never established. Hixson asked Wilson for a copy of SV’s proposal, but when it was shared, it was fairly vague with no timetable for implementation, and it contained no pricing information. (document attached). Hixson also took additional measurements of network performance while at SARI. (see attached trip report). June 2020 Hixson, Smith, Dale Smith, and NREN colleagues conducted a training seminar over the web on the importance of NARS institutions working with their home country NREN to obtain the best networking performance and to simultaneously deepen professional links with peers. https://bit.ly/3hnOWfF This webinar had 70 attendees from 23 countries who were able to participate live during the broadcast. Others have since watched the recording on YouTube. https://www.youtube.com/watch?v=SQ4fCntT2Lw&t=26s 191 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Date Activity Oct 2020 An article from the UIUC highlighted the major themes of this new ICT Connectivity Project – that Internet connectivity is the oxygen needed for modern ag research and development work. https://aces.illinois.edu/news/internet-connectivity-oxygen-research-and-development￾work Nov 2020 SIL reported on a big win on Connectivity at SARI. https://mailchi.mp/illinois/high-speed-ict￾connectivity-delivered?e=6c46063b0f SARI researchers Dr. Nicholas Denwar and Dr. Edward Martey explained how as a result of using the recommendations from the ICT Health CheckUp, SARI had switched from MTN to GARNET and as a result had been able to increase their bandwidth a full 75X over what it had been in 2018, and the cost per Mbps for connectivity had dropped from $401 USD/mbps in 2018 to $28 USD in 2020. What is not mentioned in that story is that SARI’s physical networking infrastructure still has unresolved wireless problems and the desired VPN intranet connection with the remote research stations has yet to be realized. Also unmentioned is that SARI’s administration still fails to recognize that they must increase their overall budget allocation for ICT support — and that until they do, this mission-critical component of a modern ag research institution will probably continue to suffer. Looking forward… Also unmentioned in the story cited above is that GARNET, like all relatively new NRENs in Sub-Saharan Africa (SSA) needs greater investment support from the major players in the global donor space. That’s because, like most NRENs in SSA, GARNET is only able to buy small bits of dark fiber and for the most part has to purchase the balance of their connectivity from large companies focused on selling their valuable resource to the highest bidder for use by cellular providers (like MTN or Vodaphone Ghana). As Steven Song, a Google Fellow with an appointment at the National Resources Startup Center at the University of Oregon recently stated, “there is a growing frustration with the underutilization of national fibre backbones. Fibre networks are the deep water ports of network infrastructure yet they are still treated as commodities to maximise return on rather than infrastructure which can boost entire economies.” https://manypossibilities.net/2021/02/africa-telecoms-infrastructure-in-2020/ But, when one thinks about the enormous existing investment already present at SSA’s universities and NARS institutions (both in terms of physical infrastructure and in terms of the intellectual capacity of their faculty/researchers), it is surprising the international donor community has not yet realized if they simply made major boot-strap investments in building a fiber infrastructure within each country that NRENs could operate at low cost for their member institutions. With that action alone, they would unleash the tremendous research and development potential currently locked inside those institutions. Perhaps a pilot research project to do just that might be something USAID might consider in partnership with CSIR as it tries to move all 12 of Ghana’s research institutes into the position when they can contribute most effectively to the twin goals of economic development and food security for the country. Source: Paul Hixson, Lead Researcher for MRA 11 (Connectivity), SIL 2.0 UIUC. May 8, 2021. 192 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex V.B.4. Case Study of the Collaboration Between SIL and the CSIR-SARI Technical and Financial Support Project Courtney Tamimie, Associate Director, SIL April 28, 2021 Plant Breeding Collaboration with SARI When the Soybean Innovation Lab (SIL) project was first funded in 2013, members of SIL including Brian Diers and Randy Nelson, who were heading the plant breeding Managed Research Area (MRA), visited SARI to assess the soybean breeding efforts at that station. Based on discussions with Dr. Nicholas Denwar, the SARI soybean breeder, plans were made on how SIL and SARI would collaborate to increase the capacity of the SARI program. Over the next seven years, breeders from SIL have visited SARI at least annually, except for 2020 when travel was not advised because of the COVID-19 pandemic. During these visits, SIL breeders have toured plots on the main SARI research station and other outlying research centers in northern Ghana. Based on these visits, recommendations were provided to SARI on how the program can be improved to increase its capacity to develop and release new varieties. In addition to these visits, SIL worked with Dr. Denwar on annual work plans and reviewed reports of his work and provided feedback on these documents. In addition to the technical advice, we provided new germplasm to his program from the USDA Soybean Germplasm Collection and aided in the introduction of new germplasm from other countries in Africa and throughout the world. During the interaction between SIL and SARI, there has been good and significant progress in some areas but in other areas, it has been slow. Success in plant breeding is increased through growing and testing more experimental lines and SIL has worked with the SARI program to increase the number of plots grown and the number of experimental lines developed annually. Success in increasing the number of plots is illustrated by comparing the number of yield plots grown by the SARI soybean breeding program. During 2013 to 2014, when SIL started, SARI averaged only 350 yield plots per year and during 2019 and 2020, they averaged 3,500. This order of magnitude increase in plots has dramatically improved their ability to identify high yielding, adapted varieties that are suitable for their region. In contrast, less success has occurred in increasing SARI’s ability to develop new experimental lines. There were no experimental lines developed over the 2013-2014 seasons and only an average of six over 2019-2020. The program needs to develop hundreds of experimental lines annually and the problems that have hindered their ability to generate breeding lines have not been fixed. Despite the lack of development of new lines, the increase in capacity to grow plots has allow SARI to identify experimental lines adapted to northern Ghana that were developed by other breeding programs and in 2020, SARI tested six experimental lines in on-farm trials for potential release. 193 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Pan-African Soybean Variety Trial (PAT) Collaboration with SARI SARI has also collaborated with SIL since 2018 on the implementation of the Pan- African Soybean Variety Trial (PAT) program, which fast-tracks the identification, registration, and release of new, high￾yielding materials for African growers. In the short time since the PATs have been implemented in Ghana, SARI has advanced PAT materials to on-farm trials, a necessary component of the registration pipeline, to bring new materials to Ghanaian growers. The PATs have also brought in private sector partners including Adom Seeds and the West Africa Rice Centre (WARC) to collaborate with SARI on the PATs. Attached to this write-up is the latest Pan-African Trial (PAT) Industry Extension Report from Ghana, both the agronomic summary and the protein and oil concentration analysis. The table below (Table 6) from the PAT agronomic Industry Extension Report shows that 2 varieties from Zambia and Colombia (private sector) out-perform the locally, newly-released variety Favour, followed by a variety from private sector seed company partner Seed Co based in Zimbabwe. These findings highlight the importance of the PAT program, which sheds light on the performance of local materials by comparing them with other commercial soybean varieties in an effort to identify high-yielding materials that outperform local checks. 194 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Adaptation of Soybean to Tropical Climates for Smallholder Farmers SIL supported research at SARI focused on soybean adaptation to tropical climates, such as Ghana. The collaboration with SARI on this effort generated two academic publications, including:  Molecular tools for detecting Pdh1 can improve soybean breeding efficiency by reducing yield losses due to pod shatter: https://link.springer.com/content/pdf/10.1007/s11032-019-0935-1.pdf  The effects and interaction of soybean maturity gene alleles controlling flowering time, maturity, and adaptation in tropical environments: https://link.springer.com/article/10.1186/s12870-020- 2276-y The second publication referenced above identified major and minor effect alleles of soybean genes that can be combined to control Days to Flowering, Days to Maturity, and plant height in short day tropical environments in Ghana. These phenotypes contribute to adaptation to a low latitude environment that can be optimized in a soybean breeding program with targeted selection of desired allele combinations. The knowledge of the genetic control of these traits will enhance molecular breeding to produce optimally adapted soybean varieties targeted to tropical environments. 195 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Agronomics Collaboration with SARI Soybean cultivation in Sub-Saharan Africa has numerous challenges including limited soybean varieties, poor seed quality, limited knowledge in agronomic and production practices, limited knowledge in pest management, and poor soil fertility management. Northern Ghana consisting of the savanna agroecology is the hub for cereal production, including soybean. However, soybean yields are low, on average between 800 kg/ha among smallholder farmers. The Soybean Innovation Lab (SIL) was launched in 2013 to provide the science and technical knowledge necessary for smallholder farmers to share in rising demand for soybean as a cash crop and as well address their nutritional needs. Hence, a critical component of SIL’s mandate is to improve soybean yields through proper agronomic and production practices to increase farmer incomes and improve household nutrition. Over the years, SIL adopted Soybean Management by Application of Research and Technology (SMART) farm concept as a comprehensive approach to improving soybean yield. This is accomplished through seed germination test, planting methods, soil fertility test and nutrient stewardship, and plant protection methods. Since 2013, SIL partnered with SARI to conduct soybean agronomic trials across the savanna agroecology to demonstrate the benefits of SMART Farming. Scientific data generated suggest farmers can up yields to well over 2.0 tons per ha, effectively tripling average yields. Through the partnership with SARI, over 1,200 farmers and 20 in-country soybean practitioners are informed with improved soybean agronomic and production practices. 196 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Attached to this write-up is the latest SMART Farm Industry Extension Report from Ghana (table referenced below). • Agronomic and economic assessment of input bundle of soybean in moderately acidic Savanna soils of Ghana  Awuni, G. A., Reynolds, D. B., Goldsmith, P. D., Tamimie, C. A., & Denwar, N. N. (2020). Agronomic and economic assessment of input bundle of soybean in moderately acidic Savanna soils of Ghana. Agrosystems, Geosciences & Environment, 3(1), e20085. Economic impact analyses with SARI SIL has collaborated with SARI’s agricultural economics team to conduct analyses to understand: - Farmers’ Response to COVID-19 Disruptions in the Food Systems in Northern Region of Ghana: The case of crop land allocation decision: o Martey, E., Goldsmith, P.D., & Etwire, P.M. (2021). Farmers’ Response to COVID-19 Disruptions in the Food Systems in Northern Region of Ghana: The Case of Crop Land Allocation Decision. European Journal of Development Research. Under Review. 47 pages. 197 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV - Factors that drive peer dissemination of agricultural information: Evidence from northern Ghana o Etwire, P.M., Martey, E. & Goldsmith, P.D. (2021). Factors that drive peer dissemination of agricultural information: Evidence from northern Ghana. Development in Practice. Accepted. 37 pages. - Heterogeneous Demand for Soybean Quality o Martey, E. & Goldsmith, P.D. (2020). Heterogeneous Demand for Soybean Quality. African Journal of Agricultural and Resource Economics. http://afjare.org/wp-content/uploads/2020/04/3.-Martey- Goldsmith.pdf. 15.1: 27-50. - Assessing the performance of regional soybean prices in Ghana o Martey, E., Gatti, N., & Goldsmith, P. D. (2020). Assessing the performance of regional soybean prices in Ghana. International Food and Agribusiness Management Review, 1-16. ICT Collaboration with SARI SIL developed the ICT Health Checkup, which takes quantitative performance measurements over four major areas: connectivity, infrastructure, intranet services, and staff skills. Working with the ICT Health Checkup, SARI was able to quantify their connectivity gaps and remedy these by switching to a new ISP provider, GARNET. SARI now enjoys a 155 Mbps connection across the entire station at Nyankpala, a 75x increase over their initial connection speed; and with this change, their cost per Mbps of bandwidth traffic was reduced by 93% from what it had been in 2018. The SARI journey highlights the value of working with the local NREN in your country and, equally important, using a quantitative assessment tool like the ICT Health Checkup to accurately measure existing connectivity successes and challenges. 198 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV • The ICT Health Checkup Tool: Assessing Connectivity of the National Agriculture Research System (NARS): Hixson, P., Goldsmith, P.D. & Smith, T. (2020).  The ICT Health Checkup Tool: Assessing Connectivity of the National Agriculture Research System (NARS). African Journal of Food, Agriculture, Nutrition, and Development. https://doi.org/10.18697/ajfand.93.19385. 20 (5): 16447-16470. 199 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex V.B.5. Major Phases And Events In The Implementation Plan of the CSIR-SARI Technical And Financial Support Project Phase I. The original implementation model (2014): In the original implementation plan for the USAID￾funded CSIR-SARI Technical and Financial Support Project, it was expected the project would have: 1) an independent project implementation unit that would include an independent project manager, a full￾time M&E coordinator, and a full-time accountant attached to the project by CSIR-SARI as part of is in￾kind contribution to the project ; 2) include a budget line to ensure the correct operation of this unit; and 3) a steering committee to serve as an interface between the project and the CSIR-SARI management board as was the norm in other large donor-funded projects at CSIR-SARI. Phase II. The first revision and review of the project implementation model for making CSIR-SARI a Center of Excellence (January 15, 2015-March 2017): Neither the project management unit nor the steering committee were created during the first 26 months of the project. Instead, the day-to-day management of the project was overseen by a series of acting project managers and part-time project M&E coordinators. During this time, there was no designated accountant. Although the acting project manager and M&E coordinators’ roles were recognized in the project, they were expected to execute these roles on top of their other obligations to the Institute. Because of this, the CSIR-SARI director and his staff continued to have advisory and decision-making authority for the project, as well as the responsibility of approving project work plans and budgets and monitoring activities on a regular basis. Phase III: Participatory review of and reorganization of the project implementation model to the creation of a COE model for the three priority crops (January 2017-March 2020): Concerned some of the delays and management issues the project was experiencing in 2015 would to affect the impact of some of the other USAID-funded projects (like ATT) as well as the new GoG PFJ, USAID/Ghana requested ATT conduct a participatory assessment of CSIR-SARI that would be comparative with the 2012 baseline capacity assessment conducted by Africa Lead.224 This assessment was conducted by an ATT consultant from its sub-contractor ISU and involved all of the CSIR-SARI staff. 225 This assessment found conclusive evidence that most of the recommendations of the 2012 assessment Africa Lead assessment were yet to be implemented due to lack of core funding from GoG among other institutional and managerial weaknesses at CSIR-SARI. Because of this, the report concluded:226 SARI was not in a position to “tackle all of these immediate management issues and deficits;” but “it is [however] possible to focus on establishing a ‘core of excellence; within SARI that involves a core team of researchers in the maize and soy and rice seed value chains, along with communications and business development revenue 224 Michael Dockery. 2016 (April). Email from Stephen Nutsugah to Nicholas Denwar, Gloria Boakyewaa , Wilson Dogbe, and Gilbert Nachim. Subject “SARI Assessment Report—ISU” (Dr. Dileep Guntuku ). Request for review.” 225 Dileepkumar Guntuku. 2016 (March). Assessment of the Institutional Capacity and Needs of the Savanna Agricultural Research Institute (SARI. Nyankpala, Northern Region, Ghana. Ames, Iowa: IFDC and ISU. (Final version with corrections dated April 25, 2016). Robert Kwasi Owusu. 2016 (June). “Invitation to core of Excellence Planning Meeting.” Nyankpala: CSIR-SARI and the ATT Project. Invitation to a joint meeting with the ATT Project on June 15, 2016 review the results of the needs assessment. CSIR-SARI. 2016 (June 15). Planning Meeting of “Core of Excellence at SARI.” Program. Speakers: Stephen Nutsugah, Samson Konlan, Michael Dockery, Dileep Kumar Guntuku, David Benson, Lulu Rodriguez. Not final (draft).CSIR-SARI Direct Support Project. 2017 (estimated). Implementation Arrangements (for the COE). One page document.Dileepkumar Guntuku. 2017. Core of Excellence at SARI. Status Report 2013 (Powerpoint) with inputs from ISU, ATT, USAID and SARI teams. 226 Dileepkumar Guntuku. 2016 (March). Assessment of the Institutional Capacity and Needs of the Savanna Agricultural Research Institute (SARI. Nyankpala, Northern Region, Ghana. Ames, Iowa: IFDC and ISU. (Final version with corrections dated April 25, 2016). Pg.10x. 200 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV generation leaders;” and that “this core team [be] held to a higher level of performance than what was observed from current operations.” The “immediate target [of the ATT-sponsored assessment was] to “improve the quantity and quality of rice maize and soy breeder seed to be made available to ATT and its network of private seed producers” so that “The proposed next steps [would] be undertaken as part of ISU’s subcontract with ATT [in order to] “improve the quantity and quality of rice, maize and soy breeder seed to be made available to ATT and its network of private seed producers.” It was, however, expected that the “finalized work plan (with budget) for [the] core of excellence [would] then be implemented from USAID direct support of $5.5 M to SARI” (a.k.a. the CSIR￾SARI Technical and Financial Support project being evaluated in this document). To jumpstart the project’s shift from a Centre of Excellence to a COE, the project sponsored six faculty for a two-week exchange visit to ISU and the UIUC.227 In the course of the exchange visits, the CSIR￾SARI team developed a revised plan of action for the project that focused on the achievement of five outcomes and 13 sub-outcomes to achieve the ten outputs in the original results framework (Annex V.B., Text Box 1). This shift brought about three key changes in the project’s implementation structure, including:228  The recruitment of a smaller sub-group of CSIR-SARI researchers organized and supervised as eight semi-autonomous component teams;229  The recruitment of an independent project manager, an independent M&E coordinator, and accountant ; and  The creation of an independent advisory board experts for the COE comprised of leading international experts in plant breeding and seed production for the three priority crops (rice, soybean, and maize). In July 2017, the project—with support from ATT—organized an official one day launch of the new COE model and signed an initial agreement with ISU to provide the technical assistance CSIR-SARI needed to execute the new model. Due to contracting issues, however, the MOU with ISU was never fully executed which “resulted in little or no handholding”230 for the project . In January 2019, the 227 Letter from Stephen Nutsugh to Dr. Guntuku confirming CSIR-SARI faculty participation in COE training from March 15-28, 2017. SARI COE Records. CSIR-SARI. 2017. List of participants the Proposed SARI trainings at ISU and UIUC. Nyankpala: CSIR/SARI. CSIR—SARI. 2017. Training Program to Enhance Capacities of SARI Staff. Nyankpala: CSIR-SARI. (Note a signed note from the Director confirmed the transfer of this along with the budget to ISU in February 2017).Dileepkumar Guntuku. 2017.Training Program for Core of Excellence (CoE) Leaders of Sari. March 15, 28, 2017. Hosted by the Seed Science Center, College of Agriculture and Life Sciences. Iowa State University. Training Calendar.Dileepkumar Guntuku .2017 (March). Core of Excellence at SARI . Status report. With inputs from ISU and SARI COE Team Leaders. Ames: ISU. Dileepkumar Guntku. April 19 2017. Email to Nutsugah, Dogbe, Denwar, Boakwaa, Kutah, Appiah at SARI and Lin Misra at ISU and Samson Konlan at USAID). Subject: (1) Meeting with Samson Konlan during his visit on ISU; and (2) proposed course of action. 228 See Annex V.A. for a more complete list of trainings and supports that were associated with the COE model for the project. 229 Components 1-3 focused on building the breeding programs, extension, and seed operations production for the three priority crops being supported by the project (maize, rice, soybean) (as envisioned in Components 1.2, 2.1, 2.3, and 2.4 in the original project plan [Figure 1]). Component 4 focused on accelerating the project’s original commitment to scaling up CSIR￾SARI’s research and dissemination of new technologies to promote integrated soil fertility (Component 2.2) . Component 5 focused on strengthening CSIR-SARI’s commercial processes and contacts in ways that would create the types of predictable, well-managed revenue streams that CSIR was asking all of its institutions to develop in order to be more self-sufficient (Component 3.2). A Component 6 team focused on building the global management capacity of CSIR-SARI and its senior research including SARI’s capacity to oversee infrastructure development, renovation, and maintenance (Components 1.2 and 1.3). A Component 7 team focused on helping CSIR-SARI develop and the type of institution-wide M&E system it needs for results-based programming (Component 3.3). Component team 8 focused on building CSIR-SARI’s ICT systems (Component 3.1). 230 Wilson Dogbe. 2018.. Update on the SARI Core of Excellence. PowerPoint. 201 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV project signed a second technical support contract with AgTILs to provide technical backstopping during the project’s last year (March 2019-December 2019). 202 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV Annex V.B.6. Summary Recommendations and Sub-recommendations for CSIR-SARI and USAID/Ghana Adopted by the ET for each EQ. 231 No. Recommendation & Sub-Recommendation for Future USAID/Ghana-funded Projects and CSIR-SARI CSIR￾SARI USAID/ Ghana For CSIR-SARI 1. Project Management Systems. 1.1. New donor-funded project designs. Ensure any future donor￾funded large projects (e.g., projects that represent more than 15 percent of the institute’s budget): X 1.1.1. Start-up and launch. Have a formal launch in the first few months of a project to ensure CSIR-SARI’s administrators, staff, technicians, and other key stakeholders have a good understanding of the project’s plan, work plan, and the donor’s rules and regulations for M&E, finance, and procurement. X 1.1.2. Staffing. 1) Hire a manager with a strong management background, belonging to a reputable institute, with demonstrated experience in successful capacity-building of national agricultural research institutes to ensure appropriate mentoring; 2) use that person to monitor/coach the CSIR-SARI staff person who takes over the role once the recruited manager’s tenure expires; and 3) provide project managers with full-time salaries or top-offs; 4) acknowledges the time, level of effort, and contributions of all institute staff involved in new donor-funded projects on their annual CSIR-SARI evaluations; and 5) if the CSIR-SARI staff are not eligible for top-offs (under new projects) consider what other compensations these individuals might be scheduled to receive (e.g., training, publication support, or sabbatical opportunities). X 1.1.3. Project steering committee. Have an empowered steering committee that meets regularly and includes a representative from the donor. X 1.1.4. Mid-term evaluation. Execute a mid-term evaluation to provide a participatory mechanism for CSIR-SARI and the donor to address any management, implementation, or governance issues that emerge in the first half of the LOA while there is still time to correct them. X 1.2. Internal management, finance, and governance systems. 1.2.1. CSIR-SARI management board. Strengthen the audit function of the CSIR-SARI management board. X 1.2.2. CSIR-SARI IMC. Ensure that all of the managers of large donor-funded have a seat on the institute’s IMC. X 1.2.3. Mentoring. Require any future donor-funded project that is expected to mentor CSIR-SARI on capacity to sign an MOU clarifying the type of mentoring support to be provided and how the mentorship activities will be monitored and reported to the CSIR-SARI administration, IMC, and board, as well as a representative of the project’s donor. X 1.2.4. Accounting. Continue to tighten the institute’s accounting system by using appropriate accounting software and ensuring regular audits at all levels by an independent auditor. X 2. Infrastructure and Equipment. Develop better systems for cost￾sharing or generating user fees to support the cost of routine maintenance and updating of infrastructure and equipment. X 231 (x)= times a category of recommendation was mentioned in FGDs/KIIs. 203 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV No. Recommendation & Sub-Recommendation for Future USAID/Ghana-funded Projects and CSIR-SARI CSIR￾SARI USAID/ Ghana 3. Commercialization. 3.1. Complementary private-sector investment. Advocate for private-sector investments in seed production infrastructure and then partner with CSIR-SARI to carry out research for them to replicate and commercialize. X 3.2. Supervision. Put the commercialization unit under the direct supervision of the CSIR-SARI director and management board. X 3.3. Commercialization unit budget line. Create a line budget within CSIR-SARI for commercialization linked toward the achievement of the unit’s business plan. X 3.4. Commercialization unit monitoring. Create a set of robust indicators that the institute’s top management and boards can use to track the unit’s progress toward the execution of its business plan. X 3.5. Code of conduct. Develop and enforce a professional code of conduct that clarifies CSIR-SARI’s support for plant breeders’ rights and any restrictions on research scientists and technicians setting up unrestricted side businesses to sell seed or other products. X 4. M&E. 4.1. New project PITT. Require all new CSIR-SARI projects to include a standard PITT in all of their annual reports that includes both standard indicators (required by the donor) as well as custom indicators to track the achievement of their principal expected results and outputs. X 4.2. New project M&E mentoring. Require all new CSIR-SARI projects to anticipate the need for periodic support (on an annual basis during the LOA) to help the project understand the importance of M&E, setting reasonable targets, and measuring progress toward the execution of these targets in their annual report and specific donor’s rules about changing indicators and targets over the LOA. X 4.3. New project budgets. Require all new CSIR-SARI projects to have a dedicated M&E budget that must be returned to the donor if it is not spent and that has no possibility of being converted into funds that support other activities. X 4.4. M&E unit set-up. Designate one M&E coordinator and one M&E focal point for each field station, and encourage CSIR-SARI’s new and existing donor-funded projects to help build their capacity through consistent mentoring by a professional M&E specialist (or contractors). X 4.5. M&E mainstreaming. Provide appropriate M&E baseline training to all senior and junior staff and administrators, and utilize online short courses and in-house certification programs to help new and existing staff improve their proficiency. X 5. Technology dissemination and scale-up. 5.1. Co-execution. Encourage new and existing donor-funded projects that collaborate with CSIR-SARI in northern Ghana to co-fund and co-execute joint technical and mentoring activities on issues like pest monitoring, soil mapping, training, on-farm trials, and extension. X 5.2. CSIR-MoFA collaboration. Identify ways other donor-funded projects can help scale up CSIR-SARI’s successful collaboration with MoFA for AEA training, field trials, and field days. X 204 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV No. Recommendation & Sub-Recommendation for Future USAID/Ghana-funded Projects and CSIR-SARI CSIR￾SARI USAID/ Ghana 5.3. MOUs. If projects decide to co-execute or collaborate, encourage them to formalize this collaboration through MOUs, and monitor and report on their collaboration with CSIR-SARI in order to avoid duplication and strengthen synergies. X 5.4. Partner de-briefings. Ensure the regional MoFA and PPRSD staff (as well as any donor-funded project they work with) get regular debriefings on CSIR-SARI’s current research endeavors and receive all annual reports and work plans connected with any joint programs they support. X 5.5. Training. Pilot test the feasibility of offering a limited number of fee-based training programs for key program partners (PPRSD, MoFA, NGOs, and donor-funded projects). X 6. ICT. 6.1. ICT repository. Develop a central online repository for all CSIR￾SARI’s ICT materials over the last ten years so they can be easily scaled up to new and existing projects. X 6.2. ICT monitoring. Develop gender-sensitive assessment tools for tracking the impact of different outreach methodologies (e.g., portable video communication tools and community-based theater). X 6.3. ICT baseline. Conduct a second baseline diagnostic of CSIR-SARI’s internet connectivity to assess the cost of: 1) extending internet service to its field stations; 2) making the system (including online library and website) more user-friendly; 3) lowering the routine operating costs; and 4) sub-contracting some basic maintenance, website, and database updates. X 6.4. ICT management. Identify ways that CSIR-SARI can generate the funds it needs to support the recurrent costs of its internet access, website, and documentation/database library since these costs are not covered by the GoG’s core budget for the institute. X 6.5. Advocacy. Strengthen the capacity of CSIR-SARI’s M&E and IT units to backstop its scientists in dealing with the media, publishing their own data and project M&E data, and using and the institute’s M&E data and research results for advocacy at the national level. X 6.6. Collaboration. Consider ways USAID-funded projects and innovation labs can backstop these research, M&E, and advocacy efforts through their existing budgets. X 7. Gender. 7.1. New CSIR-SARI project designs. 7.1.1. New project results frameworks. Activities with gender mainstreaming considerations should be required to include an IR or Sub-IR on gender and youth in the results framework. X 7.1.2. New project GIPs. Require any new projects with gender mainstreaming objectives to design and submit a GIP within a determined timeframe similar to the procedure for a MEL plan. New projects should also include in their staffing at least one designated gender and youth point of contact. New projects’ budgets should also make room for gender and youth activities and corresponding gender-related indicator targets for all core activities. New projects should also explore ways these GIPs can be mainstreamed by ensuring they comply with, are integrated into, and tracked and reported on as part of the institution’s strategic planning process. X 205 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV No. Recommendation & Sub-Recommendation for Future USAID/Ghana-funded Projects and CSIR-SARI CSIR￾SARI USAID/ Ghana 7.1.3. CSIR-SARI gender indicators. To facilitate a more integrated approach to gender, CSIR-SARI needs to identify a limited number of disaggregated indicators by gender and region but be generic enough to feed into the gender requirements of specific donors like USAID. X 7.1.4. Staff and budgets. Require large new project designs to: 1) have a qualified staff person serve as a gender and youth focal person; and 2) ensure there is a gender budget line to support appropriate technical backstopping, studies, and training. X 7.1.5. Evidence-based strategies. Encourage new project designs to support evidence-based strategies for women by linking the monitoring data from specific project’s and CSIR-SARI’s institute-wide M&E system to the design of new and adjustment of existing CSIR-SARI and project-specific research and commercialization strategies. X 7.2. Outreach and collaboration with MoFA on dissemination. Encourage all the associations and producer groups MoFA and CSIR￾SARI collaborate with on demonstration and adaptive trials to have women in leadership positions to strengthen the communication with the women farmers. X 7.3. Staffing and leadership training. 7.3.1. Student internships. Facilitate female university students doing internships and theses connected with SARI’s technical research or collaborating with MoFA in conjunction with the GIP. X 7.3.2. Management training. Facilitate women’s access to long- and short￾term management training in order to capitalize on the growing number of women working as SARI technicians in conjunction with the GIP. X 7.3.3. Staffing recruitment and retention. Set targets for hiring and retention of women staff and technicians in conjunction with the GIP. X 7.3.4. Core staff. Appoint one qualified CSIR-SARI staff person (and recognize this appointment by their official level of effort letter, which is the basis for their annual evaluation) to serve as the institute’s gender coordinator, and assign one staff person in each field station to serve as the station focal person. X 7.3.5. Inter-donor coordination and support. Pilot test the concept of a donor working group on gender to advise the GIP process and ensure appropriate coordination between the different international donors that support CSIR-SARI projects that include gender mainstreaming. X For USAID A. Pre-conditions for large budget transfers. Make receipt of the large initial budget transfer associated with a G2G capacity-building grant conditional on the beneficiary institution having a robust management and steering committee structure in place by the fourth month of the project, where a baseline assessment identified the need for major management reforms and the institution requested USAID/Ghana assist it in making these reforms to improve its organizational capacity. X B. Collaboration with other USAID/Ghana-funded initiatives. Require future G2G projects USAID/Ghana supports in northern Ghana to sign a detailed MOU with any other USAID/Ghana-funded project that is expected to mentor it (like ATT that mentored this project) that clarifies the type of mentoring support to be provided and how the mentorship activities will be reported to USAID/Ghana. X 206 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV No. Recommendation & Sub-Recommendation for Future USAID/Ghana-funded Projects and CSIR-SARI CSIR￾SARI USAID/ Ghana C. Mid-term evaluations. Require new G2G projects to include a detailed description of, and designated budget for, a mid-term evaluation to provide a forum to address any management, implementation, M&E, reporting, or governance issues that emerge in the first half of the project. X D. Budget transfers. Consider a more flexible system for forwarding money to G2G projects than the monthly Imprest system used under this project, or provide intensive mentoring to the beneficiary institute in these systems if it has never used them. XX E. Fee-based training. Encourage future USAID-funded projects in northern Ghana to access fee-based training programs for their staff and/or support trainings for project partners. X F. Collaboration with MoFA. Identify ways future USAID-funded projects can help scale up CSIR-SARI’s successful collaboration with MoFA for AEA training, field trials, and field days. X G. ICT internet connectivity. 1) Ask new USAID/Ghana-funded G2G projects to include an indicator of internet connectivity, like the four-variable internet health indicator SIL pilot tested on this project, in future baseline capacity assessments; 2) track any strategy developed to address internet-based issues identified through one or more custom indicators in the PITT; 3) if a baseline diagnostic indicates the institution being targeted by the G2G grant suffers from weak connectivity or systems for maintaining, repairing, or keeping the system user-friendly, encourage G2G projects to include a budget to explore mentoring from local internet cooperatives and COEs (e.g., GARNET, SIL’s ICT Connectivity Project, and/or the computer scientists in the CSIR Electronics Unit) that can lower the initial costs of installation and maintenance and help G2G partners avoid costly mistakes that are hard to fix; and 4) identify ways some of the routine costs of maintaining the system can be shared with other donors who support the institution. X H. Gender issues in design. Strengthen the guidance USAID/Ghana provides to GoG institutions that apply for G2G grants. This guidance should: 1) require new project designs to include gender and youth mainstreaming as a project sub-purpose/sub-IR in the results framework; and 2) identify some of the most important documents on the USAID websites programs can use to help incorporate gender into the initial design of their GIPs (including issues related to staffing, budget, governance, and M&E). X I. Gender issues in the PIL. If a new G2G project plan does not include a solid plan for GIP, USAID/Ghana should require this in the PIL as it did on another G2G project executed at the same time as this one. X Source: Annex IV.E. 207 | CSIR-SARI FINAL PERFORMANCE EVALUATION REPORT USAID.GOV U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523