December 2015 Revised January 2016 This publication was produced at the request of the United States Agency for International Development. It was prepared by Donald Greenberg and Christopher Root of Social Impact. African Cocoa Initiative Final Performance Evaluation Report African Cocoa Initiative Final Performance Evaluation Report December 2015 Revised January 2016 AID-OAA-I-10-00013/AID-OAA-TO-14-00021 DISCLAIMER: The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ii CONTENTS Executive Summary ..................................................................................................................................... ix Evaluation Purpose & Evaluation Questions ................................................................................................ 1 Evaluation Purpose ................................................................................................................................... 1 Evaluation Questions ................................................................................................................................ 1 Evaluation Introduction .............................................................................................................................. 2 Methods and Limitations ............................................................................................................................... 3 Methodology for Quantitative and Qualitative Data Collection ................................................................. 3 Analytical Approach .................................................................................................................................. 4 Limitations of the Evaluation Methodology ............................................................................................... 4 Evaluation Team ....................................................................................................................................... 5 Project Background ....................................................................................................................................... 6 African Cocoa Initiative Justification and Overview .................................................................................. 6 Components I-IV Background, Description and Budget .........................................................................11 Component I: Establish and Strengthen National Public Private Partnership (PPP) Platforms .........11 Component II: Address farm productivity constraints through improved planting material ................13 Component III: Enhance public and private sector extension and farmer training services ...............16 (i) Component Three Extension Activities ........................................................................................16 (ii) Component Three Ghana Quality Initiative Sub-Activity .............................................................18 Component IV: Market Driven Input Services .....................................................................................19 (i) Spray Service Providers Sub-Activity ..........................................................................................20 (ii) Input Finance Sub-activity ...........................................................................................................20 Findings, Conclusions, & Recommendations .............................................................................................23 I. Overall Findings, Conclusion and Recommendations .........................................................................23 Overall Findings ..................................................................................................................................31 Overall Conclusions ............................................................................................................................31 Overall Recommendations ..................................................................................................................31 II. Findings, Conclusions and Recommendations for Food Security ......................................................32 III. Findings, Conclusions and Recommendations for the four components ..........................................37 Component I: Establish and Strengthen National Public Private Partnership (PPP) Platforms .........37 Component II: Address farm productivity constraints through improved planting material ................42 Component III: Enhance public and private sector extension and farmer training services ...............48 (i) Component Three Extension Activities ........................................................................................48 (ii) Component Three Ghana Quality Initiative Sub-Activity .............................................................53 Component IV: Market Driven Input Services .....................................................................................59 (i) Spray Service Providers Sub-Activity ..........................................................................................59 (ii) Input Finance Sub-activity ...........................................................................................................62 iii III. Cross-Cutting Issues ..........................................................................................................................68 Findings, Conclusions and Recommendations regarding other Cross-cutting issues ........................68 Land Tenure .....................................................................................................................................68 Child Labor .......................................................................................................................................70 Environment and Human Health ......................................................................................................72 Farm Labor .......................................................................................................................................73 Design and Management .................................................................................................................74 Annexes ......................................................................................................................................................82 Annex I: Evaluation Statement of Work ..................................................................................................82 I. BACKGROUND INFORMATION .....................................................................................................82 A. IDENTIFYING INFORMATION ...................................................................................................82 B) DEVELOPMENT CONTEXT .......................................................................................................83 1. Problem or Opportunity Addressed by the Project/Activity being Evaluated ..................................83 2. Target Areas and Groups ................................................................................................................83 C) INTENDED RESULTS OF THE PROJECT/ACTIVITY BEING EVALUATED ............................84 D) APPROACH AND IMPLEMENTATION ......................................................................................85 E) DOCUMENTS .............................................................................................................................87 II. EVALUATION RATIONALE .............................................................................................................88 A. EVALUATION PURPOSE AND INTENDED USERS ................................................................. 88 B. AUDIENCE AND INTENDED USES ...........................................................................................88 C. EVALUATION QUESTIONS .......................................................................................................88 III. EVALUATION DESIGN AND METHODOLOGY .............................................................................89 A. EVALUATION DESIGN ...............................................................................................................89 B. DATA COLLECTION METHODS................................................................................................90 1. Key Informant Interviews .............................................................................................................90 2. Focus Group Discussions ...........................................................................................................91 C. DATA ANALYSIS METHODS .....................................................................................................91 D. METHODOLOGICAL STRENGTHS AND LIMITATIONS ..........................................................93 IV. EVALUATION PRODUCTS .............................................................................................................94 A. DELIVERABLES .........................................................................................................................94 B. REPORTING GUIDELINES ........................................................................................................95 V. TEAM COMPOSITION .....................................................................................................................96 VI. EVALUATION MANAGEMENT ........................................................................................................96 A. LOGISTICS ................................................................................................................................. 96 B. SCHEDULING .............................................................................................................................97 C. LOE .............................................................................................................................................97 Annex II: Data Collection Instruments .....................................................................................................98 Annex III: Sources of Information ..........................................................................................................108 Annex IV: ACI Interviewees ..................................................................................................................112 iv Annex V: ACI Budget ............................................................................................................................115 Annex VI: Disclosure of any conflicts of interest ...................................................................................116 v TABLES AND FIGURES Figure 1: Key Informant Interviews ............................................................................................................... 3 Figure 2: Focus Group Discussion Participants............................................................................................ 3 Figure 3: ACI Sub-Grants .............................................................................................................................. 7 Figure 4: African Cocoa Initiative Logic Model.............................................................................................. 9 Table 1. Summary of Outcomes, Activities, and Results ............................................................................11 Table 2. Ghana PPP Technical Working Groups........................................................................................13 Table 3. CDI PPP Technical Working Groups ............................................................................................13 Table 4. Summary of Outcomes, Activities, and Results ............................................................................13 Table 5. Seed Garden Activity ....................................................................................................................15 Table 6. Summary of Extension Outcomes, Activities, and Results ...........................................................16 Table 7. Summary of GQI Outcomes and Results .....................................................................................18 Table 8. Summary of Outcomes and Results, Spray Service Providers Sub-activity................................. 20 Table 9. Summary of Outcomes and Results, Input Finance Sub-activity .................................................20 Table 10. ACI ICS Activity Risk-Sharing Arrangements .............................................................................22 Table 11. Findings, Conclusions, and Recommendations Matrix ...............................................................23 Figure 5: Food Insecurity and Cocoa Production, Côte d’Ivoire ................................................................. 34 Figure 6: Food Insecurity and Cocoa Production, Ghana ...........................................................................34 Table 12. Trial Shipment 2014/5 Season, OFFA ........................................................................................55 Table 13. Ghana Premium Distribution for Fine Flavor Cocoa, 2014/5 Season .........................................55 Table 14. Advans Cocoa Input Credit Scheme 2013/4 to 2015/6...............................................................64 Table 15. SCOOPBAD Fertilizer Financing Program .................................................................................65 Table 16. Planned vs. Actual Awards .........................................................................................................75 Figure 7: African Cocoa Initiative Results Framework ................................................................................77 Table 17. African Cocoa Initiative Table of Indicators ................................................................................78 Table 1: Partners to be Interviewed ............................................................................................................90 Table 6: Evaluation Phases, Timeframe, Activities and Sub-Deliverables ...................................................97 Table 18. WCF/ACI Budget: Original versus Committed .........................................................................115 vi ACRONYMS 2QC Côte d’Ivoire National Cocoa Policy ACBWG African Cocoa Breeders Working Group ACI African Cocoa Initiative ANADER Agence Nationale d’Appui au Développement Rural BFS Bureau for Food Security BUREX-CI Bureau d'Expertises Côte d’Ivoire CA CocoaAction CAA Cocoa Abrobopa Association CAADP Comprehensive Africa Agriculture Development Programme CCC Conseil Café-Cacao CDI Côte d’Ivoire CFI Cocoa Fertilizer Initiative CHED Cocoa Health and Extension Department CICC Conseil Interprofessionnel du Cacao et du Café CLCCG Child Labor Cocoa Coordinating Group CLP Cocoa Livelihoods Project CMC Cocoa Marketing Company CNRA Centre National de Recherche Agronomique COCOBOD Ghana Cocoa Board CoCoShe Cocoa, Coffee, and Shea-Nut Farmers Association COPAL Alliance of Cocoa Producing Countries CPQP Cocoa Productivity and Quality Program CRIG Cocoa Research Institute of Ghana CSSV cocoa swollen-shoot virus FAAB Farming as a Business FBS Farmer Business School FF fine flavor (cocoa) FGD focus group discussion FI financial institution FTF Feed the Future GAP good agricultural practices GFFCP Ghana Fine Flavor Cocoa Project GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit GoCDI Government of Côte d’Ivoire GoG Government of Ghana GQI Ghana Quality Initiative ICCO International Cocoa Organization vii ICRAF International Center for Research in Agroforestry ICS input credit scheme IDH Dutch Sustainable Trade Initiative IITA International Institute of Tropical Agriculture KII key informant interview LBC licensed buying company M&E monitoring and evaluation MFI microfinance institution MPI Office of Markets, Partnerships and Innovations NARS National Agricultural Research System NGO non-governmental organization NRA national research agency OFFA Offinso Fine Flavor Association PETWE Productivity and Entrepreneurship Training for Women's Empowerment PPE personal protective equipment PPP public-private partnership PVR participatory variety research QCC Quality Control Company RA Rainforest Alliance RFA Request for Application SCOOPSBAD Société Coopérative Simplifiée Binkady d’Ake Douanier SDF Set Up, Deliver, and Finish SI Social Impact SPD Seed Production Division SSP spraying service provider STCP Sustainable Tree Crops Program TA technical assistance TNS TechnoServe TOR terms of reference TOT training of trainers TWG technical working group UNDP United Nations Development Programme USAID United States Agency for International Development USDA United States Department of Agriculture WCF World Cocoa Foundation WFCL worst forms of child labor viii ACKNOWLEDGEMENTS There are many people and organizations without whom this evaluation would not have been possible. The team would like to thank all parties who willingly contributed information to this evaluation in Ghana, Côte d’Ivoire, and across the United States. The team is also immensely grateful for the logistical support provided by the World Cocoa Foundation and Social Impact’s Ghana office. Mathieu Gaspard’s interpretation was essential for the KIIs and FGDs conducted in Côte d’Ivoire. The team is thankful for the research support provided by Mike Pressl at SI-HQ in the final stages of the evaluation. Finally, the team would like to extend its thanks to USAID, both in Washington D.C. and in its regional missions, for its support and facilitation of the evaluation. ix EXECUTIVE SUMMARY The African Cocoa Initiative (ACI) is a five-year Cooperative Agreement activity between the United States Agency for International Development (USAID) Bureau for Food Security (BFS) and the World Cocoa Foundation (WCF), awarded on September 30, 2011. WCF is a membership organization of more than 110 companies with a common interest with USAID in raising the productivity and food security of small-scale cocoa farmers and their communities. USAID initially committed $5,000,000 to ACI over the five-year period, and leveraged more than $4,550,000 in matching commitments from 14 WCF member companies and $3,500,000 in an in-kind contribution from the Dutch Sustainable Trade Initiative (IDH). ACI aims to double cocoa productivity for 100,000 farm households in Ghana, Côte d’Ivoire (CDI), Cameroon, and Nigeria and, in doing so, raise the household income of cocoa farmers by 150–200%. This is expected to increase food security by increasing purchasing power and thus the ability to purchase foodstuffs. The evaluation addresses five central questions, summarized below. 1. How did the ACI activities contribute or likely contribute to the objectives and results as intended for the project overall and its four components? 2. How likely are the ACI activities to positively (or negatively) affect food security among cocoa farmers and within cocoa-producing communities? 3. What other benefits or drawbacks occurred as a result of the program for each component? 4. What are the most important lessons learned from implementing ACI? 5. How might the ACI program or successor initiatives be designed to more cost-effectively and sustainably achieve and measure objectives, results, and outcomes? The project had four components, described briefly below: - Component I: Establish and strengthen national public-private partnership (PPP) platforms. The PPP platforms were intended to bring public and private cocoa stakeholders to establish priorities, set plans, and mobilize resources for investing in cocoa. - Component II: Address farm productivity constraints through improved planting material. This focused on 1) the expansion of cocoa seed gardens; 2) the creation of budwood gardens that will provide true-to-type parent lines; and 3) genetic fingerprinting, which analyzes and compares genetic material from breeding, propagating, and planted cocoa trees. - Component III: Enhance public- and private-sector extension and farmer training services. This focused on 1) the training of extension agents using updated training materials and 2) improving the overall quality of Ghana’s bulk cocoa and introducing fine flavor cocoa varieties. - Component IV: Foster market-driven farming input supply services. This component had two main activities: 1) developing spray service providers (SSPs), entrepreneurs who provide safe and effective pest-control measures on a commercial basis and 2) piloting an input credit scheme for cocoa cooperatives’ innovative risk sharing and embedded service provision. The evaluation team conducted three weeks of fieldwork in CDI and Ghana in September and October 2015. ACI operates in Nigeria and Cameroon as well, but CDI and Ghana account for a large share of the world cocoa market and ACI’s budget and were therefore chosen for the evaluation. The team used semi-structured key informant interviews (KIIs) and focus group discussions (FGDs) to collect data from a range of project beneficiaries, implementers, and other stakeholders. In total the team conducted 61 KIIs and 11 FGDs with195 people. Of the 61 key informants, 12 were female, while 28 of the 195 FGD participants were female. This fieldwork was combined with review and analysis of project monitoring data as well as a literature review of cash crops and food security. x This study’s limitations include a small sample size, which affects the degree to which the results are truly representative. Triangulation was also used to verify findings. The team attempted to mitigate this by meeting with respondents in both remote and non-remote areas. The team used before-and-after questions as well as follow-up questions to attempt to infer attribution. Finally, several potential biases may come into play in the interpretation of results, including selection bias. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS Overall The evaluation found that the ACI project activities are likely to have raised farmers’ cocoa productivity and thereby increased food security (see below). These productivity improvements have occurred through the adoption of good agricultural practices promoted by the project: improved planting materials, improved extension, spraying services, and finance. A number of factors constrained this progress. The capacity-building and enabling environment strengthening initiatives of Component I and parts of Components II and III are long-term activities that should not have been expected to contribute to increased income and food security during the project period. A greater use of evidence-based policy advocacy in Component I could have accelerated progress in achieving reforms that would have laid the groundwork for increased cocoa household incomes. The lack of a proactive market facilitation approach for parts of Component II and III, and for all of Component IV, reduced the depth, scale, and sustainability of impacts. More proactive management of sub-awardees and better integration of individual components might have yielded synergies that would have had increased impacts. Finally, a greater degree of coordination and communication within the ACI team and with other cocoa project implementers would have strengthened the impact of all project activities. These findings and conclusions have led to the following recommendations: Recommendation: Adopt a market facilitation approach to pilot initiatives. Project management should have a full-time market facilitation specialist who proactively pursues opportunities for commercially sustainable inputs and service delivery. Such pilots should be afforded the flexibility and time needed to adapt their initial business model to market realities and should be evaluated in terms of development of a sustainable and scalable model. This should focus initially on the commercialization of the spraying service provider (SSP) model. Recommendation: Proactive, evidence-based advocacy addressing key policy issues. These important policy issues include subsidized inputs, government control of planting material propagation and distribution, and land tenure. To help spur these reforms, project management should include a full￾time policy advocacy specialist. Consideration should also be given to supporting an independent regional cocoa policy research center. Recommendation: Project management that proactively identifies and promotes synergies. Management should take better advantage of synergies between components and other cocoa initiatives. Cocoa and Food Security ACI is ultimately intended to improve food security. This is measured through increased income, with the assumption that more income enables the purchase of more food. A brief literature review of cash crops and food security supports the existence of a positive relationship between cash crops and food security, not only due to higher incomes earned but also to knowledge and technology spillover between cash and food crops. The evaluation team also found evidence in the field for a positive relationship between cocoa and food security. The cash generated from cocoa can be used to purchase food that is always available in a nearby market as well as to invest in food crop productivity. Additionally, cocoa provides food security–enhancing opportunities for sharecroppers and laborers. xi Farmers are well aware of the need for crop diversification, so increasing cocoa productivity does not appear likely to lead to more risky mono-cropping. Despite this apparent positive association between cocoa and food security, the evaluation team heard from a number of farmers who had experienced periods of food insecurity in some parts of CDI. Drought appears to be the primary cause of food insecurity among cocoa farmers, and more research is needed to determine the effect of cocoa on drought resilience. No farmers reported food insecurity in Ghana. Key recommendation: Conduct more research on cocoa and food security. Research should be conducted on crop and variety choice as well as on agronomic regimes for food crops grown with cocoa that cost-effectively mitigate drought, disease, and other cocoa risks. Additionally, economic research on household expenditures could show expenditure patterns for additional income from cocoa. Findings, conclusions, and recommendations for each of the four components are summarized below. Component I: National PPP Platforms Cocoa stakeholders broadly support the concept of the PPP platforms. They appreciate the opportunity to share views, keep abreast of industry trends and policies and have a voice in national cocoa policies. The PPP platform operates most effectively in CDI, where the most notable achievement was the agreement to harmonize the disparate cocoa extension training manuals and the government’s rapid response to the cocoa swollen-shoot virus (CSSV) crisis. In Ghana, the PPP platform had been less active until recently. However, PPP platforms have not been effective at addressing sensitive policy issues that have a profound impact on cocoa productivity, such as free or subsidized distribution of inputs. This is due in part to lack of evidence-based policy advocacy initiatives by ACI but also to the fact that the platforms are dominated by the key state organizations managing cocoa. The heavy dependence of the cocoa industry on the government as a business partner likely stifles the willingness of the private sector to take on these issues in these countries. Additionally, cocoa farmers’ interests on a national level are not well represented on the platforms in either country, although individual farmers representing cooperatives/societies are members. Key recommendation: Better policy analysis and advocacy skills. The project should include strong evidence-based policy analysis and advocacy skills within the implementer’s staff or consortium partners. Component II: Address Farm Productivity Constraints through Improved Planting Material The genetic fingerprinting activity revealed that a higher-than-expected 20% of the trees sampled from the breeder and seed gardens were not true to type. However, only 5% of the trees in the breeders and seed garden trees were tested, and it is not clear how representative this 5% sample is of the rest of the trees. The National Agricultural Research System (NARS) and relevant government agencies have not yet developed a plan for further testing beyond the 5% sample financed by ACI or for corrective actions. Corrective action is required if the seed or breeder garden trees are not true to type because this affects the quality of the planting material. After considerable delays, the ACI-financed seed and budwood garden expansion is nearly completed and is on track to be completed by the end of the project. This is essential to replacing the aging, diseased, and off-type trees that are prevalent in cocoa farmers’ plantations. Significant quantities of planting materials will not be ready until 2018 or later, xii however, so the original target of 100,000 farmers receiving improved planting material will not be met during the life of project. The genetic fingerprinting activity was an essential first step toward ensuring reliable breeding and planting material propagation. However, the impacts on the availability of improved planting material will be limited, as plans to clean up the mislabeled trees in the seed gardens have not been fully formulated or implemented. Likewise, the budwood expansion and seed garden expansion were significant contributions to planting material access. However, this will not be enough to meet national planting material requirements because of the sheer size of demand caused by years of neglect and disease, inefficiencies in the supply chain to farmers, and government control on vegetative propagation. Key recommendation: Genetic fingerprinting action plans. These plans should cover: 1) analysis of findings to date, 2) determination of additional genetic fingerprinting required, and 3) mitigation or corrective action options to be considered. Component III (i): Training of Public- and Private-Sector Extension Agents Through ACI, 244 extension staff were trained in CDI and 371 in Ghana. In general, farmers expressed a high opinion of the quality of extension workers’ advice. The technical content and the new communications methods introduced in the extension agent training were well received by the government extension agencies as well as agents who received the training. Some of the new topics cited as particularly useful or interesting included pruning, record keeping, women’s empowerment, shade tree selection, and management. Primary concerns were the high level of material covered in a short time and the lack of regular refresher training. Additional extension agency trainers were not trained and there are consequently questions about institutional sustainability. The support to the extension services training provided by ACI will likely have a positive impact on cocoa productivity. However, certain factors in CDI and Ghana reduce the impact of the extension agents and thus the impact of ACI training. First, the provision of subsidized inputs and services undermines the good agricultural practices (GAP) training received by farmers. The reason is that farmers tend to delay action, waiting for subsidized services even if they are late, insufficient, and of low quality. Second, the number of cocoa farmers who can be practically reached by agents is limited by resource constraints, which likely affects more remote and poorer farmers most. Key recommendation: Pilot innovative models to expand extension coverage. Pilot introduction, with ramp-up if successful, of models to leverage resources for and increase outreach of extension, including 1) cost sharing by farmers/cooperatives; 2) better coordination of roles/geographic coverage of public- versus private-sector extension services, including through better cocoa farm–level data; and 3) concerted support for low-cost supplementary coverage such as call-in radio programs. Component III (ii): The Ghana Quality Initiative (GQI) The GQI has successfully introduced West Africa’s first cocoa flavor laboratory and companion sensory panel at the Cocoa Research Institute of Ghana (CRIG). It is approaching international standards and has generated international interest. It has successfully trained a cadre of “Q” trainers who have worked with several cocoa cooperatives that had them taste the difference between poorly and properly fermented cocoa beans and trained them on fermentation best practices for flavor. The success of the GQI, however, depends on reasonable— but largely unproven— assumptions that international buyers will reward these efforts with preferential purchasing or higher prices. Introducing new measures to improve cocoa flavor without increasing cocoa farmers’ household incomes would xiii neither be sustainable nor a productive use of USAID funds. In Ghana, differentiated pricing will depend on the willingness of the government to implement such incentives. Key recommendation: Market analysis for West African fine flavor and high-quality flavor cocoa. This market analysis would provide valuable insight to West African cocoa stakeholders and help guide the design and scope of future GQI-like activities. Component IV (i): Spraying Service Providers ACI, through sub-partner CropLife, trained nearly 3,000 spraying service providers (SSPs). Over 40,000 farmers were served by SSPs by the end of 2014, when the pilot initiative ended. Farmers valued the SSPs’ knowledge on pesticide choice and dosage, the promotion of quality pesticides, and learning how to avoid exposure to risks when spraying themselves. SSPs appreciated the training, supply of equipment, and prestige that come with graduating from an industry-recognized program. However, SSP business acumen appeared quite low (less than 10% had received business training) and they do not have clear ideas about how to finance equipment, build a market, or otherwise act as entrepreneurs. Partly as a consequence, very few SSPs own their own mist blowers, the key piece of equipment for the SSPs. Many are operating at only 50% of capacity. The participating input supply companies have not continued the training/retraining program on their own, primarily because they don’t perceive a commercial value in the program. This is in large part because of competition from free government inputs or cheap imports from non-CropLife member companies. CropLife itself has shown little ability to promote the concept commercially on its own. Key recommendation: Utilize a market facilitation approach. SSP market facilitation models should offer the potential to be commercially sustainable and scalable by deepening or broadening the SSP service offering, including to other crops or offering other cocoa services by strengthening SSPs’ linkages to input suppliers (e.g., SSPs work as input supplier employees). Component IV (ii): Credit Activity In CDI, the ACI-supported credit pilot is now a commercially sustainable success. For the initial pilot, TechnoServe (TNS) provided training and support to financial institutions that then provided input financing to co-ops. Input providers have provided training on the safe and effective use of inputs purchased under the program to ensure profitability. For the 2015/6 season, ACI partner Advans (a microfinance institution, or MFI), without ACI support, extended $2.6 million to 81 cocoa farmer co￾ops, mostly for fertilizer purchases. Advans plans to expand cocoa lending without further ACI support. An independent evaluation of the credit program from early 2015 reports that repayment rates were 100% for CDI (and 98% for Nigeria). Access to credit is mentioned by many stakeholders as a serious constraint to adoption of good agriculture practices, and the ACI credit activity was an important contribution in CDI. However, the loans are only for agrochemical inputs, largely fertilizer. Credit is not available under the program for small productivity-enhancing equipment such as mist blowers for SSPs or pruners. Key recommendation: Utilize a market facilitation approach. New cocoa credit pilot proposals should test new lending models that are scalable by a commercial partner; they should not be judged just by year-to-year metrics such as cost per farmer reached. Cross-Cutting Issues The team also looked at four cross-cutting issues: land tenure, child labor, human and environmental health, and labor. Sharecropping, especially in Ghana, reduces the incentive of both landlord and tenant to xiv invest time and money in cocoa productivity improvements. The fieldwork revealed a high level of awareness of child labor issues among all players in the value chain, including the cocoa farming households. Through the SSP program especially there were likely human and environmental health benefits of the project. Labor scarcity is a constraint to cocoa production. In CDI, hired labor often comes from Burkina Faso and in Ghana from poorer areas of the county. There are also village labor￾sharing arrangements and college student labor gangs during breaks. It should be emphasized that these cross-cutting issues were not the primary focus of the evaluation and more research on each is needed. 1 EVALUATION PURPOSE AND EVALUATION QUESTIONS Evaluation Purpose USAID’s Bureau for Food Security (BFS) contracted Social Impact to undertake a performance evaluation of the African Cocoa Initiative. This evaluation is being carried out as part of the BFS Performance Evaluations Task Order held by Social Impact (SI) and commissioned by BFS in accordance with ADS 203.3.1.3. Consistent with the Office for Learning, Evaluation, and Research’s 2011 Evaluation Policy, the ACI performance evaluation is intended to provide evidence and recommendations to respond to evaluation questions designed to support learning and continuous improvement for BFS’s work and Feed the Future (FTF) more broadly, in addition to contributing to the Feed the Future Learning Agenda. The evaluation provides findings and recommendations that BFS, ACI, Feed the Future Missions, and their sub-partners can use to improve the effectiveness of their existing and future activities to better achieve their objectives. USAID BFS will use this evaluation specifically to inform future programming and investment decisions under FTF, including activities that use cocoa or other cash crops as a means to achieve food security. The WCF, WCF/ACI partners, and individual WCF members will use this evaluation to guide how they will continue and modify this and other related income-generation and food-security activities under the context of the CocoaAction Framework, launched in 2014. Other stakeholders in the cocoa value chain (e.g., ministries, financial institutions, input suppliers, etc.) will use the evaluation to guide their own business or policy activities, models, and strategies. This performance evaluation addresses the five central questions listed below. Illustrative evaluation questions, based on discussions with BFS/MPI staff, were provided in the initial scope of work provided to the evaluation team. Based on subsequent discussions with BFS staff and a review of key project documents, these original illustrative questions and later issues raised by BFS have been incorporated into the following five evaluation questions. Evaluation Questions 1. How did the ACI activities contribute or likely contribute to the objectives and results as intended for the project overall and its four components? a. The supply of improved agricultural inputs (including planting material) and services (especially extension and finance) to target farmers. For example: o To what extent have planting materials been improved at the national level and how likely are these improved planting materials to be used profitably by farmers? o How do farmers perceive extension services to be improved or not improved and why? o How commercially viable are the input services established and strengthened through the project with and without government subsidies? b. Public/private dialogue processes and institutions leading to tangible change, through improved consensus among actors and influence on policy direction. c. The effect of ACI-facilitated extension on the understanding and implementation of improved cocoa farming practices, including practices related to environmental stewardship, safety, and child labor. 2. How likely are the ACI activities to positively (or negatively) affect food security among cocoa farmers and within cocoa-producing communities? Might other activity designs and intervention areas lead to greater impacts on food security and income? This will be based on 1) a review of the literature 2 on cocoa farming and food security, 2) review of project documentation, and 3) fieldwork using KIIs and FGDs. This will be addressed in the context of the relative mix of crops (cocoa versus other cash and food crops) and relative intensity of production (e.g., use of inputs, labor, etc.) by the target cocoa farmers. 3. What other benefits or drawbacks (i.e., unanticipated outcomes or spillover effects) occurred as a result of the program for each component? To what degree were the changes in cocoa farming practices promoted by ACI associated with changes in farming practices for other crops? 4. What are the most important lessons learned from implementing ACI (including, where feasible and appropriate, observations concerning cross-cutting issues such as child labor, environment, and health and safety for women and children) that would inform similar efforts? 5. How might the ACI program or successor initiatives be designed to more cost-effectively and sustainably achieve and measure objectives, results, and outcomes? Evaluation Introduction Given an understanding of the evaluation’s purpose and guiding questions, this report will proceed by illuminating the methodology that was used to determine findings, conclusions, and recommendations. This section will also lay out limitations given the design and context of the evaluation. The report will move on to describe the ACI’s background and design in more meticulous detail. This will include the budget, context, and activities of sub-grants under each of the project’s four components. Once the project’s design is clear, responses to the five evaluation questions will be summarized in a matrix of overall findings, conclusions and recommendations. Findings, conclusions, and recommendations particular to each component will follow along with those that apply to more cross￾cutting themes such as food security, land tenure, child labor, farm labor, health, and the environment. 3 METHODS AND LIMITATIONS Methodology for Quantitative and Qualitative Data Collection This performance evaluation was carried out using primarily qualitative data from both primary and secondary sources. The methods used were key informant interviews (KIIs), focus group discussions (FGDs), and document review. The research team conducted fieldwork in Côte d’Ivoire and Ghana from September 28, 2015 to October 16, 2015, with that time divided evenly between the two countries. The fieldwork methodology emphasized open-ended questions and question guides that allowed researchers to investigate and learn about the project while maintaining common questions across respondents and ensuring that evaluation questions were addressed. The following describes the data collection methodologies used for this evaluation. In collecting data the evaluation team made efforts to ensure the inclusion of women and that they were able to actively and freely voice their perspectives. Additionally, the team included farmers from different socioeconomic backgrounds; greater distance from main roads was used as a proxy for relatively lower socioeconomic status. Together, these were intended to ensure that the evaluation accounts for potential disparate effects related to these respondents’ different characteristics. Key Informant Interviews Key informant interviews (KIIs) were the primary data collection tool used for fieldwork. KIIs were used when a respondent represented a unique project stakeholder or was likely to possess unique information or perspectives. The evaluation team used interview guides for KIIs to ensure that key questions were answered while still allowing for open￾ended discussion. These guides gave interviewers the freedom to ask spontaneous questions to uncover important and unanticipated information. One researcher took the lead in asking questions while the other took notes and filled in with any unasked questions from the question guides. As shown in Figure 1, a total of 61 KIIs were carried out, of which 12 were with women interviewees. Many key informants were unique and therefore sampling was not necessary (for example, interviewing the person who runs the GQI at CRIG). For other KIIs such as cocoa buyers, interview requests were made on the basis of contacts, importance of respondent, and availability. Other KIIs, such as those with farmers and SSPs, were identified during focus group discussions based on a particularly interesting or relevant perspective they expressed and their availability to stay beyond the FGD for a KII. FGD sampling is described below. Focus Group Discussions Focus group discussions (FGDs) were carried out with cocoa farmers, spraying service providers (SSPs), and extension agents. Not only do FGDs allow researchers to collect information about more people in a given amount of time, FGDs also provide a unique way to triangulate and validate findings. They also allow for the identification of interesting outliers particularly useful in understanding project performance. The same interview approach was used as with the KIIs, with one evaluator taking the lead and the other taking notes and Total: 61 (Male: 49; Female: 12) Stakeholders Representation Implementers (WCF, IDH): 10 Implementing partners: 23 Private sector: 10 Public sector: 6 Beneficiaries (SSPs, farmers): 6 Co-ops/associations: 6 Total: 11 FGDs; 195 Participants (Male:167; Female:28) Stakeholders Representation Farmers SSPs Extension agents Figure 2: Focus Group Discussion Participants Figure 1: Key Informant Interviews 4 ensuring key questions were asked. As shown in Figure 2, a total of 11 FGDs were held. These FGDs included 195 farmers, SSPs, and extension agents, including 28 women. FGDs with farmers, extension workers and SSPs were organized by project partners or other cocoa stakeholders. The sample was largely a convenience sample necessitated by the time constraints of the assignment and the consequential need to reduce inter-FGD travel time. Nevertheless, in both countries the evaluation team held at least one FGD with farmers who were at least one hour off the main road in an attempt to mitigate some of the sample bias introduced by this convenience sampling. Secondary Data Collection Secondary data collection was conducted using three sources: ACI project performance reporting and data, ACI project–sponsored research including baseline studies, and a brief literature review that explored the relationship of cocoa and other cash crops with food security. Project performance reporting as well as project-sponsored studies provided quantitative and descriptive data on results that complements the evaluation team’s primary data collection, which focused mainly on qualitative data collection. A list of documents reviewed can be found in Annex III. Literature Review The evaluation team conducted a brief literature review of cocoa and food security to inform and provide context to its response to research question 2. Documents were identified online and also in academic journal databases. However, literature focusing specifically on cocoa and food security was limited, so the search was expanded to tree and cash crops and food security. Analytical Approach In analyzing KII results, the evaluation team sorted interviews by component and then looked for both similarities and divergences in responses, noting characteristics of the key informant that might contribute to these similarities and differences. Responses to unscripted questions were analyzed similarly, with researchers looking for overlap in responses as well as for where unique responses contributed to new understanding of the evaluation questions. FGD results were analyzed in the same fashion. In analyzing KIIs and FGD notes, the evaluation team was cognizant of gender and socioeconomic differences in order to explore how the project affected women and other groups differently than other beneficiaries. Triangulation across the data sources was a foundation of the evaluation team’s analysis, ensuring reliability and veracity of the findings. Limitations of the Evaluation Methodology The time available for fieldwork for this performance evaluation was limited, considering the complexity of the project design, large number of partners and sub-partners, and activities in four countries. The number of KIIs and FGDs is not large enough to ensure representative sampling of the diversity of respondent types and project locations. This introduces the possibility that findings, conclusions, and recommendations from the evaluation may miss some of these variations. Cameroon and Nigeria were omitted altogether from the fieldwork and are only briefly discussed in this report. Other limitations include the fact that the evaluation report is being prepared just before the annual ACI steering committee meetings. As a result, the perspectives of ACI steering committee members are not as comprehensive and up to date as they could be. Additionally, ACI reports incompletely document IDH’s in-kind contributions, activities, and results despite their significant budget and importance as enabling stakeholders in the cocoa sector in West Africa. 5 Another limitation is the inability to reliably attribute changes in key indicators that came about as a result of ACI activities. Attribution requires the precise identification of a counterfactual—i.e., what would have happened to project participants over the project period if they had not participated in the project. This is usually identified through a control group. However, given the limitations of the evaluation terms of reference (TOR) and lack of baseline information, it was not possible to include and evaluate a control group. Nevertheless, the evaluation team attempted to understand and qualitatively assess attribution by asking respondents before-and-after or side-by-side questions; through follow-up questions the team attempted to uncover what changes could feasibly be attributed to the project. Several types of bias might have also presented problems: selection bias, recall bias, and response bias. Selection bias occurs when researchers interview people who are more convenient to reach or who are on friendlier terms with the project implementing partners. These factors—ease of access and friendliness with the project—might be correlated with project outcomes introducing bias. Recall bias is the natural error in memory that occurs when respondents are asked about events that took place in the past. Response bias can occur when respondents give particular responses that are untrue. This may be because they want to please the interviewer or increase their chances of receiving future funding. All three biases were addressed to an extent through triangulation—getting the same information from multiple sources. Within the limits of feasibility the evaluation team took steps to minimize selection bias in arranging KIIs and FGDs by selecting of farmers and SSPs that were in relatively remote and hard￾to-access areas. Evaluation Team The primary evaluation team is composed of Team Leader Donald Greenberg and Evaluation Specialist Christopher Root. Mr. Greenberg has over 25 years of experience in international development with a focus on value chain and agribusiness. He has served as an independent consultant in a Team Leader role on multiple agribusiness, market facilitation, and rural development projects and evaluations worldwide and is very familiar with USAID-funded projects and procedures. Mr. Greenberg has the overall responsibility of ensuring the technical quality of the evaluation design, fieldwork in Ghana and Côte d’Ivoire, data analysis, and production of high-quality evaluation deliverables, including the final evaluation report. Mr. Greenberg was accompanied and supported by Evaluation Specialist Christopher Root throughout all phases of this evaluation, including fieldwork in Ghana and Côte d’Ivoire. Mr. Root is an agricultural economist with over 10 years of international development experience focusing on research and evaluation design, including performance monitoring and impact evaluations. His expertise is in value chain assessment with an emphasis on cash crops, including the cocoa sector. The evaluation team was supported by Dr. Dennis Wood (Social Impact’s Vice President for Evaluation Services and Technical Manager on this Task Order) and Dennis Marotta (Deputy Director and Senior Technical Advisor on this Task Order), who provided technical input and quality assurance for the design and implementation as well as for report findings, conclusions, and recommendations. Project Manager Isadora de Latour provided evaluation management support as well as review of technical deliverables to ensure USAID compliance and approval. Program Assistant Miguel Albornoz provided coordination and logistical support as well as copy editing support on final deliverables. SI also contracted with a local interpreter and logistician to facilitate work in Côte d’Ivoire. 6 PROJECT BACKGROUND African Cocoa Initiative Justification and Overview Within the Bureau for Food Security, the Office of Markets, Partnerships and Innovations (MPI) is charged with the mandate to promote new approaches to food security through innovative partnerships that improve market access for food-insecure households and smallholder farmers in FTF countries. West Africa is a dominant player in the world cocoa market, with a 70% share of the market. Cocoa is a key sector for West African countries, generating over eight billion dollars in annual revenue. As a smallholder crop with a typical farm size of two hectares or less, cocoa supports a significant number of farm households in the region—over two million. Global demand for cocoa is growing as consumers in emerging economies have more disposable income and evolving tastes. However, smallholder cocoa productivity and quality in West Africa are low, with some estimates placing them at less than half their potential. This stunts the sector’s role as a driver of economic growth and reduces the returns to smallholder farmers. Productivity and quality are low for a variety of reasons, including a lack of improved planting materials, low use of inputs including fertilizer, farmers’ lack of up-to-date knowledge on best cocoa practices and market opportunities, and a lack of finance. The low productivity and quality of produced cocoa resulted in lower household incomes than might otherwise have been possible, which in turn lowered households’ food security. The African Cocoa Initiative works directly and indirectly to increase household incomes derived from cocoa production, which in turn should increase food security. On September 30, 2011, USAID/Bureau for Food Security/Markets, Partnerships and Innovations awarded Cooperative Agreement # AID-OAA-A-11-00061 to the World Cocoa Foundation (WCF) to implement the five-year African Cocoa Initiative. WCF is a membership organization of more than 110 companies from around the world, with an interest in empowering and encouraging cocoa farmers through holistic, partnership-driven programs focused on farm-level training and applied research. Subject to availability of funding, USAID agreed to provide $5,000,000 over the five-year period, leveraging more than $4,550,000 in matching funding from 14 WCF member companies in the chocolate and cocoa industry, as well as an in-kind contribution of $3,500,000 from the Dutch Sustainable Trade Initiative (IDH)1. In September 2012, USAID committed an additional $647,000 to finance the Ghana Quality Initiative, matched by a $648,000 in-kind contribution from sub-awardee Tcho Chocolate. Ultimately, small-scale cocoa farmers (those with less than five hectares of cocoa) and their households are the primary ACI beneficiary group. Secondary target groups include spraying service providers (SSPs) and community or village extension workers. The ACI program aims to double cocoa productivity for 100,000 farm households and, in doing so, raise the household income of cocoa farmers by 150–200%. This increase in household income is intended to improve food security. ACI works in four countries in West Africa: Nigeria, Cameroon, Ghana, and Côte d’Ivoire. ACI works primarily through sub-grants to partners in the public, private, and non-profit sectors (see Figure 3 for a full list). Four ACI sub-grants have been issued that cover all four countries: the Alliance of Cocoa Producing Countries (COPAL), the International Institute for Tropical Agriculture (IITA), CropLife, and the Cocoa Research Institute of Ghana (CRIG). Two sub-grants have been issued that specifically focus on Ghana, to CRIG and Tcho, and two sub-grants were issued specifically focused on Côte d’Ivoire, to 7 the Centre National de Recherche Agronomique (CNRA) and Bureau d'Expertises Côte d’Ivoire (BUREX-CI). One sub-grant to TechnoServe covered both Ghana and Côte d’Ivoire. Figure 3: ACI Sub-Grants Comp. Activity Detail Awardee Name Country Start Date End Date Award Status 1 Coordinating in-country PPP activities COPAL All 1-Jan-11 31-Dec-15 Suspended 1 Cocoa household comparative economic study LMC Côte d’Ivoire & Ghana 1-Oct-13 31-Jan-14 Ended 1 Harmonization of extension training manuals in Nigeria Vicanason Nigeria 20-Apr￾15 31-Jan-16 Active 2 Coordinating DNA extraction and standardization for genetic fingerprinting IITA All 16-Aug￾12 31-Dec-13 Ended 2 African Cocoa Breeders Working Group Project CRIG All 11-Jun￾13 11-Jun-16 Active 2 Establishing 15 hectares of seed gardens and 10 hectares of budwood gardens IRAD Cameroon 11-Jun￾13 16-Jun-16 Active 2 Establishing 50 hectares of seed gardens and 10 hectares of budwood gardens CNRA Côte d’Ivoire 11-Jun￾13 11-Jun-15 Ended 2 Establishing 50 hectares of seed gardens and 10 hectares of budwood gardens CRIG Ghana 11-Jun￾13 30-Sep-15 Ended 2 Establishing 15 hectares of seed gardens and 10 hectares of budwood gardens CRIN Nigeria 11-Jun￾13 16-Jun-16 Active 3 Implementation of New Generation program (young cocoa farmers) CICC Cameroon 15-Jul-14 30-Apr-16 Active 3 Training of ANADER extension staff in Côte d'Ivoire BUREX-CI Côte d'Ivoire 15-Jul-14 30-Dec-14 Ended 3 Training of COCOBOD extension staff in Ghana Solidaridad Ghana 1-Sep-13 15-Dec-13 Ended 3 Ghana Quality Innovations project for improving cocoa quality through flavor analysis TCHO Ghana 12-Jun￾12 30-Sep-16 Active 3 Training of extension staff in Nigeria Batrop and Co. Nigeria 1-Jan-15 15-Feb-15 Ended 4 Implementing pilot agro-chemical application (SSP) program CropLife All 21-Jan￾13 31-Dec-14 Ended 4 Implementing input credit scheme TechnoServe Côte d'Ivoire & Nigeria 1-Oct-12 31-Mar-15 Ended 8 The ACI project has four interrelated components, which are briefly described below and depicted in the project’s Logic Model (Figure 4). Component I: Establishment and Strengthening of National Public-Private Partnership (PPP) 2 Platforms The PPP platforms are intended to bring together government agencies, private sector and farmer interests, and non-governmental organizations to establish priorities, set plans, and mobilize resource commitments for investing in cocoa. Component II: Address Farm Productivity Constraints through Improved Planting Material The quantity of improved planting material was to be increased through the expansion of seed gardens, from which hybrid cocoa pods are produced and eventually distributed to farmers. The seed gardens are supported by the creation/expansion of budwood gardens, which will provide buds of clones of parent lines. These are then grafted to the seed garden rootstock trees to produce true-to-type parent lines for the seed gardens. The quality of improved planting material was to be increased through genetic fingerprinting. Genetic fingerprinting included the collection and comparison of genetic material. This was carried out on a sample of trees from breeders’ gardens, existing seed gardens, and farmers’ trees to validate that the parent lines and the hybrid varieties produced from them are true to type and will therefore express the expected characteristics in the farmers’ plantation. Related activities undertaken include support to the African Cocoa Breeders Working Group (ACBWG) and on-farm trials of newly developed cocoa varieties. 9 Figure 4: African Cocoa Initiative Logic Model Farmer households in West Africa double their income from cocoa and non-cocoa sources in 5 years Support sustainable agriculture and improved food security on diversified cocoa farms in West Africa Activities Outcomes Objectives/ Components Project￾defined goal Goal ACI 1.1: (i) Grant to COPAL, (ii) In-kind contributions to plenary sessions. ACI 1.2: In-kind contributions to plenary sessions ACI 1.3: (i) Scenario Planning Workshop, (ii) WCF/ACI staff support for Technical Working Groups ACI 1.4: WCF/ACI staff support for Technical Working Groups ACI 2.3: (i) Seed and budwood garden expansion, (ii) On Farm Demonstration of improved planting materials ACI 2.1: (i) Genetic fingerprinting, (ii) ACBWG support ACI 1: Strengthened national partnership platforms ACI 2: Improved farm productivity ACI 2.1: Certified National Seed Gardens ACI 3.1: Improved training curricula via ext. services ACI 3.2: Expansion of ext. training programs in 4 countries ACI 2.3: Farmers w/ access to improved planting material ACI 2.4: Direction of ACI research on genetics ACI 1.1: Four country platforms ACI 1.2: Annual platform goal￾setting & evaluation. ACI 1.3: Cocoa sector roadmaps & investment plans ACI 1.4: Country￾level policy development ACI 2.2: Seed and budwood garden expansion ACI 2.2: Internal plant material propagation & dist. centers ACI 4: Improved market-driven farming input supply services ACI 3: Enhanced public and private sector extension and farmer training services ACI 3.1: Curricula Development ACI 3.2: Extension agent training ACI 3.3: (i) Productivity and Entrepreneurship Training for Women's Empowerment (ii) New Generation of Cocoa Farmers Program ACI 3.4: (i) Expansion of CocoaLink, (ii) Transfer of CocoaLink to COCOBOD ACI 4.1: Training of Spraying Service Providers ACI 4.2: (i) Development of new model to equip SSPs to channel input credit, (ii) Pilot and scale up of ICS program ACI 4.1: Trained agro-input dealers ACI 4.2: Expanded Credit programs for cocoa sector ACI 3.3: Expansion of succesful pilot comm. & farmer led training programs ACI 3.4: Expansion of succesful SMS & MMS pilot ext. services ACI 2.4: ACBWG support 10 Component III: Enhance Public- and Private-Sector Extension and Farmer Training Services The main activity in this component was the development of updated extension training curricula and the use of these training curricula to train public- and private-sector extension agents through the national extension services. There were also two smaller pilot activities, the Productivity and Entrepreneurship Training for Women’s Empowerment (PETWE) in CDI and the New Generation of Cocoa Farmers activity in Cameroon. The second main activity undertaken in this Component is the Ghana Quality Initiative (GQI), which aims to improve the overall quality of Ghana’s bulk cocoa and to pilot the introduction of specialty fine flavor cocoa varieties that are not yet grown in West Africa on a commercial scale. Component IV: Foster Market-Driven Farming Input Supply Services This component focused on the pilot introduction of a cadre of spraying service providers (SSPs), entrepreneurs intended to provide safe and effective pest-control measures on a commercial basis to cocoa farmers. The SSP activity was complemented by an input credit scheme (ICS) for cocoa cooperatives that included innovative risk sharing and embedded service provision by financial institutions, input providers, and exporters. This was introduced to Nigeria and was ramped up in CDI. Related Programs ACI is one of two flagship programs addressing cocoa in West Africa managed by WCF. The other, the Cocoa Livelihoods Project (CLP Phase II) is a program funded by BMGF and the German government (through BMZ). CLP also works to increase productivity but operates through a program of matching grants to individual WCF members and works directly at the farmer level. This is in contrast to ACI, which generally works on the institutional and enabling environment (Components I–III) or works to incentivize and empower value chain actors to provide inputs and services to cocoa farmers (Component IV). Most WCF/ACI steering committee members also have additional cocoa productivity and community development programs outside the ACI or CLP project envelopes. In an effort to coordinate and align all of the cocoa sustainability programs, WCF launched the CocoaAction (CA) strategy in 2014, with the goal to improve the livelihoods of 200,000 cocoa farmers in CDI and 100,000 farmers in Ghana. To join CA, one must be a member of WCF and commit to providing additional time and resources. Most ACI steering members have also joined CA. Some that haven’t joined, such as Guittard, still commit company resources to achieving one or more of the CA workstreams. CocoaAction has six workstreams:  Planting Material  Fertilizer and Soil Fertility  Community Development  Government and Donor Alignment  Innovations/Future Forms of Extensions  Shared Commitment to Measuring Progress and Impacts Most WCF members appear to be more engaged and knowledgeable about CA than ACI. There is confusion between the PPP technical working groups (TWGs) and the CA workstreams, which meant it wasn’t clear in the KIIs whether ACI steering members were able to distinguish between PPP platform and CA activities. 11 The ACI project does not completely fall under the rubric of CocoaAction. Some of the important ACI activities that have been recommended to be continued seem to fall outside the CocoaAction workstreams as currently structured, such as the credit and market-led input supply activities of Component IV. In addition, CocoaAction is only for Ghana and CDI, leaving out Cameroon and Nigeria. IDH (Dutch), one of the partners of ACI, has focused on mainstreaming the major certification systems used for cocoa (Rainbow Alliance, Utz, and Fair Trade) and more recently focused more directly on productivity issues, especially fertilizer usage. Taking a PPP approach, IDH runs two major programs dealing with cocoa livelihoods: the Cocoa Productivity and Quality Program (CPQP) and Cocoa Fertilizer Initiative (CFI), which are important complements to the two WCF projects. While there are good communications between ACI and IDH where there is co-financing, better strategic planning and linkages would strengthen impacts. GIZ (Deutsche Gesellschaft für Internationale Zusammenarbeit) is also active in cocoa in West Africa. Most notable is its Farmer Business School (FBS) training, which it has introduced through the extension systems; the training provides a very important complement to the GAP and community- and social￾oriented capacity building provided by ACI. Components I–IV Background, Description, and Budget Component I: Establish and Strengthen National Public-Private Partnership (PPP) Platforms Budget: $425,000 (7% of ACI sub-awards/direct costs), initially through a sub-award to COPAL and eventually through WCF staff and direct costs supporting the platforms.3 Table 1. Summary of Outcomes, Activities, and Results Anticipated Outcomes ACI Activities Supporting Outcomes Results Establishment and effective operation of four country platforms with broad membership and terms of reference Grant to COPAL In-kind contributions to plenary sessions CDI platform functioning Ghana platform beginning to function Annual platform goal-setting and evaluation of performance against objectives In-kind contribution to plenary sessions No formal annual platform goal-setting or evaluation process Country-level sector roadmaps and investment plans elaborated for the cocoa sector Scenario Planning Workshop support (Ghana) WCF/ACI staff support for technical working groups Platform provides stakeholder input into revisions of cocoa strategies in CDI and Ghana Country-level policy development to champion the cocoa industry and its households WCF/ACI support for technical working groups Platform provides stakeholder input into revisions of cocoa strategies in CDI and Ghana 12 Background and Context The national public-private partnership (PPP) platforms were established or strengthened by ACI to bring together government agencies, private-sector interests, and non-governmental organizations to establish priorities, set plans, and mobilize resource commitments for investing in cocoa. The platforms serve as a forum for discussing issues related to the objectives of ACI (such as extension systems), as well as more general issues important to the sector, such as transportation infrastructure beyond the scope of ACI. The PPP platforms concept was developed in the context of and to support ongoing national and regional initiatives to accomplish similar objectives. These include the Comprehensive Africa Agriculture Development Programme (CAADP)4 process initiated in 2003, which committed countries to increase investment in agriculture using PPPs and other instruments and was reinforced by the first World Cocoa Conference.5 The PPP platform concept had strong support from development partners such as GIZ and USAID. See table 1 for a summary of outcomes, activities and results of component 1. Description COPAL. Originally, ACI was to support the PPP platforms primarily through a grant to the Alliance of Cocoa Producing Countries (COPAL). COPAL, for a number of reasons, was not capable of assuming this role. WCF suspended the COPAL grant in 2012 and has assumed direct responsibility for managing this component. Because of this, WCF now plays a dual role on the PPP platforms: first, as provider of financial/technical/administrative support; second, as an advocate for the cocoa industry. In 2012, with WCF participation, COPAL undertook fact-finding missions to the four ACI countries to “establish the existence or not of national platforms” and to identify and prioritize needs for support by ACI. Most of the other support has been country-specific. Ghana. In Ghana, the ACI PPP platform has supported some of the expenses of what was initially known as the Ghana Cocoa Platform, which had been initiated and funded by the United Nations Development Programme (UNDP) Green Commodities Fund. The GCP had nearly identical objectives to the PPP concept, although with a more environmental focus. Along with other donor agencies, ACI also supported two Cocoa Sector Scenario Planning Workshops led by the World Bank. These workshops were intended to provide input for the development of a new Ghana Cocoa Sector Development Strategy. Much of the substantive PPP platform work is undertaken by TWGs working throughout the year, with membership drawn from cocoa value chain stakeholders. There are four TWGs in Ghana, as shown in Table 2. ACI supports the TWG by active staff participation. 13 Table 2. Ghana PPP Technical Working Groups TWG WCF Staff Participation Extension and Productivity Yes Environment Yes Crop Financing, Marketing, Pricing No Social Protection Yes The Extension and Productivity TWG recommended the harmonization of cocoa extension training manuals, and ACI has committed to financing this activity. CDI. ACI has provided support to the plenary sessions of the PPP platform since 2012, and WCF staff have been active participants in the plenary and TWG meetings. As in Ghana, much of the PPP platform work is delegated to TWGs, with large stakeholder participation (the size of the TWGs ranges from 17 to 35 members). There are seven TWGs in Ghana, as shown in Table 3. ACI supports these TWGs mostly through active membership and participation by WCF staff. Table 3. CDI PPP Technical Working Groups TWG WCF Staff Participation Sustainability Yes Productivity / Inputs Yes CCSV Yes Extension Yes Child Labor Yes Revamping Coffee No Infrastructure No Component II: Address Farm Productivity Constraints through Improved Planting Material Budget: $2,424,705 (39% of ACI sub-awards/direct costs) to IITA and the national research agencies of Ghana, Côte d’Ivoire, Cameroon, and Nigeria. Table 4. Summary of Outcomes, Activities, and Results Anticipated Outcomes ACI Activities Supporting Results National seed gardens certified as true￾to-type under ACBWG’s and NARS’s direction Genetic fingerprinting ACBWG supported to provide oversight to the individual NARS On track, but only 5% of seed garden material has been tested. Plans for further testing and remedial action based on results not yet well articulated. Public and private internal plant material propagation and distribution centers set up such that 10,000,000 trees (approximately 10,000 hectares) can be distributed annually in each country Seed and budwood garden expansion Public-sector propagation and distribution centers established through seed gardens, although efficiency and effectiveness constrain impact. The private sector has not been allowed to play a significant role in supporting or supplementing this. 100,000 farmers gain access to improved planting material Seed and budwood garden expansion On-farm demonstration of improved planting materials Significant amounts of seedlings will only become available after ACI project completion date. 14 Government restrictions on vegetative propagation greatly reduce access to improved planting material. ACBWG assumes primary role coordinating/directing Africa cocoa research on genetics ACBWG support ACBWG is a useful vehicle for communications among the region’s researchers but does not play a primary role in coordinating/directing cocoa research on genetics. Background and Context Yields on West African cocoa farms are only a fraction of what are routinely achieved on the region’s research stations or achieved commercially in other cocoa regions of the world. Current estimated average productivity ranges from 300 to 450 kg/ha. Low adoption of good agricultural practices (GAP), disease outbreaks, and age of trees6 are key factors in the poor yield performance in the sub-region. WCF/ACI believes that yields of up to 1,000 kg/ha are within reach if GAP are applied by the region’s farmers. Another fundamental driver of poor yield, however, is thought to be the poor genetic quality of existing plantations in West Africa. Only a fraction of farmers’ trees are modern hybrids;7 the rest are a mix of out-of-date varieties, off-type hybrids, trees older than 30 years, and unproductive self-pollinated parent line pods inadvertently distributed as hybrids by the NARS. Replacing these older varieties and off-types with current varieties would increase productivity and reduce susceptibility to disease and pests.8 Industry stakeholders believe that yields could rise to 1,500 kg/ha or more if modern hybrids were planted and GAP applied. It had been suspected that some of the cocoa trees at the NARS breeding gardens and seed propagation gardens were mislabeled and therefore not true to type. It is difficult, however, to identify such mislabeled trees solely based on physical inspection. This mislabeling would have at least two major impacts: 1) breeding of new varieties could not be done reliably if the breeders were not sure of the identify of their breeding materials (parent lines) and 2) some of the planting materials received by the farmers from the existing seed gardens would not be true crosses and therefore may not have the desired productivity or resilience factors inherent in those varieties. In addition, the physical capability of West African countries to produce enough improved planting material to meet demand is insufficient. The huge demand stems from the poor condition of the cocoa plantations. In Ghana, according to COCOBOD, 23% of trees are over 30 years old and 17% of trees are diseased. Of the remaining trees, only 31% are thought to be true-to-type modern varieties, and many of these are sub-standard in terms of productivity and resilience. In addition to the massive rehabilitation of cocoa plantations that is required, on average 3% of plantations should be replanted on an annual basis to replace aging trees. Research efforts over the past decades in the various research institutes have resulted in the development and release of improved cocoa varieties. There is a concern, though, that there has been insufficient demonstration of these new varieties under farmers’ actual production conditions. This demonstration would both assess the field performance of these new varieties over existing types and promote new varieties. 15 The African Cocoa Breeders Working Group (ACBWG), a sub-partner for this component, consists of breeders from the four national research agencies (NRAs) and works to improve cocoa productivity through increased availability of better planting materials for farmers. ACBWG was created with support from an International Cocoa Organization (ICCO) project in 2003 and has since been supported by the USDA and Mars Inc.; under the Sustainable Tree Crops Program (STCP), it has received funding from USAID and WCF. See Table 2 for a summary of outcomes, activities and results of component II. Description Activity I: Genetic Fingerprinting Genetic fingerprinting addresses the concerns that the cocoa trees in both the breeding gardens and seed production gardens were improperly labeled or not true to type. ACI, through support provided to IITA and the four NRAs, enabled genetic fingerprinting of the most important clones in the breeders’ collections and seed garden materials in Nigeria, Ghana, Côte d’Ivoire, Cameroon, and Togo.9 Activity II: Expansion of Seed and Budwood Gardens Through grants to the respective NRAs, ACI has financed a major expansion in the budwood and seed gardens of West African countries and thus their capacity to ramp up the supply of new planting materials. The expansion of the seed and budwood gardens was meant to address the shortfall in production of planting materials to meet demand. Currently, budwood gardens are used to refresh and renew the seed gardens but are not for distribution of clonal budlings to farmers. Table 5. Seed Garden Activity Country Existing Number of Hectares WCF/ACI Number of Hectares Planned Increased Capacity of Country Due to ACI Support Cameroon 85 15 18% Côte d’Ivoire 90 50 56% Ghana 138 50 36% Nigeria10 89 15 17% Total 317 130 32% Source: WCF Budwood Garden Activity The ACI grants also funded expansion of 10 hectares each of budwood gardens for the four countries. Budwood gardens are currently used to support the upgrading and expansion of the seed gardens by providing true-to-type clonal varieties, so they directly support ramping up the supply of new planting materials for farmers. See table 5 for details on seed garden activity. 16 Activity III: Enhancing Farmers’ Adoption of High-Quality Cocoa Planting Material This activity addresses the concerns that breeders are not sufficiently aware of the performance of new varieties under actual field conditions and that farmers are not sufficiently aware of the benefits of new varieties. ACI grants to the NRAs funded the establishment of demonstration plots in five communities across the cocoa-producing regions per country, with the goal of at least five farming households per community as target farmers. Each demonstration site is to include the farmers’ own varieties, cultivars currently supplied from the seed gardens, and newly developed varieties and include accompanying GAP. Community-owned nurseries will be established and used to demonstrate best practices for managing cocoa nurseries. Field Days inviting the target communities are to be conducted at the demonstration sites during critical phases of the crop to evaluate relative survival, establishment, and precocity of the varieties. The demonstration sites will be points for demonstrating GAPs to farmers and will be researcher-managed. Activity IV: ACBWG Support This activity supports the activities of the ACBWG, with a primary focus on monitoring and guiding the genetic fingerprinting exercise. ACI has supported costs for meetings of the ACBWG, where in addition to reviewing the results of the genetic fingerprinting exercise, other topics of interest including research on CSSV were discussed. Nominally, Activity III is funded through ACBWG support. Component III: Enhance Public- and Private-Sector Extension and Farmer Training Services (i) Component III Extension Activities Budget $1,108,348 (18% of ACI sub-awards) to national extension agencies. Table 6. Summary of Extension Outcomes, Activities, and Results Outcomes Activities Results Extension services in four countries adapt and develop modern farmer training curricula for application to local needs Curricula development Curricula well received by extension agents Annual objective setting and evaluation of performance exercises with national extension services that include routine master trainer development programs Was not explicitly included in design of activities In Ghana, some capacity built in the participating master trainers at the Bunson Cocoa College In CDI, some capacity built in senior trainers within ANADER Strengthened/expanded cocoa extension training programs in each of 4 countries reaching at least 100,000 households within the project term Extension agent training Training well received by extension agents and farmers Unclear to what degree farmers received improved services as a result of the training. Expand successful pilot programs in each of 4 countries to use local community and or farmer-led training approaches in concert with national services Productivity and Entrepreneurship Training for Women’s Empowerment in CDI was substituted for the Farmers Business Services Centers initiative New Generation of Cocoa Farmers Program Productivity and Entrepreneurship Training for Women’s Empowerment reviewed positively by member company Evaluate and expand successful pilot programs now underway using cellular telephony (SMS and MMS) services to deliver and monitor farmer extension services CocoaLink expanded through ACI and transferred to COCOBOD after assessment of their capacity Mobile coverage, local language, and farmer literacy pose challenges Background and Context 17 Improving agricultural practices is critical to improving the low yields of cocoa farmers in West Africa. The front line in improving GAP for cocoa farmers in West Africa is extension agents. Extension agents are also in a position to provide information to farmers on important social issues related to cocoa, including child labor and gender as well as environmental aspects. However, extension agents may not themselves know the latest GAP or be trained in the social and environmental aspects of cocoa production. In addition, many extension agents have not been trained on best practices for communicating and presenting information to farmers. In Côte d’Ivoire and Ghana, Agence Nationale d’Appui au Développement Rural (ANADER) and the Cocoa Health and Extension Division (CHED) did not have the capacity to develop and deliver these trainings themselves. ANADER is housed under the Ministère de l'Agriculture and the extension agents trained by ACI have a training and background in providing extension for a range of crops and livestock. The extension officers trained by ACI focus on cocoa but provide extension services on other crops and livestock as well. CHED is part of the Ghana Cocoa Board (COCOBOD), which itself is under the Ministry of Finance. Unlike ANADER agents, CHED extension agents provide services exclusively on cocoa. See table 6 for a summary of extension outcomes, activities and results. (i) Extension Activities Description Activity 1: Extension Agent Training Training of extension agents under Component III differed between the Côte d’Ivoire and Ghana. In Côte d’Ivoire the training was carried out by BUREX-CI and delivered to ANADER extension agents in 2014. The two-week training was based on a draft of the harmonized cocoa extension manual created through Component I. The training had five components: good agricultural practices (GAP) for cocoa, adult education and extension methodologies, child labor, HIV/AIDS and malaria, and gender issues. In total 244 extension agents including 46 women were trained; they are expected to reach 40,000 farmers with improved extension delivery by 2016. Of the trainees, two thirds were extension agents hired within the previous year. Training of extension agents was not originally planned for Ghana because its extension services were considered more advanced than those in the other three ACI countries. However, CHED, responsible for all public-sector extension in the country, requested that the project include them in their training and identified their own training needs, which informed training design. The two-week training of extension agents, coordinators, and officers was carried out by NGO Solidaridad West Africa from late 2013 to early 2014. Solidaridad contributed 50% of the training cost because the training aligns with the organization’s existing work and mission. The extension training in Ghana focused on the following areas: facilitation and communication skills, GAP, including child labor, and cocoa certification enhancement. The first component of the training followed the Set Up, Deliver, and Finish (SDF) approach. The last module focused on preparing farmers to comply with cocoa certifications, including Fair Trade, Rainforest Alliance, and Utz. The training also built the capacity of some of CHED’s monitoring and evaluation staff. The training covered 371 trainees, who are expected to reach 35,000 farmers by 2016. In Nigeria and Cameroon, Component III of ACI is implemented on a smaller scale and with a different approach. In Nigeria, 56 extension staff in two states were trained on good agricultural, environmental, and social practices for cocoa production. In addition, four employees of WCF member companies received the same training. 18 Activity II: Productivity and Entrepreneurship Training for Women’s Empowerment In addition to this training, ACI implemented another activity through ANADER called Productivity and Entrepreneurship Training for Women’s Empowerment. This activity focused on women cocoa farmer members of cooperatives that supply international cocoa buyers. This activity replaced Farmer Business Service Centers, which was taken over by another project already implementing something similar. Activity III: New Generation of Cocoa Farmers program In Cameroon, ACI works through the Conseil Interprofessionnel du Cacao et du Café (CICC) to implement the New Generation of Cocoa Farmers program. This program provides expert mentorship for 50 young cocoa farmers to demonstrate that cocoa can be a profitable and attractive business for young people. (ii) Component III Ghana Quality Initiative Sub-Activity Budget: $647,000 (10% of ACI sub-awards), matched by an additional in-kind contribution from TCHO of $648,220 and in-kind matching from CRIG. Table 7. Summary of GQI Outcomes and Results Original Outcomes ACI Activities Results Technologies/methods to improve cocoa quality Q training to farmers, coupled with tasting of cocoa samples Training to farmers underway, results are promising Development of cocoa sensory capabilities Flavor laboratory, cocoa sensory panel Flavor laboratory and sensory panel have been established at CRIG; in process at CAA Dissemination of knowledge of cocoa flavor to value chain Outreach activities at cocoa industry events and activities COCOBOD management has been sensitized and has decided to mainstream Q training in extension Creation of value-added through premium prices for better flavor cocoa Establishment of premium for fine flavor varieties Increase quality of bulk cocoa to improve sales through certified channels Premium structure for fine flavor in process of being established Improved sales of higher quality bulk cocoa through certified or other channels has not occurred GQI Background and Context Ghanaian cocoa enjoys a slight price premium over other West African cocoas, largely attributed to a superior quality control process overseen by COCOBOD through the Quality Control Company (QCC). QCC quality control is based on physical inspection and tests of the cocoa beans, which translates into fewer rejected and substandard beans for buyers; however, the standards do not include a “flavor profile” based on organoleptic (i.e., tasting) methods. Ghanaian cocoa further enjoys a reputation as having a consistent flavor profile that manufacturers can depend on in their chocolate recipes. However, there are concerns from some in the industry of the possibility that this flavor profile could erode with the introduction of new cocoa varieties that have been selected only for productivity and resilience, not flavor, thus making it relatively less desirable. There is also some interest from US specialty chocolate manufacturers to source fine flavor11 cocoa from West Africa, which only supplies commodity bulk cocoa. 19 The Ghana Quality Initiative (GQI) emerged out of the Ghana Fine Flavor Cocoa Project (GFFCP),12 which isolated fine flavor varieties of cocoa from within the CRIG germplasm collection. The GFFCP convinced a group of farmers in the Offinso area to plant one hectare each of fine flavor varieties and these farmers formed the Offinso Fine Flavor Association (OFFA). The GFFCP stalled for a number of reasons, until the initiative was taken over by GQI by Tcho Chocolate (later joined by Guittard Chocolate) as the GQI, using matching funding from ACI. Tcho, one of the more flavor-driven US companies, brought their considerable experience in introducing cocoa flavor labs in Latin America (which were jointly funded by USAID under the GDA mechanism) and expanded the scope of the original GFFCP to include measures to promote, preserve, and improve the flavor profile of Ghana’s bulk cocoa varieties. GQI is financed under ACI Component III, Extension Capacity Strengthening, but it is also supports Component II, Improved Planting Materials. This is because it 1) promotes the selection and propagation of fine flavor varieties new to Ghanaian farmers and 2) seeks to maintain Ghana’s unique flavor profile by adding organoleptic screening of new varieties. It also screens for productivity, disease resistance, drought tolerance, etc. before release to farmers. See table 7 for a summary of GQI outcomes, activities and results. GQI Activity Description GQI activities are managed by an Accra-based Tcho project manager. GQI undertakes the following key activities:  Development and operation of a flavor lab at CRIG capable of producing cocoa liquor and chocolate on a sample scale and development of a cocoa sensory panel at CRIG station capable of undertaking organoleptic testing to international standards. Establishment of a satellite laboratory and sensory panel at the Cocoa Abrobopa Association (CAA).  Flavor evaluation of CRIG cocoa varieties undergoing pre-release national variety trials to ensure that their profiles do not erode the known Ghanaian cocoa flavor profile.  Development of a cadre of cocoa quality trainers (known as “Q” trainers) from CRIG who are training farmers of the OFFA and CAA in sensory appreciation of the differences between poorly processed and properly processed cocoa. Using this sensory tasting as a guide, the Q trainers provide GAP and post-harvest techniques intended to maximize quality.  Creation of a premium-driven fine flavor supply chain in Ghana starting with the OFFA farmers, working in partnership with Transroyal (the LBC), Tachibana (specialty/certified buyer of Cocoa Marketing Company, or CMC), and Guittard. Component IV: Market-Driven Input Services Background and Context Along with better planting material and increased farmer know-how, improved farm input services were recognized in ACI as the third input for increasing productivity. Ready access to reliable fertilizer, agrochemicals, equipment, and agriculture credit are constraints to growth in the sector. Improving access to these services, for cocoa in particular and agriculture in general, is the fourth major component of ACI. 12 Managed by the Sustainable Food Lab and CIAT, 2008–2011. 20 This Component is broken down into two sections, (i) the spray service providers sub-activity and (ii) the input credit scheme sub-activity. This is because they have evolved into completely distinct activities, although they were intended to be closely linked during project design and early project implementation stages. (i) Spray Service Providers Sub-Activity Budget $ 815,500 (13% of ACI sub-awards/direct costs) granted to CropLife Table 8. Summary of Outcomes and Results, Spray Service Providers Sub-activity Original Outcomes Activity Results Trained agro-input dealers serve 100,000 cocoa farmers (outdated) Train SSPs As of Dec 31, 2014, 50,000 farmers reached Generally successful but modifications needed for scale and sustainability Background and Context The Spray Service Provider (SSP) Program was part of ACI’s Component IV, Fostering Market-Driven Farming Input Services. The program was implemented by CropLife Africa and the Middle East, a membership organization of major international crop-protection companies. The model, which has been tested on other crops in other countries in sub-Saharan Africa, is to train spray service providers to safely and expertly apply pesticides on a fee basis. The objective is to improve the safety and effectiveness of pesticide application while generating a business opportunity for the SSP. In addition, it is hoped that using SSPs will reduce over-application of pesticides by farmers. Farmers applying pesticides are thought to have a tendency to over-apply because the cost of under-application (pests) is more obvious than the cost of over-application. However, it turns out that SSPs may have an incentive to over-apply as well because they are paid by container of pesticide rather than by area sprayed. See table 8 for a summary of outcomes, activities and results of spray service providers sub-activity. SSPs Description CropLife trained four master trainers in Côte d’Ivoire and four in Ghana, who in turn trained 43 SSP trainers in Côte d’Ivoire and 49 in Ghana, respectively. These SSP trainers consisted mostly of extension agents, CropLife member companies, and NGO staff. These SSP trainers then trained 1,428 SSPs in Côte d’Ivoire and 1,346 in Ghana on the technical aspects of safe spraying. The criteria for trainee participation were willingness, literacy, and being a male (because of concerns about pesticides and pregnancy). Trainees were then issued a test, which over 90% passed. Of those trainees, 100 in CDI and 202 in Ghana received business training. Those receiving business training paid their own transportation. As of the end of 2014, SSPs in Côte d’Ivoire had reached 23,510 farmers and, in Ghana, 18,189 farmers. However, many of these farmers may have already been hiring independent sprayers (many of whom have become SSPs), so that the incremental gain in farmers covered may be somewhat overstated. (ii) Input Finance Sub-Activity Budget: $980,000 (16% of ACI sub-awards/direct costs) granted to TechnoServe Table 9. Summary of Outcomes and Results, Input Finance Sub-activity 21 Original Outcomes Methodology Results Expanded credit programs for the cocoa sector in 4 countries Develop new model to equip SSPs to channel input credit to cocoa farmers New model was not successfully introduced Modify and scale up pilot ICS program in CDI Introduce modified pilot ICS to Nigeria In CDI, modified ICS program has been adopted and rapidly scaled up by one MFI In Nigeria, ICS program model tested does not seem to be sustainable Background and Context Cocoa farmers’ access to high-quality inputs and services, delivered at the appropriate time, is limited in part because of financial institutions’ reluctance to provide appropriately structured credit packages. This reluctance is due to a number of factors, including lack of experience, lack of infrastructure and staff in the cocoa regions, and a general unwillingness to take risks on primary producers in the agricultural sector. Under the Cocoa Livelihoods Project (CLP), TechnoServe designed and successfully implemented an innovative pilot input credit scheme (ICS) for cocoa farmers in Ghana and Côte d’Ivoire from 2009 to 2013. The ICS pilot worked through selected cooperatives and linked them to input suppliers and exporters who shared risk along with TNS. Funding was provided by input suppliers in CDI and by an MFI in Ghana. Crucially, to convince them to provide funds, TNS staff acted on the suppliers’ and the MFI’s behalf as debt collection agents. Based on the success of the CLP pilot initiatives, TechnoServe was awarded an ACI grant in September 2012 to develop a new two-year pilot input credit scheme for CDI and Nigeria. See table 9 for a summary of outcomes, activities and results of input finance sub-activity. Activity Description TechnoServe’s original concept for the financing pilot to enhance the SSP model was by equipping the most creditworthy SSPs to on-lend bank credit to cocoa farmer customers, who would purchase inputs and spraying services from the SSPs on credit. In the original model, 60 of the SSPs trained and equipped by CropLife were to be selected and provided training of trainers (TOT). The TOT was to build their capacity to train farmers on credit management as part of their core business. These SSPs were then to be provided with credit by financial institutions (FIs) or supply chain companies and were expected to on-lend this credit to cocoa farmers. In a departure from earlier pilot initiatives, the SSPs were expected to act as their own collection agents and thus were expected to price the risk of a project rate of farmer non-payment into the cost of their loans. Although TNS would provide a guarantee to suppliers and FIs lending to the SSPs, the SSPs themselves were not to be provided with a guarantee for their loans to farmers. Given the lack of progress in the original concept, WCF and TNS agreed to make a fundamental shift in the ACI ICS program. The concept of SSPs as commercial agents capable of extending credit to farmers was quietly dropped, although co-ops that were recruited for the credit program did receive the full SSP training. The final design was essentially an extension and improvement on the original CLP ICS model. TNS worked with exporters that identified cooperatives with whom they had an ongoing relationship and that they thought would be creditworthy. TNS then identified and provided training and support to FIs 22 that would provide input financing to the co-ops. In the case of CDI, four new co-ops were added by TNS with ACI funding. TNS refined its risk-sharing program to the FIs, as shown in Table 10. Table 10. ACI ICS Activity Risk-Sharing Arrangements Party Share of Risk Comments Cooperative 20% Pre-payment by co-op TechnoServe 10% From ACI-financed guarantee fund Input supplier 10% Receives 90% of invoice price when inputs received by co-op, 10% upon repayment Exporter 0% On delivery of cocoa beans, deducts loan repayment before remitting the balance to farmers Financial institution 60% Funds the loans to the co-op Under this arrangement, individual members screened by the cooperative would receive funds for inputs, up to a defined maximum (in the first year, inputs for one hectare of cocoa). In principle, if any members of co-ops were SSPs, they could receive financing as well for their SSP business activities. In principle, each member of the co-op would contribute his or her share of the required 20% down payment. However, it is the cooperative’s responsibility to organize the down-payment. In a step towards sustainability of the model, in the ACI ICS, TNS staff did not support the FI debt collection activities as they did in the CLP model. 23 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS I. Overall Findings, Conclusions, and Recommendations The matrix in Table 11 summarizes findings, conclusions, and recommendations for all five evaluation questions. Note that because of the similarity between Question 1, Question 4, Question 5 and findings, conclusions and recommendations, there is some overlap in the cells. Table 11. Findings, Conclusions, and Recommendations Matrix Findings, Conclusions, Recommendations Matrix Evaluation Questions Findings Conclusions/Lessons Learned Recommendations EQ1. How did the ACI activities contribute or likely contribute to the objectives and results as intended for the project overall and its four components? Overall: Project activities appear to have raised or are likely to raise farmer cocoa productivity and increased food security at household, community, and regional levels. Significant delays in implementation of components reduced impact. Overall: 1) Components 1–3 are largely capacity-building initiatives that should not have been expected to achieve stated objectives during project life. 2) Scale and sustainability negatively affected by lack of a market facilitation approach, especially for Component IV. 3) Better integration of individual components would have yielded synergies that would have increased impact. Overall: A proactive, market facilitation approach should be applied to pilot testing. Components should be actively managed to ensure synergies, e.g., linking finance and the SSP program. EQ1, Component I Component I: Useful forum for communication among stakeholders, but lack of responsiveness by government to key proposals and ineffective farmer representation. CDI platform more effective than Ghana’s. ACI not effectively using platforms to engage in evidence￾based policy dialogue. Impact on ACI objectives not tangible. Component I: Inadequate use of evidence￾based policy advocacy meant missed opportunity in dealing with difficult but crucial problems facing cocoa sector. Component I: Explore pilot testing of market-based alternatives to subsidization and state participation in markets, such as e￾vouchers. Actively engage BFS and the USAID missions on policy dialogue. Evaluation Questions Findings Conclusions/Lessons Learned Recommendations 24 Findings, Conclusions, Recommendations Matrix Evaluation Questions Findings Conclusions/Lessons Learned Recommendations EQ1, Component II Component II: Fingerprinting a key tool for countries to support breeding and planting material propagation, but very small sample (5%) tested, and still not clear how findings will be implemented. Seed and bud garden expansion well executed but insufficient to meet demand. Impact on ACI objectives likely to come after project completion. Component II: The NARS lack clear plan on how to act on off-type information from fingerprinting; less than effective distribution channels from seed gardens to farmers will reduce impact. More emphasis should be placed on developing more productive, more flavorful and resilient varieties, not just cleaning up existing varieties. Component II: Assist NARS to prepare action plans to resolve off-types issue. Research, advocacy, and pilot testing on strengthening supply chain of planting materials to farmers, including vegetative propagation. Focused support for developing more productive and resilient cocoa varieties and farming systems. EQ1, Component III Component III: Extension trainings in both countries viewed as upgrades by both agents and farmers, but less successful building capacity with extension agencies to deliver future training. GQI successfully introduced first cocoa flavor laboratory, upgraded extension on flavor training, and revived introduction of fine flavor cocoa. Impact on ACI objectives depends on a sustainable business case for producing better flavor cocoa, which has not been made. Component III: Extension agent training doesn't address a primary constraint—ability to reach farmers because of limited resources. GQI initiative not clearly based on realistic market assessment and development of an attractive business model for farmers. Component III: Extension: Build capacity of extension agencies to do their own upgrading and refresher training, explore ways to leverage more resources for extension. GQI: Conduct market entry assessment for high-quality cocoa from Ghana/West Africa. EQ1, Component IV Component IV: SSPs are delivering more effective and safer services to farmers, but actual improvement over base case not clear and sustainability of business model not established. Gross margin analysis of SSP beneficiaries indicates income from cocoa increased by nearly 100%, although gaps in methodology and data makes this finding tentative. Input credit scheme pilot has been adopted and scaled up by MFI in Ghana; impact on ACI objectives high. Component IV: Given low industry interest in SSP model scale-up, limited business acumen of SSPs, and ineffective linkage to credit, a likely high rate of dropout of SSPs and low scalability of current model. Input credit scheme pilot focused on fertilizer, but could have been extended to other cocoa production needs such as mist blowers, pruners, which would have reinforced Component III training. Component IV: SSP: Pursue market facilitation approach for SSPs more effectively, linking them to suppliers, finance, and customers. Consider expanding SSP services offered and crops serviced. Credit: Expand financial sector support to cover other working and fixed-capital needs for cocoa production and cocoa household needs. Evaluation Questions Findings Conclusions/Lessons Learned Recommendations 25 Findings, Conclusions, Recommendations Matrix Evaluation Questions Findings Conclusions/Lessons Learned Recommendations 2. How likely are the ACI activities to positively (or negatively) affect food security among cocoa farmers and within cocoa-producing communities? Cocoa production, assuming adoption of GAP, appears in general to be slightly to highly positive for household food security, by generating surplus cash for the household to buy food, invest in growing cash crops, and protect themselves from shocks due to drought or other factors. Traditional land tenure systems not conducive to adoption of high-input, high-return cocoa production systems, reducing impact on food security. Cocoa is fairly susceptible to pests and disease relative to other key cash crops (rubber and oil palm) but offers a steadier price over time. Food insecurity reported by some cocoa farmers in CDI; none in Ghana. Food insecurity drought-related. Farmers are well aware of the need to diversify with food and other cash crops. Cocoa provides income for laborers and sharecroppers/caretakers who come from more food-insecure regions. Literature review: A positive relationship between well-managed cash crops and food security because of the cash generated, synergies between cash and food crops, better GAP for food crops. Cash crop risk should be mitigated through crop diversification at farm and community level. USAID investment in the cocoa sector is sensible as it diversifies USAID’s Feed the Future’s portfolio from an exclusive heavier focus on food crops to a potentially lucrative cash crop that is complementary to food crops. Cocoa-producing regions offer a country a means to offset food insecurity in less productive regions. However, future cocoa projects need to address the topic of food security more directly. • Research on crop choice, variety choice, and agronomic regimes for staple crops grown intercropped with or alongside cocoa that most cost-effectively mitigate drought, disease, and other risks to cocoa. • Research on cocoa varieties more resilient to drought and disease risks. • Establish and strengthen emergency social fund programs at the cooperative level, so that some cocoa earnings can be pooled to improve food security. • Establish or strengthen savings and household credit schemes that allow farmers to accumulate surplus or borrow to protect against drought and other risks. • Socioeconomic research based on a household expenditure survey on that trace use of additional cash earned from cash crops such as cocoa (i.e., on food versus other expenditures) 26 Findings, Conclusions, Recommendations Matrix Evaluation Questions Findings Conclusions/Lessons Learned Recommendations 3. What other benefits or drawbacks (i.e., unanticipated outcomes) occurred as a result of the program? Component I: PPP establishment, operations, and ability to tackle thorny issues slower than anticipated. Component I: The absence of a dedicated policy analyst on the ACI team and the ineffective use of evidence-based policy advocacy techniques reduced ACI impact on PPP platform. Component I: Embedded policy advisor and/or an independent cocoa research center to produce evidence-based research to support policy dialogue. Identification and strengthening of farmer organizations to participate effectively. Evaluation Questions Findings Conclusions/Lessons Learned Recommendations EQ3, Component II Component II: Degree of off-types and mislabeled trees higher than anticipated. Component II: The higher degree of off-types and mislabeling underscored how relevant and appropriate fingerprinting exercise was. Component II: Assist NARS to prepare action plans to resolve off-types issue. EQ3, Component III Component III: Ghana's positive interest in extension training not anticipated. The interest by cocoa buyers in the development of a “high quality stream in the river of cocoa” in Ghana was overestimated. Component III: ACI could have taken a more proactive engagement with GQI proposal to ensure it was in line with market realities. Component III: GQI: Conduct market entry assessment for high-quality cocoa from Ghana/West Africa. EQ3, Component IV Component IV: Industry CropLife members’ support for SSPs and SSPs’ ability to operate as businesses overestimated. SSPs providing pest scouting and GAP advisory services (e.g., pruning, land clearing). Original model for input credit proposed by TechnoServe completely unworkable, not anticipated. Component IV: ACI could have taken a more proactive and critical approach to the TechnoServe proposal to ensure it was based on understanding of market realities. Component IV: A proactive, market facilitation approach should be applied to pilot testing. 27 Findings, Conclusions, Recommendations Matrix Evaluation Questions Findings Conclusions/Lessons Learned Recommendations 4. What are the most important lessons learned from implementing ACI that would inform similar efforts? Overall: Project activities appear to have raised or are likely to raise farmer cocoa productivity and increased food security at household, community, and regional levels. Significant delays in implementation of components reduced impact. Overall: 1) Components 1–3 largely capacity￾building initiatives that should not have been expected to achieve stated objectives during project life. 2) Scale and sustainability negatively affected by lack of a market facilitation approach, especially for Component IV. 3) Better integration of individual components would have yielded synergies that would have increased impact. Overall: A proactive, market facilitation approach should be applied to pilot testing. Components should be actively managed to ensure synergies, e.g. linking finance and the SSP program. EQ4, Component I Component I: Useful forum for communication among stakeholders, but lack of responsiveness by government to key proposals and ineffective farmer representation. CDI platform more effective than Ghana’s. ACI not effectively using platforms to engage in evidence￾based policy dialogue. Impact on ACI objectives not tangible. Component I: Inadequate use of evidence￾based policy advocacy meant missed opportunity in dealing with difficult but crucial problems facing cocoa sector. Component I: Explore pilot testing of market-based alternatives to subsidization and state participation in markets, such as e￾vouchers. Actively engage BFS and USAID missions on policy dialogue. Evaluation Questions Findings Conclusions/Lessons Learned Recommendations EQ4, Component II Component II: Fingerprinting is a key tool for countries to support breeding and planting material propagation, but very small sample (5%) tested and still not clear how findings will be implemented. Seed and bud garden expansion well executed but insufficient to meet demand. Impact on ACI objectives likely to come after project completion. Component II: The NARS lack clear plan on how to act on off-type information from fingerprinting; less than effective distribution channels from seed gardens to farmers will reduce impact. More emphasis should be placed on developing more productive and resilient varieties, not just cleaning up existing varieties. Component II: Assist NARS to prepare action plans to resolve off-types issue. Research, advocacy, and pilot testing on strengthening supply chain of planting materials to farmers, including vegetative propagation. Focused support for developing more productive and resilient cocoa varieties and farming systems. 28 Findings, Conclusions, Recommendations Matrix Evaluation Questions Findings Conclusions/Lessons Learned Recommendations EQ4, Component III Component III: Extension trainings in both countries viewed as upgrades by both agents and farmers, but less successful building capacity with extension agencies to deliver future training. GQI successfully introduced first cocoa flavor laboratory, upgraded extension on flavor training, and revived introduction of fine flavor cocoa. Impact on ACI objectives depends on a sustainable business case for producing better flavor cocoa, which has not been made. Component III: Extension agent training doesn't address a primary constraint—ability to reach farmers because of limited resources. GQI initiative not clearly based on realistic market assessment and development of an attractive business model for farmers. Component III: Extension: Build capacity of extension agencies to do their own upgrading and refresher training, explore ways to leverage more resources for extension. GQI: Conduct market entry assessment for high-quality cocoa from Ghana/West Africa. EQ4, Component IV Component IV: SSPs are delivering more effective and safer services to farmers, but actual improvement over base case not clear and sustainability of business model not established. Gross margin analysis of SSP beneficiaries indicates income from cocoa increased by nearly 100%, although gaps in methodology and data makes this finding tentative. Input credit scheme pilot has been adopted and scaled up by MFI in Ghana; impact on ACI objectives high. Component IV: Given low industry interest in SSP model scale-up, limited business acumen of SSPs, and ineffective linkage to credit, a low likelihood of sustainability of current model. Input credit scheme pilot focused on fertilizer, but could have been extended to other cocoa production needs such as mist blowers, pruners, which would have reinforced Component III training. Component IV: SSP: Pursue market facilitation approach for SSPs more effectively, linking them to suppliers, finance, and customers. Consider expanding SSP services offered and crops serviced. Credit: Expand financial sector support to cover other working and fixed-capital needs for cocoa production and cocoa household needs. Evaluation Questions Findings Conclusions/Lessons Learned Recommendations 5. How might the project or successor initiative be designed to more cost￾effectively and sustainably achieve and measure objectives, results, and outcomes? Overall: Project activities appear to have raised or are likely to raise farmer cocoa productivity and increased food security at household, community, and regional levels. Significant delays in implementation of components reduced impact. Overall: 1) Components 1–3 largely capacity￾building initiatives that should not have been expected to achieve stated objectives during project life. 2) Scale and sustainability negatively affected by lack of a market facilitation approach, especially for Component IV. 3) Better integration of individual components would have yielded synergies that would have increased impact. Overall: A proactive, market facilitation approach should be applied to pilot testing. Components should be actively managed to ensure synergies, e.g., linking finance and the SSP program. 29 Findings, Conclusions, Recommendations Matrix Evaluation Questions Findings Conclusions/Lessons Learned Recommendations EQ5, Component I Component I: Useful forum for communication among stakeholders, but lack of responsiveness by government to key proposals and ineffective farmer representation. CDI platform more effective than Ghana’s. ACI not effectively using platforms to engage in evidence￾based policy dialogue. Impact on ACI objectives not tangible. Component I: Inadequate use of evidence￾based policy advocacy meant missed opportunity in dealing with difficult but crucial problems facing cocoa sector. Component I: Explore pilot testing of market-based alternatives to subsidization and state participation in markets, such as e￾vouchers. Actively engage BFS and USAID missions on policy dialogue. EQ5, Component II Component II: Fingerprinting a key tool for countries to support breeding and planting material propagation, but very small sample (5%) tested and still not clear how findings will be implemented. Seed and bud garden expansion well executed but insufficient to meet demand. Impact on ACI objectives likely to come after project completion. Component II: The NARS lack clear plan on how to act on off-type information from fingerprinting; less than effective distribution channels from seed gardens to farmers will reduce impact. More emphasis should be placed on developing more productive and resilient varieties, not just cleaning up existing varieties. Component II: Assist NARS to prepare action plans to resolve off-types issue. Research, advocacy, and pilot testing on strengthening supply chain of planting materials to farmers, including vegetative propagation. Focused support for developing more productive and resilient cocoa varieties and farming systems. Evaluation Questions Findings Conclusions/Lessons Learned Recommendations EQ5, Component III Component III: Extension trainings in both countries viewed as upgrades by both agents and farmers, but less successful building capacity with extension agencies to deliver future training. GQI successfully introduced first cocoa flavor laboratory, upgraded extension on flavor training, and revived introduction of fine flavor cocoa. Impact on ACI objectives depends on a sustainable business case for producing better flavor cocoa, which has not been made. Component III: Extension agent training doesn't address a primary constraint—ability to reach farmers because of limited resources. GQI initiative not clearly based on realistic market assessment and development of an attractive business model for farmers. Component III: Extension: Build capacity of extension agencies to do their own upgrading and refresher training; explore ways to leverage more resources for extension. GQI: Continue the quality initiative if industry requests and include product market entry assessment for fine flavor and high-quality bulk cocoa from Ghana/West Africa. Expand initiative to CDI, pending confirmation of industry/ government support. Better organizational and business training for participating co￾ops. Build awareness and capacity of fine flavor within the COCOBOD structures. 30 Findings, Conclusions, Recommendations Matrix Evaluation Questions Findings Conclusions/Lessons Learned Recommendations EQ5, Component IV Component IV: SSPs are delivering more effective and safer services to farmers, but actual improvement over base case not clear and sustainability of business model not established. Gross margin analysis of SSP beneficiaries indicates income from cocoa increased by nearly 100%, although gaps in methodology and data makes this finding tentative. Input credit scheme pilot has been adopted and scaled up by MFI in Ghana; impact on ACI objectives high. Component IV: Given low industry interest in SSP model scale-up, limited business acumen of SSPs, and ineffective linkage to credit, a low likelihood of sustainability of current model. Input credit scheme pilot focused on fertilizer, but could have been extended to other cocoa production needs such as mist blowers, pruners, which would have reinforced Component III training. Component IV: SSP: Pursue market facilitation approach for SSPs more effectively, linking them to suppliers, finance, and customers. Consider expanding SSP services offered and crops serviced. Credit: Expand financial sector support to cover other working and fixed-capital needs for cocoa production and cocoa household needs. 31 Overall Findings The evaluation found concrete evidence that the ACI project activities have begun to raise cocoa productivity and are likely to raise farmer cocoa productivity during and beyond the life of the project. This is happening through improvements to good agricultural practices engendered by the project’s contribution to access to improved planting materials, improved extension, spraying services, and finance. For improved planting materials activities, most of the impact on farmers will occur after project completion due to delays in implementation and the long lead time between the expansion of seed gardens and farmers producing cocoa from improved planting materials. In addition, the impacts will be reduced by bottlenecks in the supply chain from seed garden to farmers due largely heavy government involvement. For improved extension services and finance, the project had not yet collected sufficient data to measure impact. Extension workers were pleased with the training and farmers were generally satisfied with extension services, but other factors will reduce the ability of farmers to translate their improved knowledge into higher productivity. For finance and spraying services, project calculations showed that these activities were associated with increased cocoa incomes of nearly 100%; however, the gains of a control group were not measured, so it is not possible to attribute the impact of the activities versus other factors. Overall, delays in implementation of components reduced impact. Overall Conclusions USAID/BFS support for the cocoa sector is a worthwhile complement to other Feed the Future activities that focus on edible/staple crop production. Based on the studies and analysis commissioned by WCF and others, well-managed cocoa farming is profitable under the market conditions that have prevailed during the project life, generating cash that can be used to purchase food, invest in food crop production, and produce savings that cushion shocks that could lead to food insecurity. While cocoa is grown in areas with high agricultural productivity that generally have low food security concerns, it provides the region with a cushion against food security shocks in lower agricultural regions by cash transfer, opportunities for labor, and entrance into cocoa farming by sharecroppers. The capacity-building and enabling environment strengthening initiatives of Components 1, and parts of Components 2 and 3 are longer-term and should not have been expected to tangibly contribute to the increasing income and food security during the project period. Evidence-based policy advocacy was not used effectively by project management in Component I. The lack of a proactive, market facilitation approach for parts of Components II and III, and for all of Component IV, reduced the depth, scale, and sustainability of impacts on income and food security. Overall, more proactive management of sub-awardees and better integration of individual components would have yielded synergies that would have increased impact on income and food security. Overall Recommendations  A market facilitation approach to pilot initiatives for private sector scale-up, where success is measured by the development of a sustainable and scalable credit model, not just by year-by-year metrics such as cost per farmer or net credit leveraged/cost of pilot. The project 32 team should include a full-time senior market facilitation specialist who proactively pursues opportunities to facilitate commercially sustainable inputs and service delivery to cocoa farmers, focusing initially on ensuring the commercialization of the spraying service provider (SSP) model introduced by ACI through CropLife. Market facilitation pilots should be afforded the flexibility and time needed to adapt their initial business model to market realities. The pilots should be afforded the flexibility and time (e.g., two years with an option for a one￾or two-year extension) for the facilitator and partners to adapt the model to market realities. Project management should have a full-time market facilitation specialist.  Proactive, evidence-based advocacy addressing policy issues of profound importance to the cocoa sector, especially in CDI and Ghana, including subsidized inputs, government control of planting material propagation and distribution, and land tenure. Project management should include a full-time policy advocacy specialist. Consideration should be given to supporting an independent cocoa policy research unit housed in the region.  More active project management that proactively identifies and promotes synergies between components managed by disparate sub-awardees and with other cocoa development projects would greatly increase impact. II. Findings, Conclusions, and Recommendations for Food Security This section addresses research question 2: How likely are the ACI activities to positively (or negatively) affect food security among cocoa farmers and within cocoa-producing communities? Might other activity designs and intervention areas lead to greater impacts on food security and income? The project design and development logic make the assumption that increased productivity of cocoa for smallholders and accompanying increased income will lead to improved food security through the greater ability to purchase food. Household incomes were increased, according to ACI reports. Household gross margins were reported to increase from $509 at baseline to $932 last year. However, measurement of this indicator was limited to a sample of farmers who received SSP services. Nonetheless, the evaluation team’s conclusion, detailed earlier in the Component sections, is that there is evidence that all of the ACI component activities individually and holistically are likely to have a net positive impact on household incomes. This section sets out to determine the relationship between cocoa and food security and how likely ACI was to have contributed to improved food security. Findings Literature review of cocoa, tree cash crops, and food security A literature review of the relationship between cocoa and food security was undertaken to help frame the evaluation approach and methodology. No published literature could be found dealing specifically with cocoa and food security, and so the review was expanded to include other cash crops as well. Specifically, the comparative advantages of cocoa versus palm oil and rubber—the two other tree cash crops that are alternatives to cocoa in Côte d’Ivoire and Ghana—were singled out. 33 The literature review supports the ACI development hypothesis that cash crops support food security, citing general examples that are applicable to cocoa. This general support was validated by the field observations discussed in the following section. Wiggens et al. (2015) conducted a literature review and qualitative research to determine the relationship between industrial crops13 including cocoa and food security in four countries in sub￾Saharan Africa including Ghana. They find that farmers of industrial crops often themselves are concerned about food security and that food production does not drop despite increases in industrial crop output. They conclude the following: “Direct observations of food security and nutrition of households growing industrial crops rarely show them as having worse outcomes than control groups. Again, however, the few observations of workers on plantations can provide some counter-examples, but the significance of so few observations is not clear… Overall, it seems that most often growing industrial crops is likely to improve food and nutrition security, owing to higher incomes earned and the complementarities that can apply between growing industrial and food crops (Wiggens et al., 2015).” Achterbosch et al. (2014) conducted a literature on the relationship between cash crops14 and food security. They concluded that cash crops are an important part of food security strategies. Not only do cash crops provide cash that can be used for food, but the cash can also be reinvested in food crops, increasing the productivity of food for those crops. Methodologies and technologies adopted for cash crop production may also spur agricultural innovation. In addition, cash crops provide employment opportunities including for those prone to food insecurity. They caution, however, that risks associated with cash crops, such as price volatility and weather risks, must be mitigated. A key part of this mitigation strategy is crop diversification, both at the farm and community levels. LMC International (2014) carried out an ACI-commissioned report comparing cocoa, oil palm, and rubber over a five-year period, which can help with identifying cocoa’s specific role as a cash crop contributing to food security. In Côte d’Ivoire, cocoa was found to be the least profitable of the three crops. Rubber was the most profitable in both Côte d’Ivoire and Ghana. In Ghana, research found that cocoa profitability increases with input intensification; however, labor requirements were high for cocoa at these higher input levels so that returns per unit of labor declined. Relatively, oil palm and rubber were less labor-intensive. Compared to oil palm and rubber, the study found that cocoa was found to remain popular in West Africa because farmers are familiar with the crop and price volatility is relatively minimal, with profitability increasing since 2009. Farmers were reluctant to switch in and out of tree crops because of the high expense in establishment and lag terms of cash flows and subsequent return on investment. Food security observations from the field In Côte d’Ivoire, the evaluation team held an FGD where some cocoa farmers reported times when they did not have enough to eat as a result of the impacts of drought. However, these farmers noted that food was always available in the market if they had cash to purchase it. In Ghana, none of the FGD farmers reported food insecurity, even in times of drought. These observations are validated by the 34 maps in Figures 5 and 6. The Côte d’Ivoire map (Figure 5) shows that there are cocoa-producing regions that are also food-insecure; by contrast, in Ghana there is no overlap between food-insecure regions and cocoa-producing regions (Figure 6).15 This does not necessarily mean that there is no food insecurity in cocoa regions in Ghana; it just means that the problem appears to be less pronounced. The fact that WCF member companies and the government have developed school feeding programs in the cocoa regions of Ghana indicates that there are some food security issues. Figure 5: Food Insecurity and Cocoa Production, Côte d’Ivoire 16 Figure 6: Food Insecurity and Cocoa Production, Ghana 17 insecurity 35 The following key issues relating cocoa and food security were uncovered during the evaluation: Mono-cropping vs. diversification. Extension services and farmers are both aware of the risks of mono-cropping a nonedible cash crop. Crop diversification is encouraged by government extension services as well as company and donor programs and appears to be widely practiced. The main staple crops grown in cocoa-growing regions are plantains and cassava, which are often intercropped with cocoa. This is especially true during the first several years of establishment of a cocoa plantation, where these plants beneficially shade the young cocoa. After a few years, the cocoa canopy closes, and it is too shady to grow food crops except at the margins of lands or where a tree has died or been removed. Because cocoa is grown in relatively productive agricultural land, food crop yields are also relatively high, which means farmers can gain grow a food security staple crop with less land. Drought and disease. As mentioned earlier, drought is the main risk factor cited by FGD cocoa farmers who experienced food insecurity. Some cocoa cooperatives help manage this risk for their members through social emergency funds, which are funded by cocoa proceeds. In addition, traditional community social arrangements provide help for those particularly in need. However these are obviously strained by drought, which affects all member or community farmers. Cocoa is prone to disease and pests; in West Africa, CSSV and black pod are serious diseases with a significant impact on cocoa returns, but FGD participants did not consider them as serious as drought. Therefore, it appears that, in general, cocoa farmers face food insecurity more in times of drought and less in times of cocoa￾specific risks. In addition, some cocoa farmers also have oil palm, rubber, and coconut, which further protect farmers against cocoa-specific risks. Sharecropping and labor. Cocoa appears to have a positive effect on food security through sharecroppers and laborers who come from less food-secure areas to take care of farms. In Côte d’Ivoire, many caretakers and laborers are from Burkina Faso and Mali. In Côte d’Ivoire, only nationals are allowed to own land. In Ghana, labor and caretakers commonly come from the more food-insecure north of the country. However, in Ghana, caretakers and sharecroppers are not allowed to use cocoa land for food cropping under traditional sharecropping agreements. The farmers in the FGD sharecropper discussion reported that they were able to plant food crops on the margins of the lands, where trees had been removed, and with other lands that they had access to. Nevertheless, the lack of dedicated land for planting food for caretakers may indeed be an issue for food security and should be researched further. Although the evaluation team was not able to confirm it directly, it appears likely that the wages paid to day laborers by cocoa farmers also would have a positive effect on food security. Market risk. Cocoa farmers in Côte d’Ivoire and Ghana do not face much market risk. In Côte d’Ivoire, the government sets a minimum buying price every year and in Ghana the farm gate price is fixed. Most farmers do not appear to have a problem finding a buyer for beans at these minimum prices, although there are reports of farmers who are strapped for cash selling to traders below the minimum price. However, there is currently a market risk with farmers producing cocoa under one or more of the three main certification schemes. According to the Cocoa Barometer (2015), less than 50% of cocoa produced as certified worldwide is actually sold as certified. This means that co-ops and farmers who have gone to the expense of getting crops certified are only paid a premium on less than 50% of their sales. Although most chocolate companies have indicated that they will buy 100% certified cocoa by 2020, the premiums that will be paid for this certification are not known. Low productivity and shrinking farm sizes. One estimate shows that as much as 44% of cocoa farmers do not earn positive cash flows from their cocoa plantations in Côte d’Ivoire, and there is no 36 reason to think the situation is different in the other cocoa-growing countries in West Africa.18 Farmers in this category do not have positive food security benefits and in fact are likely to remove their cocoa and plant other crops if they have the means and will to do so. The main objective of ACI (and CLP, and most other cocoa-sector projects) is to increase productivity. However, cocoa is not the easiest crop to manage, and it is highly likely that there is a significant share of cocoa farmers who cannot or will not take steps to make their cocoa farming operations sufficiently productive to contribute to food security. This can be related to the increasingly shrinking size of cocoa farms, which are sub-divided upon the death of the owner. Spillover of skills and knowledge from cocoa to food crops. WCF and others stakeholders make the assertion that GAP and business management skills introduced for cocoa are generalizable and will have spillover effects with other crops. The idea is that if you become a good cocoa farmer, you will be a better food crop farmer. This was supported by our brief literature review, which cited technology and innovation spillovers from cash to food crops. The evaluation team did not find direct evidence of this during the fieldwork. However, some of the interventions introduced by ACI, such as the use of SSPs, are intended, at least eventually, to be useful for food crops. Conclusions ACI activities have positively affected or are likely to positively affect food security among cocoa farmers and within cocoa-producing communities and on a regional basis. The engine of this increased food security is greater cocoa productivity, generating surplus cash for the household to buy food, grow cash crops, and protect themselves from shocks due to drought or other factors. In addition, cocoa is labor￾intensive, providing jobs for those who may be food-insecure. It also provides sharecropping/caretaking opportunities for people from more food-insecure regions. A concern often expressed about cash crops and cocoa with respect to food security is that they may lead to more food security risk because of a greater degree of mono-cropping. However, the evaluation team found that farmers and extension staff were well aware of the risks of the need for crop diversification. Therefore, the risk of cocoa mono-cropping or even significant competition for food crop land from cocoa cultivation does not appear to be substantial. This is corroborated by the limited literature review. However, this, like the other findings and conclusions in this section, is tentative and needs to be confirmed through more rigorous research. The key factor that appears to affect the food security changes associated with cocoa farming is drought. That is, are farmers more food-secure having planted cocoa than they would have been using that land and capital for another crop? From the FGD, the answer seems to be yes, but this is from a tiny sample of farmers. The answers to this question are complex and depend on the relative resistance to drought of cocoa versus other crops, their relative profitability, the household’s ability to save cocoa surplus/borrow funds, etc. The evaluation was not able to assess whether farmers are planting the appropriate food crop varieties that optimally manage drought risk for their region, given their commitment to cocoa. While cocoa appears to have a positive effect on food security, it is less clear if the same investment in a different crop by BFS in the four ACI countries might have yielded a greater food security effect. This is a very difficult question to answer conclusively. There are, however, several potential advantages that cocoa and cash crops in general have over traditional food security staple crops. One is that donor investment can leverage significant private-sector matching and therefore increased food security return on investment. Another is that the food security that does exist among cocoa producers may be 37 relatively “low-hanging fruit,” allowing modest investment to generate significant food security, which improves increases in productivity. These increases in productivity and profitability inject more cash in the local economy, spurring community economic development and potentially alleviating some community-level food insecurity. Finally, cocoa provides employment opportunities for locals who may be food-insecure as well as sharecropping opportunities for those from more food-insecure regions. The evidence from this evaluation indicates that some investment in the cocoa sector is a prudent part of a diversified food security strategy in West Africa. However, a more definitive answer can only be obtained through more rigorous and comprehensive research on the issues raised in this section. Recommendations Most recommendations covering food security have been covered in the earlier sections. However, food security–specific recommendations for a follow-on cocoa sector project include support for:  Research on crop choice, variety choice, and agronomic regimes for staple crops grown intercropped with or alongside cocoa that most cost-effectively mitigate drought, disease, and other risks to cocoa.  Research on cocoa varieties more resilient to drought and disease risks.  Research into “best bet,” cost-effective irrigation methods for cocoa.  Establish or strengthen savings and household credit schemes that allow farmers to accumulate surplus or borrow to protect against drought and other risks.  Establish and strengthen emergency social fund programs at the cooperative level, so that some cocoa earnings can be pooled to improve food security.  Socioeconomic research on household expenditures that trace, through a household survey, how households use the additional cash earned from cash crops such as cocoa (i.e., on food versus other expenditures). III. Findings, Conclusions, and Recommendations for the Four Components Component I: Establish and Strengthen National Public-Private Partnership (PPP) Platforms Component I—PPP Platforms Overall Findings In general, cocoa stakeholders interviewed supported the principle and concept of the PPP platform and appreciated the opportunity provided by the PPP plenaries and TWGs to share views and keep abreast of industry trends and policies. WCF members have encouraged their local offices or their supply chain partners to participate both at plenary sessions and in the TWGs. Some WCF members expressed ambivalence about the PPPs. They believed that their individual and industry interests might be more effectively represented through direct representation by WCF staff or by the WCF CocoaAction steering committee. The CocoaAction strategy development and implementation process seems to have preoccupied the private sector to the detriment of their participation in the PPP platforms, and in some WCF members there is a conflation of PPP and CocoaAction activities. This concern notwithstanding, CocoaAction has expressed ongoing commitment to the PPP structures and is financing a full-time staff member to be a representative (without ACI co-funding). 38 WCF states in its annual and semi-annual reports that it is “designing and implementing evidence-based policies in the cocoa value chain” through the Component I PPP platforms. There was limited concrete evidence that this has occurred, based on discussions with WCF and stakeholders. ACI has commissioned work whose findings may have contributed to evidence-based policy advocacy under Component I. This includes the Cocoa Comparative Household Economy study, which analyzed the relative returns of cocoa, rubber, and oil palm in Ghana and Côte d’Ivoire. 19 One of the key findings of the study was that in the time period analyzed, cocoa did not generally offer the highest returns, but it did offer the least volatile returns. It is not clear what the impact of the study or this finding has been; WCF could not identify to the evaluation team what specific policies or issues were influenced by the study.20 Another example of an activity that may have been used for policy advocacy was the Soil Fertility Workshop ACI of March 2013, funded by ACI (under Component IV). Based on a review of the Workshop Report, the workshop provided potentially valuable output for policy advocacy. Some of the issues were discussed in the PPP TWGs, but little that is concrete seems to have emerged from these discussions based on the evaluation KIIs. Also, the activities financed by ACI in Components II–IV yielded important data and insights that could be used in proactive, evidence-based policy dialogue. These are pointed out in more detail in the respective Component findings below, but these include propagation and distribution of planting material, extension service delivery and coordination, and farmer credit and access to services. Finally, WCF and its members are implementing a number of project activities, both through CLP and individually, that provide a strong base of evidence for policy changes in CDI and Ghana. Specific examples of other WCF or member projects include:  Clonal propagation activities supported by Mars  Seed propagation activities promoted by Mondelez  Cocoa Academy program activities by Cargill Ghana The cocoa industry stakeholders interviewed viewed the PPP as a useful and important vehicle with which to approach the Ghana Cocoa Board (COCOBOD) with a single voice on certain key issues. This is seen as particularly important in the coordination with COCOBOD on input delivery activities such as distribution of seedlings. However, it is difficult to identify concrete actions that have resulted from the PPP structures and processes, let alone trace the impact of ACI’s support for the PPPs to concrete results. The harmonization of cocoa extension manuals is cited as an example of concrete actions resulting from the PPP, although WCF is still in the process of refining the scope of work and identifying a contractor, and work may not begin until 2016. In principle, the ACI-supported Scenario Planning Workshops in 2014 and 2015 have provided inputs to Ghana’s next ten-year Cocoa Plan. However, the only substantive output from the Scenario Planning Workshops that WCF identified was a proposal to establish a more accurate and complete farmer database. It is not clear whether this is actually a new idea or simply one validated by the workshops, 19 “Cocoa Comparative Household Economy Study: West Africa Smallholder Cocoa Farmers,” LMC International, February 2014. 39 and at any rate, the key point is whether the Government of Ghana (GoG) and COCOBOD will accept and implement this recommendation. There are concerns from PPP members outside of COCOBOD:  Many of the ideas and proposals coming from the TWGs and the PPP plenary sessions are not considered or acted upon by COCOBOD. This seems to be a particular issue when proposals address the role of COCOBOD in the distribution of subsidized inputs and services to cocoa farmers. This also appears to be the case on broader, important—but politically tricky—issues such as land tenure.  TWG members have lost their commitment to participating actively, in large part because of the perceived lack of responsiveness.  The PPP platform should be consolidated with similar initiatives (e.g., the CocoaAction Framework Platform) to reduce redundancy. There is now guarded optimism that the Ghana PPP platform will be re-energized, since the Cocoa Health and Extension Department (CHED) of COCOBOD took over coordination and management from a UNDP-funded independent consultant early in 2015. This recognizes that COCOBOD is in fact the key decider or implementer of cocoa-sector policy. There is concrete evidence of renewed action by the PPP platform under COCOBOD’s management. The PPP platform has circulated a UNDP-funded study on the impacts of the boom in artisanal, informal gold mining on the cocoa sector (e.g., environmental damage, labor shortages, and removal of land from cocoa production). Interestingly, this study found the impacts to be negligible despite stakeholder perceptions, and no major policy action was recommended. Cocoa farmers are not well represented on the PPP or the TWGs. There are individual co-ops or associations represented in the PPP institutions. The PPP platform, through COCOBOD, has consulted widely with farmers in 2014. However, there is no strong national organization representing cocoa farmers. One possibly suitable organization, the Ghana Cocoa, Coffee, and Shea-Nut Farmers Association (CoCoShe), was viewed by many to be too narrowly concerned with cocoa prices and maximizing subsidized input/service delivery by COCOBOD, as well as aligned with political interests, and therefore not a body that truly represents the broader interests of cocoa farmers. CDI As with Ghana, the PPP platform in CDI is seen as a valuable venue and process for government, the private sector, and civil society to share views and stay up to date with industry trends and government programs. The CDI PPP platform was viewed as more active from the beginning of the project than in Ghana, and this is remarkable given that the platform was established just as the country had emerged from a political crisis. Initially, there was a guarded optimism that the PPP platform would facilitate the adoption of sensible farmer- and market-oriented cocoa policies. The Government of Côte d’Ivoire (GoCDI), through the Conseil Café-Cacao (CCC, the GoCDI entity that oversees the cocoa and coffee sectors) and private stakeholders, sees the PPP platform as playing a primary role in the updating and implementation of the CDI national cocoa policy (2QC 2014–2023): 40 “This [PPP] platform will allow optimization of available resources and collaborative sharing of mutual interest programs to implemented on the ground.” 21 Stakeholders mention that the PPP has led to a greater emphasis on productivity, access to planting materials, and livelihood diversification in the new national cocoa policy. The leadership and technical support provided by the WCF team at the plenaries and TWG meetings were appreciated by respondents. Specific outcomes cited by stakeholders to be partially or completely a result of the CDI PPP platform structures and processes include:  Training manual harmonization.22  PPP agreements with cocoa value chain companies to construct school buildings and health clinics for operation by the government.  Quick action by the CCC to form a seventh TWG dealing with CSSV in 2015, cited as a recognition that the PPP platform is the “go-to” venue for public-private collaboration on key issues facing the cocoa sector.  A vehicle for interfacing with government on requests that were previously made by individual companies, such as for planting materials for farmers with whom individual companies work.  Acceptance of the importance of better collection and sharing of monitoring and evaluation (M&E) of disparate cocoa development activities.  The beginning of a more open approach to the use of grafting and somatic embryogenesis for planting material propagation. Nonetheless, significant concerns about the PPP platform were raised by private-sector stakeholders, and the initial enthusiasm in which they viewed the PPP has waned, as in Ghana. Specific issues noted include:  The lack of responsiveness or follow-through from CCC to many of the ideas and proposals that come from the plenary or TWGs. Typical comments were that CCC is a “black hole,” or that the PPP was a “rubber stamp” for CCC decisions.  The lack of commitment from CCC to provide tangible support through its staff or technical assistance to the PPP secretariat, plenary, and TWG structures and operations.  The lack of commitment from the private sector to participate in the TWGs, stemming from frustration over the lack of responsiveness from CCC.  The CSSV crisis is being used as an excuse to slow down private-sector participation in developing and distributing planting material . The CCC manager acting as chief liaison to the CCC was pleased with ACI financial support and was especially appreciative of the technical input from the WCF staff. However, he believed that ACI could offer far more in the way of technical support to the platform rather than just budgetary support and WCF staff participation. In particular, he mentioned the following areas:  Pricing policy research  Farmer financing options 21 Republic of Côte d'Ivoire Ministry of Agriculture, “2QC” Abstract, March 2014, unofficial translation from French. 22 The training manual harmonization was important, because the plethora of training materials that previously were being used and circulated by public and private extension agents were considered to be confusing and sending mixed messages to farmers. 41  Climate change response options  CSSV early detection mechanisms  Flavor lab/sensory panel (as with the GQI) As in Ghana, truly representative farmer organizations are not present in the PPP structures. There are some cooperative/society farmer representatives, but they do not (and cannot be expected to) articulate national concerns. A key problem is the lack of a national, credible organization representing cocoa farmers. One option, the CDI national office of World Cocoa Producers, is only just beginning to have a national footprint and cannot yet be considered the voice of the cocoa farmers. Component I—PPP Platforms Conclusions The goals and objectives of Component I remain relevant and important, the slow pace of implementation notwithstanding. However, a number of factors have reduced the effectiveness of ACI interventions. The ability of ACI to shape PPP policy advocacy has been significantly hampered by the low use of specially commissioned policy research or the use of evidence that has come from implementation of ACI and other WCF or individual company initiatives. As noted earlier, WCF plays a dual role in the PPP: 1) a provider of financial and objective technical assistance and 2) an advocate for the international cocoa industry. As a consequence, WCF may not be an entirely objective arbiter of policy identification, prioritization, and advocacy, especially for sensitive policy issues where stakeholder interests diverge. Furthermore, WCF members active in Ghana and CDI are heavily dependent on COCOBOD and CCC. Naturally, they would not want to offend either institution with overly forceful advocacy of sensitive policy issues, particularly if they are not directly bottom line–oriented, even though these issues might be of great importance to cocoa farmers. Component I—PPP Platforms Recommendations The following are suggested to increase the pace and depth of achieving those objectives in a follow-on activity.  The successor project should have a full-time policy advisor (as was originally anticipated), with strong evidence-based policy skills and the ability to develop trust and rapport with private and public stakeholders. An experienced COP will greatly augment and support this position but cannot do both jobs effectively.  The policy advisor would: o Develop a time-bound workplan of action with benchmarks and indicators for key policy issues. o Actively consolidate learnings from ACI project activities, other WCF activities, company programs, donors, and NGOs such as IDH or Solidaridad to use in evidence￾based policy dialogue in PPP plenary and TWG discussions. o Commission action-oriented research on key problems and issues facing the cocoa sector, with a view to using that research to build consensus and develop concrete steps for action. Ensure that both private and public sectors “own” the research by including it in design (and implementation when feasible). o Update on a regular basis progress to key stakeholders to obtain input and build consensus. Roadblocks and setbacks are just as important to communicate as successes, as they allow stakeholders an opportunity to contribute to problem solving. o Actively engage BFS and USAID missions with relevant portfolios (especially Ghana and Nigeria) to update them on policy issues and progress with policy dialogue, and seek out their technical input and advocacy support for key issues. 42  Explore the possibility of the establishment of an independent cocoa policy research institute for West Africa. The sources of funding could include foundations and donors but should exclude companies and governments to avoid partiality. This institute would publish research on a wide range of issues facing the cocoa sector, including the effects of input subsidies, how the sector will adapt to climate change, and market prospects for West African cocoa. The policy research institute may require institutional capacity building as well as program support, which could be implemented through a partnership with a US university.  Consolidate the CocoaAction Steering Committee, PPP platforms, and workstream/TWC assignments in Ghana and CDI to minimize redundancy of WCF participation.  Directly but sensitively address the “thorny” issues facing the cocoa sector; the most commonly cited were: o Planting material propagation and distribution o Subsidized input distribution o Land tenure systems and impact on cocoa productivity  Provide seed funding for pilots that could address some of the thorny issues facing cocoa industry. An example is a government e-voucher system for subsidized input distribution, which would allow precise geographic and needs targeting, facilitate private-sector supply chain development, and create valuable information on farmer input usage patterns.  Capacity-building support for national farmers’ associations to enable them to effectively represent cocoa farmers at the PPP on a national level, recognizing that in current circumstances, the development of such associations will not be a short-term objective. Component II: Address Farm Productivity Constraints through Improved Planting Material Component II—Planting Material Findings Genetic Fingerprinting About 10,000 cocoa tree samples (1,000 from breeders’ collection of germplasm and 1,000 from seed gardens) from each country were genetically fingerprinted, and where possible, the fingerprints were matched against reference samples. About 5% of the breeding lines and seed garden materials were tested. This was the first time such an exercise was undertaken in West Africa, and the results were eagerly anticipated. Through ACI support of ACBWG meetings, the results of the fingerprinting exercise were reviewed by the NARS. Breeders were provided with training and software tools that enabled them to understand the results and their implications for both breeding new varieties and propagation of hybrids already under release. The results showed that many of the trees in breeders’ collections and seed gardens were mislabeled. In some cases, it was difficult to even identify a reliable reference clone. The rate of mislabeling varied from country to country and from garden to garden, but on average it was reported to be approximately 20%. In general, the fingerprinting exercise showed that the mislabeling problem was significantly worse than expected. In CDI, approximately 2% of trees in the existing seed and bud gardens were tested, and approximately 17% were off-types. 43 In Ghana, 6 of 26 seed gardens were tested, along with 23 farms selected randomly.23 The rate of mislabeling mostly ranged from 5 to 10% depending on the garden, with the older gardens tending to be freer from contamination than the newer gardens, and some blocks within the newer gardens up to 35% mislabeled. Trees on the tested farms were found to be a mixture of traditional varieties, true-to-type hybrids, off-type hybrids from natural pollination, the farmers’ own selection actions, and off-types from the seed gardens. The genetic diversity of the NRA’s germplasm was found to be lower than expected, which negatively impacts regional breeding activities and the ability to respond to disease and pest outbreaks. The genetic fingerprinting activity was considered to be a resounding success by the governments and cocoa industry stakeholders and essential to ensuring reliable breeding and planting material propagation. The fingerprinting technology, and the means to interpret the results of the fingerprinting exercise, are well beyond the capabilities of the NARS. ACI intervention was therefore essential from both a technical and financial point of view. Also, the ACI-financed expansion of the seed gardens began before the results of the fingerprinting exercise were known, so that there would likely be mislabeled varieties in the new seed gardens (which have not been tested). In CDI, where CNRA is responsible for both breeding and planting material distribution, CNRA has indicated that it will take steps to restore the integrity of its new breeding gardens and seed gardens. In Ghana, CRIG is well aware of the problem and is taking steps to correct the problems in its breeding gardens. However, in Ghana, the Seed Production Division (SPD) of COCOBOD, actually responsible for propagation and distribution of seedlings to farmers, is organizationally separate from CRIG. The SPD director told the evaluation team that there were no off￾types in its new gardens, which does not seem credible. The ACI team is confident, however, that any mislabeling in the new seed gardens financed by ACI will eventually be identified and corrected by SPD, certainly before the gardens reach the pod production stage. The CRIG genetic fingerprint testing of farmers’ trees identified a potentially even more serious problem. While many of the farmers’ trees were off-type hybrids, this does not mean that they are any worse than traditional hybrids or even true-type hybrids. The most serious consequences occur when farmers receive planting materials from seed that was produced not from hybrids but from accidentally “selfed” parental clones that are intended only for breeding, not cocoa production. These selfed clones do not have hybrid vigor and in a cocoa plantation would likely be outcompeted by other trees, possibly not bearing any cocoa pods at all. CRIG reported that 18–20% of the farmers’ trees that were tested appeared to be from selfed seeds. Some WCF members expressed the view that as a result of the fingerprinting exercise there may be an overemphasis on cleaning up existing seed gardens, when the focus should instead be on developing new varieties with higher productivity, greater resilience, etc. Expansion of Seed and Bud Gardens In Ghana, 45 hectares of seed gardens over 26 stations have been planted; only 5 hectares remain to be completed. Seedling survival rates averaged 69% (with wide variation), attributed to the effects of drought at some of the SDP gardens that will require in-filling, which SDP commits to finishing by June 2016. CDI and Cameroon gardens have been completed, and Nigeria has its 15 hectares of seed garden remaining, although the land has already been prepared and planting materials are ready to be moved to 23 According to CRIG head breeder Dr. Francis K. Padi 44 the site. For the budwood gardens, CDI and Cameroon have finished grafting their 10 hectares each. Ghana has 4 more hectares to complete, and Nigeria has yet to begin grafting. The evaluation fieldwork confirmed that the expressed demand for high-quality planting materials from cocoa farmers in CDI and Ghana far exceeds their respective governments’ capacity to supply.24 The ACI-financed expansion of the seed gardens was a significant start to solving the shortfall in supply, and Ghana and CDI are well on their way to complete establishment as planned. The NARS (and where relevant, the seed production divisions) appears to have the internal capability of planning and executing seed garden expansion, but the budget support provided by ACI was viewed as helpful. The complementary genetic fingerprinting exercise was crucial to ensuring the integrity of this expansion. However, in both countries, meaningful production of seed pods will only occur in 2018, two years after the ACI project is finished. Demonstrations of New Varieties The ACI project assumption that farmers were not aware of the latest cocoa varieties released appeared be correct. But cocoa farmers interviewed were certainly aware of the value of improved planting materials and there is very high demand for what they consider new varieties. In CDI, farmers and ANADER agents report that less than 10% of their expressed demand was met for the so-called “Mercedes” variety (which could be one of up to eight of the varieties released in the past 10 years or so). The most desirable attribute of the “Mercedes” was that that pod production began after 18 months instead of three to four years,25 which expedites farmers’ earnings and reduces the riskiness of vulnerable and unproductive young trees. The farmers report that even when they followed ANADER’s instructions to prepare for new planting material (e.g., clear old or diseased trees, leave the right amount of shade trees, plant with banana, prepare bags with topsoil, and develop nurseries), they are only provided with a small fraction of what they were promised. Both Côte d’Ivoire and Ghana NRAs have established demonstration plots in five communities in the cocoa-producing regions. In Ghana, the communities showed little enthusiasm for the plots, as they were managed by CRIG researchers (by design), and were not “community-owned.” Only one plot per community has actually been established. Planting of the demonstration plots began only in 2014, with a higher-than-expected mortality rate in the newer varieties, which was thought to be due in part to the “selfing” problems discussed above. Any impact of this activity on farmers’ acceptance and knowledge of new varieties will only occur well after ACI project completion. ACBWG General Support The general support of the ACBWG meetings was seen as useful by participants, although determining where STCP left off and ACI picked up would be difficult. While a highly useful means of disseminating and discussing the results of the genetic fingerprinting exercise, ACBWG has yet to come up with concrete recommendations on what the next steps should be in terms of additional testing or protocols for addressing the problems in the gardens. Only one of the interviewees could point to a specific, tangible outcome or result of an ACBWG meeting (outside of genetic fingerprinting exercise issues). This outcome was a recognition that the on￾farm experience of farmers needed to be better incorporated into breeding through participatory 24 In both countries, planting material is the responsibility of the governments and is nominally free. If farmers paid a market price, the demand for planting material could be less but would still be very high. 25 It produces very fast, like a Mercedes car is perceived to go fast, hence the nickname. 45 variety research methods. This outcome appears to have led to the incorporation of the farmer demonstration activity (described above). Component II—Planting Material Conclusions Genetic Fingerprinting In principle, trees in the existing seed gardens that have been identified as off-type by the fingerprinting exercise can be replaced by using material from true-to-type tested clones from budwood gardens. The mislabeled trees could be removed and replaced with true-to-type budlings or top-worked or side￾grafted with the correct material. However, this corrective process would take years and would require further testing, which has not been funded, let alone fully considered. In the meantime, if mislabeled trees in the seed gardens are culled or grafted, as they should be, farmers’ access to planting material will be thereby reduced, not increased. Seed and Budwood Gardens The seed garden expansion will not be enough to meet cocoa farmers’ demand for planting materials, for a variety of reasons:  Seed gardens will only come into full production in 2018, and even when they do, meeting the governments’ estimates of demand would take from 4 (Ghana) to more than 11 years (CDI), according to ACI calculations. Many stakeholders, moreover, believe that government estimates of demand are on the low side.  The capacity of the old seed gardens in terms of productivity and presence of off￾types are not known. In CDI, CNRA acknowledged that the 90 hectares of seed gardens were there “on paper,” and their productivity and quality are not known. These old seed gardens will need to be rehabilitated.  The selfing issue identified by CRIG has not been resolved. As noted earlier, CRIG found that as much as 18% of its sample of farmers’ trees appeared to have been grown from selfed clones and not hybrids. Both CDI and Ghana rely on planting material production using the painstakingly slow and error-prone methods of hand-pollinated hybrid seed production, which leads to production of selfed pods. Clonal propagation of budlings from verified hybrid trees would eliminate this issue, but this has not been accepted. There are more fundamental concerns that may limit the impact of the ACI seed garden expansion, relating to the tight control on the supply chain from government nursery to the farmer.  In Ghana and CDI, the government maintains tight control over the propagation and distribution of seeds and seedlings, outside of limited private-sector pilot initiatives. In Ghana, only seedlings are distributed. Although they are nominally free, farmers are obliged to pay for transport, which is quite expensive. And, in part because of bad roads and poor equipment, the survival rate of the seedlings is low, even with 87 SDP-run “community” nurseries. In response, the government has committed to increase the number of SDP-run nurseries from 87 to 120 this year. Nonetheless, there remain concerns that the SDP nurseries are still too far away and that this is an unnecessary expenditure of time and money by SDP. Earlier, many farmers were encouraged by CHED to set up their own nurseries. With the new 46 policy, these efforts appear to be wasted, and their receptiveness to further government direction may be lessened.  In CDI, where planting material is also freely distributed by the government, the efficiency and transparency of pod distribution coordinated by ANADER, CNRA, and CCC are criticized. There are reports of farmers being asked to make informal payments in order to receive priority. It is suspected that well-connected cocoa farmers (e.g., retired civil servants) may get priority.  Private-sector offers to produce planting material on a commercial scale have not been embraced. Company pilots have been allowed to produce seed, some of them funded in part by CLP. These company pilots can provide data for an evidence-based policy dialogue that would feed into the larger issue of where and when the private sector should be allowed, or even encouraged, to do this on a larger scale, but this has not been addressed. For example, a private seed nursery project in Ghana, funded by Mondelez and operated by Tree Global, has begun to provide seedlings, and initial indications are that the seedlings produced have a higher survival rate than SDP distributed seedlings. However, this is not viewed as an official pilot; in fact, it is viewed with some suspicion within CRIG.  In Ghana and CDI, propagation of cocoa through vegetative means is not allowed outside of government facilities, except on a few carefully controlled private-sector pilot initiatives. Vegetative production would satisfy demand for new planting materials much more quickly and economically than relying only on seeds, and this is indeed a standard practice in many cocoa-producing areas outside of Africa. The main reason given is twofold: to control the spread of the deadly CCSV and to ensure genetic diversity in the national cocoa plantation. Both are justifiable concerns, though the government’s response is viewed by some as excessively conservative. It should also be recognized that CCSV has managed to spread quite extensively without vegetative propagation, and genetic diversity could still be maintained through other means. In addition, more could be done to ensure that the company-sponsored pilots are accumulating data and communicating this effectively through evidence-based policy dialogue. In CDI, Mars and the International Center for Research in Agroforestry (ICRAF) are working on a clonal propagation program that would enable virus-free planting materials, but this will take years to produce results and is still in the pilot stage.  In CDI, there was concern from stakeholders that there may be an intentional restriction in the distribution of new planting materials, to avoid a spike in production which could result in a collapse in prices. This concern was not directly voiced by CCC or other government entities, but the case of the rapid expansion of coffee in Vietnam in the 1990s and subsequent price collapse was often cited. The desire expressed was to focus on increased productivity but not necessarily production. However, the GoCDI does not control the amount of lands planted to cocoa, and there are reports of large new plantings in the west. Therefore, even if the GoCDI did restrict the amount of improved planting material distributed, some expansion in acreage would likely occur anyway. However, this would come from farmers’ own seeds, with less productivity and resilience, which would expand production but on a less competitive basis. Furthermore, the government is actively (and rightly) encouraging productivity gains through improved GAP and higher use of inputs and indeed encouraging farmers to borrow to do so when necessary. To intentionally handicap farmers who use GAP and who have borrowed for inputs by not providing improved planting materials would not be good policy. 47 Demonstrations of New Varieties The farmer demonstration activity appeared to be an attempt to combine two quite distinct goals: participatory variety research (PVR) of new varieties and farmer demonstrations of varieties already released. These two activities would likely have been more efficiently kept segregated because of the very different structure and management of these kinds of activities.  Participatory variety research is primarily used to provide input to breeders and should be jointly managed by breeders and participating farmers. The ACI initiative was a good start but, based on the field assessment, CRIG would benefit from support on designing and implementing PVR activities.  Farmer demonstrations of released varieties should be jointly managed by extension services (public and private) and participating farmers. As the research phase is complete, the involvement of NARS can be minimal. Component II—Planting Material Recommendations Genetic Fingerprinting In its remaining months, ACI should focus resources on analyzing and developing a response to the conclusions of the genetic fingerprinting exercise in the form of a concrete action plan. A first step would be to require clear, complete, and up-to-date reporting from SDP and CNRA (and the other regional agencies) on their ideas and plans for next steps. This may require outside technical support to complement the skills and experience within the ACBWG. Development of an action plan to be developed by individual NARS and coordinated by ACBWG, but with outside support as required, would address: o How much additional fingerprinting will be required in breeders’ gardens, seed gardens, and farmers’ plantations? o What are the next steps required for removing mislabeled varieties in the seed gardens? o How should the problem of selfing be addressed? o How should the low genetic diversity issue be addressed? Research In the next project, consideration should be given to a greater focus on research activities that will have a clear pay-off in the eventual development of cocoa varieties that are selected for productivity, pest and disease resistance (especially CSSV), and drought resiliency. Suggestions by stakeholders include: o Support to breeders to identify, isolate, clean, and maintain pure parent lines for breeding purposes, using advanced technological methods such as somatic embryogenesis (not currently available in the region). o Encouragement of means to facilitate the exchange of new germplasm within the region and access to international germplasm (some blockages appear to be policy-related and currently controversial, e.g., Ghana’s reluctance to adopt the 2013 Plant Breeders’ Rights Bill). Policy advocacy A follow-on project should use evidence-based policy advocacy to build consensus around research and solutions to the key problems identified by the genetic fingerprinting and seed/bud garden expansion exercise. For regional scientific issues, continued involvement of ACBWG as a key stakeholder and “owner” of policy research and advocacy would be desirable. The evidence can come from the analysis and communication of the results of pilot activities underway and commissioned research. Key issues identified include: o Clonal propagation as a means for more rapid, error-free propagation of planting materials. 48 o Private sector’s role in breeding, propagation, and distribution of planting materials. What can be learned and generalized from the various industry pilot programs? o Communities’ role in propagation and distribution of planting materials. What can be learned from the experience in rubber and oil palm industries? o Free planting materials—in what circumstances is this the most efficient and cost-effective way to rapidly distribute planting material? In what circumstances would partial or complete cost recovery make sense, considering that the purchaser may be more likely to value and take care of the planting material than if received free? ACBWG ACBWG support is valuable, but the following suggestions may encourage it to be more action- and results-oriented beyond a useful exchange of research developments. Some stakeholder suggestions: o Establishment of clear, time-bound action items, and insistence on feedback against defined indicators. o Each NRA taking on one or more research issues as “centers of excellence” or at least as those with lead responsibility. Initial suggestions by ACI steering committee members suggested that Ghana might focus on pests/disease, CDI on genetics, and Cameroon on agroforestry, but this should be determined by the ACBWG membership. o Inclusion of private-sector and academic researchers within the region and externally, not as decision makers but as observers and providers of technical support when needed. Farmer Demonstrations There should be a clear division between participatory research meant to assist breeders in selecting new varieties with feedback from farmers and demonstrations meant to expose farmers to new varieties ready for mass distribution. o Demonstrations of varieties ready for mass distribution should be managed by the national extension service and implemented with lead farmers. This should be in collaboration with the private sector, with community ownership of the results (and production!). o Participatory variety research (PVR) capacity building can be considered, which would be managed jointly by breeders and participating farmers. o A PVR training needs assessment for the regional NRAs o A PVR training program for the regional NRAs based on need o Pilot PVR roll-out, with private-sector participation as appropriate Component III: Enhance Public- and Private-Sector Extension and Farmer Training Services The evaluation report will address the two main activities under Component III in separate sections; (i) Enhanced Extension and Farmer Training Services and (ii) the Ghana Quality Initiative (GQI). This is because of the very different structure and intended results of these two activities. The first section also includes a brief description of two smaller activities, the Cameroon “Next Generation of Farmers” sub￾award and the CDI. (i) Component III Extension Activities Component III—Extension Activities Findings Because the trainings and extension departments were and are different in Côte d’Ivoire and Ghana, this section will discuss findings separately for each country. In both countries, extension administrators, agents, and farmers themselves gave very positive feedback about ACI’s extension strengthening work. However, this evaluation was not designed to be able to attribute improved practices to extension agent training, let alone assess the impact of these improvements in terms of increases in cocoa farmer 49 productivity. This would have required a more rigorous and time-consuming sampling methodology than was possible. Nevertheless, this section aims to connect training content with the impressions of extension staff on training content effectiveness and farmers’ impressions of the extension services they received in the past year. Côte d’Ivoire In Côte d’Ivoire, farmers said they learned about the appropriate timing for input application, resulting in less waste and spoilage. Others said they learned about planting and how to take care of the land. Farmers said they were usually able to implement most recommendations except advice and recommendations on inputs because they could not afford to purchase the inputs. Extension agents estimated that farmers applied about 70% of what they were taught. This was corroborated by CHED extension management in Ghana, who said their own research indicates that farmers adopt between 70 and 80 percent of what they are taught. Extension agents and their managers in Côte d’Ivoire were also positive in their assessment of the training methodology and content. One of the primary ACI training areas that extension agents found most useful was in helping them to better include and provide extension services to women. This is detailed more below. Another topic that extension agents felt was useful was how to plant, keep, and manage the right trees as shade trees, which enrich the soil and increase biodiversity. Previously farmers did not know which trees to keep and which to cut, or which to plant to complement their cocoa. However, according to some extension agents, this component of their extension is constrained by the lack of shade tree planting material. It is not clear what constrains establishment of these shade tree nurseries; perhaps the practice is not yet well enough established for there to be a significant market. On the negative side, extension agents and administrators in Côte d’Ivoire wished the two-week training had been longer, especially the practical aspects. Agents complained that the training featured more information than was possible to learn during the two-week time period. This is supported by project documentation showing that nearly 30 sub-topics were included in the training. In addition, agents and administrators identified a need for a follow-up refresher training, but that was not part of ACI and ANADER does not intend to support such a training by itself. Despite originally intending to train private-sector extension agents with ANADER agents, this training included exclusively ANADER agents. However, it was reported that these extension agents are sometimes hired by private companies, although this practice was denied by ANADER management. A weakness in ACI’s Component III is that none of the trainees in Côte d’Ivoire were trainers themselves; all 244 were extension agents. Therefore, ANADER is dependent on BUREX-CI or others if it wants to modify the curriculum, train more agents, or carry out refresher trainings. This makes it more difficult for ANADER to monitor the effectiveness of the curriculum and update it based on feedback from its agents. It also makes training on the new curriculum more expensive for ANADER. Both of these factors are likely to negatively affect the sustainability of Component III after ACI is completed. Ghana In Ghana, farmers reported learning a variety of useful practices from extension agents, including why good planting material is important, land clearing, proper cocoa tree spacing, the use of shade trees, pest and weed control, pruning, application of fertilizer, post-harvest practices, and record keeping. Farmers reported that the instruction on proper fermentation was especially useful. They were advised to increase the length of fermentation from 3 to 4 days to 6 to 7 days and, as a result, reported reducing the percentage of rejected beans from about 10% to 2%. Some farmers found the pruning instruction to 50 be very helpful, but others noted that pruners and ladders were too costly and the task of pruning too risky. Record-keeping training associated with the certification component was found to be useful even though not all farmers were part of certification schemes. Farmer respondents were positive about the communication and teaching of extension agents, which is important given that this was a priority of the training for CHED agents. They also found that the level of the material and extension agents’ communication was good. Some also noted that extension agents had become more attentive in listening and responding to their input. However, many still said they needed refresher training as they were not able to remember all they had learned. In terms of new content, several farmers said they would like to learn more about diversified livelihood options such as raising grass-cutters.26 Extension agents themselves nearly unanimously reported that the training helped them communicate and teach better. In particular, they praised the SDF approach to delivering more experiential learning to farmers, which also helped them explain better to farmers the rationale for what they were recommending. Extension agents also learned how to assess how well farmers understand what they are teaching and adjust their training based on that. As a result, some said that adoption rates increased as farmers better understood and remembered more of what they were taught. Like their Côte d’Ivoire counterparts and Ghana farmers themselves, CHED extension agents wish they had refresher trainings. And while extension agents viewed the certification training component as useful, some were not able to link farmers with certification schemes, limiting the effectiveness of that module. Gender In CDI, extension agents were very positive about the effectiveness of the gender component of the ACI training. They learned this through training modules that covered gender inequalities in the cocoa value chain; how to include women in extension outreach; strategies for extension delivery, taking gender into account; women’s leadership development; household economics; and more. In particular, extension agents found the training on how to include women in trainings to be useful and reported including as a result more women than they had previously, even married women. One way they learned to do this was to request that male farmers bring their wives to trainings. This is complemented by ANADER and NGOs’ efforts to empower women to make more agricultural decisions. Gender was not a significant component of the Ghana extension agent training. In both countries women farmers did not report feeling excluded by extension agents but, given that focus groups were primarily organized by extension services, this finding is far from conclusive. Component III—Extension Activities Conclusions Overall, the extension activities undertaken by ACI are likely to have a positive impact on cocoa farming productivity. However, there are a number of concerns. A weakness in ACI’s Component III is that none of the trainees in Côte d’Ivoire were trainers themselves; all 244 were extension agents. Therefore, ANADER is dependent on BUREX-CI or others if it wants to modify the curriculum, train more agents, or carry out refresher trainings. This makes it more difficult for ANADER to monitor the effectiveness of the curriculum and update it based on feedback from its agents. It also makes training on the new curriculum more expensive for ANADER. Both of these factors are likely to negatively affect the sustainability of Component III after ACI is completed. 26 These are raised and sold as bush meat. 51 The uncertainty surrounding the timing and assignment of free inputs undermines the effectiveness of extension services in Ghana. Many farmers who had received extension complained about not receiving inputs or receiving them too late to be effective. Not only does this crowd out the private sector, but it also means that farmers refrain from purchasing inputs because of the chance of receiving them free. This likely includes some who would otherwise purchase them if there was not a chance to receive them free. Those farmers then do not receive the inputs or receive them too late to be useful, which undermines their ability to effectively implement extension recommendations. It is not clear to what extent farmers are motivated to attend extension training by the hope of receiving free inputs. The major constraint that CHED extension agents face in delivering the training to farmers is that they are under-resourced. In addition to the typical understaffing of extension services—in Ghana one agent is responsible for 600 farmers, while in Côte d’Ivoire an agent is responsible for 500—this manifests itself in a number of ways. First, agents did not have a lot of the basic materials needed to implement trainings such as demonstration images, white boards, or paper or markers. They also do not have motorcycles to reach farmers and thus are reliant on slow and unreliable public transportation. Cell phones and computers are scarce as well, which limits communication and coordination. Likewise, there are only limited GPS units available with which extension officers can demarcate farmers’ land. With the exception of inadequate materials to deliver training, these resource constraints do not likely impact the effectiveness of ACI’s Component III extension training. Extension agents are able to deliver improved training to the same number of farmers they were serving prior to the ACI training. However, it does raise the possibility that extension officer training may not have been the most cost-effective way for the project to leverage improved extension services. Interventions to expand extension outreach, allowing extension agents to reach otherwise unreached farmers, could be more impactful than improving the quality of extension for farmers already reached with extension services. This is especially true considering the food security objectives of ACI; farmers who are harder to reach for extension agents are likely to be more food-insecure. The Cocoa Link platform, which disseminates cocoa information via SMS, is an attempt at cost-effective information dissemination in Ghana, if not a full substitute for extension. This program was previously funded by another WCF program but has been taken over by COCOBOD. However, lack of cell phone coverage, low literacy rates among farmers, and language barriers have limited its effectiveness. In Ghana, too, CHED has a program where extension agents are supposed to mentor a leader farmer for three years. After that, the agent moves on to new farmers and the leader farmer maintains responsibility for providing services to the farmers in his area. However, this system does not work well, as farmers are not willing to remunerate leader farmers for their time. In both countries, training of extension staff began after the project was halfway complete. Had training begun earlier and had internal trainers for both extension departments been trained, the component would likely have been more successful. After initially training extension agents and trainers, ACI extension training sub-contractors in each country could then mentor ANADER and CHED trainers in subsequent years as they conducted refresher trainings and trainings for new staff. This would, with minimum additional cost, help institutionalize training capacity within each agency as well as contribute to better training outcomes. In addition, such ongoing mentorship over the life of the project could have been used to build capacity to incorporate feedback into training materials and make changes to methodology and content as appropriate. Component III—Extension Activities Recommendations 52 Refresher trainings and training capacity building. The effectiveness and sustainability of the extension strengthening part of Component III can still be enhanced through refresher trainings carried out for extension agents in both countries. These should be used to strengthen the capacity of both extension agencies to carry out their own extension trainings. In the case of Côte d’Ivoire, the refresher training for ANADER should, if it has them, include some of its own trainers so that they could subsequently train other agents. If it does not have them, provision of the refresher training should be contingent on hiring the trainers. In Ghana, CHED trainers should take the lead in delivering the refresher with support from Solidaridad trainers. In addition, the capacity of both agencies should be built to incorporate revised training material, post training test results and feedback, as well as on going feedback from extension agents in the field. This strengthening of the capacity of both national extension agencies to modify and implement their own extension agent trainings should be continued in a potential follow-up project. Business training. Farmer business training material should be developed so that farmers are able to capitalize on the GAP learned through improved extension. While there have been other cocoa farmer business training programs, including GIZ’s Cocoa Farming Business School program, farmers found this to be useful and wished they had more. This point also arose in an earlier ACI review of extension services, reported in March 2014. This should focus on integrating GAP with business training so that farmers can learn the business case for why they should follow GAP practices. Another area farmers could benefit from is learning how to make better farm investment decisions. Cost recovery. A successor project should also work with both agencies to help them identify, develop, and test innovative models for expanding extension service coverage given budget constraints. These pilot initiatives would only be funded if ANADER and/or CHED were able to present realistic strategies for sustaining them and scaling them if the pilots were successful. One potentially promising area is cost-recovery models. One idea is to use local extension agents who are affiliated and working exclusively with cocoa cooperatives and societies but are trained by the national extension agency. These full-time local extension agents could have their salary jointly paid for by ANADER/CHED and the co-op or society. Since extension agencies are servicing co-ops and societies anyway, this would defray some cost. More generally, cost-sharing models should look to segment the market to find cost-sharing opportunities with the private sector (e.g., agrochemical suppliers, cocoa buyers), cooperatives/societies, as well as some farmers themselves. A more modest example that could improve information dissemination is a cocoa radio call-in show, sponsored by input supply companies and the government that would feature a host and a series of guest cocoa experts. Farmers could call in to the show and listen to it for advice. Such radio programs are popular in Tanzania and other sub-Saharan countries.27 Better integration/coordination of public and private extension (agronomists, cocoa doctors) and between regional extension systems. An important start on a national basis was made with the harmonization of training manuals for Côte d’Ivoire, funded under Component I. On a regional basis, for example, extension services in countries outside of Ghana could benefit from exposure to the SDF methodology, which CHED agents found very helpful and a high point of their training. Planning and coordinating extension and inputs. Linking extension services and government input provision would help farmers to implement input-related advice. Given that inputs only reach their full effectiveness when applied skillfully, and input advice is only useful when inputs are available, these two services should be coordinated such that all farmers receiving inputs should have already received 27For the example of Tanzania, see http://opinionator.blogs.nytimes.com/2013/11/27/in-tanzania-farmers-reap-the￾benefits-of-radio/ 53 training on how to apply them. This could be facilitated by a future project and part of a larger project initiative to improve planning and coordination in both national cocoa agencies through better farm-level data and mapping. While liberalization of inputs may ultimately be desirable, this recommendation is a short-term solution that recognizes political realities, especially in Ghana, and would improve cocoa extension service delivery. This coordination would be aided by the establishment of a farm-level cocoa database, discussed below. Develop a farm-level cocoa database. Such an initiative should be facilitated through the national￾level platforms, and data and maps should be made public or at least shared with stakeholders. Data to be included would be farm size; agronomic information, including age of trees; demographic information; farmer affiliation (e.g., cooperative, buyer); inputs received; and most recent extension service visit. Cost for a baseline could be shared between national platform members, ANADER/CHED, and a potential successor project. The project would provide a decreasing share of funding for follow-up surveys as the value of the database was demonstrated to stakeholders. This initiative would help improve service and input planning and coordination by both public- and private-sector players in cocoa in both countries. Activity 2: Productivity and Entrepreneurship Training for Women’s Empowerment The PETWE activity was not included in the evaluation fieldwork due to limited fieldwork time and WCF itself not highlighting the activity beforehand. However, one WCF member who has closely followed PETWE was very positive about the results of this training. The program trained women on aspects of cocoa production that they are most frequently involved with, including caring for young trees and post-harvest practices including fermentation, drying, and sorting. (ii) Component III Ghana Quality Initiative Sub-Activity Component III—GQI Findings Improving Cocoa Quality A number of farmer associations were approached and started Q training sessions organized by GQI. Three farmer associations have maintained interest: Organic Cocoa Growers of Suhum, CAA, and Offinso Fine Flavor Association (OFFA).These three associations share one primary economic motivation in producing higher flavor quality or fine flavor beans: to sell more cocoa beans through existing certified channels (Rainforest Alliance, Utz, Fair Trade, and Organic). Currently, there is an oversupply of certified cocoa beans in the world market, and a large share of cocoa beans that have been certified cannot actually be sold as certified. This means that the associations that went to the trouble and expense of obtaining and maintaining certification are not receiving the expected level of benefits through the premiums. These can be substantial, in the case of certified organic, as much as $600/MT. The GQI corporate sponsors believe that those co-ops with higher flavor quality or fine flavor beans will be able to sell more of their cocoa beans through certified channels. This has not yet occurred. Flavor Labs/Sensory Panel CRIG with GQI support has developed a flavor lab and recruited and trained a cocoa sensory panel, with regularly benchmarking using cocoa industry specialists doing simultaneous testing in Ghana and in the US of identical samples. The flavor lab has demonstrated the capability to produce cocoa liquor and chocolate suitable for flavor analysis. Tcho noted that the CRIG flavor lab in many ways is superior to its own facilities. According to Guittard, the CRIG sensory panel has developed a good understanding of the fundamentals of cocoa sensory analysis. They can identify basic cocoa flavor attributes (bitterness, astringency, acidity, 54 cocoa-chocolate flavor, fruit notes, and nutty, earthy, or off flavors). By the end of the activity, Guittard expects that they will be at a higher level in terms of their alignment with international counterparts on attribute identification and intensity levels. There is no guarantee that the CRIG sensory panel’s analysis will be accepted as essentially equivalent to international testing; some samples may always need to be tested internationally. However, it still could provide many useful services to the cocoa industry (advising buyers, breeders, developing post-harvest training protocols to maximize flavor, etc.). Encouragingly, CRIG reports interest in the CRIG flavor lab and sensory panel from the international chocolate industry. In CDI, both CCC and CNRA were very interested in the GQI and indicated that they would request assistance to establish similar facilities should there be a continuation of ACI. Much of the operational costs of the CRIG flavor lab are paid for by ACI and a transition plan for CRIG to assume these costs has not been developed. The satellite flavor laboratory at CAA was still in the process of being set up during the evaluation and was not visited. Development of a cadre of cocoa quality trainers GQI has successfully identified and trained Q trainers that are providing advanced training in GAP, covering harvesting, fermenting, and drying to maximize quality; these have been deployed to the CAA, OFFA, and Suhum organic farmers. The Q trainers also provide basic instruction on the taste evaluation of cocoa. According to the CRIG flavor coordinator, the COCOBOD chief executive was so impressed with the GQI activities when he visited the CRIG stand at the Cocoa Days event in October 2015 that he committed to mainstreaming the Q training into the general CHED extension training. Translating this decision into concrete action will be crucial. CRIG has started to test its current post-harvest training protocols based on flavor lab evaluation of different drying and fermentation methods, one of the planned activities of the GQI. So far, these tests have confirmed that the current protocols are fine. Fine flavor supply chain driven by premium After a slow start that even predated GQI, OFFA’s fine flavor cocoa program is beginning to show tangible results. Especially in the past year, OFFA farmers are increasingly receiving better support by GQI through CRIG, the CRIG Q trainers, the efforts of the newly recruited Tcho project manager, and the newly recruited LBC partner Transroyal). The work of the 2014/5 season was focused on strengthening OFFA as an organization, clarifying what fine flavor varieties have actually been planted, what condition they are in and getting a trial shipment out so that a level of premiums could be established. For the 2014/5 marketing year, OFFA was able to provide a trial shipment of 50 MTs of cocoa, including a small amount of FF cocoa (see Table 12). 55 Table 12. Trial Shipment 2014/5 Season, OFFA The premiums that Guittard is willing to pay for small amounts of fine flavor cocoa are quite high, significantly higher than for any of the other three major established certification systems (Rainforest Association, or RA ; Utz; and Fair Trade). A comparison is shown in Table 13. Table 13. Ghana Premium Distribution for Fine Flavor Cocoa, 2014/5 Season Partner Traceable Fine Flavor Combined Share COCOBOD (CMC) $20 $0 $20 6% Transroyal $20 $180 $200 56% Farmers $20 $120 $120 39% Total $60 $300 $360 The 2014/5 premium structure was provisional, and the share doesn’t reflect the true costs to Transroyal of moving such small quantities of cocoa (only seven bags) traceably through the COCOBOD system. As volumes increase, the share to farmers should increase significantly. Guittard proposes to make the premium payments staggered through the marketing year based on 1) ensured traceability, 2) initial quality tests, and 3) a final bonus. Because of the time between shipment and receipt of bonus, it is not clear if this bonus system will send a clear price signal back to a given farmer, rewarding him or her for ensuring high quality. Currently, the farmers are very unclear on the timing and size of the premium. For the 2015/6 marketing year, up to 10 MTs of fine flavor (FF) cocoa are expected to be delivered as the initial FF cocoa trees planted in 2011/2 enter full maturity and farmers improve their GAP and post￾harvest handling practices. The OFFA farmers have a renewed interest in the project and there is interest from additional farmers to plant FF cocoa. Of the 11 OFFA farmers interviewed in FGD, about one third of their land is planted to FF varieties, with the balance in bulk varieties. One of the participants at the OFFA FGD was not yet an OFFA member but on the basis of new enthusiasm had formed a group that was interested in affiliation and growing FF cocoa. However, there is still some concern expressed by farmers about what they see as increased costs of FF cocoa production that might erode the premium. Transroyal suspects this is probably because farmers are practicing GAP for the first time, not because of FF cocoa. Nevertheless, they are aware that the production costs and benefits of FF vs. conventional cocoa need to be studied and demonstrated more clearly. Guittard has committed to purchasing 200 MTs of FF cocoa, which is far in excess of what the existing acreage can produce if well managed (about 20 MTs). As OFFA membership increases from 60 to 250 Trial Shipment 2014/5 Season, OFFA Quantity Premium in MTs per MT Traceable Cocoa 49.5 $ 60 Fine Flavor 0.5 $ 360 Total 50.0 Source: Guittard chocolate 56 farmers and the grafting of fine flavor varieties continues, in three to five years there may be finer-flavor cocoa than Guittard can purchase. According to Guittard, with increased membership to 250 farmers, and obtaining Utz or Fair Trade certification, cocoa will move OFFA into a sustainable level of production and sales even if demand for cocoa sold under fine flavor does not increase. Transroyal is enthusiastic about acting as a FF buyer, even though actual marketed quantities will likely remain quite low for years to come. LBCs in Ghana do not compete for cocoa beans on price, so establishing a formal premium is the only way to reward quality. Transroyal sees the ability to offer a FF premium stream to cocoa growers as a value-added service to cocoa growers. Evaluation of CRIG cocoa varieties undergoing national variety trials pre-release to ensure that their flavor profiles preserve the national Ghana cocoa profile. The CRIG flavor manager Opuku indicated that cocoa samples from 10 new cocoa varieties, which have been selected for productivity, resiliency, resistance, etc. and are now undergoing national pre-release trials, are being evaluated by the sensory panel. Initial indications are that up to three of these seem to have particularly favorable flavor profiles. Testing of Regional Differences within Ghana The GQI is undertaking tests of cocoa from the various cocoa-growing districts in Ghana in an effort to determine if there are any regional differences. The first and second round of regional samples have been collected and sent for processing; 24 of those samples are now ready for sensory analysis, and the balance will be completed and distributed in mid-November. A third round of samples will be collected during the peak of the current main crop in late November or early December, and a fourth and final round of sampling will be completed in February. Sensory results will only be made available at the end of the GQI project. Component III—GQI Conclusions GQI has introduced interesting innovations in improving the flavor of bulk cocoa and fine flavor cocoa varieties that, if successful, could significantly increase returns to cocoa farmers. Improving Cocoa Quality The GQI activity assumes that if the flavor quality28 of the cocoa beans can be improved tangibly, buyers will recognize this and choose to source their certified beans from those associations producing the more flavorful cocoa. As demand for certified beans grows (e.g., Hershey is committed to 100% certified sourcing by 2020), the surplus of certified beans may disappear. The GQI activity assumes that co-ops that are producing better flavor cocoa will still have an edge and possibly enjoy a price premium. This is a fundamental assumption that has not been validated. The GQI activity also makes the assumption that if farmers can be shown the taste difference between poorly and well processed beans, they will be motivated to make changes in their post-harvest practices to improve quality. However, unless farmers are also rewarded by receiving tangible financial returns for producing better flavor cocoa, it seems doubtful that tasting alone will have much of a desired 28 To some extent, Ghana farmers are already rewarded for producing better quality cocoa by the Quality Control Corporation (QCC) of COCOBOD. More accurately, they are penalized for producing poor quality cocoa. QCC has a three-tier system of controls based on physical and visual testing. If samples don’t meet quality standards, the lots are rejected and must be reconditioned or discarded. Very importantly, however, the QCC physical and visual testing does not capture all the elements of flavor quality. Better flavor, at present, can only be rewarded by the market. 57 motivational impact. Indeed, if cocoa farmers are not rewarded for higher quality, it would seem to make more sense to focus project resources on productivity (the primary objective of ACI). Flavor Labs/Sensory Panel The costs of the CRIG flavor lab are relatively low and there is considerable interest generated in its operation. Therefore, it shouldn’t be difficult for CRIG to source funding from COCOBOD for its continued operation. A flavor lab/sensory panel associated with CHED training facilities would seem to be a low-cost investment with a potential high impact. The added value to CAA of incurring the capital and ongoing operational expenses of a satellite flavor lab at CAA may not be clear given that flavor samples could easily be sent to CRIG. However, CAA may enjoy a marketing advantage of being the only cocoa association in West Africa with its own flavor lab and sensory panel. Development of a cadre of cocoa quality trainers While CRIG may have been the appropriate body to initiate the Q training, as it was home of the flavor lab/sensory panel, it makes economic and organizational sense to mainstream Q training into CHED’s extension team rather than having separate Q trainers. Fine flavor supply chain driven by premium The GQI makes a key assumption: OFFA will have an incentive to grow new fine flavor varieties because it will receive a new fine flavor premium. There is no indication that this could happen at a meaningful scale. Fine flavor cocoa buyers currently source outside West Africa, and there is no real evidence that they are interested in looking at West Africa. Only one company (Guittard) has made commitments to source fine flavor varieties, and its projected volumes alone are not likely to be high enough to make a separate fine flavor premium economically viable. Tcho, one of the other GQI sponsors, has expressed interest but has made no firm commitments. Guittard and Tcho are relatively small chocolate companies. While their innovation and enthusiasm is the driving force behind GQI, it is probably not wise to base the success of GQI solely on the market access and connections of these two companies. There are a number of challenges that must be overcome before the GQI fine flavor initiative can be considered a success:  Weak business skills within OFFA management and membership, e.g., the ability to develop or even understand a simple crop budget comparing FF and conventional cocoa.  Premium payments are not determined at the start of the marketing season and paid only at the end of the season (except for a small upfront payment for certified beans). This sends a weaker signal to farmers than a clear pre-marketing price.  Premium structure is not clear to the farmers. Some of the farmers reported that they were told they would get an (unrealistically high) premium of 50% above market price. Apparently this was based on earlier meetings with CRIG before GQI began to be involved, but this expectation still remains. These farmers were not clear about how much of this would 1) go to support the costs of establishing and maintaining a traceable FF supply chain, 2) come in-kind in the form of inputs, equipment, tools, etc., and 3) be in the form of cash payments to the farmer.  Little knowledge about the differences between fine flavor and bulk cocoa in terms of GAP requirements, post-harvest handling, productivity, and disease/pest susceptibility. What is known is that the grafted fine flavor varieties are likely to be more productive than the typical farmer’s cocoa tree, because they are based on known parentage 58 rather than the mix that is on farmers’ fields (see Component II discussion). Grafted cocoa (not just fine flavor cocoa) can require more care in terms of pruning for several years, but after that production costs (assuming the same level of GAP) are thought to be the same. Farmers also lack knowledge about the cost differences. Evaluation of CRIG cocoa varieties undergoing national variety trials pre-release to ensure that their flavor profiles preserve the national Ghana cocoa profile. Whether this pre-release flavor evaluation will affect what cocoa varieties are actually released and promoted within Ghana is not clear. The head CRIG breeder does not seem to accept that the flavor screening is important, as he believes that the post-harvest processing and handling is a far more important determinant of flavor. The pre-release flavor screening done by CRIG could easily have been done by flavor labs in Europe or the US. However, as noted earlier, it does seem beneficial to establish this capability in Ghana, especially when the costs are low and this is only one of the many uses of the flavor labs. Testing of Regional Differences within Ghana The practical implications of regional testing have not been made clear. It is possible that the testing may suggest needs to improve GAP or post-harvest processing in certain regions. Or the data may be useful for companies that wish to identify a region for geographic designation as a marketing strategy (e.g., Champagne sparkling wine, Roquefort cheese), but under present conditions this is a long-term possibility at best. Component III—GQI Recommendations Based on conclusions, it is recommended that the new cocoa initiative include a GQI-like component, with the following suggestions to be considered:  Market analysis for fine flavor and high-quality flavor cocoa. Key assumptions underlying the potential of GQI to provide higher incomes for cocoa farmers are based on a potential demand by buyers 1) to pay higher prices for higher-quality bulk beans (or at least prefer higher quality to lesser quality for certified beans) and 2) pay a significant premium for West African fine flavor cocoa. A market analysis that would 1) segment the global cocoa market supply and demand into finer gradations of flavor qualities of bulk cocoa and fine flavor varieties and 2) identify potential buyers/supply chain partners would provide valuable insight to West African cocoa stakeholders and help guide the design and scope of future GQI-like activities. The outcome of this study should drive the decision to modify or upscale this initiative (but in the meantime the ongoing Tcho/Guittard-led activities should still be supported).  Organizational and business training for most participating co-ops is essential for proper understanding of premium structures, planting/GAP/post-harvest decisions, and premium distribution strategies, which in turn are essential for co-ops to be reliable partners for fine flavor or quality initiatives. OFFA is clearly in need of this capacity building; it is likely that the Suhum society also requires this training.  Expand the quality initiative to CDI. Both CCC and CNRA have expressed great interest in the flavor initiative.  A cost-benefit analysis of FF vs. conventional cocoa in collaboration with CHED. Farmers express concern about the added costs of production of FF cocoa, while others say that there is only a minor difference in pruning costs because of grafting and that what the farmers are experiencing is just that they are practicing GAP and good post-harvest handling for 59 the first time. This should be resolved, especially given issues with the premium price signal reaching farmers.  Plans to build out satellite cocoa flavor labs and sensory panels within co-ops should be viewed as a phased pilot initiative, which is critically appraised for cost￾effectiveness.  Build awareness and capacity for fine flavor varieties and/or better flavor bulk cocoa throughout the COCOBOD supply and marketing chain. CRIG was the right place to start, but trainings and technical assistance can build capacity and enthusiasm in the other COCOBOD components and increase supply of and demand for better flavor and/or fine flavor cocoa. Support should be structured so that COCOBOD is providing a high level of matching funding: o Quality Control Corporation of Ghana can explore the possibility of including some form of organoleptic standards or optional guidelines for Ghana cocoa, which would require training of QCC inspectors and installation of the necessary testing apparatus. o Cocoa Health and Extension Department could mainstream key aspects of Q trainers training, e.g., by setting up a flavor laboratory and sensory panel capability at the Bunson extension training college. o The Cocoa Marketing Company to incorporate Ghana’s emerging capabilities in achieving better flavor, flavor testing, regional flavor database, etc. into the marketing proposition for Ghanaian cocoa and generate demand/interest. Component IV: Market-Driven Input Services (i) Spray Service Providers Sub-Activity Component IV—SSPs Findings The SSP model demonstrated strong promise in cocoa. The purported advantages of the SSP model— safer and more effective application—were valued by farmers and appeared to be put in practice by SSPs. Farmers in particular valued not having to expose themselves to the health risks of spraying; some reported having skin and eye problems in the past when they sprayed themselves and others knew people who had died from spraying. Additionally, farmers by and large trust SSP’s recommendations on the right chemicals and dosages to use to control insects and fungus. Farmers also viewed SSPs as their link to quality pesticides. SSP use also increased profitability for farmers: ACI’s own research found that SSP users’ gross margins were increased from $509 at baseline to $932 last year based on a sample of 400 SSP users. An unexpected benefit of the SSP program that farmers appreciate is that many SSPs will scout their land in advance and advise them on proper land clearing. However, some nuance is needed in interpreting the impact of the program. CropLife’s baseline study found that in Côte d’Ivoire 62% of cocoa farmers were already using spray applicator services, and in Ghana that number was even higher (82%). It is not known how many SSP clients were already using spray applicators or how many SSPs were already spray applicators. For farmers already using the service, the impact is small; it is the difference in skill and professionalism of the SSP compared to the previous sprayer. A similar argument applies to the safety impacts on SSPs themselves. Despite the general positive impressions of the SSP programs that most stakeholders had, several issues cast doubt on the scalability and sustainability of the model. Currently SSPs are not operating at capacity. 60 Whereas rough estimates indicate that SSPs should be able to spray about 260 hectares per year,29 ten SSPs we spoke with in both countries reported spraying 112 hectares on average in the year. This affects the profitability and therefore attractiveness of being an SSP. Gender Women are prohibited from being SSPs because of the risks associated with handling chemicals during pregnancy. SSPs play a positive role in women’s health by giving women an option to apply chemicals at lower risk to their own health, especially during pregnancy. Component IV—SSPs Conclusions The SSPs introduced significant improvements in spraying services through the development of a professional cadre of spraying service providers. However, there are a number of impediments to this being a sustainable activity with scale-up potential. A primary impediment to expansion is the lack of availability of pesticide mist blowers. These are expensive capital expenditures costing upwards of 500 USD; very few SSPs purchase the equipment. Instead, they rent or borrow them from entrepreneurs, cooperatives, or cocoa communities. In Ghana, most SSPs were in Kokoo Pa (a major cocoa farmer association) communities. Each Kokoo Pa community had four SSPs trained and received one free mister from the government. However, one mister was insufficient; cocoa communities in Ghana reported that the misters they had were often broken. In addition, cocoa communities and co-ops were reluctant to purchase their own mist blowers, apparently because they don’t see it as a high enough priority and in any event hope for additional free mist blowers from the government. As a result, SSPs report not being able to meet demand for their services. In Côte d’Ivoire, CCC distributed 1,000 mist blowers to SSPs but there were still shortages reported. SSPs are interested in owning their own mister and are aware that within two years it would likely pay for itself in rental costs averted. However, they are unable to meet the requirements to access credit for purchase. Others are not interested in purchasing because they do not understand the potential business return or because they are waiting to receive one for free. Another issue is that many SSPs lack the business sense and entrepreneurial initiative needed to scale up or even maintain their business. Part of this is because the SSP cohort received training and personal protective equipment (PPE) for free from CropLife. This implies that many SSPs were trained who were not serious enough about providing the service commercially to invest their own money. Furthermore, for budget reasons business training was provided to only 11 percent of SSPs. Taken together this manifests itself in, for example, some SSPs being unclear on how they price their service or put together a business plan to get a loan from a bank. Many also lack the orientation to market their service to new customers as well as the skill to plan and coordinate their service delivery. In Ghana, SSPs face competition from free, government-sponsored spray gangs. Because coordination is poor, farmers do not know if they will receive the free spraying in a season. Some farmers would prefer to wait on the chance to receive free spraying rather than pay for on-time spraying from SSPs. This is even though farmers complained about the quality of the spray gangs in addition to the poor timing. The result, of course, is that SSPs lose business and crops are sprayed late or not at all. While SSPs undoubtedly improve the safety of pesticide handling by taking it out of farmers’ hands, there’s some question about how well the SSPs take care of their own safety. The project’s initial idea was to subsidize the purchase of PPE but found that SSPs were not willing to pay anything. Instead, in 29 Spray season is four months per year. Using a five-day workweek, that is approximately 87 days. A conservative estimate for hectares per day possible for one SSP is 3. 3 x 87 =261. 61 Côte d’Ivoire, 1,000 PPE that are distributed annually by CCC were diverted to SSPs. In Ghana, SSPs were said to already have their own PPEs. Some SSPs, however, complained about not having adequate PPEs and it is not clear to what extent SSPs own and use full PPE. Another issue is with the badge that certifies that an SSP has been trained within the past two years. In Côte d’Ivoire, there were no plans for retraining and updating of badges so they will expire shortly. In Ghana, badges were never issued. In Côte d’Ivoire, some farmers asked to see badges. It may be that badges are not needed in Ghana because SSPs were already working in communities as SSPs before training and had already established a professional reputation. On a program level it is not clear how the SSP program will be sustained or scaled after ACI. Currently, SSPs service less than 5 percent of cocoa farmers in Côte d’Ivoire and Ghana, which could be close to 10 percent if existing SSPs were to reach capacity. However, the SSP program is dependent on ACI project funding despite being implemented by an organization of pesticide companies (CropLife) that would seemingly have public relations and commercial interests in expanding the program. While ACI seems to have viewed the SSP project as a pilot that could be taken over and scaled up by the industry with success, the pesticide industry does not appear to share this view. There are several reasons for this. One is that, in general, international input companies do not view the smallholder farmer as an attractive market worth investing in to expand. They would rather deal with larger buyers such as larger farmers in Ghana but also government buyers in Côte d’Ivoire. In Ghana, one large input company estimated that the government is responsible for 80 to 85% of cocoa pesticide purchases. Those companies interested and active in selling to commercial cocoa farmers and providing valued-added services such as technical advice on spray application would rather do it themselves so they can capture all the return on their investment. Also, because of the inability to control counterfeit pesticides and limited ability to track sales to local level, companies are not confident that training SSPs will increase their sales. Component IV—SSP Recommendations A successor project should continue and possibly expand the highly promising SSP programs, with some modifications suggested: SSP market demand study. A market demand study should be carried out on cocoa farmers in both project countries covering the demand for SSP services, but this should be extended to cover other specialized functions such as pruning and soil testing/fertilizing. It should also cover other crops commonly grown by farmers. This can help to identify the size of the market for cocoa spray services as well as where the market is strongest. It should also investigate the likely effect of SSPs on pesticides sales. The study can also identify the market for spray services for other crops grown in cocoa-growing areas as well as for other services. See more on this below. Cost-sharing for SSPs. New SSP trainees should be required to bear part of the cost of the training and equipment. This would, at least in theory, mean that trainees are serious about becoming SSPs and have a better chance of being successful. It would also reduce the cost of trainer per trainee, allowing the project to train more SSPs. While some subsidy may always be necessary to train SSPs, a goal of the next project should be to shift as much of this cost on to pesticide companies themselves by convincing them of the value of the SSP program. Business training. All trainees should receive business training. This should include writing a business plan and other activities needed to access finance. 62 Market facilitation approach. CropLife or any other potential implementer of the SSP program should be required to present a market facilitation approach to bring the SSP model to scale commercially after the project. The prime contractor (e.g., WCF) should manage this activity closely and proactively and ensure that the implementer is following this approach. This should involve some initial investment by companies with a commitment to continue should the project reach certain agreed-upon targets that reflect a successful pilot. Some possibilities are presented below. It should also involve clear plans to build and maintain the quality reputation of the SSP brand, including possible spot quality control checks and badge renewals contingent on refresher trainings.  To keep SSPs engaged in the business and to encourage new SSPs, becoming an SSP needs to be more profitable. One way to do this is to expand the crop purview of SSPs beyond just cocoa so that they are expert in spraying other crops. This could provide more income throughout the year, as cocoa spraying is only over a four-month period.  Increase the revenue opportunities for SSPs by equipping them to provide additional cocoa and other crop services; details of this will be informed by the market study. The most obvious choice is fungicide, which is usually applied six times a year. Another strong candidate is pruning. Like pesticide application, this is a risk-laden activity that requires both expertise and capital. Farmers reported not pruning because of how expensive pruners and ladders were and because of the riskiness of being up high on a ladder in the tree. Soil testing/fertilizer application is another possibility, and fertilizer companies could support the SSPs with training.  Explore working through input retailers to train and manage SSPs as employees or through a franchise-like model. Pesticide companies could then provide a discount to the retailers in exchange for training and managing SSPs. This would ensure that a company’s contribution to training SSPs was proportionate to their sales through the retailer. These sales on whole would presumably be increased by having SSPs. The retailer then could either provide capital for or purchase the mist blowers themselves, perhaps with financing from their suppliers. This model could be integrated with an improved inventory tracking system to help companies track revenue sold through SSPs. Mist blower and other equipment financing for SSPs. For cooperatives and cocoa societies using SSPs, the project could explore supply chain financing involving risk sharing, such as that piloted by TechnoServe for fertilizer and other inputs as part of ACI’s Component IV. Depending on the findings of the market demand study and the ability to find suitable partners for a market facilitation pilot extending SSP services, this equipment finance pilot could be extend to pruners, soil testing kits, etc. (ii) Input Finance Sub-Activity Component IV—Input Credit Findings Design TechnoServe’s original concept for the pilot ICS was innovative but completely impractical. The original concept of SSP as micro-banker was quickly proven to be unworkable for a number of reasons, most of which should have been anticipated by TechnoServe given its long involvement in the cocoa sector regionally and with input credit schemes internationally:  The SSP business model was yet to be tested, and thus suppliers/banks were reluctant to provide funds to SSPs even with a guarantee. 63  Cocoa farmers’ demand for credit for inputs and services from the SSPs was not assessed. In fact, experience has shown that in CDI, farmers can generally pay cash for crop-protection chemicals and spraying services, which are spread out over four months of the year.  The SSPs were clearly not up to the tasks that TNS had envisioned. They had: o Little or no credit experience that enables them to be bankable (in fact, many of them lack sufficient credit to equip themselves with a mist blower and PPE). o Little or no desire to take on debt to on-lend to farmers, especially when they are assuming all risks of non-payment. o Little capacity to price the risk of non-payment by farmers into their effective interest rates. o No capacity or desire to provide credit management training to farmers. The final design was essentially an extension and improvement on the original CLP ICS model. TNS worked with exporters that identified cooperatives with whom they had an ongoing relationship and that they thought would be creditworthy. TNS then identified and provided training and support to FIs that would provide input financing to the co-ops. In the case of CDI, four new co-ops were added by TNS with ACI funding. Results The ACI ICS program ended 31 December 2014, and an independent evaluation report (Coffey report)30 was submitted in April 2015. Key positive findings from the Coffey evaluation include: o TNS achieved all targets at the output and outcome levels in both CDI and Nigeria. o Farmers report satisfaction with the quality and timeliness of inputs delivered. o Repayment rates of 100% and 98% for CDI and Nigeria, respectively. o Input suppliers extended technical assistance (TA) to participating co-ops. The Coffey team calculated that in Nigeria, $210,000 in incremental revenue was generated by the 300 participating farmers in the ACI pilot, based on extrapolations of TNS provided data. The Coffey report cited a TNS calculation that, due to the combined impact of the ACI and CLP programs, an additional $7 million in profits was generated by farmers due to the increased input use leveraged by both ICS schemes. These profit estimates, however, involve a number of assumptions and extrapolations that were not validated by the Coffey team or by this team. In their final report for the ACI grant,31 TNS farmer satisfaction surveys show that overall, a majority of farmers in Ghana, Nigeria, and CDI who borrowed under both CLP and ACI programs “perceived financial benefits derived from the inputs received on credit as sufficient to cover the cost of those inputs.” However, a closer examination of the survey findings indicates that the farmers may not have fully understood the questions. In Nigeria, for example, 100% of farmers interviewed gave a positive response to the financial benefits question. However, only about 80% responded positively to the question “quantity of inputs received on credit sufficient to have an impact.” 30 Umaru Ibrahim et al., “An Evaluation of the Input Credit System As Implemented by TechnoServe in Côte d’Ivoire and Nigeria For the African Cocoa Initiative” 31 “TechnoServe African Cocoa Initiative Final Narrative Report,” April 2015 64 Recognizing the intrinsic problem of farmer-reported data,32 TNS attempted to directly calculate a measurement of farmer yield impact by monthly pod collection for all sampled farmers in Nigeria and CDI. Unfortunately, the pod-collecting exercise did not cover the entire cropping season, and attempts by a statistician to draw conclusions from an extrapolation of the data were not conclusive. Advans initial results in CDI are compelling, as shown in Table 14. In the past three years, the Advans cocoa ICS has extended a total of about $3.7/€3.4 million to over 80 co-ops, from a base of zero in cocoa lending in the 2012/3 cocoa season, before TNS involvement. Table 14. Advans Cocoa Input Credit Scheme 2013/4 to 2015/6 Cocoa Season Number of Co-ops Participating Year-on-Year Increase in Co-ops Participating Credit extended Year-on-Year Increase in Credit Extended 2013/4 14 n/a $328,000/ €300,000 n/a 2014/5 40 167% $655,000/ €600,000 100% 2015/6 81 103% $2.5 million/ €2.7 million 317% Source: Interview with Grégoire Danel-Fedou, DG Advans CDI. Note: Credit extended to farmers was in BCEAO Francs, which have been converted to US$ using average historical exchange rates. Advans indicated that the TNS role was critical to its expansion in the cocoa sector for the following activities it provided:  Presented the risks, benefits, and key issues providing production credit to farmers  Provided an overview of the cocoa value chain and the flow/timing of funds to farmers  Lined up input suppliers willing to share risk, provide training, and market to co-ops  Marketed and sold the concept to exporters and the co-ops with whom they are associated  Providing training on input use to the co-ops  Had test data on repayment from a trial run with co-ops, with Callivoire providing input finance  The TNS guarantee fund, covering risk  Provision of TNS staff to help with credit collection As can be seen in Table 14, after TNS ended its technical and financial support in the 2014/5 cocoa season, Advans actually accelerated its level of lending to co-ops for input financing, both through increasing the number of farmers financed per co-op and the number of farmers covered, as well as through increasing the acreage covered with well-performing farmers. Advans is financing this aggressive expansion through its own balance sheet, there are no concessional funding or refinancing agreements, and there is no more technical support or risk-sharing by TNS. Farmers are using the funds mostly to finance fertilizer, which is applied once annually and thus is a large lump-sum cost. 32 Farmer-reported yield increases could be inaccurate for a number of reasons and should be considered less than reliable. Under the CLP, reported yield increases by farmers who had borrowed for crop-protection chemicals and fertilizers were an eye-popping 176 to 225%! 65 Advans has modified the model slightly to fit its own assessment of risk and its expansion goals. New co￾ops are no longer approached through the exporters; Advans and input suppliers together market the ICS to co-ops, and the input supplier still provides training to the co-ops, although on a much more basic level than the full SSP training. While exporters can still play an important role in providing a stop￾order service for Advans, the very high repayment rates experienced are giving sufficient comfort to Advans to finance this rapid expansion. In fact, this reduced reliance on exporters reflects a maturation of the business model and gives cooperatives, at least in principle, a greater opportunity to work with competitors. (Previously, credit was only available through chosen exporters.) Based on its positive experience to date, Advans is also exploring other credit products for the cocoa co-ops it finances, both for individual household requirements and for co-op working capital requirements. Advans has not done an evaluation of the impact of the credit on farmers’ productivity, and TNS is no longer involved to carry out this work. The team visited one of the co-ops33 that received financing for fertilizer through ACI for two seasons, and continued on with Advans for a third season. Table 15 shows rapid growth from essentially no fertilizer use in 2012/3. Table 15. SCOOPBAD Fertilizer Financing Program Season Fertilizer Used or Financed, MTs Number of Farmers Year-on-Year Growth in Fertilizer Usage 2012/3 0 n/a n/a 2013/4 12 34 n/a 2014/5 32 96 167% 2015/6 50 140 56% While the borrowers did not keep accurate records of the impact of the fertilizer on yields relative to their lands without fertilizer, they reported more pods per tree, larger seeds in the pods (an indicator of quality), and greater resistance to disease. Perhaps more importantly, the co-op has voted with its pocketbook—very rapid growth in farmers’ adoption of fertilizer financed by Advans, while members used their own funds for crop-protection chemicals. Key success factors noted during the co-op management FGD included:  Farmer Field School training through a GIZ partnership, with a heavy emphasis on GAP as well as farming as a business (FAAB). This included controlled trials of fertilizer usewalked through a calculation that showed that a high return on investment in fertilizer usage  Input training coordinated by TNS, which emphasized safe and efficient use of inputs34 33 Société Coopérative Simplifiée Binkady d’Ake Douanier (SCOOPSBAD), Agboville. 34 TNS has noted that the CropLife SSP multi-day training on input usage provided, which was provided to the new ACI-financed co-ops, was unnecessarily complex compared to the half-day input training provided by input companies to the CLP cooperatives. In the current Advans model, the half-day training is provided. 66  Credit management training provided by Advans  Strong discipline imposed on co-op members, reinforced by the co-op management, as farmers traditionally “don’t respect banks.” Within WCF and among some of its members, there is a perception that the TNS ACI activity was a relatively expensive exercise with little value added over what was already established by the existing CLP pilot. In their calculation, for a cost of nearly $1 million, less than $180,000 of credit was leveraged, at a cost per farmer of nearly $200.35 The figures reported by Advans show that this calculation of credit leveraged is a substantial underestimate, as over $3.4 million of credit has been leveraged, most of this under the ACI part of the program. WCF has also expressed concern that “from the point of view of farmers, real interest rates (at around 35% effective interest) were exorbitantly high.” 36 Clearly, the Advans farmers must believe that they are obtaining positive rates of returns on the fertilizer purchased with funds borrowed, even if the real interest rates are high, or they wouldn’t be borrowing for more than one season. In fact, Advans claims that with credit, the co-op was able to be linked to the bulk supplier of fertilizer. The cost per MT of the fertilizer, including all interest and transport costs, is the same as if they had purchased using cash from a local agro-dealer on an individual basis. The evaluation team was not able to validate this assertion, but this can easily be confirmed by ACI staff. TNS did attempt to work with other MFIs in CDI, without success. The key lesson learned was that choice of MFI was critical. Only those seriously prepared to make investments in staffing, infrastructure, and funding based on achievement of benchmarks in a pilot should be selected. Prior agricultural experience, surprisingly, was not important; Advans’ only prior experience in CDI was in urban lending. In Nigeria, while TNS achieved its output targets, it does not believe the climate is right yet for sustainable and scalable expansion of the model. In general, farmers are less organized, have lower adoption of GAP, and lower business skills; cooperative structures are nascent; and fertilizer costs are relatively high. Operating costs for TNS were significantly higher than originally estimated. Component IV—Input Credit Conclusions The credit ICS pilot introduced by TNS through CLP and refined during ACI has been successful enough in CDI to warrant continued support for financial services to cocoa farmers. The ACI-funded program introduced important innovations over the existing CLP pilot, and these and TNS continued support seem to have been the catalytic factor behind the adoption and rapid expansion of the model in CDI by Advans, which has occurred without any further TNS support. WCF concerns about the cost of the pilots appear to be unfounded. The TNS efforts under ACI (and CLP) were intended to be pilots to facilitate a market response that would be sustainable and scalable, and they succeeded very admirably in CDI.37 Judging a pilot during the proof-of-concept stage by donor cost per farmer reached, or program costs/credit advanced during the pilot stage, misses this essential point. 35 WCF semi-annual report FY 2015 through 30 April 2015. 36 WCF semi-annual report FY 2015. 37 In Nigeria, operating conditions became too difficult. In Ghana, the COCOBOD program of free fertilizer and input distribution has greatly suppressed development of a credit and a private-sector, commercial input supply chain. 67 There are some constraints on the current model that limit its coverage of all cocoa farmers in CDI:  TNS estimates that, at most, 25% of the registered cocoa co-ops in CDI have the minimum levels of organization and discipline that could be reached by the current ICS model.  Even within better co-ops, many members have not adopted GAP sufficiently for them to be considered productive users of fertilizer (i.e., insufficient clearing, weeding, pruning, removal of diseased/old trees, phyto treatment).  Lack of soil testing at the farm or even co-op level, and thus the inability to select fertilizers formulated specifically for need. This undoubtedly has led to a sub-optimal return on investment for fertilizer.  The pace of Advans’ expansion may be limited by the withdrawal of the partial guarantee provided by TechnoServe.  Some farmers see borrowing as leading to impoverishment rather than as a tool for escaping poverty. Component IV—Credit Activity Recommendations Proactively link experienced SSPs with a documented track record to MFIs. This could provide a motivation for newly recruited and existing SSPs to develop a customer base and keep credible records. TechnoServe and other similar NGO/consulting firms should be encouraged to develop proposals for new credit pilots that broaden the reach of the ICS pilot by extending/adapting it to other MFIs or financial institutions that are serious about agricultural credit. Proposals should also be considered that deepen the reach of credit to cocoa farmers, with a demand-based approach that could include credit for small equipment purchase (cocoa drying, mist blowers, etc.), or for household cash flow or to co-ops for other equipment purchases or leasing. Adopt a market facilitation approach to designing and evaluating credit pilots, where success is measured by the development of a sustainable and scalable credit model, not just by year-by-year metrics such as cost per farmer or net credit leveraged/cost of pilot. The pilots should be afforded the flexibility and time (e.g., two years with an option for a one- or two-year extension) for the facilitator and partners to adapt the model to market realities. Introduce soil testing in the fertilizer ICS through a market facilitation approach. For example, this could be provided as a service or cost shared by fertilizer companies as part of the tender process for selecting a fertilizer supplier. This should incorporate, expand, and leverage the work underway by IDH and companies such as Hershey and Syngenta to promote co-op– and farm-level soil testing. GAP training should incorporate soil sample protocols that will ensure useful results of the soil testing. Eventually, the co-ops/farmers may decide to take on the costs as they realize that the expense of soil testing will be more than repaid by more cost-effective, targeted use of fertilizer. Coop/society training that includes organizational development, financial/credit management, and other business skills with GAP practices would increase the number of cooperatives and farmers that could utilize credit. Include a specialized FAAB module that reinforces the idea that the net cash benefits of fertilizer are only realized or maximized when other aspects of GAP are sufficiently introduced (removal of senescent and diseased trees, land clearing, pruning, and adequate crop protection). 68 A cocoa household financial needs assessment should be undertaken that would identify key financial constraints for the cocoa household through the season and identify possible products/services that could be profitably provided. This would help focus the formulation of requests for application (RFAs) and for the evaluation of proposals. This was also requested by CCC. Improved coordination of co-op–oriented credit and related training programs with CPQP, CLP, and other WCF member programs. IV. Cross-Cutting Issues Findings, Conclusions, and Recommendations Regarding Other Cross-Cutting Issues Land Tenure Land Tenure Findings The insecurities caused by land tenure policy, law, enforcement, and cultural norms have a profound impact on the ability and willingness of cocoa farmers to invest in new measures (new planting materials, GAP, inputs). WCF members and other stakeholders also noted the impact of land tenure on siting demonstration plots, model farms, and even on their CSR activities such as school-building programs. Land tenure in rural areas of West Africa is an extraordinarily complex topic, differing not only between countries but within regions and even communities. The impact of land tenure is also linked to inheritance laws, the binding nature of oral agreements, marriage customs (especially polygamy), etc. The advent of small-scale, informal, and often illegal gold mining in parts of the cocoa region complicate land tenure practices even further, as they provide a temptation to landlords to break land agreements. The evaluation SOW did not directly address land tenure, and the evaluation team did not include land tenure specialists. However, from our fieldwork we were able to shed some light on how some of the land tenure issues affect cocoa farmers’ willingness and ability to invest, which point the way for further research, analysis, and possibly policy recommendations. In Ghana, a key land tenure issue identified was the use of the abunu and abusu systems of sharing land and produce between land owner and tenant farmer (or to use the common Ghanaian term, “caretaker”). In brief, and somewhat simplistically:  Abunu system is generally used to clear land for cocoa. The caretaker has the full responsibility to clear the land, plant the cocoa, and provide all aspects of care. When the trees begin to bear, the crop is split on a 50/50 basis. The caretaker continues to provide all care and inputs. In some regions, it was reported that the land was split on a 50/50 basis after trees begin to bear, with the landowner and caretaker individually responsible for care and inputs.  Abusu system is generally used to manage land already planted to cocoa. The caretaker has full responsibility to manage the cocoa plantation, but the necessary inputs are provided/financed by the owner. A third of the crop is retained by the caretaker, a third of the crop is used to pay for inputs, and a third of the crop is kept by the landowner. Although exact figures are not known, cocoa stakeholders estimate that as much as 60 to 70% of the cocoa land is farmed under these two systems. There are obvious benefits to these traditional land tenure systems, especially under traditional low￾input systems of cocoa production. They provide an efficient means for a landowner to clear land for 69 cocoa and allow enterprising newcomers to cocoa (as well as newcomers to the region) an opportunity to farm cocoa without having to buy land. These systems are especially beneficial in areas where rural labor is in increasingly short supply because of gold mining or urban migration. There are drawbacks to these traditional land tenure systems, which are a serious constraint to adopting higher-input and higher-cost GAP recommendations. These were flagged during an FGD with caretakers in Ghana.  Under the abunu system, the caretaker pays for 100% of inputs and management costs but only receives 50% of the proceeds. Effectively, the marginal return on investment in higher productivity is 50% of what it would be to an owner, which greatly reduces the incentives to apply GAP, borrow, etc.  Under the abusu system, the owner is supposed to use his one-third share to pay for inputs for the whole farm. However, from the owner’s perspective, the marginal return on investment in investing in higher-productivity inputs and services is effectively 50% of what it would be if he managed the fields himself. Not surprisingly, the caretakers interviewed reported that it is difficult to convince the landowner to use all of this one-third share for investing in the farm.  Under both systems, the owner of the farm tends to be preoccupied with the concern that the caretaker may be under-reporting the amount of cocoa produced. For the owner, monitoring costs are high.  Under both systems, land accessed by the caretaker may only be used for cocoa; therefore, unless the caretaker has access to other land, there is no opportunity to plant other cash crops or food security crops, with potential implications for food security of the caretaker household. Caretakers did mention that they could cultivate limited amounts of food crops on the margins of the cocoa land and in gaps in the cocoa field, and some reported that they had access to other lands for food crops or had other income streams.  Under both systems, the tenure of possession is never 100% secure; e.g., if there is a death in the landlord’s family or a gold prospector offers the owner or the chief substantial money, there is a danger that the contracts will be broken. The caretaker FGD participants did not believe there was any discrimination against them in terms of receiving COCOBOD inputs or training; however, it should be noted that the caretaker FGDs were selected from a larger group identified by CHED agents, so almost by definition they were beneficiaries. The caretaker farmers had a couple of suggestions to mitigate the drawbacks of these land tenure systems:  Training to the caretakers on the different land tenure systems and their rights and responsibilities under them.  Training on techniques of better communicating and negotiating with landlords. While a land tenure FGD was not conducted in CDI, it was reported that the common practice of the oldest male child inheriting the farm created uncertain land tenure on the part of the younger brothers, who were given plots of land on the sufferance of the older brother. In addition, it was reported that landlords and tenants may both be reluctant to invest in lands when there is uncertainty about maintaining usufruct (to the sharecropper) and continuity of service (to the landlord). 70 Land Tenure Conclusions Traditional sharecropping systems unintentionally reduce the financial incentive of both landlord and tenant to invest time and money in cocoa productivity improvements. Given that 60 to 70% of the cocoa farming in Ghana may be done under these systems, and possibly at a similar scale in CDI, land tenure is a serious constraint to the adoption of GAP and other practices that are being promoted by ACI and other programs. Follow-on projects should address this difficult and complex topic directly. Land Tenure Recommendations Training sessions targeted at caretakers (and their equivalents in the other three countries) that would provide:  Information on the different land tenure systems in their areas and their rights and responsibilities under them; where to seek further information, how to seek redress in the instance of a dispute; etc.  Techniques of better communicating and negotiating with landlords; e.g., understanding of each other’s motivations. Small-scale pilots of modifications to the sharecropping system. These pilots would test systems where the owner’s and caretaker’s investments and returns from higher productivity are matched. Cocoa-oriented land tenure research The impact of this highly complex topic on cocoa farmers and productivity investments should be more closely examined, using experienced agricultural economists and sociologists. It should include the active participation of the respective governments and local researchers as well as carefully defined terms of reference. The results of the study could provide vital findings that would inform the design of GAP and FAAB training, credit programs, selection of candidates for training, etc. Child Labor Child Labor Findings The evaluation SOW did not directly address the issues of child labor in the cocoa farming household, and the team did not include an experienced child labor specialist. At the request of BFS, however, the team reviewed ACI’s activities related to child labor and recorded observations in the field and in conversations with stakeholders, which are summarized below. The team found a high level of awareness of child labor issues among all players in the value chain, including the cocoa farming households. In particular, farmers were articulate about the need to send their children to school and to avoid using them in “the worst forms of child labor” (WFCL, e.g., handling dangerous chemicals, using sharp objects such as machetes, carrying heavy loads). Every cooperative office and every extension office had posters about child labor prominently displayed. Stakeholders mentioned that the reinforcement of these messages from multiple sources, including government programs, extension officers, NGOs, and value chain companies, was very effective. Extension agents in CDI did mention that some of the GoCDI radio messaging was not sufficiently nuanced and may have caused confusion among some farmers, because the messages seem to imply that child labor should never be allowed rather than focus on eliminating inappropriate or hazardous tasks or tasks that interfere with schooling. This is particularly relevant given labor shortages. 71 When asked casually in the context of the evaluation questions about this component, e.g., about the impact of the SSPs on children’s involvement with mixing chemicals or handling empty unclean toxic containers, none of the cocoa farmers would acknowledge that children were ever involved in these hazardous activities, at least not for the past 10 or 20 years. Older children home on school holidays from boarding schools commonly form groups and work for farmers on certain operations to raise cash, but this reportedly does not interfere with school or include activities considered inappropriate for children of this age. It should be stressed that the above observations were not substantially cross-checked or validated. The evaluation team saw no evidence of child labor on field visits to cocoa growing areas, and no instances of the worst forms of child labor were reported. However, farmer observations should take into consideration the substantial stigma attached to child labor; it would be unlikely for any government official or farming household to admit to its practice. Indeed, it is unlikely that any cocoa farmer using WFCL would even participate in an FGD or KII with an outside team. The school-building programs of WCF members in both countries are intended to support a reduction in the rate of child labor or its impact by offering educational opportunities within reach of children. In addition, through the recently completed ECHOES project, WCF with USAID and other partners have helped expand basic and adult educational opportunities and strengthen community-based organizations in the cocoa region, which also support reduction in child labor. The major certification programs (Rainforest Alliance, Utz, and Fair Trade) all include provisions to reduce or eliminate the worst forms of child labor, and many stakeholders pointed out that as the number of farmers under these certification programs grows, the problem of child labor will decrease. Industry stakeholders mentioned that they perceived the remaining prevalence of children involved in hazardous labor in Ghana to be low, due to the above factors as well as a sophisticated GoG program through the Child Labour Unit, which involves an informal community monitoring system (the Child Labor Monitoring System). However, this would seem to be contradicted by the Tulane report covering the 2013/4 season,38 where the percentage of children in Ghana involved in hazardous work had declined but was still nearly 40%. In CDI, there was a belief by stakeholders that the prevalence of child labor may be more of a problem, especially with children trafficked from Mali or Burkina Faso. However, the Tulane report indicated a lower rate of CDI children involved in hazardous labor (31%) than in Ghana but a rate that was increasing slightly over the previous year. ACI has a number of activities that directly or indirectly address the issue of child labor:  The Component I PPP platforms in Ghana and CDI have TWGs that address child labor issues.  The extension capacity-building training program of Component III included social messages about avoiding the worst forms of child labor.  The spraying service provider (SSP) activity of Component IV was justified in part by the idea that a professional cadre of SSPs would avoid the need for a farming household to involve children in hazardous spraying operations such as mixing chemicals, hauling water to the sprayer, and handling empty containers. 38 Survey Research on Child Labor in West African Cocoa Growing Areas 2013/4, Tulane University. 72 ACI did not track the impact of these activities on WFCL. In the ACI cooperative agreement, WCF agreed that “ACI shall develop a strategy to track incidences and use of child labor in unsafe conditions on cocoa farms to better understand child labor practices and how they change through the project’s implementation in the first annual workplan.” This tracking system was not developed by ACI, and it would have provided WCF, BFS, and the evaluation team with useful, actionable information about child labor prevalence and impact. Under WCF’s CocoaAction Framework, adopted in 2014, two of the “Community KPIs” to be collected include:  Number of children participating in child labor as defined by the International Labour Organization Convention 182.  Percentage of school-age children regularly attending school. However, preparation for data collection for these KPIs still appears to be in process as of this writing, which is not surprising given the sensitivity of the data. Child Labor Conclusions The extent of child labor in cocoa production could not be determined by the evaluation team for two main reasons:  The stigma and sanctions associated with the use of child labor likely led to under-reporting.  There is a lack of resources for this issue in the evaluation contract. Child Labor Recommendations USAID should proactively engage with the Child Labor Cocoa Coordinating Group (CLCCG) of the Department of Labor to better understand the issues and mitigation measures being undertaken. USAID support and the interest of CLCCG may work to the organizations’ mutual advantage. In addition, BFS may consider the following:  Explore the need for harmonization of messages on what is appropriate and inappropriate child labor, and if there is a need, fund the harmonization process.  BFS should request from WCF an updated report on the child labor indicators that have been collected under the CocoaAction and previous initiatives to cross-check against the Tulane University findings.  BFS should retain an experienced child labor specialist(s) to review the WCF data and the DOL-funded Tulane University report on child labor and other key documents. The specialist(s) would then work with WCF and other partners to develop a study that would provide better information on the scope of the child labor problem and provide suggestions for how a follow-on project or other activities could include more focused child labor mitigation activities, and a more robust reporting mechanisms on results. Environment and Human Health Environment and Human Health Environment and human health is primarily affected through the SSP sub-project under Component IV. By outsourcing spraying to trained professionals using protective equipment, pesticide spraying becomes less hazardous to human and environmental health. The human health improvement is because 1) farmers will no longer spray; 2) SSPs (mostly) have the proper personal protective equipment (PPEs) to 73 spray safely; and 3) SSPs are trained on how to safely spray, including which pesticides are safe, correct dosage, and pesticide handling. Environmental health improvements are because of reason 3. Furthermore, an empty container program through which SSPs retrieve empty pesticide containers has both human and environmental health benefits. Also, through Component III the project trained extension agents in environmental good practices for farming, such as keeping trees for shade and biodiversity as well as water, flora, and fauna protection. Findings Health and safety is one of the main motivations given by farmers for choosing to use SSP services. Some farmers even said they knew people who had died from pesticides. Just by taking pesticides out of the hands of, say, 20 farmers, an SSP has a positive human health impact. In addition, SSPs are trained and appear to be applying only approved pesticides, meaning less application of more toxic pesticides. However, some of this human benefit is offset by the fact that not all SSPs had the full PPEs. Some who already had them from previous SSP experience were not given new ones and several complained that they did not have all of the necessary safety equipment. Additionally, it is not clear that all SSPs would purchase all the PPE equipment for themselves once their equipment gets old. Even if SSPs have PPEs, it is not clear whether they wear them. Some SSPs have started their own spray teams; in these cases, it is questionable whether or not these SSPs have the training to safely apply spray or are equipped with their own PPEs. SSPs and farmers both appear confident in the SSPs’ expertise in safe and professional spraying, which should lead to environmental health benefits. In addition, both extension agents and farmers mentioned the training on shade and other tree planting or maintenance as one of the most useful parts of the new extension training. There is some practice of recycling pesticide containers, but it is not clear how extensive it is as SSPs do not always know where to return the containers. Conclusions Based on interviews with farmers and SSPs, it is likely that the SSP program has a positive impact on human and environmental health. However, it should be noted that this was not the focus of the evaluation and that the evaluation team members are not experts in this subject. More research is needed to confirm these preliminary and impressionistic findings. In particular, future research could look at how well SSPs are actually spraying and what affect this has on plant health and biodiversity. In addition, an important qualifier is that in Côte d’Ivoire CropLife’s baseline study found that 62% of farmers were already using a spray applicator, while in Ghana that number is even higher (82%). It is not known if farmers using SSPs were already using another spray provider, in which case the impact is just the difference in the skill and professionalism between the old spray provider and the SSP. Recommendations If USAID/BFS determines that it wants to verify the environmental and health impacts of the SSP activity, a study of the environmental and human health impacts of spray services should be commissioned, designed by specialists and carried out by experts in the two fields. The study should not only compare SSPs with farmer pesticide application, but also compare SSPs with others offering spray applicators on a service basis, include state-subsidized services such as the COCOBOD spray gangs. Farm Labor Findings 74 In cocoa, farm labor is especially for land clearing and weeding, which are relatively unskilled occupations. This is addressed partially through village labor-sharing arrangements between farmers, through college and boarding school youth coming home on breaks, and through hired labor. In Côte d’Ivoire, a significant portion of the hired labor comes from Burkina Faso. In Ghana, labor for cocoa comes mostly from other parts of the country. However, in both countries, labor for hire is reported to be increasingly scarce and consequently relatively expensive. Factors accounting for scarcity include the rise of informal gold mining, reduction in the use of child labor, and rural-to-urban migration of those who otherwise might accept farm labor. Conclusions Labor shortage is a significant constraint on cocoa productivity. Some farmers reported that they could not afford to hire this labor and in several of the farms visited, brush clearing, pruning, mistletoe removal, etc. had not been carried out in a long time, resulting in overgrown farms. Introduction of GAP increases the demand for labor, particularly in cocoa, as noted in the LMC 2014 study, which exacerbates the scarcity. Recommendations Given that the cost and shortage of labor are significant constraints to adopting GAP and higher productivity, a follow-up project should consider:  Labor constraint study. Analyze key labor constraints in the cocoa production and post￾harvest processing cycle and identify local or external technology and methods to cost￾effectively address these constraints.  Mechanization pilot initiatives. Based on the labor constraint study, this should be done using a market facilitation approach to ensure that proposed methods and technologies can be provided on a commercially sustainable basis. Design and Management The original ACI strategy was to operate primarily through provision of grants:  Core grants of approximately $100,000 for longer-term technical assistance partners  Annual implementation and strengthening grants ranging from $25,000 to $40,000 Core grants were to go to key implementation partners, anticipated as follows: Component I: COPAL Component II: IITA Component III: not specified Component IV: IFDC In practice, WCF assumed a much larger responsibility for management and oversight of activities than anticipated directly and through sub-contractors. Component I was taken over by ACI after it was clear that COPAL was not able to implement its workplan effectively. Component II was implemented not only by IITA, but also by the NRAs of the four countries as well as the ACBWG. Component III extension activities have been implemented by WCF 75 using specialized training contractors, while the GQI, New Generation, and PETWE programs have been implemented through grants. Component IV was implemented by grants to TechnoServe and CropLife. No annual implementation and strengthening grants were issued (see Table 16), which is not unexpected, given the high level of ACI time that would have been required to review, oversee, and evaluate sub-awards of this size and number. Many of the intended tasks for the implementation/ strengthening grants were assumed by WCF directly or through contractors. The number and size of the core grants were much greater than originally anticipated. Given the complexity and the multi-year nature of the activities, WCF appropriately adjusted its grant strategy to fit the objectives of the project. Table 16. Planned vs. Actual Awards Type of Award Planned Actual Planned Size Average Size, Actual Core 4 to 5 14 $100,000 $386,000 Annual Implementation 45 +/- 0 $25,000 to $40,000 n/a The duration of the core grants was rather short—especially the grants to TechnoServe and CropLife to fund pilot activities—approximately two years. In the agricultural sector, establishing and refining a proof of concept in two years is difficult. This is particularly true with a perennial crop such as cocoa, where some of the benefits of introducing GAP and the use of inputs may take multiple seasons to be realized by farmers. ACI implementation was rather slow in the beginning, with only four sub-awards issued in the first year of implementation; one of those (to COPAL) was quickly suspended. Component III’s extension capacity building had no activities implemented in the first year. The pace of ACI implementation picked up in the second year, with seven more awards granted, but nonetheless the slow pace of implementing sub￾awards has been a constraint to achieving objectives. The reason for the slow pace of implementation seems primarily to be the greater-than-anticipated time for WCF to negotiate sub-awards, possibly due to some initial staffing issues. ACI was WCF’s first experience as a USAID prime award recipient, and inevitably there was a learning curve that slowed down implementation. WCF’s selection of sub-awardees on the whole was commendable. The exception is the selection of COPAL, which was known to be a financially struggling regional organization, to take the lead on Component I. Fortunately, WCF recruited as COP the former secretary-general of COPAL, an experienced cocoa industry specialist, who took on the responsibility of managing Component I. Communications and Coordination Communications with ACI members and other key stakeholders could be strengthened. The ACI steering committee members interviewed were unaware of, or not clear about, the parts of the project with which they were not directly involved. There was also a recurring theme of confusion between the roles of ACI and CocoaAction. The ACI steering committee members’ better understanding and involvement with ACI may have strengthened the project by increasing the possibility of synergistic partnerships. Surprisingly, the IDH office in CDI was not aware of key ACI components such as the SSP program; their more active involvement and support could have helped with the scale-up of this pilot initiative. 76 If USAID wishes to continue supporting WCF focus on activities and countries outside of CocoaAction, WCF will need to broaden CocoaAction’s scope or agree to manage a project that does not fall within CocoaAction’s purview. Synergies with the USAID portfolios USAID/Ghana indicated its own Feed the Future program portfolio has matured, providing an opportunity to work with WCF and cocoa value chain actors on policy and project initiatives. USAID/Ghana already works closely with major agribusiness players that are ACI partners, such as Wienco and Cargill. USAID/Ghana indicated that it will consult with USAID/BFS project management and consider options on how and where to engage with ACI and its successor programs. Some specific possibilities for collaboration mentioned by USAID/Ghana include:  The FinGap loan guarantee program could consider expanding its reach to cover SSPs, especially if the SSPs would be providing services to rice, maize, or soy farmers.  Through the Business Advocacy Challenge Fund, in part financed by USAID, support could be provided to develop or strengthen associations representing cocoa farmers that seek to advocate for more market-oriented policies, laws, and implementation.  A USAID-funded advisor to the agricultural desk at the Ministry of Finance (particularly important because COCOBOD falls under this ministry) could support market-oriented policy advocacy activities for the cocoa sector. USAID/CDI was interested in the ACI project, and saw possible synergy on policy advocacy; however, its capacity at present to engage more directly is limited. The Embassy representative at the meeting mentioned that they are quite active in supporting the interests of US companies in the cocoa sector and would support policy advocacy that would benefit the industry. Staffing The original staffing anticipated included as core positions the COP, a senior PPP specialist, and a farmer training/input supply specialist complemented by other long- and short-term staff as required. The COP has essentially doubled as the PPP specialist, taking advantage of his long experience working with industry and government in the cocoa sector. He has been assisted by a highly competent farmer training/input specialist. However, there were some areas where skill gaps existed, which appear to have limited ACI’s achievements. These include:  Results-based policy advocacy. These skills include consensus building, framing the issues, identifying bottlenecks, and designing research and advocacy solutions to address these bottlenecks.  Market facilitation. These skills include the ability to: o Identify opportunities for market-based input/service delivery o Establish and nurture commercial partnerships through training, TA, and limited risk￾sharing support Monitoring and Evaluation Figure 7 and Table 17 represent the ACI results framework and table of indicators, respectively. In the results framework, the four objectives (ACI_1–ACI_4) correspond to the initiative’s four components. The table of indicators include the project’s 14 annually measured indicators, 12 of which coincide with FTF indicators. It is important to note that ACI was one of FTF’s earliest projects and therefore its results framework and indicators were still under development. 77 There are several issues with both the logic of the program as reflected in the results framework and the choice of indicators. These issues confuse program logic, which means that the indicators are not the best choices to account for and manage project performance. One issue is that the results framework is not entirely reflective of the project’s logic and intended results as understood from other project documentation including the Cooperative Agreement and conversations with project staff and implementing partners. For example, effective value chain co-management is not a key objective of the project. Likewise, Component II does not promote improved farming practices, but rather improved planting material. Under ACI_4, the two sub-IRs are confusingly similar and do not reflect the two distinct sub-components within that component. Figure 7: African Cocoa Initiative Results Framework AFRICA COCOA INITIATIVE (ACI): GOAL: Support sustainable agriculture and improved food security on diversified cocoa farms in West Africa: ACI_1: Strengthened national partnership platforms (FTF IR 1.3) ACI_2: Improved farm productivity (FTF IR 1.1) ACI_3: Enhanced public and private sector extension and farmer training services (FTF IR 1.1) ACI_4: Improved market driven farming input supply services (FTF IR 1.2) ACI_4.1: Increased access to market services ACI_2.2: Improved capacity building in cocoa research ACI_2.1: Improved adoption of new farming practices ACI_3.1: Increased knowledge, technology and information transfer ACI_1.2: Improved use of evidence-based policies and decisions FTF IR 1.3.2 ACI_1.1: Effective value chain co-management established (public/private) ACI_4.2: Increased availability of upgraded products and services In addition to the confusion and ambiguity about the project’s logic, there are issues with how results are measured. It is not clear how the indicators below, which ACI reports in its semi-annual reports, relate to the intermediate results (IR) and sub-intermediate results in the above results framework. Each sub-IR does not have its own indicator, each IR does not have its own indicator, and some indicators measure results across multiple outcomes. Therefore, it is very difficult to track, from the indicator results reported by ACI, how well the project is progressing toward its originally intended objectives as reflected in the results framework. 78 Table 17. African Cocoa Initiative Table of Indicators USAID Indicator # Indicator Associated Component39 Life of Project: FY2012–FY2016 Progress through March 2015 Target Achieved % Achieved 4.5.1-24 Number of policies, regulations, and administrative procedures in development, passed, or being implemented as a result of USG assistance 1 12 4 33% 4.5.2-2 Number of hectares under improved technologies or management practices as a result of USG assistance 2,4 300,000 242,172 81% 4.5.2-5 Number of farmers and others who have applied new technologies or management practices as a result of USG assistance 4 100,000 102,621 103% 4.5.2-7 Number of individuals who have received USG-supported short-term agricultural sector productivity or food security training 3,4 100,000 86,129 86% 4.5.2-11 Number of food security private enterprises (for￾profit), producer organizations, water users associations, women’s groups, trade and business associations, and community￾based organizations (CBOs) receiving USG assistance 3,4 200 92 46% 4.5.2-12 Public-private partnerships formed as a result of USG assistance 1,4 20 24 122% 4.5.2-23 Value of incremental sales (collected at farm level) Goal $252.11M $90.03M 36% 39 This column was added after discussion with project staff. 79 USAID Indicator # Indicator Associated Component39 Life of Project: FY2012–FY2016 Progress through March 2015 Target Achieved % Achieved attributed to FTF implementation 4.5.2-27 Number of members of producer organizations and community-based organizations receiving USG assistance 3,4 25,000 14,441 58% 4.5.2-36 Value of exports of targeted agricultural commodities as a result of USG assistance Objective $360.15M $213.7M 59% 4.5.2-42 Number of private enterprises, producers organizations, water users associations, women’s groups, trade and business associations, and community￾based organizations (CBOs) that applied improved technologies or management practices as a result of USG assistance 3,4 32 37 115% 4.5-4 Gross margin per unit of land, kilogram, or animal of selected product (crops/animals selected varies by country) Objective $750 $968 129% 4.5-11 Market discount of targeted agriculture commodities Objective <30% 37% -22% WCF Indicators Productivity Kilograms per hectare per year (at farm level)—This is an assumed component of gross margin (4.5–4) above but is reported separately for WCF member purposes (target) Objective 1000 kg/ha 636 64% 80 USAID Indicator # Indicator Associated Component39 Life of Project: FY2012–FY2016 Progress through March 2015 Target Achieved % Achieved Percent of Cocoa reference price Reported as a percentage by country and is a derived equivalent of market discount (4.5.11) above but is reported separately for WCF member purposes Objective >70% 68% 96% More importantly, it is difficult track the performance of each component. This is important in a project where each component is so distinct. For example, it is not clear how, from the above indicators, one can fully capture the performance of Component II. The only indicator that tracks Component II, Number of hectares under improved technologies or management practices as a result of USG assistance, also includes progress from Component IV. Likewise, for Components III and IV the indicators are so broad that they are not very useful in understanding how well those two components are performing. Given that each component of the project is relatively discrete, this is an important oversight. This is the result of confusingly aligned and only marginally relevant indicators. Part of the problem is that FTF indicators are not designed specifically for cocoa. Still, this is only part of the problem as the indicators are too broad (simply counting the number of organizations or participants). For this reason the indicators are trivial. To be clear, these issues apply to the FTF indicators that ACI was required to choose from. ACI and its partners track more relevant and useful indicators, such as hectares of seed and budwood gardens established, extension agents trained, SSPs trained, and farmers reached through SSPs. However, these are not included among the official project indicators and instead are scattered through the semi-annual reports as well as partners’ reports to ACI. This gives the impression that these indicators are less important than the USAID-mandated indicators, when in fact these are the indicators the project is managing to because they better measure what it is aiming to achieve. It is important to note that the project works primarily on a cocoa enabling environment but aims to achieve farmer-level impact. This obviously presents a measurement challenge; in fact, the farm-level indicators of project goals are tracked through Component IV. Farm-level impacts attributable to Component I are practically impossible to track. For Component II, they will only take place on a significant level after project completion. However, it would have been possible to better track farm￾level results from improved extension—Component III. This could have been done by increasing the M&E capacity of ANADER and CHED, which would have been a valuable substantive contribution beyond allowing ACI to improve its own M&E system. One way to improve the M&E of both extension agencies is to train agents themselves to monitor outputs and outcomes when they are in the field for training. This could be complemented by an annual sample survey that would assess the impact of training on farmers’ practices and livelihoods. Indicators are also poorly defined and explained. In Table 17, the column showing which component each indicator relates to was added only after discussion with ACI staff. Additionally, indicators are hard to interpret because performance reports do not explain how indicators are defined and measured. Even definitions and explanations given for some indicators in the performance indicator reference sheets are unclear. 81 The result of all this is that the performance indicators selected do not give a good indication of project performance and, perhaps more importantly, do not facilitate results-based management. The annual data collection period also does not lend itself to results-based management. Together this represents a missed opportunity to generate data that would assist, rather than just burden, the implementer. It is also a missed opportunity to better promote project progress. M&E Recommendations A follow-up implementer should be encouraged to choose and present custom indicators, in addition to required FTF indicators, that it believes will accurately reflect project design and be able to inform project management. It should also be required to clearly explain all indicators and their limitations so they can easily be interpreted by the reader. Finally, FTF should continue to solicit feedback from its implementers including WCF on how standard indicators can be made more relevant, practical, and useful. 82 ANNEXES Annex I: Evaluation Statement of Work EVALUATION STATEMENT OF WORK: AFRICAN COCOA INITIATIVE PERFORMANCE EVALUATION I. BACKGROUND INFORMATION A. IDENTIFYING INFORMATION 1. Project/Activity Title: BFS Performance Evaluation: African Cocoa Initiative 2. Award Number: AID-OAA-I-10-00013/AID￾OAA-TO-14-00021 3. Award Dates: Evaluation period for African Cocoa Initiative: July 27 2015 – December 31, 2015 4. Project/Activity Funding: USAID/Bureau for Food Security 5. Implementing Organization(s): Social Impact, Inc. 6. Project/Activity COR/AOR: COR: Salik Farooqi; AM: Jessica Cagley 83 B) DEVELOPMENT CONTEXT 1. Problem or Opportunity Addressed by the Project/Activity being Evaluated The U.S. Government’s Feed the Future (FTF) Initiative, led by USAID, currently targets hunger and poverty in nineteen FTF focus countries and fourteen aligned countries supported by Regional Missions in Asia, Africa, and Latin America and the Caribbean, as well as three Strategic Partner countries. FTF aims to tackle the root causes of global hunger by sustainably increasing agricultural productivity to meet demand for food, supporting and facilitating access to strong markets, increasing incomes so the poor can meet their food and other needs, and reducing under-nutrition. FTF has two key objectives that address the principal determinants of food insecurity and poverty – inclusive growth in the agriculture sector and improved nutritional status, especially for women and children. These objectives have direct causal linkages to FTF’s overarching goal of sustainably reducing hunger and poverty, and they support the United Nation’s Millennium Development Goal 1 to halve the proportion of people living in extreme poverty and suffering from hunger. Within the Bureau for Food Security (BFS), the Office of Markets, Partnerships and Innovations (MPI) is charged with the mandate to promote new approaches to food security through innovative partnerships that improve market access for food insecure households in FTF countries. West Africa is a dominant player in the World cocoa market with a 70% share of the market. Cocoa is a key sector for West African cocoa producing countries generating over eight billion dollars in revenue. As a smallholder crop with a typical farm size of two hectares or less, cocoa supports a significant number of farm households in the region – over two million. Global demand for cocoa is growing as consumers in emerging economies have more disposable income and evolving tastes. However, smallholder cocoa productivity and quality in West Africa is low, with some estimates placing it at less than half it’s potential. This stunts the sector’s role as driver of economic growth and reduces the returns to smallholder farmers. Smallholder cocoa farmers’ productivity and quality are low for a variety of reasons including lack of improved planting materials40, low use of inputs including fertilizer, farmers lacking up-to-date knowledge on best cocoa practices and market opportunities, and a lack of finance. The low productivity and quality of cocoa produced results in lower household incomes, which in turn lowers households’ food security. The African Cocoa Initiative (ACI) works directly and indirectly to increase household incomes derived from cocoa production, which in turn should increase food security. 2. Target Areas and Groups Target Areas ACI works in four countries in West Africa – Nigeria, Cameroon, Ghana and Côte d’Ivoire. USAID has requested that this evaluation focus on Ghana and Côte d’Ivoire, and fieldwork will be limited to those two countries. ACI works primarily through sub-grants to partners; public sector, private sector, and NGOs. Four ACI sub-grants have been issued that cover all four countries, to the Alliance of Cocoa Producing Countries (COPAL), the International Institute for Tropical Agriculture (IITA), CropLife, and the Cocoa Research Institute of Ghana (CRIG). Two sub-grants have been issued specifically focused on Ghana, to CRIG and Tcho, and two sub-grants were issued specifically focused on Côte d’Ivoire, to the Centre National de Recherche Agronomique (CNRA) and Burex-CI. One sub-grant to TechnoServe covered both Ghana and Côte d’Ivoire. Target Groups Ultimately, small-scale cocoa farmers (those with less than 5 hectares of cocoa), and their households are the primary ACI beneficiary group. Secondary target groups include spraying service providers (SSPs) and community or village extension workers. 40 Many cocoa farms were established 30 or more years ago, and the trees have lost productivity. 84 C) INTENDED RESULTS OF THE PROJECT/ACTIVITY BEING EVALUATED The core problem that BFS intended the ACI activity to address is the low levels of food security among small-scale cocoa farmers in West Africa. The ACI activity is based on the following four assumptions concerning cocoa farming and food security:  as a cash crop, cocoa significantly improves household incomes and thereby enhances food security by affording greater access (purchasing power) for foodstuffs;  good cocoa cultivation fosters crop diversity and the integration of other food crops, especially plantain, cassava, and fruit trees in permanent association;  cocoa-focused training and capacity development activities produce skills and benefits transferable to other crops; and  cocoa as a national export crop offsets the cost of other food imports The ACI development hypothesis is that inadequate national policies and programs, the use of unimproved planting materials, and a lack of inputs and agronomic know-how are contributing to low productivity and low prices, leading to low household incomes and therefore low levels of food security. ACI aims to address the underperformance of West African cocoa with the goal “to support sustainable agriculture and improved food security on diversified cocoa farms in West Africa”. The project is designed to address four constraints in the cocoa sector that are adversely affecting farmers: 1. Cocoa boards and other institutional arrangements inadequately performing public￾private liaising role and channeling investment to cocoa 2. Low quality cocoa planting materials 3. Lack of farmer knowhow on improved cocoa farming practices 4. Lack of farmer access to inputs and finance These constraints are to be addressed and the ACI’s goal to be achieved through four objectives shown in the in the results framework below. This results framework shows these objectives along with associated intermediate results (IRs) and sub-IRs the project is accountable for as it aims to achieve its goal. 85 Figure 1: ACI Results Framework The causal relationship between the sub IRs and the four IRs representing the project’s four objectives is logically convincing prima facie. However, the relationship between achieving the four objectives and the ACI goal of sustainable and improved food security is not as self-evident. However, it is important to note that ACI is not a stand-alone project and was designed and implemented to complement the WCF Cocoa Livelihood Project (CLP) and other activities that work more directly with small-scale cocoa farmers on the ground. The evaluation will assess how well WCF has achieved these four objectives, and how likely this translates into higher farmer incomes which is how the project defines its food security goal. Whether or not increased incomes leads to increased food security also depends on other factors such as household behavior, production of food crops, and physical access to food, in addition to having the means to purchase food. Therefore, this evaluation will assess whether achieving ACI’s four objectives and achieving higher household incomes has impacted or seems likely to impact food security. D) APPROACH AND IMPLEMENTATION ACI is a public-private partnership (PPP) intended to advance the objective of the Bureau for Food Security’s Markets, Partnerships and Innovations Office to “Develop 3-5 transformative private sector partnerships that create innovative models, drive scale across countries, or bring new actors/types of countries to the food security agenda.” Support for this Initiative will meet FTF’s food security objective of increasing the incomes of smallholder farmers. ACI follows the Sustainable Tree Crops Program (Phases 1 and 2), which was implemented by IITA. ACI was designed recognizing the need for a more systemic AFRICA COCOA INITIATIVE (ACI): GOAL: Support sustainable agriculture and improved food security on diversified cocoa farms in West Africa: ACI_1: Strengthened national partnership platforms (Contributes to FTF IR 1.3) ACI_2: Improved farm productivity (Contributes to FTF IR 1.1) ACI_3: Enhanced public and private sector extension and farmer training services (Contributes to FTF IR 1.1) ACI_4: Improved market driven farming input supply services (Contributes to FTF IR 1.2) ACI_4.1: Increased access to market services ACI_2.2: Improved capacity building in cocoa research ACI_2.1: Improved adoption of new farming practices ACI_3.1: Increased knowledge, technology and information transfer ACI_1.2: Improved use of evidence-based policies and decisions Contributes to FTF IR 1.3.2 ACI_1.1: Effective value chain co-management established (public/private) ACI_4.2: Increased availability of upgraded products and services 86 approach to building the capacity of key partners in cocoa value chain, and was designed to complement the CLP as well as other WCF member organizations programs that act more on the ground. The ACI program is implemented by USAID through a cooperative agreement with the World Cocoa Foundation (WCF), under the rubric of a Global Development Alliance (GDA), USAID’s premier model for PPPs. WCF is an international membership organization founded in 2000 that represents more than 100 member companies across the global cocoa value chain. WCF’s stated aim is to create a sustainable cocoa economy by putting farmers first. This includes promoting agricultural and environmental stewardship and strengthening development in cocoa-growing communities. The ACI program is co-funded by Dutch Sustainable Trade Initiative (IDH). IDH stated purpose is to accelerate and up-scale sustainable trade by building coalitions of companies, civil society, governments and other stakeholders that will deliver impact on the Millennium Development Goals 1 (poverty reduction), 7 (safeguarding the environment) and 8 (fair and transparent trade). USAID has committed to providing $5,000,000 over a five-year period for ACI, leveraging over $8,000,000 in matching and in-kind contributions from IDH and 14 WCF members. The ACI program is USAID’s flagship program in the cocoa sector in West Africa. The program aimed to double cocoa productivity for 100,000 farm households and in doing so raise their per capita income by 150-200%. This increased income is assumed to lead to increased food security among cocoa producing households thereby achieving ACI’s goal. ACI was also intended to advance the objectives of WCF’s members (who handle 80 percent of the world’s chocolate) by increasing the quantity and improving the quality of cocoa produced in West Africa. Although not by original design, WCF/ACI now is being implemented in large part under the framework of CocoaAction, a WCF program initiated in 2014, whose aim is to enable at least 300,000 farmers (200,000 in Côte d’Ivoire and 100,000 in Ghana) to increase productivity and advance development initiatives within their communities. WCF/ACI’s approach draws on national stakeholders’ plans and vision for their cocoa industry. ACI was designed to build national cocoa stakeholders capacity to implement their own plans for the sector rather than impose them externally. This approach stems largely from the recent experiences of other WCF implemented programs, such as the WCF ECHOES, and the Sustainable Tree Crops Program. According to the ACI agreement document, the program has four objectives. These are described below. Objective 1 “Strengthening national partnership platforms investing in agriculture and cocoa” This was to be achieved through the development and strengthening of four national platforms. Key activities envisioned included the development of country level roadmaps, investment plans, and policies. Objective 2 “Improving crop productivity through better planting material” This was to be achieved through a five-year program of regional research, led by the African Cocoa Breeders Working Group (ACBWG), and coordinated by IITA. The research would develop, pilot and test new production approaches for producing and distributing “certified/improved” planting materials, and demonstrate superior performance of “certified” planting materials under good agricultural practices in collaboration with farmers and communities. Activities envisioned included 1) a comprehensive program of genetic fingerprinting to ensure genetic purity of improved planting materials, 2) national seed and budwood gardens established to be the source of improved planting materials, and 3) public and private internal regional planting material propagation/distribution centers set up so that 10,000,000 trees (approx. 10,000 hectares) can be distributed annually in each country. Objective 3 “Enhancing public and private sector extension and farmer training services” This was to be achieved through strengthening and building up existing training institutions so that they are able to design, manage, and monitor training program. Activities envisioned included pilot programs 87 to use community or farmer led trainers, and the use of cellphones to deliver and monitor extension, and other activities to reach 100,000 households in the four countries. Objective 4 “Foster market-driven farming input supply services” This was to be achieved by strengthening the input supply system to bring quality products and services to farmers. Envisioned activities included training agro-input dealers, expanded credit programs, and soil testing/fertilizer recommendation programs, and other activities that would reach 100,000 households in the four countries. WCF/ACI is largely implemented through sub-grants to a range of partners, shown below in Figure 2. Figure 2: ACI Sub Grants E) DOCUMENTS For this performance evaluation of ACI, the evaluation team will review a wide variety of documents, many of which have already been provided by BFS and ACI. Other documents will be requested from ACI and from sub-grant partners during the fieldwork preparation phase, and others will be requested of partners and sub-partners while the evaluation team is in the field. The sub-agreement documents and reporting from awardees will be particularly important. The primary documents and types of documents to be reviewed are listed below.  ACI award documents, including the BFS cooperative agreement with WCF, performance monitoring plans and  Documents from and on sub-grant recipients, including award contracts, periodic progress reports, data for reporting on indicators, meeting notes, updates, memos, notifications, etc.  Documents on FTF objectives and implementation  ACI progress reports  ACI implementation documents, including work plans (FY13 and FY14), ACI Partnership Portfolio (current partners, completed partnerships, current partnerships, the FTF all four countries), FTF ACI Factsheet, ACI Organizational Chart Component Activity Detail Awardee Name Country Start Date End Date Award Status One Coordinating in-country PPPP activities COPAL All 1-Jan-11 31-Dec-15 Suspended Two Coordinating DNA extraction and standardization for genetic fingerprinting IITA All 16-Aug-12 31-Dec-13 Ended Four Implementing pilot agro-chemical application (SSP) program CropLife All 21-Jan-13 31-Dec-14 Ended Two Establishing 15 hectares of seed gardens and 10 hectares of budwood gardens IRAD Cameroon 11-Jun-13 16-Jun-16 Active Three Implementation of New Generation of young cocoa farmers program CICC Cameroon 15-Jul-14 30-Apr-16 Active Two Establishing 50 hectares of seed gardens and 10 hectares of budwood gardens CNRA Côte d'Ivoire 11-Jun-13 11-Jun-15 Ended Three Training of ANADER Extension Staff in Côte d'Ivoire BUREX-CI Côte d'Ivoire 15-Jul-14 30-Dec-14 Ended Four Implementing pilot input credit scheme TechnoServe Côte d'Ivoire & Nigeria 1-Oct-12 31-Mar-15 Ended Two Establishing 50 hectares of seed gardens and 10 hectares of budwood gardens CRIG Ghana 11-Jun-13 30-Sep-15 Active Three Ghana Quality Innovations project for improving cocoa quality through flavour analysis TCHO Ghana 12-Jun-12 30-Sep-16 Active Two Establishing 15 hectares of seed gardens and 10 hectares of budwood gardens CRIN Nigeria 11-Jun-13 16-Jun-16 Active One Harmonization of extension training manuals in Nigeria Vicanason Nigeria 20-Apr-15 31-Jan-16 Active Two African Cocoa Breeders'Working Group Project CRIG All 11-Jun-13 11-Jun-16 Active One Harmonization of extension training manuals in Cameroon FODEC Cameroon 1-Sep-15 31-Mar-16 Draft 88 II. EVALUATION RATIONALE A. EVALUATION PURPOSE AND INTENDED USERS BFS has contracted Social Impact (SI) to undertake a performance evaluation of the African Cocoa Initiative (ACI) program, which runs until September 29, 2016. This evaluation is being carried out as part of the BFS Performance Evaluations task order held by SI, commissioned by BFS in accordance with ADS 203.3.1.3: “Each USAID Mission/Office is required to conduct at least one evaluation of each large project it implements.” Consistent with the Office for Learning, Evaluation, and Research’s 2011 Evaluation Policy, the ACI performance evaluation is intended to provide empirical evidence to respond to evaluation questions (see section II.C. below) designed to support learning and continuous improvement for BFS’ work and FTF more broadly, in addition to providing evidence to answer questions in the Feed the Future Learning Agenda. The evaluation will provide findings and recommendations that BFS, ACI, FTF Missions, and their sub-partners can use to improve the effectiveness of their existing and future activities to better achieve intended outcomes. B. AUDIENCE AND INTENDED USES This evaluation is intended for USAID BFS, WCF, and project partners and stakeholders. USAID BFS will use this evaluation specifically to inform future programming and investment decisions under FTF, including activities that use cocoa or other cash crops as a means to achieve food security. The WCF and WCF/ACI partners, and individual WCF members will use this evaluation to guide how they will continue and modify this and other related income generation and food security activities under the context of the CocoaAction Framework, launched in 2014. Other stakeholders in the cocoa value chain (e.g. ministries, financial institutions, input suppliers, etc.) will use the evaluation to guide their own business or policy activities, models, and strategies. C. EVALUATION QUESTIONS This performance evaluation will address the five central evaluation questions, listed below. Illustrative evaluation questions, based on discussions with BFS/MPI staff, were provided in the initial scope of work provided to the evaluation team. Based on subsequent discussions with BFS staff, and a review of key evaluation documents, these original illustrative questions and later issues raised by BFS have been incorporated into the following five evaluation questions. 1. How did the ACI activities contribute or likely contribute to the objectives and results as intended for the project overall and its four components?  The supply of improved agricultural inputs (including planting material) and services (especially extension and finance) to target farmers. For example: o To what extent have planting materials been improved at the national level and how likely are these improved planting materials to be used profitably by farmers? o How do farmers perceive extension services to be improved or not improved and why? o How commercially viable are the input services established and strengthened through the project with and without government subsidies?  Public/private dialogue processes and institutions leading to tangible change, through improved consensus among actors and influence on policy direction  The effect of ACI-facilitated extension on the understanding and implementation of improved cocoa farming practices, including practices related to environmental stewardship, safety and child labor. 2. How likely are the ACI activities to positively (or negatively) affect food security among cocoa farmers and within cocoa-producing communities? Might other activity designs and intervention areas lead to greater impacts on food security and income? This will be based on a a) review of the literature on cocoa farming and food security, b) review of project documentation and c) fieldwork using KIIs and FGDs. This will be addressed in the context of the relative mix of crops (cocoa versus other cash 89 and food crops), and relative intensity of production (e.g. use of inputs, labor, etc.) by the target cocoa farmers. 3. What other benefits or drawbacks (i.e., unanticipated outcomes or spillover effects) occurred as a result of the program for each component? To what degree were the changes in cocoa farming practices promoted by ACI associated with changes in farming practices for other crops? 4. What are the most important lessons learned from implementing ACI (including, where feasible and appropriate, observations concerning cross-cutting issues such as child labor, environment, health and safety for women and children) that would inform similar efforts? 5. How might the ACI program or successor initiatives be designed to more cost effectively and sustainably achieve and measure objectives, results, and outcomes? III. EVALUATION DESIGN AND METHODOLOGY A. EVALUATION DESIGN The ACI evaluation will follow the Utilization Focused Evaluation (UFE) approach as a guiding framework to ensure that results are useful to USAID/BFS and other important users. The UFE approach uses the following 11 steps; this section focuses on evaluation design, step 741: 1. Assessing Program Readiness 2. Assessing Evaluators’ Readiness 3. Identifying Primary Intended Users 4. Situational Analysis 5. Identification of Primary Intended Uses 6. Focusing the Evaluation 7. Evaluation Design 8. Simulation of Use 9. Data Collection 10. Data Analysis 11. Facilitation of Use The evaluation design – step seven of the UFE approach -- utilizes mixed methods research. It will combine analysis of quantitative and qualitative data from primary and secondary sources. Where possible the evaluation team will use these multiple sources of data to triangulate results and provide more rigor to findings. An experimental or even quasi-experimental design is not possible given that a counterfactual to participating households was not set up ex ante and the project’s national-level objectives that make it impossible to identify a counterfactual for those national-level activities. However, wherever possible, the evaluation team will attempt to construct the counterfactual scenario of what would have happened without the project. For example, in investigating farmer productivity we may include in our analysis historical national trends in productivity as well as weather trends, subsidies and other programs that might have influenced farmer productivity over the project period but that are not related to ACI. Time permitting, we will also consider the inclusion of cocoa households or organizations that did not directly benefit from ACI activities. Together this would then allow the evaluation team to roughly estimate how much we can attribute productivity increases and other benefits to the project activities. As much as possible the evaluation will be designed to not only measure project performance but also to understand project performance. This will start with understanding the project’s logical framework and analyzing the relationships between project-reported results at different levels of the framework. The 41 For more on the UFE methodology, see Ricardo Ramirez and Dal Broadhed “Utilization Focused Evaluation”, 2013. 90 evaluation team will complement this with interviews with project implementers and stakeholders, as well as analysis of factors external to the project suspected of influencing project outcomes results. B. DATA COLLECTION METHODS In addition to reviewing data from secondary sources such as project performance reports and evaluations, the evaluation team will collect primary data using at two different methods as described below. The evaluation team will collect data disaggregated by gender and will ensure that women are included and able to actively voice their perspectives throughout data collection. Additionally, for all data collection methods the team will aim to include farmers from different socio-economic backgrounds as determined by proxies such as land size and income. Together this will make sure the evaluation accounts for potential disparate effects related to these respondents’ different characteristics. 1. Key Informant Interviews Key informant interviews (KIIs) will be used to collect information from all stakeholders. KIIs will be employed in answering all evaluation questions. In comparison with focus groups, which are discussed below, KIIs will be used when a respondent represents a unique project stakeholder or is likely to possess unique information or perspectives. The evaluation teams will use interview checklists for KIIs to ensure key questions are answered. These checklists will allow interviewers the freedom to ask spontaneous questions to uncover important and unanticipated information. The evaluation team will design a separate key informant interview guide for each set of stakeholders. Each guide will include a core set of questions that will be asked of all key stakeholders. Each set of stakeholders will also be asked a set of questions that are specific to their role in ACI. KII Sampling Plan: The evaluation team will target the stakeholders listed below for conducting KIIs. Note that that the compressed time frame of the fieldwork and the distance between sites may limit the evaluation team’s ability to reach all of the targeted stakeholders. In order to reduce the potential for bias in contributors’ responses, interviews will not mix stakeholder groups. For example, interviews conducted with sub-partner institutions will not include partner institutions, and interviews with agro-dealers will not include partner or sub-partner institution staff. 1. WCF staff and ACI Steering Committee members: The evaluation team will conduct KIIs with the WCF/ACI COP, key team members, and the senior programs director at WCF. Key steering committee members will be interviewed. 2. Partners (Sub-grant recipients): The evaluation team will endeavor to interview staff of each partner institution that received a sub-grant through ACI. Where it is not possible to conduct face-to-face meetings, other methods of collecting information (such as phone interviews) will be used. Table 1: Partners to be Interviewed a. Field b. United States i. COPAL ii. IITA iii. CropLife iv. TechnoServe v. CNRA vi. Burex CI vii. CRIG i. Tcho Ventures Inc. ii. Guittard 3. Sub-partners (organizations that work directly with sub-grant recipients): The evaluation team will interview staff involved with ACI at each of the sub-partner institutions that received funding through ACI. While the intention is to conduct interviews with as many sub-partners as possible, limited time in each country and the distance between locations where different sub-partners operate may result 91 in the evaluation team using opportunistic sampling to prioritize meeting with sub-partners that are located close to each other geographically. Where it is not possible to conduct face-to-face meetings, other methods of collecting information (such as phone interviews) will be used. Table 2: Sub-Partners to be interviewed a. Ghana b. Côte d’Ivoire i. Ghana Cocoa Board (COCOBOD) ii. Cocoa Platform Secretariat iii. Cocoa Health and Extension Division (CHED), Seed Production Division (SPD), and other COCOBOD divisions i. Conseil du Café-Cacao ii. Cocoa Platform Secretariat iii. Agence Nationale d’Appui au Developpement Rural (ANADER) iv. Agriculture Ministry departments 4. Cocoa Value Chain (CVC) stakeholders: especially licensed buying companies (LBCs), input/equipment providers, cooperatives, service providers, and financial institutions that provide inputs or services directly to small-scale cocoa-farmers, or purchase products from cocoa farmers, and that worked directly or indirectly with ACI partners/sub-partners. The final list will be developed after consultation with the ACI COP and staff, but an example follows in Table 3: Table 3: Example of Cocoa Value Chain stakeholders to be interviewed Input Providers Financial Institutions Buying Companies Research and Extension providers i. Calli Ghana ii. Chemico iii. Others i. EcoBank i. ECOM i. Cocoa Research Institute of Ghana 5. USAID staff: Relevant USAID staff in BFS, and the Ghana and Côte d’Ivoire missions. 2. Focus Group Discussions Focus groups discussions (FGDs) will be carried out with small-scale cocoa farmers and laborers and used to elicit information for questions 1-5, particularly for Components 3 and 4 Not only do FGDs allow researchers to collect information about more people in a given amount of time, FGDs also provide a unique and valid way to identify trends or conclusions. With respect to counterfactuals or external trends, for example, they can help solidify whether a drought or poor project performance caused a decrease in yields. They also allow for the identification of interesting outliers particularly useful in understanding project performance. For example, FGD participants may identify a farmer who was particularly successful under the project. The discussion can then be steered to why that farmer has had so much success, with both the farmer and other farmers offering their explanations. Then the other farmers could be led to discuss reasons they think they were not able to have as much success under the project. The research team will use checklists to ensure key information is captured from each FGD. FGD Sampling plan: Cocoa farmer groups will be identified in consultation with the ACI COP and staff as well as Partner staff responsible for management of sub-grants, taking into consideration the limitations on LOE and travel. The groups will include a mix of farmer owners, renter/sharecroppers, and laborers. The final selection will done by the evaluation team to minimize potential bias from WCF or partners C. DATA ANALYSIS METHODS In analyzing KII results, the evaluation team will sort interviews by respondent type and then first analyze responses given to the same questions on the KII question checklists. In analyzing these responses the evaluation team will look for both similarities and divergences in responses, noting characteristics of the key informant that might contribute to these similarities and differences. Responses to unscripted 92 questions will be analyzed similarly, with researchers looking for overlap in responses as well as where unique responses contribute to new understanding of the evaluation questions. FGD results will be analyzed similarly. We will analyze responses disaggregated by sex and socio economic categories to determine how the project affected women and other groups differently than other beneficiaries. This will also allow the evaluation team to what has worked well and what could have been improved in ensuring project impacts equitably reach all participants. Data analysis will employ a parallel, mixed-data approach in which quantitative data is independently analyzed from qualitative data. This approach takes the findings and analysis from the qualitative data and uses it to inform and explain findings from the quantitative data, and vice versa. For example, SI will analyze and code the qualitative data obtained from the KIIs and FGDs using a separate approach to identify response categories and patterns, and elucidate emergent themes, contextual factors, and trends. The team will then seek to expand on the findings that emerge from the quantitative data by identifying qualitative trends to provide explanations and depth for those results. Alternatively, SI will take the themes and trends emerging from the qualitative data and (a) look for existing quantitative results or (b) undertake additional quantitative analysis so as to determine whether there is evidence to generalize the qualitative findings to the target populations. Table 4, below, shows the evidence needed, the data sources, and the analysis methods for each evaluation question. Table 4: Evaluation Matrix Evaluation Questions Evidence needed Data Sources and Collection Methods Data Analysis Methods 1. How did the ACI activities contribute or likely contribute to the objectives and results as intended for the project overall and its four components?  Progress of performance indicators against targets  Perception of beneficiaries and stakeholders  Evidence mitigating factors that could explain discrepancies in performance  Review of project performance reports and data  KIIs and FGDs for corroboration, complimentary information and mitigating factors  Desk research for mitigating factors and possible counterfactual scenarios  Percentage of progress against targets  Qualitative description based on KII and FGD  Description of possible mitigating factors and rough counterfactual approximation 2. How likely are the ACI activities to positively (or negatively) affect food security among cocoa farmers and within cocoa-producing communities?  Literature review on the impact of cocoa farming on food security  Project design documents on the impact of cocoa farming on food security  Perceptions of food security of cocoa farmers and other community level respondents  Responses to indirect questions suggesting  Review of the literature on cocoa farming (and similar cash crops if necessary), and food security  Desk review of project documentation  KIIs and FGDs  Qualitative literature review synthesis  Comparison of literature review findings with project documentation and respondent perceptions of food security 93 changes in food security 3. What other benefits or drawbacks (i.e., unanticipated outcomes or spillover effects) occurred as a result of the program for each component?  Multiple stakeholders attribute unanticipated changes to project giving plausible explanations  KIIs and FGDs; if unanticipated outcomes start to emerge, the research team will pursue them through subsequent KIIs and FGDs  Identification of multiple reports of unanticipated outcomes  Analysis of plausibility of attribution to project 4. What are the most important lessons learned from implementing ACI that would inform similar efforts?  Trends emerge in what stakeholders say worked and what did not work, as supported by other sources of evaluation data  Judgment of researchers  KIIs and FGDs explicitly asking for strengths and weakness of project  All data once it has been synthesized and analyzed for the report  Identification of common insights about project performance in the views of stakeholders  Opinion of researchers after conducting research and performing analysis of project performance 5. How might the project or successor initiative be designed to more cost effectively and sustainably achieve and measure objectives, results, and outcomes?  Certain components of the project were more cost effective than others in achieving goals and outcomes  Project budget  Project performance reports and data  Results to questions 1 and 2  Cost effectiveness of different components of the project and their contribution to project goals and outcomes  Opinion of researchers after conducting research and performing analysis of project performance D. METHODOLOGICAL STRENGTHS AND LIMITATIONS The items listed below represent limitations to this performance evaluation as well as aspects of the proposed methodology that may ameliorate them. LOE: The level of effort (LOE) devoted to this performance evaluation is limited, considering the complexity of the project design, large number of partners and sub-partners, and activities in four countries. This low LOE introduces the possibility that findings, conclusions, and recommendations from the evaluation may miss key regional or national variations. Child Labor: This is a performance evaluation of ACI and focused on household level food security. BFS staff has requested that the evaluation also address child labor issues, although the evaluation team proposed by Social Impact and approved by USAID are not social scientists and do not have experience in analyzing and interpreting child labor issues. The evaluation team proposes to address this issue by an evaluation of ACI’s own data on social impact indicators dealing with children, enriched by interviews with WCF staff. If aspects of child labor arise during our data collection, we will note them as part of our work. If this occurs, we will prepare a separate memo with our observations, and any suggestions for considering this issue in the design of future cocoa activities if appropriate. This evaluation, therefore, will help BFS clarify its approach to examining child labor issues under the ACI program, but is not a 94 substitute for child-labor focused study undertaken by social scientists with experience in child labor issues in the cocoa industry. Attribution: Another limitation is the limited ability to attribute changes in key indicators as a result ACI activities. Attribution requires the precise identification of the counterfactual – i.e. what would have happened to project participants over the project period if they did not participate in the project. The gold standard for evaluations is the randomized experiment which is not possible in this evaluation. Neither are the next best solutions – quasi-experiments – which use statistical methods to estimate the precise counterfactual. Nevertheless, the evaluation team will provide an estimate of what might have happened without the project through qualitative research, research on country-level trends, and possibly interviews with cocoa stakeholders not directly benefitting from the project. This will give an indication of how project outcomes might have been affected by factors external to the project. Bias: Several types of bias might also present problems: selection bias, recall bias and response bias. Selection bias occurs when researchers interview people who are more convenient to reach or who are on friendlier terms with the project. These factors – ease of access and friendliness with project – might be correlated with project outcomes introducing bias. Recall bias is the natural error in memory that occurs when respondents are asked about events that took place in the past. Response bias can occur when respondents give particular responses that are untrue. This may be because they want to please the interviewer or increase their chance to receive future funding. All three biases can be addressed to an extent through triangulation – getting the same information from multiple sources. To the extent practical, the evaluation team will take steps to minimize selection bias in arranging KIIs. The interpretation of results will be sensitive to the inevitable low levels of bias that may remain. IV. EVALUATION PRODUCTS A. DELIVERABLES For the preparation and finalization of this evaluation, the SI evaluation team will produce the sub￾deliverables42 listed in the contract and summarized in Table 5. (See the Work Plan in Annex 3 for more detail on the timing of individual tasks.) SI will also provide trip scopes of work for each of the team members. Table 5: ACI Evaluation Contract Sub-Deliverables Documented official approval from relevant host country institutions to conduct the data collection, if required (Sub-deliverable 2) On or about September 14 (if required) Exit briefing to Mission in CDI (Sub-deliverable 8a1) On or about October 5 (depending on Mission staff availability) Exit briefing to West Africa Regional and Ghana Mission in Ghana (Sub-deliverable 8a2) On or about October 18 (depending on Mission staff availability) International Travel Debrief (Sub-deliverable 3) October 25 (within one week of departure) Capacity Strengthening Report (Sub-deliverable 7) October 30 (within one week of return from travel) Up to two rounds of draft PE reports, as required based on USAID feedback (Sub-deliverable 4) First draft – November 13, 2015 Second draft – December 11, 2015 Final PE report on evaluation findings (Sub-deliverable 5) December 30, 2015 42 This evaluation is a deliverable under the BFS Performance Evaluations Task Order, which includes four evaluations. Under the contract, the ACI evaluation itself is a deliverable, and the specific items required to be submitted to USAID for this deliverable are referred to in the contract as “sub-deliverables.” 95 In-person presentation to BFS on the PE final report (Sub-deliverable 8b) TBD (Date to be determined in collaboration with BFS) Note that contract Sub-deliverable 6 “Quantitative data sets in CSV and other formats without personally identifiably information and with accompanying documentation as described in the Open Data policy” is not required for this evaluation as quantitative data is not being collected. B. REPORTING GUIDELINES To ensure that the quality of all evaluation reports produced during the evaluation meet USAID quality standards, SI will comply with all USAID reporting criteria (including each of the criteria listed in Appendix I of the USAID Evaluation Policy) as follows:  SI commits to ensuring that all evaluation reports will represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the activity, what did not, and why.  SI commits to ensuring that all evaluation reports will address all evaluation questions included in the Scope of Work.  SI commits to ensuring that all evaluation reports will include the Scope of Work as an Annex. All modifications to the Scope of Work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer.  SI commits to ensuring that it will explain its evaluation methodology in de.tail in all evaluation reports and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.  SI commits to ensuring that findings in all evaluation reports will assess outcomes and impact on males and females.  SI commits that it will disclose the limitations to the evaluation in all evaluation reports, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).  SI commits to ensuring that findings in all evaluation reports will be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings will be specific, concise and supported by strong quantitative or qualitative evidence.  SI commits to ensuring that the sources of information in all evaluation reports will be properly identified and listed in an Annex.  SI commits to ensuring that recommendations in all evaluation reports will be supported by a specific set of findings.  SI commits to ensuring that recommendations in all evaluation reports will be action-oriented, practical and specific, with defined responsibility for the action. Additionally, as in the footnote to the proposed Roles and Responsibilities (see Annex 1), the SI evaluation will provide an unambiguous disclaimer that any suggestions from the Implementing Partners must be researched, investigated, and corroborated as objective before they are incorporated in the findings, as Implementing Partners have a stake in the outcome of the evaluation and could provide suggestions that would bias the findings. If the SI team makes a change to the findings on the basis of IPs input, then the SI team will provide the initial finding, and clearly document the change made, as well as efforts taken to investigate the IP's suggestions, corroborate their validity, and justify their inclusion to ensure findings are not affected by the potential subjectivity of the IP’s input. The elements of these criteria in Appendix I of the USAID Evaluation Policy are also incorporated and expanded upon in the Social Impact Evaluation Review checklist that the evaluation team will apply 96 before submitting the final evaluation report. This checklist, which is based on USAID’s C:19 Summary Checklist for Assessing USAID Evaluation Reports,43 is appended as Annex 2. V. TEAM COMPOSITION The primary evaluation team is composed of Team Leader Donald Greenberg, and Evaluation Specialist, Christopher Root. Mr. Greenberg brings to this evaluation over 25 years of expertise in international development with a focus on value chain and agribusiness. He has served as an independent consultant in a Team Leader role on multiple agribusiness, market facilitation, and rural development projects and evaluations worldwide and is well familiar with USAID funded projects and procedures. Mr. Greenberg will have the overall responsibility for ensuring the technical quality of the evaluation design, fieldwork in Ghana and Côte d’Ivoire, data analysis and production of high quality evaluation deliverables including the final evaluation report. Mr. Greenberg will be accompanied and supported by Evaluation Specialist, Christopher Root throughout all phases of this evaluation, including fieldwork in Ghana and Côte d’Ivoire. Mr. Root is an agricultural economist with over 10 years of international development experience focused on research and evaluation design, including performance monitoring and impact evaluations. His expertise is in value chain assessment with an emphasis on cash crops including the cocoa sector. The evaluation team will be supported by Dennis Wood (Social Impact’s Vice President for Evaluation Services and Technical Manager on this Task Order) and Dennis Marotta (Deputy Director and Senior Technical Advisor on this Task Order) who will provide technical input and quality assurance for the design and implementation as well as for reporting findings, conclusions and recommendations. Project Manager, Isadora de Latour, will provide evaluation management support as well as review of technical deliverables to ensure USAID compliance and approval. Program Assistant, Miguel Albornoz, will provide coordination and logistical support as well as review and copy editing support on final deliverables. SI will also contract a local interpreter and logistician to facilitate work in Côte d’Ivoire. VI. EVALUATION MANAGEMENT A. LOGISTICS Social Impact will be responsible for the logistical arrangements of the evaluation. SI home office staff will book flights and hotels, support arrangement of ground transportation for work in the field, assist in setting up meetings, and support communications and check-ins with the evaluation team while they are traveling. While in Côte d’Ivoire, the evaluation team will be assisted by a local interpreter and logistician who will travel within and outside Abidjan with the team, interpret and translate interviews in French and English, and assist with scheduling appointments and other logistical matters as needed. While in Ghana the evaluation team will be supported by the SI home office to confirm meetings and ensure that all evaluation tasks are completed appropriately and on time. The Social Impact home office support staff will provide other logistical support as needed throughout the conduct of the evaluation, including providing editing and formatting for deliverables, maintaining the list of persons interviewed and documents consulted and liaising with BFS. While Social Impact will minimize logistics requests to USAID/Côte d’Ivoire and USAID/Ghana staff, Social Impact will consult with Mission staff on issues such as security and approved air carriers for flights to the field. In some cases, SI may also request that BFS or Mission staff make introductions to partner or sub-partner staff or encourage them to meet with the evaluation team. 43 USID, C-19: Summary Checklist for Assessing USAID Evaluation Reports. http://usaidlearninglab.org/sites/default/files/resource/files/mod11_summary_checklist_for_assessing_usaid_evaluation_reports.pdf. Accessed April 26, 2015. 97 B. SCHEDULING Table 6: Evaluation Phases, Timeframe, Activities and Sub-Deliverables Phase Timeframe Activities Sub-Deliverables Inception Phase August – September 2015  Desk review  Team planning  Document preparation  Draft instrument design 1. Performance Evaluation design plan 2. Documented official approval from relevant host country institutions to conduct the data collection, if required Fieldwork in Ghana and Côte d’Ivoire September - October 2015  In-brief (in Ghana and Côte d’Ivoire)  Fieldwork  Preliminary analysis  Out-briefing 8(1). Exit briefing in Ghana 8(2). Exit briefing in Côte d’Ivoire Analysis & Reporting October – December 2015  Data analysis  Prepare travel debrief  Prepare capacity strengthening report  Prepare draft Performance Evaluation report  Prepare second draft PE Report  Prepare final evaluation report  Prepare final briefing 3. Travel debrief 4(1) Draft Performance Evaluation Report 4(2) Second draft PE Report 5. Final PE Report 6. Quantitative data sets 7. Capacity strengthening report 8(1) Preliminary phone or in-person briefing with BFS staff on draft Performance Evaluation findings 8(2) In-person presentation to BFS on PE final report C. LOE Table 7. LOE (Level of Effort in Days) Name (Role) Evaluators (LOE days) HQ support (LOE days) Donald Greenberg (Team Leader) 40 Christopher Root (Evaluation Specialist) 36 Dennis Wood (Senior Technical Manager) 3 Dennis Marotta (Senior Technical Advisor) 3 Isadora de Latour (Project Manager) 8.5 Miguel Albornoz (Program Assistant) 8.5 TOTAL LOE* 76 23 *LOE: Level of effort (“per days”) 98 Annex II: Data Collection Instruments Interview Guide List by Stakeholder Type44 I. Implementing Partner (Evaluation Questions 1, 3, 4, 5) i. World Cocoa Foundation II. Sub-grantee recipients (Evaluation Questions 1-545) i. CRIG Flavor Lab ii. CropLife iii. TechnoServe III. Research Organizations (Evaluation Questions 1, 3, 4, 5) i. National Agricultural Research System (NARS) IV. National Platform Members (Evaluation Questions 1, 3, 4, 5) i. National Platform Participants ii. National Platform Implementers (Secretariat) V. Service Providers i. Extension Service Agents (Evaluation Questions 1 - 5) ii. Spray Service Providers (Evaluation Questions 1, 3, 4, 5) VI. Farmers and Laborers (Evaluation Questions 1 - 5) 44 This list is not exhaustive, but rather illustrative of the kinds of organizations that fit into each category 45 Evaluation question #2 is not relevant for some of the partners such as CropLife 99 I. Key Informant Semi-Structured Interview Guide: Implementing Partner (WCF) Interviewers provide introduction that covers the following points: • Brief introduction of the evaluation team members • Purpose of the evaluation and of the interview • Main topics of the of the interview • Confidentiality of responses • Request for and receipt of permission to interview Interview Code #: Topic: Firm/Organization: Position: Male or Female: Country: Date & time of interview: Name of primary interviewer: Name of primary note taker: Common Questions 1. Which ACI initiatives or activities were most successful? Why? 2. Which ACI initiatives or activities were least successful? Why? 3. Overall, what is your opinion of the effectiveness of ACI? 4. How could ACI have been improved? 5. Can you think of anything about ACI that was unexpected or surprised you, either negatively or positively? Component 1 6. How well have the National Platforms functioned? Elaborate. 7. To what degree if at all have the platforms led to improved dialogue and consensus among cocoa policy actors? 8. To what degree if at all have the platforms influenced policy direction? 9. How could the project have improved the National Platform specifically in regard to a. effectiveness and b. sustainability? What could the project have done better to build a more effective? Component 2 10. How well have ACI’s regional research activities to improve planting materials (including genetic fingerprinting, national seed and budwood gardens, and regional planting material propagation/distribution centers) performed? 11. How could the project have done better to increase the availability of improved planting and rehabilitation materials? 12. How could the project have done better to sustainably increase the availability of improved planting and rehabilitation materials? Component 3 13. Was the curriculum and training of extension officers appropriate for the needs of farmers and the goals of the project? Explain. 100 14. Do you think the partner extension agencies in each country will continue to implement the curriculum after the project is finished? Will they train new extension workers using it? 15. What could the project have done better to improve national level extension services for cocoa? 16. Why were the farmer business service centers discontinued? 17. What lessons have you learned from the Flavor Lab work? Component 4 18. What were the lessons you learned from the SSP program? What could have been done better? 19. Do you think that SSPs will continue providing spray services? Why or why not? 20. What about the credit program, what lessons have you learned from that? 21. How did the project select spray services and credit to prioritize? What were the criteria? In retrospect, were these good choices? 22. How did the soil testing/fertilizer recommendation and related fertilizer activities perform under ACI? What could have been improved? Overall project 23. Are there other ACI interventions or activities we have not discussed that you want to mention as important to the achievement to the project objectives? 24. Do you think the mix of components and activities were well suited to achieve the project objectives? Why or why not? 25. What different interventions might have led to more success? 26. In terms of implementation, what were the lessons learned there? 27. Are there any mitigating factors outside the project’s influence that might have influenced project performance that we should look into? II. Key Informant Semi-Structured Interview Guide: Partners - Sub grantee recipients Interviewers provide introduction that covers the following points: • Brief introduction of the evaluation team members • Purpose of the evaluation and of the interview • Main topics of the of the interview • Confidentiality of responses • Request for and receipt of permission to interview Interview Code #: Topic: Firm/Organization: Position: Male or Female: Country: Date & time of interview: Name of primary interviewer: Name of primary note taker: Common Questions 1. Which ACI initiatives or activities were most successful? Why? 2. Which ACI initiatives or activities were least successful? Why? 3. Overall, what is your opinion of the effectiveness of ACI? 101 4. How could ACI have been improved? 5. Can you think of anything about ACI that was unexpected or surprised you either negatively or positively? CRIG Flavor Lab Outcomes 1. To what extent are foreign buyers aware of the flavor profile reports from the Flavor Lab? Do they see value in them and have confidence in them? 2. How is this trust being established? 3. How are variety and post-harvest practice recommendations to maximize flavor disseminated to farmers? 4. How receptive are farmers to these recommendations? Have they implemented any of them? 5. What do you think the Flavor Lab could do to improve? Crop Life Outcomes and operational 1. What was the selection criteria for SSPs to be trained? 2. Was there an excess of qualified people who wanted to participate? 3. To what degree was the training consistent with the ability of the SSPs? 4. To what degree are you confident that SSPs take seriously the human and environmental risks of spraying? Why? 5. Is the certification a differentiator for SSPs? Are you aware of other SSPs operating without the certification? 6. What would you do differently to improve the SSP program? 7. Do you think the SSPs will continue after the program is over? Technoserve Outcomes and operational 1. Why isn’t there enough affordable credit for cocoa farmers? What are the constraints in the credit market for cocoa farmers that you have to overcome? 2. How is your credit program overcoming them? 3. How do you select credit partners? 4. How likely are the credit partners you worked with to expand access to affordable credit? 5. Do you think these credit programs you have established will be continued after the project? Why or why not? What have you learned from this experience about facilitating access to credit to cocoa farmers? What worked well? What did not work well? 102 III. Key Informant Semi-Structured Interview Guide: Research Organizations Interviewers provide introduction that covers the following points: • Brief introduction of the evaluation team members • Purpose of the evaluation and of the interview • Main topics of the of the interview • Confidentiality of responses • Request for and receipt of permission to interview Interview Code #: Topic: Firm/Organization: Position: Male or Female: Country: Date & time of interview: Name of primary interviewer: Name of primary note taker: Common Questions 1. 1. Which ACI initiatives or activities were most successful? Why? 2. 2. Which ACI initiatives or activities were least successful? Why? 3. 3. Overall, what is your opinion of the effectiveness of ACI? 4. 4. How could ACI have been improved? 5. 5. Can you think of anything about ACI that was unexpected or surprised you either negatively or positively? Outcomes 1. How would you characterize the quality of cocoa planting and rehabilitation materials introduced through WCF/ACI/IITA/ACBWG compared to previous planting material?? 2. To what extent are these improved planting and rehabilitation materials being adopted by farmers? 3. How do improved planting and rehabilitation materials reach farmers i.e. what is the distribution system? 4. What has been or do you think is likely to be the outcome of the DNA fingerprinting exercise undertaken through the project? 5. What has to happen to increase availability of improved planting and rehabilitation materials? 6. What has to happen to increase adoption of improved planting and rehabilitation materials? 7. What has to happen to sustain increased adoption of improved planting rehabilitation materials? IV. Key Informant Semi-Structured Interview Guide: National Platform Members Interviewers provide introduction that covers the following points: • Brief introduction of the evaluation team members • Purpose of the evaluation and of the interview • Main topics of the of the interview 103 • Confidentiality of responses • Request for and receipt of permission to interview Interview Code #: Topic: Firm/Organization: Position: Male or Female: Country: Date & time of interview: Name of primary interviewer: Name of primary note taker: Common Questions 1. Which ACI initiatives or activities were most successful? Why? 2. Which ACI initiatives or activities were least successful? Why? 3. Overall, what is your opinion of the effectiveness of ACI? 4. How could ACI have been improved? 5. Can you think of anything about ACI that was unexpected or surprised you either negatively or positively? Outcomes 1. How would you characterize national public-private coordination and planning in cocoa compared to 5 years ago? 2. How did coordination and planning in the sector take place before ACI? 3. Overall how has the National Platform performed in terms of improving coordination and planning and facilitating new investments and partnerships in the sector? Elaborate. 4. What in your opinion are the most important achievements of the National Platform? 5. What do you think will happen to the National Platform after the project is completed? Operational 6. How well has WCF done in forming and facilitating the National Platform? 7. Which ACI initiatives or activities were most successful in forming and facilitating strong National Platforms? Why? 8. Which ACI initiatives or activities were least successful in forming and facilitating strong National Platforms? Why? 9. How could the project have done better to help build a more effective National Platform? 10. How could the project have done better to help build a more sustainable National Platform? 104 VII. Key Informant Semi-Structured Interview Guide: Extension Service Providers and SSPs Interviewers provide introduction that covers the following points: • Brief introduction of the evaluation team members • Purpose of the evaluation and of the interview • Main topics of the of the interview • Confidentiality of responses • Request for and receipt of permission to interview Interview Code #: Topic: Firm/Organization: Position: Male or Female: Country: Date & time of interview: Name of primary interviewer: Name of primary note taker: Common Questions 1. Which parts of the ACI training and curriculum were most successful? Why? 2. Which parts of the ACI training and curriculum were least successful? Why? 3. Overall, what is your opinion of the effectiveness of the ACI training and curriculum? 4. How could the ACI training and curriculum have been improved? 5. Can you think of anything about ACI training and curriculum that was unexpected or surprised you either negatively or positively? Extension Service Providers Outcomes - Training 1. Does the curriculum you learned from WCF allow you to give better advice to farmers? Please elaborate. 2. What particular aspects of the curriculum are the most useful? Why? 3. Was the approach that each extension agent learns about all the different aspects of cocoa production useful? Why or why not? Outcomes – Farmers 4. Have you noticed any changes in farmer cocoa practices since you have worked with them using the new curriculum? If so, what changes? 5. In your experience, how do farmers respond when cocoa becomes more profitable? 6. Follow-up: Does it affect their food production? 7. Do cocoa farmers use the skills you train them on with other crops? Explain. 8. Roughly what percent of the farmers you work with regularly are women? 9. Are there any particular challenges in working with women farmers? If yes, how do you overcome them? 105 10. Are farmers receptive to your messages about the negative aspects of using child labor in cocoa farming? Explain please. 11. In your experience, do farmers use pesticides safely? Explain. Operational 12. How suitable do you think the curriculum is for the cocoa farmers you work with? 13. How well did the training and other support from ACI prepare you to implement the curriculum? Spray Service Providers Outcomes 1. How much net cash do you earn from spraying services in a year? 2. How many hours/days a week/month/year do you spend on spraying services 3. Approximately how many farmers have you sprayed for this year?___ 4. Who was typically applying spray in your clients’ household before you? (man, woman, children) 5. Are your clients typically wealthier or poorer farmers? 6. Are you confident that you are well trained in applying the spray safely? Please explain. 7. Do you have any problem finding farmers willing to pay for your service? 8. Do farmers trust that you know what you are doing because of your certification? 9. How long do you see yourself continuing as a spray service provider? VIII. Focus Group Discussion Guide: Farmers Interviewers provide introduction that covers the following points: • Brief introduction of the evaluation team members • Purpose of the evaluation and of the interview • Main topics of the of the interview • Confidentiality of responses • Request for and receipt of permission to interview Interview Code #: Topic: Ghana or Côte d’Ivoire (circle) FGD location/description: Total number in FGD: Total females in FGD: Date & time of interview: Name of primary interviewer: Name of primary note taker: Common Questions 1. Which parts of the SSP service or new extension services were most successful? Why? 2. Which parts of the SSP service or new extension services were least successful? Why? 3. Overall, what is your opinion of the effectiveness of the SSP service or new extension services? 4. How could the SSP service or new extension services have been improved? 5. Can you think of anything about SSP service or new extension services that was unexpected or surprised you either negatively or positively? 106 Outcomes – Extension 1. How many of you have received advice on cocoa production, marketing or business management from extension agents in the past few years? _______ 2. Who has provided the extension services you have used? 3. If you receive extension services, what information have you found to be the most useful? What did you do with the information, and which information have you put into practice? 4. To what degree has the extension information you have put into practice change productivity of your cocoa trees, quality of your cocoa, or prices you have received from cocoa? 5. What business concepts did you learn from extension? What have you put into practice and what impact has it had on your farming as a business? 6. Has anything you learned in your cocoa training changed your farming practices for your other crops? Elaborate. 7. What practices that you have learned from extension will you continue to put into practice and which will you discontinue? Please elaborate. Outcomes – Inputs 8. How many of you have used pesticide spray on your cocoa trees in the past few years? _____ 9. Where did you learn how to apply the spray? 10. Who usually applies the spray? If child, how old are they? Has this changed over the last few years because of anything you have learned from spraying service providers? 11. How many of you have used a spray service provider in the past few years? _____ 12. How did you like this service? Will you use it again? Why or why not? 13. Do you use fertilizer? How do you determine when, how much, and what types of fertilizer to use? Where do you access the fertilizer from? Outcomes – Food security 14. Why do you grow cocoa? 15. How do you view cocoa versus other farming or other economic activities in terms of return on land/money/time? Riskiness? Has this changed over time? 16. How has your income from cocoa changed over the last few years? What is responsible for that change? 17. How are different members of your family and hired laborers (including women and children) are involved in growing and harvesting your cocoa? Who contributes to the different tasks of growing cocoa (e.g., planting, pruning, harvesting, weeding, spraying, applying fertilizer)? 18. If cocoa became more profitable what would you do? 19. Is there enough food available/affordable in local markets year around? Where does this mostly come from? 20. Are you better able to provide food for your family because of cocoa, or would you be better off using more of your land/time for food or other cash crops. 21. What are your plans for growing cocoa vs other crops and activities over the next 5 years? Focus Group Discussion Guide: Laborers Interviewers provide introduction that covers the following points: • Brief introduction of the evaluation team members • Purpose of the evaluation and of the interview • Main topics of the of the interview • Confidentiality of responses • Request for and receipt of permission to interview Interview Code #: Topic: Ghana or Côte d’Ivoire (circle) 107 FGD location/description: Total number in FGD: Total females in FGD: Date & time of interview: Name of primary interviewer: Name of primary note taker: Common Questions 1. Have any of the farms you worked on recently used spray service providers, received credit for cocoa or received training on cocoa from extension officers? Continue only if yes 2. Which parts of the SSP service or new extension services were most successful? Why? 3. Which parts of the SSP service or new extension services were least successful? Why? 4. Overall, what is your opinion of the effectiveness of the SSP service or new extension services? 5. How could the SSP service or new extension services have been improved? 6. Can you think of anything about SSP service or new extension services that was unexpected or surprised you either negatively or positively? Outcomes 7. Approximately how much money did you make from laboring on cocoa farms last year? How has this changed over the last few years? 8. Approximately what percentage of your wages came from cocoa farming last year? 9. Does cocoa work pay better, the same, or worse than your other agricultural work? 10. What about compared to working on food crops? Approximately what is the difference in daily rate? 11. How does laboring on cocoa farms affect your income and where does it fit with your other economic activities? 12. How did you learn how to do cocoa work? 13. Do you apply pesticide to cocoa trees? 14. Have you received any training or information on how to safely apply it? 15. Generally would you happy if there was more cocoa work available? Why or why not? 108 Annex III: Sources of Information Achterbosch, Thom, Siemen van Berkum and Gerdien Meijerink. “Cash crops and food security: Contributions to income, livelihood risk and agricultural innovation.” Dutch Ministry of Economic Affairs. Report by LEI Wageningen UR. April 2014. “Analyse régionale de la situation de l’insécurité alimentaire aiguë – Situation courante.” Cadre Harmonise. 2014. Asamoah, Mercy et. al. “Insight into the Standard of Living of Ghanaian Cocoa Farmers.” Greener Journal of Agricultural Sciences, Volume III. May 2013. Benjamin, Swayne and Angus Deaton. “Household Welfare and the Pricing of Cocoa and Coffee in Côte d’Ivoire: Lessons from the Living Standards Surveys.” The World Bank Economic Review, Vol. 7, No. 3. 1993. Budiansky, Ethan and Holly Houston. “Global Development Final Report.” Bill & Melinda Gates Foundation. March 2014. Burex-CI. “Renforcement des capacities techniques de l’ANADER en matiere de cacaoculture durable.” December 2014. Central Intelligence Agency. “The World Factbook: Ghana.” 2015. Central Intelligence Agency. “The World Factbook: Ghana.” 2015. “Climate Change Knowledge Portal.” The World Bank Group. 2015. “Côte d’Ivoire 2013: Besoins Humanitaires en phase de transition.” Besoins Humanitaires. September 2012. “Côte d’Ivoire Dashboard: Natural Hazards.” The World Bank Group. 2015. CropLife Africa Middle East. “Market-Driven Farm Input Services: End-of-Project Report.” World Cocoa Foundation. March 2015. CropLife Africa Middle East. “Market-Driven Farm Input Services: Quarterly Progress Report.” World Cocoa Foundation. April 2013. CropLife Africa Middle East. “Market-Driven Farm Input Services: Quarterly Progress Report.” World Cocoa Foundation. July 2013. CropLife Africa Middle East. “Market-Driven Farm Input Services: Quarterly Progress Report.” World Cocoa Foundation. September 2013. CropLife Africa Middle East. “Market-Driven Farm Input Services: Quarterly Progress Report.” World Cocoa Foundation. December 2013. Désiré, Pokou N. and Tahi Mathias. “Enhancing the quality of cocoa planting materials and its accessibility to farmers in West Africa.” Centre National de Recherche Agronomique. September 2014. Diegert, Paul, Gwynne Zodrow, Leona Ba, and Fion de Vletter. “Cocoa Livelihoods Program: Phase I Evaluation/Phase II Baseline.” World Cocoa Foundation. Report by Management Systems International. April 2014. 109 Dohmen, Manon Mireille, Willem Adriaan van den Andel and Okechukwu Felix Nwoche. “Baseline study for Africa Cocoa Initiative: Cameroon.” CropLife. February 2014. Dohmen, Manon Mireille, Willem Adriaan van den Andel and Okechukwu Felix Nwoche. “Baseline study for Africa Cocoa Initiative: Ghana.” CropLife. December 2013. Dohmen, Manon Mireille, Willem Adriaan van den Andel and Okechukwu Felix Nwoche. “Baseline study for Africa Cocoa Initiative: Ivory Coast.” CropLife. February 2014. Dohmen, Manon Mireille, Willem Adriaan van den Andel and Okechukwu Felix Nwoche. “Baseline study for Africa Cocoa Initiative: Nigeria.” CropLife. November 2013. Dongo, Lelia N. “Training of Extension Agents in Oyo and Kogi States.” World Cocoa Foundation. Report by Matrop & Co. NIG Ltd. 2015. Ebai, Sona. “Outcome Assessment Report.” World Cocoa Foundation. April 2014. “Food Security and Humanitarian Implications in West Africa and the Sahel.” World Food Programme. April 2014. Fountain, A.C. and Friedel Hütz-Adams. “Cocoa Barometer 2015.” VOICE Network. 2015. “Global Weather Hazards Summary.” FEWS Net. October 2015. Hardman & Co. “Giant On A Pinhead: A Profile of the Cocoa Sector.” May 2014. Hjelm, Lisa and Wuni Dasori. “Comprehensive Food Security & Vulnerability Analysis: Ghana 2012.” World Food Programme. 2012. Ibrahim, Umaru, Serge Eric Yakeu and Leonard Garden. “An Evaluation of the Input Credit System As Implemented by TechnoServe In Côte d’Ivoire and Nigeria For the African Cocoa Initiative.” April 2015. International Institute of Tropical Agriculture. “Bi-annual Report of ‘African Cocoa Initiative’ Project (May 2013 – October 2013).” World Cocoa Foundation. November 2013. International Institute of Tropical Agriculture. “Bi-annual Report of ‘African Cocoa Initiative’ Project (November 2013 – April 2014).” World Cocoa Foundation. April 2014. International Institute of Tropical Agriculture. “Quarterly Report of ‘African Cocoa Initiative’ Project.” World Cocoa Foundation. November 2012. International Institute of Tropical Agriculture. “Quarterly Report of ‘African Cocoa Initiative’ Project (August 2013 – October 2013).” World Cocoa Foundation. November 2013. International Institute of Tropical Agriculture. “Technical report of the ‘African Cocoa Initiative’ Project (September 2014 – December 2014).” World Cocoa Foundation. December 2014. International Institute of Tropical Agriculture. “Technical report of the ‘African Cocoa Initiative’ Project (January 2015 – March 2015).” World Cocoa Foundation. April 2015. International Institute of Tropical Agriculture. “Technical report of the ‘African Cocoa Initiative’ Project (May 2014 – August 2014).” World Cocoa Foundation. September 2015. “IPC support to West Africa.” IPC Brief. September 2013. Kolavalli, Shashi and Marcella Vigneri. “Cocoa in Ghana: Shaping the Success of an Economy.” World Bank. 2011. 110 Larson, Bruce et. al. “Household Health and Cocoa Production: A Baseline Survey of Smallholder Farming Households in Western Region, Ghana.” Center for International Health and Development. Boston University School of Public Health. November 2005. Le Conseil du Café-Cacao. “Manuel technique de cacaoculture durable: a l’attention du technicien.” February 2015. Le Conseil du Café-Cacao. “MManuel du planteur de cacao.” February 2015. LMC International. “Cocoa Comparative Household Economy Study: West Africa Smallholder Cocoa Farmers.” World Cocoa Foundation. February 2014. Marks, Jackie. “’Farmer-owned’ seed gardens take shape in Cameroon.” World Cocoa Foundation.2015. Marks, Jackie. “Cocoa Extension in Ghana Receives Boost with New Training.” World Cocoa Foundation. 2015. Marks, Jackie. “Fostering Innovation to Deliver Input Credit to Nigerian Cocoa Farmers.” World Cocoa Foundation. 2015. Marks, Jackie. “Major Boost to Cocoa Extension in Côte d’Ivoire with New Training.” World Cocoa Foundation. 2015. Marks, Jackie. “SSPs: The Game-Changing Rural Entrepreneurs.” World Cocoa Foundation. 2015. Mathias, Tahi G. et. al. “Intensification de la distribution et de l’utilisation de materiel vegetal performant de cacao.” Centre National de Recherche Agronomique. November 2013. Mathias, Tahi G., et. al. “Verification de la conformite des clones, intensification de la distribution et de l’utilisation de materiel vegetal performant de cacao.” Centre National de Recherche Agronomique. April 2014. Padi, Francis Kwame. “Increasing the capacity of the Seed Production Unit of Ghana Cocoa Board for production of cocoa seed pods.” Cocoa Research Institute of Ghana. April 2015. “Predicting the Impact of Climate change on the Cocoa-Growing Regions in Ghana and Côte d’Ivoire.” Climate Change Agriculture and Food Security. September 2011. Rademacher-Schulz, Christina and Edward Salifu Mahama. “Rainfall, Food Security and Human Mobility Case Study: Ghana.” United Nations University, No. 3. November 2012. “Supply Chain Risk Assessment: Cocoa in Ghana.” The World Bank. 2011. TechnoServe. “African Cocoa Initiative Biannual Narrative Report.” January 2014. TechnoServe. “African Cocoa Initiative Final Narrative Report.” April 2015. TechnoServe. “African Cocoa Initiative Quarterly Narrative Report, Q2.” 2013. TechnoServe. “African Cocoa Initiative Semiannual Narrative Report.” March 2013. Terazono, Emiko. “Plantation groups eye cocoa farming.” Financial Times. June 2014. The African Cocoa Breeders Working Group. “Enhancing farmers’ adoption of high quality cocoa planting materials and good agricultural practices in West and Central Africa: First Annual Report.” World Cocoa Foundation. October 2014. 111 The African Cocoa Breeders Working Group. “Enhancing farmers’ adoption of high quality cocoa planting materials and good agricultural practices in West and Central Africa: Third Quarterly Report.” April 2014. Tulane University. “Survey Research on Child Labor in West African Cocoa Growing Areas.” School of Public Health and Tropical Medicine. July 2015. Tulane University. “Two-Page Summary of Research Findings.” July 2015. UNDP. “Support for Development and Operation of COCOBOD’s Ghana Cocoa Platform.” 2012. Vigneri, Marcella. “Drivers of cocoa production growth in Ghana.” ODI Project Briefing No. 4. December 2007. Wiggins, Steve, Giles Henly and Sharada Keats. “Competitive or complementary? Industrial crops and food security in sub-Saharan Africa. ODI.April 2015. World Cocoa Foundation. “Annual Report October 2012 – September 2013.” October 2013. World Cocoa Foundation. “Annual Report October 2013 – September 2014.” October 2014. World Cocoa Foundation. “CocoaAction At A Glance.” February 2015. World Cocoa Foundation. “Ghana Cocoa Extension Agents’ Training Reports. 2014. World Cocoa Foundation. “Semi-Annual Report October 2014 – March 2015.” April 2015. World Cocoa Foundation. “Symposium of the Next Generation of Cocoa Research for the West and Central African Cocoa Region.” World Cocoa Foundation. “True-to-type DNA fingerprinting: Implications towards Cocoa and Chocolate economy and sustainability.” July 2015. World Cocoa Foundation. “WCF/ACI Seed Garden Site Inspection Report.” 2014. 112 Annex IV: ACI Interviewees Key Informant Interviews (KIIs) - Ghana Organization Location Ghana Cocoa Board Accra Seed Production Division Accra Cocoa Marking Company (GH.) LTD. Accra Quality Control Company LTD. Accra CropLife Accra Transroyal (Gh.) Ltd. Accra Tcho Accra ECOM Ghana Kumasi None Mofra None Mofra 10 Kookoo Pa Farmers Assocation Offinso Transroyal Offinso SPU Bunsu Cocoa Research Institute of Ghana New Tafo￾Akim Cocoa Research Institute of Ghana New Tafo￾Akim Cocoa Research Institute of Ghana New Tafo￾Akim Hershey Accra Weinco Ghana Ltd. Accra WCF Accra WCF Accra 10 WCF Accra WCF Accra USAID/Ghana Accra Total 23 (3) Females 113 Focus Group Discussions (FGDs) – Ghana Type # participants (females) Location Extension management 14 (2) Accra Farmers 5 (1) Mofra Farmers (Fine Flavor Lab) 21 (6) Ofinso Farmers 18 (8) Ofinso Extension agents 10 (1) Ofinso Farmers 24 (3) Bunsu Extension agents 5 (2) Bunsu Key Informant Interviews (KIIs) - Cote D’Ivoire Organization Location World Cocoa Foundation Abidjan World Cocoa Foundation Abidjan World Cocoa Foundation Abidjan Technoserve Abidjan Advans Côte d’Ivoire Abidjan Center National De Researche Agronomique (CNRA) Abidjan Center National De Researche Agronomique (CNRA) Abidjan CNRA Abidjan CNRA Abidjan CropLife Abidjan 10 CropLife Abidjan BASF Abidjan NA Ble NA Ble NA Ble Coop-CA Le Roches Sinfra Coop-CA Le Roches Sinfra ANANDAR Yamoussoukro ANANDAR Yamoussoukro ANANDAR Divo 10 Societe Cooperative Simplifie Binkady d’Ake Douanier (SCOOPSBAD), Advans partner Abgoville 114 Le Conseil du Café-Cacao Abidjan Le Conseil du Café-Cacao Abidjan BUREX Abidjan IDH Abidjan Olam Abidjan World Cocoa Producers Organization Abidjan USAID/Côte d’Ivoire Abidjan Department of State Abidjan WCF Yamoussoukro 10 WCF Abidjan Total 31 (7) females Focus Group Discussions (FGDs) – Côte d’Ivoire Type # participants (females) Location Farmers and SSPs 60 (2) Ble Extension agents 8 (3) Divo SSPs 11 (0) Champo Farmers 19 (0) Champo Key Informant Interviews – Phone and San Francisco Organization Location Barry Callebaut Phone Mars Phone Hershey Phone Mondalez Phone CropLife Africa/Middle East Phone Tcho Chocolate San Francisco Guittard Chocolate San Francisco Total 7 (2) Females 115 Annex V: ACI Budget Overall, programmatic commitments are approximately $1.8 million less than planned, a shortfall of 23%. However, within this overall shortfall, the relative funding of ACI components shifted from what was anticipated, as reflected in Table 18 below.46 There are three major reasons for the slower than anticipated commitments:  WCF’s successful cost-leveraging with IDH (whose support is not completely reflected in ACI budget reports) and with Solidaridad.  The delayed start of many activities,  Opportunities to build on the initial successes SSP and credit pilots were not exploited. The commitments for Component One is about half of what was anticipated. This in part reflects the considerable challenges of working with the PPP platforms in a dynamic and political context, but it also reflects potential missed opportunities to faciliate evidence-based policy advocacy. Component Two commitments are virtually the same as anticipated, but given the overall drop in funding, its relative share of project budget climbed to nearly 40%. This would seem reasonable, given the importance of Component Two’s fingerprinting and seed/bud garden activities. Component Three commitments were $1.1 million less than anticipated, and if not for the unexpected Ghana CHED training, and the GQI activity (which has many aspects beyond extension and farmer training services) it would been an even greater drop in emphasis. Component Four commitments increased reflected in the large grants to TechnoServe and CropLife. The efficiency of the pilots, particularly the TechnoServe pilot has been questioned; in the evaluation team’s view the potential impacts are well worth the cost. However, TechnoServe and CropLife were certainly effective n proving a concept. Indeed, there were many opportunities for ACI with additional spending to have leveraged these successes with follow-on activities that were missed. These should be considered for the follow-on project. Table 18. WCF/ACI Budget: Original versus Committed Component Originally Earmarked (US $ 000) Original Emphasis Commitments as of Sep 15 Actual Emphasis Reduction from originally Earmarked One --- Platforms 950 12% 425 7% -525 Two – Planting Material 2466 30% 2425 39% -41 Three -- Extension 2866 35% 1756 28% -1100 Four – Input and Credit 1805 22% 1617 26% -188 Total 8087 6255 -1832 Source: WCF/ACI. Note: Does not include the IDH component, which is not reported by WCF to USAID. 46 While this table shows commitments as of September 2015, it is unlikely that ACI will make significant additional commitments before project wind-down in June 2016. 116 Annex VI: Disclosure of any conflicts of interest The evaluation Team Leader, Donald Greenberg, and Evaluation Specialist, Christopher Root, disclosed that they had no potential conflicts of interest to conduct this evaluation. The team also signed a Non￾Disclosure Agreement upon beginning work on the evaluation. A record of the COI and NDA forms are saved in the project folder and available upon request. U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523