MozaCajú End of Project Evaluation Report A report to TechnoServe evaluating the results of MozaCajú, a USDA project. Award Number TNS FCC-656-2013-038-00 The Project: MozaCajú was a four-year grant (including no-cost extension) funded by the United States Department of Agriculture (USDA), implemented jointly by TechnoServe (TNS) as the principal grantee, and the Aga Khan Foundation-Mozambique (AKF) as sub-grantee. The project supported small cashew producers, processors, small-service providers and foreign buyers by addressing issues relating to production, innovation, input supply, processing, and marketing. These activities, affecting over 23,500 poor farming households, aligned with poverty reduction efforts by the Government of Mozambique Acronyms AKF Aga Khan Foundation BRC British Retail Consortium (standard) CNSL Cashew Nut Shell Liquid FFS Farmer Field School FTE Full-time equivalent (jobs) GAPs Good Agricultural Practices HACCP Hazard Analysis and Critical Control Points (standard) ICT Information and Communication Technology INCAJU Institute of Cashew Promotion, Mozambique MT Metric Tonnes MTE Midterm Evaluation MZN Mozambican Meticais (also ‘Mts’) RCN Raw Cashew Nut SME Small and Medium Enterprise SDAE District Service for Economic Activities (principally agriculture) SPEED+ Supporting the Policy Environment for Economic Development TNS TechnoServe USDA United States Department of Agriculture Acknowledgements The project team and its evaluator acknowledge the generous and constant support of USDA in developing, implementing and overseeing the highly ambitious MozaCajú project to revitalize the leading export crop in Mozambique. The author of this report specifically thanks Will Rabieh of USDA for his design inputs and review of this evaluation. The author of this report thanks all the interviewees for their engagement and willingness to share their experience and ideas. Special appreciation goes to the TechnoServe and Aga Khan Foundation professional staff engaged jointly in MozaCajú and led by Project Director Daria Gage, for their detailed, frank and helpful insights, and to the administrative staff for their help with logistics. The author also acknowledges the collaboration of the farmers, processor managers and staff, grant recipients, international buyers, and the officers and staff from INCAJU and SDAE offices in the three provinces who gave their time to be interviewed and provide access to further information. As a result, this evaluation was consistently conducted in an independent, open and objective style. In particular, the author acknowledges the extended assistance provide by MozaCajú M&E Assistant Clerico Constantino and Graham Sherbut, Director of M&E (AKF), for their sustained assistance in the Endline as well as the MTE; Tinashe Nyahwedegwe, Regional M&E Advisor for Southern Africa for quantitative support, Denny Newhouse of TechnoServe for providing generous logistical and translation support for the Endline interviews; and James Tinker, Manager of TechnoServe Corporate Measurement, for verification review discussions of the major quantitative findings. This evaluation was conducted as an independent study and is certified as such by the author; all opinions expressed and any omissions herein are solely those of the author. (Cover photo, lower: MozaCajú Field Promoters in a training session) Contents Background: Relationship of this Report to Midterm Evaluation........................................................... 1 1 Executive Summary......................................................................................................................... 2 1.1 Design of Endline Assessment ................................................................................................ 2 1.2 Lessons Learned ...................................................................................................................... 4 2 Project Background......................................................................................................................... 6 2.1 Global overview and history ................................................................................................... 6 2.2 Local economic context .......................................................................................................... 7 3 MozaCajú Objectives and Strategy ................................................................................................. 8 3.1 Project scope......................................................................................................................... 10 3.2 Endline Evaluation Aims and Design ..................................................................................... 11 3.2.1 Project Management Context........................................................................................... 12 3.3 Structure of this Report ........................................................................................................ 12 4 Component findings: Improved Production ................................................................................. 13 4.1 Farmer Engagement.............................................................................................................. 13 4.2 Delivery of GAPs and Commercial Training .......................................................................... 14 4.3 GAPs Training and Adoption ................................................................................................. 16 4.4 Farm Management Plan/Commercial Training and Adoption.............................................. 19 4.5 Does Longevity in a Project Matter?..................................................................................... 21 4.6 Sales Volumes and Revenue ................................................................................................. 22 4.6.1 Year-to-Year Project Performance .................................................................................... 22 4.6.2 Life of Project Performance .............................................................................................. 22 4.6.3 Farmer Investment............................................................................................................ 23 4.6.4 Performance by Farmer Segment..................................................................................... 23 4.6.5 Sustainability ..................................................................................................................... 26 4.7 Comparative analysis............................................................................................................ 26 4.7.1 Challenges ......................................................................................................................... 27 4.7.2 Revenue Analysis .............................................................................................................. 28 5 Component findings: Improved Inputs......................................................................................... 30 5.1 Chemical spraying ................................................................................................................. 30 5.2 Seedlings and Polyclonal Seeds: ........................................................................................... 31 6 Component findings: Traceability and Innovation........................................................................ 33 6.1 Progressing towards traceable cashew with batch processing ............................................ 33 6.2 Ensuring food safety through HACCP certification ............................................................... 33 7 Component findings: Improved Financing/Grants ....................................................................... 35 7.1 MozaCajú’s matching grant fund .......................................................................................... 35 7.2 Illustrative grants projects .................................................................................................... 36 7.3 Results and Suggestions for Grant Designs:.......................................................................... 38 7.4 Jobs Creation......................................................................................................................... 39 8 Component Findings: Marketing & Market Linkages ................................................................... 43 8.1 Better market linkages for farmers....................................................................................... 44 8.2 Better market linkages for factories..................................................................................... 45 8.3 Jobs Benefitted ..................................................................................................................... 47 9 Adjunct Activities: Policy and ICT.................................................................................................. 48 9.1 Knowledge-building within the sector.................................................................................. 48 9.2 Cashew law and regulation reform....................................................................................... 49 9.3 Enabling infrastructure via ICT (Information and Communication Technology) ..................49 10 Lessons Learned ............................................................................................................................ 50 10.1 Production............................................................................................................................. 50 10.2 Inputs: ................................................................................................................................... 51 10.3 Traceability and Innovation .................................................................................................. 51 10.4 Access to Finance: ................................................................................................................. 51 10.5 Marketing & Market Linkages:.............................................................................................. 52 10.6 Program Management:......................................................................................................... 52 10.7 Policy & Knowledge Building................................................................................................. 53 Appendix 1: Terms of Reference ........................................................................................................... 54 Background onTechnoServe and the Program.............................................................................................54 Endline Evaluation ....................................................................................................................................................56 Endline Objectives.....................................................................................................................................................56 Audience........................................................................................................................................................................62 Evaluation Methods ..................................................................................................................................................62 Key Activities andDeliverables............................................................................................................................63 Appendix 2: Evaluation Methodology................................................................................................... 66 Relationship to MTE.......................................................................................................................... 66 Quantitative Survey of Beneficiary and Non-Beneficiary Farmers................................................... 66 Quantitative Farmer Survey - Questionnaire................................................................................ 66 Quantitative Farmer Survey - Sampling........................................................................................ 67 MozaCaju sample weightings related to Population .................................................................... 68 Quantitative Farmer Survey – Implementation............................................................................ 68 Quantitative Farmer Survey – Data Cleaning & Analysis .............................................................. 69 Quantitative Farmer Survey – Limitations & Challenges in the Field ........................................... 70 Qualitative Study Design ................................................................................................................... 72 Qualitative Study 1 – Producer Groups, Nurseries and Promoters.............................................. 73 Qualitative Study 2: Larger Farmers, Processors, Buyers, Nurseries............................................ 75 Qualitative Interview Guide for Commercial-scale Farmers ............................................................. 80 Qualitative Interview Guide for Processors...................................................................................... 81 Qualitative Interview Guide for Global Cashew Buyers.................................................................... 83 Appendix 3: Table of Respondents ....................................................................................................... 84 Appendix 4: Performance Indicators and Results................................................................................. 87 Appendix 5: Grants Profiles .................................................................................................................. 91 Appendix 6: List of Key Documents Reviewed ...................................................................................... 96 Appendix 7: List of Datasets provided .................................................................................................. 98 1 Background: Relationship of this Report to Midterm Evaluation The following report is designed to be read as a stand-alone document, with further details on the comparative information cited available in the Midterm Evaluation submitted to USDA on 27 April 20161 . The Midterm Evaluation (MTE) findings provided a context for mid-course changes in the direction and emphasis of elements in the project and its final outcomes; the effects of these are noted in the current report. By the time of the independent MTE, we could discern a number of initial indications of positive effects from the project. The 23,000 registered farmers had been exposed to new and improved agricultural practices, including the use of improved inputs. A small income improvement was suggested by initial findings at the time of the MTE, supported by a favorable price effect of up to 5% being reported by farmers, attributed by them to being better organized for selling. As an important part of overcoming the problems of access to inputs, the project had established community nurseries and seed providers across the cashew production zones of the country; these were at an early stage of producing seedlings in volume, and attempts were being made to see whether nurseries could develop local markets beyond supplying MozaCajú. Several grants had been awarded but most were still only at early implementation stage. Significant preparatory work had been completed with the processing companies to upgrade their facilities and practices towards earning certification of international food safety standards, complementing the project’s efforts to develop the willingness of international buyers to place orders despite the poor reputation of Mozambique for quality and reliability -- with considerable early success, as reflected in the 938MT of kernel export sales recorded in 2015-16. At that time, the report noted that the overall trajectory was encouraging and the project was likely to meet all its major targets, while acknowledging that several significant challenges remained. Farmer training and organization seemed to have resulted in some differential adoption relative to non-participating farmers, and this seemed to be related to modest but positive commercial effects being reported by those beneficiary farmers. With cashew traditionally exhibiting considerable variability in its yields per tree, year to year, one major aim for the Endline was to identify whether these adoption behaviors and the commercial benefits were being maintained. Processor certifications for HACCP or otherstandards were being affected by long lead times for training, documentation and factory changes; the major companies showed widely differing levels of interest or ability to stay the course. Grant recipients were hampered by the very high cost of financing, when it was available, along with the many other start-up challenges of new businesses. Community nurseries had produced a pleasing number of seedlings in their first year, but support requirements were high and the overall target required expansion of effort and relied heavily on direct MozaCajú technical and financial support. Related to all of this, the MTE report noted several areas in which improvements to the project operation seemed necessary; in particular, the project’s internal processes and operational communication needed improvement, and final-year activities required careful prioritization. Management were of a similar mind and developed applicable plans in a number of these areas; consideration of the effects of those plans is included in the current evaluation. 1 Reference: MozaCajú Project Midterm Evaluation: A report to TechnoServe evaluating the progress of MozaCajú, a USDA project; Award Number TNS FCC-656-2013-038-00, June 2016. 2 1 Executive Summary MozaCajú, the “Mozambican Cashew Project,” was a 4-year initiative2 funded by the United States Department of Agriculture (USDA). On 26 September 2013 USDA contracted TechnoServe as the project’s prime implementer, with the Aga Khan Foundation (AKF) serving as sub-grantee and with the Government of Mozambique’s Cashew Promotion Institute (INCAJU) as a key partner. The project conducted its main production and processor-based activities from late-2013 to the end of 2016, funded by monetization totalling USD 9.322 million drawn down in three tranches. The award was later extended to 31 March 2018 as a no-cost extension for a subset of activities. MozaCajú operated in the three annual seasons (2014-16) of the original award across the three northern provinces of Nampula, Cabo Delgado and Zambézia, in line with the Government of Mozambique’s effortsto reduce poverty. Spanning thislarge geography3 , MozaCajú aimed to support the Mozambican cashew nut industry by harnessing rising global market demand for premium cashew. This in turn meant addressing long-standing obstacles that USDA and TechnoServe had recognized throughout the value chain, in particular in the areas of production, inputs, processing, finance, and market linkages that extend from the isolated farm gate to the global commodity buyer. MozaCajú also worked to support the enabling environment for the cashew nut industry through knowledge-building and policy reform. 1.1 Design of Endline Assessment The Endline evaluation was designed to resemble the MTE as far as practicable, to provide a basis for comparison of performance and the development of learnings. This involved two rounds of qualitative interview data collection and a large survey of farmers at the end of the project to collect the quantitative data (please see Appendix 1 for the ToR and Appendix 2 for Methodology details). The quantitative survey study provided demographic, activity, production and income data for the farmers, while the interview processes collected confidential business information from processors; perception and behavioral data from farmer groups, promoters, nurseries and staff; and explanatory insights from the handful of producer groups and commercial-scale farmers interviewed in-depth. Analysis of these data indicate that the project has substantially met or exceeded its headline targets for production, grants, inputs and exports; with targets relating to the processors largely met but slowed by the many contingent factors in making capital and operational investments. * The project has improved production. The project’s 23,566 farmers are now applying at least a small number of improved cashew farming practices e.g. pruning, cleaning, spraying, and some post￾harvest handling. This is supported by enabling 15,559 farmers to adopt at least two improved farm management practices in combination with spraying to prevent disease, resulting in 84,719 hectares now being under improved farm management techniques. * The project has increased access to inputs: the project helped establish 85 community-based, locally owned cashew tree nurseries, enabling 5,566 farmers to plant 542,807 tree seedlings to rejuvenate or expand their future production. By the time of the Endline, a small number of nurseries had developed potential pathways to viability post-project by making some local sales to farmers. Spraying chemicals and services are scarce resources; the project enabled 219 individualsto acquire sprayer machines with the potential to protect 328,500 trees against pests and disease. 2 The initial contract was established to operate between 1 October 2013 and 31 March 2017; in 2016, the parties agreed to a further 12 months of limited operation (i.e. closing farmer support in the extension period) to 31 March 2018 on a no-cost extension basis. 3 The three provinces involved have an aggregate land area equal to the state of Colorado. 3 In total, 309 micro-scale agro-dealers (including the 219 mentioned) were established to supply various inputs and services (spraying, seedlings, tools) to cashew farmers. * The project improved the competitiveness of the local processing companies: the project supported 6 factories to implement batch-processing training and systems, with 2,522 factory staff and management trained on batch processing and food safety systems. To date, 3 factories have received HACCP food safety certification (against a target of 5), and 2 factories are also progressing towards further certification, namely the British Retail Consortium or BRC certification that provides assurance of food safety, legality and quality, a qualification recognized by the Global Food Safety Initiative (GFSI). * The project increased access to finance and innovation to some extent, despite challenges: in a climate where credit is very difficult for smaller businesses to secure, 13 grants were awarded to 11 enterprises and agencies involved in the cashew nut industry, comprising 7 small and medium agri-businesses, 3 large cashew factories, and 1 government agency. The largest recipients successfully implemented new technologies, and all recipients welcomed the grants funding and - especially - the technical support. Many of the smaller recipients found the grant application process and operational implementation of their plans challenging; three additional approved grants were subsequently cancelled, reflecting the many obstacles these businesses face. * The project supported better market linkages and marketing: the project introduced 3 new international buyers to Mozambican processors, resulting in over 1,600 tonnes of cashew kernel export sales that can be significantly attributed to this work to identify, engage and facilitate at technical and commercial levels. Processor top management sentiment also appeared to improve between the MTE and Endline interview periods; despite their many criticisms of the local production chains in terms of quality and reliability, processors acknowledged improvements in supply and within their own operations, and all were eager for ongoing support. In a market where no premium is paid yet for higher quality, the surveys provided evidence of modest price improvement for cashew sold by the project’s farmers. Several farmer groups noted that they saw important intangible benefits in becoming organized, helping them secure better prices; but the project was poorly-equipped to track and record the quantitative benefits. In the 2016-17 season, MozaCajú supported USD 10.97 million of farmer gross revenues4 , building on the USD 5.7 million reported in the MTE for the eligible subset of trained farmers that had been supported for at least one full season in 2015-16, thereby summing to USD 16.7 million of gross farmer revenue supported over the life of the project. * The project thussubstantially met or exceeded the key numerical targets in the USDA agreement, principally the productivity, farmer income, processor upgrading and export sales outcomes, as summarized in Table 1: 4 We derived a range of income results depending upon judgement of the quality of the survey findings. The project farmers sold 13.8 thousand MT of RCN (based on the sample 2016-17 sales of 496 MT). At self-reported prices averaging 60Mts/kg, and assuming 100% sales completion, this would be worth approximately USD 11.9m at the exchange rate of MZN 71:USD 1.00 used for the year. INCAJU reference pricing was lower, however, at 56.4Mts/kg (weighted average across the three Provinces). At that price level, gross revenues would be USD 10.97m. Either approach involves potential biases; to maintain the conservative approach adopted in the MTE, we report the lower figure for the year; with the USD 5.7m MTE result, this sums to USD 16.7 million in total for the project. 4 Table 1: Summary Results Outcomes Target Description Target Achieved Result % Total individuals directly benefitting from the project 23.6 thousand 23.6 thousand 100% Farmers applying improved technologies/ techniques 17.3 thousand 21.2 thousand 122% Farmers applying improved farm management 5.8 thousand 15.6 thousand 268% Tree Productivity improvement (measured in RCN kg/tree, final year versus the 2013-14 baseline) 15% 53% 335% Total farmer income (cumulative, USD, life of project) $7.2 million $16.7 million 231% Factories/processors selling via international certification (HACCP; BRC) 5 3 (+2 ongoing) 60% Factories/processors applying new technologies (principally batch processing for basic traceability) 6 6 100% Export sales of kernel (sector-wide, cumulative incremental relative to 2014) in metric tonnes 1.9 thousand 3.7 thousand (attribute 1.6k) 195% * The project implemented a satisfactory exit strategy: producers largely expressed a newly￾positive outlook towards cashew in the Endline interviews with producer group and individual farmers, aided by high global commodity prices and favorable growing conditions in 2015 and 2016 that boosted sector production by 34%. Farmers widely acknowledged the support delivered by the project in organizing them for training and commercialization, albeit usually at a basic level. Processors reflected that improvements are visible in the supply chain, but wanted much more done: they continued to point to bad farmer practices and aggressive foreign buying of RCN as their chief risks. Policy development initiatives to mitigate these issues included the project team engaging the SPEED+ policy development project with evidence that there was an opportunity for legislative reform, resulting in multi-faceted collaboration with INCAJU to develop policy-shaping ideas. The project supported this by arranging educational visits to West African counterparts. The last round of interviews confirmed that these initiatives had helped change the industry discourse towards private sector investment, technology upgrades to remain competitive, and movement towards constructive regulation, themes that were cap-stoned with the project arranging a two-day national industry conference in December 2017 (see Section 9.2). The information and communication technology (ICT) component of the project inspired the Norwegian Embassy to sponsor a follow-on project for INCAJU and TechnoServe supporting 100,000 cashew farmers. 1.2 Lessons Learned Unlike the many previous projects in the Mozambican cashew sector, MozaCajú aimed to address the chronic production, quality and competitiveness issues in a holistic way. Interview respondents at all levels – producer, processor, global buyer, government official – welcomed this and stated that the MozaCajú duration was too short for such an ambitious scope. Project duration had an effect in the quantitative evaluation as well; the 2-3 year data timeseries was too short to enable us to ascertain how much of the reported income changes were sustainable versus how much were due to season-specific factors (e.g. favorable climatic conditions 2016-17 in most areas; the weak harvest in Southeast Asia in that period; abrupt currency depreciation increasing the nominal farmgate price). The design of the project to address this broad and ambitious scope had double-edged effects. The industry association (AICAJU) and international buyers acknowledged the value of tackling some of the highest priority and obstacles to farmer productivity while concurrently improving the competitiveness of local processors: better supply encourages better local buying, improving the supply and demand calculus. Engaging with global actors made sense, resulting in international buyers “rediscovering” Mozambique as a new source of initially tentative supply. The broad project 5 scope led to a splitting of responsibilities through a prime- and sub-award arrangement, largely separating the value chain reorganization work led by TechnoServe from the farmer production work led by AKF. Structured as separate grants, this is described by management as resulting in significant coordination issues, with the very different operating history and style of both organizations taxing management’s capabilities to deepen the intervention and align some key processes. Management responses to the MTE recommendations had ameliorated these issues but remained constrained by the sometimes-incompatible operating cultures between the main contractor and subcontractor, inflexible internalsystems of thatsub-contractor and project communication processes that provided inadequate monitoring for senior management. The bifurcated grantstructure and related project coordination issues resulted in an inadequate M&E capability for the project as a whole. Heavy reliance was placed on the collection of key data through surveys at the MTE and Endline; these consistently proved challenging to implement and provided poor comparative data for a fuller assessment of the project’s impact. We can report, for example, that farmers in the project reported a significant difference in adoption rates compared to non￾beneficiary farmers, and that these beneficiary farmers experienced good volume and price increases in 2015 and 2016, doubling their incomes; but methodological issues prevent any attribution of a direct correlation of income outcomes to specific project actions. Despite generally positive results in the major components of the project,several activities remained at an immature stage by termination date. The lesson is that complex value chainsrequire a sufficient runway, even once project efficiency has been addressed. In these results, we see that production yields are still well below potential, and post-harvest losses remained high, averaging 0.5kg per tree; farmer resilience is generally low and they lack the resources to renovate their plantations fast enough to reverse the sector’s overall productivity decline; the small cohort of inputs providers fostered require further support; processors have only partly stepped up to making the investments needed; and policy development, which began late, is bearing fruit but requires ongoing support. Access to finance remained a major and largely unsolved hurdle. While the Grants program provided a stimulus for a range of new ventures and introductions of technology, those recipients without foreign currency revenues to support forex loans struggled to secure matching and working capital; even with a renewed willingness to lend by banks, local borrowing interests rates are effectively 30% and thus prohibitive in start-up years. The project therefore reduced budget for this component. The recipients often required significant technical inputs, placing a greater than anticipated demand on project resources, and recipients struggled to implement on time. Despite this, the overall response from the principals of recipient organizationsremained very positive and the program leveraged over USD 1 million in funds, most of it through private investment of equity or retained earnings. The project developed and implemented an exit strategy that had been enhanced by the granting of a no-cost extension to allow specific field trials to continue, such as with a small number of nurseries, supporting an innovative new information technology project to launch in an overlap period, and enabling a strong collaboration to develop in policy reform discussions with INCAJU and other stakeholders. Knowledge transfer seemed only partly completed. Field level actors expressed concern in the Endline interview meetings that the project momentum would be lost, with several interviewed officials from INCAJU and the economic development arm of the government (SDAE) asking how continuity of support and knowledge could be maintained. Some private sector actors had similar questions. Pleasingly, some village level promoters have committed to voluntarily continue their work beyond the project. ‘Lessons learned’ for future project designs are thus provided (Section 10); better engagement with the sector stakeholders at an earlier stage to communicate respective roles, timelines and exit pathways is recommended for future projects. 6 2 Project Background 2.1 Global overview and history The overall prospects for cashew as a cash crop are good. As incomes rise and diets change globally, cashew trading globally has experienced solid growth in recent years and looks likely to continue (Antonio & Griffiths, 2017; INCAJU 2017). As a cash crop, cashew has proven over the last half-century to be well-suited to the landholding size, technical capacities and resources of smallholder production; as a result, African cashew nut producers supply around half the world’s supply of raw cashew nut (RCN). While these factorsshould represent a valuable opportunity forsmall farmers, the great majority in Africa remain trapped at subsistence level, reflecting their lack of access to productivity enhancing techniques, resources and services, together with the absence of any price premiums for improving quality. This is due to fragmented value chains, weak public extension, and the very low participation of African countries in value-addition; less than 10% of global processing is done within Africa, and only 10% of the end-consumer value is captured by producers in Mozambique (Grobe-Ruschkamp and Seelige, 2010). The great majority of African RCN production is exported as bulk commodity, primarily to India and Vietnam, the leading importers of RCN in the shell. With 3 million tonnes of processing capacity (in 2017)5 , these countriesthen process and exportto markets in North America and Europe, creating a high dependency on single sources for this supply. Global demand is growing consistently at 6-7% per annum. These factors represent a significant opportunity for local high-volume producers with their own processing capability. Mozambique was a top global producer of cashew nuts in the 1970s and later developed a thriving processing sector that exported primary processed cashew nuts. The effects of history still weigh upon Mozambique’s capacity to compete in cashew. Following a prolonged civil war from the 1970s to the mid-1990s, and a cyclone in 1994 that destroyed 40 percent of the productive cashew trees, Mozambique’s cashew production levels fell dramatically. The cashew sector’s recovery is impaired by structural obstacles that mean public extension reaches few farmers, efforts to rejuvenate the tree stock are falling behind and most of the sector profits are claimed by processors and traders, with minimal returns accruing to farmers themselves. The farmer base is extensive – well over 1 million households – and almost all are small-scale producers and poor; many are discouraged or, at best, passive gatherers, resulting in stubbornly low yields averaging just 3.2 kg per tree nationally, far below best practice in other producing countries. At the same time, the Mozambican cashew processing industry, which had become highly inefficient and overly centralized, declined significantly as government protections were temporarily lifted with market liberalization in the mid-1990s. As noted in the MTE Report, those decades-ago policy shifts towards market liberalization led to significant deterioration of the competitiveness of local processorsto provide a local market for RCN and to generate wage employment. Despite later efforts to rehabilitate the downstream processing sector, processors have struggled to diversify their markets and capture the full potential of these markets due to both food safety concerns and weak market linkages between processors and farmers, making product traceability impossible. Cost structures remain high, labor productivity and absenteeism further erode margins, and markets are becoming increasingly demanding of food safety certifications (see e.g. the MozaCaju 2015 report: Competitiveness of the Mozambican Cashew Industry6 .). The need for sector-wide investment has been heightened by many countries developing stricter food safety compliance and traceability requirements for imports. Most European buyers today 5 Jim Fitzpatrick keynote address at the MozaCaju National Cashew Conference 2017; accessible at: http://www.mozacaju.com/presentations/ 6 Accessible at: http://www.mozacaju.com/publications/ 7 require the implementation of a food safety management system based on internationally recognized standards; at minimum, this entails certification for Hazard Analysis and Critical Control Points (HACCP). Similarly, the U.S. has increased its food safety regulations through the Food Safety Modernization Act of 2011, which aims to ensure the safety of U.S. food supply by shifting the focus from responding to contamination to preventing it. However, at project commencement most existing Mozambican processors had no food safety management systems in place to regulate the quality and safety of the cashew nuts for export, increasing risk of contamination-based recalls and losses, thereby limiting their market opportunities. These processors were also in need of major technology upgrades. At the time, processors still relied heavily on manual labor, despite improvements in technology becoming more widely available. These processors were hesitant to adopt these new technologies and techniques due to scepticism of the benefits, and as such they were not therefore able to compete with other processors who did upgrade. TechnoServe had worked in prior projects to help the processing industry rehabilitate its capabilities, mostly with a small processorstrategy, but even aslate as 2011-14 severe slumps in RCN volumes were still occurring regularly, overhangs from the complexity of reconstructing the sector (see reports by e.g. FAO7; leading development economists8) and inadequacy of doing it piece-meal. FAO’s study of the sector from 2005-13 showed (and MozaCajú’s experience more recently has directly confirmed) that a large proportion of the national tree population is aged and poorly maintained. INCAJU is approaching the end of a 10-year recovery program with its ambitious goal of doubling raw cashew nut (RCN) production to 181,000 metric tonnes (MT) by 2020, and is contemplating its new goal of 300,000 MT by 2033. The most recent fully-recorded year saw 139,000 MT commercialized, a volume 2.1 times that of the sector five years prior. But as noted in the MTE interview findings, processors have remained concerned about the availability, cost and moisture content of the RCN sourced locally, while INCAJU resources to improve supply are chronically constrained, meaning that at best perhaps 15% of productive trees receive spraying support9 . The situation has not improved at the date of thisreport: the national tree stock continues to decline, and INCAJU continues to only be able to supply far less than one million seedlings per year, barely improving on figures from 5 years ago10, with poor seedling survivorship rates based on the scant tracking information available, due to the lack of irrigation and high mortality of grafted seedlings. 2.2 Local economic context In Mozambique today, INCAJU estimates that as many as 1.4 million households grow cashew — though much of it is grown in very small plots and cultivated relatively passively. Cashew11 is one of the few reliable cash cropsthat farmers have historically been able to grow in the country; as a result, these smallholder farmers account for more than 95 percent of the country’s cashew production. In Nampula province alone, cashew accounts for nearly 20% of total household income and approximately two-thirds of total cash income (in cashew producing areas). As a result, Mozambique accounts for less than 3 percent of African RCN production, seen by international buyers as having 7 FAO. 2014. ‘Analysis of price incentives and disincentives for cashew nuts in the Republic of Mozambique’ 2005- 2013. Technical notes series, MAFAP, by Zavale, H., and Monroy, L.; Rome. 8 ‘When Economic Reform Goes Wrong: Cashewsin Mozambique’, McMillan, M., Horn, K., Rodrik, D. 2002, Harvard. 9 INCAJU can reliably support less than 3 million of the 30-40 million national tree stock with its annual supply of chemicals for spraying – typically the single most significant intervention for improving yields. 10 Source: INCAJU presentation ‘Desempenho Do Sector- 2013’, October 2013. Figures for INCAJU seedling production vary widely. The ‘Cashew Competitiveness Report’ (2015) cited the planned 2015-16 figure of 3 million seedlings, but it is unclear that these volumes can be produced and distributed in the target provinces, and survival rates from INCAJU nurseries are currently low. 11 Source: INCAJU (Instituto de Fomento do Caju) presentation ‘Desempenho Do Sector- 2013’, October 2013. 8 the lowest quality within Africa due to old acreage and recurrent disease outbreak. The economic importance of cashew is also seen off the farm; processing jobs provide income for Mozambicans by way of factory wages. When a cashew factory opens, usually in a rural area, it employs workers and it procures its RCN from the local community. A typical factory will have over 1,000 workers at any one time, about 60% of them female (as recorded in our Endline study of processors) engaged throughout the various Sections of the cashew factory. In northern Mozambique, there are now 14 large industrial processing facilities and several smaller facilities, representing 12-14,000 jobs in these communities where alternative cash employment is difficult to find. In recent years, however, the Mozambican cashew processing sector has operated at 70 percent of capacity, largely due to significant supply-side challenges and the lack of competitiveness due to quality issues. For example, in the 2015-16 period, one processor experienced the rejection of over 20,000 boxes (equivalent to 28 containers, value nearly $3 million) due to quality problems. Some improvement in factory investment is more evident lately, with the opening of two additional factories by OLAM and a new factory built by ETG. The sector has effectively doubled in capacity since 2013, with a total cited capacity of 46,000 MT of RCN compared with 25,000MT in 2013-14, with 62,000MT of RCN procured by the processors in the latest fullseason. In the interviews conducted for thisreport, three processor companies stated they had further investment plans to expand production capacity by 2020. MozaCajú’s involvement in support of this overall development is given in Sections 6 and 8 below. At a policy level, there is now a growing conviction that step-change is possible and required, as reinforced by MozaCajú-organized visits for INCAJU and processor officials to witness the growth in countries such as Cote d’Ivoire, and by ongoing policy development work in association with the SPEED+ initiative (see Section 9.2). The consensus is forming that in 10 to 15 years (2028-2033) cashew production in Mozambique can increase by three times above the recent-years’ average of 100,000 MT, moving to 300,000 MT and above. This would entail producing and planting 10 million cashew seedlings per year for the next 5 to 10 years, with steps taken to improve survivorship including boosting the capabilities of the sector to implement pest and disease control for something like 15 million cashew trees annually rather than the perhaps 3 million treated reliably currently. 3 MozaCajú Objectives and Strategy In September 2013, USDA’s Food for Progress program granted to TechnoServe 8,540 MT of crude degummed soybean oil (CDSO) to monetize anticipated proceeds of $8,540,000 for use over a period of approximately three years ($2.85 million per year) through September 2016 to implement the Mozambique Cashew Project (‘MozaCajú’) in the three northern provinces of Cabo Delgado, Nampula and Zambezia. In a subsequent modification (July 2015), the commodity allocation was increased to 11,110 MT of CDSO in order to guarantee the project a needed 6-month extension to see out the 3rd harvest season, and fund a number of special initiatives including CommCare and additional grants. However, a lower commodity sale price than expected for the final tranche has resulted in a final monetary budget of $9,335,602 over 3.5 years ($2.67 million per year) through March 2017, with a corresponding reduction in some activity budgets and targets (see Table 2). Table 2: Monetization Summary Monetization Tranche Amount Realized (USD) Date Monetization 1 3,250,427.75 First Sale November 2013 Monetization 2 2,973,750.00 Second Sale May 2014 Monetization 3 3,111,423.80 Third Sale February 2016 Total Monetization Proceeds 9,335,601.55 9 In 2016, the parties agreed that elements of the project could be extended for a further duration, ultimately set at March 2018, on a no-cost basis, so that key activities could be wrapped up in a satisfactory way. This entailed tracking a further season of commercialization and seedling production, while executive-level policy development work proceeded in parallel. The aim was to provide a solid basis for continuation of key initiatives, post-project. MozaCajú was an innovative sector-wide project that sought to improve the capabilities of key stakeholders, at scale, in ways that naturally align disparate interests and thereby enable their activities to integrate in value-adding ways. Building on and extending various prior initiatives in the sector by parties ranging from USAID, the Bill & Melinda Gates Foundation and the Government of Mozambique, implemented by a range of international NGOs including TechnoServe and GIZ, MozaCajú aimed to link these previously separately addressed elements of the poorly-functioning value chain to support the re-building of the export-oriented cashew kernel supply chain in Mozambique. TechnoServe came to the project with extended experience (2002-09) in working with cashew processing factories in Mozambique to upgrade capabilities, attract new investment for smaller, labor-intensive processing plants, and to help establish an industry association to assist with sector development12. MozaCajú was designed to build on this expertise and introduce new elements to build the capabilities and resources of small farmers to engage in cashew production with greater productivity, increased commerciality, and with an improved level of planning based on confidence that an accessible market will reward enhanced quality and reliability of supply. In these ways, the project planned to establish a foundation forsupporting the broader development of a more capable and resilient rural economy. The project’s key objectives were stated in the award granted by USDA as: 1. To increase agricultural productivity in the cashew sector by providing training to improve agricultural practices and post-harvest handling, expanding access to inputs, establishing nurseries, and strengthening smallholder producers; and 2. To expand sector trade through training in phytosanitary and sanitary standards, certification, and traceability; facilitating market linkages; increasing access to finance and investment. MozaCajú’s strategy was to leverage the existing expertise and resources of government and commercial actors while demonstrating more participative, commercially-oriented models for delivery of support. This was implemented through services that attempted to recognize the differing needs of farmers ranging from the vulnerable subsistence household to those seeking to become true farming businesses. This strategy was defined by five component-level project objectives: 1. Improved Production: improve the capabilities of farmers through agricultural, post-harvest and farm management training, supported by a network of extension agents and demonstration plots. 2. Access to Inputs: improve the supply of seedlings, and related inputs and services, including through the direct establishment of community level nurseries. Initially under-estimated, the importance of the supply and use of chemicals was highlighted in the work to negotiate chemical supply priorities with INCAJU and the development of policy changes to attract private investment. 3. Traceability and Innovation: address hazards and encourage investment in quality improvement practices by primary processing facilities, including through selective grant-making, to thereby enhance competitiveness in export kernel markets. 12 ‘Factories in the Fields’, Paul, Brad, PhD., TechnoServe Mozambique 2008. Available at: http://www.technoserve.org/files/downloads/factoriesinthefield.pdf 10 4. Access to Finance: facilitate the development of lending and investment opportunities in the sector and encourage technology innovation through targeted grants. 5. Marketing and Market Linkages: support the restoration of Mozambique as a credible and dependable supplier of kernel through facilitation of sales and the development of better linkages between farmers, processors, and global buyers, to reinforce the value of sector upgrading. These components, and the integrated approach adopted in the project design, drew on the many years of TechnoServe experience in Mozambique and lessons it has learned in the cashew sector across Sub-Saharan Africa, where TechnoServe supports approximately 80% of the main processors13. The original project award included a sub-grant made to the Aga Khan Foundation Mozambique (AKF) to perform the bulk of the field-level work in organizing and supporting the project’s producer base across the three provinces (Figure 1 below). AKF employed and managed MozaCajú field staff and field operations(procurement and distribution of inputs and extension tools, and collection of field-level data). Production activities aimed to draw on Aga Khan Foundation’s long experience in Cabo Delgado specifically, extended for this project to Nampula and Zambezia provinces with newly-established field teams and managers. These experiences all shared a focus on helping defined target groups achieve sustainable socio￾economic benefits -- including improved farming practices, market access for smallholders, job creation in competitive local enterprises, financial services, improved public services infrastructure and the introduction of appropriate information and communication technologies. Please see the final set of Performance Indicators and Results in Appendix 4 for target and achievement details. 3.1 Project scope Over three annual seasons of planting, cultivating, spraying, harvesting and commercialization, MozaCajú worked with over 23,500 cashew farmers, one in five of whom were women, to improve their productivity and commercial opportunities. This scale of activity represented the potential to improve the prospects of the 100,000 or more additional individuals in the households of those farmers. The partnersrecognized from the outset that improved production and more organized selling would not be sufficient to address the many structural and environmental impediments to improved household outcomes. The project therefore aimed to complement the work with farmers by stimulating an ecosystem of local inputs and services, while improving the competitiveness of the local processors whose factories represent the ultimate market for much of the smallholder production. The project aimed to support small and sometimes medium-scale enterprises such as agrodealers and including the community nurseries established and supported by the project, aiming to improve access to inputs and related services by generating local investment. The project partnered its expert food safety and training resources with external technical services providers to engage with seven industrial cashew processing facilities to provide hands-on technical assistance that would support the introduction of batch processing and international food safety standards. The project team saw that these upgrades were essential for local factories to access foreign buyers. Complementing this technical work, the project’s market facilitation activities brokered introductions to global and regional buyers, and provided ongoing technical and marketing support to the factories’ pursuit of contracts with buyers. 13 Source: Shakti Pal, TechnoServe Regional Cashew Specialist, Nampula; corroborated independently in interviews with several global buyers in the MTE and Endline evaluations. 11 For all actors, the challenges of accessing finance for cashew production or related services are substantial in Mozambique. MozaCajú included a grants fund aimed at demonstrating the ability of sector participants to introduce innovation at micro, meso and macro levels of the sector. The grant program focused initially on farmers and processors, but found that even the largest farmers were not ready to provide the business plans and matching funds to enable new projects to be approved, with a few exceptions (e.g. DDI, Emaju, Langa). The program then belatedly focused on the further acceleration of small and medium agribusiness, the trials of new plantations by commercial farmers or cashew related businesses by medium-sized entities, and the adoption of new technology by the major processors. The mutual desire to see certain initiatives through to a satisfactory completion led to the agreement of a no-cost extension that constituted year 4 of the project. Farmer support terminated at the completion of the original term, at which time most of the project’s facilitator staff and local promoter positions were eliminated. A small number of the highest performing field workers and nurseries (4 facilitators: 2 in Cabo Delgado, 2 in Nampula; 19 promoters; and 14 nurseries) were retained for an additional period to late 2017 to support one further season of seedling production and additional training for nursery managers on quality management. 3.2 Endline Evaluation Aims and Design The organizational structure of the project reflected what management believed to be the most effective and pragmatic split of work. That allocation drew on what management had perceived to be the relative strengths of the partners, arising from the relatively complementary distribution of prior project experiences between the organizations – AKF historically in Cabo Delgado, working with smallholders in diverse crops, and TechnoServe in Nampula and Zambezia spanning multiple sectors, mostly at the commercial end. Throughout the project, AKF focused on production activities whereas TNS focused more on the commercial aspects, grant-making and processor upgrading. As a result, the MTE had contained both an analysis of component-level attainment against results indicators and a lengthy report to management on identified process or resource issues with recommended actions to address them in a prioritized manner. These latter elements were actively engaged with by the Project Director and team, by means of a comprehensive workshop with the lead evaluator, subsequent improvement plan development for agreed priority areas and periodic monitoring of major improvement initiatives by the Project Director and team following the MTE. Figure 1: Intervention Geography 12 The quantitative Endline survey for the production and processing project components was conducted during the last reporting period, since direct technical assistance to producers and processors ended March 2017, and preparations for it were made for the first of two visits by the Endline evaluator. The full qualitative Endline evaluation (comprising finance, marketing, and sector￾level interventions) was conducted in the second half of 2017, and addressed issues of sustainability as well as the project’s broader innovation and impact through the grants program, policy initiatives, and work with investors and retailers. The methodology is set out in Appendix 2. 3.2.1 Project Management Context The organizational context to the scope and approaches adopted in the MTE and Endline have a bearing on the design described below. As noted above and discussed earlier in the MTE, the project had adopted a restricted approach to Monitoring and Evaluation (M&E) from the outset. The project made only limited resources available for M&E deployment: TechnoServe appointed an individual as M&E Manager, but that role became vacant after several months and was not back￾filled; AKF did not allocate a specific budget for M&E until the end of the first full year of the project, at which time a single M&E Assistant was located in Pemba and part of the national M&E Director’s time was allocated. These restricted resources lacked the processes or people to provide “antennae” in the field, affecting the frequency of data collections and analysis, restricting the data available for assessing activities and outputs, and especially limiting visibility at the outcome level. This was in part due to a combination of challenges finding experienced resources in the north of the country and the separate workings of TechnoServe and AKF field activities to a large degree, exacerbated by the attenuated communication and reporting lines across the very large geographical span. The latter impeded regular reporting of updates, while the former affected what could be done with the data collected. The M&E strategy instead evolved to rely on AKF resource for production and for a hybrid monitoring strategy for the other components, with operational monitoring by the functional leaders in the project supplemented by independent data collection and evaluation at both the mid-term and Endline periods by the external evaluator. 3.3 Structure of this Report The remainder of this report is set out as follows. The findings for each of the five components are each set out in a separate Section: Section 4: Improved Production Section 5: Access to Inputs Section 6: Traceability and Innovation Section 7: Access to Finance Section 8: Marketing and Market Linkages Based on these findings, an overall evaluation assessment is made in Section 9. From these discussion points, specific lessons learned are suggested in Section 10, as future project design inputs for cashew projects. The seven Appendices provide: (1) the ToR for the evaluation, (2) the Methodology for the quantitative and qualitative data collection and analysis, (3) Informants, (4) the principal Results Indicators, (5) Grants profiles, (6) Documentation referenced, and (7) a list of Datasets supplied electronically. 13 4 Component findings: Improved Production 4.1 Farmer Engagement Since 2014, MozaCajú has worked with cashew farmers across the three northern-most cashew￾producing regions in Mozambique, training them on improved production techniques and technologies in order to increase their yields. As Table 3 below shows (see also Appendix 4), a total of 23,556 farmers were registered and trained. The project segmented these farmers into four categories by number of reported productive trees14 and the small and medium-sized farmers were organized into farmer groups. By project-end, around 21,500 active participants remained involved. Table 3: Farmer base demographics Province # of Groups Number of Producers by Segment Total # of Producers Sum of Province Productive Trees Mean/Prod. # of Trees cf. MTE Trees cf.MTE Trees Small Medium Large Commercial Delta: TOTAL Mean Zambezia 103 1,132 1,117 117 1 2,367 361,024 153 25,024 107% Nampula 642 9,028 3,613 757 194 13,592 2,382,240 175 350,167 118% Cabo Delgado 318 3,100 3,433 1,006 58 7,597 1,210,319 159 -126,269 91% Total 1,063 13,260 8,164 1,880 253 23,556 3,953,583 168 248,362 107% % of Total Farmers 56% 35% 8% 1% 100% Comments: Overall, tree count has expanded, in part due to a Ave % Male 78% 84% 87% 79% 80.2% renewed effort to include larger farms headed by female farmers Ave % Female Ave # Tree/Male Ave # Tree/Female 22% 62 53 16% 178 178 13% 447 448 21% 1,622 2,701 19.8% who, on average, report having higher tree counts, in particular in Nampula. Total female participation reached 20% . The 1,063 groups comprised 93% of the total farmer count. The average size of each group was 21 farmers, with fluctuations in size as some farmers dropped out due to health or other employment. The Nampula cohorts represented 58% of the whole project, proportionate to the overall farmer population; as Table 3 reflects, Nampula has a higher average number of productive trees per farmer due to the disproportionately higher incidence (77%) of “commercial” farmers. Female farmers comprised 19.8% of the total project participants, a slight increase on the MTE result of 19.3%. As the numbers of registered farmers grew between MTE and the final year of the project, we identified an overall increase of 7% in the average tree holding, due to a combination of tree planting by project participants15 and the recruitment of farmers with larger farms in the later phases of the registration process in Nampula. Female representation as farmers is not well-understood; average tree holdings on farms registered to female farmers were smaller in the case of “small” farms, but elsewhere matched or exceeded male averages. From survey results, only around 65% of trees 14 Segmentation categories: ‘Small’ (<133 trees); ‘Medium’ (133-400 trees); ‘Large’ (401-1,000 trees); ‘Commercial’ (>1,000 trees). As noted in the MTE, this proved to be a highly unsatisfactory way of segmenting a farmer base, in particular because it ignores the behavioral differences that may constitute a stronger predictor of a farmer’s capacity to absorb new knowledge, response to quality improvement techniques and make investments. The MTE qualitative interviews and overall interpretation suggested that a behavioral segmentation model would be more effective; Endline interviews confirmed this, however It was not feasible to change the segmentation scheme mid- project as no baseline data existed for recasting the earlier periods in an alternative schema. 15 60% of the project’s farmers planted an average 42 trees each in 2015; in the Endline survey, the result improved to 65% of project farmers planning an average of 59 trees annually, effectively 2 times the rate by control farmers. These trees were typically reported as “productive” in the survey by many of the farmers, but most were too immature to yield commercial quantities of nuts within the project timeframe. 14 regarded as “productive” (i.e. not fatally burnt, diseased or aged) bore sellable quantities of nuts in any one season, with farmers discriminating accordingly in allocating labor and inputs use. MozaCajú deployed four regional supervisors supporting 16 field technicians that worked with 176 farmer promotion agents. These promotion agents, more commonly known as “promoters,” were members of the community selected by MozaCajú according to specific criteria including literacy and approval from that community. Each promoter was trained on the designated improved production techniques and technologies and was subsequently certified and responsible for training 7-8 groups of 20-30 farmers, averaging about 130 farmersin total per promoter. In return the promoter received training, materials and a small incentive. These farmer trainings, which took place on local demonstration plots using the Farmer Field School (FFS) method, were organized seasonally to ensure that farmers used the practices soon after they were taught. The scheme seemed effective overall in broad capacity building, as seen in e.g. GAPs adoptions (see Section 4.2); while a number of promoters expressed their dissatisfaction with the incentives in the MTE and Endline interviews, several others expressed enthusiasm to continue providing the farmer support voluntarily following the end of the project. Given their training, this could be combined with e.g. support to establish a small inputs business such as spraying services, to make that ongoing support sustainable -- an MTE recommendation partly realized through the AgriFocus micro-grant process (see Section 7.1 below). 4.2 Delivery of GAPs and Commercial Training These production activities have achieved or exceeded the GAPs targets, with 21,323 farmers (90% of the total) applying at least three GAPs (such as pruning, cleaning, spraying, and post-harvest handling), and 15,559 farmers (66% of the total) adopting at least one improved farm management practice. This adoption of GAPs has led the project’s farmers to cultivate a total of 84,719 hectares under improved techniques, on this modest set of criteria. These results have been accompanied by a 35% increase in production volumes in the final year of the project, but the latter is not clearly distinguishable from the performance of typical farmers in the three provinces – the latter experienced similar average increases of 34%. This was due to two good years for producers generally, thanks largely to timely rains and perhaps lower disease pressure (INCAJU 2017). Beyond the numerical results, 80% of the farmers reported through the survey and the producer group qualitative interviews that they are fully satisfied with the project versus their expectations in joining. In the in-depth interviews with larger farmers, the study found that while they often have very specific requirements, their technical and market linkage requirements were largely met by the project. The lack of access to credit was perceived as a significant issue but one that was beyond the project scope, apart from grant-making. In the following Sections we report on the specific technical and commercial results obtained in the Endline evaluation process. The project delivered most GAPs training through the Farmer Field School (FFS) model of in-situ collective training. Over the course of three years, 1,168 demonstration plots were created to train the 23,556 cashew farmers as part of the FFS program, with e.g. 99.4% (23,414) reporting they were trained on the major issue of post-harvest handling. ‘Small’ and ‘medium’ farmers (i.e. those ranging from c. 50 trees up to 300 productive trees) were supported collectively, through farmer producer groups supported by the promoters. The support processes faced some challenges: recruitment and retention issues had resulted in a slight under-representation by women in the promoter network (21% female field promoters, compared to a target of 25%), and the project saw a pronounced turnover of staff in Nampula province mid-way, including the replacement of the provincial supervisor (with a lengthy bridging of two provinces by one supervisor until the role was filled). The farmers classified as ‘large’ or ‘commercial’ tended to receive more direct support by the facilitator staff, with the largest commercial farmers being directly supported by the supervisors themselves. 15 One of the GAPs training objectives was to enable replanting and expansion of farms, but farm replanting and expansion remained heavily constrained by farmers’ lack of financial resources. Of the 17,410 Ha. recorded in the final project database as available for future expansion of cashew planting, 62% of that available land is held by smallholders (74% male, 26% female) who almost universally lack the resources to prepare the land and cultivate seedlings through the four-year development cycle before those new plantations generate cash. One of the most common constraints to expansion found in the Endline interviews was that labor costs had increased over 100% in the prior 12-18 months, making land preparation increasingly unaffordable for many. GAPs adoption during the season was similarly constrained by lack of working capital. While all farmers (beneficiary or control) who hired labor reported a similar average cost per season (approximately 3,700 Mts per farm, or USD 52), the beneficiary farmers were found to be much more likely to hire labor for that work: 43% reported doing so in the final season, compared to only 7% of control farmers. This is plausibly a reflection of improved sensitivity to applying GAPs, as promoted by the project, an important behavioral influence especially considering the relatively aged farmer demographics of cashew production (Figure 2). The project did not attempt to mobilize financial services providers to provide farmer credit facilities to address these issues. Delivery of commercial (or “farm management plan”) training reached fewer farmers; from the survey results, we calculate that 17,502 farmers completed the farm management plan training course, a shortfall compared to GAPs due partly to field staff turnover and the relative over-weighting of agronomic support by those agronomy-trained and relatively junior-level field staff. This staff profile has implications for promoting farm investment and income resilience; lack of training may have prevented some farmers from being able to develop plans in order to access resources such as working capital loans. As noted, working capital scarcity is especially significant with an aging farmer population: around 30% of farmers reported being 55 years or older, with e.g. the average Cabo Delgado beneficiary farmer notably older than the norm. Where hired labor was used, often the farmer owed the wages in RCN harvest; as with spraying on similar terms, at recent RCN prices this represented significant opportunity costs16. At the other extreme, commercial farmers whose holdings range from 1,000 to upwards of 15,000 trees or more, and with presumably the greatest capacity for more systematic farming aided by mechanization, reported that they have very little land available for expansion, and limited ability to Figure 2: Age distribution of farmers (Endline survey) irrigate new plantations, in the absence of financing. 16 Farmers in the MTE frequently cited a cost of 2kg of RCN per tree for spraying services; in the Endline process, we heard some farmers saying they now pay 3Kg (effectively, $3 per tree). They reported that labor costs for cleaning had increased from 20-25Mts/tree to 50 Mts/tree in 2016. For a small farmer with 120 trees, this could represent outlays equivalent to $300-450 per year, several months before average revenue of $550-600 at typical yields. 16 As with the MTE, only a minority of interviewed commercial farmers had sought (and even fewer secured) loans; two of them however exhibited skill in accessing grants as well asloans from multiple sources, enabling them to set up sizable spraying and transportation businesses as adjuncts to cashew growing that one participant described as decidedly more profitable than the farming itself. 4.3 GAPs Training and Adoption As evidenced in the MTE evaluation and by this Endline, MozaCajú has worked with a very large base ofsmall cashew farmers to increase their adoption of improved production techniques, while helping to build capacity and leverage investments by larger commercial cashew farmers. Overall, the evaluation showed MozaCajú farmers were 2.6 times more likely to adopt most or all of the suite of promoted Good Agricultural Practices, for at least part of their farm, with 50% adopting this level of GAPs. Exposure to GAPs training by cashew farmers surveyed was near￾universal, but that was expected; there have been multiple cashew projects over recent decades and farmers made frequent reference to them, claiming repeated re-trainings in basic techniques. The MTE had reported that some degree of difference could be identified between the actual adoption of various combinations of those practices, as shown by the beneficiaries of the project relative to proximate control farmers. The surveys recorded the reported adoption of each GAP; as anticipated, pruning and cleaning practices in combination figured prominently with all farmers. The MTE noted that the more interesting results lay in the combinations of GAPs adopted. This difference seemed to emerge more clearly in the Endline for each major combinations of GAPs, as shown in Figure 3 where the 12 most commonly adopted combinations of GAPs are summarized in terms of the relative rate of their adoption reported in the surveys, between the beneficiary and control groups. Figure 3: GAPs adoption (in discrete combinations; top 12 shown) Whereas the MTE was conclusive for only some combinations of GAPs the Endline results reflected a significant separation in the rates of adoption between the groups surveyed.17 Table 4 (below) shows that farmers in the MozaCaju project reported 44% higher adoption of more comprehensive sets of GAPs techniques (adopting, 4, 5 or more such GAPs). In the Endline results, the difference between treatment and control adoption (for half or more of the practices) is statistically significant at the 99% confidence interval. 17 Note: while very few farmers openly stated zero adoption, 5% of project farmers and 41% of control farmers did not provide any response to the questions on adoptions – see below. The graphs in this chapter relate only to those 17 farmers providing a response of at least one adoption of a relevant practice or combination of practices. 18 Table 4: Overall Adoption Differences GAPs Adoption Beneficiary Control Difference Two or less 32.1% 59.1% -27.0% Three or more 66.6% 41.4% 25.2% Four or more 37.2% 13.8% 23.4% (or 2.7X) Five or more 17.9% 6.9% 11.0% (or 2.6x) Another way to understand this is in terms of relative propensity to adopt. As noted, we found that very few farmers only adopt a single GAP, and that once we move beyond the simplest techniques (pruning, cleaning), the survey results18 recorded substantially higher rates of adoption by beneficiary farmers with respect to pest and disease control; and even more so for the full range of practices promoted. The frequency of such adoption is summarized for combinations of GAPs reported for the final season of the project in Figure 4. Of those farmers that responded that they had adopted at least one GAP, the graph below presents the respective frequencies of the most common 12 combinations of GAPs adopted19. In total, 23,414 farmers applied at least one GAP. By recasting these data in terms of the frequency of the most common combinations adopted, beginning with the most popular combination, the difference between beneficiary and control groups can be clearly seen (Figure 4 below). Figure 4: Frequency of GAPs combination adoptions (from most popular to least): for all adopters. For example, while one in every five farmers simply prunes and cleans (combinations 1, 2), project farmers were twice as likely to find resources for spraying, pest/disease and fire management than the general farming population (i.e. reflected in the 41% cumulative result in the graph, for the second most popular GAPs combination - contrasting with just 22% for control group farmers). Even in the absence of chemicals to enable spraying, the next two most popular combinations took the cumulative difference to 65% adoption versus 31%, a 2X difference that was maintained thereafter. 18 Survey samples: Beneficiary n=851; Control n=242. See Appendix 2 for details of sampling approach. 19 An insignificant number of “other” responses were received and are also ignored. 19 We should note that these results are skewed by different response rates; the charts above reflect only the responses of farmers who had adopted at least one GAP. Within the beneficiary dataset, when asked whether they had adopted GAPs they had been trained on, less than 2% of project farmers failed to respond or had adopted no GAP; in the case of the control dataset, 41% failed to respond or had adopted none, with the typical explanation being some combination of a lack of cash for labor or spraying services, the age or infirmity of the farmer, lack of tools, damage to the trees and the perennial issue of the lack of supply of chemical and/or its late distribution by INCAJU. While many of the beneficiary farmers faced similar constraints, the project team provided evidence that they had worked with INCAJU for two seasons to prioritize the distribution of chemical, organize the farmers for some degree of self-help and the attraction of service providers, and had provided constant local reinforcement of the need for GAPs adoption through the promoter corps. Farmers confirmed these claims in interviews, albeit there was still misunderstanding by some that the project would rectify the chemical shortage directly (as also noted in the MTE report). The key lesson here seems to be that it is critically important to find ways to remove obstacles to farmer enablement, if training is to be converted into timely adoption. Projects can achieve this in part – for example, MozaCaju’s efforts to guide INCAJU distribution of spray to better-trained farmers; but much of the initiative taken was carried out independently by farmers themselves, according to interview results; for example, in sourcing chemical spray directly from Tanzania to optimize production. Such self￾reliance promotes sustainability, but may be the preserve of the relatively better-off farmer. Though small farmers carry out the bulk of cashew farming (in numbers of actors and in tonnage), MozaCajú had hypothesized that commercial cashew farmers would have an important role to play and had gauged that these farmers would be more responsive to improved access to technical and commercial support. Based on their field experience, the MozaCajú management team concluded that “commercial farmers demonstrate far higher rates of adoption of good practices in response to direct technical assistance, and serve as role models for neighboring smallholders. Smallholders, in turn, exhibit much higher adoption rates of improved techniques when a successful commercial farmer is located nearby, whereas they are less likely to take risks and change practices without first seeing a larger neighbor succeed. 20” The project also concluded that “commercial farmers are a focal point for procurement and can be aggregators for neighboring production, which enables expedited and more efficient procurement for traders and processors, enhancing the overall competitiveness of the export-focused industry. Therefore, one key lesson learned from MozaCajú is that focusing efforts on these larger farmers will increase impact and help reach more cashew farmers overall”. These reasonable-sounding hypotheses were subsequently supported in broad terms by the Endline quantitative evaluation based on the survey results21. From the survey, commercial farmers did adopt a relatively higher count of GAPs than smallholders (average of 3.5 GAPs per commercial farmer, versus 3.0 for smallholders), in large part because a high percentage of smallholders adopted just 1-2 GAPs -- or in rare cases within the project, none at all. Furthermore, the in-depth interviews suggested that the commercial farmers tended to apply these practices to a higher proportion of their tree stock, in a prioritized fashion that should maximize yield. Some commercial farmers were observed to be engaging with neighboring smaller farmers; this was typically in connection with the former operating a spraying business or nursery, or having e.g. a tractor available for hire. The segmentation nomenclature can be misleading, however (see footnote 14, Section 4.1); as observed in the MTE and again in the Endline, some medium-scale farmers displayed greater commercial awareness and capacity for adoption of new seedling varieties and techniques -- for example, buying 20 MozaCajú Impact Report, December 2017. Accessible at: http://www.mozacaju.com/publications/ 21 Within the limits of a short time series of data in a complex context involving multiple variables; we should note the qualification that further analysis of these survey data was not able to produce correlation results with statistical significance (see Appendix 2 for discussion). 20 chemical when it was not available from INCAJU, because they actively invest in their farms – whereas some farmers with large tree holdings lacked the capacity or aptitude for much change. “Large” and “medium”-scaled farmers overall landed in the middle, with almost all adopting at least some GAPs, and some exhibiting strong adoption intent. The survey did not try to capture what proportion of any farm was treated with GAPs, nor the quality or direct effectiveness of that application: previous survey experience indicated that the responses would have wide variation in accuracy, and tests of the survey instrument prior to use in this evaluation had shown that complicated questions became unworkable22. Instead, qualitative interviews with a small sample of smaller farmers (in meetings with their producer groups) and large or commercial farmers (by direct 1:1 interview) produced findings that were consistent with those of the MTE. Ad-hoc informal learning or spillover effects were detected in the survey responses and were confirmed in the interviews; this extended not just to learnings being transferred farmer to farmer within the project, but also affected some aspects of the comparison between beneficiary and control group, a methodological issue [see Appendix 2 for a discussion] that invites further research into knowledge diffusion patterns and techniques, to more rigorously test the hypothesis noted above. While direct correlation to immediate financial benefit could not be discerned in the income results, farmers themselves tended to report that they saw this extended adoption of GAPs as better preparing them against pests, diseases and other climate-related effects. 4.4 Farm Management Plan/Commercial Training and Adoption The project invested significant resources into complementing technical practices training and support with commercial training, marketing organization and mobilization to help farmers improve their negotiating position relative to buyers of RCN and suppliers of inputs. Figure 5: Farm Management Practices adoptions (combinations) Training themed on “cashew as a business” -- including the associated practices of keeping receipts, managing costs, planning the farm and improving the marketing of RCN -- was conducted for all farmer groups and supported with related cashew farm management planning materials. Larger farmers, especially the largest, received much more individual attention, with extensive consulting on business planning, including plans for related activities such as establishing and operating nurseries, irrigation and planning for value addition businesses such 22 Even then we were surprised to find anomalies; questions relating to reporting overall revenue figures for a season were later checked with a later series of questions on actual production and details of the iterations of prices x quantities sold. Frequently these did not match the overall revenue reported for that season; data cleaning identified simple recording errors, but overall, we conclude these reflect the challenges of recall-based survey 21 methods in this context. See Appendix 2 for further details and the partial mitigation steps available. 22 as small-scale processing or juicing23. Farmers again typically recalled receiving previous training in basic bookkeeping and organizing for sales from prior projects; but few of the interviewed farmers were able to demonstrate that they currently practiced previously-trained methods or techniques, beyond basic record-keeping. The survey results for adoption of such practices in MozaCajú (Figure 5 above) indicated that 15,559 farmers adopted at least one farm management practice (representing 89% of the 17,502 farmers that the survey data suggest had been trained in these practices. This meant that only 74% of the total project base had received such training, and the effective adoption generally was weak; simple cost calculation was sufficient to be recorded as an “adoption” (and 20% of farmers solely adopted this one practice). By comparison, comprehensive adoption of commercial practices was the exception rather than the rule: barely 11% of project farmers reported adopting all three of the promoted commercial practices. Considered in terms of the frequency of adoption of combinations of practices (again, looking only at farmers who reported adopting at least one practice), the contrast between beneficiary and control groups of farmers is pronounced, albeit from a low base (Figure 6 below): Figure 6: Frequency of Farm Management Practice combination adoptions (from most popular to least) As also reported in the MTE, field observations of supervisors, their field facilitator staff and their interactions with the village promoters suggested that the low (74%) training coverage and low adoption of trained business practices waslargely attributable to the limited commercial experience and capability of the frontline facilitator staff. Front-line field staff were typically young and trained in basic agronomic topics and community group organization. The MTE report had noted the lack of commercial training and job focus on commercial development amongst the field teams; this was essentially unchanged at the time of the Endline. In only three of the 17 farmer group or commercial 23 Two examples illustrate the range of venture developments: (i) Emaju acquired a disused cashew farm and abandoned small-scale processing factory. With the aid of the project’s grants and technical assistance, a small processing business and a medium-sized commercial farm have now been set up. But farmers may also over-reach; for example, (ii) Orlando Langa planned to complement his 20,000 tree farm and large nursery with value-addition via a juice factory; technical consultancy by MozaCaju led to a grant being arranged, perhaps too hurriedly; subsequently the farmer’s other business priorities impeded proceeding and the grant was cancelled (Section 7). 23 farmer visits conducted for this Endline analysis did we see the facilitator or supervisor produce any form of written records or other documentation to support the meeting. A similar number of farmers presented documents, in two cases in quite elaborate detail. From MTE to Endline, the field management focus appearsto have continued to be on the agronomic support. While the project provided annual support to farmers in the commercialization process specifically, this was an activity largely separated from the ongoing field support and was identified as solely a TechnoServe responsibility. The qualitative interviews with producer groupssuggested however that the (modest) reported rates of adoption of these commercial practices can be qualified with a more positive interpretation. Farmers interviewed cited the intangible benefits of being organized into selling groups by the project as an advantage, such as in increasing their confidence in selling. In particular, the farmers perceived that their negotiating capability was improved in relation to the buyers, stating price differentials not fully borne out in the numerical data24 but indicative of a greater awareness to look for alternative buyers and to check current prices. Nevertheless, Endline interviews with larger farmers and the findingsfrom focus group meetings with producer groups suggested that the lack of commercial skills adoption and farm planning left farmers vulnerable to shocks: for example, in not diversifying their activities sufficiently in anticipation that global cashew prices can fall rapidly, or that incomes can be severely affected by rising labor costs. 4.5 Does Longevity in a Project Matter? Another hypothesis offered by project staff and assessed as part of this evaluation was that improvements in performance by farmers might show a relationship between length of time in the project and the observed year-to-year change. Farmers were registered beginning late 2013 in small numbers, with the bulk joining in 2014 and the final 8.5 thousand in 2015. As seen in Table 5, the data are suggestive that there may be positive effects of more sustained experience and access to the project’s services and support. Treated as cohorts, the year-to-year [YTY] change in productivity [“Prod”] indicates some differential effect for the earliest cohort, particularly in Zambezia and Cabo Delgado. For example, this is mirrored in the relatively higher application of GAPs and higher change in year￾to-year revenues [“Rev”] for the earliest members in those two provinces. In interviews, farmers reflected that repeated trainings and the improved access to spraying resources and services progressively increase their confidence in farming and help them deal with setbacks. This was also reflected in the higher rate of planting of new seedlings by farmers with more than one full season of project support. Table 5: Longevity Effects (duration in the project) Summary: YTY Prod Cabo Delgado Nampula Zambezia Cohorts 2013 72% 54% 114% 2014 28% 28% 42% 2015 48% 43% 29% Summary: GAPS Applied 2013 3.1 2.7 3.6 2014 2.9 3.2 3.4 2015 2.5 3.2 3.1 Summary: YTY Rev (MZN) 2013 224% 143% 138% 2014 145% 143% 112% 2015 139% 92% 72% 24 A similar bullishness beyond the empirical price data was reflected by the commercialstaff of MozaCajú who cited price differences of “up to 30%”. Our best estimate is a price difference of 3-4.9Mts/kg on average, broadly consistent with the 5% differential detected in the MTE findings (2016 prices averaged 55-60Mts/kg in the three provinces of interest; 5% equates to 3Mts). The lack of good quality market price data, and the lack of a project 24 ‘antenna’ for collecting such data routinely, contributed to ambiguity persisting for all actors. 25 These findings are suggestive of a strong relationship between extent of exposure and potential learning effects, but could not be reliably demonstrated with this current dataset. A longer time￾series of data would have been useful to analyze the relationship and reduce the potential selection bias. In the project, recruitment was continuous, blending the initial two cohorts given the late operational start of the project. Initial farmer recruitment may for example have been of “low hanging fruit”, skewing the results. More longitudinal data could have isolated this effect. The short project time series impeded the emergence of statistically significant effects in the project generally; in addition, the results suggest that multiple factors are in play. Nampula for example presented relatively weaker indications of a linkage, potentially due to higher incidence of turnover in MozaCajú field staff and the long management hiatus that occurred at the midpoint of the project. These factors suggest that interesting hypotheses about relative learning-based impacts of different multiyear designs, and their relation to costs per GAP adoption, should be tested in a future project. 4.6 Sales Volumes and Revenue In this Section the experience of the farmers in the project is analysed in terms of their results relative to theirstarting position at project inception and at the mid-term period; in the following Section 4.7, the counterfactual, i.e. performance relative to that of non-participants, will be discussed. 4.6.1 Year-to-Year Project Performance In the final commercialization season for MozaCajú’s field activities, the project’s typical farmer generally saw a solid improvement in both their average yields and in prices (in both dollar and metical terms25) for cashew sold. The project’s farmers experienced an increase in income versus the MTE result of +41% in local currency and +28% in USD, driven by a combination of MZN market price increases and increased volume of RCN produced and sold. Our analysis of farmer income was made by comparing the project Midterm average revenue for project beneficiaries of approximately $363/farmer; in the finalseason, the corresponding figure was $464/farmer (and because direct costs are typically negligible, this represents a 28% improvement in income – see Section 4.6.3). As with the MTE, these Endline revenue figures are based on conservative assumptions: we adopt the INCAJU reference price in calculating these 2016-17 revenue figures for attribution to the project26. Initially one aim in the evaluation was to provide a full three-year statistical analysis for the project’s farmers, but this proved infeasible for two reasons: (i) year-ago recall data proved to be low-quality, with high rates of nonresponse to key questions and great variability in responses, including contradiction; and (ii) the late start of the project meant that 2014-15 production and commercialization were barely affected in many communities. Because of these concerns about the quality of both the baseline data and the survey-based recall data for year one of the project (2014- 15 season), plusthe effects of late commencement of implementation, the above revenue attribution for the project is based on solely on two years of performance, namely the MTE and Endline years (2015-16 and 2016-17 seasons). This is consistent with the approach taken in the MTE report. 4.6.2 Life of Project Performance Our conservative estimate is that over the life of the project, the beneficiary farmers earned gross revenues of $16.7 million27. Due to the late start of the project’s operations, no revenues were attributed for year one of the project, 2014-15, in this calculation. For the second year of the project, 25 Average exchange rates used per commercialization season: 2015-16 season: USD1.00:MZN 43.92; 2016-17 season: USD1.00:MZN 70.88 26 INCAJU weighted average reference price for 2016-17 for the three provinces: 56.4Mts. The survey results gave an average 61.5Mts/kg; we selected the more conservative figure for the YTY change in beneficiary farmer revenues, consistent with the approach in the MTE. At 583kg/farmer median sale, revenue is 32,881 Mts (USD 464) at this lower price (used in the attribution calculation); or 35,906 Mts at the claimed price for 2016-17. We take the lower. 27 See Section 1.2 above and footnote 4 for pricing assumptions used to derive a conservative estimate. 26 the MTE had reported farmer revenues of $5.7 million (based on the more conservative of two calculation approaches discussed), for the large subset of farmers that had received support for at least a full season, i.e. for the 2015-16 year. In the final year of the project term (2016-17) the full farmer cohort generated a further $10.97 million. The project's farmers sold 13,788MT in the 2016- 17 season, an average of 583 kg per farmer earning an average of 32,960 Mts, compared to a per￾farmer average of 431kg per farmer earning an average of 15,614Mts in the MTE result for 2015-16. This was calculated based on survey findings of RCN volumes for that sample, (3.6% of the project population), extrapolated to the population for project total production, and multiplied by the weighted INCAJU reference price, which was 5-8% lower than the self-reported (but typically undocumented) responses given in the survey (i.e. averaging 61.5 Mts/kg). Table 6 summarizes this calculation in extrapolating from the sample to the entire project: Table 6: Farmer Total Gross Revenues – 2016-17 Year Sample production Sample (3.6%) extrapolation factor to project population Project Total Production 2016-17 (est.) Weighted (INCAJU) Average 2016-17 Reference Price Total Revenue LCY Total Revenue in USD (2016-17 e/rate 1:70.88 ave.) 496,006 Kg 27.8X 13,787,681 kg 56.41 Mts/kg 777.5m Mts 10.97m 4.6.3 Farmer Investment While not a formal target for the project, cost data were collected in the surveys to try to gauge profitability, and this was discussed at length in the qualitative interviews. The findings present a picture of relatively very low investment in the sector. The notable exceptions are most clearly seen with some of the larger farmers (”large” or “commercial” segments) who were investing in related services such as spraying businesses or in clearing land for new plantations. • For instance; a large farmer, with over 1,200 productive trees, the farmer has invested in a tractor, vehicles and seven spraying machines; • The survey also found a commercial scale farmer planting 20,000 trees from his own nursery in collaboration with the project and planning several value addition businesses. Beyond these few exceptions, the average spending on labor or other supplies by any farmer is modest, largely due to the unavailability of working capital. Of those farmers that reported any spending for the most recent season, the average outlay was around 3,700 Mts per farm – this was equally true for beneficiary farmers and for control group farmers. The difference lay in frequency: as noted in Section 4.2, farmers in the project were six times more likely to make these outlays than neighboring farmers not in the project. The interesting question was why. While no correlation to immediate benefit could be detected, many farmers themselves reflected that they recognized their overall resilience and capacity for expansion were improved by making investments in improving the tree stock and in taking preventative steps to reduce their risks (e.g. fire breaks). Costs are high in relative terms, however, as noted in footnote 16, Section 4.2 above. Even for the larger farmers that spent for production, the outlays overall represented about 10% of their average gross revenue from cashew farming, paid with RCN or in cash once revenue was subsequently realized at season-end. 4.6.4 Performance by Farmer Segment One of the strengths that TechnoServe brought to the project was its long experience in the commercial aspects of value chains and the improvement of the position of smallholder farmers. These aspects included the ability of farmers of all sizes and levels of organization to access competitive markets, to produce more sellable output for those markets, and to negotiate better prices, along with a crosscutting concern for addressing gender imbalances in these activities and in related aspects such as access to inputs and credit. Taking a view over the life of the project, average project farmer annual revenue increased 66% from an adjusted baseline of USD 280 (see footnote 31, page 26) to USD 464. The major drivers of 27 this revenue increase were: (i) a 35% increase in average production in the latter years, including an increases in harvested yield per tree totalling 53% over the baseline for the project and measured as 14% since the MTE28 itself, (ii) a 7% increase in the “productive” tree stock count, some of which remained immature, and (iii) an average (USD) price increase of 15% earned in the final 2 seasons. Sold volume per tree increased at a similar rate, the balance being lost through post-harvest handling and storage problems (16-20% stock losses are common), own consumption of nuts, and theft. Breaking this down by farmer segment level as defined by the project, the findings suggest that in financial terms commercial farmers tended to benefit more than other segments over the duration of the project. Through the survey, smallholders reported 2016-17 prices +80% higher in local currency than the overall average of the year before, whereas Commercial farmers reported average prices 114% higher than the year-ago MZN price for the equivalent period; the average difference being 12 Mts/kg. These simple averages included those farmers who did not sell; when the lower participation rate of smallholders in completing sales is taken into account29, this difference halved to around 6Mts/kg for the 92% of small farmers that made sales, i.e. excluding zero sales/price results. Of that 92% of small farmers, reported prices varied widely: even excluding outliers, responses ranged from 28Mts to 100Mts per kg, whereas by comparison larger farmers had more stable and typically higher prices. Location also mattered: the highest prices were reported for the early weeks of the season when supply was limited, especially in the north of the country where foreign buyers flooded into the districts as global prices peaked for short periods30 (see Figure 7). Figure 7: Price trends in 2016-17 commercialization season by time - source: INCAJU 2017 One positive finding is that revenue improvements seem potentially sustainable, (assuming prices remain relatively stable), based on the higher rate of new planting engaged in by the Project farmers. The 65% of farmers who planted in 2015-16 added 59 trees each that year, 3 times the target rate. 28 The average harvested yield per tree over the project is reported as increasing by 53%, but this is inflated by the low baseline of 3.2kg; relative to MTE (2015-16), yield per tree increased by 14% in the final year, from 4.3 to 4.9kg. 29 6% ofsmallholders and 3% medium reported selling no product during the season, whereas almost all of the larger farmers recorded sales. Lack of sales was mostly attributed to fire (including one large plantation), disease or theft. 30 In an interview with one major foreign buyer (SKN), the principal reason in the 2016-17 year was a poor harvest in Vietnam, creating factory utilization shortfalls. Local processors confirmed this heightened level of competition. 28 As noted, the survey was susceptible to data quality issues, and steps were taken to improve that quality (see Appendix 2). Given the short time series and lack of written farmer records, figuresshould therefore be understood in terms of ranges rather than absolutes. Nevertheless, combined with the qualitative findings, we conclude that the evidence suggest that commercial farmers are (i)able to negotiate a price above average; and (ii) much more likely to produce sellable harvests and protect it post-harvest. It is noteworthy also that Zambezia (representing 13% of the farmer sample overall) had zero commercial farmers for inclusion in the survey and reported lower prices across theboard, reinforcing the conclusion that size of selling cluster, distance from foreign buyers and the timing of the harvest combine to create a significant disadvantage for some geographies. Taking a longer view for a directional assessment cognizant of data quality, the MTE findings indicated that in all farmer segments the beneficiaries achieved substantial revenue growth in local currency terms over the three years from the baseline period (2013). Extending this same series out to a four-year analysis by including 2016, the Endline findings revealed a large uptick in local currency revenues driven by intensive buying by international buyers at prices often in excess of one US dollar per kilo of RCN, even as the Metical almost halved in value with a precipitous devaluation in 2016. As reflected in Table 7 below, reported prices varied significantly according to farmer segment in the most recent year (noting again that commercial farmers are primarily found in Nampula province and are well known to the processors and formal/informal buyers based there). Working back from 2016- 17 prices, the table eliminates exchange rate effects to show that the average 2016-17 price was 1.74 times that for 2013-14 (baseline year) in USD terms (i.e. $0.87/kg in 2016-17 versus $0.50 in 2013- 14), representing an increase of 3.1 times in local currency terms over that same period. Table 7: Sales Results and Prices - per Farmer segment and gender (Endline survey data, self-report basis) Adjusting for exchange rate, the devaluation over four years accounts for a 115% change in nominal revenues; e.g. a “large” farmer earning 22,000 Mts on average in 2013 needed to be earning 47,000 Mts in 2016 to be standing still. However, the survey findings showed that latter result was in fact almost doubled, reaching 93,000Mts on average, with male “large” farmers achieving more (+5%). As noted, the largest farmers secured higher prices and much greater average revenues (Table 7: all in local currency). They were the first sellers to be approached by the foreign buyers, obtaining prices as high as 90 to 100 Mts/kg in the early phases of the 2016-17 procurement season. The results also indicated continuing improvement in revenue generally for the middle segments of farmers (those Segments: Ave. Mzn/Kg 2016 Ave. Mzn/Kg 2015 Ave. Mzn/Kg 2014 Ave. Mzn/Kg 2013 Commercial 70.59 33.18 23.39 16.86 Large 61.56 33.42 22.39 15.94 Medium 63.38 33.92 23.07 14.91 Small 59.63 32.77 23.16 14.71 Weighted Ave. 61.55 33.13 23.05 14.94 (@ 2013 e/rate) 25.85 22.45 21.62 14.94 Ave RCN P in $: 0.87 0.75 0.71 0.50 Note: Pricing and farmer revenues trended up, in MZN terms; four-year changes were lowest for the smaller farmers but remained positive for all segments. 29 segmented as ‘medium’ and ‘large’). Gender featured as a factor for the next-to-largest farmers; the few farms owned by women excelled at the top end but enjoyed lower rates of growth in the “large” category, with lack of access to capital given as the major reason by project staff. By contrast, small farmers enjoyed substantial gains in the three-year picture to 2016, but their averages stood still at best in the final year of the project, in local currency terms. We could identify no specific reason for this disparity, it seems due to a combination of factors. Recall-based data introduces issues of data quality; this is often heightened with small farmers, despite the majority being organized into groups – this is due to the lack of reliable records, plus the reality that many are only ‘occasional’ cashew farmers with other sources of income from other cash crops and casual labor, making recall less clear. Of the small farmer segment, 5.4% reported no sellable production at all (with 4.1% of the project’s farmers overall reporting no sales), a decrease on the previous survey where 8% of small farmers made that report31. 4.6.5 Sustainability During the no-cost extension period, MozaCajú personnel conducted one final round of commercialization trainings in August and September 2017 in advance of the final commercialization season. These trainings were attended by selected cashew farmers and promoters, INCAJU technicians and, in the case of Mocímboa da Praia, also attended by Customs agents and representativesfrom the National Parks. In addition to refresher training on group commercialization practices, the theme of improving quality and its analysis at point-of-sale was introduced for the first time in the project. This latter theme could have been developed and implemented earlier in the project; a constant criticism from processors is that the moisture content of locally sourced RCN is too high, increasing their costs. The commercialization support team were not able to furnish fully satisfactory data from the results of their monitoring of the final season in the initial project; where absent, these have instead been inferred from the survey data. The low capital intensity of typical cashew farming as practised in Mozambique means that almost all costs are borne in local currency, except for those actors acquiring imported atomizers, tractors, irrigation equipment or other assets. Interviewed farmers were very enthusiastic about expanding their production, particularly those in the northern regions that had experienced the most intensive foreign buying and the highest prices. Thus, most farmers who had productive years in 2015-16 and 2016-17 seasons and who had managed to sell at good prices, experienced a windfall result that led some to state they would invest significantly in expansion (while others interviewed were more intent on improving their housing, buying a second vehicle, or other uses of the receipts). The risk ofhubris came to mind in some of these discussions; most interviewed ‘commercial’ farmers and almostall the smaller farmers interviewed had no business planning tools for making appropriate decisions including building reserves for when markets are less favorable (as noted in Section 4.4 above). 4.7 Comparative analysis MozaCajú acknowledgesthat the initial consulting project that developed the baseline for the project was inadequate32, with some key data later being adjusted based largely on empirical experience of the project management. One aim of the current evaluation was to determine whether we could identify differential revenue effects for the farmers in the project relative to their peers in the neighboring areas. Ideally this would be analyzed descriptively and any correlations between a specific project activity and its revenue effects would be identified, against which statistical tests of 31 Fire, pest attacks, theft and the aging of tree stock were cited by staff and interviewed farmers as the causes, with a higher number of trees being affected by fire especially, according to interviews, in the final year of the project. 32 For example, the baseline analysis gave an average farmer revenue figure of USD 104.52 collected during the 2013/14 campaign; management identified that as being too low for a baseline, with the consequent risk of exaggerating project achievement, and adjusted it to the INCAJU reference figure of USD 280 per farmer. 30 significance could be applied. As set out in Appendix 2, the learnings from the earlier conduct of the MTE farmer survey and subsequent analysis were used to inform the design, selection of consultant company, data collection tools used and validity checks to be applied in the Endline analysis. As the Methodology discussion describes, this proved challenging in practice, largely because of fourfactors: (i) the lack of M&E data on commercial results in the periods between the MTE and Endline to inform the latter; (ii) difficulties with satisfactory execution by the hired survey consultant companies; (iii) flaws in both the design and implementation of the questionnaire; and (iv) the compression of timelines to collect the data, inadequately managed in the data collection process. While all attempted correlation analyses failed to produce a usable (significant) result relating projectoutputs to income effects, some comparative observations can still be made. This is set out below. 4.7.1 Challenges It is useful to first briefly recap what we do not know. INCAJU as the responsible government agency aims to provide diligent levels of service to a sprawling, widely diverse farmer base; but its severe resource constraints mean that each of its 68 extension agents in the three targeted provinces notionally supports around 12,000 farmers with very little logistical means to reach those farmers. Thus, to a large extent farmer activity is supported in a sub-optimal way (e.g. in the distribution of inadequate amounts of chemical) and the monitoring of that activity and its commercialization is largely inductively derived from reports filed by buyers and from fragmented data collection in the producing districts. Even when data collection is the focus of an initiative, it is very challenging. While the project could attempt to monitor as much as 50% of the project’s farmer commercialization activities (as recorded for the MTE; but less than 40% for the Endline), the bulk of the performance data are self-reported via the survey, and farmgate activity for surveyed farmers outside the project was entirely unmonitored. The MTE survey process had encountered similar, significant issues of data quality, in particular in relation to the control group sample (see Appendix 2). While these caveats have been noted in Section 4.2 relating to how to interpret the consistently higher adoption of both GAPs and basic commercial practices by beneficiary farmers, farming practices are at least to some degree observable as a form of potential validation. The survey self￾reports from the project’s farmers were broadly consistent with the anecdotal interview evidence and the ongoing observation of management and staff during the course of the project, as declared in interviews with the staff and indicated in project documentation – for example, in records of seedling distribution and the planting experience of certain closely watched larger farmers. Behaviorally, then, many of the output-level results in terms of training and practice adoption were as anticipated. While all farmers in the sector had been trained at some time on pruning, cleaning, fire control etc., it seemed reasonable from field observations and the survey that organized efforts during the project had doubled the relative rate of adoption by beneficiary farmers. By comparison, while commercial practices were actively promoted by the project, albeit at lower levels of intensity, they were much more difficult to empirically observe, as were their actual transactional outcomes (prices and volumes sold); instead, our findings largely rely on survey responses. Control group farming practices could potentially be checked through anecdotal evidence of behaviors observed by project field staff, INCAJU and SDAE agents and other actors as they moved through communities; these data were collected from interviews. However, any comparison with the control group requires the disclaimer that little or no systematic observational evidence had been collected to validate the self-reports from the survey responses by the control group. This is especially acute in the case of revenue where the multiple concerns and motivations relating to disclosing financial information may affect responses for events which are largely unobservable. Reliability is therefore problematic: control farmers are unknown to the project 31 and its enumerators, and conversely the project and the person bearing the questionnaire form are usually unknown to the control respondent33. In the Endline survey findings, the quantitative data suggest at face value that farmers across the sector, on average, saw two good seasons -- with yields, total production and prices improving year on year; this is validated by official reports (INCAJU, 2017). The quantitative analysis also indicated that no discernible difference could be identified between the improvement in production of farmers in the project and the sampled farmers outside the project; in the two focal seasons of 2015-16 and 2016-17, increases in yields per tree were effectively indistinguishable between beneficiary and control. This was typically explained by farmers and staff as being due to the multiple factors that affect production with a cyclically-flourishing, rain-fed crop such as cashew; only some of these production drivers were potentially manageable within the scope and resources of the project. 4.7.2 Revenue Analysis The project farmer sampling frame was designed to enable a longitudinal comparison with the MTE, augmented by questionnaire responses from farmers who joined the project later to deepen the quantitative data set (see Appendix 2, Section 2.2). This aimed to provide a proxy three-year time series. The summary results are given in Table 8 below: Table 8: Revenue per farmer: Beneficiary versus Control (Endline survey data, self-report basis) Farmer Revenue Beneficiary Control (Mts/farmer) 2014 2015 2016 2016 (adj.) EL vs MTE 2014 2015 2016 2016 (adj.) EL vs MTE Overall Average rev. 11,020 6,000 15,968 32,960 20,426 28% 7,045 3,000 12,725 42,466 26,317 107% Median rev. 7,436 14,500 8,986 21% 6,800 12,706 7,874 16% Excluding Average rev. 8,762 5,000 11,742 23,738 14,711 25% 4,242 2,400 10,520 20,489 12,697 21% top outliers Median rev. 6,900 13,600 8,428 22% 6,200 12,000 7,437 20% (2016 adjusted: 2016 values normalised to MTE exchange rate (USD1.00:MZN44) for Endline versus MTE comparison; 2014 figures unreliable) Table 8 presents the summary revenue findings, set out as a three-year series based on the MTE and Endline surveys. Figures for 2014 are for reference but are subject to the concerns noted earlier – they are 12-14 month-old recall responses. Because of the inherent unreliability that introduces, we chose to use as 2015 figures the findings from the MTE rather than base it on recall in the Endline survey. The overall average revenue for beneficiary farmers, calculated using INCAJU reference prices, was an average of 32,960 Mts in 2016 (or 20,426 Mts in 2015 exchange rate terms), representing the 28% year-to-year change in revenues expressed in USD terms noted in Section 4.6. On an overall basis, the figures for 2016 indicated a significant skew that results in beneficiaries being initially recorded as having lower average revenue per farmer than the control group in the final year. Conversely the overall median revenue per farmer represented a 14% higher result for beneficiary farmers relative to control. The principal explanation for this disparity lies in two factors: (i) Different rates of participation in commercialization: control group respondents consistently reported 1.5 to 2 timesthe rate of non-participation in commercialization (i.e. no sales) compared to the project’s farmers(Table 9). Non-participation occurred predominantly with smaller farmers, muting the impact on averages but being reflected better in the median results -- e.g. in the 2016 figures, the median income overall for beneficiary farmers is 14% higher, and grew 21% between 33 Some exceptions were noted; 14% of control respondents named MozaCajú as a project they were aware of, and nearly 5% stated that they had received training from MozaCajú; the latter we interpret as a combination of hearsay learning [spillovers] and perhaps opportunistic joining of training sessions, consistent with the fluidity of people movement and extended family memberships in rural communities. 32 2015 and 2016 (versus 16% for control), representing a benefit for project participants despite the skewed average across the whole sample, distorted by a small number of very large farms. Table 9: Non-participation in commercialization (MTE and Endline survey data, self-report basis) 2014 2015 2016 Beneficiary Control Beneficiary Control Beneficiary Control No Sales (%) 4.3% 8.7% 10.5% 27% 4.1% 6.6% (ii) Selection bias effects: the control group selection problems discussed in the Methodology Appendix 2 (i.e. biasto larger farmers)strongly skewed the overall averages, despite data cleaning. Standard deviation differences support this hypothesis of selection error34, when we instead exclude the top 5% of results for both beneficiary and control data sets, the average and median difference for beneficiary farmers relative to control group farmers becomes +16% (average) and +13% (median) respectively. These results suggest an overall benefit for the typical project participant, with slightly higher year-to-year revenue growth rates (25% and 22%, for average and median revenue respectively). Given that production volume showed effectively no difference for all farmers, the differential revenue result would therefore be driven by differential prices. This conclusion was consistent with the finding in the MTE that price improvements for beneficiary farmers were evidenced; these continued to be suggested in the analysis of the final year of activity in several ways. The project’s farmers reported prices that they believed were slightly better than prices obtained by non-participating neighbors. Survey responses indicated an average price of 61.5 Mts/kg for beneficiary farmers; the commercialization monitoring activity in 2016-17 suggested an average of 59 Mts/kg, and in interviews with commercial or large farmer a price of around 60 Mts/kg seem to be a consensus. Interviews with farmer producer groupsindicated that beneficiaries believed they achieved better average and median price figures due to better organization for selling; those qualitative investigations suggested a 5-8% positive price differential to market rates (as represented by INCAJU’s reference prices), equivalent to an additional 3-4.9Mts/kg. Our conservative quantification of the difference is based on taking the lower price differential (“low case”) of 3 Mts/kg (approximately 5% of sales price) and applying it to production volume sold. For the average beneficiary, that represented the increase in revenue to 35,832 Mts; or a benefit in local currency of 3,042 Mts/USD 54 per farmer over the two principal years of the project (Table 10). Table 10: Revenue Differential (average per farmer): Two Views Difference 2015-16 (MTE) 2016-17 (Endline) Two-Year Benefit (Average) Volume (kg) Price delta Volume (kg) Price delta Beneficiary Revenue ‘High case’ 431 4.9 (Mts/kg) 583 4.9 (Mts/kg) 1,014 x 4.9 4,969 Mts (USD 88) ‘Low case’ 431 3.0 (Mts/kg) 583 3.0 (Mts/kg) 1,014 x 3.0 3,042 Mts (USD 54) The final-year benefit, taken as an annualized run-rate of differential revenue accruing to the population of beneficiary farmers, amountsto USD 579,20035. The qualitative indicators noted earlier suggests that many farmers would continue accruing such differentials by remaining organized and 34 The skewing caused by the unplanned, convenience sampling of particularly large control group farmers is seen in the standard deviations for the overall dataset averages of farm sizes: e.g. Beneficiaries SD: 927 productive trees; Control SD: 1,633 productive trees (i.e. 1.75x). 35 Calculation: 3 Mts/Kg x 583 Kg/farmer in final year = 1,749 Mts or ~USD 25; USD 25 x 23,556 farmers = USD 580K (all figures rounded) annually and potentially recurring. 33 continuing to invest in the use of GAPs techniques and procure inputs, in particular in response to continuing high market demand. Section 5 describes the findings relating to inputs use. 5 Component findings: Improved Inputs The project focused on two areas of inputs; the rationalization of chemical access and use; and the supply of improved seedlings and polyclonal seeds. 5.1 Chemical spraying: Inputs supply shortages are a chronic issue in the sector, with anti-fungal chemical spray to combat white powdery mildew the first demand by most farmers. In the absence of significant private sector activity in providing chemicalsupplies (an area that project management have argued requires policy reform to provide farmers with choice), the project worked pragmatically with INCAJU to rationalize the distribution of chemical and to demonstrate that better application improves cost-effectiveness. The Endline study confirms that the project is widely acknowledged to have facilitated increased access to chemical spray for the project farmers. For example, 79% of beneficiaries in Cabo Delgado reported having applied chemical treatment versus 58% in the case of non-beneficiaries, the control group; in Nampula province, the difference was 89% versus 78% respectively; whereas in Zambezia, no differential result was reported. Anecdotally, the project farmers also applied the chemical more appropriately, and this would have contributed to the 53% yield increase per harvested tree (albeit from a low baseline figure) recorded overall during the life of the project. Observationally, even the larger farmers sometimes displayed confusion, however, about what the appropriate application rates and timings were, reinforcing the need for periodic retraining and a more comprehensive analysis of yield issues. These concerns of reliable supply and appropriate timing and dosage become even more pronounced if the national strategy mentioned in Section 1 is to be achieved, trebling the tonnage of RCN production over the next decade. TechnoServe management believe this will require Mozambique to develop the capacity to spray 15 million trees per annum, a 5- to 6-times increase in the current level that will require structural change that enables private sector entry for supply and more accessible credit services for procuring inputs. As noted in the production component findings, the correct application of antifungal spray was one of the promoted GAPs. In the majority of cases, regardless of designated segment or their on-farm resources, we observed a very high reliance by farmers on free supply of chemicals36 and seedlings in their activities, together with some willingness to pay for essential related services such asspraying (but seldom for the chemical itself). INCAJU policies that emphasize the subsidization of these materials seem to have crowded out any attempts by the private sector to offer better supply of inputs to Mozambican cashew farmers. The initial project proposal did not fully anticipate these issues relating to inputs and therefore did not include an advocacy component whereby the project would work to encourage INCAJU to phase out these subsidies, at least for seedlings. This was later recognized as a distinct need, and some progress has been made; it required a long-term policy advocacy effort that had been deemed to not be feasible within the project timeframe. As Section 4.3 also showed, we also observed that combinations of GAPs are more effective. Where spray is made available, the Endline analysis reveals that beneficiary farmers are twice as likely to use chemical in association with other complementary GAPs, which in turn should produce a higher 36 INCAJU provides the physical chemicals but not the spraying equipment, fuel or services that are needed for application. Similarly, large numbers of seedlings are distributed, but it appears seedling mortality is as high as 50% due to lack of support services, irrigation and good handling in transit. 34 return on investment by avoiding other issues. For example, as Table 11 indicates, 17% of the project’sfarmers combined spraying (item ‘5’ in Table 11 above) with the full suite of GAPs, while only 5% of their counterparts outside the project reported during the same. Table 11: GAPs combinations featuring spraying: (Endline) Across an entire population of farmers, this may have significant economic implications in terms of the cost and benefit of cashew farming and the ultimate RCN price (and its quality) to the processor or exporter. The complementary activity in promoting small businesses to invest in sprayers was significant, but remains a fraction of what is needed nationally. MozaCajú facilitated access to 219 sprayer machines for service providers along with training. These machines have the potential to treat 328,500 trees with protection against pests and disease, benefitting 5,566 farmers. Nampula province is estimated to have around 1,700 spraying machines currently37, many in poor repair, with a need for probably ten times that number to service the tree stock adequately. Supporting 85 nurseries, 5 inputs providers and 219 spraying services providers meant that a total of 309 inputs services businesses were assisted in establishment by the project. 5.2 Seedlings and Polyclonal Seeds: MozaCajú developed and supported a network of distributed, locally-owned and operated cashew nurseries as a trial to demonstrate the feasibility of constructing and operating an alternative source of supply of sustainably sourced cashew seedlings. The network of 85 community-based nurseries across the three northern provinces (18 in Cabo Delgado, 56 in Nampula and 11 in Zambézia) was designed to locate nurseries close to farming communities and at a distance from established INCAJU nurseries. This was intended to ensure that a wider set of farmers could be supplied and commercial sales of seedlings could be experimented with. Proximity to an INCAJU nursery, it was believed, could confuse farmers and any attempts to test a commercial model would be frustrated by the attraction of the free supply from the INCAJU nursery. The fledgling nurseries were set up in consultation with capable local farmers and their communities; the project trained the owner/manager in seedling production and nursery management, and help them train their staff. The project also provided 85 micro-grants of materials and equipment, valued at approximately $850 each or $72,000 in total - for example plastic pots, seeds (including polyclonal seeds), shovels, pruning shears, wheelbarrows, knives, hoes, and water bottles. The project then acquired most of the output at the end of each growing season, creating some incentive for reinvestment but also a dependency on the project that was mitigated in the no-cost extension year by ceasing procurement. The production results and first-hand observations during the evaluation suggest that a good skills transfer has occurred across the seedling program. While most nurseries remained at a modest size, producing up to 5,000 seedlings per year, two nurseries demonstrated the capacity to produce more than 15,000 seedlings per year. The project also acted asthe destination for the seedlings; acceptable seedlings were procured by the project for distribution, and essential materials were refreshed for each new season. In addition to this quasi-market, MozaCajú also encouraged nurseries to try to develop a local demand, which in a few cases resulted in some sizable selling activity (for example, one nursery supplying 5,000 seedlings to DDI, a project grant recipient). As noted in the MTE report, it continued to be the case that none of the visited nurseries showed a good sense of their true costs 35 37 Source: discussion with Dep. Tecnico S. Dario, Agrifocus, Nampula 2017 36 or had a business plan for becoming commercially viable; production volumes however remained on target, and by the end of the project’s production of 542,807 seedlings exceeded the target by 35%. In the 2014-2015 season, 162,000 seedlings were produced, followed by 307,000 in 2015-2016. An additional 73,119 seedlings were produced over the course of the 2016-17 planting season; while this latter production sits outside the official timeframe for this activity (e.g. seedling production in relation to the target was to be completed by the end of the 2015/16 planting season), these 73,119 seedlings represent additional contributions made by the project and have validly been counted in the overall total of 542,807 achieved. These supported 5,500-9,000 farmers to plant significant new areas of plantation depending upon take-up rates and distribution logistics (the higher figure equating to the 59 trees planted by 65% of farmersin the final year of the project). At a broader level, the project’s farmers overall achieved a weighted average of 33 tree plantings per year, double the target of 15.6 new trees per year (see footnote 15). The widespread adoption of bokashi (sidebar), promoted by the project,supported highersurvivorship rates helping farmersimprove their planting. Following the final year of the project, in the no-cost extension period MozaCajú supported the 14 most capable nursery owners with further technical and commercial assistance, so that they could assess whether to continue in operation. These locations were as follow: 3 (Cabo Delgado), 5 monitored by Queiroz (Nampula), 5 monitored by Lupatiua (Angoche). The initial emphasis on grafted seedlings transitioned to stronger interest in polyclonal seeds, developing both a policy discussion and a future role for distributed nurseries as part of the ongoing collaboration with INCAJU that now continues beyond MozaCajú. Before the end of project, MozaCajú and INCAJU worked to train 10 seed providers on these production techniques, and to also negotiate with two processor organizations that had begun distributing free seedlings in areas near the continuing project nurseries, to encourage supply from those local sources and to align messaging about the purchase of inputs. Maintaining a positive relationship with INCAJU ranked as a high priority in the conduct of the project and interviews with several INCAJU executives and field managers in the provinces confirmed that a good level of dialogue and collaboration was maintained throughout. Nursery operation was one area where some divergent viewpoints were expressed by INCAJU personnel. In two interviews held 12 months apart, one Provincial Delegate expressed dissatisfaction with the initial planning and coordination regarding the location of MozaCajú community nurseries, stating that some were sited too close to existing nurseries, while others were less effective by being sited in overly-remote locations. Even in the closing stages of the project, that senior official remained sceptical that community nursery managers would have the business mentality, and smallholders would have the technical skill to use bokashi, to sustain the initiatives established by the project. Other INCAJU managers were more comfortable with the nursery trials, and this is evidenced in the close collaboration between INCAJU and TechnoServe in the planning for a seedling tracking module for CommCare and its current realization in the follow-on ConnectCaju Project. We conclude that the relationship is sound and has progressed further in post-project activities – on the policy front and in projects such as ConnectCaju for ICT expansion – but initial MozaCajú implementation pressures had impeded some elements of that collaboration in the early stages. Bokashi: multiple use input ‘Bokashi’ is an organic material used to fertilize plants. MozaCajú taught nursery owners how to make bokashi using only materials locally available, improving the development of seedlings and protecting them in transportation to farmers; its light weight enables a 3X increase in seedlings that can be delivered on a motorbike. Every nursery visited was actively producing their own bokashi and farmers were observed learning how to adopt it themselves from the local nursery technicians. 37 In summary, the Endline confirmsthat MozaCajú supported farmersto apply chemical in combination with other GAPs at better than double the rate of application of farmers generally. We also conclude that 85 cashew nurseries benefitted at least 5,566 farmers through seedling distribution at standard allocations per farmer, for a sum of 542,807 new cashew trees planted in local communities (135% of target). This contributed improved seedling and polyclonal resources that helped the project farmers to report having planted an average of 33 new trees per year over the final two years of the project, double the target rate for the project. 6 Component findings: Traceability and Innovation 6.1 Progressing towards traceable cashew with batch processing MozaCajú conducted food safety audits inside five factories (3 Condor, 2 ETG) in order to identify obstacles that impede certification. Food safety management involves applying and integrating the policies, procedures, practices, controls, and documentation that ensure that the food sold by a business is safe to eat and free from contaminants. Traceability supports this: from the first step of purchasing RCN from farmers all the way to the final 50-pound box of processed, branded export kernel. Extensive factory investments of personnel and time were required for the audits. Four factories passed the food safety audits within the project timeframe. Basic traceability requires factory management and documentation systems that allow processing facilities to keep cashew sourced from a single district segregated throughout the various stages of processing. This is particularly challenging as in Mozambique smallholder farmers typically sell their RCN to a local trader in the informal market, and these traders in turn aggregate the RCN up to several times before selling to processors. Improvement depends heavily on warehouse organization, well￾designed records and registers, and the commitment of staff and management. MozaCajú trained senior management and factory supervisors on the various requirements for batch processing. Following this, MozaCajú technicians worked with these factory supervisors and managers to incorporate and adapt the new processes into the existing factory management and documentation systems. MozaCajú technicians could then conduct training sessions for all the workers to learn the various batch processing components and to create systems for recording and tracking. By project-end, six participating processing facilities had implemented new technologies and practices to establish batch processing systems, investing in the reorganization of plant processes and equipment that enable traceability to the district level within their supply chain, setting up a foundation for development of the systems and skills needed to implement traceability throughout the entire supply chain. This broadly represents about 60% of cashew volume being able to be traced to the district level within the factories supported directly by the project. 6.2 Ensuring food safety through HACCP certification MozaCajú worked with Mozambican processing facilities to help them install food safety management systems that are compliant with internationally recognized standards. A food safety management system includes the policies, procedures, practices, controls, and documentation that ensures safety and freedom from contaminants. To accomplish this, MozaCajú assisted processing facilities to become certified according to the HACCP system. HACCP is a food safety management system and risk management tool that identifies, evaluates, and controls hazards that are significant for food safety at all stages of the food manufacturing operation. Achieving certification entails making tangible and demonstrable improvements in food safety performance across every aspect of the manufacturing process, through detailed training and process development. This provides a greater level of legal compliance, and additionally, results in less waste thanks to more efficient 38 operations. The ultimate objective however, is to guarantee food safety by replacing low-efficiency, retroactive quality control processes with proactive identification of food safety hazards and best management practices. MozaCajú contracted a food safety company with over 20 years of experience in the food industry, Merieux NutriSciences, to assess the status of food safety by conducting HACCP pre￾assessment audits in seven of the factories. The findings from the pre-assessment audits informed the necessary corrective actions to be taken by each factory to prepare them for HACCP certification. Following the pre-assessment audits, Merieux NutriSciences and MozaCajú technicians formed HACCP Committees in each of the factories, which consisted of approximately five senior-level factory employees who became responsible for the implementation and maintenance of the HACCP system in the factory. MozaCajú conducted workshops for these HACCP Committees to orient and train them on the various requirements for obtaining HACCP certification, before commencing a prolonged training processfor all workers in each Figure 8: Managing Partner S. Martins, Condor Nuts, with HACCP Section of the factories. The main topics covered in these training sessions included: • Cleaning, sanitization, and maintenance procedures for infrastructure and equipment • Waste management procedures • Pest control procedures • Document control procedures • Maintenance plans, with emphasis on cleaning and sanitizing • Inventory management • Hygiene codes • Quality control records In total, over 100 documents were developed to assist the HACCP Committee roll out HACCP in each section of the factory. These included documents detailing rules and regulations, registration sheets for tracking the movement of people and product within the factory, and verification sheetsto ensure that maintenance, cleaning, and sanitization occurred when necessary. By end of the project, three of MozaCajú’s participating cashew processing facilities have received HACCP certification, 60% of the target. While MozaCajú did not achieve that target of 5 certifications, two of these newly HACCP-certified factories are currently seeking further certification by the British Retail Consortium standard, which builds on but exceeds HACCP standards. The combined HACCP and BRC training results (for activities that included further work conducted during the no-cost extension period), which are key to achieving certifications, are summarized in Table 12 below. Table 12: Certification-Related Training Performance Training: Project Totals Korosho (ETG) Condor Nuts Condor Caju Caju Ilha TOTAL Workers trained on phyto-sanitary procedures 663 941 843 0 2,447 Supervisors trained on process and product quality control procedure 10 8 16 2 36 Supervisors trained on food safety management programs 20 23 21 11 75 INCAJU technicians trained on food safety management systems 17 39 In total, 128 stakeholders (111 processor representatives and 17 INCAJU representatives) received in-depth training relating to certification. Participation was not uniform. Notably, one processor chose to only partially participate in the training and technical consulting process; Caju Ilha hosted management training and multiple technical and international buyers’ visits arranged by MozaCajú but were not prepared during the project duration to engage in the preparatory investment for certification. Conversely, Olam largely proceeded independently. Processors were trained on post￾harvest handling as part of the quality control training; in total 36 representatives received training￾of-trainer (ToT) instruction in post-harvest processing; we understand they would subsequently have trained others upon return to their units but no figures were recorded for their numbers. Reluctance to invest in certification more aggressively is at least partially attributable to the mixed signals processors received from international buyers; while some EU and US buyers have progressively emphasized certification and the near-term requirement for traceability, other buyers are less demanding in a supply-constrained market where few incentives exist that consistently reinforce the need for higher quality. Quality-based price signals are still not being seen in the local supply chain, reducing the incentive to overcome some of the problems of reliability that processors cite as significant issues impeding their investment. Conversely, the processors themselves can do more to motivate that quality improvement in local RCN supply, and some steps are occurring: ETG has invested in a network of warehouses and Condor is considering a plantation initiative, the latter subject to a late-starting grant award that terminated due to the imminent expiry date of the project. 7 Component findings: Improved Financing/Grants 7.1 MozaCajú’s matching grant fund MozaCajú launched its matching grant fund in May 2015, soliciting grant applications from a diverse range of actors in the cashew nut industry, from processors and commercial farmers to agrodealers. The grants provided funding for a variety of projects, most especially for the procurement of much￾needed equipment or technologies that would have been otherwise unavailable to the grantee due to high cost and risk. MozaCajú’s matching grantfund had a cost-share requirement in which the grantees had to putsome of their own resources toward the project along with MozaCajú funds. The intention was that this would ensure that the grantee was committed to the project and would also leverage the investment made by MozaCajú. This cost-share was generally a minimum of 30 percent of the total grant value. With a total fund of $250,000, each MozaCajú grant ranged from just over $1,000 to under $50,000. Overall, MozaCajú awarded and completed 13 implemented grantsto 11 granteesranging from SME to corporations, leveraging over $1 million in investments along the value chain. Half of the awarded grants funded innovations or technologies in the processing sector, while the other half advanced production. MozaCajú’s grantees who completed the process of draw-down consisted of: • three large industrial cashew factories (Olam, Condor Cajú and Condor Nuts) • one government agency (INCAJU) • three agricultural inputs companies (Agrifocus, Soluções Verdes, Ngala Investimentos) • two small-scale farms and factories (ADPP and EMAJU) • one agribusiness (Sociedade Comercial Messalo) • one large commercial farm (DD Investimentos). Grantee profiles and updates were included in each of the MozaCaju semi-annual narrative reports provided to USDA during the project. For convenience, grants profiles are repeated in Appendix 5. The Endline evaluation was conducted through interviews (with several in late 2016 being repeated 40 in late 2017 for longitudinal follow-up); most of the data collection was reserved for the 2017 round of interviews due to the not-unexpected delays experienced by several grantees in getting ready to draw-down on the grant and make their investments. Three of the four recipients interviewed in the MTE were re-interviewed in 2016 (one project, Matanuska, had subsequently cancelled due to major business issues elsewhere), to acquire some longitudinal perspective on the opportunities and challenges, the support processes from the project and any changes in attitude towardsthe program. The evaluation findings for the complete grants program are summarised in Table 14 below. 7.2 Illustrative grants projects While each grant noted in Section 7.1 (and summarized in Table 14) had a unique trajectory and encountered specific obstacles and opportunities, the study was designed to interview at least 75% of the active recipients, spanning the wide range of interventions; this was achieved. As an illustration of that range of candidates and their requirements, and the approaches taken to create what was ultimately a valuable intervention from the perspective ofstakeholders interviewed, an example is given below of interventions we may characterize as being characteristic macro, meso and micro levels of project complexity. 1. Macro intervention: DDI – launching a commercial farm DDI Investmentos Limitada (http://www.ddi-agri.com/) is a new company set up in 2011 by experienced investors. In 2015 DDI applied for assistance to establish a cashew plantation on 200 Ha of land identified and obtained after a 12-month scoping exercise, with the intention of planting 50 Ha per annum with improved cashew varieties. The Endline evaluation confirmed that DDI presented as a well-prepared candidate. They had developed a network of contacts, obtained the land, had a crop plan suited to the dry conditions and have comprehensive plans for community engagement, mechanized clearance, a fair local compensation scheme for relevant stakeholders and a future outgrowing scheme. At the time of evaluation, they had completed their initial season, planting 50 Ha. (polyclonal and grafted) with 5,000 seedlings, mostly grafted and sourced from a MozaCajú nursery; the polyclonal seeds came from INCAJU via TechnoServe. Challenges of lack of readily-usable water and a higher-than￾anticipated mortality rate of seedlings sourced from the nursery have caused a temporary pause in expansion to the next 50 Ha. block in 2018. DDI cite the consistent technical support received from the project in all aspects noted above; including help with the hiring of technical expertise: “You’ve been restless in your assistance” (farm manager). The financial grant and technical assistance, including training transfers, were crucial to overcoming the initial obstacles. “To make us independent and ready to act in a better way”, the project aided development of a diversified business plan including a future processing plant for value addition; and the project provided business resilience support, including diversifying seedling sourcing and grant support to obtain a tractor and water tank to overcome the problems with water supply that slowed their planting plans for 2018-19. 41 Rural Small Business Capital Formation Generating Re-investment Funds: Ngala At full operational potential, Ngala will collect 45,000 kg of cashew (2 kg/tree, for 22,500 trees owned by approximately 150 farmers). Sold at an average price of 50Mts/Kg, this would earn 2.25m Mts in gross revenues. Based on the cost structure they described in detail of 63Mts per tree serviced, including labor, fuel, maintenance and spray, the business can generate up to 800,000 Mts annually. Allowing for future contingencies, amortization reserves and price/cost fluctuations, this represents at least a 1X payback per annum, generating capital for further expansion including their planned further diversification into the otherinputsthey already sell. 2. Meso intervention: Condor Nuts - capital investment in a new technology Condor Nuts received a MozaCajú grant to encourage the first-ever purchase of a Cashew Nut Shell Liquid (CNSL) extraction machine in Mozambique. Through the installation of proven extraction equipment, the CNSL can provide fuel for industrial purposes and the dense, dry matter or “cake” that remains after oil extraction can then be safely and more efficiently burned. The Endline evaluation confirmed that with this CNSL machine, Condor Nuts has been able to sell some liquid and cashew shell cake, to such an extent that the company intends to invest further in the sale of stoves fuelled by the CNSL cakes and in pelletization equipment. The company advised that the CNSL process will produce 4,000MT of cake this year, considerably in excess of what can currently be absorbed without better access to stoves. The opportunity is seen as significant: “We estimate that an employee can reduce their fuel costs from 700-800 Mts per month using charcoal to 200-300 Mts per month in this way” (CEO, 2017) but adoption is constrained by access to credit schemes available to acquire the stoves. The project is a work-in-progress. Wider credit is necessary to stimulate the market for efficient and safe stoves. Technical and operational constraints have so far inhibited using the oil as fuel for the factory itself. The project team have therefore contributed ongoing advice regarding technology options, for which senior management expressed their appreciation in the interviews. 3. Micro intervention: Ngala Investimentos – agri-dealer Due to the large role of the government in chemical distribution and the difficulty in securing business credit, there are far too few private spraying services available even in those cashew￾growing communities prepared to pay for these services. While the few larger actors (notably, Agrifocus) can fill some of the supply gap, many more local service providers are needed to stimulate adoption of spraying, thereby improving farmers’ financial returns, in turn creating ongoing demand for more services. Ngala Investimentos is a small, energetic actor who started with one sprayer, grew to three machines, declared itself a “business” and sought a grant. The grant award enabled the acquisition of 12 new machines and creation of 24 jobs, meaning they could support around 150 smallholders and their 22,500 trees with 42 the recommended three rounds of spraying per season. With a cost structure of 62.5 Mts/tree and revenues of 2Kg RCN (or approximately 100-120 Mts), Ngala can generate sufficient cashflow to invest in further machines with a payback of around one season (see sidebar). MozaCajú provided grant funds, some technical advice and further assistance to identify capable operators from the farmer database. Ngala’s manager concluded the Endline interview by expressing a commonly-heard wish that the project would not end: “Our one complaint is that farmers are still not applying GAPs sufficiently to reap the full benefits”, with project extension seen as being essential to encourage higher usage of Ngala’s services. 7.3 Results and Suggestions for Grant Designs: One significant aim of the grants program wasto stimulate processor investment in new technologies through matching grants. The project aimed to result in USD 250,000 of incremental, technology￾related investment that had met the grant criteria – e.g. investments that would improve competitiveness, thereby excluding routine capacity expansion. Condor, Olam and start-up Emaju made investments (Table 14 below), summing to MZN 23.45 million of grantee contributions. Thus, USD 390,839 of grantee investment was stimulated through the grants program, a 56% increase above the target and a good indication that key objectives were met for the program; grant contribution from USDA funds totalled $70,766 to match that investment (i.e. an 18% match). Each of these technology investments was accompanied by an analysis of the viability of the technology expansion project, supported by technology insights shared by the TechnoServe Regional Cashew Director and through study tours to West Africa (Section 9). In total three analyses were completed. These findings also suggested several observations that may specifically guide future grant program design (see also Section 10 below for “lessons learned”) generally: 1. Early-stage grants must be realistic about measures of success in the context. The MozaCajú grants seldom generated any financial outcome in the timeframe beyond the simpler cases (such as agri-businessspraying operations like Ngala, where a machine can pay itself back within 1-2 seasons). Even where matching funds seem adequate, the many operational and ongoing working capital issues in rehabilitating an old factory (e.g. Emaju, ADPP) or farming block (e.g. DDI, Emaju), or in establishing a network of remote workers to enable women to earn from their homes (e.g. S.C. Mesalo), result in many obstacles to applying the funds. 2. Related to # 1, one implication is the sometimes-surprising amount of ongoing technical and business planning support that each grant entails. These support skills must be rationed across a wide canvas of different types and scale of business in geographically-dispersed locations, diffusing attention and presenting constant prioritization challenges. 3. Thissupport challenge is exacerbated by the constant requirement for such assistance in making the grant application at the outset. Applicantsstruggled with the technicalities; some, including one major processing company, presented several unacceptable submissions before their application was finally deemed unsuccessful and refused. The project team struggled with the paperwork load required for compliance. As an example, an MTE Report recommendation to help capable field promoters be sustainable by becoming local service agents was rejected because individual micro￾grants could not be supported. A partial solution already in development was the ‘master grant’ agreed with Agrifocus to enable 148 spraying machines to be bought on a subsidized basis by small farmersto provide localservices during the grant; these sales were completed within the timeframe. 43 4. Good selection, conversely, generates positive feedback for the program even when conditions remain challenging. The interviewed grant recipients were almost universally as appreciative of the technical support received as they were for the financial subsidy. 5. The periodic updates provided in the project’s narrative reports are essentially accurate, but ‘lessons learned’ should be encouraged during a project, rather than being reserved as an evaluation contribution. The program experienced useful lessons in e.g.: - false starts(e.g. initial focus on large commercial farmers unable to source matching capital); - flaws in due diligence (e.g. Langa’s unreadiness for a juice factory establishment). It is important therefore to establish a mechanism for feeding back to the donor the challenges in adapting a compliance-oriented process to the needs of a highly diverse clientele. 7.4 Jobs Creation While jobs creation objectives were not restricted to the Finance and Grants component of the project, the grants program recipients were the major contributor to the project creating 740 jobs (FTE basis). These are noted in respective Component Sections of this report where appropriate, but in consolidated form the jobs creation breakdown is as follows (Table 13): Table 13: Jobs Attributable to the Project. Source: Endline analysis Project element Attributable Jobs Comment Nurseries (85) 198 2.33 FTE created per nursery on average Processor Jobs 96 64 in micro-scale ventures (SC Messalo); 31 in medium ventures (Emaju); 11 roles in factories (but see Section 8.3) Agribusinesses 375 296 FTE in AgriFocus-related sprayer businesses (2 FTE/business); 64 FTE in other spraying businesses (via Ngala, Messalo, Condor); 15 FTE in inputs and spray businesses Plantation 74 Pebane establishment: 67 FTE (smallholders converting to full-time cashew farming); DDI set-up and operation: 7 FTE TOTAL: (FTE) 740 All roles calculated on an FTE basis No detailed wage information was recorded by the project. At the broadest level, the factories responded to the Endline and MTE interviews with summary wage data that indicated the total annual factory wage bill grew by MZN 109 million between 2014 and 2016 (i.e. USD 1.89 million); some part of this could be considered indirect project benefit (e.g. 43% of the wage increase is suggested by one approach [see Section 8.3 below]). Taking just the 740 directly attributable jobs, most (712) are labouring jobs, understood to earn MZN 60 thousand/year – effectively, USD 712 thousand in total, at recent exchange rates. The other roles are technical and managerial; no salary information was provided per role, but a feasible average including benefits is USD 20 thousand per role. The incremental wages for the 740 attributed jobs would therefore conservatively sum to USD 768 thousand per year. The project benefitted directly the 23,556 farmers trained by the project (Section 4.1) and the 740 formal employment jobs that should be attributed to the project directly within processor, inputs services and nursery operations (Section 7.3). We conclude that a total of at least 24,296 people benefitted directly from the USDA￾funded interventions in the project, which translates to 121,480 individuals benefitting in total, with inclusion of household members 38assumed to benefit from the wage income generated. 38 The project assumed a typical household of 5 including the worker. The project based this on conventional assumptions for the demographics of rural Mozambique and developed no independent method of verification or tracking (e.g. by survey or diary). 44 Table 14: Grants Program Summary. Source: MozaCaju and Evaluator Interviews 2016, 2017 Company Project Location Amount Funded Cost Share (Grantee %) Disbursed (MZN) Status Endline Evaluator Comments OLAM ECO2 Rapid Treatment Facility Monapo, Nampula MZN 1,700,000 MZN 6,888,000 (80.1%) MZN 1,700,000 Closed Implemented and fully operating 2017, following lengthy procurement design delays. The first example of this pest control technology in a Mozambique cashew processingfactory. No evidence of emulation by other factories to date. Condor Nuts Purchase of CNSL extractor Anchilo, Nampula MZN 1,700,000 MZN 15,403,352 90% MZN 1,700,000 Closed Implemented and operating 2017. Oil use in the factory is still an unresolved challenge. Cake sales occurring; despite marketing challenges due to the costs of stove acquisition without a credit scheme, and prohibitions currently on use of the fuel in the factory itself, the company plans future expansion. EMAJU Tractor and processing plant refurbishment Monapo, Nampula MZN 846,000 MZN 999,000 54.15% MZN 846,000 Closed Series of support steps provided by the project for this multi￾pronged production/processing project. Active clearing and planting on the farm, and expansion of the small processing set￾up; constrained by lack of working capital. Grant 1 of 2 received. SC Messalo Hygienic boxes for artisanal processing Mueda￾Nangade, Cabo Delgado MZN 1,692,275 MZN 4,299,681 (71.8%) MZN 1,692,275 Closed Social enterprise arm of a commodity trader, supported to develop and deploy micro-scale primary processing tools to village women, for further processing at a micro-factory. Deployment scaled-back due to the support overheads in maintaining quality, operational challenges with villagers, and difficulties in finding end-markets for their branded products. INCAJU M&A of cashew seedling distribution using CommCare Nampula Province MZN 1,683,275 MZN 1,967,800 (56.63%) MZN 1,620,799 Closed CommCare implemented and registered around 3,500 farmers. Despite its modest functionality, it inspired the development of the ‘ConnectCaju’ M&E tool project (Norges Vel) to support 100,000 farmers (2018-19) as an example of scaling through demonstration effects. Agrifocus Increased productivity through access to atomizers forsmall and medium producers Cabo Delgado, Nampula, Zambézia MZN 2,250,000 1,539,000 (40.62%) MZN 2,220, 000 Closed Grant to leverage the largest agri-inputs business network with a distribution scheme to reach small and medium farmers with a spraying ‘business in a box’. Over 200 machines sold (subsidized basis); “with 2 more years, we could have a lot of joint success” (Agrifocus MD). 45 DD Investimentos GROWCAJU: establishment of 50 ha of cashew production Chiure, Cabo Delgado MZN 3,011,178 MZN 1,891,500 (38.5%) MZN 3,011,178 Closed Full commercial farm plan for 9,500 ha in total, with thoughtful community engagement processes and highly disciplined farming practices. Initial 500 ha cashew farm hampered by lack of water, seedling mortality and technical challenges. They complimented strong technical support by the project: advice, mechanization and water harvesting support, business planning, mechanization grant. Matanuska Mozambique Lda Reinvigorating small￾scale cashew production in Monapo district Monapo, Nampula MZN 2,500,000 MZN 853,300 (25.45%) N/A Cancell￾ed Social engagement program for small-scale cashew as an adjunct to the primary banana plantation (1500 Ha.). Business challengesin the plantation slowed and finally led to the project cancelling the grant in consultation with Matanuska. EMAJU Innovative methods to grow and process cashew Monapo, Nampula MZN 3,304,650 MZN 1,526,900 (31.6%) MZN 3,304,650 Closed Second grant received, to develop the small-scale processing operation in a disused factory. The operation seems to have applied the funds well to establish an integrated operation, but it remains challenged by working capital and skills shortages. Condor Nuts HACCP certification inspection Anchilo, Nampula MZN 80,000 MZN 80,000 (50%) MZN 80,000 Closed Support for the certification development activities involved in attaining HACCP. Successfully completed with certification awarded 2017. Condor Caju HACCP certification inspection Nametil, Nampula MZN 80,000 MZN 80,000 (50%) MZN 80,000 Closed Support for the certification development activities involved in attaining HACCP at the second Condor facility. Certification awarded 2017. ADPP Increased profitability in the Itoculo Cashew Center (cashew nut and juice factory) Itoculo, Monapo, Nampula MZN 1,247,371 MZN 844,899 (40%) MZN 1,247,371 Closed NGO-based social enterprise adding value to locally-sourced cashew by processing, juicing and marketing locally supported by grants for equipment and basic infrastructure to make and sell branded product. Part-year employment for 6-8 women; unclear yet whether it will be financially sustainable. Orlando Langa Cashew juice factory "Tio Langa" Meconta, Nampula MZN 840,850 MZN 438,000 (34%) MZN 398,560 Cancell￾ed Grant support agreed to enable a large commercial farmer (20,000 trees) to add value with a juice factory for cashew and off-season crops. Due diligence process seemed to have under￾estimated Langa’s over-commitment to too many alternative business lines, resulting in delays that led to cancellation. 46 Soluções Verdes Construction of an inputshop and small warehouse Mueda, Cabo Delgado MZN 1,259,800 MZN 1,374,250 (52%) MZN 1,039,168 Closed Grant to support set-up of a ‘new model’ input store owned by a highly-skilled cashew agronomist and former MozaCajú Supervisor. Appears to have been an arms-length process: selection seemed to be objective – strong technical qualifications, large network of contacts and long personal history in cashew, matching funds, and a strong business plan. Ngala Investimentos Purchase of atomizer machines and other inputs Nampula MZN 1,495,200 MZN 2,146,480 (59%) MZN 1,400,498 Closed Grant to support a nascent SME (formerly micro-level with 3 spraying machines); acquisition of 12 new machines completed. Principals expressed a clear picture of costs and pricing strategy. Condor Caju Development of a Community Cashew Nut Plantation and support to existing trees Nametil, Nampula MZN 2,992,500 MZN 4,529,375 (60%) MZN 258,863 Cancell￾ed Support agreed for a community outreach program that S.V. Martins (Managing Partner) told the evaluator would be made “open-source” to diffuse the knowledge across the sector. Business delays took implementation outside the window for continuation within the MozaCajú project. Source: MozaCaju database and Grants managers; interview and observation comments by the author (2016; 2017). All balances of funds not yet disbursed as of 31 March 2018 were cancelled in writing by the project. Access to program records was at times administratively slow, but all due processes seemed to have been conducted in a satisfactory manner. 47 8 Component Findings: Marketing & Market Linkages The sector has experienced periodic deep slumps in production even in relatively recent years, deterring investment at both the farmer and the factory levels. Since the time of the most recent slump (2011-12 and 2013-14 growing seasons), total production tracked by INCAJU had more than doubled to 139,000 MT by 2016-17 (source: INCAJU, 2018). Value-addition through basic processing is the single avenue of differentiation for export currently (although some attempts are being made to add branded products – for example, Sunshine Nuts in southern Mozambique, and some small producers such as SC Messalo). It is notable that processor share of procurement has risen from typically 30-40% of total production to nearly 50% of the greatly expanded production volume, reflecting a multiyear program of investment in factory capacity. Over this period, local processor procurement has increased nearly 3 times (290%) on their baseline 2013-14 volumes (that is, from 22,300 MT to 64,300 MT). The latest full year figures indicate that the total commercialization volume of 139,000 MT was split almost equally between exports of RCN and local purchasing for processing, the latter decomposing into roughly 60% informal buying by means of local traders and 40% direct purchasing through their own buying networks (see Figure 9 below: the blue line represents processor procurement, the red line represents export of unprocessed RCN). The latest full-year preliminary data from INCAJU for the 2017-18 season indicates that this upturn in volume is sustained, with something like the same share for formal procurement volumes overall being recorded. Figure 9: RCN Buying Destinations - Source: INCAJU 2018 The volatile global market is a significant driver of demand, with Mozambique remaining a still largely undifferentiated price-taker, albeit some foreign buyers are encouraging the development of local capability and direction towards higher food safety certifications and traceability (see Section 8.2). While RCN export volumes seem to have exploded in 2016, the problems of data quality in tracking commercialization suggest that much of the apparent shrinkage denoted by the “other” element of each year’s sector performance is likely to represent mostly un-traced exports of RCN (please see overleaf; these dynamics are illustrated in Figure 10). INCAJU has previously reported a sizable component of “other”, principally undeclared exports; these should be added to formal exports in the chart below for a more accurate picture. In addition, significant weight losses occur in transit to ultimate sale. The most recent year shrinks this proportion of ‘other’ dramatically; this is partly due to a classification change and highlights the motivation expressed by INCAJU for initiatives such as the follow-on information and communication technology project, ConnectCaju, to focus on creating better transparency for the entire commercialization chain. 48 Figure 10: Dynamics of Commercialization: 2009-10 to 2016-17. Source: INCAJU 2018 8.1 Better market linkages for farmers MozaCajú was designed with a focus on stimulating and reinforcing the demand for processed cashew nut, recognizing that a major “competitor” is the export – legal or illegal – of RCN for overseas processing, typically bought by informal traders on a purely transactional basis. Given the perennial issues of supply reliability, quality and the many disincentives (low prices, uncertain selling) experienced historically for smaller farmers to maintain interest in growing cashew, the project addressed the supply-side through the activities outlined in the earlier Sections. Having helped generate additional production and reduce costs, the commercialization activity organized farmers to communicate with potential buyers (in particular, through CommCare, whereby a farmer group could message the availability of harvest to a panel of buyers) and to negotiate in a more informed manner. The underlying belief was this would create incentives for improved production and quality. Not too much should be read into any individual finding based on a single year, but the magnitude of the 35% increase in 2016-17 supply (see Section 4.6) was consistent with similar uptick findings in the MTE report, suggesting an underlying positive trend. Contextual factors are highly relevant to the interpretation, however; in the second and third years of the project, growing conditions for cashew were generally very good leading to a 34% increase in commercialized production reported by INCAJU across the entire sector for the 2016-17 year. Global prices were high and remained relatively high in local currency terms, in large part due to supply shortfalls in Vietnam. Farmers in the two northern￾most provinces thus benefited from the intensive buying activity of foreign buyers in the pre-launch before the announcement of the 2016-17 commercialization season. The data collection process for the Endline therefore produced some ambiguous results. On the self￾reported data provided by survey respondents, the project’s farmers performed somewhat better than results presented by the control group; but as discussed in Sections 4.6-4.7 we have reason to be cautious about accepting control group survey information given to an unknown enumerator by farmers frequently selected by convenience (when no one with the targeted farmer profile could be found by the enumerator); these factors create methodological issues that are very difficult to resolve in the field (see Appendix 2 for a discussion of the issues and the mitigation steps taken). RCN Destination Dynamics (proportion of total RCN) Source: INCAJU 2018 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 2009/10 2010/11 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 RCN procured by Processors (Formal) RCN Procured by Processor (Informal) Exported RCN Weight losses, stocks, etc Others (unknown) 49 The qualitative findings suggest that the project’s farmers experienced a roughly 5% price premium from being better organized; the project also created more confidence in making a sale by attracting more buyers, and a renewed enthusiasm by farmers for applying the practices and new seedlings to expand their production. This aligns with the quantitative discussion in Section 4.7 (above); by excluding outliers, we can reasonably report a median revenue differential, based on the survey results, to the control group for the final year of the project. A further support for this benefit of linkages is seen in the lower volatility of prices experienced by the project’s farmers; the range of price variations was much narrower (by almost 50%), meaning that more farmers were closer to the average and therefore more assured of benefiting. These findings in combination suggest that the project did support the development of better linkages of farmers to competitive markets. 8.2 Better market linkages for factories By leveraging the extensive private sector network developed by TechnoServe globally in its export projects, MozaCajú worked to connect Mozambican processors to several large, international buyers who were not currently engaged in the Mozambican cashew nut market. The verbal reports received from factories indicate that 8 of the 15 substantial introductions (53%) led to transactions; this is consistent with the findings reported in the MTE. Some of these buyers had previously procured some cashew from Mozambique, but in the interviews conducted for this report and for the MTE, several buyers revealed they had previously ceased sourcing due to chronic quality control issues. Encouraged and supported by MozaCajú to accelerate their plans to install food safety management and quality control systems, these processors became ready to re-enter the international market to sell their kernel, and valued the new introductions to buyers. While no single action of feature opened the door to substantial new sales, the combination of trusted referrals and the proof of progression by some of the processors towards earning the key international food safety certifications discussed in Section 4, implementation of initial batch processing systems, commitment of thousands ofstaff to training, and willingness to become more transparent, encouraged progressively-minded buyers to investigate Mozambique as a valid option. In early 2015, MozaCajú contacted Caro Nut, a nut roaster and wholesaler that supplies cashew products to the major U.S. retailer Costco, and invited them to Mozambique to learn more about the Mozambican cashew industry. MozaCajú managers hosted Caro Nut representatives in-country and introduced them to several Mozambican processors, including Condor Nuts, Condor Cajú, and ETG. In 2016, Caro Nut placed its first order of cashew kernel from those three cashew factories. MozaCajú helped to ensure quality of those initial orders by checking for variances between grades and conducting random nut counts before they were shipped to Caro Nut’sroasting facility in Fresno, California. Caro Nut testimonial Caro Nut Company supplies Costco Wholesale and other retail stores with high-quality and traceable finished goods. Though they source cashew from all over the world due to rising demand, Caro Nut is a dedicated and active buyer of cashew nut in West and East Africa, aiming to strengthen local African processing industries for a more resilient value chain. From a start in 2009 with just two suppliers, factories; they now source nearly half their African volumes from Mozambican factories, largely due to the diligent work of MozaCajú: “The MozaCaju team has always been a tremendous resource for Caro in East Africa” (private correspondence, 2018). With MozaCajú’s support, the company has been able to identify good business linkages, and develop a clear view of the cashew value chain. Caro Nut believes that MozaCajú has been a key factor in the company’s ability to source confidently from the region. In the years 2016 and 2017, they purchased nearly 80 containers from Condor and 4 from Korosho, for a total value of almost $12m. 50 Similarly, MozaCajú reached out to both Nutrade, a South African bulk supplier of fortified functional foods, grains, and nuts, and to Red River Foods, a U.S. based global supplier of a wide variety of tree nuts, dried fruits, seeds, and specialty snack products. As verified by the senior buyer interviewees, MozaCajú introduced Nutrade to Olam, ETG and Caju Ilha factories and introduced Red River Foods to Condor Nuts and Condor Cajú factories; these factories are now currently supplying multiple containers per year to these new buyers. Four of these factories are now supplying export markets under international certification; 11 factories showed evidence of some branded product being marketed. In quantifying these facets of the support, it proved very difficult to identify how many representatives of factory operations were directly benefitted by these interventions in support of marketing; export sales tend to be managed very closely by just 2-3 managers per factory, rotating work as they see fit; no reliable calculation could be finalized for ‘representatives benefitted’. Communication brokerage was a substantial aspect of the support program. As both Red River Foods and Caro Nut were interested in traceability and sourcing sustainable cashew from farmers, MozaCajú worked with these companies to collect information on the farmers from whom the factories source their RCN. MozaCajú also acted as a filter for requests to purchase Mozambican kernel through its online presence (www.mozacaju.com). MozaCajú received dozens of inquiries from companies monthly seeking to source kernel from Mozambique, and through communication with these companies the project passed along vetted leads to the Mozambican processors. The Endline evaluation process found that these key introductions and technical support resulted in 1,615 incremental tons(cumulative) of cashew kernel sales that can be attributed, i.e. that resulted from sales to buyers introduced and supported to some substantial degree by MozaCajú. This contributed to the sector enjoying sales increases summing to a total incremental 3,474 MT of exported kernel by 2016 versus 2014 (Figure 11). These outcomes translate to the healthy overall increases in sector annual revenues during the three years of the original project term from USD 25 million (2014) to USD 44 million by 2016 (75% growth). Over this period, cumulative three-year export volumes were 15,465 MT, representing cumulative export revenues of USD106.7 million. Figure 11: Kernel Exports by four main local processing companies – in Volume (MT) and Value (USD) as reported by the Processors. Source: Endline interviews and INCAJU 2017 verification Growth in exports is also a reflection of the progressive, albeit still-cautious (with the exception of OLAM), expansion of capacity of the factories. The project had technical influence but limited direct 51 say over the timing and scale of investment in production expansion; global shortages meant factories’ focus remained on volume, and this constrained time or attention for e.g. preparation for formal traceability. All buyers voiced concerns about ongoing assurance of supply, quality (e.g. Korosho were affected in early 2016, Condor in 2017), and international cost competition for RCN. Condor and Korosho demonstrated the most collaborative and open attitude receiving significant technical and commercial inputs, whereas OLAM pursued certifications on its own behalf while joining Caju Ilha in welcoming MozaCaju’s ongoing technical consultation and the facilitation of introductions to buyers. During this period investments were made in three new plants by OLAM and ETG/Korosho and in the upgrading and expansion of existing facilities; for example, Condor Nuts has invested over $1 million per year in new shelling and other equipment throughout this period. The project’s major contribution has been in reinforcing the factory’s abilities to find willing buyers, and in particular, buyers who take a longer-term view. The strengthening of thislinkage to global markets provides valuable feedback loops that reinforce the value of technical upgrading and investment in certifications. This in turn can create a more reliable market for smallholders that are organized and who communicate appropriately so that they become more accessible to the buying networks; and that in turn reduces the margin-taking by the current layers of middlemen. 8.3 Jobs Benefitted Factory expansion also creates a linkage for villagers to the formal labor market; while some new technology investments reduce the demand for labor, overall the processors remain highly manual. Jobs growth has been significant as a result, although structural issues such as the high rates of absenteeism (as high as 50% in certain periods) create cost pressures that may not be sustainable in the medium-term39. In aggregate, recent expansion has generated significant new job growth, adding nearly 5,000 jobs since 2014 (Figure 12). Figure 12: Factory Jobs - source: four major local processing companies Whether this growth is sustainable is an open question, very largely dependent on factory utilization following recent capacity expansion. Necessary technology upgrades were found to reduce labor employment in specific areas of the factory, as e.g. new shelling machines and other automation 39 Competitiveness of the Mozambican Cashew Industry, by MozaCaju. Accessible at: http://www.mozacaju.com/publications/ Gender: overall, 60% female 52 were implemented by Condor, Olam and ETG. The full-time equivalent, non-salaried jobs information collected during the end line process is summarized in Figure 12 above; the data are from company responses (verbal and by email) and, as with wage information, could not be verified directly. Directly-attributable job data were collected during the Endline evaluation of the Processors. Quality control initiatives had resulted in a net addition of 3 personnel, while technology upgrades accounted for a further 8 jobs (6 at Condor, 2 at Olam). The small processing factory at Emaju created 16 plant￾based jobs and 5 field extension jobs, and the micro-processing jobs in the SC Messalo venture created 34 factory roles and 30 roles in villages. These sum to 96 FTE related to processing of RCN. At a broader level, the project was tasked with benefitting factory employees from capital accessed by the processors. This was very challenging to interpret and the project struggled to collect that data; the factories did not keep records attributing job changes to factory expansion influenced by MozaCaju specifically. A proxy measure that may suffice is to relate employee numbers benefitted to the proportion of export sales growth in the focal period that can be attributed to the market facilitation and ongoing support the project provided; as noted in Section 8.2, that latter represented 1,615 MT of cumulative exports, in the context of a total cumulative increase (compared to 2014) of 3,747 MT, i.e. 43%. Applying that percentage to the total jobs growth reported by the processors (4,911), we may reasonably thereby attribute 2,117 employees that benefitted through job creation from the capital accessed by processors to this expansion of production to meet growing demand. 9 Adjunct Activities: Policy and ICT Policy constraints lie at the heart of many of the structural issues that impede the future development of the sector, as well as the broader economic and household food resilience of the million or more farming families that are dependent on cashew as a source of cash income but are often discouraged or passive in relation to cashew farming. Policy development was not an explicit headline target of the original USDA Award but is a crosscutting issue affecting almost all areas of intervention beyond the simple transmission of specific techniques. As the project progressed and relationships between stakeholder parties solidified, more scope became available for policy-related discussions; the work leading up to and including the no-cost extension placed a more explicit priority on fostering actions to create policy reform. These principally lay in three areas: (i) knowledge building, (ii) law and regulation development, and (iii) the creation of an enabling information infrastructure. These have previously been profiled in the project’s narrative reports in some detail, the main implications for this evaluation are discussed in the following sub-sections. 9.1 Knowledge-building within the sector The project invested in a combination of analytical reports, executive visits and conferences with the intention of creating a broad knowledge base and sector alignment relating to key themes identified by the project. As noted in the MTE, the project published a 2015 study -- the Competitiveness of the Mozambican Cashew Industry report -- analyzing the comparative costs of cashew processing in Mozambique in relation to its competitors. Cost and quality issues led to a persuasively-argued recommendation for strong monitoring and evaluation capabilities to guide seedling variety selection, distribution and planting, along with more systematic attention to inputs and postharvest handling by opening up the choices available to farmers. This was complemented by a Cost￾benchmarking database that TechnoServe claims benchmarks Mozambique against other cashew￾producing countries, leveraging TechnoServe’s global cashew program. These sources underpinned the development in 2016 of a Cashew Sector Industry Strategic Plan40 that addresses systemic issues 40 Accessible at: www.mozacaju.com/s/Cashew-Strategic-Analysis_2016_Final.pdf 53 for improving the competitiveness of the Mozambican cashew industry, notably the lack of entry points for private-sector actors in inputs and services provision. To reinforce these arguments, two senior-level industry stakeholder learning trips were invested in during late 2016 and second quarter 2017. The trips combined public and private sector leaders in a program of learning about the structural, regulatory and technical drivers of the high rates of growth in certain competing cashew producing countries in West Africa; interviewed leaders commented favourably on the learnings and insights gained in terms of policy development and technology. 9.2 Cashew law and regulation reform In 2016, MozaCajú informally partnered with USAID’s project SPEED+ (Supporting the Policy Environment for Economic Development41), whose goal is to provide expert technical services to the Government of Mozambique to support economic and structural reform in the areas of agriculture, trade, business enabling environment, energy, water, and biodiversity conservation. The emerging collaboration with SPEED+, INCAJU and the industry association AICAJU aimed to foster development of the Government of Mozambique’s new Cashew Law and Regulation, with MozaCajú providing a leading technical contribution to the consultative process. This contribution was evidenced with the distinguished participation of national and international actors in the December 2017 MozaCajú national cashew conference in Maputo (pictured below). MozaCajú has used these forums effectively to advocate for reformsto be centered on a re-definition of INCAJU’s role in the cashew industry towards becoming a regulator rather than service provider. 9.3 Enabling infrastructure via ICT (Information and Communication Technology) MozaCajú realized early in the project that recommendations for policy and behavioral change will not progress unless means are provided for assuring greater visibility for regulators and more effective access and reach for service providers. For example, phasing out INCAJU’s direct role in chemical distribution is only feasible if private sector agri-input investment can be encouraged and can be monitored. In this highly underdeveloped sector, the information and communication linkages in place are weak or non-existent across most of Mozambique. This frustrates regulatory oversight, makes market discovery difficult for the private sector, and impedes the introduction of efficient methods such as electronic vouchering to ensure that the poor are not disadvantaged. 41 See: http://www.speed-program.com/ 54 CommCare, the early experiment in ICT within the project, showed promise and this informed the design of a new project to scale up the reach and deepen the range of services provided using ICT, as seen in the year-old ConnectCaju project. That latter project, with which the author is very familiar, provides good evidence of actual development of policy-directed alignment between MozaCajú and INCAJU in particular: the critical topics of chemical monitoring, seedling distribution and commercialization tracking are being jointly addressed by INCAJU and TechnoServe in that project, with strong senior leadership commitment that reflects the progressive development of a common vision for the policy and structural reforms to improve the sectors competitiveness and its legitimacy with the farming population. Many implementation details remain, and timelines for the transition of INCAJU’s roles are still development; the project is nevertheless a robust proof point for the trusted relationship built up through MozaCajú and related activities. 10 Lessons Learned Beyond the evaluation of performance against target, this Endline report aims to provide evidence￾based suggestions that may serve as guidance for future project design and implementation. Some commentary on lessons learned has already been provided in earlier Sections and will not be repeated here; for convenience, the brief comments that follow are set out under the subheadings of the components that they most explicitly address, with full acknowledgment that some are crosscutting and apply therefore in more than one area. 10.1 Production: 1. (Genuinely) ‘commercial’ farmers asleaders in adoption: Though MozaCajú focused on reaching a large base of smaller cashew farmers, it also worked with cashew farmers with holdings of considerable size, ranging up to 2,000-10,000 or more trees. MozaCajú formed the view that larger commercial farmers were better at adopting improved practices and contributed disproportionately to the project’s impact. This suggested that they could act as “knowledge nucleus farmers”, on the assumption that smaller farmers in the community were more likely to adopt better practices from the perceived role modelling. While this segmentation approach seems too simplistic – not all very large farmers were “commercial” in behavior42 – overall the larger farmers’ slightly higher adoption of better practices and better access to resources (such as their own nurseries) seemed to create a potentially valuable mechanism for orchestrating spillovers and maintaining the adoption of better practices sustainably. A more explicit nucleus program, of segmenting on behavioral terms and with design elements to encourage mentoring via additional training, (non-cash) recognition and periodic monitoring, could strengthen this networked approach. 2. Timeframe and geography: As noted in the MTE, MozaCajú was initially designed as a three-year intervention due to USDA program parameters (with the subsequent extension granted towardsthe later phases of implementation). As observed by various actors cited in the MTE, and reconfirmed during the Endline by economic officials as well as extension officials and processor managers, the minimum viable intervention for a perennial crop such as cashew should be at least five years. This is needed in order to allow new practices and technologies or varieties to come to fruition and produce observable production and economic outcome. This applies in particular where large geographies are involved, making the deployment and management of the geographically dispersed teams challenging. Some of the key learnings of the project could have been better obtained with more focused interventions in smaller geographies, allowing much closer monitoring and clearer 42 This was later (2018) reflected upon by the MozaCajú Program Director who argued that TechnoServe had long favoured a ‘Small Commercial Farmer’ model of technology diffusion by the most motivated farmers; this however seemed to have been an emerging development, at least partly lost in translation to project field activities. 55 evaluation. Subsequent USDA cashew programs seem to have acknowledged the underlying lessons with five-year programs seeming to become the norm. 10.2 Inputs: 1. Build in policy advocacy in tandem with agronomic training: Despite its importance for crop protection, chemical fungicide is not commercially available in rural markets in Mozambique, as farmers expect to receive these inputs from the government rather than to purchase them. Similarly, improved seedling varieties are also not available in sufficient quantities for the level of planting that cashew farmers require. Because of this, the local agri-input industry for seedlings, chemicals and related services are vastly underdeveloped. These constraints must be addressed in tandem: donor and government support is needed both for private-sector agri-input distribution and the development of the market for these products. This can be accomplished by helping farmers to understand the dramatic returns on investment they may see from proper spraying and planting. Without this type of multi-pronged approach, any intervention will not have sustained impact. 2. Over-communicate early (e.g. regarding Nursery location): The project initially seems to have over-weighted the importance of the location of facilities (in this case, community nurseries) close to the farmer communities it aimed to serve. At least two significant constituencies should be consulted in an open communicative style to understand the context beyond ideals of operational efficiency: the incumbent operations (INCAJU nurseries) and the farmers whose behavior in relation to the nursery is paramount. In this project it was necessary for example to establish nurseries in locations that were far from the government’s nurseries but also in communities where there was enough market demand and willingness to pay in order to test the hypothesis of sustainability. 10.3 Traceability and Innovation: 1. Creative incentives for adoption: Cashew processing is a high-volume, low-margin business operating in a global market that very largely buys in bulk despite the growing influence and market demand in the EU and US for safe and ethically produced product certified under the various seals. MozaCajú learned that while specific segments of the global buyer network are very interested in HACCP certification and traceability, with a small number willing to invest by placing orders with new sources of supply that demonstrate an appetite to earn certification, there is limited willingness to pay a premium for product that has attained only minimal certification (HACCP). There is also only niche market demand that will support pricing reflecting the investment costs of fully implemented traceability. MozaCajú successfully identified and cultivated like-minded buyers; a longer project may have created more of a platform for processors to continue that non-traditional engagement. Quality is crucial; processors in turn must be shown why they should offer premium pricesfor assured quality through their buying chain, something only partly achieved in the project. 2. Investment business case: MozaCajú learned that obtaining buy-in from the highest levels of senior management in cashew factories is critical to implementing sustainable changes at the operational level of the factory. The technical advantages do not speak for themselves, and training alone achieves nothing. The four processing companies demonstrated very different attitudes and aptitudes at this time for investing in the innovations proposed. A longer project duration should include a careful, frequently updated monitoring of the business outcomes, to demonstrate the business case; this was only partially achieved in this project. 10.4 Access to Finance: 1. Supporting, but not leading the process: MozaCajú aimed to reach a variety of value chain actors with its grant fund – including new and smaller agribusinesses. However, many of these agribusinesses required significant support just to start the application process. MozaCajú had to balance this assistance with room for participants to take the lead on the details of the project, even 56 though that frequently lengthened the implementation. The due diligence process must involve careful evaluation of stakeholder commitment to the business idea and commercial assessment beyond the technical and compliance issues. Further suggestions were made in these regards in Section 7.1 above. While the Grants program was closely tied to targets for job creation in the project, a broader understanding of the linkages would have been beneficial. Mozambique is currently poorly-prepared for addressing the creation of employment in and around the sector. Beyond the activities noted above and outlined in Section 8.3 below in relation to the factories, the project did not attempt to address the issues of job creation, job-seeking and worker skills upgrading directly within its scope. Grants recipients noted that labor costs, low skill levels and high absenteeism rates dictate that these issues require attention in the near future. 2. Technical assistance increases impact of financial assistance: Though the main purpose of the grant fund was to dispense financial assistance in the form of in-kind or cash grants, for many of the grantees, the technical assistance that MozaCajú provided proved to be just as important. MozaCajú technicians’ insights on production, processing, or business planning skills were paramount to each project’s success prospects, especially for those grantees using the grants to enter new sectors. The workload significantly exceeded the project managements’ expectations. Future project designs should give higher weighting to the workload involved in supporting often-complex, bespoke grants. 10.5 Marketing & Market Linkages: 1. Identifying how to transition facilitation work: MozaCajú successfully brokered several important relationships between international cashew buyers and Mozambican processors, leading to tangible contributions to over 1,600 MT of incremental sales within the project duration. MozaCajú aimed to not take the lead role in these relationships beyond the introduction, so as to build ongoing sustainability; but the range of issues that arise in such transactions in actuality pulled MozaCajú into ongoing firefighting and technical support tasks. A clearer set of negotiated understandings for engagement between the project and the processor including focused development of the required capabilities on the processors’ side, is recommended. Learning trips arranged at an earlier stage, for example, may have lifted participants’ sights sufficiently to improve the terms of engagement. 2. Providing transparency and monitoring infrastructure: As noted above, MozaCajú’s CommCare system was a useful tool developed as something of an afterthought to share information between farmers and factories’ buyers via SMS; a simple way to begin to improve market linkages. Farmers were sometimes confused however about the role of the mobile information tool, ‘CommCare’ and whether it provided a marketplace or just a communication channel. CommCare was limited by being essentially one-way communication, with no feedback loop that could report on whether a sale occurred; the sometimes frustrated both buyer and seller. Where geographies are large and resources limited, an appropriate ICT infrastructure can overcome visibility issues for the beneficiary and the project team by providing the access to information, monitoring capability and two-way communication that enables projects to react in real time and to scale at less cost. 10.6 Program Management: 1. Contractual Design: The contrast observed between the generally good and open collaboration at senior management levels and the very different and sometimes deeply conflicting operating expectations methods and communication in the execution, was evident in both the MTE and was observed to continue to persist at the time of the Endline. The key lesson is that donor and implementing organizations should ensure that in any proposed partnership, the parties are not just technically complementary but that they are truly aligned philosophically and share a vision for success. Further, the sub-award structure impeded an internal coherence and consistency to execution (see #3 below). 57 2. Project Design: as noted in relation to production and other facets of the project, the design of the project was constrained by a project duration that was inadequate to allow the evaluation data to filter out sufficient “noise” so as to draw some statistically significant conclusions. All attempted correlation analysesfailed to produce a usable result; this was not unexpected, it does not take away from the empirical observations and preliminary performance measures collected but it does raise more questions than it can answer and leaves these questions for an opportunity to collect a data set with a longer time series. This was heightened by the absence of periodic operational monitoring and evaluation; this impeded timely management decision-making and the assembly of a better dataset. Given the time constraint, a more useful design may have focused on more specific questions with a narrower range of intervention in a more limited geography so that the deeper learnings needed in a troubled sector and an uncertain policy environment could be derived. 3. Management systems: From the MTE and Endline interviews and field observations over three weeks’ total duration, the major issues identified in the MTE persisted – inflexible, paper-based operational management chains; field processes stretched over wide geographies with field staff accustomed to basic agronomic but not commercial support; infrequent management monitoring due to distances, and then primarily to larger farmers. The subcontractor managing the extension network was weighed down by internal bureaucracy and slow transmission of information from the field, with too many approvals taking time and delaying implementation. This improved over the life of the project, but some chronic delays (e.g. in making payments to promoters) had a negative impact on morale. The parties should have operated under a single management process rather than the parallel, frequently arms-length processes that created some significant information gaps and execution challenges. 10.7 Policy & Knowledge Building: 1. Policy development investment is a necessary complement: As noted in the MTE and thisreport, the initial project proposal did not fully anticipate the entrenched policy-related issues relating to inputs supply and access, for example, and therefore did not include an advocacy component whereby the project would work to encourage INCAJU to phase out these subsidies, for seedlings and potentially for chemical. This was later recognized as a distinct need; these projects operate in a complex ecosystem and their effectiveness is highly dependent on regulation and complementary incentives e.g. for private sector investment. The project demonstrated that achieving more than the direct interventions such as technical training and technology transfer requires investment in long￾term policy advocacy efforts that for most of the project had been deemed to not be feasible. From the end of 2016, the original project term, TechnoServe’s policy work with SPEED+ has successfully brought that initiative into partnership with INCAJU, convening the wider stakeholder discussions and international expert inputs to help develop a better framework for the sector. 2. Knowledge Building Pays Dividends: while difficult to measure within the limits of a project term, knowledge-building is an important investment. Beyond the mechanics of organizing efficient delivery of training to farmers and extension agents, a funded knowledge-building component provides new resources and channels for learning across the range of issues that affect production and livelihoods, even though they go well beyond those specific topics. The project seemed to bring multiple elements to bear effectively, including a cost benchmarking database and technology tours that gave processors and other commercial parties new insights; (i) industry strategy and visioning tools (a detailed competitiveness study; and industry strategic plan) to provide international benchmarks and create a new discourse of investment and choice. These elements reinforce the essential policy development initiatives by helping various stakeholders see the connections and the potential for private sector investment in greatly-underserved areas such as inputs supply. MozaCajú Endline Evaluation Page 58 Appendix 1: Terms of Reference ToR for Evaluation: Endline Terms of Reference PROGRAM EVALUATION TERMS OF REFERENCE (Revised December 2015; August 2017) I. PROGRAM TO BE REVIEWED Program Name MozaCajú Program Location Cabo Delgado, Nampula, Zambezia Mozambique Program Budget $9,036,600 Program Start Oct 2013 Issue Program Addresses Program Goal Increased financial benefits for cashew processors and farmers Implementing Agency and Partners TechnoServe, AKF Evaluation Type (Mid- course) Mid-course Evaluation timeframe October 15 2016 to March 2018 (revised August 2017) Final Report deadline April 30, 2018 (revised - Final Report Loaded to USDA- FAIS) Background on TechnoServe and the Program TechnoServe is an international non-governmental organization that promotes business solutions to poverty in the developing world. TechnoServe's mission is to work with enterprising people in the developing world to build competitive farms, businesses and industries. It does this by linking people to information, capital and markets. Active in Mozambique since 1997, TechnoServe is registered in the US as a 501(c)3 nonprofit corporation and headquartered in Washington, DC. Its staff of over 1,000 employees operate from 30 country offices in Africa, Latin America and Asia. For more details on TechnoServe and its work in Mozambique see: http://www.technoserve.org/ MozaCajú is a three-year grant agreement funded by the United States Department of Agriculture (USDA), and implemented jointly by TechnoServe (TNS) as the principal grantee, and the Aga Khan Foundation-Mozambique (AKF) as a sub- grantee. The project supports small cashew producers and processing factories in line with the Government of Mozambique’s ongoing efforts to reduce poverty. This Food for Progress-funded project supports the cashew industry by harnessing global market demand for premium cashew kernel and addressing obstacles throughout the value chain, including in the areas of inputs, production, processing, and marketing. In total, this project will help support approximately 26,000 small cashew farmers by increasing their productivity and competitiveness as well as connecting them to new markets. MozaCajú Endline Evaluation Page 59 The project responds to increased demand for traceable cashew kernel as well as new commitments by several key global retailers to pay a premium for this guarantee. The project is introducing a sustainable system of shared value and traceability throughout the cashew value chain in Northern Mozambique. Among other benefits, traceability down to the individual farmer level facilitates organic certification and supports food safety audits. Toward this end, MozaCajú is working with seven (effectively all) of the top Northern Mozambican processors to achieve the highest level of food safety systems in order to supply some of the world’s most demanding markets, including the United States which represents 40 percent of all global cashew demand. Over 40 percent of Mozambican farmers or nearly one million households grow cashew, one of the few reliable cash crops that farmers are able to grow in the country. Smallholder farmers are responsible for more than 95 percent of the country’s cashew production. In Nampula province, cashews account for nearly one-fifth of total household income and approximately two-thirds of total cash income. This project supports producers directly through agricultural extension, nursery development, and demonstrationplots. Several findings from this baseline survey of cashew farmers in the MozaCajú project areas are worth noting: The average number of cultivated trees per household is around 145, with almost 200 trees per farmer in Cabo Delgado, 120 in Nampula, and 100 in Zambezia. Productivity is very low, with average productivity per tree of only about 3 kg/tree, and a median production of only 1.6 kg/tree. Interestingly, the productivity per tree is significantly higher for households in the highest category of sales, at over 4.5 kg/tree compared with less than 2.5 kg/tree for the other sales categories. This may be explained by the fact that a larger proportion of farmers in the highest sales category apply fertilizer and spray their trees. Thus, there appears to be significant scope for increasing the productivity of cashew producers within the MozaCajú project area, and smaller farmers appear to have particular potential to catch up with the performance of the largerfarmers. Access to support and services is very limited for cashew producers in the project area. Overall access to training is very low. Only about three percent of interviewed farmers reported that they received any agricultural productivity training. Likewise membership in groups is low, with fewer than 20 percent of interviewed farmers reporting participation in groups. Only about 10 percent of farmers report that they have access to any sources of credit to invest in cashew production. The overall situation of cashew producers in the MozaCajú production area is one of low productivity, but with much scope for adoption of key techniques, and provision of training, access to improved inputs andcredit. The processors interviewed in the baseline do not face major marketing problems for whole kernels, but several of the firms reported that they had difficulties in finding buyers for pieces. However, even though they can presently find buyers for whole kernels, all the companies are aware that the international market for cashews is changing and that they need to make changes in their operations to meet more stringent demands. Since all of the companies interviewed sell the majority of their kernels to US and European markets, they are particularlyaware ofthe growingdemands ofbuyers inthese two markets. This perceived need falls squarely within the scope of the MozaCajú project, which has a focus of helping Mozambican processors to meet needs of buyers – traceability, organic, and fair trade. The impact of MozaCajú will be reinforced by the implementation of the M&E plan, its products will support the project management team and provide USDA, local partners and other stakeholders with critical information to track progress and the best alignment of the MozaCajú Endline Evaluation Page 60 resources to activities. Its evaluations will include three major surveys, baseline, mid-term evaluation and final evaluation. The project baseline was conducted during October to December 2013; Midterm evaluation will be done from October 2015 to February (revised to March) 2016 and final evaluation during the final four months of the program’s implementation. However, while the agreement dates place the project end date at September 30, 2016, TechnoServe has obtained a no-cost extension to March 2017 to include the third harvest season, in order to properly measure the full impact of this 3-year program. The fieldwork for the Endline study will be conducted October – January, and the final report delivered in March 2017. Endline Evaluation The purpose of the Endline evaluation is to review and assess implementation results of MozaCajú’s activities on cashew farmers, processors, and consumer goods retailers in achieving the intended outcomes, and to help project management determine course corrections needed to fully achieve project results in the remaining half of the project. For each component, the Endline evaluation will compare progress against benchmarks set during the baseline study and document lessons learned. The Endline evaluation will cover all three target provinces and, through direct data collection of stratified samples of project beneficiaries and non-beneficiaries, will provide estimates of the project impacts on project beneficiaries by comparing the results of beneficiaries and non-beneficiaries at the time of the due project end in relation to the baseline values of performance indicators (adoption of technologies, yields, profits from cashew production). A structured survey questionnaire similar in format to the baseline and mid-term as well as select qualitative research will be used. In addition, the Endline evaluation will examine the quality of project design and the appropriateness of resource allocation. The Endline evaluation will draw lessons learned to create a set of recommended course corrections to relate all targeted results and identify any issues that might have prevented project activities from being sustainable upon the project’s completion. The review process will take a participatory approach, involving interviews with key stakeholders including USDA/FAS/MES staff, and FAD program administration, the Mozambican government agencies involved in the cashew industry such as INCAJU, implementing staff at TNS, including MozaCajú staff, and AKF, and relevant private sector firms involved in the project. These lessons learned will also be disseminated to all project partners as describe below under “Audience” fed into USDA’s learning agenda. Endline Objectives The evaluation process is intended to be participatory, and result in findings and recommendations that are valid, insightful and useful. While relevant program staff and key program stakeholders will be involved cooperatively to the extent possible, as this is an independent, third-party evaluation, the selected evaluator will control the design and implementation of evaluation activities. Overall, this evaluation intends to assess the extent to which the objectives of the program are in the process of being attained, help capture and articulate lessons learned so far, and make observations and recommendations on project improvement to TechnoServe, AKF, and USDA. Accordingly, the evaluation will focus on the following 5 componentareas: MozaCajú Endline Evaluation Page 61 Overall: Key questions to answer How well does the overall project design match the needs of the targeted beneficiaries? How well does the project align with national and regional development priorities as well as the development objectives of USDA and the US Government? Have project resources been allocated in such so as way as to allow activities to achieve the desiredoutcomes? What are the benefits key Mozambican stakeholders such as INCAJU (the national body responsible for the development and promotion of cashew) receives from the project’s influence and how can these be sustained? Component 1: Facilitate Buyer/Seller Activities: Market Access: Facilitate Buyer-SellerRelationships Relationships Market Access: Facilitate Private and Public Partnerships Training: Improved marketing andbranding Key questions to answer How many new sales to new global buyers have been achieved by MozaCajú’s partners as a direct result of the project’s marketing work? How have attitudes of targeted international consumer goods retailers changed towards Mozambican cashews with regards to the factors identified during the baseline to affect sourcing decisions? Have consumer goods retailers set up any community investment projects, and what business value do the retailers intend for these projects to provide? How would targeted social investments affect the value drivers of brand trust established during the baseline consumer market research? MozaCajú Endline Evaluation Page 62 Methodology The evaluation process is intended to be participatory, and result in findings and recommendations that are valid, insightful and useful. While relevant program staff and key program stakeholders will be involved cooperatively to the extent possible, as this is an independent, third-party evaluation, the selected evaluator will control the design and implementation of evaluation activities. Hired external evaluator will survey participating processors to determine the number and nature of any purchase agreements signed in the project. The evaluator will also review data pertaining to the targeted consumer goods retailers to determine whether attitudes towards Mozambican cashews have changed with regards to the key factors identified as affecting sourcing decisions. Interviews with a subset of retailer representative with regards to the reasons behind the change in attitude (i.e. whether the change is due directly to a factor affected by project activities) will complement this data. MozaCajú Endline Evaluation Page 63 Component 2: Set Up a Traceability System and Support Innovation Activities: Technical Assistance: Certifications Technical Assistance: Traceability Training: Post-Harvest Processing Training: Sanitary and PhytosanitaryStandards Key questions to answer Are firms making the necessary investments to up traceability systems in order to run traceability trials, food safety audits, and to receive organic or Fair Trade certifications? How does progress in setting up these systems compare to benchmarks established during the baseline? Methodology External evaluator will examine project records and survey participating processors to determine the extent to which the firms have made the investments in resources (personnel, technology, etc.) necessary to establish the various systems. This progress will be compared against the benchmarks determined in the baseline. Component 3: Expand SME Access to Finance Activities: Financial Services: Facilitate Agricultural Lending Financial Services: Leverage Private and Public Investment Financial Services: Provide SME Finance Key questions to answer Are firms accessing increased financing due to the loan guarantees and other project financing support? Have the attitudes of actual and potential private sector financers towards providing financing to the processors changed due to MozaCajú’s financing support activities? How does this progress compare against benchmarks established during the baseline? Methodology External evaluator will examine project records as well as surveying both processors and financers to determine what financing has been achieved by the activities, how attitudes of financers have changed, and how this compared against benchmarks. Component 4: Train Farmers on Improved Production Methods and Farm Management Activities: Training: Improved Agricultural Production Techniques Training: Post-harvest handling Training: Improved farm management Training: Demonstrationplots Capacity Building: Agricultural extensionagents/services MozaCajú Endline Evaluation Page 64 Key questions to answer What practices or technologies are being adopted by beneficiary farmers through the model? Hasthe FPA extension approach proven effective in encouraging the uptake of new practices ortechnologies? How did this practice or technology affect the yield of beneficiary farmers? What farm management practices have been adopted by beneficiary farmers through the model? What is the cost of the FPA model per farmer reached? Methodology The third-party independent evaluator will measure the impact training on adoption of improved practices, and yields of farmers who have received training compared with those who have not received training. At the time of the Endline survey, the treatment and comparison groups will be identified as follows: - Participants in the treatment group will be randomly selected from farmers who have received training - Participants in the comparison group will be randomly selected from communities located out of the reach of training activities. This geographic separation will prevent spillover and crossover effects from occurring due diffusion of training to farmers. A third party evaluator will create an Endline survey tool to test interim outcomes and early indicators of success. A structured survey questionnaire, incorporating some of the key questions as in the baseline, but focused more exclusively on data relevant for the core outcome-level indicators, as well as select qualitative research will be used. Final results of the treatment and beneficiary groups will be compared to provide an initial estimate of the impacts of access to nurseries and other inputs. Note that this estimation may be confounded by underlying differences in characteristics of cashew farmers in the beneficiary and control groups. Component 5: Improve Access to Inputs and Support Services for Farmers Activities: Inputs: Develop agrodealers and/or other input suppliers In-kind Grants: Inputs MozaCajú Endline Evaluation Page 65 Key questions to answer Methodology How many farmers were planting new cashew trees due to the nursery system and at what rate are trees planted? Were these trees being planted in such a way as to lead to likely long-term survival (i.e. with proper germiling/grafting methods)? Were the nurseries sustainable? Were they viable long-term as individual enterprises, and as a collaborativenetwork? Were other agro-dealers (e.g. rural shops) being developed and sustained in such a way that they can act as market-oriented input suppliers over the long-term? The evaluator will measure the impact of the nurseries model on tree survival rates and costs to farmers. The third-party independent evaluator will establish whether there is a basis for an end of project quasi-experimental, differences-in-differences analysis to evaluate the impact of improved access to inputs and support services, if data quality is found to be sufficient. After nursery locations are determined, the treatment and comparison groups will be identified as follows: 1. Participants in the treatment group will be randomly selected from communities within reach of nurseries 2. Participants in the comparison group will be randomly selected from communities located out of the reach of nurseries. This geographic separation will prevent spillover and crossover effects from occurring due to nursery outreach and local radio advertising, while allowing for open selection. An independent evaluator will create an Endline survey tool to test interim outcomes and early indicators of success. Final results of the treatment and beneficiary groups will be compared to provide an initial estimate of the impacts of access to nurseries and other inputs. Note that this estimation may be confounded by underlying differences in characteristics of cashew farmers. In order to address this issue, if feasible a differences-in-differences methodology would ultimately establish independence of the treatment variable (access to inputs) from other factors affecting farmer income by controlling for all observed and unobserved characteristics that are consistent over time between farmers in the treatment and comparison group. It will also account for external, environmental changes that have similar impacts over time in the treatment and comparison groups, and confirm that behavior change over time between the two groups. The difference-in-differences comparisons will be made between the mid￾term and final rounds of the farmer surveys. Comparison of results between the beneficiary and control groups across the mid-term and final survey rounds will provide estimates the differences-in-differences of the identified indicators. MozaCajú Endline Evaluation Page 66 Audience The main audiences for this Endline evaluation are TechnoServe’s Mozambique and global leadership teams, the USDA, and other USG agencies, AKF, global retailers, and program partners –the Mozambican Ministry of Agriculture, INCAJU, the Ministry for Coordination of Environmental Action (MICOA, the Center for the Promotion of Agriculture (CEPAGRI), National Directorate of Rural Development, and the Institute for Agriculture Research (IIAM). This evaluation is intended to measure results against outcome indicators and issues around efficiency of resource use and quality ofimplementation. Evaluation Methods TechnoServe seeks an evaluation approach that is appropriate for the scope of the program, intended audience and available resources. Evidence to be used will come from both quantitative and qualitative research methods, and include: primary sources – key informant interviews, focus groups, beneficiary stories, etc., and secondary sources – planning documents, program documents and reports, monitoring records, and baseline data, etc. MozaCajú’s activities are carried out in three provinces, Cabo Delgado, Nampula and Zambézia and across 16 districts from which some will be randomly selected. Farmer selection will be done based on Farmer Field Schools. This strategy will allow us to sample across the districts. The main objective is to stratify samples into farmers trained by the project according to “beneficiaries” versus “non- beneficiaries” and an additional methodology shall include “sprayed trees” versus “non-sprayed trees.” Upon evaluation start-up, MozaCajú’s program director and TNS M&E staff will support the lead evaluator to review program documentation and other relevant information and develop the evaluation work plan. A list of reference material that will be sent to the evaluator in advance of field work includes the following: • Project documents • Baseline report • Semi-annual M&E report(s) • Q&A or Interviews with key projectstaff MozaCajú Endline Evaluation Page 67 Key Activities and Deliverables The dates below presents an approximate summary work plan. TechnoServe anticipates a total lead evaluator level of effort of approximately 40-45workdays. Evaluation Key Dates (revised December 2017, February 2018) December 22- January 13: document review and remote management briefings January 14 – January 30: Staff interviews and Fieldwork for Data Collection February 5: MozaCajú Steering Committee Meeting presentation and feedback February 6-28: remote interviews with key stakeholders; analysis February 29-March 25: Report-writing March 31: Draft Deliverable Due to TNS April 15: Final Deliverable Due to TNS April 22: Report sent to USDA April 22-May 5: USDA review period May 10: Response to USDA comments (by TNS and Evaluator) May 15: Final Midterm Evaluation upload to FAIS Findings from the evaluation will be presented by contractor, or jointly with TNS, to USDA Washington, DC, USDA Pretoria and USDA Maputo,Mozambique. Additionally, TNS will produce a Midterm evaluation summary for external audiences, including retailers and a Portuguese version for the Governmentof Mozambique. Evaluation Logistics: The external evaluator will be responsible for: • Managing the evaluation process; • Developing, testing and applying any instruments/questionnaires • Leading the interview process (TNS to provide field assistance); TNS/AKF staff will facilitate the process rather than assume direct involvement, i.e. identify and introduce key individuals • Analysis of data; • Preparing and presenting all draft and finalreports. MozaCajú Endline Evaluation Page 68 TNS-Mozambique and AKF staff will be responsible for: • Providing the evaluator with copies of all TechnoServe background documents (proposal, grant agreement, reports, work plans,training materials, etc.) • Introduce evaluator to key informants as needed, assist in identifying/engaging suitable individuals for agreed-upon special data collection. • Assisting with logistics. . Unless otherwise agreed, all travel and logistical arrangements will be made by TechnoServe business advisors and administrative staff. Deliverables: All deliverables shall be submitted electronically, in both MS Word and PDF files. As possible, all photographs or other graphics/figures in the reports will also be submitted as separate editable files. Finally, the evaluator should also submit the final cleaned dataset used to inform the analysis. Evaluation Report Structure • Title Page • Executive Summary (maximum three pages) • Program description (including goals, objectives, andoutcomes) • Underlying program strategies that were used to achieve the program’s goals • Maps of program geographies (Annexed) • Evaluation purpose, methodology and approach • Evaluation findings, with reference to documented evidence. Use of specific examples, anecdotes and photographs to illustrate key points isdesirable. • Conclusions: insights into the findings; reasons for successes and failures; innovations • Lessons learned, barriers to success • Industry challenges and opportunities goingforward • Recommendations(based on evidence and insights) for both TechnoServe and the USDA Annexes to the evaluation report o Terms of Reference for the evaluation o Evaluation work plan with timetable o Sampling strategy o Data collection tools, including questionnaires, interview guides and other tools as appropriate o List of individuals interviewed and of stakeholder groups and/or communities consulted o List of supporting documentation reviewed Selection Criteria/Profile of the Evaluator The ideal candidate to lead the external evaluator woulddemonstrate: • Strong relevant experience in designing and leading multi-faceted program evaluations with experience working in private sector agro-industry; • Background in international development and poverty-reduction projects; • Ability to facilitate and relate to stakeholders at multiple levels (e.g., TechnoServe leadership and field staff, private sector business owners and managers, senior and junior government staff, program participants, community leaders workers and farmers; • Ability in quantitative, qualitative and participatory evaluation methods; • Data analysis and presentation skills; • Sensitivity to cultural/historical contexts. Additional training or expertise in anthropology, sociology or historical research viewed favorably; • Knowledge of Portuguese language helpful. Submission and review of proposals to implement the evaluation Evaluators meeting the above criteria are invited to a proposal via email to TechnoServe at bpaul@tns.org (Use the subject line “Application and Proposal for Cashew Evaluation.” Application should include the following: • CV of the applicant; • A writing sample; • An outline of a proposed evaluation design, methodology and general evaluation approach • Estimated evaluation budget, including the applicant’s daily rate, required additional non￾TechnoServe personnel, etc. • List of 2 referees who can attest to the applicant’s experience and expertise as it relates to this project and this ToR. Proposals are expected from interested parties that meet the requirements by COB on August 21, 2015 (replacement evaluator appointed in December 2015 and contracted 11 January 2016, for mid-term and Endline evaluations). Supervision The evaluator will work with the MozaCajú M&E staff, which includes TNS and AKF. The independent evaluator will be supervised by TNS’s M&Eteam. End of Document Appendix 2: Evaluation Methodology This Appendix describes the methodology employed for this evaluation. Relationship to MTE: Similar to the Midterm Evaluation (MTE), this end of project evaluation was focused on: (i) assessing the degree to which MozaCajú succeeded in achieving its stated objectives, across its different components, as well as identifying where the project fell short of its aspirations and why; and (ii) capturing representative data from a sample of project beneficiaries that could be extrapolated to the project population. In doing so, it would then be possible to establish final measures for the project’s output and outcome-level performance indicators. The Endline evaluation was designed to directly build-on the MTE, utilizing – wherever possible – similar assumptions and methods. Like the MTE, the Endline employed three complementary data collection approaches: (i) a quantitative survey of beneficiary and non-beneficiary farmers (ii) a complementary qualitative study of small/medium farmers, nurseries and project promoters, as well as (iii) qualitative interviews with larger farmers, processors and buyers. These approaches were augmented by interviews with the MozaCajú project team (from both TNS and AKF) and with key partners, including local government economic development authorities, major processing companies and, in particular, INCAJU. Extensive document reviews complemented these actions. Used together, these approaches provided a relatively good triangulation of data and constituted a validity check for the Endline findings and recommendations. We describe the selected quantitative and qualitative approaches in the following Sections. Quantitative Survey of Beneficiary and Non-Beneficiary Farmers The following Sections describe the design of the survey based data collection that aimed to relate the data collected to those obtained in the MTE, providing a longitudinal series (where appropriate) for analysis. Quantitative Farmer Survey - Questionnaire The Endline farmer survey was focused primarily on collecting outcome-level data relating to the project’s six key production indicators. The structure of the questions asked in the Endline, as with the MTE, related to the six outcome indicators by following the guidelines set down in the project PMP – e.g. the PMP expects that yield will be determined by dividing total production by the # of productive trees, and that farmer income will be determined on the basis of determining sales and multiplying this by a reference price; that # of hectares under improved techniques will be determined based on asking the # of trees to which farmers are applying these techniques and multiplying this by an average stocking density; and so on. The Endline varies from the MTE in aiming to be more expansive. In the MTE, the farmer survey questionnaire was tailored to focus on these indicators rather than trying to be a more broadly evaluative tool, whereas in the Endline attempts were made to modestly expand the scope of the Endline survey questionnaire to capture a wider variety of data. These included questions around MozaCajú Endline Evaluation Page 67 whether MozaCaju met the expectations of the farmers, farmers’ perceived benefits from applying good agricultural and/or farm management practices, germination rates for planted seedlings, as well as key project impacts for farmers. The written questionnaire was developed in Portuguese and transposed to a flexible software platform for use on mobile devices. In the field, the questions themselves were administered orally in either Portuguese or in one of Northern Mozambique’s three local languages: Macua, Makonde or Kimwani. The enumerator training that took place prior to data collection allowed for the verbal translation and back translation of the questionnaire between Portuguese and the different local languages, with enumerator teams being selected in part on the basis of local language ability. This was particularly the case in Cabo Delgado, where each of the three local languages is prominent. Quantitative Farmer Survey - Sampling The Endline farmer survey was similar in scale to the survey carried out for the MTE. However, a decision was made to slightly reduce the beneficiary sample size for the Endline to a more manageable but still representative 850 beneficiaries (down from 1,000 beneficiaries in the MTE) while keeping the MTE sample size of 200 non-beneficiaries to serve as a control. From the outset, the Endline survey was intended to be, at least in part, a longitudinal study in relation to the MTE, with a sub-sample of Endline farmer respondents being inclusive of farmers also sampled in the MTE. Through this approach, it was assumed that the Endline would enable the project to collect a wide range of key data, particularly on production and incomes, which could be compared for the same beneficiaries over time. An important point of consideration in planning the Endline farmer survey, however, was that at the time of conducting the MTE, not all project beneficiaries had been active in the project long enough to be deemed eligible for inclusion: a cohort of only around 8,000 beneficiaries had been with the project for the 8-12 months deemed necessary for the project to influence their production and incomes at the time of the MTE (conducted November 2015 – February 2016). The remaining project beneficiaries (around 15,500) were not eligible for inclusion in the MTE and instead constituted a second cohort for whom the full effects of project interventions were expected to be seen only at Endline. Since an exclusively longitudinal approach would have allowed for the inclusion of the first but not second cohort, a compromise was decided whereby part of the total sample of 850 beneficiaries would be drawn from this first cohort (namely, 290 respondents) and comprise a longitudinal sample while a second part of the beneficiary sample would be drawn randomly from the second cohort (560 respondents). The beneficiary sample was selected through the use of two sampling frames: the sample for the initial cohort also sampled in the MTE was drawn from the MTE survey database, while the sample from the second cohort was drawn from the MozaCaju project farmer database (excluding that initial cohort). The original sample was under-sampled for Cabo Delgado due to logistical issues relating to distance and the time available. To ensure that the results represent the project’s population, the analysis wasre-weighted using the weights below. Asfar as possible, the sample actually interviewed aimed to match the Treatment (or beneficiary) group with Control (or non-beneficiary) group, but farmer availability was sometimes problematic, as discussed further below. Gender representation also presented challenges; while the Treatment sample mirrored the project population’s 19.3% women farmers, enumerators struggled to engage with that proportion of female farmers in the Control group due largely to the lack of sufficient female enumerators. MozaCajú Endline Evaluation Page 68 MozaCaju sample weightings related to Population Farmerselection was done randomly through a systematic sampling approach, with the sample being drawn proportionally to the total beneficiary population in the three project provinces of Cabo Delgado, Nampula and Zambezia. Following the sample selection, the evaluation team also reviewed the sample on the basis of district and farmer category (small, medium, large, commercial) to check sample proportions vs. proportions in the overall population. The Treatment sample was found to have been appropriately allocated in relation to population actuals. Inactive farmers (some 2,000 of them, by the time the Endline was to be conducted) were not filtered out of the population to be sampled; this was because farmers can alternate between active and inactive states, and the project operations did not monitor those changes routinely. The sample therefore would likely include inactive farmers, representative of the project as a whole. This would allow extrapolations to be made from the project sample back to the population without filtering manipulation. No sampling frame existed with a comparable list of non-beneficiary farmers; the data do not exist. As a result, the selection of the Control sample was intended to be done purposively through a process of ‘matching’ on the basis of specified creiteria. The evaluation team in the field sought to identify cashew farmers, in villages outside (but near) MozaCajú’s intervention area, and select farmers in consultation with community leaders, who broadly matched them with equivalent beneficiary farmers on key characteristicssuch as age, gender, number of productive trees and access to extension support. Province Population Sample Re￾Weight Cabo Delgado 35% 27% 128% Nampua 58% 61% 96% Zambezia 7% 12% 58% MozaCaju sample participation by Province Province Sample Category Control Treatment n % n % Cabo Delgado 90 37.2% 210 24.7% Nampula 127 52.5% 535 62.9% Zambezia 25 10.3% 106 12.5% Total 242 100% 851 100% MozaCaju survey participation by Gender of Producer Gender of Producer Sample Category Control Treatment n % n % Male 217 89.7% 687 80.7% Female 25 10.3% 164 19.3% Quantitative Farmer Survey – Implementation Following an inception meeting and orientation with the MozaCajú lead evaluator and AKF’s core M&E staff (held in November 2016), a competitively-selected Mozambican external survey company and its enumerator team conducted the survey in February and March 2017. The external survey company organized a three-day enumerator training workshop in Pemba in mid-February, allowing for the training of enumerators, the translation/back translation of the questionnaire, as well as piloting with non-sampled project beneficiaries in the district of Metuge near Pemba. MozaCajú Endline Evaluation Page 69 One of the problems encountered in the MTE was the number of errors in the data that were seen as being attributable to the use of paper questionnaires. Missed skips, illegible recording ofresponses as well as an overreliance on field supervisors to spot and resolve all relevant errors in the field – e.g. relating to range checks and basic logic checks – were all problems identified in the MTE.More important was that the process of data entry from paper questionnaires into a database system introduced additional opportunities for transcription error, and this necessitated a more laborious process of data cleaning. On the other hand, a random spot-check of questionnaire quality was made easier by having hard-copy completed surveys in hand – once they had been physically retrieved, transported and deciphered. The use of paper questionnaires also presented AKF with problems in regards to transportation and secure data storage. To offset these problems as much as possible, we made the decision in the Endline design to move the survey questionnaire to a digital platform. Using open source software (Open Data Kit - ODK), the external survey team adapted the paper questionnaire so that it could be usable on smart phones distributed to each survey enumerator. The pre-execution training sessions included extensive practice with the digital forms and handsets. The paperless questionnaire had a number of advantages, in that the skip patterns were automatically built into the system, as were range and validity checks. Use of the smart phone allowed for the automatic entering of the data and its immediate uploading to a central database also linked to ODK, which eliminated the need for additional data entry and was seen as likely reducing opportunities for introducing additional errors into the dataset. Even with this level of testing and checking, however, we found thatsome additional logic checks would have obviated some of the subsequent data cleaning (for example, capture of claimed total production or total revenue earned did not trigger any alerts in subsequent Sections where unit volumes being teased out exceeded the totals earlier stated – described in the data cleaning and analysis Section). The survey was administered over approximately four weeks from mid-February to mid-March 2017. Once the external survey company had selected the sample, members of the project team helped develop itineraries to orient data collectors in the field and to try and maximize the efficiency of the data collection process. For the beneficiary sample, a process of replacement was agreed prior to the start of data collection. Farmers from the MTE cohort who were not available at the time of the survey were replaced with randomly chosen farmers who were also sampled in the MTE; unavailable farmers from the second cohort who were not part of the longitudinal sub-sample were replaced by randomly selected farmers from lists generated from the project database. Control group did not involve replacement – there was no pre-existing panel – so the project team relied on the management of the consultant company to implement the purpose of search and identification of relevant control respondents. This reliance seemed necessary, given the large geography being canvassed, but therefore entailed a risk of quality control in the actual implementation. Consultants from the external survey company acted as field supervisors of the enumerators, ensuring that the selected beneficiary and control farmers were being interviewed, that replacement – when necessary – was done correctly, and that enumerators were fulfilling their obligations in producing carefully completed questionnaires. The consultants were tasked with reviewing questionnaires to identify obvious errors and to carry out essential logic checks. Quantitative Farmer Survey – Data Cleaning & Analysis It should be noted that the external survey company was initially contracted, following a competitive bidding process, to perform data cleaning and analysis. Their references from other projects (not, it should be noted in agriculture – primarily health) were superior to any agriculture-oriented MozaCajú Endline Evaluation Page 70 consultancy shortlisted. However, as the company’s performance was not deemed to be satisfactory (as detailed further in the next Section), we made the decision to cancel the consultancy contract following the initial round of data cleaning had been done and prior to the start of analysis. Further data cleaning was done by the lead evaluator, AKF’s Director of M&E, as well as by an M&E-specialist TNS member of staff based in Zimbabwe, who was contracted to provide much-needed assistance. The contracted member of staff from TNS-Zimbabwe performed the majority of data analysis, with a number of reviews being done by the lead evaluator and the AKF M&E Director. The lead evaluator performed the majority of higher-level analysis, including cross-tabulations, comparative analyses (including longitudinal, between segments, between beneficiary versus control) and some limited efforts around correlation analysis. Quantitative Farmer Survey – Limitations & Challenges in the Field The farmer survey has several limitations, noted above, largely resulting from unexpected challenges encountered in the field. These include: Overall Data Quality The externalsurvey company did not perform their role to the level of quality that we expected based on detailed briefings. Data quality assurance in the field was found, upon initial reviews of the rough dataset by the lead evaluator and AKF’s M&E Director, to have been particularly poor. There were large numbers of data records for which noticeable and often considerable discrepancies were evident – e.g. between farmer production values and their sales. Most of the glaring errors in the early versions of the dataset were fixable, with some rework of data collection and repair of data errors, but some errorsremained. As much as possible, clearly erroneous values have been excluded from the analysis. The presence of these errors, in spite of the move to a paperless approach to data collection, reveals an important limitation of the paperless approach. Namely, there is still a vital need for field supervisors to understand and carry out logic checks (such as the totals versus breakdowns issue mentioned above); this is however difficult over large geographies within a limited time-span and budget. These checks are difficult to build comprehensively into a digital questionnaire, but the supposed benefits gained from the paperless approach in limiting the opportunity for introducing data errors means little if at least the human logic checks are not performed with enough rigour. Problems in the dataset were often logical in nature – e.g. between levels of production and overall revenues – and these were not given enough attention by the external survey company. Notably, a key area of concern in terms of data quality relates directly to issues around production and sales. Both of these figures are difficult to ascertain with absolute precision, particularly when they are being asked of farmers on a recall basis, and when farmers (very typically) lack written sales records (as noted in the Production Component analysis in the body of this report). This was exacerbated by the lack of any periodic M&E data collection that could have provided trigger thresholds for early warning that the data collection was at times quite substantially deviating from any norms. Even though we had anticipated these issues and designed the questionnaire to approach key calculations from different directions (e.g. recall of total income received; then later, unit-level counts of what was sold and at what price), we saw that such a wide range of data reported were potentially problematic. As a quality improvement contingency step, the lead evaluator and projectteam decided to use project staff members to re-collect a subset of data – particularly focusing on MozaCajú Endline Evaluation Page 71 production and sales – from a very small number ofsurvey respondents(40 in total) in late June/early July 2017, around three months after the survey data collection had been completed. This small data re-collection exercise, while revealing some minor issues with data collected during the survey, found fewer major problems with the data than expected. The general takeaway from this review, admittedly based on a very small number of survey respondents, was that the quality of enumeration was not ideal, but nor was it catastrophic. Problems remained within the data, but the lead evaluator and AKF M&E Director determined that at least we can be confident that there were no major anomalies in the conduct of the interviews. The questions about the content of the findings remained; some of these would need to be attributed to limitations around beneficiary farmer recall, farmer willingness to divulge information, etc. At the very least, the data re-verification exercise established confidence that the dataset, while far from perfect, was workable, in particular for the beneficiary group. Some final data cleaning was done by the AKF M&E Director following this re￾verification work and a final dataset was produced. Understanding the Control Sample results At the outset of the Endline, and based on experiences with the MTE where this issue was anticipated, it was apparent to usthat a fully quasi-experimental differences-in-differences approach to comparing outcomes between a beneficiary and control sample was not realistic. This was noted in the MTE report. We saw evidence in the quantitative data, supported qualitatively in farmer comments, that good advice travels; this occurs as farmers or family members themselves relocate, as farmers swap stories with (non-beneficiary) contacts, and as extension agents visit multiple communities. MozaCajú was not designed to work in clearly delineated and segregated ‘intervention areas’; informal transmission was evident, along with some degree of more formal spillover actively promoted through project activities like radio campaigns which any farmer can access. These blur any attempted clear lines between farmers who have or have not been exposed to the project. Nevertheless, the evaluation team approached the Endline with the desire of at least improving on the MTE, where the control sample was selected poorly and even a general comparison between beneficiary and control was problematic. This situation did not improve appreciably at Endline. A comparison of beneficiary and control data revealed a number of anomalies, particularly in terms of prevailing cashew sales prices and production yields; whereas the numbers of GAPs applied, according to the self-reports, were low, the production claimed to have been sold in 2016-17 year were high. Overall sampling was skewed by 12% with the control group, meaning that on average the farms were larger. The short duration of the study meant that by the time we observed these anomalies in the data, the survey process had already been completed. One hypothesis we formed was that a number of the control farmers, being encountered “cold” by an unknown enumerator, not knowing the project formally and not knowing how the data would really be used, offered data of varying quality – some understated, many perhaps aspirational. Indications in support of this hypothesis included the number of recordsthat needed to be discarded before we had accumulated the targeted sample of 240 control group farmers; those discarded records typically had many incomplete Sections, and sometimes wildly divergent data [e.g. the relationship between number of trees and claimed production, with impossible yields]. Data cleaning on the records that remained was more extensive with the control group, to remove typographical errors and other markers of unsatisfactory and perhaps rushed questionnaire completion. It was impracticable to try to test this rigorously. MozaCajú Endline Evaluation Page 72 Another likely explanation is that the selection of the control sample was done on the basis of convenience rather than through a rigorous attempt at ‘matching’. This convenience sampling would have seen the controlsample be selected from among those non-beneficiaries who were the simplest to access – e.g. located in district centers, concentrated in particular locations, and larger farmers who may be more likely to be at home when the enumerator passed by rather than pursuing some alternative form of employment at that moment, etc. this might help explain the relatively higher variance and results [prices obtained, yields per tree, etc.] within the control group findings relative to the beneficiary group. The average control group farmer had 12% more trees (187, versus 167) than the beneficiary group, suggesting a significant skew to larger farmers; similarly, we observed many more. farmers with no sales in the control group than in the beneficiary group. Overall, the anomalies we detected cast doubt on the reliability of the control data, even after cleaning. We believe that any subsequent analysis should footnote that findings may only be indicative, rather than definitive; the body of this report includes this disclaimer in relevant Sections. With these limitations noted, for commercial data in particular, we found that the behavioral data collected – e.g. planting of new trees, GAPs applied to trees – were more consistent (with the MTE findings, and within the overall control dataset) and gave us a higher confidence in making comparisons with beneficiaries in the analysis. Qualitative Study Design The restricted approach to Monitoring and Evaluation (M&E) from the outset provided challenges beyond gathering timely data that might affect decision-making by the project management. In the absence of more frequent programmed data collection, management relied extensively on field visits and interactions with leading actors – commercial farmers, nurseries, processors and periodic visits to representative farmer producer groups, to gain better understanding of perspectives, process effectiveness, issues and opportunities, and overall satisfaction with the project. In doing so, the analysis was continuously aiming to iteratively consider how and in what ways the data – derived from different project actors – could be fit together to construct causal explanations and, in turn, explanatory theories around project performance that could be tested against the complementary quantitative data. This approach heightens the reliance on the MTE and Endline to provide a formalized and independently objective assessments of progress, challenges and learnings. In addition to the quantitative surveys and analyses, outlined in the previous Section, the PMP identified the need for qualitative study and analysis to develop insights into the causes of phenomena observed, the perceptions of key players internally and externally, and the learning that can be taken from the project. Given the wide span of communities in the value chain addressed by the project – from the smallest farmer to large exporting factories – the design called for a range of analyses and tools to be deployed for that qualitative evaluation, broadly clustered into four focal topics: 1. farmer producer groups, nurseries and promoters, as a closely linked network (qualitative study number 1) 2. larger/commercial farmers, processors and international buyers, as a looser but frequently directly transacting network; plus nurseries, as a semi-commercial trial to stimulate inputs 3. grant recipients 4. government authorities directly related to cashew activities, project team and its processes (ii-v: qualitative study 2). The evaluation methodology is set out for each of these in the following Sections. MozaCajú Endline Evaluation Page 73 Qualitative Study 1 – Producer Groups, Nurseries and Promoters Prior to the MTE, the smaller farmers were studied in a separate qualitative exercise led by the project’s local M&E Assistant, due to logistical considerations (travel duration, local languages). The study’s results were subsequently validated with the lead evaluator and incorporated into the MTE in early 2016. Focusing on the perceived benefits that different stakeholders identified from their participation in the MozaCajú project, the aims of this small study were consistent with the MTE, i.e. to gain insights into how stakeholders perceive the project, its intervention activities, actual results and any drawbacks or negative learnings to take from the project. The interviewed stakeholders were Farmer Promoter Agents (FPAs), nursery owners and Farmer Producer Groups themselves, withthe farmers representing a range of levels of investment in their cashew orchards up to ‘medium’ size. AKF’s M&E team undertook a second round of qualitative data collection and analysis of these groups was carried out in September 2016, approximately twelve months following the similarly scoped study in 2015 and around six months prior to the implementation of the Endline. This was done with the objective of understanding the extent to which (if any, and how) MozaCajú interventions were benefiting the project’s targeted farmers. The focal interventions related primarily to trainings on topics that moved past the basics to encompass improved production, post-harvest handling and farm management techniques. The study again complemented the farmer-level assessment by examining changes in their access to inputs resulting from the introduction of community nurseries; and reciprocally, we wanted to understand how nursery owners perceived their emerging businesses. The third grass-roots constituency was the village-level Promoters (FPAs); the study was designed to garner the perspectives of FPAs on both their contributions to the project and benefits from the project, as well as understand their motivation (average tenure in role: 2.1 years) and their satisfaction with the support to them provided by project management. A further aim of this study was to identify, in a limited fashion, the extent to which issues raised during the project’s mid-term evaluation appeared to have been addressed. In short, the 2016 qualitative study was conducted to provide evidence on the progress of project activities and to shed light on issues that the external evaluator could then focus on in more depth as part of the project Endline evaluation. Methodology Since this was intended to be a follow up study, the methodology adopted is the same as in the September 2015 study: a qualitative methodology was adopted, consisting of a mix of Focus Group Discussions (FGDs) and in-depth interviews (IDIs), via a purposive sampling approach: (i) Focus Group Discussions: This design aimed to capture the complexities of human experiences, attitudes, perceptions and values, which in fact are crucial to effectively and coherently materialize the project goals. The study would be limited logistically in scope to 8 FGDs with members of project-supported producer groups: 2 in Cabo Delgado, 5 in Nampula and 1 in Zambezia. The study focused on the production component of the project and provides in-depth information on the extent to which MozaCajú interventions are benefiting the project’s targeted farmers. (ii) In-Depth Interviews: In addition, the M&E Assistant conducted 5 individual interviews with producers (2 in Cabo Delgado, 2 in Nampula and 1 in Zambezia), and with 16 FPAs (6 in Cabo MozaCajú Endline Evaluation Page 74 Delgado, 6 in Nampula and 4 in Zambezia) as well as with one nursery owner. Respondents sometimes had multiple roles: of the 16 FPAs interviewed, 6 served also as nursery owners. Given that one of the objectives of the study was to analyze the sustainability of both the FPA model and the nurseries, we decided the most productive use of time wasto purposively interview those individuals who were most closely involved in both activities. (iii) Sampling: Interviewee selection was carefully considered: the selected interviewees were chosen from a list of FPAs nominated by MozaCajú facilitator staff as people most likely to continue operating as cashew-focused extensionists and/or nursery operators in the future, making their inclusion logical from the standpoint of a specific focus on issues of project sustainability. The producer groups included in the study were selected through a systematic sampling process from a list of all the groups MozaCajú was assisting; as far as practicable, given travel distances and costs, the design tried to avoid sampling more than one group per district. For similar reasons, FPAs and nursery owners were purposively selected based on their proximity to the selected producer groups in a given area. Each FGD lasted for about an hour while the IDIs took about 30 minutes each. In most of the districts, the team began by interviewing FPAs, which helped forestall the research team’s blank spots, provided good contextual information and raised important questions that should be geared towards the different conditions and experiences of each of the producer groups, including seeking expressions of negative experiences with the project. The AKF M&E Assistant for MozaCajú, an MBA based in Pemba, conducted the FGDs and interviews, as he had done for the MTE. The study adopted semi-structured questions designed by the M&E team, based on that MTE experience. To allow the respondents to freely speak up and in an unbounded matter fully express themselves, the data were collected in the respondents’ preferred language, which varied between Portuguese and Macua. All the data were collected face-to-face and audio-recorded with the consent of the respondents. Key Sections of the tape-recorded data were verbatim transcribed to enable systematic analysis combined with analysis of field visits and notes taken during the FGDs and IDI served to triangulate the data, thus ensuring the trustworthiness of the data set. Limitations: As with the MTE, we would not over-rely on the findings from any one study due to the logistical, time and participation constraints. The major limitations that characterized the smallholder-oriented study outlined above were similar to those of the MTE: (i) Methodology: As with the iteration at MTE time, the research was conducted again in a relatively high period of cashew cultivation, meaning that participation (including gender balance) was to an extent uncontrollable and subject to ‘whoever turned up on the day’. FPAs helped mitigate this by encouraging participation so that adequate numbers of farmersdid participate, mitigating against biases. (ii) Reach: Due to limited resources and time the study only reached a limited number of people and districts. This was heightened somewhat by the imminent departure of the M&E Assistant who was soon to resign from AKF. (iii) Generalizing conclusions: Consistent with the nature of most qualitative research, the results cannot be generalized and there is susceptibility to biases on the part of the researchers – the risk that they were unconsciously oriented towards success cases. In mitigation of that reflexive concern, the lead evaluator concurred that the study found a number of plausible similarities in responses between producer groups across the different sites. MozaCajú Endline Evaluation Page 75 Qualitative Study 2: Larger Farmers, Processors, Buyers, Nurseries Taking inputs from the small/medium producer study described above, periodic status reports from the project, and from half-yearly status updates on the action plans developed by the Project Director following the MTE, the lead evaluator designed an interview and inspection data collection process that would complement the above with a small number of targeted in-depth interviews and visits to test the key operational and commercial assumptions of the project. A second objective was to collect the quantitative measurements that the project’s M&E process are unable to gather – such as processor volumes, investment and costs; and buyer perceptions of the market opportunity and intentions to procure. The evaluator was supported throughout by the M&E Assistant from the project. Most interviewees agreed to be audio recorded, documents were collected, extensive notes were taken in meetings, and these artefacts were jointly analyzed in an iterative process of analysis. The motivation for this qualitative study remained the same as with the MTE: to deepen the understanding possible from the data to be collected from the farmer survey. The small farmer and field promoter study, together with the initial round of qualitative interviews with larger farmers, provided inputs for survey design and questions, as did the many accounts given in the 18 staff interviews and a wide range of documents provided by MozaCajú staff in advance of the survey. Analysis and interpretation were similarly improved by taking into account the related project records of e.g. grants disbursed and progress reported for SME grant recipients and nurseries, and from of dealings with otherstakeholdersincluding the more commercial-scale farmers, the processor and the kernel export buyers. As with the MTE, production of RCN is at least one key element for success in the whole project: poor production harms farmers and destabilizes the downstream decisions and results of other players such as the processors and the local authorities. An overarching question of these qualitative studies, performed at both the MTE and Endline, was: how can we better understand the relationship between the behavioral perspectives and aptitudes of farmers, for example, and the project’s approach to promoting the adoption of new practices and techniques? Beyond the farmer, the project had aimed to influence the supply of necessary inputs, including services such as spraying and mechanization, and an improved opportunity for farmers and the RCN buyers to transact more constructively and efficiently. A second key question of the evaluation, consistent with the MTE, was: what are the constraints, opportunities and motivations of upstream stakeholders [inputs providers, service providers and extension support agents] downstream stakeholders (processors, buyers), and how well does the project align to create positive linkages for the farmers? The major design difference with the end line responded to the staged termination dates agreed in the final year of the project for specific components. The no-cost extension for specific components or subsets [e.g. a subset of nurseries] allowed the design to take account of an additional year for a more complete longitudinal analysis of the affected components. A two-phased qualitative evaluation was arranged: (i) Mid-November to early December 2016: interviews and observation with larger farmers, farmer￾run nurseries, processor management and projectstaff [field and HQ] prior to expiry of the initial term; assess CommCare ICT service; interview representatives of INCAJU and SDAE in Cabo Delgado and Nampula provinces. (ii) October 2017: visit and interview the project’s grant recipients, revisit processor management for specific topics, interview additional representatives of INCAJU and SDAE in Cabo Delgado and Nampula provinces; process interviews with remaining functional staff of the project. MozaCajú Endline Evaluation Page 76 Larger Farmers The data collection processes and documentary sources noted above had yielded extensive data, often in standardized form; but the issues experienced in the project, such as the vastly different production outcomes of various farmers even within the same segment, prompted a closer investigation of what farmers do and how they make decisions. We chose to investigate this with the farmers operating more complex, multi-crop and often multi-site farms, to tease out the various factors and the related decision processes of actors. In designing the production- oriented component of this qualitative study of larger farmers, and their relations with input suppliers and the formal local buyers, we continued the MTE orientation to focus on developing insights into behavioral elements. The project production base was segmented on a demographic basis – location and threshold levels of productive tree counts. In reviewing the design, we were informed by the small/medium farmer producer group qualitative study noted above, and this reinforced the need to understand behavioral characteristics, triggers and support. For example, the September 2016 study reported large variances in actual scale of farming operations, commercial awareness, motivation to expand production and adopt new techniques, and level of sustained engagement with the farmer field schools and other programs. In-depth interviewing, augmented by observation and inspection of records, allows for an exploration of decision processes and possible causal connections. As with the MTE, for practical reasons we selected a small number of interviewees chosen from listings that allowed in-depth interviews to be conducted with one or two large farmers in each of five selected districts of Cabo Delgado and Nampula, with farmers chosen by the evaluator and be project M&E team for a maximum feasible diversity of activities and resource conditions. Factors in the selection included whether the farmer was known to have invested in mechanization, whether she or he was actively planting new trees, and whether we could discern other indicators of behavioral differences that could lead to insights about farmer decision making. Logistical factors played a part in the ultimate selection; distances are great, making travel times prolonged. Asfar as possible, interviews with other stakeholders would be arranged in the vicinity: farmer-established nursery operations in the project, grant recipients, INCAJU offices and in several locations, the district SDAE official for perspectives on local economic development related to the project activities. These are described in separate Sections below. The farmer interviews ranged from 90 minutes to over 2.5 hours, and followed a similar protocol as with the MTE: • application of the quantitative survey, as used in the broad survey study. This gave us a set of current and its historical data to prompt further questions. This took approximately 30 minutes. • semi-structured interviewing using an interview guide (appended), notes taken in-vivo during the quantitative discussion and points arising in a free-flowing discussion. • inspection of part of the farm and any ancillary operations to understand some of the specific practices and characteristics of the farming being conducted, including diversification in crops, into services or into non-farming businesses. Processors We arranged semi-structured interviews with top management of each processor group. Each had agreed to provide the required commercial data, to be handled in confidence, as part of the evaluation; in these interviews any questions were answered and clarifications made to ensure comparable data to that collected for the MTE would be made available. Interviews ranged from 60- 90 minutes and included lengthy inspection tours at each processing factory visited. The respective MozaCajú Endline Evaluation Page 77 interview guides remained consistent with those used in the MTE; they are appended. Beyond business data collection, which could be partially triangulated by comparing data with reports received by interview to the extent they were made available to us, the interviews explored investment appetite (e.g. further food safety upgrades), the risks and concerns for ongoing expansion, the value of the projects interventions, and reflections on further work needed in the sector for it to maintain or improve its competitiveness internationally. Buyers Four international buyers and had been contacted and interviewed for the MTE. The semi-structured interview guide consisted of requests for business volumes and operational metrics [e.g. rejection rates], in questionnaire format, followed by more open-ended questioning about the competitiveness of Mozambique supply, quality and price issues, perspectives on the project performance and willingness to collaborate in any future activities. The same for buyers where contacted and invited to respond for the end line; due to what appears to be a particularly volatile and busy year due to international supply constraints, none responded with agreement to be interviewed, instead secondary sources were used to confirm that the uptick in buying reported in the MTE has been maintained. Nurseries The producer group qualitative study [above] investigated farmer level attitudes and use of nursery supply of seedlings from the network of 85 distributed nurseries. The nurseries visited were selected from a list identified by the project management as most likely to achieve a level of commercial viability from seedling sales. Project monitoring indicated that the nurseries would achieve or exceed their production target of seedlings; we wanted to know whether the nursery operators themselves viewed this as a successful experience. The MTE made an initial assessment of progress in both adoption of nursery techniques and the development of any signs of becoming self-sufficient. To address the technical, logistical and commercial questions involved, we selected five nurseries proximate to the large farmers selected for interview and conducted in-depth interviews with the owners, supplemented by inputs from farmers and workers on site, and inspected the nursery sites. Each visit typically comprised a 90 minute interview followed by a 60 to 90 minute inspection. Grant recipients The grant program has been relatively closely monitored from its inception, facilitated by the small number of grants made [13 grants, to 11 recipients] and close working relationships between the project and the grantee. Grants were split roughly equally between processing applications and production- related [including inputs]; a simple sampling was not appropriate however due to the very wide spectrum of grant applicants, their scale and the nature of their intended use. Accordingly, the study was designed to interview at least 75% of the active recipients, spanning the micro, meso and macro scale of those recipients; this was largely reserved for the 2016 data collection program, due to the not-unexpected delays experienced by several grantees in getting ready to draw down on the grant and make investments. Three of the four recipients interviewed in 2015 were re￾interviewed in 2016 (one project, Matanuska, had subsequently cancelled due to business issues elsewhere), to acquire some longitudinal perspective on the opportunities and challenges, the support processes from the project and any changes in attitude towards the program. Interviews ranged from 45 to 70 minutes, with inspections of more complex sites each taking up to three hours. Government Authorities The MTE data collection process had limited successin engaging with government authorities, largely due to officials’ lack of availability and logistical challenges in the rainy season. The end line process MozaCajú Endline Evaluation Page 78 was adjusted to take account of this experience. In late 2015, four engineer-managers and one provincial delegate of INCAJU were interviewed, followed up with the scheduling of interviews with two provincial delegates in late 2016. Local economic development advisors were interviewed in 2015 [2: Macomia and Angoche), with a further three in 2016 (Macomia – replacement, Chiure, Meconta/Rapale). A semi-structured interview protocol was used with each, in interviews of typically 60 minutes duration. The MTE had identified that officials responded positively to the early ICT initiative in the project, CommCare; in the 2016 interviews, additional questions were asked in relation to the successor ICT project to identify whether officials were supportive of the operational and policy improvements possible with this extension from the project. Project Staff An extensive interview program of MozaCajú project top management, functional management, advisors and field staff comprising 14 interviews in three locations (2015) and a repeat set of 9 interviews effectively a year later (2016), provided a good survey of the end-to-end activities of MozaCajú, its diverse and sometimes conflicting internal processes. Various process checks were conducted through discussion and inspection – in particular the commercialization process, the CommCare farmer support service, and the grant management process -- sitting with the responsible function heads and stepping through the job steps, records and reports. These steps supported the triangulation of data collected and provided information for assessing the implementation of agreed suggestions for improvement, where applicable, in the MTE. At each farmer and nursery interview site, we requested that a local field promoter and the MozaCajú facilitator joined; where time permitted, regional supervisors joined us at the larger farmer sites. These individual staff members were interviewed to understand their working practices, the processes implemented, the support they received and the services they in turn provided to their farmers. Observation of records kept and training tools used by staff supplemented the interviews and provided a corroboration check. As with the MTE, two MozaCajú field supervisors joined the visits for 2-3 days each; both were interviewed and observed at length to understand their working practices, the support they received and in turn provided to their staff and the promoters, internal processes for technical, logistical and commercial work elements and their views on the project and its areas of impact. This supplemented an extensive individual interview program with MozaCajú project staff and management to collect performance data, request documents and trace the workings of key processes and management systems. Limitations: Any evaluation process involves trade-offs and inherent limitations in what can be achieved, verified and understood in the timeframe and at any particular point in time. As noted earlier, this was heightened by the extensive scope of the project, the very large geography and its communication challenges, and the limitations of the monitoring and evaluation strategy and resources deployed. We made efforts to ensure in the design that each key actor in the chain addressed in the project was given adequate weighting in the data collection and analysis, and attempted to identify confirming or disconfirming sources of information where available. In addition, specific limitations are acknowledged, outlined below together with the steps taken to try to mitigate each one. 1. Language: Throughout these data-gathering exercises, we set as a priority that respondents should be allowed to freely speak in an unbounded matterso they could fully express themselves. MozaCajú Endline Evaluation Page 79 The data were therefore collected in the respondents’ preferred language, which varied between Portuguese, Macua, English and other local languages. This mix of languages even presented difficulties for the project’s highly educated M&E assistant; in one community, our driver needed to step in as translator. The buyer interviews, most of the grant recipient interviews, half of the INCAJU interviews and three ofthe four CxO-level processor interview were conducted in English; written business results were provided in English or translated independently from Portuguese. In the 2015 field-oriented study, data were collected face-to-face and farmer, nursery, field staff and SDAE/relevant INCAJU interviews were audio-recorded with the consent of the respondents, summary findings transcribed and translated into English. Translation was provided by the MozaCajú M&E Assistant (Nov-Dec 2015 study) and the postgraduate, multilingual TechnoServe volunteer/consultant (October 2016), introducing the possibility of bias or embellishment. Key findings were written up at the end of each day and exchanged as a way of identifying any misunderstandings; this facilitated a wider scrutiny by TechnoServe and AKF managers, upon request, with a few simple clarifications being identified as a result. 2. Validation: the paucity of written records at field level, and the difficulty in ascertaining the accuracy of data reported in infrequent project data collection for reporting purposes, along a lengthy paper-based reporting trail, presented challenges in ascertaining the validity of data. Farmers almost universally kept little in the way of written records; some embraced the trainings on record keeping but sometimes demonstrated that their processes were not effective in assessing accurate volumes or profitability. This was evidenced in the extensive data cleaning required in the quantitative survey, even amongst the beneficiary respondents. Several validity check processes were used to ensure – to the extent possible – the trustworthiness of the dataset. In the 2015 field study, all key segments of the recorded data were verbatim transcribed to enable systematic analysis and combined with analysis of field visits notes and documents to triangulate the data. A series of discussions with relevant people – the Project Director, the Regional Cashew Specialist, government authorities and external advisors – helped validate the findings. At a more strategic level, the validity of the design was tested in a collaborative way that the project management. The risk of success bias – e.g. going only to farmers achieving good results, or grantees making best use of funds - was candidly discussed: in selecting farmers, nurseries, grantees to visit the evaluation was constrained by logistical factors, but the project gave full access to its database and lists of candidates were freely exchanged and openly filtered, with final selection remaining with the lead evaluator. Further validity checks were conducted in partnership with the M&E team during the ongoing analysis phase, resulting in cooperation in selecting a few additional candidates to interview in the 2016 activity. The Project Director was asked to review earlier drafts of this report, helping to confirm that the data analysis was accurate and fair, even where critical; no requests were made for material changes of any sort. These steps provided a solid endorsement of the overall findings. Qualitative Interview Guide for Commercial-scale Farmers 1. Dimensions and trajectory: (refer to amended Comm’l Farmer Quantitative instrument). 2. Your History as a Cashew Farmer 2.1. When did you start farming cashew? Why did you choose cashew? 2.2. When you started, did you acquire an existing farm or plant it all yourself? 2.3. What other activities (crops, businesses, official duties) do you carry out? How much time do you dedicate to the cashew business? 2.4. When was the major expansion of your cashew farm (e.g. in early years; recent years)? 2.5. What are the major challenges you face as a cashew farmer? 3. Resources: 3.1. Do you have mechanization (tractor, truck, sprayers, irrigation pump…) for cashew? 3.2. Do you have other facilities such as a warehouse or seedling nursery for cashew? 3.3. If you don’t own these, are they available to you from someone else (borrow/rent)? 3.4. Would any of those resources help your farming (tractor, truck, warehouse, irrigation pump, …)? If yes: why have you not invested in that resource? 4. Expansion Plans 4.1. You are / are not currently planting new seedlings. Do you plan to expand your tree numbers in the next 2 years? Why and by how much? 4.2. Would you expand by (e.g.) 1000 trees? Would that make you money? What would you need to expand like that? 5. MozaCajú Support Provided: 5.1. Are you receiving the Materials, Services and Training you need? 5.2. Are you receiving any other Support you need? 5.3. For 5.1 – 5.3, for Yes or No: “Please give a specific example of a technique/technology you have received from MozaCajú that makes a difference for your farming”. Probe. 5.4. Are you registered on CommCare? If No: heard of it? If Yes: use it (SMS volumes to sell)? 5.5. What could be improved in the overall MozaCajú support you receive? 5.6. What are specific examples. How much would that help you specifically (e.g. by increasing volumes, lowering costs, reducing risk…)? 6. Nursery: 6.1. Do you have your own nursery? If Yes: who do you supply? Selling them? Price/volume? 6.2. If No: Do you buy seedlings? If yes: How much do you pay? Have you considered starting one? Would farmers buy from that nursery? Would it be viable, financially? 7. Attractiveness & Confidence: 7.1. Do you want your children or grand-children to farm cashew? Why? END Qualitative Interview Guide for Processors 1. What opportunities are there in the world market for growing Mozambique’s market share of processed cashew and obtaining higher prices? 2. What are the main constraintsto doing more business with the Mozambique cashew sector? 3. RCN Processing: what were your 206 processing volumes (RCN MT)? Please also provide the 2013 figure if possible. 4. Export sales: volume (boxes) and USD value of export kernel sold from 2016? Also for 2017 if available. 5. Factory Employment: for 2016: number of workers (average), and percentage that are women? Similar estimate for 2017 if possible. 6. Wages: Total factory wages cost: 2016 in Mzn or USD? Similar data for 2013 also if possible. 7. Capital equipment or factory upgrade: amount invested in USD, in 2016 for the cashew factories? Did these result in any increase or decrease in factory employment - if so, please indicate the numbers? 8. HACCP and any other certifications: what are your current status and plans? Do buyers expect certification? 9. Export Sales Facilitation: has MozaCajú introduced buyers to you; and has MozaCajú facilitated any actual sales? 10. Business outlook over the next 3-5 years: how do you see its cashew business volumes growing in Mozambique over that mid-range period, what obstacles or issues do you face and how will you tackle them? 11. Support to you: Are there any other ways that MozaCajú is helping you? Are there any support efforts that have not helped as expected? 12. Suggested Improvements: Can you suggest any areas for potential MozaCajú improvement during its third and final year of the project? MozaCajú Endline Evaluation Page 81 Processor Qualitative Findings – Data Capture Guide. Processor: Topic Actual Reasons Given Plans Capital Investment Decisions Made Jobs and Wages changes Upgrades in their Processes – food safety, traceability: - Motivation - Commitment Opportunities seen for their business . Obstacles/Concerns seen for their business Role of MozaCaju Specific Support By MozaCaju • Describe • Evaluate • Rank • Issues? Reasons for support taken/not taken Improvements suggested for future Project Outlook on the Sector in the near future. END MozaCajú Endline Evaluation Page 83 Qualitative Interview Guide for Global Cashew Buyers 1. What volumes of cashew have you bought from Mozambique overall. How much of it was in some way facilitated by MozaCajú (such as, via direct introductions and in-person support to identify and start transacting supply): (Containers) 2016 2017 2018 (Plan Total purchases made overall Purchases facilitated by MozaCajú 2. What opportunities, if any, are there for growing Mozambique’s global market share of processed cashew, and for obtaining higher prices? - e.g. what market niches are best suited to Mozambique’s sector? a) Short term b) Long term Please give a sentence or two to explain your outlook? 2. How important are Certifications (HACCP, ISO22000, EU) to your current procurement in Mozambique; and to those future opportunities you noted? 3. What are the main constraints to doing more business with the Mozambique sector today? 4. What specific MozaCajú services or support have you received from the project to develop the opportunities you foresee? (e.g. introductions to processors; facilitation of meetings and factory/farmer visits; market intelligence; risk reduction through information flows; etc... a) Has this assistance been of value to your organization? b) Would you have made this contribution from your own resources or was it a genuine added￾value from the project? 5. How would you characterize your interactions with the project, giving your reasons? (if you have different types of interaction, please rate each one) 6. Does your organization intend to engage more closely with the sector in Mozambique to address the opportunities or constraints you’ve identified? (e.g. by providing direct support for increased RCN production, or by supporting bi-product processing, etc…) 7. Considering the whole of the MozaCajú project, what if anything could Daria and the team have done to improve their support for the Buyers or the sector more generally? What could that achieve? END Yes / Limited / No MozaCajú Endline Evaluation Page 84 Appendix 4: Performance Indicators and Results The Targets and Results are set out below in a similar format to that used in the MTE. Section numbers are provided for references to the descriptive and explanatory text in the report. OUTCOME INDICATORS Result Indicator Target Result Reference Contribution to Feed the Future # of farmers who have applied improved new techniques and technologies as a result of USDA assistance 17,318 farmers 23,414 farmers Sec 4.2 # of farmers who have applied improved farm management practices 5,773 farmers 15,559 farmers Sec. 4.4 # of hectares under improved techniques or technologies as a result of USDA assistance 20,020 ha 84,719 ha1 Sec. 4.2 # of jobs attributed to USDA assistance 865 jobs 740 jobs directly attributable Sec. 7.4 Increased Agricultural Productivity % increase in cashew tree productivity 15% increase 53% increase2 Sec. 4.6.4 Value of farmer income from cashew sales (USD) USD 7,205,426 USD 16,690,000 Sec. 4.6.2 Total # of individuals benefiting directly from USDA-funded interventions 24,092 individuals 24,296 individuals Sec. 8.3 Total # of individuals benefiting indirectly from USDA-funded interventions 120,461 individuals 121,480 individuals Sec. 8.3 Increased Availability of Improved Inputs # of new trees planted (annual average per farmer) 15.6 new trees per year 59 new trees above baseline over two years, by 65% farmers = 33/year (ave.) Sec. 5.2 MozaCajú Endline Evaluation Page 85 OUTCOME INDICATORS Result Indicator Target Result Reference Improved Farm Management (Operational and Financial) # of farmers who have applied improved farm management practices 5,773 farmers 15,559 farmers Sec. 4.4 % increase (in USD) in farmer income as a result of cashew sales (average sales per producer for all cashew products) 25% increase 66% increase3 Sec. 4.2 & Sec. 4.6.4 Expanded Trade of Agricultural Products (Domestic, Regional and International) Value (in USD) of cashew sold into regional and international markets USD 12,700,000 USD 106,673,000 Sec. 8.2 Volume (MT) of processed cashew kernel sold into regional and international markets 1,942 MT increase 15,465 MT cumulative / 3,747 MT incremental to 2014 level Sec. 8.2 Increased Value Added to Post-Production Agricultural Products # of jobs attributed to USDA assistance 865 jobs 740 jobs directly attributable Sec. 7.4 Value of wages from jobs resulting from USDA assistance (USD) USD 1,301,300 USD 768,000 Sec. 7.4 Increased Adoption of Established Standards by Industry # of processors/factories passing food safety audits 4 processors/factories 4 processors/factories Sec. 6.1 Increased Use of Improved Post-Production Processing and Handling Practices # of processors/factories that have applied new technologies and/or management practices as a result of USDA assistance 6 processors/factories 6 processors/factories Sec. 6.1 Improved Marketing of Agricultural Products # of processors/factories pursuing exports under international certifications 5 processors/factories 3 processors/factories Sec. 8.2 Average % volume of traceable cashews 60% 60% Sec. 6.1 # of processors/factories participating in international marketing and branding 7 processors/factories 11 processors/factories Sec. 8.2 Increased Access to Market to Sell Agricultural Products % of introductions made between distinct retailers and processors/factories that resulted in a sale 50% 53% Sec. 8.2 1 area (in ha) under improved techniques calculated based on those farmers applying 3 or more improved agricultural practices. 2 refers to the difference in yield between the 2014 and 2016 endline harvests: 2014 baseline 3.2kg/tree, MTE finding (2015) 4.3kg/tree, Endline (2016) 4.9kg/tree. It should be noted that the yield figures fluctuate and are cyclical from one harvest to the next; yield also seems to be affected more by factors such as the availability of chemicals and climatic factors as much as to project-specific interventions around other improved agricultural practices. 3 % increase in income has been influenced primarily by changing price dynamics, with higher prices being offered for RCN in the 2015 & 2016 campaigns as compared to the 2013 and 2014 campaigns. MozaCajú Endline Evaluation Page 86 OUTPUT INDICATORS Activity Indicator Target Result Reference In-Kind Grants (Inputs) # of farmers benefitting from the procurement of seedlings/germilings 19,963 farmers (pending approval by USDA to reduce to 4,000 farmers) 5,566 farmers planted 542,807 seedlings Sec. 5.2 # of seedlings/germilings procured by farmers 399,253 seedlings/germilings 542,807 seedlings/germilings Sec. 5.2 Capacity Building: Agricultural Extension Agents/Services # of agricultural extension agents who have received training in improved cashew production techniques and technologies 145 extension agents 176 extension agents Sec. 4.1 # of agents who are certified in the use of improved cashew production techniques and technologies 116 extension agents 176 extension agents Sec. 4.1 Financial Services: Facilitate Agricultural Lending # of loans, grants, etc. disbursed to direct beneficiaries as a result of USDA assistance 6 loans/grants 0 loans/ 13 grants Sec. 7.1 Financial Services: Leverage Private and Public Investment Value of capital accessed by processors for expansion or improvement of operations by leveraging project matching grants USD 250,000 USD 390,839 Sec. 7.1 (table 14) # of analyses completed on the viability of expansion projects or other improvement investments 3 analyses 3 analyses Sec. 7.3 Number of financial investments made by processors for expansion or improvement of operations by leveraging the project matching grants 3 investments 5 investments Sec. 7.1 # of employees benefitting from capital accessed by processors 950 employees 2,117 employees Sec. 7.1 Financial Servces: Provide SME Finance # of loans and grants disbursed to processors/factories as a result of USDA assistance 5 loans/grants 0 loans/ 13 grants Sec. 7.1 Value of matching grants to processors USD 250,000 USD 70,766 invested; stimulated USD 390,839 matching funds Sec. 7.3 MozaCajú Endline Evaluation Page 87 OUTPUT INDICATORS Activity Indicator Target Result Reference Financial Services: Provide SME Finance to Inputs Value of subsidies provided to community nurseries (USD) USD 68,000 USD 72,000 Sec. 7.1 # of subsidies provided to agrodealers for the distribution of seedlings/germilings 85 subsidies 85 subsidies Sec. 5.2 # of micro-grants provided to communitry nurseries 85 micro-grants 85 micro-grants Sec. 5.2 # of micro-grants provided to tool workshops 5 micro-grants 0 micro-grants Sec. 7.1 Inputs: Develop Agrodealers and Other Input Suppliers # of agrodealers created and operational 90 agrodealers 309 agrodealers Sec. 5.1 # of nurseries created 85 nurseries 85 nurseries Sec. 5.2 Market Access: Facilitate Buyer-Seller Relationships # of processor-retailer introductions made 12 introductions 15 introductions Sec. 8.1 # of processor representatives benefitting from introductions to retailers 24 representatives n/a - not measurable Sec. 8.2 Technical Assistance: Certifications # of industry stakeholders trained in certifications 90 stakeholders 128 stakeholders Sec. 6.2 Technical Assistance: Traceability # of traceability audits conducted 5 audits 5 audits Sec. 6.1 Training: Demonstration Plots # of demonstration plots created 660 plots 1,168 plots Sec. 4.2 # of farmers benefitting from demonstration plots 23,091 farmers 23,556 farmers Sec. 4.2 Training: Improved Agricultural Techniques # of farmers who have received training in agricultural production techniques 23,091 farmers 23,556 farmers Sec. 4.2 Training: Improved Farm Management # of farmers who have received training on improved farm management practices (e.g. governance, administration, financial management) as result of USDA 23,091 farmers 17,502 farmers Sec. 4.2 Training: Post-Harvest Handling # of farmers who have received training in post-harvest handling 23,091 farmers 23,414 farmers Sec. 4.2 Training: Post-Harvest Processing # of processor representatives trained in post-harvest processing 50 representatives 36 representatives (ToT) Sec. 6.2 Training: Sanitary and Phytosanitary Standards # of processor representatives trained in sanitary and phytosanitary standards 46 representatives 75 representatives Sec. 6.2 MozaCajú Endline Evaluation Page 88 Appendix 6: List of Key Documents Reviewed PROJECT: MozaCajú Award Agreement and Revisions • USDA/FAS Grant Agreement FCC-656-2013-038-00 September 2013 • Revision request October 2014 TNS MozaCajú Monitoring & Evaluation Plan MozaCajú Project Midterm Evaluation: A report to TechnoServe evaluating the progress of MozaCajú, a USDA project; Award Number TNS FCC-656-2013-038-00, June 2016 MozaCajú Narrative Reports to USDA: • October-March 2014 • April-September 2014 • October-March 2015 • April-September 2015 • October-March 2016 • April-September 2016 • October-March 2017 • April-September 2017 MozaCajú Indicators reports (accompanying the above narrative reports) AKF Component Reports to TNS Quarterly since April 2014 to mid-2017 ‘A Qualitative Assessment of MozaCajú Progress’, 2015; MozaCajú ‘Competitiveness of the Mozambican Cashew Industry’, MozaCajú report, September 2015 (word document and powerpoint; related internal policy brief) MozaCajú implementation plan and revisions TechnoServe MozaCajú Sub-Grants Manual MozaCajú training materials – technical, commercial CommCare Evaluation Report Email correspondence between MozaCajú management and Global Buyers ‘Factories in the Fields’, Paul, Brad, PhD., TechnoServe Mozambique 2008. ‘Value on the Margins: Wealth and Poverty in the Cashew Commonwealth’, Paul, Brad, PhD., TechnoServe Mozambique 2009.