MOZACAJÚ PROJECT MID-TERM EVALUATION A report to TechnoServe evaluating the progress of MozaCajú, a USDA project. Award Number TNS FCC-656-2013-038-00 Project: MozaCajú is a three-year grant funded by the United States Department of Agriculture (USDA), being implemented jointly by TechnoServe (TNS) as the principal grantee, and the Aga Khan Foundation￾Mozambique (AKF) as a sub-grantee. The project supports small cashew producers, processors and international buyers by addressing issues relating to production, input supply, processing, and marketing. These activities, affecting over 23 thousand farming households, align with poverty reduction efforts by the Government of Mozambique. MozaCajú Mid-Term Evaluation Acknowledgements The project team and its evaluator acknowledge the generous and constant support of USDA in developing, implementing and overseeing the highly ambitious MozaCajú project to revitalize the leading crop export sector in Mozambique. The author of this report specifically thank USDA for its input and review of this evaluation. The author of this report thanks all the interviewees for their engagement and willingness to share their experience and ideas. Special appreciation goesto the TechnoServe and Aga Khan Foundation professional staff engaged jointly in MozaCajú for their detailed, frank and helpful insights, and to the administrative staff for their help with logistics. As a result, this evaluation was consistently conducted with an open and objective process. In particular the author acknowledges the extended assistance provide by MozaCajú M&E Assistant Clerico Constantino for his assistance in coordinating requested meetings, providing translation, facilitating communication with the independent quantitative analyst, and generally providing an excellent level of collaboration to gather, record and jointly analyze information. All opinions expressed and any omissions herein are solely those of the author. MozaCajú Mid-Term Evaluation Page 1 1 Executive Summary Mozambique has worked hard for the last 20 years to rebuild a cashew industry that was broadly successful until it was sharply disrupted after a period of intensive market liberalization in the 1990s. The cashew sector’s recovery is impaired by structural obstacles that mean public extension reaches few farmers, efforts to rejuvenate the tree stock are falling behind and most of the sector profits are claimed by processors and traders and with minimal returns accruing to farmers themselves. The farmer base is extensive – over 1 million households – and almost all are small-scale producers and poor; many are discouraged or, at best, passive gatherers, resulting in stubbornly low yields averaging just 3kg per tree nationally. The local processing sector, which constitutes a competitive advantage for Mozambique at a time when world markets increasingly seek traceable, single-origin crops with a smallholder ‘story’, is now expanding production capacity in response to rising global market demand. In turn, processors depend on ever-increasing raw cashew nut (RCN) production volumes from a smallholder base that is too unorganized to realize the returns that would motivate better cultivation. Multiple prior projects have tried to improve farmer productivity and make new technologies available; but these have been short-term, usually agronomy-focused, and have not addressed the capacity deficits and poor linkages that isolate the farmer. Even the profitability of traders and processors is fragile, due to inefficient operations, competition from larger-scale African and Asian sources, and slow adoption of productivity and quality-enhancing technologies. MozaCajú is an ambitious attempt to pull many of these threads together, based on previous history by TechnoServe (with processors) and the Aga Khan Foundation (with communities in rural areas) in an integrated program that combines efforts to broaden the provision of extension services with the development of improved commercial capabilities that enable farmers to secure better inputs and sell their outputs for higher prices. At the end of the second year of the project, MozaCajú is on track to successfully meet all targets in the agreement by the end of the project. This includes targets related to farmer income, processor volumes and certifications, and market linkages with new global buyers. In addition, the project has launched new, ambitious initiatives and is actively looking ahead to what will happen after project completion. It is working on a sustainable exit strategy for some project components, and laying the groundwork for future stakeholder collaboration on others. The networks of farmers, promoters, nurseries and demo plots are fully in place. The targeted 23,091 farmers are organized, attending trainings on good agricultural practices (GAPs), and are within reach of the network of 85 nurseries established by the project. Most farmers have chosen to be organized into groupsfor selling, enhancing their ability to negotiate. In the 2015-16 season, MozaCajú supported USD 5.7 million of farmer gross revenues1 . The CommCare information system announced 1,064 MT of RCN to buyers (14% of total sales volume) to facilitate sales linkages and is being trialed for seedling distribution tracking in collaboration with the National Cashew Institute (INCAJU). The nurseries and the 176 field-level volunteer promoters are actively engaged with farmers. The project has committed 1 We have study findings ranging from USD 5.7 million based on direct discussions by MozaCaju staff with 60% of the producers who commercialized their RCN in 2015-16, to USD 8.3 million based on survey responses from 961 beneficiary farmers representing the project population. Either research method involves potential biases; to be conservative, we report the lower figure and will improve data triangulation in the 2016-17 season. MozaCajú Mid-Term Evaluation Page 2 significant resources to upgrading processors to achieve international certifications, and to facilitating the introduction (orreintroduction – due to the sector’s historical failures in kernel delivery and quality) of new international buyers that resulted in incremental sales of 37 containers to date , while building credibility and improved linkages for small farmer supply. The project has in addition developed an exit strategy to improve the potential for technology transfer that reinforces the adoption of improved practices. The project addresses growing international interest in organic production, certification and export, through a sustained collaboration with potential investors relating to an idle factory in Pemba. This report evaluates whether these core activities are having the desired outcome effects of increasing agricultural productivity and thereby farmer income while simultaneously improving the capacity of other stakeholders to improve competitiveness. The main findings of the mid-term evaluation are as follow: 1. All the project’s targeted farmers (25% female) are receiving frequent trainings, in local language, in agronomy, farm management plans and, for smaller farmers, group organization. 2. Most of the project farmersshow a higher rate of adoption of GAPs compared to a control sample: for example, significantly2 better adoption rates (5-20+% above control group) for six discrete GAPs. One set of specific practices has a pronounced effect on per-tree yield: farmers that spray achieve 25% better yields overall; MozaCajú’s better project coordination and training enabled 71% of farmers to spray their trees in 2015, contrasting with just 52% for control farmers. Furthermore, twice as many beneficiary women farmers were able to spray their trees compared to their non-beneficiary female counterparts. However, chemical distribution by INCAJU is erratic and inadequate for farmer needs, but the private sector is ‘crowded out’ by these free inputs. 3. The project strives to achieve the above in collaboration with INCAJU, through district-level engagement that improves the distribution of INCAJU’s free chemicals and seedlings to farmers, while also partly filling the holes in public extension support. 4. Farmer nurseries supported by the project are cultivating and supplying improved seedlings, with the target of 400,000 seedlings looking achievable. One project outcome is an improved rate of planting: 60% of the project’s farmers planted an average 42 trees each in 2015, amounting to almost 8,000 Ha of new plantation by the 15,760 farmers supported for the 2015-16 season; by contrast, control group figures are 39% of all farmers planting just 33 trees each. 5. Processor factories have engaged with varying degrees of commitment in food safety and certification initiatives, with the two factories of one group ready to be certified for HACCP by the February 2016 date of completing this report and further certifications planned. 6. Aims of improving access to finance have been frustrated by high interest costs and a lack of willing financial institutions. Some farmers have financing needs but field staff lack the business skills to fully support them. Processors finance expansion through their own credit lines and retained earnings. The project has redirected its efforts to the grant-making with some success. 7. Sustained commercial linkages efforts are welcomed both by global buyers and their processor counterparties. The project closes important information and trust gaps on e.g. quality that had 2 Throughout this report, ‘significant’ is treated as a defined term, denoting a statistical significance of p<0.05. MozaCajú Mid-Term Evaluation Page 3 deterred many buyers from considering Mozambique; the project is educating processors on market demands for traceability and certification, but decision horizons remain short-term. A recurring concern is expressed by actors at all levels: will the integrated development momentum and improved visibility of actors continue when MozaCajú ends? This arises in several forms: ● Can the networks of organized farmers, nurseries and promoters be maintained? The needed support infrastructure is demanding and expensive, but INCAJU capacity is limited. ● Are the nascent ‘commercial’ services (nurseries, local promoters) able to generate a market, despite the lack of commercial planning and training provided by the project to date? ● Will the private sector, which is dependent on reliable and growing supply, pick up the reins? With limited exceptions, the buyers and processors do not see themselves taking a direct role. There is fragile progress in the sector to date. In recognition of this, many stakeholders, especially the private sector, appreciate that MozaCajú’s current project term is a valuable starting point for what must be a lengthy process to change the sector’s dynamics and thereby meet the ambitious national goals: “The aims of MozaCajú are good but not feasible in the time, for a crop like cashew; it requires a minimum of 7 to 8 years… [The sector] has a need for a serious and honest restructuring.” (CEO of a major Mozambican processor). This evaluation therefore makes two recommendations to achieve the highest return on investment for USDA: (i) The project, in its remaining year, should focus on two or three major achievements that may provoke sustained and impactful change. A summary of recommended actions, their rationale and the decisions made in response by MozaCajú management, is provided in section 5.2 below. In particular, based on what has been learnt to date, the highest priority recommendations are to: a. Agree the 2 to 3 primary measures of success for the end of project outcome; b. Focus more resources on the more capable nurseries and processors especially, by rationalizing the MozaCajú support package; and c. Strengthen those pivotal players so that they can continue to promote improvements post￾project. (ii) There is considerable justification for developing plans for a longer, more targeted follow-on project, in partnership with key actors, to complete the key initiatives started to date. Valuable networks of resources have been put in place and are operating effectively. The challenge is to avoid shuttering yet another extension services initiative, as with so many prior projects. But simply doing more of the same is not recommended either; the project learnings have pin-pointed some elements that are pivotal, and others that are less valuable. Instead, the evaluator would recommend consideration of developing targeted plans to: a. Enable existing stakeholders, especially INCAJU, to absorb and effectively manage the pivotal resources established by the project, including viable nurseries and promoters; MozaCajú Mid-Term Evaluation Page 4 b. Deepen the value-addition capabilities of genuinely committed processors through targeted technology upgrades matched by their contributions to support capable farmers; c. Create channels for improved communication between the farmers and INCAJU; d. Stimulate a competitive private inputs distribution sector to improve production volumes. Preparatory steps for the above can be completed in the final year of the current project; the requirements for achieving a transformation of the sector would of course exceed its timeframe. For example, small farmer advocacy support was not included in the original design of the project but its need has emerged as a significant learning during the project, in particular in relation to subsidized inputs. Subsidies distort the supply of inputs, deterring the private sector from providing farmers with alternatives. These issues can be only partially addressed within MozaCajú’s current scope, resources and project timeframe; opportunities for creating sustainable change by instigating and supporting policy change are complex and involve multiple stakeholders. Given the achievements and learnings to date from MozaCajú, as outlined in this report, there seems to be much to gain by finding a way to effectively mobilize the sector around such opportunities. MozaCajú Mid-Term Evaluation Page 5 2 The Needs, Opportunities and Challenges for Cashew Some 1.4 million smallholders, representing 40 percent of Mozambican farmers or approximately one million households, grow cashew3 , one of the few reliable cash crops that farmers have historically been able to grow in the country. These smallholder farmers are responsible for more than 95 percent of the country’s cashew production. The country has a long history in cashew production and export, resulting in a large production base of some 34 million trees, a number of processing plants for value￾addition, and a broad legacy knowledge base – “everyone knows cashew” is a common refrain – albeit produced at chronically low yields and with variable quality by the vast majority of farmers. The effects of history still weigh upon Mozambique’s capacity to compete in cashew. Market liberalization and the dismantling of processing facilities in the 1990’s, need only be acknowledged here, as these past policy effects – evaluated as ranging from “disappointing” to “disastrous” -- have been well documented by various commentators (e.g. FAO4 ; leading development economists5 ). The most direct effects are still apparent today; as FAO’s study of the sector from 2005-13 shows, a large proportion of the tree population is aged and poorly maintained. INCAJU is over half-way through a 10 year recovery program with an ambitious goal of doubling raw cashew nut (RCN) production to 181,000 metric tonnes (MT) by 2020, but can only support 4-4.5 million trees annually with chemicals for spraying, and with several hundred thousand seedlings per year, on their own figures6 ; they admit the tree population continues to deteriorate. A number of local processors failed as a result of this abrupt policy turn, but the sector did not completely exit Mozambique. With the opening of a further two factories by OLAM and a new factory being built by ETG, the sector has now a total cited capacity of 40,000 MT per annum of raw cashew nut (RCN). Standards of quality remain low, however, and it has taken well over a year to get two factories fully prepared to receive certification expected soon after the date of writing for the HACCP food safety standard. Other processors will follow more quickly once this occurs. While export kernel prices in 3 Source: INCAJU (Instituto de Fomento do Caju) presentation ‘Desempenho Do Sector- 2013’, October 2013. 4 FAO. 2014. ‘Analysis of price incentives and disincentives for cashew nuts in the Republic of Mozambique’ 2005-2013. Technical notes series, MAFAP, by Zavale, H., and Monroy, L.; Rome. 5 ‘When Economic Reform Goes Wrong: Cashews in Mozambique’, McMillan, M., Horn, K., Rodrik, D. 2002, Harvard WP. 6 Source: INCAJU presentation ‘Desempenho Do Sector- 2013’, October 2013. Figures for INCAJU seedling production vary widely. The ‘Cashew Competitiveness Report’ (2015) cited the planned 2015-16 figure of 3 million seedlings, but it is unclear that these volumes can be produced and distributed in the target provinces and survival rates are low. A Competitive Value Chain? A recent detailed study commissioned by MozaCajú (‘Competitiveness of the Mozambican Cashew Industry – September 2015’) identified that the country’s cashew kernel export industry remains competitive despite inefficiencies caused by poor infrastructure and high labor absenteeism, even at the recent high RCN costs (27Mzn/Kg presented a 5-7% processor pre-tax margin in that analysis). The actual 2015-16 RCN farmgate average price identified in our later survey came in at 33Mzn, compressing margins and causing concern among global buyers. The dilemma is that higher RCN prices are attracting farmers to ‘rediscover’ cashew and accelerate planting precisely at a time when the industry’s efficiency and quality needs to improve, for fear of losing hard￾won global buyer interest and international market competitiveness. Productivity of RCN supply, quality improvement along the supply chain, and policies that better support the development of effective markets for inputs and services, are all required; but entrenched skepticism and the resulting short-term decision horizons of most players hold back industry buy-in. MozaCajú Mid-Term Evaluation Page 6 the last two years have increased, RCN costs have also increased considerably and labor costs remain relatively high due in part to labor absenteeism rates of 30-40%. International buyers unanimously report that processors have shown little motivation to invest in anything but additional processing capacity until very recently, and even then in muted terms (see section 4.6 below). Most importantly, the sector continues to struggle with reliability of supply because farmers lack the incentives and the meansto increase volumes and improve quality. MozaCajú’s own 2015 study7 found that cashew was seldom a priority crop – the opportunity cost associated with investing in cashew wasn’t seen as worthwhile by many farmers if it meant taking resources away from investing in more lucrative cash crops such as sesame. For many farmers producing cashew for sale, the outcome of cultivation is hit and miss – a good harvest (such as in 2013) saw only modest returns to farmers as prices were very low (average 15 Meticais (Mzn) per kilogram of RCN); that harvest was followed by a strong season where both yields and prices increased (23 Mzn/Kg reported), followed by the latest season where prices again jumped, to an average of 33 Mzn/Kg; but many farmers in Figure 1: Mozambican RCN supply and Processor Buying Cabo Delgado suffered a lack of rain that slashed their yield per tree by 50%. The much higher RCN farmgate prices in 2015 were due to supply constraints in Asia, competition between expanding processors, and a sharp currency devaluation, events that can change rapidly. Historically, many of the smaller producers are farmers of cashew only in name, being effectively gatherers of dropped fruit with a resulting lack of adoption of cultivation practices.8 Those who are active farmers have few resources available for expanding production: mechanization and irrigation are almost completely absent (see section 4.2 below). Farmers are aware that spraying of fungicides and pesticides can boost yields, but they universally rely on an inconsistent and erratically-timed distribution of free chemicals from INCAJU’s stocks – with free input supply described by a leading farmer as breeding a “dangerous mentality”. As a result, the 3Kg/tree yields for most farmers are low in comparison to global smallholder cashew producers, with e.g. Vietnam’s reported smallholder yields that are approaching an average of 10Kg/tree. In response to these issues, various projects in the recent past have addressed parts of the challenges identified. With the exception of TechnoServe’s work with processors and the industry association, other projects have typically focused at the level of technical advice to farmers – training, agronomic support services. A Bill & Melinda Gates Foundation-funded program (implemented by Germany’s GIZ) to promote cashew tree planting ended in 2012, and the USAID-funded AgriFuturo project, providing technical assistance (through TechnoServe) to cashew processors ended in 2013. Value addition, such as roasting and traceability systems, was not developed beyond the conceptual stage. 7 ‘A Qualitative Assessment of MozaCajú Progress’, Constantino, C. 2015; MozaCajú 8 Aksoy, M. A. and F. Yagci. 2012. ‘Mozambique Cashew reforms revisited’. W.P. 5939. World Bank, Wash. DC MozaCajú Mid-Term Evaluation Page 7 Furthermore, the problem of assured supply identified in the Government’s national cashew strategy remains: “to increase cashew production (in quantity and quality) and domestic processing of raw cashew nuts and cashew derivatives, ensuring the generation of higher added value”9 . To date these linkages and interdependencies have not been systematically addressed by the sector. The MozaCajú project brings several of these threads together in an ambitious attempt to directly support 23,000 small cashew producers10 by linking a highly participative model of agricultural extension, nursery development and field school/demonstration plot activities with concurrent efforts to support the local processors to improve their competitiveness. In this way farmers may have more assured markets for RCN and the possibility of better prices as incentives for improving their practices and active management of crop quality. In return, processors can better assure their future supply by affording to buy at better prices from increasingly more capable farmers because of the returns realized by being able to invest in more value-addition capacity locally. The mutual ‘win’, on purely pragmatic grounds, may help dampen the volatility of supply historically seen in the sector, improving planning and operational utilization rates (see Figure 1). Perthe Terms of Reference (attached as Annex (i)), this Mid-term Evaluation is intended to: (i) describe the program’s strategy and key facets of implementation; (ii) set out progress made against program milestones and targets; (iii) summarize the key implementation challenges faced and their resolution and (iv) identify ‘best practices’ and key lessons learned in order to improve program performance and assist with any future program design. The remainder of this document is structured as follows: 3. MozaCajú Objectives and Strategy 4. MozaCajú Progress Evaluation • Design and Methodology • Component-level evaluation 5. Suggested Improvements Annexes i. Terms of Reference ii. Targets and Achievements (abbreviated) iii. Map of Program Area iv. Methodology and List of Interviewees v. List of key documents reviewed 3 MozaCajú Objectives and Strategy In September 2013, USDA’s Food for Progress program granted to TechnoServe 8540 MT of crude degummed soybean oil (CDSO) to monetize at an anticipated proceeds of $8,540,000 for use over a period of approximately three years ($2.85 million per year) through September 2016 to implement the Mozambique Cashew Project (‘MozaCajú’) in the three northern provinces of Cabo Delgado, Nampula and Zambezia. In a subsequent modification (July 2015), the commodity allocation was increased to 11,110 MT of CDSO in order to guarantee the project a needed 6-month extension to see out the 3rd harvestseason, and fund a number ofspecial initiatives including CommCare and additional 9 As stated by INCAJU’s 2011-2020 Master Plan. 10 All statements of targets are based on the revised Indicators submitted to USDA in February 2016. MozaCajú Mid-Term Evaluation Page 8 grants. However, a lower commodity sale price than expected on the final tranche has resulted in a final monetary budget of $9,275,651 over 3.5 years ($2.65 million per year) through March 2017, with a corresponding reduction in some activity budgets and targets. The project award includes a sub-grant made to the Aga Khan Foundation Mozambique (AKF) to perform the bulk of the field-level work in organizing and supporting the project’s producer base across the three provinces. AKF employs and manages MozaCajú field staff and field operations (procurement and distribution of inputs and extension tools, and collection of field-level data). Please see the latest project Performance Indicators in Annex (ii) and project map in Annex (iii)). MozaCajú is an innovative sector-wide project that seeks to improve the capabilities of key stakeholders, at scale, in ways that naturally align disparate interests and thereby enable their activities to integrate in value-adding ways. The project does not operate in a vacuum; it builds on but significantly deepens and extends some of the various prior initiatives in the sector by parties ranging from USAID, the Bill & Melinda Gates Foundation and the Government of Mozambique, implemented by a range of international NGOs including TechnoServe and GIZ, each of which has addressed discrete elements of the poorly-functioning value chain. MozaCajú was designed - and continues to evolve - to support the re-building of this export-oriented cashew kernel supply chain in Mozambique. TechnoServe had extended experience (2002-09) in working with cashew processing factories in Mozambique to upgrade capabilities, attract new investment for smaller, labor-intensive processing plants, and to help establish an industry association to assist with sector development11. MozaCajú builds on this expertise and introduces a new element in working to build the capabilities and resources of small farmers to engage in cashew production with greater productivity, increased commerciality, and with an improved level of planning based on confidence that an accessible market will reward enhanced quality and reliability of supply. In these ways, the project sets a foundation for supporting the broader development of a more capable and resilient rural economy. The project’s key objectives are stated in the award granted by USDA as: 1. To increase agricultural productivity in the cashew sector by providing training to improve agricultural practices and post-harvest handling, expanding access to inputs, establishing nurseries, and strengthening smallholder producers; and 2. To expand sector trade through training in phytosanitary and sanitary standards, certification, and traceability; facilitating market linkages; increasing access to finance and investment. MozaCajú’sstrategy isto leverage the existing expertise and resources of government and commercial actors while demonstrating more participative, commercially-oriented models for delivery of support. This is implemented through services that attempt to recognize the differing needs of farmers ranging from the vulnerable subsistence household to those seeking to become true farming businesses. This strategy is defined by five component-level project objectives: 1. Improved Production: improve the capabilities of farmers through agricultural, post-harvest and farmmanagementtraining,supported by a network of extension agents and demonstration plots. 2. Access to Inputs: improve the supply of seedlings, and related inputs and services, including through the direct establishment of community level nurseries. Initially under-estimated, the 11 ‘Factories in the Fields’, Paul, Brad, PhD., TechnoServe Mozambique 2008. Available at: http://www.technoserve.org/files/downloads/factoriesinthefield.pdf MozaCajú Mid-Term Evaluation Page 9 importance of chemicals is now highlighted in the work to negotiate chemical supply priorities with INCAJU. 3. Traceability and Innovation: address hazards and encourage investment in quality improvement practices by primary processing facilities, including through selective grant-making, to enhance competitiveness in export kernel markets. 4. Access to Finance: facilitate the development of lending and investment opportunities in the sector and encourage technology innovation through targeted grants. 5. Marketing and Market Linkages: support the restoration of Mozambique as a credible and dependable supplier of kernel through facilitation of sales and development of better linkages between farmers, processors, and global buyers. These components, and the integrated approach being used, all draw on years of TechnoServe experience in Mozambique and lessons it has learned in the cashew sector across Sub-Saharan Africa, where TechnoServe supports approximately 80% of the processors12. Production activities draw on Aga Khan Foundation’s long experience in Cabo Delgado specifically, extended for this project to Nampula and Zambezia provinces with new field teams. These experiences all share a focus on helping defined target groups achieve sustainable socio-economic benefits -- including improved farming practices, market access for smallholders, job creation in competitive local enterprises, financial services, improved public services infrastructure and the introduction of appropriate information and communication technologies. 4 MozaCajú Progress Evaluation 4.1 Design and Methodology The evaluation was designed to meet two major needs: (i) to capture representative data for extrapolation to the project population so that a wide range of output and outcome measurements could be established; and (ii) to perform a mid-course evaluation of whether the project is on track to have any material effects on the key stakeholders within the project scope. The initial baseline survey at project commencement was unsatisfactory to support the evaluation, as outlined in Annex (iv); we therefore designed this mid-term evaluation to include some remedial data gathering as part of an extensive survey of project beneficiaries and a smaller number of non-beneficiaries. Three complementary data collection approaches were used, augmented by project team interviews and extensive document review these provide triangulated inputs that constitute a validity check for the mid-term findings, recommendations on course correction and the establishment of ‘cohort’ datasets of beneficiaries and non-beneficiaries, combining recall (2013-14) and actuals (2015 and 2016 years), for comparative Final Evaluation analysis. 4.1.1 Quantitative survey The farmers included in the study were randomly selected from the project database through a sampling process that took care to ensure no district was over-sampled. This systematic approach 12 Source: Shakti Pal, TechnoServe Regional Cashew Specialist; corroborated indirectly in interviews with several global buyers. MozaCajú Mid-Term Evaluation Page 10 ensured that the surveyed farmers would be representative of the MozaCajú producer population overall. This was especially important as Monitoring and Evaluation (M&E) processes in the project to date have tracked essentially only demographic and output-based indicators – e.g. basic farmer profiles, trainings administered. Because of the timing of the project start, the first harvest (September-November 2014) occurred shortly after the project was fully staffed; because of this timing, management chose to treat this first harvest as non-attributable to the influence of MozaCajú (see timeline in Figure 2 below). Recording of outcomes such as farmer yields, revenues and costs could only be made from the second harvest (late 2015), and the Mid-Term Evaluation was therefore designed to span all of these aspects, starting with checking the extent to which M&E output data captured to date were accurate. For this evaluation, we used a mixed-method approach that included a survey involving 1,000 respondents from the project, allocated proportionately to the project population distribution; we included a further 200 ‘control group’ non-participant farmers, to provide this evaluation with a high degree of confidence in its representativeness (please see Annex (iv) for the methodology). The survey was conducted in the latter half of January 2016 using a Mozambican external survey company and its enumerator team for data collection and cleansing. We implemented a quality assurance process to validate that hardcopy results were transcribed correctly, and then made efforts to check the veracity of basic data with calls to randomly selected respondents, albeit most went unanswered. Figure 2: M&E Timeline of MozaCaju Project As noted in the fuller description of the design provided in Annex (iv) we recognize several limitations of the study design and implementation. The underlying data used for the selection of the sample is imperfect; the project team acknowledge logistical challenges in obtaining and verifying data from some areas due to communication issues and personnel turnover (three facilitators and one supervisor to date) within the sub-awardee’s field staff, across a large geography. The control group of 214 non￾beneficiary farmers was selected to be broadly similar and proximate to the project while avoiding spillover effects; once the data were collected however we found that many of these non-beneficiaries operate smaller farms than expected, with lower average numbers of productive trees and therefore potentially lower activity levels - factors borne in mind as far as practicable in the analysis to avoid introducing bias. 4.1.2 Qualitative study ofsmall/medium producers, nurseries and promoters The MozaCajú M&E team had conducted a study of grassroots actors in the project in September 2015 to gain a better understanding of some of the issues and successes noted by that date. This was an internally-conducted but critical study comprising a series of in-depth interactions with trained and active participants -- 7 promoter interviews, 4 nursery interviews and 7 hour-long producer focus groups with a diverse makeup, including a purposeful garnering of female points of view -- spanning 6 districts. Thisstudy was evaluated and found to have used sound qualitative research protocols:semi- MozaCajú Mid-Term Evaluation Page 11 structured question guides, local language dialogues, recording and transcription routines were followed, and an active seeking of negative cases, i.e. instances where the project was not meeting expectations. The data collection faced challenges: in particular, the timing of the focus groups coincided with a high activity period, reducing participation numbers to an average of 15 per group. The analysis findings have been documented in a separate publication13. 4.1.3 Qualitative interview study of larger producers, processors, buyers and staff Having retrospectively validated the small/medium producer study and the proposed quantitative study (see above) the replacement evaluator designed an interview and inspection data collection process that would complement the above with a small number of targeted in-depth interviews and visits to test the key operational and commercial assumptions of the project. A second objective was to collect the quantitative measurements that the project’s M&E process are unable to gather – such as processor volumes, investment and costs; and buyer perceptions of the market opportunity and intentions to procure. The evaluator was supported objectively throughout by the MozaCajú M&E Assistant. Most interviewees agreed to be recorded, extensive notes were taken in meetings, documents were collected and all of these artifacts were jointly analyzed in an iterative process. One overarching question of this study was: how can we better understand the relationship between the behavioral perspectives and aptitudes of farmers or nurseries, and the project’s approach to promoting the adoption of new practices and techniques? The project segments farmers pragmatically on a simple count of productive trees; but anecdotally, farmers within each segment can display very different behaviors and some are clearly outliers in terms of production and commercial outcomes. In￾depth interviewing allows for an exploration of decision processes and possible causal connections. For practical reasons a small number of in-depth interviews were planned, involving 6 large farmers and 6 nurseries across five districts of Cabo Delgado and Nampula, with respondents chosen by the evaluator and M&E team for the greatest feasible diversity of activities and resource conditions. A second key question was: what are the constraints, opportunities and motivations of downstream stakeholders (processors, buyers) and how well does the project align to create linkages for the farmers? Factory visits and interviews with factory management and then top management of each group were arranged. Senior representatives of four diverse international buying organizations were interviewed; calls with an investment group and a food safety expert provided further perspectives. Finally, we asked: are resources being used effectively, and what might need to change? We conducted an extensive interview program of project senior management, functional management, advisors and field staff, comprising 17 interviews in work locations, provided insights into the end-to-end activities of MozaCajú, its diverse and sometimes conflicting internal processes. We arranged an early validity check on the initial findings of this work via a two hour presentation and discussion with the Steering Committee of MozaCajú on 3 February; this resulted in endorsement of the findings and supported the development of a set of prioritized actions for the project in its remaining year of operation (section 5.2 below). 13 ‘A Qualitative Assessment of MozaCajú Progress’, Constantino, C. 2015; MozaCajú MozaCajú Mid-Term Evaluation Page 12 4.2 Component-level evaluation: Improved Production By the date of submission of the MozaCajú narrative report for April to September 2015, the project was able to report that the target now set for producer recruitment had been met, based on the known pipeline of additional farmers still to be registered. As of January 2016, the project has 23,635 registered farmers(101% of target), of whom 25.4% are female, which is 4.6% lower than planned due Table 1: Producer Key Statistics per Segment to attitudinal challenges in identifying and recruiting a sufficient number of female￾headed farms in a number of communities. The project segments farmers into four categories based solely upon their number of productive trees, with the aim of directing a differentiated set of services and channels to support them in both their production and commercialization activities, thereby making best use of project resources. ‘Small’ and ‘medium’ farmers (i.e. those with from near zero to 300 productive trees) are supported collectively, through farmer production groupings served through the deployment of 176 village level promoters allocated proportionately to the distribution of farmers. The promoters are managed and supported by a field staff of 16 facilitators and 4 full-time supervisors. Recruitment and retention issues have resulted in a slight under-representation by women in the promoter network (21%, compared to a target of 25%). The farmers classified as ‘large’ or‘commercial’ tend to receive more direct support by the facilitator staff, with the largest commercial farmers being directly supported by the supervisors themselves. In the vast majority of cases, regardless of designated segment, the training given or productive assets available to the farmer, we observed a very high reliance on free supply of chemicals14 and seedlings in the planning processes of the farmers, together with a low willingness to pay for essential related services such as spraying. INCAJU policies that emphasize the subsidization of these materials have crowded out any attempts by the private sector to offer better supply of inputsto Mozambican cashew farmers. The initial project proposal did not fully anticipate these issues relating to inputs and therefore did not include an advocacy component whereby the project would work to encourage INCAJU to phase out these subsidies, at least for seedlings. This was later recognized as a distinct need; it requires a long-term policy advocacy effort deemed not feasible within the timeframe.. The project’s total number of productive trees has increased significantly from the 2.9 million reported in late 2015, to the current total of 3.7 million, spanning 74,000 Ha. of land. While only 6,000 women participate in the project, matrilineal structures result in their average holdings being typically within 14 INCAJU provides the physical chemicals but not the spraying equipment, fuel or services that are needed for application. Similarly, large numbers of seedlings are distributed, but it appears mortality is high due to lack of support services. MozaCajú Mid-Term Evaluation Page 13 5% of their male counterparts, regardless of segment; for example, the average ‘large’ farmer has 437 productive trees, split between males (441) and females (415) – see Table 1. Given that one of the priorities of the project is to promote the planting of new trees, an additional resource constraint for the development of the whole sector is the availability of land for potential planting of cashew and associated rotational crops. Of the 17,000 Ha. recorded in the database as being available for new planting by the 23,634 project farmers, 60% of that available land is held by smallholders(74% male, 26% female) who typically lack the resources to prepare the land and cultivate seedlingsthrough a four year development cycle before the trees bear a harvest. At the other extreme, commercial farmers whose holdings range from 1,000 to upwards of 15,000 trees or more, and with presumably the greatest capacity for more systematic farming augmented by mechanization, report that they have little land available for expansion in the absence of financing. Only two of the six interviewed commercial farmers had sought (and secured) loans. For these reasons, despite an initial focus on larger farmers the project has given increasing attention to the medium-size farmers, those with 200-300 productive trees, and large farmers, with up to 1,000 trees. These medium and large farmers plant new trees more rapidly and they claim access to some 6,000 Ha. of available land between them (5,000 Ha. and 1,000 Ha. respectively); these are indicators of the potential for accelerating the development of the sector provided resources can be unlocked for smaller farmers. Farmers experience a strong cyclicality in cashew harvest yields overall, as suggested by Figure 1: one or more high seasons will be followed by low seasons, with other potential drivers including the timing and amount of pest and disease treatments and the availability of rainfall. Farm management planning and responsive extension can mitigate the effects of first two factors – for example, through intercropping planning and better coordination of chemical spraying. Mulching for seedlings and water harvesting for targeted irrigation can mitigate the reliance on rain-fed cultivation of cashew. Farm management planning is conducted in standardized manner in MozaCajú; mostly due to their lack of literacy and numeracy skills, only 25% of farmers surveyed reported adopting any element of the training, whereas interviews with larger farmers revealed that the tools are too basic for their needs; these larger farmers lack decision tools for prioritizing investment in resources, and they lack mitigation strategies for pest/disease, drought and fire problems. While improved coordination of supply of INCAJU chemical is widely acknowledged as a benefit of the project (see section 4.3 below), little in the way of water harvesting training or related demonstration facilities have been incorporated into the current farmer field school resources and demo sites in the project. MozaCajú technical assistance is perceived favorably by producers, particularly at the level of the small and medium farmers who report having historically received few, if any, visits by INCAJU. In contrast to the baseline study, the survey respondents, whether beneficiary or control group, reported almost universally receiving at least some training in GAPs from various sources, mostly non-government organizations (NGO); most also reported adoption of at least one such GAP. The survey attempted to gauge the intensity of the claimed adoption of GAPs. While it may be too early to meaningfully relate this finding to potential yield changes or other outcomes, our survey analysis indicated that a significant difference in adoption rates does exist for these key practices by beneficiary farmers relative to other farmers outside the project. Expressed as the percentage difference in adoption frequency, correct tree spacing was reported as 300% more frequently practiced by beneficiaries than non-participating farmers, while (e.g.) mulching and post-harvest processing were 120% and 65% more frequently practiced by beneficiaries, respectively (see Figure 3). MozaCajú Mid-Term Evaluation Page 14 Figure 3: Differential Adoption of GAPs by Beneficiary Farmers(% above Control) While overall producer response to MozaCajú is favorable, the farmers’ expressed continuing frustrations with the inadequate distribution of chemical from INCAJU; they sometimes (wrongly) identify the fact that it is unresolved as a shortcoming of the MozaCajú intervention itself; this viewpoint was especially pronounced with the larger farmers in the qualitative interviews. While many of the largest farmers have found their own way to work through the local INCAJU system to secure free chemical on a reasonably reliable basis, they were vocal in their dissatisfaction with what they regard as inconsistent, erratic and in their view typically insufficient supply of the chemical on a timely basis. Smaller farmers more clearly perceived that MozaCajú was having some effect in influencing INCAJU’s chemical supply to be more equitable and transparent, albeit the project can oversee only a fraction of that supply. Leading farmers recognize that this as a policy level issue and look specifically to MozaCajú to address it. Please see section 5.3 for a related recommendation. All processorsinterviewed emphasized the importance to them of an assured pipeline of RCN. Reasons given included their need to increase the utilization of their existing capacity, the value of diversifying RCN sources of supply, concerns to reduce the costs of purchasing RCN, and underpinning for their investment in factory capacity expansion or upgrades. Several interviewees stated they saw MozaCajú’s technical assistance to producers as effective and welcome; improving the supply of RCN is beyond their perceived corporate remit and none declared an interest in providing direct support to improve the production base. MozaCajú’s production support in Cabo Delgado, comprising 32% of the project farmer base, was singled out by three processers as notably effective – “amazing work!” (Condor Nuts Managing Partner), based on buyer reports and visits by management. The effects of the project’s activities in Nampula province were less obviousto the processorsso far, despite the majority of project activities being located there. The field promoters are deployed and active. The project’s 2015 qualitative study reported that promoters are broadly effective in communicating the training modules and reminders based on farmer field school interactions twice monthly, but that the workload and lack of a fixed remuneration plan were impediments to securing their continued support. Promoters encountered as part of this evaluation demonstrated a readiness to train and seemed visible to their farmers, but expressed disappointment at delays in the provision of promised incentive payments due to processissues within (300%) MozaCajú Mid-Term Evaluation Page 15 the sub-awardee. None saw their role as being sustainable post-project because of the workload involved and the need for ongoing training support. The design of the training packages for farmers is effective, according to reports by promoters and facilitators. Farmers acknowledged this as generally the case. The GAP training rates recorded through the survey seem to reflect that basic technical assistance is being delivered, and this is borne out by some marked differences in cited adoption rates of GAPs (Figure 2 above). The qualitative study of small and medium farmers provided some insight into how well these training practices met farmers’ needs. One strong finding: farmers reported the primary motivation to join producer groups organized by MozaCajú was the opportunity to learn improved practices. Some aspects of training delivery (for example, the frequency of farmer field school sessions and the time commitment involved) were problematic for some farmers, because of the distances involved to attend trainings. Overall, our study noted a good level of farmer satisfaction and evidence of better commercialization prices, in particular where smaller farmers had become more organized and market-aware (Table 2). Table 2: Income based on recall of sales volumes & prices 2013-15 The project anticipated that the absence of an ecosystem of services providers and motivated financial services institutions would pose challenges. In the evaluation we found that smaller farmers often could apply improved practices to only part of their orchard due to the physical difficulty of the work involved with simple tools, exacerbated by a lack of working capital to hire labor, the uncertainty of availability of spraying chemicals, and the lack of spray atomizers, pruning saws, watering cans and other equipment to perform that work productively. The generality of these findings was corroborated by interviews with promoters and facilitators. By contrast, the evaluation fieldwork indicated that the larger producers -- those classified as ‘large’ or ‘commercial’ with typically 1,500-3,000 or more trees at various stages of growth and productivity – are already largely self-sufficient in relation to basic agronomic knowledge and access to services. These larger farmers demonstrated a relatively high awareness and extended record of implementing practices for establishing plantations, securing and applying inputs, harvesting and commercialization, typically as independent actors. These trees are often intercropped with what are frequently viewed as more profitable crops, such as sesame, to provide more income and better resilience in poor seasons. Nevertheless, the poor ecosystem and fragmented land tenure typical in most areas meant that few of even the largest producers had productive assets – such as tractors, transportation or storage facilities – and most stated that the density of small land parcels and lack of land for potential expansion made those assets less valuable. These larger farmers identified MozaCajú’s main value for them as potentially being threefold: (i) improve the timing and certainty of the supply of chemical from INCAJU that they typically had individually negotiated; (ii) provide support for accessing credit that could be used to clear more land MozaCajú Mid-Term Evaluation Page 16 or to buy a tractor; and (iii) help them develop a commercial plan for improving the profitability of their farms. Intuitively these farmers recognized that purchasing chemicals would provide a good return on investment; but the unavailability of accessible agrodealers distributing chemicals prevents large farmers from considering this option. The broader issues with chemical inputs are discussed in section 4.3 below. Our survey indicates that the project supported the 15,760 farmers who had been trained by the 2015-16 harvest season to earn gross revenue of $5.7 million15, with their 3 year income at least doubling in nominal terms on average for small, medium and large farmers (and more so for females in these segments, albeit mostly from a much lower base); whereas commercial farmers saw only 50- 70% increases, in part presumably because many had already adopted more GAPs and had learned how to negotiate with traders (Table 2). The opportunity to provide better commercial support to all farmer segments was starkly illustrated by two interviewed large farmers who had acquired tractors by independently accessing credit schemes: one was compelled to use all of the proceeds of his cashew farming to meet repayment obligations. Very little revenue opportunity was available from potentially providing services to neighbors due to their small landholdings. The second larger farmer, by contrast, had combined capital asset purchases with a lucrative spraying business to generate strong profits. The project field staff have been able to support this category of farmers with both basic and specialized problem-solving on agronomic issues (for example, fire damage recovery through an acclaimed expert on staff), but only limited commercial training and templates for very basic record￾keeping. One of the most progressive and perhaps profitable of the large farmers interviewed, credited the project with cashew agronomy support and basic record-keeping training; he now keeps records and training notes assiduously, in neat notebooks. He had not been shown how to record items in ways that permit himto break out the costs for cashew versus his other four major crops and therefore gauge his profitability from cashew as a decision tool. Another farmer, plans to plant over 100,000 trees and has received expert agronomic support to establish his nursery and to respond to problems (fire, low soil fertility). But he also noted that he lacks business tools and timely support for his commercial issues with his 18,000 seedling nursery. The potential value of now adopting a more behaviorally-aware 15 Total revenue used in this report is based on the MozaCajú commercialization results, extrapolated to the 15,760 farmers supported in the 2015-16 season (see footnote 1). These farmers sold 7,800 MT in the season. MozaCajú Mid-Term Evaluation Page 17 targeting of support is evidenced here; by contrast, other large farmers simply wanted more free chemicals and help with accessing finance. Thus, while many of the largest farmers were being individually supported by facilitators or even supervisors, the field staff interviewed did not demonstrate the commercial business planning capabilitiesthat is required to address the issues and opportunitiesfacing these farmers - such as, how to optimize between cultivating and spraying productive but ageing trees versus investing in the planting of higher yielding varieties and complementary crops. These types of commercial demands equally impede the more commercially-minded farmers classified as ‘medium’ based on their number of productive trees, but whose planting activity levels, related behaviors and resource availability - in particular, land - make them prime candidates for expansion of their production and good targets for demonstrations of more productive farming and farm management techniques. 4.3 Component-level evaluation: Accessto Inputs Chemicals: Chemicals availability is a ubiquitous concern for the farmers. INCAJU’s funded supply of chemicals can at best support less than 15% of the total tree population in the country, with different policies for distribution (direct to farmer; direct to spraying service provider) in force depending upon the province. The interview feedback on the timing of distribution is frequently negative; most farmers say they cannot count on receiving any chemical with certainty, or sufficient quantities to apply three treatments as prescribed. Even the more influential larger farmers complain about perceived short delivery and erratic timing that may miss the crucial early flowering season. Several of the larger farmers acknowledged that MozaCajú’s efforts to prioritize the list of participating farmers, to direct resources to farmers that are adopting complementary practices, is having a positive effect, although the project does not have a process to capture the effects of these interventions in a timely way. The survey data reveal that 71% of all project farmers are able to spray at least part of their tree holdings, versus 52% in the case of non-beneficiary farmers. Where spraying does occur, the project is able to identify clear and significant yield benefits for farmers. For example, surveyed beneficiary small farmers (N=603) achieved a 25% yield improvement in 2015 over their peers who did not spray, with essentially identical results seen in each province. Large and Medium farmers showed at least equally significant differences through the application of spray, relative to peers who did not spray. Control group farmers who applied chemicals also had improved results overall; this suggests that focusing on improving the availability of chemical and related services, by coordinating INCAJU services to reach those farmers ready to spray, has a high payoff. This can likely be enhanced by increasing chemical availability by encouraging private sector sources of supply such as agro-dealer shops, improving the quantity and timeliness of treatment available to farmers. Policy discussion with INCAJU is needed to open up chemical availability; farmers look to MozaCajú to promote these opportunities in the final year of the project. Whereas average land holdings are effectively equal between the genders (section 4.1), access to other resources is not; this applies demonstrably to spraying. While one positive result for the project is that female beneficiaries are nearly twice as likely to spray their trees as non-beneficiaries (50% of project females vs. 27%), they remain significantly under-served relative to males (where the rates of usage are 78% versus 61% respectively). Respondents believe thisis due to a combination of the issues MozaCajú Mid-Term Evaluation Page 18 outlined above – the distortions in chemical distribution and the lower ability by female farmers to pay for spraying equipment or costly spraying services16; these are issues to be monitored in the final year of the project. Farmer complaints of perceived chemical short-delivery may also be symptomatic of training needs that are only being partly met in the project. In interviews with farmers at all levels, we observed some confusion amongst farmers, promoters and even the MozaCajú facilitators as to the appropriate dosages for scarce chemicals, as correct dosage varies with factorssuch asthe age and size of the tree. This indicates shortcomings in the planning materials used with the farmers; much of it is text-based, ignoring differences in literacy; and staff themselves need training to be more capable diagnosticians. Actions to address this need are summarized in section 5.2 below. Seedlings: INCAJU’s ten year development plan to 2020 aims to distribute seedling numbers equivalent to 10%-15% of the total Mozambican Cashew tree population every year; meaning that around 3.5 million trees were to be distributed in the 2015-16 season17. In addition, MozaCajú adds a further 150,000+ seedlings per year to distribute a total of around 400,000 new trees by the end of the project. These two, fully subsidized programs comprise the majority of all seedling distribution in the country. Despite these developments, the planned INCAJU rate of net replacement (after mortality) is failing to keep pace with the ageing of the tree stock. Furthermore, monitoring of whether INCAJU’s trees are actually distributed in a timely manner, whether they are actually planted and their survival rates remains a significant challenge. The planned CommCare ‘seedling tracker’ pilot using mobile technology may help with traceability of seedlings and their outcomes; at the time of writing, that initiative was still implementing and relies on collaborative operationalization with INCAJU. Figure 4: MozaCajú nurseries: from small group-owned (left & middle) to large, privately owned (right) MozaCajú’s farmer-owned nurseries network is deployed and actively producing, with the target level of 85 nurseries reached by the end of 2015, providing over 160,000 seedlings in 2015-16, with 240,000 planned for 2016-17 to achieve the target of 400,000 overall. The earlier qualitative study reported that promoter activities broadly supported these nurseries by sensitizing farmers to variety selection and seedling cultivation practices such as spacing and mulching for water conservation. Nursery operations range from the many tidy but rudimentary plots with 2-3000 seedlings but few 16 The reference price set by INCAJU to spraying services is 35Mzn/tree. Farmers are typically cash-poor during the season; they instead pay in-kind, at 2-3Kg of RCN/tree, equivalent to c. 100Mzn at recent farmgate prices. 17 INCAJU presentation ‘Desempenho Do Sector- 2013’, October 2013. MozaCajú Mid-Term Evaluation Page 19 tools, to a few with large shade houses and dedicated staff with advanced training who tend 10-20,000 seedlings (Figure 4 above). As seen with the field promoters, the administrative processes of the project have had mixed success in providing timely support to nurseries. Several of the nursery managers interviewed as part of the evaluation stated that the supply of materials has not been consistent, and in some cases supply was lagging far behind the requirements for production of seedlings. It is not unusual of course, in any project, for farmers and nursery operators to insist that they require ever more materials and tools, for free; most of these nurseries made similar requests. The more salient point is that the contracted support – whether incentives or materials – have not been consistently supplied, and this is especially problematic where activities are time-sensitive, such as in the preparation and grafting of seedlings. Nursery owners were initially viewed by the project as engaged in an inherently entrepreneurial activity that would naturally seek to become commercially sustainable; project personnel now believe some of the more capable operators can become viable over time; “it would be a great result if 1/3 of them survive post-project” (MozaCajú senior manager). At present, nursery operators express considerable skepticism about the possibility of sustainability without the project as the contracted buyer; “there is no culture of farmers paying for seedlings” (Nursery owner), with the free distribution of seedlings by INCAJU continuing to distort farmer expectations18. Table 3: Nursery Income Statement (illustrative) Interviewed nurseries stated that they had experienced limited success to date in addressing these challenges. Several had made a few sales, typically of around 50 seedlings per sale at 10 Mzn each, but were uncertain of near-term repeat buying; they believed that farmers would wait to see whether the grafted varieties turned out to be higher yielding and therefore worth the investment before committing more funds: “the farmers don’t like the grafted seedlings, they don’t know the characteristics of the tree they will get” (Nursery owner). The owners have had little or no commercial training in promoting quicker behavioral change. They saw the farmers’ wait-and-see attitude as curtailing near-term demand, and have not engaged in real promotion; none of them saw their operation being sustainable if MozaCajú were to stop buying. Our preliminary modelling indicates a significant shift in farmer willingness to pay would be needed in order to make a nursery viable (see Table 3). Within the project timeframe, selected nurseries will be supported with business planning training (see section 5.2) to begin to address this need. A further issue for potential sustainability is the nursery owner’s lack of business skills and market awareness. As with the farmers interviewed, none of the nursey principals brought records or 18 Source: ‘Mozambique Cashew Competitiveness Study. Note that MozaCajú’s seedlings are also distributed free to farmers. Assumes 5000 seedlings sold/year. Year 1 set-up losses are funded by grant/loan, with modest year-to-year improvements in productivity and therefore costs. Key assumption: farmers will buy at greater than break-even price of 22 Mzn/seedling. Today very few farmers buy any seedlings, even at 10Mzn/unit. Nursery Income Statement (Mzn ‘000) Item Year 1 Year 2… Revenue 120.0 135.0 less Seed, inputs 30.3 30.3 Tools, equip’t. 26.7 7.5 Labor 72.0 65.1 Other costs 15.2 13.3 Operating Profit -24.2 18.8 MozaCajú Mid-Term Evaluation Page 20 planning tools to the interviews; none had a sales plan, and most of them could not begin to state the costs or likely sources for obtaining materials commercially, in the event that MozaCajú stopped providing the materials to them for free. In the final 12 months of the project, business planning and related upskilling are being developed with a handful of the most viable nurseries to support their efforts to become sustainable (section 5.2). A sophisticated farmer/nursery operator stated that his large nursery was clearly losing money at 10- 15 Mzn/seedling; for him, a true costing would mean a selling price of 60 Mzn, which he saw as inconceivable for purchasers currently. That farmer said he would nevertheless continue to produce seedlings because he valuesthe continuing accessto bettertechnology through MozaCajú for hisfarm. 4.4 Component-level evaluation: Traceability & Innovation Sub-Saharan Africa (SSA) is credited by international buyers with a competitive advantage of relative traceability to the country level, differentiating SSA nuts from Indian and Vietnamese purchases of blended RCN from multiple sources. East African sources, and in particular Mozambique, enjoy a further advantage of relatively larger local processing capabilities than West Africa, appealing to those buyers who seek to purposively buy single-origin with local value-addition, whether to fulfill a social mission or to sell a differentiated narrative to end markets. Supply from Asian sources is typically mixed with kernel from multiple sources at various stages of distribution by the trading companies, reducing the nut to a simple commodity. These factors are very promising for re-establishment of Mozambique as a leading source for the global markets. The project has stated an aspiration, in response to growing market demand and due to issues around ensuring good food safety management, of developing traceability capabilities down to the individual farmer level. Currently, the major processing groups all claim a strong degree of traceability, to the district level, on their own accounting, of around 40% of their volumes. This is seen positively by buyers. For general cashew “traceability to the individual is impossible” according to one local processor CEO. If organic production is embraced, however, then the certification and traceability will need to extend to the individual farm. No processor has a current plan for organic, although the business case for re-starting Cabo Cajú in Pemba is premised on organic (see next section), and OLAM has expressed an interest that includes a test farm of 300Ha currently in operation and a forward plan of potentially dedicating one of their four factories solely to traceable production. MozaCajú is providing traceability support in three ways: (i) the promotion and enabling of HACCP certification; (ii) use of the CommCare information service to support commercialization of farmer production; and (iii) extending CommCare’s tracking capabilities to seedling distribution. In brief: (i) HACCP: initial training efforts in 2014 were organized using expert resources from Merieux Nutriscience. Events were however under-subscribed by two target processing groups, in that management failed to show up and instead sent lower-level people; a third group, Condor Nuts, engaged more productively at the CEO level, and has proceeded to hire Quality Control expertise and to actively participate in lengthy training and upgrading consultancy offered by MozaCajú staff. This has been successful to the point where HACCP certification is now imminent. The three other processors have excused their lack of initial buy-in as being due to resource and priority issues; OLAM and ETG now plan HACCP certification using their own resources, whereas Cajú Ilha/GANI group show little commitment despite frequent visits by MozaCajú staff. In response, the project has begun a quiet MozaCajú Mid-Term Evaluation Page 21 campaign of discussions with international buyers to mobilize them to send a clearer signal of market requirementsto all processors relating to global market trends in favor of traceability and certification. Our current evaluation is that overall, Condor Nuts have bought into the development path around traceability as something that will lead to a fundamental improvement of their operations rather than a marketing ploy, and this is recognized by several of the buyers interviewed as being attributable to MozaCajú. By contrast, the project’sinfluence on the other three major processing groups is less direct and in the case of at least one group, it is seen as peripheral. (ii) CommCare: the mobile-based information platform has now operated for one full season, with the MozaCajú commercialization team reporting in December 2015 that 1,064MT of RCN had been announced to potential buyers through the service. This is not a trading platform, and MozaCajú is unable to directly track the source of sales and whether sales were facilitated directly by CommCare. Anecdotally, in two weeks of fieldwork we saw farmers struggle with the tool when asked to demonstrate, probably because it is only used at selling time. Farmer group SMS’s are nevertheless successfully being received, aggregated and passed on to the RCN buyer list; interviews with buyer management indicated they see both benefits and costs in this service. Two processor managers criticized receiving what they saw as false signals from CommCare; advertisements of RCN availability led to buying trips where agents discovered the harvest had been sold to traders, immediately (they believed) after sending the CommCare SMS. Conversely, the general manager of another processor saw a valuable market intelligence benefit in CommCare, with volume announcements providing a useful check on the processor’s own internal buying network. Figure 5: Group Leader demoing CommCare (iii) Seedling Tracker: a technical evaluation of CommCare’s ability to better link farmers and buyers ran into the obstacle that factories were not willing to pay to have bags tagged for traceability. Instead, an alternative proposal isfor CommCare tracking capability to be used to trace the distribution and survival rates of seedlings. This could help alleviate a major resource loss: up to 50% of INCAJU’s current seedling distribution failing to survive, with little tracking and understanding why this occurs. The technical feasibility of CommCare application to tracking seedling distribution has been evaluated and is very high, with only minor development required. A pilot is currently being prepared in collaboration with INCAJU; at this time, there is no way to evaluate its feasibility or value. 4.5 Component-level evaluation: Accessto Finance The original project design included MozaCajú working to enable better accessto financing for farmers and processors. High interest cost, lack of farmer collateral in most cases, and the unwillingness of many commercial banks to engage resulted in these targets being eliminated. The project emphasis in the area of financing support has instead turned to: (i) collaboration to attract new investment, via efforts to restart Cabo Cajú plant with the collaboration of the Clinton Foundation; and (ii) grant￾making to encourage investment in new technologies by farmers and the processors. (i) Cabo Cajú / Organics: MozaCajú management recognized the potential opportunity for organic cashew processing at an early stage in the project and a candidate site was identified in the form of the disused Cabo Cajú factory owned by a Pemba based international businessman. As confirmed by the Clinton Giustra Enterprise Partnership (CGEP) investor analyst involved, MozaCajú commissioned MozaCajú Mid-Term Evaluation Page 22 CGEP to conduct due diligence and rewrite a business plan and enhance the business case for investment in a rehabilitated facility geared to organics. The investment proposal is currently on hold, however, due to CGEP funding constraints; the owner of the facility and the former CGEP analyst both acknowledged the quality of the communication maintained by MozaCajú between the parties. At the date of writing, a new attempt is being launched to bring the investment proposal forward for full approval by CGEP, a process the former analyst described as “100% reliant on MozaCajú” to support the 4-5 years efforts in farmer linkages and training needed to achieve organic certification and build sales volume. This poses an unresolved challenge of mismatched project tenors. (ii) Grant-making: one target of the project is to make available grants totaling up to $450,000 on a matched basis in order to support the introduction of production or processing equipment that is either new technology to Mozambique and/or that is otherwise unavailable to the grantee. The project grant program has evolved rapidly based on the learnings from three rounds conducted to date, overseen by a selection committee; the evaluation documentation review indicated that a high level of compliance and due diligence in that selection and award process is being exercised. The initial two rounds attempted to reach out to farmers to generate their adoption of technologies and mechanization based on a business plan for each farm. While potential interest initially looked strong, the project discovered that farmers lacked the business planning skills and the financial resources to achieve the promised match of funds, along with an underlying mentality that NGO funds are effectively ‘free’. The evaluator saw evidence of downstream private investment decisions being prompted or supported by the grants – e.g. the construction of a privately￾owned green-field sorting facility by the trading company Messalo in Mueda, for nuts that can be peeled hygienically at home by women who are supplied with an innovative, grant￾sponsored processing box. Staff resource and travel budget constraints meant that not all farmers were canvassed in these earlier rounds however: in the evaluation fieldwork, one of the largest farmers in the project was unaware grants were potentially available for his capital investment plans. Figure 6: Private investment spurred by complementary grants: sorting facility The third round of grants directed attention instead to injecting new technologies into companies, specifically the processors. Various proposals were received, and widely differing business case competencies were revealed in the multiple iterations of discussion of each proposal. By the end of this round, seven candidates had been approved in principle, ranging from individual farmers to cooperatives and also processors, as indicated in the most recent semi-annual narrative report. The outcome to date is that 5 grants have been disbursed. The two grantsto processors provide assistance to procure new technologies; one is for equipment to extract cashew nut shell oil byproduct at Condor Nuts, reducing fuel costs and environmental footprint; while the second is for the implementation of a new pest control technology that is operationally and environmentally beneficial at one of the OLAM facilities. There is a cost in terms of extra relationship management involved in grant-making, however; a third processor submitted an inadequate proposal in several iterations that did not address core issues, and that proposal was finally rejected; this has caused a slight sense of (misplaced) grievance, an issue being managed but requiring significant management attention. MozaCajú Mid-Term Evaluation Page 23 The project team plans to focus the grant-making activity in 2016 on spraying equipment, addressing the single most critical production constraint on yields, and thus improving potential income for farmers. The due diligence and administrative burden of grant-making leads the project to seek to negotiate just a single grant with an equipment provider or service provider, with a plan to be developed for distributing those resources to nurseries. As noted in section 5.1 below, the evaluator has discussed extending those resources to selected promoters in order to enhance the sustainability of their activities beyond the life of this project. All four major processor groups reported making capital investments in 2014 and 2015 in capacity expansion and equipment upgrades. Some of these, such as the investment in deshelling equipment made by ETG, were reported as being influenced by MozaCajú. These investments ranged from $0.5- $2 million per year, per processor. While some credit facilities were drawn on to finance these investments, all four processors had secured any loans involved without requiring facilitation support from MozaCajú. All of the major processors welcomed the introduction of new buyers and other support to facilitate trial procurement from Mozambique; and all they welcome technical interactions with MozaCajú, in particular the technology scanning insights provided by regional specialist Shakti Pal. A concern expressed by some projectstaff and by two international buyers is that the project may not sufficiently demand that these processors match the contribution of MozaCajú effort with equal commitment of resources. For example, while most processors sent the appropriate executives and managers to participate in expert certification training provided by the project, others sent only low-level staff. Similarly, the project has seen varying commitments by processors in hiring quality management staff to leverage the training and consultancy advice given; for example it is still unclear how one grant recipient, OLAM, who alone will process 15,000MT of RCN in 2016, will now move on to meaningfully accelerate their development of food safety certification and trialing of organics. This overall issue of matching commitment is being addressed in the current year (see Section 5.2 below). 4.6 Component-level evaluation: Marketing & Market Linkages In flowing from tree to trade, the Mozambique value chain operates at different levels of actors with different interests and practices. At what we might call the ‘micro’ level, the farmer segments in the project have been commercially trained and most of them sell collectively, seeing an average 1.5Mzn/Kg (or +5%) RCN price advantage from participation in the project, relative to the surveyed control group. This suggests that after even only a Table 4: Average RCN Selling Prices - Beneficiaries. single season of project participation for most farmers, the commercialization support is helping a significant number of farmers to achieve a better price for their product (see Table 4). Organized collective selling and the related training that sensitizes farmers to seek better prices, appear to have improved their negotiating position overall. This seems to be supported by the observation that average prices achieved acrossthe four segments of farmers were very similar within the project (average varied only by 1 Mzn – Table 4), whereas non-beneficiary farmers see a Segments: Ave. Mzn/Kg 2015 Ave. Mzn/Kg 2014 Ave. Mzn/Kg 2013 Commercial 33.18 23.39 16.86 Large 33.42 22.39 15.94 Medium 33.92 23.07 14.91 Small 32.77 23.16 14.71 Weighted Ave. 33.13 23.05 14.94 (@ 2013 e/rate) 22.45 21.62 14.94 MozaCajú Mid-Term Evaluation Page 24 strong skew of prices to the larger farmers (the average varied by over 10 Mzn in the survey). In USD exchange rate-adjusted terms, RCN prices have for the last two years been relatively stable at 50% higher than 2013. Around 20% of last season’s MozaCajú farmer production was flagged for sale at local level via CommCare SMS messages that are sent to a central MozaCajú team for electronic distribution; this advertising of volume and location is only indirectly associated with facilitating actual sales, but the process is seen as helping the farmers to negotiate with more confidence. The typical dynamic was recounted by a local group leader, during the study: “I was away from the village, and my neighbors were approached by a trader offering 35 Mzn per kilo. Even with a group total of 115MT volume for sale, he wouldn’t improve the price. Before they sold, I came back and called them together, I made some calls on my cell phone to the buyer’s HQ and confirmed that the market price was higher. We sold at 39.” At the ‘meso’ level, the major actor for non-export RCN buying is the processor. Mozambique’s production is largely concentrated in the north with 4 industrial processors now operating 8 sites that are likely to account for around 40,000 tons of processed RCN in 2016 within the project geography. In the fieldwork interviews, all four processor groups reported that their procurement books from 2015-16 were effectively full. The four locally-based processors reported that the project has brought them active introductions to buyers and each attributes some resulting sales to the project, estimated as totaling 37 containers of incremental sales over the period 2014-2015. Meanwhile, farmgate RCN prices (in Mzn) during the 2015-16 commercialization season were significantly higher than recent years. While this seems to represent a ‘win-win’, domestic RCN prices were seen as unsustainably high by top management of two of the processor groups, a sentiment echoed separately by two major international buyers. The chairman of the Mozambican cashew industry group AICAJU noted in an interview that RCN costs delivered to the factory door were now around 48 Mzn or USD 1.00, a price-point identified by global buyers as deterring orders. This poses a dilemma for the sector: high RCN prices encourage farmer investment in cashew, potentially boosting production supply in the near to medium term through new planting and accelerated adoption of practices; however, the sector has only fragile profitability due to relatively inefficient operating costs at the factory and high upgrade financing costs, making sustained high farmgate RCN prices a potentially major concern19. MozaCajú’s activities across the whole chain are seen by processors and buyers as demonstrating useful approaches to improve the sustainability of these actors. These outcomes roll up to begin to address the ‘macro’ level issues that are crucial for re-establishing Mozambique globally as a supplier of traceable and branded nut. Buyers’ interests range from the transactional to developmental; whereas a buyer such as Red River Foods focuses very closely on the cost differential between kernel of specific grades from Mozambique versus West Africa or Vietnam, other buyers such as Nutrade and Caro Nut actively engage in issues such as how to overcome quality unreliability and encourage processors to develop an active pathway to certification to satisfy US and European markets. These latter buyers say they seek a closer collaboration with the project, as a “partner” in the development of supply rather than simply a counterparty. Just what this partnering amounts to varies, from an indefinite request to be “part of the dialogue” (Caro Nut), to suggestions that are more concrete. Nutrade expresses an interest in developing an overall Mozambique nut 19 The Cashew Competitiveness Study (2015) dissected the costs structures and concluded that for all players, “The equilibrium between profitability and loss is still a thin one.” MozaCajú Mid-Term Evaluation Page 25 strategy that utilizes Nutrade’s new 3,000MT groundnut processing and roasting facilities in Mozambique, to also develop differentiated, consumer cashew product for the South African market off the same production line; for Nutrade’s CEO “MozaCajú’s registered farmer network is golden”, a route to targeted interaction with motivated, trained farmers. In addition to the incremental volume achieved for the processors, the project’s efforts to make introductions, solve problems and improve communication – all needed to successfully complete an initial small order with a cautious new buyer – are seen as particularly valuable for overcoming international buyer skepticism of the quality and reliability of supply from Mozambique. One major US-based buyer put it this way: “We placed a first order with [processor] in 2013, just two containers to start. The product quality was bad; it was moldy on arrival. But [processor] didn’t step up (to settle the claim), they showed little interest; so as a result in 2014 we placed no orders in Mozambique at all. Because of MozaCajú’s encouragement and support, we took the plunge in 2015 and placed small orders again; this time we are happy.” The work led by the project director of MozaCajú to support these disparate interests was strongly acknowledged by all the contacted parties, especially for transaction support. A broad wish was expressed by two buyers that they “seek to do more” than just transactions, but apart from Nutrade (as noted above), buyers struggled to articulate just what they are willing to engage in, materially. While the processors, with the exception of Condor Nuts, tended to see the project in mostly instrumental terms (i.e. as a useful but relatively short-lived intervention to boost production volumes and improve communication with the field), the international buyers placed a high value on the improved visibility gained into these sources of supply and their trajectories. Reciprocally, they appreciate the way the project communicates end-market developments and consumer preferences to stakeholders in the Mozambique supply chain, starting to align them around common issues and interests. Valued buying relationship are being forged; each buyer nominated particular processors with whom they have now developed closer relationships. MozaCajú will continue to strengthen those linkages, improving post-project sustainability. As noted, several buyers identified the opportunity for the project, as an independent convener of various interest groups, including policymakers, around key themes to address broader sector challenges and opportunities that the buyers themselves can only indirectly influence. Examples of such themes include the development of organic supply and collaboration to launch specialized products; elucidating these opportunities can build the basisfor ‘shared value’ commitmentsto a future project leveraging the gainsfrom MozaCajú. The MozaCajú competitiveness study is one expression of that ability to influence the policy dialogue, with the development of organic production proposals as another. 5 Overall Evaluation and Suggested Improvement Areas The projectseeksto demonstrate that effective production support can be delivered to a large number of distributed, resource-poor farmers by creating pivotal commercial and technical nodes of capability in the field, linked to public resources and specialist effortsto upgrade local commercial value addition. This is reflected in the complex organization and operation of the project. TechnoServe is responsible for overall technical strategy and implementation and directly provides the technical staff and back office support for the work with processors, buyers and other institutional stakeholders. AKF Mozambique provides the frontline staff and back office support for all the geographically far-flung MozaCajú Mid-Term Evaluation Page 26 production component activities including producers and nurseries, funded through a sub-grant. The initial year of activity revealed a number of challenges in these arrangements, leading to a top-level realignment effort in early 2015. During the fieldwork, we saw good evidence of the goodwill and overall strategic alignment of the project team; recent management processes such as a weekly alignment meeting have been helpful. As subsequently discussed with the Steering Committee, a number of learnings have been identified, and many challenges remain; it seems imperative to prioritize these learnings in order to determine what MozaCajú should try to achieve in its final year. Suggested improvement areas are outlined below by principal actor or process, followed by a table of consolidated recommendations and planned management actions. 5.1 Learnings and Suggested Improvement Areas Project Management: the MozaCajú senior management and functional teams seem clear on the overall objectives of the project but they sometimes express varying interpretations of what the priorities amount to in execution. In particular, the desired ‘end state’ of the project seems to require a unified management decision regarding what to emphasize in the final year of the project, based on learnings to date. There seem to be two possible pathways for exit from each project activity: by (i) generating commercial dynamics whereby key actors change their behaviors and form viable linkages; or (ii) by transferring what works best to INCAJU. The former alternativessuggeststhe projectshould aim to leave, asfar as possible, a set of commercial linkages and capabilities in place, to bypass the inadequate and inconsistent support to farmers otherwise available. The project has experimented with this possibility of commercial viability in critical production-level activities, encouraging e.g. the nurseries to see whether they could find buyers amongst the local farmers. Processors have been encouraged to take a longer view towards higher volumes of certified product, including organic, sourced from the small farmer base. But not surprisingly, the project is finding that willingness to pay is low in the current environment, making such viability at the farmer/nursery level highly unlikely in the short to medium term. Processors have begun to invest gradually in value-adding technologies and buyers are seeing improved quality of supply as a result, but none are currently willing to invest directly in improving the productivity of smallholders. Alternatively, the focus of the final year could instead be on delivering sound extension processes and technologies that could readily be absorbed as a valued legacy by INCAJU itself, effectively amounting to a technology transfer handover of the established project network This is an approach that the project team has already developed specifically with regard to CommCare; management might also seek to engage INCAJU in a wider-ranging transition proposal and negotiation that includes some or all of the nurseries, promoters and perhaps even project staff transitioning. Either strategy is potentially feasible, over the medium term. The project combines the commercial capabilities required by the former proposed approach, with its requirements for a market-oriented, demand-side focus, with the supply side expertise to improve small farmer production and the diffusion of extension services and basic technologies. The author suggests that any decision to pursue a wider transition plan with INCAJU would benefit greatly from further support by USDA in lending its authority through high-level Ministry discussions for what would amount to a significant, multi-year change management process for INCAJU. MozaCajú Mid-Term Evaluation Page 27 Field level operations: there is good evidence that the project is effectively delivering technical assistance to the over 23,000 farmerstargeted by the project through the innovative delivery networks of community level promoters and nurseries. This innovation enables service delivery by practicing farmers in local language, supported by an active but dispersed team of facilitators. This leveraged model allows one facilitator, equipped with a motorcycle and CommCare, to recruit, train and support 10 to 15 promoters, each equipped with bicycle, training materials and CommCare. Each promoter in turn supports around seven groups of 15 to 25 farmers each. Beyond the usual logistical and communication challenges of field networks, the project is characterized by several key issues and opportunities, many already noted by facilitators, their supervisors and farmers themselves: 1. Working relationship with INCAJU: the initiative taken by the project to promote better coordination and transparency with INCAJU in the distribution of chemical is widely acknowledged as having a positive initial effect. Many farmers are looking for greater consistency as well as commercial alternatives for purchasing chemical, as well as access to resources to perform their own spraying. Farmers clearly want choice of input supply, including some that expressed an interest in buying from competitively-priced private suppliers; however, INCAJU’s current policies crowd out any meaningful private sector entrance. These farmers look to MozaCajú to resolve these issues with INCAJU; they believe that MozaCajú has an independent convening power that could be used to begin this process. 2. Nursery viability: the nurseries will meet the overall project plan of producing 400,000 new seedlings and the adoption of improved techniques was clearly evident from the visits. Field staff, supervisors and most nursery operators interviewed cannot see a path to viability for the nurseries that extends beyond the term of the project. One nursery operator described the uncertainty about MozaCajú as making them “feel helpless”. Some logistical processes of the sub￾awardee AKF had created unexplained delays for receipt by nurseries of receiving materials and of payment for seedlings delivered to MozaCajú; one large nursery operator had received only 20% of the current season materials promised by the late January interview date, leaving just a few weeksto plant before the end of the rainy season. These process issues need to be addressed. Several nursery operators expressed an interest in developing a commercial basisfor their nursery and recognized that they required commercial training; this has not been made available by the field team to date. The project field staff should be equipped with the skills and be made accountable to help nursery operators develop the promotional and business management skills required to stand a chance of success. This is challenging in the time remaining. 3. Field Promoters: the field promoters are recruited, trained and managed under the AKF sub￾grant, and all goals for the number deployed and trained have been met. The promoters encountered in the course of the fieldwork at various farmer or nursery sites appear engaged and carry training material which they state is useful. They are the day-to-day face of the project, and claim that the unpaid workload involves typically a 3 day effort per week. Some issues have been reported in the payment of promised performance incentives, an issue being addressed in the AKF back office operations. Further process improvement and monitoring by management is recommended. 4. The network of field promoters and nurseries is active and on target to achieve the basic production targets of the project. The most capable promoters and nursery operators are demonstrating strong potential but lack a services model that can create a sustainable set of interactions between these parties and the large base of registered farmers at the termination of MozaCajú Mid-Term Evaluation Page 28 the project. Promoters have grassroots presence but lack a service scope that can provide them with a paid income from supporting farmers. Nurseries have good project-provided initial resources and technical training but have very low commercial capability to generate demand and to source affordable materials. These activities might become viable if promoters and nurseries had access to additional services that they could sell, enabling them to generate a paid income. Facilitatorstaff are able to influence INCAJU chemical distribution to varying degrees, but they cannot help with then providing the spraying services to farmers. These factors led to the evaluator proposing that requested resources and support could be provided to develop those ancillary services, in particular spraying services. Spraying has a known market demand, and the going rate of 2-3 kg of RCN per tree sprayed (see footnote 14, page 14) can be reduced through competition. The potential opportunity to enable capable local-level actors to make a living by offering valued farmer services, while presumably reducing the cost to farmers of these services through competition, seems worthwhile. But the grant-making mechanism in the program is not geared to the provision of multiple micro-grants (a sprayer costs approximately $600-700). An intermediate approach is being considered, based on grant-making to one or more commercial service providers who would in turn equip nurseries and/or farmers within the project to conduct spraying services. As of the time of writing, that initiative was still in planning; its resolution and its potential extension to promoters is recommended as a first-order priority. 5. Facilitators: the facilitators attending various farmer meetings displayed a good basic technical assistance capability, but with varying degrees of energy and productivity in the farmer encounters witnessed. The facilitators have been trained in the set curriculum and equipped with CommCare and manuals. Potential areas of improvement that they suggested seemed appropriate for the context: e.g. knowledge exchange for specific topics relevant to their farmers. Some process improvements would enhance their value to the farmers. No facilitator carried a farm management plan, or tracking tool, relying (fallibly) in each of the pre-meeting briefings on memory. Their role with producer groups and individual farmers is crucial to farmer investment; the facilitators should be equipped and accountable to deliver or arrange commercial support beyond simple record keeping. 6. Supervisors: each region spans a wide geography, with supervisors also managing some promoters and providing support to the largest farmers directly. The supervisors work would be enhanced by improved logistics and payment processes provided by AKF back office departments; these lead to delays that supervisors report feeling unable to improve. Facilitators see the need for additional training, including commercial, but the supervisors lacked concrete proposals for addressing training needs due to lack of resources; neither of the supervisors interviewed and observed for a day each in the field has commercial experience. The facilitator team should be equipped to deliver commercial support, through e.g. arranging specialist training or other support from commercial staff in Pemba or Nampula, with the facilitator accountable for making this happen. 7. Project coordination: has clearly improved as the project progressed, aided by weekly management alignment meetings and other efforts to build the intra-project collaboration. One frequently cited example noted positively by facilitators, processors and field staff, was the joint field visits by senior management of the project. Even with this, the project is spread across so many levels of activity and types of issue that greater attention to the few core objectives seems MozaCajú Mid-Term Evaluation Page 29 to be needed. This highlights the need for clearer alignment of the project onto just two or three defining objectives for the project’s final year. 8. Information flows: communication of project status and attainment is openly shared within the project team. Operational information processes from field level through the AKF field structure are less timely than needed. Field operational data all flow through the supervisor, these reports are paper-based, and the paper then flows to the central M&E team. This is the case with both measurement data and operational data. The M&E team naturally do not have the requisite skills or work priorities to act on or to route this information to the appropriate managers or specialists in a timely way, creating delays in managerial visibility of events and problems in the field. Conversely, all the commercial data, which is only collected at the annual commercialization period, bypasses the M&E team entirely, divorcing production activity from outcomes. The hierarchical flow of information has also meant that, at times, informed opinion is not sought within the project team. These processes should be addressed as a first-order issue, given the compressed timeline to project-end. 9. Information quality: emerged as problematic when investigating the lack of a feedback loop of commercial outcomes that would benefit operational management and their decision making. Survey preparation and implementation shows that farmer registration data are not always recorded adequately, and completion of details is significantly lagging for a proportion of records. Commercial outcomes are not routinely measured, there is no routine data collection even at simple levels of purchasing or selling. This leads to the project having a very high reliance on the Mid-Term Evaluation and Final Evaluation data collection to create those linkages. Process changes are recommended. An adjusted field staff work flow that collected key commercial data routinely from producer group meetings and individual farmers and nurseries, formatted and communicated electronically even if batched up to the end of the week, would start to close this gap. 5.2 Recommendations and Actions Summary The project is on track to meet or exceed all targets in the USDA agreement such as productivity, farmer income, processor upgrading and export sales outcomes. This is indicated by the actual achievement against project-end targets as of the conclusion of the first full season of project operation (2015-16 season), for example: Outcomes Target Description Project- End Target Achieved to Date End Target Achieved Total individuals benefitting from the project 24.1 thousand 23.6 thousand 98% Farmers applying improved technologies/techniques 17.3 thousand 15.8 thousand 91% Farmers applying improved farm management 5.8 thousand 3.9 thousand 67% Tree Productivity improvement (measured in RCN Kg/tree) 15% 31% 207% Total farmer income (cumulative) in USD to project end $7.2 million $5.7 million 79% Factories/processors selling via international certification 5 3 60% Factories/processors applying new technologies 6 2 33% Export sales of kernel (incremental) in metric tonnes 1.9 thousand 1.2 thousand 63% MozaCajú Mid-Term Evaluation Page 30 The full set of results is summarized in Annex (ii) of this report. Progress is less quantifiable at thisstage for a small number of targets. In particular, data are not reliably available at present for attributable net jobs growth and their economic value. Process improvements, capacity upgrades andgradual technology improvements are improving processor efficiency and volumes; some of these upgrades and expansions have led to new jobs being created, while other improvements have improved efficiency and permitted job numbers to remain static or even reduce. Data collection of attributable changes is further complicated by factorssuch as very high absenteeism rates of typically 40% and large variations month to month. The data available for the mid-term evaluation were too anecdotal to report with sufficient confidence. Converting raw figures to attributable full time equivalents requires improved data collection processes throughout the season; this cannot be left until the time of the evaluation. It is recommended that a quarterly data collection process is implemented for processor employment, with particular focus on net employment effects as new technologies or certification￾compliant processes are implemented. Based on the above findings (a number of which were already discerned to various degrees by project members), the key evaluation recommendations made to the Steering Committee Meeting (SCM) and their own subsequent decisions coming from the 3-6 February, 2016 meeting are summarized below. Achieving these objectives will fulfil the spirit of the USDA award, but realization of some key opportunities to build on this project will require support that extends well beyond the project term. Recommendation Rationale SCM Decisions 1. Prioritize final year resource allocations based on learnings. Focus more resources on pivotal players by further rationalizing the MozaCajú support package, aligned to # 4 & # 5 below. Catalyze change for high￾growth farmers, capable promoters, best nurseries. Bring private sector into the inputs and finance supply. Create business support for ‘nodes’ of key potential: i.e. income opportunities for promoters and nurseries. Design strategy to work with each producer segment. Bring private sector in to chemical distribution; bring FinAgro in for financing. Business plans for nurseries. Targeted field training Facilitator and promoter management review 2. Improve Production Component effectiveness of the sub-awardee: • Information flows • Commercial tools • Production admin and payment processing The farmers and nurseries that behaviorally can make the most improvements need targeted, commercially relevant, timely support. Cashew seedling and RCN production are time-sensitive. Improve information process within l the sub￾awardees operations. Enable commercial support for actors aspiring to sustainability post-project. Nursery commercial and technical guidance tools. 3. With initial credibility built, be more demanding on partners. Invest scarce resources more selectively and with shared commitment. Widely differing levels of commitment to e.g. HACCP (processors) or actively supporting the development of better producers (buyers). Focus tasks identified: Set commitment targets with processors receiving support; get an additional factory certified directly. MozaCajú Mid-Term Evaluation Page 31 5.3 Beyond Contractual Fulfilment Once the project has achieved all it principal objectives in terms of farmer income, new trees planted, processor upgrades and so on, it has an opportunity to build upon thisfoundation and achieve an even higher level of industry competitiveness and sustainability. Fuller realization of the benefits of MozaCajú relies largely on the capabilities and incentives of whoever will pick up the respective elements of the project at the end of the contract. While the remit of this evaluation report is to assess progress to date against commitments for the current award and suggest areas of tangible improvement for the remaining life of the project, this work naturally provided a basis for making tentative proposals for maximizing the value of the project prospectively. The final sub-section of this report does so by highlighting the exit plan priorities and identifying those learnings or resources that seem to warrant investment in further projects. To these ends, additional recommendations (numbers 4 & 5 below), going beyond those above for the committed project targets, aim to strengthen the exit plan. These are summarized below: Additional Recommendations Rationale SCM Decisions 4. Decide what this project must leave as its legacy. Agree the 2 to 3 primary measures of success for the end of project outcome. Resolve project design tension: is this ultimately delivering extension services as a public good, or an experiment in commercial enablement? Project Director to define and enroll the team. All-staff meeting (March). 5. Hand over to capable actors who can realize the progress made. Define a realistic but bold exit plan for MozaCajú. Focus on creating commercial sustainability of pivotal players or incorporation into INCAJU. Get a definitive understanding with INCAJU on transition. Project Director to define and enlist sector partners. Develop plan for technology transfer to INCAJU. The ‘legacy’strategy and related exit plan for MozaCajú have now been defined by the Project Director as the following three principal exit strategy actions: 1) Transfer CommCare (commercialization sales tracking and seedling tracker tool) to INCAJU; 2) Mobilize retailers, processors, donors, government for a post-MozaCajú effort in organic certification; 3) Formalize and document agronomic and farming-as-a-business training materials in a field manual, along with business planning training for nurseries, all of which can be utilized by promoters and producers beyond the life of the project. These plans appear realistic to anchor some key achievements of the project, by combining elements of commercial development and a strengthening of extension capabilities. Delivery of ‘lessonslearned’ briefings, a published lessons learned document and targeted additional training can also help to strengthen some of the more capable actors and to coalesce the stakeholders around some key issues, to help with that anchoring. MozaCajú Mid-Term Evaluation Page 32 Based on the learnings to date, there are strong arguments for continued donor support to targeted initiatives for the sector through the current MozaCajú partners and other partner agencies. Ongoing support seems warranted to further strengthen and accelerate the change processes set in motion by MozaCajú, by developing the participative model further while exploring potential sustainability via public-private relationship building and facilitation, based on demonstration of both policy-level and commercial benefits delivered. As observed by many of the institutional stakeholders interviewed for this evaluation, systemic change is needed; but they also note that realizing the farmer benefits of those changes will require sustained and targeted additional investment to build INCAJU’s absorptive capacity, to establish waysfor farmers to advocate for changes on their own behalf, and to enable the entry of private sector alternatives for the competitive supply of vital inputs and services. A longer, more targeted follow-on project would be required, in partnership with key actors, to complete the key initiatives started to date. In particular, the evaluator recommends consideration of developing a targeted follow-on project in order to: a. enable existing stakeholders, especially INCAJU, to absorb and effectively manage the pivotal resources established by the project, including viable nurseries and field promoters; b. create channels for farmer communication and advocacy with INCAJU; c. stimulate a competitive private inputs distribution sector to improve production volumes; d. deepen the nascent value-addition capabilities of genuinely committed processors through targeted technology upgrades matched by their contributions to support capable farmers. Achieving an effective transition of basic extension resources and capabilities to INCAJU could require up to 3 additional years, during which time a top priority would be the rapid transition of the day-to￾day operational support of the network. If INCAJU can secure the budgetary resources, the entire network could be transitioned relatively quickly, once management capabilities are strengthened. Failing that, a more focused follow-on project could work with a selected subset of (e.g.) 5,000 of the more responsive farmers (segmented behaviorally), along with say 50 of the more viable nurseries and a similar number of the more effective promoters, to establish an intensive testbed for layering on additional services and reconfiguring resources, in order to test specific hypotheses. Any robust transition plan – whether to INCAJU directly or by way of first establishing a further project in parallel to INCAJU’s routine work -- would need to be complemented by a program of INCAJU management system development, including better communication with farmers, so that progressively less grassroots intervention is required by the successor to MozaCajú. Encouraging INCAJU to exit the distorting supply of free inputs will enable private sector actors to offer more choice to farmers. The deadening culture of free inputs can, with some effort be overturned -- as has been witnessed in India, generally to the advantage of all farmers. This extended project would lead to deeper learnings about the conditions under which the sector can achieve greater efficiencies, to see whether more engaged farmers build more viable farming businesses, and to ascertain how the sector might attract relevant investment and financial services. The processors have had good technical and commercialsupport from MozaCajú; deeper collaboration to further upgrade their value-addition capabilities would be very attractive to some of them, and in MozaCajú Mid-Term Evaluation Page 33 return they would be required to make clear commitments to tangibly support the deepening of farmer linkages and the development of transferable learnings that INCAJU can also then adopt acrossthe farmer base. While only some preparatory steps for the above could be completed in the final year of the current project, MozaCajú will certainly contribute credible learnings to allstakeholders, by the end of itsshort project life, through workshops and documentation of lessons learned. Taking additional steps to convene stakeholders around these core, fact-based issues is also recommended for this final year – but this requires some stepping back from the intense focus on ‘getting the numbers’ in order to develop the clearer exit strategy. Effective communication of these findings and participative engagement of all stakeholders on the identified common issues and opportunities, should encourage policy makers, the private sector and leading farmers themselves to step up to support further developments that leverage what has been achieved. A final, more general recommendation based on the MozaCajú experience is that USDA may wish to revisit the way in which any awards of substantial sub-grants are made for entire project components. In this project, the up-front award of a large funding amount to the sub-awardee through a grant mechanism has seen some significant challenges emerge in aligning project priorities, timelines for task completion and understanding of performance interdependencies at a granular level. These sorts of issues could potentially be mitigated by e.g. a more graduated, milestone-based contract that is more closely aligned to the implementing lead’s evolving understanding of the project priorities as a complex project unfolds. MozaCajú Mid-Term Evaluation Page 34 Annex (i): MTE Terms of Reference PROGRAM EVALUATION TERMS OF REFERENCE (Revised December 2015) I. PROGRAM TO BE REVIEWED Program Name MozaCajú Program Location Cabo Delgado, Nampula, Zambezia Mozambique Program Budget $9,036,600 Program Start Oct 2013 Issue Program Addresses Program Goal Increased financial benefits for cashew processors and farmers Implementing Agency and Partners TechnoServe, AKF Evaluation Type (Mid- course) Mid-course Evaluation timeframe October 15 2015 to March 2016 (revised) Final Report deadline March 31, 2016 (revised - Final Report Loaded to USDA- FAIS) 1. Background on TechnoServe TechnoServe is an international non-governmental organization that promotes business solutions to poverty in the developing world. TechnoServe's mission is to work with enterprising people in the developing world to build competitive farms, businesses and industries. It does this by linking people to information, capital and markets. Active in Mozambique since 1997, TechnoServe is registered in the US as a 501(c)3 nonprofit corporation and headquartered in Washington, DC. Its staff of over 1,000 employees operate from 30 country offices in Africa, Latin America and Asia. For more details on TechnoServe and its work in Mozambique see: http://www.technoserve.org/ 2. Program Background MozaCajú is a three-year grant agreement funded by the United States Department of Agriculture (USDA), and implemented jointly by TechnoServe (TNS) as the principal grantee, and the Aga Khan Foundation-Mozambique (AKF) as a sub￾grantee. The project supports small cashew producers and processing factories in MozaCajú Mid-Term Evaluation Page 35 line with the Government of Mozambique’s ongoing efforts to reduce poverty. This Food for Progress-funded project supports the cashew industry by harnessing global market demand for premium cashew kernel and addressing obstacles throughout the value chain, including in the areas of inputs, production, processing, and marketing. In total,this project will help support approximately 26,000 small cashew farmers by increasing their productivity and competitiveness as well as connecting them to new markets. The project responds to increased demand for traceable cashew kernel as well as new commitments by several key global retailers to pay a premium for this guarantee. The project is introducing a sustainable system of shared value and traceability throughout the cashew value chain in Northern Mozambique. Among other benefits, traceability down to the individual farmer level facilitates organic certification and supports food safety audits. Toward this end, MozaCajú is working with seven (effectively all) of the top Northern Mozambican processors to achieve the highest level of food safety systems in order to supply some of the world’s most demanding markets, including the United States which represents 40 percent of all global cashew demand. Over 40 percent of Mozambican farmers or nearly one million households grow cashew, one of the few reliable cash crops that farmers are able to grow in the country. Smallholder farmers are responsible for more than 95 percent of the country’s cashew production. In Nampula province, cashews account for nearly one￾fifth of total household income and approximately two-thirds of total cash income. This project supports producers directly through agricultural extension, nursery development, and demonstrationplots. Several findings from this baseline survey of cashew farmers in the MozaCajú project areas are worth noting: The average number of cultivated trees per household is around 145, with almost 200 trees per farmer in Cabo Delgado, 120 in Nampula, and 100 in Zambezia. Productivity is very low, with average productivity per tree of only about 3 kg/tree, and a median production of only 1.6 kg/tree. Interestingly, the productivity per tree is significantly higher for households in the highest category of sales, at over 4.5 kg/tree compared with less than 2.5 kg/tree for the other sales categories. This may be explained by the fact that a larger proportion of farmers in the highest sales category apply fertilizer and spray their trees. Thus, there appears to be significant scope for increasing the productivity of cashew producers within the MozaCajú project area, and smaller farmers appear to have particular potential to catch up with the performance of the largerfarmers. Access to support and services is very limited for cashew producers in the project area. Overall access to training is very low. Only about three percent of interviewed farmers reported that they received any agricultural productivity training. Likewise membership in groups is low, with fewer than 20 percent of interviewed farmers reporting participation in groups. Only about 10 percent of farmers report that they MozaCajú Mid-Term Evaluation Page 36 have access to any sources of credit to invest in cashew production. The overall situation of cashew producers in the MozaCajú production area is one of low productivity, but with much scope for adoption of key techniques, and provision of training, access to improved inputs andcredit. The processors interviewed in the baseline do not face major marketing problems for whole kernels, but several of the firms reported that they had difficulties in finding buyers for pieces. However, even though they can presently find buyers for whole kernels, all the companies are aware that the international market for cashews is changing and that they need to make changes in their operations to meet more stringent demands. Since all of the companies interviewed sell the majority of their kernels to US and European markets, they are particularly aware of the growing demands of buyers in these two markets. This perceived need falls squarely within the scope of the MozaCajú project, which has a focus of helping Mozambican processors to meet needs of buyers – traceability, organic, and fair trade. The impact of MozaCajú will be reinforced by the implementation of the M&E plan, its products will support the project management team and provide USDA, local partners and other stakeholders with critical information to track progress and the best alignment of the resources to activities. Its evaluations will include three major surveys, baseline, mid-term evaluation and final evaluation. The project baseline was conducted during October to December 2013; Midterm evaluation will be done from October 2015 to February (revised to March) 2016 and final evaluation during the final four months of the program’s implementation. However, while the agreement dates place the project end date at September 30, 2016, TechnoServe has obtained a no-cost extension to March 2017 to include the third harvest season, in order to properly measure the full impact of this 3-year program. The fieldwork for the endline study will be conducted October – January, and the final report delivered in March 2017. 3. Midterm Evaluation The purpose of the midterm evaluation is to review and assess implementation progress and the results of MozaCajú’s activities on cashew farmers, processors, and consumer goods retailers in achieving the intended outcomes, and to help project management determine course corrections needed to fully achieve project results in the remaining half of the project. For each component, the midterm evaluation will compare progress against benchmarks set during the baseline study and document lessons learned. The midterm evaluation will cover all three target provinces and, through direct data collection of stratified samples of project beneficiaries and non-beneficiaries, will provide estimates of the project impacts on project beneficiaries by comparing the results of beneficiaries and non-beneficiaries at the time of the mid-term in relation MozaCajú Mid-Term Evaluation Page 37 to the baseline values of performance indicators (adoption of technologies, yields, profits from cashew production). A structured survey questionnaire similar in format to the baseline as well as select qualitative research will be used. In addition, the midterm evaluation will examine the quality of project design and the appropriateness of resource allocation. The midterm evaluation will draw lessons learned to create a set of recommended course corrections to ensure that all targeted results are achieved by the end of the project and address any issues that might prevent project activities from being sustainable upon the project’s completion. The review process will take a participatory approach, involving interviews with key stakeholders including USDA/FAS/MES staff, and FAD program administration, the Mozambican government agencies involved in the cashew industry such as INCAJU, implementing staff at TNS, including MozaCajú staff, and AKF, and relevant private sector firms involved in the project. These lessons learned will also be disseminated to all project partners as describe below under “Audience” fed into USDA’s learning agenda. 4. Objectives The evaluation process is intended to be participatory, and result in findings and recommendations that are valid, insightful and useful. While relevant program staff and key program stakeholders will be involved cooperatively to the extent possible, as this is an independent, third-party evaluation, the selected evaluator will control the design and implementation of evaluation activities. Overall, this evaluation intends to assess the extent to which the objectives of the program are in the process of being attained, help capture and articulate lessons learned so far, and make observations and recommendations on project improvement to TechnoServe, AKF, and USDA. Accordingly, the evaluation will focus on the following 5 component areas: Overall: Key questions to answer ● How well does the overall project design match the needs of the targeted beneficiaries? ● How well does the project align with national and regional development priorities as well as the development objectives of USDA and the US Government? ● Have project resources been allocated in such so as way as to allow activities to achieve the desiredoutcomes? ● What are the benefits key Mozambican stakeholders such as INCAJU (the national body responsible for the development and promotion of cashew) receives from the project’s influence and how can these be sustained? MozaCajú Mid-Term Evaluation Page 38 Component 1: Facilitate Buyer/Seller Activities: ● Market Access: Facilitate Buyer-SellerRelationships Relationships ● Market Access: Facilitate Private and Public Partnerships Training: Improved marketing and branding Key questions to answer ● How many new sales to new global buyers have been achieved by MozaCajú’s partners as a direct result of the project’s marketing work? ● How have attitudes of targeted international consumer goods retailers changed towards Mozambican cashews with regards to the factors identified during the baseline to affect sourcing decisions? ● Have consumer goods retailers set up any community investment projects, and what business value do the retailers intend for these projects to provide? ● How would targeted social investments affect the value drivers of brand trust established during the baseline consumer market research? Methodology Hired external evaluator will survey participating processors to determine the number and nature of any purchase agreements signed in the first half of the project. The evaluator will also review data pertaining to the targeted consumer goodsretailers to determine whether attitudes towards Mozambicancashews have changed with regards to the key factors identified as affecting sourcing decisions. Interviews with a subset of retailer representative with regards to the reasons behind the change in attitude (i.e. whether the change is due directly to a factor affected by project activities) will complement thisdata. MozaCajú Mid-Term Evaluation Page 39 Component 2: Set Up a Traceability System and Support Activities: ● Technical Assistance: Certifications ● Technical Assistance: Traceability ● Training: Post-Harvest Processing ● Training: Sanitary and PhytosanitaryStandards Key questions to answer ● Are firms making the necessary investments to up traceability systems in order to run traceability trials, food safety audits, and to receive organic or Fair Trade certifications? ● How does progress in setting up these systems compare to benchmarks established during the baseline? Methodology External evaluator will examine project records and survey participating processors to determine the extent to which the firms have made the investments in resources (personnel, technology, etc.) necessary to establish the various systems. This progress will be compared against the benchmarks determined in the baseline. Component 3: Expand SME Access to Finance Activities: ● Financial Services: Facilitate Agricultural Lending ● Financial Services: Leverage Private and Public Investment ● Financial Services: Provide SME Finance Key questions to answer ● Are firms accessing increased financing due to the loan guarantees and other project financing support? ● Have the attitudes of actual and potential private sector financers towards providing financing to the processors changed due to MozaCajú’s financing support activities? ● How does this progress compare against benchmarks established during the baseline? Methodology External evaluator will examine project records as well as surveying both processors and financers to determine what financing has been achieved by the activities, how attitudes of financers have changed, and how this compared against benchmarks. Component 4: Train Farmers on Improved Production Methods and Farm Activities: ● Training: Improved Agricultural ProductionTechniques ● Training: Post-harvest handling ● Training: Improved farmmanagement ● Training: Demonstration plots ● Capacity Building: Agricultural extensionagents/services MozaCajú Mid-Term Evaluation Page 40 Key questions to answer ● What practices or technologies are being adopted by beneficiary farmers through the model? ● Is the FPA extension approach proving effective in encouraging the uptake of new practices or technologies? ● How is this practice or technology affecting the yield of beneficiary farmers? ● What farm management practices have been adopted by beneficiary farmers through the model? ● What is the cost of the FPA model per farmer reached? Methodology The third-party independent evaluator will measure the impact training on adoption of improved practices, and yields of farmers who have received training compared with those who have not received training. At the time of the mid-term survey, the treatment and comparison groups will be identified as follows: 1. Participants in the treatment group will be randomly selected from farmers who have received training 2. Participants in the comparison group will be randomly selected from communities located out of the reach of training activities. This geographic separation will prevent spillover and crossover effects from occurring due diffusion of training to farmers. A third party evaluator will create a Midterm survey tool to test interim outcomes and early indicators of success. A structured survey questionnaire, incorporating some of the key questions as in the baseline, but focused more exclusively on data relevant for the core outcome-level indicators, as well as select qualitative research will be used. Mid-term results of the treatment and beneficiary groups will be compared to provide an initial estimate of the impacts of access to nurseries and other inputs. Note that this initial estimation may be confounded by underlying differences in characteristics of cashew farmers in the beneficiary and control groups. Component 5: Improve Access to Inputs and Activities: ● Inputs: Develop agrodealers and/or other inputsuppliers ● In-kind Grants: Inputs MozaCajú Mid-Term Evaluation Page 41 Key questions to answer Methodology ● How many farmers are planting new cashew trees due to the nursery system and at what rate are trees planted? ● Are these trees being planted in such a way as to lead to likely long-term survival (i.e. with proper germiling/grafting methods)? ● Are the nurseries sustainable? Are they viable long-term as individual enterprises, and as a collaborativenetwork? Are other agro-dealers (e.g. rural shops) being developed and sustained in such a way that they can act as market-oriented input suppliers over the long-term? The evaluator will measure the impact of the nurseries model on tree survival rates and costs to farmers. The third-party independent evaluator will establish the basis for an end of project quasi-experimental, differences-in-differences analysis to evaluate the impact of improved access to inputs and support services, if data quality is found to be sufficient. After nursery locations are determined, the treatment and comparison groups will be identified as follows: 1. Participants in the treatment group will be randomly selected from communities within reach of nurseries 2. Participants in the comparison group will be randomly selected from communities located out of the reach of nurseries. This geographic separation will prevent spillover and crossover effects from occurring due to nursery outreach and local radio advertising, while allowing for open selection. An independent evaluator will create a Midterm survey tool to test interim outcomes and early indicators of success. Mid-term results of the treatment and beneficiary groups will be compared to provide an initial estimate of the impacts of access to nurseries and other inputs. Note that this initial estimation may be confounded by underlying differences in characteristics of cashew farmers. In order to address this issue, if feasible a differences-in-differences methodology would ultimately establish independence of the treatment variable (access to inputs) from other factors affecting farmer income by controlling for all observed and unobserved characteristics that are consistent over time between farmers in the treatment and comparison group. It will also account for external, environmental changes that have similar impacts over time in the treatment and comparison groups, and confirm that behavior change over time between the two groups. The difference-in￾differences comparisons will be made between the mid-term and final rounds of the farmer surveys. Comparison of results between the beneficiary and control groups across the mid-term and final survey rounds will provide estimates the differences-in-differences of the identified indicators. MozaCajú Mid-Term Evaluation Page 42 VII. Audience The main audiences for this mid-term evaluation are TechnoServe’s Mozambique and global leadership teams, the USDA, and other USG agencies, AKF, global retailers, and program partners –the Mozambican Ministry of Agriculture, INCAJU, the Ministry for Coordination of Environmental Action (MICOA, the Center for the Promotion of Agriculture (CEPAGRI), National Directorate of Rural Development, and the Institute for Agriculture Research (IIAM). This evaluation is intended to measure progress against outcome indicators and issues around efficiency of resource use and quality ofimplementation. 6 Evaluation Methods TechnoServe seeks an evaluation approach that is appropriate for the scope of the program, intended audience and available resources. Evidence to be used will come from both quantitative and qualitative research methods, and include: primary sources – key informant interviews, focus groups, beneficiary stories, etc., and secondary sources – planning documents, program documents and reports, monitoring records, and baseline data, etc. MozaCajú’s activities are carried out in three provinces, Cabo Delgado, Nampula and Zambézia and across 16 districts from which some will be randomly selected. Farmer selection will be done based on Farmer Field Schools. This strategy will allow us to sample across the districts. The main objective is to stratify samples into farmers trained by the project according to “beneficiaries” versus “non- beneficiaries” and an additional methodology shall include “sprayed trees” versus “non-sprayed trees.” Upon evaluation start-up, MozaCajú’s program director and TNS M&E staff will support the lead evaluator to review program documentation and other relevant information and develop the evaluation work plan. A list of reference material that will be sent to the evaluator in advance of field work includes thefollowing: • Project documents • Baseline report • Semi-annual M&E report(s) • Q&A or Interviews with key project staff VIII. Key Activities and Deliverables The dates below presents an approximate summary work plan. TechnoServe anticipates a total lead evaluator level of effort of approximately 35-40workdays. MozaCajú Mid-Term Evaluation Page 43 Evaluation Key Dates (revised December 2015) • December 22- January 13: document review and remote management briefings • January 14 – January 30: Staff interviews and Fieldwork for Data Collection • February 5: MozaCajú Steering Committee Meeting presentation and feedback • February 6-28: remote interviews with key stakeholders; analysis • February 29-March 15: Report-writing • March 16: Draft Deliverable Due to TNS • March 27: Final Deliverable Due to TNS • March 28: Report sent to USDA • March 29-April 4: USDA review period • April 8: Response to USDA comments (by TNS and Evaluator) • April 11: Final Midterm Evaluation upload to FAIS Findings from the evaluation will be presented by contractor, or jointly with TNS, to USDA Washington, DC, USDA Pretoria and USDA Maputo,Mozambique. Additionally, TNS will produce a midterm evaluation summary for external audiences, including retailers and a Portuguese version for the Government of Mozambique. IX. Evaluation Logistics: The external evaluator will be responsible for: • Managing the evaluation process; • Developing, testing and applying any instruments/questionnaires • Leading the interview process (TNS to provide field assistance); TNS/AKF staff will facilitate the process rather than assume direct involvement, i.e. identify and introduce key individuals • Analysis of data; • Preparing and presenting all draft and final reports. MozaCajú Mid-Term Evaluation Page 44 TNS-Mozambique and AKF staff will be responsible for: • Providing the evaluator with copies of all TechnoServe background documents (proposal, grant agreement, reports, work plans, training materials, etc.) • Introduce evaluator to key informants as needed, assist in identifying/engaging suitable individuals for agreed-upon special data collection. • Assisting with logistics. . Unless otherwise agreed, all travel and logistical arrangements will be made by TechnoServe business advisors and administrative staff. X. Deliverables: All deliverables shall be submitted electronically, in both MS Word and PDF files. As possible, all photographs or other graphics/figures in the reports will also be submitted as separate editable files. Finally, the evaluator should also submit the final cleaned dataset used to inform the analysis. Evaluation Report Structure • Title Page • Executive Summary (maximum three pages) • Program description (including goals, objectives, andoutcomes) • Underlying program strategies that were used to achieve the program’s goals • Maps of program geographies (Annexed) • Evaluation purpose, methodology and approach • Evaluation findings, with reference to documented evidence. Use of specific examples, anecdotes and photographs to illustrate key points isdesirable. • Conclusions: insights into the findings; reasons for successes and failures; innovations • Lessons learned, barriers to success • Industry challenges and opportunities goingforward • Recommendations (based on evidence and insights) for both TechnoServe and the USDA Annexes to the evaluation report o Terms of Reference for the evaluation o Evaluation work plan with timetable o Sampling strategy o Data collection tools, including questionnaires, interview guides and other tools as appropriate o List of individuals interviewed and of stakeholder groups and/or communities consulted o List of supporting documentation reviewed XI. Selection Criteria/Profile of the Evaluator The ideal candidate to lead the external evaluator woulddemonstrate: • Strong relevant experience in designing and leading multi-faceted program evaluations with experience working in private sector agro-industry; • Background in international development and poverty-reductionprojects; • Ability to facilitate and relate to stakeholders at multiple levels (e.g., TechnoServe leadership and field staff, private sector business owners and managers, senior and junior government staff, program participants, community leaders workers and farmers; • Ability in quantitative, qualitative and participatory evaluation methods; • Data analysis and presentation skills; • Sensitivity to cultural/historical contexts. Additional training or expertise in anthropology, sociology or historical research viewed favorably; • Knowledge of Portuguese language helpful. XII. Submission and review of proposals to implement the evaluation Evaluators meeting the above criteria are invited to a proposal via email to TechnoServe at bpaul@tns.org (Use the subject line “Application and Proposal for Cashew Evaluation.” Application should include thefollowing: • CV of the applicant; • A writing sample; • An outline of a proposed evaluation design, methodology and general evaluation approach • Estimated evaluation budget, including the applicant’s daily rate, required additional non-TechnoServe personnel, etc. • List of 2 referees who can attest to the applicant’s experience and expertise as it relates to this project and this ToR. Proposals are expected from interested parties that meet the requirements by COB on August 21, 2015 (replacement evaluator appointed in December 2015 and contracted 11 January 2016). 7 Supervision The evaluator will work with the MozaCajú M&E staff, which includes TNS and AKF. The independent evaluator will be supervised by TNS’s M&Eteam. End of Document MozaCajú Mid-Term Evaluation Page 45 Annex (ii): Indicators Charts OUTCOME INDICATORS Result Indicator Target Status Contribution to Feed the Future # of farmers who have applied improved new techniques and technologies as a result of USDA assistance 17,318 farmers 15,760 farmers (full 2015--16 campaign) # of farmers who have applied improved farm management practices 5,773 farmers 3,940 farmers # of hectares under improved techniques or technologies as a result of USDA Assistance 20,020 ha 76,842 ha1 # of jobs attributed to USDA assistance 865 jobs 0 jobs (to be collected at endline) Increased Agricultural Productivity % increase in cashew tree productivity 15% increase 31% increase2 Value of farmer income from cashew sales (USD) USD 7,205,426 USD 5,720,000 Total # of individuals benefiting directly from USDA-funded interventions 24,092 individuals 23,556 farmers Total # of individuals benefiting indirectly from USDA-funded interventions 120,461 individuals 117,780 individuals Increased Availability of Improved Inputs # of new trees planted (annual average per farmer) 15.6 new trees per year 25.0 new trees per year Improved Farm Management (Operational and Financial) # of farmers who have applied improved farm management practices 5,773 farmers 3,940 farmers % increase (in USD) in farmer income as a result of cashew sales (average sales per producer for all cashew products) 25% increase 71% increase3 Expanded Trade of Agricultural Products (Domestic, Regional and International) Value (in USD) of cashew sold into regional and international markets USD 12,700,000 >USD 12,000,000 Volume (MT) of processed cashew kernel sold into regional and international Markets 1,942 MT increase 1,252 MT increase (continued overleaf) MozaCajú Mid-Term Evaluation Page 46 MozaCajú Mid-Term Evaluation Page 47 OUTCOMES (2) OUTCOME INDICATORS Result Indicator Target Status Increased Value Added to Post-Production Agricultural Products # of jobs attributed to USDA assistance 865 jobs 0 jobs (to be collected at endline) Value of wages from jobs resulting from USDA assistance (USD) USD 1,301,300 0 wages (to be collected at endline) Increased Adoption of Established Standards by Industry # of processors/factories passing food safety audits 4 processors/factories 0 processors/factories Increased Use of Improved Post-Production Processing and Handling Practices # of processors/factories that have applied new technologies and/or management practices as a result of USDA assistance 6 processors/factories 2 processors/factories Improved Marketing of Agricultural Products # of processors/factories pursuing exports under international certifications 5 processors/factories 3 processors/factories Average % volume of traceable cashews 60% ~40% claimed by processors # of processors/factories participating in international marketing & branding 7 processors/factories 3 processors/factories Increased Accessto Market to Sell Agricultural Products % of introductions made between distinct retailers and processors/factories that resulted in a sale 50% 53% 1 area (in Ha) under improved techniques calculated based on those farmers adopting 3 or more GAPs, meaning improved agricultural practices. 2 refers to the difference in yield between the 2013 and 2015 harvests. It should be noted that the yield figures fluctuate and are cyclical from one harvest to the next. Also, yield seems to be affected at least equally by factors such as the availability of chemicals and climatic factors as much as to project-specific interventions around other improved agricultural practices. 3 % increase in income has been influenced primarily by changing price levels, with +40% higher prices being offered for RCN in the 2015 campaign as compared to 2014 campaign, and higher again versus 2013. MozaCajú Mid-Term Evaluation Page 48 OUTPUTS OUTPUT INDICATORS Activity Indicator Target Status In-Kind Grants (Inputs) # of farmers benefitting from the procurement of seedlings/germilings 19,963 farmers (pending approval by USDA to reduce to 4,000 farmers) 2,539 farmers # of seedlings/germilings procured by farmers 399,253 seedlings/germilings 186,858 seedlings/germilings Capacity Building: Agricultural Extension Agents/Services # of agricultural extension agents who have received training in improved cashew production techniques and technologies 145 extension agents 176 extension agents # of agents who are certified in the use of improved cashew production techniques and technologies 116 extension agents 146 extension agents Financial Services: Facilitate Agricultural Lending # of loans, grants, etc. disbursed to direct beneficiaries as a result of USDA Assistance 6 loans/grants 5 loans/grants Financial Services: Leverage Private and Public Investment Value of capital accessed by processors for expansion or improvement of operations by leveraging project matching grants USD 250,000 USD 1,500,000 # of analyses completed on the viability of expansion projects or other improvement investments 3 analyses 2 analyses Number of financial investments made by processors for expansion or improvement of operations by leveraging the project matching grants 3 investments 2 investments # of employees benefitting from capital accessed by processors 950 employees 200 Financial Servces: Provide SME Finance # of loans and grants disbursed to processors/factories as a result of USDA Assistance 5 loans/grants 5 loans/grants Value of matching grants to processors USD 250,000 USD 157,446 Financial Services: Provide SME Finance to Inputs Value of subsidies provided to community nurseries (USD) USD 68,000 USD 62,000 # of subsidies provided to agrodealers for the distribution of seedlings/germilings 85 subsidies 85 subsidies # of micro-grants provided to communitry nurseries 85 micro-grants 85 micro-grants # of micro-grants provided to tool workshops 5 micro-grants 0 micro-grants (continued overleaf) MozaCajú Mid-Term Evaluation Page 49 OUTPUTS (2) OUTPUT INDICATORS Activity Indicator Target Status Inputs: Develop Agrodealers and Other Input Suppliers # of agrodealers created and operational 90 agrodealers 90 agrodealers # of nurseries created 85 nurseries 85 nurseries Market Access: Facilitate Buyer-Seller Relationships # of processor-retailer introductions made 12 introductions 25 introductions # of processor representatives benefitting from introductions to retailers 24 representatives 3 representatives Technical Assistance: Certifications # of industry stakeholders trained in certifications 90 stakeholders 87 stakeholders Technical Assistance: Traceability # of traceability audits conducted 5 audits 2 audits Training: Demonstration Plots # of demonstration plots created 660 plots 1,168 plots # of farmers benefitting from demonstration plots 23,091 farmers 23,556 farmers Training: Improved Agricultural Techniques # of farmers who have received training in agricultural production techniques 23,091 farmers 23,556 farmers Training: Improved Farm Management # of farmers who have received training on improved farm management practices (e.g. governance, administration, or financial management) as a result of USDA assistance 23,091 farmers 23,556 farmers Training: Post-Harvest Handling # of farmers who have received training in post-harvest handling 23,091 farmers 23,556 farmers Training: Post-Harvest Processing # of processor representatives trained in post-harvest processing 50 representatives 25 representatives Training: Sanitary and Phytosanitary Standards # of processor representatives trained in sanitary and phytosanitary standards 46 representatives 34 representatives MozaCajú Mid-Term Evaluation Page 50 Annex (iii) MAP of the Project Note: detailed planning investigation following the baseline and initial design work led to the decision to eliminate three districts shown in Cabo Delgado, namely Muidumbe, Palma and Metuge because of lower farmer number density than expected. MozaCajú Mid-Term Evaluation Page 51 Annex (iv): MozaCajú Baseline & Mid-Term Farmer Surveys – Study Designs20 Baseline Farmer Survey The baseline survey was a study of the general population of cashew-producing farmers in select districts of the three provinces(Cabo Delgado, Nampula, Zambezia) rather than being a survey of MozaCajú project farmers – the latter had not yet been defined and delimited at the time of conducting the baseline. An external evaluator, Mark Langworthy from TANGO International, led the design of the baseline survey – including the sampling strategy and questionnaire. Both TNS and AKF had considerable input into the content of the questionnaire, although the questionnaire’s overall structure, and the approach for obtaining data on key indicators like yield and farmer income, was ultimately determined by the external evaluator. Data collection was overseen by TNS (in Nampula and Zambezia) and AKF (in Cabo Delgado), with internal M&E staff serving as field supervisors managing four teams of hired enumerators. The TNS M&E Manager, Mily Devji, was responsible for carrying out initial rounds of data cleaning, with the external evaluator doing further checks on the cleaned data and signing off on the dataset. The TNS M&E Manager also supervised data entry, carried out by two hired data entry clerks at the TNS office in Nampula. Data analysis and the drafting of the baseline report were done by the external evaluator, though the poor quality of the draft report meant that TNS and AKF both contributed to polishing it. The sample was inclusive of 1,008 farmers – 72 farmers each sampled in 14 districts. The sample size was calculated with yield per tree as a reference indicator and to detect a difference in cashew productivity per tree (kg/tree) of 30% between any pair of the 14 districts with a 95% statistical confidence and 80% statistical power. With an equal number of farmers being sampled from each district, sample weights were applied to adjust to the differences in district population. However, we would ask you to note that the sample, while representative of the overall project area (the three provinces) was not designed to be fully representative of producers at the district level; a considerably larger sample size would have been required to obtain the requisite level of precision. The baseline study was done very quickly - within three months of the project contract being signed as per USDA regulations. At this point, very little detailed planning or direct scoping work had been done to determine what geographies MozaCajú should work in or what types of farmers it should work with (though the TNS and AKF program staff at the time did develop a rough list of districts/admin posts in which it was suggested the baseline focus its efforts). No groups had been established at the time of the baseline since the majority of project staff had yet to be hired. As such, the baseline was carried out in some districts that MozaCajú did not end up working in (e.g. Palma, Muidumbe and Metuge in Cabo Delgado). Within the administrative posts recommended by the program team, a random selection of villages was done through systematic sampling (by the external evaluator) and the overall selection of farmers per village was done, in the absence of household/farmer lists, through a random walk exercise. The focus of the baseline survey was on capturing data relating to the production outcome indicators noted in the project Performance Monitoring Plan (PMP). These indicators were: i. # of hectares under improved techniques or technologies; ii. # of farmers who have applied new techniques and technologies; iii. # of farmers who have applied farm management practices; 20 Please refer to the ‘USDA FFP – TNS MozaCajú Evaluation Plan’ for further details. MozaCajú Mid-Term Evaluation Page 52 iv. % increase in cashew tree productivity (yield per tree); v. # of new trees planted; and vi. % increase in farmer income as a result of cashew sales. Unfortunately, the inadequate project planning that had occurred at the time the evaluator conducted the baseline meant there was a lack of clarity that resulted in some data being unusable. Yield per tree was the key indicator of interest at the time of the baseline and the figure obtained – an average yield of 3.2 kg/tree – was cross-referenced with INCAJU and deemed to be reasonably accurate. However the quality of the baseline data analysis was found to be questionable because of ambiguities in definitions of variables such as which production techniques/technologies or farm management practices to include in the survey, and because the design focus on making district-wide projections for the key indicators (based on weights) proved to be confusing and misaligned (since the project geography changed post-baseline). The baseline questionnaire included additional data on potentially interesting topics such as the sale of cashew by-products, approaches to dealing with different pests and diseases and farmer access to credit. The baseline study itself was ultimately seen as a falling short of expectations, but the structure of some of the questions, particularly relating to yield and income, were deemed reasonably useful and were maintained for the Mid-Term survey. For this Mid-Term Evaluation we therefore drew on three approaches. 1. Mid-Term Farmer Survey Initially, the Mid-Term survey was supposed to be carried out by an external evaluator, Andrew Lambert, who was also responsible for the rest of the Mid-Term evaluation. However, once on the ground, this evaluator was not prepared to carry out this exercise and did not seem to understand the reference to it in the evaluation ToR. A quick decision was made to then look for a local consultant who would be able to undertake the farmer survey while the lead external evaluator, who had by this time been replaced by John Tull, would focus on the qualitative aspects of the Mid-Term and provide overall survey guidance where it was deemed necessary. The local consultant selected, was chosen through anopen bid process, but also on the recommendation of TNS – who had previous experience working with him on (non-survey) work in the cashew sector. The Mid-Term farmersurvey was focused very specifically on collecting outcome-level data relating to the six production indicators noted above. While some additional data were included on topics relating to farmer costs and post-harvest handling, the questionnaire was tailored to focus on these indicatorsrather than to be a more broadly evaluative tool – a decision that was made on the basis of trying to contain costs and keep the time required for data collection and analysisto a minimum. Unfortunately, this means that opportunities may have been lost to collect some further data that would have complemented the broader Mid-Term evaluation. The baseline analysis for the above six outcome indicators, as well as for post-harvest handling, was replicated as part of the Mid-Term evaluation. However, since the baseline had been performed before the project had any real understanding of what it would do (e.g. what the focal content of its trainings would be), response categories in the improved production techniques and improved farm management practices questions do differ. Overall formal comparability between the baseline and Mid-Term is limited by the fact that the two studies are essentially looking at two different sub-sets of farmers – overall farmers selected from a range of districts (some no longer relevant to MozaCajú) and from very specific MozaCajú Mid-Term Evaluation Page 53 admin posts within these districts; whereas the Mid-Term sampled population focuses specifically on MozaCajú farmers (and a small non-treatment ‘control’ group). Note that the project evaluation plan itself does not outline or seemingly expect any real comparison between the baseline and Mid-Term evaluations – though it does expect comparison between the Mid￾Term and Final evaluations. The structure of the questions asked in the Mid-Term relating to the six outcome indicators follows the guidelines set down in the project PMP – e.g. the PMP expects that yield will be determined by dividing total production by the # of productive trees and that farmer income will be determined on the basis of determining sales and multiplying this by a reference price; that # of hectares under improved techniques will be determined based on asking the # of trees to which farmers are applying these techniques and multiplying this by an average stocking density; and so on. Neither the evaluation plan nor the PMP make any reference to district-based projections. Projections are expected to be made on the basis of the overall project population – e.g. taking the % of farmers whoare adopting farm management practices and multiplying this by the overall # of project farmers to get abroad project-wide estimate, segmented in various ways (e.g. by province, gender, age). Therefore, despite the baseline design, we have decided to disregard district weights and district projections for the Mid-Term and the Final evaluations. As noted above, the Mid-Term (and Final) studies were to move away from the baseline approach and instead carry out a comparison of MozaCajú project beneficiaries and non-beneficiaries. USDA expressed a desire for the sample size to be kept at 1,000 beneficiary farmers, which both TNS and AKF agreed would provide a reasonable level of representativeness/precision in relation to the entire project area – though not representative at the district level (as noted, never the expectation). However, the final decision on the sample size was made on the basis of the recommendation by the external evaluator for a sample of 1,000beneficiary farmers and a further 200 to serve as a control. This was communicated to USDA in final revisions of the evaluation plan in December 2015. In the actual conduct of the survey, we obtained acceptable survey responses from 1,045 beneficiaries and 214 control group farmers. Approximately 80 treatment group records were subsequently not used in the analyses, as those farmers had joined the project after May 2015 and probably only received partial support in the 2015-16 season. We considered those farmers to likely be too proximate to the project participants, however, making them ineligible for inclusion in the control group. The beneficiary sample was drawn from the project farmer database and includes beneficiaries from across each of the project provinces/districts. The overall number of beneficiaries from each province was determined proportional to the overall size of the beneficiary population in each, meaning that 60%of the sample is from Nampula, 33% from Cabo Delgado and 7% from Zambezia. This database does haveflaws in relation to field staff (in some districts) not being reliable in providing updated information (and M&E not always having the time/resources to follow up and regularly verify). We accordingly resolved that verifying the database will need to be a focus over the final year of the project to ensure that the sampling frame is asstrong as possible forthe Final study. The controlsample was obtained by the surveyteam, with the assistance of district-level officials, in MozaCajú project districts but away from areas where the project is active (to avoid ‘spillover’ effects). As much as possible, attempts were made to ensure that the broad characteristics of the control sample matched those of beneficiaries in terms of MozaCajú Mid-Term Evaluation Page 54 their underlying characteristics (e.g. that control farmers were broadly similar to MozaCajú farmers pre￾project in terms of their access to extension support). We recognize several limitations of the study design and implementation. The underlying data used for the selection of the sample is imperfect; the project team acknowledge persistent challenges in obtaining and verifying data from some areas due to communication issues and personnel turnover. The control group of 214 non-beneficiary farmers was selected to be broadly similar and proximate to the project while avoiding spillover effects; once the data were collected however we found that many of these non￾beneficiaries operate smaller farms than expected, with lower average numbers of productive trees and therefore potentially lower activity levels - factors borne in mind in the analysis to avoid bias. The project has struggled to register and profile all farmers being supported. This was seen in the survey implementation itself; because of in-vivo substitution suggestions made by a facilitator in Mueda, a district with a significant number of larger producers, the survey ended up recruiting substitute survey participants when listed farmers could not be located; seven of the twelve largest producers across the survey were then found to have not yet been registered in the project. We believe these decisions were pragmatic and set a good foundation for a Mid-Term versus Final comparison. However we openly acknowledge the limitations. The fact that the baseline was done in at least a few very marginal districts as well as the fact that the selection of specific villages and farmers within these villages (e.g. through random walks) limited the geographic spread of the baseline sample, means that baseline figures could be skewed low. Similarly, the relatively small ‘control’ group, selected from farmer clusters at some distance from the MozaCajú project area in order to avoid spillover effects, is more indicative than strictly representative. One advantage of the approach being taken is that a substantially representative cohort of qualified MozaCajú farmer respondents (N=961), along with a control cohort (N=214) have now been analyzed for the Mid-Term and can be re-surveyed as part ofthe Final evaluation. This will provide a longitudinal record of two years of ‘immediate’ data (on a mostly recall/verbal basis, admittedly; some with records in hand), for 2015-16 and 2016-17 seasons, and two years of already-recorded recall data (2013 and 2014) for those same farmers, whether treatment or control. Expectations of limited comparability is why the Mid-Term survey tool asks farmers, on a recall basis, to provide information on their production and incomes as far back as 2013 – essentially trying to replicate (in an imperfect way) a revised baseline for the project farmers. 2. Qualitative Study – Producer Groups, Nurseries and Promoters Project M&E resources have been narrowly deployed to date; AKF has focused on production, whereas TNS has focused more on the commercial aspects but mid-way lost its M&E resource who has not been replaced. The project therefore needed insights into how stakeholders perceive the project, its intervention efforts, results and any drawbacks. For this reason, AKF’s Monitoring, Evaluation and Research (M&E) team undertook a qualitative data collection exercise in September 2015, with the objective of understanding the extent to which (and how) MozaCajú interventions are benefiting the project’s targeted farmers. These interventions relate primarily to trainings on a range of topics that encompass improved production, post-harvest handling and farm management techniques. The study also complements this farmer assessment by examining changes in the accessibility of inputs resulting MozaCajú Mid-Term Evaluation Page 55 from the introduction of community nurseries and nursery owners’ perceptions of their emerging businesses. Finally, the study notes the perspectives of FPAs on both their contributions to and benefits from the project, as well as on their motivation and satisfaction with the support they have been provided by project management. In short, the study was conducted to provide evidence to inform on the progress of project activities and to shed light on issues that the external evaluator(s) could then be asked to focus on in more depth as part of the project midline study to be carried out in November/December 2015. Methodology Since the objective of the study was to go beyond collecting numerical data regarding the project indicators, a qualitative methodology was adopted. This methodology consisted of a mix of Focus Group Discussions(FGDs) and in-depth interviews (IDIs) (see Appendix 1). This approach was adopted to capture the complexities of human experiences, attitudes, perceptions and values, which in fact are crucial to effectively and coherently materialize the project goals. The study focused on the production component of the project and provides in-depth information on the extent to which MozaCajú interventions are benefiting the project’s targeted farmers. Seven (7) FDGs were conducted with seven (7) producer groups located in six (6) different districts, six (6) IDIs with seven (7) FPAs and four (4) IDIs with nursery owners. The producer groups included in the study were randomly selected through a systematic sampling process from a list of all the groups MozaCajú is assisting. In case producer groups were from a district that already had one producer group selected, that group wouldn’t be considered so as to avoid sampling more than one group per district. Due to lack of resources and time, F PAs and nursery owners included in the study were purposively selected based on their proximity to the selected producer groups in a given area. Each FGD lasted for about an hour while the IDIs took about 30 minutes. In most of the districts, the team begun by interviewing FPAs, which helped forestall the research team’s blank spots, provided good contextual information and raised important questions that should be geared towards the different conditions and experiences of each of the producer groups, including seeking expressions of negative experiences with the project. The AKF M&E Assistant for MozaCajú, an MBA based in Pemba, conducted the FGDs and interviews. The study adopted semi-structured questions designed by the M&E team, after piloting in Mocimboa da Praia. To allow the respondents to freely speak up and in an unbounded matter fully express themselves, the data were collected in the respondents’ preferred language, which varied between Portuguese and Macua. All the data were collected face-to-face and audio-recorded with the consensus of the respondents. All the tape-recorded data were verbatim transcribed to enable systematic analysis combined with analysis of field visits and notes taken during the FGDs and IDI served to triangulate the data, thus ensuring the trustworthiness of the data set. Three major limitations characterize the study: • Methodology: The research team failed in some cases to stick to the ideal number of participants in FDGs(8-12 people). This was because each producers group is composed of an average number of 25 producers and the study was conducted in high period of cashew cultivation (ending of chemical spraying and approaching of commercialization period), thus, almost all the producers wanted to be involved. The average number of producers per FGD was 15. To mitigate this, the MozaCajú Mid-Term Evaluation Page 56 interviewer kept a balanced conversation by including points of view of the diverse producers including women, avoiding in this way dominance from the group’s leader. • Reach: Due to limited resources and time the study only reached a limited number of people. • Generalizing conclusions: Due to the nature of qualitative research, the study is limited by the fact that results cannot be generalized and there is susceptibility to biases on the part of the researchers. To address these potential limitations the assessment team has attempted to not over-generalize conclusions. However, it should be noted that the study found a number of similarities in responses between producers groups across the different sites. 3. Qualitative Study of Larger Farmers, Nurseries, Processors, Buyers, Advisers and Staff Having retrospectively validated the small/medium producer study (section 3.2 above) and the proposed quantitative study (section 3.1 above) the replacement evaluator designed an interview and inspection data collection process that would complement the above with a small number of targeted in-depth interviews and visits to test the key operational and commercial assumptions of the project. A second objective was to collect the quantitative measurements that the project’s M&E process are unable to gather – such as processor volumes, investment and costs; and buyer perceptions of the market opportunity and intentions to procure. The evaluator was supported throughout by the M&E Assistant from the project. Most interviewees agreed to be audio recorded, documents were collected, extensive notes were taken in meetings, and these artefacts were jointly analyzed in an iterative process of analysis. The motivation for this qualitative study was to deepen the data collected from the farmer survey, the small farmer and field promoter study, and from the many accounts given in interviews and documents by MozaCajú staff of dealings with other stakeholders including the more commercial-scale farmers, the processor and the kernel export buyers. Production of RCN is arguably the key element of the whole project: poor production harms farmers and destabilizes the downstream decisions and results of other players. Therefore one overarching question of this study was therefore: how can we better understand the relationship between the behavioral perspectives and aptitudes of farmers, for example, and the projects approach to promoting the adoption of new practices and techniques? The data collection processes and documentary sources noted above had yielded extensive data, often in standardized form; but the issues experienced in the project, such as the vastly different production outcomes of various farmers even within the same segment, prompted a closer investigation of what farmers do and how they make decisions. We chose to investigate this with the farmers operating more complex, multi-crop and often multi-site farms, to tease out the various factors. The project segments farmers pragmatically but simplistically on a count of productive trees; but anecdotally, farmers in each segment can display very different behaviors and some are clearly outliers in terms of production and commercial outcomes. In-depth interviewing allows for an exploration of decision processes and possible causal connections. For practical reasons a very small number of six in- depth interviews were planned with large farmers in five districts of Cabo Delgado and Nampula, with farmers chosen by the evaluator and be project M&E team for a maximum feasible diversity of activities and resource conditions. We selected interview candidates on the basis of recorded profile information, MozaCajú Mid-Term Evaluation Page 57 which was often incomplete, including factors such as whether the farmer was known to have invested in mechanization, whether she or he was actively planting new trees, and whether we could discern other indications of behavioral differences that could lead to insights about farmer decision making. Logistical factors played a part in the ultimate selection; distances are a great, and with the onset of the rainy season, travel times would be prolonged. The farmer interviews ranged from 120 minutes to over three hours, and proceeded as follows: (i) application of the quantitative survey, as used in the broad survey study. This gave us a set of current and its historical data to prompt further questions. Approximately 30 minutes. (ii) semi-structured interviewing using an interview guide (appended), notes taken in-vivo during the quantitative discussion and points arising in a free-flowing discussion. (iii) inspection of part of the farm to understand some of the specific practices and characteristics of the farming being conducted. The establishment of a network of 85 distributed nurseries, with the design philosophy of placing them as far from INCAJU nurseries as possible in order to test the commercial possibilities of seedling sales, raises many interesting questions. Can the farmers operating the nursery be trained effectively on grafting techniques? Would nursery owners find the workload acceptable? Could these private nurseries be productive in cultivating viable seedlings? Would the nursery owners discover ways to stimulate demand for sales to local farmers, or do they just see themselves as suppliers to MozaCajú? To address this mix of technical, logistical and commercial questions, we selected five nurseries proximate to the large farmers selected for interview and conducted in-depth interviews with the owners, supplemented by inputs from farmers and workers on site, and inspected the nursery sites. At each nursery and the single producer group interview site, a local field promoter and the MozaCajú facilitator joined; where time permitted, these individuals were interviewed to understand their working practices, the support they received and the services they in turn provided to their farmers. Two MozaCajú field supervisors joined the visits for 2-3 days each; both were interviewed at length to understand their working practices, the support they received and in turn provided to their staff and the promoters, internal processes for technical, logistical and commercial work elements and their views on the project and its areas of impact. This supplemented an extensive individual interview program with MozaCajú project staff and management to collect performance data, request documents and trace the workings of key processes and management systems. An extensive interview program of MozaCajú project top management, functional management, advisors and field staff comprising 17 interviews in five locations provided a good survey of the end-to-end activities of MozaCajú, its diverse and sometimes conflicting internal processes. These steps supported the triangulation of data collected and the development of suggestions for improvement. A second key question of the evaluation was: what are the constraints, opportunities and motivations of downstream stakeholders (processors, buyers) and how well does the project a line to create linkages for the farmers? Factory visits and interviews with factory management of one site from each processor group were conducted. Semi-structured interviews with top management of each group were arranged. Senior representatives of four international buyer organizations, supplemented by an interview with a representative of one investment group and with a food safety technical specialist deeply involved in the processor training for certification provided a diversity of perspectives. Interviews ranged from 60-90 minutes and included lengthy inspection tours at each processing factory visited. The respective interview guides are appended. Attempts to meet with the Director of INCAJU were unsuccessful due to the Director’s travel outside of the country at the time. The Project Manager was unable to secure a suitable alternative interview for the evaluator. Throughout these data gathering exercises, we set as a priority that respondents should be allowed to freely speak in an unbounded matter so they could fully express themselves. The data were therefore collected in the respondents’ preferred language, which varied between Portuguese, Macua, English and other local languages. Translation was provided by the MozaCajú M&E Assistant. With the exception of the buyer interviews and one CEO-level processor interview, which were via telephone and all in English, all the data were collected face-to-face and audio-recorded with the consensus of the respondents. Several validity check processes were used to ensure the trustworthiness of the data set. All key segments of the recorded data were verbatim transcribed to enable systematic analysis and combined with analysis of field visits notes and documents to triangulate the data. A series of discussions with relevant people – the Project Director, the Regional Cashew Specialist, and external advisors – helped validate the findings. An early validity check on the overall set of detailed initial findings of this work occurred through a two hour presentation and discussion with the steering committee of MozaCajú on 3 February, followed by extensive review of data with the M&E team. Project Director review of earlier drafts of this report also helped to confirm that the data analysis was accurate and fair, even where critical; no requests were made for material changes of any sort. These steps provided a strong endorsement of the overall findings and supported the development of the set of prioritized recommended actions proposed for the project in its remaining year of operation. Qualitative Interview Guide for Commercial-scale Farmers 1. Dimensions and trajectory: (refer to amended Comm’l Farmer Quantitative instrument). 2. Your History as a Cashew Farmer 2.1. When did you start farming cashew? Why did you choose cashew? 2.2. When you started, did you acquire an existing farm or plant it all yourself? 2.3. What other activities (crops, businesses, official duties) do you carry out? How much time do you dedicate to the cashew business? 2.4. When was the major expansion of your cashew farm (e.g. in early years; in recent years)? 2.5. What are the major challenges you face as a cashew farmer? 3. Resources: 3.1. Do you have mechanization (tractor, truck, sprayers, irrigation pump…) for cashew? 3.2. Do you have other facilities such as a warehouse or seedling nursery for cashew? 3.3. If you don’t own these, are they available to you from someone else (borrow/rent)? 3.4. Would any of those resources help your farming (tractor, truck, warehouse, irrigation pump, …)? If yes: why have you not invested in that resource? 4. Expansion Plans 4.1. You are / are not currently planting new seedlings. Do you plan to expand your tree numbers in the next 2 years? Why and by how much? 4.2. Would you expand by (e.g.) 1000 trees? Would that make you money? What would you need to expand like that? 5. MozaCajú Support Provided: 5.1. Are you receiving the Materials, Services and Training you need? 5.2. Are you receiving any other Support you need? 5.3. For 5.1 – 5.3, for Yes or No: “Please give a specific example of a technique/technology you have received from MozaCajú that makes a difference for your farming”. Probe. 5.4. Are you registered on CommCare? If No: heard of it? If Yes: use it (SMS volumes to sell)? 5.5. What could be improved in the overall MozaCajú support you receive? 5.6. What are specific examples. How much would that help you specifically (e.g. by increasing volumes, lowering costs, reducing risk…)? 6. Nursery: 6.1. Do you have your own nursery? If Yes: who do you supply? Selling them? Price/volume? 6.2. If No: Do you buy seedlings? If yes: How much do you pay? Have you considered starting one? Would farmers buy from that nursery? Would it be viable, financially? 7. Attractiveness & Confidence: 7.1. Do you want your children or grand-children to farm cashew? Why? END MozaCajú Mid-Term Evaluation Page 58 Qualitative Interview Guide for Processors 1. What opportunities are there in the world market for growing Mozambique’s market share of processed cashew and obtaining higher prices? 2. What are the main constraints to doing more business with the Mozambique cashew sector? 3. RCN Processing: what were your 2014 and 2015 processing volumes (RCN MT)? Please also provide the 2013 figure if possible. 4. Export sales: volume (boxes) and USD value of export kernel sold from 2014 and 2015? Also for 2013 if available. 5. Factory Employment: for 2014 and 2015: number of workers (average), and percentage that are women? Similar estimate for 2013 if possible. 6. Wages: Total factory wages cost: 2014 and 2015 in Mzn or USD? Similar data for 2013 also if possible. 7. Capital equipment or factory upgrade: amount invested in USD, in 2014 and in 2015 for the cashew factories? Did these result in any increase or decrease in factory employment - if so, please indicate the numbers? 8. HACCP and any other certifications: what are your current status and plans? Do buyers expect certification? 9. Export Sales Facilitation: has MozaCajú introduced buyers to you; and has MozaCajú facilitated any actual sales? 10. Business outlook over the next 3-5 years: how do you see its cashew business volumes growing in Mozambique over that mid-range period, what obstacles or issues do you face and how will you tackle them? 11. Support to ETG: Are there any other ways that MozaCajú is helping you? Are there any support efforts that have not helped as expected? 12. Suggested Improvements: Can you suggest any areas for potential MozaCajú improvement during its third and final year of the project? END MozaCajú Mid-Term Evaluation Page 62 Qualitative Interview Guide for Global Cashew Buyers 1. What volumes of cashew have you bought from Mozambique overall. How much of it was in some way facilitated by MozaCajú (such as, via direct introductions and in-person support to identify and start transacting supply): (Containers) 2013 2014 2015 2016 (plan) Total purchases made overall Purchases facilitated by MozaCajú 2. What opportunities, if any, are there for growing Mozambique’s global market share of processed cashew, and for obtaining higher prices? - e.g. what market niches are best suited to Mozambique’s sector? a) Short term b) Long term Please give a sentence or two to explain your outlook? 2. How important are Certifications (HACCP, ISO22000, EU) to your current procurement in Mozambique; and to those future opportunities you noted? 3. What are the main constraints to doing more business with the Mozambique sector today? 4. What specific MozaCajú services or support have you received from the project to develop the opportunities you foresee? (e.g. introductions to processors; facilitation of meetings and factory/farmer visits; market intelligence; risk reduction through information flows; etc... a) Has this assistance been of value to your organization? b) Would you have made this contribution from your own resources or was it a genuine added-value from the project? 5. How would you characterize your interactions with the project, giving your reasons? (if you have different types of interaction, please rate each one) 6. Does your organization intend to engage more closely with the sector in Mozambique to address the opportunities or constraints you’ve identified? (e.g. by providing direct support for increased RCN production, or by supporting bi￾product processing, etc…) 7. In the remaining 12 months of the MozaCajú project, what if anything could Daria and the team do to improve their support for the Buyers or the sector more generally? What could that achieve? END MozaCajú Mid-Term Evaluation Page 63 Yes / Limited / No MozaCajú Mid-Term Evaluation Page 64 Annex (v) : List of Key Documents Reviewed MozaCajú Award Agreement and Revisions • USDA/FAS Grant Agreement FCC-656-2013-038-00 September 2013 • Revision request October 2014 TNS MozaCajú Monitoring & Evaluation Plan MozaCajú Narrative Reports to USDA: • October-March 2014 • April-September 2014 • October-March 2015 • April-September 2015 ‘A Qualitative Assessment of MozaCajú Progress’, 2015; MozaCajú ‘Competitiveness of the Mozambican Cashew Industry’, MozaCajú report, September 2015 (word document and powerpoint; related internal policy brief) MozaCajú Indicators reports MozaCajú implementation plan and revisions TechnoServe MozaCajú Sub-Grants Manual AKF Component Reports to TNS • Quarterly since April 2014 INCAJU Master Plan 2015 INCAJU Presentation ‘Desempenho Do Sector- 2013’, October 2013 Processor Audit Reports • ETG • Condor Nuts MozaCajú training materials – technical, commercial CommCare Evaluation Report Email correspondence between MozaCajú management and Global Buyers MozaCajú Mid-Term Evaluation Page 65 ‘Factories in the Fields’, Paul, Brad, PhD., TechnoServe Mozambique 2008. ‘Value on the Margins: Wealth and Poverty in the Cashew Commonwealth’, Paul, Brad, PhD., TechnoServe Mozambique 2009. ‘Analysis of the Cashew Value Chain in Mozambique’, Bill & Melinda Gates Foundation, February 2010 ‘Value Chain Finance Assessment of the Cashew Nuts Sector In The Province of Nampula, Mozambique: Final Report’, April 2011, GIZ, African Cashew Initiative (ACI)