This publication was produced at the request of the United States Department of Agriculture. It was prepared independently by ANED Consultores. Foreign Agricultural Service, United States Department of Agriculture Farmer trainer for the USDA-funded MAS+ program demonstrates best practices for ensuring coffee quality in a solar dryer. Honduras Sustainable Agricultural Improvement Project (MAS+) MID-TERM EVALUATION NOVEMBER 5, 2021 DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Mid-term Evaluation of the Sustainable Agricultural Improvement Project (MAS+) The United States Department of Agriculture (USDA) Food for Progress (FFPr) project in Honduras is implemented by TechnoServe (2017 to 2022). Mejoramiento Agrícola Sostenible (MAS+) aims to improve the livelihoods of coffee and bean farmers in Honduras by improving their quality and yields on the farm; and by facilitating their access to direct marketing relationships, favorable financing mechanisms, and other important services across the market system. By supporting smallholders to become reliable suppliers of high-quality products, MAS+ demonstrates the business case for buyers to invest in financial- and technical-assistance to farmers. To date, the project has supported $20 million in incremental sales for farmers, with a $3.14 increase in farmers’ gross incomes for every $1 invested by USDA. Agreement Number: FCC-522-2017/023-00 Project Duration: 2017-2022 Implemented by: TechnoServe Evaluation Authored by: ANED Consultores i b Presented by: MID-TERM EVALUATION OF THE SUSTAINABLE AGRICULTURAL IMPROVEMENT PROJECT (MAS+) i LlST OF ACRONYMS AIMC All Inclusive Marketing Contract ANED Consultores Asesores Nacionales Especializados para el Desarrollo COMSA Café Orgánico Marcala S.A. COPECO Comité Permanente de Emergencias (Permanent Emergency Committee) CSA Climate Smart Agriculture CSB Community Seed Bank CT Community Trainer DICTA Dirección de Investigación, Ciencia y Tecnología Agropecuaria (Directorate for Agricultural and Livestock Science, Research and Technology) FFPr Food For Progress FFS Farmer Field School FOGAFRIJOL Fondo de Garantía de Frijol (Dry-beans Guarantee Fund) FUNDER Fundación para el Desarrollo Empresarial Rural (Honduran Foundation for Rural Enterprise Development) GAP Good Agricultural Practice GFSS Global Food Security Strategy GOH Government of Honduras iDE International Development Enterprises IFPRI International Food Policy Research Institute IHCAFE Instituto Hondureño del Café (Honduran Institute of Coffee) IHMA Instituto de Mercadeo Agrícola (Honduran Institute of Agricultural Marketing) IT Information Technologies LoP Life of Project MAS+ Mejoramiento Agrícola Sostenible Plus MEL Monitoring, Evaluation and Learning MSU Michigan State University MTE Mid-Term Evaluation PO Producer Organization qq Quintales (hundredweight) SENASA Servicio Nacional de Sanidad Agropecuaria (National Sanitary Service for Agricultural and Livestock) SHF Small Holder Farmer SME Small and Micro Enterprise SMS Short Message Service TNS TechnoServe ToC Theory of Change US United States USDA United States Department of Agriculture USDA-FAS United States Deparment of Agriculture – Foreign Agricultural Service WP Wet Parchment ii TABLE OF CONTENTS I. Introduction............................................................................................................................... 1 1. Purpose of the review and main questions.......................................................................... 1 II. Project background.................................................................................................................. 2 2.1 MAS+ Theory of change............................................................................................................3 III. Evaluation methodology ......................................................................................................... 3 3.1 Effectiveness of the Evaluation and main obstacles.............................................................5 IV. Conclusions and relevant findings relating MAS+ objectives and indicators................ 6 4.1 Relevance ....................................................................................................................................6 4.2 MAS+ results to date..................................................................................................................6 4.3 MAS+ effectiveness....................................................................................................................8 Activity 1: Training to Facilitate Improved Farmer Productivity,..................................................8 Activity 2: Capacity of PO Building: POs and Cooperatives......................................................10 Activity 3: Develop Input, Service, and Equipment Providers ...................................................12 Activity 4: Training of Agricultural Extension Agents/Services..................................................12 Activity 5: Facilitate Agricultural and SME Lending ....................................................................13 Activity 6: Develop Buyer-Seller Relationships............................................................................13 Activity 7: Building Government Institutions capacity and support Policy Regulatory Framework improvements. .............................................................................................................15 Cross cutting themes .......................................................................................................................15 4.4 Monitoring, Evaluation and Learning .....................................................................................18 4.4 Communications .......................................................................................................................19 4.5 Efficiency....................................................................................................................................19 4.6 Sustainability..............................................................................................................................19 V. CONCLUSIONS ....................................................................................................................... 20 VI. Lessons learned and recommendations. ........................................................................... 26 6.1 Lessons Learned ......................................................................................................................26 6.2 Recommendations....................................................................................................................26 General ..............................................................................................................................................27 On access to finance .......................................................................................................................27 On the Community Seed Banks.....................................................................................................28 On environmental stewardship and Climate Smart Agriculture ................................................28 On women and youth.......................................................................................................................28 On child labor....................................................................................................................................28 On the MEL plan...............................................................................................................................29 ANNEXES ........................................................................................................................................... 30 Annex 1. ............................................................................................................................................ 30 Comparison of Mid-term results against baseline..................................................................................30 I.1 Total production.....................................................................................................................31 I.2 Yields ......................................................................................................................................34 I.3 Total Sales...............................................................................................................................35 I.4 Income from Sales..................................................................................................................36 I.5 Make up of sales for coffee producers...................................................................................38 1 I. Introduction This report summarizes the results for the mid-term evaluation of the five-year Mejoramiento Agrícola Sostenible Plus (MAS+) Project financed the United States Department of Agriculture Food for Progress (FFPr) Program and implemented by a consortium led by TechnoServe. The evaluation was carried out by the firm Asesores Nacionales Especializados para el Desarrollo (ANED Consultores), a local consulting firm that specializes in designing, implementing and providing integrated technical support to development programs. Based in Tegucigalpa, Honduras, ANED Consultores has worked for more than 15 years throughout Central and Latin America implementing and evaluating development projects. During that time, ANED Consultores has acquired significant expertise in many areas, including carrying out socio-economic studies on health and nutritional services, food security, local and municipal development. The evaluation was carried out between late July and February 20211 . MAS+, initiated in FY2018 is designed to build upon achievements of a previous MAS program also implemented by TechnoServe between 2012 and 2017. MAS+ is designed to: (1) Increase the production and quality of coffee and dry￾beans through training of producers, expansion of financial services and inputs, and capacity building for producer organizations to increase producer incomes and build future resilience to environmental risks, and (2) Expand agricultural trade and increase volume of sales in the coffee and dry-bean value chains through marketing initiatives aimed at expanding international premium coffee and domestic commercialization of dry-beans. At the end of the project life (LoP), MAS+ aims to increase productivity and incomes of 32,020 producers (22,920 coffee producers and 9,100 dry-beans producers) and strengthen 1,000 Producer Organizations (PO)s. To achieve this, MAS+ implements seven activities: (1) Training to Facilitate Improved Farmer Productivity, (2) Capacity of PO Building: POs and Cooperatives, (3) Develop Input, Service, and Equipment Providers, (4) Training of Agricultural Extension Agents/Services, (5) Facilitate Agricultural and SME Lending, (6) Develop Buyer-Seller Relationships and (7) Building Government Institutions capacity and support Policy Regulatory Framework improvements. The successful implementation of these activities should result in a 30% yield increase and a $10,300 incremental revenue for coffee farmers, and a 30% yield increase and $1,170 incremental revenue for dry-beans farmers. 1. Purpose of the review and main questions The intervention of MAS+ is almost half-over, thus the mid-term evaluation is fundamental for evaluating the progress of the project to this date, to identify opportunity areas and to provide useful recommendations for the implementation during the rest of the Life of Project (LoP). The general questions answered in this evaluation are:  Have incomes of participating coffee and dry-beans producers increased to date?  What progress has the project achieved towards fulfilling its goals?  Is the current implementation strategy favoring that the project reaches its results? If not, what changes in the implementation strategy could increase the chances of reaching the results?  Are producers satisfied with training and services provided by the project. 1 The MTE consisted of 4 phases: planning (July-September), field work (September), analysis and evaluation (October-December) and reporting (January –February). There was active participation of MAS+ Monitoring, Evaluation and learning (MEL) and technical personnel throughout the four phases of the MTE. 2  Are there significant differences to date between MAS+ participant indicators and the baseline control group? Should these differences warrant review of current project strategy? II. Project background The Sustainable Agricultural Improvement Project (MAS+) is funded by the U.S. Department of Agriculture Food for Progress (USDA-FAS) and the J.M. Smucker Company, building on the success of TechnoServe’s previous MAS program, implemented from 2012 to 2017. The key objective of the five-year MAS+ program is to increase agricultural productivity and quality in the coffee and bean value chains in Honduras. To achieve this objective MAS+ works through a variety of activities including: developing strong market relationships between small farmers and buyers, strengthening the capacity of producer organizations (POs), building the capacity of partners to provide extension services, expanding producers’ access to credit, and increasing the capabilities of the government to strengthen input supply and services to smallholder farmers. A guiding principle for MAS+ is to demonstrate the business case for buyers to invest in financial and technical assistance to small farmers by supporting them to become reliable suppliers of high￾quality coffee and beans. The project formalizes this win-win arrangement through the All￾Inclusive Market Contract (AIMC) model, which aligns the interests of value chain stakeholders (including farmers, buyers, input providers, and financial and government institutions) within comprehensive and economically sustainable marketing agreements. Partnering with the Honduran Foundation for Rural Enterprise Development (FUNDER), and Michigan State University (MSU); MAS+ aligns the interests, skills, and resources of USDA and TechnoServe, large private-sector coffee exporters such as Molinos de Honduras, Beneficio Santa Rosa and Café Orgánico Marcala S.A. (COMSA), and IHCAFE, SENASA and IHMA around the shared objectives of increasing incomes for smallholder coffee and dry-beans farmers, strengthening their connections to input and output markets, and protecting the environment while developing profitable and sustainable coffee and dry-beans market systems. At the end of year 5, when key project initiatives have transitioned to market actors, MAS+ will have trained targeted farmers to adopt a minimum of three good agricultural practices, increasing their yields by 30 percent for bean and 30 percent for coffee and mutually beneficial trade relationships developed by MAS+ will allow for sustained marketing of these crops. At the end of the project, MAS+ will have also linked POs to formal markets based on favorable price contracts. Successful implementation of the seven MAS+ activities should translate into a more sustainable coffee and dry-beans market system with participating coffee and dry-beans farmers having increased their revenues by $10,300 and $1,170 respectively. 3 2.1 MAS+ Theory of change MAS+ theory of change (ToC) is depicted in Figure 1 below. The MAS+ consortium partners identified four key levers of change to address the needs, challenges, and constraints in the coffee and dry-beans value chains. These include: (1) provision of cost-effective, scalable, and high￾quality training; (2) access to inputs and services for climate change resiliency; (3) improved marketing platforms in diverse value chains; and (4) a policy and enabling environment that facilitates these levers of change. Figure 1. MAS+ Theory of change. Despite the dramatic changes in the operating conditions for MAS+ during the period: lowest prices in over a decade for coffee, droughts and disastrous hurricanes, the levers of change and activities still hold true and need to suffer no changes going forward. However, MAS+ would do well in reviewing its accomplishments to date on access to services (including training) for climate change and resiliency, and policy enabling environment to either modify its activities in support of these changes or set more attainable goals given the remaining project life. III. Evaluation methodology The design of the MAS+ mid-term evaluation was based on the standpoint of sequential mixed methods with a quantitative orientation. The criteria for analysis were based on the OECD-DAC2 : relevance, effectiveness, efficiency and sustainability. Because MAS+ is less than half-way over, the focus of the evaluation is not on causality (impact attributable to the project) but rather focuses on implementation. However, a comparison with baseline data collected for participants and a 2 Organization for Economic Co-operation and Development - Development Assistance Committee Goal Expanded trade and increased sustainability of coffee and dry-beans producers Levers of Change MAS+ Activities Provision of cost￾effective, scalable and High quality training Access to inputs and services for climate change resiliency Improved marketing platforms in diverse value chains Policy and enabling environment that facilitates these levers of change Activity 1 Training: Facilitate imporved farmer productivity. Avtivity 4 Capacity Buliding: Agricultural Extension agente Cross-cutting Building on what works Cross Cutting: Market driven solutions Activity 3 Inputs: Develop input, service, and equipment providers Activity 5 Financial Services: Facilitate agricultural and SME lending Activity 2 Capacity Buiding: Producer grous and cooperatives Activity 6 Market Access: Develop buyer and seller relations Activity 7: Government institutions and regulatory framework 4 control group of producers in 2018, was carried out to see if, at this stage of the project, there are already noticeable differences between the groups on 5 key indicators: (1) total production, (2) productivity, (3) difference in coffee prices received between sales to exporters and to intermediaries, (4) volumes sold and (5) gross income, and determine if there is need to review the project’s final evaluation strategy. Results of this comparison are presented in Annex I: Impact to date. To carry out the MTE, a joint work plan with MAS+ personnel was developed prior to initiating field work. The plan included four phases that include the mixed research carried out. The methodology for sample selection followed the analysis of the baseline study carried out between July and August 2018 by the International Food Policy Research Institute (IFPRI), maintaining the same variables so that a comparison between baseline and MTE results could be comparable3 . MAS+ initiated in FY2018 incorporating a first cohort of 20,034 producers (13,393 in coffee and 6,641 in dry-beans) from the departments of Comayagua, El Paraíso, Francisco Morazán, Olancho and Yoro. Throughout the document, these will be referred to as Cohort 1 producers. Also incorporated into MAS+ at that time, were 10,170 from the previous MAS program in those same departments.4 At the time of the MTE, these producers had received training and technical assistance through two coffee crop cycles. Dry-beans producers received training during the two yearly crop cycles known as “primera” and “postrera”. In 2019 a further 4,308 producers (3,818 in coffee and 490 in dry-beans) were incorporated from the remaining four departments (Cortés, Intibucá, La Paz and Santa Bárbara); these producers received training and technical assistance during only one crop cycle and will be referred to as Cohort 2 producers throughout the document. At the request of MAS+, ANED Consultores, collected data in a way that comparisons could be made against baseline data collected by IFPRI. For the baseline, after a sample of potential participant villages was established, half were randomly chosen to be in the treatment group and the other half in the control group. These same producers and villages were revisited to collect information for the MTE. Given the different goals for the use of the baseline data and those of the MTE data, there are some differences in the data collection instruments and methodology but not in that of the sample selection process. For the MTE a subsample of those producers (except for the coffee producers in the department of Yoro where the baseline study did not collect any data) included in the baseline study was taken to evaluate production levels, productivity, comparison of sales to exporters and intermediaries, sales volume, and income from sales. Since, overall, the analysis for the baseline study shows that the randomization was properly implemented and that the treatment and control groups are comparable at baseline across all the key outcomes of the evaluation, there is confidence in that comparisons between participants and non-participants at MTE will provide an accurate picture of progress to-date on these five indicators. To evaluate progress towards objectives, a sample of 552 producers, previously interviewed for the baseline study (276 for coffee and 276 for dry- beans) from the first cohort was selected for this evaluation. Using the same procedure for sample selection as that in the baseline study, an additional 312 producers (156 for coffee and 156 for dry-beans) from the second cohort were selected. This sample size provides a 95% confidence level on each of the key parameters 3 Despite using the same methodology as the baseline study, there are some differences between the MTE data and that of the baseline that require further analysis beyond the scope of this evaluation. 4 These producers were not considered in the sample of producers to be interviewed for the MTE, but their results on sales, total production, access to finance, and other indicators are considered for the global MAS+ results for the MTE period. 5 evaluated. This sample size provides a minimum detectable effect of 33% for the coffee yields and 29% for the dry-beans yields (mean detectable errors for yields in the baseline study were 30% for coffee and 25% for dry-beans). During the field work 282 coffee producers from the first cohort (136 participants and 146 non￾participants) and 284 dry-beans producers (133 participants and 151 non- participants) were interviewed5 ; for those producers in the second cohort, only participating farmers were interviewed (159 for coffee and 158 for dry-beans). A further 109 interviews through focus groups and semi-structured interviews were held for producers, the TechnoServe and partner institutions’ technical teams, and 11 focus groups involving cajas rurales (Rural Banks), dry-beans community seed banks and coffee cuppers. 3.1 Effectiveness of the Evaluation and main obstacles. Despite the field level restrictions imposed by the COVID-19 pandemic, the survey for the evaluation was carried out smoothly. Willingness to participate in the survey was 86% for coffee producers and 90% for dry-beans producers6 . In many of the coffee communities it was not possible to complete the number of selected interviews, however, replacements were quickly found. In the end, of the 864 programmed field level interviews in the 9 departments where MAS+ operates, 883 were carried out, providing a safety net for incomplete or failed interviews. For the semi structured interviews, given the restrictions imposed by the GOH given the COVID-19 situation, a methodology to carry them out virtually was designed. Restrictions at the field level regarding participation of selected producers did not reduce the number of participants as back￾up participants had been selected in case the selected participant was not able to attend. Focus groups were carried out using the GOToMeeting platform. Unfamiliarity with this methodology of interviewing for both interviewers and participants may have caused some bias regarding responses, as in general, people try to please the moderator or interviewer rather than freely voice their opinions. Without the accompaniment of MAS+ personnel for the interviews, enumerators, even though experienced in applying national level surveys, found it difficult to quickly arrive at the different communities, making the process slower than anticipated. It is recommended that for future surveys involving outside personnel, MAS+ designate field level personnel to help provide directions on routes and location of farms in order to speed up the process of data collection. A further difficulty with the MTE was the fact that it included two producer cohorts. Though these two groups are kept separate, there had been no baseline prepared for those producers in Cohort 2 (other than that collected by TNS at the time of entry) such that comparisons cannot be made regarding their progress towards production, productivity and other farm specific indicators. While the agreed upon survey focused on collection of those indicators requested by the MAS+ team, those indicators do not reflect the full spectrum of activities and cross-cutting themes the MAS+ program is implementing (in part because there are no indicators collected on these topics 5 More producers than those in the required sample size were interviewed to account for errors or incomplete surveys. 6 Due to COVID-19, some communities did not allow enumerators to enter the community, in other cases, the producer did not want to be interviewed. In all cases, a replacement was found using the agreed upon methodology for replacements between MAS+ and ANED. 6 by MAS+) and thus, this MTE does not provide a complete picture of the work MAS+ is implementing. IV. Conclusions and relevant findings relating MAS+ objectives and indicators 4.1 Relevance MAS+ objectives align well with USDA’s Global Food Security Strategy (GFSS) related to inclusive and sustainable agriculture-led economic growth, resilience, food security, and nutrition. The GFSS calls for implementing partners to coordinate to sustainably increase food security and nutrition, with a focus on scalable, evidence-based approaches that reduce poverty, hunger, and malnutrition and develop partnerships to improve the reach, effectiveness, and sustainability of these interventions. In particular, the seven MAS+ activities contribute to USDA’s Strategic Objectives of Increased Agricultural Productivity and Expanded Trade of Agricultural Products (Domestic, Regional, and International. Production of MAS+ participating coffee and dry-beans producers represents a sizable portion of Honduras’ total production of these crops (20% and 11% respectively) and thus are highly relevant for the country’s export and domestic markets. Additionally, as over 60% of assisted producers are SHFs with farm less than 2.0 Ha, MAS+ is very much aligned with the goals of the 2018-2022 Honduran Government’s Strategic Plan for the agricultural sector. Regarding the relevance of the MAS+ training and activities, 99% of those surveyed in this category, indicated complete satisfaction with the strategy, training content and project activities which directly concern them (Activities 1 through 6). Agreement among coffee growers was the need to include infrastructure financing activities in the project. For dry-beans growers, although only 6% of producers have irrigation systems, a needed change requested was to include training on irrigation use as part of the curricula. 4.2 MAS+ results to date Given the results to date on contractual indicators provided in the table below. It is highly likely, despite the recent highly disruptive events in market, health and climate conditions, that MAS+ will reach and surpass its main objective of increasing production, quality and income of both coffee and dry-bean producers. These results can be sustainable given the innovative approach to producer organization, access to finance, and market insertion strategies applied by the project that have resulted in increased volume of sales of higher quality coffee for the international markets and commercialization of dry-beans in the local market. However, it is not clear that this sustainability will also include the environmental, gender equity, and climate resiliency aspects of the project, as the project has not defined or collects indicators providing a clear pathway towards a sustainable state for these issues. 7 Table 1. MAS+ progress to date on contractual indicators. ACTIVITY PROGRESS TO DATE PROJECT GOAL PROGRESS (%) Training: Facilitate Improve Crop Productivity and Quality Number of hectares of land under improved techniques or technologies as a result of USDA assistance (Coffee) 15,870 27,200 58% Number of hectares of land under improved techniques or technologies as a result of USDA assistance (Bean) 10,042 4,480 224% Number of individuals benefiting directly from USDA￾funded interventions 34,512 32,020 108% Number of individuals benefiting indirectly from USDA-funded interventions 205,717 100,736 204% Number of individuals who have applied new techniques or technologies as a result of USDA assistance (Coffee Male) 6,745 13,134 51% Number of individuals who have applied new techniques or technologies as a result of USDA assistance (Coffee Female) 1,895 3,284 58% Number of individuals who have applied new techniques or technologies as a result of USDA assistance (Bean Male) 8,998 4,057 222% Number of individuals who have applied new techniques or technologies as a result of USDA assistance (Bean Female) 2,106 1,016 207% Volume of coffee sold by project beneficiaries 209,819 224,742 93% Volume of dry-beans sold by project beneficiaries 24,275 6,256 388% Capacity Building: Producer Groups/Organizations Number of jobs attributed to USDA assistance 49,931 3,392 1472% Number of private enterprises, producers organizations, community-based organizations (CBOs), and government offices trained to adopt new techniques and technologies as a result of USDA assistance 981 828 118% Inputs: Improve Access to Inputs and Services Number of public-private partnerships formed as a result of USDA assistance 21 23 91% Financial Services: Facilitate Agricultural Lending Number of individuals receiving financial services as a result of USDA assistance (Coffee Male) 9,944 5,664 176% Number of individuals receiving financial services as a result of USDA assistance (Coffee Female) 2,487 1,416 176% Number of individuals receiving financial services as a result of USDA assistance (Bean Male) 3,387 2,640 128% Number of individuals receiving financial services as a result of USDA assistance (Bean Female) 885 660 134% Number of loans disbursed as a result of USDA assistance (Coffee) 68,479 21,240 322% Number of loans disbursed as a result of USDA assistance (Bean) 13,526 6,699 202% Value of loans provided as a result of USDA assistance 61,605,617 24,281,629 254% Market Access: Facilitate Buyer￾Seller Relationships Value of new public and private sector investment leveraged as a result of USDA assistance 29,789,375 4,399,559 677% Value of sales by project beneficiaries (Coffee) 165,094,278 189,981,432 87% Value of sales by project beneficiaries (Bean) 18,792,472 4,306,120 436% 8 MAS+ has developed other custom indicators to measure progress towards objectives, among these are increases in income and productivity. Based on the survey results, gross incomes for coffee producers that went through two training cycles increased by 52.4% over baseline, but considering adverse price factors during the 2018-2019 crop cycle and drought conditions during the 2019-2020 crop cycle, a more illustrative comparison of project benefits is that against the control group at the time of the MTE. At the time of the MTE, gross income for coffee producers was 122% higher than that of the control group 7 . The same can be said for productivity8 , where a modest 3.3% increase over baseline was achieved, however, that of the control group diminished by 42.1% during the same period representing a 49.6 % productivity increase over the control group (productivity for the participant group was slightly higher at the time baseline was collected)9 . For dry-beans producers, gross sales increased by 22.8% over baseline, while those of the control group increased by only 11%. With regards to productivity, drought conditions severely affected the first harvest (primera) resulting in a 4.4% decrease over baseline (but a 47% over that of the control group), for the second harvest (postrera) productivity increased by 16.5%, a 24% increase over that of the control group. 4.3 MAS+ effectiveness This section evaluates progress to date and effectiveness of the seven MAS+ activities in the achievement of the project goals. Activity 1: Training to Facilitate Improved Farmer Productivity, For coffee, MAS+ has designed a training curriculum with a market, agronomy, post-harvest and commercialization orientation. Training is carried out through Farmer Field Schools (FFS) involving small groups of producers (25 on average) around a demonstration plot in which activities taught are demonstrated using a learning-by-doing approach. Topics dealt with at the FFS follow the crop life cycle. Farmers interviewed indicated that the topics and training methodology are responsive to their needs. Survey of participating farmers in the coffee value chain farmers show that recollection of participation in the training and delivery of technical assistance on each of the topics has been modest (considering two training cycles have been provided), ranging between 36% for the Finance and Contracts module to 68% for the Farm Planning module10 . However, MAS+ records show that 65% of producers have completed at least seven of the nine modules and over 79% have completed at least four modules. Discrepancy between survey recall and actual attendance 7 A control group was identified during the baseline study for this group of producers. There were no significant statistical differences between control and participating group in any of the variables compared. 8 Productivity comparisons here are made on the commercial area planted for coffee and do not include areas not in full production (“plantia”) 9 An analysis of probable causes for the decrease in control group yields is carried out in Annex 1. 10 The nine modules considered for the survey were Soil Preparation, Farm Planning, Equipment Maintenance, Harvest, Postharvest, Quality and Certification, Commercialization, Contracts and Finance, and Access to Markets 9 is likely due to the fact that the same module receives a different name every time it is provided. This change of names for the modules is a cause of confusion for both producers and evaluators. For the 2019-2020 coffee crop cycle, producers in the first cohort attending the trainings sold, on average, 5.27MT of wet parchment equivalent11, a 28.2% increase over baseline data. There was a notable increase from baseline on the quantity sold as cherry coffee (332%) and that sold as dry parchment (908%). Productivity for these first cohort farmers increased a modest 3.3% from the baseline. However, during the same period, productivity of the control group had decreased by 42.1% due to climate and market related occurrences, resulting in a difference of 49.6% between both groups (at baseline, participating producers had a slightly higher productivity than that of control group)12. Given the impact of weather and market prices on productivity, perhaps it would be convenient for MAS+ to revisit its productivity increase goals, now set at 30% over baseline, to a comparison against control producers at the time of measurement. MAS+ has been successful in increasing the level of coffee sold as dry parchment which commands a higher price than that sold as cherry or wet parchment. Producers incorporated into MAS+ during the 2019-2020 crop cycle sold very little coffee (1.29 MT WP equivalent) during the 2019-2020 crop cycle compared to those in the first cohort. Also, their yields, one year after receiving training were still much lower than those having already received training over 2 crop cycles (21.2 vs 38.33 qq WP/Ha in their area under full production)13 . While too early to fully evaluate the effect of training or other project activities on this recent cohort, their results contribute to the overall goals of the project regarding indicators other than those related to productivity and training. In the case of dry-beans producers, MAS+ has designed a 4-module training program using the same training methodology as that of coffee. Survey results indicate that attendance in the training modules ranges from 36 to 63%. MAS+ records show that attendance in the full set of modules has been 70%, and 79% to three modules at least. As mentioned for the coffee training, changes in the name of the module every time the course is repeated generates confusion among producers and evaluators. As with coffee, farmers feel that training is focused to their need but would like to add the irrigation training to the curricula as indicated above. Producers were particularly satisfied with the training on the use of seed inoculant at the time of planting Sales of dry-beans for the first cohort, averaged 1.78 MT for those producers reporting sales. While there is little change in the level of production (+9.6%), overall productivity did not decrease in spite of a severe drought during the first part of the 2019 crop cycle (ciclo de primera). For this reason, comparison with baseline data is not useful to evaluate project impacts. However, comparison with a control group of producers during the same crop cycle yields substantial differences in productivity (919 kg/ha vs. 625 kg/Ha for the primera crop and 811 kg/Ha vs. 65 11 To measure coffee production, we standardize all measures to húmedo or wet parchment using conversion factors proposed by IHCAFE (Instituto Hondureño del Café) and TechnoServe that adjust for the humidity and processing in the weight. 12 Cost-cutting techniques promoted by MAS+ helped maintain yields while reducing production costs by 23% and 33% for coffee and dry-beans, respectively. Cost-cutting techniques involve the use of soluble fertilizer, improved weeding and programmed harvesting for coffee producers and the use of inoculum and improved seed for the dry-beans producers. 13 There is a large discrepancy between MAS+ internal data and those of the EMT. A review of MTE data by MAS+ indicates production is 2.26 MT WP Coffee equivalent total production and yields are 50 WP Coffee/ha. This discrepancy needs to be addressed. 10 kg/Ha for the postrera crop) thus reflecting the benefits of the training. Comparison of productivity increase against baseline data under these stressful circumstances is not meaningful. For those producers incorporated during the 2018-2019 crop cycle yields were slightly higher than those in the first group for the first crop cycle (941 kg/Ha) but substantially higher for the second crop cycle (1014.1 kg/Ha) reflecting the different agro climatic conditions between the two groups of departments. MAS+ collects information on the percentage of families covering their yearly projected dry-beans consumption, but this is an unreal indicator, as, from the survey, most of the dry-beans produced by participants are sold. MAS+ should consider reporting actual volume of dry-beans produced for home consumption and assigning a value to this production as it impacts on the family income. Adoption of topics taught during the reflect the effectiveness of training. In spite of relatively modestly recall of receiving technical assistance and training. MAS+ has succeeded in coffee producers implementing the key technologies aimed at improving quality, productivity and environmental stewardship.14 To date, some 43% of the first cohort producers, and close to 40% of those in the second cohort. producers apply between 4 and 5 of these techniques (mainly fertilization, integrated pest management, weed control and tissue management). For the control groups, only 31% of producers apply between 4 and 5 these techniques. The above notwithstanding, there still remain a large group of producers (25%) implementing only two or three of the practices taught. Given progress to date, MAS+ would do well in narrowing training to the key practices producers should implement and provide a better description of what it means to implement these good agricultural practices (i.e. the use of organic fertilizer vis a vis use of chemical fertilizers). Another topic mentioned by dry-beans producers is that of marketing and commercialization, as currently MAS+ is taking over much of these activities. For the dry-beans producers, seven techniques15 were considered as key technologies to improve productivity and quality. Over 43 of dry-beans producers in the first cohort are applying between 4 and 5 of these practices and 25% applying 6 of these practices. Again, MAS+ would do well in narrowing and describing these practices to reflect those that are key for productivity and quality. Activity 2: Capacity of PO Building: POs and Cooperatives MAS+ builds the capacity of POs to deliver commercial and financial (through cajas rurales) services to participants in the project. To date training and strengthening of 1,324 different types of POs ranging from small POs created around FFS to cajas rurales, women associations, and indigenous groups are provided. The process of organizational strengthening is clear for the cajas rurales as MAS+ partner FUNDER carries out a standardized strategy to strengthen and move them from inception to self-sustainability. FUNDER is currently working with 292 cajas rurales (119% over LoP goal), of which 255 have completed 60% or less of the FUNDER standards to be considered as self-sustainable (i.e., not requiring further accompaniment), and 58 POs. Besides the training directly aimed at creating and organizing a caja rural from inception to self-sustainability, FUNDER provides direct training to all 350 POs on governance and organizational management, and commercialization and financing. 14 Technologies include: postharvest management, fertilization, pest and disease control, tissue management, selective harvesting, use of organic fertilizers, shade management, irrigation and record keeping. 15 Use of inoculant, chemical treatment of seeds to control pests and diseases, weed control, correct harvesting, postharvest, pre-drying and quality selection. 11 To complement training provided by FUNDER, MAS+ includes training specifically designed for coffee and dry-beans POs. The objective of these trainings is to improve their efficiency in the in commercialization through formal, high paying markets; channeling the AIMC financing to members; and in the case of 350 POs strengthening PO governance mechanisms. At the end of the project, the increased capacity of the PO in the delivery of these services will ensure its sustainability. By the timeframe of the MTE, the progress towards sustainability of POs is that shown in Table 2. Table 2. Level of development of Producer Organizations. In addition to training, MAS+ has worked with POs on the delivery of their commercialization services by strengthening their commercialization committees so that they can deliver quality control services, take on the negotiation with coffee exporters and bean buyers and inform membership of earnings and price differentials obtained. For the 2019-2020 crop cycle, 977 POs commercialized 15,600 MT of coffee and 7,700 MT of dry-beans thanks to these services. POs also promote and coordinate loans provided to members though the AIMC mechanism. For the dry-beans value chain MAS+ establishes or strengthens community seed banks (CSB) producing certified dry bean varietal seed that will improve yields, producer profitability and drought tolerance. While technically the CSB strategy is included under Activity 3: Develop Input, Service, and Equipment Providers, the organizational aspects and potential of a CSB go beyond that of input provision and warrant their inclusion in this Activity’s discussion. A CSB involves 25 producers around a qualified certified seed grower belonging to the group. During the LoP, MAS+ was to establish 150 CSBs through its partnership with MSU16. To date, 183 of these CSBs have been established, surpassing the LoP goal by 22%. Additionally, the owner of the CSB can act as a distributor, not only of seed but of inoculant and other required inputs. According to the activity description, MAS+ should prioritize the inclusion of women and youth in the development of these services, this is not clear from the data collected by MAS+ or this MTE Establishment of seed banks serves both to strengthen producer organization as well as to ensure availability of certified seed for producers in the group. However, interviewed producers indicate they do not belong to an established CSB but rather individual producers have been trained as seed producers and those have been considered as belonging to a CSB. This creates problems when seed supply is scarce as certified seed growers, even though, in theory, belong to a CSB, 16 MSU leads the Feed the Future Dry Grain Pulses Collaborative Research Support Program. Stage of development Stage Definition Number of register POs Developed PO exercises good governance and provides efficient services to its members. 233 In-development PO provides services to their members, and are positioned to improve management systems. 371 Start-ups PO is in the process of completing basic legal documentation; will require additional support in governance, management, and service provision. 448 Total 1,052 Source: InfoMAS and FUNDER records. 12 prefer to sell outside the CSB and obtain higher prices. MAS+ should revisit their strategy on the CSBs moving forward. Activity 3: Develop Input, Service, and Equipment Providers Through this activity, MAS+ should support local and regional agricultural input supply firms and market actors to stock improved inputs, equipment and other services to improve yields and quality of coffee and dry-bean production. Additionally, MAS+ will train POs to comply with sanitary and phytosanitary regulations and selection of appropriate inputs as well as facilitating access to sustainably managed water, inputs and equipment. For coffee producers, MAS+ has designed an effective strategy to increase access to quality inputs specifically prepared to service their needs. As part of the AIMC, financing for fertilizer, to be paid at the end of the crop cycle, is included. At the beginning of the crop cycle, fertilizer is delivered to each PO to be distributed among its membership. Fertilizers are formulated by large distributors (DISAGRO and CADELGA) according to soil analyses carried by MAS+ or the coffee exporter signing the AIMC. Under the AIMC, the exporter also finances other inputs according to the POs’ requests. For the 2019-2020 coffee crop cycle 3,300 MT of improved fertilizers were provided to 628 coffee POs. Additionally, coffee exporters and buyers have financed or donated 628 solar driers which help improve the price and quality received by producers. For dry-beans, MAS+ has reached agreements with input distributors (CADELGA, Bayer DISAGRO, Syngenta and others) to finance fertilizer and agrochemical packages specific for dry￾beans production. Some 3,000 MAS+ producers have benefitted from this mechanism. Lack of organic input providers has been addressed by training over 7,000 MAS+ producers in the preparation and use of organic herbicides and 6,250 in the preparation of organic fungicides to combat leaf rust and American leaf spot (Ojo de Gallo). Through training on cupping and specialized postharvest techniques, MAS+ has created a critical mass of service providers in their zone of influence). These specialized service providers are, for the most part young men and women. MAS+ has also provided training and technical the provision of certified seed (3,209 qq in 2018 and 2019) and introduced two drought resistant dry￾beans varieties. Regardless of the need to evaluate results concerning their associativity aspects, the CSB concept has provided excellent results regarding availability of quality seed to the sector as whole by driving a change of attitude regarding the importance of seed quality, has improved the sector’s seed production capability, and favored inter institutional coordination in support of seed production, dissemination and availability. However, organization of dry-beans producers is still weak and may prove unsuccessful in the long run if adequate demand incentives are not created in the remaining project implementation period. Availability of quality seed has resulted in over 70% of MAS+ dry-beans producers in the first cohort using certified seed (as opposed to 41% in the control group). Use of certified seed by dry-beans producers in the second cohort remains low, with 50% of producers still utilizing non-certified seed varieties. Activity 4: Training of Agricultural Extension Agents/Services 13 To ensure sustainability beyond MAS+ LoP, the project should support public and private sector organizations to develop their own technical assistance services in support of activity 1. This activity is closely linked to activity 3 above. MAS+ has trained technicians working with coffee exporters and DICTA on the FFS methodology and curricula and on CSBs respectively. Beginning 2020, some coffee exporters are sharing the costs of training and technical assistance for producers under their AIMCs. MAS+ still has sufficient time to strengthen this activity during the remaining project time. Activity 5: Facilitate Agricultural and SME Lending Through this activity, MAS+ facilitates agricultural lending from microfinance institutions, private banks, input suppliers, and other market system actors to producers and producer organizations. To carry out this activity, MAS+ has implemented several strategies, the establishment of cajas rurales through its partner FUNDER (252 created to date), a revolving fund partnership (MAS+, coffee exporters, Fundación José María Covello) which is used by exporters to finance POs under the AIMCs mechanism, PRONTO-PAGO factoring for dry-beans producers and direct loans by coffee exporters to POs for inputs or infrastructure. This highly innovative strategy has allowed for more direct linkages between value chain actors and financial inclusion of small producers. Other mechanisms that MAS+ has implemented are the dry-beans guarantee fund (FOGAFRIJOL), technology packages for dry-beans by large input suppliers (Bayer, Syngenta, DISAGRO and CADELGA). Regardless of the mechanisms introduced for the dry-beans sector, there are still many producers without access to finance in the value chain; during the remainder of the project, MAS+ should consider creating a similar model to the AIMC for the dry-beans value chain or strengthening the cajas rurales system to provide at least part of the required financing. To date, the MAS+ access to finance approach has resulted in $38.1 MM in financing mobilized though 46,754 credit operations. The bulk of this financing (90%) is provided by coffee exporters through the revolving fund, with FUNDER providing 9.5% of loans and only 0.5% from other sources. MAS+ should consider recording loans by cajas rurales since, despite the low individual volumes, they make up a source of funds not tied to coffee or bean production and may constitute an important source of resources in between harvests. Average credit was $642 for coffee producers and $306 for dry-beans producers. The level of these credits reflects the size of producers and the difficulty on the part of formal financial institutions to manage these micro￾credits at reasonable costs. It is not clear that MAS+ or its partner FUNDER have provided financial training to project participants other than those directly involved in the cajas rurales. Activity 6: Develop Buyer-Seller Relationships For this activity MAS+ would develop domestic and international market opportunities for coffee and dry-bean producers and producer organizations by engaging both buyer and seller in creating contracts that stipulate the high-quality standards that coffee and bean producers must meet in order to access these markets. MAS+ should also engage buyer and seller parties to negotiate mutually beneficial marketing contracts that include terms for financing, technical assistance, and preferential prices for premium coffee. For this activity, at a national level, MAS+ should coordinate efforts with IHCAFE and the private sector to elevate international recognition of 14 Honduran coffee quality. TechnoServe will also identify and encourage international buyers to contribute resources to finance components related to increased production sustainability. For the coffee sector, MAS+ has been very successful in the implementation of this activity. By organizing producers into POs and cajas rurales that engage into direct contracts with coffee exporters through an All-Inclusive Marketing Contract (AIMC). The contract details, besides quantities to be sold, quality specifications that must be met. AIMCs are implemented, on the part of the producer through a PO or a caja rural. After one year of fulfilling their contract, the caja rural or PO has access to credit from the exporter provided as inputs at the beginning of the crop cycle and as cash during the harvest, both for labor at the farm level and as aggregation funds for the PO or caja rural. Through this innovative mechanism, financing, as described previously, has reached producers that would otherwise not have been able to access formal financing. The AIMCs also result in higher prices paid to the producer; according to MAS+ records17 , prices for coffee producers included in the AIMCs are 40% over those paid by intermediaries; this includes volume and quality bonuses. For dry-beans, the mechanism is a fixed-base indexed price contract with IHMA that has resulted in a price increase of 33% for the primera cycle and of 28% for the postrera cycle in the 2019- 2020 crop cycle. Currently MAS+ works with 19 coffee buyers including exporters coops and cajas rurales and 6 dry-bean buyers (IHMA, exporters and wholesalers) through which 628 AIMCs were been signed for 10,600 MT of dry parchment coffee equivalents and 712 sales contracts for 10,100 of dry￾beans for the 2019-2020 crop cycle. All of these operations result in loans or convertible instruments (through factoring) from the buyer. Coffee farmers in the second cohort, though they have already signed AIMCs, were not eligible for loans from the exporters for the period previous to the MTE. Even though producers may have an AIMC, they would still sell to an intermediary at certain times during the harvest when their coffee does not meet quality standards required by the exporter. Thus, despite the AIMCs, 96.5% of MAS+ producers still sell some of their product through intermediaries. Finally, in spite of the low coffee prices during the 2018-2019 crop cycle ($107.09 per qq), MAS+ coffee farmers were able to upkeep their farms for the next crop cycle, maintaining their level of production and thus their income. During the 2019-2020 crop cycle (MTE period), the average MAS+ producer in the first cohort received 122% more income from coffee sales than that of the control group with similar farm characteristics ($5,249.46 vs. 2,368.29) and their yields were 44.7% higher. This attests to the effectiveness of the MAS+ program on ensuring the resiliency of coffee producers to market and weather shocks. Regarding the recognition of Honduran coffee quality, little has been done on a continuous basis, however, MAS+ has involved IHCAFE on some of its activities during the Specialty Coffee Association of America shows and other trade fairs in Europe and Asia in which tastings of high￾quality coffees have been carried out. 17 MAS+ has a detailed close-out system for sales that measures price differential between project sponsored sales and those of intermediaries for each sale that a participant makes to a coffee exporter, IHMA or another producer. The system is monitored by the CT to ensure real-time recording. For the 2019-2020 crop season price increase over that of intermediaries averaged 40% for the 150,000 qq of coffee and 90,000 qq of dry-beans recorded. 15 Activity 7: Building Government Institutions capacity and support Policy Regulatory Framework improvements. To implement this activity, MAS+ should help build the Government of Honduras’s capacity to strengthen input supply markets and services to producers. MAS+ will provide technical assistance to the Honduran Ministry of Agriculture (SAG) to validate bean varieties that can help farmers improve yields, and to help decentralize access to the rhizobium inoculant it produces. Limited resources available as a result of the monetization/budget shortfall have limited MAS+’ capacity to improve input supply or service capacities of GOH or IHCAFE for the coffee market. However, MAS+, through the CSBs, has worked extensively with the GOH in the production of certified seed with SENASA, which is then sold to DICTA, to distribute nationally (instead of going to the CBS members). CSBs supplied 75% of GOH’s certified seed demand and contributed to 65% of IHMA’s strategic dry-beans reserve. Additionally, MAS+ has contributed to field level validation of two new dry-bean varieties resistant to drought and a variety with high productivity and market demand. Also, dry-bean farmers are now familiar with the use of inoculant as a way to increase productivity. Because there are no indicators established or collected for this activity, it is uncertain of how much MAS+ truly accomplished on this front despite active participation in numerous platforms and government activities. Equally, under this activity, MAS+ should collaborate with the Permanent Emergency Contingency Committee (COPECO) and other actors to train targeted local and national government personnel to strengthen areas that enable smallholder producers to respond more effectively to climatic crises, including drought and excess rain. While there is no indication that this has occurred, MAS+, through a subagreement with the International Development Enterprises (iDE) that has since been phased out, has worked with 25 municipalities in the development of municipal environmental plans. These plans are coordinated with MiAmbiente, the national level organization overseeing these plans. The plans include an inventory of natural resources, environmental control, tourism, environmental training, risk management and basic sanitary infrastructure. iDE’s work has fallen short of expectations as, to-date, only 10 municipalities (of the 25 projected) have initiated the process; 5 of these have already signed an agreement with the project for the preparation of the plan while the rest have still to sign these agreements. The preparation and approval of these plans is a slow process, and each municipality faces a different set of challenges to prepare and let alone implement these. MAS+ should consider, in view of the limited progress and recent changes in resource allocation within the municipalities due to COVID-19, natural disasters and the upcoming elections, the revision of this indicator, focusing iDE’s efforts on environmental stewardship training for MAS+ CTs and producer leaders and in adapting MAS+ curricula to incorporate an environmental lens for the general training. Cross cutting themes a. Improve the participation of women and youth. Other than disaggregated reporting of indicators referring to participation of women and youth and access to finance for women, MAS+ has not set indicator goals that reflect a clear strategy to improve the participation of women and youth. In spite of incentives being provided to CTs to increase women participation, participation is far below that of TechnoServe’s 40% target for women participation. 16 For the MTE, a set of questions to evaluate women access to resources at the household level were prepared. While there is no basis for comparison as these questions were not included in the baseline study, the answers provide us with a picture of the status of women regarding use of and disposition of resources within MAS+ participants in both value chains. In the coffee value chain, decisions on production, marketing and income disposal are taken by men in over 60% of cases; women make these decisions only in some 3% of cases and joint decision-making ranges between 25% for use of resources to 35% for disposal of income. For the dry dry-beans value chain, participation of women is slightly higher, notably on decisions about income disposal, where 38% of those interviewed indicated that the decision is taken jointly. There is clearly a long way to go on these issues but, without a stated strategy and dedicated resources to this activity, no big changes in the status quo of women within the coffee and dry-beans value chain will be achieved. There is no clear access to finance indicators for women other than those in the FUNDER agreement where the goal is set at 35% of loans going to women. To date, the proportion of loans going to women is close to 30% for the dry-beans value chain and slightly lower for the coffee value chain. b. Avoiding child labor Coffee and dry-beans farming by the SHF is a family activity. SHF use family labor either directly or indirectly, women, youth and children actively participate in harvest and postharvest activities in both value chains MAS+ participates. Over 9,000 participant coffee growers have signed contracts that include provisions against child labor and child labor is monitored by the project as well as messages against that practice included during training. Nevertheless, some interviewed CTs were in agreement that school age children can participate in these crops if it does not interfere with their education and the chores, they are assigned to are not strenuous or dangerous. While this reflects the reality of the sectors in which MAS+ operates, the project should double its efforts on training its personnel on child-sensitive indicators to ensure that children working alongside their parents, while contributing to the household income, is not to the detriment of their personal development. c. Migration prevention From the field level interviews, 19% of households indicated that that someone from their home had migrated, this is a small decrease from the baseline data (19.4%) but it basically indicates that drivers of migration (i.e., access to better economic opportunities) remain in spite of the project advances. Of those coffee households reporting migration, 57% did so internally, 34% to the US and the rest to other countries. For the dry-beans value chain, migration figures are slightly higher having increased by 3 percentage points over baseline, however, internal migration represents 70% of migration, 25% to the US and the rest to other countries. The overwhelming majority of migrants in the dry-beans value chain are men (63%) while in the coffee value chain distribution is more even (58% men and 42% women). As there are many other causes for migration (internal or external), it is unclear what MAS+ can do at the household level, as, for instance, there are no labor opportunities for better educated youth in the coffee and dry-beans communities (a logical consequence of higher household income from their crops) forcing them to migrate. However, MAS+ can focus on ensuring that 17 intention of coffee and dry-bean farmers (not household members) to migrate diminishes as they increase their incomes through professionalization, access to markets and access to finance18 . This can be done through annual surveys to evaluate intention on the part of registered coffee and dry-beans farmers to migrate. d. Environmental stewardship, climate change In spite of having 1,150 coffee producers Rain Forest Alliance certified and 8,600 coffee producers having signed marketing agreements with environmental protection clauses, dissemination of drought resistant dry-bean varieties and use of inoculum, and training on irrigation techniques, the project has not yet applied a strategy to monitor implementation of environmentally friendly and CSA practices taught in the FFSs. Thus, at this time incorporation of environmentally friendly or CSA practices into the training curricula appears to be an ad-hoc rather than an intentional practice as no indicators have been set from the onset of the project. For this reason, practices, other than those related to varietal and use of inoculum cannot be causally tied to the project. For the remaining of the project, MAS+ has decided to incorporate the monitoring of these practices and is developing the indicator descriptions to share with the CTs in charge of collecting this information. MAS+ partner organization iDE is in charge of disseminating and promoting the adoption of CSA practices among producers and other actors in the value chains. Additionally, as mentioned, iDE supports municipalities in developing their environmental and water management plans. By MTE, iDE had trained 1,103 participants (not clear if they are all different) in ten different topics. It also is not clear whether or not these topics are part of a concerted training curricula or respond to a particular training plan. Of those ten topics, the bulk of trainings has been delivered on introduction to CSA (459 participants) and adapting farm planning to climate (434 participants). Additionally, 100 POs have been strengthened on CSA but it is not clear what this strengthening entails. One hundred nineteen MAS+ and FUNDER technicians (114 and 5 respectively) have been trained on one or more of the ten topics in which producers were trained, however, no follow up on whether these topics are being incorporated into the training sessions delivered to the 34,512 MAS+ participants. It is clear from the interviews that producers are aware of climate change effects and measures to mitigate these at their farms, as shown by their response to adverse climate variations in the period between the baseline study and the MTE. However, until now, there has been no concerted effort on the part of MAS+ to collect information on environmentally friendly and climate smart practices being implemented by producers. For instance, although trained in the conversion of coffee pulp waste to organic fertilizer, producers shun this practice, and there is no evidence that training included the treatment of honey water resulting from the wet-milling process or safe use of pesticides. These are among the environmentally friendly practices that will be monitored by project CTs during the second half of the project. 18 In a 2018 study about intent to migrate, MAS participants showed a 38% less intention to migrate, than that of a control group. MAS+ would benefit from following that approach as opposed to a more holistic approach that is beyond the scope of the project 18 4.4 Monitoring, Evaluation and Learning MAS+ monitoring rests on its INFOMAS information system. INFOMAS allows real time collection and processing of data to provide the necessary tools for managers to evaluate up-to￾date progress make necessary adjustments where required. Each participant has his/her own QR code and farms are georeferenced. This allows for easy follow-up of activities at the farm-level, presentation of data using maps, and, if required at a later date, set-up a traceability system. Data collection is integrated throughout all levels of the project in real time; thus, the project leadership always has current information at their fingertips. MAS+ has managed to integrate CTs, project middle management, partners and allies into their reporting system which is then processed into the INFOMAS information system. An important feature of INFOMAS is that partners are able to enter data on their activities through a dedicated interphase to INFOMAS. Their data is then processed into the different reports required by themselves or MAS+ management Data is fed to INFOMAS using mobile and web applications and information is processed into reports as required by the user. An important feature of INFOMAS is that required data is available to partners such as coffee exporters to evaluate progress in the completion on the AIMCs. Data is coded so that confidentiality and safety of information is assured. The process of generating information on the INFOMAS system is presented in Figure 2 below. Figure 2. INFOMAS data collection and information generation platform. The MEL component of a project is traditionally conceptualized in the initial MEL Plan, which is considered a living document meant to evolve and change along with the project natural evolution. In the case of the case of MAS+, given the progress to date, with many indicators far surpassing LoP goals, this plan requires changes for some of the indicators and activities. Therefore, among the recommendations that will be put forth at the end of this report, a detailed and strategic revision of the MEL plan as soon as a possible is included. A thorough revision of the MEL plan will open the opportunity to make changes to the project logic regarding indicators that may no longer be considered valid, add/remove/update indicators, review assumptions and resubmit targets, which seem to be unrealistic to the current realities in Honduras and project progress to date. 19 4.4 Communications Communications with participants, given the current limitations caused by the COVID-19 pandemic, plays an important role in the implementation of MAS+. Even prior to the restrictions caused by the pandemic, MAS+ has been successful in implementing a solid technical communications strategy. The communication strategy MAS+ is implementing rests on three pillars: messaging, social media and virtual technical assistance. Messaging using SMS, is used to provide technical and market information to POs, individual participants, and PO leadership; the initial goal of reaching 14,000 participants has been surpassed by 987 at mid-term. Information reaches 11,390 coffee producers and 3,597 dry-beans producers. Seventy-nine percent (79%) of registered recipients are male. Information is also provided through social media through WhatsApp messaging (9,500 people reached) and 95 technical publications on the Facebook platform (208,000 visits). Remote technical assistance is provided via the Zoom and WhatsApp platforms for those producers that cannot be reached in-person. For this purpose, MAS+ has designed training sessions not lasting more than one hour focusing on specific agronomic recommendations according to the crop cycle. Training is later accompanied by short videos sent to the attending producers. For the commercialization training, CTs assemble a small group of producers, and using their device, participate in a 1.5-hour session where the AIMC model is explained by MAS+ and exporters’ senior personnel. Through this strategy, 20% of producers who could not be reached are covered with training and technical assistance services. MAS+ has generated substantive learning both technically and operationally. However, none of this is yet documented so that it can be scaled-up by actors other than TechnoServe itself. It is highly recommended that these learnings be documented and shared with the donors, GOH, allies and the general public. 4.5 Efficiency MAS+ has been shown to be a highly efficient project, despite a substantial budget reduction due to much lower revenue than that expected for the second monetization, not only successfully achieving but also surpassing key LoP indicators this early into the project. Results of MAS+ to date have also provided high returns for USDA’s investments; for every dollar invested by USDA there has been a $2,87 average increase in farmer income, $7.24 available in financing for POs and a $9.39 average increase in total sales. Very few development projects can boast these levels of return on investment. 4.6 Sustainability It is clear that the strategy of organizing producers in to POs and cajas rurales to subsequently link them to the market via AIMCs ensures sustainability of the commercial process. Furthermore, recent agreements with coffee exporters to provide technical assistance, however minimal, also ensures the continuous professionalization of the SHF coffee producer currently in MAS+. The same however cannot be said for the dry-beans producers, while there is already a strong link to IHMA as the main market player, any change in the GOH’s policy regarding dry-beans can wipe 20 out the gains to date. In the absence of a strong player coordinating the link with DICTA and SENASA for the production of certified seed and distribution of Rhizobium inoculant, MAS+ should ensure POs and CSBs are sufficiently strengthened and empowered to carry out that process by themselves at the end of the project. On the issue of environmental sustainability, while substantial progress that cannot be directly ascribed to the project (due to lack of indicators) has been made, there are no clear indications, other than the limited work by iDE, that there is a commitment on the part of MAS+ on the implementation of environmentally friendly and CSA practices at the farm level19. During the interviews, farmers receiving fertilizer from coffee exporters indicated they preferred to let the waste from wet milling decompose naturally creating a source of contamination rather than use it to prepare organic fertilizer. To implement the strategy already designed for the remainder of the project, MAS+ would do well in assigning this role to a high-level technical person to ensure that all training includes environmental and CSA aspects and that CTs are responsible for ensuring these practices are implemented with equal emphasis as those that increase productivity. V. CONCLUSIONS Criteria Review questions Findings Relevance How does the project align to beneficiaries’ needs – both men’s and women’s - and responds to economic and social context in the implementation zone? How relevant is the Theory of Change (ToC) to the current contextual dynamics of the project? MAS+ aligns with USDA and Government of Honduras’ priorities and to the needs of coffee producers in the nine departments of influence. In particular, MAS+ aligns with the GFSS and with the Honduras 2018-2022 Government Strategic Plan for Agriculture Based on surveys and interviews with the project's allies, MAS+ aligns well he needs of coffee and dry￾bean producers, responding to their needs regarding technical assistance, access to market and access to finance, and taking into account their socioeconomic conditions. Despite changes in the operating conditions for MAS+ during the period, the levers of change and activities still hold true and need to suffer no changes going forward. However, MAS+ would do well in reviewing its accomplishments to date on access to services (including training) for climate change and resiliency, and policy enabling environment to either modify its activities in support of these changes or set more attainable goals given the remaining project life. 19 During the remainder of the project, MAS+ Community Trainers will monitor the implementation of key environmental and CSA practices at the farm level. 21 Effectiveness How do project strategies have contributed to achieving its goals and/or objectives? . Regarding activity 1: Training to Facilitate Improved Farmer Productivity Modest recall of participation level in trainings: after two cycle crops first cohort producers indicate their participation in training varies from 38% to 67% in each of the nine training modules. However, MAS+ data indicates that participation is much higher. For FY2020, this indicator has been negatively affected by the limitations imposed on field activities caused by the COVID-19 pandemic. Producers are satisfied with trainings. 99% of surveyed producers expressed that they are satisfied with the training they have received and the topics covered. Dry-beans producers would like more emphasis on training on irrigation techniques. . Producers are adopting key GAP practices to increase yields; 54% of first cohort coffee producers have adopted more than 5 GAP practices. However, despite the training on organic fertilization, some producers shunned their preparation use in favor of chemical fertilizers provided by exporters under the AIMCs. Despite low prices during the 2018-2019 crop cycle and drought conditions and restrictions due to COVID￾19, coffee producers participating in the first cohort were able to maintain their productivity and sales volume at baseline levels. Compared to a group of control farmers, participants show 49.6 higher productivity and sold 136% more coffee. Dry-bean farmers having participated in the MAS+ project for two years, showed substantial differences with regards to a similar group of producers in the control group. For the first crop in the cycle, following severe drought conditions, participants had 47% higher productivity. For the second crop in the cycle, weather conditions improved yet participants had 23.9% higher productivity than that of the control group. With respect to activity 2: Capacity of PO Building: POs and Cooperatives, MAS+ is working with 1,324 different types of POs ranging from POs created around a FFS to cajas rurales and women associations. The strengthening of cajas rurales follows a defined path line which is not the case for the other POs. MAS+ partner FUNDER works with 264 cajas Rurales at different stages of development. To date, 255 of these have achieved 22 60% or less of the standards developed by FUNDER to be considered a self-sustaining organization. For the dry-beans participants, MAS+ organizes producers around CSBs. While the goal of 150 CSBs has been surpassed, interviewed producers indicate that they do not belong to a CBS but rather individual producers have been trained in seed production and are considered a CSB. In regards to Activity 3: Develop Input, Service, and Equipment Providers MAS+ has focused on the CSB concept for the dry￾beans value chain. This has had excellent results in driving a change of attitude regarding the importance of seed quality and has improved the sector’s seed production capability, favored inter institutional coordination and availability of quality seed. While no direct support to agricultural supply firms has been provided by MAS+, improved region-specific fertilizer formulations for coffee and dry-beans, based on soil analysis have been demanded and supplied by these firms. The volume demanded has allowed them to economically prepare region-specific fertilizer formulations that they have been able to market to non-participant farmers in the different coffee regions. With the help of MAS+, input suppliers have been able to prepare a fertilizer and agrochemical package for dry-beans producers that is also available to non￾participant dry-beans producers. On Activity 4: Training of Agricultural Extension Agents/Services, While MAS+ has trained agricultural extension agents and services (cuppers, wet mill operators, etc.), this aspect was not considered in the MTE. Yet, it is clear that with over 2 years remaining in the project, this activity can be strengthened and appropriate indicators established to measure progress. On Activity 5: Facilitate Agricultural and SME Lending MAS+ has implemented a highly innovate strategy aligning buyers, implementing factoring for dry-beans, establishing a revolving fund and cajas rurales, this has resulted in $38.1 MM financed through 46,754 credit operations. The bulk of this financing (90%) is provided by coffee exporters through the revolving fund mechanism. Average credit amount was $642 for coffee producers and $306 for dry-beans producers which reflects the difficulty for these producers to participate in formal financial markets at reasonable costs. 23 With regards to Activity 6: Develop Buyer-Seller Relationships Little has been done to elevate international recognition of Honduran coffee jointly with IHCAFE. MAS+ has participated in two SCA fairs conducting cupping sessions with international buyers. Despite the good results at these cuppings in terms of specialty coffee sales, the perception of uneven quality of Honduran coffee still persists. On the internal front, MAS+ has been very successful by organizing coffee producers into POs and cajas rurales and linking them to exporters through AIMCs. This innovative mechanism has resulted in 475 AIMCs being signed and 10,600 MT of dry parchment coffee being sold directly to exporters at prices 40% above those of intermediaries in the 2019-2020 crop cycle. Despite the success of AIMCs, 96.5% of MAS+ producers sell some of their production to intermediaries and only 17% of those interviewed are selling only to exporters, implying that there are still many producers still not participating in the mechanism. On Activity 7: Building Government Institutions capacity and support Policy Regulatory Framework improvements MAS+ has worked extensively with the GOH in the production of certified seed with SENASA which is then sold to DICTA and distributed nationally through the “Bono Solidario”. There are no established indicators to ascertain how much MAS+ has truly accomplished on this front. There is no evidence that MAS+ has contributed with IHCAFE of the GOH on improving policy framework. MAS+ has worked, through its partner iDE on developing municipal level environmental and water management plans with 25 municipalities in project areas. Progress to date is limited and success doubtful given the restrictions caused by the COVID￾19 pandemic, resource allocation following severe natural disasters and upcoming elections. MAS+ should consider refocusing resources currently assigned to iDE, on training CTs in environmentally friendly practices, ensuring incorporation of an environmental lens into the training curricula, and overseeing implementation of these activities at the producer level. On the cross-cutting themes Improve participation of women and youth – Other than reporting disaggregated indicators reporting to the participation of women and youth, and access to finance by women, there are no specific indicators that reflect a clear strategy to achieve this goal. 24 Avoiding child labor – MAS+ should re-think the training provided to CTs and participants to include more child-friendly practices and emphasize children’s rights to education and a healthy childhood. Migration prevention – There are many factors outside of MAS+ control that impact on migration. Rather than focusing on migration as a whole, MAS+ should probably focus on reducing the intention to migrate among registered participants Environmental stewardship, climate change – While interviewed producers are aware of climate change effects and measures to mitigate these there are no indicators on implementation of these measures at the farm level. MAS+ should speed up the implementation of its strategy for this component during the remainder of the project by having a dedicated high-level staff member in charge of this component. Monitoring, evaluation and learning – MAS+ has developed a sophisticated information system: INFOMAS, that allows management to evaluate up-to￾date information on indicators and project progress. INFOMAS data is also available to partners such as coffee exporters. MAS+ MEL plan should be updated to reflect the project’s current advance and circumstances and make the necessary changes in targets, assumptions and indicators. Communications – MAS+ has implemented an aggressive and successful IT based communications strategy to provide producers with technical and market information, reaching 14,987 coffee and bean producers through an SMS platform and 9,500 through WhatsApp groups. The COVID-19 pandemic has not stopped the provision of technical assistance and training to the 20% of producers that could not be reached due to various restrictions. For that purpose, MAS+ has designed a remote training strategy focusing on key messages during a one-hour online training and following up with short videos on the topics taught. Efficiency To date, has the team efficiently managed project resources? Despite budget limitations stemming from lower than anticipated revenues from the second monetization, MAS+ has been able to efficiently manage its resources to achieve or surpass the key LoP indicator goals. 25 Are project activities carried out in the most cost-efficient and timely manner? MAS+ as shown highly efficient use of resources. For every dollar invested by USDA there has been a $2.87 average increase in farmer income, $7.24 available in financing for POs and a$9.39 average increase in total sales Sustainability What evidence does exist to predict if benefits from the project will be maintained over time, once CAFE has ended? The Evaluation Team identified the following project features that have the chance to drive sustainability beyond the MAS+ LoP: Adoption of sustainable agricultural practices: strengthening technical capacities of beneficiaries, focusing on improving yields, selling more dry coffee, improving quality and use of inoculum and certified seed by dry-beans producers, is expected to create the right incentives for small coffee producers to increase incomes from coffee sales sustainably. Benefits of associativity: Associativity, in particular through cajas rurales, ensures that farmers can more easily negotiate with buyers and vice versa, creating a sustainable win-win relationship Access to finance: MAS+ AIMC is an innovative approach to ensure SHF have access to finance from coffee exporters. This market-led solution to access finance is highly sustainable in the long run. Market opportunities: By linking coffee producers directly to exporters through the AIMCs, producers are assured of the best and fair prices for their quality coffee. Sustainability in the dry-bean sector: Through the introduction of improved seed varieties and inoculum use, yields have increased to profitable levels, linkages to IHMA for direct sales contracts and the introduction of factoring under these contracts, provides the path for sustainability in the sector. Resiliency: Despite adverse price and weather conditions over the past two growing seasons, and mayor disruptions stemming from the COVID-19 pandemic. MAS+ participants in both value chains have maintained their production and productivity levels. Other How has the COVID-19 pandemic affected MAS+ activities? While initially disrupting field level activities, in response to COVID-19, MAS+ adapted operations to support farmers through a combination of remote and socially distanced training modalities, including small group training, farm visits (based on crop cycle and participant needs), and phone-based follow-ups. This strategy has proved successful in ensuring no loss of production or productivity levels for producers in both value chains 26 What are the cases of innovation and entrepreneurship that have arisen thanks to the project intervention? The main innovation introduced by MAS+ has been in the introduction of the AIMCs linking POs and cajas rurales directly to coffee exporters. A second innovation having to do with the above has been the establishment of a revolving fund including MAS+, coffee exporters and the formal financial sector to allow those POs signing an AIMC access inputs and funds at reasonable costs. Innovations in the dry-bean sector have been the establishment of community seed banks and use of inoculum at the time of planting. These two innovations have increased yields, incomes and resiliency of dry-bean producers participating in the project. VI. Lessons learned and recommendations. 6.1 Lessons Learned  Marketing practices developed for the coffee sector by MAS+ have evolved to the point where little intervention for the higher-level marketing activities (coordination, linkages to buyers, and finance institutions) on the part of project staff are required. This is no so for the dry-beans sector as major coordination with the GOH, the largest buyer is still required of MAS+ staff. Regardless the differences in the markets, MAS+ should consider directing more efforts to influence government policies regarding the dry-beans market and financing.  MAS+ has improved the livelihoods of participating coffee and dry-bean farmers focusing on a key level for migration: lack of profitable economic opportunities. This has great impact on producers themselves but does not necessarily extend to other household members, particularly the young, more educated - as a result of increased incomes – members. While not a direct project responsibility, migration affects all sectors of Honduran economy MAS+ and its prevention must be considered by every development project. Currently MAS+ evaluates through periodic surveys, including the MTE, the percentage of households where a family member has migrated. Rather than at the household level, MAS + should consider focusing its migration prevention efforts on those directly involved with the project and seek alliances with other specialized projects dealing with migration prevention in its zone of influence to address this situation.  MAS+ should define more frequent mechanisms to evaluate the financial services and products being promoted with other financial actors (other than partner actors) to determine their penetration, demand and appropriateness to participant’s conditions and make the necessary adjustments to improve their use by participants. 6.2 Recommendations 27 General  The CSB concept is an excellent strategy to strengthen associativity among dry-seed producers and ensure availability of certified seed and inputs (inoculum primarily) to its members. Additionally, the CSB members can more readily establish marketing relationships with buyers – other than government – as is being done in the coffee sector.  Given the success of the AIMCs, it could be of interest to PO leadership to be trained intensively in those aspects that will enable them to successfully lead their groups after the project ends. Possible topics for this training are: (1) importance of quality standards, (2) negotiation, (3) contracts, (4) aggregation strategies and control, and (5) control and settlement.  It is not clear that MAS+ has established a strategy to more equitably incorporate women and youth into project activities and thus its benefits. In the absence of a strategy, based on a gender needs identification at the onset of the project, the limitations of women and youth to access project benefits cannot be defined. MAS+ should consider assigning project personnel to exclusively focus on gender and youth. This group would design the projects gender strategy and define the indicators and goals for the remaining of the project.  In the same vein, although there are clear indications that producers are knowledgeable about CSA and environmentally friendly practices, this knowledge cannot directly be ascribed to the project. MAS+ should consider, as above, in assigning dedicated personnel to this function and develop the necessary indicators to measure progress in environmental stewardship and CSA.  While the initial goal of GAP implementation has been fulfilled, qualitative indicators regarding these practices should be considered such that the professionalization aspect of the training can be noticed. That is, for example, fertilization is a good practice, but adequate fertilization considering soil analysis and correct dosages at the correct times is a good practice that indicates a more prepared producer and more likely to be sustainable in the long-run. On access to finance  Current MAS+ MEL data can detail number of minimucredit operations and amounts disbursed but, it does not identify those producers with access to credit. MAS+ should consider including this indicator as part of its regular monitoring activities and reporting.  MAS+ should consider sharing its access to finance models with BANHPROVI as the main second-tier bank providing funds destined for agriculture. Given the current situation where the bank is now providing direct loans, similar products as those designed for the revolving fund may be of interest to the bank.  Factoring developed by the project has proven to be effective in the dry-bean value chain, there is a unique opportunity to extend this model to other rural activities and modify it to be operated by the cajas rurales.  Given the success achieved by MAS+ in developing working capital financial products (including fertilizer supplied by buyers or the GOH) for SHFs, it is suggested that MAS+ explore the possibility of adding financial products geared towards the improvement of 28 on-farm infrastructure related to productivity and quality (irrigation, wet-mills, pulpers, etc.). These on-farm investments will help implement taught during the trainings. On the Community Seed Banks  As indicated before, in the absence of a strong of a strong player coordinating the link with DICTA and SENASA for the production of certified seed and distribution of Rhizobium inoculant, MAS+ should ensure POs and CSBs are sufficiently strengthened and empowered to carry out that process by themselves at the end of the project.  Dependency on DICTA for the access to and selling of certified seed and on IHMA as the main buyer of dry-beans is risky. MAS+ should consider identifying other possible market outlets for the dry-beans produced by the project participants.  Given the degree of technical knowledge involved in the production of certified seeds, MAS+ should consider establishing closer links between CSBs, academia and research institutions so as to provide feedback to seed developers and provide data that will improve future research and development activities. On environmental stewardship and Climate Smart Agriculture  The project does not track environmental or CSA indicators, as such, the resiliency shown by participants in the wake of adverse weather effects, and their knowledge of these topics cannot be ascribed to project activities. Furthermore, when participants indicate their unwillingness to prepare organic fertilizer from their coffee pulp, and bean producers indicate their desire to receive training in irrigation management there appears to be a clear need of emphasizing environmentally friendly and CSA practices during training and the development of project level indicators and goal towards these practices. For the remainder of the project, MAS+ has already developed a strategy to track implementation of environmental and CSA practices at the field level. It is highly desirable that a high-level staff member be assigned exclusively to ensure successful implementation of this strategy. On women and youth  MAS+ reports data on women participation in its different activities: training (25% women), sales to more formal markets (34% women) and access to credit (25% women). MAS+ also collects periodic information on the level of women participation in decisions about resource use. Additionally, MAS+ has provided a one-time training to the majority of its personnel on gender equity topics and includes some of these topics during the FFS training. Clearly these actions are not enough to reach TechnoServe desired women participation target of 40%. In the absence of a dedicated staff member for this activity it is unlikely that more progress on this front will be made. On child labor  Responses on the part of MAS+ technicians regarding child labor in the MTE focus groups indicate that further and continuous training on this topic are required. While child 29 labor may be the reality at the field level in Honduras, it is counter TechnoServe and USDA views that children be involved in on-farm work at the detriment of their future development. MAS+ should re-think the training provided to CTs and participants to include more child-friendly practices and emphasize children’s rights to education and a healthy childhood. On the MEL plan  The progress MAS+ has shown towards the achievement of LoP goals for all contractual indicators is impressive. At the same time, it is obvious that indicators on some of the cross-cutting themes, notably gender and youth, environmental stewardship, and CSA are missing. Given the current stage of project accomplishments and shortcomings, a strategic revision of the MEL plan as soon as a possible recommended. A thorough revision of the MEL plan will open the opportunity to make changes to the project logic, add/remove/update indicators, review assumptions and resubmit targets, which seem to be unrealistic to the current realities in Honduras. 30 ANNEXES Annex 1. Comparison of Mid-term results against baseline. This annex compares the results between those of the baseline survey conducted in July-August 2018 by IFPRI for five key indicators and outcomes of the project and evaluate the progress achieved on these indicators at the mid-term of the project. The baseline provided estimates for these indicators for those prospective beneficiaries and a control group of non-beneficiaries having similar characteristics. The baseline data was collected to specifically attempt to measure the increase in productivity among beneficiaries relative to a control group that will not be participating in project activities and assess the project impact after the five years of MAS+. At the request of MAS+, ANED Consultores, the firm selected to carry out the mid-term evaluation, collected data in a way that comparisons could be made against baseline data collected by IFPRI. For the baseline, after a sample of potential participant villages was established, half were randomly chosen to be in the treatment group and the other half in the control group. These same producers and villages were revisited to collect information for the MTE. The baseline data focused on three of the MAS+ activities: training, capacity building, and financial services (only for the coffee value chain) with the intent to answer, at the end of the project, three primary research questions: (1) What is the effect of training on agricultural productivity? (2) What is the additional effect of finance on agricultural productivity? and (3) What practices are more likely to be adopted by smallholder farmers, and why? Answers to these three questions will be used at project end to evaluate the impact MAS+ has had on coffee and dry-beans producers. On the other hand, the MTE seeks to evaluate project progress in general regarding project activities and cross-cutting themes on relevance, effectiveness, efficiency and sustainability. The general questions answered in this evaluation are: (1) Have incomes of participating coffee and dry-beans producers increased to date? (2) What progress has the project achieved towards fulfilling its goals? (3) Is the current implementation strategy favoring that the project reaches its results? If not, what changes in the implementation strategy could increase the chances of reaching the results? and (4) Are producers satisfied with training and services provided by the project. Given the different goals for the use of the baseline data and those of the MTE data, there are some differences in the data collection instruments and methodology but not in that of the sample selection process. For the MTE a subsample of those producers (except for the coffee producers in the department of Yoro where the baseline did not collect any data) included in the baseline study was taken to evaluate production levels, productivity, comparison of sales to exporters and intermediaries, sales volume, and income from sales. Since, overall, the analysis for the baseline study shows that the randomization was properly implemented and that the treatment and control groups are comparable at baseline across all the key outcomes of the evaluation, there is confidence in that comparisons between participants and non-participants at MTE will provide an accurate picture of progress to-date on these five indicators. 31 The baseline ended up with a sample of 961 coffee producers and 968 dry-beans producers being surveyed, from these, a sample of 282 coffee producers and 284 dry-beans producers were selected for the MTE and distributed as indicated in Table I.1. Again, it is stressed that all of those producers in the MTE sample, both for the control group as well as those in the participant group, had been interviewed for the baseline study except for those previously noted. With the realized results, the minimum detectable effects for the baseline data are estimated as 25% of the mean for the beans yields and 30% of the mean for the coffee yields and are similar for the MTE sample. Table I.1 Sample size comparison between Baseline study and Mid-Term Evaluation study. Department Baseline Mid-Term Evaluation Control Participants Control Participants Coffee Sample Comayagua 151 153 60 73 El Paraíso 163 110 51 0 Francisco Morazán 22 0 12 Olancho 150 190 23 51 Yoro 0 0 12 0 Total Coffee 464 475 146 136 Dry-beans Sample Comayagua El Paraíso 66 57 27 Francisco Morazán 22 33 12 12 Olancho 228 240 51 84 Yoro 159 163 61 37 Total dry-beans 475 493 151 133 Five performance indicators were to be compared to evaluate participant progress against baseline data and against that of a control group selected at baseline: total production by farm, productivity, proportion of sales to intermediaries vis a vis sales to exporters, total sales, and income from sales. During the period between the project start-up and the MTE, there occurred market and weather changes that have impacted the dry-beans and coffee markets. The 2017-2018 coffee crop received lower prices than anticipated and the 2018-2019 crop received the lowest prices in the last 15 years. Prices for the 2019-2020 crop rebounded to 2017-2018 levels but still below the $1.30 per pound expected by coffee producers. In 2019, a severe drought caused the government to declare a national emergency as tens of thousands of hectares of maize and dry￾beans were lost; drought was followed by intense rains in the later months, causing flood problems in certain areas. These price and weather changes have an impact on progress comparisons against baseline data but still allow us to compare the impact of the program if comparisons are made with those of the control producers. Comparing any variable, say yields per ha, between participants and control producers would determine if the project training and connected activities has had any real benefits on participants over those not participating in the project. It is with this in mind that the comparisons that follow are approached. I.1 Total production Coffee 32 Coffee can provide multiple products and depending on the stage of the processing have different weight and densities. For example, coffee may be sold as either berries, wet or dried, etc. To measure coffee production, we standardize all measures to húmedo or wet parchment using conversion factors proposed by IHCAFE (Instituto Hondureño del Café) and TechnoServe that adjust for the humidity and processing in the weight. All production values for each presentation were converted to quintales (qq) and then presentations were converted from quintales to quintales in pergamino húmedo or wet parchment to obtain a standardized measure of production for coffee. Total coffee production depends on a combination of area under production and productivity (qq WP Coffee/ha in this case). If total coffee production is to be used as a variable to measure impact, certain assumptions must be made about land: it is either a constraint, or it is not. In the case of coffee, increasing the area planted (and thus obtain a higher production) may not be the best-case scenario from an environmental standpoint, as new coffee areas are generally located at higher elevations (due to climate change) using previous forest areas and, in most cases, altering the water recharge capacity of the ecosystem in which they are established. Because the period between Baseline and MTE is not sufficient for new planted areas to begin production, we can assume that area planted with coffee by a particular producer has not changed and thus, the effect on total volume is mostly as a result of yield variations in areas already under commercial production and plantía20 areas or stumped areas (recepa) coming into production. Values of total coffee production between Baseline and MTE shown in Table I.2 indicate that while participants have increased their total production by 22%, that of the control group has decreased by 16.8%. There could be many reasons for these differences, none of which are apparent by just by looking at these figures. Without understanding the reasons behind these changes, these differences cannot be ascribed to the project. Table I.2. Comparison between Baseline Study and Mid-Term Evaluation total coffee production in wet parchment equivalents. Coffee production Baseline Study Mid-Term Evaluation Control Participants Control Participants Total Farm Production qq WP Coffee 64.40 71.39 53.61 87.14 A quick analysis of the change in planted areas can be made from Table I.3. As expected, given the price and weather effects during the measurements, the area under stumping grew proportionately more for the control group as low prices would tend to favor that practice as a means of obtaining higher production later rather than investing in fertilizer in the short run. Table I.3. Breakdown of average coffee planted areas for Baseline and EMT in hectares Area Planted Baseline Mid-Term Evaluation Control Participants Control Participants Average total area planted 2.12 2.46 2.14 2.51 Average area in plantía 0.50 0.67 0.32 0.29 20 Area still not under full production is considered as plantía 33 Area Planted Baseline Mid-Term Evaluation Control Participants Control Participants Average area in commercial production 1.61 1.83 1.82 2.22 Stumped area 0.12 0.16 0.43 0.29 Given the price and weather situation, and the distribution of planted areas between the two measurements, the differences in production can be presumed to come from differences in yields, with a slight difference due to the differences in total area and the distribution of planted areas between plantía, commercial production and stumped. Thus, a first project impact could be considered that despite the adverse price and weather effects, production for participant producers has increased while that of the control group has not only decreased, but the gap between control and participants has widened. This increase in total production will, cet par, result in higher incomes for the control group. Dry-beans Among dry-beans producers, the Baseline study found that some farmers primarily plant in the primera season, while others in the postrera. Households producing in the primera tend to sell more of their harvest than those in the postrera and produce for home consumption; those who sell beans typically do so to an intermediary. While this still holds true for the MTE, no data was collected for home consumption. Data for average total production is shown in Table I.4. Table I.4. Comparison between Baseline Study and Mid-Term Evaluation total dry-bean production Baseline Study Mid-Term Evaluation Control Participants Control Participants Total Production primera kgs 801.18 921.20 813.64 1010 Total production postrera kgs 607.87 672.01 826.36 768.18 Because of the small areas involved in planting, land use is not a limiting factor thus, by simply evaluating differences between the total production without considering land used and yields, project impacts cannot be inferred from the total production data. To further complicate an analysis of land use, baseline data report that area planted was lower than harvested, a clear mistake that cannot be addressed at this time. Suffice to say, by looking at the data above, that except for an increase in production for the control group, the situation between the baseline study and the MTE for this impact indicator, remains the same. 34 I.2 Yields Coffee Yield comparisons can be made from the data in Table I.5. To compare yields, we chose to use the yields of areas under commercial plantation as opposed to the average yield used in the Baseline study. We chose this value because the goal of agricultural project is to increase the full production yield as opposed to increasing the yields of areas coming into production (plantía) as they can be of different ages and therefore different production levels. The hypothesis here is that the steady state of a coffee plantation is that in which all the planted is under full production with a certain number of stumped rows depending on the rejuvenation cycle desired by the producer. Table I.5. Comparison between Baseline Study and Mid-Term Evaluation yield for coffee in commercial production areas. Baseline Study Mid-Term Evaluation Control Participants Control Participants Coffee yields in commercial production areas qq WP Coffee/ha 36.41 37.10 25.62 38.33 It is clear from watching the above that yields have not changed very much for the participant group (approximately 3%), with the project goal of a 30% increase, this represents only a 10% increase towards goal. This, however, cannot be construed as lack of project impact as, when comparing the situation with that of the control group at the time of the MTE, these have dropped their productivity by 42.1%. This drop is likely due to the neglect of their plantation due to the low prices and adverse weather in the period between measurements. Thus, while project impact cannot be measured in yield increases, it can be measured in terms of resiliency in maintaining yields. Despite adverse market and weather conditions, project participants fared substantially better than those of the control group. Dry-beans According to the Baseline study, both control and participants had received technical assistance through extension services, however, the participant group had received substantially more extension with similar themes as MAS+. For MAS+ promotion of two yield enhancing techniques for dry-beans producers are expected to bring about the desired 30% yield increase: use of seed inoculum and certified seed – including drought resistant seed. Table I.6. Comparison between Baseline Study and Mid-Term Evaluation yields for dry￾bean production Baseline Study Mid-Term Evaluation Control Participants Control Participants Dry-beans yield primera kgs/ha 839.86 961.58 625.45 919.55 Dry-beans yield postrera kg/ha 643.34 696.46 654.54 811.36 35 Data for the mid-term evaluation and shown in Table I.6 was collected after a severe drought affecting the primera cycle and intense rains in the latter part of the year affecting the postrera crop. Comparing MTE yields with those of the baseline, these have fallen for the primera cycle for both the control group and the participant group, however, the drop in yield for participants was only 4.6% while that of the control group decreased by 34.2%. For the postrera cycle, the control group remained at the productivity level (1.7% increase), but the participants group increased its productivity by 16.4%. As with the coffee producers, project impact on yields may not be significant as compared to baseline data, but given the adverse climate conditions, the project has provided the tools to make participant dry-beans producers more resilient and maintain the same level of production and productivity. I.3 Total Sales Coffee Total sales determine gross income received by the producer and impact on their livelihoods. Total gross impact will impact on the producer’s ability to meet family needs, so they are important measure of project impact. Producers sell their coffee in different stages of postharvest processing, each commanding a different price, depending on the infrastructure at the farm level and, in many cases, depending on the different prices and weather conditions that can impact on drying and transportation from the farm to the exporter. Because the composition of sales may vary from year to year, total sales reporting is done in WP Coffee equivalents. Table I.7. Comparison between Baseline Study and Mid-Term Evaluation total sales by coffee producers Baseline Study Mid-Term Evaluation Control Participants Control Participants Average sales MT of WP Coffee equivalent 6.29 4.11 2.34 5.27 From Table I.7, we come to the same conclusions about project income as those regarding total sales. Participating producers have been able to increase their sales volumes levels by 28.2% while those of non-participants have dropped dramatically by 168.8%. In the wake of very low coffee prices, participant producers have managed to keep their farms running at the same levels as before those shocks, obviously through better agronomic practices and using, adequate levels of fertilization despite low coffee prices21 . 21 MAS+ has introduced cost-cutting techniques like the use of soluble fertilizer, reduced postharvest losses and reduce the price of fertilizer by volume purchase through the AIMCs. 36 Dry-beans There were no statistically significant differences in total bean sales among participating and non￾participating farmers during the baseline study. We should expect the same at the MTE evaluation measurement. The MTE did not differentiate between production for home consumption and production for sale. Therefore, little conclusions about the effect of project impact on dry-beans producers can be extracted from the total sales of dry beans. According to the baseline study, dry-beans sales represent 60% of dry-bean producer’s income, we should expect that if higher paying markets, yield increases and access to finance as promoted by MAS+ are effective, that producers would plant more areas and higher amounts after joining the program. This is not the case, as shown in Table I.8, so that there must be other reasons, since land is not a limiting factor, why this has not occurred. MAS+ would do well in investigating the causes so that dry-beans producers generate sufficient income to cover their family needs. Table I.8. Comparison between Baseline Study and Mid-Term Evaluation total sales for dry-bean producers Baseline Study Mid-Term Evaluation Control Participants Control Participants Average dry-beans sales primera MT 0.77 0.83 0.86 1.03 Average dry-beans sales postrera MT 0.59 0.62 0.84 0.75 I.4 Income from Sales Total income is an important impact indicator for the MAS+ project. Income from sales depends on two factors: total sales and prices received. A main objective of MAS+ is to increase producer incomes with one of the pillars being that of receiving better prices for their products through a combination of improvements in quality, linkages to higher paying markets and reducing the intermediation chain. We have already discussed the production aspects above. Here, we discuss the price component for the total sales. MAS+ follows a clear strategy to increase prices: quality improvement at the farm level, associativity to gain market power, and AIMCs with large firms to shorten the intermediation chain and achieve better prices, comparisons between Baseline and between control and participants’ groups should provide an indication of progress towards objectives and impact to-date on those indicators. Coffee Better coffee prices are to be achieved by MAS+ through improvements in quality, increasing the quantity of dry parchment coffee sold by the producer and selling higher volumes directly to the exporter rather than through the intermediary. To achieve this, MAS+ follows various strategies at the agricultural, postharvest, organizational and marketing levels. MAS+ has worked extensively on improving coffee quality at the farm level, changing the mix producers market in 37 favor of dry parchment coffee as opposed to wet parchment, organizing producers into POs and cajas rurales so that they can negotiate directly with exporters, and providing services to ensure that quality standards required by exporters are met. The data in Table I.9 indicate that the MAS+ strategy has been successful. When examining the prices received per MT of WP Coffee equivalent (dividing the income data by the sales volume), it is clear that the increase in gross incomes is both a produce of yield and price increases, the two pillars of MAS+ intervention. Table I.9. Comparison between Baseline Study and Mid-Term Evaluation coffee income between participant and non-participant producers Baseline Study Mid-Term Evaluation Control Participants Control Participants Average income from coffee sales US$ 2677.46 3445.27 2368.29 5249.46 Table I.10 Prices received by MAS+ participant and non-participant producers at Baseline and MTE. Baseline Study Mid-Term Evaluation Control Participants Control Participants Average price per MT of WP Coffee US$ 425.67 838.267 1089.76 1264.70 Because prices received are dependent on the world market price, comparison against baseline may not be an appropriate comparison to evaluate progress towards this objective, but it is obvious from the data in Table I.10 that participant farmers fared better than participating farmers regarding price increases for their product (16% difference).22 Dry-beans MAS+ uses the same strategy to increase total incomes for dry-beans producers as it does for coffee: associativity to increase market power, better quality through drying and AIMCs with large buyers such as IHMA. In Table I.10 total income for both primera and postrera cycles are compared. As total income from dry-beans sales depends on the willingness of producers to sell their product or keep it for home consumption, evaluation of cash income may be a misleading indicator as to the wellbeing of producer families. MAS+ must consider including the dry-beans volume kept for home consumption as part of the total income if comparisons against Baseline or controls are to be meaningful. 22 MAS+ data indicate that the price differential between prices to exporters and those of intermediaries is 40%. Some 40% of all MAS+ coffee producers sell to exporters through the AIMC mechanism. MAS+ has also introduced cost-cutting techniques for farm activities, lower the cost of fertilizer and reduce postharvest losses that may boost net incomes by 23%. 38 Table I.11. Comparison between Baseline Study and Mid-Term Evaluation average gross income received for dry-beans sales between participant and non-participant producers Baseline Study Mid-Term Evaluation Control Participants Control Participants Average income from dry￾beans sales primera US$ 562.18 680.85 650.56 850.47 Average income from dry￾beans sales postrera US$ 1,215.81 1,347.14 646.78 631.98 Table I.12. Comparison between Baseline Study and Mid-Term Evaluation prices received for dry-beans between participant and non-participant producers Average price US$/MT Baseline Study Mid-Term Evaluation Control Participants Control Participants Average dry-beans sales primera 730.10 820.30 757.16 835.89 Average dry-beans sales postrera 2,060.69 2,172.81 771.15 842.64 Looking at the prices received, there appears to be differences in the Baseline postrera prices that require an explanation that is beyond the scope of this comparison. Difference in prices received by dry-beans producers participating in MAS+ and those of the control group are 9 to 11% higher, depending on the crop cycle. This difference is also holding for the baseline study comparison between participants and control. As prices are highly influenced by government decisions, and 70% of MAS+ producers selling to IHMA, price comparisons against baseline are not as meaningful to determine project impact on this variable as those against control at the time of the survey. I.5 Make up of sales for coffee producers. MAS+ encourages participating producers to sell more of their coffee as dry parchment because it commands a higher price. Here we examine how the sales mix has changed between the Baseline study and the MTE. From the discussion above, it has already been determined that overall average prices received by participants are higher than those received by non-participants, and even higher than those received if they were to sell to intermediaries in their communities. Table I.13 shows that the bulk of sales income of participants at the time of the MTE come from selling dry parchment coffee and that there has been a substantial shift in the amount of coffee 39 sold as dry parchment by these producers (dry parchment sales representing 69% of volume sold and 75% of income from sales, vs. 6% and 5.8% respectively at Baseline). Table 1.13. Income received from the different coffee presentations by MAS+ participants and non-participants Baseline Study Mid-Term Evaluation Control Participants Control Participants Total sales US$ 2677.46 3445.27 2,342.33 5,242.26 Cherry 71.73 253.77 112.35 522.83 Wet parchment 1746.33 2064.19 30.74 550.47 Dry parchment 151.17 201.43 1,759.35 3,996.54 Green 708.23 920.03 439.9 172.42 Prices received by MAS+ participating producers, except for those of cherry coffee are higher than those received by non-participants. Because of price changes in the world market price, comparison against baseline is not an appropriate method to evaluate project impact on prices. Comparison, at the time of measurement, provides a better understanding of the project impact on price variables. In the case of MAS+, the overall increase in average sales price for those participating in the project is 16% with differentials varying from -5.3% for the cherry coffee to 14% for the wet parchment coffee. The overall difference in income is made by the amount of coffee sold as dry parchment which represents 76% of total sales. Table 1.14. Breakdown of sales in MT of wet parchment coffee equivalent for participant and non-participant producers at Baseline and MTE. Baseline Study Mid-Term Evaluation Control Participants Control Participants Average sale price in $/MT of WP Coffee equivalent 425.67 838.27 1089.76 1,264.70 Cherry 652.09 551.67 302.20 287.03 Wet parchment 420.80 852.97 1081.89 1223.80 Dry parchment 604.68 805.72 1951.84 2035.79 Green 395.66 948.48 1691.92 1915.78 40 Summary Impact to date of MAS+ has been positive. Particularly as it refers to the resiliency of both coffee and dry-beans producers to adapt to disruptive weather shocks, and in the case of coffee farmers to market price shocks. Despite these occurrences, yields, as compared with those of non￾participant farmers are much higher as are sales. MAS+ has also managed to increase prices and, though not a part of this comparison, managed to introduce cost-cutting techniques that have allowed farmers to continue investing in their coffee and dry-beans farms.