Mejoramiento Agrícola Sostenible MAS+ Impact Evaluation Baseline Report November 2018 b Prepared for TechnoServe by the International Food Policy Research Institute Last revision: January 10, 2019 Contacts: Alan de Brauw Markets, Trade and Institutions Division International Food Policy Research Institute 1201 Eye Street NW, Washington, DC Miguel Almanzar Markets, Trade and Institutions Division International Food Policy Research Institute 1201 Eye Street NW, Washington, DC Impact Evaluation Baseline Report for FY 2017 Food for Progress Mejoramiento Agrícola Sostenible (MAS+) Project in Honduras Program: Food for Progress Agreement Number: FCC-522-2017/023-00 Funding Year: Fiscal Year 2017 Project Duration: 2017-2022 Implemented by: TechnoServe i Contents List of Tables ...................................................................................................................................iii List of Figures..................................................................................................................................vi List of Acronyms.............................................................................................................................vii Executive Summary...................................................................................................................... viii Data Sources...............................................................................................................................ix Coffee Farmers ...........................................................................................................................ix Beans Farmers.............................................................................................................................x Summary of Key Indicators..........................................................................................................x 1 Introduction.............................................................................................................................. 1 2 Background.............................................................................................................................. 1 Primary Research Questions .................................................................................................. 3 Theory of Change.................................................................................................................... 4 3 Impact Evaluation Methodology .............................................................................................. 5 Randomized Control Trial Methodology: Coffee Producers Impact Evaluation ..................... 6 Propensity Score Methodology: Beans Producers Impact Evaluation ................................... 7 4 Data Sources ........................................................................................................................... 8 4.1 Survey Design .................................................................................................................. 8 4.2 Data Collection: Training, Pilot and Fieldwork............................................................... 13 4.3 Statistical Power and Sample Selection ........................................................................ 14 Survey Response Rates........................................................................................................ 16 Effective Minimum Detectable Effects................................................................................... 17 5 General Characteristics of the Intervention Area.................................................................. 18 6 Analysis of Coffee Producers in MAS+ Evaluation ............................................................... 20 6.1 Demographic Characteristics......................................................................................... 20 6.2 Agricultural Outcomes.................................................................................................... 22 Size of Cultivated Land ......................................................................................................... 22 Coffee Production.................................................................................................................. 23 Coffee Yields.......................................................................................................................... 25 Coffee and Markets: Buyers, Sales and Prices .................................................................... 29 Evaluation Validity Tests at Baseline: Agricultural Productivity for Coffee............................ 32 ii Inputs, Technologies and Management Practices ................................................................ 39 Coffee Extension ................................................................................................................... 44 Coffee Pests .......................................................................................................................... 45 Investments and Equipment.................................................................................................. 46 Group Membership and Participation ................................................................................... 47 Evaluation Validity Tests at Baseline: Inputs, Extension and Pests for Coffee .................... 48 7 Analysis of Beans Producers in MAS+ Evaluation ............................................................... 52 7.1 Demographic Characteristics......................................................................................... 52 7.2 Agricultural Outcomes.................................................................................................... 54 Size of Cultivated Land ......................................................................................................... 54 Beans Production .................................................................................................................. 55 Beans Food Security ............................................................................................................. 56 Beans Yields .......................................................................................................................... 56 Beans Sales........................................................................................................................... 60 Evaluation Validity Tests at Baseline: Agricultural Productivity - Beans............................... 61 Inputs ..................................................................................................................................... 67 Practices ................................................................................................................................ 68 Beans Extension.................................................................................................................... 70 Beans Pests........................................................................................................................... 71 Investments and Assets of Beans Farmers .......................................................................... 71 Group Membership and Participation ................................................................................... 73 Evaluation Validity Tests at Baseline: Inputs, Extension and Pests for Beans..................... 74 8 Discussion and Conclusions ................................................................................................. 79 9 Recommendations for Monitoring ......................................................................................... 81 References.................................................................................................................................... 82 Appendix 1: Collective Action, Transfers, Credit, Shocks, Aspirations and ICTs......................... 83 Coffee Sample........................................................................................................................... 83 Beans Sample ........................................................................................................................... 90 iii List of Tables Table 1 Projected Baseline Sample Characteristics and Minimum Detectable Effect................. 15 Table 2 Coffee Sample Design ..................................................................................................... 16 Table 3 Beans Sample Design ..................................................................................................... 16 Table 4 Completed sample and Response Rates........................................................................ 17 Table 5 Minimum Detectable effects of Sample ........................................................................... 17 Table 6: Demographic Characteristics for Coffee Sample ........................................................... 20 Table 7: Demographic Characteristics.......................................................................................... 21 Table 8: Human Capital Characteristics, by Gender.................................................................... 21 Table 9: Agricultural and Non-Agricultural Income ....................................................................... 21 Table 10: Size of planted and cultivated area for Coffee Sample................................................ 23 Table 11: Number of total plots and owned plots for coffee sample ............................................ 23 Table 12 Units and Processing Levels selected by respondents................................................. 24 Table 13: Coffee production.......................................................................................................... 25 Table 14: Coffee Yields ................................................................................................................. 26 Table 15 Coffee Buyers and Quantity Sold .................................................................................. 29 Table 16 Coffee Buyers and Sale Prices...................................................................................... 30 Table 17: Coffee Sales: Value....................................................................................................... 31 Table 18 Coffee Sales: Quantity by Processing Level ................................................................. 32 Table 19: Balance Test for Coffee Area Size................................................................................ 33 Table 20: Balance Test for Coffee Production .............................................................................. 34 Table 21: Balance Test for Coffee Yields...................................................................................... 35 Table 22: Balance Test for Coffee Sold and Value of Sales......................................................... 37 Table 23: Balance Test for Quantity Sold by Buyers of Coffee .................................................... 38 Table 24: Share of farmers that applied coffee inputs.................................................................. 39 Table 25: Cost of inputs and labor costs ...................................................................................... 40 iv Table 26: Coffee Practices............................................................................................................ 40 Table 27: Coffee Practices: Plant and tissue management ......................................................... 41 Table 28: Coffee Practices: Quality .............................................................................................. 41 Table 29: Coffee Practices: Soil Care........................................................................................... 42 Table 30: Coffee Practices: Post-Harvest..................................................................................... 43 Table 31: Coffee Practices: More on Quality................................................................................ 44 Table 32: Coffee Extension........................................................................................................... 45 Table 33: Pests in Coffee.............................................................................................................. 46 Table 34: Investments in the agricultural plots ............................................................................ 46 Table 35: Investment in Equipment .............................................................................................. 47 Table 36: Groups present in the community................................................................................. 48 Table 37: Balance Test for Share of Farmers that applied inputs ................................................ 49 Table 38: Balance Test for Cost of Inputs and Labor Costs......................................................... 49 Table 39: Balance Test for Coffee Extension................................................................................ 50 Table 40: Balance Test for Pests in Coffee................................................................................... 51 Table 41: Demographic Characteristics, Beans Sample.............................................................. 52 Table 42: Demographic Characteristics........................................................................................ 53 Table 43: Demographic Characteristics by Gender ..................................................................... 53 Table 44: Agricultural and Non-Agricultural Income..................................................................... 53 Table 45: Area Size of planted and harvested area with beans................................................... 54 Table 46: Beans Production.......................................................................................................... 55 Table 47: Beans Production Final use, Primera........................................................................... 55 Table 48: Beans Production Final Use, Postrera ......................................................................... 56 Table 49: Beans Food Security..................................................................................................... 56 Table 50: Beans Yields ................................................................................................................. 57 Table 51: Beans sales, quantity and value of sales ..................................................................... 60 v Table 52: Balance Test for Beans Area Size Planted and Harvested .......................................... 62 Table 53: Balance Test for Beans Production .............................................................................. 63 Table 54: Balance Test for Beans Yields ...................................................................................... 64 Table 55: Balance Test for Beans Sales, quantity and value of sales ......................................... 66 Table 56: Share of farmers that used beans inputs ..................................................................... 67 Table 57: Total costs of Inputs and Labor Costs .......................................................................... 67 Table 58: Practices Beans: Access to Markets, Varieties and Selling Points .............................. 68 Table 59 Sales of Bean Production by buyers ............................................................................. 68 Table 60: Practices in Bean parcels ............................................................................................. 69 Table 61: Beans Farming Practices.............................................................................................. 70 Table 62: Extension received for beans sample........................................................................... 70 Table 63: Pests that affected Beans crops................................................................................... 71 Table 64: Investments and Asset Counts ..................................................................................... 72 Table 65: Eigenvalues for first component of PCA for Investments............................................. 72 Table 66: Eigenvalues for first component of PCA for Equipment............................................... 73 Table 67: Groups in the Community, beans sample .................................................................... 73 Table 68: Balance Test for Shared of Beans Farmers that applied inputs................................... 75 Table 69: Balance Test for Input Costs and Labor Costs ............................................................. 75 Table 70: Balance Test for Beans Extension................................................................................ 77 Table 71: Balance Test for Pests in Beans................................................................................... 78 Table 72 Summary Baseline Measures: Coffee Key Variables.................................................... 80 Table 73 Summary Baseline Measures: Beans Key Variables .................................................... 81 Table 74: Coffee Practices: Access to Market and Expectations................................................. 83 Table 75......................................................................................................................................... 84 Table 76: Remittances and transfers received by coffee sample ................................................ 86 Table 77: Credit for coffee sample................................................................................................ 87 vi Table 78: Shocks experienced in the last 12 months................................................................... 88 Table 79: Sources of received agricultural information ................................................................ 88 Table 80: Internet and Cellular Use .............................................................................................. 89 Table 81: Collective Action............................................................................................................ 90 Table 82: Remittances and Transfers received by beans sample ............................................... 92 Table 83: Credit received by beans sample ................................................................................. 93 Table 84: Shocks experienced during the last 12 months, beans sample................................... 94 Table 85: Sources of agricultural information ............................................................................... 94 Table 86: Internet and cellphone use ........................................................................................... 95 List of Figures Figure 1: Coffee and Beans Sample Geographic Distribution ..................................................... 18 Figure 2 Distribution of coffee productivity ................................................................................... 27 Figure 3: Coffee area and coffee production, total and for commercial production .................... 28 Figure 4 Distribution of Prices of Coffee: Calculated by Processing Level ................................. 31 Figure 5 Distribution of coffee sales ............................................................................................. 32 Figure 6 Area Planted Distribution across Treatment Assignment .............................................. 34 Figure 7 Yield Distribution across Treatment Assignment ........................................................... 36 Figure 8 Quantity sold across treatment assignment................................................................... 38 Figure 9 Distribution of Beans productivity by season................................................................. 57 Figure 10: Beans Area and Beans Production in Primera and Postrera ..................................... 59 Figure 11 Distribution of Beans Sales by season......................................................................... 61 Figure 12 Distribution of Area Planted by Treatment Assignment ............................................... 63 Figure 13 Distribution of Yields by Treatment Assignment .......................................................... 65 Figure 14 Distribution of Beans Quantity Sold by Treatment Assignment................................... 66 Figure 15: Aspirations, locus of control and self-esteem perceptions ......................................... 85 Figure 16: Aspirations, locus of control and self-esteem perceptions ......................................... 91 vii List of Acronyms ANCOVA Analysis of Covariance CAPI Computer-Assisted Personal Interviews COPECO Permanent Emergency Contingency Committee CRT Cluster Randomized Controlled Trial CSA Canadian Standard Association CSB Community Seed Bank ES Executive Summary EV Eigen-Values ESA ESA Consultores. Economics, Society, Environment and Engineering GPS Geographic Positioning System ICT Information and Communication Technologies IHCAFE Honduran Institute for Coffee IFPRI International Food Policy Research Institute MAS Mejoramiento Agrícola Sostenible - Sustainable Agricultural Improvement MDE Minimum Detectable Effects MDG Millennium Development Goals MT Megagram MT Metric Ton MSU Michigan State University PCA Principal Component Analysis POs Producer Organizations PPP Purchasing Power Parity PSM Propensity Score Matching RCT Randomized Controlled Trial SAG Honduran Ministry of Agriculture SD Standard Deviation SE Standard Error USAID United States Agency for International Development USD United States Dollar USDA United States Department of Agriculture USG United States Government WP Wet parchment viii Executive Summary The project Mejoramiento Agrícola Sostenible (henceforth referred to as ‘MAS+’) is a five-year project funded by the United States Department of Agriculture (USDA) and implemented by TechnoServe in Honduras. The MAS+ project is a continuation and expansion of proven interventions first implemented in the FY2012 MAS project1 . It seeks to work with 32,000 smallholder coffee and bean producers across nine departments in Honduras. The MAS+ project’s integrated approach is based on seven discrete but interrelated activities that aim to increase coffee and beans producers’ incomes through: training to improved crop productivity and quality, expansion of financial services and inputs, capacity building for producer organizations, and marketing initiatives aimed at expanding the coffee and dry-beans value chains. This report describes the impact evaluation approach being applied to the MAS+ project, as well as the results of the impact evaluation baseline survey conducted in 2018. The impact evaluation described in this report is an important component of the overall monitoring and evaluation plan for MAS+. The goal of the impact evaluation is to provide rigorous evidence about how specific activities lead to improved agricultural productivity related to coffee and beans production. The impact evaluation will measure intermediate outcomes along the hypothesized causal chain to understand how impacts may have come about; furthermore, by measuring against a control group, it will help us understand what gains in production and/or productivity can be attributed to the MAS+ project. The impact evaluation is focusing on three MAS+ activities: training, capacity building, and financial services (for the coffee value chain). It seeks to answer three primary research questions: 1) What is the effect of training on agricultural productivity? 2) What is the additional effect of finance on agricultural productivity? and 3) What practices are more likely to be adopted by smallholder farmers, and why? The impact evaluation takes slightly different approaches to answering questions 1 and 3 for both value chains and all three questions for the coffee value chain. For the coffee value chain, the impact evaluation methodology will be a cluster randomized controlled trial. After a sample of potential participant villages was established, half were randomly chosen to be in the treatment group and the other half in the control group. The impact evaluation in the beans sample is being conducted with non-experimental methods. To further improve the design, beneficiary groups were initially chosen, and then non-participants were selected to be similar to the participants at the start of the project by initially pairing the prospective groups with similar ones. 1 This includes leveraging the private and public sector partnerships developed during the MAS Program and expanding on that project’s technical assistance model, focus on higher quality markets and use of proper inputs, access to finance model, and efforts to strengthen producer organizations and incorporate the participation of youth in the program’s activities. ix Data Sources The baseline survey was jointly designed by IFPRI, TechnoServe, and ESA Consultores, the local firm that conducted the survey. It was conducted using computer assisted personal interviews software called Survey Solutions. The content of the survey focused on three themes: agricultural production and commercialization, extension and technology practices, and producer association and credit access, which mirrors the research questions noted above. It was conducted between July and August of 2018. IFPRI researchers, jointly with ESA Consultores, conducted a 14-day training between June 4th to June 17th: 7 days with field managers and groups’ supervisors and the remaining 7 days with the whole team of enumerators to review agricultural concepts facilitated by TechnoServe. We designed the sample to target an increase in coffee yields of 24% (40% among participants, at a 60% participation rate) and bean yields of 21% at the 5 percent significance level and 80 percent power, through the baseline and endline surveys; the goal was to survey 1000 households for each arm. We ended up with a sample of 961 coffee producers and 968 beans producers. With the realized results, the minimum detectable effects are estimated as 25% of the mean for the beans yields and 29% of the mean for the coffee yields, at the same significance and power levels. If covariates can help explain yields at endline, these minimum detectable effects sizes will decrease somewhat. Coffee Farmers Among coffee producers, we find that coffee income represents about 69 percent of total income on average. Coffee farmers produce for commercial purposes and usually sell in wet parchment presentation. Coffee farmers mainly sell to an intermediary and very few to exporters. Yields are around 70 quintales per hectare in wet parchment on average and 34 quintales per hectare in wet parchment at the median, highlighting the wide range in coffee farmer productivity found in the sample. Not all coffee farmers use inputs, the most common inputs used are chemical fertilizers (63 percent) and herbicide (36 percent). Few coffee farmers, 27 percent, use foliar fertilizer. Further, less than 10 percent have received extension in the past 12 months. Forty percent of coffee farmers report having problem with pests, particularly coffee rust and broca. Given that the intervention will be cluster randomized, not surprisingly very few differences were found between the treatment and control groups at conventional significance levels (specifically, due to farmers’ coffee being affected by the minador pest). Overall, the analysis shows that the randomization was properly implemented and that the treatment and control groups are comparable at baseline across all the key outcomes of the evaluation and project. The distribution of prices across different coffee processing levels had a bi-modal shape after standardizing the quantities and processing levels. This indicates that there are two different pricing regimes for coffee; which price a farmer obtains depends on other attributes of their production. In the endline, we will characterize these attributes, to help explain differences in prices within each processing level. By helping to explain these differences, the power to detect differences will increase. x Beans Farmers Among beans producers, we find that some farmers primarily plant in the primera season, while others in the postrera. This tendency is mainly due to inherent characteristics of the bean crop cycle and the primera season’s more favorable weather conditions, including temperature, rainfall, and soil drainage capabilities, among others. For this reason, primera season generally yields higher volumes than postrera. Thus, households producing in the primera tend to sell more of their harvest than those in the postrera, and many produce for home consumption. Households that sell beans typically do so to an intermediary. Yields are around 888 kilograms per hectare in the primera and 670 in the postrera. While not all households use inputs, the most common inputs used are herbicides and fertilizer. Few households use any inoculant when planting, which boosts yields; very few farmers (17 percent of the sample) have received any extension in the past 12 months either. Some farmers report having problems with pests; therefore it will be important to monitor whether pest exposure differs between treatment and control areas. Despite the fact that the beans intervention is not randomized, we found very few statistically significant differences between the treatment and control groups. The main difference we found is that the beans treatment group has received more extension in the past twelve months than the control group. We hypothesize that some farmers may have received extension with similar themes as MAS+ in the past 12 months; however, it is a relatively small number of households. We will control for this exposure in the endline, and plan to tailor the survey form to ensure that we can directly capture the types of extension that are provided by MAS+ partners. We also find less exposure to pests, specifically losses, among the treatment group relative to the control group. However, the magnitude is small. For this second difference, we will keep in mind at endline to learn whether pests are more endemic to control areas, so that we can control for this difference if necessary as well. Summary of Key Indicators We summarize the baseline findings for important agricultural productivity outcomes—the production, yields, volume sold, and the total reported value of sales- in Table ES-1 for coffee and Table ES-2 for beans. Farmers in the sample have a wide spread in their agricultural productivity, which creates an opportunity for MAS+ to provide a diverse set of training that can respond to different characteristics of farmers. Namely, for farmers that are below the median of the yield distribution, training aimed at increasing productivity can bring about large impact across these outcomes. In the case of those farmers that are above the median in the productivity distribution, an intervention aimed at increasing their access to better markets (including export), credit, certifications, etc. can have a large impact on the total income they derive from agriculture. xi TABLE ES-1 SUMMARY BASELINE MEASURES: COFFEE KEY VARIABLES Mean SD Median Observations Coffee production in qq WP, total 70 125 34 925 Total yields (qq/ha WP) 30.9 30.1 21.0 925 Quantity of coffee sold in Total MT WP 4.08 12.95 1.41 898 Total Sale value (USD) 3,220 6,887 1,500 873 Notes: Sample for beans consists of 961 farmers. Difference in sample sizes is due to farmers that do not have positive quantities for the indicators. TABLE ES-2 SUMMARY BASELINE MEASURES: BEANS KEY VARIABLES Mean SD Median Observations Production, Primera (kg) 862 943 544 657 Production, Postrera (kg) 645 761 454 738 Yields, Primera (kg/ha) 888 596 782 657 Yields, Postrera (kg(ha) 670 440 586 738 Volume Sold, Primera (MT) 0.81 0.92 0.54 535 Value of Sales (USD), Primera 631 724 369 487 Volume Sold, Postrera (MT) 0.61 0.76 0.36 541 Value of Sales, Postrera (USD) 1311 6637 277 491 Beans Production Home Consumption kg in Primera 140 192 91 633 Beans Production Home Consumption kg in Postrera 144 219 91 708 Notes: Sample for beans consists of 968 farmers. Difference in sample sizes is due to farmers that do not have positive quantities for the indicators. 1 1 Introduction The project Mejoramiento Agrícola Sostenible (henceforth referred to as ‘MAS+’) is a five-year project funded by the United States Department of Agriculture (USDA) taking place in Honduras, aimed at increasing coffee and beans producers’ incomes through training to improved crop productivity and quality, expansion of financial services and inputs, capacity building for producer organizations, and marketing initiatives aimed at expanding the coffee and dry-beans value chains. The impact evaluation described in this report is an important component of the overall monitoring and evaluation plan for MAS+. The first part will monitor activities throughout the project to provide evidence about changes in agricultural productivity and expanded sales of agricultural products taking place among beneficiaries. The impact evaluation will specifically attempt to measure the increase in productivity among beneficiaries relative to a control group that will not be participating in project activities. The goal of the impact evaluation, then, is to provide rigorous evidence about how specific activities lead to improved agricultural productivity, related specifically to coffee and bean production. The impact evaluation will also measure intermediate outcomes along the hypothesized causal chain to understand how impacts may have come about; furthermore, by measuring against a control group, it will help us understand what gains in production and/or productivity can be attributed to the MAS+ project. This report presents the results of the baseline survey conducted in July-August 2018 to measure the key indicators and outcomes of the project and evaluate the impact of the activities under MAS+. The report describes the survey design, baseline estimates for key indicators, and validation of the proposed impact evaluation design and the comparability between prospective beneficiaries and non-beneficiaries. 2 Background The MAS+ project, led by TechnoServe, is a continuation and expansion of proven interventions first implemented in the 2012 MAS project to work with 32,000 smallholder coffee and bean producers in Honduras. The integrated approach is based on seven discrete but interrelated activities. MAS+ will expand its successful comprehensive training curriculum for farmers and producer organizations (POs), handing over implementation to private and public-sector extension workers by Year 4. MAS+ will also provide advisory services to firms supplying improved agricultural inputs and climate smart agriculture equipment and services to increase access. It will promote innovative financing mechanisms to increase access to finance for farmers and POs and develop opportunities to link strengthen these relationships. Finally, it will provide capacity building to government institutions to address key issues in the policy environment inhibiting trade and productivity in the agricultural sector. When key project initiatives have transitioned to market actors in Year 5, project goals include training targeted farmers to adopt a minimum of three good agricultural practices that will increase their yields by 30 percent and decreasing costs by 10 percent on average. 2 The MAS+ project has seven main activities:  Activity 1: Training: Facilitate Improved Farmer Productivity. MAS+ will implement a suite of on-farm training activities tailored to the specific needs of targeted farmers to facilitate improved productivity of coffee and bean parcels. TechnoServe will also support targeted producers to produce secondary horticultural crops for additional income.  Activity 2: Capacity Building: Producer Groups and Cooperatives. MAS+ will build the capacity of 435 POs to provide effective marketing, financial, and technical assistance to farmers. In addition, we will help POs implement value-added services.  Activity 3: Inputs: Develop Input, Service, and Equipment Providers. Market-oriented inputs and equipment suppliers will understand and supply the input and infrastructure needs of farmers and POs. MAS+ will provide business advisory services to existing agricultural input, equipment and service providers, with a particular focus on those providing CSA products for water-harvesting, irrigation, and soil analysis. TechnoServe and its partner MSU will continue to scale its community seed bank (CSB) model with additional refinements.  Activity 4: Capacity Building: Agricultural Extension Agents/Services. To ensure the sustainability of MAS+, TechnoServe will help public sector, private sector, and civil society organizations develop their own agricultural extension services and take over implementation of Activity 1.  Activity 5: Financial Services: Facilitate Agricultural and SME Lending. MAS+ will strengthen Cajas Rurales, rural savings and loans groups, to facilitate access to finance at the PO level. The program will continue to promote innovative agreement mechanisms that can successfully offset risk to exporters, such as factoring, input-supplier loans and harvest advances.  Activity 6: Market Access: Develop Buyer and Seller Relationships. MAS+ will promote marketing contracts among farmers and anchor firms. Furthermore, our partners will coordinate efforts with IHCOFFEE and the private sector to elevate international recognition of Honduran coffee quality.  Activity 7: Capacity Building: Government Institutions and Policy Regulatory Framework. TechnoServe will provide technical assistance to the Honduran Ministry of Agriculture (SAG) to validate bean varieties that can help farmers improve yields. TechnoServe will collaborate with the Permanent Emergency Contingency Committee (COPECO) and other actors to train targeted local and national government personnel to strengthen areas that enable smallholder producers to respond more effectively to climatic crises, including drought and rain excess. Not all the activities are appropriate for an impact evaluation, as a counterfactual is not available to measure the impacts of some of these activities. Specifically, the impact evaluation will focus on activities 1, 2, and 5, as training and cooperative capacity are considered as constraining productivity in both the coffee and bean sectors. For activity 5, the impact evaluation will specifically focus on the coffee value chain, as availability of finance to the smallholder is not thought to be a constraining factor in increasing bean yields. 3 Clearly, the other 4 activities may also lead to increased impacts on smallholder agricultural productivity. Rather than trying to isolate each activity on its own, we consider the impact estimates that will be generated by the impact evaluation as inclusive of those four activities. We study these activities through project monitoring and a mid-term process evaluation. Primary Research Questions The impact evaluation then will focus on three primary research questions, all of which relate to the USDA Food for Progress Learning Agenda. The key evaluation questions from the perspective of the quantitative impact evaluation are: 1. What is the effect of agricultural training on productivity? Two activities of MAS+ relate directly to the provision of agricultural extension. Therefore, the first primary research question is what the impact of training is on productivity. As trainings will take place through groups of farmers, this question relates to priority learning Question 15 in the Learning Agenda, “Do cooperatives, associations, federations, or collectives impact producers’ abilities to optimize sales to markets at the local, regional, or international level? What services provided by cooperatives lead to results? What types of cultural environments are best for these different models to thrive?” 2. What is the additional effect of access to finance on productivity? In the coffee value chain, the additional effect of access to finance may be important to increasing productivity. We will randomize this component in the coffee value chain in addition to provision of extension. This question relates to priority learning Question 14 in the Learning Agenda: “What are the best linkage models to help small and medium sized producers, traders, and post-harvest market actors, who frequently lack collateral, registration, and credit history to access loans or other financial instruments to effectively expand their businesses?” 3. What practices are more likely to be adopted by smallholder farmers, and why? Within the training, farmers will be exposed to new or different practices than they typically use, with the goal of increasing productivity and ensuring the quality and marketability of their produce. Both through process evaluation and impact evaluation, we plan to study which practices are more likely to be adopted by smallholders. This goal indirectly speaks to priority learning Question 21: “In what context is it profitable for agricultural actors, particularly producers and processors, to adopt higher product quality standards for sales in higher value markets, including international markets?” In this report, we discuss baseline measure of the concepts underlying these three research questions. Namely, to answer question one we are particularly interested in the indicators that describe coffee and bean productivity (e.g. production over cultivated area). To answer question two, we are interested in understanding how much access to finance farmers have, primarily access to credit, particularly among coffee farmers. To answer question three, we describe the practices farmers are already implementing that should help them increase agriculture 4 productivity, as well as the techniques they have learned from extension workers in the past and whether farmers have adopted them or not. Theory of Change To answer these research questions, it is valuable to have in mind a theory about how changes among producers will occur. The primary outcome of interest is in increasing productivity within the crop (coffee, beans) that each farmer is growing. In this subsection we describe our theory, and assumptions behind it, about how productivity at the farm level would increase. Recall that there are two mechanisms by which the MAS+ will interact directly with farmers that are described above—first, through agricultural training, and second, through finance in the context of the coffee intervention. So we first consider how agricultural training can influence productivity. A first way that trainings can increase productivity is by raising yields. To do so, we first assume that farmers are not using optimal techniques or practices on their coffee and/or beans; hence, it is possible for them to raise productivity by adopting techniques. Second, farmers must either attend trainings on those techniques, or learn about the techniques from their neighbors or peer farmers, who learn about them in the trainings conducted by MAS+. Third, farmers must either test adoption or adopt the techniques, and fourth, they must find the techniques both effective at making them more productive (e.g. in terms of yields) holding effort constant, or by making their farms more profitable, by reducing the amount of labor or effort it takes to produce the same or larger amounts. For coffee, it is not easy to adjust the amount of land used in the short term, but it is for beans. Therefore, bean farmers may also adjust the amount of land they use for beans; they may expand it if growing beans becomes more profitable relative to other crops using new techniques or practices. If not, farmers may continue to farm the same amount of land in beans or reduce the amount of land they use for beans. The second pathway through which trainings can boost productivity is by improving quality, which should increase the prices received by farmers for their products. Some of the assumptions here are similar; for example, that farmers are not optimizing the quality of their produce, and that by adopting new techniques they can produce higher quality, which can be differentiated by the market, leading to higher prices. Farmers must again both learn about and then adopt such techniques, and the techniques must work (meaning lead to higher prices) so that farmers continue to use them. If market power on the buyer side of the market exists (monopsony or oligopsony power), it could dampen returns to adopting such techniques, which would lessen the incentives for adoption. Throughout the theory so far, we have assumed that farmers are interested only in raising actual yields or the net value of production. Note that farmers are likely risk averse and may also worry about whether new techniques would increase the variability of yields, particularly in the context of climate change, which is raising temperatures and changing rainfall patterns. If so, then we must consider “risk adjusted” returns to the techniques. If farmers believe techniques they learn would both increase their incomes but also increase the variance of their incomes, the increase 5 in income must be more valuable than the increase in variance. Alternatively, the techniques taught to farmers may be designed specifically to reduce variance of returns. For example, farmers may be encouraged to use bean seeds that are more adapted to higher temperatures or less predictable rainfall. If so, then farmers may adopt even if they do not perceive that yield increases would occur from adoption. The second component to the theory of change is how added finance could increase coffee production. Here, we assume that finance implies credit (or liquidity); farmers do not have cash on hand to pay for either material or labor inputs when they are most needed to increase production (or the value of their crop). By making agricultural loans available, farmers can then take loans out and pay for missing inputs, increasing their productivity. We therefore are assuming that farmers would take out the loans and understand loan terms if made available, and would be able to obtain the needed inputs, hence increasing their productivity. This discussion informs the selection of key indicators that we discuss in section 4.4. Before discussing key indicators, though, it is important to review the impact evaluation methodology and surveys to be conducted to measure outcomes. 3 Impact Evaluation Methodology Impact evaluations are best defined as the response to question: “how is the outcome different than it would have been if the project had not been implemented?” However, we never observe a given beneficiary in both situations – with the project and without the project. Good impact evaluations resolve this problem through the use of a counterfactual. The counterfactual identifies what would have happened to the beneficiaries in the absence of the program. Understanding and accurately estimating this “parallel world” is a key challenge of impact evaluation, and is critical in understanding whether or not the intervention in question is effective and should be continued or expanded. A rigorous impact evaluation is one that combines the most appropriate and feasible evaluation design with the use of carefully selected indicators to document the impact pathway. There are several methods for conducting impact evaluations, with the use of random assignment to create treatment and control groups producing the most rigorous results. Using random assignment, the control group will have – on average – the same characteristics as the treatment group. Thus, the only difference between the two groups is the program, which allows evaluators to measure program impact and attribute the results to the project. Random assignment is a preferred impact evaluation methodology and, in the case of coffee, we use a cluster randomized controlled trial (CRT) to identify the impacts of specific activities of MAS+. Random assignment is not always possible. For the activities of MAS+ for beans producers we conduct an evaluation with a control group but no randomization design by comparing the outcomes among participants to the outcomes among comparable non-participants. Namely, a difference in difference analysis combined with propensity score methods to match participants to non-participants. 6 Both methodologies are explained in more detail in what follows. Randomized Control Trial Methodology: Coffee Producers Impact Evaluation Clustered randomized trials are a type of Randomized Controlled Trials (RCTs) in which groups of subjects, rather than individual subjects, are randomized in order to evaluate the impact of a particular intervention, policy or project. Succinctly, RCTs have become recognized as the best way to run ex ante impact evaluations as randomly selecting beneficiaries should imply that any improvement in outcomes is due to the program or project benefits, and not due to something else changing over time, since nothing else would affect the beneficiary group differently from the control group. Relative to individual level RCTs, CRTs reduce the risk of biases in the evaluation due to spillover and crossover effects between individuals and are a natural fit when the intervention is applied at the group level. When projects involve information sharing, community mobilization, access to trainings and extension through farmer groups, financial services provided by, e.g. village savings, they take advantage of the capacity communities (or farmer group) have to coordinate actions, disseminate and share information, and distribute resources. Therefore, a CRT is a natural fit for evaluating components of the MAS+. Several criticisms of RCTs have been advanced in both the economics and the more general program evaluation literature (Harrison, 2011; Ravallion, 2015; Deaton and Cartwright, 2017). One argument is that RCTs may not be ethical, as some potential beneficiaries are excluded from an intervention to measure its impacts. RCTs may generate unbiased estimates of average treatment effects for outcomes of interest, but they often lack information about how those outcomes came about. Finally, specific to agricultural projects, there are potential spillovers to neighboring areas. We plan to minimize these criticisms in the design of this RCT. First, we acknowledge that the MAS+ design is complex and it would not be possible to randomize access to the entire intervention; we therefore plan the impact evaluation around specific activities within the whole project. To ensure we understand how the whole project worked, the ongoing monitoring and the mid-term process evaluation will provide information about the effectiveness of other activities. Second, we randomize at the farmer group or community level so jealousies do not arise within communities based on who gets access to specific components, as would occur with individual randomization. Furthermore, since randomization generates unbiased estimates of the benefits of programs, in combination with cost data the use of randomization can help us understand what projects create better value-for-money. We note that external validity is always a threat to the proper interpretation of results from impact evaluations, whether randomized or not. We will test for impact heterogeneity, particularly with regards to gender and region. The latter might help us understand how interventions would work in other contexts. Finally, we note that there are specific difficulties related to agricultural interventions. There is a great deal of chance for spillovers to control groups. We plan to deal with these issues in two 7 ways; first, we will design the survey form to learn about potential spillovers to the control group. Second, we will ask for cell phone numbers of household members at baseline; in a previous IFPRI work tracking migrants, we found that phone numbers were the most effective way to track migrants. Propensity Score Methodology: Beans Producers Impact Evaluation Under the assumption that we can identify “treated” communities or areas reasonably well, we will apply propensity score methods to estimate impacts of MAS+ activities in bean growing areas. Through comparisons with experimental estimators, Heckman, Ichimura and Todd (1997, 1998) show that propensity score matching can provide reliable, low-bias estimates of program impact, under the following assumptions: (i) the same data source is used for participants and non￾participants, (ii) the data include meaningful explanatory variables capable of identifying program participation and outcomes, and (iii) participants and non-participants have access to the same markets. This logic can be extended to any estimates based on either covariate matching or reweighting estimators. To measure impacts of MAS+ in the beans value chain we explicitly make the following assumptions. First, we assume that the households in the control group are similar to participating households. Second, the treatment and control groups have access to roughly the same markets (as above), and we assume that the explanatory variables that are included can identify hypothetical program participation among the control group. Note that we use retrospective variables for estimating propensity scores, so we need to assume that any recall biases that exist are similar between the treatment and control groups for the period under study. Third, we assume that households in the control group contain enough households that would have been group members to make meaningful comparisons between the treatment and control groups. Fourth, we assume there are no spillovers from the treatment to control groups, which could be an innocuous assumption, but may not be if growing techniques spread rapidly. Unlike an RCT, in which the treatment and control groups are randomly selected, in using propensity score methods, the goal is to find control observations that are as similar as possible to project participants but did not have access to the program. The matching of control observations to treated observations must take place with observable variables, and then we must assume that unobservable characteristics are also roughly equivalent between the treatment and control groups. This assumption is quite strong; in this context, an important challenge will be finding control group observations that would have participated in trainings had they been offered, since training groups will not be set up in villages in the control group. Second, matching methods (including PSM) can be criticized as being easy to manipulate, based on the covariates that go into the matching process. To ensure that matching is not manipulated, we will use an algorithm to choose variables to include in propensity scores (Hirano and Imbens, 2002; Imbens, Newey, and Ridder, 2005). For the algorithm to function properly, it needs a rigorously collected list of baseline characteristics. 8 To further to improve the design we ensure that the control group is comparable to the project beneficiaries using a purposely selected baseline to identify non-participants and construct a control group that is similar to the participants at the start of the project. The method to select the baseline for the beans producers involves matching of just two sampling units (clusters) to construct matched pairs. With this approach, the number of treatment and comparison units is the same and the selection of individual items is correlated, since either both members of a matched pair are included in the sample, or neither is. Not only is the selection event correlated (among sampling units), but the two items in each cluster pair are highly correlated with respect to the matching variables. In addition, to account for external selection bias given that the projects purposively target particular household types, in the evaluation we compare the changes – instead of the level – of a given indicator between the group of participants and the control group. Assuming the change in the indicator in the control group is a good representation of what the change in the indicator would have been among the beneficiaries, this “difference-in-differences” estimate can provide a valid way to neutralize the external selection bias and hence provide an unbiased assessment of the project’s impact. 4 Data Sources Small farmers often keep no records and much information collected about agricultural production activities depends on farmer recall. However, the easiest and often least expensive way of obtaining agricultural production data is to simply ask farmers and other producers directly. The evaluation and this report main source of information comes from personal interviews with farmers. Below we describe the instrument and platform used in data collection, and statistical survey design. 4.1 Survey Design IFPRI designed the survey instrument, in consultation with both TechnoServe and the local survey firm that was contracted to conduct the survey by TechnoServe, ESA Consultores. The software application to administer the questionnaire through computer-assisted personal interviews (CAPI) was written by ESA Consultores. The survey instrument was translated into a CAPI application in the software package Survey Solutions. This software was chosen by its flexibility to handle complex modules, which facilitated the order and flow of the interview. The software includes all necessary crosschecks and directions of how to answer questions for the enumerators, i.e. help screens and error messages are coded so that information can be corrected on the field. This electronic platform supports complex skip patterns and has simple questionnaire navigation capabilities. The platform also automates error-prone aspects of data collection, such as the filling in of ID codes or selecting at which level would each section be asked. Finally, this platform allows the capture and integration of non-traditional data, such as audio-visual media and geographical coordinates. The finalized surveys are validated in extenuating detail by the time the supervisors see them, and then they go to the server after a final check. IFPRI made a thorough review of the survey application before fieldwork to ensure that the CAPI application minimize errors in the data 9 collection stage, maximize the internal consistency of the questionnaire and provided opportunities to identify and correct errors while in the field. The content of the questionnaire revolves around 3 themes:  Agriculture Production and Commercialization. Measuring the production, productivity, and commercialization of coffee and dry-beans  Extension and Technology Practices. Measuring the prevalence of agricultural practices of coffee and dry-beans, participation in extension programs, and access to other sources of agricultural information.  Producer Association and Credit Access. Measuring and its participation in community groups, communal activities, producer associations, participation in the credit market, and household savings. The questionnaire design benefited from IFPRI’s experience in Honduras, from the inputs from the monitoring and evaluation team of TechnoServe, and the inputs of ESA Consultores. The questionnaire adopted was the product of this exchange The survey instrument includes the following survey modules:  Module 1: Household composition, education and demography  Module 2: Dwelling characteristics  Module 3: Land access: Coffee  Module 4: Coffee production and commercialization o Inputs and labor o Agricultural technologies and practices o Pests o Extension  Module 5: Land access: Beans  Module 6: Beans production and commercialization o Inputs and labor o Agricultural technologies and practices o Pests o Extension  Module 7: Agriculture production and commercialization of other crops  Module 8: Investments, assets, social organizations and collective action  Module 9: Access to credit markets  Module 10: Opinions, perceptions and access to information  Module 11: Off-farm labor  Module 12: Transfers and remittances Below we briefly summarize the objectives of each module. Cover 10 The objective of this section is to identify each interview with a unique sample frame identification number, the geographic location, and directions to arrive to the household. GPS coordinates are collected for each household. The GPS coordinates allow us to locate the household and obtain other geographic information. Consent An informed consent script was read to the survey respondent explaining the main objective of the survey, the voluntary nature of it, that the information would remain strictly confidential, and that the information would be used only for research purposes. Farmers were asked if they agreed to participate and provided a verbal consent, which was recorded by the software. Module 1: Household composition, education and demography Collects information on the household structure, determine household members, and the education level of these members. We also obtained this demographic information and education levels for non-household members in charge of crop management. Module 2: Dwelling characteristics Collects information about the living situation of the household. We obtain information on the value of the house where they live and household related expenses; as well as characteristics of the dwelling. These include: materials use on the floors, walls and ceiling, access to electricity, potable water and energy use. Module 3: Land access for coffee Collects information on the land ownership for coffee producers. It provides general characteristics of parcels, like accessibility to roads and markets, irrigation status, size, etc. Module 4: Coffee production and commercialization Collects information about coffee production, commercialization, practice and techniques, pests and training and extension. It is exclusively asked of coffee producers. Sub-Section 4.1: Coffee production Collects information on coffee production at the variety level. For each variety planted, it gathers information about: total area, area in plantía, area in commercial production and area in recepa, plant density, age of plants, and total production. It also gathers information on losses, both field losses and post-harvest losses and productivity expectations under good, regular and bad conditions. Sub-Section 4.2: Coffee sales and inputs Collects information about the total number of sales in the last 12 months. Once the number of cuts a year is obtained, detailed information about the buyers, quantity sold per type of processed coffee, value of the sale, timing, contract, quality and price is obtained 11 at the cut level. We also gather information about expectations of sales under good, regular and bad market conditions, labor use and input use. Sub-Section 4.3: Coffee practices and techniques Collects information about important practices targeted by the MAS+ program and suggested by the TechnoServe technical team. These practices are divided by the activity level and include: plot management, harvesting practices, post-harvest practices and coffee quality. Sub-Section 4.4: Pests in coffee Collects information about the main pests that affected the coffee crop during the last 12 months. For each pest, collects information about estimated losses, control measures, cost of control measures and if the farmers received any help to prevent such pests. Sub-Section 4.5: Participation in coffee training and extension Collects information about the participation of household members in extension programs or training, on or off the farm. It inquiries about the entity that provided extension, costs, specific practices learned, frequency and use of such information learned. Module 5: Land access for beans Collects information on the land ownership for bean producers. It provides general characteristics of parcels, like accessibility to roads and markets, irrigation status, size, etc. Module 6: Beans production and commercialization Collects information about beans production, commercialization, practice and techniques, pests and training and extension. It is exclusively asked of beans producers. Sub-Section 6.1: Beans production Collects information about beans production at the variety level for each season (primera 2017 and postrera 2017-2018). For each season, it obtains information for varieties planted, area planted and harvested, total production, sales, price and production expectations under good and bad conditions. Sub-Section 6.2: Beans sales and inputs Collects information about sales in the last 12 months the buyer, and expectation of prices under good and bad market conditions. It also collects information labor use and input use and costs associated. Sub-Section 6.3: Beans practices and techniques Collects information about important practices targeted by the MAS+ program and suggested by the TechoServe technical team. These practices are divided by the activity level and include: planting, harvest and post-harvest practices. 12 Sub-Section 6.4: Pests in beans Collects information about the main pests that affected the bean crop during the last 2 agricultural seasons. Then for each pest, it asks specific questions, such as estimated losses per season, control measures, cost of control measures and if the farmers received any help to prevent such pests. Sub-Section 6.5: Participation in beans training and extension Collects information about the entity that provided extension, the training or extension cost, specific practices learned in such training activities, frequency and use of the information learned, post-harvest, quality and certification, marketing, finance and contracts and market access). Module 7: Production and commercialization of other crops Collects information about all other crops planted. For each crop the module collects the number of harvests in the last 12 months, the value of production, whether it was consumed by the household, and the sales value. Module 8: Investments, assets, social organizations and collective action Collects information about the main investments and equipment of the household. In addition, this section asks about each type of organization that exists in the community, the level of involvement in such organizations, and the main benefits they obtain from such organizations. Module 9: Access to credit markets Collects information about the sources of credits used by households, the degree of participation in the credit market, and household savings. Access to credit is of paramount importance for agricultural production and the household economic welfare. We do not limit question to formal credit; we include informal sources, input, services/machinery, and other assets that the members received against a promise to pay in the future- be it in product, cash, or labor. Module 10: Opinions, aspirations and information and communication technology Elicits the opinion of the interviewee with respect to agricultural technologies and events, and the diffusion of new techniques in the household’s social network, farmers’ perceptions of aspirations for the future, self-esteem and locus of control. We end the section asking about sources of agricultural information and use of communication technologies. Module 11: Off-farm labor Collects information about the productive activities of each individual in the household over 12 years old. It serves to know the different sources of income of the household besides agricultural income, both salaried and self-employment activities. Module 12: Transfers and remittances 13 This section asks about the main sources of transfers and remittances, frequency of transfers, amounts transferred and use of such transfers. Back Cover Collects information about the result of the survey, the personnel that participated, and any further comments from the enumerators and the supervisors. The survey instrument is provided as an attachment/appendix to this document. 4.2 Data Collection: Training, Pilot and Fieldwork The Baseline Survey was conducted between July and August of 2018. IFPRI researchers, jointly with ESA Consultores, conducted a 14-day training between June 4th to June 17th: 7 days with field managers and groups’ supervisors and the remaining 7 days with the whole team of enumerators to review agricultural concepts facilitated by TechnoServe. Enumerators and supervisors alike were evaluated to be able to be part of the final team of interviewers. The training also included 2 pilot activities with both coffee and beans farmers; these pilot farmers were in areas outside of the impact evaluation survey sample and were not anticipated to be part of the control or treatment groups. Pilot activities were supervised by ESA, IFPRI and TechnoServe teams. These pilot activities were useful to refine the survey instrument, collect farmer’s impressions, refine technical questions, adjust the interview length, and further evaluate enumerators interviewing skills. Training was based on the following topics:  Introduction to the survey: survey objectives, sample, survey modules, survey implementation, confidentiality, interviewer’s role, assignment to supervisors  Conducting the interview: giving general guidance, approaching the household, building rapport, converting refusals, obtaining informed consent, ensuring privacy, asking questions, probing, following interview instructions on the questionnaire and tablet, noting differences between the printed questionnaire and tablet screens, and flagging issues to be discussed with the Field Supervisor  Questionnaire content: household roster; informed consent; dwelling characteristics; improved agriculture technologies and productivity, etc.  Fieldwork procedures: following field team members’ roles and responsibilities, managing the household interview, reporting to the field supervisor, following up missed interviews, ensuring high-data quality, and monitoring and review of interviewers’ performance  Entering and managing data on the tablet: understanding the tablet and screen components, starting a questionnaire on the tablet, navigating the questionnaire, advancing through survey modules and groups, entering responses, dealing with refusals, troubleshooting, transmitting data  Completing survey modules: knowledge of general instructions, administering each survey module, asking questions, and entering responses question by question 14  Improved agriculture technologies: understanding improved agriculture technologies, measuring crop productivity At the conclusion of the interviewers’ training, the entire field team conducted a pilot test in rural communities and included individuals who are similar to the planned survey respondents. The pilot areas were not part of the sample. At the end of the pilot, everyone participating in the pilot test met to discuss issues and challenges and to identify solutions. Issues with the survey instrument and program were communicated to IFPRI and corrections were made, documented, and tested before the fieldwork started. Revisions to the instrument and data entry program and other preparations for fieldwork took several days, so there was a brief period between the pilot and the initiation of fieldwork. Interviewers worked in teams of 5: 4 interviewers and 1 supervisor, for a total of 8 teams (mixed female and males). The teams covered equally both the coffee and beans samples. Two additional field supervisors were in charge of visiting different areas, homogenizing the protocol across teams and providing general coordination. The ESA team was also responsible for reviewing the quality of the data. ESA delivered weekly reports to the IFPRI team and TechnoServe. Throughout the data collection, IFPRI provided timely responses to issues found in the field to ensure homogeneous protocols across survey teams. 4.3 Statistical Power and Sample Selection The evaluation will primarily use two surveys: a baseline survey before the beginning of activities and endline survey after the conclusion of the MAS+. The baseline survey will serve two purposes. First, it provides a representative baseline estimates for measurement of performance indicators that will be measured among beneficiaries throughout the project. Second, it provides a baseline for impact estimates, which will compare treatment groups to a control group of non-participants. As such the sample design follows an analytical survey design. The purpose of an analytical survey is to develop detailed models that describe the relationship of dependent variables to a variety of explanatory (independent) variables (including treatment variables). The principal objective of the surveys is to provide high precision for estimates of the difference-in-difference estimates of program impact. We designed the sample to target an increase in coffee yields of 24% (40% among participants, at a 60% participation rate) and bean yields of 21% at the 5 percent significance level and 80 percent power, through the baseline and endline surveys. Assuming sampling clusters with an average size of 25 producers; a 20 percent of the variance in the endline outcomes using covariates collected at baseline, which includes the baseline outcome in an ANCOVA model (e.g. McKenzie, 2012); and intracluster coefficient of 0.05; we estimated the minimum detectable effects for coffee and beans yields. In Table 1 we find that for bean yields, the minimum detectable effect is 21% yield increase with a sample of 500 farmers per treatment condition; for coffee, we find that with below 12 clusters per treatment the difference could be detected with 80% power. However, for small numbers of clusters (e.g. less than 30) 15 standard t-tests are likely to over-reject the null hypothesis for a given level of statistical significance, and require adjustment (e.g. Cameron and Miller, 2015). As a result, we included 20 clusters per treatment arm in the coffee sample, so the minimum detectable effect improves and drops to 19 percent of the mean. The resulting sample is 40 total clusters per treatment arm; the minimum detectable effect on the logarithm of the value of agricultural output is then about 19 percentage points. TABLE 1 PROJECTED BASELINE SAMPLE CHARACTERISTICS AND MINIMUM DETECTABLE EFFECT Variable Mean Standard Deviation (after 20% variance reduction) Clusters per Treatment Condition Size of Clusters Total Sample Size Minimum Detectable Effect Achievable % of Mean Bean Yields 18.1 12.2 N/A N/A 1000 3.9 21.5 Coffee Yields 63.9 29.1 40 25 1000 12.1 19.0 Notes: Yields in quintales per hectare (qq/ha). At the time of sample selection, and due to constraints in obtaining a sample frame or listing of all farmers in the treatment and control clusters from where to select the treatment and control sample producers, the effective sample varies slightly from the number of clusters and the size of each cluster that was initially proposed. Table 2 and Table 3 show the sample distribution in the initial sample design for coffee and beans, respectively, at the start of the fieldwork. In the survey design, we set out to interview 500 producers in 20 clusters in each treatment condition and for each commodity. Treatment and control producers were selected from the list of potential participants provided by TechnoServe when available. This was the case for all treatment and control farmers in the case of coffee. For the coffee sample we listed 550 control coffee producers in 40 clusters and 572 treatment coffee producers in 38 clusters (with additional pre-selected farmers as replacements to reduce attrition). In the case of beans, we could not obtain the list of farmers in advance of the fieldwork. Control farmers in the beans sample that were not listed at the beginning of fieldwork were selected randomly by IFPRI from a listing of beans producers obtained before visiting the cluster. For the beans sample we listed 71 control beans producers in 34 clusters and 503 treatment beans producers in 34 clusters prior to data collection (with additional pre-selected producers as replacements to reduce attrition). During data collection, more complete lists of farmers were developed in control areas; data were then collected among the more complete list within the 34 clusters. 16 TABLE 2 COFFEE SAMPLE DESIGN Control Treatment Department Producers Replace￾ments Total Clusters Control Producers Replace￾ments Total Clusters Treatment COMAYAGUA 168 20 188 12 164 26 190 11 EL PARAÍSO 187 23 210 14 122 15 137 8 FRANCISCO MORAZÁN 30 2 32 2 LA PAZ 10 10 1 15 3 18 1 OLANCHO 129 9 138 9 166 20 186 11 YORO 56 4 60 4 75 6 81 5 Total 550 56 606 40 572 72 644 38 TABLE 3 BEANS SAMPLE DESIGN Control Treatment Department Producers Replacements Total Clusters Control Producers Replacements Total Clusters Treatment EL PARAÍSO 29 29 5 69 12 81 5 FRANCISCO MORAZÁN 15 15 2 30 6 36 2 OLANCHO 16 16 16 240 38 278 16 YORO 11 11 11 164 26 190 11 Total 71 71 34 503 82 585 34 Survey Response Rates Table 4 shows the distribution of the completed surveys and the response rates by department for coffee in panel A and for beans in panel B. The overall attrition was low, with above 95 percent completed in both the coffee and beans sample. The table shows the difficulty we had identifying control producers in the beans samples in some departments and that we recuperated some of the sample by selecting additional clusters in other departments. For example, in Yoro the coffee farmers listed were re-assigned to Olancho, thus the response rate is above one. 17 TABLE 4 COMPLETED SAMPLE AND RESPONSE RATES Control Treatment Completed Expected Response Rate Completed Expected Response Rate Panel A: Coffee Sample COMAYAGUA 151 168 0.90 153 164 0.93 EL PARAÍSO 163 187 0.87 110 122 0.90 FRANCISCO MORAZÁN 22 30 0.73 LA PAZ 9 10 0.90 13 15 0.87 OLANCHO 150 129 1.16 190 166 1.14 YORO 0 56 0.00 0 75 0.00 Total 473 500 0.95 488 500 0.98 Panel B: Beans Sample EL PARAÍSO 66 57 69 0.83 FRANCISCO MORAZÁN 22 33 30 1.10 OLANCHO 228 240 240 1.00 YORO 159 163 164 0.99 Total 475 500 0.95 493 503 0.98 Note: The total “expected” is not equal to the sum by department as we listed (“expected”) more potential households to be able to arrive at the 500 interviewed farmers per treatment condition. Effective Minimum Detectable Effects Given the changes in the sample and the difference in the survey instrument used in the surveys, it is important to verify the minimum detectable effects (MDE) within the realized sample. The power analysis using estimated means and variances from the survey is presented in Table 5. The MDE is 25% of the mean for the beans yields and 30% of the mean for the coffee yields. Compared to the initial sample design above (21% for beans and 19% for coffee) these are larger and highlight the need to maintain low attrition in the endline and ensure that the production and area measurements are of good quality. In summary, the results show that the sample design is well powered to detect economically significant differences in yields across the treatment and control groups in the sample. TABLE 5 MINIMUM DETECTABLE EFFECTS OF SAMPLE Variable Mean Standard Deviation Clusters per Treatment Condition Avg Size of Clusters Total Sample Size MDE Achievable MDE after (after 20% variance reduction) % of Mean Bean Yields (Primera) 12.73 8.6 37 T/39 C 12.7 968 4.06 3.2 25% Bean Yields (Postrera) 9.31 6.4 37 T/39 C 12.7 968 3.02 2.42 26% Coffee Yields 45.78 36.22 38 T/39 C 12.6 961 16.97 13.55 30% Notes: Coffee yields in qq/ha wet parchment; for the MDE calculation the standard deviation is the important quantity. 18 5 General Characteristics of the Intervention Area Honduras is the second poorest country in the Western Hemisphere. In 2017, it reported a poverty rate of 64.3 percent below the national poverty line, according to national official records. Two out of three people are poor in the country poverty is more severe in rural areas than in urban areas. In 2013, 46.6% of the population or 3.8 million people lived in rural areas2 . With a poverty rate of 69.3% in rural areas, this is 2.6 million people living in poverty in rural areas. Comayagua, El Paraíso, Olancho and Yoro, the departments where most of the MAS+ beneficiaries are concentrated, are home to approximately 1.5 million of such families living in poverty. Figure 1 shows the geographic distribution of the sample. FIGURE 1: COFFEE AND BEANS SAMPLE GEOGRAPHIC DISTRIBUTION 2 XVII Censo de Poblacion 2013, INE(http://www.ine.gob.hn/) 19 In 2017, 40.7% of the country’s population lived in extreme poverty. In rural areas, with a poor population being 58.8% many farmers to afford the basic food basket3 . Poor households in rural areas are further below the poverty line than in urban areas. According to the last National Census, in 2013, 54.3% of the population had unsatisfied basic needs. This share has barely decreased since the previous census in 2001, where the share of the population with unsatisfied basic needs reached 55.4% of the population. Since 2001, there has been a decrease in the share of the population that has no access to most basic needs or services: access to water, sanitation, access to basic education and overcrowding. But there are two particular necessities that have seen an increase in the share of the population that has no access to these services: the share of the population without subsistence capacity (which increased from 17 to 30% from 2001 to 2013) and the share of the population with dwellings in bad conditions (which increased from 0.5 to 9% from 2001 to 2013). A third of the country’s population is employed in the agricultural sector. Agricultural households in the area of intervention cultivate three main traditional crops: maize, beans and coffee on small plots, often on hillsides. The country is also vulnerable to national disasters, such as hurricanes and droughts, and the area of intervention is particularly characterized by little rainfall and variable climatic conditions. The poor economic circumstances of the area condition its people to struggle under such variable environments and do not allow them to mitigate shocks with risk management mechanisms, such as credit or insurance. The lack of resources constraints people in the area to invest in improved technologies, such as irrigation systems, that will also allow people’s resilience to environmental shocks. The use of traditional cultural practices produces poor yields, which leads to poor income and nutrition, and depletes soil of nutrients, which also leads to environmental degradation. On top of this, farmers’ access to markets is hindered by poor roads and long distances. The coffee sector is also vulnerable to external shocks, such as the declining of international coffee prices. 3 INE, LVIII Encuesta Permanente De Hogares de Propósitos Múltiples - EPHPM, June 2017, www.ine.gob.hn 20 6 Analysis of Coffee Producers in MAS+ Evaluation In this section, we describe the sample of coffee growers. After describing the households demographically, we examine agricultural statistics among households related to coffee. We include balancing tests throughout the themes in these sections to show the validity of the RCT design for the coffee samples and that the evaluation design balances important outcomes across treatment and control groups. 6.1 Demographic Characteristics First, we examine demographic characteristics among the sample in Table 6. The average household has 4.77 members, distributed as follows. On average, the households include 0.69 members aged 5 years old or under, split nearly evenly by gender. Members include 1.1 boys and girls combined, on average, aged between 6 and 14 years old (Table 6). Most other members are working age adults (15 to 64 years old), as there are only 0.2 elderly members on average. There are fewer working aged women on average than working aged men in coffee sample households, but otherwise households are relatively gender balanced. TABLE 6: DEMOGRAPHIC CHARACTERISTICS FOR COFFEE SAMPLE Mean SD Min Median Max Observations Number of people living in the house 4.77 2.12 1 5 14 961 Boys under 6 0.35 0.64 0 0 5 961 Girls under 6 0.34 0.62 0 0 6 961 Boys between 6 and 14 years old 0.52 0.92 0 0 11 961 Girls between 6 and 14 years old 0.58 1.43 0 0 14 961 Men between 15 and 64 years old 1.18 0.71 0 1 3 961 Women between 15 and 64 years old 0.88 0.42 0 1 3 961 Men older than 65 0.11 0.32 0 0 2 960 Women older than 65 0.09 0.29 0 0 2 961 People who manage agricultural activities within the coffee sample range from 11 to 87 years old (Table 7). On average, the primary farmer is 41 years old. About 42% of people working in agriculture in the coffee sample are women. The average number of years of education is 5 years; most farmers have completed primary school. The literacy rate is 85%. Looking at the same characteristics by gender (Table 8), we observe that women are just a little older on average than men (42.5 years old as opposed to 40 years old for men). Average years of education and literacy ratios for both genders are about the same. 21 TABLE 7: DEMOGRAPHIC CHARACTERISTICS Age Female Years of Education Literacy Mean 41.13 0.42 4.67 0.85 SD 16.11 0.49 3.30 0.36 Min 11 0 0 0 Median 40 0 6 1 Max 87 1 18 1 Observations 2198 2197 2197 2198 Note: These demographic characteristics have been calculated for all people in charge of agricultural decisions in the 961 coffee households. TABLE 8: HUMAN CAPITAL CHARACTERISTICS, BY GENDER Male Female Age Years of Education Literacy Age Years of Education Literacy Mean 40.16 4.68 0.85 42.48 4.65 0.85 SD 17.14 3.28 0.36 14.45 3.33 0.36 Min 11 0 0 14 0 0 Median 39 6 1 40 6 1 Max 87 18 1 85 17 1 Observations 1280 1280 1280 917 916 917 Note: These demographic characteristics have been calculated for all people in charge of agricultural decisions in the 961 coffee households. Besides asking about income earned from coffee sales (analyzed below), we asked farmers if they had other sources of income. We use this information to calculate total household monthly income and estimate what percentage coffee sales represent out of total household monthly income. We find that income earned from coffee sales represents on average 69% of household income (Table 9). It is worth mentioning that these shares are only estimated for households reporting positive earnings from either agricultural or non-agricultural activities (907 farmers). TABLE 9: AGRICULTURAL AND NON-AGRICULTURAL INCOME Monthly Non-Agricultural Income in USD Monthly Agricultural Income in USD Share that Coffee Income represents from Total Income Mean 448.74 267.21 0.69 SD 1261.02 570.36 0.37 Min 4.10 0.03 0.00 Median 147.60 124.76 0.98 Max 14760.00 10284.17 1.00 Observations 478 886 907 Note: Only 478 farmers from the coffee sample report earning income from other non-agricultural activities; 886 farmers report earning income from selling coffee to the market. The share that coffee income represents from total income is calculated only for farmers earning positive amounts from either of both sources (907 farmers) 22 6.2 Agricultural Outcomes We next examine agriculture practiced by sample households, focusing on coffee production. First, we examine the amount of land being cultivated in coffee, before discussing production and yields. We then discuss coffee sales in the past 12 months. Sample for coffee consists of 961 farmers; across the different estimates the sample may vary because of non-response, non￾applicability or the answer were considered extreme outliers. Size of Cultivated Land Measures of area are fundamental components of agricultural statistics, as they are required for calculating many indicators of productivity. Ideally, measures of both production and area should be highly accurate. However, errors in the denominator (area) magnify any errors in the numerator (production); thus, accurate measures of area are arguably more critical to minimizing potential errors in calculating agricultural yield. As many farmers in developing countries have no real means of accurately determining how much land they use to produce crops or other agricultural products, accurate measures of area can be difficult to obtain. The relevant measure for area is the area planted (cultivated), rather than the area harvested, or owned. This is an important distinction since not all parts of a field or farm that are planted will necessarily produce any yield or be harvested. Area cultivated is determined by farmer estimates (self-reported) of how much area they cultivate in the previous 12 months. Table 10 shows the estimates of land use for coffee production. The average household has 2.3 hectares (ha) planted with coffee; there is substantial variation in the total area planted, with the standard deviation slightly larger than the mean.4 The median household in the sample has 1.4 ha planted with coffee, so the average is skewed by some larger farms. For area planted with coffee either in plantía, recepa, or in commercial production, we present conditional averages; only non-zero values. Note that not all households have coffee currently in commercial production; 54 surveyed farmers fit this category. We estimated that the area planted with coffee in commercial production is 1.9 ha, which is lower than the total area planted. About half of households have coffee in plantía, and average area planted in plantía is 1.9 ha. Medians are somewhat lower, suggesting that there are some households with larger holdings. 4 Most of the responses in the survey were given in manzanas, a standard unit of land area in Honduras; a Honduran manzana is equivalent to approximately 0.7 hectares. 23 TABLE 10: SIZE OF PLANTED AND CULTIVATED AREA FOR COFFEE SAMPLE Total area planted with coffee (in HA) Area planted with coffee in plantía (in HA) Area planted with coffee in commercial production (in HA) Area planted with coffee in recepa (in HA) Mean 2.3 1.1 1.9 0.9 SD 2.9 2.1 2.2 0.9 Min 0.0 0.0 0.0 0.1 Median 1.4 0.7 1.4 0.7 Max 55.7 27.8 31.3 6.3 Observations 925 506 871 150 Notes: The Total sample for coffee consists of 961 farmers. Differences in sample sizes for total area are due to 3 farmers that do not report area planted and observations that were determined to give extreme outliers in the yield estimation. All other differences are due to farmers that have all their plantation either in plantía, in commercial production, or for recepa, but not all of them together. The modal and median farmer has one plot of coffee; as some farmers have more than one plot, the average is 1.7 plots across the sample (Table 11).5 A few farmers have substantially more coffee plots. Coffee farmers in the sample do not all have title to their plots; rather, farmers of less than half of the plots in the sample have title, which might affect their access to finance. TABLE 11: NUMBER OF TOTAL PLOTS AND OWNED PLOTS FOR COFFEE SAMPLE Number of plots for coffee Number of plots that are owned with property title Mean 1.71 0.78 SD 1.12 1.12 Min 1.0 0.0 Median 1.0 0.0 Max 10.0 9.0 Observations 961 961 Note: Coffee sample consists of 961 farmers. All farmers have at least one plot where they plant coffee. Coffee Production Asking farmers to estimate their total production is perhaps one of the most convenient and least expensive ways to gather data on agricultural production. It is often employed through surveys, relying on the ability of farmers to remember how much they might have harvested of a crop or from a plot. The accuracy of production estimates from farmer recall varies tremendously; evidence of farmer error has been based primarily on differences between farmer estimates of production and those calculated with individual measurement approaches. Additionally, if there is 5 The average landholdings are 2.3 hectares and there are 1.7 plots held per farmer, the average plot size is 1.35 hectares. 24 not bias in farmers’ recall, the mean from a large enough sample is an unbiased estimate of the true mean. In the absence of bias or if bias is constant over time, recall estimates can provide accurate estimates of change over time. Farmers may express production in local units, which must then be converted to standard units such as pounds, kilograms, etc. Errors easily accumulate through multiple conversions and rounding. The information was collected using the units used by the farmer and selected with from a pre-loaded list of units that can be accurately converted to standard units to minimize errors and ensure that the units could be accurately converted in the analysis. In the analysis that follows we convert all other weight measurements to quintales (qq) (1 quintal = 100 lbs= 45.359 kg). Coffee can provide multiple products and depending on the stage of the processing have different weight and densities. For example, coffee may be sold as either berries, wet or dried, etc. To measure coffee production, we standardize all measures to pergamino húmedo or wet parchment using conversion factors proposed by IHCAFE (Instituto Hondureño del Café) and Technoserve that adjust for the humidity and processing in the weight. The following conversion factors are used:  100 lbs café uva or coffee in berries = 43.7 lbs wet parchment  100 lbs café uva or coffee in berries = 21.53 lbs dry parchment  100 lbs pergamino seco = 81.92 lbs green parchment All production values for each processing level were converted to quintales (qq) and then processing levels were all converted from quintales to quintales in pergamino húmedo or wet parchment to obtain a standardized measure of production for coffee. For example, if the respondent said total production was 50 cargas in uva or berries, we convert this to 100 qq in berries, since each carga has 2 qq; then 100 qq in berries is equivalent to 43.7 quintales in wet parchment (qq WP) using the conversion factor above. In practice, most farmers selected the standard unit of measurement quintal and wet parchment as the units of production and presentation, as shown in Table 12. This minimizes the effect of possible errors in the conversions. TABLE 12 UNITS AND PROCESSING LEVELS SELECTED BY RESPONDENTS Units Processing Level Freq. Percent Freq. Percent Quintal 1,347 93.0 Uva/Berry 138 11.1 Carga (2 quintales) 24 1.7 Wet parchment 862 69.2 Libras 6 0.4 Dry parchment 67 5.4 Arrobas 1 0.1 Oro/Green 179 14.4 Cans/Latas (sp. no. Per quintal) 54 3.7 Other (sp. no. Per quintal) 16 1.1 Total 1,448 100 Total 1,246 100 Notes: Observations at the variety level; each household can have multiple varieties. 25 Total production is defined as the amount that is produced, regardless of how it was ultimately used. It also includes any postharvest loss (i.e. postharvest losses are not subtracted from total production). All production quantities are considered only for farmers having positive plantía and commercial production or field or post-harvest losses. To identify outliers, we processed the production and area information to compute yields and then identified yields with implausible values given the observed distribution. Once we identified these observations, we also excluded those observations from the average area and average production estimates (fewer than 10 observations on the top of the distribution of yields were excluded). Table 13 shows coffee production in quintales of wet parchment coffee. We estimate total production, production in the area designated as plantía, production in the area designated for commercial production, and then losses in the field and post-harvest. The spread of total production is consistent with the skewed distribution we observe in area planted. Total production at the median is 34.3 qq, and commercial production is 31 qq at the median; almost half the estimated mean, showing that there is a long tail over high production values after excluding extreme outliers. Production itself is quite variable, with standard deviations above the average. Finally, it is notable that only a small share of households report post-harvest losses; even then, average post-harvest losses are small or around 10 percent. Note that as the production estimates are all conditional (they must be greater than zero), so the subcategories do not add up to the total mean in the first column. TABLE 13: COFFEE PRODUCTION Coffee production in qq WP, total Coffee production in qq WP, plantía Coffee production in qq WP, commercial production Coffee production in qq WP, field losses Coffee production in qq WP, post harvest losses Mean 70.4 27.4 65.5 7.2 7.6 SD 125.0 49.5 117.8 13.4 16.8 Min 0.3 0.1 0.1 0.1 0.1 Median 34.3 10.0 31.0 2.6 2.6 Max 1,529.3 300.0 1,376.3 152.9 179.2 Observations 925 177 873 239 179 Notes: See text for conversion procedure. All figures are in qq and húmedo/wet processing level. Sample for coffee consists of 961 farmers. All production quantities are considered only for farmers having positive plantía and commercial production or field or post-harvest losses. Outliers above the 99th percentile in yields were excluded from the production and area averages, and categorized as extreme outliers. Coffee Yields Agricultural commodities measure of the total output of production of an agricultural commodity relative to the amount of land used to produce it. Yield is calculated as the ratio of the total production in quintales of wet parchment coffee and the total area planted in hectares (qq WP/ha) and is a measure of productivity from that farm or producer. 26 Table 14 shows some sample statistics of yields; the average productivity is 30.9 qq/ha WP, with a median of 21 qq/ha WP. Evidenced by the difference between the mean and the median, it is important to analyze the shape of the yield distributions. Figure 2 shows the distribution of the yields where we can appreciate that most farmers are below 50 qq/ha WP6 . Note that once we divide production by cultivated land, the standard deviation drops below the mean indicating the yield distribution is less spread out than the production distribution. The figure shows both the household level distribution and the distribution of the yields at the variety level (for households with more than one variety). We find lower yields both in commercial area and in plantía; yields in plantía are more variable in general. These yields are consistent with averages measured by IHCAFE, which estimates 85 qq/ha WP in Comayagua and 61.5 qq/ha WP in El Paraiso, for example. TABLE 14: COFFEE YIELDS Total yields (qq/ha WP) Yields in commercial area (qq/ha WP) Yields in plantía (qq/ha WP) Mean 30.9 36.8 23.0 SD 30.1 36.7 26.6 Min 0.3 0.4 0.2 Median 21.0 23.9 12.9 Max 157.2 201.6 123.1 Observations 925 861 177 Notes: See text for conversion procedure. All figures are in qq and húmedo/wet processing level. Sample for coffee consists of 961 farmers. All production quantities are considered only for farmers having either positive plantía and commercial production or field/post-harvest losses. Outliers above the 99th percentile in yields were excluded from the production and area averages, and categorized as extreme outliers. 6 The last bars in the figure represent the Winsorized values. Winsorizing consists in decreasing the effect of outliers by smoothing the tails of the distribution by placing values above a value x at x or on the values below x sequentially to maintain the shape of the distribution. 27 FIGURE 2 DISTRIBUTION OF COFFEE PRODUCTIVITY To check whether the yields are reasonably sensible, we inspect the production values against area in Figure 3. If all yields were the same, we would see the production increase linearly with area with a strong correlation. However, we expect some dispersion from a line, based on differences in productivity of farmers and land, shocks, or other factors. We find a marked dispersion in total productivity, perhaps more than one would expect; however, commercial production appears to have a more linear pattern. In general, the yields look reasonable for self-reports. We observe some bunching around what heuristic estimates of plot sizes (e.g. around 1.4, 2 or 2.8 hectares, equivalent to reports of 2, 3 or 4 manzanas in the survey). Farmers are estimating the amount of land they have rather than knowing precise amounts, which will affect the denominator in the yields calculation. Abate et al. (2018), highlights that inaccurate estimates of land area can affect the estimated effects of extension programs on yields, so we will consider whether an alternative method of measuring land area is needed for the endline. Otherwise, the pattern is consistent with expectations. 28 FIGURE 3: COFFEE AREA AND COFFEE PRODUCTION, TOTAL AND FOR COMMERCIAL PRODUCTION 29 Coffee and Markets: Buyers, Sales and Prices We next look at some markets variables to better understand the agents that interact in the coffee markets represented in the sample. Table 15 shows the distribution mean of quantity sold to different type of coffee buyers. The principal message of this table is that most farmers sell to intermediaries and not exporters and in wet parchment and not green/oro or uva/berry for the most part. This indicator will be important in the future to gauge how successful was MAS+ linking farmers to exporters. Quantities sold are presented in metric tons (MT or 1,000 kgs.) and the prices are in USD per MT in each processing level. The aggregate or total price is calculated as the value of sales over the quantity sold, after converting to wet parchment units. On average farmers sold 4 MT WP to intermediaries and 7.8 MT WP to exporters, but there are few farmers in this group, indicating that big farmers are the few selling to exporters7 . TABLE 15 COFFEE BUYERS AND QUANTITY SOLD Who did you sell your coffee to? Quantity of coffee sold in Berries/ Uva, MT WP Quantity of coffee sold in Wet/Húmedo, MT WP Quantity of coffee sold in Dry/Seco, MT WP Quantity of coffee sold in Green/Oro, MT WP Quantity of coffee sold in Total MT WP Intermediary or Coyote Mean 2.6 3.1 3.7 10.0 4.0 SD 8.5 9.2 8.3 27.9 13.3 Observations 96 581 40 104 809 Exporter Mean 0.7 3.4 8.3 5.3 7.8 SD . 3.0 14.5 6.3 10.4 Observations 1 9 7 7 16 Other Buyers Mean 3.4 2.5 3.0 6.7 4.3 SD 1.5 2.9 3.4 13.0 8.2 Observations 4 48 8 24 73 We expect that after standardizing prices, coffee that has undergone more processing will sell for higher amounts. 8 In Table 16 we estimate the price that coffee farmers are receiving for each level of processing, and confirm that the price increases by processing level, reflecting the value added. 7 We note that the average quantity sold, approximately 4 MT, is above the 3.2 MT in the production data after converting the figures to WP units. This is due to the farmers that sell their coffee green and the conversion factors used. The figures could suggest that farmers are trading coffee but we doubt this is the case. In the endline, we will include questions to explore if farmers are also acting as intermediaries of others or aggregating their coffee. Further note that the average for all other types of processing (none/berry, wet, and dry) are consistent with the production values (around 3 MT). 8 The prices (calculated) presented are obtained by dividing the total sales by the quantity sold; as opposed to farmers reporting the price per ton or per qq they received. The distribution of reported vs. calculated prices are similar and we presented the calculated figures to directly link the sales values and quantities presented before. 30 Coffee sold through intermediaries in uva/berry is 332.60 USD per ton; coffee in wet parchment is sold at 1,322 USD per ton. In contrast, if sold in green/oro, the price per ton is 1,743.80 USD per ton. We note that the number of observations in some categories is small, since not all farmers sell to different buyers; thus, the price estimate for those categories is less precise, evidenced by the large standard deviation (SD) compared to the estimated average price. Converting all sale quantities to wet parchment unit to obtain an average price per wet parchment that included the conversion factors and price differentials across processing levels, the average price for coffee sold to intermediaries is 1,143.60 USD per ton9 . These estimated prices are interesting, in that farmers are receiving very different prices even within a specific processing level. We present Figure 4 with the calculated prices in USD per MT from the transactions reported by farmers in the 12 months before the survey. The figure shows that there is wide variation in prices depending on the state in which coffee was sold, and moreover that the distribution of the aggregate price in the sample is bi-modal; the first mode representing the prices of coffee sold in uva/berry and the second mode representing coffee sold in húmedo/wet. This indicates that there are two different prices and that the price that a farmer obtains depends on some other attributes of their production. Note that this also the case for those that sold as wet parchment, thus these differences not driven by the conversions due to humidity and density. TABLE 16 COFFEE BUYERS AND SALE PRICES Who did you sell your coffee to? Price of coffee sold in Berry/ Uva, per MT Price of coffee sold in Wet/Húmedo, per MT Price of coffee sold in Dry/Seco, per MT Price of coffee sold in Green/Oro, per MT Price of coffee sold in Total per MT WP Intermediary or Coyote Mean 332.6 1,322.9 1,437.9 1,743.8 1,143.6 SD 234.2 429.5 808.6 1,853.5 499.3 Observations 95 564 40 98 787 Exporter Mean . 1,381.8 1,718.9 2,014.1 1,031.6 SD . 386.3 515.5 1,116.1 398.2 Observations - 9 6 7 16 Other Buyers Mean 212.0 1,404.2 1,981.2 3,461.2 1,196.4 SD 27.7 470.3 698.3 4,742.1 536.1 Observations 4 48 8 23 72 Note: Prices are winsorized at the 2nd and 98 th percentile to decrease the influence of extreme outliers. Prices are calculated as the ratio of total sales and total quantity sold in each processing level. 9 This price is a weighted average of the prices received in each processing level, where the weights are a function of the proportion sold in each processing level and of the conversion factor to WP from each specific processing level. In the figures, this price is identified as “Total-Aggregate WP.” 31 FIGURE 4 DISTRIBUTION OF PRICES OF COFFEE: CALCULATED BY PROCESSING LEVEL Most coffee farmers reported sales in the last 12 months before the survey. In Table 17 we estimate the average value of total sales is 3,220 USD per year. Most sales are as wet parchment, and the median farmer sold 1,500 USD in the previous 12 months. Comparing production to sales, median production is 1.6 MT per year and the median quantity sold is 1.4 MT, for a 12.5 percent loss at the median (Table 18). Figure 5 shows the distribution of sales, with the right side of the tail winsorized to better see the spread. TABLE 17: COFFEE SALES: VALUE Sale value coffee Berry/Uva (USD) Sale value coffee WP (USD) Sale value coffee Dry/Seco (USD) Sale value coffee Green/Oro (USD) Total Sale value (USD) Mean 1,520.7 2,819.5 3,009.2 5,839.8 3,220.5 SD 4,071.5 3,659.5 6,519.4 14,671.4 6,886.5 Min 24.6 0.9 46.7 0.4 0.4 Median 476.6 1,640.0 1,102.7 1,482.7 1,499.8 Max 36,080.0 28,103.0 42,213.6 123,410.0 123,410.0 Observations 99 621 54 128 873 Notes: Sample for coffee consists in total of 961 farmers. In this table, only farmers selling positive quantities are included. 39 farmers made no coffee sales, and 74 farmers reported no monetary revenue from coffee sales. 32 TABLE 18 COFFEE SALES: QUANTITY BY PROCESSING LEVEL Quantity of coffee sold in Berry/ Uva, MT WP Quantity of coffee sold in Wet/Húmedo, MT WP Quantity of coffee sold in Dry/Seco, MT WP Quantity of coffee sold in Green/Oro, MT WP Quantity of coffee sold in Total MT WP Mean 2.6 3.0 4.2 9.2 4.1 SD 8.3 8.8 8.8 25.1 13.0 Min 0.0 0.0 0.1 0.0 0.0 Median 0.8 1.4 1.4 2.7 1.4 Max 78.3 147.3 50.6 216.8 216.8 Observations 101 638 55 135 898 Notes: Sample for coffee consists in total of 961 farmers. In this table, only farmers selling positive quantities are included. 39 farmers made no coffee sales, and 74 farmers reported no monetary revenue from coffee sales. FIGURE 5 DISTRIBUTION OF COFFEE SALES Evaluation Validity Tests at Baseline: Agricultural Productivity for Coffee In what follows we provide balance tests for the treatment and control groups to validate the impact evaluation design. By balance tests, we mean that we are testing for statistically significant differences in the average values for outcomes we plan to study in later survey rounds. In the 33 coffee sample, we expect that the unconditional variables should be balanced; e.g. the null hypothesis that the mean for the treatment and control groups are equal should be accepted. From a statistical perspective, we should expect a few statistically significant differences. Recall that if we test 20 independent null hypotheses at the 5 percent level, in expectation we should reject one hypothesis out of chance. So, if a handful of rejections occur, it does not necessarily mean that there is a lack of balance; rather, if we find a pattern of rejections, we need to develop a strategy to overcome that pattern. First, we examine the variables measuring the coffee area (Table 19). On the left side, we show the number of observations and clusters for the control group followed by the mean and the standard error of the mean. On the right side, we show the same information for the treatment group, and finally we provide a p value for the t-test that the two means are equivalent. Note that where we presented conditional averages in the previous section, here we provide unconditional averages, since in a randomized control trial we can best identify the intent-to-treat effect. Whereas we find that the point estimates for each type of area under coffee cultivation are slightly larger in the treatment group, the averages are sufficiently similar that none of the differences are statistically different. To examine if the balance is similar across the distribution and not just the means, we present Figure 6 with the distribution of both area planted and commercial area planted for treatment and control groups. The distributions are very similar, with treatment and control lines intertwine throughout the range of the distribution. TABLE 19: BALANCE TEST FOR COFFEE AREA SIZE (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Total coffee area (in ha) 473 2.12 488 2.46 0.23 [39] [0.17] [38] [0.23] Coffee area in plantía (in ha) 472 0.50 487 0.67 0.13 [39] [0.05] [38] [0.10] Coffee area in commercial production (in ha) 473 1.61 488 1.83 0.34 [39] [0.15] [38] [0.18] Coffee area in recepa (in ha) 473 0.12 487 0.16 0.40 [39] [0.02] [38] [0.04] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. 34 FIGURE 6 AREA PLANTED DISTRIBUTION ACROSS TREATMENT ASSIGNMENT Second, we examine average production levels and losses across the treatment and control groups (Table 20). We find averages across the two groups that are broadly similar; the smallest p value is in fact 0.24, far from significance at the 5 or 10 percent level. Potentially interesting is that with the unconditional figures, we can now clearly observe that post-harvest losses are below 2 percent on average at the farm level. TABLE 20: BALANCE TEST FOR COFFEE PRODUCTION (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p￾value Total Production in qq WP 473 64.40 488 71.39 0.60 [39] [8.51] [38] [10.22] Total Production in qq WP, Total (exclude loss) 473 61.26 488 68.11 0.60 [39] [8.24] [38] [10.13] Total Production in qq WP, Plantia 473 5.63 488 4.57 0.51 [39] [1.27] [38] [1.01] Total Production in qq WP, Commercial 473 55.63 488 63.54 0.53 [39] [7.81] [38] [9.73] Total Production in qq WP, Field loss 473 2.03 488 1.56 0.36 [39] [0.47] [38] [0.27] Total Production in qq WP, Post-harvest loss 473 1.11 488 1.72 0.25 [39] [0.22] [38] [0.48] 35 Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. Since neither area cultivated nor coffee produced was statistically different between the treatment and control groups, it follows that the yields should not be statistically different from one another either (Table 21). Indeed, we test for differences in yields for total area planted, the area in commercial production, and the area in plantía, and we find no significant differences. Point estimates for the control group are slightly higher than the treatment groups, but as we have discussed this difference is not statistically meaningful. Figure 7 shows that the yield estimates are balanced across the distribution. TABLE 21: BALANCE TEST FOR COFFEE YIELDS (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Yields qq/ha WP, Total area 453 30.73 475 31.16 0.93 [39] [3.31] [38] [3.38] Yields qq/ha WP, Commercial 422 36.41 442 37.10 0.91 [39] [3.94] [38] [4.29] Yields qq/ha WP, Plantía 96 21.15 82 24.92 0.47 [34] [3.44] [34] [3.83] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. 36 FIGURE 7 YIELD DISTRIBUTION ACROSS TREATMENT ASSIGNMENT To complete this line of thought, we examine the amount of coffee that was sold, as well as the value of those sales (Table 22). The quantity and value of coffee sold in berry form is marginally significant (at the 10 percent level), indicating that farmers in the treatment group are receiving higher prices. As mentioned, we expect some differences and this difference is not statistically different across groups at conventional significance levels. Figure 8 shows that the quantity sold is balanced across the distribution. Finally, Table 23 shows that the proportion of farmers that sell to different buyers is balanced, including exporters. 37 TABLE 22: BALANCE TEST FOR COFFEE SOLD AND VALUE OF SALES (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Quantity sold in berry in MT (1000 kgs)- WP units 455 0.11 473 0.46 0.09* [39] [0.04] [38] [0.21] Quantity sold in wet in MT (1000 kgs)- WP units 455 4.15 471 2.42 0.48 [39] [2.40] [38] [0.44] Quantity sold in dry in MT (1000 kgs)- WP units 455 0.25 473 0.25 0.99 [39] [0.12] [38] [0.13] Quantity sold in green/oro in MT (1000 kgs)- WP units 455 1.79 473 0.97 0.26 [39] [0.64] [38] [0.34] Quantity sold total in MT(1000 kgs)- WP units 455 6.29 471 4.11 0.39 [39] [2.44] [38] [0.65] Sale value berry (USD) 455 71.73 473 253.77 0.10* [39] [21.78] [38] [107.41] Sale value wet parchment (USD) 455 1746.33 471 2064.19 0.35 [39] [186.53] [38] [282.70] Sale value dry parchment (USD) 455 151.17 473 201.43 0.71 [39] [57.75] [38] [123.13] Sale value green/oro (USD) 455 708.23 473 920.03 0.60 [39] [203.84] [38] [350.52] Total Sale value (USD) 455 2677.46 471 3445.27 0.18 [39] [284.82] [38] [489.86] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. 38 FIGURE 8 QUANTITY SOLD ACROSS TREATMENT ASSIGNMENT TABLE 23: BALANCE TEST FOR QUANTITY SOLD BY BUYERS OF COFFEE (1) (2) t-test Control Treatment p-value Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE (1)-(2) Sold to intermediary or coyote 455 0.92 473 0.88 0.21 [39] [0.02] [38] [0.03] Sold to exporter 455 0.01 473 0.03 0.25 [39] [0.01] [38] [0.01] Sold to other buyers 455 0.06 473 0.09 0.32 [39] [0.02] [38] [0.02] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. 39 Inputs, Technologies and Management Practices We seek to track progress in the introduction of improved technologies and management practices. We consider two concepts—application and adoption. Application is the use of technology or management practice by a farmer or other producer over at least one production period. Adoption is the use of technology or management practice by a farmer in a sustainable way over an extended period of time. The fact that farmers or other beneficiaries have applied a technology or management practice for a year or two does not mean that they have sustainably adopted it – or will continue to do so after a project ends. In addition, improved technology and management practices are often promoted as packages comprising several independent technologies or practices. For example, “conservation agriculture” is often promoted as an improved technology/management practice package and may include any combination of several independent elements (e.g., zero-tillage, use of cover crops, integrating livestock, direct seeding), each of which can lead to improved production outcomes but are more effective when applied together. In the survey we track independent elements of a technology or practices to allow identification of barriers to application of some technologies relative to others. Finally, note that the application or adoption of many of these techniques will be monitored among beneficiaries. The goal here is to measure the application of those techniques relative to the control group. Also, important to growing coffee are the types of inputs that are used by farmers. Note that inputs may be important to yield, but not necessarily to profits; farmers should ideally be optimizing their profitability rather than yields. We first report the proportion of farmers that applied different types of inputs (Table 24). We find that the largest proportion of farmers used chemical fertilizers; a smaller proportion also used either foliar fertilizer or compost. A substantial share of farmers also used herbicides (36 percent); fewer famers used fungicides or other inputs. Note that farmers might only use these chemicals if they were affected by some fungus or pest TABLE 24: SHARE OF FARMERS THAT APPLIED COFFEE INPUTS Compost / Organic fertilizer Chemical fertilizer Foliar fertilizer Fungicide Herbicide Parasi toids Other inputs Share of farmers that applied 0.15 0.63 0.27 0.19 0.36 0.01 0.02 SD 0.36 0.48 0.44 0.40 0.48 0.10 0.14 Observations 961 961 961 961 961 961 961 Notes: Sample for coffee consists of 961 farmers. Farmers that did not respond to the inputs module because they reported not having used any input have also been included in this table as farmers that did not apply any inputs. The total cost of inputs used by farmers appears to be substantial (Table 25). Conditioning on any positive spending, we find the total cost of inputs other than labor is 373 USD lempiras on average; labor costs for households that use hired labor is over 3,000 USD. These figures suggest that coffee production, in the way that coffee is produced, is either not very profitable or farmers are overestimating their labor costs. 40 TABLE 25: COST OF INPUTS AND LABOR COSTS Total cost of inputs Annual cost of male labor Annual cost of female labor Total annual labor cost Annual cost in USD 373 2487 1168 3163 SD 755 3966 2302 5111 Observations 775 782 515 805 Notes: Only farmers spending positive monetary value on inputs are included in this table. Table 26 shows practices across the production cycle. Thirty percent of farmers perform tissue management and 21 percent dry their coffee. The tissue management practice that farmers mention is sanitary pruning, with 70 percent implementing it (Table 27). In other words, among all farmers doing any tissue management, 70 percent do sanitary pruning; so only 21 percent of the full sample is doing sanitary pruning. Most farmers report harvesting when the coffee berries are in a uniform maturity level and sell their coffee cleaned but not dried or in wet parchment. Also 70% of farmers that have their coffee in shade have it permanent shade (Table 28). TABLE 26: COFFEE PRACTICES During last 12 months, Mean SD Observations did you clean, control weeds? 0.98 0.13 961 did prune plantations, tissue management? 0.30 0.46 959 did you do pruning of excess shoots? 0.30 0.46 958 did you do soil fertilization? 0.64 0.48 959 did you do foliar fertilization? 0.24 0.42 957 did you do shade regulation? 0.48 0.50 957 did you do pest management? 0.21 0.41 960 did you do disease control? 0.17 0.38 960 did you cut coffee? 0.94 0.24 960 did you de-pulp coffee? 0.82 0.38 961 did you wash and ferment coffee? 0.81 0.39 960 did you dry coffee? 0.21 0.41 958 did you store coffee? 0.07 0.25 953 Notes: These questions were asked of the entire sample of 961 coffee farmers; differences in sample sizes due to missing values on specific questions. 41 TABLE 27: COFFEE PRACTICES: PLANT AND TISSUE MANAGEMENT Mean SD Observations Did you perform tissue management during the last 12 months? 0.3 0.2 961 For tissue management: did you do sanitary pruning? 0.7 0.5 255 did you do recepa (0.40m from ground)? 0.2 0.4 255 did you do descope/pruning (1.70m)? 0.2 0.4 255 did you do medium pruning (1.50m )? 0.1 0.2 255 did you do other type of pruning? 0.0 0.2 255 did you do pruning of excess shoots? 0.4 0.5 255 Note: These questions were asked of the entire sample of 961 coffee farmers; differences in sample sizes due to missing values to specific questions. TABLE 28: COFFEE PRACTICES: QUALITY Mean SD Observations Did you do any shade management practice? 0.5 0.5 960 How many bloomings did you have in 2017? 2.1 1.5 956 Do you monitor bloomings to program harvesting? 1.8 0.4 961 Do you harvest when coffee cherries have same maturity level? 0.9 0.3 959 When you harvest, do you separate the green beans? 0.7 0.4 959 Do you sample your harvest to scrutinize coffee grains? 0.0 0.2 959 % of coffee beans ripe 76.3 20.8 42 % of coffee beans green 13.7 13.8 42 % of coffee beans over ripe 8.8 19.5 42 % of coffee beans dry 6.9 17.0 42 Do you have shade of type: permanent? 0.7 0.5 436 Do you have shade of type: temporal? 0.0 0.2 436 Do you have shade of type: both? 0.3 0.4 436 Do you have shade at a level of: 10-30%? 0.3 0.5 436 Do you have shade at a level of: 31-50%? 0.5 0.5 436 Do you have shade at a level of: more than 50%? 0.2 0.4 436 Is your coffee grain small size? 0.1 0.3 957 Is your coffee grain medium size? 0.5 0.5 957 Note: These questions were asked of the entire sample of 961 coffee farmers; differences in sample sizes due to missing values to specific questions. 42 TABLE 29: COFFEE PRACTICES: SOIL CARE Mean SD Observations Did you perform a soil analysis in any of your plots? 0.11 0.31 954 Did you add lime to soil? 0.16 0.36 961 Did you perform a water analysis? 0.01 0.10 958 How much space in cm do you leave between coffee plants? 111.47 78.72 955 How much space in cm do you leave between grooves? 174.08 135.54 955 Note: These questions were asked of the entire sample of 961 coffee farmers; differences in sample sizes due to missing values to specific questions. Only 11% of farmers perform a soil analysis and only 1% of farmers performed a water analysis, showing these practices are still low in the area (Table 29). Seventy percent of coffee farmers own a de-pulping machine, but only 35% report doing pulp managing; the majority of farmers compost it. To sell their “beneficiado húmedo” or wet processing, half the farmers use water from a source such as river, creek, canal, lake; showing that practices for water collection are not used extensively. Eighty seven percent of farmers sell their coffee humid and the same share of farmers de-pulp the coffee before selling (Table 30). Only about 30 farmers in the coffee sample report having received training in coffee quality, and half of them have tested for quality at baseline level. Most farmers (86 percent) report that their coffee grain is white, followed by 30 percent that also report it being brown and 25 percent dark brown. Given the small number of farmers that performed quality tests at baseline (15), sample sizes on quality are still very small to draw conclusions on both the quality results they got from tests and the quality classification. Although anecdotally, 9 out of those 15 farmers don´t know or were not informed about the quality classification. 43 TABLE 30: COFFEE PRACTICES: POST-HARVEST Mean SD Observations Do you have a cherry coffee receipt hopper? 0.49 0.50 961 Do you have your own de-pulping machine? 0.69 0.46 835 Did you manage coffee pulp during 2017? 0.35 0.48 835 What do you do with the coffee pulp? Compost 0.54 0.50 294 What does he do with the coffee pulp? Leave if on the farm 0.33 0.47 294 What do you do with the coffee pulp: Accumulate it 0.17 0.38 294 Source of water for wet processing: Rainwater / water harvest 0.06 0.25 835 Source of water for wet processing: River, creek, canal, lake, dam 0.50 0.50 835 Source of water for wet processing: Private well 0.08 0.27 835 Source of water for wet processing: Public well 0.01 0.08 835 Source of water for wet processing: Spring 0.20 0.40 835 Did you perform any water management during 2017? 0.20 0.40 835 What do you do with the honey waters? Compost 0.08 0.27 168 What do you do with the honey waters? Pour them on the farm 0.18 0.39 168 What do you do with the honey waters? Pour them into fountains / streams 0.10 0.30 168 What do you do with the honey waters? Pour them in oxidation lagoon 0.35 0.48 168 What do you do with the honey waters? Pour them on the street, ditch 0.21 0.41 168 Do you sell humid? 0.87 0.34 835 Do you dry the coffee bean? 0.92 0.28 108 Do you store coffee? 0.36 0.48 99 Do you sell your coffee immediately? 1.31 0.47 99 Do you transport your harvested coffee in your own vehicle? 2.13 0.76 31 Type of tolva: In water 0.33 0.47 475 Type of tolva: Dry 0.67 0.47 475 Do you de-pulp coffee? 0.87 0.34 960 Do you have shade at a level of: more than 50%? 0.36 0.48 835 Is your coffee grain small size? 0.14 0.35 835 Is your coffee grain medium size? 0.25 0.43 835 Note: These questions were asked of the entire sample of 961 coffee farmers; differences in sample sizes due to missing values to specific questions. 44 TABLE 31: COFFEE PRACTICES: MORE ON QUALITY Mean SD Observations Have you received training on coffee quality tests? 0.30 0.46 99 Did any of the lots of coffee you produce has been tested for quality? 0.50 0.51 30 What was the color of the coffee grain: white 0.86 0.34 835 What was the color of the coffee grain: light yellow 0.09 0.28 835 What was the color of the coffee grain: green yellow 0.02 0.13 835 What was the color of the ferment: brown 0.30 0.46 835 What was the color of the ferment: dark brown 0.25 0.43 835 What was the color of the ferment: black 0.05 0.22 835 What result did you get? from 80 to 83 points 0.47 0.52 15 What result did you get? more than 83 points 0.40 0.51 15 What was the classification it got: standard 0.07 0.26 15 What was the classification it got: High altitude coffee HG 0.27 0.46 15 What was the classification it got: special coffee/gourmet 0.07 0.26 15 What was the classification it got: you were not informed 0.60 0.51 15 Note: These questions were asked of the entire sample of 961 coffee farmers; differences in sample sizes due to missing values to specific questions. Coffee Extension Coffee farmers report in general very little exposure to extension (Table 28). The survey form asked specifically about exposure to 10 different topics related to extension, and only 182 out of 961 farmers, or about 19 percent of farmers, report receiving extension on any of the 10 topics. Among farmers receiving some coffee extension, the two most common topics are coffee pruning and crop maintenance; very few farmers have received any extension on marketing, finance and contracts, or market access. Among farmers reporting receiving extension, they tend to rate the extension as very good. Since a main goal of MAS+ is to increase productivity through extension, the finding of little exposure is quite positive if it will lead to stronger impacts. 45 TABLE 32: COFFEE EXTENSION Mean SD Observati ons Number of topics of received extension 0.35 0.93 961 Share of farmers that received extension about: Coffee Pruning 0.09 0.29 961 Shade regulation 0.06 0.24 961 Maintenance 0.10 0.30 961 Coffee cut 0.03 0.17 961 Post-harvest 0.01 0.12 961 Quality and certification 0.02 0.14 961 Marketing 0.02 0.14 961 Finance and contracts 0.01 0.07 961 Market access 0.00 0.06 961 Duration of coffee extension in days 55.1 254.23 182 Quality of extension, 1 to 4 3.57 0.56 175 Notes: All coffee farmers have been considered for this table. All farmers who claimed not having received extension in any topic, and so they were not asked these questions directly, are coded as “no” for each topic. Sample for duration of training and quality of extension include only farmers that received extension. Coffee Pests We next examine damages that farmers report being caused by pests, as one specific type of negative shock affecting coffee farmers (Table 33). Farmers claim substantial losses to pests when they claim losses; on average 24 percent of the crop was loss when they had a loss. Among the more common insects doing damage to crops were coffee rust and broca. Among farmers reporting that they incurred in some cost to manage a pest, the average expenditure was 5,478 lempiras; note that not all farmers attempted to solve pest problems with additional inputs. 46 TABLE 33: PESTS IN COFFEE Mean SD Observations Percent of damage done by pests 23.97 22.94 628 Share of farmers that had problems by: Broca 0.40 0.49 961 Coffee rust 0.40 0.49 961 Minador 0.02 0.16 961 Ojo de gallo 0.15 0.36 961 Mal de hilachas 0.05 0.22 961 Antracnosis 0.03 0.17 961 Mancha de hierro 0.10 0.29 961 Other 0.03 0.17 961 Average cost to control pest 5,478.17 16,947.47 246 Notes: Information on having encountered each specific pest includes all farmers. Sample sizes for percent of crop damage caused by pests and control costs includes only farmers that had positive values for these two variables. Investments and Equipment Next, we examine what investments and equipment the farmers in the sample have. The most reported investment are plant nurseries, with 18 percent of coffee farmers having them. The proportion of farmers that have post-harvest equipment for coffee is low, with only 7 percent reporting having a pulping machine and 2 percent having a designated place to dry their coffee (Table 34). TABLE 34: INVESTMENTS IN THE AGRICULTURAL PLOTS Investment in the last 12 months: Mean SD Observations Live barriers 0.04 0.19 952 Dead barriers, fences 0.07 0.26 956 Water harvests 0.00 0.06 954 Alley cultivation 0.02 0.13 956 Plant nursery 0.18 0.39 957 Greenhouses 0.00 0.05 956 Irrigation canal 0.00 0.00 953 Irrigation system 0.01 0.10 953 Solar dryer 0.02 0.14 953 Pulping machine 0.07 0.26 955 Other investments 0.01 0.07 944 In the survey we asked 19 different types of durable goods that could either be associated with higher incomes or production (Table 35). We find that the most important are primarily consumer durables, such as televisions, radios, motorcycles, and cell phones (though the latter could also 47 be considered useful for production). Equipment associated with agriculture, such as sprayers, are not own by many farmers in the sample. TABLE 35: INVESTMENT IN EQUIPMENT Type of machinery and equipment: Mean SD Observations Sprayer with motor 0.04 0.18 961 Motor mounter 0.03 0.16 961 Tractor 0.00 0.00 961 Car 0.02 0.15 961 Truck or pick-up 0.17 0.38 961 Motorcycle 0.33 0.47 961 Bicycle 0.08 0.27 961 Radio 0.64 0.48 961 Television 0.47 0.50 961 Refrigerator 0.30 0.46 961 Computer 0.06 0.23 961 Cell phones 0.85 0.35 961 Gas stove 0.09 0.28 961 Livestock (oxen, cattle) 0.17 0.37 961 Small livestock (goats, pigs, sheep) 0.21 0.41 961 Hens, Ducks, Turkeys, Pigeons 0.81 0.39 961 Horses, mares, mules, males, donkeys 0.35 0.48 961 Group Membership and Participation Group membership can play a role in fostering improvements in agricultural production, though it is not sufficient (e.g. Waddington et al., 2014). Learning about the extent of these organizations is important, especially given the focus of MAS+ in fostering group participation. We therefore ask farmers if they participate in any of such organizations at baseline (Table 36). To be counted as participants, they can be either active or inactive organization members, or leading such organizations. We find that the most common type of group are water user associations (with 20% participation), followed by civic or charitable groups (with similar participation). Productive groups are rarer; only about a quarter of households report the existence of credit or microfinance groups, and only 17 percent report the existence of producer groups. Other organizations have close to zero participation. Therefore, there is a lot of scope for organizing coffee farmers within the sample. 48 TABLE 36: GROUPS PRESENT IN THE COMMUNITY Mean SD Observations Share of farmers that are members Exists in the community: Productive agricultural groups or org. of producers 0.17 0.38 946 0.13 Exists in the community: Productive agricultural groups for commercialization 0.02 0.13 946 0.01 Exists in the community: Water users groups 0.72 0.45 959 0.20 Exists in the community: Civic or charitable groups 0.77 0.42 961 0.19 Exists in the community: Women groups 0.06 0.24 949 0.01 Exists in the community: Credit or micro-finance groups 0.25 0.44 958 0.14 Exists in the community: Insurance or mutual aid groups 0.00 0.00 951 0.00 Exists in the community: Commerce and business associations 0.00 0.00 957 0.00 Exists in the community: NGO / External Project 0.01 0.11 956 0.01 Exists in the community: Other 0.01 0.12 938 0.01 Evaluation Validity Tests at Baseline: Inputs, Extension and Pests for Coffee Our next tests of the validity of the evaluation design focuses on inputs, extension participation and the prevalence of pests in the coffee. We study whether the share of households using specific types of inputs in coffee differs between the treatment and control group (Table 37). In all seven cases, the average proportion of households using specific inputs is not statistically different between the treatment and control groups. We additionally examine whether material or labor input costs differ by treatment group (Table 38). As with the discrete indicators, we find no evidence of statistically significant differences. To this point, as expected through the randomization, we have found no differences between the treatment and control groups for any indicators related to coffee inputs, outputs, or sales by sample households. 49 TABLE 37: BALANCE TEST FOR SHARE OF FARMERS THAT APPLIED INPUTS (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Applied Compost/ Organic fertilizer 473 0.14 488 0.16 0.43 [39] [0.02] [38] [0.02] Applied Chemical fertilizer 473 0.63 488 0.64 0.89 [39] [0.04] [38] [0.05] Applied Foliar fertilizer 473 0.25 488 0.28 0.51 [39] [0.03] [38] [0.04] Applied Fungicides 473 0.17 488 0.22 0.29 [39] [0.03] [38] [0.03] Herbicides 473 0.38 488 0.34 0.41 [39] [0.03] [38] [0.03] Parasitoids 473 0.01 488 0.01 0.31 [39] [0.01] [38] [0.00] Other inputs 473 0.02 488 0.02 0.80 [39] [0.01] [38] [0.01] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. TABLE 38: BALANCE TEST FOR COST OF INPUTS AND LABOR COSTS (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Input costs (USD) 387 332.31 399 401.51 0.40 [39] [55.18] [38] [61.44] Annual labor cost (USD)- men 392 2416.33 421 2369.42 0.91 [39] [285.38] [38] [292.57] Annual labor cost (USD)- women 392 626.65 421 845.10 0.29 [39] [107.66] [38] [177.64] Annual labor cost (USD)- total 392 3042.97 421 3214.52 0.76 [39] [349.28] [38] [427.12] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. The next set of variables that relate to one of the research questions involve access to extension for growing coffee. We test first for whether the average number of topics covered through extension differs by treatment and control groups; it does not (Table 39, row 1). We then test whether there are differences in receipt of extension by practice; again, we find no differences. We finally examine extension by days and rating; there are no statistical differences between treatment and control groups. 50 TABLE 39: BALANCE TEST FOR COFFEE EXTENSION (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Number of topics of received extension 473 0.30 488 0.39 0.34 [39] [0.06] [38] [0.07] Received extension about Coffee pruning 473 0.08 488 0.10 0.48 [39] [0.02] [38] [0.02] Received extension about Shade regulation 473 0.05 488 0.06 0.65 [39] [0.01] [38] [0.01] Received extension about Maintenance 473 0.09 488 0.11 0.32 [39] [0.02] [38] [0.02] Received extension about Coffee cut 473 0.03 488 0.03 0.84 [39] [0.01] [38] [0.01] Received extension about Post-harvest 473 0.01 488 0.02 0.09* [39] [0.00] [38] [0.01] Received extension about Quality and certification 473 0.02 488 0.02 0.62 [39] [0.01] [38] [0.01] Received extension about Marketing 473 0.02 488 0.02 0.95 [39] [0.01] [38] [0.01] Received extension about Finance and contracts 473 0.00 488 0.01 0.26 [39] [0.00] [38] [0.00] Received extension about Market access 473 0.00 488 0.01 0.32 [39] [0.00] [38] [0.00] Duration of coffee extension in days 81 25.39 101 78.89 0.25 [28] [7.38] [29] [45.46] Quality of extension from 1 to 4 77 3.53 98 3.60 0.39 [28] [0.07] [29] [0.04] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. Our final balance tests for coffee relate to experience with pests (Table 40). We test whether households have differences in the percent of their crop damaged; we also test whether the cost of pest control differs between the treatment and control groups. We find that there are no significant differences in the percent of damages or the cost of pest control. However, for one specific pest (the minador), we find a significant difference. We believe this difference is not pertinent for two reasons. First, it is the only difference we find, out of dozens of tests. We expected, in fact, to find more differences. Second, the share of households affected by the minador are extremely small, at about 3 percent of all households. Since the share is so small, it would not have meaningful effects on production over the entire sample. 51 TABLE 40: BALANCE TEST FOR PESTS IN COFFEE (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Percent of damage 308 24.40 320 23.56 0.78 [39] [2.27] [38] [1.98] Had problems with broca 473 0.39 488 0.40 0.91 [39] [0.05] [38] [0.03] Had problems with roya 473 0.41 488 0.39 0.67 [39] [0.05] [38] [0.04] Had problems with minador 473 0.04 488 0.01 0.01** [39] [0.01] [38] [0.00] Had problems with ojo de gallo 473 0.16 488 0.15 0.80 [39] [0.02] [38] [0.03] Had problems with mal de hilachas 473 0.06 488 0.04 0.33 [39] [0.02] [38] [0.01] Had problems with antracnosis 473 0.03 488 0.03 0.59 [39] [0.01] [38] [0.01] Had problems with mancha de hierro 473 0.10 488 0.09 0.74 [39] [0.02] [38] [0.01] Had problems with other 473 0.03 488 0.03 0.67 [39] [0.01] [38] [0.01] Cost of pest control in Lps 136 3986.26 110 7322.70 0.11 [38] [1312.67] [30] [1582.25] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. In summary, we find almost no statistically meaningful differences between the treatment and control groups for outcomes among the coffee sample, demonstrating that the randomization worked quite well. Even without controlling for other variables in analysis, the implication is that the difference in means either midway through the project, or at endline between treatment and control groups is an unbiased estimate of the program impacts. Controlling for other factors in analysis may remain useful, however, to explain some of the variance in the outcome, to improve statistical inference on the difference in means. 52 7 Analysis of Beans Producers in MAS+ Evaluation As with the coffee subsample in the previous section, in this section we describe the sample of beans growers. After describing the households demographically, we examine agricultural statistics among households from the perspective primarily of growing beans, including both outputs and inputs. We include balancing tests throughout the themes in these sections to show the validity of the propensity score matching evaluation design to verify that the selected sample is balanced for important outcomes across treatment and control groups. 7.1 Demographic Characteristics We first examine demographic characteristics among the beans sample; households are slightly larger than among the coffee sample, but the different is not statistically significant (Table 41). The households are also largely nuclear, meaning that there are few elderly residents; there are 0.2 total residents older than 65, and there are on average around 2 children who are under 15 living in the households. As with the coffee sample, young and elderly household members are relatively gender balanced. However, we find that adults of working age are not gender balanced; we find more males than females in households, at 1.16 males on average versus 0.87 females. TABLE 41: DEMOGRAPHIC CHARACTERISTICS, BEANS SAMPLE Mean SD Min Median Max Observations Number of people living in the house 4.84 2.11 1 5 15 968 Boys under 6 0.43 0.70 0 0 5 968 Girls under 6 0.35 0.70 0 0 9 968 Men between 6 and 14 years old 0.54 1.01 0 0 13 968 Women between 6 and 14 years old 0.55 1.25 0 0 15 965 Men between 15 and 64 years old 1.16 0.66 0 1 4 968 Women between 15 and 64 years old 0.87 0.42 0 1 3 968 Men older than 65 0.11 0.31 0 0 1 968 Women older than 65 0.09 0.29 0 0 2 968 People managing the farm in the beans sample range between 11 to 85 years old (Table 42). The average age is 41 years old. About 42% of beans farmers are women. The average number of years of education is 4.3 years and the average literacy ratio is 82%, just a little below the coffee sample. Looking at the same characteristics by gender (Table 43), we see that women are about two years older on average than men (42 years old vs 40 years old for men), similar to the coffee sample. Average years of education and literacy ratios for both genders are about the same. 53 TABLE 42: DEMOGRAPHIC CHARACTERISTICS Age Female Years of Education Literacy Mean 41.00 0.42 4.32 0.82 SD 16.28 0.49 2.98 0.38 Min 11 0 0 0 Median 40 0 5 1 Max 85 1 16 1 Observations 2188 2188 2188 2188 Note: Demographic characteristics are calculated among all persons in charge of agricultural decisions in the 968 beans households. TABLE 43: DEMOGRAPHIC CHARACTERISTICS BY GENDER Male Female Age Years of Education Literacy Age Years of Education Literacy Mean 40.26 4.30 0.81 42.03 4.33 0.83 SD 17.26 3.02 0.39 14.76 2.92 0.38 Min 11 0 0 11 0 0 Median 38 5 1 42 5 1 Max 85 16 1 84 15 1 Observations 1273 1273 1273 915 915 915 Note: Demographic characteristics are calculated among all persons in charge of agricultural decisions in the 968 beans households. We also calculate non-agricultural income for the beans sample. But as opposed to the coffee sample, there is a smaller share of beans farmers that sell their product to the market, so we have information for 448 farmers that report earning positive income from non-agricultural activities, and 438 farmers that report earning income from selling beans. Considering the sample of beans farmers that earn positive amounts of either agricultural or non-agricultural activities, we find that bean sales represent 60% of total monthly income (Table 44). TABLE 44: AGRICULTURAL AND NON-AGRICULTURAL INCOME Monthly Non-Agricultural Income in USD Monthly Agricultural Income in USD Share that Coffee Income represents from Total Income Mean 302.51 126.27 0.60 SD 574.16 504.94 0.44 Min 0.21 0.06 0.00 Median 147.60 49.20 1.00 Max 6150.00 8200.00 1.00 Observations 448 438 513 Note: Only 448 farmers from the beans sample report earning income from other non-agricultural activities; 438 farmers report earning income from selling beans to the market. The share that beans income represents from total income is calculated only for farmers earning positive amounts from either of both sources (513 farmers). 54 7.2 Agricultural Outcomes We next explore bean production, first from the perspective of cultivated area, and then examine production and yields. Before turning to agricultural inputs, the section also discusses beans sales. Note that we expect different outcomes with sales, as beans are not necessarily a cash crop. Size of Cultivated Land There are two main seasons for growing field crops in Honduras, and most of Central America— the primera (April-July) and postrera (August-October) seasons. Decisions about which season to plant in depend upon several factors, such as temperature, rainfall, and soil drainage capabilities, among others. Since beans can be grown in either season, we split the sample to look at area planted and harvested by season (Table 45). We find that conditional on planting, households tend to plant about 1 ha in beans, regardless of the season, though more households plant in postrera than in primera. The distributions are not as skewed as with coffee; the largest areas planted and harvested are 10.4 ha and 12.5 ha in the primera and postrera, respectively. TABLE 45: AREA SIZE OF PLANTED AND HARVESTED AREA WITH BEANS Total area planted with beans in Primera (ha) Total area harvested with beans in Primera (ha) Total area planted with beans in Postrera (ha) Total area harvested with beans in Postrera (ha) Mean 1.00 0.93 1.04 0.97 SD 1.01 0.93 1.18 1.09 Min 0.04 0.01 0.01 0.01 Median 0.70 0.70 0.70 0.70 Max 10.44 10.44 12.53 12.53 Observations 657 656 766 747 Notes: Sample for beans consists of 968 farmers. Difference in sample sizes is due to farmers that only plant in primera or postrera. 683 farmers report planted areas in Primera, and 795 farmers in postrera. Outliers were dealt with in the following ways: If commercial area reported was greater than total area, it was replaced with total area. We replaced total area by commercial area. 14 cases in primera, 23 in postrera. 28 extra observations were dropped from total area and 16 from harvested area because they are yield outliers or have missing values in the primera; similarly, we removed 49 for total area and 23 for harvested area in the postrera. 55 Beans Production Next, we examine the total amount of beans produced by season (Table 46). We find that among producing households, beans production is a bit higher in the primera than the postrera season, at about 863 kg on average versus 646 kg in the latter season. As with area cultivated, we find that the distribution is somewhat skewed; the medians are smaller than the means, at 544 and 453 kg for the primera and postrera, respectively. TABLE 46: BEANS PRODUCTION Beans production in Primera (in kg) Beans production in Postrera (in kg) Mean 862.56 645.80 SD 943.82 761.87 Min 0.191 0.340 Median 544.31 453.59 Max 8890.36 6667.77 Observations 657 739 Notes: Only farmers with positive production in either primera or postrera have been considered for this table; 14 observations were dropped from production in primera and 25 from production in postrera because they contained yield outliers or missing observations. Given that beans are not necessarily a cash crop and can constitute an important share of home consumption, and as such, have an important role in food security, we look at how farmers report using their total production. Table 47 and Table 48 illustrate how the beans production was distributed for primera and postrera, respectively. The possible uses included in the survey were: sale, consume in the home, make sub-products, animal feed, and losses. We find that most farmers use their beans production for home consumption and sales. TABLE 47: BEANS PRODUCTION FINAL USE, PRIMERA Sales kg- primera Home consumption kg - primera Exchange kg - primera Sub products kg - primera Animal feed kg - primera Gifts kg - primera Lost kg -primera Mean 106.14 140.03 162.19 90.72 110.56 85.02 259.03 SD 303.32 191.81 402.31 45.36 90.26 212.07 215.30 Min 0.91 0.23 18.14 45.36 22.68 1.36 0.91 Median 45.36 90.72 90.72 90.72 68.04 45.36 181.44 Max 4535.90 3039.05 1814.36 136.08 272.15 2177.23 907.18 Observations 537 633 19 3 8 195 55 Notes: Sample for beans consists of 968 farmers. Difference in sample sizes is due to farmers that only plant in primera or postrera. 657 farmers had planted area in Primera, and 739 farmers in postrera. 56 TABLE 48: BEANS PRODUCTION FINAL USE, POSTRERA Sales kg- postrera Home consumption kg - postrera Exchange kg - postrera Sub products kg - postrera Animal feed kg - postrera Gifts kg - postrera Lost kg - postrera Mean 87.74 144.44 115.30 60.48 101.76 57.18 310.86 SD 192.07 218.65 104.32 26.19 103.17 96.99 288.05 Min 0.34 6.80 14.51 45.36 22.68 2.27 18.14 Median 45.36 90.72 90.72 45.36 90.72 45.36 226.80 Max 2902.98 3655.94 362.87 90.72 453.59 1088.62 1814.36 Observations 539 708 16 3 15 144 60 Notes: Sample for beans consists of 968 farmers. Difference in sample sizes is due to farmers that only plant in primera or postrera. 657 farmers had planted area in Primera, and 739 farmers in postrera. Beans Food Security To investigate whether a household’s consumption meets minimum requirements, we use the Standard of Per Capita Consumption of Basic Grains for countries affected by Mitch hurricane, which for Honduras is estimated in 25.7 kg per capita / year10. We find that 48% of households have a yearly per capita consumption that is greater than the recommended threshold (Table 49). The average consumption in the area is estimated to be greater than the standard, at 35.6 kg per capita a year. TABLE 49: BEANS FOOD SECURITY Beans Consumption (KG/PC/year) % of HH above the Standard for Per Capita Consumption in Honduras (25.7) Mean 35.56 0.48 SD 44.57 0.50 Min 0.57 0.00 P50 22.68 0.00 Max 731.19 1.00 Observations 749 749 Notes: These statistics are produced only for the sample of farmers that reported having home consumption. If farmers only produced in primera and not in postrera, only their consumption in primera was considered; and vice-versa Beans Yields Combining the production and land plated information, we then measure yields for beans, by season (Table 50). Yields are higher in the primera on average than the postrera among sample households at 888 and 670 kg/ha, respectively. Median yields are slightly lower, at 782 and 586 kg/ha. In general, these averages are in line with expectations. 10 http://www.fao.org/docrep/003/y2784s/Y2784S07.htm 57 Figure 9 shows the distribution of the yields for primera and postrera. Most farmers are below 3,000 kg/ha. Note that the yields distribution is more spread out in primera, and primera has higher productivity estimates; evidenced by the thicker tail on the right (green). We find lower yields than we would expect using a semi-technical growing system, which would be between 1,000 kg/ha and 1,824 kg/ha (around 1038 and 1818 kg/ha depending on the variety). TABLE 50: BEANS YIELDS Yield in Primera (kg/ha) Yield in Postrera (kg/ha) Mean 888.47 670.44 SD 596.68 440.58 Min 0.27 0.98 Median 781.94 586.45 Max 3127.76 2345.82 Observations 657 738 Notes: Sample for this table consists only of farmers having positive area planted and positive production in either primera or postrera. 311 farmers did not have any area in commercial production or had production in primera, and 230 did not have commercial area planted or had production in postrera. FIGURE 9 DISTRIBUTION OF BEANS PRODUCTIVITY BY SEASON 58 As with coffee yields, we next graph production against area reported planted in both seasons (Figure 10). A difference between the graphs of coffee and bean yields is that we observe clear bunching around what are clear estimates of plot sizes (e.g. around 1.4, 2 or 2.8 ha -equivalent to the reported: 2, 3 or 4 manzanas in the postrera). Clearly, farmers are estimating the amount of land they have rather than knowing precise amounts, which will affect the denominator in the yields calculation. As shown by Abate et al. (2018), inaccurate estimates of land area can affect impact estimates from programs on yields, so we will consider whether an alternative method of measuring land area is needed for the endline. Otherwise, the pattern is consistent with expectations, as production in both seasons grows with land area in a linear relationship. 59 FIGURE 10: BEANS AREA AND BEANS PRODUCTION IN PRIMERA AND POSTRERA 60 Beans Sales To complete this section, we examine the quantity and value of bean sales (Table 51). As discussed at the outset, not all households sell beans, and we would not expect households to sell all their production in most cases. Indeed, more farmers producing beans in the primera appear to also sell beans after the season (81 percent of producers); those who sell do tend to sell what appears to be a large share of their production, as the average sales are 0.8 MT (1 metric tons/MT= 1,000 Kgs). In the postrera season, 73 percent of households recorded sales; among those selling, the average amount sold was 0.6 MT. Households report an average sales value of around 631 USD in the primera season, which implies a price of 1,051 USD per ton; the average price reported was about 2,150 USD per MT in the postrera, higher than in primera. Figure 11 show the distribution of these sales in each agricultural season. In primera, the distribution is more concentrated to the right; this thicker tale shows that primera production is more likely to be sold. In the case of postrera we find that the mass of sales is more concentrated below 1 MT. TABLE 51: BEANS SALES, QUANTITY AND VALUE OF SALES Beans quantity sold in Primera (in MT) Value of Sales in Primera (in USD) Beans quantity sold in Postrera (in MT) Value of Sales in Postrera (in USD) Mean 0.81 631 0.61 1311 SD 0.92 724 0.76 6637 Min 0.00 0.41 0.00 1.48 Median 0.54 369 0.36 277 Max 8.66 5437 6.53 98400 Observations 535 487 541 491 Notes: Sample for beans consists of 968 farmers. Difference in sample sizes for columns due to farmers that did not sell positive quantities or have positive revenues. 61 FIGURE 11 DISTRIBUTION OF BEANS SALES BY SEASON Evaluation Validity Tests at Baseline: Agricultural Productivity - Beans In this section, we provide balance tests for the treatment and control groups for variables that reflect agricultural productivity: area planted, production, yields, and sales. The balance tests, look to detect statistically significant differences in the average values for outcomes we plan to study in later survey rounds across treatment and control groups. In the beans sample, we might expect more differences initially, as the beans sample was not randomly selected. For the propensity score matching methodology this validation test is very important not just a confirmation and in the coffee evaluation that uses an RCT. The purpose of these test is to, first, see if the matching exercise in the sample selection resulted in a sample that is comparable to across the treatment assignment. Second, for the variables that are not balanced we can obtain some context to inform the matching exercise using the baseline sample, to create weights that adjust for this imbalance at baseline and adjust the impact estimates at endline using these weights. In what follows we focus on the first of these aspects. From a statistical perspective we should expect a few statistically significant differences if the treatment and control groups are randomly chosen from the same underlying population. This is not the case for the beans evaluation design; which uses a propensity score matching. We visited treatment areas because we had them on a selected list to be treated, so they are bound to be different from the control areas we visited because we knew they were not going to be touched by MAS+. To minimize this selection bias, we implemented a pairing or matching of areas which should decrease the differences across the treatment and control groups. Recall that if we test 20 independent null hypotheses at the 5 percent level, in expectation we should reject one hypothesis out of chance. Thus, if a handful of rejections occur, it does not 62 necessarily mean that there is a lack of balance and thus no valid comparisons can be made; rather, if we find a pattern of rejections, we need to develop a strategy to overcome that pattern. The balance tests for beans are slightly more complicated than the tests for coffee. Farmers can plant beans in either the primera or the postrera season, or both. So substantial numbers of farmers do not grow beans in one of the two seasons. We adjust our methodology to account for this complication. We present tests for whether the conditional means are different from one another (e.g. production in postrera, conditional on growing beans in postrera), but we initially test whether there is a difference in the proportion of households conducting each activity (e.g. planting beans, producing beans) between the treatment and control groups. We can then isolate whether any differences would be due to differences in participation in the activity, or due to the level of participation. We first test for differences in area planted and harvested, by season (Table 52). The p-values associated with null hypotheses that each variable in the table have the same mean between the treatment and control groups are statistically similar at conventional levels (5 percent). As a result, there are no statistically significant differences between the treatment and control groups either for planting in the primera or postrera seasons, or for the amount produced conditional on planting. Figure 12 shows the distribution of area planted for the treatment and control groups. The figure shows that the distributions are very similar and near each other across the distribution. TABLE 52: BALANCE TEST FOR BEANS AREA SIZE PLANTED AND HARVESTED (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Dummy for whether farmer planted beans in Primera 475 0.68 493 0.68 0.92 [39] [0.04] [37] [0.04] Beans Area Planted in Primera (in ha) 321 0.97 336 1.03 0.54 [39] [0.08] [36] [0.07] Beans Area Harvested in Primera (in ha) 321 1.87 335 1.55 0.30 [39] [0.29] [36] [0.12] Dummy for whether farmer planted beans in Postrera 475 0.84 493 0.77 0.19 [39] [0.03] [37] [0.05] Beans Area Planted in Postrera (in ha) 401 1.07 380 1.00 0.64 [39] [0.10] [37] [0.10] Beans Area Harvested in Postrera (in ha) 375 0.98 375 0.95 0.79 [39] [0.10] [37] [0.09] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. 63 FIGURE 12 DISTRIBUTION OF AREA PLANTED BY TREATMENT ASSIGNMENT We next test whether household production differs between the treatment and control groups (Table 53). Because households may have differentially lost all their production, we also include dummy variables for whether or not they produced in each season; as in Table 52, we find no systematic differences in production patterns. We find no significant differences. We further find that we cannot reject the null hypothesis of equal production levels. So, we find no differences in either cultivated area or production for both seasons. TABLE 53: BALANCE TEST FOR BEANS PRODUCTION (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Farmer had beans production in Primera 475 0.68 493 0.68 0.92 [39] [0.04] [37] [0.04] Beans Production in Primera (in kg) 321 801.18 336 921.20 0.33 [39] [93.16] [36] [80.92] Farmer had beans production in Postrera 475 0.78 493 0.77 0.85 [39] [0.03] [37] [0.05] Beans Production in Postrera (in kg) 371 607.87 380 672.01 0.54 [39] [69.17] [37] [80.21] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. 64 ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. Consequently, there should not be statistical differences in mean yields, conditional on growing beans, between the treatment and control groups. Sure enough, we cannot reject the null hypothesis of equivalent mean yields, for either season (Table 54). So, studying production from the perspective of cultivated area, total beans production, and yields, we find no statistically significant differences. Figure 13 shows the distribution of yields by treatment assignment. The distributions are similar; however, the primera yields seem to be higher between 1,000 – 2,000 kg/ha, with treatment farmers having higher yields. These distributional differences are not reflected in the balancing test. However, these are the kind of differences in the distribution that should be accentuated in the endline impact estimates if the MAS+ has a large impact on yields on different parts of the distribution. TABLE 54: BALANCE TEST FOR BEANS YIELDS (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Yields kg/ha Primera 321 839.86 336 961.58 0.15 [39] [54.59] [36] [64.69] Yields kg/ha Postrera 366 643.34 372 696.46 0.27 [39] [31.79] [37] [36.47] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. 65 FIGURE 13 DISTRIBUTION OF YIELDS BY TREATMENT ASSIGNMENT Our final comparison related to production is among bean sales. We initially test whether different proportions of beans producers sell any beans, by primera and postrera seasons (Table 55, rows 1 and 4). We find no statistical differences in proportions. We then test whether either the sales volume or value differs over the treatment and control groups; we again find no significant differences. So, we can extend the earlier statement about production to also include sales. Figure 14 shows the distribution quantities sold. Reflecting the distribution of yields, the quantity sold in primera for the treatment is slightly higher around 2 MT. 66 TABLE 55: BALANCE TEST FOR BEANS SALES, QUANTITY AND VALUE OF SALES (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Dummy for whether farmer sold beans in Primera 475 0.52 493 0.59 0.35 [39] [0.05] [37] [0.05] Total sales beans in MT - Primera 249 0.77 291 0.83 0.57 [37] [0.09] [36] [0.07] Value of sales in Lempiras - Primera 224 562.18 268 680.85 0.23 [37] [55.38] [36] [80.47] Dummy for whether farmer sold beans in Postrera 475 0.53 493 0.61 0.14 [39] [0.04] [37] [0.04] Total sales beans in MT - Postrera 250 0.59 302 0.62 0.80 [38] [0.07] [37] [0.08] Value of sales in USD - Postrera 227 1215.81 274 1347.14 0.86 [38] [398.06] [37] [611.69] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. FIGURE 14 DISTRIBUTION OF BEANS QUANTITY SOLD BY TREATMENT ASSIGNMENT 67 We next turn to input use (Table 68). Recall from the descriptive tables that some input use was heavy, such as specific fertilizers, whereas other inputs were seldom used, such as inoculant. Whether or not input use was heavy or light, we find no significant differences between the treatment and control groups for the use of any specific inputs. We further study both total material input costs and labor costs by gender (Table 69) in conditional figures; we find no significant differences for these variables, either. Even though the sample clusters were not chosen randomly, we find no significant differences between the treatment and control groups for outcomes related to either production, area planted or productivity. Inputs Bean farmers appear to be more likely to report using inputs than coffee farmers in the sample (Table 56). Whereas few farmers are using inoculants to promote crop rooting, many farmers use one or more types of fertilizer, particularly chemical or foliar fertilizers. Moreover, a relatively large proportion of farmers use fungicides (47 percent), herbicides (68 percent), or insecticides (43 percent) in producing beans. TABLE 56: SHARE OF FARMERS THAT USED BEANS INPUTS Inoculant Compos/ fertilizer Chemical fertilizer Foliar fertilizer Fungicide Herbicide Insecticide Other Inputs Mean 0.06 0.09 0.41 0.56 0.47 0.68 0.43 0.01 SD 0.23 0.28 0.49 0.50 0.50 0.47 0.50 0.09 Observations 968 968 968 968 968 968 968 968 Notes: Sample for beans consists of 968 farmers. Farmers that did not respond to the inputs module because they reported not having used any inputs, are included as “no” for all inputs in figures above. However, the amount of inputs used, when they are used, appears to be much smaller among bean farmers than among coffee farmers (Table 57). The total cost of inputs among those with positive values 56 USD. When farmers hire labor, they hire substantially more male labor than female labor to work on their beans crops. The total cost of labor, among households hiring any labor, is around 688 USD, though note that the figures are highly variable with large SD, implying that a few farmers in the sample are hiring substantial amounts of labor whereas most may be hiring very little labor. TABLE 57: TOTAL COSTS OF INPUTS AND LABOR COSTS Total cost of inputs Annual cost in male labor Annual cost in female labor Total annual labor cost Annual cost in USD 55.94 665.33 22.33 687.66 SD 93.37 1628.17 194.75 1715.92 Observations 848 739 739 739 Note: Only farmers spending positive monetary value on inputs are included. 68 Practices Next, we examine practices related to bean production (Table 58). The average farmer in the sample reports living far from a main road, at 31 minutes away (row 1). The market at which they can sell beans is not surprisingly much farther; the average farmer reports needing to travel more than 80 minutes, though the median in this case is 30 minutes. Most farmers report that they do not irrigate their bean plots. TABLE 58: PRACTICES BEANS: ACCESS TO MARKETS, VARIETIES AND SELLING POINTS Mean SD Observatio ns Time from dwelling to main road in minutes 31.79 41.30 966 Time in minutes to place where they sell beans 82.33 323.10 731 Do you have irrigation in any of your plots? 1.95 0.21 965 Did you plant seed: Criolla 0.70 0.46 800 Amadeus 0.19 0.39 726 Carrizalito 0.01 0.08 717 Deorho 0.04 0.19 719 Azabache 0.00 0.00 715 Paraisito mejorado 0.02 0.13 717 Honduras nutritivo 0.00 0.00 714 INTA Cárdenas 0.00 0.04 717 Other improved 0.03 0.17 723 Other 0.16 0.37 726 Notes: These questions were asked of the sample of 968 beans farmers; differences in sample sizes due to missing values to specific questions. The most common seed used that is reported is the criolla or native; the Amadeus seed is also used by most farmers. Some of the seeds that we asked about were not planted at all, and about 16 percent of farmers noted using a seed that we did not list. Table 59 shows to whom households sell their produce and the quantities. Most beans farmers sell to intermediaries, estimated at 83 percent; 12 percent of farmers also sell their product at a local main square. TABLE 59 SALES OF BEAN PRODUCTION BY BUYERS Point to buy or sell: Mean SD Observations Local main square or farm 0.12 0.33 692 Intermediary or coyote 0.83 0.37 755 Exporter 0.00 0.00 686 IHMA 0.01 0.11 688 Point to buy or sell: Industrial 0.00 0.04 687 Organization 0.01 0.10 687 Other 0.05 0.22 684 Notes: These questions were asked of the sample of 968 beans farmers; differences in sample sizes due to missing values to specific questions. 69 We next asked about growing practices, which gives a baseline for primary research question 3 (Table 60). We find that almost all farmers state that they cleared the bean plot in the past year (97 percent). There is clearly some confusion about zero or minimum tillage, as 56 percent of farmers state they used one of those two techniques, but they should really be mutually exclusive with reporting having prepared soil (61 percent). Whereas few farmers use mechanized planting, which would optimize the planting rate, most farmers fertilize their plots (53 percent), control weeds (77 percent), or try to limit pests (80 percent). Fewer farmers attempted to do disease control (43 percent); however, they may not have experienced a recent disease outbreak. TABLE 60: PRACTICES IN BEAN PARCELS During last year in your plot, Mean SD Observations Did you clean? 0.97 0.17 967 Did you prepare the soil? 0.61 0.49 964 Did you plant with zero tillage? 0.36 0.48 963 Did you plant with minimum tillage? 0.20 0.40 966 Did you do mechanized planting? 0.07 0.26 964 Did you do fertilization? 0.53 0.50 963 Did you do weed control? 0.77 0.42 960 Did you do pest management? 0.80 0.40 965 Did you do disease control? 0.43 0.50 961 Did you harvest? 0.94 0.25 967 Did you pre-dried? 0.56 0.50 963 Did you do aporreado? 0.94 0.23 965 Notes: These questions were asked of the sample of 968 beans farmers; differences in sample sizes due to missing values to specific questions. Finally, we ask about a few remaining practices that could also improve productivity, largely related to purchased input requirements (Table 61). We ask first about whether seeds were mixed or coated with inoculant, which helps rooting and productivity. Only 5 percent of farmers use inoculants. Second, we ask about the number of seeds used per hole; the average is 3, but there is some variation to that number. Twenty-five percent of households use either a balanced formula fertilizer, but 59 percent do apply a foliar fertilizer (which is a liquid fertilizer applied to leaves of a plant). In general, there is room for improvement to yields; however, recall that those improvements are costly, and might only improve yields and not profits. 70 TABLE 61: BEANS FARMING PRACTICES Mean SD Observatio ns Did you mix the seeds with inoculant? 0.05 0.22 913 How many seeds did you use per time? 3.00 0.72 912 Did you apply fertilizer, balanced formula? 0.24 0.43 913 Did you apply foliar fertilizer? 0.59 0.49 913 Did you pre-dry? 0.52 0.50 913 Notes: These questions were asked of the sample of 968 beans farmers; differences in sample sizes due to missing values to specific questions. Beans Extension Few farmers have received extension on beans, also important for realizing the goal of the third research question (Table 62). We asked about whether farmers had received extension about nine different topics related to beans. Households reported receiving extension messages on 0.41 topics on average; most farmers (83 percent) have received no extension on beans. Among farmers reporting that they received extension, the most common topics that come up are about planting and soil preparation, then maintenance and harvest techniques. Interestingly, there are very few farmers who have received advice on marketing or market access, so a strong entry point with these farmers could be ensuring that they have better market access. Finally, among farmers who have received extension, they tend to rate it highly. TABLE 62: EXTENSION RECEIVED FOR BEANS SAMPLE Mean SD Observation s Number of topics of received extension 0.41 1.15 968 Share of farmers that received extension about: Soil Preparation 0.09 0.29 968 Planting 0.12 0.33 968 Maintenance 0.06 0.24 968 Harvest 0.06 0.24 968 Post-harvest 0.02 0.14 968 Quality and certification 0.02 0.13 968 Marketing 0.02 0.14 968 Finance and contracts 0.01 0.08 968 Market access 0.01 0.10 968 Duration of beans extension in days 40.63 97.84 167 Quality of extension, 1 to 4 3.57 0.56 160 Notes: All beans farmers have been considered for this table. All farmers who claimed not having received extension in any topic, and so they were not asked these questions directly, are coded as “no” for each topic. Sample for duration of training and quality of extension include only farmers that received extension. 71 Beans Pests We next examine damages that farmers report being caused by pests, as one specific type of the negative shocks affecting beans farmers (Table 63). About 31 percent of all farmers state that some damage was done during the primera season, whereas 43 percent of all farmers claim pest￾caused damages during the postrera season. Farmers claim substantial losses due to pests when they claim losses; an average of 7.6 quintales in the primera and 8.8 in the postrera, assuming that any losses occurred. Among the more common insects causing damage to crops were weevils, white files, and gallina ciega (a type of beetle). Other pests were not as common though a type of fungus (mancha angular) was mentioned by 11 percent of all farmers as well. Total costs reported by farmers of dealing with pests were 25.8 USD on average, conditional on any spending; note that not all farmers attempted to solve pest problems with additional inputs. TABLE 63: PESTS THAT AFFECTED BEANS CROPS Mean SD Observations Percent of damage done by pests in primera 32.41 25.40 300 Percent of damage done by pests in postrera 36.08 26.69 418 Losses in QQ due to pest damages in primera 7.57 9.47 316 Losses in QQ due to pest damages in postrera 8.79 11.29 418 Share of farmers that had problems by: Picudo de la vaina 0.19 0.39 968 Mosca blanca/White Fly 0.20 0.40 968 Gallina ciega/Fall armyworm 0.18 0.38 968 Lorito verde 0.04 0.19 968 Lepidópteros 0.03 0.18 968 Mustia hilachosa 0.09 0.29 968 Roya/Rust 0.08 0.27 968 Mancha angular 0.11 0.31 968 Mosaico viral 0.08 0.27 968 Other 0.11 0.31 968 Average cost to control pest 25.8 34.26 266 Notes: Information on having encountered each specific pest includes all farmers. Sample sizes for percent of crop damage caused by pests and pest control costs includes only farmers that had positive values for these two variables. Investments and Assets of Beans Farmers To measure investments and agricultural equipment used in bean production, we count a number of different types of investments that farmers could have made over the past 12 months (Table 64). We specifically ask about 11 types of investments. Similarly, we ask how many consumer or producer assets out of 19 farmers might own, some of which could be important in producing beans. We find that few households made investments in the past year; the average number of investments is 0.23. This number might not be all that surprising, if investments last for several years, then they do not need to be made each year. Households own, on average, 4.8 different 72 consumer or producer durables (assets) out of the 19 categories enumerated. We explain both investments and equipment further below. TABLE 64: INVESTMENTS AND ASSET COUNTS Mean SD Min Median Max Observations Total no. of investments in the last 12 months (out of 11 options) 0.23 0.56 0.00 0.00 6.00 968 Total no. of equipment owned for beans (out of 19 options) 4.82 2.23 0.00 5.00 16.00 968 A standard way to create an index from a number of potentially related variables is principal component analysis. We conduct principal component analysis around investments, and report eigenvalues for the first principal component in Table 65. The eigenvalues effectively tell us which investments were the more important in generating a variable that incorporates variation in the investments across households, with larger values being more important. We find that the “alley cultivation” investment is the most important in this regard, with barrier creation being the second and third most important. Other investments, such as plant nurseries, greenhouses, or types of investments that are tied to irrigation, are less common. TABLE 65: EIGENVALUES FOR FIRST COMPONENT OF PCA FOR INVESTMENTS Investment in the last 12 months: EV Live barriers 0.41 Dead barriers and fences 0.35 Water harvest system 0.30 Alley cultivation 0.51 Plant nursery 0.38 Greenhouses 0.13 Irrigation canal 0.19 Irrigation system 0.37 Solar dryer 0.16 Pulping machine -0.05 Other investments 0.00 Similarly, households could own 19 different types of durable goods that could either be associated with higher incomes or production (Table 66). We find that the most important in developing the first principal component are primarily consumer durables, such as televisions, radios, and cars. Equipment associated with agriculture, such as sprayers, are somewhat less important in the first principal component. 73 TABLE 66: EIGENVALUES FOR FIRST COMPONENT OF PCA FOR EQUIPMENT Type of machinery and equipment: EV Sprayer with motor 0.22 Motor mounter 0.25 Tractor 0.22 Car 0.39 Truck or pick-up 0.18 Motorcycle 0.14 Bicycle 0.01 Radio 0.41 Television 0.43 Refrigerator 0.34 Computer 0.23 Cell phones 0.29 Gas stove 0.09 Livestock (oxen, cattle) 0.08 Small livestock (goats, pigs, sheep) -0.12 Hens, Ducks, Turkeys, Pigeons -0.03 Horses, mares, mules, males, donkeys 0.05 Other -0.04 Group Membership and Participation Group membership can play a role in fostering improvements in agricultural production, though it is absolutely not sufficient (e.g. Waddington et al., 2014). Given that MAS+ has a focus on fostering group participation, particularly within the beans value chain, we investigate the existing degree of group participation in the beans sample (Table 67). Participation is defined as either active or inactive group membership, or group leadership. We find that the most common type of group are water user associations (about 25% of farmers), followed by civic or charitable groups (18%). Productive groups are rarer; only about a quarter of households report the existence of credit or microfinance groups, and only 13 percent report the existence of producer groups. TABLE 67: GROUPS IN THE COMMUNITY, BEANS SAMPLE Mean SD Observations Share of farmers that are members Exists in the community: Productive agricultural groups or org. of producers 0.13 0.34 958 0.08 Exists in the community: Productive agricultural groups of commercialization 0.02 0.13 960 0.01 Exists in the community: Group water users 0.83 0.38 967 0.25 Exists in the community: Civic or charitable groups 0.67 0.47 965 0.18 Exists in the community: Women groups 0.11 0.31 961 0.04 Exists in the community: Credit or micro-finance groups 0.25 0.43 966 0.11 Exists in the community: Insurance or mutual aid groups 0.00 0.06 965 0.00 Exists in the community: Commerce and business associations 0.00 0.03 966 0.00 Exists in the community: NGO / External Project 0.01 0.11 964 0.01 Exists in the community: Other 0.02 0.13 950 0.01 74 Evaluation Validity Tests at Baseline: Inputs, Extension and Pests for Beans Our next tests of the validity of the evaluation design focuses on inputs, extension participation and the prevalence of pests in beans farmers. First, we turn to input use (Table 68). Recall from the descriptive tables that some input use was heavy, such as specific fertilizers, whereas other inputs were seldom used, such as inoculant. Whether or not input use was heavy or light, we find no significant differences between the treatment and control groups for the use of any specific inputs. We further study both total material input costs and labor costs by gender (Table 69) in conditional figures; we find no significant differences for these variables, either. Even though the sample clusters were not chosen randomly, we find no significant differences between the treatment and control groups for outcomes related to either production outputs or inputs. 75 TABLE 68: BALANCE TEST FOR SHARED OF BEANS FARMERS THAT APPLIED INPUTS (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Applied Inoculant to promote rooting 475 0.04 493 0.07 0.40 [39] [0.01] [37] [0.03] Applied Compost/Organic fertilizer 475 0.08 493 0.09 0.81 [39] [0.02] [37] [0.02] Applied Chemical fertilizer 475 0.41 493 0.40 0.94 [39] [0.04] [37] [0.05] Applied Foliar fertilizer 475 0.56 493 0.55 0.81 [39] [0.03] [37] [0.05] Applied Fungicide 475 0.46 493 0.48 0.65 [39] [0.03] [37] [0.04] Applied Herbicide 475 0.68 493 0.67 0.74 [39] [0.03] [37] [0.04] Applied Insecticide 475 0.44 493 0.43 0.94 [39] [0.04] [37] [0.04] Applied Other Inputs 475 0.01 493 0.01 0.59 [39] [0.00] [37] [0.01] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. TABLE 69: BALANCE TEST FOR INPUT COSTS AND LABOR COSTS (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Input costs 414 1378.32 434 1350.94 0.92 [39] [196.16] [37] [182.90] Annual labor cost - men 353 18096.83 386 15488.36 0.50 [37] [2579.15] [37] [2931.83] Annual labor cost - women 353 885.48 386 284.84 0.17 [37] [421.15] [37] [112.91] Annual labor cost - total 353 18982.31 386 15773.21 0.44 [37] [2902.29] [37] [2961.16] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. We find significant differences between the treatment and control groups related to extension participation in Table 70. Most households in the treatment group have received some form of extension in the previous 12 months before the survey. We find that 145 of the 160 households reporting any extension are in the treatment group. So not surprisingly, they report having received 76 some extension on several topics, whereas the control group rarely report having received any extension. Notably, the extension appears to have been received by at least one household in many of the treatment groups, as 34 of the 37 treatment clusters have at least one household answering “yes.” It is likely that some of the farmers in the treatment areas have participated in extension programs with similar topics. However, this slight imbalance will not prevent the evaluation from providing information on the effects of MAS+. To ensure that we can understand the impacts of MAS+, it will be important to be very specific on the types of practices and themes that are being discussed in MAS+ in any midline and final questionnaires, so the lessons are accurately reflected in the evaluation. In addition, we need to be careful to document any similar programs that might occur in the same areas, and to be sure we measure the amount of participation in MAS+, so we can provide context and explore heterogeneity between those who stay in the program from beginning to end and those that leave at some point. Luckily, we find no differences between the treatment and control groups in terms of production above, so we do not have to be concerned that the relatively small share of households that have already received some extension by the baseline have received enough (or properly timed) extension to affect their beans production. The main consequence of this finding is that we will have to control for extension received prior to baseline in an appropriate way when measuring impacts of the program, for example, by directly controlling for the months or years that the farmers have participated in extension programs. To do so, we plan to collect data—both administrative and recall through the survey—on extension participation, and if it varies substantially, we can use continuous treatment effect models to measure impacts of additional participation (e.g. Imbens and Hirano, 2004). 77 TABLE 70: BALANCE TEST FOR BEANS EXTENSION (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Received extension 475 0.05 493 0.76 0.00*** [39] [0.02] [37] [0.10] Received extension about Soil preparation 475 0.01 493 0.17 0.00*** [39] [0.00] [37] [0.02] Received extension about Planting 475 0.01 493 0.23 0.00*** [39] [0.01] [37] [0.03] Received extension about Maintenance 475 0.01 493 0.12 0.00*** [39] [0.00] [37] [0.02] Received extension about Harvest 475 0.00 493 0.12 0.00*** [39] [0.00] [37] [0.02] Received extension about Post-harvest 475 0.00 493 0.04 0.00*** [39] [0.00] [37] [0.01] Received extension about Quality and certification 475 0.00 493 0.03 0.00*** [39] [0.00] [37] [0.01] Received extension about Marketing 475 0.01 493 0.03 0.01*** [39] [0.00] [37] [0.01] Received extension about Finance and contracts 475 0.01 493 0.01 0.96 [39] [0.00] [37] [0.00] Received extension about Market access 475 0.00 493 0.02 0.06* [39] [0.00] [37] [0.01] Duration of coffee extension in days 15 16.53 152 43.00 0.02** [11] [6.19] [34] [9.22] Quality of extension from 1 to 4 15 3.53 145 3.57 0.81 [11] [0.16] [34] [0.06] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. Finally, we examine balance related to exposure to pests (Table 71). We first examine whether damage differs between the treatment and control groups (rows 1 and 2), by season. We find that the treatment group appears to have suffered less damage in the postrera season than the control group, and in terms of production losses, they are lower in the treatment group in both seasons (rows 3 and 4), at the 10 percent level or better. However, we find no differences in exposure to any specific types of pests, nor in the cost of treating crops for pests. 78 TABLE 71: BALANCE TEST FOR PESTS IN BEANS (1) (2) t-test Control Treatment (1)-(2) Variable N/[Clusters] Mean/SE N/[Clusters] Mean/SE p-value Percent of damage in primera 243 20.00 280 17.37 0.32 [36] [2.32] [37] [1.26] Percent of damage in postrera 243 33.42 280 24.86 0.01*** [36] [2.61] [37] [1.64] Loss quantity due to pest in primera 137 9.04 179 6.44 0.05** [35] [1.11] [37] [0.70] Loss quantity due to pest in postrera 195 10.18 223 7.56 0.07* [34] [1.13] [35] [0.87] Had problems with picudo de la vaina 475 0.17 493 0.21 0.33 [39] [0.03] [37] [0.03] Had problems with mosca blanca 475 0.17 493 0.22 0.17 [39] [0.03] [37] [0.02] Had problems with gallina ciega 475 0.17 493 0.18 0.88 [39] [0.03] [37] [0.03] Had problems with lorito verde 475 0.04 493 0.04 0.67 [39] [0.01] [37] [0.01] Had problems with lepidópteros 475 0.03 493 0.04 0.55 [39] [0.01] [37] [0.01] Had problems with mustila hilachosa 475 0.10 493 0.09 0.79 [39] [0.03] [37] [0.02] Had problems with roya 475 0.07 493 0.10 0.17 [39] [0.01] [37] [0.01] Had problems with mancha angular 475 0.10 493 0.12 0.45 [39] [0.02] [37] [0.02] Had problems with mosaico viral 475 0.08 493 0.08 0.85 [39] [0.02] [37] [0.02] Had problems with otra 475 0.09 493 0.12 0.39 [39] [0.02] [37] [0.02] Does not know 475 0.02 493 0.02 0.90 [39] [0.01] [37] [0.01] Cost of inputs to control pest 280 305.84 320 255.70 0.58 [37] [67.59] [37] [61.91] Notes: Values displayed for t-tests are p-values. Standard errors are clustered at the primary sampling unit level. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels, respectively. The implication is that we will also need to continue to monitor whether pest exposure differs between treatment and control areas. If treatment areas randomly had fewer pests in the previous 12 months, it should not affect outcomes at the endline or after households are exposed to the intervention. However, if the control areas just have endemic problems with pests that are worse than those in treatment areas, we will have to find a way to control for those differences at endline as well. 79 In sum, we find two main differences between the treatment and control groups related to beans production. First, the beans treatment group has received substantially more extension in the past twelve months than the control group. We hypothesize that some of these farmers may have received extension with similar themes as MAS+ in the past 12 months; however, it is a relatively small number of households. We will control for this exposure in the endline by directly controlling for experience in extension programs for farmers in the households and including practices that are uniquely promoted by MAS+. Second, we find less exposure to pests, specifically losses, among the treatment group relative to the control group. For this second difference, we will keep in mind to understand at endline whether pests are more endemic to control areas, so that we can control for this difference if necessary as well. In this case, we would include a refined matching based on the prevalence of these pests in treatment and control areas. 8 Discussion and Conclusions This report has discussed the plan for the impact evaluation associated with the MAS+ project and the baseline survey conducted for that evaluation. For evaluating interventions among households growing coffee and beans, respectively, we split the sample into the households growing coffee and those growing beans. The evaluation of the coffee interventions was randomized at the cluster level, meaning that specific geographic areas were randomly selected for participation, while others will be excluded for the purpose of developing a credible counterfactual. The evaluation of the beans interventions, on the other hand, has not been randomized. To attempt to reduce selection bias, the research team pre-matched the clusters chosen for treatments with potential control groups using the available monitoring data, and selected households within control groups that provided the best match possible. The pre￾matching procedure helps minimize any unobservable differences between the targeted group for participation and the control group. After describing how the impact evaluation fits into the overall monitoring and evaluation plan for MAS+, the report describes the overall research questions and the survey modules that we use to answer those research questions. The survey was developed collaboratively between IFPRI, TechnoServe, and ESA Consultores, the firm that collected the data. Data collection took place using CAPI and specifically the Survey Solutions application. The main challenge encountered during data collection revolved around the control groups for the beans sample; some of the communities pre-selected were quite sparsely populated and required replacement. The samples that were reached are just short of the projected samples, at 961 and 968 households for coffee and beans, respectively; differences will not materially affect statistical power. The remainder of the report studies both some household characteristics of the sample, and many potential outcomes that could be affected by the interventions. After trimming some outliers in both samples, production levels seem quite comparable to other studies in Honduras. We notably find that neither the coffee nor the beans sample has been exposed to much extension, and in particular neither group show much experience with help marketing their crops. Since a main projected outcome of the MAS+ is to help farmers improve their connections to agricultural value chains, there is clearly scope for interventions helping farmers through this channel. 80 We finally provide balance tests for outcomes among both the coffee and beans samples of farmers, which measure whether observable variables differ between the prospective participant and control groups. We find that the coffee sample is quite well-balanced. The beans sample is balanced among outcomes related to area under cultivation and production, as well as inputs, but we find differences in exposure to extension. During the implementation period of MAS+ it will be important to track the areas where the program is being offered and be careful to exclude the areas that we assigned to control for this intervention. As a result, we will need to keep this point in mind during the endline analysis; it will be important to control for this exposure in data analysis after the program is complete. We summarize the baseline findings for important agricultural productivity outcomes—the production, yields, volume sold, and the total reported value of sales- in Table 72 for coffee and Table 73 for beans. Farmers in the sample have a wide spread in their agricultural productivity, which for MAS+ creates an opportunity to provide a diverse set of training that can respond to different characteristics of farmers. Namely, for farmers that are below the median of the yield distribution, training aimed at increasing productivity can bring about large impact across these outcomes. In the case of those farmers that are above the median in the productivity distribution, an intervention aimed at increasing their access better markets, to export markets, credit, certifications, etc., have the opportunity to have large impact in the total income they derive from agriculture. TABLE 72 SUMMARY BASELINE MEASURES: COFFEE KEY VARIABLES Mean SD Median Observations Coffee production in qq WP, total 70 125 34 925 Coffee production in qq WP, plantía 27 49 10 177 Coffee production in qq WP, commercial production 66 118 31 873 Coffee production in qq WP, field losses 7 13 3 239 Coffee production in qq WP, post-harvest losses 8 17 3 179 Total yields (qq/ha WP) 30.9 30.1 21.0 925 Yields in commercial area (qq/ha WP) 36.8 36.7 23.9 861 Yields in plantía (qq/ha WP) 23.0 26.6 12.9 177 Quantity of coffee sold in Wet/Húmedo, MT WP 2.60 8.33 0.78 101 Quantity of coffee sold in Berry/Uva, MT WP 3.02 8.83 1.36 638 Quantity of coffee sold in Dry/Seco, MT WP 4.19 8.80 1.40 55 Quantity of coffee sold in Green/Oro, MT WP 9.21 25.09 2.71 135 Quantity of coffee sold in Total MT WP 4.08 12.95 1.41 898 Sale value coffee berry/uva (USD) 1,521 4,071 477 99 Sale value coffee WP (USD) 2,820 3,660 1,640 621 Sale value coffee Dry/Seco seco (USD) 3,009 6,519 1,103 54 Sale value coffee oro (USD) 5,840 14,671 1,483 128 81 Total Sale value (USD) 3,220 6,887 1,500 873 Notes: Sample for beans consists of 961 farmers. Differences in sample sizes due to farmers that do not have positive quantities for the indicators. TABLE 73 SUMMARY BASELINE MEASURES: BEANS KEY VARIABLES Mean SD Media n Observations Production, Primera (kgs) 862 943 544 657 Production, Postrera (kgs) 645 761 454 738 Yields, Primera (kg/ha) 888 596 782 657 Yields, Postrera (kg(ha) 670 440 586 738 Volume Sold, Primera (MT) 0.81 0.92 0.54 535 Value of Sales (USD), Primera 631 724 369 487 Volume Sold, Postrera (MT) 0.61 0.76 0.36 541 Value of Sales, Postrera USD) 1311 6637 277 491 Beans Production Home Consumption kg in Primera 140 192 91 633 Beans Production Home Consumption kg in Postrera 144 219 91 708 Notes: Sample for beans consists of 968 farmers. Differences in sample sizes due to farmers that do not have positive quantities for the indicators. 9 Recommendations for Monitoring The results of the baseline study suggest two potential considerations for monitoring. First, note that in most cases the selected sample have demonstrated good balance between the treatment and control groups, both in the randomized coffee sample and in the non-randomized beans sample. It is important to make sure that control groups are left out of the intervention; if the intervention were to use those groups or areas, the study design would potentially be compromised. It is important to put into place measures that ensure that implementation teams do not move into control areas. Second, it is worth considering investing in measuring beans plots using GPS. The plot of production against area for beans clearly demonstrates bunching around ordinal land areas, which are almost surely farmer estimates and affect the measurement of yields. If a more accurate measure of land area can be available, it will likely tighten up the distribution of yields for beans. However, the challenge is that enumerators must be further trained, and the activity can be costly and time consuming in terms of going to the field and walking around it. If the monitoring data is clearly suggestive of impacts on yields close to the endline, we strongly suggest taking the opportunity to attempt to accurately measure beans plots. 82 References Abate, G. T., Bernard, T., de Brauw, A., & Minot, N. (2018). The impact of the use of new technologies on farmers' wheat yield in Ethiopia: evidence from a randomized control trial. Agricultural economics (Amsterdam, Netherlands), 49(4), 409-421. Cameron, C and Miller, 2015 “A Practitioner's Guide to Cluster-Robust Inference,” Journal of Human Resources, Spring 2015, Vol.50, No. 2, pp.317-373. Deaton and Cartwright, 2017, “Understanding and misunderstanding randomized controlled trials,” Social Science & Medicine, Volume 210, August 2018, Pages 2-21 Harrison, Glenn W., 2011, “Randomisation and Its Discontents,” Journal of African Economies, 20(4), 2011b, 626-652. Heckman, J, Ichimura, H and Todd, P 1997, ”Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Program,” Review of Economic Studies, pp. 605-654. Heckman, J, Ichimura, H, Smith, J and Todd, P 1998, “Characterizing Selection Bias Using Experimental Data, ” Econometrica, pp. 1017-1098. Hirano, K and Imbens, GW 2002, “Estimation of Causal Effects using Propensity Score Weighting: An Application to Data on Right Heart Catheterization,” Health Services & Outcomes Research Methodology, vol 2, pp. 259-278. Imbens, G.W., and K. Hirano, 2004. “The Propensity Score with Continuous Treatments,” working paper, Harvard University. Imbens, G. W., Newey, W. K.,, and Ridder, G. , 2005 “Mean-Square-Error Calculations for Average Treatment Effects,” IEPR Working Paper No. 05.34. McKenzie, D., 2012. "Beyond baseline and follow-up: The case for more T in experiments," Journal of Development Economics, Elsevier, vol. 99(2), pages 210-221. Ravallion, M., 2015. "On the Implications of Essential Heterogeneity for Estimating Causal Impacts Using Social Experiments," Journal of Econometric Methods, De Gruyter, vol. 4(1), pages 1-7, January. Waddington, H., Snilstveit,, B., Hombrados, J., Vojtkova, M., Phillips, D., Davies, P., White, H., 2014, “Farmer Field Schools for Improving Farmer Practices and Farmer Outcomes: A Systematic Review,” Campbell Systematic Reviews 2014:6. 83 Appendix 1: Collective Action, Transfers, Credit, Shocks, Aspirations and ICTs Coffee Sample Productivity and price expectations of coffee farmers Table 74 shows the productivity and price expectation of farmers. Their expectation of their productivity is above the realized productivity we found. For example, the median farmer expects to produce 20 quintales green per manzana under good conditions and 12 quintales green per manzana under bad conditions while the median productivity we estimate it at 11 quintales green per manzana. The prices expectations are also above the realized prices with the median price expected under bad market conditions being 50 percent above the median price in the sample. Under these measures the farmers are having a bad year. TABLE 74: COFFEE PRACTICES: ACCESS TO MARKET AND EXPECTATIONS Mean SD Min Median Max Observations Under good conditions, what would be the production in next season (QQ/MZ) 38.07 54.57 0.83 20.00 490.00 904 Under bad conditions, what would be the production in next season (QQ/MZ) 26.18 47.08 0.40 12.00 450.00 904 Under good market conditions, what would be the price next season? 2,434.27 1556.38 5.00 2250 25,000 587 Under bad market conditions, what would be the price next season? 1,592.47 717.37 2.20 1500 5,030 586 Notes: These questions were asked to sample of 961 coffee farmers. Difference in sample sizes is due to missing values to specific questions Collective Action and Aspirations We next ask about whether communities participate in a variety of different types of collective action (Table 75). We find very few examples of collective action; the most common one is either shared work or work exchanges and water conservation community work, but only 7 percent of households report them in their communities. Other types of community based collective action are rare, which suggest that community cohesion is not strong in rural Honduras. 84 TABLE 75: COLLECTIVE ACTION Mean SD Observations Community participated in: Construction for water conservation 0.07 0.25 960 Community participated in: Joint purchase of agricultural inputs 0.02 0.13 958 Community participated in: Shared work or hand back 0.06 0.23 960 Community participated in: Joint agricultural certification 0.00 0.03 960 Community participated in: Gathering / selling joint agricultural products 0.01 0.07 957 Community participated in: Other collective development activity 0.01 0.11 957 We next explore a set of questions asked about aspirations, control of one’s fate, and self-esteem (Figure 15). The bars left to right measure the proportion of the sample answering strongly agree to strongly disagree. The different questions show farmers tend to agree with the statements that mark independence and agency, having access to help in the community, and life improving in the future. 85 FIGURE 15: ASPIRATIONS, LOCUS OF CONTROL AND SELF-ESTEEM PERCEPTIONS 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% You can solve problems with your crops without help You can decide what you are going to do in your life If an adverse weather event occurs, however much you do, you… You are very good at taking care of the crops you have You can negotiate the sale price of your products and get better… In the future, your life condition will improve It is very important that parents keep their promises to their children You have specific goals and plans for future growth of your crops You are satisfied with the production level of your plot / business Your neighbors don't like you You do not have much to be proud of Other people like to work with you Most neighbors would be willing to help you if needed You are afraid to try new things / something new You have had conflicts with neighbors for when to sell or who to… You have had conflicts with neighbors for those who participate in… Strongly agree Agree Disagree Strongly disagree 86 Transfers and Remittances Farmers can have different sources of income. Some of them are transfer from social programs or remittances from migrants abroad. We find that fully 20 percent of the sample received remittances from outside the country. The most common types of transfers received from public programs are the Bono Vida Mejor and Bono 10,000 programs; about 20 percent of households received one of these transfers. TABLE 76: REMITTANCES AND TRANSFERS RECEIVED BY COFFEE SAMPLE Mean SD Observations During the last 12 months, received: Remittances from relatives out of the country 0.20 0.40 958 During the last 12 months, received: Remittances from relatives within the country 0.05 0.22 959 During the last 12 months, received: Bono Vida Mejor/Bono 10,000 0.20 0.40 959 During the last 12 months, received: Stove, Ecofogón Vida Mejor 0.08 0.28 959 During the last 12 months, received: 3rd age bonus 0.00 0.06 957 During the last 12 months, received: Bolsa Solidaria 0.08 0.28 959 During the last 12 months, received: Fees, alimony 0.00 0.06 959 During the last 12 months, received: Other 0.00 0.07 932 Credit We track household participation in group-based savings, microfinance, or lending programs. Group-based savings programs are formal or informal community programs that serve as a mechanism for people in poor communities with otherwise limited access to financial services to pool their savings. The specific composition and function of the savings groups group vary and can include rotating loan disbursement. The definition is inclusive of all of the different types of group-based savings programs. According to the World Bank, microfinance can be defined as approaches to provide financial services to households and microenterprises that are excluded from traditional commercial banking services. Typically, these are low-income, self-employed or informally employed individuals, with no formalized ownership titles on their assets and with limited formal identification papers. The benefits of financial inclusion include: lower transaction costs of day to day interactions (e.g. Mobile Money), ability to grow savings to smooth consumption and mitigate against shocks, and access to credit to invest in Micro, Small and Medium enterprises (MSME). Group-based savings programs are formal or informal community programs that serve as a mechanism for people in poor communities, with otherwise limited access to financial services, to pool their savings. The specific composition and function of the savings groups group vary and can include rotating disbursement as well as accumulating savings models. A household is considered to be participating in group-based savings, micro-finance or lending program if any member of the household saved money with or took a loan or borrowed cash or in-kind from a group-based savings, micro-finance or lending program in the past 12 months. Only 27 percent (or 258 of 961 households) have received credit in the past 12 months (Table 77). Most of the credit they received comes from formal banks and credit and savings associations. 87 TABLE 77: CREDIT FOR COFFEE SAMPLE Mean SD Observatio ns In the last 12 months, did you receive credit from a public or private institution? 0.27 0.44 961 Institution that gave you credit: BANHCOFFEE 0.02 0.15 258 Institution that gave you credit: Another bank 0.26 0.44 258 Institution that gave you credit: Savings and Credit Cooperative 0.24 0.43 258 Institution that gave you credit: Another regulated formal institution 0.14 0.35 258 Institution that gave you credit: Agricultural Cooperative or Producers Association 0.05 0.21 258 Institution that gave you credit: Communal bank or other unconventional 0.02 0.14 258 Institution that gave you credit: Caja Rural / Caja Municipal 0.17 0.38 258 Institution that gave you credit: Grupo solidario 0.01 0.09 258 Institution that gave you credit: NGO or Project 0.02 0.12 258 Institution that gave you credit: SAG program or government 0.00 0.00 258 Institution that gave you credit: Other non-regulated institution 0.02 0.14 258 Institution that gave you credit: Farming companies 0.00 0.06 258 Institution that gave you credit: Agricultural input store 0.00 0.06 258 Institution that gave you credit: Another store / merchant 0.01 0.11 258 Institution that gave you credit: Coyote / Local Copier 0.03 0.17 258 Institution that gave you credit: Coyote / foreign collector 0.00 0.00 258 Institution that gave you credit: Lender 0.01 0.11 258 Institution that gave you credit: Another farmer of the municipality 0.00 0.00 258 Institution that gave you credit: Family member or friend 0.02 0.14 258 Institution that gave you credit: Another informal lender 0.00 0.06 258 Shocks Negative shocks can have deleterious effects on household production, and so it is important to measure shocks to understand whether they might have affected households. We ask households in the sample whether they have experienced negative shocks over the past 12 months (Table 78). Many households noted experiencing shocks in the past 12 months; the most common reported negative shock was an increase in agricultural input prices (64 percent), followed by low agricultural product prices (46 percent). Notably, 20 percent also reported being affected by heavy rains, so both a lack of rain and too much rain affects different parts of the sample. These measures of shocks may be helpful in controlling for differences in productivity among farmers. 88 TABLE 78: SHOCKS EXPERIENCED IN THE LAST 12 MONTHS Events that affected you in the last 12 months Mean SD Observations Drought 0.23 0.42 947 Fires / burning 0.03 0.18 947 A lot of rain / floods 0.20 0.40 947 Migration of a household members 0.03 0.17 947 Pests / animal disease 0.14 0.35 947 Sick people working at home 0.16 0.37 947 Low agricultural product prices 0.46 0.50 947 Road strikes 0.07 0.25 947 Increase in agricultural input prices 0.64 0.48 947 Less work available than before 0.27 0.44 947 Lack of contracted labor 0.30 0.46 947 Robbery / assaults on house 0.02 0.13 947 Violent death of a household member 0.01 0.11 947 Information and Communication Technologies The survey form asked about where households received information about agriculture, more generally (Table 79). The most common source of agricultural information is the radio, followed by the TV; however, only 19 percent of households mentioned the radio, and only 8 percent mentioned the TV. Other sources of agricultural information were mentioned very infrequently, meaning by only 1 or 2 percent of sample households at most. In general, the majority of sample households appear not to receive much widely disseminated agricultural information. TABLE 79: SOURCES OF RECEIVED AGRICULTURAL INFORMATION Mean SD Observations Sources of agricultural information: Radio 0.19 0.39 957 Sources of agricultural information: TV 0.08 0.28 959 Sources of agricultural information: Internet 0.01 0.12 957 Sources of agricultural information: Messages 0.00 0.06 960 Sources of agricultural information: Newspapers 0.00 0.06 960 Sources of agricultural information: Merchant 0.02 0.14 956 Sources of agricultural information: Supplies 0.01 0.08 959 Sources of agricultural information: Association 0.02 0.14 959 Sources of agricultural information: Other 0.01 0.11 938 Finally, we asked about some specific uses of the internet and cell phones (Table 80). Although not specifically asked, through the averages presented 28 percent of households have smartphones, since they report both accessing the internet and using Whatsapp, a smartphone application. A few households report owning a computer or using a neighbor’s computer to access the internet. Most of the sample households do not have internet access. Far more households have cell phones; whereas only a few use cell phones to buy or sell agricultural products, the majority of the sample reports either sending or receiving messages by phone. 89 TABLE 80: INTERNET AND CELLULAR USE Mean SD Observatio ns Internet access by: Own computer with internet 0.03 0.17 960 Internet access by: Computer of a negihbor or family 0.01 0.07 957 Internet access by: Mobile internet (cellphone) 0.28 0.45 960 Internet access by: Cybercoffee 0.00 0.03 958 Internet access by: Library, school, other institution 0.00 0.05 959 Internet access by: Other 0.00 0.00 953 Use cellular to: Make purchases of agricultural products or processed food 0.07 0.25 950 Use cellular to: Sales of agricultural products or processed food 0.08 0.28 950 Use cellular to: Send messages in general 0.54 0.50 960 Use cellular to: Receive messages in general 0.61 0.49 958 Use cellular to: use Whatsapp 0.28 0.45 955 90 Beans Sample Collective Action and Aspirations We ask about whether communities participate in a variety of different types of collective action (Table 81). We find very few examples of collective action; the most common one is either shared work or work exchanges, but only 11 percent of households report them in their communities, which does not mean they participate in labor exchanges. Other types of community based collective action appear quite rare, which is consistent with the idea that community cohesion is not strong in rural Honduras. TABLE 81: COLLECTIVE ACTION Mean SD Observations Community participated in: Construction for water conservation 0.05 0.22 968 Community participated in: Joint purchase of agricultural inputs 0.01 0.12 967 Community participated in: Shared work or hand back 0.11 0.31 967 Community participated in: Joint agricultural certification 0.00 0.06 965 Community participated in: Gathering / selling joint agricultural products 0.01 0.12 967 Community participated in: Other collective development activity 0.02 0.14 965 As with the coffee sample, we next explore a set of questions asked about aspirations, control of one’s fate, and self-esteem (Figure 16). And as with the coffee sample, the bars left to right measure the proportion of the sample answering strongly agree to strongly disagree. 91 FIGURE 16: ASPIRATIONS, LOCUS OF CONTROL AND SELF-ESTEEM PERCEPTIONS 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% You can solve problems with your crops without help You can decide what you are going to do in your life If an adverse weather event occurs, however much you do, you can not… You are very good at taking care of the crops you have You can negotiate the sale price of your products and get better price… In the future, your life condition will improve It is very important that parents keep their promises to their children You have specific goals and plans for future growth of your crops You are satisfied with the production level of your plot / business Your neighbors don't like you You do not have much to be proud of Other people like to work with you Most neighbors would be willing to help you if needed You are afraid to try new things / something new You have had conflicts with neighbors for when to sell or who to sell in… You have had conflicts with neighbors for those who participate in… Strongly agree Agree Disagree Strongly disagree 92 Remittances High migration rates in Honduras make it important to ask about remittances, as they can affect production either positively or negatively. We also ask about other types of transfers that farm households might have received, largely related to social programs (Table 82). We find that fully 21 percent of the sample received remittances from outside the country, which is similar to the coffee sample; however, more households in the bean sample receive remittances from within the country (9 percent) than within the coffee sample (5 percent). The most common types of transfers received from public programs are the Bono Vida Mejor and Bono 10,000 programs; about 19 percent of households received one of these transfers, whereas 13 percent received transfers from Bolsa Solidaria. TABLE 82: REMITTANCES AND TRANSFERS RECEIVED BY BEANS SAMPLE Mean SD Observations During the last 12 months, received: Remittances from relatives out of the country 0.21 0.41 966 During the last 12 months, received: Remittances from relatives within the country 0.09 0.29 967 During the last 12 months, received: Bono Vida Mejor/Bono 10,000 0.19 0.39 963 During the last 12 months, received: Stove, Ecofogón Vida Mejor 0.10 0.30 967 During the last 12 months, received: 3rd age bonus 0.00 0.06 966 During the last 12 months, received: Bolsa Solidaria 0.13 0.33 967 During the last 12 months, received: Fees, alimony 0.00 0.03 966 During the last 12 months, received: Other 0.00 0.05 949 Credit Whereas finance is not seen as a constraint to bean production as it may be for coffee production, nonetheless we are interested in whether bean producing households have received any credit. We find that only 14 percent of households have received formal or informal credit in the past 12 months (Table 84). Among those reporting they received credit, banks other than the BANHCOFFEE, perhaps not surprisingly, are the most common source at 35 percent. Other answers were the credit and savings association at 20 percent, and then a caja rural or caja municipal, which is an informal institution, at 16 percent. Nonetheless, the beans sample appears to have very little access to credit. 93 TABLE 83: CREDIT RECEIVED BY BEANS SAMPLE Mean SD Observatio ns In the last 12 months, did you receive credit from a public or private institution? 0.14 0.35 968 Institution that gave you credit: BANHCOFFEE 0.01 0.08 140 Institution that gave you credit: Another bank 0.35 0.48 140 Institution that gave you credit: Savings and Credit Cooperative 0.20 0.40 140 Institution that gave you credit: Another regulated formal institution 0.13 0.34 140 Institution that gave you credit: Agricultural Cooperative or Producers Assoc. 0.02 0.15 140 Institution that gave you credit: Communal bank or other unconventional 0.04 0.20 140 Institution that gave you credit: Caja Rural / Caja Municipal 0.16 0.37 140 Institution that gave you credit: Grupo solidario 0.01 0.12 140 Institution that gave you credit: NGO or Project 0.01 0.08 140 Institution that gave you credit: SAG program or government 0.01 0.08 140 Institution that gave you credit: Other non-regulated institution 0.01 0.08 140 Institution that gave you credit: Farming companies 0.01 0.08 140 Institution that gave you credit: Agricultural input store 0.01 0.08 140 Institution that gave you credit: Another store / merchant 0.01 0.12 140 Institution that gave you credit: Coyote / Local Copier 0.00 0.00 140 Institution that gave you credit: Coyote / foreign collector 0.00 0.00 140 Institution that gave you credit: Lender 0.01 0.08 140 Institution that gave you credit: Another farmer of the municipality 0.00 0.00 140 Institution that gave you credit: Family member or friend 0.02 0.15 140 Institution that gave you credit: Another informal lender 0.00 0.00 140 Shocks Again, negative shocks can have a very important impact on agricultural production, so we also ask households in the bean sample whether they have experienced negative shocks over the past 12 months (Table 84). Many households noted experiencing shocks in the past 12 months; the most common reported negative shock was an increase in agricultural input prices (51 percent), followed by drought (47 percent). Notably, 41 percent also reported being affected by low agricultural output prices; it is not clear from this table whether farmers actually experienced decreases in output prices or increases in input prices, or if they are noting what they perceive as regular conditions. Also notably, 28 percent also reported being affected by heavy rains, so both a lack of rain and too much rain affects different parts of the sample. These measures of shocks may be helpful in controlling for differences in productivity among farmers. 94 TABLE 84: SHOCKS EXPERIENCED DURING THE LAST 12 MONTHS, BEANS SAMPLE Events that affected you in the last 12 months Mean SD Observation s Drought 0.47 0.50 959 Fires / burning 0.12 0.32 959 A lot of rain / floods 0.28 0.45 959 Migration of a household members 0.03 0.18 959 Pests / animal disease 0.14 0.35 959 Sick people working at home 0.12 0.33 959 Low agricultural product prices 0.41 0.49 959 Road strikes 0.08 0.27 959 Increase in agricultural input prices 0.51 0.50 959 Less work available than before 0.21 0.41 959 Lack of contracted labor 0.17 0.37 959 Robbery / assaults on house 0.02 0.15 959 Violent death of a household member 0.01 0.10 959 Information and Communication Technologies As with the coffee sample, we ask about how farmers receive agricultural information and about their use of the internet and other information and communication technologies. First, we ask about whether farmers receive agricultural information from different sources, such as the radio, television, etc. We find very low response rates, suggesting that bean farmers are not receiving much information from widely disseminated sources (Table 85). The most common source of information mentioned is radio, followed by television, but only 10 and 6 percent of the sample mentioned those two sources, respectively. TABLE 85: SOURCES OF AGRICULTURAL INFORMATION Mean SD Observations Sources of agricultural information: Radio 0.10 0.31 967 Sources of agricultural information: TV 0.06 0.23 967 Sources of agricultural information: Internet 0.00 0.06 967 Sources of agricultural information: Messages 0.00 0.06 967 Sources of agricultural information: Newspapers 0.01 0.08 964 Sources of agricultural information: Merchant 0.01 0.10 964 Sources of agricultural information: Supplies 0.02 0.14 965 Sources of agricultural information: Association 0.01 0.10 966 Sources of agricultural information: Other 0.05 0.21 956 Similar to the coffee sample, we find that around 26 percent of the beans sample appear to have a smartphone, as 26 percent use the internet on a mobile device and 27 percent report using Whatsapp (Table 86). Otherwise, few respondents appear to have access to the internet. 95 Households report using their cell phones to send and receive messages most often; as with the coffee sample, few households appear to be either purchasing or selling agricultural products on their phones, at 11 and 16 percent, respectively. TABLE 86: INTERNET AND CELLPHONE USE Mean SD Observations Internet access by: Own computer with internet 0.03 0.16 966 Internet access by: Computer of a negihbor or family 0.00 0.06 967 Internet access by: Mobile internet (cellphone) 0.26 0.44 968 Internet access by: Cybercoffee 0.01 0.09 967 Internet access by: Library, school, other institution 0.01 0.08 967 Internet access by: Other 0.00 0.00 965 Internet access by: Not applicable 0.01 0.10 911 Use cellular to: Make purchases of agricultural products or processed food 0.11 0.31 967 Use cellular to: Sales of agricultural products or processed food 0.16 0.37 965 Use cellular to: Send messages in general 0.48 0.50 966 Use cellular to: Receive messages in general 0.51 0.50 964 Use cellular to: use Whatsapp 0.27 0.45 960