Nicaragua FY 2014 Food for Progress
PROGRESA Caribe Project 
Baseline Evaluation 
2015
Nicaragua FY 2014 Food for Progress
PROGRESA Caribe Project
Program: Food for Progress
Agreement Number: FCC-524-2014/056-00
Funding Year: Fiscal Year 2014
Project Duration: 2014-2022
Implemented by: CRS
Evaluation Authored by: FUNIDES (Fundacion Nicaraguense Para el Desarrollo 
Economico y Social)
DISCLAIMER: This publication was produced at the request of the United States Department of 
Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views
expressed in this publication do not necessarily reflect the views of the United States Department of 
Agriculture or the United States Government.
Accessibility Note: An accessible version of this document can be made available by contacting 
fas.monitoring.evaluation@usda.gov
PROGRESA Caribe Baseline Report1
1. Introduction
This report presents the results of the baseline of the Program for Rural Enterprise, Health and 
Environment – Caribbean Zone (hereinafter PROGRESA Caribe), which will be implemented in 
consortium by Catholic Relief Services (CRS), TechnoServe (TNS) and Lutheran World Relief (LWR).
The document is composed of eight sections. The second section describes the background and 
description of the intervention. In the third section, the objectives of the evaluation are 
presented. Section four discusses the evaluation design (original design and actual program 
participants and nonparticipants). Section five deals with the sampling strategy and data. The sixth 
section explains the validation of evaluation design. Section seven presents comprehensive 
descriptive statistics on beneficiaries, producer organizations and private enterprises. Conclusion 
and recommendations for implementation are presented in section eight.
2. Background and description of the intervention
CRS, TNS and LWR are organizations with vast experience in agricultural development projects. 
Among these projects the following are highlighted: a) PROGRESA I, which is executed by CRS in 
the North-Central region of the country, benefitting 4,700 producers of dual-purpose livestock and 
other value chains; b) Livestock Business (GANE), which is executed by TNS benefitting around 
4,000 livestock farmers in Matagalpa, Jinotega, RACN and RACS; and c) ACORDAR, which was 
implemented by CRS with LWR as a key partner.
PROGRESA Caribe is a project to be carried out by the consortium formed by CRS, TNS and LWR. It 
will have a duration of five years and it aims to strengthen the value chain of cocoa and (dual 
purpose) livestock in the Nicaraguan Caribbean Region of Nicaragua. PROGRESA Caribe builds 
upon those current programs by expanding into new areas of need in RAAN, RAAS, and Rio San 
Juan, and by continuing to work with producers participating in current programs to build a critical 
mass of technical knowledge and commercial capacity that will transform the cacao and livestock 
value chains in Nicaragua and allow producers to capture greater shares of income and profit as 
their value chains become better articulated and their participation more proactive and organized. 
The decision to work along geographic boundaries has generated a selection of beneficiaries by 
administrative rule. The methodological implication of this type of selection will be discussed in 
section five.
 
1
The baseline report of PROGRESA Caribe has been prepared by FUNIDES team (hereinafter evaluation 
team) with the support of CRS´s MEAL team and the Knowledge Management Division.
The project has as target population of 4,247 small cocoa and/or livestock producers. The activities 
will be focused on existing producers, instead of promoting new plantations. The selected 
producers to participate in the project are from 13 municipalities: Waslala, Siuna, Bonanza and 
Rosita in RACN; El Ayote, La Cruz de Rio Grande, Muelle de los Bueyes, Bocana de Paiwas, El Rama, 
Bluefields, and Nueva Guinea in RACS; and San Carlos and El Castillo in Rio San Juan.
2.1. Strategic objectives
The project activities will be aimed at fulfilling two strategic objectives: 
1) Increase agricultural and livestock productivity.
2) Expand trade of cacao and cattle products.
Improvement in productivity will be addressed by providing inputs and facilitating genetic 
material, credit to the productive activity, technical assistance and training. This will enable 
improved production techniques and enhance the administration capacity of the farms. The 
expansion of cacao trade, dairy products and meat will be sought through training in business 
management to producers and cooperatives; facilitating construction and improvement of the 
post-harvest processing infrastructure; to increase access to markets, the activities will promote 
the participation in national tasting contests and the establishment of public-private and private￾private alliances, as well as the leveraging of resources from the private sector and investments 
from the public sector.
2.2. Project activities
CRS and its consortium partners will implement a cohesive set of activities that will mitigate value 
chain weaknesses in four key areas of the cacao and livestock value chains: production, post￾harvest handling, processing and marketing. Activities to increase agricultural productivity in the 
target value chains will be focused on training to improve agricultural production techniques and 
farm management, as well as fomenting farm-level financial and non-financial services that 
producers need in order to implement the best practices. 
Activities to expand trade will focus on increasing market access for smallholder producers 
through improving post-harvest handing and processing, training on certifications and standards, 
and facilitating private and/or public partnerships for buying and selling, co-investment in 
infrastructure, and the exchange of market information. Complementary foundational activities 
include increasing the capacity of cooperatives and trade associations to serve their smallholder 
members.
PROGRESA Caribe will help farmers achieve an increase in agricultural productivity of cocoa and 
dairy/beef products through activities that increase the availability of improved inputs, and 
increase farmers' knowledge of improved techniques and technologies and farm management.
The increase in productivity of cacao and livestock will occur through improved knowledge among 
producers of improved technologies and techniques. In cacao, the supervisors and the technical 
promoters will use a tested and proven toolkit developed by LWR in coordination with cacao 
producers. Composed of ten practical guides, it addresses the following themes: 1) agroforestry 
models, 2) cacao plantations, 3) propagation in nurseries, 4) soil fertility, 5) pruning and 
maintenance, 6) integrated pest management, 7) management of vegetative cover, 8) harvest, 
fermentation, and drying, 9) certification, and 10) commercialization. 
In livestock, the consortium partners will promote rotational grazing using improved grasses and 
cut-grass, using mineral salts, planting leguminous plants in pastures and cacao fields to provide 
additional protein, and incorporating trees in grazing systems. In addition, farmers will be trained 
in animal management, nutrition, and health.
The project will offer technical assistance using cascade training and demonstration plots. Each 
technical supervisor will be assigned to work with a group of community promoters who will each 
in turn work with neighboring producers. This cascade focus has been previously implemented in 
the ACORDAR project (financed by USAID), PROGRESA I (CRS - USDA) and GANE (TechnoServe -
USDA).
PROGRESA Caribe will help Nicaraguan farmers expand trade of cocoa, dairy and beef products in 
domestic, regional and international markets through activities that strengthen productivity and 
product quality, increasing the use of improved post-production processing and handling 
practices, and improving post-harvest infrastructure.
The Project will train producer cooperatives and their members in improved post-harvest handling 
techniques for cacao and dairy products. Utilizing the tested LWR Cocoa Toolkit, and building upon 
proven CRS experience working with cacao producers in Matagalpa and Siuna, cacao producers 
will learn appropriate harvest, sorting, pod splitting and transport techniques for superior quality. 
CRS and its partners will also train producer cooperatives to improve fermenting techniques, 
utilize tools, such as moisture meters to ensure quality, and to dry fermented beans in sufficient 
quantities to meet buyer demand. 
Similarly, dairy producers and cooperatives will be trained in hygienic milking techniques, 
improved storage technologies, transport and spoilage times, and first-level processing such as 
cheese production and chilling. Cooperatives will be trained to test for contamination, water 
content and eventually milk solids composition to ensure high quality milk is delivered to buyers. 
The Project will also promote improved post-harvest handling techniques to milk buyers, and 
facilitate negotiations for quality price premiums for cooperatives utilizing post-harvest best 
practices.
Producers will also be trained to better manage their farms as a business, including improved 
record keeping, tracking of true costs of production, including family labor, transportation costs, 
sales prices and volumes, and ultimately net income of agricultural activities. As a result of this 
training, producers will be encouraged to adopt Farm Plans to formalize production plans, to 
project income and costs, and to measure performance against pre-established farm-level targets. 
Through cooperative enterprises in the cacao value chain, the project will facilitate the 
construction and improvement of post-harvest processing infrastructure including fermentation 
trays, drying tunnels, ovens, moisture control tools and thermometers. This will be accomplished 
through direct project investments, potential co-investments with private companies (such as 
Ritter Sport in RAAS and Norteak in RAAN), and by facilitating credit to cooperatives and 
companies interested in value adding processes. CRS, LWR, and TNS have already identified 
several key opportunities to establish or improve cacao processing infrastructure throughout the 
Atlantic Region. For the milk value chain, infrastructure investments include small milk collection 
centers closer to target producers and improved cheese processing equipment. 
To support all activities and particularly market access objectives, CRS will facilitate private-private 
and private-public partnerships to increase market access and leverage investment in the cacao 
and livestock value chains, linking producer cooperatives to businesses that can provide inputs in 
exchange for purchase contracts, co-invest in post-harvest handling and processing facilities, and 
provide technical assistance to increase producer knowledge of quality standards.
3. Objectives of the evaluation 
3.1. Hypothesis and theory of change2
The theory of change of PROGRESA Caribe has been built from the results framework, budget 
headings detailed in the program´s implementation plan and of the indicators established in the 
partnership agreement of the consortium partners with USDA. It is also consistent with the theory 
of change of agricultural livelihoods established in CRS´ Latin America Regional Office´s strategy.
The theory of change has been modeled in a results chain, where the casual logic is defined from 
the initiation of the project to the end, taking into account the following elements: activities (the 
how), products (why), results (if), final results (then).3 Graphics 1 and 2 show the results chain for 
the two strategic objectives: 1) improving agricultural and livestock productivity and 2) expand 
trade. In addition, graphics in annexes A and B present the project-level results framework.
Based on the results chain, we want to test the following hypothesis in the impact evaluation:
 If the program activities as a whole improve agricultural and livestock productivity and if the 
trade of cacao, dairy products and meat expands, this will increase the income of the farmer 
families and will contribute to poverty reduction in the areas of intervention.
In this sense, the hypothesis to test must identify the particular effect of the intervention as a 
whole compared to which would be the outcome in terms of family income and poverty level if 
the beneficiaries had not received the services of the project.
3.2. Key outcome indicators
The indicators to evaluate include those (intermediate) outcomes that are achieved once the 
beneficiary population uses the project outputs, as well as the final outcomes of the intervention, 
which are in correspondence with the strategic objectives. Also, the implementation indicators 
established in Annex E of PROGRESA Caribe´s cooperative agreement will be evaluated. In this 
manner, an evaluation throughout the results chain and a better identification of the causes of 
program impact will be guaranteed. Graphics 3 and 4 present the intermediate indicators matrix 
 
2
The CRS team has prepared an expanded version of PROGRESA Caribe´s theory of change, which deals with 
the chain results for each strategic objective, product, input and foundational result.
3
The sequence of the chain results can also split the inputs, i.e., the resources available for the project 
(budget, staff, etc.). PROGRESA Caribe´s theory of change takes into account this element in its formulation.
and the outcome matrix, respectively. The description of each indicator, calculation formula, 
frequency of measurement and means of verification is discussed.
Graphic 1. Results chain – strategic objective 1
Graphic 2. Results chain – strategic objective 2
Graphic 3. Matrix of Intermediate indicators
Description Calculation Formula Measuring Frequency Means of Verification
Liters of milk per cow per day
This indicator measures the productive
efficiency of cattle herd
Baseline survey, mid-term
survey, final survey
Module: Livestock
production
Yields in the cacao production
The total cacao production volume in
(Metric Tons) by producers divided by the
total number of production hectares 
Baseline survey, mid-term
survey, final survey Module: Agriculture
Average cattle weight per hectare 
This indicator will measure the total kilos of
live animals that the producer has in the
herd inventory, and are supported by
USDA
Baseline survey, mid-term
survey, final survey
Module: Livestock
production
Cocoa Production Costs
Total production cost per hectare measured
in dollars for the cacao category. All sales
related with cacao (dry and baba)
production are included, as well as genetic
material
Baseline survey, mid-term
survey, final survey Module: Agriculture
Production Cattle Costs
Total cost of milk sales, derived from milk
and cattle, by hectare of pasture and
measured in dollars
Baseline survey, mid-term
survey, final survey
Module: Livestock
production
Gross Margin per Hectare (cocoa)
Increase in income attributable to
investments made in improving production
techniques and technology implementation
Baseline survey, mid-term
survey, final survey Module: Agriculture
Gross Margin per hectare
(livestock)
Increase in income attributable to
investments made in improving production
techniques and technology implementation
Baseline survey, mid-term
survey, final survey
Modules: 1) Livestock
Production and 
2) Cattle Sales
Number of hectares managed
under sustainable agricultural
practices
Number of hectares of cocoa and livestock
managed under sustainable practices, such
as using eco-friendly fertilizers and
pesticides, erosion control, agroforestry, etc.
Baseline survey, mid-term
survey, final survey
Modules: 1) Livestock
Production and 
2) Cattle Sales
Indicator
Intermediate 
Indicators
 
 
 
 
 
 
 
 
 
 
*100
 
 
*100
 
 
 
 
 
Graphic 4. Matrix of outcome indicators
Description Calculation Formula Measuring Frequency Means of Verification
Progress out of Poverty Index
The PPI is a simple index rating of poverty
that estimates the probability that a home
has costs less than a certain poverty line.
The index uses ten simple indicators for
collection on-site.
Baseline survey, mid-term
survey, final survey
Modules: 
Poverty Index PPI
Family Income
Family income received by the sale of
production and other income from some
family member(s)
Baseline survey, mid-term
survey, final survey
Modules: 1)
Agriculture 2) Sale of
Livestock and 3) Food
Security (socio￾demographic 
characteristics
** HDDS: Total number of food groups
consumed by families 
Months of adequate supply of
food in the household (MAHFP)
Capture changes in the home ability to
approach the vulnerability so as to
guarantee the procurement above a
minimum level throughout the year.
Measuring the MAHFP has the advantage of
obtaining the combined effects of a series of
interventions and strategies, for a better
agricultural production, storage and
activities that increase the families’
purchasing power.
Baseline survey, mid-term
survey, final survey
Modules:
Food Security 
Return of the producer’s
investment and processing of
cooperatives activities
It measures the reduction in production
costs and the increase of income as a result
of the quality achieved through the
efficiency of processing activities. Evaluates
the performance of the investment made by
the producer or cooperative in operation
(efficiency of spending to maximize sales)
Annual
Administrative 
Records 
Number of individuals receiving
financial services as a result of
USDA´s assistance
Number of producers who have access to
financial services, as a result of USDA´s
support. Financial services include those
that help identify and access funds through
formal and alternative channels.
Bi-annual
Administrative 
Records 
Indicator
Outcome 
indicators
Household dietary diversity scores
It collects information about the families´
food safety. Focuses on the composition of
the regular family diet, if they suffer some
kind of food shortage and in which months
this scarcity predominates.
Baseline survey, mid-term 
survey, final survey
Modules:
Food Security 
 
 
 
 
 100
 
 
 
*100
 
 
 
 
 
3.3. Key evaluation questions
The impact evaluation will also provide answers to key questions about the relevance, 
effectiveness, efficiency, impact and program sustainability, as well as specific learning questions 
in value chains that are important to USDA, consortium partners, donor community and other key 
stakeholders. These questions are listed below:
Relevance:
 To what extent did the project meet the needs of the project beneficiaries and take into 
account the economic, cultural and political context?
 To what extent did the project fit within the Government of Nicaragua’s economic and 
agricultural development policies and programs?
Effectiveness:
 To what extent has the project achieved its stated objectives?
 Did the project increase farmers’ productivity in cacao and dairy/beef products?
 Did the project enhance the quality and safety of the food produced by participating farmers?
 Did the project increase participating farmers’ sale of products?
 Did the project ensure food security of participating farmers’ family?
 Did the project increase participating farmers’ family income?
 Did the project reduce the incidence of poverty in the intervention areas?
Efficiency:
 To what extent has the project achieved benefits more cost-effectively than other efforts to 
achieve the same ends?
Impact:
 Which are the medium and long-term effects, both intended and unintended, of the project 
intervention?4
Sustainability of outcomes and impacts:
 To what extent are the project’s activities sustainable?
 Will project activities continue in the absence of support from both USDA and consortium 
partners? If so, how? If not, why not?
 Will the effects of the project on producers, producer organizations, partner organization 
sustain overtime after the project ends?
 To what extent are the outcomes and impacts of the program resilient to risk over time?
Key value chain learning questions:
 What is the return on investment (technology, infrastructure, etc.) that the participating 
families (and their organizations) achieve as a result of their participation in the value chains?
o What is the relationship between infrastructure investment and changes in sales and 
income at both the household and cooperative level?
 
4
The effects can be both direct or indirect and positive or negative. The evaluation will assess to which 
extent the effects are due to the project intervention and not to other factors.
 How do the different enterprise models established for value chain development affect impacts 
at the producer and cooperative levels?
o In what measure do the enterprise models generate incentives for private sector businesses 
to invest in productive infrastructure and provision of services (technical and financial) for 
smallholder producers?
o What is the household-level impact of affiliation with cooperatives that are connected with 
value chains?
 What is the relationship between compliance with GPP norms (cattle) and organic or other 
certifications (cacao), changes in productivity, and income?
o What are the most effective incentives to promote compliance with sanitary and 
phytosanitary norms?
o What are the key success factors for leveraging sanitary and phytosanitary practices into 
access to higher value markets?
o Do certified producers have higher net income than non-certified producers?
All these questions will be answered in the final evaluation of the program, however, those 
questions related to effectiveness and efficiency as well as questions of learning in the value chain 
can be addressed from the mid-term evaluation.
4. Evaluation Design
The evaluation of PROGRESA Caribe will follow a mixed-method approach, i.e., both quantitative 
and qualitative methods will be integrated. Bamberger (2012) states that this approach allows for:
a) triangulation of evaluation findings, which strengthens the validity and credibility of the results; 
b) using information from one method to develop the instrument for the other; c) 
complementarity of the results; d) diversity in the value dimensions of the evaluation; and e) 
generating new knowledge into evaluation findings.
This section is divided in two parts. The first part discusses the experimental methodology to 
measure the impact of the project on families’ welfare (quantitative approach). The second part
addresses the participatory approach to evaluation (qualitative approach) that will measure the 
impact on producers’ organizations (cooperatives) and private companies and the satisfaction of 
services provided by the program.
4.1. Experimental Design
This sub-section addresses the targeting of the program and the enrolment procedure, the 
selection bias that arises from the program design, the strategy of identification of the project 
impact and complementary econometric models.
Targeting and enrolment procedure
PROGRESA Caribe will work with 4,247 smallholder cacao farmers, cattle ranchers and those that 
can work in both chains. In the design phase of the program, the project selected 13 vulnerable 
municipalities on the Caribbean Coast and in Río San Juan province. These are: Waslala, Siuna, 
Bonanza and Rosita in the North Caribbean Coast Autonomous Region (RACN); El Ayote, La Cruz 
del Río Grande, Muelle de los Bueyes, Bocana de Paiwas, El Rama, Bluefields, and Nueva Guinea in 
the South Caribbean Coast Autonomous Region (RACS); and San Carlos and El Castillo in Río San 
Juan.
The project targets the RACN, RACS and Río San Juan because they are areas with a high incidence 
of extreme poverty5
and, historically, they have received fewer resources for socio-productive 
projects, but they have great productive potential in the selected crops and have adequate 
security conditions. In addition, they are areas of interest for the assistance provided by USDA in 
Nicaragua, where the members of the consortium and the implementing partners have extensive 
experience. The municipalities were selected after consultation processes with local organizations 
and implementing partners, local leaders and other key stakeholders, taking into consideration the 
criteria mentioned above. 
The selection of beneficiaries was based on the following criteria: a) cacao growers, dual purpose 
cattle ranchers, and producers that work in both chains, whose production is fundamental to their 
livelihood strategy; b) producers that have between 5 and 100 cattle in production (and less than 
20 hectares allocated to livestock production) and/or with less than 5 hectares for agriculture; c) 
families that are willing to actively participate in the program; and d) identification of women 
farmers who work in these chains in order to achieve 20% participation by women in the program.
The program also decided to continue strengthening producers who have participated in prior 
interventions by the consortium members in the area and who meet the selection criteria. The 
remaining producers were identified by the members of the consortium and by the implementing 
partners. That said, in practice, the selection of the communities was based on the geographic 
location of the producers that meet the eligibility criteria and who accepted being part of the 
program. 
As a result of the participant selection process, 27.5% of the beneficiaries will be served by CRS 
(13.6% through ADDAC and 13.9% by UCM), 28.8% by LWR (17.65 by IPADE, 7.0% by CACAONICA 
and 4.2% through ACAWAS), and 43.6% by TNS. As for the value chain beneficiaries, 46.05 of the 
beneficiaries belong to the cacao chain, 24.4% to the dual-purpose livestock and 29.6% work in 
both chains. Most of the selected cacao producers’ are concentrated in RACN (mainly in Waslala 
and El Castillo) while the livestock farmers are located in RACS (mainly in Nueva Guinea and 
Bocana de Paiwas).
Selection bias
In practice, the selection of beneficiaries generated two types of selection bias, due to: a) 
administrative rule (based on observable characteristics) and b) self-selection. The first is related 
to the fact that 14% of the targeted beneficiaries come from current and previous interventions of 
the consortium partners (57.7% of those repeating will be served by CRS). These producers may 
differ in characteristics compared to those working for the first time with the consortium partners, 
as they would have better results due to the capacities built in previous interventions similar to 
the current one. However, there is no reason to believe that both groups have differences in their 
motivations to participate in the program. In addition, another slight bias is identified due to 
 
5
INIDE (2005). 
observable characteristics, given that 16% of the beneficiaries selected are women, who were 
encouraged to participate in the program6
.
Another source of bias comes from the selection – under the consortium´s partner criteria – of 
participants that are apparently willing to actively participate in the program (program placement 
bias). These beneficiaries probably have a different behavior from those that were not selected to 
participate in the project, which makes it difficult to compare the results of the beneficiaries with 
those that are not involved in the project. Furthermore, their behavior may also be correlated with 
other non-observable characteristics (e.g. motivation). Finally, a potential self-selection bias might 
appear if some families with motivation to receive a benefit approached more actively local 
partners in order to get selected into the program.
The different sources of bias in the selection of beneficiaries or, in other words, the absence of a 
random selection, makes it difficult to identify a causal relationship between program 
participation and the expected results. This is because the treatment status (being a beneficiary or 
not) is correlated to observable and non-observable characteristics of beneficiaries. In this sense, 
comparing beneficiaries to other populations does not identify the particular impact of the 
intervention as the expected result is not statistically independent of the treatment status. To be 
able to make a proper identification, the treated individuals must be compared with individuals 
that have the same characteristics on average (statistically equivalent); in other words, where the 
only difference that exists between the two groups is that only one group benefits from the 
program.
Identification of impact
The key aspect of all impact evaluations is to identify what would be the outcome of the 
beneficiaries in the absence of the program. Obviously, it is not possible to observe the 
counterfactual outcome because individuals can only have one treatment status (being treated or 
not).
In the case of PROGRESA Caribe, the selection of beneficiaries did not follow an experimental 
design, because it is very complicated to have a “pure” counterfactual outcome in agricultural and 
livestock programs, as only certain producers can be selected to participate for ethical reasons, 
program objectives and budgetary issues (Winters et al., 2010).
Nevertheless, as the activities will not begin with all producers immediately, it is proposed to have 
an intervention strategy that will make it possible to randomly identify a temporary control group 
and also to randomize the duration of time in the program. This strategy is called randomized 
order of phase-in and has been applied to the case of Nicaragua in the Social Protection Network 
Program (see Maluccio and Flores, 2004) and in the Rural Business Development program of the 
Millennium Challenge Account (see Carter, Toledo and Tjernström, 2012).
The evaluation team and the consortium partners agreed the intervention will be implemented in 
two cohorts, making it possible to randomize the beneficiaries in early and late treatment. For the 
late treatment cohort, services will initiate from the 19th month of the execution of activities.
 
6
The PROGRESA Caribe project document determined that 20% of the program beneficiaries would be 
women. 
Nevertheless, one of the main problems in agricultural programs are the spillover effects (or 
contamination), i.e., late treatment could be indirectly exposed to the program (Winter et al., 
2010). For example, if a producer is selected to be early treatment while another from the same 
community corresponds to late treatment, the latter can learn from the producer that is receiving 
technical assistance and/or any other type of services. This would generate a contaminated 
comparison group and could dramatically underestimate the program effects.
Recently, the experimental approach in several agricultural programs has been designed to 
capture spillover effects for eligible and non-eligible units (double randomization). However, 
PROGRESA Caribe was not designed for this purpose, so in order to diminish the contamination 
risk in the project, it was decided to randomize the treatment allocation at the level of 65 groups 
(clusters) of adjacent communities (cluster randomization). These groups were created in 
coordination with the consortium partners based on geographical location criteria. Within each 
group, the communities are adjacent and not among the targeted groups, since the geographical 
distances are very far. At this level the communities within each group have the same probability 
of being selected so the risk of contamination would be marginal.
One relevant issue is that when the experiment was designed TechnoServe had not identified 500 
beneficiaries that work in cacao and livestock production in the South Caribbean region. 
Therefore, the evaluation team and consortium partners agreed that these producers would be 
late treatment and the community groups will be selected randomly based on a producers list 
(that comply with the selection criteria) from Nueva Guinea, Muelle de los Bueyes, El Rama and El 
Ayote.
The consortium partners have stressed that in order to meet the goals of beneficiaries per year 
they have to initiate activities with more than 50% of beneficiaries. For this reason, it was decided 
a (cluster) random selection of 65% for early treatment and 35% for late treatment, which 
generated an early and late assignment of 42 and 23 clusters, respectively. At beneficiary level this 
means 2,676 beneficiaries for early treatment (63%) and 1,571 for late treatment (37%).
Since all clusters have a similar opportunity to be early treatment, we can have a treatment group 
(early treatment) and a temporary conventional control group (late treatment) with similar 
characteristics. In this regard, the first cohort becomes the treatment group and the second one 
corresponds to their comparison group.
The diagram below presents the treatment selection process. The information regarding the 
excluded producers is not available because they did not comply with the selection criteria or 
declined their participation. In the baseline this scheme will be updated with information on the 
effective allocation, and subsequently, in the follow-up surveys the diagram will also include 
information about the producers that have deserted from the program and/or who have been left 
without monitoring.
Graphic 5. Treatment selection process
Targeting and elaboration of 
potential beneficiary list
istpotenciales beneficiarios
Eligibility analysis
Cluster random assignment
Group 1
Offering participation in the 
project 
Group 2
Ineligible Eligible
Early Treatment 
(18 months)
Late Treatment 
(month 19 to the end)
A limitation of this strategy is that it makes it difficult to estimate the effects of the program for 
the final evaluation (and also for the long-term), as it is only possible to have a comparison group 
when there are beneficiaries awaiting to receive the services of the project.
Therefore, a continuous treatment model is proposed for the final evaluation, following the 
strategy of Carter, Toledo and Tjernström (2012). The insight behind this approach is that the 
program’s impact may evolve over time and, therefore, it is possible to identify the way that this 
evolution occurs (linear, quadratic, cubic, etc.) and test the impact depending on the duration of 
time in the program. According to these authors, continuous evolution occurs when the farmers 
experience a learning process that causes their results to gradually improve; this is consistent with 
the intervention logic for PROGRESA Caribe, in which there will be producers (treated early) who 
are exposed to the intervention during a longer period.
This analysis is also justified by the fact that the multiplier effects of the program will not be 
observed instantaneously. For example, the result from investments in infrastructure will not 
necessarily be observed in the first year of implementation. Furthermore, it cannot be rule out the 
possibility that the effects of the project will dissipate for the second cohort (the intervention 
would last approximately two years), especially in the cacao value chain, where the effects of the 
interventions take longer to occur.
A successful experiment requires a clear identification strategy. In this regard, the evaluation team 
has explained the experimental design and its challenges to the consortium partners, 
implementing partners and their technicians, so that they inform all beneficiaries and producer 
organizations about the treatment allocation process.
The cluster random assignment generated the following beneficiaries’ distribution by treatment 
status, partner, value chain, region, sex and repeating producers:
Chart 1. Distribution of beneficiaries by treatment assignment.
F M F M F M F M F M F M F M F M F M F M F M F M F M F M F M F M
Both chains 25 194 15 87 28 139 12 21 127 3 43 84 476 4 1258
Cocoa 32 171 11 81 38 242 75 296 17 127 65 175 3 5 17 94 7 61 36 161 43 124 21 43 10 1955
Livestock 9 99 5 27 108 571 3 16 4 35 3 13 140 1 1034
Total 66 464 31 195 38 242 0 0 75 296 17 127 201 885 6 33 42 256 10 107 36 161 0 0 43 124 21 43 97 626 0 5 4247
Source: Evaluation team calculations based on PROGRESA Carib administrative records.
New Repeaters New Repeaters New Repeaters New Repeaters
TNS
RACNS RACNS Río San Juan RACS
LWR CRS
New Repeaters
LWR
Total
Early treatment Late treatment
CRS
RACNS
New Repeaters
RACNS
New Repeaters New Repeaters
Río San Juan RACS
TNS
18
Issues related to the experimental design
The randomized order of phase-in has the advantages that treatment assignment for each cohort 
is fair and transparent and provides a natural identification of the beneficiaries (Maluccio and 
Flores, 2004). In addition, due to the expectation that it creates in future beneficiaries (who know 
that they are part of the project), this approach allows them to keep in contact with the 
researchers and those responsible for the monitoring and evaluation of the program; this will help 
to reduce potential attrition (Duflo, Glennerster and Kremer, 2007). This latter advantage will be 
very useful in the context of PROGRESA Caribe, because the intervention areas are in remote 
territories, which makes it difficult to monitor the beneficiaries.
A potential disadvantage of this approach is that later treatment may be affected by future 
treatment expectation (Winters et al., 2010). For example, farmers may delay some activities that 
they normally would have undertaken if they were not part of the project (e.g., seeking technical 
assistance and access to external markets). In addition, it is possible that producers for late 
treatment become beneficiaries from other projects, which would seriously affect the comparison 
in the mid-term evaluation.
Moreover, the targeting and selection bias imply that the results cannot be generalized to other 
populations, economic settings or related treatments (external validity). Therefore, we cannot 
suggest that the impact refers to the Average Treatment Effect (ATE). Nonetheless, given the 
nature of the intervention, we can identify the Average Treatment Effect on the Treated (ATET), 
provided that the experiment is successful.
In this sense, the experiment will be conducted under the assumption that there is no evidence of 
targeting errors. In other words, under the targeting criteria defined for the program, the selected 
beneficiaries are similar in characteristics.
Econometric methodology
Difference in Differences
If the experiment is successful and we have a considerable sample evaluation size, then the ATET 
under a standard evaluation of a binary treatment would be:
𝐴 𝐸 = 𝐸[∆ 
| = ] = 𝐸[𝑌1 𝑌0 
| = ] = 𝐸[𝑌1 
| = ] 𝐸[𝑌0 
| = ]
The key is to identify the counterfactual 𝐸[𝑌0 
| = ], i.e., the potential outcome as untreated, 
for the treated ones.
Given that randomization solves the selection problem because it ensures that the treatment 
assignment be statistically independent of potential outcomes, then:
𝐴 𝐸 = 𝐸[𝑌1 
| = ] 𝐸[𝑌0 
| = ]
𝐴 𝐸 = 𝐸[𝑌1 
| = ] 𝐸[𝑌0 
| = 0]
Randomization causes that the potential outcome as untreated for those who were actually not 
treated (𝐸[𝑌0 
| = 0]) is a valid counterfactual.
19
Alternatively, the difference-in-means estimator could be specified through the following 
regression:
𝑌 = 𝛽0 𝛽1 𝜀 
Where i corresponds to the unit of analysis (the producer). Randomization ensures that 
is 
independently distributed from the unobservable characteristics (𝜀 
), so it accomplishes with the 
zero conditional mean assumption (𝐸[𝜀 
| = 0]). 𝛽1 measures the causal effect (or average 
treatment effect):
𝐴 𝐸 = 𝛽1 = 𝐸[𝑌 
| = ] 𝐸[𝑌 
| = 0]
Given that the data will be collected for three periods (baseline and two follow-ups), we can 
exploit the panel data structure by computing the ATET in the mid-term evaluation using the 
difference-in-differences methodology. This method consists of the double difference calculation:
∆̂ = (𝑌̅
 , 𝑓 𝑌̅
 , 𝑓 ) (𝑌̅
 , 𝑓 𝑌̅
 , 𝑓 )
The first difference corresponds to the difference in the after-and-before outcomes for the 
treatment group (early treatment), which controls for time-invariant characteristics. The second 
difference relates to the difference in the after-and-before outcomes for the comparison group 
(late treatment). Subtracting the first difference from the second, it has isolated the effect of time￾varying factors, under the assumption that both treatment and comparison groups were exposed 
to the same environmental conditions (parallel trend assumption); this assumption is plausible in 
the context of PROGRESA Caribe, given the targeting in the Caribbean Coast and Rio San Juan, 
which are locations where the producers tend to face the same environmental conditions.
This method provides a more precise estimate of the impact and also allows the treatment and 
comparison group to differ in characteristics, provided that the parallel assumption trend is 
accomplished and that the unobservable characteristics of the beneficiaries are time-invariant.
In the potential outcome framework, the difference-in-differences parameter can be expressed as 
follows:
𝛿𝐴 𝐸 = 𝐸[𝑌 ,2 𝑌 ,1 | = ] 𝐸[𝑌 ,2 𝑌 ,1 | = 0]
Where 𝑌 ,2 is the outcome of producer at the mid-term survey and 𝑌 ,1 is the potential outcome 
of the producer at the baseline survey. The potential outcome depends on whether the producer 
was assigned to be treated early ( = ), i.e. in the first 18 months of the program 
implementation, or while he was assigned to be treated late ( = 0), that is, from the 19th month.
The double difference can also be specified in the following regression, 
𝑌 = 𝛽0 𝛽1 𝛽2 𝛿 × 𝜇 𝜀 
20
Where 𝑌 represents the outcome of producer in cluster at time . 𝛿 corresponds to the 
impact coefficient, i.e. the mid-term program impact would be given by the interaction between 
the variable that captures the time of data collection ( ) and the binary variable that captures the 
early or late treatment assignment ( ). 𝜇 corresponds to time-invariant observable and 
unobservable characteristics of producer families, for example education and producer skills, sex, 
soil quality, etc. 𝜀 is a time-varying idiosyncratic error. The parameters 𝛽 y 𝛿 are unknown.
For the proposed experimental design, the difference-in-differences technique will allow to isolate
the self-selection bias (previously discussed), as it can be assumed that the motivation to 
participate in the program is a time-invariant unobservable characteristic.
In order to ensure robust results and reduce the residual variance of the estimate, additional 
regressors will be included to test the validity of the random sample.7
 Therefore, the above 
equation can be rewritten as follows:
𝑌 = 𝛽0 𝛽1 𝛽2 𝛿 × 𝛾
′𝑋
′
 𝜇 𝜀 
Where 𝑋
′
corresponds to a vector of time-varying observable producer characteristics that may 
affect the potential outcome.
In addition, the potential heterogeneity in treatment effects will be explored, i.e. individuals with 
different characteristics respond differently to treatment (for instance, those who have previously 
received a similar intervention). In order to capture these effects, the double difference equation 
can be rewritten as follows:
𝑌 = 𝛽0 𝛽1 𝛽2 𝛿 × 𝜓
′
( × × 𝑍
′
𝑘, ) 𝜇 𝜀 
Where 𝑍
′
𝑘, are observable time-invariant characteristics in which there could be heterogeneous 
treatment effect, captured in the parameters 𝜓
′
. These characteristics correspond to the 
beneficiaries’ sex, consortium partners, producers attended in previous interventions, and value 
chain. Finally, the statistical inference will also take into account impacts on multiple outcomes 
and the cluster randomization.
For the final evaluation, the difference-in-differences estimator will be also applied, with the aim 
to evaluate if producers who were treated earlier show higher outcomes in comparison to late 
treatment. If this is not the case, it is likely that the treatment effect had dissipated or lessened, 
even if the early cohort continues to receive benefits until program ends. Therefore, following 
Carter, Toledo and Tjernström (2012), it will be estimated using an equation that takes the 
following form:
𝑌 = 𝛽0 𝛽1 2 𝛽2 3 𝛿1
 2 
 𝛿2
 3 
 [𝛾 𝜙
′𝑋 
]
Where 𝑋 corresponds to a vector of baseline characteristics, 2 and 3 are binary variables that 
represent the medium-term and final surveys, respectively. 𝛾 and 𝜙 control for any difference in 
 
7
If the program effect changes significantly by including additional regressions, then it is probable that the 
randomization has failed in some way.
21
the baseline the early and late treatment groups; these variables are in brackets as they are time￾invariant, so the above equation can be stated as follows:
𝑌 = 𝛽0 𝛽1 2 𝛽2 3 𝛿1
 2 
 𝛿2
 3 
 𝛼 
Where 𝛼 
is the fixed effect. 
Continuous treatment Model
Since the duration of time in the program will be randomized, the final evaluation will also 
estimate the continuous treatment model (fixed effects) proposed by Carter, Toledo and 
Tjernström (2012). As mentioned previously, the insight of this approach is that program impact 
evolves over time, so we can identify the way in which this evolution occurs (linear, quadratic, 
cubic, etc.) and test the impact depending on the duration of time in the program.
We begin by generalizing the binary response function to the case of continuous treatment:
𝑌 [ ] = 𝜆2
 
 2 𝜆3
 
 3 ∆ 𝛼 𝜀 
Where is the number of months that producer has received the services of the project and 
Δ corresponds to a flexible function that captures the sort of non-linearity in the impact. 
Basically, it is intended to measure the duration at each follow-up survey as the number of months 
between when the producer initiated activities and the date of the survey. Following the 
approach proposed in Carter, Toledo and Tjenström (2012), a semi-parametric analysis will be 
drawn to identify the shape of the function Δ . The term of error 𝛼 controls for all observable 
and unobservable time-invariant characteristics of the producer families. The fixed effects 
estimator controls for any correlation between observables characteristics and the duration of 
treatment (Carter, Toledo and Tjernström, 2012).
The above equation is estimated based on the correlated effects model of Mundlak (1978) and 
Chamberlain (1982, 1984), in which the individual fixed effects are a linear projection on the 
observables plus a disturbance:
𝛼 = 𝜓 𝑋 1
′
𝜆 𝑋 2
′
𝜆 𝑋 3
′
𝜆 𝜐 
Since there is no reason to believe that the form in which time-varying observable characteristics 
affect the individual effects or differ between survey rounds, we use the averages of these 
variables, so the fixed effects are denoted as follows:
𝛼 = 𝜓 𝑋̅
 
′
𝜆̅ 𝜐 
Combining the last three equations we obtain:
𝑌 [ ] = 𝜆2
 
 2 𝜆3
 
 3 ∆ 𝜓 𝑋̅
 
′
𝜆̅ [𝜐 𝜀 ]
This equation can be estimated by Ordinary Least Squares (OLS). Finally, this model will be 
extended to analyze the continuous effect of treatment in different quintiles of the observed 
outcomes (e.g., income) in the way proposed in Carter, Toledo and Tjernström (2012).
22
The alternative plan: matching estimators
In practice, it is difficult to ensure that the treatment and comparison groups are totally 
comparable, as it is common in experimental approaches to have some faults in treatment 
assignment. In these cases, Winters et al. (2010) suggest to not only consider which approach 
could be better, but also to analyze different alternatives in order to test the sensitivity of the 
identified impact.
In this sense, even after randomizing treatment assignment, we cannot rule out the possibility that 
the early and late treatment are statistically different, either due to targeting errors or difficulties 
in the implementation of the project; for example, that a share of the beneficiaries to be treated 
late are treated early. In this case, the combination of two techniques may be used to isolate the 
potential biases: matching estimators (which mitigates the bias by observable characteristics) with 
difference-in-differences (which isolates the self-selection bias)8
.
The idea behind matching estimators is to statistically identify (based on observable 
characteristics) two “identical” individuals in the data, with the exception that one is treated and 
the other individual is untreated, so that any difference in the result between the two may be 
attributed to the project.
Matching requires the assumption that the selection in the program is only based on observable 
characteristics, which also affect the expected results (conditional independence assumption). If 
this assumption is fulfilled, conditional on a set of covariates, the allocation of treatment is 
random. Therefore, it is assumed that two individuals with the same characteristics, one treated 
and the other not, have the same counterfactual (Caliendo and Kopeing, 2008). The key in this 
method lies on the careful identification of the “clone” for the treated individual.
Nonetheless, is difficult to identify for each treated individual an untreated one (curse of 
dimensionality); therefore the propensity score technique would be applied. This method is one of 
the most popular in the impact evaluation literature for non-experimental designs. It estimates the 
probability of participation in the program for the sample of treated and controlled individuals, 
using observable characteristics as covariates, which jointly affect the probability of participation 
and the potential outcome. Using the estimated probabilities for each individual, the sample is 
restricted to a region of common support; in other words, to individuals in both groups (treatment 
and control) that have similar probability of being treated. For this task different matching 
algorithms (e.g., Nearest-neighbor, Kernel, Caliper and Radius, among others) will be used. Once 
the matching is successful, we can compute the impact of the program, i.e. the difference in the 
mean outcome (weighted) between the treated and control group.
The success of an eventual application of matching techniques would require an extensive 
knowledge of the selection process of beneficiaries. For this, the evaluation team had different 
meetings with CRS' monitoring and evaluation team in order to document program’s eligibility 
criteria and selection mechanisms.
Complementary econometric models: factors associated with productivity
 
8
The insight behind the application of this method has been explained in the previous section. 
23
The impact evaluation identifies what is the effect of the program as a whole on the outcome of 
the beneficiaries. However, it does not allow to derive conclusions regarding the impact in 
outcome due to changes in some of the program components or the specific impact of a particular 
activity (Mallucio and Flores, 2004). This makes it difficult to identify which activities should be 
addressed as the program execution progresses.
One way to identify the need to adjust or adapt some activities is through the econometric 
analysis of the factors that explain the performance in some intermediate results. For PROGRESA 
Caribe, the evaluation team will analyze the factors explaining productivity in milk, livestock and 
cocoa.
The choice to carry out a productivity analysis is due to different reasons. Firstly, the increase in 
productivity is one of the strategic objectives of the project. Secondly, the increase in productivity 
represents one of the major challenges for Latin American development, since there are significant 
gaps between different sectors that reflect the structural heterogeneity and the lack of systemic 
competitiveness (CEPAL, 2012). In general, producers are facing different problems affecting their 
productivity such as production technology, the scale of operations, the investment climate and 
improvement in efficiencies (Fried et al., 2008). Therefore, the idea is to identify the determinants 
of the production and suggest areas of improvement which constitute a guide to reinforce the 
intervention. Thirdly, it will contribute to the empirical evidence on productivity issues for the case 
of Nicaragua, as there are not many studies in this field due to lack of information about 
productive characteristics.
It is important to discuss some concepts associated with the methodology adopted for this task. 
The term production function is used as the relation between the total production and the inputs 
such as land, labor, machinery and equipment, and others. In a simplified manner, considering the 
production and only one input (e.g. labor), graphic 6 shows a production function that represents 
the optimal level of output per worker.
The points below the production function are considered technically inefficient from two 
perspectives: technical inefficiency from inputs (if other farmers have the same production but 
with fewer inputs) or technical inefficiency from production (if other farmers produce more from 
the same amount of inputs). It is important to mention that the producer´s size does not 
necessarily influence the level of efficiency (Kuhmbhakar and Know, 2000).
9
 
9
The production frontier does not take into account important elements such as input prices that are being 
used and producer behavior of cost minimization. This analysis cannot be carried out in the present 
evaluation, as there is no data available at this time on production costs for each producer. 
24
Graphic 6. Production Function, one product and one input 
Based on the above, we will discuss the factors that explain the production in the value chains that 
the project will work. Then, we will estimate the technical efficiency and the elements that affect 
it.
The type of production functions to analyze are Cobb-Douglas and Trans-Logarithmic, and these 
functions are estimated using parametric Stochastic Frontier (SF) models, which have its origins on 
the contributions of Meeusen and van den Broeck (1977) and Aigner, Lovell and Schmidt (1977). 
These models are inspired by the theoretical idea that no economic agent may exceed the “ideal 
frontier” of production and the deviations from the frontier represents individual inefficiencies. 
(Belotti et al., 2012). A technically efficient producer is on the frontier of production, while an 
inefficient one is found below the frontier. The Stochastic Frontier models allow us to identify: a) 
the factors influencing the production, b) average efficiency levels, and c) factors affecting the 
distribution of technical inefficiency (risk inefficiency).
To estimate the SF model, cross-section data will be used in the baseline analysis, whereas for the 
mid-term and final evaluation the panel data structure will be used. In the baseline study, the 
models will not differentiate between early and late treatment, while at the mid-term and final, 
the efficiency will be analyzed according to treatment status.
4.2. Participatory approach to evaluation
The consortium partners will work on the implementation of different participatory 
methodologies with benefitting producers, cooperative members and staff, territorial leaders and 
private sector representatives.
Sometimes, it is assumed that participatory evaluation refers exclusively to obtaining qualitative 
information on the views of participants using methods such as maps or stories; however, this is 
only an option. This type of evaluation also allows the measure of the impact of different 
Producción
Trabajadores
A
B
Production
Workers
25
indicators. Catley, Burns, Abebe and Suji (2007) explain that the use of participatory methods to 
assign scores and establish a hierarchy order allows linking of numerical information to the 
qualitative indicators (based on opinions or perceptions). Assigning a score provides a point of 
reference through which it is possible to measure the program impact over time.
The World Bank indicates that not all qualitative methods are participatory, although many 
participatory techniques can be quantified. The indicators obtained using participatory methods 
are not the primary purpose of this type of exercise, but the process of reflection and analysis 
arising from the application of the instrument.
Guijt (2014) mentions that participatory approaches can be applied in any type of impact 
evaluation design, as they are not unique to a specific method of evaluation. Thus, the following 
discussion explains the participatory methodologies that will measure the impact of PROGRESA 
Caribe in the performance of cooperatives and private companies, and of the satisfaction of the 
services provided by the program.
Tool: Self-assessment for the management of rural associative enterprises 
For evaluating the impact of the project on producer organizations the “Self-assessment tool for 
the management of rural associative enterprises” (facilitated self-assessment) will be used, which 
was developed by the Learning Alliance in Nicaragua. This tool will allow producer organizations to 
carry out a quick analysis of their business management and organizational process.
Gottret, Junkin and Ugarte (2012) state that this methodology guides a participatory process of 
self-evaluation that includes the governance structure, associates, management, and the 
administrative and technical teams.
The tool has six areas of assessment: 1) strategic orientation; 2) business management; 3) 
technical services; 4) financial services; 5) structure and functionality; and 6) governance in 
partnership processes. It includes 126 criteria outlined in 34 indicators, which are detailed in 
graphic 7. For each criterion a score of 1 to 5 is assigned, where 5 is the optimum state of the 
organization. In addition, the methodology provides a set of quantitative indicators, which can be 
triangulated with the perception indicators in order to validate the results.
26
Graphic 7. Qualitative indicators of the Facilitated Self-Assessment Tool
The methodology indicates that initially we have to form groups or have separate sessions with 
the board, management, administrative and technical staff of the cooperative and with the 
associates. Once this stage is finished, we proceed to carry out a meeting where the different 
groups share their results and decide by consensus the final scores for each area of assessment. 
These scores are incorporated into an Excel file, which calculates the average value obtained. In 
this sense, the consortium partners will act as facilitators of this process but not as evaluators or 
verifiers.
The results obtained through this methodology will allow producer organizations to classify 
themselves according to their level of development in terms of business and key organizational 
capacities, identify needs, current and future opportunities for improvement, and design an action 
plan to attain specific goals. Also, it will allow the comparison of results over time for the same 
organization and compared to other organizations.
The participation of management, administrative and technical teams, and a representative 
sample of associates is very important not only for the validity of the self-assessment, but also for 
evaluating the management and organizational process periodically at the different levels of 
decision making. This exercise will also provide a reflection about the institutional strengthening 
needs, and identify opportunities and actions to improve the capacity of the organization.
Additionally, this tool will facilitate planning sessions to establish clear strategies to achieve the 
objectives that were developed through the self-assessment and to establish follow-up 
mechanisms to guarantee the progress towards the fulfillment of those goals.
1) Strategic 
Orientation
Market Skills
Strategic 
Management 
Skills
Strategic 
Planning
Business Plan
Business 
Alliances
Access and Use 
of Information
2) Business 
Management
Economic 
Analysis
Economic and 
Financial 
Management
Administrative 
Management
Commercial 
Management
Human 
Resource 
Management 
and Gender
3) Technical 
Services
Access and 
Coverage
Partnerships for 
Innovatin in 
Service Delivery
Investment for 
the Provision of 
Services
Management for 
the provision of 
Services
Satisfaction 
with Technical 
Services
Skills 
development for 
Sustainable 
Production
Provisions of 
Operational 
Services
4) Financial 
Services
Financial 
Skills for 
Saving
Planning for the 
Provision of 
Financial Services
Partnerships for 
Financial Access
Management of 
Financial Services
Access and 
Coverage
Satisfaction with 
Service
5) Structure and 
Functionality
Legal Status
Organizatiional 
Chart and 
Functions
Rules and 
Regulations
Transparency 
and 
Accountability
Communication
Influence on 
Policies and 
Practices
6) Governance in 
Partnership 
Processes
Organizational 
Skills
Membership 
and 
Commitment
Organizational 
Practices
Resolution of 
Conflicts
27
This tool will be applied in the baseline, mid-term and final evaluation to 25 cooperatives located
in the municipalities where the program will intervene. Data on expenditures, income and 
organizational and business processes of the organization will also be recorded on a regular basis 
as part of the project´s monitoring system. These data will support the mid-term and final 
evaluation.
Link Tool
The stakeholders in the value chain are linked by trade relations in which each one acts as buyer 
and seller at the different stages. For example, producers are sellers and the cooperatives are 
buyers in the first link, but in the following the cooperatives are sellers and the companies are the 
buyers.
For the purpose of PROGRESA Caribe, the Link methodology will be used to carry out the analysis 
to capture the influence of the project on the participation and business practices of the private 
sector. 
The Link method involves the use of four tools: 1) mapping of the value chain, which helps identify 
the key stakeholders, process and services within a value chain and how each one interacts with 
the others, 2) the template of the business model, which allows organizations and/or businesses 
to identify their current status as an entity and to identify areas for improvement and/or 
intervention; 3) the principles for inclusive business models, in which it is determined the level of 
business inclusion between producers and buyers; and 4) the prototype cycle, in which the 
previous tools are unified to derive strategies for expansion or to implement new innovations to 
promote the participation of small farmers in the chain. These tools facilitate a better 
understanding of the key players, processes and relations within a value chain.
This exercise will take place in the baseline, mid-term and final evaluation. For the baseline study 
the tool #3 will be applied to the companies that have worked with the cooperatives targeted by 
the project. It is expected to get different perceptions and opinions as well as a better 
understanding of the context of the value chain. This information will provide inputs to partners 
for decision making in order to improve the inclusion of small producers to the value chain.
Evaluation to implementing partners
The efficient and high-quality technical, business and financial services provided to producers and 
organizations will be a key factor for the program´s success. Therefore, it is proposed to evaluate, 
at mid and final term, the degree of satisfaction of the services provided by the consortium and 
implementing partners. This will ensure a better accountability to beneficiaries. For this purpose, 
two tools are proposed, whose content must be agreed with the consortium partners in the mid￾term evaluation.
The first one refers to a participant satisfaction survey. In this regard, the evaluation team 
proposes the application of a survey that allows the measuring of perceptions and the level of 
satisfaction of the beneficiaries regarding the services provided by the partners. The survey will be 
applied to all the cooperatives and a sub-sample of beneficiaries. For the sub-sample of 
beneficiaries the questions will be incorporated in the household survey. This tool will be applied 
28
by external evaluators, in the mid-term and final evaluation. The mid-term evaluation will serve as 
a management tool for the consortium partners, because it will provide an opportunity to make 
changes and improve services.
The second tool corresponds to a Participatory Mid-Term Evaluation. Botelo et al. (2005) explain 
that the objectives of this tool are: a) measure the progress of the project from the beneficiaries’ 
perspective in terms of progress in the execution of activities and satisfaction for the services
received; and b) generate information that serves to adjust the project from the perspective of the 
beneficiaries. This information will allow consortium partners to adjust the projects´ activities, 
towards the achievements of the planned objectives.
This methodology is highly participatory, since it allows beneficiaries to take decisions about the 
future of the project. Service providers are informed of the valuation made by the beneficiaries on 
project implementation, their satisfaction with the services, and their recommendations for the 
actions that are necessary to achieve the project´s objectives. The application of this methodology 
requires information related to the objectives, results and schedule for each activity of the project.
A representative sample of all strata of beneficiaries should be selected to attend a participatory 
assessment workshop. In the workshop, the beneficiaries can be grouped according to the 
different areas of intervention. They rate the results carried out by the project. This gives an idea 
of the importance that recipients give to each outcome. The socialization of activities allows the 
respective parties to know if the outcomes have been met or not, and measure the degree of 
satisfaction of the beneficiaries.
4.3. Units that Participate and Do Not Participate in the Program
The evaluation team and the CRS Monitoring, Evaluation, Accountability and Learning (MEAL) 
team jointly developed the criteria to ensure assignment by treatment status in accordance with 
the experimental design. To a great extent, the assignment of program beneficiaries occurred in a 
way that was compatible with the planned design. However, it is important to emphasize some 
details, which will be discussed below. 
The following table shows that not all the producers registered in the program will participate in it. 
According to the PROGRESA-Caribbean records, the true assignment at the time of this report is 
97.3% of the beneficiaries selected; thus the percentage of desertion is fairly low. 
Chart 2. Distribution of PROGRESA Caribe beneficiaries
Early departure occurs for different reasons, depending on the intervention area. CRS has reported 
that, in Waslala and the Mining Triangle (Bonanza, Rosita and Siuna), desertion occurs because 
there are producers that decided to participate in a program serving cacao growers that is 
implemented by the Ministry of the Family, Community, Cooperative and Associative Economy 
Early 
treatment
Late 
treatment
Total
Early 
treatment
Late 
treatment
Total
Early 
treatment
Late 
treatment
Total
CRS 756 415 1171 29 39 68 727 376 1103
LWR 795 428 1223 0 0 0 795 428 1223
TNS 1125 728 1853 18 27 45 1107 701 1808
Total 2676 1571 4247 47 66 113 2629 1505 4134
Source: Evaluation team calculations based on Baseline Household Survey - PROGRESA Caribe.
Partner
Enrolled Atrittion Efective allocation
29
(MEFCCA). The characteristics of this program are similar to those of PROGRESA-Caribbean, 
although its potential effectiveness is unknown. TNS reports that early departure in the southern 
Caribbean region is because some producers have little confidence in any governmental or non￾governmental institution, given the uncertainty in that area due to the potential construction of 
the interoceanic canal. On the other hand, although LWR does not report deserters in the Río San 
Juan, it is known that MEFCCA will soon begin to implement a similar program in this area; 
therefore, ongoing monitoring is important to prevent a larger number of dropouts that may 
affect the internal validity of the impact evaluation. To a certain extent, these limitations reveal 
the difficulty of good targeting in the selected intervention areas.
Regarding the 500 beneficiaries that TNS has yet to identify for late treatment, it selected only 170 
for the evaluation sample in the municipalities of Nueva Guinea, Muelle de los Bueyes, El Rama
and El Ayote. TNS has made a commitment to quickly identify the rest of the producers. Given that 
there are communities that correspond to the early treatment in the first three municipalities, 
they sought new beneficiaries from communities for late treatment or from new communities.
5. Sampling and Data
5.1. Sampling Strategy
The information for the evaluation will come from panel data that will compile the characteristics 
of producers and their families for three evaluation rounds (baseline, mid-term and final). The idea 
is to compare the outcomes of interest before and after the intervention. The sampling strategy 
has been designed so as to allow the collection of the necessary data to answer key evaluation 
questions and to analyze the effects of the intervention on intermediate and final outcomes.
For this it is necessary to have a representative sample of beneficiaries from the early and late 
treatment. To calculate a preliminary sample size the identification of the following information is 
required: a) expected effect on outcome variable; b) outcome indicator standard deviation; c) 
confidence level; and d) statistical power. 
In regard to the expected effect on outcome variables, there is no official information on key 
variables such as the poverty indicator and family income in the areas of program intervention. 
Therefore, it was decided to rely on second-hand information in order to obtain the reference 
values for the intermediate indicators of milk and cacao productivity. Given that there is a strong 
correlation between productivity and family income and therefore, with the incidence of poverty, 
the values of these indicators are used to obtain a preliminary estimate of the sample size.
The cacao yields come from a combined sample of beneficiaries in two projects, one carried out in 
the municipality of Siuna by CRS and the other carried out in the municipality of Waslala, El 
Castillo, San Carlos and Rancho Grande by LWR. Milk yields data corresponds to information from
the 2011 National Agricultural Census (CENAGRO) for the program´s intervention zones. Based on 
this information and the goals discussed with the partners, for cacao it was estimated a minimum 
level of impact of 0.22 metric tons per hectare, while for livestock it was estimated a minimum 
level of impact of 0.45 liters of milk per day. According to discussions held with the CRS monitoring 
and evaluation team, it is very likely that the impacts to be identified will be greater, so this 
conservative estimate would give a greater precision to the evaluation sample.
30
In reference to the standard deviation of the outcome of interest, it is assumed that the variance 
between early and late treatment is the same. On the other hand, as it is usually made in impact 
evaluation designs, it uses a confidence level of 95% and a statistical power of 80%. A power of 
80% means that we will find an impact in 80% of the cases where one has occurred (Gertler et al., 
2011). This information will allow us to compute a preliminary sample size and power using the 
noncentral t-distribution.
However, the experimental design was based on a cluster randomization with the aim to reduce 
the contamination risk. The intra-cluster correlation is uncertain since it cannot be calculated 
because of lack of information. As an approximation, the intra-cluster correlation was estimated 
using the cacao and milk data discussed above, under the assumption that each community 
corresponds to one cluster. This estimate yields an intra-cluster correlation of 0.127 and 0.104 for 
cacao and milk, respectively. 
It is noticeable that these associations are low and most likely even lower in the clusters created 
for the experiment, as the number of beneficiaries in each cluster is unequal and because within 
these clusters the geographical distances are very far. Therefore, it is expected that there will not 
be a strong intra-cluster correlation.
As potential intra-cluster correlation increases the sample requirements, we proceeded to correct 
the sample by the cluster effect following a conservative strategy that implied taking 100 to 150 
average observations by cluster and use an intra-cluster correlation of 0.15 and 0.12 for cacao and 
milk, respectively.
Assuming a balanced random sample (50/50) to maintain efficiency and statistical power, we use 
the sample size estimated by the indicator that required the largest sample size. The results 
suggest a representative sample of 980 observations (490 for each treatment status). Assuming a 
rate of non-response (and attrition) of slightly more than 20% the total sample would be of 1,210 
producers (605 for each treatment status). This strategy is conservative in the sense that it 
generates a greater sample to ensure greater statistical accuracy.
As there will be three survey rounds (baseline and two follow-up surveys), we will have in practice 
a much greater statistical power given the estimated sample size, as there will be covariates on
the baseline associated with the outcome results. In fact, the minimum sample size required, that 
arises from adjusting the calculations considering three survey rounds, is 50% less than previous 
estimation.
The sample design will have as a framework the list of project participants. The selection of the 
sample will take place at one stage and using a simple random sampling. It was discarded to carry 
out a two-step cluster sampling with a proportionate stratification, as given the rules of selection 
into the program, this method would not ensure a representative sample in sub-populations to 
measure heterogeneous treatment effects. In addition, this would have caused a very uneven 
distribution in the number of surveys to apply by partner. The consortium partners expressed their 
interest in a similar distribution of the sample to ensure their presence in the different 
municipalities, and thus to be able to have a first contact with a greater number of producers.
It could be argued that this sample design may involve higher costs. However, as we have a small 
number of strata and due to the fact that geographical distances in the areas of intervention are 
31
rather great, this suggests that there would not be much difference with respect to a stratified 
sampling. Besides, the evaluation team did an exercise of a stratified sampling controlling the size 
of the sample in each stratum - in order to guarantee precision in some sub-population groups -
and found that the distribution of the sample was not very different from the one obtained by 
simple random sampling.
On the other hand, the cluster sampling was not used because in practice the intra-group 
correlation is much smaller than the estimated with the available information, the size of the 
clusters are not equal among themselves and this method would not ensure enough statistical 
precision in some sub-populations, given the nature of the program.
Regarding the latter, the evaluation team made an estimate of the minimum required sample 
adjusting for the three survey rounds and taking into account a lower (and more realistic) intra￾group correlation (5%) and assuming an average of 50 observations per cluster. This calculation 
yielded a minimum sample of 132 observations for each treatment group. When comparing this 
estimation with the results of the simple random sampling we get a high precision for most of sub￾population domains for which the heterogeneous treatment effect will be evaluated.
Since agricultural programs tend to have a certain level of attrition even before the beginning of 
the activities, a random sample of a 10% of replacements has been estimated, which will be 
distributed proportionally to each implementing partner. Nevertheless, the evaluation team has 
recommended caution in the use of replacements and therefore has stated specific criteria with 
the aim that replacements are used only in very special circumstances.
On the other hand, as mentioned previously, when the experiment was designed TechnoServe had 
not identified 500 beneficiaries. To ensure a random selection of the evaluation sample in the 
outstanding areas, it was agreed that a random sampling will be done from the lists of producers 
(that comply with the selection criteria) identified in the municipalities of Nueva Guinea, Muelle 
de los Bueyes, El Rama y El Ayote.
5.2. Data Collection
This subsection documents the data collection process for the baseline study for the household 
survey, their assessment and the difficulties identified, the nonresponse and its implications for 
the impact evaluation.
5.2.1. The Process
Data collection in the different phases of the evaluation was a shared responsibility among the 
technical advisors from the consortium, the evaluation team, the area supervisors and the
surveyors10
. The process was supervised by the CRS MEAL team.
CRS developed the instruments to be used to collect information from the producer families. The 
evaluation team and the CRS monitoring and evaluation unit jointly reviewed this instrument to 
ensure that the questions include all the elements necessary to perform the evaluation, both 
 
10 The surveyors are the same people who will work as field trainers in the framework of the program. It was 
designed in this way in order to introduce them to the program and to familiarize them with the context in 
which the activities will be implemented. 
32
those related to the key indicators (intermediate indicators and result indicators), as well as the 
impact mechanisms. 
The preparation for the field work included training workshops, the preparation of a users’ guide, 
pilot tests, and the creation of route sheets to collect the information. Two general training 
workshops were conducted for the entire consortium and three bilateral training events were held 
with each partner, along with specific advising sessions. The main objective of these workshops 
was to train the technicians, advisors and promoters in the use of the digital survey system. The 
evaluation team explained the evaluation design and the sampling strategy in the second general 
workshop. It also clarified doubts and concerns about the relevance of the questions to be asked 
in the household survey. As a complement to the training process, the CRS monitoring and 
evaluation team prepared a users’ guide that addresses the use of the system, the input of 
information into the iPads, the criteria for identification of the agricultural and livestock 
households, and the use of replacement in the survey process. Parallel to the training process, a 
pilot test of the digital survey was conducted in the municipality of Nueva Guinea and, later, each 
of the partners conducted pilot tests in their intervention areas.
In addition, the CRS team prepared the protocol for systematic review for data cleansing. This 
protocol presents the flow chart for quality control, which is undertaken at five levels: extensionist 
technicians (first level of review), project coordinators (second level), partners’ monitoring and 
evaluation technicians (third level), CRS MEAL team (fourth level), and the PROGRESA MEAL 
management (fifth level). This system endeavors to reduce measurement errors and ensure 
quality in information management. 
The following databases were obtained from the household survey:
1. Progress out of Poverty Index (PPI) Information
2. Food security
3. Agriculture: production and sales of cacao
4. Cattle production
5. Cattle inventory
6. Vaccination
7. Cattle sales
8. Infrastructure
9. Credit
10. Employment
11. Socio-demographic information
5.2.2. Assessment of the Process and Difficulties Identified11
The evaluation team and the CRS MEAL team agree that, during the process of field data 
collection, the same procedures were applied for both the early treatment and late treatment 
groups. The digital survey system makes the standardized application of the survey possible. This, 
together with the protocol for systematic review for data cleansing (prepared by CRS), minimized 
measurement errors.
Nevertheless, although the information available is of very good quality, there were many 
constraints on the collection of information. In part, this is reflected in the high percentage of use 
 
11 This section has been prepared by the evaluation team and the CRS MEAL team. 
33
of the programmed replacements (65%). Even when the surveyors are originally from the 
intervention areas, it is clear that there were difficulties in applying the instrument with the 
producers, given the distances between communities, the poor access to the farms (which often 
prevented them from arriving on time) and problems in the communications networks. This 
implies a major challenge for the follow-up surveys as outside surveyors will be required. 
Therefore, the evaluation team suggests that, in the follow-up rounds, the program should 
develop a strategy in conjunction with the consortium partners to support the surveyors in 
identifying the key informants, traveling in the intervention areas (given the logistical difficulties), 
and determining the survey route. In addition, the hiring of a firm of external surveyors is 
recommended; it should be a firm with experience in data collection on the Caribbean Coast and 
in the application of surveys similar to the Living Standards Measurement Survey (LSMS) and the 
National Agricultural and Livestock Census (CENAGRO). It should be noted that most of the survey 
firms in the country have little or no experience in the use of the digital survey system; therefore, 
the follow-up rounds must be scheduled enough in advance to conduct an exhaustive training 
process on its use. 
Another significant difficulty was in the use of the forms. The survey had nine sections and various 
sub-sections to collect information. Although this ensured the quality of the information by 
reducing measurement errors (given the presence of filters), in many of the databases, it seems 
that the filters were not applied correctly by the surveyor and, in some cases, the instrument was 
applied very quickly and the surveyors forgot to complete some of its sections. 
Furthermore, there were certain difficulties in the understanding of several sections of the form, 
especially in the socio-demographic characteristics, infrastructure and cattle inventory. On several 
occasions, this resulted in the CRS MEAL team instructing the supervisors to have the surveyors 
return to the farms to collect the missing information; they were not always able to do this, due to 
difficulties in access and communications. The above clearly illustrates the heterogeneous nature 
of the application of the survey by partner, sub-partner and even among surveyors. It shows that 
the training process did not impact all the surveyors equally. For example, the training explained 
that the survey teams must charge the device to 100% power to ensure the collection of survey 
data, but this did not always occur; many forgot to charge it or they were located in communities 
where there was no electricity. 
On the other hand, the quality of Internet service in the project impact areas is deficient, which 
caused some problems in the synchronization of the survey (outdated sections were downloaded) 
and of the data (the surveys were incompletely uploaded to the system). 
The following additional constraints were observed during the process:
• The consortium partners did not respect the weekly synchronization dates, which resulted in 
the accumulation of information, leaving less time for quality control. 
• The considerable distances between one producer and another meant that no more than three 
surveys could be conducted per day.
• There was no general script for the surveyors to introduce themselves. Often the producer did 
not know about the program and not all the surveyors made a proper introduction. 
34
• The questions were not always adapted to the context of the survey (for example, using less 
complex words) and extensive use was made of the survey headings. 
• The surveying was conducted in the morning and, generally, producers are working at that 
time; therefore, the surveyor had to make return visits to the farms. 
5.2.3. Data Collected
Overall, data was collected in 82 (of 86) clusters. The coverage of surveys in each cluster varied, 
with an average response rate per cluster of 80%.
Of the random sample of 1,210 families, 1,040 were already registered in the program and 170 
were yet to be identified by TNS. In addition, a sample was prepared of 165 substitutes that were 
randomly selected by partner and implementing partner. Regarding the latter, initially a substitute 
sample of 10% was selected; however, at the request of the consortium partners, this was 
expanded to 14%, given the desertion problems mentioned above and because some of the 
beneficiaries were not home at the time of the survey or were temporarily off the farm12
.
Taking the section on the Progress out of Poverty Index (PPI)13 as the basis for determining the 
final sample, information was collected for 922 (88.7%) of the families registered, data was 
collected for 172 families pending identification, and 104 substitutes (63%) were used. The final 
sample is 1,198 families and 12 of the families were not interviewed, due to the difficulties 
mentioned above. 
To estimate the number of complete interviews, it was determined that each survey should have 
information in the sections on the PPI, cacao production and sales, cattle production, cattle 
inventory, and cattle sales14
. These sections were selected because they contain the data 
necessary for calculating the major intermediate and result indicators. When this information is 
cross-checked, there is a total of 945 complete interviews, of which 534 correspond to early 
treatment (56%) and 411 to late treatment (44%).
This imbalance is also identified when separating by chain and treatment status, with larger 
numbers for the producers that work in both chains and those that only work in cattle. This is, in 
part, because when the survey was performed, many producers who were registered in both 
chains ended up working in only one of them, especially in the case of cacao. The above suggests 
that there were limitations in the targeting process for the program. This meant a reduction in the 
amount of information expected for cacao and cattle in 427 and 111 observations, respectively. 
 
12 This situation is very common in the intervention areas, given the distances between producers and 
between communities and the difficulties in communication. However, this does not imply that all the 
producers who were not interviewed and who had substitutes in the sample will not be part of the program. 
13 This criterion is established because the PPI is the most important long-term indicator for the program 
and because the PPI section had the largest number of families compared to the other 10 sections. If 
information was collected for some of the sections on production and not for the PPI, the interview was not 
recognized. 
14 When the producer only works in one value chain, the information required is on the PPI and the 
production and sales for his/her respective chain. 
35
Completed interviews represent 78% of the planned sample, distributed by partner as follows:
Chart 3. Evaluation survey response
The loss of 253 observations is because there were surveys that had information on the PPI but 
not on cacao or cattle and vice versa, or even some that had no information in the other sections. 
For example, there are 70 producers for whom information was collected about cattle production 
and sales, but for whom there was no cattle inventory as it was not possible to collect the 
disaggregated composition of the herd. Another example is the case of 13 surveys that had 
information on cattle practices but none on cattle sales and inventory. Moreover, for some of 
these 945 observations, there is incomplete information for other input indicators (for example, 
the indicators on credit and economic infrastructure).
Nonresponse is calculated as the total of those that were not interviewed plus incomplete 
interviews. The nonresponse rate is the percentage of nonresponse relative to the planned 
sample, which is 22% (265 observations) for the PROGRESA-Caribbean baseline study. This is a 
high percentage given that this figure practically represents the proportion for which the sample 
was adjusted to reduce the effects of potential attrition due to nonresponse and desertion from 
the program and in the face of uncertainty due to the limited information on the indicators to 
calculate the statistical power and correlation within groups. In this sense, in order to not lose 
more statistical power in the mid-term and final evaluations, the evaluation team and the 
consortium partners should develop new field procedures to ensure a compliance rate of more 
than 95% in subsequent follow-up surveys. One advantage is that the randomized phase-in 
approach makes it possible for the researchers and those responsible for program monitoring and 
evaluation to have greater contact with the beneficiaries, which helps reduce potential departure 
from the program. 
Early 
treatment
Late 
treatment
Total
Early 
treatment
Late 
treatment
Total
CRS 170 172 342 166 128 294
49.7% 50.3% 100.0% 56.5% 43.5% 100.0%
LWR 186 185 371 135 152 287
50.1% 49.9% 100.0% 47.0% 53.0% 100.0%
TNS 251 246 497 233 132 364
50.5% 49.5% 100.0% 64.0% 36.0% 100.0%
Total 607 603 1,210 534 411 945
50.5% 49.5% 100.0% 56.5% 43.5% 100.0%
Source: Evaluation team calculations based on Baseline Household Survey - PROGRESA Caribe.
Partner
Target sample Completed interview
36
Chart 4. Evaluation survey: final sample and nonresponse
Although the non-response rate at the program level is higher for the late treatment group, the 
evaluation team argues that this is not a systematic problem that is correlated with being in the 
early or late treatment group. This loss was, practically, exogenous to the treatment status and, in 
fact, is related to operational issues in the field data collection, such as the lack of standardization 
in the capacity to use the forms by the consortium partners. This is confirmed in the analysis of 
non-responses by implementing partner, where LWR lost more of the sample in relative and 
absolute terms in the early treatment and CRS and TNS lost more of the sample in the late 
treatment. For the latter partner, the difficulties were mainly due to the random search for 170 
producers who are yet to be identified. Later discussion in this report will show that this did not 
affect the internal validity of the experiment, although it did reduce the statistical power in the 
impacts to be identified. It should be mentioned that, during field supervision, the evaluation team 
confirmed that the producers surveyed were informed that the instrument was applied to better 
understand the characteristics of the farmers in order to supply program services in accordance 
with their needs and without mentioning that they would be subjects of evaluation. 
Baseline
(1) Target sample 1210
(2) Initial target sample 1040
(3) Efective sample 922
(4) Replacements 165
(5) Use of replacements 104
(6) Sample identified by TNS 172
(7) Not interviewed 12
(8) = (3) + (5) + (6) Final sample 1198
(9) Incomplete interviews 253
(10) = (7) + (9) Nonresponse 265
Nonresponse rate 22%
(11) = (8) - (9) Completed interviews 945
(10) Early treatment 534
Percent intervention 57%
(11) Late treatment 411
Percent intervention 43%
Source: Evaluation team calculations based on Baseline Household Survey - PROGRESA Caribe.
37
5.3. Constraints on Power and Inference by Cluster
Constraints on Power
The effective power of the evaluation sample varied from the plan for different reasons: 1) the 
outcomes (intermediate) of the productive yields of the beneficiaries differed from the estimates 
using secondary sources of information, such as data from the 2011-2012 CENAGRO, and from 
cacao projects previously implemented in similar areas by CRS and LWR; 2) the secondary 
information provided slightly downward estimates of intra-cluster correlation; 3) the reduction of 
anticipated information for cacao and cattle in 427 and 111 observations respectively, due to the 
re-composition of the value chains relative to the planned sample; 4) the 22% reduction of the 
planned sample and its slight imbalance (56/44) due to nonresponse. It should be noted that these 
types of difficulties are present in all impact evaluations. 
Considering that these problems would be latent, the calculation of the planned sample continued 
to be a conservative strategy in the sense that it generated an intentionally larger sample and
considering that there was only one round of data available to conduct the evaluation. In practice, 
the information for the evaluation will come from a panel data that will compile the characteristics 
of the producers and their families for three points in time (the baseline, mid-term and end of the 
program); therefore, this will compensate, to a certain extent, for the losses in power due to the 
constraints mentioned above. In addition, to gain more statistical power, the impact estimates can 
be controlled by the covariables from the baseline study that are associated with the outcome 
indicators. 
The evaluation team has conducted simulations based on the evaluation sample in order to 
explore whether it is possible to identify the expected effects of the intervention with one round 
of data, using the targets in the PROGRESA-Caribbean indicators from the contract with the United 
States Department of Agriculture (USDA) as a reference. According to estimates, it is possible to 
identify the impacts on poverty, food security and cacao yields without difficulty. It is more 
difficult to identify the minimal impacts for milk yields and sales volumes. It was not possible to 
conduct these simulations for other intermediate indicators (e.g., gross margins, value of 
production per hectare) because the baseline study sheds light on indicators that differ from those 
in the program plan15
. 
The above reiterates the importance for the evaluation scheme to be based on a panel data. 
McKenzie (2012) explains that having one baseline study and two follow-up surveys provides 
greater statistical power than having two rounds of data, even when the sample is smaller. On the 
other hand, following this same author, the evaluation team recommends that the mid-term 
evaluation analyze the autocorrelation of the data in the different measurement rounds 
(particularly for the data on sales and income), as well as the viability of applying an analysis of 
covariance (ANCOVA) to replace the difference-in-differences method. 
In addition, the evaluation team has agreed with the CRS monitoring and evaluation team to add 
the data from the two rounds of monitoring campaigns in the first 18 months of implementation 
to the mid-term evaluation. These campaigns collect the same information as the baseline survey 
and will provide greater statistical power to the evaluation of some indicators.
 
15 The end of the baseline report discusses the need to modify some program targets. 
38
Finally, the evaluation sample has clusters with very unequal sizes and when the analysis is 
performed by partner, the number of clusters is low. This is because the consortium partners did 
not follow an equal size cluster approach in the recruitment of beneficiaries. Despite this, to 
prevent contamination of the experimental design, the random assignment to early and late 
treatment continued to be a cluster approach. The evaluation sample did not follow a cluster 
approach because this would have generated serious discrepancies in the sample quantity by 
partner and value chain, which would affect the impact measurement in these strata. 
Nevertheless, the inference to be made with these data should be adjusted for the intra-cluster 
correlation, which is much greater than 10% in different indicators.
Inference by Cluster
Inference based on samples with clusters of unequal sizes has been widely debated in the 
literature. On the one hand, there is the argument that heteroskedasticity-robust standard errors 
are biased when the presence of intra-cluster correlation is not taken into account; in other words, 
that the observations within each cluster are correlated in some unknown way, but there is no 
presence of correlation between clusters. Nevertheless, adjusting for cluster-robust standard error 
may further affect inference when the size of the clusters is very unbalanced (Cameron and Miller, 
2014). Recently, several authors have demonstrated that the wild cluster bootstrap developed by 
Cameron, Gelbach and Miller (2008) performs better in the presence of clusters with very 
different sizes and in samples with few clusters. 
The discussion of the validation of the evaluation design will be based on the estimates adjusted 
for the wild cluster bootstrap, given the strong presence of intra-cluster correlation and the 
acceptance of this approach in econometric literature for samples such as the one being analyzed 
here16
.
6. Validation of the Evaluation Design
If the randomized order of phase-in approach was successful, baseline characteristics for the early 
treatment group should be similar (statistically equivalent) to those of the late treatment group. 
This may also be true upon disaggregating the treatment status by implementing partners. To 
validate the evaluation design, differences in socio-demographic and productive characteristics are 
estimated by treatment status for the entire evaluation sample17
.
 
16 No marked differences for most of the indicators are found in the statistical inference using the different 
adjustments mentioned. This results are not presented in this report but are available upon request.
17 These characteristics correspond to the outcome indicators and the intermediate indicators for the 
program.
39
The table below reports the average outcomes for the late treatment group (temporal control
group) as well as the difference between early and late treatment.
Chart 5. Baseline differences between early treatment and late treatment at program level
The results suggest that the two samples are fairly balanced; in other words, there are no 
statistically significant differences in the characteristics of the early treatment group and the late 
treatment group. This indicates that the randomization at the program level worked well.
In performing this same analysis by implementing partner, no significant differences are found in 
the socio-demographic characteristics of the producer families. The only exception is observed in 
the score for household dietary diversity for TNS, which is slightly greater for the early treatment 
sample18
.
 
18 This difference is marginal in economic terms. Indicator
Late treatment 
mean
Early treatment - 
Late treatment 
difference
p-value Observations
Progress out of Poverty Index (PPI) - general 64.13 -6.35 0.145 945
Progress out of Poverty Index (PPI) - extreme 25.26 -5.40 0.130 945
MAHFP 11.13 0.02 0.985 923
HDDS 6.57 0.46 0.115 923
Percentage of families with insufficient food 0.31 0.04 0.670 923
Total sales 4,714.77 -64.45 0.995 900
Cocoa sales 850.62 -79.90 0.655 527
Livestock sales 9,668.53 -1800.00 0.705 442
Milk yields 2.76 -0.17 0.475 468
Average cattle weight per hectare 365.76 7.73 0.870 503
Gross income - cattle 9,232.88 -1470.00 0.635 485
Value of production per hectare - cattle 174.56 21.74 0.395 440
Gross margin per unit of land - cattle -44.44 5.87 0.605 485
Cocoa in pulp yields 0.70 0.07 0.610 532
Dry cocoa yields 0.23 0.02 0.610 532
Hectares of cocoa in production 1.45 -0.21 0.135 533
Hectares of cocoa in development 0.93 -0.11 0.270 241
Gross income - cocoa 858.15 -84.12 0.605 532
Value of production per hectare - cocoa 565.04 71.50 0.670 527
Gross margin per unit of land - cocoa -208.45 -52.86 0.650 532
Notes: The table reports the difference in each variable between the early treatment and late treatment groups.
Inference is adjusted by wild cluster bootstrap standard errors. Penultimate column reports the p-value of tests of
differences for each variable.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.
Source: Evaluation team calculations based on Baseline Household Survey - PROGRESA Caribe.
40
Chart 6. Baseline differences between early treatment and late treatment
by implementing partner – socio-demographic characteristics
There are no significant differences in the productive characteristics for TNS and CRS in cattle and 
for LWR in cacao between the early and late treatment. Nonetheless, there are differences for CRS 
in the yields per hectare for dry cocoa beans and cocoa beans in pulp, gross income and the value 
of production of cacao per hectare. Even when the random allocation of early treatment took into 
consideration the partner, the number of clusters was not sufficient to generate homogeneous 
treatment groups in the cacao crop for this partner. On average, the early treatment group has 
higher yields and income than the late treatment group. 
Indicator
Late treatment 
mean
Early treatment - 
Late treatment 
difference
p-value Observations
CRS
Progress out of Poverty Index (PPI) - general 61.37 4.55 0.235 294
Progress out of Poverty Index (PPI) - extreme 21.97 4.00 0.210 294
MAHFP 11.07 -0.23 0.645 292
HDDS 5.75 0.48 0.575 292
Percentage of families with insufficient food 0.32 0.17 0.330 292
LWR
Progress out of Poverty Index (PPI) - general 71.78 -10.93 0.175 287
Progress out of Poverty Index (PPI) - extreme 31.05 -10.49 0.185 287
MAHFP 10.55 0.08 0.805 286
HDDS 7.13 0.20 0.510 286
Percentage of families with insufficient food 0.52 -0.03 0.785 286
TNS
Progress out of Poverty Index (PPI) - general 57.96 -7.75 0.360 364
Progress out of Poverty Index (PPI) - extreme 21.77 -6.67 0.215 364
MAHFP 11.85 -0.15 0.305 345
HDDS 6.72 0.7404** 0.045 345
Percentage of families with insufficient food 0.06 0.09 0.180 345
Notes: The table reports the difference in each variable between the early treatment and late treatment groups.
Inference is adjusted by wild cluster bootstrap standard errors. Penultimate column reports the p-value of tests of
differences for each variable.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.
Source: Evaluation team calculations based on Baseline Household Survey - PROGRESA Caribe.
41
Chart 7. Baseline differences between early treatment and late treatment
by implementing partner – productive characteristics
Indicator
Late treatment 
mean
Early treatment - 
Late treatment 
difference
p-value Observations
CRS
Total sales 1,955.58 409.05 0.660 263
Livestock sales 3,268.71 473.31 0.685 124
Milk yields 3.36 0.11 0.845 143
Average cattle weight per hectare 396.63 -94.65 0.135 178
Gross income - cattle 2,903.40 506.42 0.720 160
Value of production per hectare - cattle 200.05 -17.50 0.605 123
Gross margin per unit of land - cattle -67.49 15.85 0.150 160
Cocoa sales 545.06 267.9137** 0.015 195
Cocoa in pulp yields 0.59 0.3387*** 0.000 195
Dry cocoa yields 0.20 0.1129*** 0.000 195
Hectares of cocoa in production 1.35 -0.16 0.470 195
Hectares of cocoa in development 0.95 -0.25 0.160 75
Gross income - cocoa 552.24 271.3176** 0.015 195
Value of production per hectare - cocoa 447.00 309.0294*** 0.005 195
Gross margin per unit of land - cocoa -137.51 69.42 0.200 195
LWR
Cocoa sales 943.77 -45.65 0.830 282
Cocoa in pulp yields 0.81 -0.04 0.675 285
Dry cocoa yields 0.27 -0.01 0.675 285
Hectares of cocoa in production 1.49 -0.17 0.445 286
Hectares of cocoa in development 0.93 -0.10 0.555 147
Gross income - cocoa 955.66 -60.31 0.745 285
Value of production per hectare - cocoa 662.15 -7.81 0.870 282
Gross margin per unit of land - cocoa -157.16 -67.60 0.550 285
TNS
Livestock sales 12,600.00 -3,340.00 0.540 318
Milk yields 2.42 -0.16 0.660 325
Average cattle weight per hectare 344.05 62.27 0.150 325
Gross income - cattle 13,000.00 -3,410.00 0.485 325
Value of production per hectare - cattle 162.69 38.33 0.310 317
Gross margin per unit of land - cattle -30.57 -2.38 0.940 325
Notes: The table reports the difference in each variable between the early treatment and late treatment groups.
Inference is adjusted by wild cluster bootstrap standard errors. Penultimate column reports the p-value of tests of
differences for each variable.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.
Source: Evaluation team calculations based on Baseline Household Survey - PROGRESA Caribe.
42
In part, the above is due to the targeting by CRS in this crop and the design by cluster of adjacent 
communities. First, most of the CRS clusters that were randomly selected for early treatment are 
from Waslala, which is the municipality that has the highest cacao productivity in the country. This 
is because, at request of the consortium partners, the probability of being selected for early 
treatment was greater and a large number of CRS clusters are located in Waslala. Although the 
situation is similar for LWR, the number of beneficiaries per cluster is more unequal than in CRS, 
which caused a greater balance as the larger clusters were randomly selected for late treatment. 
Furthermore, no significant differences were found by treatment status in most of the indicators 
when the sample is restricted only to women. In the case of total sales, milk yields and hectares of 
cacao in production, the treatment groups did have different results. The evaluation team 
recommends caution in the interpretation of how women respond to the treatment. 
In summary, the conclusion is that randomization worked well at the program level and for TNS 
and CRS in cattle and for LWR in cacao. It also worked well for some indicators for women. 
Therefore, the experimental design generated internal validity for the evaluation to be conducted, 
because, on average, the early and late treatment groups are exposed to the same series of 
external factors, with the exception of the time to which they will be exposed to the program. 
In the case of CRS in the cacao crop, significant differences were found in some indicators between 
the early and late treatment groups; therefore, the results for this group should be treated with 
caution. As the mid-term evaluation will apply the difference-in-differences methodology to 
calculate impact, it could measure the impact of CRS on this crop, as that method allows the 
treatment and comparison groups to differ in characteristics, provided that the parallel trend 
assumption is fulfilled and that the non-observable characteristics of the beneficiaries are time￾invariant. These assumptions must be verified by the evaluation team in the mid-term evaluation. 
At the time of this baseline report, there is no additional information about the true assignment to 
the program in practice, but rather the assignment in the regular CRS monitoring system. 
Therefore, the evaluation team argues that the impact to be identified refers to the average 
treatment effect (ATE). With more information in the mid-term evaluation, it will be possible to 
determine if this will be maintained or if the local average treatment effect (LATE) or the intention 
to treat (ITT) will be estimated. It should be remembered that the results to be identified cannot 
be generalized to other similar populations, contexts or programs (external validity), for the 
reasons mentioned in the evaluation design.
7. Characteristics of Beneficiaries and Key Stakeholders
This section presents comprehensive statistics on beneficiaries, producer organizations and 
private enterprises. The analysis of the characteristics of the beneficiaries is based on a household 
survey that provides information on socio-demographic and productive characteristics. The 
performance of producer organizations is addressed using the ADA Toolkit. The analysis of 
inclusive business models relies on the Link Toolkit.
43
7.1. Characteristics of the Beneficiaries19
The baseline study collected information about the socio-demographic and productive 
characteristics of the beneficiaries. The discussion below addresses the intermediate and outcome 
indicators, as well as other socio-demographic characteristics, such as the schooling levels of 
heads of households and their children and the size of the household. The results are 
disaggregated by partner, value chain and sex of the beneficiary. This analysis endeavors to 
provide a profile of the main characteristics of the beneficiaries, information that will be useful to 
the consortium partners in improving the implementation of the program. 
Socio-demographic Characteristics
57.8% of the households that will be benefitted live in conditions of poverty, while 19.9% face 
extreme poverty. These data differ considerable from the latest official results on poverty for the 
Caribbean Coast from 201420
, where overall poverty is situated at 39.0% and extreme poverty at 
11.5%21
. Nevertheless, these discrepancies suggest that the project has good targeting for the 
condition of poverty. There are no pronounced statistical differences observed in poverty between 
those to be served by CRS and LWR, but there are for TNS, whose beneficiaries are 7% and 5% less 
likely to be poor and extremely poor relative to the average. TNS will serve families with less 
incidence of poverty in all the value chains compared to the rest of the partners. 
Graphic 8. General poverty likelihood, %
 
19 The discussion in this section is on the results for early treatment, as only these beneficiaries will be 
exposed to the program in the first 18 months of activity. 
20 EMNV (2015). Resultados de la “Encuesta Nacional de Hogares sobre Medición de Nivel de Vida - 2014”. 
Managua, Octubre 2015.
21 The PPI methodology for Nicaragua was updated in 2013 by Mark Schreiner using the 2009 LSMS as a 
point of reference to estimate the probability of being poor. 
65.92
60.85
50.2
66.16
62.02
51.85
58.59
52.83
57.78
0
10
20
30
40
50
60
70
80
90
100
CRS LWR TNS Both Cocoa Cattle Male Female Mean
Parther Chain Sex General
Source: Baseline household survey 
44
Graphic 9. Extreme poverty likelihood, %
The value chain with the lowest levels of poverty is the cattle chain, while there are no significant 
differences among the cacao growers and those producer families that work in both chains. It 
should be noted that there are no serious divergences in the incidence of poverty by sex of the 
beneficiary, which suggests good targeting by the program. 
The families to be served by TNS have greater food security than those to be served by the other 
partners. The cattle ranching families also face better food security conditions. Dietary diversity is 
very similar by value chain and there is less diversity in the families to be supported by CRS.
Graphic 10. Dietary Diversity Score
A particular point of interest is that women reported slightly more dietary diversity in their 
families than men. Also, a higher percentage of women state that they have had insufficient food 
in some month of the last year. While this might be the true situation in the household, the fact 
that women generally have better knowledge about the food situation in the home must be 
considered, which may suggest some bias in the results from male beneficiaries. The evaluation 
team observed that, during the process of field data collection, only the LWR surveyors had good 
25.96
20.56
15.1
25.52 22.63
15.93
20.38 16.66
19.86
0
10
20
30
40
50
60
70
80
90
100
CRS LWR TNS Both Cocoa Cattle Male Female Mean
Parther Chain Sex General
Source: Baseline household survey
6.22
7.33 7.46
6.9 6.91
7.18 6.96
7.42
7.03
1
2
3
4
5
6
7
8
9
10
CRS LWR TNS Both Cocoa Cattle Male Female Mean
Parther Chain Sex General
Source: Baseline household survey
45
practice in asking the male beneficiaries to always have their wives present when the food security 
form was applied. This practice should be taken into account in the follow-up surveys. 
Graphic 11. Months of adequate food supply in the household
It is estimated that the planned performance objective22 for food diversity and months of 
adequate household food provisioning should be 7.4 and 11.7, respectively. At the level of the 
partners and by value chain, these indicators are slightly lower than those objectives; therefore, 
the evaluation team considers that it is feasible to achieve them in the framework of the program. 
The average family size is 4.8 people per household; this figure is lower for the families to be 
served by CRS (4.1). The average number of years of schooling for heads of households is 3.7 years 
(fourth grade of primary school); this is slightly lower for CRS (3.1 years). The sons and daughters 
over 15 years of age have average schooling levels of 7.0 years (finished primary school). These 
data clearly reflect the low educational levels of the beneficiaries and their families, which 
suggests the need to create educational mechanisms that are adjusted to the cognitive capacities 
of the producers for training on administrative practices and farm records. 
Productive Characteristics23
PROGRESA-Caribbean will serve producers that work in the cattle or cacao value chains, or in both 
chains. The analysis below discusses the results from the producers in each value chain, but also 
takes into account when the beneficiary works in both chains.
The indicator for milk yields, or liters of milk per cow per day, is 2.59 liters. By partner, the yields 
are higher – by slightly more than one liter – for the beneficiaries from CRS compared to those 
that will work with TNS. The producers that work in both chains claim higher yields than those that 
work only in cattle (one additional liter). Women obtain lower yields than men (by a little more 
 
22 These objectives are calculated using the USDA methodology, which specifies that the informative 
performance objective is calculated as the average of the indicator for the highest tercile for income. 
23 The average indicators by producer for the project discussed in this section differ slightly from the table of 
indicators for PROGRESA-Caribbean according to the contract with USDA, as the results in that table are 
constructed using the totals for the indicators, not the averages for the beneficiaries. 
10.84 10.63
11.7
10.82 10.84
11.54
11.17 11.03 11.15
1
2
3
4
5
6
7
8
9
10
11
12
13
CRS LWR TNS Both Cocoa Cattle Male Female Mean
Parther Chain Sex General
Source: Baseline household survey
46
than one-half liter). The average weight of the cattle is 374 kilograms per hectare. The 
beneficiaries with TNS have a higher average weight per hectare and there are not statistically 
significant differences in this indicator according to the sex of the producer. There is significant 
divergence in the average number of cows in production with CRS (11) compared to TNS (22) and 
throughout the distribution of this indicator24
.
Graphic 12. Milk yields, LTS per cow per day
Graphic 13. Average livestock weight per hectare, KG/HA
The analysis for cacao yields is divided by production in pulp and dry production. The yields for 
cacao in pulp are 0.77 metric tons per hectare (MT/Ha), while yields for dry cacao are 0.26 MT/Ha. 
There are not significant differences among the producers that will work with CRS and LWR (which 
have the larger percentage of beneficiaries in this chain). This is partly because both partners have 
a strong presence in the municipality of Waslala. TNS has lower yields in this chain. Despite this, 
there are not significant differences in the number of hectares in production among the partners; 
thus the difference in yields is in the volume of production. There are no significant differences in 
 
24 For example, while in the 95th percentile the average heads of cattle for CRS is 54, it is 70 for TNS.
3.47
2.26
3.55
2.4
2.67
2.04
2.59
0
0.5
1
1.5
2
2.5
3
3.5
4
CRS TNS Both Cattle Male Female Mean
Parther Chain Sex General
Source: Baseline Household survey
301.98
406.32
334.4
382.4
368.76
403.63
373.5
0
50
100
150
200
250
300
350
400
450
CRS TNS Both Cattle Male Female Mean
Parther Chain Sex General
Source: Baseline Household survey
47
the yields per hectare by the sex of the beneficiary or between those that work only in the cacao 
chain and those that work in both chains. 
Graphic 14. Cocoa in pulp yields, TM/HA
Graphic 15. Dry cocoa yields, TM/HA
Sales are a key indicator for measuring the economic activity of the families. On average, the levels 
of total sales (totaling the income from sales of cacao and cattle) by the PROGRESA-Caribbean 
beneficiaries are US$4,650 annually. The sales vary depending on the chain in which the product is 
commercialized. The cattle ranchers report an average of US$8,402 in annual sales, while the 
cacao growers have an average of US$761 in annual sales. The producers that work in both chains 
have average sales of US$4,842, with the cattle activity driving that average. By sex, women sell 
much less than men, regardless of the chain analyzed. 
0.93
0.77
0.23
0.84
0.75 0.77 0.79 0.77
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
CRS LWR TNS Both Cocoa Male Female Mean
Parther Chain Sex General
Source: Baseline Household survey
0.31
0.26
0.08
0.28
0.25 0.26 0.26 0.26
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
CRS LWR TNS Both Cocoa Male Female Mean
Parther Chain Sex General
Source: Baseline Household survey
48
Graphic 16. Total sales, Annual USD
These differences by chain are similar throughout the distribution of income from sales. 
Nevertheless, it should be noted that, for the 40% of the beneficiaries who sell the least, the 
income from cacao is very similar to the income from total sales, which suggests that a large 
percentage of the beneficiaries that generate the lowest incomes work in this chain. In fact, the 
producers located at the top of the distribution of income from cacao have similar levels as the 
beneficiaries in the middle of the distribution of income from cattle ranching. 
Graphic 17. Total sales by value chain, Annual USD
By partner, the producers with higher levels of sales throughout the distribution of income are 
those that will be served by TNS, which is consistent with the fact that this partner will work with 
larger scale cattle ranchers. In particular, the income from sales in the first part of the distribution 
is similar for LWR and CRS, although the distribution separates at the higher levels due to the 
beneficiaries in the cattle chain that will be served by the latter partner, generating significant 
differences in the mean income from sales for both partners. 
2364.64
898.11
8335.83
4842.3
761.26
8402.38
4987.52
2545.21
4650.32
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
CRS LWR TNS Both Cocoa Cattle Male Female Mean
Parther Chain Sex General
Source: Baseline Household survey
0
5000
10000 15000 20000 USD
.1 .2 .3 .4 .5 .6 .7 .8 .9
Percentiles
Total
Cacao
Livestock
Source: Evaluation team calculations based on Baseline Household Survey - PROGRESA Caribe.
49
Graphic 18. Total sales by implementing partner, Annual USD
An indicator that is more associated with the household standard of living is the gross income, as it 
includes the value of the production that is consumed by the household as well as the income 
from sales. Generally, gross income follows the same pattern as total sales in all the different 
areas analyzed. In the case of cattle, gross income is much higher for the TNS beneficiaries than for 
the CRS beneficiaries. In contrast, no large differences are observed between CRS and LWR in the 
cacao chain, but the TNS producers claim less gross income compared to their peers. By sex, 
women earn less gross income in both value chains, mainly in cattle. 
Graphic 19. Cocoa gross income, Annual USD
0
5000
10000 15000 20000 USD
.1 .2 .3 .4 .5 .6 .7 .8 .9
Percentiles
CRS
LWR
TNS
Source: Evaluation team calculations based on Baseline Household Survey - PROGRESA Caribe.
823.56
895.36
147.42
818.16
762.26
810.57
521.3
774.03
0
200
400
600
800
1000
1200
1400
CRS LWR TNS Both Cocoa Male Female Mean
Parther Chain Sex General
Source: Baseline Household survey
50
Graphic 20. Cattle gross income, Annual USD
The differences in the levels of sales and income are closely related to the size of the producer’s 
holdings. One indicator of productive efficiency, regardless of size, is the value of production per 
hectare25
. The average for the program is US$196 per hectare for cattle and US$637 per hectare 
for cacao. No significant differences are identified between partners or by sex for cattle. In cacao, 
the TNS producers have much less production value per hectare compared to the other partners, 
mainly due to their low yields, for both dry cacao and cacao in pulp.
Graphic 21. Value of production per hectare – cocoa, Annual USD
 
25 This is analyzed by hectare of pasture in the case of cattle and by hectare in production for cacao. 
3409.81
9627.5
4896.78
8410.96 8261.18
4442.12
7757.99
0
2000
4000
6000
8000
10000
12000
CRS TNS Both Cattle Male Female Mean
Parther Chain Sex General
Source: Baseline Household survey
756.03
654.34
157.26
686.64
622.93 634.67 650.08 636.54
0
100
200
300
400
500
600
700
800
CRS LWR TNS Both Cocoa Male Female Mean
Parther Chain Sex General
Source: Baseline Household survey
51
Graphic 22. Value of production per hectare – cattle, Annual USD
The average gross margins, defined as the proportion of net income (gross income minus costs of 
production) relative to gross income, are negative in both chains and for all the consortium 
partners. While average losses in cattle ranching are 39%, they are over 250% for cacao. In both 
chains, only 30% of the beneficiaries have positive margins and the highest margin in cacao is 
around 62%. In the cattle chain, the TNS beneficiaries have the smallest margin for losses, while 
the cacao producers from this partner report the greatest losses, followed by those that will work 
with LWR.
Graphic 23. Gross margin per unit of land – cocoa, %
182.55
201.01
250.33
185.49
201.66
163.56
196.29
100
120
140
160
180
200
220
240
260
CRS TNS Both Cattle Male Female Mean
Parther Chain Sex General
Source: Baseline Household survey
CRS LWR TNS Both Cocoa Male Female Mean
Parther Chain Sex General
Series1 -68.09 -224.76 -1046.1 -110.98 -301.39 -262.41 -253.65 -261.31
-1100
-1000
-900
-800
-700
-600
-500
-400
-300
-200
-100
0
100
Source: Baseline Household survey
52
Graphic 24. Gross margin per unit of land – cattle, %
Approximately 4% of the beneficiaries use sustainable farming and/or cattle ranching practices. 
TNS has a higher percentage of producers with sustainable practices. No differences are observed 
in this indicator by sex of the producer. In analyzing the difference in yields for cacao and cattle 
among the farmers that comply with the requirements for sustainable practices versus those that 
do not, no statistically significant differences are identified. In the case of those that will work with 
CRS, an important aspect is that the producers with sustainable practices report smaller yields in 
both chains compared to those that do not use such practices, especially in milk production. 
Graphic 25. Sustainable farming practices, %
CRS TNS Both Cattle Male Female Mean
Parther Chain Sex General
Series1 -51.64 -32.95 -24.54 -41.77 -36.02 -55.37 -38.57
-60
-20
20
60
Source: Baseline Household survey
1.0%
4.0%
11.0%
3.0%
4.0% 4%
3%
4%
0.0%
5.0%
10.0%
15.0%
CRS LWR TNS Both Cocoa Male Female Mean
Parther Chain Sex General
Source: Baseline Household survey
53
Graphic 26. Cattle ranching practices, %
The intermediate and result indicators discussed above clearly show a great deal of heterogeneity 
among the consortium partners, by chain and by sex. Therefore, it will not be possible to compare 
the performance of partners or chains with their peers. However, this does not affect the 
exploration of the potential heterogeneous effects in the treatment; in other words, that a group 
of individuals may respond differently to the treatment than its comparison group. 
7.2. Performance of producer organizations
7.2.1. Introduction
The “Self-assessment tool for the management of rural associative enterprises” (facilitated self￾assessment) was applied in order establish a baseline for future evaluation of the impact of the 
project on producer organizations. This tool allowed producer organizations to carry out a quick 
analysis of their business management and organizational processes. Gottret, Junkin and Ugarte 
(2012) state that this methodology guides a participatory process of self-evaluation that includes 
the governance structure, associates, management, and the administrative and technical teams.
7.2.2. Methodology
The tool has six areas of assessment: 1) strategic orientation; 2) business management; 3) 
technical services; 4) financial services; 5) structure and functionality; and 6) governance in 
partnership processes. It includes 126 criteria outlined in 34 indicators, which are detailed in 
Graphic 7. For each criterion a score of 1 to 5 is assigned, where 5 is the optimum state of the 
organization. In addition, the methodology provides a set of quantitative indicators, which can be 
triangulated with the perception indicators in order to validate the results.
The score for the sub-areas is obtained by averaging the value of the qualitative indicators, and 
the score for the areas is obtained through a simple average of the sub-areas that comprise it. 
Likewise, the overall score (henceforth known as the Learning Alliance score) was obtained 
through a simple average of the six standardized areas:
1.0%
5.0%
2.00%
4.00%
3%
7%
4%
0.0%
1.0%
2.0%
3.0%
4.0%
5.0%
6.0%
7.0%
8.0%
CRS LWR TNS Both Cattle Male Female Mean
Parther Chain Sex General
Source: Baseline Household survey
54
 𝑦 𝐴 =
 
 
𝐿 𝐴 =
 𝐸 𝐴 
 𝐸 𝐴 
For the interpretation of the results and comparative analysis among areas, the results were 
standardized to obtain values that range from 0 to 100.
 𝑥 =
 
 𝑥 𝑥 00
In addition, the tool includes the analysis of a set of quantitative performance indicators for the 
organization. The tool makes it possible to compare the self-assessment results with the results 
obtained by other rural associative enterprises and with the results of the same organization over 
time. 
The consortium partners acted as facilitators of this process (but not as evaluators or verifiers). 
Following the methodology, they formed groups with the board and management, with 
administrative and technical staff of the cooperative, and with the members. Once this stage was 
finished, a meeting was conducted in which the different groups shared their results and arrived at 
a consensus to determine the final scores for each assessment area. 
The participation of management, administrative and technical teams, and a representative 
sample of members was very important not only for the validity of the self-assessment, but also 
for evaluating the management and organizational processes at different decision-making levels. 
The exercise provided a reflection about the institutional strengthening needs and identified 
opportunities and actions to improve the capacity of the organization.
The results obtained through this methodology allowed producer organizations to categorize 
themselves according to their level of development in terms of business and key organizational 
capacities and to identify needs as well as current and future opportunities for improvement. 
These results will be an asset in designing an action plan to attain specific goals.
7.2.3. Management Results
The following section will analyze the major results obtained from the application of the Learning 
Alliance self-assessment tool to 27 cooperative enterprises that will participate in the PROGRESA￾Caribbean Program. The results are divided into: individual analysis by cooperative, by the 
organization from which it will receive support (partner), and by the chain or chains with which it 
works. 
7.2.3.1.Analysis by Cooperative Enterprise
This section will describe the results obtained overall for each of the 27 cooperatives, grouped 
according to the partner (TNS, LWR and CRS) that will serve them in the framework of the project. 
According to the overall Learning Alliance score, the cooperative enterprises with the highest 
scores were the La Pradera Cooperative with 58.5%, the Multi-service Cacao Cooperative in the 
55
Indio Maíz Reserve (COOSEMUCRIM) with 59.29%, and the Ahmed Campos Union of Agricultural 
Cooperatives (UCA) with 68.74%. The UCA was the only one that reached close to 70% fulfillment 
of the indicators, as can be seen in the graph below. Likewise, it is important to emphasize that 13 
cooperatives, mostly to be served by CRS, have a management capacity of less than 20%. Of 
particular note are the Buena Esperanza Multi-service Cooperative of Pispis (COOMBESPIS) and 
the Miguel Martínez Cooperative for their low management capacity of less than 10%.
Graphic 27. ADA score by cooperative and partner
7.2.3.2.Analysis by Partner
Regardless of the partner, the cooperative enterprises that will be attended within the framework 
of the program have their greatest strengths in the areas of “governance of partnership 
processes” and “structure and functionality”, and their greatest weaknesses in “financial services”. 
In addition, differences were found in the level of development of the cooperative enterprises to 
be served by partner. Those working with CRS show the greatest weaknesses in the areas of 
strategic orientation, business management and technical services, which includes the capacities 
for strategic planning, financial and accounting management, and the provision of technical 
services. 
5.54
7.29
11.82
12.88
13.76
15.1
15.44
15.6
16.26
16.35
16.57
17.2
17.85
26.77
28.61
30.09
31.67
34.65
36.59
36.89
47.91
52.01
53.38
53.64
58.5
59.29
68.74
Miguel Martínez
COOMBESPIS
COMULVAN
COOPELACTME
COOPAPROMU…
COOMUSALWI
UCM
Escolástico Barrera
COOMULCOB
COOMUCOR
COOMUNSOL
Juan Rodriguez
Etanislao Lira
COOSEMUVIS
COMPROMUB
ASIHERCA
COOPROCAR
COOSAGRO
COODEPROSA
COOSEMUP
ACAWAS
Nueva Waslala
CACAONICA
COOPROCAFUC
La Pradera
COOSEMUCRIM
UCA
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
TNS LWR CRS
56
Graphic 28. ADA evaluation areas by partner
Catholic Relief Services (CRS)
Disaggregating by area, 10 of the 11 cooperative enterprises that CRS will support have 
weaknesses in “strategic orientation”, “business management”, “technical services” and “financial 
services”. They show greater development in “governance of partnership processes” and 
“structure and functionality”26, although with potential for improvement. The Nueva Waslala 
Multi-service Cooperative is the only cooperative enterprise that scores above 40% in fulfillment 
of all the areas evaluated in the Learning Alliance assessment. 
 
26 See Table 1 in annex C for more details. 
CRS
TNS
LWR
Partner
Strategic Orientation
Business Management
Technical Services
Financial Services
Structure and Performance
Organizational Processes
0-10 10-20 20-30
30-40 40-50 50-60
60-70
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROBRESA Caribe.
57
Graphic 29. CRS’s cooperatives according to ADA evaluation areas
TechnoServe (TNS)
TechnoServe will work with 10 cooperative enterprises. Of these, two – the Ahmed Campos UCA 
and La Pradera – have high management levels. However, the Augusto Winchang Multi-sector 
Cooperative (COOMUSALWI), the Mejía Dairy Cooperative (COOPELACTE), the Multi-sector 
Organic Producers Cooperative of Muelle de los Bueyes (COOPROMUB) and the Multi-service 
Cooperative of Villa Siquia (COOSEMUVIS) have a pronounced weakness in the supply or 
channeling of technical and financial services for their members, achieving scores of less than 10% 
in this area. This problem is more acute in the United Women’s Agricultural Cooperative for Family 
Economic Development (COPAPROMUDEF) because it also has weaknesses in strategic orientation 
and business management. This would suggest that the cooperative enterprise does not have 
clear and assertive strategies for what it wants to achieve in the medium and long term, as well as 
strategies that coordinate the stakeholders involved for the administration and preservation of 
their resources. 
Escolástico Barrera
COOMULCOB
Etanislao Lira
COOMUCOR
Juan Rodriguez
COOMUNSOL
COOMBESPIS
COMULVAN
Nueva Waslala
Miguel Martínez
UCM
Strategic Orientation Business ManagementTechnical ServicesFinancial Services Structure and Performance Organizational Processes
0-10 10-20 20-30
30-40 40-50 50-60
60-70
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
58
Graphic 30. TNS’s cooperatives according to ADA evaluation areas
Lutheran World Relief (LWR)
In general, the six cooperative enterprises to be supported by LWR show greater management 
capacity than those to be supported by TNS and CRS, with percentages around 50% in almost all 
the areas evaluated by the tool. The homogeneity of the strengths and weaknesses in financial 
services is outstanding in this group of cooperative enterprises. 
Of the cooperative enterprises that LWR will serve, the Multi-service Cacao Cooperative in the 
Indio Maíz Reserve (COOSEMUCRIM) has the best performance due to the way it is internally 
organized to develop its functions and due to compliance with strategies aimed at the effective 
administration of its resources. It achieves scores above 70% in those areas. The cooperative 
enterprises with weaker performance in this group are the Productive Development Cooperative 
of San Juan (CODEPROSA) and the Association for Initiatives and Partnerships of El Castillo 
(ASIHERCA). They have weaknesses in the supply and channeling of technical and financial 
services. Nevertheless, their performance is good compared to the rest of the cooperative 
enterprises that will be served in PROGRESA-Caribbean.
COOMUSALWI
COMPROMUB
COOPAPROMUDEF
COOPELACTME
COOSAGRO
La Pradera
COOPROCAR
UCA
COOSEMUVIS
COOSEMUP
Strategic Orientation Business ManagementTechnical ServicesFinancial Services Structure and Performance Organizational Processes
0-10 10-20 20-30
30-40 40-50 50-60
60-70 70-80 80-90
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
59
Graphic 31. LWR’s cooperatives according to ADA evaluation areas
7.2.3.3.Analysis by Productive Chain
Of the 27 cooperative enterprises, 17 work in the cacao chain, nine work in the dual purpose cattle 
chain, and one works in both chains. The cooperative enterprises that works with both chains 
(COOMUSALWI) has a lower level of management than those that only work in one chain, whether 
that is cacao or cattle. 
CACAONICA
ASIHERCA
COODEPROSA
COOPROCAFUC
COOSEMUCRIM
ACAWAS
Strategic Orientation Business ManagementTechnical ServicesFinancial Services Structure and Performance Organizational Processes
0-10 10-20 20-30
30-40 40-50 50-60
60-70 70-80
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
60
Graphic 32: ADA evaluation areas by value chain
On the other hand, it is clear that the cooperative enterprises that work in the cacao chain are 
relatively stronger in almost all the areas of analysis than those that work in the cattle chain. 
However, regardless of the chain, all have a low level of development in “financial services”.
7.2.4. Analysis of Weaknesses
Given the weakness observed overall in the area of “financial services” or the control and 
management of money, the results were disaggregated by sub-areas for that category. The results 
for “strategic orientation”, “business management” and “technical services” were also 
disaggregated, as they have low levels of development, in order to specifically identify where the 
weaknesses occur. 
7.2.4.1. Financial Services
The low levels of fulfillment by the cooperative enterprises are evident in all the sub-areas under 
“financial services”, particularly for those that will work with CRS and TNS. As the table below 
illustrates, the cooperative enterprises to be served by CRS have the lowest results in all the sub￾areas; specifically, in financial skills for saving. In other words, they do not implement training 
processes on financial planning for their members or for their personnel responsible for the 
finances within the organization. In addition, there are weaknesses in access to and coverage by 
the few or non-existent services provided and in the implementation of mechanisms for feedback 
on customer satisfaction. 
Livestock
Cacao
Both
Value Chain
Strategic Orientation
Business Management
Technical Services
Financial Services
Structure and Performance
Organizational Processes
0-10 10-20 20-30
30-40 40-50 50-60
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
61
Chart 8. Evaluation sub-areas of “Financial Services” by partner
The cooperative enterprises that will be served by TNS need to work mainly on the strengthening 
of access and coverage of their financial services; the establishment of alliances with financial 
institutions, buyers or suppliers of inputs that can provide services; the formulation of policies and 
procedures for providing and paying for these services; and implementation of a system for 
evaluation of satisfaction with the services offered by the cooperative enterprises to their 
members. 
While the cooperative enterprises with which LWR will work obtain superior scores in all the sub￾areas compared to the cooperatives that CRS and TNS will serve, there is also potential for 
improvement in all of the sub-areas, particularly in financial skills for saving. 
7.2.4.2. Strategic Orientation 
The cooperative enterprises that will receive support from CRS have the greatest weaknesses in 
the area of “strategic orientation”. CRS could support the cooperative enterprises in the creation 
and implementation of strategic and business plans, as well as in the establishment of a regular 
monitoring and evaluation system. It could also help the cooperative enterprises in seeking stable 
alliances that benefit their commercial relations and negotiating power. TNS could support the 
cooperative enterprises that it will serve during the project in the creation of business plans that 
explain the overall purpose of the organization, along with the implementation and monitoring of 
those plans.
Chart 9. Evaluation sub-areas of Strategic Orientation by partner
Although the cooperative enterprises to be attended by LWR have a greater level of development 
in the sub-areas than the rest of the cooperative enterprises, they have potential for improvement 
in competencies for “business management”, “strategic planning” and “business plans”. 
Financial skills 
to saving
Planning for 
financial services
Partnership for 
financial access
Financial 
Services 
Access and 
Coverage
Service 
Satisfaction
CRS 2.27 5.81 4.55 6.57 2.53 2.42
TNS 11.25 12.5 5.83 7.5 5.83 4.5
LWR 11.6 18.43 22.69 25 18.52 18.06
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
Partner Market skills
Business 
Management Skills
Strategic 
Planning
Business 
Plan
Business 
Alliances
Access and use 
of information
CRS 16 11.93 3.6 7.01 8.52 12.31
TNS 32.5 35 28.13 12.5 40 46.88
LWR 58.68 41.32 42.85 47.83 60.27 71.23
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
62
7.2.4.3. Business Management 
The cooperative enterprises that CRS will support present the greatest challenges in the area of 
“business management”. Within this area, the major weaknesses are in administrative and 
commercial management. In other words, these cooperative enterprises do not have enough 
personnel or trained personnel responsible for the administrative and commercial areas, 
respectively. An additional influence on this area is that the organizations do not have a manual on 
administrative management or, if there is one, it is not used. In terms of commercial management, 
they also do not have stable relations with buyers in the markets where they commercialize their 
products. 
One of the sub-areas in which the cooperative enterprises with which LWR will work can improve 
is “financial and accounting management”. 
Chart 10. Evaluation sub-areas of “Business Management” by partner
7.2.4.4. Technical Services 
Finally, there is potential for improvement in the area of technical services, mainly in the 
cooperative enterprises that CRS will serve, which have problems in facilitating technical services 
for all of their members. They have few or no alliances with public and/or private suppliers that 
could facilitate innovation and help with providing technical services. In addition, they indicate 
that there is no ongoing re-investment of part of the profits to provide those services. This makes 
them less competitive with services offered by the competition in the market. 
Chart 11. Evaluation sub-areas of “Technical Services” by partner
It is important to emphasize that, while there are weaknesses at the group level in these areas, 
some of the cooperative enterprises obtain high scores in these sub-areas at the individual level. 
This is true for the Nueva Waslala Cooperative27 and the Ahmed Campos UCA28 that will be 
attended by CRS and TNS, respectively.
 
27 See Table 2 in annex C.
28 See Table 3 in annex C.
Partner
Economic 
Analysis
Financial and Account 
Management
Administrative 
Management
Comercial 
Management
Human Resource 
Management
CRS 15.91 14.58 9.77 8.52 15.06
TNS 42.5 28.75 43 32.5 16.25
LWR 66.75 43.13 56.13 60.09 44.77
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
Socio
Access and 
Coverage
Partnership for 
innovation and 
services provision
Investment for 
services 
provision
Managemen
t for services 
provision
Services 
Satisfaction
Skill development for 
sustainable 
production
Operational 
services 
provision
CRS 10.61 3.79 5.81 5.56 5.05 17.53 8.03
TNS 33.75 35.83 13.33 15.83 25 25.71 27.5
LWR 50.49 43.26 30.93 29.31 55.83 38.79 50.5
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
63
Graphic 33. ADA score for Nueva Waslala and UCA
Although in general the cooperative enterprises that LWR will support have a greater level of 
development in the sub-areas evaluated than the average for the cooperative enterprises that will 
be served by the project, the Agroforestry Services and Cacao Commercialization Cooperative 
(CACAONICA)29 obtains scores that are under 10%. This indicates that it should implement 
differentiated actions and would suggest that, in practice, the consortium partners should work 
with differentiated strategies among their cooperatives, in accordance with the level of 
development of their management.
7.2.5. Relations among Areas Evaluated in the Learning Alliance Survey
To complement the findings, a correlation analysis was performed among the six areas evaluated 
by the Learning Alliance methodology, in order to determine whether one area is positively or 
negatively associated with the result of another. The results indicate that there are areas that are 
correlated, some more strongly than others. 
As the graph below shows, Area 1, “strategic orientation”, has a high association (represented in 
pink) with Area 2, “business management”, and Area 3, “technical services”. Area 2 also has a high 
degree of association with Area 3 (“technical services”), and Area 5 (“structure and functionality”) 
with Area 6 (“governance in socio-organizational processes”). However, it is not possible to 
determine the direction of causality between them. In contrast, Area 4, “financial services”, is least 
associated with Area 3 (“technical services”), Area 5 (“structure and functionality”) and Area 6 
(“governance in socio-organizational processes”), represented in green30
. 
 
29 See Table 4 in annex C.
30 To analyze the correlation results among areas, see Table 5 in annex C.
64
Graphic 34. Correlation between ADA evaluation areas
7.2.6. Quantitative Analysis
7.2.6.1.Members
The cooperative enterprises have an average of 100 members. The cooperative enterprises with 
the largest number of members are: Ahmed Campos UCA, Nueva Waslala, UCM and CACAONICA, 
while the cooperatives with the fewest members are COMPROMUB and COOPELACTME. On 
average, most of the members are older than 30 years of age and 30% of the members in all of the 
cooperative enterprises are women. 
Strategic Orientation
Business Management
Technical Services
Financial Services
Structure and Performance
Organizational Processes
Strategic Orientation
Business Management
Technical Services
Financial Services
Structure and Performance
Organizational Processes
1.0
.8-.99
.6-.8
.4-.6
0-.4
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
65
Chart 12: Members characterization
It is clear that the cooperatives in the cacao production chain have a higher average number of 
members than the rest of the production chains. 
Chart 13. Average member by value chain
Members % Women % Men
% < 30 years 
old
% > 30 years 
old
UCA Ahmed Campos 546 31.5 68.5 13.0 87.0
Nueva Waslala 334 25.4 74.6 8.7 91.3
UCM 285 40.0 60.0 n/d n/d
CACAONICA 243 8.6 91.4 0.0 100.0
ACAWAS 157 27.4 72.6 1.9 98.1
COOSEMUCRIM 155 14.8 85.2 0.6 99.4
COOSEMUP 103 10.7 89.3 1.0 99.0
COOMUNSOL 100 43.0 57.0 12.0 88.0
COOPROCAFUC 96 27.1 72.9 4.2 95.8
La Pradera 84 20.2 79.8 11.9 88.1
COOPAPROMUDEF 72 100.0 0.0 2.8 97.2
COMULVAN 50 32.0 68.0 20.0 80.0
COOMUSALWI 50 18.0 82.0 14.0 86.0
COOSEMUVIS 47 10.6 89.4 17.0 83.0
Escolástico Barrer 47 51.1 48.9 6.4 93.6
COODEPROSA 46 17.4 82.6 2.2 97.8
ASIHERCA 45 13.3 86.7 0.0 100.0
COOSAGRO 45 4.4 95.6 4.4 95.6
COOMULCOB 42 19.0 81.0 23.8 76.2
COOMBESPIS 36 47.2 52.8 25.0 75.0
COOMUCOR 35 11.4 88.6 17.1 82.9
COOPROCAR 26 15.4 84.6 7.7 92.3
Etanislao Lira 24 54.2 45.8 20.8 79.2
Juan Rodriguez 23 17.4 82.6 13.0 87.0
Miguel Martínez 21 52.4 47.6 28.6 71.4
COOPELACTME 10 40.0 60.0 40.0 60.0
COMPROMUB 3 33.0 67.0 67.0 33.0
Average 100.9 29.1 70.9 14.0 86.0
Source: ADA Toolkit.
Cocoa 128.9
Both 50.0
Livestock 53.7
Source: ADA Toolkit.
66
7.2.6.2. Fulfillment of Commercialization Targets
On average, the cooperative enterprises have met 97% of their targets for commercialization in 
the current cycle. Some cooperative enterprises, such as COODEPROSA, La Pradera and, especially, 
UCM, surpassed their targets, while ASIHERCA has the lowest fulfillment. ASIHERCA’s lack of 
infrastructure to receive the cacao harvest forces it to resort to local brokers, making it difficult to 
control quality. In recent deliveries to its major buyer, it did not meet the quality standards 
demanded by that buyer; therefore it decided to sell to the local market where the price is lower. 
Consequently, some members have started to sell their cacao on their own. 
Chart 14. Fulfillment of commercialization targets
7.2.6.3.Evolution of Income from Sales
The cooperative enterprises with higher income from sales are COOSEMUVIS, La Pradera, Nueva 
Waslala, COOSEMUP, CACAONICA and ACAWAS, while COOMUNSOL has the lowest income from 
sales among the cooperatives enterprises that the project will serve. If the income is disaggregated 
by the number of members, the same enterprises also have the highest income per member, with 
the exception of Nueva Waslala and ACAWAS. Excluding COOSEMUP because it is an atypical case, 
sales by the enterprises have experienced an average growth of 19%. ACAWAS and CACAONICA 
present the highest rates of growth in their income from sales, while the income for 
COOPROCAFUC, COOSEMUCRIM and ASIHERCA decreased relative to the past cycle. It is possible 
that this is because they received smaller volumes of the harvest. 
Current Cycle
UCM 279.1%
COODEPROSA 138.5%
La Pradera 100.3%
Nueva Waslala 90.9%
CACAONICA 88.5%
COOMUCOR 80.0%
COOSEMUCRIM 74.2%
ACAWAS 73.0%
UCA Ahmed Campos 66.7%
COOPROCAFUC 59.6%
ASIHERCA 20.8%
Average 97.4%
Source: ADA Tookit.
67
Chart 15. Sales by cooperative
7.2.6.4.Differentiated Markets
Differentiated markets offer better prices to their suppliers. This encourages the cooperative 
enterprises to increase their participation in them. Organizations such as the Ahmed Campos UCA 
and UCM place 100% of the production that they sell in this type of market. Although they 
commercialized relatively little production in these markets, CACAONICA and Nueva Waslala, show 
increases compared to the previous cycle. In contrast, income for COODEPROSA and 
COOPROCAFUC, which commercialize a good part of their production through differentiated 
markets, decreased compared to the previous cycle. COOSEMUP, which sells little production in 
this type of market, shows a decrease relative to the previous cycle. 
Current Import
Mean import by 
member Grow rate
COOSEMUVIS C$21900,000.00 C$465,957.45
La Pradera C$18700,000.00 C$222,619.05 0.9%
CACAONICA C$14900,000.00 C$61,316.87 81.4%
Nueva Waslala C$11200,000.00 C$33,532.93 12.0%
COOSEMUP C$9942,518.00 C$96,529.30 216.8%
ACAWAS C$5834,948.00 C$37,165.27 53.1%
UCM C$2640,828.00 C$9,266.06 21.4%
COOSEMUCRIM C$2252,956.00 C$14,535.20 -5.1%
COOPROCAFUC C$1939,241.00 C$20,200.43 -9.5%
COOSAGRO C$1907,908.00 C$42,397.96 24.9%
UCA Ahmed Campos C$600,000.00 C$1,098.90 25.0%
COODEPROSA C$578,334.90 C$12,572.50 35.3%
ASIHERCA C$330,000.00 C$7,333.33 -38.9%
COOMUCOR C$200,000.00 C$5,714.29 33.3%
COOMUNSOL C$26,467.31 C$264.67
Mean C$6196,880.08 C$68,700.28 34.7%
Source: ADA Toolkit.
68
Chart 16. Differentiated Markets
7.2.6.5.Participation by Members in the Final Sales Price 
A part of the price at which the cooperative enterprises sell the production goes into the hands of 
the producer and another part is used to cover operating costs, reinvestment in infrastructure, 
increasing funds, and other items. On average, 78% of the final sales price for the cooperative 
enterprises that will be served by the project remains in the hands of the producer, although this 
varies at the individual level. In organizations such as La Pradera and the UCA, the members 
receive a good portion of the final sales price, while the members of COODEPROSA and 
COOSEMUCRIM receive a lower proportion. Nevertheless, almost all the cooperative enterprises 
have increased the proportion of the final price that remains in the hands of the producers, with 
the exception of COODEPROSA, COOMUCOR and La Pradera, although the decrease is minimal for 
the latter enterprise. 
Chart 17. Producer’s participation on the final sale price
Current 
participation (% of 
total production) Grow rate
UCA Ahmed Campos 100.0 0.0%
UCM 100.0 0.0%
COODEPROSA 80.5 -4.9%
COOSEMUCRIM 73.8 94.9%
COOPROCAFUC 72.2 -20.0%
Nueva Waslala 19.0 394.3%
CACAONICA 17.3 261.5%
COOSEMUP 9.1 -41.1%
Promedio 59.0 86%
Fuente: ADA Toolkit.
Current Paricipation Grow rate
La Pradera 92.25 -0.1%
COOMUCOR 90.00 -3.6%
UCM 86.52 30.8%
ACAWAS 83.74 3.7%
UCA Ahmed Campos 83.33 6.7%
CACAONICA 76.10 34.5%
COOPROCAFUC 74.08 11.1%
Nueva Waslala 72.90 0.4%
COOSEMUCRIM 68.33 380.5%
COODEPROSA 58.29 -11.4%
Average 78.55 45%
Source: ADA Toolkit.
69
7.2.6.6. Financial Ratios
Liquidity measures the capacity to face debts over the short term with floating assets (money) that 
is on hand. It is clear that most of the cooperative enterprises that reported this indicator have the 
capacity to cover their debts. ACAWAS, COOMUNSOL and COOSEMUCRIM have the highest 
liquidity indices. This is because they have a large amount of floating assets or little short-term 
debt. Only COOSAGRO shows a negative index; in other words, in any eventuality, it would not be 
able to pay its debts with its current level of floating assets. 
Chart 18. Financial Ratios
The ratio of liabilities and assets explains what proportion of total liabilities are backed by the total 
assets. The cooperative enterprises have an average coverage of 14.2 times. COMPROMUB, 
COMULVAN, COOSEMUP and La Pradera have coverage above the average. With the exception of 
COOSEMUP, the other cooperative enterprises with high coverage indices have low liquidity 
indices; in other words, most of their assets are fixed, not floating assets. 
7.2.6.7.Employment
An average of only 27% of the workers in the cooperative enterprises are women. It was also 
determined that, on average, women earn less than men. This difference may be because the 
women have less access to high level or management positions in the cooperative enterprises, 
which are associated with higher income. However, this indicator does not clearly show whether 
there is a salary gap between men and women, as it does not make comparisons between the 
same positions or educational levels. 
Liquidity Assets/liabilities
ACAWAS 43.2 1.7
COOMUNSOL 11.4 36.7
COOSEMUCRIM 5.4 5.4
COOPROCAFUC 3.3 2.1
COOSEMUP 3.0 10.2
COODEPROSA 2.6 2.2
COMPROMUB 1.9 54.0
CACAONICA 1.2 1.2
La Pradera 1.2 32.5
COMULVAN 1.0 40.6
UCA Ahmed Campos 0.2 4.5
COOSAGRO 0.1 3.9
Nueva Waslala n/d 1.6
UCM n/d 2.2
Average 6.2 14.2
Source: ADA Toolkit.
70
Chart 19. Gender gap on enployment
7.2.6.8.Management – Performance Link
Not all the cooperative enterprises keep records of the quantitative indicators included in the tool 
and those that do may have weaknesses in the way in which they keep them. Therefore, caution is 
suggested in the interpretation of the results. The absence of the calculation of performance 
indicators in the cooperative enterprises corresponds to the low levels of management achieved. 
Overall, the cooperative enterprises that achieve a higher score in business management are the 
ones that keep some type of record. At the same time, these enterprises are closer to achieving 
their commercialization targets, which suggests a greater capacity for planning and/or projection. 
They also demonstrate greater participation in differentiated markets, with UCM as the exception. 
Chart 20. Management-Performance Link
% female employees
Wage equity 
index
COOSEMUP 50.0% 0.83
UCA Ahmed Campos 37.5% 0.49
ACAWAS 35.7% 0.37
COOPELACTME 33.3% 0.50
CACAONICA 32.1% 0.49
La Pradera 28.6% 0.35
UCM 28.6% 0.26
COOSEMUVIS 25.0% 0.40
COOSEMUCRIM 20.0% 0.35
Nueva Waslala 18.2% 0.18
COOPROCAFUC 14.3% 0.01
COOSAGRO 12.5% 0.09
Average 28.0% 0.36
Source: ADA Toolkit.
71
Business 
Management Score Sales
Fulfillment of 
Commercialization Targets Sale's grow rate
Differentiated 
Markets
COOSEMUCRIM 3.9 2252956.0 74% -5.1% 0.7
UCA Ahmed Campos 3.7 600000.0 67% 25.0% 1.0
La Pradera 3.6 18700000.0 100% 0.9% 1.0
COOPROCAFUC 3.5 1939241.0 60% -9.5% 0.7
CACAONICA 3.4 14900000.0 89% 81.4% 0.2
Nueva Waslala 3.3 11200000.0 91% 12.0% 0.2
COOSEMUP 3.1 9942518.0 216.8% 0.1
ACAWAS 2.9 5834948.0 73% 53.1%
COODEPROSA 2.7 578334.9 139% 35.3% 0.8
COOSEMUVIS 2.6 21900000.0
ASIHERCA 2.6 330000.0 21% -38.9%
COOSAGRO 2.3 1907908.0 24.9%
UCM 2.3 2640828.0 279% 21.4% 1.0
COOMUCOR 1.7 200000.0 80% 33.3%
Source: ADA Toolkit.
72
7.3. Inclusive Business Models: Link Toolkit
7.3.1. Introduction
The stakeholders in the value chain are linked by trade relations in which each one acts as buyer 
and seller at the different stages. For example, producers are sellers and the cooperatives are 
buyers in the first link, but in the following the cooperatives are sellers and the companies are the 
buyers.
For the purpose of PROGRESA Caribe, the LINK methodology will be used to carry out the analysis 
to capture the influence of the project on the participation and business practices of the private 
sector. The LINK method involves the use of four tools: 1) mapping of the value chain, which helps 
identify the key stakeholders, process and services within a value chain and how each one 
interacts with the others, 2) the template of the business model, which allows organizations 
and/or businesses to identify their current status as an entity and to identify areas for 
improvement and/or intervention; 3) the principles for inclusive business models, in which the 
level of business inclusion between producers and buyers is determined; and 4) the prototype 
cycle, in which the previous tools are unified to derive strategies for expansion or to implement 
new innovations to promote the participation of small farmers in the chain. These tools facilitate a 
better understanding of the key players, processes and relations within a value chain.
For the baseline study, tool #3 has been applied to the companies that have worked with the 
cooperatives targeted by the project. The tool was applied to a total of 12 vendors (cooperative 
enterprises) and seven buyers (five companies, two cooperative enterprises). As expected, the 
evaluation team got different perceptions and opinions as well as a better understanding of the 
context of the value chain. These issues are discussed in detail in the following sections.
The analysis is structured as follows: First, the perspective of the cooperative enterprises (sellers) 
is addressed generally, by partner, by chain and by linked enterprise. Secondly, the perspective of 
the companies (buyers) is discussed by company and by chain. Then we analyze the relationship of 
the perspective on the principles of vendor cooperative enterprises and purchasing companies. 
Later, the connection between the Learning Alliance tool and the LINK methodology will be 
explored, by cooperative enterprise and consortium member, in order to identify factors for 
synergy. Finally, in the section on conclusions and recommendations, the most important findings 
and recommendations will be described. This information will provide input to partners for 
decision-making in order to improve the inclusion of small producers in the value chain.
7.3.2. Methodology
LINK Tool #3 is a participatory instrument designed to analyze the principles for inclusive business 
models. Its purpose is to help the buyers and the sellers to self-evaluate the status of the 
commercial relationship using inclusion criteria. The assessment offers input for improving 
inclusion of small scale producers in the value chain. 
LINK Tool #3 evaluates six principles:
1. Collaboration among Stakeholders: establishes the level of shared goals that exist among 
buyers and sellers. 
73
2. Effective Market Linkages: evaluates the relations that are formed in both parties to access 
market information and quality products. 
3. Transparent and Consistent Governance: evaluates quality and commitment in the rules of 
negotiation. 
4. Access to Services: measures the impact of the company in collaborating with the cooperative 
in access to services. 
5. Inclusive Innovation: evaluates the joint promotion of innovation in products and good 
practices. 
6. Measurement of Outcomes: incorporates customized indicators and monitoring plans to 
evaluate the health of commercial relations of a for-profit company, as well as its effectiveness 
as a driving force for development. 
Each principle is disaggregated into criteria, which vary depending on the principle. Each criterion 
is assessed, based on the perception of the key stakeholders from the cooperative enterprise and 
the for-profit company. In the case of the cooperative enterprises, when several stakeholders 
participate in the evaluation, the facilitators discuss the assessments with them to arrive at a 
consensus. Thus, a reasoned process is followed and this also helps to contextualize the 
commercial situation of the cooperative, which is unknown in many cases by those stakeholders 
who are linked to the board of directors but who do not have active involvement in the 
management of the cooperative. 
The score for each criterion is on a six-point scale (0 to 5), with 0 representing “very strongly 
disagree” and 5 representing “very strongly agree”. The scale is designed to not have a neutral 
score; in other words, each response will have a positive or negative slant. There are some criteria 
that may not apply (N/A), depending upon the context in which the business model is developed. 
To maintain standardization in the application of the tool, the evaluation team agreed with the 
CRS business development management area on the filters for criteria that can be considered “not 
applicable”.
7.3.3. From the Perspective of Vendor Cooperative Enterprises
This section addresses the status of the commercial situation of each vendor cooperative 
enterprise, from the general context to the evaluation by principles, relating this to the production 
chain, associated company and project partner.
7.2.3.4. Cooperative Enterprises (Sellers)
7.2.3.4.1. Overall Score
Fifty percent (50%) of the cooperative enterprises are above the overall average. Nueva Waslala 
and COOPROCCAFUC, both cooperatives that work with cacao and are associated with Ritter Sport 
Company, obtain the highest scores. No systematic and statistically significant differences are 
observed in the scores related to the associated company or in the value chain that works with the 
cooperative31
.
 
31 This result was based on linear regressions, using the overall score as a dependent variable and binary 
variables of the production chains, partners and associated company, separately. 
74
Graphic 35. Link Score by cooperatives grouped by partner and value chain
7.2.3.4.2. Disaggregation by Principle
The heat map below helps to more deeply explore the results by principle for each cooperative 
enterprise. The map classifies as “good” (green) those scores that are above the average for each 
principle and the color gradually turns to red when the score is lower. 
2.24
2.31
2.12
2.14
1.76
2.26
2.58
2.64
2.75
3.09
3.18
3.24
0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
COOSEMUP
COOSEMUVIS
CACAONICA Ethicable
UCA Ahmed Campos
ASIHERCA
UCM
COODEPROSA
COOSEMUCRIM
Acawas
CACAONICA Ritter
COOPROCAFUC
Nueva Waslala La Perfect
a Nilac Eticabl
e Gente del Cacao Ritter Livestock Cacao Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe
75
Chart 21. Inclusive business principles by sellers
The principle with the lowest score for all the cooperatives is “collaboration among stakeholders”. 
Nevertheless, the Ahmed Campos Union of Agricultural Cooperatives (UCA) and COOSEMUP 
present the greatest weaknesses in this principle, as they have no aligned environmental 
objectives and lack incentives to stimulate collaborative behavior between seller and buyer. In the 
exclusive case of the Ahmed Campos UCA, this cooperative also has a weakness in the alignment 
of commercial goals. It should be added that, although COODEPROSA has a high score for this 
principle, it has weaknesses in the alignment of both commercial and environmental objectives. 
Lastly, the group of cooperatives with the highest assessments for this principle has low scores on 
the criterion of informal exchange of information. Nonetheless, this is not necessarily negative, as 
the organizations prefer the formal exchange of information. 
The principle with the greatest weakness for the cooperative enterprises is “inclusive innovation”. 
Seven of 12 sellers score 0 on this. With the exceptions of Acawas, Nueva Waslala and CACAONICA 
(in relationship with Ritter Sport), this principle represents the Achilles’ heel for the sellers in 
terms of inclusive commerce. While COOSEMUCRIM and COOPROCAFUC score above the average, 
this is because the overall average is low for the principle, not because their assessments are high. 
The evaluation team believes that the low scores on this principle are due to the few efforts for 
inclusive innovation that exist in the chains analyzed and the fact that the greatest portion of 
existing innovations are implemented/imposed by the buying company; it is not a mutual 
improvement process. Furthermore, in the case of the cacao chain, there is noteworthy 
differentiated treatment on this principle by Ritter Sport, as most of the inclusion depends, in part, 
on the long-standing nature of the commercial relationship and the quality of the product that it 
buys. 
The second strongest principle for the cooperative enterprises is “market linkages”. Nueva Waslala 
and COOPROCAFUC have the highest scores, above the average for this principle. Nueva Waslala 
scores 4 points on six of the nine criteria. Most of the weaknesses in this principle are related to 
the conservation of environmental resources and the capacity to identify market opportunities. 
The self-evaluation of COOPROCAFUC is 5 points in the stability of its supply of production and the 
profitability of commercial relations, while its lowest scores are for the capacity to identify market 
opportunities, environmental conservation and the stability of income from the commercial Buyers-Sellers 
Sinergy
Market 
Link Governance
Services 
Access
Inclusive 
Innovation
Results 
Tracking Link Score
Acawas 3.77 3.33 2.36 1.9 3.14 2 2.8
COOSEMUVIS 2.77 2.78 2.91 2.2 0 3.17 2.3
ASIHERCA 3.46 1.78 2 1.3 0 2 1.8
UCM 3.69 3.22 2 2.2 0 2.45 2.3
UCA Ahmed Campos 2.54 2.78 2.92 1.4 0 3.17 2.1
Nueva Waslala 4.08 3.67 2.75 3.1 3.57 2.25 3.2
COOSEMUCRIM 3.23 3.11 3 1.8 1.14 3.56 2.6
COOPROCAFUC 3.77 3.78 3.45 2.8 1.71 3.58 3.2
COODEPROSA 4.08 3 3 1.4 0 4 2.6
COOSEMUP 2.62 2.89 2.36 2.9 0 2.67 2.2
CACAONICA Ethicable 4.15 2.78 3 0.78 0 2 2.1
CACAONICA Ritter 3.31 3 2.73 2.3 3.71 3.5 3.1
Average 3.46 3.01 2.71 2.01 1.11 2.86 2.52
Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe.
76
relations with its customer. ASIERCA has the lowest assessment, with major weaknesses in 
maintaining a stable product supply, cooperation with its customer to conserve natural resources, 
reaction to the needs of the customer, and command of its market position. 
In terms of the principle on “transparent and consistent governance”, COOPROCAFUC has the 
highest score, which is because it scores 5 points on knowing the quality standards of the 
customer, understanding how the customer prices the products, and compliance with informal 
agreements. It should be emphasized that, although these sellers obtained the highest 
assessments for this principle, they do not have formal contracts in their commercial relations and 
do not have access to insurance against the risks of production. Meanwhile, ASIERCA and UCM 
have the lowest scores. The self-evaluation of both is 0 regarding formal and informal contracts, as 
well as in sharing the productive and market risk with their buyers. In addition, ASIERCA has 
serious weaknesses in applying traceability to its products. 
Nueva Waslala, COOPROCAFUC and COOSEMUP score above the average for the principle of 
“access to services”, although their scores are around 3 points. The remaining cooperatives have 
fairly low scores, especially CACAONICA in its relations with Ethiquable. This principle can be 
classified as the second weakest, after inclusive innovation. The low score by CACAONICA in its 
relationship with Ethiquable is because this company is located in France, making it a mainly 
virtual commercial relationship and limited to the sale of the product. For the remaining 
cooperatives that work with cacao, again the difference in the provision of services is related to 
the differentiated treatment by Ritter. Nevertheless, an important feature is that all the 
cooperative enterprises state that they have opportunities for financial services with this company 
at very affordable interest rates, although with limitations in the amounts and in the number of 
loans that they receive at one time. Moreover, some cooperatives receive bonuses for 
infrastructure. The major difference between the cooperatives is in the provision of technical 
assistance and in the access to production and post-harvest technologies. Although the sellers 
state that they receive few services, on average they say that they are 80% satisfied with the set of 
services that they receive from their buyers. 
Lastly, the “measurement of outcomes” receives the third lowest score, although there are high 
assessments for CACAONICA-RITTER, COOPROCAFUC, COOSEMUCRIN and CODEPROSA. The 
lowest scores for this principle are around 2 and 2.5 points. The evaluation team notes that the 
majority of the sellers keep a basic record system (amounts sold, price and income from sales); 
however, the most important measurements mainly depend upon the interests of the buying 
companies. For example, in the case of cacao, the most important aspect is its quality; however, 
most of the cooperatives do not have the equipment or experts to measure the quality. Therefore, 
the measurement is performed indirectly by the buyer at the time the product is sold. 
7.2.3.5. By Partner
To perform the analysis by implementing partners, a simple average is taken of the score of each 
cooperative enterprise to be attended by each partner for each principle. The previous section 
mentioned that no systematic differences were found in each principle by implementing partner. 
However, the assessment by partner helps identify the general areas for improvement that each 
of these should prioritize. The results are as follows:
77
Graphic 36. Link Score by partner
The enterprises that will work with Catholic Relief Services (CRS) as their partner have the highest 
scores, with slightly lower scores by those that will work with Lutheran World Relief (LWR) and 
TechnoServe (TNS). None of the partners achieves a score of 3.
Chart 22. Inclusive business principles by partner
The heat map shows the order of scores for each principle. It remains practically the same as the 
previous analysis; in other words, “collaboration among stakeholders” continues to be the 
principle with the best evaluation and “inclusive innovation” continues to be the weakest. 
The cooperatives to be attended by CRS achieve scores above the average in each principle, with 
the exception of “transparent and consistent governance” and “measurement of outcomes”. The 
assessment of the latter principle is lower compared to its peers. 
The cooperative enterprises associated with LWR do not surpass the average on “market linkages” 
and “access to services”. “Collaboration among stakeholders” is its major strength, while “inclusive 
innovation” is its main weakness. It should be emphasized that it obtains the best score among its 
peers on “transparent and consistent governance”. 
The cooperatives working with TNS surpass the average for “transparent and consistent 
governance” and “measurement of outcomes”; achieving the highest score among its peers for 
2.59
2.23
2.75
0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
LWR
TNS
CRS
Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe
Buyers-Sellers 
Sinergy Market Link Governance
Services 
Access
Inclusive 
Innovation
Results 
Tracking Link Score
LWR 3.68 2.97 2.79 1.75 1.39 2.95 2.59
TNS 2.64 2.81 2.73 2.17 0 3 2.23
CRS 3.88 3.44 2.38 2.65 1.79 2.35 2.75
Average 3.40 3.07 2.63 2.19 1.06 2.77 2.52
Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe.
78
the latter. Nevertheless, its LINK score is the lowest among the consortium partners. Its weakest 
principles are “access to services” and “inclusive innovation”. 
These results reflect that the partners are heterogeneous in terms of strengths and weaknesses of 
the cooperatives that they will serve. This suggests that the strategy for strengthening inclusive 
businesses will be differentiated among the partners, although it is evident that they all must place 
priority on attending to the task of “inclusive innovation”. 
7.2.3.6. By Chain
The calculation for this discussion was based on the simple average of the sellers that belong to 
each production chain for each principle, obtaining the following:
Chart 23. Inclusive business principles by value chain-Sellers
The cooperatives that work in the cacao chain, on average, surpass those that work with 
smallholder cattle ranchers, by a difference of 15%. This is because the cooperative cacao 
enterprises that have a commercial relationship with Ritter have better scores. 
In the principles on “measurement of outcomes” and “access to services”, the scores of the 
cooperatives in the cattle chain surpass the scores of those that work with cacao. Regarding 
“access to services”, the difference is a little more than 30%. The evaluation team considers that 
this may be because there is a greater dependency by the cooperative enterprise on the buying 
company in this chain. For example, in some cases, the buying company is the owner of the 
cooling tanks in the cooperative, an essential input for the operation of the cooperative. This 
enables the cooperative to have greater control of the milk quality in the collection system and 
also means that it has a greater probability of receiving technical assistance and support for post￾milking management. 
7.2.3.7. By Linked Company
This analysis involved a simple average of the cooperative enterprises grouped according to the 
company to which they sell. Later, the data from this section will be used as input for the 
comparison between buyers and sellers. 
The cooperatives that sell to Ritter lead the scoring with 2.69, 15% more than the next-best score 
for commercial relations. None of the commercial relations from the perspective of the seller 
achieves a score of 3.
Buyers￾Sellers 
Sinergy
Market 
Link Governance
Services 
Access
Inclusive 
Innovation
Results 
Tracking Link Score
Cacao 3.61 3.04 2.72 1.9 1.33 2.85 2.58
Livestock 2.69 2.83 2.64 2.55 0 2.92 2.27
Average 3.15 2.94 2.68 2.23 0.67 2.89 2.42
Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe.
79
Graphic 37. Link Score by linked company
While the cooperative enterprises associated with Ritter have the best evaluations, they do not 
surpass the average for the principle on “transparent and consistent governance”, which is the 
second lowest score. This result is low due to the lack of formal agreements with most of the 
cooperative enterprises and a low assessment of compliance with informal agreements in the 
period evaluated. The evaluation team was informed that Ritter will establish formal agreements 
with market quotas with each of its sellers, beginning in 2016. 
Regarding “inclusive innovation”, while the cooperatives associated with Ritter are the only ones 
that have some type of joint innovation with that company, this figure is only 33% of the 
maximum possible (5).
CACAONICA, in its association with Ethiquable, has the highest score for “collaboration among 
stakeholders” and “transparent and consistent governance”, achieving a score of 4.15. However, it 
obtains the lowest scores in “access to services” and “measurement of outcomes”, mainly because 
the commercial relations are developed virtually. The combination of these principles places it as 
the commercial relationship with the fewest business linkages. 
The cooperatives associated with La Perfecta, Gente del Cacao and Nilac have similar scores 
overall. Although there are differences in the performance by principle, all obtained 
scores/assessments with a negative trend. 
2.69
2.31
2.12
2.14
2.24
0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
Ritter
NILAC
Eticable
Gente del cacao
La Perfecta
Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe
80
Chart 24. Inclusive business principles by linked company
7.3.4. From the Perspective of the Companies (Buyers) 
This section addresses the current status of commercial relations from the perspective of the 
buyers; in other words, the for-profit companies. For those companies that have more than one 
associated seller in the framework of the program, the section addresses the general perception of 
these, without individual distinctions. It discusses the overall assessment for each company and 
according to the value chain in which it operates. It should be stated that three companies are 
milk collection enterprises (Quebar, Carsa and Coagropek) and purchase from smallholder 
producers; a small percentage of these producers will be served by the program32
.
7.3.4.1.By Company
Graphic 38. Link Score by Buyers
 
32 The information about which of the producers that will work with the program sell to these collection 
companies is not available. Buyers-Sellers 
Sinergy
Market 
Link Governance
Services 
Access
Inclusive 
Innovation
Results 
Tracking Link score 
Ritter 3.67 3.11 2.66 2.1 1.66 2.92 2.69
NILAC 2.77 2.78 2.91 2.2 0 3.17 2.31
Gente del cacao 2.54 2.78 2.92 1.4 0 3.17 2.14
La Perfecta 2.62 2.89 2.36 2.9 0 2.67 2.24
Eticable 4.15 2.78 3 0.78 0 2 2.12
Average 3.15 2.87 2.77 1.88 0.33 2.79 2.30
Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe.
3.82
3.62
3.37
3.29
3.26
2.26
2.25
0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
Ritter
Gente del cacao
Carsa
La Perfecta
Nilac
Coagropek
Quebar
Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe.
81
Only Quebar and Coagropek, which buy from smallholder producers, are below the average 
performance for buyers. This result is expected as this type of company does not usual operate 
under inclusivity criteria, which suggests that there is room for improvement in this link of the 
cattle value chain. Ritter and Gente del Cacao achieve the highest scores. This result is influenced 
by the good assessment in all the principles evaluated, except “inclusive innovation” for Gente del 
Cacao. These companies believe that they have a fairly high degree of inclusivity with their sellers. 
Chart 25. Inclusive business principles by Buyers
The heat map shows that the principle with the highest score for all the companies is 
“collaboration among stakeholders”, which coincides with the self-evaluation results from the 
sellers. It is noteworthy that the companies are conscious that “inclusive innovation” is a pending 
task. One of the four companies that buys from the cooperatives reports the minimum score for 
this principle. Ritter assesses the degree of innovation that it undertakes with its suppliers as 
positive. 
7.3.4.2. By Chain
This analysis was based on a simple average of the buying companies by principle, depending on 
the chain in which they operate, with the following results: 
Buyers-Sellers 
Sinergy
Market 
Link Governance
Services 
Access
Inclusive 
Innovation
Results 
Tracking Link Score
Ritter 4.08 4.00 3.27 3.70 3.43 4.42 3.82
Gente del cacao 3.62 4.78 3.91 4.40 0.00 5.00 3.62
La Perfecta 4.08 3.56 3.82 4.10 0.00 4.17 3.29
Nilac 4.23 4.00 3.36 3.80 0.00 4.17 3.26
Coagropek 3.54 2.78 2.18 2.00 1.57 1.50 2.26
Carsa 4.46 3.44 2.91 3.10 3.57 2.75 3.37
Quebar 3.50 3.33 2.45 1.89 0.00 2.33 2.25
Average 3.93 3.70 3.13 3.28 1.22 3.48 3.12
Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe.
82
Graphic 39. Link Score by value chain-Buyers
On average, the companies that purchase cacao surpass the cooperatives that buy milk, by a 
difference of almost half a point. Exploring more deeply by principle and associating the buyers 
with the chain in which they operate, the study identifies a similar pattern as that found in the 
section on the perspective of the sellers. The highest assessment of the buyers is in “collaboration 
among stakeholders” and the lowest is in “inclusive innovation”, where the buying companies 
that work in the cattle chain score 0 and those in the cacao chain score 1.71 (due to the influence 
of Ritter). The rest of the assessments are situated around the median scale. The companies that 
buy cacao, on average, surpass those that buy in the cattle chain in all the principles except for 
“transparent and consistent governance”, where there is a minimal difference.
Chart 26. Inclusive Business principles by value chain-Buyers
7.3.5. Relationship of the Perspective on the Principles of the Buyer and Seller by Commercial 
Relation
This section discusses the difference between the assessment of the sellers and the buyers, to 
measure the level of agreement between the parties in the perception of inclusivity in the 
commercial relationship. It should be recalled that, for the vendors, the result is an average of all 
the suppliers for the companies that will work with the program. 
There is a wide difference between the scores of sellers and buyers, ranging from 0.96 to 1.48 
points of difference. The sellers have a perception toward the negative plane and the buyers are 
more positive. 
3.27
3.72
0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00
Livestock
Cacao
Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe.
Buyers-Sellers 
Sinergy
Market 
Link Governance
Services 
Access
Inclusive 
Innovation
Results 
Tracking Link Score
Livestock 4.15 3.78 3.59 3.95 0 4.17 3.27
Cacao 3.85 4.39 3.59 4.05 1.71 4.71 3.72
Average 4.00 4.09 3.59 4.00 0.86 4.44 3.50
Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe.
83
Graphic 40. Comparison between buyer and seller perspective
Ritter and its suppliers have a better assessment of the commercial relationship compared to their 
peers. An interesting feature of this commercial relationship is that Nueva Waslala has an overall 
score of 65%, which reflects that the overall results are influenced by this cooperative. This 
confirms the prior observation of the evaluation team about the differentiated treatment of its 
suppliers by Ritter. 
In contrast, while the Gente del Cacao enterprise assesses itself very positively, its supplier has a 
negative perception. There is less inequality in the assessments by buyers and suppliers for Nilac 
and La Perfecta.
Chart 27. Disaggregated comparison between buyers and sellers
The buyer always has a better evaluation than the seller in all the principles, with the exception of 
“inclusive innovation”. Also, at this level of disaggregation, the greatest disparity is found in the 
relationship with Gente del Cacao. This difference mainly involves the principles of “market 
linkages” and “access to services”; in the latter, the buyer has an evaluation that is twice as high as 
that of the seller. 
2.69
2.31 2.14 2.24
3.82
3.26
3.62
3.29
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
5.00
Ritter NILAC Gente del cacao La Perfecta
Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe.
Seller perspective Buyer Perspective
Seller Buyer Seller Buyer Seller Buyer Seller Buyer
Buyers-Sellers Sinergy 2.77 4.23 2.62 4.08 2.54 3.62 3.67 4.08
Market Link 2.78 4.00 2.89 3.56 2.78 4.78 3.11 4.00
Governance 2.91 3.36 2.36 3.82 2.92 3.91 2.66 3.27
Services Access 2.2 3.80 2.9 4.10 1.4 4.40 2.1 3.70
Inclusive Innovation 0 0.00 0 0.00 0 0.00 1.66 3.43
Results Tracking 3.17 4.17 2.67 4.17 3.17 5.00 2.92 4.42
Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe.
Nilac La Perfecta Gente del cacao Ritter
84
Ritter claims little difference in the first three principles and a moderate discrepancy in the 
remainder. Nilac and La Perfecta present moderate differences in the six principles evaluated. It is 
very important to indicate that, with the exception of Ritter, all the parties recognize the absence 
of inclusive innovation in their business model. 
7.3.6. Connection between the Learning Alliance and LINK
This section addresses the connection between the Learning Alliance and LINK tools in the 
cooperative enterprises in which both instruments were applied. The score for each cooperative 
enterprise for both tools was evaluated, along with the average by partner that will attend each 
cooperative. In order to make the two tools comparable, the LINK results are standardized in the 
same spirit as the Learning Alliance methodology, to create a scale of 0 to 10033
.
7.3.6.1.By Cooperative Enterprise
The scores achieved in the Learning Alliance methodology are not determined by the scores 
obtained in LINK and vice versa34. The cooperative enterprises with the best evaluations are 
COOPROCAFUC and Nueva Waslala, while these same cooperatives achieve middling scores in the 
Learning Alliance. The most interesting case is that of the UCA, which has the highest evaluation in 
the Learning Alliance, while it is in the last positions in the LINK methodology. 
Chart 28. Comparison between ADA and Link score
7.3.6.2.Comparison by Principle
The objectives of Learning Alliance and LINK tools are different. The Learning Alliance is an internal 
self-evaluation, while LINK measures the relationship with a specific buyer. Nevertheless, the 
 
33 𝑥 =
𝐶 𝑉 𝑀 𝑉 
𝑀 𝑉 𝑀 𝑉 
34 To confirm this, two linear regressions were performed; one with the LINK score as a dependent variable 
and the Learning Alliance score as an independent variable and vice versa. None of the coefficients obtained 
were statistically significant. ADA score LINK score
Acawas 47.9 55.0
ASIHERCA 30.1 35.1
CACAONICA Ethicable 53.4 52.1
COODEPROSA 36.6 51.6
COOPROCAFUC 53.6 63.6
COOSEMUCRIM 59.3 52.8
COOSEMUP 36.9 44.8
COOSEMUVIS 26.8 46.1
Nueva Waslala 52.0 64.7
UCA Ahmed Campos 68.7 42.7
UCM 15.4 45.2
Source: Evaluation team calculations based on Baseline ADA 
and Link Toolkits - PROGRESA Caribe.
85
correlation between the LINK principles and the Learning Alliance areas suggests that there is 
some degree of association in some of them:
Chart 29. Association between and Link principles ADA areas
“Collaboration among stakeholders” in LINK is not associated with any Learning Alliance area as 
this principle exclusively measures the buyer-seller relationship and is not involved with the 
internal development of the cooperative. 
The principle of “Market linkages” endeavors to measure the profitability and stability of the 
supply of products by the cooperative, as well as the market opportunities. This is somewhat 
related to “strategic orientation”, as the latter addresses issues of access and use of information, 
strategic alliances and market competencies. It is also related to “business management”, 
especially in issues connected to commercial management, which ensure compliance with 
production processes that are reflected in both the supply and quality of the product, heavily 
impacting the profitability of the cooperatives. Almost to the same degree, “market linkages” is 
also associated with “governance in organizational processes”, as this area places a priority on the 
leadership needed for proper operations and to undertake processes and improvements. Lastly, 
“market linkages” has a stronger relationship to “technical services”, mainly due to the sub-areas 
of alliances for innovation in providing services and the development of competencies for 
sustainable production. This directly affects the volume and quality of production. 
The principle of “transparent and consistent governance” in the LINK methodology, which 
measures how the commercial relations with the customer are managed, has the greatest 
connection to the Learning Alliance. It is strongly related to all areas of the Learning Alliance, with 
the exception of “technical services” and “financial services”. It is linked closely to the “strategic 
orientation” area as it measures the business plan, strategic planning and the use of information. 
“Business management” is also connected to this as it evaluates commercial management, which 
includes the relationship with buyers and the commercialization of products. Finally, “structure 
and functionality” is connected to governance as it evaluates standards and regulations, 
transparency, accountability and communications. 
“Access to services” does not have any significant connection to the Learning Alliance as it directly 
evaluates the services that the cooperative directly or indirectly accesses due to the commercial 
relationship with its buyer, while the Learning Alliance seeks to evaluate all the internal relations 
necessary for the operation of the cooperative. 
Strategic 
Orientation
Business 
Management
Technical 
Services
Financial 
Services
Structure and 
Performance
Organizational 
Processes
Buyers-Sellers Sinergy -0.1075 -0.3212 -0.0594 -0.115 -0.372 -0.3258
Market Link 0.2842 0.3138 0.4539 0.0475 -0.09 0.3114
Governance 0.7234 0.6261 0.3703 0.3255 0.5243 0.6708
Services Access -0.1767 0.076 0.1744 -0.4727 -0.2339 0.0095
Inclusive Innovation 0.3633 0.3298 0.6842 0.0173 0.294 0.4876
Results Tracking 0.5564 0.467 0.3043 0.0238 0.3041 0.4816
Source: Evaluation team calculations based on Baseline Link Toolkit - PROGRESA Caribe.
86
The LINK principle of “inclusive innovation” is most closely related to the Learning Alliance in 
“technical services”, which suggests that the joint development of processes for innovation with 
the buyers facilitates the technical training by the cooperative of its members. 
Finally, the “measurement of outcomes” is connected to “strategic orientation”, “business 
management” and “governance of organizational processes”. These associations suggests that 
greater organizational capacity will increase the interest of the cooperative in measuring the 
results of its commercial relations. 
7.3.6.3.By Project Partner
Chart 30. Comparison cooperatives average grouped by partner
As mentioned above, there is no direct relationship between the LINK scores and those of the 
Learning Alliance, as the focus of the two tools is different. At the same time, the scores are not 
influenced by the partner with which the cooperatives work, as each partner serves cooperatives 
that had strong and weak results in each tool. Given the above, it is expected that there will be 
disparities between the two tools when they are compared by partner. 
Partner ADA Link
CRS 33.7 55.0
LWR 49.9 50.4
TNS 31.9 45.5
Source: Evaluation team calculations based on Baseline ADA and 
Link Toolkits - PROGRESA Caribe.
87
8. Conclusions and Recommendations
Conclusions
This report addressed the major findings of the PROGRESA-Caribbean baseline study, which are 
summarized below according as follows: a) data collection process, b) participation in the program 
and validation of the experimental design, c) results at the family level, d) results at the level of the 
producer organizations, and e) results of the commercial relations between the cooperative 
enterprises and for-profit companies. 
Data Collection Process
The information collection process in the cooperatives and companies was successful. In the case 
of the cooperatives, participants included the leadership, management, administrative personnel, 
technicians and members.
Field data collection for the families followed the same procedures for both the early and late
treatment groups. In general, the information available is of good quality. Nevertheless, there 
were some difficulties in applying the instrument to the producers due to the distances between 
communities and even between producers in the same community, difficult access to farms, 
problems in the communications networks, difficulties in the use and understanding of the digital 
survey forms by some of the surveyors, and problems in the implementation of the activities. The 
combination of all these factors resulted in high use of the planned replacements (104 of 165) and 
a 22% lack of response. The latter is due to problems in the application of the survey and is not 
correlated with the treatment status. The expected information by chain decreased sharply as 
there was an updating of the value chains in which the producers work. In addition, the amount of 
information in the different sections of the survey differs slightly for all the completed interviews. 
Participation in the Program, Validation of the Experimental Design and Characteristics of Early 
Treatment
The evaluation team has confirmed that, to a great extent, the assignment of program 
beneficiaries was undertaken in a way that was compatible with the planned design. At the time of 
this report, a low percentage of desertion was identified, with a real assignment of 97.3% of the 
selected beneficiaries. The causes of desertion are exogenous to the treatment status; therefore, 
there is no effect on the internal validity of the evaluation. 
A total of 945 completed interviews were obtained for the evaluation sample; of these, 534 
correspond to the early treatment group (56%) and 411 to the late treatment group (44%). 
Overall, the results suggest that the two samples are fairly balanced; in other words, there are no 
statistically significant differences between the characteristics of the early treatment group and 
those of the late treatment group. The evaluation team concludes that the randomization worked 
well at the program level and for TNS and CRS in cattle and LWR in cacao, as well as for some of 
the indicators in the case of women. Therefore, the experimental design generated internal 
validity for the evaluation to be conducted because, on average, the early and late treatment 
groups are exposed to the same series of external factors, with the exception of the time that they 
will be exposed to the program. However, the evaluation team recommends caution in the 
analysis of the response to treatment by the women, as the sample of women is somewhat small, 
88
which may limit the statistical power to identify program impact. In the case of CRS in the cacao 
crop, significant differences were found for some indicators between the early and late treatment 
groups. Therefore, the results for this group should be treated with caution. The program may also 
seek to identify this impact with non-experimental methods. 
At the time of this baseline report, there is no additional information about the true assignment to 
the program in practice, but rather the assignment in the regular CRS monitoring system. 
Therefore, the evaluation team argues that the impact to be identified refers to the average 
treatment effect (ATE). With more information in the mid-term evaluation, it will be possible to 
determine if this will be maintained or if the local average treatment effect (LATE) or the intention 
to treat (ITT) will be estimated. It should be remembered that the results to be identified cannot 
be generalized to other similar populations, contexts or programs (external validity), for the 
reasons mentioned in the evaluation design. 
Results at the Family Level
Some 57.8% of the households that will be benefitted live in conditions of poverty, while 19.9%
face extreme poverty, which suggests that the project has good targeting for poverty conditions. 
The baseline study finds that the families of the beneficiaries are fairly likely to achieve the 
performance objective planned for dietary diversity (7.4) and for months of adequate household 
food provisioning (11.7). 
The average size of the families is 4.8 people per household. The average amount of schooling is 
through fourth grade of primary school for heads of households and completion of primary school 
for sons and daughters over 15 years of age. These data clearly reflect the low educational levels 
of the beneficiaries and their families.
In terms of intermediate indicators, milk yields are identified at 2.59 liters per cow per day. The 
average weight of cattle is 374 kilograms per hectare. Yields for cacao in pulp are 0.77 MT/Ha, 
while yields for dry cacao are 0.26 MT/Ha. 
The levels for sales and gross income varied depending on the chain in which the products are 
commercialized. The cattle ranchers report far more resources than the cacao growers. It is worth 
noting that the producers located in the highest part of the income distribution for cacao have 
similar levels as the beneficiaries that are in the middle part of the income distribution for cattle 
ranching. By sex, women sell much less than men, regardless of the chain analyzed. 
The value of production by hectare is greater for cattle than for cacao. The average gross margins 
are negative in both chains and for all the consortium partners. In both chains, only 30% of the 
beneficiaries have positive margins. Finally, around 4% of the beneficiaries use sustainable 
agriculture and/or cattle ranching practices.
The intermediate and result indicators reflect a great deal of heterogeneity among the consortium 
partners, by chain and by sex. While TNS has the larger scale producers (cattle ranchers), CRS and 
LWR are more similar in their strong targeting of cacao. The results for women are lower than for 
men in most of the indicators. The above suggests that it will not be possible to compare the 
performance of partners or chains with their peers. However, this does not affect the exploration 
of the existing potential for heterogeneous effects in the treatment.
89
Results at the Level of Producer Organizations: Learning Alliance Tool
The cooperative enterprises to be served by the project have the potential to improve their 
management as none of them obtained higher than 70% in its evaluation. The strengths observed 
in most of the cooperative enterprises are in Areas 5 and 6. These areas establish the internal 
organization of the stakeholders involved in the operation of the enterprises, compliance with 
established laws and standards, accountability and transparency, as well as some aspects related 
to the practices of the members with the board of directors and personnel aimed at fulfilling the 
goals and objectives agreed upon in the vision of the organization. 
On the other hand, the major weaknesses are located in Areas 1, 2, 3 and 4. The cooperative 
enterprises do not have a clear and coordinated vision of their medium-term goals; therefore they 
do not have a guide to establish strategies and planned actions that enable them to effectively 
administer their resources. In addition, they have weaknesses related to providing technical and
financial services, such as savings, training on financial administration, etc. There are also 
weaknesses in good management by the personnel responsible for financial management and 
ensuring the commitment of the members to pay for the services provided.
In terms of the performance of the cooperatives measured through the quantitative indicators, it 
is important to emphasize that not all the cooperatives keep records. For example, regarding the 
fulfillment of the target on commercialization, many do not even record income from sales; those 
that do have weaknesses in the way in which they keep these records. Therefore, caution is 
suggested in the interpretation of their performance indicators. This is one aspect on which the 
program should work with the cooperative because the lack of capacity to manage these 
indicators corresponds to the low levels of management achieved. 
Results of the Commercial Relations between Cooperative Enterprises and For-profit Companies:
LINK Tool
In evaluating the degree of inclusivity in commercial relations between the cooperative 
enterprises and for-profit companies, most of the cooperative enterprises (sellers) are situated 
below the overall average, which suggests that there are many weaknesses in the relations with 
their buyers. No significant differences were identified in the performance by production chain, 
implementing partner or linked company; therefore the effectiveness in the relationship between 
buyers and sellers is specific for each case. 
The principle with the best assessment is collaboration among stakeholders; in other words, there 
is good communication and interaction between buyers and sellers. The principle on market 
linkages has a relatively good assessment, which suggest that the cooperative enterprises have a 
certain organizational level to face their production responsibilities. The main problem in the 
principle on measurement of outcomes is the inability of many organizations to measure the 
quality of production, which is the major indicator of interest in improving the profitability of 
commercial relations.
Transparent and consistent governance received a moderate evaluation, mainly due to 
weaknesses in the transmission of information about quality standards and terms of purchase and 
sale, as well as the direct involvement of both parties in the production chain. The second weakest 
90
principle is access to services, which suggests little involvement of the companies in supporting 
improvements in the production of the cooperatives and in the management of the cooperative 
enterprises. Finally, inclusive innovation is the greatest challenge for the cooperatives, particularly 
for the cacao cooperatives as Ritter does not support this activity in all the cooperatives equally, 
rather it depends upon the length of the commercial relationship and the market quota. 
The purchasing companies give a higher assessment to the commercial relations compared to the 
perception of their suppliers. This may suggest two things: 1) the purchasing companies perceive 
their action as positive, but it really does not generate the expected results; or 2) the cooperative 
enterprises do not efficiently take advantage of the relationship with their buyers. This aspect 
should be addressed in detail in the framework of the program through greater interaction by the 
implementing partners with the stakeholders involved in the value chain. 
Finally, there is not an overall direct connection between the LINK and Learning Alliance tools, 
although there is a certain degree of association between some principles and areas. Transparent 
and consistent governance is the principle – in LINK – that has the greatest connection to the 
Learning Alliance instrument. Its relationship is strong for the areas of strategic orientation, 
business management, and structure and functionality. This indicates that improvements in 
governance in commercial relations are positively influenced by better internal management of 
the cooperative enterprises. Another important aspect is that the development of innovation 
processes in conjunction with the buyers facilitates the technical training by the cooperative of its 
members. 
Recommendations
Below are some recommendations on the process to be undertaken for the implementation of 
activities and for the mid-term evaluation:
- Review the targets for some PROGRESA-Caribbean indicators according to the contract with the 
USDA, to ensure a more objective mid-term evaluation. 
- Conduct new rounds of training before the monitoring campaigns to standardize the capacity of 
the trainers and promoters. 
- Strengthen communication between the CRS monitoring and evaluation team and the 
monitoring teams of the consortium partners in order to reduce the percentage of incomplete 
information in the family surveys. The evaluation team suggests a compliance rate of more than 
95% in subsequent follow-up surveys to prevent the loss of greater statistical power. 
- Take into consideration the communications difficulties in the intervention areas during the field 
data collection processes. 
- As the mid-term evaluation will require outside surveyors, begin evaluation activities promptly to 
develop the strategy for accompaniment and training of the outside surveyors in the digital 
system. The survey firm must have experience in field data collection in the Caribbean Coast and 
Río San Juan areas. 
91
- Conduct ongoing monitoring of the beneficiaries, especially in the areas where similar programs 
or projects may intervene, in order to prevent potential bias in the impact evaluation due to 
program desertion. 
- Improve the control of the number of completed interviews. For this, alternative mechanisms to 
the monitoring system must be developed in order to implement that control. For example, the 
evaluation team used the Stata statistical package to identify the complete sample in all the 
sections of the survey. 
- Ensure greater involvement of the members in the evaluation of the cooperative enterprises. In 
practice, the leadership in the organizations is often reduced to the manager and the president 
of the board of directors. 
- Respect the assignment of late treatment; in other words, do not intervene in this group in the 
first 18 months of program implementation, in order to prevent contamination and bias in the 
impact evaluation.
- Ensuring accurate record of the date of commencement of activities of each beneficiary in the 
program in order to apply the model of continuous treatment in the final evaluation. 
92
Project target for outcome and intermediate indicators
Source: Evaluation team calculations based on Baseline Household Survey - PROGRESA Caribe.
Objective/Outcome/ Indicator Baseline
Project 
target 
Progress out of Poverty Index (PPI) (Adapted from FTF Goal 2) % 57.8 4
Months of adequate food supply in the household (MAHFP) 11.1 11.7
Dietary Diversity Score (HDDS) 7.0 7.4
Volume of selected crops produced/harvested per hectare (yield –
MT/ha) (cacao) (FFpr SO1 illustrative)
0.24 1
Average cattle weight per hectare (CRS custom)(kg/ha) 314.30 395
Gross margin per unit of land - cacao (FTF IR 1 (1)) -13.96 25
Gross margin per unit of land - cattle (FTF IR 1 (1)) 15.42 30
Number of hectares managed under sustainable agriculture 
practices - cattle (CRS custom)
2,097 30,816
Increased Use of Financial Services
Number of individuals receiving financial services as a result of 
USDA assistance (female) (FFPr 4)
# 74 482
Number of individuals receiving financial services as a result of 
USDA assistance (male) (FFPr 4)
# 438 1,926
Number of loans disbursed as a result of USDA assistance (FFPr 5 
Standard)
# 512 2,769
Number of hectares managed under sustainable agriculture 
practices - cacao (CRS custom)
226 6,741
Improved Quality of Land and Water Resources
Value of production per hectare - cacao (FFPr SO1 (#3) Illustrative) 609.34 1,262
Value of production per hectare - cattle (FFPr SO1 (#3) Illustrative) 175.53 1,277
FTF Goal: Sustainably Reduce Global Poverty and Hunger
FFPr. SO 1. Increased agricultural productivity
Volume of selected animal products per animal per unit of time
(liters/unit of time)(milk) (FFPr SO1 (#2) illustrative) 
2.11 7
93
Annex
Graphic A. project-level results framework
94
Graphic B. project-level results framework
95
Annex C
Table 1: Cooperatives Served by CRS According to Evaluation Areas in the Learning Alliance 
Survey
Table 2: Cooperatives Served by CRS According to sub-areas of “Financial Services” Area
Strategic 
Orientation
Business 
Management
Technical 
Services
Financial 
Services
Structure and 
Performance
Organizational 
Processes
Escolástico Barrera 0 0 0 0 49.86 43.75
COOMULCOB 6.25 11 0.51 1.39 46.11 32.29
Etanislao Lira 7.29 5 0 0 65.14 29.69
COOMUCOR 17.71 16.25 5.54 0 39.31 19.27
Juan Rodriguez 0 0 17.86 0 41.6 43.75
COOMUNSOL 11.46 10 0 2.78 42.92 32.29
COOMBESPIS 0 1.25 1.02 0 30.56 10.94
COMULVAN 2.08 3.75 1.53 0 32.29 31.25
Nueva Waslala 43.23 56.46 50.9 40.09 67.06 54.34
Miguel Martínez 0 3.75 2.04 0 13.89 13.54
UCM 20.83 33 9.18 0 20.28 9.38
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
Financial skills 
to saving
Planning for 
financial services
Partnership for 
financial access
Financial Services 
Management
Access and 
Coverage
Service 
Satisfaction
Escolástico Barrera 0 0 0 0 0 0
COOMULCOB 0 0 8.33 0 0 0
Etanislao Lira 0 0 0 0 0 0
COOMUCOR 0 0 0 0 0 0
Juan Rodriguez 0 0 0 0 0 0
COOMUNSOL 0 8.33 8.33 0 0 0
COOMBESPIS 0 0 0 0 0 0
COMULVAN 0 0 0 0 0 0
Nueva Waslala 25 55.56 33.33 72.22 27.78 26.67
Miguel Martínez 0 0 0 0 0 0
UCM 0 0 0 0 0 0
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
96
Table 3: Cooperatives Served by TNS According to sub-areas of “Financial Services” Area
Table 4: Cooperatives Served by LWR According to sub-areas of “Financial Services” Area
Table 5: Correlation Coefficients among the Evaluation Areas in the Learning Alliance Survey
Financial skills 
to saving
Planning for 
financial services
Partnership for 
financial access
Financial 
Services 
Access and 
Coverage
Service 
Satisfaction
COOMUSALWI 0 0 0 0 0 0
COMPROMUB 0 0 0 0 0 0
COOPAPROMUDEF 25 0 0 0 0 0
COOPELACTME 0 0 0 0 0 0
COOSAGRO 0 0 0 0 0 0
La Pradera 62.5 50 0 0 0 0
COOPROCAR 0 0 0 0 0 0
UCA Ahmed Campos 25 75 58.33 75 58.33 45
COOSEMUVIS 0 0 0 0 0 0
COOSEMUP 0 0 0 0 0 0
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
Financial skills 
to saving
Planning for 
financial services
Partnership for 
financial access
Financial 
Services 
Access and 
Coverage
Service 
Satisfaction
CACAONICA 0 0 0 0 0 0
ASIHERCA 0 25 0 25 29.17 32.5
COODEPROSA 0 2.78 16.67 27.78 13.89 10
COOPROCAFUC 20.83 36.11 40.28 33.33 13.89 15
COOSEMUCRIM 32.08 30 45.83 30.56 13.89 15
ACAWAS 16.67 16.67 33.33 33.33 40.28 35.83
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
Strategic 
Orientation
Business 
Management
Technical 
Services
Financial 
Services
Structure and 
Performance
Organizacional 
Processes
Strategic Orientation 100
Business Management 96.41 100
Technical Services 81.5 80.93 100
Financial Services 72.84 71.44 57.7 100
Structure and 
Performance 75.79 67.72 62.19 63.85 100
Organizacional 
Processes 74.19 65.16 74.15 57.7 86.22 100
Source: Evaluation team calculations based on Baseline ADA Toolkit - PROGRESA Caribe.
97
References
Aigner, D. J., Amemiya, T., and Poirier, D. J. (1976), On the Estimation of Production Frontiers: 
Maximum Likelihood Estimation of the Parameter of Discontinuous Density Function, International 
Economic Review 17 (2), pp. 377-96.
Bamberger, M. (2012), Introduction to mixed methods in impact evaluation, Impact Evaluation 
Notes No. 3. InterAction and The Rockefeller Foundation.
Belotti, F., Daidone, S., Ilardi, G. and Atella, V. (2012), Stochastic frontier analysis using Stata. The 
Stata Journal, vv, Number ii, pp. 1-39.
Bonbright (2012). Use of Impact Evaluation Results, Impact Evaluation Notes No. 4. InterAction 
and The Rockefeller Foundation.
Botello et al. (2005). Mid-Term Participatory Evaluation or Assessment PITAs implemented, Based 
on Satisfaction of the Plaintiffs.
Caliendo M. and Kopeinig S. (2008). Some Practical Guidance for the Implementation of Propensity 
Score Matching, Journal of Economic Surveys, No 22, pp. 31-72.
Cameron, A. C, Gelbach, J. B. and Miller, D. L. (2008). Bootstrap-based improvements for inference 
with clustered errors.' Review of Economics and Statistics, vol. 90 (3), pp. 414-427 
Cameron, A.C. and Miller, D.L. (2014). A practitoner's guide to cluster robust inference, Journal of 
Human Resources. Forthcoming.
Carter, M. R., Toledo, P. and Tjernström, E. (2012). The Impact of Rural Business Services on the 
Economic Well-being of Small Farmers in Nicaragua. Mimeo.
Catley, Burns, Abebe and Suji (2007). Participatory Impact Evaluation or Assessment. Guide for 
Professinals. Feinstein International Center.
CEPAL (2012). Structural Change for Equalities. An inegrated Vision of Development. United 
Nations, Santiago.
Duflo, E., Glennerster, R. and Kremer, M, (2007), Using Randomization in Development Economics 
Research: A Toolkit, en Handbook of Development Economics, vol. 4.
Fried, H., Lovell, K. and Schmidt, S. (editors) (2008). The Measurement of Productive Efficiency and 
Productivity Growth. Oxford University Press, New York.
Gertler, P. J., Martinez, S., Premand, P., Rawlings, L. B. and Vermeersch, C. M. J. (2011), Impact 
Evaluation in Practice, World Bank Publications, The World Bank, No. 2550.
Gottret, Junkin and Ugarte (2011). Self-assessment provided for the Management of Rural 
Enterprises. CATIE.
98
Guijt, I. (2014). Methodological Briefs Impact Evaluation No. 5. Participatory Approaches. UNICEF. 
Available in: 
http://devinfolive.info/impact_evaluation/ie/img/downloads/Participatory_Approaches_ENG.pdf
Kuhmbhakar S.C. and Knox C. A. (2000). Stochastic Frontier Analysis. Cambridge University Press, 
United Kingdom.
Lundy, Mark. Metodología LINK: A participatory guide to inclusive business models with small 
farmers. / Mark Lundy, Gertjan Becx, Nancy Zamierowski, Alexandra Amrein, Jhon Jairo Hurtado, 
Erika Eliana Mosquera, Fernando Rodríguez. -- Cali, CO: International Center for Tropical 
Agriculture (Centro Internacional de Agricultura Tropical) (CIAT), 2012.
Maluccio, J., and Flores, R. (2004), Impact Evaluation of a Conditional Cash Transfer Program: The 
Nicaraguan Red for Social Protection l, Discussion Paper 184, International Food Policy Research 
Institute, Washington, D.C. 
Meeusen, W., and J. van den Broeck (1977), Efficiency Estimation from Cobb– Douglas Production 
Functions with Composed Error, International Economic Review 18: 2 (Junio), pp. 435-44.
McKenzie, D. (2012). Beyond baseline and follow-up: The case for more T in experiments, Journal 
of Development Economics, Elsevier, vol. 99(2), pp. 210-221.
Winters, P., Salazar, L. and Maffioli, A. (2010), Designing Impact Evaluations for Agricultural 
Projects, SPD Working Papers 1007, Inter-American Development Bank, Office of Strategic 
Planning and Development Effectiveness (SPD).