Village Enterprise
Development Impact
Bond Evaluation
Findings
30 November 2021
Authors
Miguel Angel Jimenez Gallardo:
miguel.jimenez@idinsight.org
Winfred Kananu:
winfred.kananu@idinsight.org
Christy Lazicky:
christy.lazicky@idinsight.org
Jeffery McManus:
jeffery.mcmanus@idinsight.org
Frida Njogu-Ndongwe:
frida.njogu-ndongwe@idinsight.org
About IDinsight
IDinsight uses data and evidence to help leaders combat
poverty worldwide. Our collaborations deploy a large
analytical toolkit to help clients design better policies,
rigorously test what works, and use evidence to implement
effectively at scale. We place special emphasis on using the
right tool for the right question, and tailor
our rigorous methods to the real-world constraints
of decision-makers.
IDinsight works with governments, foundations, NGOs,
multilaterals and businesses across Africa and Asia.
We work in all major sectors including health, education,
agriculture, governance, digital ID, financial access,
and sanitation.
We have offices in Bengaluru, Dakar, Johannesburg, Lusaka,
Manila, Nairobi, New Delhi, San Francisco,
and Washington, DC. Visit www.IDinsight.org
and follow on Twitter @IDinsight to learn more.
Abbreviations
BM – Business Mentor
BG - Business Group
BSG – Business Saving Group
DIB – Development Impact Bond
FCDO – the Foreign, Commonwealth & Development Office
GDI – Global Development Incubator
ITT – Intention to Treat
PPI – Poverty Probability Index
RCT – Randomized Controlled Trial
TOT – Treatment on Treated
USAID-DIV – United States Agency for International Development - Development Innovation Ventures
VE – Village Enterprise
Table of Contents
Table of Contents 3
Executive Summary 7
Key Findings 7
1. Background 8
a. The Village Enterprise Development Impact Bond 8
b. The Village Enterprise Program 8
2. Study Design 11
a. Design Overview 11
b. Main Outcomes 13
c. Data Collection 14
d. Analytical Model 15
3. Results 17
a. Main Results 17
b. Subgroup Results 18
c. Descriptive Analysis by Business Type 24
d. COVID and Livelihoods 28
6. Discussion 31
Limitations of the study 32
Nonresponse and replacement sampling 32
Missing values 33
Outliers 33
COVID-19 impact 33
7. References 34
8. Appendices 35
Appendix A: Outcome payment and impact projection 35
Appendix B: Regression results for main outcomes 37
Appendix C: Self-reported effects of COVID on livelihoods 38
Appendix D: Nonresponse and replacement sampling 41
Appendix E: Missing values and imputation approach 46
Appendix F: Treatment effects on individual components of consumption and assets 50
Appendix G: Differences in treatment effects on types of assets for households that
received larger versus smaller grants 56
Appendix H: Treatment effects by age of household head 59
Appendix I: Average outcomes by business type, country, and cash transfer size 60
Appendix J: Household groupings 61
Appendix K: Cohort-wise effect sizes in Uganda for households surveyed pre-lockdown
vs post-lockdown 62
Appendix L: Treatment effects in this study versus Sedlmayr et al (2020) 63
Appendix M: Evaluation team 64
Appendix N: Evaluation Discussion 66
4
List of figures
Figure 1 - Village Enterprise Programme Timeline
Figure 2 - Village Enterprise Theory of Change
Figure 3 - Village Enterprise cohort timelines
Figure 4 - Effects of Village Enterprise program on consumption
Figure 5 - Effects of Village Enterprise program on net assets
Figure 6 - Grant-size effects
Figure 7 - Cohort effects
Figure 8 - Cohort effects by country
Figure 9 - Treatment effects by gender
Figure 10 - Treatment effects by baseline wealth level, by country
Figure 11 - Treatment effects by disability
Figure 12 - Outcome means by business type
Figure 13 - Outcome means by business health
Figure 14 - Outcome means by business owner gender
Figure 15 - Outcome means by business owner disability status
Figure 16 - Reported impact of COVID-19 on HH livelihoods
Figure H1: Treatment effects by age of household head
Figure I1: Average outcomes by business type & country
Figure I2: Average outcomes by business type & cash transfer size (Kenya only)
5
List of tables
Table 1: Balance checks on baseline characteristics (N=9,888)
Table 2: Correlation coefficients between business health indicators and outcomes
Table B1: Treatment effects of main outcomes
Table D1: Balance across treatment and control groups for nonrespondents
Table D2. Balance across treatment and control groups for nonrespondents (Kenya)
Table D3. Balance across treatment and control groups for nonrespondents (Uganda)
Table D4: Balance across treatment and control groups for nonrespondents and replacements
Table D5. Balance across treatment and control groups for nonrespondents and replacements (Kenya)
Table D6. Balance across treatment and control groups for nonrespondents (Uganda)
Table F1: Treatment effects on consumption items in Kenya
Table F2: Treatment effects on consumption items in Uganda
Table F3: Treatment effects on asset items
Table G1: Treatment effects on asset items, small-grant versus large-grant households, Kenya only
Table J1: Household groupings
Table K1: Cohort-wise effect sizes, Uganda only
Table L1: Comparison of this evaluation’s vs Sedlmayr et al study outcomes
6
Acknowledgements
This evaluation was possible thanks to the contribution of many teammates and project stakeholders who
were integral to the evaluation’s successful implementation and always providing valuable insights. The
authors are grateful to many IDinsight team members who supported the evaluation over the past four
years, including: Sarah Carson, Nisha Hariharan, Michael Otieno, Cassandre Pignon, Penny Davis, Gabrielle
Posner, Mercy Cheptoo, Hannah Blackburn, Lerato Rametse, Patricia Pena, Maureen Stickel, and the entire
team of enumerators in Kenya and Uganda. Special thanks to the Laterite team for excellent support and
flexibility during data collection in Uganda.
The program stakeholders were also critical to the evaluation’s success: Village Enterprise facilitated access
to important information and data needed for the evaluation. They also supported the team when we were
seeking local approvals for fieldwork. The other stakeholders: Instiglio; the United States Agency for
International Development-Development Innovation Ventures (USAID-DIV); the Foreign, Commonwealth &
Development Office (FCDO); and Global Development Incubator (GDI) provided key support and feedback
throughout the exercise.
Finally, we thank all of the respondents who took part in the study, responding to surveys and sharing their
opinions and thoughts which were valuable in generating evidence for this impact evaluation.
7
Executive Summary
This report describes the results of the evaluation of the Village Enterprise poverty graduation program,
conducted in Kenya and Uganda between November 2017 and December 2020. The objective of this
evaluation was to measure the causal impact of the program on household consumption and assets. The
Village Enterprise program was randomized across 241 treatment villages and 241 control villages in
western Kenya and eastern Uganda and rolled out to cohorts of treatment households from January
2018 to December 2020. We surveyed 9,888 households from all study villages from May to August
2021, which was 6 months to 2.5 years after treatment households had completed the one year
program. All analyses were pre-specified and registered on the American Economic Association’s registry
for randomized controlled trials.
1 The findings from this evaluation determine the final payments of the
Village Enterprise Development Impact Bond (DIB) from the Trustee of the Outcome Payers, Global
Development Incubator, to the Service Provider, Village Enterprise, for the achieved results.
Key Findings
● Consumption: The Village Enterprise program had a positive and statistically significant impact
on monthly consumption for households that were offered the program. On average, treatment
households consumed 9.9 USD (or 6.3%) more per month than the control group. Effect sizes
were larger in Kenya (15.2 USD or 7.3%) than in Uganda (3.9 USD or 3.6%).
● Assets: The Village Enterprise program also had a positive and statistically significant impact on
household net assets after the end of the intervention. On average, households in the treatment
group have USD 40.5 (or 5.8%) more in net assets than those in the control group. Effect sizes
were larger in Kenya (60.9 USD or 8.5%) than in Uganda (15.6 USD or 2.3%).
● Larger versus smaller grants: In Kenya only, some households received larger grants (150
USD/household) while others received smaller grants (50 USD/household). Households that
received larger cash grants did not have significantly different treatment effects for consumption
than households that received smaller grants, though they did have significantly larger
treatment effects for assets (87.7 USD for larger grants vs 5.9 USD for small grants).
● Household wealth: Treatment effects on consumption were generally larger for households that
had more wealth prior to the introduction of the program. Treatment effects on assets are not
strongly correlated with baseline wealth.
● Other subgroups: There were no clear patterns of differential program effects for other
pre-specified subgroups (cohort of treatment household, gender of household head, household
member with disability).
● COVID-19 impacts on well-being: 87% of respondents reported that the COVID-19 pandemic
has had an effect on their economic well-being. Households that reported being negatively
affected by COVID-19 achieved similar treatment effects as other households.
Based on the evaluation results, we find that the DIB Outcome Payment Type 2 (P2
) = $2,493,964.36,
such that Village Enterprise maxes out the total DIB payment of $4,280,618.
1 https://www.socialscienceregistry.org/trials/7168
8
1. Background
a. The Village Enterprise Development Impact Bond
Village Enterprise pursues a mission of ending extreme poverty in rural Africa through entrepreneurship
and innovation. The United States Agency for International Development-Development Innovation
Ventures (USAID-DIV); the Foreign, Commonwealth & Development Office (FCDO); Instiglio; Global
Development Incubator (GDI); an anonymous funder; and Village Enterprise have designed a
development impact bond (DIB) to enable Village Enterprise finance their graduation model intervention
and focus on achieving measurable results. The DIB model harnesses private risk capital so that outcome
payers, such as donor agencies or foundations, pay only for the achievement of agreed-upon outcomes.
Village Enterprise launched this first DIB in Sub-Saharan Africa in 2017, with a maximum outcome
payment of $4,280,618. IDinsight was the independent, third-party evaluator of the DIB. We conducted
a four-year randomized controlled trial (RCT) to measure the causal impact of the Village Enterprise
program financed through the DIB on household consumption and assets. In this report, we report the
results of the RCT and we use these results to determine the payments from the Trustee of the Outcome
Payers to the Service Provider, Village Enterprise, for the achieved results.
b. The Village Enterprise Program
Village Enterprise is a non-profit that administers a micro-enterprise development program in East
Africa. The goal of the Village Enterprise graduation program is to lift the poorest East Africans out of
extreme poverty. Village Enterprise’s program runs similar to an ultra-poor graduation model, as it
targets the poorest members of the communities they operate in and provides them with an integrated
package of cash transfers and training. The program places a large emphasis on business skills and
microenterprise development. Prior evidence suggests that this integrated package, compared to cash
transfers alone, is critical to enabling ultra-poor households to generate enough income to break out of
poverty (Banerjee et al., 2018).
The Village Enterprise program lasts for approximately one year, as illustrated below in Figure 1. Village
Enterprise helps participating individuals form business groups (BGs) of two to three entrepreneurs and
provides an initial cash transfer as seed capital. Program participants receive the first grant ($100 per
BG, or ~$33 per household) after successfully completing the 3 months training and a second grant ($50
per BG, or ~$17 per household) is delivered 6 months after the first grant.
2 Some BGs in Kenya received
a larger grant size ($300 for the first grant and $150 for the second) to assess the impact of the grant size
on outcomes. These cash transfers are accompanied by training and ongoing mentoring by a local
business mentor (BM). Entrepreneurs are also organized into business savings groups (BSGs) of 30
entrepreneurs to allow access to growth capital, provide a safe space for savings, and build social capital.
2
If a BG only included two entrepreneurs, then it received a grant amount equal to ⅔ of the grant amount received by
3-member BGs in the same village, so that the amount received by each household is consistent across 2-member and
3-member BGs.
9
Figure 1. Village Enterprise Programme Timeline
Source: Village Enterprise
Figure 2 shows a simplified Theory of Change for how Village Enterprise envisions the program
improving household well-being and standards of living.
Figure 2. Village Enterprise Theory of Change
Source: Village Enterprise
The program funded by the Village Enterprise DIB was implemented in 241 villages across western Kenya
and eastern Uganda from 2017 to 2020. A total of 14,772 ultra-poor households were offered to
participate in the program.
3 These households were divided into seven cohorts, which were offered the
program at different times, as detailed in Figure 3. Out of those offered the program, 14,130
households, or 95.7% received at least the first cash grant, which is how we define ‘participation’ for
the purposes of estimating the treatment-on-the-treated effect. A total of 13,839 households received
both the first and second grants.
3 This number includes the original sixty households per treatment village selected to participate in the program, as well as any
alternate households who were offered the program due to initial households refusing to participate. See appendix J for the
breakdown of original vs alternate households offered the program.
10
Figure 3: Village Enterprise Cohort Timelines
4
IDinsight independently verified the number and amount of grants disbursed to each cohort. Verification
involved several processes, including IDinsight enumerators witnessing disbursements, financial audits
of Village Enterprise’s bank statements, cash retirement documents, M-Pesa
5
statements, and phone
surveys with randomly sampled program participants. We successfully verified grant disbursement in all
treatment villages. Some aspects of implementation of the program were slightly delayed for Cohort
6-Jul2020
6 and Cohort 7-Dec2020 due to COVID-19.
7
The evaluation was originally intended to include two endlines: one for Cohort 1-Dec2018 through
Cohort 4-Dec2019 in April/May 2020 and one for Cohort 5-Apr2020 through Cohort 7-Dec2020 in
April/May 2021. This timeline would have ensured that data was collected within 1.5 years of the
conclusion of the program for all cohorts. However, due to the COVID-19 pandemic, the two rounds of
data collection were combined into a single endline conducted from May to August 2021. As a result,
the outcomes for Cohort 1-Dec2018 through Cohort 4-Dec2019 were measured more than a year later
than planned. The results for these early cohorts should therefore be interpreted more in light of the
sustainability of the program’s impact over time (within the context of a global health and economic
shock), rather than reflecting the short-to-medium term impact of the program as initially intended.
7The second cash disbursement for Cohort 6-Jul2020 in Uganda and the first cash disbursement for Cohort 7-Dec2020 in Kenya
were delayed for about a month due to the implementation of new procedures during COVID-19. Business mentoring was also
delayed and had to be extended until the end of December 2020 for Cohort 5-Apr2020 through Cohort 7-Dec2020 to ensure
they receive in-person mentoring after the lockdown was lifted.
6 Cohorts in this report are referenced as [cohort number]-[MonthYear when the cohort finished the program].
5 M-Pesa is a mobile phone-based money transfer service that allows users to store and transfer money through their mobile
phones
4 This represents households that were offered the program (ITT), which is more expansive than the population who actually
received the first transfer. ITT analysis includes every household that was randomized, maintaining the balance generated from
the original randomization assignment. This analysis is slightly more conservative compared to treatment-on-treated analysis
(TOT), which is affected by noncompliance and withdrawal from the program after randomization. In this evaluation,
noncompliance was low but non-zero (95.7% of households that were offered the program opted to participate and receive
the first grant), and thus ITT impact estimates are slightly lower than TOT impact estimates.
11
2. Study Design
a. Design Overview
The main objective of this evaluation was to determine the impact of Village Enterprise’s program on
household consumption and assets. The results from this evaluation were used to determine the
appropriate outcome payments for the DIB. A secondary goal for this study was to contribute to the
body of knowledge on graduation-style interventions and to serve as an example for similar
interventions in the future.
This study aimed at answering the following research questions:
● Primary question: What was the impact of the Village Enterprise program on household
consumption and asset value, relative to control households? (Intention-to-treat effect)
● Secondary questions:
○ What was the impact on household consumption and assets for those households that
received at least the first business grant? (Treatment-on-the-treated effect)
○ Were there differences in the impact on household consumption and asset value, based
on: cash transfer size; cohort; country; gender of the household head; the presence of a
household member with a disability; baseline wealth levels?
○ How does endline consumption and asset vary by: business type; business group size;
business health;
8 participant gender; participant disability?
These secondary questions gave additional important insights on the impact of the Village Enterprise
program along these different dimensions.
To assess the impact of the program, we designed and conducted a cluster randomized controlled trial
(RCT) in Trans-Nzoia and Bungoma counties of Kenya, and Soroti, Amuria and Dokolo districts of Uganda.
For all major research questions outlined, the village is the unit of treatment and the household is the
unit of analysis.
Sampling and randomization proceeded as follows: first, Village Enterprise produced a list of eligible
villages in study areas, which included information on the number of households, trading centers, cattle
dips, and water sources used. IDinsight analyzed the village data to produce a final list of villages that
met the minimum requirements for inclusion in the study: had at least 70 households, and either did not
share a trading center or water source (Kenya) or did not share a water source or cattle dip (Uganda).
9
Village Enterprise then allocated study villages to cohorts and, within cohorts, to Business Mentors
(BM). To identify eligible households within selected study villages, Village Enterprise conducted a
targeting exercise, which consisted of a participatory wealth ranking exercise conducted with local
leaders. All households ranked as ‘poor’ or ‘very poor’ in this exercise were administered a Poverty
9 We ensured that study villages did not share a trading center, water source, or cattle dip to limit spillover effects. Selection of
eligible study villages was conducted twice: first for Cohort 1-Dec2018 through Cohort 4-Dec2019, and later for Cohort
5-Apr2020 through Cohort 7-Dec2020.
8 Business health variables include business value, profit, record keeping, number of active members, and the proportion of
grant funds invested in the business.
12
Probability Index (PPI)
10
survey by Village Enterprise enumerators.
11 Villages that did not have 70
qualified households were determined ineligible for the study.
Next, within each pair of villages under a BM, IDinsight randomly assigned one village to the treatment
group and the other village to the control group.
12
In addition to improving statistical power,
stratification at the BM-level ensured that each BM had the same number of treated villages on the
same programmatic timeline and that BMs did not need to be reallocated after the targeting process
and randomization. Households in the control villages received no treatment.
Within treatment villages, IDinsight randomly selected 60 of the 70 poorest households to be invited to
the program. The remaining ten households were designated as alternates in case any of the original 60
households declined to participate. Offers to alternates were made in the order of IDinsight’s
randomized ranking.
13 The original 60 households and any alternates invited to the program were
included in the study sample frame. In control villages, all 70 of the poorest eligible households were
included in the study sample frame. In total, our final sampling frame consisted of 31,642 households.
Of these, we randomly selected 21 households per village to participate in our endline survey, for a total
of 10,122 households. We successfully collected data from 4,935 households in control villages and
4,953 households in treatment villages, for a total of 9,888 households.
14 Table 1 below provides
descriptive statistics for baseline variables across treatment groups for the households for which data
was collected.
Table 1: Balance checks on baseline characteristics (N=9,888)
Variable
15 Treatment Control Difference P-value
PPI score 25.600 25.348 0.253 0.237
Household size 6.492 6.505 -0.013 0.718
Household head age 48.955 49.585 -0.630 0.004
Number of cows 0.828 0.882 -0.053 0.083
Number of bodas
16 0.045 0.035 0.009 0.002
Owns a radio 0.327 0.308 0.019 0.007
All have shoes 0.096 0.085 0.011 0.033
Has iron roof 0.559 0.555 0.004 0.653
Number of beds 0.885 0.891 -0.006 0.720
Number of pigs 0.176 0.190 -0.014 0.188
Number of bikes 0.377 0.386 -0.009 0.338
16 This is the number of motorcycles a household reported to have during baseline
15 These variables represent household data collected during baseline
14During data collection, 16% of sampled households were unavailable for the endline survey despite field team efforts to track
them. Out of this number, we replaced 14% with randomly-selected alternates from the list of eligible households. We
describe our protocol for replacement sampling and implications for estimation in Appendix D.
13 Alternate households were offered the program in 80 of the 241 treatment villages. In five villages all ten alternates were
offered the program.
12 Village Enterprise’s PWR exercise and IDinsight’s subsequent randomization were conducted separately for each cohort.
11 VE enumerators could also administer the PPI survey to additional households, at their discretion.
10 www.povertyindex.org
13
Has frying pan 0.006 0.006 0.000 0.767
Number of jerry cans 3.118 3.169 -0.050 0.160
Has phone 0.681 0.674 0.007 0.354
Number of school years of
household head
17 6.358 6.232 0.126 0.048
Missing school years of
household head
18 0.064 0.067 -0.003 0.371
Although we observe imbalance in some baseline variables, the differences are small in magnitude and
not in a consistent direction.
19 Therefore, we find no material concerns with the randomization process.
In keeping with our pre-analysis plan, we include these variables as controls in our analytical model.
20
b. Main Outcomes
This evaluation estimated the effects of treatment on two primary outcomes of well-being: household
consumption and assets.
Household monthly consumption, measured at endline included the sum of
21
:
1. Total food, beverage, and temptation goods expenditure over the preceding seven days
including food prepared at home, purchased outside, and given in-kind.
2. Total recurrent expenditure, such as fuel and transportation expenditures, utilities, personal
hygiene and health over the preceding four weeks.
3. Total infrequent expenditure on larger social and religious activities (e.g., wedding, funeral),
clothing, taxes, housing maintenance, migrations, travel and educational costs and fees over
the last 12 months.
Household net asset ownership, measured at endline included the sum of:
1. Total durable assets
2. Total home improvements
3. Total productive assets
4. Total household savings, including savings generated through the Village Enterprise business
savings groups
22
5. Total business assets, accounting for business ownership share by the household
22 Survey instrument aimed to make sure that only the household’s share of business assets and savings were captured rather
than the full business group assets in order to provide an accurate measure of asset ownership.
21 Consumption measured over different recall periods were converted to monthly amounts. Note, the sum of these
components was approximated through a subset approach. See Appendix E for details.
20 With the exception of PPI score, since its component parts were included in the lasso covariate selection model.
19 For instance, treatment households are slightly more likely to have a boda, radio, and shoes, while control households are
slightly more likely to have cows, pigs, and jerry cans. We also note that differences with similar magnitudes and statistical
significance levels were found in many of the same variables among all eligible households at baseline (bodas, radios, shoes,
jerry cans, and years of schooling were significant at the 5% level for eligible households), so these slight imbalances are an
artifact of the randomization process rather than the household sampling process.
18 This variables represents missing number of years of school for household head at baseline
17 Number of school years of the household head is missing for 650 observations, and so the values in this row come from
9,238 non-missing observations.
14
6. Total household and share of business liabilities (subtracted from 1 – 5 above to arrive at total
net ownership).
Owing to the ongoing COVID-19 pandemic during the data collection period, the length of the survey
was reduced from 2.5 hours to 1 hour to prevent prolonged interactions and reduce exposure between
enumerators and respondents. This involved eliminating items from several sections of the survey. For
the consumption modules, we identified the most commonly purchased goods from recent
representative consumption and expenditure surveys in each country that accounted for 85% of total
consumption.
23 This process resulted in cutting 222 items down to 41 in Kenya and 144 items down to
31 in Uganda. During analysis, we multiplied the consumption value in each country by approximately
(1/0.85) to estimate total consumption.
24 Prior studies have shown that shortening consumption
modules in this way and rescaling aggregate values produces accurate measurements of total
consumption.
25 Therefore, we believe that the estimated effect sizes are reasonable given previous
work.
For the household asset module, we cross-checked our initial list of 33 assets with prior studies
26 and
identified the 15 most commonly measured assets, which accounted for 95% of asset ownership in the
previous RCT of Village Enterprise’s program in Uganda (Sedlmayr et al. 2020).
27 For the business asset
module, we identified the 10 most common categories of assets used by businesses in Village
Enterprise’s business asset database, accounting for 95% of business asset ownership.
28 We further
shortened the survey by eliminating the full household roster and the disability module.
29
c. Data Collection
To measure the causal impact of the Village Enterprise program on household consumption and assets,
IDinsight conducted one round of data collection across all 482 study villages in Kenya and Uganda
between May and August 2021.
30 Depending on each treatment village’s cohort, data collection
occurred 6 months to 2.5 years after the conclusion of the Village Enterprise program. Data was
collected through enumerator-administered in-person surveys on tablets using SurveyCTO. Despite
government restrictions and various challenges presented by COVID-19 during the course of data
collection, we surveyed 98% of our target sample.
31
31 See details on our replacement sampling protocol in Appendix D.
30 Data collection occurred in Kenya from June 2 to July 29, 2021. Data collection started in Uganda on May 6, 2021 and
continued until government-imposed lockdowns went into effect on June 18. Data collection resumed in Uganda on August 9,
2021, and continued until all villages were completed on August 31. A first endline was initially planned for April and May 2020
but this was canceled due to the COVID-19 pandemic.
29
In the subgroup analysis below, we use data collected by Village Enterprise during the household targeting exercise to
estimate treatment effects by whether a household included a member with a disability at baseline.
28
ibid. Note that we administered the business asset module separately for each household business.
27 Since we also included an ‘other’ asset category to capture any large household assets not included in this list, we did not
rescale asset measurements. Note that household savings and loans were captured separately, and no savings or loans survey
items were cut from the final survey.
26
Including Banerjee et al (2015), Dupas et al (2018), Haushofer & Shapiro (2016), and Sedlmayr et al (2020).
25 See for instance Beegle et al (2012) and Natali & de Neubourg (2014).
24
In Kenya we estimated that the consumption module would include 85.06% of total consumption, so we multiplied
consumption values by 1/0.8506 = 1.176. In Uganda we estimated that the consumption module would include 85.18%. of
total consumption, so we multiplied consumption values by 1/0.8518 = 1.174.
23
In Kenya we used the Kenya Integrated Household Budget 2015-16 dataset
(https://sun-connect-news.org/fileadmin/DATEIEN/Dateien/New/KNBS_-_Basic_Report.pdf) and in Uganda we used the
Uganda National Budget Survey 2015-16 (https://microdata.worldbank.org/index.php/catalog/3460).
15
To ensure data quality, IDinsight conducted audio audits, back-checks, supervisor spot-checks, and daily
checks for outliers on key variables, incomplete surveys, and other anomalies. Surveys that were flagged
for possible anomalies were conducted again.
d. Analytical Model
The causal effect of the Village Enterprise program on assets and consumption was estimated using the
following Ordinary Least Squares model:
𝑌
𝑖𝑗 = β
0 + β
1
𝑇
𝑗 + 𝑋
𝑖𝑗
'
γ + α𝑚
'
δ + ε
𝑖𝑗
Where:
● Yij denotes the outcome variable (either monthly consumption or net assets) for household i in
village j.
● Tj denotes the treatment status of village j (1 for Treatment Group; 0 for Control Group).
● β1
is the estimated treatment effect of the Village Enterprise program that is used to calculate
DIB payments.
● Xij
is a vector of controls for household i in village j as measured during the baseline targeting
exercise.
32
● α’m denotes a vector of dummy variables corresponding to the Business Mentors-village pairs in
which we randomized, which is 1 when household i is allocated to Business Mentor-village pair
m and 0 otherwise.
● εij denotes the household error term i, clustered at the village-level j to reflect the randomization
process.
We pooled data for all seven cohorts. The impact of the program was measured using an
intention-to-treat (ITT) analysis in which we assess the impact of the program on all households that
were offered the Village Enterprise program. We also estimated the treatment effect on the treated
(ToT), which is the impact for households that actually took up the program, by using treatment
assignment as the instrumental variable for receiving the treatment.
33
We also conducted analyses for key subgroups including: whether households received regular grant
sizes ($50/household) or increased grant sizes ($150/household); country; cohort; gender of the head of
the household; household baseline wealth level (as proxied by PPI scores); and whether a household
member has a disability (as measured during Village Enterprise’s household targeting). These analyses
sought to address the secondary questions outlined above with the aim of understanding the effects of
the program in these dimensions.
33 Treatment was defined as households that at least received the first small business grant.
32 The control variables were pre-specified in the PAP and include: # of people living in HH, age of HH head, # of cows/cattle the
HH owns, # of bodas/motorcycles the HH owns, whether the HH owns a radio, whether every HH member has a pair of shoes,
whether the HH has an iron roof, # of beds the HH owns, # of pigs the HH owns, # of bicycles the HH owns, whether the HH
owns a frying pan, # of jerry cans the HH owns, whether the HH owns a mobile phone, and # of years of school for the HH
head. Note that the variables for radios, frying pans, and mobile phones were initially intended to capture the number of each
item; however, given the data collected by Village Enterprise during baseline targeting, these were updated to whether the HH
owned at least one of each item. We also pre-specified that if any households were missing covariate information, we would
replace the missing observation values with zero and add a missing indicator for that covariate. In the final dataset the only
covariate with missing data was the number of years of school for the HH head, which was missing for 7% of observations.
Accordingly, we created a dummy variable for missing values in this variable, and replaced missing values in the original
variable to zero.
16
During the course of data collection, Kenya was experiencing the third wave of COVID-19 while Uganda
was in the second wave. To better understand the impacts of COVID in our study population, IDinsight
included questions in our survey about whether and how the pandemic had an effect on households’
economic well-being.
34
34 We asked respondents whether their household's livelihood had been in any way impacted by the ongoing COVID-19
pandemic; and if so, if the impact was mostly positive or mostly negative; and finally requested them to briefly explain their
responses (the full report with the findings from this analysis is on Appendix C).
17
3. Results
a. Main Results
The Village Enterprise program had a positive and statistically significant impact on household monthly
consumption. Figure 4 below shows how consumption compares between treatment and control
villages in both countries, as well as for each country where the program was rolled out.
35
Figure 4: Effects of Village Enterprise program on consumption
On average, those households that were offered the program consumed 9.9 USD (or 6.3%) more per
month than those in the control group across both countries. The program effect in Kenya (15.2 USD or
7.3%) was larger than the effect in Uganda (3.9 USD or 3.6%).
The Village Enterprise program also had a positive and statistically significant impact on household net
assets. Figure 5 shows the effect size on net assets value for those households that were offered the
program.
35
In all figures that show control & treatment values, the control value is the average value of that outcome in the control
group, whereas the treatment value is the control average plus the treatment effect estimate from the regression model
described in the Analytical Model section.
18
Figure 5: Effects of Village Enterprise program on net assets
On average, households in the treatment group had USD 40.5 (or 5.8%) more in net assets than those in
the control group across both countries. Similar to what we observed for consumption, the program
effect for households in Kenya (60.9 USD or 8.5%) was larger than the effect for households in Uganda
(15.6 USD or 2.3%).
In Appendix F we report treatment effects for individual components of consumption and net assets,
while correcting for multiple hypothesis tests by following the procedure described in Benjamini et al
(2006) and implemented in Anderson (2008).
Since 95.7% of households that were offered to participate in the program accepted the invitation and
received at least one grant, the treatment-on-the-treated (ToT) results closely mirror the ITT estimates.
The ToT estimate for consumption is 10.3 USD (p < 0.01) and for assets is 42.2 USD (p < 0.01) across both
countries.
b. Subgroup Results
Cash transfer size
The Village Enterprise program implemented two different cash transfer sizes in Kenya.
36 Some of the
BGs in Kenya received $100 during the first disbursement while others received $300.
37 For the second
grant, the BGs that received a smaller grant in the first round received $50, while the BGs that received
37 Treatment villages were non-randomly selected by Village Enterprise to receive small or large grants. Thus any difference in
outcomes or treatment effects between villages that received small or large grants is suggestive but should not be interpreted
as causal evidence of the impact of larger grants. Differences between control villages in village pairs where the treatment
village received large grants versus village pairs where the treatment village received small grants reflect differences in the
underlying characteristics of those types of villages. All BGs within a village received the same cash transfer sizes, unless a BG
had only two members instead of three, in which case it received ⅔ of the grant amount disbursed to 3-member BGs in the
same village.
36
In Uganda, all BGs received the same cash transfer sizes, unless the BG had only two members instead of three, in which
case it received ⅔ of the grant amount disbursed to 3-member BGs.
19
the larger grant in the first round received $150. Figure 6 below shows differences in the impact based
on the cash transfer size.
Figure 6: Grant-size effects
38
There was no statistically significant difference in treatment effects on consumption for households that
received larger vs smaller grants. However, households that received larger grants had on average 87.7
USD more in net assets than households that received smaller grants (p = 0.02). Appendix G provides
details on how the treatment effects for specific asset components differ for households that received
large versus small grants, and which components are driving the difference in the aggregate index.
Cohort of household
As shown in Figure 3, cohorts received cash grants and participated in the Village Enterprise program at
different points in time. As of endline in May-August 2021, treatment households in Cohort 1-Dec2018
had completed the Village Enterprise program 2.5 years prior, whereas treatment households in Cohort
7-Dec2020 had completed the Village Enterprise program only 6 months prior. If the impact of the
program grows or diminishes over time, we may find different treatment effects for different cohorts. If
the COVID-19 pandemic affected the impact of the Village Enterprise program, we may find different
treatment effects for cohorts that completed the program prior to pandemic, by December 2019,
(Cohort 1-Dec2018 through Cohort 4-Dec2019) compared to those that completed the program during
the pandemic in 2020 (Cohort 5-Apr2020 through Cohort 7-Dec2020). Figure 7 shows treatment effects
(with 95% confidence intervals) for each cohort.
38 The difference-in-differences or “DiD” effect reported in these graphs refers to the difference in treatment effects between
the group on the left and the group on the right. A significant DiD effect indicates that treatment effects are different across
subgroups.
20
Figure 7: Cohort effects
Surprisingly, given the differences in program timing across cohorts, there are no significant differences
in treatment effects on household consumption across cohorts, except for Cohort 4-Dec2019. The
treatment coefficient on consumption for Cohort 4-Dec2019 is significantly smaller than the coefficient
for the other six cohorts (p < 0.01), and is statistically indistinguishable from zero. This may be driven in
part by timing: Cohort 4-Dec2019 completed the program shortly before the COVID-19 pandemic and
ensuing lockdowns, which may have given inadequate time for new businesses to become established.
There is slightly more variation in treatment effects on net assets across cohorts. Effects for Cohort
2-Apr2019, Cohort 3-July2019, Cohort 5-Apr2020, and Cohort 7-Dec2020 are statistically
indistinguishable from each other or from the average effect across all cohorts. The treatment effect for
Cohort 6-Jul2020 is slightly greater than the cross-cohort average, though the difference is marginally
significant (p = 0.07). The treatment effect on net assets for Cohort 4-Dec2019 is significantly smaller
than the coefficient for the other six cohorts (p = 0.01) and is statistically indistinguishable from zero.
The treatment effect on net assets for Cohort 1-Dec2018 is also significantly smaller than the coefficient
for the other six cohorts (p < 0.01). Surprisingly, the coefficient for Cohort 1-Dec2018 is negative and
marginally significant (p = 0.07).
In Figure 8 we show cohort-wise results disaggregated by countries.
21
Figure 8: Cohort effects by country
An interesting pattern emerges in consumption effects: whereas treatment effects are greater for the
last three cohorts than the first three cohorts in Kenya, the reverse is true in Uganda. We posited that
the pattern of consumption treatment effects in Uganda could be driven by interactions between the
data collection schedule and the timing of the lockdown, where households in later cohorts tended to
be surveyed more after the lockdown was lifted than before the lockdown was put into place. However,
the evidence to support this hypothesis is mixed. In Appendix K we show cohort-wise effects in Uganda
for households surveyed pre-lockdown vs post-lockdown. While we do see that households surveyed
post-lockdown in Cohort 6-Jul2020 and Cohort 7-Dec2020 had lower effect sizes than households in the
same cohorts surveyed pre-lockdown, which partially explains the lower effect sizes for those cohorts in
Uganda, we do not see a consistent pattern in effect sizes across cohorts (e.g. effect sizes are actually
larger for household surveyed post-lockdown in Cohort 1-Dec2018, Cohort 4-Dec2019, and Cohort
5-Apr2020).
In both countries, the consumption effect is smallest for Cohort 4-Dec2019 and statistically
indistinguishable from zero. There are less clear patterns in asset effects across cohorts, though we see
that the negative coefficient on assets for Cohort 1-Dec2018 is driven by Kenya.
Gender of the head of household
Figure 9 below shows differences in program impact based on the gender of the household head as
reported by respondents during the survey. Within the study sample, 29.5% of households have a
female household head.
22
Figure 9: Treatment effects by gender
We find no statistically significant differences in treatment effects for households headed by men versus
women. However, female-headed households in both treatment and control groups have lower
consumption and fewer assets.
HH baseline wealth level (as proxied by PPI scores)
We analyzed the effect of the program on households for each wealth quintile, proxied by PPI scores
determined by Village Enterprise during the targeting exercise. We divided households into wealth
quintiles, separately for each country, and estimated treatment effects for each country-quintile. Figure
10 below illustrates the effects on consumption and net assets. We perform this analysis by country
since wealth levels differ substantially between the Kenya sample (wealthier) and the Uganda sample,
and since PPI scores and national poverty lines are country-specific.
23
Figure 10: Treatment effects by baseline wealth level, by country
Treatment effects on consumption are generally greater for households that had more wealth at
baseline. The one exception is that households in the middle quintile in Kenya had the highest treatment
effect for any quintile in either country. There is less of a clear pattern for assets. We observe the largest
treatment effects on assets for the 2nd and 3rd quintiles in Kenya and the 4th quintile in Uganda. Effects
are smallest for the 1st quintile in Kenya and the 5th quintile in Uganda.
Presence of a household member with a disability
39
Figure 11 below shows the differences in impact between households that reported having a member
with a disability compared to those that did not have any reported cases of disability. Within the study
sample, 47.4% of households report having at least one member with a disability.
39 Village Enterprise administered the Washington Group Short Set of Questions to households at baseline, which defines a
disability as having a lot of difficulty with the following abilities: eyesight, hearing, language, memory, mobility, and self-care.
24
Figure 11: Treatment effects by disability
We find no statistically significant differences in treatment effects for households that had a member
with a disability versus those that did not.
c. Descriptive Analysis by Business Type
In this section we report descriptive statistics for treatment households belonging to different types of
businesses (control households and villages are excluded from the analysis in this section). Range bars in
all graphs in this section denote 95% confidence intervals.
Outcomes by business type
In the first and second grant application, households reported the type of business in which they were
involved. Figure 12 shows average outcomes for treatment households according to the business type
that they reported in their second grant application.
40
In the second grant application, 60% of
households reported running a retail business, 21% crops, 8% livestock, 3% service, 2% skilled labor, and
5% reported running a business that fell into multiple categories.
41
41 2% of BGs did not have a business type listed in the second grant application. The percentages above correspond to
the 98% of BGs who reported a business type.
40 We report business type according to the second grant application since we believe that this would correspond more closely
to the business type at endline than what BGs reported in the first grant application.
25
Figure 12: Outcome means by business type
Average consumption is similar across business types, though households with a crop business
consumed less, and households with a business that falls into multiple categories consumed more.
Households in the livestock business and with a business across multiple categories reported higher
average net assets.
In Appendix I we show average outcomes by business type disaggregated by country and, within Kenya,
by cash transfer size.
Outcomes by business group size
Since only 2% of BGs in treatment villages had 2 members, the confidence intervals on outcomes for
two-member BGs are very large and not informative, and so we omit this analysis from the report.
Outcomes based on business health
During progress visits, Village Enterprise tracked key performance indicators of business health,
including the proportion of grant funds invested by the business, the total business value, the total
business profit, the number of members dropped from the BG, and whether the business had
up-to-date records. Each indicator was rated on a “Red-Yellow-Green” scale, and then the overall
business was rated as the worst of its indicators.
42 Figure 13 shows average outcomes for treatment
households by the business’s color rating in the most recent progress survey. In this survey, 23% of
businesses were rated as Green, 65% Yellow, 10% Red, and 2% of businesses had failed and were not
given a color rating.
42 For instance, if a business had a Red rating in one indicator and a Green rating in the other four indicators, the business
would have had an overall rating of Red.
26
Figure 13: Outcome means by business health
Households with more successful businesses had higher consumption levels and higher net asset levels.
To explore which business health indicators are driving this pattern, we estimate correlation coefficients
between outcomes and business indicators (the same five indicators that are used by Village Enterprise
to assign the business health color) and report the results in Table 2.
Table 2: Correlation coefficients between business health indicators and outcomes
Indicator Consumption Net Assets
Proportion of PR grant funds
invested in business
0.05*** 0.01
Total business value 0.35*** 0.08***
Total business profit 0.28*** 0.03**
Any members dropped from
group
-0.01 -0.02
Up do date business records 0.03** 0.01
* p < 0.10, ** p < 0.05, *** p < 0.01
Business value is most strongly correlated with outcomes, followed by business profits.
Business owner gender
Three out of four business owners were women. Figure 14 shows average outcomes by whether the
business owner in the household was a man or a woman.
27
Figure 14: Outcome means by business owner gender
Households with female business owners had higher average consumption, though net assets were
similar for households with male versus female business owners.
Participant disability
Starting with Cohort 2-Apr2019, Village Enterprise administered the Washington Group Short Set of
Questions on Disability to all business owners. This survey included six questions about whether the
respondent had difficulty with various activities, including seeing, hearing, mobility, concentration,
self-care, and communication, and if so, how much difficulty the respondent faced in attempting each
activity. A respondent was considered as having a disability if they responded that they had ‘a lot of
difficulty’ with an activity or ‘cannot do it at all’. Figure 15 shows average outcomes for business owners
by disability status. Four percent of business owners reported a disability.
28
Figure 15: Outcome means by business owner disability status
Consumption levels were similar for businesses with an owner who reported a disability versus others.
Average net asset levels were higher for businesses with an owner who reported a disability, however,
due to the relatively small number of business owners with a disability in our sample, estimates are
imprecise and the difference in net assets between business owners with and without a disability is not
statistically significant.
d. COVID and Livelihoods
Endline data was collected during the period when Kenya was experiencing the third wave of COVID-19
(July-September 2021) while Uganda was in the second wave (May-July 2021). We included questions at
the end of the survey asking respondents how the ongoing COVID-19 pandemic affected their economic
well-being. The aim was to find out how the pandemic had impacted the livelihoods of households in
the study sample. Out of the 9,888 households that we surveyed, 87% of respondents reported that
their household's livelihood had been in one way or other impacted by the ongoing COVID-19 pandemic.
The vast majority of these households (96.4%) reported that the impact had been mostly negative, while
3.6% expressed that the impact was mostly positive for them.
Figure 16 shows program impact by whether a household reported being negatively affected by the
COVID-19 pandemic.
29
Figure 16: Reported impact of COVID-19 on HH livelihoods
First, it’s notable that households that were more likely to report being negatively affected by COVID
have better outcomes than households that reported not being negatively affected. This likely reflects
geographic differences (87% of households in Kenya reported being negatively affected versus 81% of
households in Uganda) as well as differences in households’ sources of livelihood (85% of treatment
households engaged in retail, which tends to be a higher-earning business type, reported being
negatively affected versus 76% of laborers, who tend to be lower-earning).
Second, there is no statistically significant difference in treatment effects between households that
reported being negatively affected by COVID-19 compared to those that reported a positive effect or no
effect.
43 This may indicate that Village Enterprise was successful in helping treatment households to
realize the potential of the program even when those households were navigating the economic shocks
of the pandemic. At the same time, we advise against interpreting this result to mean that the pandemic
had no effect on the program’s impact. Our metric for COVID-19’s impact on a household is
self-reported, and it may be that a household was financially impacted by the pandemic even if they did
not realize it (and similarly some households may not have been as impacted as they perceived).
Instead, we regard this metric as distinguishing between households that were more versus
less-obviously impacted, and providing suggestive evidence that even the most impacted households
still benefited from the program.
Qualitative responses shared by respondents show that the pandemic affected the economic well-being
of both treatment and control groups. Respondents mentioned that the restrictions put in place due to
the COVID-19 pandemic - which included curfews; market, road, and border closures; as well as
restriction on social gatherings - affected their economic activities and increased the cost of living.
Sellers mentioned that market closures interrupted their operations since they were not able to meet
their customers, which reduced their income significantly. Some respondents also mentioned limited
ability to meet their basic needs due to a decrease in income caused by loss of formal and casual labor,
43 Additionally, the fraction of households that report being negatively affected by COVID are similar in treatment
(84.4%) and control (83.5%) villages.
30
as well as a decrease in remittances for those who were dependent on family or organizational support.
The pandemic also affected the price levels of different items, with some basic food items, as well as
medicine, becoming more expensive, while demand for other non-food items like apparel decreased,
forcing sellers to reduce prices.
We provide more details about the self-reported effects of the COVID-19 pandemic on respondents’
livelihoods in Appendix C.
31
6. Discussion
Six months to 2.5 years after the conclusion of the program, the Village Enterprise program continued
to have a positive, statistically significant, and meaningful impact on the livelihoods of its ultra-poor
recipients. Households in the treatment group reported an increase in consumption and assets, relative
to the control group, despite disruptions caused by the COVID-19 pandemic. The program had a positive
and statistically significant impact on both countries, though the impact estimates show a larger effect in
Kenya.
We hypothesize that treatment effects were larger in Kenya for several reasons. First, the study sample
in Kenya was wealthier at baseline than the study sample in Uganda, and consumption effects were
positively correlated with baseline wealth levels. Second, consumption effects were lower on average for
households surveyed after the lockdown vs before the lockdown in Uganda (-0.1 USD vs 5.9 USD per
month). However, the consumption effect for the poorest quintile in Kenya (12.8 USD) was still larger
than the consumption effect for the richest quintile in Uganda after removing households surveyed after
the lockdown (7.0 USD per month), suggesting that there may be other reasons for larger treatment
effects in Kenya. For instance, it may be that BGs in Kenya had better access to markets than their
counterparts in Uganda. We do not think that the difference is explained by the larger transfer sizes to
certain households in Kenya, since consumption effects are actually larger for households that received
smaller transfers (though the difference is not statistically significant).
Although effect sizes were smaller in Uganda, in nominal terms, the effect on consumption was nearly
identical to the previous impact estimate of the Village Enterprise program in Uganda. Sedlmayr et al
(2020) reported that the Village Enterprise program increased per capita annual consumption by UGX
26,061. Using the 2017 exchange rate of 1 USD to 3,580 UGX and average household size of 6.5
members (as observed in our Uganda sample), Sedlmayr et al’s estimate in terms of household-level
monthly consumption converts to 3.94 USD, compared to our impact estimate of 3.87 USD. On the other
hand, Sedlmayr et al (2020) found larger effects on net assets in Uganda: 29.67 USD per household, or
roughly twice the effect that we observed among Ugandan households (and half the size of the effect
among Kenyan households). The replication of impact on consumption is particularly impressive given
the effects of the COVID-19 pandemic on livelihoods, though the reduction in asset effects may reflect
households coping with the effects of the pandemic. Appendix L provides more details on the
comparison of effect sizes between our study and Sedlmayr et al (2020).
While treatment effects on average were positive and significant, magnitudes varied by program cohort.
In particular, we observe smaller effect sizes on consumption and assets in both countries for Cohort
4-Dec2019 that are statistically indistinguishable from zero. This may be driven in part by timing: Cohort
4-Dec2019 completed the program shortly before the COVID-19 pandemic and ensuing lockdowns,
which may have given inadequate time for new businesses to become established. Moreover, Village
Enterprise provided extended mentoring to Cohort 5-Apr2020 through Cohort 7-Dec2020 to help offset
COVID disruptions; this mentoring may have been a critical input to enable these cohorts to achieve
sustained consumption effects (especially in Kenya) that was not available to Cohort 4-Dec2019.
In Kenya, where some BGs received larger grants ($150 per household) than others ($50 per household),
the size of the grant appears to have had little influence on consumption effects. This is consistent with
recent evidence from cash transfer programs, where larger cash transfers do not translate into
32
meaningfully larger or longer-term program impact (Kondylis & Loeser 2021).
At the same time, households that received larger grants had significantly larger treatment effects on
net assets than households that received smaller grants. Treatment effects for households that received
larger grants are driven primarily by new home purchases, household assets (especially large electronics,
furniture, chickens, and goats), savings with community savings groups, and business stock. These
results mirror the results for the full Kenya sample, suggesting that households that received larger
grants invested in 'more of the same' as households that received smaller grants. It is therefore
somewhat surprising that we do not observe a concomitant increase in consumption effects for
households that received larger grants. Perhaps such consumption differences would emerge over a
longer time horizon as households that received smaller grants spend down their asset values more
rapidly than households that received larger grants.
Consistent with previous poverty graduation programs (e.g. those studied in Banerjee et al 2015),
treatment effects on consumption are generally larger for households with more baseline wealth. Of
course, all households in the study were categorized as ultra-poor at baseline, so differences in wealth
levels are relative to an already low distribution in each country. Nevertheless, our results are consistent
with previous evaluations, such as Banerjee et al 2015, suggesting that poverty graduation programs
may be most effective for households that are not at the very bottom of the wealth distribution.
The results from this study show that the Village Enterprise program had a statistically significant
positive impact on the livelihoods of its beneficiaries and provides valuable insights for future similar
interventions. First, the use of an RCT for the evaluation of the DIB provides an unbiased estimate of the
impact of the program at the time of data collection to determine the outcome payments to be made to
Village Enterprise. These results may have been different if data collection had not been delayed or the
lockdown in Uganda had not occurred. Second, to our knowledge this is the first evaluation that looks at
the impact of graduation style interventions in household consumption and assets during the COVID-19
pandemic. Our findings show that, in spite of the pandemic, beneficiaries of the Village Enterprise
program were still able to increase their consumption and assets, which points to the success of the
Village Enterprise model in promoting self-reliance amongst its beneficiaries. Lastly, the use of
administrative data sources containing information on business groups and business owners, shared by
Village Enterprise to complement the data that was collected during field work, allowed the team to
look at the different dimensions of analysis that provide a richer and better understanding on the impact
of the program at different levels.
Limitations of the study
Nonresponse and replacement sampling
During data collection, some sampled households were unavailable for the endline survey and had to be
replaced with randomly-selected alternates from the list of eligible households. These replacements
occurred for various reasons, though they were exacerbated by the COVID-19 pandemic. The overall
replacement rate was 14.0% (13.0% in treatment, 14.7% in control).
Although nonresponse and replacement rates were similar across treatment arms, nonresponse could
introduce bias if nonrespondents differ between treatment arms, or could reduce the external validity of
impact estimates if nonrespondents differ from replaced households. In Appendix D we show that
nonrespondents were similar across treatment arms and the final sample with replacements was
33
statistically representative of the population of eligible households.
Missing values
Missing values in the endline dataset occurred when the respondent said that their household or
business consumed or owned an item but did not know the cost, quantity, or total value of that item.
Although enumerators encouraged respondents to give their best estimate when the respondent was
not certain of a value, some respondents could not provide a reasonable guess for some survey items
and the value was coded as missing. The average survey item is missing 0.3% of responses or 2.0% of all
non-zero responses.
As specified in our PAP, for missing costs or prices, we used the median of reported sub-county cost or
price for the same item. For missing covariate information, we replaced the missing observation values
with zero and added a missing indicator for that covariate.
44 However, for missing quantities or total
values, corresponding to 0.3% of all values from these variables, we deviate from our pre-specified
approach of replacing the missing value with zero. Instead, we replaced missing quantities or total
values with the median reported by other households in the same village. We do this because
households with missing values reported consuming the good in question, but they were unsure of the
amount. If a different percentage of households in treatment and control reported consuming a good
but are unsure of the amount, then replacing missing values with zero would bias estimates of
differences in consumption between treatment and control groups.
We describe the prevalence of missing data and our approach to imputation in detail in Appendix E. We
note that since missingness is relatively rare, imputation has a small effect on outcome estimates: for
the average household, imputed values comprise 1.0% of total consumption and 2.0% of net assets.
Outliers
As pre-specified in our PAP, to limit the influence of outliers on variance estimators, we winsorized
consumption and net assets variables at the 97.5
th and 2.5
th percentiles.
45
COVID-19 impact
Conducting fieldwork during the COVID-19 pandemic exacerbated some technical risks. In particular, it
may have affected the generalizability of the findings to a non-pandemic context. From the analysis of
the self-reported effects of COVID-19, we noted that the COVID-19 crises reportedly affected livelihoods
for 87% of our respondents. As such, while the evaluation results may still be internally valid for this
context, there is a risk that impact estimates may not generalize to the non-COVID-19 context.
45 Winsorization is a technique used to limit the influence of outliers on variance estimators. It replaces extreme values above a
predefined percentile with the values at that percentile. In this case, all values above the 97.5th percentile were replaced with
the 97.5th percentile of the estimate in question for our primary outcomes, and all values below the 2.5th percentile were
replaced with the 2.5th percentile of the estimate in question for our primary outcomes.
44
In practice this only occurred for one covariate - number of school years of household head - which was missing data for 650
or 6.6% of observations.
34
7. References
Anderson, M. (2008). Multiple inference and gender differences in the effects of early intervention: a
reevaluation of the abecedarian, perry preschool, and early training projects. Journal of the American
Statistical Association, Vol 103.
Banerjee, A., Karlan, D., Osei, R.D., Trachtman, H., & Udry, C. (2018). Unpacking a multi-faceted program
to build sustainable income for the very poor. NBER Working Papers 24271.
Beegle, K., De Weerdt, J., Friedman, J., & Gibson, J. (2012). Methods of household consumption
measurement through surveys: Experimental results from Tanzania. Journal of Development Economics.
98(1), 3-18.
Benjamini, Y., Krieger, A., & Yekutieli, D. (2006). Adaptive linear step-up procedures that control the false
discovery rate. Biometrika, Vol. 93.
Blattman, C., Green, E., Jamison, J., Lehmann, M.C., and J. Annan (2016). The returns to microenterprise
support among the ultrapoor: A field experiment in postwar Uganda. American Economic Journal:
Applied Economics, 8(2): 35-64.
Duflo, E., Glennerster, R., and M. Kremer (2007). Using randomization in development economics
research: A toolkit. Centre for Economic Policy Research Discussion Paper Series, No. 6059.
Fafchamps, M., McKenzie, D., Quinn, S., and C. Woodruff (2014). Microenterprise growth and the
flypaper effect: Evidence from a randomized experiment in Ghana. Journal of Development Economics,
106(2014): 211-226.
Gertler, P., Martinez, S., Premand, P., Rawlings, L., and C. Vermeersch (2016). Impact Evaluation in
Practice: Second Edition. World Bank Group and Inter-American Development Bank.
Kondylis, F., & Loeser, J. (2021). Intervention size and persistence. World Bank Policy Research Working
Paper No. 9769.
Natali, L., & de Neubourg, C. (2014). Household welfare measurement in Bangladesh: a tale of two short
consumption modules. Unicef. Office of Research Working Paper. WP-2014-No. 17. December 2014
Sedlmayr, R., Shah, A., and M. Sulaiman (2020). Cash-plus: Poverty impacts of alternative transfer-based
approaches. Journal of Development Economics, Volume 144.
35
8. Appendices
Appendix A: Outcome payment and impact projection
Outcome payment under OPA-A2
In this appendix we calculate the DIB Outcome Payment (payment type 2, or P2
) to be made to Village
Enterprise. Based on the evaluation results, we find that P2 = $2,493,964.36, such that Village Enterprise
maxes out the total DIB outcomes payment of $4,280,618. Village Enterprise will use these funds to
repay investors. All payment formulas are copied from the Outcomes Payment Agreement--2nd
Amendment (OPA-A2), effective as of January 1, 2021.
Previously, two payments had been made to Village Enterprise: reimbursement of seed capital to
business groups for all seven cohorts (payment type 1, or P1
) = $1,138,715.64 and reimbursement for
intervention costs for Cohort 1-Dec2018 through Cohort 4-Dec2019 (payment type 1.5, or P1.5
) =
$647,938. P1.5 was equivalent to 60% of implementation costs that were incurred due to the COVID-19
pandemic.
Since the average increase in assets as of the evaluation am = $40.49 ≥ 0, we use the payment formula
corresponding to the “Optimistic scenario” in the OPA-A2:
P2B = P2A + cm , where P2A = cmm - , and
1
𝑠
𝑎
0
● cm -> ITT estimate of consumption = $9.88
● m -> weighted average of number of months from the seed transfers to data collection,
accounting for the size of transfer and timeline of transfers made = 24.3
● s -> annual discount rate of 10%, as defined in OPA-A2, corresponding to a monthly discount rate
10%/12 = 0.83%
● 𝑎 -> weighted average seed capital size made to each household for all cohorts = $77.14
0
Thus,
P2A = cmm - = 9.88*24.3 - 77.14 = $162.77,
46 𝑎 and
0
P2B = P2A + cm = $162.77 + 9.88*(1/0.0083) = $1,347.82.
1
𝑠𝑚
According to OPA-A2, P2B
is subject to a payment cap of $265 per household, thus we have P2B = $265,
and the sum of all three payments is:
P1total + P1.5total + P2*ITT population = $1,138,715.64 + $647,938 + $265*14,772 = $5,701,233.64
Since the sum of all three payments is subject to a payment cap of $4,280,618, we have:
P2total = $4,280,618 - P1total - P1.5total = $2,493,964.36.
46 Note that for all equations, we list rounded values in the text of this appendix, but we do not round in any of the
calculations. Thus the values in the equations as written in this appendix do not necessarily sum to the exact values listed after
each equality.
36
Outcome payment under OPA-1
In this section we estimate what the outcome payment would have been under the original Outcomes
Payment Agreement (OPA-1). This calculation is purely for illustration; it has no bearing on the DIB
payments. Note that OPA-1 intended for there to be two outcome payments: P2(1)total
, corresponding to
the outcome payment for Cohorts 1, 2, 3, and 4 based on data collected during the first endline in
April/May 2020, and P2(2)total
, corresponding to the outcome payment for all cohorts based on data
collected during the second endline in April/May 2021 minus P2(1)total
. Since IDinsight conducted a single
endline in May-August 2021, we set P2(1)total = 0 in the calculations below. This calculation differs from
what P2(1)total+P2(2)total would have been under OPA-1 to the extent that treatment effects vary over time.
Since the average increase in assets as of the endline is am = $40.49, and the weighted average seed
capital size made to each household for all cohorts 𝑎 = $77.14, we have > am > 0, corresponding to
0
𝑎
0
the “Medium Scenario” in OPA-1. To calculate the appropriate payment in this scenario, we first
estimate the depreciation rate:
d = 1-(am / )
(1/m) = 1-(40.49/77.14)
(1/24.3) 𝑎 = 2.6% 0
Thus,
P2A = cmm - 𝑎 = $162.77 (as before under OPA-A2), and
0
P2B = P2A + cm = $162.77 + 9.88*( ) = $441.38
47
.
1−𝑑
𝑠+𝑑
1−0.026
0.0083+0.026
Since P2B was still subject to a payment cap of $265 per household under OPA-1, we have P2B = $265, so
P2total = $265*14,772 = $3,914,580. However, since P1total = $1,138,715.64, then P1total+P2total > $4,280,618,
which is the maximum DIB payment, and so we have P2total = $4,280,618 - $1,138,715.64 = $3,141,902.
Thus, even without P1.5
(the reimbursement of 60% of programming costs due to COVID-19), VE would
still have maxed out the total DIB payment of $4,280,618 under OPA-1.
Projection of total lifetime impact
We estimate total lifetime impact of the program as the sum of consumption effects at the time of
endline data collection, plus a projection of future consumption effects:
● Impact from program start to endline data collection: At the time of endline data collection in
mid-2021, the average household had started the program 24.3 months ago, and so program
impact to that point is estimated as ($9.88 treatment effect on monthly consumption)*(14,772
treated households)*(24.3 months) = $3.55M.
● Projected future impact: Future impact is estimated as ($9.88)*(14,772 treated households)*(12
months per year)/(0.1 annual discount rate) = $17.51M.
Thus we estimate total lifetime impact of the Village Enterprise program is $3.55M + $17.51M =
$21.06M.
47
In this calculation, we use unrounded figures. If rounded figures are plugged into the formula, the result would be
$443.33
37
Appendix B: Regression results for main outcomes
Table B1: Treatment effects of main outcomes
(1)
Control
Mean
(2)
Treatment
Effect
(3)
p-value
(4)
Std. Error
(5)
N
(6)
R2
ITT Consumption 158.007 9.875*** 0.000 1.700 9888 0.3297
ITT Assets 703.296 40.490*** 0.003 13.638 9846 0.1606
TOT Consumption 158.007 10.290*** 0.000 1.752 9888 0.3297
TOT Assets 703.296 42.185*** 0.003 14.017 9846 0.1606
* p < 0.10, **p < 0.05, *** p< 0.01
38
Appendix C: Self-reported effects of COVID on livelihoods
During endline data collection, Kenya was experiencing the third wave of COVID-19 while Uganda was in
the second wave. For this reason, we added a short section on the effect of COVID-19 on the economic
well-being of the households we were surveying, consisting of the following questions:
● Has your household's livelihood been in any way impacted by the ongoing COVID-19 pandemic?
● Would you say that the impact has been mostly positive or mostly negative?
● Please briefly explain your answer in a few sentences
87% of our respondents reported that their household's livelihood had been in one way or another
impacted by the ongoing COVID-19 crisis. Out of this number, 96.4% reported that the impact had been
mostly negative, while 3.6% expressed that the impact was mostly positive for them. We then requested
the respondents to share more information to support their responses and below is a summary of the
key themes that came out of all the responses.
1. Access to markets
Just over a third (38%) of the respondents who reported that the pandemic affected their livelihoods
negatively mentioned accessibility to markets. Market closures were one of the directives by the
Governments of Kenya and Uganda as part of efforts to contain the spread of the virus. This affected
both buyers and sellers in different ways. Sellers were not able to reach their usual customers as before,
and some small-scale sellers who would typically get their stock from the open-air markets were left
without a source for their stock. Also, owing to the pandemic, the number and the purchasing power of
customers went down. People were no longer able and willing to pay as before because of the
uncertainty that came with the different COVID waves; demand for some items decreased. Others
expressed that there was the loss of casual labor occurring in markets, like for those who make their
daily living from transporting items from the market at a fee or are employed as extra hands or to attract
customers by those who own businesses during the market day. Additionally, there was a 7pm curfew in
both countries that reduced the “selling time”. Some businesses would sell more in the evenings but this
was restricted since they had to close early. Others, like the alcohol business, were directed to
completely close which left the owners without a source of income.
2. Change in the price level
Respondents mentioned that the cost of living increased due to the pandemic. Basic food items,
transportation, and medicine became more expensive due to disruptions in the supply chain caused by
market and border closures, as well as the increased transport costs incurred by sellers transporting the
commodities. Public transport was required to only carry half as many passengers to ensure social
distancing. As a result,fare prices increased to cover the cost of fuel. On the other hand, some
respondents noted a decrease in the prices of some commodities, such as apparel, due to a decrease in
demand.
3. Business activity
Many business owners reported that their businesses shrunk or closed as a result of COVID-19. Roads,
border closures, and lockdowns in both countries affected business activities. In Uganda, bodaboda
riders faced restrictions to only carry one passenger which reduced their income. Some respondents
expressed that they needed to hire a middleman to transport their goods which reduced their profit. In
39
some cases respondents no longer had access to a middleman and thus could not sell their goods in the
market. There was also an increased number of customers taking products on credit and failing to pay in
good time which affected restocking. This was the case for some casual workers who reported to work
but were not paid or their pay was delayed. Some respondents stated that they were forced to use
business capital to pay for household expenses which led to business failure. Others who had plans to
expand their businesses or start a business canceled their plans due to COVID. For farmers, a number
reported that they had to end group farming as they would not be able to pay casual workers, and some
reverted to subsistence farming. Many respondents also mentioned that they could no longer meet in
their local savings group since people didn’t have much to save, and hence they could not borrow from
the groups to boost their businesses as before.
4. Impact on livelihoods
We had several self-reported cases of increased poverty/hunger attributed to the effects of COVID-19.
One of the reasons given for this was the increased cost of food. Household income also decreased as
business profit was reduced due to COVID or the breadwinner of the family lost their job. Many survey
respondents depended on casual work and, due to COVID, the demand for casual labor declined while
the supply increased since those who were formally employed and lost their jobs joined this pool.
Therefore, even if one was lucky enough to get casual work, the salary was much lower compared to
pre-pandemic rates. Others, especially the elderly who depended on support from family, friends, and
the church reported that there was a decrease in financial assistance since the onset of COVID-19. There
were households that were receiving support from the government and NGOs as a form of social
protection prior to COVID, and they also reported that this was no longer happening or its frequency
decreased.
Schools were closed to prevent the spread of the virus. This meant that those households with children
in boarding schools as well as those who ate some meals from school now had to cover their children’s
food expenses at home. This significantly increased the household’s expenditure on food items and
some struggled to provide for their families. Some parents also expressed frustration that the school
fees they had already paid as well as the other school-related expenses they had incurred before their
children were sent home due to COVID would go to waste. Unfortunately, there were also a few cases
where it was reported that the teenage pregnancy rate increased due to COVID. Further, respondents
expressed that they were burdened by the need to spend additional money to purchase masks,
sanitizers, or pay police fines during the lockdown.
5. Positive impact
COVID-19 did not entirely have negative effects; 3.6% of our respondents reported that it mostly had
positive effects on their livelihoods. Some sellers who were able to navigate the challenges presented by
COVID ended up benefiting more due to lack of or reduced competition when their counterparts’
businesses failed. There are others who had commodities that became more expensive due to COVID
and as a result, they were able to make more profit. Additionally, demand increased for some
respondents’ products, like firewood to accommodate an increase in cooking since children were out of
school and needed to take their meals at home. Some respondents started businesses in response to
COVID, like selling masks. Some household heads reported having an additional set of hands to help with
farming and house chores as a result of the closure of schools. For those respondents who were farming
in addition to other business activities, since they could not go out as much due to COVID, they reported
40
that they put more time and focused on farming which increased their productivity. Also, the closures of
businesses and decreased activity forced some respondents to learn to limit or reduce their
discretionary spending. Furthermore, some respondents reported having been beneficiaries of
COVID-related initiatives by NGOs and humanitarian organizations.
6. No COVID impact
13% of respondents reported that their livelihoods were not impacted by the ongoing pandemic.
Respondents who sold produce from their homes and subsistence farmers were not affected by the
closure of markets. The elderly and other dependents who mostly stayed at home pre-pandemic were
less affected by the lockdown.. Respondents living in extreme poverty without a business, a job, or many
assets were less affected by the pandemic. Further, some respondents reported that their sectors of
work were not affected, like those offering essential services.
41
Appendix D: Nonresponse and replacement sampling
This appendix clarifies issues over nonresponse, attrition and replacement sampling in the Village
Enterprise DIB evaluation. Since the evaluation did not include a baseline, there is conceptual ambiguity
over the meaning of attrition in the pre-analysis plan. In this appendix we introduce the related but
distinct concept of replacement sampling, or substituting initially sampled households with randomly
selected replacements. The final replacement rate of 14.0% (13.0% in treatment, 14.7% in control) was
largely driven by COVID restrictions on data collection, especially curfews enforced in response to surges
in COVID cases, as we predicted may happen in our pre-analysis plan. We show that nonrespondents are
similar across treatment and control groups and the final sample with replacements is statistically
representative of the population of eligible households. As a result, the initially specified analytical
model with our final sample, including replacements, continues to offer the most robust estimate of the
impact of the Village Enterprise program.
1. Attrition protocols in the pre-analysis plan
In our pre-analysis plan,
48 we defined attrition as being unable to collect data from participants who
dropped out of the study between baseline and endline (p. 15). We specified possible actions in
response to various levels of attrition, citing 10% as our target threshold (p.16). We also added a section
in the latest version of the PAP related to technical risks due to collecting data during the COVID-19
pandemic (p. 17), in which we noted that attrition may be higher than projected (>10%) due to
respondent reluctance to speak with enumerators or transportation and mobility challenges during the
pandemic.
While these protocols provided important guidance for our data collection approach, we realized after
data collection that there was conceptual confusion over the meaning of ‘attrition’ in the PAP. Attrition
refers to the inability to collect data from individuals who were in the original study sample.
49
In this
evaluation, although Village Enterprise conducted a census of eligible households, there was no sample
taken at baseline, and instead the study sample was taken at endline. As a result, ‘attrition’ is not a
meaningful concept for this evaluation. Instead, the PAP should have described the related but distinct
concept of nonresponse at endline.
The distinction between nonresponse and attrition is a common feature of RCTs. For instance, the World
Bank’s Impact Evaluation in Practice describes ‘unit nonresponse’ as “aris[ing] when it becomes
impossible to collect complete data for some sampled units” and ‘attrition’ as “a common form of
nonresponse that occurs when some units drop from the sample between data collection rounds”.
While researchers often do not report nonresponse rates in their RCTs, some do and, if applicable, will
differentiate the nonresponse rate from the attrition rate. For instance, in the poverty graduation RCT
literature, Sedlmayr et al (2018) distinguish between sampled respondents who “opted to participate in
the study” (p. 9) (pertaining to 6.4% of the sample) from those for whom “follow-ups were successful”
(p. 20). Blattman et al 2016 distinguish between “nonresponse” (p. 42) (pertaining to 5.5% of the
sample) and “attrition” (p. 44). Fafchamps et al 2014 distinguish between sampled units that could be
found and willing to participate in the study (p. 215) and attrition (p. 216).
Nonresponse at endline should be tracked and investigated since missingness resulting from
49 This definition is given on p.58 in Duflo, Glennerster, and Kremer (2006) “Using Randomization in Development Economics
Research: A Toolkit” CEPR No. 6059, and elsewhere.
48 https://www.socialscienceregistry.org/trials/7168
42
nonresponse at endline can reduce study power, different nonresponse rates at endline in the treatment
and control groups can bias impact estimates, and different characteristics among those who did and did
not respond can reduce external validity of the study. In the following section we describe how
replacement sampling was successfully used to ensure that non-response at endline did not harmfully
reduce power, that missingness is independent of potential outcomes (MIPO), and that the final sample
with replacements likely remains representative of the population of target households.
2. Replacement sampling in the Village Enterprise DIB evaluation
Replacement sampling refers to substituting households that were in the initial sample with randomly
selected replacements. Had a baseline survey been conducted, it is likely that some of the households
that were replaced at endline would have been replaced at baseline as well for similar reasons (e.g. if a
household refused consent at endline, it is likely that they would have refused consent at baseline as
well). Of those in the initial sample, the nonresponse rate was 16%. We executed a random replacement
sampling approach that led to an endline replacement rate of 14.0% (13.0% in T, 14.7% in C), and
collection of data from 98% of the target number of households. Using random sampling ensured that
nonresponse at endline would not harmfully reduce study power.
These replacements occurred for a variety of reasons, but were exacerbated by the COVID pandemic. In
particular, the most common reason for replacement, accounting for ~50% of replacements, was that no
one was available to be surveyed at the household until after COVID-induced curfews, which were
enforced at 7pm in both countries. In such cases our enumerators attempted to track someone from the
household to another location in the village and complete the survey at that location. Enumerators also
returned to villages to reach households that were unavailable on the first visit. Households were only
replaced if no one was available within a reasonable distance, or couldn’t be reached during up to 3
revisits, until after field teams had to return to their hotels due to curfews. Our protocols were based on
standards that are commonly used in deciding whether to replace households in evaluations, such as
those described in the section on “Conducting Fieldwork and Undertaking Quality Control” in the World
Bank’s Impact Evaluation in Practice: Second Edition (p. 308-311).
3. Missingness, Internal Validity, and External Validity
Village Enterprise collected targeting and covariate data at baseline that we use to explore the
independence of missingness coming from nonrespondents with potential outcomes. The nonresponse
rates were similar across treatment (14.8%) and control (16.8%) groups.
50 We explore MIPO conditioned
on different indicators that are likely predictive of household consumption and assets, such as PPI scores
and household characteristics, making comparisons between treatment and control groups as well as
between nonresponse and replacement households.
50 One concern raised was that households may have been unavailable because they were engaged in economic activities (such
as buying/selling at the market) resulting from participation in the Village Enterprise program, and thus we would be missing
out on the households most impacted by the program. The fact that replacement was slightly higher in the control group
suggests that this was not occurring.
43
Table D1. Balance across treatment and control groups for nonrespondents
Variable Treatment Control Diff p-value
PPI score 27.24 27.44 -0.19 0.88
Number of people living in the HH 5.97 6.08 -0.11 0.48
Number of cows the HH owns 0.55 0.57 -0.02 0.59
Number of bodas the HH owns 0.04 0.04 -0.01 0.64
Whether HH owns a radio 0.29 0.27 0.02 0.71
Whether every HH member has a
pair of shoes 0.11 0.10 0.01 0.53
Whether HH has an iron roof 0.63 0.64 -0.02 0.87
Number of beds the HH owns 0.76 0.82 -0.05 0.14
Number of pigs the HH owns 0.15 0.10 0.04 0.60
Number of bicycles the HH owns 0.29 0.29 0.00 0.83
Whether HH owns a frying pan 0.01 0.02 -0.01 0.23
Number of jerry cans the HH
owns 2.98 3.20 -0.22 0.05
Whether the HH owns a mobile
phone 0.66 0.68 -0.02 0.78
Number of years of school for the
HH head 6.08 6.21 -0.12 0.94
N 750 849
Table D1 shows that the characteristics across treatment and control groups for nonrespondents are
very similar. None of the differences are significant at the 5% level. This, along with the similar
replacement rates across treatment and control groups, indicates that the replacement strategy
employed did not jeopardize internal validity.
Table D2. Balance across treatment and control groups for nonrespondents (Kenya)
Variable Treatment Control Diff p-value
PPI score 31.08 31.75 -0.68 0.12
N 453 533
44
Table D3. Balance across treatment and control groups for nonrespondents (Uganda)
Variable Treatment Control Diff p-value
PPI score 21.40 20.16 1.25 0.06
N 297 316
Table D4 below presents the findings from the comparison between nonrespondents and replacement
households.
Table D4. Balance across treatment and control groups for nonrespondents and replacements
Variable Nonrespondents Replacements Diff p-value
PPI score 27.35 26.05 1.29 0.00
Number of people
living in the HH 6.03 6.60 -0.58 0.00
Number of cows the HH
owns 0.56 0.81 -0.25 0.00
Number of bodas the
HH owns 0.04 0.04 0.01 0.46
Whether HH owns a
radio 0.28 0.32 -0.04 0.02
Whether every HH
member has a pair of
shoes 0.11 0.10 0.01 0.82
Whether HH has an
iron roof 0.63 0.64 0.00 0.22
Number of beds the HH
owns 0.79 0.90 -0.11 0.00
Number of pigs the HH
owns 0.12 0.14 -0.02 0.48
Number of bicycles the
HH owns 0.29 0.36 -0.08 0.00
Whether HH owns a
frying pan 0.01 0.01 0.00 0.39
Number of jerry cans
the HH owns 3.09 3.28 -0.19 0.01
Whether the HH owns a
mobile phone 0.67 0.71 -0.04 0.01
45
Number of years of
school for the HH head 6.15 6.02 0.13 0.45
N 1599 1365
Table D4 shows that there are some differences between nonrespondents and replacement households.
PPI scores, for instance, are slightly lower for replacement households (27.3 vs 26.1).
51 Replacement
households have more household members, cows, radios, beds, bikes, jerry cans, and phones. However,
the differences are generally small in magnitude, and given that the replacement rate for the study is
low overall (14%), these differences are likely to have a limited effect on external validity. The final
sample with replacements has nearly the same PPI average score as the list of all eligible households
(25.5 and 25.7, respectively).
Similarity across these indicators for both sets of comparisons provides suggestive evidence that
differential nonresponse across treatment and control groups is not introducing bias to the study, and
that the final sample with replacements likely remains representative of the population of target
households. For these reasons, we are confident that the original model with the final sample (including
replacements) continues to offer the most robust estimate of the impact of the Village Enterprise
program.
Table D5. Balance across treatment and control groups for nonrespondents and replacements
(Kenya)
Variable Nonrespondents Replacements Diff p-value
PPI score 31.44 29.79 1.65 0.00
N 986 831
Table D6. Balance across treatment and control groups for nonrespondents and replacements
(Uganda)
Variable Nonrespondents Replacements Diff p-value
PPI score 20.76 20.24 1.25 0.06
N 613 534
51 This magnitude is small relative to the range of PPI scores among eligible households at baseline, which vary from 0 to 75
with a standard deviation of 10.3.
46
Appendix E: Missing values and imputation approach
This appendix describes our process for imputing missing values in the endline dataset. While
missingness occurs in our dataset, it is not common: the average survey item is missing 0.3% of all
responses or 2.0% of all non-zero responses. As a result, imputed values comprise a relatively small
fraction of primary outcome values: for the average household, 1.0% of total consumption and 2.0% of
net assets rely on imputed quantities or values within component items. However, as described below,
we deviate slightly from our pre-specified approach in how we impute missing values.
1. Prevalence of missing values in the Village Enterprise DIB evaluation dataset
Missing values in the endline dataset occurred when the respondent said that their household or
business consumed or owned an item but did not know the cost, quantity, or total value of that item.
Although enumerators encouraged respondents to give their best guess when the respondent was not
certain of a value, some respondents could not provide a reasonable guess for some survey items and
the value was coded as missing.
The average survey item relating to quantity or total value is missing 0.3% of all responses or 2.4% of all
non-zero responses.
52 23% of survey items are not missing any responses, 90% of survey items are
missing fewer than 1% of all responses, and 59% of survey items are missing fewer than 1% of non-zero
responses. The survey item with the highest prevalence of missingness is the quantity of alcohol
consumed by households in Kenya, which is missing 4% of all responses or 22% of responses where the
household reported consuming at least some alcohol.
The average survey item relating to cost is missing 0.4% of all responses or 1.2% of responses from
individuals who consumed or owned those items.
53 91% of survey items relating to cost have at least
some missing values, though only 11% of survey items are missing more than 1% of all response values.
As with survey items relating to quantity or total value, the survey item with the highest prevalence of
missingness is the cost of alcohol consumed by households in Kenya, with 21% of respondents who
reported consuming some alcohol failing to provide a cost for that consumption.
2. Deviation from pre-specified approach to dealing with missing data
In our PAP we specified three methods for addressing missing values:
● “the median of reported sub-county prices will be used when a price is unknown” (p. 12)
● “If any households have missing covariate information, we will replace the missing observation
values with zero and add a missing indicator for that covariate” (p. 13)
● “If a participant does not know the answer to a response or refuses to give an answer, we will
treat that category as a zero value for the household consumption and asset value.” (p. 17)
We document one update to the pre-specified approach of dealing with missing quantities/total values,
relating to #3 above.
53 A table listing all 75 consumption and asset survey items relating to costs, along with the same statistics listed in the
previous footnote, is available upon request. We omit it from this report due to space constraints.
52 A table listing all 172 consumption and asset survey items relating to quantities or total values, along with the number of
responses, the number of non-zero responses, the number of missing values, and the frequency of missing values as a percent
of all responses and all non-zero responses, is available upon request. We omit it from this report due to space constraints.
47
For quantities and total values, we were concerned that the pre-specified approach of replacing missing
values with zero could bias treatment effect estimates toward zero. Households with missing values
reported consuming the good in question, but they were unsure of the amount. If a different percentage
of households in treatment and control reported consuming a good but are unsure of the amount, then
replacing missing values with zero would mis-estimate true differences in consumption between
treatment and control groups.
Instead, our preferred and updated approach is to replace missing values with the median reported by
other households in the same village.
54 Where there are no non-missing observations for a particular
item from the same village, we replace missing values with the median reported by other households in
the same treatment group and subcounty. We believe that this updated approach results in more
accurate estimates of the amount of each good consumed or owned than replacing missing values with
zero, and less biased treatment effect estimates.
3. Missing data imputation process
In this section we describe the process followed for imputing missing values for each category of survey
items.
Consumption (one-week recall items)
For one-week recall items, respondents reported the amount consumed, the units of consumption, and
the price per unit. For most items, respondents in our study reported different units of consumption. For
instance, 77% of households in Kenya reported milk consumption in cups, while 9% reported milk
consumption in liters, 9% reported milk consumption in bottles, and the remainder reported milk
consumption in other units.
For missing cost values for these items, we calculated the median cost per unit in each subcounty, and
replaced missing cost values for respondents that reported consumption in the same units.
For missing quantities, we first calculated the total value of consumption (quantity*price per unit) for
each item among households that have non-missing quantity and price values. We then imputed missing
total values for households with missing quantities, rather than imputing quantities directly, since
households reported consumption in different units. We calculated the median total value spent on
each item within each village, and used these values to impute missing total values for households in the
same village.
Consumption (one-month and one-year recall items)
For one-month and one-year recall items, respondents only reported the total amount spent on each
item, and not the amount consumed or the price per unit. For this reason, we only impute total values,
using the median total value for each item reported by households within the same village.
Construction services
Since construction services (such as new homes, additions to homes, plumbing additions and repairs,
etc.) are relatively rare in our sample, we do not have sufficient data to impute median costs of different
54 Since all households in a village have the same treatment status, this implies that quantities and total values are imputed
within the treatment group.
48
types of construction projects in each village. Instead, we calculate the median cost per construction
type within each subcounty X treatment group, and apply those values to missing values for the same
types of construction projects in the same subcounty X treatment group.
Household assets
Respondents reported the number of household assets owned and the total amount that they would
expect to earn if they sold all of those assets in their current condition. For households that reported the
number but not the total value of an asset type, we calculated the median value-per-unit of that asset
type within the same sub-county, and multiplied the result with the number of assets reported by the
household. More rarely, a household did not know the number of assets of a particular type.
55
In these
instances, we calculated the median number of that type of asset owned by other households in the
same village, and replaced the missing value with the result.
Household savings
For household savings, respondents reported all of the places where they saved money (at home,
mobile banking, in a savings group, etc.) and the amount saved in each location. For most savings types,
there are insufficient observations to impute within the village, and so we calculate the median amount
saved within each subcounty X treatment group and apply those results to missing values for the same
savings type. We make an exception to this approach for savings with savings groups, since nearly half of
households held this type of savings. For the households that reported savings with a savings group but
did not know how much was saved, we calculated the median amount saved with savings groups by
other households in the same village, and applied the result to the missing savings amount.
Household loans
For household loans, respondents reported the source of the loan (e.g. from a private bank, mobile
banking, etc.), the original loan amount, and the amount still owed. For respondents that knew the
original loan amount but not the amount still owed, we calculated the median fraction still owed on the
same type of loans within the same subcounty X treatment group, and multiplied the result with the
respondent’s original loan amount to estimate the amount still owed. For respondents who did not
know the original loan amount or the amount still owed, we calculated the median amount still owed on
loans of the same type within the same subcounty X treatment group, and applied the result to the
missing value.
Business assets
Respondents reported the total value of each type of asset owned by each household business, along
with the number of business partners outside of their household for each business, and whether each
asset was owned solely or with other business partners.
56 We do not know the cost-per-unit or the
number of units of each asset type, and so we imputed the total value for missing asset types. Since
there are relatively few business assets reported in our data, we do not have sufficient data to calculate
total values for each asset type within each village. Instead, we calculated the median total value of each
business asset type within each subcounty X treatment group, and applied those values to missing
56 29% of respondents reported no household business, 48% reported 1 business, 18% reported 2 businesses, and the
remaining 5% reported more than 2 businesses. The most businesses, reported by 2 respondents, was 7.
55 The asset type with the most missing values is chickens; 7 respondents or 0.09% of all 7,599 respondents with chickens did
not know how many chickens they owned.
49
values for the same asset types within the same subcounty X treatment group.
Business savings
For business savings, respondents reported all of the places where they saved money for each business,
the total amount saved, and the portion saved belonging to the household. Similar to household loans,
we first imputed missing values for respondents who knew the total amount saved but not the portion
belonging to the household. For these respondents we calculated the median fraction of each business
savings type belonging to the household in each subcounty X treatment group and applied the result to
missing values. For respondents who did not know the total amount saved or the portion belonging to
the household, we calculated the median amount of each type of business savings belonging to the
household in each sub-county X treatment group and applied the result to missing values.
Business loans
For business loans, respondents reported the source of each loan, the total loan amount, and the
amount that the household owes. Similar to business savings, we first imputed missing values for
respondents who knew the total loan amount but not the amount owed by the household. For these
respondents we calculated the median fraction of each business loan type owed by the household in
each subcounty X treatment group and applied the result to missing values. For respondents who did
not know the total amount owed or the amount owed by the household, we calculated the median
amount of each type of business loan owed by the household in each sub-county X treatment group and
applied the result to missing values.
4. Imputed values in consumption and net assets variables
In this section we report the fraction of each outcome (monthly consumption, net assets) that relies on
imputed data.
For the average household, 1.0% of total consumption relies on imputed quantities or values in its
component items. 1.6% of total consumption relies on imputed quantities, total values, or costs. The
median household has no imputed consumption data. For 95% of households, less than 5.4% of total
consumption relies on imputed quantities or values in its component items, and less than 9.7% relies on
imputed quantities, values, or costs.
For the average household, 2.3% of net assets relies on imputed quantities or values in its component
items. Similarly, 2.3% of net assets relies on imputed quantities, total values, or costs.
57 The median
household has no imputed net assets. For 95% of households, less than 6.3% of net assets relies on
imputed quantities, values, or costs.
57 The amount of net assets that involves imputed costs is very small (< 0.01%) but non-zero since costs could be calculated for
a small number of asset items.
50
Appendix F: Treatment effects on individual components of
consumption and assets
In this appendix we report treatment effects on individual components in the consumption and asset
indices. As a reminder, to put the following effect sizes into context, the ITT effect on monthly
consumption was USD 9.9 across both countries (USD 15.2 in Kenya, USD 3.9 in Uganda), and the ITT
effect on net assets was USD 40.5 across both countries (USD 60.9 in Kenya, USD 15.6 in Uganda).
Treatment effects on consumption are driven mostly by food expenditures. Table F1 reports treatment
effects on consumption items in Kenya. Treatment effects in Kenya are observed across many food
items, and are largest (in absolute values) for green maize, beef, dried and smoked fish, maize grain or
flour, and chicken. Treatment households in Kenya consumed significantly less alcohol. Positive
treatment effects are also observed on some non-food items, including vehicle fuels, domestic services,
and health expenses, and mobile airtime.
Table F2 reports treatment effects on consumption items in Uganda.
58 Treatment effects in Uganda are
relatively smaller but are similarly concentrated in food items. Treatment households in Uganda
consumed significantly more chicken, dried and smoked fish, and tomatoes. Treatment effects on
non-food expenditures are positive on average but not significant for any particular items.
Treatment effects on assets are driven more by increases in household assets than in other types of
assets or savings, though effects are observed in other categories as well. Table F3 reports treatment
effects on asset items in both countries. In Kenya, treatment households spent more on new roof
construction, new homes, and durable assets. Treatment households were also more likely to save with
savings groups, and have larger mobile money and household savings.
In Uganda, treatment households spent more on livestock assets. In contrast to Kenya, treatment effects
on cattle and sheep in Uganda are large and significant. Interestingly, treatment effects are negative for
the asset value of bodas/motorcycles. We also observe positive and significant effects on savings with
savings groups in Uganda.
In terms of business assets, positive and significant effects are only observed in Kenya, where treatment
households had significantly more business stock, as well as more machines and tools. We do not
observe significant effects on business assets in Uganda, or on business savings in either country.
Treatment effects on household and business loans are mostly small and non-significant.
58 Treatment effects on consumption items are reported in separate tables for Kenya and Uganda since the list of items in the
consumption modules differed across the two countries. However, the list of items in the asset modules were the same in the
two countries, and so we print a single table for treatment effects on asset items (though we still report the country-wise
treatment effects in separate columns).
51
Table F1: Treatment effects on consumption items in Kenya
Item Avg in Control ITT % Increase q-val
59
TOTAL CONSUMPTION
60 207.41 15.21 7.3% 0.00
Food & beverage consumption 111.72 9.19 8.2% 0.00
Rice 2.77 0.58 20.8% 0.00
Maize grain or flour 20.43 0.64 3.1% 0.05
Green maize 7.79 1.22 15.6% 0.00
Bread 1.70 0.47 27.7% 0.00
Wheat grain or flour 2.13 0.55 25.9% 0.00
Sweet potatoes 1.19 0.28 23.5% 0.00
Matooke/cooking bananas 4.01 0.54 13.6% 0.00
Beans 7.95 0.43 5.4% 0.05
Onions/leeks 1.49 0.07 4.8% 0.04
Tomatoes 1.94 0.16 8.2% 0.00
Sukuma/kale 3.55 -0.01 -0.2% 0.34
Traditional vegetables 4.21 0.27 6.4% 0.01
Ripe bananas 2.10 0.37 17.8% 0.00
Avocado 5.26 0.56 10.6% 0.01
Fresh milk 10.10 0.31 3.1% 0.06
Beef 3.70 1.17 31.5% 0.00
Chicken 2.35 0.62 26.4% 0.00
Dried/smoked fish 2.74 0.74 27.1% 0.00
Cooking oil/fat 4.53 0.22 4.9% 0.02
Sugar 7.18 0.02 0.3% 0.31
Salt 0.74 0.02 2.1% 0.09
Tea 1.71 -0.01 -0.6% 0.28
Soda/soft drink 0.67 0.16 23.4% 0.00
Alcohol 2.56 -0.42 -16.3% 0.01
Food prepared outside of home 1.36 0.07 5.4% 0.15
Recurring consumption 37.41 2.94 7.9% 0.00
Fuel/kerosene/paraffin 0.59 0.06 9.9% 0.03
Soap/laundry bar/detergents 3.49 0.06 1.7% 0.14
Match boxes 0.34 -0.01 -2.1% 0.09
Barber/beauty shops 2.10 0.07 3.4% 0.14
Personal care/cosmetics 1.41 -0.02 -1.1% 0.28
Vehicle fuels (petrol, diesel, engine oil) 2.81 0.59 21.1% 0.03
60 The ITTs of the component items do not sum to the ITT of total consumption for two reasons. First, total consumption is
scaled by 1.176 in Kenya and 1.174 in Uganda to account for missing items in the shortened consumption module. Second, for
the analysis in this appendix we winsorized individual components of consumption to limit the influence of outliers on ITTs and
more clearly identify patterns. For total consumption we winsorized the aggregate index, but not the individual components,
to limit the number of post-hoc adjustments to the values reported by respondents.
59 q-values can be interpreted like p-values but with an adjustment for multiple hypothesis testing. Because of the number of
statistical tests performed in this item-wise analysis, the likelihood of incorrectly rejecting the null for any individual item is
high. To account for the multiplicity of hypotheses being tested and to reduce the likelihood of incorrectly rejecting null
hypotheses, we control for the false discovery rate (FDR) and report ‘sharpened q values’, as defined in Anderson (2008). These
q-values can be interpreted as the expected FDR in the family of outcomes if we reject the null at that level. We define four
families for these corrections: consumption in Kenya and Uganda, and assets in Kenya and Uganda.
52
Matatus/buses 3.14 0.20 6.4% 0.11
Bodaboda/tuk-tuk 4.16 0.08 1.9% 0.24
Mobile phone airtime 4.73 0.31 6.6% 0.05
Firewood 9.52 0.16 1.6% 0.23
Domestic services (gardener, cook,
guard)
2.65 0.75 28.2% 0.00
Infrequent consumption 27.50 0.66 2.4% 0.15
Men's clothing/footwear 1.22 0.08 6.9% 0.04
Women's clothing/footwear 1.50 0.09 5.8% 0.04
Children's clothing/footwear 1.69 0.06 3.6% 0.10
Educational expenses (school fees,
uniforms, books, supplies)
16.88 0.00 0.0% 0.35
Health expenses 4.90 0.42 8.6% 0.03
Table F2: Treatment effects on consumption items in Uganda
Item Avg in Control ITT % Increase q-val
TOTAL CONSUMPTION 106.63 3.87 3.6% 0.04
Food & beverage consumption 65.12 2.55 3.9% 0.14
Rice 0.74 0.11 14.8% 0.21
Maize grain or flour 3.65 -0.05 -1.5% 1.00
Sweet potatoes 1.26 0.05 3.6% 0.92
Matooke/cooking bananas 0.82 0.05 5.9% 0.75
Chapati 0.37 0.06 16.3% 0.07
Beans 8.37 0.24 2.8% 0.47
Cassava/cassava flour 11.64 0.39 3.4% 0.39
Onions/leeks 0.88 0.04 4.7% 0.35
Tomatoes 1.61 0.19 12.0% 0.01
Greens (dodo, nakati, gyobyo,
malakwang, other)
4.31 -0.19 -4.4% 0.35
Groundnuts 2.62 0.13 4.9% 0.67
Fresh milk 0.89 0.00 -0.2% 1.00
Beef 2.90 -0.19 -6.6% 0.67
Chicken 5.57 0.92 16.5% 0.00
Fresh fish 2.43 -0.04 -1.7% 1.00
Dried/smoked fish 4.31 0.43 9.9% 0.03
Cooking oil/fat 2.29 0.03 1.3% 0.92
Sugar 1.85 0.09 4.6% 0.39
Salt 1.05 0.01 1.2% 0.67
Tea 0.13 0.01 4.4% 0.75
Alcohol 1.85 -0.10 -5.4% 0.67
Food prepared outside of home 0.56 -0.02 -3.3% 0.92
Recurring consumption 3.98 0.04 1.0% 0.95
Washing soap 1.98 0.03 1.3% 0.75
Match boxes 0.14 0.00 -0.6% 1.00
Personal care/cosmetics 1.77 0.02 0.9% 1.00
53
Infrequent consumption 18.91 0.17 0.9% 1.00
Men's clothing/footwear 0.91 -0.01 -1.0% 1.00
Women's clothing/footwear 1.00 0.03 2.7% 0.69
Children's clothing/footwear 1.10 0.03 2.4% 0.75
Educational expenses (school fees,
uniforms, books, supplies)
5.58 0.41 7.3% 0.35
Funerals/other social functions 2.47 0.05 1.9% 0.99
Health expenses 6.17 -0.08 -1.3% 0.92
Table F3: Treatment effects on asset items
Item Kenya Uganda
Avg in
Control
ITT % Incr q-val
Avg in
Control
ITT % Incr q-val
TOTAL ASSETS
61 718.25 60.94 8.5% 0.00 687.68 15.60 2.3% 0.43
Livestock assets 288.34 -0.03 -0.0% 0.48 338.02 29.34 8.7% 0.02
Durable assets 290.15 21.48 7.4% 0.01 161.31 -6.62 -4.1% 0.24
Home improvements
Construction of new
home
N/A
62 N/A
New room 5.32 0.47 8.8% 0.25 1.01 0.23 23.0% 0.07
New roof 1.70 0.49 28.5% 0.02 5.47 0.96 17.6% 0.00
Wall N/A 0.30 -0.03 -11.0% 0.43
Latrine N/A 0.20 0.01 7.1% 0.73
Plumbing N/A N/A
Wiring N/A N/A
Repair N/A 0.63 -0.03 -4.3% 0.73
Floor N/A 0.03 0.01 27.7% 0.33
Plaster N/A N/A
Other N/A N/A
New home purchase
New home purchase 42.15 6.23 14.8% 0.03 38.53 -3.48 -9.0% 0.33
Household assets
Mattresses/beds 57.52 3.18 5.5% 0.05 29.35 2.26 7.7% 0.04
Large electronics (TVs,
radios)
22.58 2.03 9.0% 0.09 2.27 0.03 1.2% 0.73
Tables 17.69 0.28 1.6% 0.31 5.10 0.12 2.4% 0.58
Chairs 51.16 2.93 5.7% 0.05 6.76 0.25 3.6% 0.28
Cabinets (cupboards) 17.19 2.06 12.0% 0.01 N/A
Mobile phones 22.15 0.66 3.0% 0.26 9.28 0.12 1.3% 0.73
Solar panels 15.73 -0.23 -1.5% 0.35 5.37 0.14 2.7% 0.67
Bicycles 5.71 0.34 5.9% 0.14 19.10 -0.07 -0.4% 0.77
Bodas/Motorcycles 29.06 1.20 4.1% 0.31 29.54 -4.94 -16.7% 0.04
62 Some components have insufficient data for estimation of treatment effects. This occurs when fewer than 2.5% of
households own that item, and after winsorization there are no positive values remaining for that item.
61 The ITTs of the asset items do not sum to the ITT of total assets since, as with consumption, we winsorized the
individual components of assets for this appendix but winsorized the aggregate asset index for the top-level estimates.
54
Pigs 1.04 0.16 14.9% 0.15 15.46 0.74 4.8% 0.42
Chickens 18.21 1.83 10.0% 0.00 16.16 1.67 10.4% 0.00
Goats 8.58 1.52 17.7% 0.02 29.20 3.51 12.0% 0.00
Sheep 8.95 -0.48 -5.4% 0.27 10.70 1.52 14.2% 0.03
Cattle 238.55 -3.36 -1.4% 0.31 254.97 22.48 8.8% 0.04
Agricultural inputs 1.02 0.33 32.7% 0.00 6.97 0.86 12.3% 0.18
Other 19.35 0.91 4.7% 0.14 25.38 1.98 7.8% 0.00
Business assets (share owned by HH)
Buildings 1.11 0.13 12.0% 0.25 0.00 0.00
Vehicles 4.73 1.08 22.8% 0.12 1.31 -0.14 -10.7% 0.43
Machines 1.77 0.57 32.4% 0.01 0.47 0.05 9.6% 0.43
Farming tools 0.24 0.12 51.4% 0.00 2.46 -0.35 -14.4% 0.04
Tools for carrying
materials
1.47 0.59 40.2% 0.00 2.69 0.02 0.8% 0.77
Furniture 0.43 0.28 65.1% 0.00 0.84 -0.05 -5.7% 0.67
Chickens N/A 0.17 0.00 -0.3% 0.83
Sheep & goats N/A N/A
Pigs N/A N/A
Stock 19.54 9.41 48.2% 0.00 23.93 0.69 2.9% 0.73
Other 0.12 0.12 93.7% 0.00 1.91 0.65 34.3% 0.03
Household savings
At home 0.91 0.50 54.2% 0.00 2.06 -0.58 -28.1% 0.01
With relative N/A N/A
With neighbor/friend 0.05 0.00 2.1% 0.40 N/A
With shopkeeper N/A N/A
With SACOs, ROSCAs,
other savings groups
16.94 4.53 26.7% 0.00 15.67 4.91 31.3% 0.00
With NGO N/A 0.02 0.19 826.3% 0.00
In bank account 1.41 0.06 4.2% 0.35 N/A
With M-PESA/mobile
money
3.95 1.24 31.4% 0.00 N/A
With MFIs N/A N/A
Other source N/A N/A
Cash 1.56 0.80 51.3% 0.00 2.38 0.42 17.6% 0.01
Business savings (share owned by HH)
At home N/A N/A
At the business N/A N/A
With relative N/A N/A
With neighbor/friend N/A N/A
With shopkeeper N/A N/A
With SACOs, ROSCAs,
other savings groups
N/A N/A
With NGO N/A N/A
In bank account N/A N/A
With M-PESA/mobile
money
N/A N/A
55
With MFIs N/A N/A
Other source N/A N/A
Household loans (amount remaining)
From
friend/neighbor/relative
3.10 -0.34 -11.0% 0.08 2.99 -0.09 -3.0% 0.73
From trader/shopkeeper 7.96 1.39 17.4% 0.14 0.01 0.00 20.3% 0.21
From work place N/A N/A
From savings group 5.30 0.92 17.4% 0.07 14.10 -0.18 -1.3% 0.73
From moneylender N/A N/A
From mobile banking 1.45 0.20 13.6% 0.09 N/A
From NGO 16.93 0.79 4.7% 0.32 N/A
From MFI N/A N/A
From private bank N/A N/A
From government bank N/A N/A
From other source 8.36 0.71 8.5% 0.26 N/A
Business loans (amount remaining owed by HH)
From
friend/neighbor/relative
N/A 0.19 -0.04 -23.3% 0.06
From trader/shopkeeper 0.13 0.00 2.1% 0.39 N/A
From work place N/A N/A
From savings group 0.09 0.07 73.7% 0.00 2.24 -0.01 -0.4% 0.83
From moneylender N/A N/A
From mobile banking N/A N/A
From NGO N/A N/A
From MFI N/A N/A
From private bank N/A N/A
From government bank N/A N/A
From other source N/A N/A
56
Appendix G: Differences in treatment effects on types of assets
for households that received larger versus smaller grants
Households in Kenya that received large grants ($150/household) had significantly larger treatment effects
on net asset value than households that received smaller grants ($50/household). In Table G1 we show how
treatment effects on the individual components of net assets differ across households that received small
versus large grants.
Larger treatment effects for households that received large grants are driven primarily by new home
purchases, household assets (especially large electronics, furniture, chickens, and goats), savings with
community savings groups, and business stock. Households that received large grants also had significantly
larger treatment effects for business assets and savings, though the magnitudes are smaller. The only asset
type where small-grant households had significantly larger treatment effects than large-grant households
was construction of new roofs.
Table G1: Treatment effects on asset items, small-grant versus large-grant households, Kenya only
63
Item Small grants Large grants
Avg in
Control
ITT
%
Increase
q-val
Avg in
Control
ITT
%
Increase
q-val
Home improvements
Construction of new home N/A N/A
New room 4.61 1.38 29.9% 0.26 5.68 -0.04 -0.7% 0.59
New roof 1.45 1.11 76.3% 0.01 1.83 0.18 9.8% 0.37
Wall N/A N/A
Latrine N/A N/A
Plumbing N/A N/A
Wiring N/A N/A
Repair N/A N/A
Floor N/A N/A
Plaster N/A N/A
Other N/A N/A
New home purchase
New home purchase 43.27 1.43 3.3% 0.61 41.58 9.38 22.6% 0.01
Household assets
Mattresses/beds 56.45 3.66 6.5% 0.38 58.06 3.26 5.6% 0.11
Large electronics (TVs, radios) 22.70 -3.15 -13.9% 0.35 22.53 4.63 20.5% 0.00
Tables 17.97 0.33 1.8% 0.60 17.54 0.32 1.8% 0.44
Chairs 50.30 1.73 3.4% 0.59 51.60 3.79 7.4% 0.05
Cabinets (cupboards) 16.95 1.87 11.0% 0.35 17.32 2.47 14.2% 0.02
Mobile phones 21.68 -0.05 -0.2% 0.77 22.39 1.13 5.0% 0.23
Solar panels 14.16 -0.89 -6.3% 0.60 16.54 0.19 1.2% 0.57
63 See Appendix F for why the ITTs of individual components do not sum to the ITTs of net assets, and also for how to
interpret q-values.
57
Bicycles 5.13 0.58 11.3% 0.35 6.00 0.22 3.6% 0.36
Bodas/Motorcycles 27.72 4.53 16.3% 0.40 29.75 -0.41 -1.4% 0.57
Pigs 1.72 -0.02 -0.9% 0.77 0.69 0.24 34.5% 0.09
Chickens 17.58 1.62 9.2% 0.21 18.54 1.93 10.4% 0.01
Goats 9.20 -0.07 -0.7% 0.77 8.25 2.46 29.8% 0.00
Sheep 9.71 -1.95 -20.1% 0.35 8.56 0.30 3.5% 0.47
Cattle 263.34 -15.09 -5.7% 0.40 225.84 3.42 1.5% 0.46
Agricultural inputs 0.83 0.48 57.3% 0.04 1.11 0.30 26.7% 0.01
Other 20.24 0.71 3.5% 0.60 18.90 1.10 5.8% 0.12
Business assets (share owned by HH)
Buildings 0.98 0.08 7.7% 0.60 1.18 0.18 15.6% 0.25
Vehicles 4.37 0.75 17.1% 0.60 4.92 1.36 27.8% 0.12
Machines 1.62 0.20 12.6% 0.60 1.84 0.80 43.6% 0.00
Farming tools 0.26 0.06 22.7% 0.47 0.22 0.16 69.8% 0.00
Tools for carrying materials 1.45 0.39 27.1% 0.09 1.48 0.71 47.9% 0.00
Furniture 0.42 0.14 33.7% 0.33 0.44 0.36 81.9% 0.00
Chickens N/A N/A
Sheep & goats N/A N/A
Pigs N/A N/A
Stock 16.16 8.39 51.9% 0.02 21.27 10.23 48.1% 0.00
Other 0.13 0.08 57.3% 0.26 0.12 0.13 113.6% 0.00
Household savings
At home 1.00 0.31 30.8% 0.38 0.87 0.59 68.3% 0.00
With relative+A6A61:A68 N/A N/A
With neighbor/friend 0.06 -0.02 -28.7% 0.52 0.04 0.01 21.4% 0.24
With shopkeeper N/A N/A
With SACOs, ROSCAs, other
savings groups
17.58 2.04 11.6% 0.49 16.62 5.79 34.8% 0.00
With NGO N/A N/A
In bank account 1.41 -0.38 -27.1% 0.40 1.40 0.29 20.4% 0.23
With M-PESA/mobile money 3.79 -0.07 -1.8% 0.77 4.04 1.93 47.7% 0.00
With MFIs N/A N/A
Other source N/A N/A
Cash 1.61 0.27 16.6% 0.20 1.54 1.09 71.0% 0.00
Business savings (share owned by HH)
At home N/A N/A
At the business N/A N/A
With relative+A6A61:A68 N/A N/A
With neighbor/friend N/A N/A
With shopkeeper N/A N/A
With SACOs, ROSCAs, other
savings groups
N/A N/A
With NGO N/A N/A
In bank account N/A N/A
With M-PESA/mobile money N/A N/A
58
With MFIs N/A N/A
Other source N/A N/A
Household loans (amount remaining)
From
friend/neighbor/relative
2.98 -0.38 -12.8% 0.40 3.16 -0.31 -9.9% 0.20
From trader/shopkeeper 7.84 4.24 54.0% 0.25 8.02 -0.14 -1.7% 0.57
From work place N/A N/A
From savings group 4.64 0.74 15.9% 0.55 5.65 0.94 16.6% 0.13
From moneylender N/A N/A
From mobile banking 1.15 0.55 48.1% 0.09 1.60 0.02 1.4% 0.57
From NGO 18.61 -1.87 -10.1% 0.60 16.07 2.28 14.2% 0.23
From MFI N/A N/A
From private bank N/A N/A
From government bank N/A N/A
From other source 7.47 1.13 15.1% 0.59 8.82 0.57 6.4% 0.44
Business loans (amount remaining owed by HH)
From
friend/neighbor/relative
N/A N/A
From trader/shopkeeper 0.14 -0.02 -11.1% 0.60 0.12 0.01 8.1% 0.44
From work place N/A N/A
From savings group 0.08 0.03 38.2% 0.40 0.09 0.08 90.4% 0.00
From moneylender N/A N/A
From mobile banking N/A N/A
From NGO N/A N/A
From MFI N/A N/A
From private bank N/A N/A
From government bank N/A N/A
From other source N/A N/A
59
Appendix H: Treatment effects by age of household head
In Figure H1 we show treatment effects disaggregated by the age of the head of household.
Figure H1: Treatment effects by age of household head
60
Appendix I: Average outcomes by business type, country, and
cash transfer size
Figure I1: Average outcomes by business type & country
Figure I2: Average outcomes by business type & cash transfer size (Kenya only)
61
Appendix J: Household groupings
Table J1: Household groupings
# HHs offered the program (ITT population) 14,772
# HHs from original 60 14,311
# HHs from alternates 461
# HHs received grant 1 (TOT population) 14,130
64
# HHs from original 60 13,710
# HHs from alternates 420
# HHs received grant 2 13,839
# HHs from original 60 13,438
# HHs from alternates 401
64 16 households in the grant transfer dataset do not have a HH rank in the baseline data, so we include them in the
count of the original 60.
62
Appendix K: Cohort-wise effect sizes in Uganda for households
surveyed pre-lockdown vs post-lockdown
Table K1: Cohort-wise effect sizes, Uganda only
63
Appendix L: Treatment effects in this study versus Sedlmayr et
al (2020)
Table L1: Comparison of this evaluation’s vs Sedlmayr et al study outcomes
This study, full sample
This study, Uganda
only
Sedlmayr et al (2020)
USD per
HH
65 SD USD per HH SD
UGX per
capita
USD per HH
66 SD
Consumption
(per month)
9.88
(1.70)
0.09
(0.02)
3.87
(1.84)
0.04
(0.02)
26,061
(11,248)
3.94
(1.70)
0.07
Net assets
40.49
(13.64)
0.05
(0.02)
15.60
(19.50)
0.02
(0.02)
16,343
(5,449)
29.67
(9.89)
0.12
Standard errors in parentheses
66 We convert the per capita values reported in Sedlmayr et al (2020) to per household values. In Sedlmayr et al (2020),
the preferred treatment effect estimates for the Village Enterprise microenterprise program are 26,061 UGX per capita
per year for consumption and 16,343 UGX per capita for assets. We use the 2017 nominal exchange rate of 1 USD to
3,580 UGX and average household size of 6.5 members (as observed in our Uganda sample) to make this conversion.
65 All USD values are nominal
64
Appendix M: Evaluation team
Frida Njogu-Ndongwe - East & Southern Africa Regional Director
Frida is the East & Southern Africa Regional Director, based in Nairobi. Frida is a medical doctor, public
health specialist, entrepreneur, and development and management consultant. Prior to IDinsight, Frida was
the Director of Programs at the Center for Health Solutions – Kenya, a consultant at McKinsey & Company’s
Africa Delivery Hub, and CEO and co-founder of Afyakit Technologies, a Kenya-based start-up that provides
data and evidence in health. She has also served as a consultant for governments, NGOs, civil society and
multilateral agencies across a multitude of sectors including health, human rights, gender, housing, youth
employment, city design, and other areas.
Frida holds a Master of Public Health degree from the University of Auckland and a Bachelor of Medicine
and Bachelor of Surgery (MBChB) degree from the University of Nairobi.
Jeff McManus - Senior Economist
Jeff McManus is a Senior Economist on the Technical Team at IDinsight, based in Lusaka, Zambia. Jeff
oversees the technical design and analysis of impact evaluations, process evaluations, and machine
learning applications at IDinsight. Jeff co-founded the Technical Team at IDinsight and served as its Director
for 3 years, during which he set the organizational strategy for the technical rigor of the company’s
portfolio. He continues to mentor and manage evaluation specialists on the Technical Team. Jeff holds a
bachelor’s degree in Economics from Swarthmore College and an MPA in International Development
(MPA/ID) from Harvard Kennedy School.
Christy Lazicky - Consultant Economist
Christy Lazicky is a former Manager at IDinsight. Christy brings more than five years experience in the
international development impact evaluation field. Prior to joining IDinsight, she worked as a Research
Manager at London Business School leading the operationalization and implementation strategy of three
impact evaluations in Uganda, Rwanda, and Ghana. She has also worked with MIT’s Abdul Latif Jameel
Poverty Action Lab (J-PAL) in South Africa and Mathematica Policy Research in Washington, DC. Christy
holds a MPA in International Development (MPA/ID) from Harvard Kennedy School of Government and a
bachelor’ s degree in Mathematics from Dartmouth College.
Miguel Angel Jimenez Gallardo - Manager
Miguel is a manager at IDinsight. Before joining IDinsight, Miguel was Country Economist at the
International Growth Centre in Mozambique. Before that, he worked in the Mexican Government and for
various international organisations and NGOs, including the World Bank, and Innovations for Poverty
Action.
Miguel Angel holds a Master in International Economics and a Master in Development Studies from the
Graduate Institute of International and Development Studies (IHEID).
65
Winfred Kananu - Associate
Winfred is an Associate at IDinsight, based in Nairobi, Kenya. She supports projects through research, field
team management, and statistical analysis.
Prior to IDinsight, Winfred worked as a Junior Strategy Consultant where she helped clients improve their
operational efficiency and impact.
Winfred is an Economist and a Statistician by training.
66
Appendix N: Evaluation Discussion
This evaluation has been conducted with the support of the following stakeholders:
1) Village Enterprise
2) Wellspring Philanthropic Fund
3) The Foreign, Commonwealth and Development Office (FCDO)
4) United States Agency for International Development - Development Innovation Ventures
(USAID-DIV)
5) Global Development Incubator (GDI)
6) Instiglio
This publication has been approved by all the different stakeholders. An initial draft of this publication was
subjected to two different review rounds during which the different stakeholders had the opportunity to
comment on the findings, recommendations, and lessons learned during the evaluation. This final
document incorporates all the feedback that was received from the various stakeholders.
The evaluation design was discussed and agreed upon with all the relevant stakeholders before conducting
data collection. Various rounds of discussions were conducted between all stakeholders to agree on the
evaluation methodology (Section 2). All differences of opinion were fully acknowledged and the final
evaluation design was discussed and approved by all stakeholders.