Improving Health Service Delivery Through
Community Monitoring and Non-Financial
Awards: Report to USAID
January 2015
Acknowledgements
The interventions being evaluated have been developed in partnership with the Gov￾ernment of Sierra Leone’s Decentralization Secretariat (DecSec) and have been funded
by the World Bank’s Decentralized Service Delivery Program (DSDP) and Institutional
Reform and Capacity Building Program (IRCBP). The interventions are being imple￾mented by three non-governmental organizations: the International Rescue Committee,
Concern Worldwide, and Plan International.
Data collection, staffing, clinic mapping, piloting activities and other operating costs
related to the impact evaluation were funded by the International Growth Centre (IGC),
Namati, the World Bank, and the United States Agency for International Development
(USAID). Innovations for Poverty Action (IPA) and the Centre for the Study of African
Economics (CSAE) at Oxford provided logistical support, and managed the development
of survey instruments, data collection, and analysis.
This report was written by IPA, with technical oversight from Oeindrila Dube (New York
University), Johannes Haushofer (MIT-JPAL) and Bilal Siddiqi (Oxford University).
Gieltje Adriaans, Ali Ahmed, Alix Bonargent, Fatu Conteh, Sarah Dykstra, Caroline Fry,
Niklas Heusch, Anthony Mansaray, and Samantha Zaldivar-Chimal provided excellent
research assistance, and Mike Duthie, Joshua McCann, Andrew Tedesco, and Abebual
Zerihun provided in-country supervision.
1
Contents
Acknowledgements 1
List of figures 3
Abbreviations 4
1 Introduction 5
2 Program and evaluation overview 6
2.1 The health sector in Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Incentivizing health service delivery: a review of the evidence . . . . . . . 6
2.3 Social accountability interventions . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Evaluation design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.5 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3 Results 11
3.1 Key outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 Overview of the main results . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.3 Utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.4 Immunization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.5 Institutional deliveries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.6 Antenatal care visits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.7 Illegal fees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.8 Nurse absenteeism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.9 Staff attitude . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.10 Anthropometric outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.11 Maternal and under-five mortality . . . . . . . . . . . . . . . . . . . . . . 25
4 Cost-effectiveness analysis 27
5 Outreach 30
5.1 Partners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
6 Conclusion 32
References 33
2
List of Figures
1 Time trend. % of households that had visited the target clinic in the
previous month . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2 Treatment effects. % of households that had visited the target clinic in
the previous month . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3 Time trend. Average vaccination scores . . . . . . . . . . . . . . . . . . . 15
4 Time trend. Proportion of households delivering in government clinics . . 17
5 Treatment effects. Proportion of households delivering in government clinics 17
6 Time trend. Proportion of households paying an illegal fee at least once . 20
7 Treatment effects. Proportion of households paying an illegal fee at least
once . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
8 Time trend. Proportion of households experiencing absenteeism at least
once . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
9 Treatment effects. Proportion of households experiencing absenteeism at
least once . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
10 Time trend. Average attitude score . . . . . . . . . . . . . . . . . . . . . . 23
11 Treatment effects. Current nutritional status: % of children wasted . . . . 25
12 Time trend. % of households experiencing the death of an under-five child 26
13 Time trend. % of households experiencing the death of a mother during
childbirth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3
Abbreviations
ANC Antenatal Care
CM Community Monitoring
CSAE Centre for the Study of African Economics
DecSec Decentralization Secretariat
DSDP Decentralized Service Delivery Program
GIS Geographic Information Systems
GoSL Government of Sierra Leone
IGC International Growth Centre
IPA Innovations for Poverty Action
IRCBP Institutional Reform and Capacity Building Program
MoHS Ministry of Health and Sanitation
NFA Non-Financial Award
NPS National Public Services survey
OLS Ordinary Least Squares
PNC Postnatal Care
SLDHS Sierra Leone Demographic and Health Survey
USAID United States Agency for International Development
4
1 Introduction
Over the past decade, the Government of Sierra Leone (GoSL) has made a concerted
effort to improve health care services throughout the country, including an ambitious
free health care initiative launched in 2010. Key to the success of the initiative was
strengthening the weak incentives faced by frontline health providers—for example, in
early 2010, a large share of health staff were compensated entirely through informal
fee payments from users and margins from drug sales. To alleviate basic resource con￾straints, the GoSL increased worker salaries and the flow of resources to clinics na￾tionwide, and sought to introduce non-financial incentive schemes, which recent studies
have shown to be highly effective in improving worker performance in a range of envi￾ronments (Bj¨orkman & Svensson, 2009, Kosfeld & Neckermann, 2011, Ashraf, N. et al.,
2011). However, could such schemes be applied to Sierra Leone’s health sector, and if
so, what types of non-financial incentives would work best?
To answer this question, the GoSL, in conjunction with the World Bank, the Centre
for the Study of African Economies, and Innovations for Poverty Action, launched a
randomized controlled trial to test the effectiveness of two innovative interventions using
non-financial incentives to improve health care outcomes. The first intervention involved
community monitoring of health clinics through the use of health scorecards and collabo￾rative meetings between community members and health staff. The second intervention
used non-financial awards, such as public commendations, with the aim of improving
worker motivation and promoting greater efficiency within health clinics. This report
outlines findings on key indicators of the randomized evaluation.
Since the beginning of 2014, Sierra Leone has been facing an unprecented Ebola outbreak
that revealed the limitations of the health care system and has already deeply affected
the public health sector. The outbreak notably highlights a chronic lack of trust in
the health authorities and more broadly in the public sector. In this respect, initiatives
aiming at reinforcing accountability at the local level and improving relationship between
the community and health care providers are likely to play a central role in the rebuilding
of the health care system in the post-Ebola landscape.
5
2 Program and evaluation overview
2.1 The health sector in Sierra Leone
Sierra Leone ranks close to last in the UN’s Human Development Index and faces some
of the lowest health indicators in the world. The health system is plagued by chronic
worker absenteeism, low utilization of health clinics for essential services such as as￾sisted deliveries1
, and under-provision of critical basic health care such as vaccinations
of children under five2
. This poor service delivery is in part due to unrealistically weak
incentives for service providers, which do not align the interests of public health staff
with the interests of local communities. Often, public health staff work on a voluntary
basis, or are compensated at very low levels, and use margins from drug sales and in-kind
community contributions to supplement their income (according to MoHS). In addition,
many clinics do not receive operating budgets and finance investments such as facility
repairs out of drug profit margins and by charging informal user fees, even to women
and children under five, who are supposed to receive free health services by law.
Over the past decade, the GoSL has made a concerted effort to improve health care ser￾vices within the country. Alongside a national decentralization program introduced in
2004, the Government of Sierra Leone launched an ambitious policy in 2010 to institute
free healthcare for pregnant women, new mothers and children under-five. The policy
abolished user fees, while at the same time raising workers’ salaries. The reforms were
introduced along with programs - performance-based financing and interventions relying
on non-financial incentives - aiming at improving oversight of health workers and chang￾ing underlying incentive systems in order to address the challenges faced by the public
health sector (illegal fee-charging, nurse absenteeism and poor health outcomes).
2.2 Incentivizing health service delivery: a review of the evidence
Recent experiments have highlighted the power of non-financial incentives to improve
the performance of individuals performing public service. Such mechanisms utilize the
power of social sanctions and rewards, through mechanisms such as public recognition
and community monitoring, and are attractive for three reasons. First, they are cost￾effective: both theory and empirical evidence suggest that the likely channels through
which these interventions operate are concerns for status and reputation (Tirole, 1996;
Besley and Ghatak, 2008; Frey and Neckermann, 2010; Banerjee et al., 2004; Yared,
2009), and it has been shown that such concerns drive behavior even in the absence of
material benefits (Kosfeld and Neckermann, 2011). Thus, the deeply rooted human desire
1Results from the 2011 Integrated National Public Service Survey and the 2008 Demographic and
Health Survey
2The 2008 Sierra Leonean Demographic and Health Survey reveals very low level of vaccination
coverage among children under five years of age. However, preliminary results from the 2013 Demographic
and Health Survey suggest that immunization has substantially improved over recent years.
6
for recognition can act as a free-of-charge incentive that can replace financial incentives.
Second, non-financial awards avoid the potential crowding-out effect that performance￾based payment schemes can have on intrinsic motivation (B´enabou and Tirole, 2003;
Camerer and Hogarth, 1999; Kreps, 1997). This concern carries weight because intrinsic
motivation has been shown to be a major component in the job motivation of health
workers in developing countries (Mathauer and Imhoff, 2006; Stilwell, 2001). Finally,
non-financial incentives also obviate the significant monitoring and other administrative
costs that are incurred by performance-based incentive schemes, and can operate even
in the presence of limited liability and moral hazard (Kohn, 1999; Besley and Ghatak,
2008).
In the area of public recognition, Ashraf et al. (2011) find that social recognition was a
more powerful performance motivator and more cost-effective than both financial com￾pensation and voluntary contracts. Moldovanu et al. (2007) provide a simple theoretical
framework showing that under simple assumptions (awards increase the utility of the
recipients, and decrease that of non-recipients; and the utility of receiving the award
decreases in the number of workers who receive it), awards should increase the effort
provided by workers. In line with this prediction, Kosfeld & Neckermann (2011) show
in a randomized field experiment that purely symbolic awards boost the productivity of
students by 12%. Since material benefits from the awards were ruled out, this study also
showed that the performance improvement was driven by social recognition and status
alone. Similarly, Markham et al. (2002) showed that a public recognition program in a
U.S. manufacturing firm substantially improved employee attendance.
In the area of community monitoring, Bj¨orkman and Svensson (2009) and Bj¨orkman￾Nyqvist, de Walque and Svensson (2014) observe remarkable improvements on a vector of
health outcomes from training communities to effectively monitor their health providers
in Uganda, in both the short and long run. Similary, Duflo et al. (2012) find that training
PTA committees in India to effectively monitor schools reduced teachers’ absenteeism
and increased student test scores.
However, the finding that such non-financial incentives work leaves a crucial question
unanswered: did community monitoring improve clinic performance because it was a
‘bottom-up’ intervention which made clinic personnel socially accountable to their im￾mediate neighbors? Or did it work because clinic performance was being evaluated per
se, without it being necessary that this evaluation was performed within the commu￾nity? Put differently, might the incentives as well have been ‘top-down’ rather than
bottom-up? The answer to this question is important because top-down incentives are
potentially much cheaper and more efficient than bottom-up incentives. Furthermore,
one the few studies that compares top-down to bottom-up monitoring (Olken, 2007) in
the context of road provision in Indonesia, finds evidence in favor of top-down audits of
public works. Yet there is little data about whether the same conclusions would hold in
resource-poor environments such as Sierra Leone, where top-down monitoring capacity
is limited.
7
2.3 Social accountability interventions
In 2011, the GoSL requested support under the World Bank’s Decentralized Services
Delivery Project (DSDP) for social accountability activities designed to monitor per￾formance and motivate workers within the free health care initiative. As part of this
effort, the GoSL partnered with Centre for the Study of African Economies (CSAE),
Innovations for Poverty Action (IPA), and researchers from Oxford, NYU and JPAL to
conduct a rigorous, scientific investigation of two innovative social accountability inter￾ventions designed to incentivize health workers, stimulate demand for health services,
and strengthen the link between providers and communities. The interventions were
designed and implemented by the GoSL’s Decentralization Secretariat (DecSec) along
with three international NGOs—the International Rescue Committee, Concern World￾wide and Plan International. The MoHS has remained a partner throughout.
The first intervention, involving non-financial awards, facilitates yardstick competition
among groups of maternal and child health clinics, and rewards workers at both the
best performing and the most improved facilities. At baseline, a relative ranking of
clinics by district on key measures of performance, such as worker absenteeism, staff
attitude and charging of illegal fees, and utilization for maternal and child health ser￾vices was calculated. The competition, entitled “Respect Pass Money”, was advertised
through district-wide clinic meetings, posters on clinics and through individual meet￾ings at clinics held by trained facilitators from partnering NGOs. Facilitators discussed
clinic performance and the competition with clinic staff; however, the indicators used
to produce clinic rankings were not revealed to clinics or the public, in order to prevent
‘teaching to the test.’ Clinics were revisited three times throughout the course of the
9-month competition in order to sustain interest and at the end of the nine months, an
audit of reported clinic results was conducted, and any clinic found to be misrepresent￾ing information were disqualified. At endline, those clinics (i) which performed best in
absolute terms on these indicators, and (ii) which showed the greatest improvement over
the course of the intervention, were declared winners and received non-financial awards.
Specifically, staff at winning clinics received certificate of commendation and a wall clock
for the clinic from high-ranking politicians at a public ceremony.
The second intervention builds upon evidence from the recent community monitoring ini￾tiative in Uganda. A ‘bottom-up’ community monitoring intervention introduced health
scorecards that provide information regarding the state of health care in each commu￾nity, and facilitated interface meetings between community members and health facility
staff. Prior to the meetings, communities were surveyed on their perception of service
provision by the local clinic, and focus groups held to discuss the state of health care
in the community. During the interface meetings, information about the state of health
care was disseminated via a community scorecard and mutual commitments made to
improve services through a joint action plan addressing such areas as staff absenteeism,
maternal mortality and vaccination rates. Additional meetings were held one month,
three months and nine months after the initial meeting to review the joint action plan
8
and progress made since the previous meeting. This framework aimed to ensure partici￾patory decision-making and hold both health care workers and the community mutually
accountable, fostering increased access to and utilization of maternal and child health
services.
2.4 Evaluation design
The evaluation employed a randomized, controlled field experiment to assess the causal
impact of the interventions. Clinics and their catchment area were randomly selected to
participate in one of the interventions (they constitute the “treatment group”) or to act as
control. This methodology ensures that observed and unobserved characteristics likely to
affect health outcomes and clinics effectiveness are similarly distributed across treatment
and control groups before the intervention. Consequently, any difference in indicators
of interest between treatment and control groups observed after the intervention can be
interpreted as the effect of the intervention itself.
Two hundred and fifty-four health clinics in 4 districts of Sierra Leone were selected
to participate in the evaluation. The districts of Bo, Bombali, Kenema and Tonkolili
were selected to provide regional balance. Prior to selection, a mapping exercise of
330 clinics and their catchments was undertaken in order to select clinics based on the
least amount of overlap between catchment populations and minimize the possibility of
spillovers between treatment and control clinics. The starting point was conducting GPS
coding to track and establish the physical location of maternal and child clinics. This
involved mapping the list of villages in each facility’s catchment area and integrating it
with existing Geographic Information Systems (GIS) data for Sierra Leone, so that the
location and population of each facility’s designated catchment area could be established.
Eleven clinics were dropped due to the fact that their locations could not be verified.
The remaining 319 clinics were narrowed down to 254 clinics after eliminating clinics
based on distance criteria to minimize spillovers.
Using a non-bipartite matching algorithm created specifically for this project, the clinics
within the sample were split into triplets based on similar utilization and performance
characteristics gathered during baseline data collection. Clinics were then randomly
assigned within these triplets to participate in either the community monitoring inter￾vention, the non-financial awards intervention or act as a control, so that each group
comprise a third of the clinics. In total, the sample consisted of 254 MCHP and CHP
health clinics, 508 communities (two villages are selected in each clinic’s catchment) and
10 households per community, or 5080 households in total.
2.5 Data collection
Baseline data collection was performed in September 2011. The endline surveys were
conducted in May and June of 2013.
9
Four types of survey were administered – a clinic survey, community survey, household
survey and user-feedback survey. The surveys were conducted as follows:
1. Clinic Survey - All clinics participating in the evaluation were surveyed to assess
clinic services, resources and staffing, and underwent an audit of drug stocks and
registers.
2. Community Survey - A community survey was administered to leaders in each
of the 508 target villages. The questionnaire inquired into their communities’
remoteness, public health status and any on-going government and NGO projects.
3. Household Survey - All the households surveyed in the National Public Services
survey (NPS) 2008 (five per community) were included in the sample.
4. User-Feedback Survey - Fifteen randomly sampled individuals in two villages (within
2 miles of the clinic) were administered a user-feedback questionnaire to collect
information on recent health episodes, and feedback on service provision and sat￾isfaction
The four surveys were piloted in June 2011 in three districts that were not selected for the
intervention - Western Rural, Pujehun, and Port Loko. The pilot study helped to address
issues with the structure and content of the surveys as well as the audit verification which
was planned to verify the information provided by the clinic. For the collection of endline
data, the household survey and user-feedback survey were combined. The research
team ensured a high level of data quality by closely monitoring the data while surveys
were taking place. Extensive back-checking was performed and the incoming data was
examined on a regular basis for measurement error. Supervisors and monitors provided
feedback to enumerators and in-field refresher trainings were conducted as necessary.
Range and internal consistency checks were completed to yield a clean dataset.
10
3 Results
3.1 Key outcomes
The effectiveness of the interventions was assessed through indicators of maternal and
child health outcomes, clinic utilization, and the quality of services provided. More
precisely, eight key outcomes were specified: number of children completing first year of
required vaccinations, number of institutional deliveries, number of women completing
fourth antenatal care visit, whether fees are charged for maternal and under-five health
services, nurse absenteeism, staff attitude, number of maternal deaths, and number of
under-five deaths. Furthermore, the effects of the interventions on clinic utilization and
the nutritional status of under-five children were evaluated.
The analysis that follows focuses on these indicators. The first six indicators and clinic
utilization describe the quality and quantity of health care services; mortality and nutri￾tional outcomes refer to health outcomes directly. We run Ordinary Least Squares (OLS)
regressions at the household level in which the endline outcomes are compared across
the different groups (Control, Community Monitoring and Non-Financial Awards). The
analysis presented here utilizes data from the endline household survey and not the clinic
and baseline survey; as such, all results are based on cross-sectional regression analyses.3
The analyses of mortality and utilization are exceptions in this regard: repeat cross￾sectional regressions, including both baseline and endline data, were used as the sample
was slightly unbalanced at baseline. For all other outcomes presented in the report, the
sample was balanced at baseline.
3.2 Overview of the main results
Community Monitoring has promising effects on the quality of health services delivered
by the target clinics. The findings indicate that the intervention increased utilization
by the households of the catchment area and the proportion of pregnant women giving
birth in government clinics, reduced the prevalence of illegal fee charging and improved
the nutritional status of children under five (in the short run). As expected, there was
no impact observed on health outcomes varying in the longer run, such as stunting or
mortality. The Non-Financial Award intervention does not appear to have substantially
affected service provision and health outcomes.
The table below summarizes the results of the randomized evaluation. Each result is
discussed in further details below.
3Only half of the households included in the baseline sample - or 5080 households - were selected
to participate to the endline survey. Among those selected households, 87 % (4437 households) were
located by the survey teams and reinterviewed. When a household could not be found by the surveyors,
it was replaced by another household from the baseline sample which was not initially selected to be
reinterviewed. These replacement households were not taken into account in the following analysis.
11
Table 1: Overview. Treatment effects of the CM and NFA interventions
Outcome Community Monitoring Non-Financial Awards
Utilization Households 11% more likely
to have utilized the target
clinic in the previous month.
No change
Immunization* No change No change
Institutional deliver￾ies*
Households 10% more likely
to deliver in government clin￾ics.
No change
Antenatal care visits* No change No change
Illegal fees* Households 27% less likely to
have paid an illegal fee.
No change
Nurse absenteeism* No change Households are 69% more
likely to have encountered
nurse absenteeism.
Staff attitude* No change No change
Wasting Proportion of wasted children
is 44% lower, at 4.2% (com￾pared to 7.5 percent).
No change
Stunting No change No change
Mortality* No change No change
* included in the eight core indicators
3.3 Utilization
Summary: The period 2011-2013 saw a rapid rise in clinic utilization across all treatment
groups. In addition, the community monitoring intervention had a significant positive
impact on clinic utilization, with households being 11% more likely to have utilized the
target clinic in the previous month.
Ensuring that adequate care is sought from a skilled provider whenever medically nec￾essary is a crucial first step for improving health outcomes. Reduced absenteeism, fewer
illegal fees, and a good attitude of clinic staff, all outcomes which the interventions
aimed to obtain, are likely to affect the utilization of clinics. Clinic utilization is hence
a natural starting point for assessing the interventions.
12
Data on clinic utilization and care-seeking behavior is recorded in the household surveys.
Apart from vaccinations, antenatal care, and births, the surveys recorded for each episode
of illness and injury whether and where the affected household member obtained medical
care. From this information a dichotomous variable, equal to 1 if any household member
utilized the target clinic, was calculated. The effect of the interventions on this outcome
allows to draw inferences on the utilization of target clinics and the quantity of care
obtained. To analyze whether the interventions caused patients to simply take their
business to the target clinic (i.e. to utilize the target clinic instead of other clinics) or to
obtain a greater quantity of care from clinics in total (i.e. to utilize the target clinic when
they would not have utilized any clinic otherwise), both outcomes are also generated for
the utilization of government clinics (which include target clinics) and the utilization of
any type of clinic (which includes government clinics).
Figure 1: Time trend. % of households that had visited the target clinic in the previous
month
Time trends figures display the change in outcome mean - here the difference in the proportion of
households having visited the target clinic in the past month - overtime in the control group.
Figure 1 shows the time trend in target clinic utilization between 2011 and 2013. Target
clinic utilization increased dramatically: while in 2011 only 12.3% of households had
visited a target clinic in the previous month, 47.4% of households in the control group
had done so in 2013. This increase is likely to be driven by Sierra Leone’s provision of
free health care for under-five children and pregnant and lactating mothers. The overall
13
utilization of government clinics (which includes target clinics) increased similarly. These
findings suggests that the increase in target and government clinic utilization is driven
by a higher clinic utilization overall: in 2013, households visited a clinic when they would
not have done so in 2011. However, this does not exclude the possibility of substitution
towards target clinics among treatment households.
Figure 2: Treatment effects. % of households that had visited the target clinic in the
previous month
Figure 2 shows the effects of the interventions on clinic utilization at endline. Community
monitoring lead to a statistically significant 5.4 percentage-point increase in the number
of households that had visited the target clinic in the past month compared to the
control group. The NFA intervention did not lead to a significant increase in target
clinic utilization compared to the control mean of 47.1%.
3.4 Immunization
Average vaccination scores increased between 2011 and 2013. Neither intervention af￾fected child immunization.
According to information collected in the Sierra Leone Demographic and Health Survey
14
(SLDHS), only 68% of children aged 12 to 23 months were fully vaccinated4
in 2013.
Vaccination coverage for the first dose of vaccine is always above 95% of children for
BCG, DPT and Polio but the figures start declining with the subsequent doses (to
around 78% for the third doses of DTP and Polio). Thus, the challenge in reaching full
vaccination lies in ensuring that all children receive the complete immunization cycle for
each type of vaccine.
Figure 3: Time trend. Average vaccination scores
In the household questionnaire, information on vaccination history is collected for each
under two child living in the household. The date of each vaccination received is taken
from the child’s Growth Monitoring Card5
for 7 different vaccines: BCG, Polio, Pen￾tavalent (diphtheria, tetanus, whooping cough, hepatitis B and Haemophilus influenzae
type b), Pneumococcal (PCV), Rotavirus (RVV), Measles and Yellow Fever. A variety
of household level indicators are computed from the vaccination information collected at
the individual level. The results here presented are the average vaccination scores (on
a scale from 0-7) for children aged 12 to 23 months. The vaccination score counts the
4The World Health Organization guidelines, a child is fully vaccinated when (s)he has received a
vaccination against tuberculosis (BCG), three doses each of diphtheria, pertussis, and tetanus (DPT)
and polio vaccines, and a measles vaccination by the age of 12 months.
5
70% of children aged 9-23 months had a Growth Monitoring Card, which surveyors asked to see;
there was no difference between control and treatment groups in the percentage of children that had a
Growth Monitoring Card.
15
number of vaccinations for which a child has received all required doses. Hence, a child
with a score of 7 has received all doses of all recommended vaccinations, whereas a child
with a score of 4 has only received all required doses for four (out of seven) recommended
vaccinations.
Average vaccination scores increased between 2011 and 2013, as Figure 3 shows. While
in 2011, children aged 12 to 23 months had on average completed the vaccination sched￾ule for 3.9 vaccines, children in the same age range in the control group had completed
the schedule for 4.9 vaccines in 2013. However, neither intervention produced an addi￾tional impact on vaccination scores or other measures of vaccination status. At endline,
there was no significant difference in outcomes between the control and the treatment
groups.
3.5 Institutional deliveries
Institutional deliveries increased by 10% between 2011 and 2013. Additionally, house￾holds in community monitoring villages were 10% more likely to deliver in government
clinics.
According to the 2013 SLDHS, 54% of childbirths occur in a health facility and 60%
are performed by a skilled providers (doctor, nurse, midwife or MCH aide). Provision
of safe delivery services is an essential condition to improve maternal and infant health
and especially to reduce maternal mortality at childbirth.
Each woman in the household who gave birth in the past year was administered a survey
module on childbirth, covering assistance received during labor and the final place of
delivery. The vast majority of households experienced only one childbirth in the past
year. A dichotomous variable, equal to 1 if the birth took place at a government clinic
and equal to 0 if the birth took place elsewhere, was hence calculated for each household.
For households experiencing more than one birth, the variable was set equal to 1 if at
least one birth took place at a government clinic.
Figure 4 shows a clear upward trend in the proportion of households delivering at gov￾ernment clinics between 2011 and 2013. While in 2011 70.5% of births took place in
government clinics, this figure increased to 80.4% in the control group in 2013.
The Community Monitoring intervention produced an additional and significant increase
in the proportion of deliveries occuring in government clinicsm, as Figure 4 displays:
households in CM villages are an additional 8 percentage-points more likely to deliver
in a government clinics. This corresponds to a 10% increase in the likelihood of giving
birth in a government clinic as a result of the CM intervention. In contrast, the NFA
intervention does not lead to a significant increase in the proportion of deliveries in
government clinics.
16
Figure 4: Time trend. Proportion of households delivering in government clinics
Figure 5: Treatment effects. Proportion of households delivering in government clinics
17
3.6 Antenatal care visits
Neither intervention affected antenatal care.
Another key determinant of maternal health and infant development is access to ade￾quate antenatal care (ANC). ANC aims to provide the necessary information to prepare
future mothers for pregnancy, birth and parenthood, and aims to help prevent and treat
problems encountered during the pregnancy. Consequently, monitoring the frequency,
the regularity, and the timing of the visits is extremely important. As an illustration,
deliveries at a health facility are common for women having received ANC, especially
for women having completed four or more ANC visits.
Women who received antenatal in the past month were asked detailed questions about
the location and the type of health facility attended, the number of visits for ANC in the
past month and the date of the last consultation. This information allows to identify the
women having completed three and four ANC visits among all pregnant women having
completed at least one ANC in the past month.
At baseline, 22% of the 85 women in the relevant sample had completed at least three
ANC visits; at endline this proportion was slightly higher at 27% in the control group,
but the difference was not statistically significant. Of the 85 women at baseline, 12% had
completed four ANC visits, while 21% of the 78 women in the control group at endline
sample had done so. This difference is also not statistically significant.
Neither intervention had an impact on the proportion of women completing at least
three ANC visits. This finding is robust - when considering the proportion of women
having completed all four ANC visits, neither intervention produced a significant in￾crease. However, one must keep in mind that this apparent absence of results could
potentialy driven by the limited sample size for this indicator.
3.7 Illegal fees
Illegal fee payments increased slightly between 2011 to 2013, by about 3%. However,
households in community monitoring villages were 27% less likely to have paid an illegal
fee.
Among the factors impeding access to health care services, the cost of treatment is the
first reason cited by Sierra Leonean women. Up to 80% of women report that paying for
treatment was the main obstacle preventing them from seeking treatment. In addition,
charging of illegal fees by health workers can weight on already strained relationship
between the community and the formal health system. The Free Health Care Initiative
was aimed precisely at eliminating fees for maternal and under-5 care, and we are able to
measure if the interventions influence illegal fees levied for these types of care. Payment
of illegal fees
18
In the module specific to target clinic attendance in the household questionnaire, infor￾mation regarding fees and in-kind payments was collected for each household member
for each type of episode. The Free Health Care Initiative provides virtually all treat￾ment free of charge for lactating and pregnant women and children under five. For the
purpose of this research, the payment of illegal fees is therefore defined as any payment,
either in cash or in-kind, given during a visit to the target clinic for childbirth, child
vaccination, antenatal and postnatal care (ANC/PNC), or treatment for illness/injury
of an under-5 child. At the household level, a dichotomous variable indicating if any
household member paid illegal fees in the past month is hence calculated.
In 2011, 22.9% of households paid at least once in the previous month an illegal fee for
a supposedly free visit; in 2013, 26.6% of households in the control group did so.
The Community Monitoring intervention had significant success in reducing the solicita￾tion of illegal fees, as Figure 7 shows. The proportion of visits in which an illegal fee was
solicited was 7.3 percentage-points lower, translating into a large 27% reduction in the
likelihood of households having to pay an illegal fee. In contrast, the NFA intervention
did not produce a significant change in the likelihood of having to pay an illegal fee. In
parallel, the control group experienced a slight upward trend between 2011 and 2013 in
the proportion of households experiencing illegal fees (Figure 6).
3.8 Nurse absenteeism
Absenteeism more than halved between 2011 and 2013. Households in NFA villages
were 71% more likely (although from a low base probability) to have encountered nurse
absenteeism.
The availability of health care providers at the facility is another essential determinant
of access to health care. Over a third of women interviewed in the SLDHS declare that
staff absenteeism is a serious problem in getting treatment.
The measure of absenteeism was derived from the information collected in the target
clinic specific module. It is a dichotomous variable indicating if any household members
experienced nurse absenteeism - i.e., came to the target clinic during regular hours
and found no staff present at least once - in the past month. It is possible that both
interventions may have increased clinic utilization. Since the probability of facing nurse
absenteeism could be positively correlated with clinic visits, it is important to account
for the number of visits to the target clinic. Regressions controlling for the number of
clinic visits were hence run as robustness checks; and yielded similar results.
Figure 8 shows the time trend in the proportion of households experiencing absenteeism
at least once between 2011 and 2013. The figures suggest that absenteeism more than
halved between 2011 and 2013.
Figure 9 examines the impact of the two interventions on nurse absenteeism. 2234
households reported at least one visit of a household member to the target clinic in the
19
Figure 6: Time trend. Proportion of households paying an illegal fee at least once
Figure 7: Treatment effects. Proportion of households paying an illegal fee at least once
20
past month. Only 5.8% of households in the control group ever found a nurse absent
in regular opening hours during any such visits of members. However, households in
the NFA group were a significant 4.1 percentage-points more likely (a 71% increase over
the control group, albeit from a low base) to have encountered nurse absenteeism at
least once during visits to the target clinic in the past month. This result is surprising
and needs further explanation. One possibility is that nurses increased the number
of visits to households in an effort to improve their clinic’s ranking and, as a result,
ended up being less present to the clinic. Another possibility is that learning about their
rankings demotivated poorly performing nurses even further, and had no effect on higher
performing nurses.
In contrast to the NFA intervention, the CM intervention did not have a significant
impact on nurse absenteeism.
3.9 Staff attitude
Staff attitude was positively viewed by patients in both 2011 and 2013. Neither interven￾tion affected perception of staff attitude.
Medical problems involving women and children often touch on sensitive issues. As
such, staff attitude may play a significant role in determining whether or not patients
feel comfortable enough to bring their medical issues to public health clinics. This in
turn, will affect the level of health services patients receive.
For each type of health episode, individuals who attended the target clinic in the past
month were asked to rank the attitude of the staff on a four-level Likert scale (ranging
from ‘very polite and respectful’ to ‘very rude and disrespectful’). An individual attitude
score was then computed by averaging the episode specific scores. At the household level,
the final outcome used is the average attitude score among household members having
attended the target clinic in the past month (see Figure 10). This average attitude score
ranges from 1 to 4, with a score close to 1 indicating a good performance for the staff of
the clinic.
Reported average satisfaction did not differ significantly between 2011 and 2013, or
between the treatment and control groups in 2013. Similarly, Figure 10 displays the
time trend of the average reported satisfaction with staff attitude between 2011 and
2013. Overall, households were very satisfied with the attitude of clinic staff, as the
average score of 1.3 suggests.
3.10 Anthropometric outcomes
The community monitoring intervention had a significant positive impact on the current
nutritional status of under-5 children. The proportion of children considered wasted is
44% lower in CM villages, at 4.2% (compared to 7.5 percent).
21
Figure 8: Time trend. Proportion of households experiencing absenteeism at least once
Figure 9: Treatment effects. Proportion of households experiencing absenteeism at least
once
22
Figure 10: Time trend. Average attitude score
Malnutrition and poor anthropometric outcomes are considered to be both a cause and
a consequence of poor health. Worms and frequent diarrhea, to name only two, lead to
nutrient deficiency and malnutrition in children. Malnutrition and poor health, already
undesirable on their own, in turn also affect a wide variety of socio-economic outcomes
such as labor supply and income.
Through improved medical care in target clinics, both interventions can plausibly affect
malnutrition in children. Anthropometric outcomes, such as height and weight make
it possible to draw inferences on children’s past and present nutritional status and can
thereby shed light on past and present (protein-energy) malnutrition. During the endline
data collection, information on height, weight, and arm circumference was hence collected
for all children under the age of five.
Following the WHO’s methodology, height-for-age, weight-for-height, and weight-for￾age were calculated for every child. These anthropometric indices are informative about
both acute and chronic malnutrition. Height-for-age, a measure of linear growth, is
considered to capture the effects chronic malnutrition. Weight-for-age, on the other
hand, is considered to be a measure of current nutritional status. Weight-for-height, a
composite measure of the two, is considered to be an overall indicator of a population’s
nutritional health. For all indices, the outcomes for each child were compared to those of
children of the same sex and age in the NCHS/WHO international reference population.
23
In the case of height-for-age, for example, the height of a child from the survey is
compared to that of a child of the same sex and age from the international reference
population; this simple comparison allows to establish whether a child in Sierra Leone is
tall or short by international standards. Similarly, for weight-for-height, the weight of a
child from the survey is compared to the weight of a child of the same sex and height from
the international reference population. Low weight-for-height hence simply means that
a child weighs little, considering how tall s/he is. As is common methodology, differences
in outcomes between Sierra Leonean children and children of the international standard
population are expressed as z-scores, allowing for easy comparison across age and sex.
A z-score states by how many standard deviations a given outcome for a Sierra Leonean
child differs from the same outcome for an average child (of same sex and age) in the
reference population.
Chronic malnutrition, as measured by height-for-age, is very severe among the chil￾dren surveyed. Children in the sample are on average 2.24 standard deviations shorter
than comparable children in the reference population; 57.5% of children are considered
stunted (i.e. more than 2 standard deviations shorter than a comparable child in the
reference population). These figures are more dramatic than those of the 2013 DHS,
which reports an average z-score for height-for-age of -1.4, and 37.9% of children stunted
for its nationally representative sample. Neither intervention had a significant impact
on height-for-age. This was to be expected, since height-for-age is a long-term mea￾sure of chronic malnutrition, unlikely to change quickly (the endline data collection was
conducted around a year after the launch of the intervention).
The current nutritional status of children is measured by weight-for-age. A child that is
more than two standard deviations below the reference mean is considered too thin for
her/his age, or wasted, a condition reflecting acute or recent nutritional deficit. 7.5% of
children in the sample are wasted, and 3.8% of children are severely wasted (z-score <
-3). The average z-score is 0.53. The 2013 DHS reports similar figures, finding 9.3% of
children wasted and 4.0% of children severely wasted. The average z-score reported by
the DHS is 0.0.
Figure 11 shows the study’s finding on the interventions’ impacts on the current nutri￾tional status of children, as measured by weight-for-age.
Wasting is reduced among children in villages subject to the community monitoring in￾tervention: children in CM villages weigh on average 0.175 standard deviations more
than similar children in control villages, this effect is statisticaly significant. Further￾more, the number of stunted children in CM villages is 3.3 percentage points lower.
Taking into account that 7.5% of children in the control group are classified as wasted,
this means that the CM intervention has reduced the number of wasted children by 44%.
It hence appears that community monitoring has improved the current and recent nu￾tritional status of under-5 children. The non-financial awards intervention, on the other
hand, does not appear to have improved children’s short-term nutritional status.
Weight-for-age is a composite measure of both weight-for-height and height-for-age. It
24
Figure 11: Treatment effects. Current nutritional status: % of children wasted
is hence affected both by acute malnutrition (wasting) and chronic malnutrition (stunt￾ing); a child can be underweight for her/his age because s/he is stunted, wasted, or
both. Weight-for-age is hence an overall indicator of a population’s nutritional health.
Of the children surveyed, 21.3% were underweight (z-score < -2), and 8.8% were severely
underweight (z-score < -3); the mean z-score was -0.82. Based on these figures, mal￾nutrition appears worse than in the 2013 DHS, which finds that 16.4% of children were
underweight, and 5.6% of children were severely underweight, while the average z-score
was also -0.8.
3.11 Maternal and under-five mortality
Maternal and under-five mortality reduced significantly between 2011 and 2013. Neither
intervention had a significant impact on either type of mortality.
Although progress has been made, under-five and maternal mortality remain high in
Sierra Leone; the 2013 DHS reports an under-five mortality of 156 deaths per 1,000 live
births. Improved delivery of public health services can plausibly affect both under-five
and maternal mortality. The impact of the interventions on both types of mortality is
hence evaluated. The household surveys collected information on deaths of under-five
children and mothers during childbirth. For each household, a dichotomous variable
25
(indicating if any death had occurred) was computed for both under-five and maternal
deaths.
Figures 12 and 13 display the time trends graphically. At the 2011 baseline, 5.1% of
households declared at least one death of an under-five child in the past six months,
for a total of 254 under-five deaths. At the 2013 endline, the proportion of households
declaring at least one death of an under-five child in the past six months was lower at
3.4%, for a total of 166 under-five deaths. In the baseline survey, 0.65% of households
experienced a maternal death during childbirth; 0.07% of households did so in the endline
survey.
Figure 12: Time trend. % of households experiencing the death of an under-five child
These reductions in mortality between baseline and endline are large and significant.
However, the size of the reductions in mortality does not differ between the treatment
and control groups; neither intervention had a significant (additional) impact on reducing
mortality.6
6An in-depth study of changes in mortality rates would require a larger sample size: the occurance of
death within the household is much rarer than the occurance births, pregnancies or illnesses, for example.
This relatively small sample size can only allow to capture important changes in mortality rates across
treatment and control groups.
26
Figure 13: Time trend. % of households experiencing the death of a mother during
childbirth
0.6
0.1
0
1
2 % of HH experiencing death of a mother during childbirth
Baseline (2011) Endline (2013)
4 Cost-effectiveness analysis
Comparison of Community Monitoring and Non-Financial Awards
The total direct cost of the intervention sums the costs of the implementing partner
NGOs (Concern, IRC, Plan International) and IPA’s costs in generating scorecards and
rankings, and monitoring the intervention. Implementing partner costs were derived
from partner financial reports,7 while expenses associated with the generation of score￾cards and clinics ranking, as well as IPA monitoring costs, were projected.8 The total
direct costs are divided into four categories: Personnel (35% of the total cost on average
for both interventions), Equipment and Supplies (16% of the total cost), Reimbursable
Expenses (28.5% of the total cost) and Other Costs (including direct project activity
7The implemented costs incurred by Plan International were projected using the financial reports
provided by Concern and IRC.
8Since the scorecards and the rankings have been generated using the data collected during the clinic
survey, it was not possible to dissociate the cost associated to the collection of information relative to
scorecards and rankings and total survey cost. The costs has been projected on the basis of a two weeks
data collection period involving three enumerators, and including survey training. The monitoring costs
were projected on the basis of one monitoring visit conducted by one surveyor in half of the community
monitoring intervention clinics.
27
costs, 20.5 percent).
For the Community Monitoring intervention, the total cost of implementation in four
districts is $513,436 (SLL 2,156,430,914). This figure is lower for the Non-Financial
Awards intervention, with $316,103 (SLL 1,327,631,041) spent in total in the implemen￾tation districts.
The cost-effectiveness analysis focuses on four main outcomes: institutional deliveries,
the likelihood of paying illegal fees, clinic utilization and the prevalence of wasting among
under-five children. More precisely, in the Community Monitoring treatment villages,
the number of households declaring that at least one pregnant woman had given birth
in a government clinic in the past year (among households declaring one birth in the
past year) increases by 10%. Households were 1% more likely to go to the target clinic.
Moreover, the number of households reporting that at least one member had paid illegal
fees in the past month (among households having visited the target clinic in the past
month) decreases by 28%. Finally, the number of children considered as wasted is re￾duced by 44%.
To determinate the absolute impact of the Community Monitoring intervention, pro￾jections of the catchment area population were made based on the data collected on
one hand, and the 2004 population census data on the other hand.9
In absolute terms,
the impact of the Community Monitoring intervention on the four main outcomes is as
follow.
• Institutional deliveries: 707 additional households have at least one woman who
has given birth in a government clinic rather than at home or in another health
facility (994 households with census data).
• Utilization: 2645 additional households are using the target clinic (3791 households
with census data).
• Illegal fees charging: the number of household declaring illegal fee charging de￾creases of 974 households (1390 households with census data).
• Wasting: the number of children considered as wasted is reduced by 1578 chil￾dren.10
Based on these impact figures and on the expenses reported by the implementing part￾ners, it is possible to compare effectiveness and cost of the intervention, for each positively
affected outcome:
• Institutional delivery: with $513,436 spent in the Community Monitoring treat￾9Using the data collected at endline, the catchment area population was deduced from the figures
provided by the clinics in the PHU questionnaire. Using the census data, the catchment population was
deduced from the average population by locality and average number of localities by health center for
each intervention district.
10Information on specific age groups is not readily available from the 2004 population census. For this
reason, there are no projections of the under five population in the catchement area based on the census
data.
28
ment clinics, institutional delivery in government clinics increased by 10%. This
represents a cost of $726 per additional household declaring having a woman having
given birth in a government clinic ($517 with census data).
• Utilization: target clinic utilization at the household level increased by 11 percent,
representing $194 per additional household ($135 with census data).
• Illegal fees charging: the $513,436 spent on the Community Monitoring interven￾tion resulted in a 28% decrease in illegal fees charging, which corresponds to $527
per household ($ 369 with census data).
• Wasting: wasting was reduced by 44 percent, representing $325 per child.
On the other hand, the Non-Financial Award intervention failed to substantially affect
the quality and the quantity of health services provided by public health centers and
had no impact on the nutritional status of under-five children. The comparison between
the two treatment groups is therefore straightforward: Community Monitoring is more
cost-effective than Non-Financial Awards in the context of this study.
Comparison with other interventions
To put in perspective the cost-effectiveness of the intervention, it is possible to compare
these results with the findings of other works studying the effect of community moni￾toring on the quality of health service provison and heath outcomes. Here, the analysis
will focus on the impact of several interventions on children’s malnutrition. As of now,
there is no study providing rigorous evidence on the impact of another intervention
implemented in Sierra Leone and focusing on health service provision and nutritional
outcomes. Therefore, it is not possible to put these results in perspective in the Sierra
Leonean context. An alternative is to assess how well the Community Monitoring in￾tervention performs internationally. One must keep in mind that malnutrition is a
multi-dimensional concept that can be defined in several ways, and that implementation
costs (wages, equipment, transport etc.) differ across countries.
In Indonesia, the evaluation of the Generasi program, relying on explicit – financial –
performance incentives and aimed at improving maternal and child health and education,
shows that the intervention reduced malnutrition by 15 percent11 (Olken et al., 2012).
Overall, the cost of pulling a child out of malnutrition was $ 385 to $ 528 per child. It is
worth noticing that the program also improved child mortality and education outcomes,
but did not affect institutional deliveries.
Another program based on Community Monitoring was the subject of an RCT in Uganda
(Bjorkman & Svensson, 2009). This intervention was shown to increase the weight-for￾age z-scores of children under 18 months of 0.14 units on average, which corresponds to
a reduction in risk of mortality of 7%. The study only provides cost-effectiveness figures
11Defined as weight-for-age z-score lower than -2 standard deviation from the standard distribution
mean.
29
based on the overall effect of the intervention: the estimated cost of averting the death
of a child under five is around $300.
5 Outreach
IPA will work closely with key partners who are in the best position to disseminate the
results of the experiment and implement the program at scale. Specifically, these are (1)
the Government of Sierra Leone, through the Decentralization Secretariat (DecSec) and
the Ministry of Health and Sanitation (MoHS), which have been central to the design
and implementation of the intervention to date, and are interesting in scaling up the
components that work; (2) The World Bank’s Justice for the Poor program, which has
through the Decentralized Service Delivery Project committed to funding the scaled-up
program; and (3) IRC, Concern Worldwide and Plan International, the implementing
partners operating in the four target districts. Details of each partner’s contribution and
future involvement are provided below.
In addition, the rich and extensive survey data collected as part of this research will be
used to inform post-Ebola response in Sierra Leone. The two rounds of the survey (2011
and 2013) provide a snapshot of Sierra Leone’s health system before and two years after
the injection of donor funding related to the Free Health Care (FHC) Initiative. As such,
these data offer insights into the prospects for post-Ebola recovery, as well as highlight
systemic weaknesses that were not addressed by the FHC Initiative. IPA is currently
working with researchers at the World Bank’s Development Impact Evaluation Initiative
(DIME) and Health, Nutrition and Population Global Practice to incorporate insights
from the data into the World Bank’s health sector strategy for Sierra Leone.
5.1 Partners
DecSec and the World Bank’s Justice for the Poor (J4P) program have significantly
contributed to the substance and design of this report. The J4P program has also
provided technical assistance to the implementing partners on the methodology and
design of the intervention, and has been involved in the data collection process. IPA has
regularly sent drafts of the reports for review to both institutions, and received feedback
on the empirical strategy and overall research framework.
Going forward, IPA and the Principal Investigators will leverage their links to these
institutions by making strategic presentations to decision-makers. In addition, IPA
will facilitate and support the decision-making process between the Decentralization
Secretariat and World Bank’s Justice for the Poor program. IPA has tailored the final
report to a policymaker audience by concentrating on the relevance of these results to
policy and displaying results in a non-technical manner.
30
The results will be presented to both the Decentralization Secretariat and the World
Bank. We intend these presentations to be a forum for further discussion and a medium
for ensuring that key partners understand the main takeaways from the evaluation. IPA
and the Principal Investigators will facilitate, support and participate in subsequent
decision-making process by the World Bank and the Decentralization Secretariat.
For wider dissemination, IPA will create a policy brief and presentations highlighting
the main results of the study, emphasizing the relative effectiveness of the Community
Monitoring intervention and discussing whether the cost effective analysis provides a
case for scale-up that will be used to inform a broader audience, including within the
Government of Sierra Leone and the World Bank, and outside. Finally, the results of this
experiment will be written up in a technical paper and presented to research audiences
through conferences and seminars.
31
6 Conclusion
This document summarizes the findings of the Community Monitoring and Non-Financial
Award interventions in 254 rural clinics in Sierra Leone. We find that the Community
Monitoring intervention has a significant impact on the “extensive” margin of fees paid
by pregnant women, mothers and children: fewer households declare having been charged
with illegal fees for treatment. In addition, we find that households in the catchments of
Community Monitoring clinics report significantly more births in a government clinic,
rather than at home or another health facility. Thus, the intervention motivated mothers
to give birth in government facilities. Furthermore, households in villages undertaking
Community Monitoring report higher clinic utilization. Lastly, the current nutritional
status of under-five children, as measured by weight-for-age, is significantly better in
Community Monitoring villages. Considering that the main objective of the Free Health
Care Initiative is to increase the provision of primary health care to these vulnerable
populations, this result suggests that Community Monitoring can be effective in fur￾thering this goal of the Free Health Care Initiative. However, immunization, antenatal
care, absenteeism, and staff attitude were not affected by the Community Monitoring
intervention.
In addition, we find little impact of Non-Financial Awards on our indicator variables, sug￾gesting that this intervention may not be effective in affecting health and service delivery
outcomes. In fact, households living in the catchments of Non-Financial Award clinics
display a somewhat higher probability of experiencing nurse absenteeism, although this
effect departs from a low baseline probability in the control group.
The analysis shows that Community Monitoring affects some health and service deliv￾ery outcomes, such as illegal fee charging, institutional delivery, clinic utilization, and
children’s nutritional status, while non-financial awards do not lead to significant im￾provements along these and other dimensions. Overall, this suggests that initiatives
focusing on reinforcing the accountability and monitoring structures at the community
level can play an essential role in improving the quality and the quantity of health service
delivery but also health outcomes, like acute malnutrition. The need for intervention
favoring relationship between the community and the health workers has only been re￾inforced by the recent Ebola outbreak, that has revealed a general distrust of the formal
health sector in the population. Therefore, community monitoring could be a relevant
and effective approach to rebuid and strengthen the health care system in post-Ebola
Sierra Leone.
32
References
Ashraf, N., Bandiera, O. and Jack, K. (2011). No Margin, No Mission? A Field Ex￾periment on Pro-Social Tasks (Working paper). Retrieved from the Harvard Business
School website: http://sticerd.lse.ac.uk/dps/eopp/eopp35.pdf
Banerjee, A., Deaton, A., & Duflo, E. (2004). Wealth, Health, and Health Service
Delivery in Rural Rajasthan. American Economic Review Papers and Proceedings,
94(2): 326-330.
Basinga, P., Gertler, P., Binagwaho, A., Soucat, A., Sturdy, J., Vermeersch, C. (2010).
Paying Primary Health Care Centers for Performance in Rwanda (World Bank Pol￾icy Research Working Paper 5190). Retrieved from http://siteresources.worldbank.org/
EXTDEV DIALOGUE/ Images/ 537296-1238422761932/5968067-1269375819845/Rwan
da P4P.pdf
B´enabou, R., & Tirole, J. (2003). Intrinsic and Extrinsic Motivation. Review of Eco￾nomic Studies, 70(3), 489–520.
Besley, T., & Ghatak, M. (2008). Status Incentives. American Economic Review, 98(2):
206–11.
Bilardi, J. Fairley, C., Temple-Smith, M., Pirotta, McNamee, K., Bourke, S.,. . . Hock￾ing, J. (2010) Incentive Payments to General Practitioners Aimed at Increasing Oppor￾tunistic Testing of Young Women for Chlamydia: A Pilot Cluster Randomised Controlled
Trial. BMC Public Health, 10(70).
Bj¨orkman, M., & Svensson, J. (2009). Power to the People: Evidence from a Randomized
Field Experiment on Community-Based Monitoring in Uganda. The Quarterly Journal
of Economics, 124(2), 735.
Bj¨orkman, M., de Walque, D., & Svensson, J.Working paper. Community Based Mon￾itoring: When Does It Work (even in the long run)? Experimental Evidence from
Uganda
Camerer, C., & Hogarth, R. (1999). The Effects of Financial Incentives in Experiments:
A Review and Capital-Labor-Production Framework (California Institute of Technology
Social Science Working Paper 1059). Retrieved from: http://www.springerlink.com/
content/wh218110256r8t00/
Christianson, J., Leatherman, S., & Sutherland, K. (2007). Financial Incentives, Health￾care Providers and Quality Improvements: A Review of the Evidence. London: Health
Foundation, 2007.
Frey, B., & Neckermann, S. (2010). Awards as Signals (CREMA Working Paper Series
2010-21). Center for Research in Economics, Management and the Arts (CREMA).
Retrieved from: http://www.iew.unizh.ch/wp/iewwp513.pdf
33
Greevy R, Xu X., Lu B., Beck C. (2011). Optimal Nonbipartite Matching and its
Statistical Applications. The American Statistician. 65(1): 21-30.
Huntington, D., Zaky, H., Shawky, S., Fattah, F., & El-Hadary, E. (2010). Impact of a
Service Provider Incentive Scheme on Quality of Reproductive and Child-Health Services
in Egypt. The Journal of Health, Population and Nutrition, 28(3): 273-280.
Kosfeld, M., & Neckermann, S. (2011). Getting More Work for Nothing? Symbolic
Awards and Worker Performance. American Economic Journal: Microeconomics, Amer￾ican Economic Association, 3(3): 86-99.
Kreps, D. (1997). Intrinsic Motivation and Extrinsic Incentives. American Economic
Review Papers & Proceedings, 87: 359-364.
Mathauer, I., & Imhoff, I. (2006). Health Worker Motivation in Africa: The Role of
Non-Financial Incentives and Human Resource Management Tools. Human Resources
for Health, 4(24).
Olken, B., Onishi, J., & Wong, S. (2012). Should Aid Reward Performance? Evidence
from a Field Experiment on Health and Education in Indonesia. NBER Working Paper
17892.
Stilwell, B. (2001). Health Worker Motivation in Zimbabwe. In an unpublished report
for the Department of Organization of Health Care Delivery Geneva: World Health
Organization.
Tirole, J. (1996) A Theory of Collective Reputations (with Applications to the Per￾sistence of Corruption and to Firm Quality). The Review of Economic Studies, 63:
1-22.
Van Herck, P., De Smedt, D., Annemans, L., Remmen, R., Rosenthal, M., & Sermeus, W.
(2010). Systematic Review: Effects, Design Choices and Context of Pay-for-Performance
in Health Care. BMC Health Services Research, 10(247).
Witter, S. Zulfiqur, T., Javeed, S., Khan, A., & Bari, A. (2011). Paying Health Work￾ers for Performance in Battagram District, Pakistan. Human Resources for Health,
9(23).
Yared, A. (2009). Non-Financial Incentives for Voluntary Community Health Workers:
A Qualitative Study ( Working Paper No. 1). Retrieved from the JSI Research & Train￾ing Institute, Inc. website: http://l10k.jsi.com/Resources/Docs/nfi workingpaper vol
2.pdf
34