Foreign Agricultural Service, United 
States Department of Agriculture
This publication was produced at the request of the United States Department of Agriculture. It 
was prepared independently on behalf of Catholic Relief Services by School-to-School 
International. The views expressed in this publication do not necessarily reflect the views of the 
United States Department of Agriculture or the United States Government.
BASELINE EVALUATION REPORT 
PHASE IV OF THE McGOVERN-DOLE 
INTERNATIONAL FOOD FOR EDUCATION AND 
CHILD NUTRITION PROGRAM IN SIERRA 
LEONE
September 2019
ii
ALL PIKIN FOR LEARN IN SIERRA 
LEONE PHASE IV BASELINE 
EVALUATION REPORT 
Program: McGovern-Dole International Food for Education and Child Nutrition 
Program 
Agreement Number: FFE-636-2018/007-00
Funding Year: Fiscal Year 2018
Project Duration: 2018-2022
Implemented by: Catholic Relief Services
Evaluation Authored by: School-to-School International
Crystal Chiglinsky
Candace Debnam 
Casey McHugh 
Aimee Reeves 
Autumn Thomas 
Hetal Thukral 
Accessibility Note: An accessible version of this report can be made available upon request. 
Please contact FAS.Monitoring.Evaluation@usda.gov 
iii
ACKNOWLEDGEMENTS
The School-to-School International and Catholic Relief Services teams would like to thank the 
United States Department of Agriculture for their continued support of the All Pikin for Learn 
project in Sierra Leone. 
We would particularly like to recognize the Catholic Relief Services’ Monitoring, Evaluation, 
Accountability, and Learning team, the Sierra Leone Ministry of Basic and Senior Secondary 
Education, and all the government officials and partner stakeholders who contributed their time 
and expertise to this evaluation. 
The following persons played a major role in the process:
Daniel Mumuni (Catholic Relief Services Sierra Leone)
Santigie Kabia (Catholic Relief Services Sierra Leone)
Stacy Prieto (Catholic Relief Services USA)
iv
ACRONYMS
APFL All Pikin for Learn Project
APFL IV Phase IV of the All Pikin for Learn Project
CRS Catholic Relief Services
CTA Community Teacher Association
FGD Focus Group Discussions
GoSL Government of Sierra Leone
ICC Intracluster Correlation
IR Intermediate Result
IYCF Infant and Young Child Feeding
KII Key Informant Interviews 
MBSSE Ministry of Basic and Senior Secondary Education
MGD McGovern-Dole
MSG Mothers Support Group
NSFS National School Feeding Secretariat
PMP Performance Monitoring Plan
SILC Savings and Internal Lending Communities
SMC School Management Committee
SO Strategic Objective
STS School-to-School International
USDA United States Department of Agriculture
v 
CONTENTS
Acknowledgements .................................................................................................................. iii
Acronyms ................................................................................................................................. iv
Contents .................................................................................................................................... v
List of Figures..........................................................................................................................viii
List of Tables ...........................................................................................................................viii
Executive Summary .................................................................................................................. 1
Introduction................................................................................................................................ 7
Education Context in Sierra Leone ........................................................................................ 7
All Pikin for Learn Project Background .................................................................................. 7
Intervention Logic and Theory of Change.............................................................................. 9
Strategic Objectives.............................................................................................................10
Evaluation Purpose .................................................................................................................13
Evaluation Purpose..............................................................................................................13
Evaluation Questions...........................................................................................................13
Key Performance Indicators ................................................................................................14
Methodology............................................................................................................................15
Evaluation Tools ..................................................................................................................16
Literacy Assessment ........................................................................................................16
School-based Surveys and Observation Checklists ........................................................17
Community Focus Group Discussions and Key Informant Interviews..............................18
Sampling Approach and Sample Size .................................................................................18
Data Collection ....................................................................................................................18
Recruitment and Training of Enumerators .......................................................................18
School-based Data Collection..........................................................................................19
Focus Group Discussions and Key Informant Interviews.................................................20
Data Quality, Management, and Analysis............................................................................20
Data Monitoring and Quality Assurance...........................................................................20
vi
Data Analysis ...................................................................................................................21
Limitations............................................................................................................................21
Findings................................................................................................................................... 23
SO1: Improved Literacy of School-aged Children ...............................................................23
IR1.1 Improved Quality of Literacy Instruction ................................................................. 28
IR1.2 Improved Attentiveness ..........................................................................................35
IR1.3 Improved Pupil Attendance ....................................................................................37
SO2 Increased use of good health and dietary practices ....................................................40
IR2.1 Improved Knowledge of Health and Hygiene Practices .........................................42
IR2.2 Increased Knowledge of Safe Food Preparation and Storage Practices................43
IR2.4 Increased Access to Clean Water and Sanitation Services ...................................46
IR2.5 Increased Access to Preventative Health Services ................................................48
IR2.6. Increased Access to Requisite Food Preparation and Storage Tools and 
Equipment ........................................................................................................................49
Program Relevance .............................................................................................................49
Program Effectiveness.........................................................................................................50
Sustainability and Impact.....................................................................................................54
Conclusions and Recommendations.......................................................................................56
Conclusions .........................................................................................................................56
Recommendations...............................................................................................................57
Program Relevance .............................................................................................................57
Program Effectiveness.........................................................................................................57
Sustainability and Impact.....................................................................................................58
References ..............................................................................................................................60
Annexes ..................................................................................................................................61
Annex A. Sampling Approach..............................................................................................61
Annex B. Supplementary Findings ......................................................................................62
Annex C. Updated Indicator Performance Tracking Table (IPTT).......................................67
vii
Annex D. Scope of Work and Evaluation Questions – REMOVE BEFORE MAKING 
PUBLIC ..................................................................................Error! Bookmark not defined.
Annex E. Pupil Literacy Assessment ...................................................................................74
Annex F. Pupil Survey .........................................................................................................74
Annex G. Teacher Survey and Classroom Observation Tool..............................................74
Annex H. School-Based Surveys.........................................................................................74
Annex I. School Observation Tool .......................................................................................74
Annex J. Focus Group Discussion Guide ............................................................................74
Annex K. List of Sample schools visited During Baseline – REMOVE BEFORE MAKING 
PUBLIC ..................................................................................Error! Bookmark not defined.
Annex L. School-to-School International Contributor Bios – REMOVE BEFORE MAKING 
PUBLIC ..................................................................................Error! Bookmark not defined.
viii
LIST OF FIGURES
Figure 1: Intervention Chiefdoms within the Koinadugu and Falaba Districts............................... 9
Figure 2: APFL IV Theory of Change..........................................................................................10
Figure 3: AFPL IV Project Results Framework ...........................................................................11
Figure 4: Percentage of Pupils Reaching Literacy Threshold by Sex.........................................24
Figure 5: Percentage of Pupils Receiving Zero Scores by Sex ..................................................24
Figure 6: Total Number of Teachers and Number of Teachers in Attendance ...........................29
Figure 7: Classroom Resources Observed.................................................................................30
Figure 8: Observed Use of Textbooks or Readers in the Classroom .........................................30
Figure 9: Teacher Knowledge of Teaching Techniques .............................................................32
Figure 10: Teacher Motivating Factors .......................................................................................34
Figure 11: Percentage of Head Teacher Techniques and Tools Observed................................35
Figure 12: Percentage of Pupils Receiving Food by Sex............................................................36
Figure 13: Attendance Rates by Class and Sex .........................................................................37
Figure 14: Average Pupil Enrollment by Class and Sex .............................................................39
Figure 15: Dropout Rates by Class and Sex...............................................................................40
Figure 16: Knowledge of Infant and Young Child Feeding Behaviors ........................................41
Figure 17: Percentage of Pupils Consuming Selected Food Groups .........................................41
Figure 18: Percentage of Pupils Consuming a Minimum Acceptable Diet (Four of Seven Food 
Groups) by Sex ...........................................................................................................................42
Figure 19: Percentage of Pupils Naming Good Health and Hygiene Practices ..........................43
Figure 20: Knowledge of Safe Food Preparation and Storage Practices ...................................46
Figure 21: Percentage of Pupils Reporting Receiving Preventative Health Services by Sex .....48
LIST OF TABLES
Table 1: Timeline of All Pikin for Learn Project Phases ................................................................ 8
Table 2: Key Performance Indicators at Baseline.......................................................................14
Table 3: Literacy Assessment Subtasks .....................................................................................16
Table 4: School-based Surveys and Observation Checklists .....................................................17
Table 5: Sample Targets and Response Rates ..........................................................................19
Table 6: Alphabet Naming Mean Scores by Sex (Correct out of 51) ..........................................25
Table 7: Phonemic Awareness Mean Scores by Sex (Correct out of 10)...................................25
ix
Table 8: Familiar Word Reading Mean Scores by Sex (Correct out of 40)................................. 26
Table 9: Invented Word Reading Mean Scores by Sex (Correct out of 25)................................26
Table 10: Reading Passage Mean Scores by Sex (Correct out of 41) .......................................27
Table 11: Reading Comprehension Mean Scores by Sex (Correct out of 5)..............................27
Table 12: Distribution of Correct Reading Comprehension Questions by Sex ...........................28
Table 13: Listening Comprehension Mean Scores by Sex (Correct out of 4).............................28
Table 14: Classroom Teacher Characteristics............................................................................31
Table 15: Classroom Teacher Training Participation..................................................................32
Table 16: Coaching and Mentoring Frequency...........................................................................33
Table 17: Pupils’ Reported Hunger Status or Access to Food by Sex........................................36
Table 18: School Infrastructure Characteristics ..........................................................................38
Table 19: School Kitchen Characteristics ...................................................................................44
Table 20: School Storeroom or Storage Facility Characteristics ................................................45
Table 21: School Water Facility Characteristics .........................................................................47
Table 22: School Sanitation Facility Characteristics ...................................................................47
Table 23: Presence of Kitchen Equipment and Tools................................................................. 49
Table B.1: Literacy Assessment Mean Scores for Pupils at New or Continuing APFL Schools . 62
Table B.2: Distribution of Reading Comprehension Scores for Pupils at New or Continuing APFL 
Schools .......................................................................................................................................62
Table B.3: Head Teacher Supplementary Findings ....................................................................63
Table B.4: Teacher and Classroom Observation Supplementary Findings ................................63
Table B.5: Pupil Attention Supplementary Findings....................................................................63
Table B.6: Head Teacher School Feeding Response Supplementary Findings.........................64
Table B.7: School Observation Enrollment, Attendance, and Dropouts .....................................65
Table B.8: Classroom Observation Headcounts.........................................................................65
Table B.9: School Garden Findings ............................................................................................66
Table B.10: Head Teacher Preventative Health Services Supplementary Findings...................66
Table C.1: Updated Indicator Performance Tracking Table........................................................67
1 
EXECUTIVE SUMMARY
Project Background 
Catholic Relief Services (CRS) is implementing the fourth phase of the McGovern-Dole Food for 
Education project All Pikin for Learn (APFL) in Sierra Leone. The project aims to reduce hunger 
while improving literacy and primary education, especially for girls. Since 2008, CRS has rolled 
out APFL, funded by the United States Department of Agriculture (USDA), in chiefdoms within 
the Koinadugu and Falaba districts. The project aligns with the education strategy of the
Government of Sierra Leone (GoSL) by seeking to improve the quality and relevance of 
education, encourage the completion of primary education for vulnerable and marginalized 
children, and increase community involvement in education. It is designed to achieve these 
goals by providing school meals, training teachers and school administrators, improving water 
and sanitation facilities, providing school infrastructure, and building skills and knowledge within 
the communities. 
In 2018, USDA approved APFL for four additional years of implementation, expanding its reach 
to 69,731 primary school pupils in the Koinadugu and Falaba districts from 32,684 pupils in the 
previous phase. Phase IV of the project builds on the progress and lessons learned throughout 
Phases I, II, and III. It prioritizes activities that leverage gains achieved from prior McGovern￾Dole investments to achieve sustainability and graduation by the end of the project. The 
objectives of Phase IV of APFL (APFL IV) align with the standard strategic objectives (SO) of
the McGovern-Dole project: 
• Improved literacy of school-aged children; and 
• Increased the use of health and dietary practices of school-aged children in 309 schools. 
Report Purpose
This report presents the findings of the baseline evaluation of APFL IV. The evaluation
establishes baseline values for all performance indicators, generates data for comparative 
analysis, and validates project strategies and assumptions. Findings from this report will 
elucidate contextual factors that can improve pupil health and literacy in the most food-insecure 
chiefdoms in Koinadugu and Falaba districts. It will also allow the McGovern-Dole APFL IV
project team to establish questions to test APFL’s theory of change and to refine indicator 
targets. 
Methodology
The external evaluation of APFL IV is a four-year study. Baseline data collection took place in 
June 2019; the midline data collection will take place in June 2021 and the endline in June
2022. Each evaluation uses a mixed-method approach with six tools:
• Literacy assessment
• School observation checklist
• Classroom observation
2 
• School-based surveys with seven stakeholder groups: pupils, head teachers, teachers, 
food preparers, school management committees (SMC) chairpersons, community 
teacher associations (CTA) chairpersons, and mothers support groups (MSG) heads
• Key informant interviews (KII) with government officials and partner organizations
• Focus group discussions (FGD) with community members, leaders, and parents 
The study includes 70 sample schools across 15 chiefdoms where the project is intervening. 
Ten randomly selected pupils enrolled in class 2 took the literacy assessment and the school￾based survey of pupils. Enumerators conducted additional school-based surveys with each 
school’s head teacher and appropriate group representative; this could include up to three 
teachers and two food preparers. In-depth interviews with key stakeholders at the Ministry of 
Basic and Senior and Secondary Education (MBSSE), the National School Feeding Secretariat 
(NSFS), the Ministry of Health and Sanitation, and implementation partners rounded out the 
qualitative component of the assessment. Finally, 12 FGD collected the critical perspectives of 
community members, leaders, and parents. 
Limitations
The following limitations should be considered when reviewing the findings of the APFL IV 
baseline:
• Use of tools from previous APFL phases. The baseline data collection utilized several 
tools from previous APFL evaluations. While these tools were reviewed and updated to a 
certain level, the tools were not fully aligned with the most recent recommendations from 
literacy research. Additionally, the instructions for the data collection were in English. 
Based on the results from the listening comprehension task, it is likely that many 
students struggle with listening comprehension so may not have understood the task.
• Delays in uploading data due to poor connectivity. Poor connection in more remote 
communities was a critical challenge during data collection as enumerators were unable 
to upload the data to the server each day. This limited the ability of the team to properly 
monitor the data as it was coming in and resolve problems quickly. Despite these 
challenges, the data quality is strong thanks to redundant data quality processes put into 
place to address this potential challenge. 
• Lower response rates than anticipated for school surveys. The sampling approach 
called for three teachers to be surveyed and three classrooms to be observed at each 
school—one each for classes 2 through 4. However, there were fewer eligible teachers 
and classrooms to include as a result of the multi-level classrooms. There were also 
fewer MSG heads and food preparers to survey than recommended as many sampled 
schools are new to the APFL project and have not yet established these positions. 
• Inherent bias in sampling children present on day of assessment. Pupils’ literacy 
assessment results may be biased towards the types of students who attend regularly 
and may exclude those pupils who are enrolled by do not attend regularly. 
3 
Findings
Strategic Objectives 
The baseline literacy assessment of surveyed APFL IV class 2 pupils revealed low literacy
levels—only 5.58 percent met the project’s reading comprehension benchmark. Pupils 
performed best on the alphabet naming subtask; however, the majority of pupils performed 
poorly on each of the other subtasks. In general, boys outperformed girls. 
Teachers play a critical role in pupils’ learning and are, therefore, instrumental in meeting 
APFL’s project goals. There is a clear need for rigorous teacher training and coaching on 
effective teaching techniques. Fewer than half of teachers are certified; therefore, CRS’s 
teacher training and certification efforts will help fill a critical gap in Koinadugu’s and Falaba’s 
workforce. According to surveys, positive mentorships between teachers and head teachers 
could be an impactful activity. 
It will also be critical for CRS to support the school feeding program. Without CRS’s support
during this gap in the school feeding implementation between APFL III and APFL IV, only 6.12 
percent of schools provided a meal on the day of the assessment; 60.46 percent of head 
teachers of those that did not said the absence was due to food supplies running out. With less 
than half of pupils reporting that they maintain the minimum acceptable diet, keeping school 
feeding programs well stocked will be essential for pupils’ nutrition and, in turn, their attendance 
and literacy outcomes. 
Knowledge on good health and hygiene should be reinforced as less than half (42.74 percent) 
of pupils could name six out of 12 hygiene practices and only 37.25 percent of food preparers 
could name eight of eleven best food preparation and storage practices. However, the 
infrastructure at the schools did show improved sanitation resources at 228 of 309 intervention 
schools (73.79 percent) with nearly three-quarters of sampled schools having a functioning 
latrine (74.26 percent). 
Relevance
Qualitative data from the community FGDs and the stakeholder KIIs indicate that the holistic 
approach and design of APFL IV is responsive to the most prominent needs of children in their 
community, specifically feeding support, improving teacher quality, water wells and latrines, and 
improving school building infrastructure. Of these needs, school feeding support was cited the 
most often in FGDs and echoed prominently in stakeholder KIIs. Overall, community members
are appreciative of APFL’s programming and want to see the project succeed. 
Effectiveness
Participants in both the community FGDs and stakeholder KIIs view the project’s overall design 
as appropriate—if not critical— to the context of their communities. However, stakeholders and 
community members alike expressed concerns over the effectiveness of the project if all 
4 
implementation aspects are not strictly adhered to and monitored; specific areas of concern 
include teacher training, food provision, and community sensitization. Additionally, participants 
drew attention to gender considerations within the project that should be examined and 
addressed, such as girls’ higher chore burden, risk of pregnancy and early marriage, and 
gender roles within APFL IV activities. 
Sustainability and Impact
CRS’s coordination with partners will be critical to ensuring that APFL’s activities are 
sustainable beyond the life of the project. There are opportunities to include all partners—from 
community members and school support groups at the local level through the NSFS at the 
national level—in each APFL activity. Indeed, the NSFS is eager to learn from CRS’s successes 
as it prepares to take on the responsibility for school feeding activities. 
Conclusions
Overall, the findings from the baseline evaluation for the APFL IV project show that the design 
of the project is relevant and properly addresses the needs of the communities CRS serves. 
The findings provide both a challenge and encouragement for CRS. Communities are very 
supportive of the APFL programming and want it to continue. The NSFS, as an entity, is ready 
to learn and prepare itself to take on the school feeding activities. 
However, with such a large expansion of the APFL project in Phase IV, the project’s needs are 
great. Based on the baseline literacy assessment, APFL IV pupils’ literacy outcomes are low—
only 5.58 percent of class 2 pupils achieved the reading comprehension benchmark. In general, 
boys outperformed girls across the subtasks, which indicates a gender-based performance gap. 
Importantly, low performance on the listening comprehension sub-task indicates very low 
English abilities that are foundational for reading acquisition. 
Teachers will likely play a critical role in the project’s success. While teacher attendance is 
relatively high, there is a clear need for rigorous training and coaching. Less than one-quarter of 
teachers felt any confidence in their ability to use the techniques in their own classrooms. Fewer 
than half of teachers are certified, so CRS’s teacher training and certification efforts will help fill 
this gap in the teaching workforce in Koinadugu and Falaba districts. One encouraging finding 
concerning teacher coaching was that nearly all teachers (94.85 percent) had been observed by 
their head teacher in the academic year; 87.15 percent had been observed more than twice in 
the year. With one of the top three reported motivation factors for teachers being their 
relationship with the head teacher, supporting positive relationships between teachers and their 
head teachers can be an impactful activity for CRS.
Supporting the feeding program will be critical for the APFL IV project. Only 6.12 percent of 
head teachers reported being able to provide a school meal on the day of the baseline data 
collection visit. This indicates that without CRS support, provision of meals at schools will be 
limited. Additionally, with less than half of pupils reporting that they are reaching the minimum 
acceptable diet, keeping the school feeding programs well stocked will be essential for pupils’ 
5 
nutrition. 
Recommendations
Relevance
TAILOR APFL IV INTERVENTIONS TO COMMUNITY NEEDS.
The project should adapt its approach in different communities based on previous level of 
support from APFL as well as current needs and priorities. More targeted interventions would 
likely lead to greater impact as well as encourage stronger community buy-in and ownership, 
thus promoting future sustainability. Tailoring approaches would also allow CRS to leverage 
existing resources and potential partnerships with other entities working within the communities 
to be more efficient. 
EXAMINE EXISTING STUDENT AND TEACHER ENGLISH LANGUAGE ABILITIES.
Student performance on the listening comprehension sub-task may indicate that students have 
a limited ability to understand spoken English. The project may want to consider undertaking 
more targeted research into the reasons for this gap in comprehension. Specifically, this may 
mean a deeper investment in coaching for basic skills for literacy instruction for early grade 
teachers, who’s English language proficiency was not addressed in this baseline data collection. 
Improving the English abilities of teachers may be a necessary step to ensuring they can 
confidently teach students to read in English. 
Effectiveness 
CLOSELY MONITOR PROJECT IMPLEMENTATION. 
At every stage of the APFL IV project, it will be important to monitor the implementation of 
activities in order to ensure fidelity to the project design. Close monitoring should help alleviate 
community concerns around theft or ineffectiveness, increase CRS’s responsiveness to 
challenges that arise, and allow the project team to link specific interventions with outcomes
observed during the midline and endline evaluations.
ADDRESS GENDER CONSIDERATIONS WITHIN THE PROGRAMMING. 
CRS should examine and address gender considerations in all aspects of APFL IV 
programming. Baseline findings show that boys outperform girls on the literacy assessment; 
furthermore, community members shared concerns that challenges around gender are 
prevalent in targeted communities. CRS should consider completing a thorough gender and 
social inclusion analysis to ensure that the project’s design, implementation, monitoring, and 
evaluation accounts for existing gender disparities and that the interventions designed to 
remove or reduce such disparities are appropriate for the community context. 
6 
Sustainability and Impact
BUILD THE CAPACITY OF THE NATIONAL SCHOOL FEEDING SECRETARIAT. 
Considering the APFL IV goal of transitioning the school feeding program to the GoSL, CRS 
should play a role in building the capacity of the nascent NSFS. This could include frequent 
collaboration and communication between the two groups, as well as CRS-led trainings. 
Potential trainings could focus on building the capacity of monitoring-and-evaluation teams to 
oversee implementation of the school feeding program as well as address the program’s 
administrative and logistical components. 
INTEGRATE TRANSITION STRATEGY FROM START-UP FOR SUSTAINABLE IMPACT. 
Clear communication and expectations setting around CRS’s transition from the region must be 
central to the early discussions with communities, partners, and the GoSL. CRS should develop 
a transition strategy for AFPL IV in partnership with relevant stakeholders. Such a strategy
should clearly articulate transitions and phase-out approaches for each activity, identify 
responsible parties to continue implementation after APFL IV ends, delineate direct and indirect 
costs, establish monitoring and oversight guidelines, and set clear timelines. A successful 
strategy should also outline contingency plans, as needed.
7 
INTRODUCTION
EDUCATION CONTEXT IN SIERRA LEONE
The Government of Sierra Leone (GoSL) declared basic education “free and compulsory” with 
the Education Act of 2004.1 While school enrollment has increased in recent years, Sierra 
Leone faces high dropout rates and low literacy rates.2, 3 According to the most recent 
Demographics and Health Survey, only 31.9 percent of males over the age of six and 23.7 
percent of females have completed primary school or higher.4 Furthermore, Sierra Leone’s 
education system was devastated by the 2014–15 Ebola virus outbreak; schools closed for 
more than nine months, resulting in nearly one year of lost schooling.5 
Despite these challenges, the GoSL has committed to increasing its investment in the education 
sector. It allocated 21.00 percent of the national budget to support the launch of the GoSL’s 
Free Education Program in August 2018. The program provides free education from pre-primary 
through secondary school and strengthens schools’ infrastructure, supply chains, and services.6
The Ministry of Basic and Senior Secondary Education’s (MBSSE) 2018–20 Education Strategy 
aims to increase access, equity, and completion rates; improve the quality and relevance of 
pupils’ education; and strengthen the education system. Key interventions of the robust strategy 
include bolstering the national school feeding program, upgrading school infrastructure through 
maintenance or construction, improving teaching and learning materials in the classroom, and 
investing in teachers’ skills and motivation.7 
ALL PIKIN FOR LEARN PROJECT BACKGROUND 
Catholic Relief Services (CRS) has been implementing the All Pikin for Learn (APFL) project in 
northern Sierra Leone since 2008. APFL, funded by the United States Department of 
Agriculture’s (USDA) McGovern-Dole Food for Education program, strives to reduce hunger and 
improve literacy and primary education. McGovern-Dole projects around the world provide 
school meals, teacher training, and other support activities to boost school enrollment and 
academic performance.8 
1 Parliament of Sierra Leone, “The Education Act, 2004,” signed March 29, 2004. http://www.sierra-leone.org/Laws/2004-2p.pdf
2
 UNESCO Institute of Statistics, accessed August 6, 2019, http://uis.unesco.org/country/SL#slideoutmenu 
3
 Statistics Sierra Leone - SSL and ICF International. Sierra Leone Demographic and Health Survey 2013. Freetown, Sierra Leone: 
SSL and ICF International. 2014. 
4
 Statistics Sierra Leone - SSL and ICF International. Sierra Leone Demographic and Health Survey 2013. Freetown, Sierra Leone: 
SSL and ICF International. 2014. 
5
 Sierra Leone Ministry of Basic and Senior Secondary Education, “2018–20 Education Sector Plan,” 2017. 
6
 State House Media and Communications Unit, “President Bio Launches Free Education, Calls on Parents and Teachers to 
Support the Initiative,” last modified August 20, 2018, https://statehouse.gov.sl/president-bio-launches-free-education-calls-on￾parents-and-teachers-to-support-the-initiative/. 
7
 Sierra Leone Ministry of Basic and Senior Secondary Education, “2018–20 Education Sector Plan,” 2017. 
8
 United States Department of Agriculture, “McGovern-Dole Food for Education Program,” accessed August 6, 2019, 
https://www.fas.usda.gov/programs/mcgovern-dole-food-education-program. 
8 
Under the McGovern-Dole program, CRS has provided nutritional support to communities within 
Sierra Leone’s Koinadugu and Falaba districts. This support includes providing nutritious meals 
to pupils, teachers, and food preparers at intervention schools each school day, as well as take￾home rations for pupils who demonstrate high academic performance and teachers who support 
APFL’s after-school activities. APFL has sought to improve pupils’ educational outcomes 
through the provision of teaching and learning materials, the creation of after-school reading 
clubs, literacy training and coaching for teachers and school administrators, and school 
infrastructure improvements. Additionally, the project has worked with local communities to 
create and strengthen school management committees (SMC), community teacher associations
(CTA), mothers support groups (MSG), and savings and internal lending communities (SILC). 
APFL went through three project 
phases between 2008 and 2018. 
Phase 1 targeted the four most food 
insecure chiefdoms in Koinadugu 
with critical food distribution via daily 
school meals and take-home rations 
to more than 5,000 girls in upper 
primary classes. CRS also trained SMCs, provided schools with teaching and learning 
materials, and supported key infrastructure improvements. In Phase II, the APFL expanded to 
include a fifth chiefdom and 75 additional schools. In-service teacher training on diagnostic 
teaching methods was added to improve literacy instruction within the intervention schools. CRS 
also established SILCs to improve the financial stability of households. Lastly, in Phase III, CRS 
supported intervention schools in rebuilding after the Ebola crisis. Twenty-five literacy coaches 
were hired and equipped to provide training and coaching to teachers, and after-school reading 
clubs were created to support pupils’ interest in reading. All Phase III activities ended on June 
30, 2019. 
Now, CRS is launching the fourth phase of the project (APFL IV), which will run from September
2018 through September 2022. Under this $25 million phase, CRS will double its reach from 
32,684 primary school pupils to 69,731 pupils in 309 schools across 15 chiefdoms in the 
Koinadugu and Falaba districts. The 32,684 pupils who benefited from APFL’s Phase III with 
continue to be supported in APFL IV. Building upon the progress made, APFL IV will relaunch
its core project activities, add new support activities, and intensify advocacy efforts to achieve 
sustainability at the local, district, and national levels. CRS will partner with the MBSSE, the 
Ministry of Health and Sanitation (MoHS), CARITAS Makeni, Ernest Bai Koroma University, the 
Association of Language and Literacy Education (TALLE), District Councils, and World Hope 
International. A key goal of APFL IV is to successfully transition support of the schools’ feeding 
programs to the GoSL’s national school feeding program by the end of the project in 2022 so 
these partnerships with be critical to the sustainability of the efforts. 
Table 1: Timeline of All Pikin for Learn Project Phases
APFL 
Phases
Timeline Geographic 
Area
Pupils 
Reached
I 2008–12 4 chiefdoms 25,128
II 2012–16 5 chiefdoms 28,585
III 2016–18 5 chiefdoms 32,684
IV 2018–22 15 chiefdoms 69,731
9 
Note: Stars indicate APFL IV intervention chiefdoms within the Koinadugu and Falaba districts.
INTERVENTION LOGIC AND THEORY OF CHANGE 
Since its inception, the APFL project has been a school feeding program aiming to improve the 
education and nutrition outcomes of children. Due to the poor state of education in the 
intervention districts, CRS has focused resources on improving and jumpstarting literacy
through the APFL project. However, after the Ebola crisis, the project intensified efforts to 
reinforce positive health and dietary practices in pupils and communities. The APFL IV Theory 
of Change (TOC) is outlined in Figure 2 below. 
Falaba District
Koinadugu District
Figure 1: Intervention Chiefdoms within the Koinadugu and Falaba Districts
10
OVERALLL SYSTEM NUTRITION LITERACY
Figure 2: APFL IV Theory of Change
STRATEGIC OBJECTIVES
The APFL IV project centers around the two USDA MGD project SOs: 
• SO 1: Improved literacy of school-age children; and
• SO 2: Increased use of health and dietary practices of school-aged children.
Both SOs will be supported as outlined in the APFL IV Project Results Framework (Figure 3). 
• THEN There will be sustained delivery of quality 
literacy education in a healthy and enabling school, 
home, and community environment for children in the 
vulnerable Koinadugu and Falaba Districts 
IF School systems are strengthened and function overall at a 
higher level, and Children benefit from safe, nutritious meals, 
AND Children benefit from high-quality literacy instruction, 
• THEN schools systems will be strengthened and will 
function overall at a higher level and children will 
benefit from safe, nutritious meals provided by a 
sustainable national school feeding program 
IF national and local governments prioritize increased spending 
for approving new schools, if policy and regulatory frameworks 
are established to fund and institutionalize a national school 
feeding program inclusive of complementary project activities 
(e.g. WASH, deworming, nutrition), AND national and local 
governments take responsibility for implementing and 
monitoring activities with the support of communities, 
•THEN children will benefit from safe, nutritious 
meals. 
IF students have access to a school feeding program, if 
vulnerable groups improve nutrition knowledge, attitudes and 
practices (MCH and SBC), and if nutrition activities link to 
complementary project activities (e.g. WASH, deworming), 
•THEN students will improve their literacy, as 
evidenced by assessment results. 
IF teachers and school administrators improve competency and 
commitment to teach literacy, if children are ready to start and 
thrive in school at the appropriate age, and if communities 
sustain literacy activities both inside and outside of the 
classroom, 
•THEN children will benefit from high-quality 
literacy instruction.
IF teachers remain motivated to continue teaching at their 
current schools, if teachers and administrators are trained, 
coached, and adopt new pedagogical practices, and if through 
SILC, agriculture, nutrition and livelihood activities, communities 
are engaged to actively support children’s literacy learning and 
contribute to the necessary inputs for school feeding, 
11
Under the project’s first SO, APFL IV will implement several school-based activities to improve 
the literacy of school-aged children in 309 intervention schools. CRS recognizes the critical role 
of teachers in pupils’ learning and will focus on teachers’ professional development through
training, coaching, and performance incentives. With an emphasis on sustainability through the 
GoSL, CRS will conduct a two-year, distance-education certification program to increase the 
number of certified teachers who are eligible to enroll in the MBSSE system and become 
Figure 3: AFPL IV Project Results Framework
12
government-salaried teachers. CRS will also support existing qualified teachers in the 
enrollment process. 
Direct support to pupils will include the establishment of 117 new after-school reading clubs that 
promote a culture of reading. As the heart of the McGovern-Dole program, daily school meals
will be provided at all intervention schools—in collaboration with the MBSSE at all government￾supported schools and independently at non-government schools—to encourage pupils’ 
attendance and attentiveness. Take-home rations will be provided to the top-performing male 
and female pupils in each class level at intervention schools during an annual ceremony. 
Teachers who facilitate after-school activities will receive quarterly rations. 
The project’s second SO seeks to increase the use of health and dietary practices. CRS’s 
activities will focus on promoting health, nutrition, and personal hygiene initiatives within the 
schools and communities. Health clubs for pupils will be expanded and reinforced. Food 
preparers, school administrators, SMC chairpersons, and local leaders will receive training on 
proper food preparation, storage, and sanitation practices. MSG members will receive training 
on maternal and child health and nutrition. Twenty school gardens will be established with 
support from the Ministry of Agriculture to provide hands-on learning opportunities for both 
pupils and community members. 
CRS will improve school water and sanitation facilities, which will allow pupils to put proper 
health behaviors into practice. In consultation with the Ministry of Sanitation and Water 
Resources, the project will build and repair gender-segregated latrines in accordance with 
national standards; ten new wells will be built at schools currently without access to water. To 
further mitigate the effects of poor sanitation, CRS will also collaborate with the Ministry of 
Health and Sanitation in school-based deworming efforts under their national Neglected Tropical 
Disease Program.
To achieve these ambitious goals and move towards local and national sustainability by the end 
of this phase, the APFL IV project team will consistently work alongside local communities, 
organization partners, and GoSL ministries, departments, and agencies, including the MBSSE, 
Ministry of Health and Sanitation, National School Feeding Secretariat, and Ministry of 
Sanitation and Water Resources. National advocacy efforts will be interwoven throughout Phase 
IV to promote teacher professional development and incentives, share best practices with the 
national school feeding program, and inform and reinforce national priorities around child health, 
nutrition, and sanitation. 
13
EVALUATION PURPOSE
EVALUATION PURPOSE
CRS contracted School-to-School International (STS) as the independent external evaluator for 
the APFL IV project. The evaluation plan for Phase IV includes a baseline data collection in the 
June 2019, a midline evaluation in June 2021, and an endline evaluation in June 2022. This 
report outlines the results of the 2019 baseline evaluation. The baseline evaluation does not 
compare changes over time between phases of the APFL project, only within Phase IV. 
The purposes of the baseline evaluation are to establish baseline values and define targets for 
APFL IV’s program performance indicators, to generate data to be used for comparative 
analysis, and to help CRS validate the project’s strategies and assumptions. Evidence from this 
report will elucidate some of the contextual factors for improving pupil health and literacy in the 
Koinadugu and Falaba districts, allowing CRS to make evidence-based decisions in their 
programming to maximize the effectiveness, relevance, efficiency, sustainability, and impact
over the life of the project. 
Partners within the MSBBE may also use the results in order to inform their national policies, 
programs, and practices. As examples, the National School Feeding Secretariat may adopt the 
best practices demonstrated during APFL IV or the members of the MBSSE focused on early 
grade reading may better understand contextual factors underlying pupils’ literacy performance. 
At the community level, pupils’ performance can be used in discussions with the SMCs, CTAs, 
MSGs, and parents to reinforce the need of community support around pupil’s education and 
reading. 
EVALUATION QUESTIONS 
To support the previously stated purposes, this baseline evaluation report explores the following 
questions within its findings, conclusions, and recommendations. 
OVERALL RELEVANCE
• To what extent does the program design respond to the needs identified by the 
baseline? 
PROGRAM IMPLEMENTATION
• How appropriate is the design to the context, and to what extent is the project likely to 
adopt different approaches in different communities? 
• What challenges can already be foreseen with the design of the project in achieving its 
objectives, and what improvements could be made to the design and plan? 
• How could integration of the two SOs and the foundational results be enhanced or 
further leveraged to deeper positive program impact? 
• How could integration of the gender considerations be enhanced or further leveraged to 
deepen positive project impact? 
14
• To what extent will the project respond to the needs of stakeholders and facilitate their 
participation? What design changes could be made to improve this? 
• Which interventions are likely to be the most critical or effective in achieving the project’s 
intermediate results and objectives?
• What are the key assumptions related to program’s theory of change that need to be 
monitored and specifics questions to test the theory of change during APFL IV 
implementation?
• Are there any foreseeable negative impacts or unintended consequences of the project 
that need to be addressed or mitigated against since the beginning, and how? 
SUSTAINABILITY AND IMPACT
• What are the key considerations to ensure sustainability and impact? 
• What could be barriers to achieving sustainability and impact? 
• How can an exit strategy be better integrated from project startup? 
• How can CRS effectively improve the project’s design and implementation to achieve a 
more sustainable impact?
KEY PERFORMANCE INDICATORS
The APFL IV PMP requires that the majority of performance indicators be set to zero for the 
baseline evaluation. Indicators that are not set to zero are recorded in Table 2 below. These 
values represent data from both the external baseline evaluation conducted by STS and from 
CRS’s internal monitoring data. Data provided by CRS is shaded grey and represents the full list 
of 309 intervention schools, while the evaluation data represents only the 69 sample schools. 
Table 2: Key Performance Indicators at Baseline
Indicator Name Indicator 
No.
Baseline (2019) Comments Target Male Female Total
Percentage of students who, 
by the end of two grades of 
schooling, demonstrate that 
they can read and understand 
the meaning of grade-level 
text
MGD 1 25% 7.77% 3.11% 5.58%
Percentage of students in 
target schools who indicate 
that they are hungry or very 
hungry during the school 
days
CRS 
Custom 
3 
N/A 1.08% 0.43% 0.78%
CRS school 
feeding not active 
at time of 
baseline
Average student attendance 
rate in USDA supported 
classrooms/schools
MGD 2 89% 70.9% 52.7% 61.8% Per CRS 
monitoring data
Number of students enrolled 
in schools receiving USDA 
assistance (Primary School)
MGD 9 31,546 23,356 21,155 44,511 Per CRS 
monitoring data
15
Indicator Name Indicator 
No.
Baseline (2019) Comments Target Male Female Total
Percentage of pupils in 
classes 3 to 6 who dropped 
out of school at the end of the 
school year
CRS 
Custom 
9 
N/A 4.47% 4.42% 4.45%
Percentage of participants of 
community-level nutrition 
interventions who practice 
promoted infant and young 
child feeding behaviors
MGD 21 TBD 98.27%
Calculated as 
number of MSG 
heads that 
identified at least 
one IYCF
behavior
Percentage of students in 
target schools who achieve a 
passing score on a test of 
good health and hygiene 
practices
CRS 
Custom 
10
0 42.04% 43.53% 42.74%
Percentage of food preparers 
at target schools who achieve 
a passing score on a test of 
safe food preparation and 
storage
CRS 
Custom 
12
0 37.25%
Sex of food 
preparers not 
captured
Number of schools using an 
improved water source MGD 27 46 161 Per CRS 
monitoring data
Number of schools with 
improved sanitation facilities MGD 28 169 228 Per CRS 
monitoring data
METHODOLOGY
This section describes the methods used to address the study’s research questions, including 
the research approach, evaluation tools, sampling method, data collection and analysis, and 
study limitations.
Research Approach 
The baseline evaluation of the APFL IV project was a mixed-method study that established 
baseline values and targets for the project’s performance indicators and provides information for 
evidence-based decision making regarding the design and assumptions of the APFL IV project. 
Data were collected from a clustered sample of the 309 APFL IV intervention schools in June 
2019 using a suite of quantitative and qualitative tools. Diverse groups of stakeholders were 
included to provide broad perspectives for the project—pupils, teachers, head teachers, SMC 
chairpersons, CTA chairpersons, school food preparers, the heads of the MSGs, community 
members, and staff from partner organizations. Pupils’ enrollment and attendance rates, literacy
16
and attentiveness levels, and knowledge and use of health, hygiene, and dietary practices were 
measured. Similarly, data on teachers’ attendance, motivation, and knowledge and use of 
teaching practices were collected. 
EVALUATION TOOLS
The APFL IV baseline data collection utilized evaluation tools from previous phases of the APFL 
project. The tools include a literacy assessment and pupil survey, classroom and school 
observation checklists, school-based stakeholder surveys, and community FGD questionnaire, 
and government and partner KII questionnaires. The tools were reviewed by STS and CRS prior 
to data collection. Specific revisions were made to ensure that the literacy assessment followed 
best practices in assessment design and that the survey tools were responsive to APFL IV’s 
performance monitoring plan (PMP) and project-level implementation questions. 
Literacy Assessment 
STS administered a baseline literacy assessment to class 2 pupils to measure their core early 
grade reading skills. The assessment was adapted from a national literacy assessment tool 
originally developed by UNICEF. The assessment contained seven untimed subtasks, which 
were administered in English: alphabet naming, phonemic awareness, familiar word reading, 
invented word reading, reading passage, reading comprehension, and listening comprehension. 
Table 3 provides a summary of the subtasks. 
Table 3: Literacy Assessment Subtasks
Subtask Core Reading Skill Subtask Description
Alphabet Naming Alphabet knowledge
Provide the name of 51 letters presented in 
both uppercase and lowercase in a random 
order. 
Phonemic Awareness Phonemic awareness
Identify the words represented by 10 pictures 
and give the sound of the first letter of each 
word represented. 
Familiar word reading Word recognition
Read 40 familiar words that are randomly 
ordered and drawn from a list of frequent 
words. 
Invented word reading Decoding Make letter-sound correspondences through 
the reading of 25 simple nonsense words. 
Reading passage Decoding and reading Read a short, grade-appropriate passage of 
36 words with accuracy and little effort. 
Reading
comprehension
Reading
comprehension
Respond correctly to five questions, including
four literal questions and one inferential
question, about the passage read in the 
previous subtask.
Listening
comprehension
Listening
comprehension and
Listen to a text the enumerator reads aloud, 
and respond correctly to four questions, 
17
Subtask Core Reading Skill Subtask Description
oral language including three literal questions and one
inferential question, about the text. 
The literacy assessment was administered by enumerators at the school on tablets using 
Tangerine®, an electronic data collection software.
School-based Surveys and Observation Checklists
For a comprehensive picture of a sampled school’s environment, numerous data points were 
collected at the school level. Seven surveys and two observation checklists from previous 
phases of the APFL project were used to collection information at each school (Table 4). 
Table 4: School-based Surveys and Observation Checklists
Tool Types of information collected 
Pupil Survey
Availability of teaching and learning materials and activities; 
frequency and sufficiency of meals at home and school; 
knowledge of and demonstration of good health and hygiene 
practices. 
Teacher Survey 
Levels of teacher certification; in-service training and 
coaching; knowledge and use of teaching techniques; 
motivating factors; satisfaction with the APFL project. 
Head Teacher Survey
MBSSE status; enrollment and attendance data; teacher 
training, attendance, and retention information; school 
infrastructure details; teaching and learning materials
available; school activities and support structures. 
Food Preparer Survey Training received; knowledge of safe food preparation and 
storage practices; challenges in role, 
SMC Chairperson Survey Trainings received; committee operations; role in school 
feeding program. 
CTA Chairperson Survey Association operations; school engagement. 
MSG Head Survey Group operations and activities; knowledge of and use of 
nutrition, health, and sanitation practices. 
Classroom Observation 
Checklist
Physical attributes of the classroom; presence and use of 
teaching and learning materials in the classroom; evidence of 
pupil attentiveness. 
18
Tool Types of information collected 
School Observation Checklist 
Physical attributes of the school, including those overall and 
with classrooms, sanitation facilities, food preparation and 
storage areas, and other spaces; inventory of teaching and 
learning materials in the classrooms. 
Community Focus Group Discussions and Key Informant Interviews 
In addition to the surveys conducted at the school, the baseline evaluation included focus group 
discussions (FGD) with community members and in-depth key informant interviews (KIIs) with 
stakeholders from the Ministry of Basic and Senior and Secondary Education (MBSSE), the 
National School Feeding Secretariat (NSFS), and partner organizations. Enumerators 
conducted interviews using structured questionnaires from previous phases of the APFL project 
that were adapted by STS for the baseline evaluation. The interview guides for FGDs and KIIs 
can be found in Annex J. 
SAMPLING APPROACH AND SAMPLE SIZE
For the baseline evaluation, a two-stage cluster sampling approach was used. Attributes such 
as district location and APFL status—continuing versus new—were accounted for within the 
sample. First, schools were randomly selected as clusters. With each sampled school unit, 
enumerators conducted KIIs with the head teacher, the SMC chairperson, the CTA chairperson, 
the head of the MSG, and two food preparers. For the second stage of sampling, STS randomly 
selected ten pupils from those present in class 2 at each sample school, who participated in the 
literacy assessment and pupil survey, and up to three teachers representing classes 2, 3, and 4, 
who participated in the teacher survey and classroom observation. A list of appropriate 
replacement schools was also created in case the original sample schools were unavailable or 
difficult for enumerators to reach. 
The resulting target sample size was 70 schools and 700 pupils—ten pupils, five girls and five 
boys, from each school. Of the 70 schools, 43 schools (61.43 percent) are continuing from 
APFL III while 27 are new schools under APFL IV (38.57 percent). With one representative from 
each school, the target sample size for the head teachers, SMC chairpersons, CTA 
chairpersons, and MSG heads was 70 per survey. The project sought to survey all food 
preparers and teachers for classes 2, 3, and 4, leading to a maximum of 140 food preparers and 
210 teachers in the sample. A more detailed description of the sampling approach can be found 
in Annex A. 
DATA COLLECTION
This section describes baseline operational data collection, including enumerator training, data 
collection, and data management and analysis. 
Recruitment and Training of Enumerators
A local firm, Dalan Development Consultants, was contracted to conduct the baseline data 
collection in June 2019. Dalan recruited 46 participants for the data collection from their pool of 
19
data collectors, 16 of whom had prior experience collecting data for the APFL project during a 
previous phase. 
From June 10 to 14, 2019, STS and CRS trained the participants on the evaluation tools and 
protocols. The five-day training in Freetown covered the contents of the literacy assessment
subtasks and school-based surveys, administration protocols for the data collection software 
and use of tablets, ethical considerations, and the responsibilities of enumerators and 
supervisors during data collection. Although some sessions were held with the full group of 
participants, three concurrent trainings were held during the week—one for the enumerators 
responsible for the literacy assessment, one for the enumerators responsible for conducting the 
school-based surveys and observation checklists, and one for the enumerators responsible for 
the FGDs and KIIs. The training included one day of field testing in nearby non-intervention 
schools in Freetown. 
School-based Data Collection 
The baseline data collection was conducted in the Koinadugu and Falaba districts from June 17
to 24, 2019. Thirteen teams of three—consisting of two enumerators who administered the 
literacy assessment and pupil survey and one enumerator who conducted the school-based 
surveys—visited one school per day. Within each team, one enumerator was designated as the 
supervisor responsible for introducing the teams to the school and conducting the classroom 
and pupil sampling. 
Of the 70 sample schools, 69 were visited during the data collection. One school had been 
completely destroyed by cattle, and an appropriate replacement school was not available in the 
area. Further impacting the response rates, the enumerators found many schools did not yet 
have food preparers or MSGs as implementation of the food service had not yet begun under 
APFL IV. Enumerators also encountered many multi-level classrooms, which limited response 
rates for the classroom observations and teacher surveys. 
Table 5: Sample Targets and Response Rates
Group Target sample
number
Actual number of 
responses Response rate
Schools 70 69 98.57%
Pupils 700 682 97.43%
Teachers* 210 145 69.05%
Classroom observation 210 179 85.24%
School Observations 70 68 97.14%
Head Teachers 70 68 97.14%
Food Preparers** 140 87 62.14%
SMC Chairpersons 70 67 95.71%
CTA Chairpersons 70 61 87.14%
Mothers Support Group 
representatives** 70 66 94.29%
20
* Many schools had multi-level classrooms and teachers, so it was often not possible to interview three teachers at a sample school. 
** As of the baseline data collection, not all sampled schools had employed food preparers or MSGs yet. 
Focus Group Discussions and Key Informant Interviews 
During data collection, two teams of enumerators facilitated twelve FGDs—one all-male group 
and one all-female group in six communities within the intervention districts. The team from 
CRS and STS conducted a purposeful sample of communities for the FGD sessions in advance
of the data collection in order to ensure a wide range of geographic and programmatic 
experiences were represented in the qualitative data. Upon their arrival in each community, 
enumerator teams sought the approval of the community chief to lead FGDs. CRS provided a 
letter of introduction to support this process. If approval was granted, the community chief
assisted with the mobilization of participants—drawing from parents or caregivers of pupils in 
classes 1 through 6, teachers, youth leaders, religious leaders, members of the SILCs, and 
school representatives. Participants of the school-based surveys were excluded from FGDs. 
The community chief was also exempted from FGD sessions to avoid any bias or influence. 
One enumerator conducted four in-person KIIs with key stakeholders from APFL’s partners at 
Caritas Makeni, Ernest Bai Koroma University, the MBSSE, and the Ministry of Agriculture, 
Forestry, and Food Security. A member of STS’s team conducted three remote interviews, 
using Skype© or the phone, with members of the NSFS and the MoHS in Freetown.
Enumerators transcribed the FGDs and KIIs, and sessions were recorded digitally through audio 
recorders to minimize potential transcription errors.
DATA QUALITY, MANAGEMENT, AND ANALYSIS
Data Monitoring and Quality Assurance
Throughout data collection, Dalan’s field coordinator and STS’s program coordinator monitored
the incoming data. Dalan’s field coordinator visited multiple schools in person to conduct on-site 
spot checks and troubleshoot any issues encountered by teams in the field. Communication 
with the enumerator teams was maintained through a WhatsApp© group comprised of team 
supervisors; this allowed for wider communication and faster responsiveness when issues arose 
in the field. However, due to severe connectivity issues in the communities, many enumerator 
teams were unable to upload their data electronically each day. In some cases, data could not 
be uploaded until the teams returned to Freetown. This impeded the real-time data tracking 
activities of the field coordinator and program coordinator. 
Dalan’s staff ensured data collection procedures were followed and submitted a field report that 
logged any discrepancies in the number and type of data collected prescribed in the targeted 
sample. STS later cross-referenced these reports against the uploaded data from the Tangerine
software. Disposition codes were applied to categorize any issues that emerged during the data 
collection process. The coding and flagging procedures helped to ensure the nuanced contexts 
of data collection at the school level were sufficiently cataloged and considered during the data 
cleaning, analysis, and reporting process.
21
Data Analysis 
Sample Weighting
The analysis used sampling weights to produce more representative estimates in the sample of 
pupils. Although random sampling does not acknowledge that some pupils have a lower 
probability of being selected when they represent smaller subgroups within the population, 
sampling weights allow the analysts to account for these differences in probabilities.
STS computed the weights using background data available from each school in the sample 
populations, including the number of class 2 classrooms at the school and the number of pupils
in each classroom. CRS collected this information via the head teacher survey. Weights were 
applied when analyzing the literacy assessment and survey results. A combined school and 
pupil weight was used for all pupils, while the school weight was applied to all school-based 
surveys. 
Generation of Findings
Throughout July and August 2019, STS generated the following descriptive statistics using the 
baseline data:
• Mean scores: Average percentage of items answered correctly on a given subtask; 
• Zero scores: Proportion of pupils who were unable to answer a single item correctly on 
a given subtask; 
• Proportions: Proportion of respondents who replied in a specific way to an item; and
• Means: Average score on survey items. 
Analysts computed inferential statistics on subtask mean scores to determine differences in 
performance between girls and boys. Where detected, statistically significant differences are 
noted in tables. 
LIMITATIONS
The following limitations should be considered when reviewing the findings of the APFL IV 
baseline:
• Use of tools from previous APFL phases. The baseline data collection utilized several 
tools from previous APFL evaluations. While these tools were reviewed and updated to a 
certain level, the tools were not fully aligned with the most recent recommendations from 
literacy research. For example, the subtasks were untimed which allowed STS to 
calculate pupils’ accuracy—scored as the average number of items correct—but did not 
allow for pupils’ fluency to be measured, which is a critical component of pupils’ literacy.
Furthermore, 51 letters were included for the alphabet naming subtask. Current best 
practice is to include 100 letters arranged in a grid format. Lastly, a picture stimulus was 
used for the phonemic awareness subtask with is out of alignment with current practice 
for this subtask. 
Additionally, since the evaluation team relied on prior pilots of the surveys, STS was 
unable to examine the extent of any potential social desirability bias inherent in the tools, 
22
as well as their cultural relevance and appropriateness. STS believes no additional bias 
was introduced as a result of the training and recruitment of the enumerators was done 
in a way to ensure they had no inherent interest in the outcomes of the study. 
Lastly, the instructions for the data collection were in English. Based on the results from 
the listening comprehension task, it is likely that many students struggle with listening 
comprehension and therefore may not have understood the tasks. 
• Delays in uploading data due to poor connectivity. Poor connection in more remote 
communities was a critical challenge during data collection. While the data being 
collected was stored within the software on the tablets, enumerators were unable to 
upload the data to the server each day as instructed. This limited the ability of the field 
coordinator and program coordinator to properly monitor the data as it was coming in 
and resolve problems as they were happening. Despite these challenges, the data 
quality is strong thanks to redundant data quality processes put into place to address 
this potential challenge. 
• Lower response rates than anticipated for school surveys. The sampling approach 
called for three teachers to be surveyed and three classrooms to be observed at each 
school—one each for classes 2 through 4. However, when enumerators arrived, there 
were fewer eligible teachers and classrooms to include as a result of the multi-level 
classrooms. There were also fewer MSG heads and food preparers to survey than 
recommended as many sampled schools are new to the APFL project and have not yet 
established these positions. 
• Inherent bias in sampling children present on day of assessment. Pupils’ literacy 
assessment results may be biased towards the types of students who attend regularly 
and may exclude those pupils who are enrolled by do not attend regularly. However, this 
random method of sampling on the day of the assessment is preferable to sampling 
pupils in advance, as it may create opportunities for manipulation to have only high 
performers participate. This sampling approach will remain the same at future 
assessments and therefore the comparison across timepoints will be valid. 
23
FINDINGS
The findings presented here correspond to the APFL IV project’s results framework, SOs, and 
PMP’s indicators. Although baseline values are not required of all indicators, this report presents 
descriptive baseline findings to provide supplemental contextual information for the project.
SO1: IMPROVED LITERACY OF SCHOOL-AGED CHILDREN
The first SO of the APFL IV project is the improved literacy of school-aged children. 
Achievement of this SO is measured through indicator 1.0.0.1: Percentage of students who, by 
the end of two grades of schooling, demonstrate that they can read and understand the 
meaning of grade-level text (McGovern-Dole Indicator #1). According to the project’s PMP, this 
indicator will measure the proportion of pupils who attain a specified threshold at the end of 
class 2. The specified threshold used in this analysis was defined as the ability to correctly 
answer at least four of the five reading comprehension questions correctly.9
Baseline values for indicator 1.0.0.1 were captured by administering the APFL IV literacy
assessment to boys and girls at the end of class 2. Results for indicator 1.0.0.1 are presented in 
Figure 4. Per the definition of indicator 1.0.0.1, 5.58 percent of pupils at the end of class 2 could 
read and understand the meaning of grade-level text at baseline. The proportion meeting the 
threshold was lower for girls (3.11 percent) than it was for boys (7.77 percent). Overall, of the 
682 pupils assessed, 94.42 percent did not correctly answer at least four reading 
comprehension questions.10 
9 Catholic Relief Services, “Final Evaluation of Phase III of The McGovern-Dole Food For Education Project (MGD III) 
In Sierra Leone,” August 8, 2018. 10 Additional results comparing continuing and new APFL schools can be found in Annex B. 
24
Figure 4: Percentage of Pupils Reaching Literacy Threshold by Sex
The proportions of pupils who did not answer a single item correct across subtasks—known as 
zero scores—are presented in Figure 5. The proportion of pupils receiving zero scores was 
lowest on the alphabet naming subtask (8.66 percent) and highest on the phonemic awareness 
subtask (79.67 percent). Across all subtasks, boys had a lower proportion of zero scores than 
did girls.
Figure 5: Percentage of Pupils Receiving Zero Scores by Sex
3.11%
7.77%
5.58%
96.89%
92.23%
94.42%
Girls
Boys
Total
Met benchmark Did not meet benchmark
Note: Girls (n=334); Boys (n=348); Total (N=682)
10.15%
80.51%
61.78%
85.63%
84.64%
50.23%
89.41%
7.35%
78.93%
43.14%
63.73%
60.31%
45.94%
68.23%
8.66%
79.67%
51.89%
74.01%
71.73%
47.95%
78.17%
Alphabet naming (out of 51)
Phonemic awareness (out of 10)
Familiar word reading (out of 40)
Invented word reading
Reading passage (out of 36)
Reading comprehension (out of 5)
Listening comprehension (out of 6)
Girls Boys Total
Note: Girls (n=334); Boys (n=348); Total (N=682)
25
Mean scores for other literacy assessment subtasks are presented in the following section to 
allow for a better understanding of pupils’ reading performance. Statistical significance tests 
were performed to analyses the difference in mean scores between boys and girls; statistically 
significant differences are noted under each table.
Alphabet Naming
In the alphabet naming subtask, enumerators presented pupils with a grid of 51 letters in 
uppercase and lowercase and asked students to say the name of the letter.11 The alphabet 
naming subtask measures pupils’ knowledge of letters of the alphabet and their ability to 
recognize the graphemic features of each letter. 
Baseline results for the alphabet naming subtask are presented in Table 6. On average, pupils 
named 36.57 letters correctly out of 51. Boys had statistically significantly higher mean scores 
than did girls; boy, on average, named nearly five more letters than girls did. 
Table 6: Alphabet Naming Mean Scores by Sex (Correct out of 51)
Sex N Mean Score Standard Error
Girls 334 34.06 0.94
Boys 348 38.79** 0.83
Total 682 36.57 0.63
Note: Two asterisks (**) denotes that boys’ scores are statistically significantly higher than girls’ scores at the p<0.01 level.
Phonemic Awareness
For the phonemic awareness subtask, enumerators provided pupils with clipart pictures of 10 
common objects and read the name of the object out loud to the pupils. Pupils were asked to 
say the initial sound of the object’s name. The phonemic awareness subtask measures pupils’ 
awareness of phonemes and their ability to distinguish among multiple phonemes. 
Baseline results for the phonemic awareness subtask are presented in Table 7. Out of a total of 
10 possible items, pupils correctly identify the initial sound of 1.06 items on average. There was 
no significant difference between girls’ and boys’ scores.
Table 7: Phonemic Awareness Mean Scores by Sex (Correct out of 10)
Sex N Mean Score Standard Error
Girls 334 0.94 0.13
Boys 348 1.16 0.14
Total 682 1.06 0.09
11 This subtask was modified from the APFL III literacy assessment, which contained 26 letters in a diamond-shape. The number of 
items was increased for the APFL IV literacy assessment to ensure that each letter appeared both in its lowercase and uppercase 
forms. 
26
Familiar Word Reading
The familiar word reading subtask consisted of a list of 40 familiar words that were presented to 
pupils in a grid. Pupils were asked to read as many of the familiar words as they could out 
loud.12 Familiar word reading measures pupils’ sight-word recognition and decoding skills. 
Baseline results for the familiar word reading subtask are presented in Table 8. Pupils correctly 
read an average of 11.60 familiar words at baseline. Boys read nearly twice as many words 
correctly as girls did—14.98 familiar words versus 7.78 familiar words, respectively. The 
difference between girls’ and boys’ mean scores was statistically significant.
Table 8: Familiar Word Reading Mean Scores by Sex (Correct out of 40)
Sex N Mean Score Standard Error
Girls 334 7.78 0.67
Boys 348 14.98** 0.80
Total 682 11.60 0.54
Note: Two asterisks (**) denotes that boys’ scores are statistically significantly higher than girls’ scores at the p<0.01 level.
Invented Word Reading
For the invented word reading subtask, pupils were presented with a grid of 25 made-up words 
that follow the phonological and spelling rules of English but are not actual words in the 
language. Enumerators asked pupils to read aloud as many nonwords as they could.13 Invented 
word reading measures pupils’ decoding skills. 
Baseline results for the invented word reading subtask are presented in Table 9. Out of 25 
items, pupils correctly read 3.31 invented words on average. Boys outperformed girls, reading 
4.88 invented words correctly compared with 1.54, on average; this difference was statistically 
significant.
Table 9: Invented Word Reading Mean Scores by Sex (Correct out of 25)
Sex N Mean Score Standard Error
Girls 334 1.54 0.24
Boys 348 4.88** 0.38
Total 682 3.31 0.24
Note: Two asterisks (**) denotes that boys’ scores are statistically significantly higher than girls’ scores at the p<0.01 level.
12 The items included in the familiar word subtask were consistent across the APFL III and IV literacy assessments. The familiar 
words on the APFL IV assessment were rerandomized within lines. One item—“play”—appeared twice in the grid. 
13 Several updates were made to this subtask from APFL III to APFL IV. Four invented words that were homophones of either 
familiar words or proper nouns were modified by changing one letter. All invented words were presented in lowercase letters. 
27
Reading Passage and Reading Comprehension
For the reading passage and reading comprehension subtasks, pupils were presented with a 
short story of 41 words and were asked to read as much of the story out loud as they could. 
After finishing, enumerators read five comprehension questions—four direct and one 
inferential—out loud to pupils to test their understanding of the content of the story.14 These two 
subtasks measure decoding and reading comprehension.
Baseline results for the reading passage subtask are presented in Table 10. From a short story 
of 41 words, pupils correctly read 7.59 words on average. While boys correctly read about 26 
percent of the words (10.89 words), girls only read about 9 percent of the words (3.87 words) 
correctly. The difference in mean scores between girls and boys was statistically significant.
Table 10: Reading Passage Mean Scores by Sex (Correct out of 41)
Sex N Mean Score Standard Error
Girls 334 3.87 0.54
Boys 348 10.89** 0.76
Total 682 7.59 0.49
Note: Two asterisks (**) denotes that boys’ scores are statistically significantly higher than girls’ scores at the p<0.01 level.
Baseline mean scores for the reading comprehension subtask are presented in Table 11. 
Overall, pupils were able to answer less than one reading comprehension question correctly at 
baseline. Boys had statistically significantly higher mean scores than girls did on the reading 
comprehension subtask. 
Table 11: Reading Comprehension Mean Scores by Sex (Correct out of 5)
Sex N Mean Score Standard Error
Girls 334 0.31 0.05
Boys 348 0.86** 0.08
Total 682 0.60 0.05
Note: Two asterisks (**) denotes that boys’ scores are statistically significantly higher than girls’ scores at the p<0.01 level.
The distribution of pupils able to answer reading comprehension questions correctly is detailed 
in Table 12. Overall, nearly four out of five pupils (78.17 percent) were unable to correctly 
answer a single reading comprehension correctly.15 
14 Three items on the reading passage were updated from the APFL III to the APFL IV literacy assessment. All five comprehension 
questions were also updated to better align with the story and with common early grade literacy assessment guidance. 
15 Significance tests were not conducted on the distribution of correct reading comprehension questions by sex. 
28
Table 12: Distribution of Correct Reading Comprehension Questions by Sex
Number of Questions 
Correct Girls Boys Total
0 89.41% 68.23% 78.17%
1 1.19% 7.22% 4.39%
2 2.69% 7.05% 5.00%
3 3.60% 9.73% 6.85%
4 1.95% 3.56% 2.80%
5 1.17% 4.21% 2.78%
Listening Comprehension
The listening comprehension subtask consists of a short story of 40 words, which was read out 
loud by the enumerator to the pupils. The enumerator then asked the pupil four comprehension 
questions related to the story—three direct and one inferential.16 Listening comprehension 
measures pupils’ overall oral language comprehension and vocabulary. Listening 
comprehension subtask complements the reading passage and comprehension subtasks as it 
enables a better understanding of whether pupils’ comprehension difficulties are a result of 
reading skills or of overall language comprehension.
Baseline results for the listening comprehension subtask are presented in Table 13. Out of a 
possible four questions, pupils correctly answered, on average, 1.06 questions. There was no 
statistically significance difference between boys’ and girls’ performance.
Table 13: Listening Comprehension Mean Scores by Sex (Correct out of 4)
Sex N Mean Score Standard Error
Girls 334 1.00 0.07
Boys 348 1.11 0.07
Total 682 1.06 0.05
IR1.1 Improved Quality of Literacy Instruction
The first intermediate result (IR) under SO1 is the improved quality of literacy instruction. Four 
outputs are associated with this IR:
• 1.1.1 More consistent teacher attendance
• 1.1.3 Improved literacy instructional materials
• 1.1.4 Increased skills and knowledge of teachers
• 1.1.5 Increased skills and knowledge of school administrators
Baseline findings are presented for each output to provide relevant contextual information at the 
start of APFL IV. 
16 One of the listening comprehension questions was updated from the APFL III to the APFL IV literacy assessment. 
29
1.1.1 More Consistent Teacher Attendance
At baseline, head teachers were asked a series of questions about teacher attendance and 
documentation of teacher attendance at the school level. Of the 68 head teachers interviewed, 
62 (92.06 percent) reported that their school had a time book for recording daily teacher 
attendance. 
Teacher attendance results are reported in Figure 6. On the day of the interviews, 320 of an 
employed 363 (88.17 percent) teachers were present. Overall, 88.73 percent of female teachers
and 88.04 percent for male teachers were present on the day their school was interviewed. 
Figure 6: Total Number of Teachers and Number of Teachers in Attendance
1.1.3 Improved Literacy Instructional Materials
To understand the resources available in schools at baseline, enumerators inventoried
classroom resources and furniture. Key findings from their observations are presented.
The classroom resources observed at baseline are detailed in Figure 7. Of the 179 classrooms 
observed, roughly three in four (72.52 percent) had a chalkboard or a blackboard. Less than half 
(41.92 percent) had a teacher’s table and chair. Posters or messages about health or sanitation
were present in 22.72 percent of classrooms.
363
294
69
320
259
61
0 50 100 150 200 250 300 350 400
Total
Male
Female
Total Present
Note: Total (N=68)
30
Figure 7: Classroom Resources Observed
Enumerators also recorded the presence and prevalence of textbooks or readers in the 
classroom (Figure 8). In more than half of the classrooms (52.87 percent), textbooks or readers 
were only used by the teacher. When textbooks or readers were used by pupils, they were more 
often available one per pupil (11.93 percent) than shared by two (11.62 percent) or by three or 
more (8.38 percent). In 15.20 percent of classrooms, no textbooks or readers were observed.
Figure 8: Observed Use of Textbooks or Readers in the Classroom 
72.52%
41.92%
28.54% 31.69%
22.72%
27.48%
58.08%
71.46% 68.31%
77.28%
Chalkboard or
blackboard
Teacher's table and
chair
Children's work on
wall
Vocabulary words or
alphabet strips
Posters or messages
about health or
sanitation
Yes No
Note: Total (N=179)
52.87%
11.93%
11.62%
8.38%
15.20%
By teacher only
By pupils, one each
By pupils, shared by two
By pupils, shared by three or more
No books/readers
Note: Total (N=179)
31
1.1.4 Increased Skills and Knowledge of Teachers
At baseline, 145 classroom teachers—50 from class 2, 46 from class 3, and 49 from class 4—
were interviewed to gain an understanding of their credentials; their knowledge of good 
instructional practices and teaching techniques; the type of support they receive from coaches, 
head teachers, and MBSSE supervisors; and their teaching motivations.
Table 14 provides a summary of the demographic and educational characteristics of the 
classroom teachers interviewed. About four in every five teachers (82.60 percent) interviewed 
were male, and nearly half (46.15 percent) reported having a teaching certificate. Of those
individuals with a teaching certificate, 57.90 percent had a Teachers Certificate (TC), and 33.99 
percent had a TC-lower certificate. The majority (61.18 percent) of teachers report that their 
highest qualification completed was a West African Senior School Certificate Examination
(WASSCE). 
Table 14: Classroom Teacher Characteristics
Characteristic
Total
N % 
Sex
Female 26 17.40%
Male 119 82.60%
Type of teaching certificate
Has teaching certificate 68 46.15%
Teacher Elementary Certificate (TEC) 4 6.32%
Teachers Certificate Lower 24 33.99%
Teachers Certificate 39 57.90%
Teachers Certificate Higher 1 1.49%
Highest qualification
Basic Education Certificate Examination 29 19.72%
West African Senior School Certificate 
Examination 88 61.18%
Ordinary Level 27 18.41%
Other 1 0.70%
Note: Total (N=145)
Teachers were also asked about their participation in the APFL IV project and in other types of 
trainings, and their responses are presented in Table 15. About half (50.73 percent) of teachers 
participated in the APFL IV project. Of the 145 teachers interviewed, 35.61 percent participated 
in a diagnostic teaching methodologies training during the academic year, and 29.88 percent 
responded that they or another teacher in their school had been trained in life skills areas.
32
Table 15: Classroom Teacher Training Participation
Characteristic N Percentage of total
Has participated in the Food for Education
program in the last academic year 75 50.73%
Has participated in a diagnostic teaching 
methodologies training by a literacy coach in 
academic year
54 35.61%
Engaged in a distance education course that 
will lead to a teaching certificate 32 21.15%
Has been trained in life skills areas in this 
school (responding teacher or any other 
teacher in the school)
46 29.88%
 Note: Total (N=145) 
Enumerators asked teachers about their level of knowledge of teaching techniques critical to the 
APFL IV program; results are presented in Figure 9. Over half of teachers reported knowing 
nothing about fluency, developing independent learners, grouping for instruction, and adapting 
for individual differences. Less than one-quarter of teachers reported having knowledge and 
confidence in their abilities across all techniques. The only techniques in which any teachers 
reported having “excellent” knowledge and skill were assessment (1.39 percent) and effective 
questioning (0.58 percent).
Figure 9: Teacher Knowledge of Teaching Techniques
50.35%
59.69%
43.23%
45.91%
34.46%
46.98%
43.09%
58.49%
61.74%
67.53%
39.19%
35.58%
48.31%
40.72%
41.79%
34.58%
39.07%
33.31%
32.71%
26.93%
10.46%
4.74%
8.46%
13.37%
22.36%
17.86%
17.85%
8.20%
5.55%
5.54%
1.39%
0.58%
Word recognition and phonics
Fluency
Vocabulary
Comprehension
Assessment
Effective questioning
Motivation
Developing independent learners
Grouping for instruction
Adapting for individual differences
I know
nothing about
this
I know about
this, but I
don't know
how to use
I know about
this and have
some
confidence in
my abilities in
this area
I have
excellent
knowledge
and skill in
this area
Note: Total (N=145)
33
Of the 145 teachers interviewed, 75.97 percent reported having a lesson plan for that day, and 
85.01 percent had access to a teacher’s guide, either in the head teacher’s office or in their 
classroom. 
Table 16 reports on teachers’ responses regarding the frequency of coaching and mentoring 
sessions they experienced during the past month and the previous academic year. Just over 
half (50.94 percent) of teachers said they were observed or mentored by a CRS literacy coach 
in the past month, and 56.43 percent of teachers reported being observed or mentored by an 
MBSSE inspector during the prior academic year. Almost all teachers (94.85 percent) said they 
had been observed or mentored by their head teacher during the previous academic year; of 
those, 87.15 percent said they were observed or mentored more than twice during the year.
Table 16: Coaching and Mentoring Frequency
Type N Percentage of 
total
Observed or mentored by CRS literacy coach in past month 78 50.94%
Once in the month 38 46.99%
Twice in the month 18 24.12%
More than twice in the month 22 28.89%
Observed or mentored by head teacher in academic year 138 94.85%
Once in the year 11 7.35%
Twice in the year 8 5.50%
More than twice in the year 119 87.15%
Observed or mentored by MBSSE inspector this year 80 56.43%
Once in the year 20 23.60%
Twice in the year 28 36.19%
More than twice in the year 32 40.21%
Note: Total (N=145) 
Finally, teachers were asked about what aspects of their job they find motivating; results are 
presented in Figure 10. Out of 18 categories included, the largest proportion of teachers were 
somewhat or highly motivated by interpersonal relationships with colleagues (82.39 percent) 
and interpersonal relationships with the head teacher (81.25 percent). Roughly half (50.15 
percent) said they were somewhat or highly unmotivated by salary, and 30.47 percent said they 
were somewhat or highly unmotivated by job security.
34
Figure 10: Teacher Motivating Factors
1.1.5 Increased Skills and Knowledge of School Administrators
At baseline, 68 head teachers were asked questions about the type of training they received 
during the school year, and enumerators took note of specific improved tools or techniques 
demonstrated by head teachers. Out of those, 38 head teachers (55.39 percent) reported that 
they benefited from training by CRS or TALLE in diagnostic teaching methodologies in the past 
12 months. 
Enumerators also noted if specific techniques and tools were visible in the head teacher’s office, 
and findings are presented in Figure 11. Out of the five techniques, the most frequently 
observed was a logbook or visitors’ book, which was seen in 72.03 percent of head teacher’s 
offices. About half (50.43 percent) of head teachers had a teacher duty roster clearly displayed 
in their office.
15.80%
12.79%
15.69%
9.96%
50.15%
30.47%
20.69%
13.20%
13.66%
26.11%
23.81%
18.33%
12.28%
14.52%
15.58%
11.64%
6.87%
8.72%
7.64%
9.03%
8.31%
7.64%
19.00%
14.70%
15.94%
5.55%
11.39%
9.69%
16.09%
16.24%
12.09%
15.45%
14.74%
12.65%
10.42%
14.73%
76.56%
78.18%
76.00%
82.39%
30.84%
54.83%
63.38%
81.25%
74.95%
64.21%
60.10%
65.43%
75.63%
70.03%
69.68%
75.72%
82.71%
76.55%
Recognition
Potential for professional growth
Supervision by superiors
Interpersonal relationships with colleagues
Salary
Job security
Status
Interpersonal relationships with head teacher
Sense of achievement
Working conditions
MBSSE policies
Teacher evaluation
Responsibility
Potential for advancement
Work itself
Factors in personal life
Interpersonal relationships with students
Sense of accountability
Somewhat or highly unmotivating Neither Somewhat or highly motivating Note: Total (N=145)
35
Figure 11: Percentage of Head Teacher Techniques and Tools Observed
IR1.2 Improved Attentiveness
The second IR under SO1 is improved attentiveness. Enumerators observed pupils’ 
attentiveness during class and evaluated it on a three-point scale of little, moderate, or 
extensive evidence of pupils following the teacher’s instructions during the class period. This 
measure included observations of students listening and working without distraction, students' 
participation in lessons, and students asking questions or seeking help. Boys and girls were 
evaluated separately. On average, about 64.16 percent and 62.92 percent of girls and boys, 
respectively, were attentive to instructions.
 
Additionally, two outputs are associated with this IR:
• 1.2.1 Reduced short-term hunger
• 1.2.2 Increased access to food (school feeding)
1.2.1 Reduced Short-term Hunger
Enumerators asked pupils about their access to food and feeding both at home and at school 
throughout a week. Results are presented in Figure 12. About four in five pupils reported that 
they had eaten at home or elsewhere before coming to school. While 11.12 percent of pupils 
said they were given food in school in the previous day, only 1.42 percent reported having been 
given food on the day of the assessment. At the time of data collection, CRS was not 
implementing their school feeding program at any of the sampled schools, as APFL III activities 
had ended in continuing schools and the APFL IV activities had not yet begun in new schools. 
As such, this low percentage of students reporting receiving food at school was expected. 
72.03%
51.67% 50.43%
60.21% 63.15%
Log book (visitor book)
available
Teaching master￾timetable displayed
Teacher duty roster
clearly displayed
Visual teaching & learning
materials displayed
Inventory Book or other
school records properly
organized & updated
Note: Total (N=68)
36
Figure 12: Percentage of Pupils Receiving Food by Sex
Pupils who reported having received food at school on the day of the assessment were asked 
about their level of hunger after eating; this question is in response to indicator 1.2.1.1
Percentage of students in target schools who indicate that they are hungry or very hungry 
during the school days. Findings in Table 17 indicate that only 10 pupils reported receiving food 
in school on the day of the assessment. These results are corroborated by head teachers—only 
five (6.12 percent) head teachers reported that pupils had have or will receive food from CRS on 
the day of the assessment. When asked the reason that no food will be provided, 60.46 percent 
said that food supplies had run out and 36.43 percent said that there was no feeding program 
yet established at the school. This was anticipated as APFL IV activities had not yet begun at 
any school. 
Of those pupils who received food at school on the day of the assessment, five (0.64 percent) 
reported not being hungry at all or had enough food and five (0.78 percent) reported being 
somewhat hungry or had some food but not enough. No pupils reported being very hungry after 
receiving food at school on the day of the assessment.
Table 17: Pupils’ Reported Hunger Status or Access to Food by Sex
Response
Girls Boys Total
n % n % N % 
Not hungry at all (had enough food) 2 0.36% 3 0.90% 5 0.64%
Somewhat hungry (had some food 
but not enough) 2 0.43% 3 1.08% 5 0.78%
Very hungry 0 0.00% 0 0.00% 0 0.00%
Missing/did not receive food at school 
on day of assessment 328 99.21% 342 98.03% 670 98.58%
83.28%
12.28%
0.79%
8.08%
84.12%
10.09%
1.97%
13.59%
83.73%
11.12%
1.42%
11.01%
Ate at home or elsewhere before
coming to school this morning
Given food in school yesterday
Given food in school today
Given food every day of the week
Girls Boys Total
Note: Girls (n=332); Boys (n=348); Total (N=680)
37
IR1.3 Improved Pupil Attendance
The third IR under SO1 is improved pupil attendance. According to CRS’s baseline monitoring 
data across the 309 intervention schools, the overall attendance rate is 61.8 percent for pupils—
70.9 percent for boys and 52.7 percent for girls. For this monitoring data, head count attendance 
data were collected and averages were taken.
While the data above is representative of the overall monitoring data on attendance collected by 
CRS across all project schools, the study was able to collect more detailed information on the 
sampled schools. Analysis of the data from sampled schools only is included below.
For sampled schools only, enumerators asked head teachers of sample schools to report the 
total number of pupils enrolled and attending school, by class and sex, to respond to indicator
1.3.0.1 average student attendance rate in USDA supported classrooms/schools. Attendance 
rates, calculated as the number of pupils attending divided by the number of pupils enrolled, are 
presented in Figure 13. Within the sample schools, the average school-level attendance rate 
was 67.56 percent, with girls attending at a rate of 66.42 percent and boys at a rate of 68.56 
percent. Boys appeared to have slightly higher attendance rates overall (68.56 percent) and 
across class, except for class 4, in which girls had higher attendance rates.
Figure 13: Attendance Rates by Class and Sex
Three outputs are associated with this IR:
• 1.3.3 Improved school infrastructure
• 1.3.4 Increased pupil enrolment
• 1.3.5 Increased community understanding of the benefits of education
64.37%
65.66%
66.52%
72.95%
69.78%
62.25%
66.42%
65.07%
67.85%
69.76%
72.03%
73.32%
74.49%
68.56%
64.74%
66.79%
68.30%
72.45%
71.67%
69.23%
67.56%
Class 1
Class 2
Class 3
Class 4
Class 5
Class 6
Total (school)
Girls Boys Total
Note: Total (N=68)
38
1.3.3 Improved School Infrastructure
Enumerators observed the physical infrastructure of the 69 sample schools, and key findings 
are presented in Table 18. A majority of sample schools (89.77 percent) had corrugated metal 
sheets made of zinc as the main material of the roof; most schools had polished concrete walls 
(60.57 percent) and concrete floors (68.93 percent). About half (51.09 percent) of schools had 
kitchens available for cooking food, while 62.81 percent of schools had storerooms or storage 
facilities.
Table 18: Sample School Infrastructure Characteristics
Characteristic n % of Total
Main material of roof
Corrugated metal sheets (zinc) 61 89.77%
Asbestos 0 0.00%
Concrete 0 0.00%
Thatch 6 8.78%
Tarpaulin (plastic sheet) 1 1.45%
Main material of walls of building
Concrete polished wall 42 60.57%
Mud polished 11 15.45%
Concrete unpolished wall 1 1.47%
Mud unpolished 8 13.50%
Metal sheets (pan body) 0 0.00%
Thatch 3 4.66%
Tarpaulin (plastic sheet) 0 0.00%
Other 3 4.35%
Main material of floor of building
Concrete floor 48 68.93%
Earth floor 20 31.07%
Kitchen available for cooking food 36 51.09%
Storeroom or storage facility in school 45 62.81%
Note: Total (N=68) 
1.3.4 Increased Pupil Enrollment 
For the purposes of this baseline evaluation, indicator 1.3.4.1—Number of students enrolled in 
schools receiving USDA assistance—is calculated using the number of pupils formally enrolled 
in APFL IV schools. Across the 309 APFL IV intervention schools, the total enrollment at 
baseline is 44,511 pupils—23,356 boys and 21,155 girls—according to the CRS baseline 
monitoring data. 
39
While the data above is representative of the overall monitoring data on attendance collected by 
CRS across all project schools, the study was able to collect more detailed information on the 
sampled schools. Analysis of the data from sampled schools only is included below.
For sampled schools only, enumerators asked head teachers of sample schools to provide the 
total number of enrolled pupils by class and sex. At the sample schools, the overall average 
enrollment per school (classes 1 through 6) was 176.51 pupils; the average school enrollment 
was 82.35 girls and 94.16 boys. 
The average enrollment by class and sex is presented in Figure 14. The average total 
enrollment was highest for Class 1, with 63.54 pupils. Average enrollment decreased as the
class level increased, with the smallest average class size in reported in class 6 (8.87 pupils). 
The average enrollment of girls was lower than boys within each class level.
Figure 14: Average Pupil Enrollment by Class and Sex
1.3.5 Increased Community Understanding of the Benefits of Education
Dropout rates by class and sex were calculated to respond to indicator 1.3.5.3: Percentage of 
students in classes 3 through 6 who dropped out of school at the end of the school year. Head 
teachers provided the total number of enrolled pupils by class and sex and the total number of 
dropouts; the dropout rate is calculated as the total dropouts divided by the total enrollment.
Dropout rates by class and sex are presented in Figure 15. The average dropout rate across all 
classes and both sexes was 4.56 percent. Class 5 had the highest dropout rate—5.56 percent 
of pupils dropped out by the end of the school year. There was no observable trend in dropouts 
by sex and class, although the average dropout rate for boys across all classes was higher than 
that for girls—4.87 percent versus 4.21 percent, respectively.
30.06
17.93
13.52
10.19
6.84
3.81
33.47
18.93
16.44
12.37
7.88
5.06
63.54
36.86
29.95
22.56
14.72
8.87
Class 1
Class 2
Class 3
Class 4
Class 5
Class 6
Note: Total (N=68) Girls Boys Total
40
Figure 15: Dropout Rates by Class and Sex
SO2 INCREASED USE OF GOOD HEALTH AND DIETARY PRACTICES
The second SO of the APFL IV project is the increased use of good health and dietary 
practices. Results for indicator 2.0.0.3 Percentage of participants of community-level nutrition 
interventions who practice promoted infant and young child feeding behaviors were calculated 
by asking the heads of MSGs about their knowledge of infant and young child feeding (IYCF) 
behaviors. Enumerators asked respondents to name things that a mother can do to support 
good IYCF practices; they marked the practices that were mentioned.17 Findings are presented 
in Figure 16. Out of the 59 MSG members responding to this item, 98.27 percent were able to 
name at least one IYCF behavior. Over half of respondents named early initiation of 
breastfeeding within one hour of birth (61.99 percent) and exclusive breastfeeding for the first 
six months of life (56.17 percent). Only 15.97 percent of respondents named the use of fortified 
complementary foods or vitamin-mineral supplements; 1.73 percent of respondents were unable 
to mention any behaviors. 
17 A total of 66 MSG heads were interviewed. 
4.50%
3.33%
4.40%
4.01%
5.49%
3.68%
4.21%
5.41%
4.78%
4.63%
3.99%
5.62%
3.34%
4.87%
4.98%
4.08%
4.53%
4.00%
5.56%
3.49%
4.56%
Class 1
Class 2
Class 3
Class 4
Class 5
Class 6
Total (school)
Note: Total (N=68) Girls Boys Total
41
Figure 16: Knowledge of Infant and Young Child Feeding Behaviors
At baseline, pupils were asked about the types of foods that they ate the previous day; this was 
done to better understand their dietary intake and the types of foods consumed. Results are 
presented in Figure 17. Nearly all pupils (98.24 percent) reported having eaten grains, roots, 
and tubers in the previous day. About four in five pupils (78.47 percent) consumed vegetables, 
and just over half (55.81 percent) consumed flesh foods. Only 4.05 percent had consumed eggs 
the previous day.
Figure 17: Percentage of Pupils Consuming Selected Food Groups
61.99%
56.17%
47.55%
29.60%
26.40%
15.97%
20.87%
38.01%
43.83%
52.45%
70.40%
73.60%
84.03%
79.13%
Early initiation of breastfeeding within 1 hour of birth
Exclusive breastfeeding for the first 6 months of life
Introduction of complementary (solid) foods at 6 months together
Continue frequent, on-demand breastfeeding until 2 years of age or beyond
Gradually increase food consistency and variety
Use fortified complementary foods (solid foods) or vitamin-mineral
supplements as needed
During illness, increase fluid intake including more breastfeeding, and offer
soft, favourite foods
Mentioned Not mentioned
Note: Total (N=59)
98.24%
48.96%
20.11%
55.81%
4.05%
66.17%
78.47%
2.93%
Grain, roots
and tubers
Legumes and
nuts
Dairy
products
Flesh food Eggs Fruits Vegetables Other foods
eaten
Note: Total (N=679)
42
Using these results, pupils were grouped into those receiving a minimum acceptable diet—
defined as consuming four of the seven food groups in the previous day.18,19 The percentage of 
pupils consuming a minimum acceptable diet is presented in Figure 18.20 Out of 682 pupils
surveyed, 44.72 percent consumed at least four of seven food groups in the previous day.
Figure 18: Percentage of Pupils Consuming a Minimum Acceptable Diet (Four of Seven Food Groups) by Sex
IR2.1 Improved Knowledge of Health and Hygiene Practices
The first IR under SO2 is improved knowledge of health and hygiene practices. To respond to 
this IR at baseline, pupils were asked to name things they can do to have good health and 
hygiene; enumerators coded their responses based on a set of 12 possible good practices. 
Responses are presented in Figure 19. Pupils most frequently mentioned wearing clean 
clothes—three in four (75.34 percent) named this practice. About the same proportion of pupils
mentioned eating good food and cutting and keeping nails clean—69.40 percent and 69.02 
percent, respectively. The practices that the fewest pupils mentioned included brushing teeth 
twice a day (30.98 percent) and hair braiding (24.66 percent).
Using these results, results for indicator 2.1.1 Percentage of students in target schools who 
achieve a passing score on a test of good health and hygiene practices was calculated. 
Assuming a passing score of 50 percent of practices named, 287 pupils (42.74 percent) 
achieved a passing score for good health and hygiene practices. The proportion of girls and 
boys achieving passing scores were 43.53 percent and 42.04 percent, respectively.
18 CRS has based this definition on the World Health Organization’s (WHO) guidelines for minimum dietary diversity for infants and 
young children. This definition was also used for the APFL Phase III endline evaluation. 
19 “Data Sources and Inclusion Criteria.” World Health Organization. World Health Organization, December 23, 2013. 
https://www.who.int/nutrition/databases/infantfeeding/data_source_inclusion_criteria/en/. 20 The seven food groups were grain, roots, and tubers; legumes and nuts; dairy products; flesh foods; eggs; fruits; and vegetables. 
The “other foods eaten” category was not considered in the indicator calculation. 
45.80%
43.76%
44.72%
54.20%
56.24%
55.28%
Girls
Boys
Total
Received minimum acceptable diet Did not receive minimum acceptable diet
Note: Girls (n=334); Boys (n=348); Total (N=682)
43
Figure 19: Percentage of Pupils Naming Good Health and Hygiene Practices
IR2.2 Increased Knowledge of Safe Food Preparation and Storage Practices
The second IR under SO2 is increased knowledge of safe food preparation and storage 
practices. To respond to this IR at baseline, enumerators interviewed food preparers to learn 
about their knowledge of safe food preparation and storage. Enumerators also observed kitchen 
and storage facilities at each school. Key findings are presented. 
As previously reported in Table 18, only 51.09 percent of schools had a kitchen available for 
cooking food; characteristics of those school kitchens are presented in Table 19. A majority 
(59.34 percent) of school kitchens had roofs of corrugated metal sheets made of zinc, but just 
over half (52.16 percent) did not have any walls. Nearly three in four kitchens (74.13 percent) 
had earth floors, and one-third (30.15 percent) of school kitchens had spoon and plate shelves.
75.34%
69.40%
69.02%
63.93%
57.64%
51.72%
49.48%
48.28%
42.36%
36.07%
30.98%
24.66%
Wear clean clothes
Eat good food
Cut and keep our nails 
clean
Keep our enviroment 
clean and safe
Drink safe water that has 
been treated, stored and 
retrieved properly
Deposit trash into a 
trash/dust bin
Wash hands with soap and 
clean water after using the 
latrine
Wash our body daily
Wash hands with soap and 
clean water before eating
Keep latrines clean
Brush our teeth twice a 
day
Hair braiding
Note: Total (N=679)
44
Table 19: Sample School Kitchen Characteristics
Characteristic n Percentage of total
Material of roof of kitchen
Corrugated metal sheets (zinc) 21 59.34%
Asbestos 0 0.00%
Concrete 0 0.00%
Thatch 15 40.66%
Tarpaulin (plastic sheet) 0 0.00%
Material of wall of kitchen
Concrete polished wall 4 10.97%
Mud polished 3 8.62%
Concrete unpolished wall 7 20.12%
Mud unpolished 1 2.87%
Metal sheets (pan body) 0 0.00
Thatch 2 5.26%
Tarpaulin (plastic sheet) 0 0.00
No wall 19 52.16%
Material of floor of kitchen
Concrete floor 9 25.87%
Earth floor 27 74.13%
Wooden floor 0 0.00%
Has spoon and plate shelves 11 30.15%
Has rack/pallet for drying plates and spoons 19 52.65%
Note: Total (N=36) 
As reported in Table 18, 62.81 percent of the schools observed had a storeroom or storage 
facility; the characteristics of those spaces are presented in Table 20.21 All storerooms and 
storage facilities had a door with a lock, and all had ventilation blocks, either with mesh (70.84 
percent) or without mesh (29.16 percent). Nearly all (97.42 percent) storerooms and storage 
facilities had roofs of corrugated metal sheets made of zinc and most (92.26 percent) had 
polished concrete walls. 
21 Although 45 schools had a storeroom or storage facility, the enumerators were not able to visit four of these spaces because of 
their distance from the school. 
45
Table 20: Sample School Storeroom or Storage Facility Characteristics
Characteristic n % of Total
Has door with lock 41 100.00%
Has ventilation blocks
Yes, with mesh 29 70.84%
Yes, without mesh 12 29.16%
No ventilation blocks at all 0 0.00%
Material of roof of storeroom/storage facility
Corrugated metal sheets (zinc) 40 97.42%
Asbestos 0 0.00%
Concrete 1 2.58%
Thatch 0 0.00%
Tarpaulin (plastic sheet) 0 0.00%
Material of wall of storeroom/storage facility
Concrete polished wall 38 92.26%
Mud polished 3 7.74%
Concrete unpolished wall 0 0.00%
Mud unpolished 0 0.00%
Metal sheets (pan body) 0 0.00%
Thatch 0 0.00%
Tarpaulin (plastic sheet) 0 0.00%
Material of floor of storeroom/storage facility
Concrete floor 41 100.00%
Earth floor 0 0.00%
Wooden floor 0 0.00%
Food stacked on pallet 27 65.65%
Food store clean 38 92.26%
Food store has been fumigated in last 6 
months 30 71.63%
Note: Total for the first item (N=45); total for the remaining items (N=41) 
To calculate results for indicator 2.2.1 Percentage of food preparers at target schools who 
achieve a passing score on a test of safe food preparation and storage—enumerators asked 
food preparers to name things they can do to prepare and store food safely. Assuming a 
passing score means the respondent named eight of the eleven monitored practices, 33 out of 
87 food preparers (37.25 percent) achieved a passing score at baseline.22 
22 As per the APFL IV PMP, indicator 2.2.1 should be disaggregated by sex; however, at baseline, data on the food preparers’ sex 
were not captured. 
46
The specific practices named by the food preparers are detailed in Figure 20. Across schools, 
89.83 percent of food prepares named washing hands with clean water and soap before 
handling food, and 81.53 percent named washing cooking utensils and all dishes with clean 
water and soap and drying. 
Figure 20: Knowledge of Safe Food Preparation and Storage Practices
IR2.4 Increased Access to Clean Water and Sanitation Services 
The fourth IR under SO2 is increased access to clean water and sanitation services. According 
to CRS’s baseline monitoring data of all intervention schools, 161 of the 309 schools are using 
an improved water source. 
For sampled schools only, enumerators observed the presence of water and sanitation services 
at each of the 68 sample schools. Findings on water source characteristics from the sample 
schools, which correspond to indicator 2.4.1 Number of schools using an improved water 
source, are presented in Table 21. About two out of every five sample schools (39.98 percent) 
had a functional drinking water facility; of those, 66.95 percent were chlorinated. The most 
prevalent type of water facility was a hand pump well (68.08 percent), followed by a borehole 
with a pump (12.81 percent). For those schools without a functional water facility, 28.64 percent 
had a broken facility, 25.60 percent had no water, and 45.76 percent had facilities that were not 
functioning for other reasons. 
89.83%
81.53%
80.98%
67.12%
61.82%
60.28%
37.84%
41.80%
39.87%
38.74%
15.61%
10.17%
18.47%
19.02%
32.88%
38.18%
39.72%
62.16%
58.20%
60.13%
61.26%
84.39%
Wash hands with clean water and soap before handling food
Wash the cooking utensils and all dishes with clean water and soap; and then
dry them
Sweep the kitchen and environment where food is prepared
Wash the food items before cooking
Kitchen or environment should be free from animals
Cover the cooked food after dishing
Store the cooked food in a clean place (room or dinning)
Storage should be free from flies
Put cleaned utensils on a platform (rack/pallet) – i.e. do not put cleaned 
utensils on ground 
Wear kitchen apron or apparel when handling food
Cooked, or ready-to-eat food shouldn’t be handled with bare hands (Use 
tongs, spatulas, spoons, or disposable gloves)
Mentioned Not mentioned Note: Total (N=87)
47
Table 21: Sample School Water Facility Characteristics
Characteristic n Percentage of total
Functional drinking water facility at school 28 39.98%
Main water facility is chlorinated 19 66.95%
Type of water facility
Tap/pipe borne water 1 3.67%
Hand pump well 19 68.08%
Borehole with pump 4 12.81%
Ordinary well (protected) 1 3.67%
Ordinary well (unprotected) 1 4.41%
Others 2 7.35%
Reason for water facility not functioning
Broken down 12 28.64%
No water/dried 10 25.60%
Other 18 45.76%
Note: Total for the first item (N=68); total for second item (N=28); total for the third item (N=28); total for the fourth item (N=40) 
For sanitation facilities, CRS’s baseline monitoring data shows 228 of the 309 intervention 
schools have improved sanitation facilities. Findings on sanitation facilities at the 68 sample 
schools, which correspond to indicator 2.4.2 Number of schools with improved sanitary facilities, 
are presented in Table 22. Out of the 68 schools observed, 74.26 percent had a functioning 
toilet or latrine; of those, 86.94 percent were separated by sex. About three in every five schools 
(61.05 percent) had toilets or latrines with all rooms cleaned, and 57.10 percent of schools with 
toilets or latrines had a place for handwashing. 
Table 22: Sample School Sanitation Facility Characteristics
Characteristic n Percentage of total
Functioning toilet/latrine in school 51 74.26%
Latrines/toilets separated by sex 45 86.94%
Pupils and teachers have separate 
toilets/latrines 37 72.07%
Cleanliness of toilets/latrines
All rooms are clean 32 61.05%
Some rooms are clean 14 27.87%
No rooms are clean 5 11.08%
Place for hand washing 30 57.10%
Availability of water and soap at handwashing facility
Water and soap available 10 33.5%
Water available only 20 66.5%
Soap available only 0 0.0%
48
Characteristic n Percentage of total
No water and soap available 0 0.0%
Note: Total for the first item (N=68); total for items 2–5 (N=51); total for the sixth item (N=30) 
IR2.5 Increased Access to Preventative Health Services
The fifth IR under SO2 is increased access to preventative health services. To respond to this 
IR at baseline, pupils and head teachers were asked about their receipt of deworming and 
vitamin A capsules during the school year.
At baseline, enumerators asked pupils if they had received a dose of deworming medicine or a 
vitamin A capsule during the school year since September 2018. Results are presented in 
Figure 21. Out of a total of 680 pupils, 62.90 percent responded that they had received 
deworming medicine and 48.39 percent responded that they had received a vitamin A capsule 
during the school year.
To triangulate these findings, head teachers were also asked if pupils in the school had been 
given the same preventative health services during the school year and if yes, the frequency 
with which they were administered. Of the 68 head teachers interviewed, 69.86 percent reported 
that pupils had received deworming medicines during the year—an average of 1.36 times during 
the academic year. Just over half (54.47 percent) reported that pupils had received vitamin A 
capsules during the year—1.31 times, on average, during the academic year.
Figure 21: Percentage of Pupils Reporting Receiving Preventative Health Services by Sex
59.46%
48.92%
65.93%
47.92%
62.90%
48.39%
Received deworming medicine Received vitamin A capsule
Girls Boys Total
Note: Girls (n=332); Boys (n=348); Total (N=680). Remaining percentage includes pupils that responded "No" or 
"Don't know".
49
IR2.6. Increased Access to Requisite Food Preparation and Storage Tools and Equipment
The sixth IR under SO2 is increased access to requisite food preparation and storage tools and 
equipment. To respond to this IR at baseline, enumerators inventoried the food preparation and 
storage tools and equipment present at the school (see also Table 19 and Table 20). Results 
are presented in Table 23. 
Table 23: Presence of Kitchen Equipment and Tools
Tool Average Minimum Maximum
Big pots 1.82 1 3
Big bowls for cooked food 0.96 0 7
Big bowls for sauce 0.79 0 8
Wooden spoon 1.34 0 6
Scooping spoon 1.97 0 15
Serving plates 14.35 0 283
Spoons 14.28 0 283
Buckets 2.67 0 15
Towels 2.46 0 9
Cups 12.80 0 150
Knives 0.62 0 3
Mortar 0.39 0 2
Mortar pestle 0.48 0 3
Note: Total (N=68) 
PROGRAM RELEVANCE
The findings from the FGDs and KIIs inform the qualitative research questions and are 
summarized in the following sections, as well as the Conclusions and Recommendations 
section. It is important to note that these findings should not be considered representative of the 
entire population, but rather the communities sampled. 
To what extent does the program design respond to the needs identified by the baseline? 
Qualitative data from the community FGDs and the stakeholder KIIs indicates that, overall, the 
holistic approach and design of the APFL IV project is responsive to the most prominent needs 
of pupils in their education: 
• School feeding support, 
• Improving teacher quality, 
• WASH needs, including water wells and latrines, and 
• Creating or improving school building infrastructure.
Of these needs, school feeding support was cited the most frequently in FGDs and was echoed 
prominently in stakeholder KIIs. Other, less cited barriers to pupils’ education included sickness, 
lack of learning materials, lack of parental encouragement, negative peer influences, stigma and 
50
bullying from peers, fear of corporal punishment from teachers, being required to do farm work, 
and lack of interest in education. In the stakeholder KIIs, the lack of government-approved or 
government-supported schools was also cited as a barrier, along with parents’ low literacy rates.
Differences between girls and boys’ barriers are addressed more in the gender section of this 
report.
One area that emerged that is not currently supported through APFL IV is the issue of uniforms 
and clean clothes. Clothing was identified across multiple FGDs as a key barrier pupils face, as
they may not go to school or they may be sent home from school without a uniform or clean 
clothes. Participants also saw it as a broader concern, citing stigmatization and bullying around 
clothing will lead pupils to skip school or drop out. 
From the pupils’ literacy assessment, the low performance of students in the listening 
comprehension subtask indicates there may be an unmet need to reinforce English literacy 
within the classrooms. Teachers may be uncomfortable or are not trained to teach English and 
may rely on mother tongue language instruction in the classrooms. 
PROGRAM EFFECTIVENESS 
Participants in both the community FGDs and the stakeholder KIIs provided their perceptions of 
the strengths, weaknesses, and value of the APFL IV project. Recommendations on how to 
improve the effectiveness of the project—including ways to enhance and leverage gender 
considerations, mitigate challenges and unintended consequences, and leverage existing 
resource—are discussed in the Recommendations section of this report. 
How appropriate is the project design to the context? 
The APFL IV design was viewed by parents and community member FGD participants and KII 
respondents as appropriate for the context, because it closely responds to the challenges, 
barriers, and needs of the communities. In communities where support is ongoing, parents 
expressed high appreciation and gratitude for the work that CRS has been doing and a desire 
for it to continue. Furthermore, in communities where work will be beginning under APFL IV,
parents expressed a high level of interest and commitment to supporting the activities.
In addition, the design is seen as appropriate because it builds on and leverages existing 
channels and practices around parent and community support and engagement. For example, 
SILCs have already been established—sometimes independently and at other times with 
support from other NGOs—in some of the new APFL IV communities. Community group work 
has been leveraged in both old and new APFL communities to support the building of school 
infrastructure, cleaning up of school grounds, and other activities in a similar means that APFL 
IV employs. 
What challenges can already be foreseen with the design of the project in achieving its 
objectives? 
Both participants of the FGDs and the KIIs expressed concern around teachers. They felt that 
the quality of teaching may be impacted by a lack of trained teachers; teacher absenteeism due 
to inconsistent, late, or missing salaries; and an insufficient number of teachers at some 
51
schools. Quantitative data from the school-based surveys supports these concerns. Less than 
half of teachers (46.15 percent) report having any type of teaching certificate. Furthermore, the 
self-reported knowledge of teaching techniques is very low with less than one-quarter of 
teachers reporting any level of knowledge and confidence across all techniques. 
More specifically, within the KIIs there were concerns over the need for community members—
parents and sometimes pupils—to support teachers through additional fees or assisting with 
farm work. These methods of teacher support were noted as potentially negatively impacting 
access to and quality of education. Concerns over indebtedness to teachers may reduce 
parents’ engagement in supporting schools or even sending their children to school. 
Another area of concern was with regards to the school feeding program. Participants felt that 
pupils’ school attendance might be impacted if school feeding is inconsistent or not present due 
to pupils’ “lack of interest” in school. Additionally, there was minor concern that poor monitoring 
or handling of the distribution of supplies may result in stealing or siphoning off of food, 
materials, or funds. 
How could integration of the two strategic objectives and the foundational results be 
enhanced or further leveraged to deeper positive program impact? 
No clear findings emerged from the analysis at the time of the baseline evaluation. 
How could integration of the gender considerations be enhanced or further leveraged to 
deepen positive project impact? 
Some of the gender-specific barriers to girls’ education identified in FGDs and KIIs include
pregnancy, early marriage, and household chore burden. 
• Pregnancy was the most commonly cited barrier specific to girls in FGDs and KIIS, 
especially in relation to their dropping out of school when they become pregnant. 
Broader discussions of girls’ engagement in sexual activity and in some cases, 
transactional sex, were noted in FGDs as having a negative impact on girls’ education; 
this was at times accompanied by FGD participants’ perceptions that girls are more 
interested or influenced by money than by their education. There was also an 
undercurrent within FGDs that educating a daughter was not done because she would 
just get pregnant anyway and education would be wasted. According to one female 
participant, 
“[Men] said when you put the girls to school, they will end up getting pregnant, 
[men] also said no matter how you educate a girl child they will end up getting 
married. That is why they do not educate girls. They will leave everything in the 
hands of the mother, which is the reason we go the extra mile in other to support 
our girl child.”
• Similar to pregnancy, early marriage was noted across several KIIs as a barrier for girls. 
Respondents shared that, while overall rates of early marriage are declining, there 
remain people who believe the purpose of a girl is to marry and that school is a waste of 
time. 
52
• It was also noted several times in FGDs and KIIs that girls have a higher household 
chore burden. For example, a girl may be held back from school so she can cook for 
the family while her mother works at the farm. 
However, in some FGDs—particularly within the all-male groups—respondents expressed that 
they see no difference in the barriers for boys and girls and that the challenges are the same. 
This may be supported by the similar drop-out rates recorded within sampled schools—4.87 
percent for boys and 4.21 percent for girls. However, it does not explain the differences in 
pupils’ performance on the literacy assessment wherein boys outperformed girls in most 
subtasks. 
When examining how APFL IV programming can be enhanced, it is important to look at the 
differences in gender roles for the parents as well as the pupils. In some all-female FGDs, 
women expressed their perceptions that the burden and responsibility to support their 
daughters’ education—and boys to some extent as well—falls solely on the mothers. Mothers
buy school materials, send their children to school, and support their education. According to 
one mother, 
“I have given birth to my child, and I said she has to go to school, and her father said ‘I 
will not be responsible for her school charges. You will be the one paying the school 
charges if you want her to go to school.’ That is a clear indicator that a girl child should 
not be educated, but if you still persist in sending her to school you will be responsible 
for her. The mother is the only one buying all the school materials for her daughter. If 
you don’t have the money to do all that, your daughter will not go to school. The men are 
not supporting their girl child. If you don’t believe, ask any of these girls standing there 
and they will tell you, ‘my mother is the one gives me everything.’ These are the reasons 
why the girls end up being a drop out, because they have left all the responsibility of the 
girl child in the hands of their mothers. The men said they will not educate a girl child; a 
mother will not be able to educate her girl child.” 
Similar sentiments that children’s education is considered the responsibility of women and 
mothers were expressed in KIIs as well. In one KII, the respondent noted that the burden falls 
disproportionately on women. Moreover, CRS’s approach towards engaging parents often 
mirrors traditional gender roles and expectations. MSGs, SILCs, food preparers training, and 
other activities are perceived to involve women and mothers as caregivers and, therefore, the 
main person responsible for children’s education; fathers and men are typically engaged to 
support with physical or manual labor. From the view of one of the KII respondents, this
approach to engaging parental involvement may even reinforce traditional stereotypes about 
women’s and men’s roles and responsibilities within society and put an undue burden of 
volunteering and support on mothers and female community as well. According to one KII 
respondent: 
“In order to address gender issues, I think that the mothers should be properly 
remunerated. We need to properly pay them rather than just give them dry rations—
[CRS] give them family rations to take home—because if these moms are being taken 
away from supporting their husbands and the farm, supporting them with the household 
53
chores and they are volunteering and cooking meals at the school, it needs to be 
remunerated properly... rather than just giving them dry rations. I know they give them 
five portions of dry rations equivalent to a meal for a single-family unit, but then there are 
responsibilities that are not just limited to food. They need to be remunerated so they 
can partake in things like looking after themselves, providing… having an income that 
would address other issues that they need.”
This concern was not consistent across all FGDs, however, as some men and fathers 
highlighted the various ways they support their children’s education—including that of their girls. 
Given the variations, another interpretation may be that a degree of variation in the level of 
support and engagement for daughters’ education exists between mothers and fathers—and is 
likely to vary both within and across communities. 
To what extent will the project respond to the needs of stakeholders and facilitate their 
participation? 
In communities where APFL has already been intervening, the qualitative data indicates that the 
project is highly responsive to the needs of parents and community members and that it has 
engaged their participation in numerous ways. However, several barriers to parent and 
community member engagement and support of the APFL project have been identified: 
• Hardships related to poverty. In areas where parents help support community 
teachers by paying school fees, poverty can make this financial arrangement
unsustainable. In turn, children may be turned away from school, not sent to attend, or 
feel shame related to indebtedness to the teacher. In addition, when parents are not able 
to contribute financially or provide relevant learning materials, uniforms, or clean clothes, 
they may feel ashamed, withdraw their child from school, or not participate in school 
activities or meetings for fear of being asked to provide financial support.
• Farm work. Parents often need to focus on their farms as their first priority. 
• Lack of value placed on education by parents. While all participants in the FGDs 
displayed a strong level of interest and commitment to their children’s education, they 
noted that this sentiment and belief is not necessarily universal. They felt that there are 
parents who do not value education for their children and parents who may place a 
higher value on children supporting farm work than attending school.
• Lack of knowledge of project activities. Parents and community members noted a 
lack of knowledge about the program as a potential barrier to participation. 
Other less frequently cited barriers include concern over bullying, lack of accountability for non￾contributors to group work within the community, lack of monitoring of materials or funds 
contributing to stealing, parents’ low literacy levels, lack of incentives for parental engagement,
and a sense of pride wherein wealthier parents may prefer to send money than contribute 
physically. 
Which interventions are likely to be the most critical and effective in achieving the 
project’s intermediate results and objectives?
Parents and community members discussed all aspects of the project’s intervention based on 
the outlined activities provided by CRS. In this discussion, parents and community members—
54
as well as KII respondents—most frequently cited school feeding as the project’s most critical 
intervention. School feeding was perceived as increasing pupils’ attendance and enabling them 
to be able to focus and pay attention in the classroom and, therefore, supporting their learning. 
Another intervention that was frequently cited across FGDs and KIIs as critical was teacher 
training, coaching and mentoring of teachers, and distance learning. Parents noted that if a 
teacher is trained properly, they will be able to teach the children better. WASH-related 
interventions were also mentioned across multiple FGDS and several KIIs as critical. 
Specifically, these interventions included the construction of latrines, water wells, and pumps as 
need is high for improving these hygiene practices.
Other interventions that were noted less frequently included supporting of SILCs due to 
prevalence of poverty, training on food storage and handling to ensure hygienic practice as well 
as reduce waste and contamination, school gardens to support and supplement the school 
feeding, provision of learning materials to help children learn, and the construction of school 
building infrastructure given high level of need.
In almost half of the FGDs, participants mentioned that all the activities were needed or critical.
Are there any foreseeable negative impacts or unintended consequences of the project 
that need to be addressed or mitigated against since the beginning? 
When asked about potential negative impacts or unintended consequence, parents and 
community members identified several potential issues and concerns: 
• Stealing of food or other supplies by food preparers, teachers, and others; 
• Poor food preparation leading to sickness; 
• Pupils skipping afternoon lessons after receiving their meal; 
• Pupils eating too much and being sleepy and unable to pay attention to the lessons in 
the afternoon;
• Children from nearby communities are abandoning their schools in favor of enrolling in 
APFL-supported schools, which could also contribute to potential safety issues for 
children on the way to and from the schools due to the distance; and
• Donor dependency or fatigue especially when considering the sustainability of the 
schools after the APFL program ends. 
SUSTAINABILITY AND IMPACT
Much of the discussion around sustainability came from discussions with stakeholders 
participating in the KIIs. The findings from these discussions are summarized here. More 
detailed recommendations around sustainability are presented in the Recommendations 
section, which also answers to the following evaluation questions: 
• How can an exit strategy be better integrated from project startup? 
• How can CRS effectively improve the design and implementation to achieve a more 
sustainable impact?
55
What are the key considerations to ensure sustainability and impact? 
The evaluation identified the following areas of focus for sustainability of the APFL IV project: 
• Coordination across partners was noted as critical to sustainability in multiple KIIs. At the 
national level, it was noted that given the holistic nature of the program, it is essential 
that the project coordinates with key partners and that parents are involved in and attend 
important meetings. When asked which key partners must be included, one respondent 
indicated partners in education, health, social welfare, agriculture and community 
structures, and counsels. 
• Respondents indicated that strengthening and building the capacity of the MBSSE and 
other relevant government partners to prepare them for transitioning into the 
implementing role for the school feeding program at CRS-supported schools would be 
important.
• Placing a greater emphasis on school gardens and farms—as well as a reduced 
emphasis on food imports—was also seen as important. One respondent highlighted 
that by buying local food from the community rather than importing food, the program 
would create employment and support local farmers by creating a demand for their 
products. While this may not be possible due to programmatic limitations, such a barrier 
could indicate that more stakeholder education is needed regarding the structure and 
scope of the project.
• Respondents also named continuing to leverage and strengthen existing structures to 
take on increasing responsibility for program activities. At the community level, this 
would include MSGs, SILCs, SMCs, and community workgroups. 
What could be barriers to achieving sustainability and impact?
The transition of CRS’s school feeding support to the GoSL may be challenging due to overall 
implementation costs, limited experience and capacity, and the complicated logistics of getting 
food to schools due to the poor road conditions and limited accessibility. 
Poverty levels will likely limit parents’ and community members’ ability to meet any financial 
demands associated with supporting or supplementing teachers not on MBSSE’s payroll as well 
as the other hidden costs of education beyond school fees, such as learning materials, 
uniforms, opportunity costs related to household chores or farm work, and others. 
If teachers are not on the government payroll or receiving payment in a timely fashion, this may 
contribute to teacher absenteeism. It may also limit teachers’ commitment and engagement in 
their teaching roles, which would have potentially negative implications on the quality of 
teaching and pupils’ learning gains. 
If community schools are not approved to become a part of the MBSSE system, this may impact 
their ability to get the necessary support from the government. The lack of government support 
would increase the burden placed on the community. This is especially an area of concern when 
CRS is no longer available to supplement finances and advocate for governmental support.
56
CONCLUSIONS AND
RECOMMENDATIONS
CONCLUSIONS
Overall, the findings from the baseline evaluation for the APFL IV project show that the design 
of the project is relevant and properly addresses the needs of the communities CRS serves. 
The findings provide both a challenge and encouragement for CRS. Communities are very 
supportive of the APFL programming and want it to continue. The NSFS, as an entity, is ready 
to learn and prepare itself to take on the school feeding activities. 
However, with such a large expansion of the APFL project in Phase IV, the project’s needs are 
great. Based on the baseline literacy assessment, APFL IV pupils’ literacy outcomes are low—
only 5.58 percent of class 2 pupils achieved the reading comprehension benchmark. Pupils 
performed best on the alphabet naming subtask, but more than half of all pupils struggled with 
the other subtasks. In general, boys outperformed girls across the subtasks, which indicates a 
gender-based performance gap. Importantly, low performance on the listening comprehension 
sub-task indicates that students cannot comprehend a story told to them at a level that was 
deemed appropriate which would indicate very low English abilities that are foundational for 
reading acquisition. 
Teachers will likely play a critical role in the project’s success. While teacher attendance is 
relatively high, there is a clear need for rigorous training and coaching. When asked about 
effective teaching techniques, less than one-quarter of teachers felt any confidence in their 
ability to use the techniques in their own classrooms. Fewer than half of teachers are certified, 
so CRS’s teacher training and certification efforts will help fill this gap in the teaching workforce 
in Koinadugu and Falaba districts. One encouraging finding concerning teacher coaching was 
that nearly all teachers (94.85 percent) had been observed by their head teacher in the 
academic year; 87.15 percent had been observed more than twice in the year. With one of the 
top three reported motivation factors for teachers being their relationship with the head teacher, 
supporting positive relationships between teachers and their head teachers can be an impactful 
activity for CRS.
Supporting the feeding program will be critical for the APFL IV project. Only 6.12 percent of 
head teachers reported being able to provide a school meal on the day of the baseline data 
collection visit. This indicates that without CRS support, provision of meals at schools will be 
limited. Additionally, with less than half of pupils reporting that they are reaching the minimum 
acceptable diet, keeping the school feeding programs well stocked will be essential for pupils’ 
nutrition. 
From the community FGDs and stakeholder KIIs, there is clear support for the APFL IV project 
and a desire to see it succeed. Participants shared potential challenges and unintended 
57
consequences of the project, as well as recommendations on how to further strengthen the 
design and implementation of the project. Many of these recommendations are outlined here. 
RECOMMENDATIONS
Program Relevance
TAILOR APFL IV INTERVENTIONS TO SPECIFIC COMMUNITY AND SCHOOL NEEDS. 
Based on slight variations in specific needs within different communities, it will be appropriate 
for the project to include minor adaptions to its approach to different communities based on their 
previous level of support from APFL as well as their current needs and priorities. More targeted 
interventions would likely see the greater impact and encourage stronger community buy-in and 
ownership, thus ensuring greater sustainability. 
There may also be unique needs in specific communities that warrant additional, specific 
attention—such as one community wherein both the male and female FGDs cited the 
destructive impact of cattle as a major issue in their area. Proposed engagement to support 
fencing around the school could be an appropriate deviation from project activities. 
Additionally, in communities with other organizations are already in place, it may make sense to 
not focus on certain activities—such as SILCs—to prevent duplication. For example, according 
to the qualitative data, SILC-like activities are being supported by Oxfam in Fudu Kalia and
Medicoast in Wara Wara Bafodia in Koinadugu. CRS should create and leverage partnerships 
within these communities to ensure their education work is not duplicative. This could also be 
the case with the distribution of Vitamin A supplementation, which was reported to be taking 
place in some intervention schools. Additionally, as CRS launches more SILC groups, they may 
need to be sensitive to community perceptions that poverty is a barrier to the development of 
SILCs. 
EXAMINE EXISTING STUDENT AND TEACHER ENGLISH LANGUAGE ABILITIES.
Student performance on the listening comprehension sub-task may indicate that students have 
a limited ability to understand spoken English. The project may want to consider undertaking 
more targeted research into the reasons for this gap in comprehension. Specifically, this may 
mean a deeper investment in coaching for basic skills for literacy instruction for early grade 
teachers, who’s English language proficiency was not addressed in this baseline data collection. 
Improving the English abilities of teachers may be a necessary step to ensuring they can 
confidently teach students to read in English. 
Program Effectiveness 
CLOSELY MONITOR PROJECT IMPLEMENTATION.
At every stage of the APFL IV project, it will be important to monitor the implementation of 
activities in order to ensure fidelity to the project design. This would include documenting any 
differences in implementation across communities and schools, as CRS strives to meet the 
specific needs of each school. This may help alleviate concerns shared in the FGDs and KIIs 
58
around the mismanagement of supplies, and it will allow CRS to more nimbly respond to 
challenges that arise, such as the insufficient food supplies. Close monitoring of the fidelity of 
implementation will also allow the project to link specific interventions with outcomes during the 
midline and endline evaluations of the APFL IV project. 
ADDRESS GENDER CONSIDERATIONS WITHIN THE PROGRAMMING. 
Several recommendations around gender considerations came out of the FGD and KII 
discussions: 
• Informing their daughters about family planning practices to help prevent unwanted 
pregnancies; 
• Leveraging life skills activities to link pupils to youth-friendly, sexual and reproductive 
health services for girls and boys; 
• Community-wide sensitization on the importance of education, especially for girls, 
including promoting a sharing chore burden across siblings regardless of gender 
• Engaging fathers and men more broadly in pupils’ education, possibly in nontraditional 
roles; and
• Examining potential alternatives and expansion of remuneration practices for food 
prepares to ensure that the work is not exploitative nor taking advantage of women’s 
volunteer role. 
CRS may consider completing a more thorough gender and social inclusion analysis to ensure 
that the project design, planning, implementation, monitoring, and evaluation consider existing 
gender disparities and that the implementations designed to remove or reduce them are the 
right ones for the project context. For example, a gender analysis could further examine 
challenges and solutions related to: 
• Girls’ extra household chore burden; 
• Girls who continue not to enroll, have high levels of absenteeism, or drop-out despite the 
school feeding and other interventions; 
• Lack of female teachers or role models within the school; 
• Teenage pregnancy and early marriage in primary education drop-out; and 
• Transactional sex in relation to education. 
Sustainability and Impact 
BUILD THE CAPACITY OF THE NATIONAL SCHOOL FEEDING SECRETARIAT. 
Considering the APFL IV goal of transitioning the school feeding program to the GoSL, it was 
proposed within the KIIs that CRS could play a role in building the capacity of the nascent 
NSFS. 
Recommendations for capacity building activities include: 
• Increased collaboration with NSFS to prepare them to take over through CRS-led
trainings and frequent communication; 
59
• Supporting study visits to other countries in West Africa to observe and learn from the 
challenges and success of other government-run school feeding programs; 
• Building the capacity of the monitoring-and-evaluation teams to provide oversight to 
school feeding implementation, as well as building capacity related to the administrative 
and logistical wings on the actual implementation of school feeding programs; 
• Identifying any potential role for CRS to play in support NSFS, while recognizing that
there may be a perception within the Secretariat that CRS is less engaged in the GoSL’s 
preferred homegrown school feeding programs; and 
• Building NSFS fundraising capacity in order to provide access to supplemental funding 
opportunities and support, including developing an understanding of the donor 
landscape and the networking within that space.
INTEGRATE TRANSITION STRATEGY FROM START-UP FOR SUSTAINABLE IMPACT. 
Clear communication and expectation setting around the transition strategy must be central to
APFL IV’s early discussions with communities, partners, and the GoSL. The transition strategy 
should be developed in partnership with relevant stakeholders. Such a strategy should clearly 
articulate transitions and phase-out approaches for each activity, identify responsible parties to 
continue implementation after APFL IV ends, delineate direct and indirect costs, establish 
monitoring and oversight guidelines, and set clear timelines. A successful strategy should also 
outline contingency plans, as needed.
60
REFERENCES
Catholic Relief Services, “Final Evaluation of Phase III of The McGovern-Dole Food For 
Education Project (MGD III) In Sierra Leone,” August 8, 2018.
“Data Sources and Inclusion Criteria.” World Health Organization. World Health Organization, 
December 23, 2013. 
https://www.who.int/nutrition/databases/infantfeeding/data_source_inclusion_criteria/en/.
McConnell, Brendon, and Marcos Vera-Hernandez. “Going beyond Simple Sample Size 
Calculations: a Practitioner's Guide.” Institute for Fiscal Studies, September 28, 2015. 
https://www.ifs.org.uk/publications/7844
“McGovern-Dole Food for Education Program.” US Department of Agriculture Foreign 
Agricultural Service. Accessed August 6, 2019. 
https://www.fas.usda.gov/programs/mcgovern-dole-food-education-program. 
Parliament of Sierra Leone, “The Education Act, 2004,” signed March 29, 2004. 
http://www.sierra-leone.org/Laws/2004-2p.pdf
“President Bio Launches Free Education, Calls on Parents and Teachers to Support the 
Initiative.” The Republic of Sierra Leone State House, August 20, 2018. 
https://statehouse.gov.sl/president-bio-launches-free-education-calls-on-parents-and￾teachers-to-support-the-initiative/.
“Sierra Leone.” UNESCO UIS, April 12, 2017. http://uis.unesco.org/country/SL#slideoutmenu. 
Sierra Leone Ministry of Basic and Senior Secondary Education, “2018–20 Education Sector 
Plan,” 2017.
Statistics Sierra Leone and ICF International. “Sierra Leone Demographic and Health Survey 
2013,” Freetown, Sierra Leone: SSL and ICF International. 2014. 
61
ANNEXES 
ANNEX A. SAMPLING APPROACH
For the baseline evaluation, a two-stage cluster sampling approach was used to select pupils, 
teachers, and food preparers. First, schools were randomly selected as clusters; then pupils, 
cooks, and teachers within schools were selected at the second stage. The sample size for the 
sampling units was computed using the indicator “Percentage of pupils who, by the end of two 
grades of primary schooling, demonstrate that they can read and understand the meaning of the 
grade-level text.” The outcome is binary—taking the value 1 if a pupil can read grade-level text 
and 0 otherwise. The pupil’s sex was further used as a covariate to gain additional explanatory 
power. 
The evaluation used the equation in McConnell and Vera-Hernandez23 to calculate sample 
sizes for a binary outcome with a binary covariate.
𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 𝑠𝑠𝑠𝑠𝑠𝑠 = 2 ∗ 𝑘𝑘 = (𝒈𝒈𝑴𝑴−1𝒈𝒈′)
� 𝛽𝛽 + 𝛼𝛼/2�
2
( 1 − 0)2 (1 + ( − 1)𝜌𝜌𝑥𝑥)
where = 10, the number of pupils sampled per school; 𝑘𝑘 = the number of schools sampled; g
and M are defined in equation of McConnell and Vera-Hernandez.24 Based on APFL Phase III 
endline results and assumptions about baseline values for the new schools entering in APFL IV, 
analysts assume a pass rate for boys at baseline of 39.4 percent and 41.5 percent for girls, with 
53 percent of the sample being boys. The project targeted an endline pass rate of 58 percent for 
both boys and girls. 
𝛽𝛽 = One-tailed value of the normal distribution at (1–0.80), which is the sample’s power. 
Analysts have a 20.00 percent probability of not finding an effect from intervention, despite there 
being a genuine effect—Type II error. 𝛼𝛼= Two-tailed value of the normal distribution at 0.05, 
which is our significance level. Analysts have a 5.00 percent chance of rejecting the null 
hypothesis when it should not—Type I error. In this case, the null hypothesis is there is no 
difference between reading scores from baseline to endline. 𝜌𝜌𝑥𝑥 = 0.52, or the conditional 
intracluster correlation (ICC) from the APFL Phase III endline.25
23 McConnell, Brendon, and Marcos Vera-Hernandez. “Going beyond Simple Sample Size Calculations: a Practitioner's Guide.” IFS. 
Institute for Fiscal Studies, September 28, 2015. https://www.ifs.org.uk/publications/7844. 
24 McConnell, Brendon, and Marcos Vera-Hernandez. “Going beyond Simple Sample Size Calculations: a Practitioner's Guide.” IFS. 
Institute for Fiscal Studies, September 28, 2015. https://www.ifs.org.uk/publications/7844. 
25 An ICC value of 1.00 would mean that all the variation in the data could be explained by differences between the schools, thus we 
would need to visit many schools, but only one pupil per school. An ICC of 0.00 would mean that all the data variation could be 
explained by differences between the pupils, thus researchers could visit fewer schools, but more pupils in each. See McConnell 
and Vera-Hernandez on calculating the ICC and conditional ICC. 
62
ANNEX B. SUPPLEMENTARY FINDINGS 
Table B.1: Literacy Assessment Mean Scores for Pupils at New or Continuing APFL Schools
Subtask Total Possible 
Items Correct
Continuing Schools New Schools
n Mean n Mean
Letter Name 51 458 36.80 224 33.98
Phonemic Awareness** 10 458 1.40 224 0.70
Familiar Word 40 458 11.56 224 10.77
Invented Word** 25 458 18.05 224 18.77
Reading Passage** 41 458 8.69 224 4.75
Reading Comprehension** 5 458 0.74 224 0.30
Listening Comprehension 4 458 1.09 224 1.04
Note: Two asterisks (**) denotes that the scores of pupils at continuing schools are statistically significantly higher than those at new schools at the p<0.01 level.
Table B.2: Distribution of Reading Comprehension Scores for Pupils at New or Continuing APFL Schools 
Zero Correct One Correct Two Correct Three Correct Four Correct Five Correct Total Meeting 
Benchmark of 
4+ Correct
n % n % n % n % n % n % n %
Pupils from 
new APFL IV 
schools
222 83.91% 10 3.68% 13 5.08% 11 4.00% 6 2.30% 3 1.02% 264 3.32%
Pupils from 
continuing 
APFL schools
311 74.53% 20 4.84% 21 4.95% 36 8.66% 13 3.12% 16 3.90% 418 7.02%
Total Pupils 533 78.17% 30 4.39% 34 5.00% 47 6.85% 19 2.80% 19 2.78% 682 5.58%
63
Table B.3: Head Teacher Supplementary Findings
Item
Yes No
n % n % 
Does the school have a time book for 
recording daily teacher attendance? 62 92.06% 5 7.94%
Is there a logbook (visitor book) in the 
head teacher’s office? 48 72.03% 19 27.97%
Is there a teaching master-timetable 
displayed in the head teacher’s office? 35 51.67% 32 48.33%
Is there a teacher duty roster clearly 
displayed in the head teacher’s office? 34 50.43% 33 49.57%
Are there visual teaching and learning 
materials displayed in the head 
teacher’s office?
40 60.21% 27 39.79%
Are inventory book or other school 
records properly organized and
updated in the head teacher’s office?
42 63.15% 25 36.85%
Has the head teacher benefited from 
training in diagnostic teaching 
methods?
37 55.39% 30 44.61%
Table B.4: Teacher and Classroom Observation Supplementary Findings
Item Response Options n % 
Class level observed
Class 2 50 34.32%
Class 3 46 32.05%
Class 4 49 33.63%
Do you have a lesson plan or note for 
the class you just conducted?
Yes 110 75.97%
No 35 24.03%
Do you have access to a teacher's 
guide?
Yes, in head 
teacher's office 94 65.26%
Yes, in classroom 30 19.75%
No 21 14.99%
Pupils follow instructions
Little evidence 44 24.62%
Moderate evidence 78 43.57%
Extensive evidence 57 31.81%
Table B.5: Pupil Attention Supplementary Findings
Item n Mean
Number of boys in class 179 13.13
Number of girls in class 179 11.64
64
Item n Mean
Number of boys attentive to instructions 179 8.26
Number of girls attentive to instructions 179 7.47
Table B.6: Head Teacher School Feeding Response Supplementary Findings
Item Response Option n % 
Have or will pupils in this school 
receive/be served meal/food 
provided by CRS today?
Yes 5 6.12%
No 47 67.08%
Do not know/Not 
applicable 16 26.79%
When was meal/food served to 
pupils today?
Morning (before 
11:30 am) 5 100.00%
At lunch time (exactly 
at 11:30–12:30 pm) 1 20.00%
Afternoon (after 
12:30 pm) 0 0.00%
Why have pupils in this school 
not been served meal/food 
today?
No feeding program
yet established at 
school
21 36.43%
Food supplies run out 40 60.46%
No condiments for 
cooking 3 4.77%
No fuel (firewood) to 
cook food 0 0.00%
No food preparer
available 1 1.89%
No cooking utensils 
(pot) 0 0.00%
No access to store 0 0.00%
Other 3 4.68%
Are the community people 
supporting this school with 
stipulated level of food 
contribution for cooking 
materials (such as condiments, 
firewood, vegetables, etc.) to the 
school feeding program? 
Yes 46 64.36%
No 6 8.85%
Do not know/Not 
applicable 16 26.79%
Are teachers of this school 
currently receiving meals 
(school feeding program)? 
Yes 23 32.18%
No 29 41.03%
Do not know/Not 
applicable 16 26.79%
65
 
Table B.7: School Observation Enrollment, Attendance, and Dropouts
Item N Girls
(sum)
Boys
(sum)
Total
(sum)
Class 1 enrolled 68 2044 2276 4321
Class 1 attending 68 1316 1481 2797
Class 1 attendance rate (%) 68 64.37% 65.07% 64.74%
Class 1 dropouts 68 92 123 215
Class 2 enrolled 68 1219 1287 2507
Class 2 attending 68 801 873 1674
Class 2 attendance rate (%) 68 65.66% 67.85% 66.79%
Class 2 dropouts 68 41 61 102
Class 3 enrolled 68 919 1118 2037
Class 3 attending 68 612 780 1391
Class 3 attendance rate (%) 68 66.52% 69.76% 68.30%
Class 3 dropouts 68 40 52 92
Class 4 enrolled 68 693 841 1534
Class 4 attending 68 505 606 1112
Class 4 attendance rate (%) 68 72.95% 72.03% 72.45%
Class 4 dropouts 68 28 34 61
Class 5 enrolled 68 465 536 1001
Class 5 attending 68 324 393 717
Class 5 attendance rate (%) 68 69.78% 73.32% 71.67%
Class 5 dropouts 68 26 30 56
Class 6 enrolled 68 259 344 603
Class 6 attending 68 161 256 418
Class 6 attendance rate (%) 68 62.25% 74.49% 69.23%
Class 6 dropouts 68 10 12 21
Total enrolled 68 5600 6403 12003
Total attending 68 3719 4390 8109
Total attendance rate (%) 68 66.42% 68.56% 67.56%
Total dropouts 68 236 312 548
Table B.8: Classroom Observation Headcounts
Item N Girls
(sum)
Boys
(sum)
Total
(sum)
Class 2 attending 50 857 964 1821
Class 3 attending 46 638 819 1457
Class 4 attending 49 585 565 1150
66
Table B.9: School Garden Findings
Item Response Options n % 
Does the school have a school 
garden?
Yes 22 30.64%
No 46 69.36%
What types of vegetables are grown?
Green beans 5 24.25%
Potatoes 12 51.94%
Peppers 10 45.76%
Tomatoes 4 17.68%
Cassava leaves 16 72.07%
Pumpkin 1 5.75%
Lettuce 2 7.95%
Spinach 0 0.00%
Other 4 19.17%
Table B.10: Head Teacher Preventative Health Services Supplementary Findings
Item Response Options n % 
Have pupils in this school received de￾worming medicine/worm medicine 
during this school year (since 
September 2018)?
Yes 48 69.86%
No 20 30.14%
Number of times 
(average) 68 11.36
Have pupils in this school received 
vitamin A supplementation during this 
school year (since September 2018)?
Yes 38 54.47%
No 30 45.53%
Number of times 
(average) 68 22.34
67
ANNEX C. UPDATED INDICATOR PERFORMANCE TRACKING TABLE (IPTT) 
Table C.1: Updated Indicator Performance Tracking Table
Performance Indicator
Standard 
MGD 
Indicator 
Number
Baseline 
Target
Baseline 
Actual
Outcomes (Data available only for Baseline, Midterm, and Final Evaluation years)
Percent of students who, by the end of two grades of primary 
schooling, demonstrate that they can read and understand the 
meaning of grade level text MGD # 1 25% 5.58%
Percent of students who, by the end of two grades of primary 
schooling, demonstrate that they can read and understand the 
meaning of grade level text (Male) MGD # 1 27% 7.77%
Percent of students who, by the end of two grades of primary 
schooling, demonstrate that they can read and understand the 
meaning of grade level text (Female) MGD # 1 22% 3.11%
Percent of participants of community-level nutrition 
interventions who practice promoted infant and young child 
feeding behaviors
MGD # 21 TBD 98.27%
Outcomes (Data available only at baseline and annually)
Average student attendance rate in USDA supported 
classrooms/schools MGD # 2 89% 61.8%
Average student attendance rate in USDA supported 
classrooms/schools (Male) MGD # 2 90% 70.9%
Average student attendance rate in USDA supported 
classrooms/schools (Female) MGD # 2 89% 52.7%
Number of teachers/educators/teaching assistants in target 
schools who demonstrate use of new and quality teaching 
techniques or tools as a result of USDA assistance MGD # 4 0
Number of teachers/educators/teaching assistants in target 
schools who demonstrate use of new and quality teaching 
techniques or tools as a result of USDA assistance (Male) MGD # 4 0
Number of teachers/educators/teaching assistants in target 
schools who demonstrate use of new and quality teaching 
techniques or tools as a result of USDA assistance (Female) MGD # 4 0
Number of school administrators and officials in target schools 
who demonstrate use of new techniques or tools as a result of 
USDA assistance MGD # 6 0
Number of school administrators and officials in target schools 
who demonstrate use of new techniques or tools as a result of 
USDA assistance (Male) MGD # 6 0
68
Performance Indicator
Standard 
MGD 
Indicator 
Number
Baseline 
Target
Baseline 
Actual
Number of school administrators and officials in target schools 
who demonstrate use of new techniques or tools as a result of 
USDA assistance (Female) MGD # 6 0
Number of students enrolled in schools receiving USDA 
assistance (Primary School) MGD # 9 31,546 44,511
Number of students enrolled in schools receiving USDA 
assistance (Male) MGD # 9 16,266 23,356
Number of students enrolled in schools receiving USDA 
assistance (Female) MGD # 9 15,281 21,155
Number of individuals who demonstrate use of new child health 
and nutrition practices as a result of USDA assistance MGD # 19 0 
Number of individuals who demonstrate use of new child health 
and nutrition practices as a result of USDA assistance (Male) MGD # 19 0 
Number of individuals who demonstrate use of new child health 
and nutrition practices as a result of USDA assistance (Female) MGD # 19 0 
Number of Individuals who demonstrate use of new safe food 
preparation and storage practices as a result of USDA 
assistance
MGD # 20 0 
Number of Individuals who demonstrate use of new safe food 
preparation and storage practices as a result of USDA 
assistance (Male)
MGD # 20 0 
Number of Individuals who demonstrate use of new safe food 
preparation and storage practices as a result of USDA
assistance (Female)
MGD # 20 0 
Percent of participants of community-level nutrition 
interventions who practice promoted infant and young child 
feeding behaviors
MGD # 21 TBD 
Output (Data available only at Baseline and Annually)
Number of policies, regulations, or administrative procedures in 
each of the following stages of development as a result of 
USDA assistance
MGD # 10 0 
Number of policies, regulations, or administrative procedures in 
each of the following stages of development as a result of 
USDA assistance Education
MGD # 10 0 
69
Performance Indicator
Standard 
MGD 
Indicator 
Number
Baseline 
Target
Baseline 
Actual
Number of policies, regulations, or administrative procedures in 
each of the following stages of development as a result of 
USDA assistance Child Health and Nutrition
MGD # 10 0 
Value of new USG commitments, and new public and private 
sector investments leveraged by USDA to support food security 
and nutrition
MGD # 11 0 
Value of new USG commitments, and new public and private 
sector investments leveraged by USDA to support food security 
and nutrition (Host Government amount)
MGD # 11 0 
Value of new USG commitments, and new public and private 
sector investments leveraged by USDA to support food security 
and nutrition (Other Public amount)
MGD # 11 0 
Value of new USG commitments, and new public and private 
sector investments leveraged by USDA to support food security 
and nutrition (Private amount)
MGD # 11 0 
Number of social assistance beneficiaries participating in 
productive safety nets as a result of USDA assistance MGD # 18 0 
Number of social assistance beneficiaries participating in 
productive safety nets as a result of USDA assistance (Male) MGD # 18 0 
Number of social assistance beneficiaries participating in 
productive safety nets as a result of USDA assistance (Female) MGD # 18 0 
Number of social assistance beneficiaries participating in 
productive safety nets as a result of USDA assistance (New) MGD # 18 0 
Number of social assistance beneficiaries participating in 
productive safety nets as a result of USDA assistance 
(Continuing)
MGD # 18 0 
Number of children under five (0-59 months) reached with 
nutrition-specific interventions through USG-supported 
programs 
MGD # 24 0 
Number of children under five (0-59 months) reached with 
nutrition-specific interventions through USG-supported 
programs (Male)
MGD # 24 0 
Number of children under five (0-59 months) reached with 
nutrition-specific interventions through USG-supported 
programs (Female)
MGD # 24 0 
70
Performance Indicator
Standard 
MGD 
Indicator 
Number
Baseline 
Target
Baseline 
Actual
Number of children under two (0-23 months) reached with 
community-level nutrition interventions through USG-supported 
programs
MGD # 25 0 
Number of children under two (0-23 months) reached with 
community-level nutrition interventions through USG-supported 
programs (Male)
MGD # 25 0 
Number of children under two (0-23 months) reached with 
community-level nutrition interventions through USG-supported 
programs (Female)
MGD # 25 0 
Number of pregnant women reached with nutrition-specific 
interventions through USG-supported programs. MGD # 26 0 
Number of pregnant women reached with nutrition-specific 
interventions through USG-supported programs. ( <19 years of 
age)
MGD # 26 0 
Number of pregnant women reached with nutrition-specific 
interventions through USG-supported programs. ( ≥19 years of 
age)
MGD # 26 0 
Number of individuals participating in USDA food security 
programs MGD # 30 0 
Number of individuals participating in USDA food security 
programs (Male) MGD # 30 0 
Number of individuals participating in USDA food security 
programs (Female) MGD # 30 0 
Number of individuals benefiting indirectly from USDA-funded 
interventions MGD # 31 0 
Output (Data available only at baseline and Biannually)
Number of teaching and learning materials provided as a result 
of USDA assistance MGD # 3 0 
Number of teachers/educators/teaching assistants trained or 
certified as a result of USDA assistance MGD # 5 0 
Number of teachers/educators/teaching assistants trained or 
certified as a result of USDA assistance (Male) MGD # 5 0 
Number of teachers/educators/teaching assistants trained or 
certified as a result of USDA assistance (Female) MGD # 5 0 
Number of school administrators and officials trained or certified 
as a result of USDA assistance MGD # 7 0 
71
Performance Indicator
Standard 
MGD 
Indicator 
Number
Baseline 
Target
Baseline 
Actual
Number of school administrators and officials trained or certified 
as a result of USDA assistance (Male) MGD # 7 0 
Number of school administrators and officials trained or certified 
as a result of USDA assistance (Female) MGD # 7 0 
Number of educational facilities (i.e. school buildings, 
classrooms, improved water sources and latrines) 
rehabilitated/constructed as a result of USDA assistance
MGD # 8 0 
Number of educational facilities (classrooms) 
rehabilitated/constructed as a result of USDA assistance MGD # 8 0 
Number of educational facilities (improved water sources) 
rehabilitated/constructed as a result of USDA assistance MGD # 8 0 
Number of educational facilities (latrines) 
rehabilitated/constructed as a result of USDA assistance MGD # 8 0 
Number of educational facilities (Kitchens, cook areas) 
rehabilitated/constructed as a result of USDA assistance MGD # 8 0 
Number of educational facilities (Other school grounds or 
building (storerooms) rehabilitated/constructed as a result of 
USDA assistance
MGD # 8 0 
Number of public-private partnerships formed as a result of 
USDA assistance MGD # 12 0 
Number of public-private partnerships formed as a result of 
USDA assistance (Education) MGD # 12 0 
Number of public-private partnerships formed as a result of 
USDA assistance (Health) MGD # 12 0 
Number of public-private partnerships formed as a result of 
USDA assistance (Nutrition) MGD # 12 0 
Number of public-private partnerships formed as a result of 
USDA assistance (Multi Focus) MGD # 12 0 
Number of Parent-Teacher Associations (PTAs) or similar 
"school" governance structures supported as a result of USDA 
assistance
MGD # 13 0 
Quantity of take home rations provided (in metric tons) as a 
result of USDA assistance MGD # 14 0 
72
Performance Indicator
Standard 
MGD 
Indicator 
Number
Baseline 
Target
Baseline 
Actual
Quantity of take home rations provided (in metric tons) as a 
result of USDA assistance (Rice) MGD # 14 0 
Quantity of take home rations provided (in metric tons) as a 
result of USDA assistance (Vegetable Cooking Oil) MGD # 14 0 
Quantity of take home rations provided (in metric tons) as a 
result of USDA assistance (Lentils) MGD # 14 0 
Number of individuals receiving take-home rations as a result of 
USDA assistance MGD # 15 0 
Number of individuals receiving take-home rations as a result of 
USDA assistance (New) MGD # 15 0 
Number of individuals receiving take-home rations as a result of 
USDA assistance (Continuing) MGD # 15 0 
Number of individuals receiving take-home rations as a result of 
USDA assistance (Male students) MGD # 15 0 
Number of individuals receiving take-home rations as a result of 
USDA assistance (Female students) MGD # 15 0 
Number of individuals receiving take-home rations as a result of 
USDA assistance (Pregnant and Lactating Women) MGD # 15 0 
Number of individuals receiving take-home rations as a result of 
USDA assistance (Others) MGD # 15 0 
Number of daily school meals (breakfast, snack, lunch) 
provided to school-age children as a result of USDA assistance MGD # 16 0 
Number of school-aged children receiving daily school meals 
(breakfast, snack, lunch) as a result of USDA assistance MGD # 17 0 
Number of school-aged children receiving daily school meals 
(breakfast, snack, lunch) as a result of USDA assistance (Male) MGD # 17 0 
73
Performance Indicator
Standard 
MGD 
Indicator 
Number
Baseline 
Target
Baseline 
Actual
Number of school-aged children receiving daily school meals 
(breakfast, snack, lunch) as a result of USDA assistance 
(Female)
MGD # 17 0 
Number of school-aged children receiving daily school meals 
(breakfast, snack, lunch) as a result of USDA assistance (New) MGD # 17 0 
Number of school-aged children receiving daily school meals 
(breakfast, snack, lunch) as a result of USDA assistance 
(Continuing)
MGD # 17 0 
Number of individuals trained in safe food preparation and 
storage as a result of USDA assistance MGD # 22 0 
Number of individuals trained in safe food preparation and 
storage as a result of USDA assistance (Male) MGD # 22 0 
Number of individuals trained in safe food preparation and 
storage as a result of USDA assistance (Female) MGD # 22 0 
Number of individuals trained in child health and nutrition as a 
result of USDA assistance MGD # 23 0 
Number of individuals trained in child health and nutrition as a 
result of USDA assistance (Male) MGD # 23 0 
Number of individuals trained in child health and nutrition as a 
result of USDA assistance (Female) MGD # 23 0 
Number of schools using an improved water source MGD # 27 46 161
Number of schools with improved sanitation facilities MGD # 28 169 228
Number of students receiving deworming medication(s) MGD # 29 0 
Number of schools reached as a result of USDA assistance MGD # 32 0
74
The following annexes are included in a separate document. 
ANNEX E. PUPIL LITERACY ASSESSMENT
ANNEX F. PUPIL SURVEY
ANNEX G. TEACHER SURVEY AND CLASSROOM OBSERVATION TOOL
ANNEX H. SCHOOL-BASED SURVEYS
ANNEX I. SCHOOL OBSERVATION TOOL
ANNEX J. FOCUS GROUP DISCUSSION GUIDE