August 2018
This publication was produced at the request of the United States Department of Agriculture. It was 
prepared independently on behalf of Catholic Relief Services by Converge SL.
Final Evaluation Report: Phase III of the McGovern-Dole 
Food for Education Program in Sierra Leone
i 
FINAL EVALUATION OF 
PHASE III OF THE 
MCGOVERN-DOLE FOOD 
FOR EDUCATION 
PROJECT (MGD III) IN 
SIERRA LEONE
August 8, 2018 
Program: McGovern-Dole International Food for Education and Child Nutrition 
Agreement Number: FFE-636-2015/018-00
Funding Year: Fiscal Year 2015
Project Duration: 2015-2018
Implemented by: Catholic Relief Services
Evaluation Authored by: Converge SL
ii
DISCLAIMER
The author’s views expressed in this publication do not necessarily reflect the views of 
the United States Department of Agriculture or the United States Government. 
Accessibility Note: An accessible version of this report can be made available upon 
request by emailing FAS.monitoring.evaluation@usda.gov. 
iii
CONTENTS
Final Evaluation Report: Phase III of the McGovern-Dole Food for Education Program in Sierra Leone .. 1
Contents..................................................................................................................iii
Table of Figures .......................................................................................................iv
List of Tables ............................................................................................................ v
Acronyms ................................................................................................................ vi
Executive Summary ................................................................................................... 7
Background ........................................................................................................... 7
Purpose ................................................................................................................. 7
Methodology.......................................................................................................... 7
Findings and Conclusion........................................................................................ 7
Recommendations ...............................................................................................10
Introduction............................................................................................................13
Country Context ..................................................................................................13
Project Background..............................................................................................14
Intervention Logic................................................................................................15
Review Purpose & Questions ...................................................................................18
Evaluation Purpose ..............................................................................................18
Review Questions................................................................................................. 18
Evaluation methods................................................................................................. 20
Sampling and Sample Size ...................................................................................20
Data Quality, Management, and Analysis..............................................................24
Findings..................................................................................................................24
EFFECTIVENESS: To what extent has the project achieved its OBJECTIVES? ............25
iv
RELEVANCE: To what extent does the project conform to the needs and priorities 
of target groups and the policies of the country and donor? ................................63
EFFICIENCY: How could integration of the two strategic objectives be enhanced or 
further leveraged to deepen positive program results? .........................................64
SUSTAINABILITY: To what extent will the benefits of the project continue after the 
end of the project life? .........................................................................................65
IMPACT: What were the positive or negative effects, both intended and unintended, 
of the project? .....................................................................................................68
Conclusion & recommendations..............................................................................69
Conclusion ..........................................................................................................69
Recommendations ...............................................................................................70
References ..............................................................................................................73
Annexes .................................................................................................................74
TABLE OF FIGURES
Figure 1. Results Framework for the MGD III Program.................................................16
Figure 2. Percentage of boys and girls who can read and understand grade-level text 27
Figure 3. Mean Number of letters identified correctly (out of 26)................................28
Figure 4. Mean scores in Phonemic Awareness subtasks (out of 10) ...........................29
Figure 5. Mean number of familiar words correctly read (out of 40)............................30
Figure 6 Mean Listening Comprehension Score (out of 4) ...........................................30
Figure 7. Teacher attendance rate on the day of the survey. .......................................32
Figure 8. Level of Teachers’ Satisfaction with Various elements of the MGD III program 
(% of teachers VERY satisfied).....................................................................................33
Figure 9. Availability of teaching and learning materials and supplies.........................33
Figure 10. Availability of desks and benches in observed classrooms .........................34
Figure 11. Availability of textbooks or readers in observed classrooms ......................35
Figure 12. Supervisory visits to teachers in the month and year prior to the survey ....37
Figure 13 Area 1: Teachers Demonstrate good instructional practice..........................39
Figure 14. Area 2 Teacher uses a variety of pupil assessment techniques...................40
v 
Figure 15. Area 3 Teacher engages pupils throughout the lesson..............................41
Figure 16. Teacher proficiency across all 8 teaching practice areas ............................42
Figure 17. Aspects of their jobs that teachers find “motivating” or “highly motivating”43
Figure 18. Percentage of classrooms that had (little/moderate/extensive) evidence of 
student attentive behaviors ........................................................................................45
Figure 19 Hunger status of children ...........................................................................46
Figure 20. Status of School Infrastructure...................................................................47
Figure 21. Status of water and toilet facilities in observed schools..............................48
Figure 22 Pupils’ knowledge of good health and hygiene ...........................................51
Figure 23. Cook’s knowledge of safe food preparation and storage practices .............53
Figure 24. Status of Kitchens in Schools .....................................................................54
Figure 25. Status of Storage Facilities.........................................................................55
Figure 26. Food groups contained in pupils’ diet on the day prior to the survey .........56
LIST OF TABLES
Table 1. Number of sample units allocated by chiefdom.............................................22
Table 2. Response Rates for different units of analysis ...............................................23
Table 3. Distribution of scores in the reading comprehension test (%) ........................26
Table 4 Summary of Teacher Characteristics (N=164) (Percentage) .............................35
Table 5 Teacher Training Profile.................................................................................36
Table 6 Scoring guidelines for teacher observations ...................................................38
Table 7 Trends in Enrollment, 2016-2018 ..................................................................49
Table 8. Factors that predict whether a student can read and understand grade-level 
text............................................................................................................................49
Table 9 Availability of kitchen equipment and Utensils ...............................................57
Table 10. Key Performance Indicators: Midline, Baseline and End of Project Targets ..59
vi
ACRONYMS
CRS Catholic Relief Services
CSR Country Status Report (on Education)
CTA Community Teacher Association
DEO District Education Office
DTM Diagnostic Teaching Methods
FWR Familiar Word Reading
GOSL Government of Sierra Leone
LOP life of project
M&E Monitoring and Evaluation
MC Mother’s Club
MEST Ministry of Education, Science and Technology
MGD McGovern-Dole
MICS Multiple Indicator Cluster Survey
NP Northern Polytechnic
PA Phonemic Awareness
PQTR Pupil Qualified Teacher Ratio
PTR Pupil-Teacher Ratio
SCR School Census Report
SILC Savings and Internal Lending Committee
SMC School Management Committee
TALLE The Association of Language and Literacy Educators
TLM Teaching and Learning Materials
USDA United States Department of Agriculture
WAEC West African Examination Council
WASSCE West African Senior Secondary Certificate Examination
WFP World Food Programme
7 
EXECUTIVE SUMMARY 
BACKGROUND 
This report presents the findings of the final evaluation of Catholic Relief Services’ 
(CRS) implementation of Phase III of the McGovern-Dole Food for Education (MGD III) 
project in Sierra Leone. CRS has been implementing this project, which is funded by 
the United States Department of Agriculture (USDA), in selected chiefdoms in 
Koinadugu district since 2008. The McGovern-Dole Program aims to reduce hunger 
and improve literacy and primary education, especially for girls. The MGD III project in 
Sierra Leone has two overarching goals: (1) to improve the literacy skills of children in 
the 192 schools targeted by the program and (2) to improve health and hygiene 
practices in targeted school communities. It is a 3-year project and this evaluation 
comes at the end of the third year. The project aims to achieve these goals by 
providing school meals, teacher training, improving water and sanitation 
environments, and improving skills and knowledge of various beneficiaries. For Sierra 
Leone, which is one of the poorest countries in the world that is also recovering from a 
recent Ebola epidemic, this partnership with CRS/USDA has been constructive. 
PURPOSE
The purpose of this final evaluation was two-fold: (1) to measure the overall results of 
the project strategies in terms of their effectiveness, relevance, efficiency and 
sustainability; and (2) to recommend actions for future iterations of the project. 
METHODOLOGY 
The final evaluation used a mixed-method approach, which combined the use of 
qualitative and quantitative data. Quantitative data was collected from pupil surveys, 
reading assessments, classroom observations, and school observation checklists. The 
qualitative data came from semi-structured interviews with key informants and focus 
group discussions. The team interviewed head teachers, government officials, CRS 
staff, and staff of other partner organizations and held focus group discussions with 
community members. 
FINDINGS AND CONCLUSION
8 
The most important findings from the final evaluation are provided below.
Effectiveness
The project met and surpassed its overall goal of having 25 percent of boys and girls 
being able to read and understand grade-level text at the end of grade 2. The project 
achieved 56 percent of girls and 61 percent of boys reading with understanding; a 
remarkable improvement from the baseline evaluation where only 8 percent of boys 
and girls could read with understanding. 
In addition to meeting its overall goals, the project also saw remarkable improvements 
in its intermediate results such as increased skills and knowledge of teachers, 
improved student and teacher attendance, reduced hunger, increased attentiveness, 
and improved teaching and learning conditions in schools. The changes were all 
positive and large, even though not all the LOP targets were met. 
 
Relevance
The MGD III project, to a very large extent, conforms to the needs and priorities of the 
government, communities, and donor. There was a change in government during the 
implementation of the MGD III program, and the new government has made a 
commitment to education, recently unveiling its flagship project – providing free, 
quality education for all children in government-supported schools. Parents, teachers, 
and community members are all very satisfied with their participation in the program, 
mostly because they can see improvements in the schools and the capabilities of 
students. Students are not hungry in school, and they can read. Where communities 
expressed dissatisfaction, it was because they wanted MGD III to do more for the 
schools, whose needs are greater than this one project could satisfy. 
Efficiency
The MGD III integrated interventions in education and health to improve the overall 
wellbeing of students in the school, and ultimately improve reading outcomes in the 
schools. Children who are healthy and do not miss school due to illness have a higher 
opportunity to learn. Efficiency also has to do with cost-effectiveness of the 
9 
intervention – trying to understand the cost-benefit of the interventions and whether 
the results could have been achieved at a lower cost. Because of the complexity of the 
theory of change and the limitations of the available funding data, it was difficult for us 
to do a cost-benefit analysis. However, there are some areas in which some efficiencies 
could be realized such as (1) improving the pass rates of teachers who take the teacher 
certification exam by NP; (2) providing targeted support to schools who need it the 
most rather than spreading scarce resources unnecessarily; (3) transitioning
government-supported schools from the MGD III program; and (4) reducing number of 
indicators being tracked to focus only on the most important ones. 
Sustainability
Sustainability is an important feature of McGovern-Dole programs worldwide, and for 
USDA, this means that schools served by MGD III can continue the activities on their 
own or with support from other sources such as the government or local community 
partners after USDA’s support ends. 
CRS took several steps to ensure sustainability of the MGD III project, including 
aligning the project objectives to the goals of the government, developing the capacity 
of various stakeholders, and helping schools through the formal approval process that 
qualifies them to receive government support. 
During the focus group discussions, communities expressed a willingness to continue
the activities after the project ends by developing school gardens, maintaining school 
infrastructure, and continuing to send their children to school. While their willingness 
is encouraging, communities still struggle till now with providing basic condiments and 
utensils to support the CRS school feeding program. It is unlikely that they will 
continue with many of the MGD III activities.
For government-approved schools, it is possible that the Ministry of Education, Science 
and Technology (MEST) and its partners will continue some of these activities. With the 
new Free Quality Education program that will launch in September 2018, the GOSL will 
provide tuition-free education, textbooks and uniforms for students, and expand the 
national school feeding program. Schools that are not approved will not be eligible for 
10
these supports. A key strategy for sustainability will be to intensify advocacy efforts to 
get these schools approved and government-supported. 
RECOMMENDATIONS 
The recommendations in this section are derived from the findings and suggest ways 
to strengthen key areas of the project to improve the chances of meeting or surpassing 
the end of project targets. The main recommendations are as follows: 
Improved Literacy Skills 
• Gender differences in reading: there are still some gender differences in 
reading, with boys performing better than girls. The project should promote 
gender equality by ensuring that there is no discrimination in terms of gender in 
teaching practices, curriculum materials, or school culture. Gender 
responsiveness should be an integral part of all training. The teaching staff is 
overwhelmingly male, and some affirmative action may be needed to encourage 
more women to enter teaching as that has been shown to have positive effect on 
the learning outcome for girls. 
• Use of teaching and learning materials: in at least a third of classrooms, only 
the teacher had and used a textbook. The project team should continue to 
encourage teachers to distribute textbooks in the classrooms for children to 
use. Use of textbooks should be an integral part of teacher training and 
coaching. 
• School approval: the MGD III has made good progress in terms of getting 
schools to submit an application for approval from the GOSL. However, there are 
still too many schools that are not approved. It is critical that all schools submit 
their application, even as they work towards meeting the eligibility criteria. 
• Community engagement: engage parents and communities in support reading 
(and not just the feeding program). Parents and caregivers, who can, should 
read to their children. Those who can't, should encourage their children to read 
to them, tell them stories, or encourage their children to join a reading club. 
• Reading clubs: encourage children to join reading clubs as children who are 
members of reading club tend to perform better. This is likely because these 
clubs provide children with an opportunity to practice their reading and 
introduce them to different kinds of reading materials. For children who cannot 
join reading clubs, they should be encouraged to take some reading materials 
home. 
11
Improved health and hygiene practices
• Intensify efforts to complete the construction of wells and reduce delays in the 
process. Access to safe water is needed. 
• Care should be taken while constructing wells to ensure that it is done at the 
right time of the year. Many schools had wells that had dried up at the time of 
the evaluation, which sometimes suggests that the well was dug at the wrong 
time, and not when the water table is at its lowest.
• Work with communities to ensure that the needed cooking and eating utensils 
are provided.
• Ensure that children from the poorest families are not penalized because their 
parents cannot afford to pay the cost of condiments. 
• Better measures are needed for some of the indicators – for example, asking 
children about the foods they ate the previous day as a proxy for increasing
knowledge of nutrition is not a very good measure as children don’t usually 
have control over what food they eat in these contexts. 
Other recommendations
• Develop an exit and sustainability plan with communities, local government and 
MEST. 
• The role of literacy coaches has proved to be instrumental in providing much￾needed support for teachers. The sustainability strategy should identify who in 
the school community (head teacher, deputy head teacher, another teacher) can 
provide peer coaching to other teachers. 
• Evaluation of any future MDG project should include an experimental or quasi￾experimental design with a control group to improve the project’s ability to 
make causal claims. 
• Simplify the theory of change for this intervention. The current theory of change 
is complex and difficult to test. 
• Continue to work with the current government to design and test the national 
school feeding program and gain commitments that they will include the MGD III 
schools in the national school feeding programs when CRS phases out. 
• Encourage cross-chiefdom learning as there are differences in performance on 
both objectives across the various chiefdoms. For example, some communities 
have strategies for ensuring utensils are provided for children and cooks, while 
others do not. This provides an opportunity for learning across chiefdoms that 
could be facilitated by CRS. 
12
13
INTRODUCTION
COUNTRY CONTEXT
Sierra Leone is one of the poorest countries in the world, ranking 179 out of 188 
countries in the 2016 UNDP Human Development Index (United Nations Development 
Programme UNDP, 2016). It is also one of the most food insecure countries in the world, 
ranking 109 out of 113 countries on the 2017 Global Food Security Index from the 
Economic Intelligence Unit (The Economist Intelligence Unit, 2017). Part of the reason 
for the poor social and economic conditions is that the country was engulfed in a civil 
war lasting over a decade that officially ended in 2001. Unfortunately, in 2014, even as 
Sierra Leone was trying to rebuild its social and economic systems, the country was hit 
by the largest Ebola outbreak the world has ever seen. This outbreak led to a period of 
economic decline that the country is just trying to recover from. 
Education performance is low, and progress in achieving learning outcomes has been 
slow. A national assessment of reading in the early grades showed that many children 
could not read simple words or comprehend simple passages after three years of 
schooling (Montrose International, 2014). In addition, while remarkable progress had 
been made in enrolments at the primary level, the 2015 Population and Housing Census 
showed that 35 percent of children of primary-school age were not attending school
(Statistics Sierra Leone, 2016). Literacy rates are also among the lowest in the world, and 
the same census data showed that just 51 percent of the population over aged 10 was 
literate, with a mere 41 percent literacy rate for women. 
Presidential elections held in April 2018 was won by Julius Maada Bio of the Sierra 
Leone People’s Party (SLPP), while the All People’s Congress (APC) retained their 
majority in Parliament. President Bio has said that education is a key priority for his 
administration and declared Free Quality Education as the flagship programme of his 
government (Julius Maada Bio, 2018). During the budget speech by the new Minister of 
Finance, additional funding was allocated to the education sector to fund the Free 
Education Programme, which will become active in 2018/19 school year. The 
increased allocation will support free tuition for primary and secondary schools, 
14
provision of teaching, learning materials and textbooks; school feeding for primary 
schools, and examination fees for NPSE, BECE and WASSCE. While it is unclear exactly 
how this Free Quality Education programme will be implemented, education is a 
priority for the new administration. 
The United States Department of Agriculture (USDA) has been supporting Sierra Leone 
through the McGovern-Dole Food for Education (MGD) program for almost a decade in 
the Koinadugu District of Northern Sierra Leone. The MGD program helps support 
education, child development and food security in low-income, food-deficit countries 
around the globe. Its key objective is to reduce hunger and improve literacy and 
primary education, especially for girls, by providing school meals, teacher training, and 
related support, and boosting the health and nutritional status of primary school 
pupils. 
PROJECT BACKGROUND 
Catholic Relief Services (CRS) has been implementing the MGD project in Koinadugu 
District of Sierra Leone since 2008. Koinadugu was chosen because of the district’s 
food insecurity status, high malnutrition rates amongst children under age five, and 
below average education performance. The MGD program has been implemented in 
three distinct phases.
Phase 1: The first phase of the MGD program (MGD I) ran from 2008 to 2012 in four 
chiefdoms in Koinadugu - Sulima, Mongo, Neini, and Neya. These chiefdoms were 
chosen because they were the most marginalized in the district. Between 2008 and 
2012, the MGD I program distributed almost 1,500 metric tons of food, corresponding 
to 5,780,201 meals served to 18,610 pupils. The project also included take-home 
rations for girls in upper primary from which over 5,000 girls benefitted. In addition to 
food aid, the MGD I project trained school management committees (SMCs) and 
improved on school infrastructure. The MGD I project also distributed teaching and 
learning materials (TLMs) and furniture to schools. 
Phase II: The second phase of the program (MGD II) began in September 2012 and 
ended in January 2016. In Phase II, the program expanded to include a fifth chiefdom 
15
(Dembelia Sinkunia) and to include an additional 75 schools from existing program 
chiefdoms. Phase II also included additional activities such as in-service teacher 
training on Diagnostic Teaching Methods (DTM) to improve literacy instruction and 
established Savings and Internal Lending Communities (SILC) to help strengthen the 
financial status of households. The food aid component continued in Phase II using 
the same two modalities: (1) two in-school meals per day; and (2) take-home rations for 
girls in upper primary who maintain at least an 85 percent attendance rate. The Ebola 
outbreak created a public health emergency that challenged the implementation of 
MGD II activities and diminished its potential impact. 
Phase III: The third phase (MGD III), which is the focus of this final evaluation is being 
implemented in the same five districts as MGD II and will run from December 2015 to 
December 2018. The MGD III was intended to help the schools and communities 
rebuild after the effects of the Ebola epidemic, which included a 1-year closure of 
schools. Most of the activities remain the same as in MGD II, but there are important 
differences. First, MGD III provides one school meal daily, whereas MGD II provided 
two. The project also switched from bulgur to fortified rice as the staple food. These 
changes were made to align the school meals component more closely with the 
Government of Sierra Leone’s (GOSL) planned school feeding program. Second, MGD III 
intensified the attention paid to the literacy component of the program by hiring 25 
literacy coaches, who provide professional development support (training, coaching, 
mentoring) to teachers of the early grades. Third, the MGD III introduced reading clubs 
in all the schools, which provides a space for students to come together after school to 
read and study. The project also provided 2 solar light per school as the reading clubs 
tended to meet in the evening. Lastly, in response to the Ebola crises, MGD III includes 
a second strategic objective that focused on improving health and dietary practices of 
learners and the wider school communities. 
INTERVENTION LOGIC
The MGD program is a school feeding program with two main goals: (1) improved 
literacy of school-aged children and (II) improved health and dietary practices of pupils 
and communities. Figure 1 illustrates the intervention logic of the MGD III program. 
16
Figure 1. Results Framework for the MGD III Program
Under goal 1 (improved literacy), there are three sub-objectives (improved quality of 
literacy instruction, improved student attentiveness, and improved student attendance) 
and multiple activities (e.g. teacher training, school feeding, provision of teaching and 
learning materials). Under goal 2 (increased use of good health and dietary practices), 
there are six sub-objectives; three are related to increased knowledge of good health 
17
and hygiene practices, safe food preparation, and nutrition, and the other three are 
related to increased access to clean water, food preparation and storage tools, and 
preventative health services. CRS summarizes the theory of change of the MGD III project 
as follows:
In the implementation of the MGD III, CRS partners with local and international 
organizations such as The Association of Language and Literacy Educators (TALLE), 
Northern Polytechnic (NP), and Caritas. TALLE supports the coaching on literacy 
instruction, Caritas provides Life Skills Training and implements the WASH activities, and 
NP provides teacher certification through a distance education program. Throughout the 
implementation, all parties focus on the development of capacity in schools and 
communities, national and local government agencies and departments. 
IF the quality of literacy instruction is improved, AND children, 
having received school meals, have their short-term hunger 
reduced AND their attentiveness and attendance improved,
THEN the literacy of school-aged children in the project area 
will be improved.
Also, IF pupils learn about improved health and hygiene practices
at school AND cooks learn about safe food preparation and 
storage practices AND schools have improved access to clean 
water and sanitation sources AND pupils have increased access 
and knowledge to preventative health interventions AND food 
preparers have increased access to requisite food preparation and 
storage facilities, tools and equipment, THEN the use of health 
and dietary practices will improve. 
18
REVIEW PURPOSE & QUESTIONS 
EVALUATION PURPOSE 
The purpose of the final evaluation was to measure the overall results of the project 
strategies in terms of their relevance, effectiveness, efficiency, impact, and 
sustainability. The final evaluation also documents successes and lessons learned in 
implementation and management. 
REVIEW QUESTIONS 
The main questions for the final evaluation, which followed the DAC evaluation criteria, 
were:
EFFECTIVENESS: To what extent has the project has achieved its objectives? 
• To what extent are the project results and the yearly benchmark indicators 
achieved?
• Have the implementation strategies been relevant and effective enough to 
improve: 
• student’s literacy level
• enrolment and attendance among pupils, particularly girls
• community participation and engagement
• Are there some internal and/or external factors that are hindering the efficient 
implementation of project activities?
• Is the management system effective?
• Has program implementation been effectively monitored? How well has the 
monitoring and evaluation mechanism facilitated project implementation?
RELEVANCE: To what extent does the project conform to the needs and priorities of 
target groups and the policies of the country and donor? 
• Are the activities and outputs of the project consistent with the overall goals and 
the attainment of its objective?
• Are the activities and outputs of the project consistent with the intended 
impacts and effects?
• Does the program meet community and government expectations(?)? 
19
• Are stakeholders (school management committee, parents, teachers, local 
authorities) satisfied with their participation in the program? Why or why not?
• How well does the project complement and link to activities of other donors at 
the local level?
EFFICIENCY: To what extent did project inputs lead to expected results, and could the 
results have been achieved with fewer inputs or resources? 
• Have activities been cost efficient?
• Are results achieved on time?
• Is the project being implemented in the most efficient way compared to 
alternatives?
SUSTAINABILITY: To what extent will the benefits of the project continue after the end 
of the project life? 
• What activities and/or outcomes (both expected and unexpected) of the 
program are likely to be sustained? What evidence is there to suggest this?
• What are the major factors that can influence the achievement or 
nonachievement of the sustainability of the project?
• How do the government’s capacities, policies, procedures, and priorities 
contribute to sustainability?
• What strategies should be used to obtain long-lasting support from 
communities and local/central administration that goes beyond the time of the 
project
IMPACT: What were the positive or negative effects, both intended and unintended, of 
the project? Are these effects the direct result of the project intervention?
• What were the intended and unintended positive and negative effects of the 
intervention on children, communities, and institutions?
• How did the intervention affect the well-being of different groups of
stakeholders?
• What do beneficiaries and other stakeholders affected by the intervention 
perceive to be the effects of the intervention on themselves?
• To what extent can identified changes be attributed to the intervention?
20
EVALUATION METHODS 
The final evaluation adopted a mix of both quantitative and qualitative research 
methods. The quantitative survey design consisted of pupil survey, reading 
assessment, cook assessment, teacher and classroom observation and school 
observation checklist. The qualitative component had two parts: in-depth interviews 
with key informants (head teachers, project staff, MEST officials, and partner 
organizations) and focus group discussions with community members. Insights from 
the qualitative interviews and discussions complemented data obtained through the 
surveys and allowed researchers to examine certain topics in more depth.
The final evaluation, like the interim review, did not include a comparison group 
because it was difficult to find schools that were disadvantaged as much or in the same 
ways as the targeted MGD III schools. Because the available communities didn't 
provide a good comparison, the ethical issues of continuing to visit schools who did 
not receive the same resources while taking information from them became 
problematic. Therefore, management decided to use a pre/post design rather than a 
comparison group design. 
SAMPLING AND SAMPLE SIZE
A two-stage cluster sampling approach was employed to select pupils, teachers and 
cooks. At the first stage, schools were randomly selected as clusters and then pupils, 
cooks, and teachers within schools were selected at the second stage. The advantages 
of cluster sampling are that it is more economical, time-efficient, and easier to manage 
than a simple random survey. The disadvantage of this sampling technique is that 
individuals (teachers or pupils) within a cluster (school) tend to have similar 
characteristics, and there is a chance that they may not be representative of the overall 
population. 
Sample Size 
Pupils, teachers and food preparers were the sampling units of the quantitative survey. 
The sample sizes for the sampling units were computed using four (4) key indicators. 
21
MGD III baseline and midline results for these indicators were used for computing 
sample sizes for the sampling units. 
Pupils
The formula below was used to compute the sample size for pupils:
where,
• n is the required sample size (i.e., number of pupils to be surveyed)
• 4 is the factor required to achieve a 95% level of confidence;
• p is the value of the key indicator; in this case, the ‘percent of pupils identified as 
attentive during classroom activities (boy & girls) (0.58)1
; 
• deff (design effect) is the factor by which the precision of the estimates of the key 
indicators decrease due to the sampling design relative to a survey that uses 
simple random sampling (2)2
; 
• 1.2 is the factor necessary to raise the sample size by 20% for the expected non￾response3
. This assumes 80% response rate;
• The margin of error (± 5 percentage points) i.e., Alpha α=0.05 . 
Using the formula, the sample size for pupils was estimated at 900 for pupil survey 
and the reading assessment. The decision was made to select 15 children per school 
across grades 2-6 as too many learners from one school would reduce the variability in 
our sample, which meant that we needed to have 60 schools in our sample. 
The schools were the primary sampling unit and were targeted for the following 
surveys:
• Pupils survey,
• Reading assessment,
• School observation checklist, 
• Key informant interviews (with head teachers, SMCs, CTA & Mothers club)
 
1 MGD III baseline result. 2 The design effect, estimated at 2, is chosen because of the large cluster sample size (i.e. 15 pupils per school).
3 The 20% non-response is assumed to increase the sample size sufficiently large to be able to detect an effect on 
the following groups: meals only, meals + THR, meals + SILC and nothing.
( )
( )
2
arg
14 2.1
M in of error
pp deeff n
××−×× =
22
• Teacher and classroom observation. 
Once the schools were selected, a maximum of 3 teachers and 3 cooks per school were 
also selected. The surveys used are included in full in Annex B, and, except for a few 
questions, they are the same as those used during the baseline and midline. This 
makes it possible to compare the responses at final evaluation to those of earlier 
evaluations. 
Table 1 shows the number of schools and pupils to be sampled by chiefdom.
Table 1. Number of sample units allocated by chiefdom
Chiefdom No. of 
MGD 
Schools
Number of 
schools to 
sample
Number of 
pupils to 
sample
Number of 
Teachers 
Number of 
cooks
Dembelia Sinkunia 18 6 90 18 18
Mongo 43 13 195 39 39
Neya 37 12 180 60 60
Neini 65 20 300 36 36
Sulima 29 9 135 27 27
Total 192 60 900 180 180
The schools were selected by systematic random sampling, while the pupils, teachers, 
and cooks were chosen by simple random (ballot) method. 
Focus Group Discussions (FGDs)
FGDs were held in all survey clusters/school communities with community 
stakeholders including women and men from SILC and non-SILC households. 
Discussions were held separately for men and women and each FGD had between 7 
and 12 participants. Enumerators were instructed to select a wide range of ordinary 
community members as well as community leaders (chiefs, women’s leaders, and 
headmen). Community members who had already been interviewed because of their 
membership in SMCs and Mothers Groups were not included to increase the diversity 
of opinions.
Table 2 gives the response rates for schools, teachers, cooks, and pupils. Response 
rates were high and ranged from 91 to 100 percent.
23
 
Table 2. Response Rates for different units of analysis
Group Target 
number
Number of 
responses
Response 
rate 
Schools 60 60 100%
Teachers 180 164 91%
Pupils 900 900 100%
Cooks 180 163 91%
Recruitment and Training of Enumerators
Forty-two data collectors (36 enumerators and 6 supervisors) were recruited for the 
data collection. The supervisors and most of the enumerators had participated in 
previous evaluations of the MGD program, and therefore had experience with the tools 
and methodology. All enumerators had at least a post-secondary qualification and 
many had participated in other surveys and/or had taught in schools. 
The supervisors and enumerators attended a 5-day training on the administration of 
the evaluation tools/questionnaires and general survey protocols to equip them to 
collect appropriate and quality data for analysis. For this survey, quantitative data 
collection was done using mobile devices (iPad minis), and the enumerators had 
additional training on collecting data using mobile devices. Training also included role￾play to deepen practice on the administration of the tools and translation of keywords 
into the lingua franca (Krio) to help in the focus group discussions (FGD) and pupil 
interviews. 
Another focus of the training was on ethical issues in conducting surveys with children 
and vulnerable populations. The training included the importance of consent and 
protecting the rights and dignity of all those interviewed. This meant that each child 
had to agree to proceed with the survey and we did not take personally identifiable 
information (e.g., names) from them. We also asked that each child be interviewed in a
public place, but away from their teacher and others who could hear their answers. 
Finally, if it appeared children could not do a subtest, the enumerators were trained to 
stop that subtest and move to something else. 
24
Furthermore, at the end of the training, arrangements were made for enumerators and 
supervisors to practice administering the tools in nearby schools and communities. 
This allowed for data collectors to practice administering the tools in ‘real-life’ 
situations and gave an opportunity to test the appropriateness of the questions. After 
the pretesting, final revisions were made to the tools and rendered on the tablets 
before the start of the field research. 
DATA QUALITY, MANAGEMENT, AND ANALYSIS
The first step in ensuring that the evaluation produced good quality data was to 
develop well-defined questions that helped to capture accurate and relevant data for 
analysis. The evaluation tools/questionnaires had been reviewed multiple times since 
many of them were used in previous evaluations. The new questions added also went 
through multiple reviews. In addition, the questionnaires and surveys were piloted in 
the field before being adopted and administered to collect the data for the evaluation. 
During the administration of the survey, supervisors visited enumerators in their 
various sites to ensure that data collection was proceeding as planned and to provide 
quality assurance to the evaluation process. The use of mobile devices to collect data 
helped to produce good quality quantitative data. 
Quantitative data collected in the iPads were exported to Excel and then into SPSS 
software package in which further cleaning of data was carried out. Errors and 
mismatches due to inconsistencies during data collection were checked and corrected; 
this enhanced logical consistency and quality of data. The cleaned data was then 
exported to Stata 14 (a statistical software) for analysis. All programs created to 
analyze the data and produce tables have been saved so that the findings can be 
reproduced by other researchers.
FINDINGS
25
This section summarizes the major findings from the final evaluation organized by the 
evaluation questions. The data used in this analysis were collected between Monday, 
June 4th and 14th, about a month before the end of the 2017/18 school year. 
EFFECTIVENESS: TO WHAT EXTENT HAS THE PROJECT ACHIEVED ITS 
OBJECTIVES? 
The MGD III project has two main strategic objectives and numerous sub-objectives. 
Below are the findings organized based on these objectives and sub-objectives. Where 
possible, we have reported the starting point of the indicators, as measured during the 
baseline, the status during this interim review, and the final target at project end. 
MGD S.O.1 Improved Literacy of School-aged children
One of the main strategic objectives (SO) of MGD III is to improve the literacy skills of 
children in the program. USDA’s standard indicator to measure literacy skills is: the 
“percentage of children, who at the end of two grades of schooling, demonstrate that 
they can read and understand grade-level text.” Although the children in class 2 were 
the population of interest, children in class 3 were also sampled because previous 
assessments had shown that children in class 2 had minimal reading skills and scored 
zero in several of the reading subtests. 
The literacy assessment tool used was adapted from a nationally developed and 
validated test developed by Dr. Johanna Kuyvenhoven, a literacy educator from Calvin 
College. As a consultant with UNICEF, the assessment tool was used to assess children 
all over Sierra Leone. The skills assessed include: phonemic awareness, letter 
identification, familiar word reading, and listening and reading comprehension. It does
not test oral reading fluency since the reading tests are not timed. For ease of 
comparison, the reading assessment remained the same across the baseline, midline 
and final evaluations, with only changes to the order of letters and words. 
In order to test reading comprehension, learners were given a short, grade-appropriate
story to read aloud after which the enumerators asked five comprehension questions. 
The grade 3 text was slightly more difficult than the grade 2 text to account for grade 
progression. Only learners who could read the first sentence of the text correctly 
26
proceeded with the reading comprehension test. The number of questions asked from
each pupil depended on how far in the story each child could read. Children were 
asked only those questions that could be answered from the text that they read. Four
questions in this subtask were literal and could be answered directly from the 
information provided in the story. The last question was an inferential question and 
required pupils to combine information from the story with their background 
knowledge to derive a correct answer. A learner is said to be able to “read and 
understand grade-level text” if they answer at least 4 of 5 questions correctly on the 
reading comprehension test. 
In total, 951 pupils (474 in class 2 and 477 in class 3) took the reading assessment, of 
which 54 percent were boys. Table 3 shows the distribution of scores on the reading 
comprehension. About 17 percent of class 2 students and 6 percent of class 3 
students scored zero on the comprehension subtest; 37 percent of class 2 students 
and 46 percent of class 3 students answered all the 5 questions correctly. 
Table 3. Distribution of scores in the reading comprehension test (%) 
Number 
Correct
Class 2 Class 3
Boy Girl Total Boy Girl Total
0 16.2 18.6 17.3 3.5 8.1 5.7
1 3.6 3.6 3.6 0.4 2.7 1.5
2 4.7 6.3 5.5 4.3 11.3 7.6
3 14.6 15.4 15.0 13.3 13.6 13.4
4 21.3 23.1 22.2 29.7 22.6 26.4
5 39.5 33.0 36.5 48.8 41.6 45.5
Based on the indicator definition, the percentage of children who can read and 
understand grade-level is depicted in Figure 2 below. The table shows remarkable 
achievement over the course of the MGD III. At the final evaluation shows that 61 
percent of boys and 56 percent of girls could read and understand grade-level text at 
the end of class 2, compared with only 8 percent during the baseline. The project has 
far exceeded its target of 25 percent of class 2 students being able to read and 
understand grade-level text. A lower percentage of girls than boys can read with 
understanding in both classes (32 percent vs. 39 percent in class 2), but the difference 
is statistically significant for only class 3 learners (64 percent vs. 79 percent). 
27
Figure 2. Percentage of boys and girls who can read and understand grade-level text 
In addition to the reading comprehension sub-test, we also assessed learner’s progress 
in several reading skills such as alphabet naming, phonemic awareness, phonics, and 
listening comprehension. 
Alphabet Naming
In this task, children were presented with a random list of the 26 letters of the 
alphabet (in mixed upper and lower case) and were asked to name the letters. The 
average number of letters identified correctly was 25, improved from 19 at the 
baseline (Figure 3). Boys scored higher than girls and students in class 3 higher than 
those in class 2. However, these differences are not statistically significant. 
8 8
15 12
39
32
44
37
61
56
79
64
0
10
20
30
40
50
60
70
80
90
100
Boy Girl Boy Girl
Class 2 Class 3
% of learners
Baseline Midline Final Target
28
Figure 3. Mean Number of letters identified correctly (out of 26)
Phonemic Awareness (PA)
There were two types of PA tasks. In the first, the learner was given pictures of 10 
common objects, told the names of the object, and asked to say the initial sound. In 
the second PA task, the learner was asked to read a set of invented words or nonwords 
(e.g.’ kum’). The second task is a new task introduced in the final evaluation. 
Figure 4 compares the mean scores in the test of beginning sound and shows strong 
improvement over the year. The mean score at the final was 8 out of 10 compared to 3 
at the baseline, and there was no gender difference in the score. 
20
18 17
21
19
23 23 22
24 23
25 24 24 25 25
0
5
10
15
20
25
30
Boy Girl Class 2 Class 3 Total
Baseline Midline Final
29
Figure 4. Mean scores in Phonemic Awareness subtasks (out of 10)
Familiar Word Reading 
In this sub-task, learners were presented with a list of 40 familiar words and were 
asked to read the words. The test was discontinued if the pupil couldn’t read any of 
the first ten words. Learners in class 2 and class 3 had a different set of words, but 
with some overlap. Again, there was a remarkable improvement in this sub-task over 
the course of the project, as shown in Figure 5. There was no gender difference in 
performance. 
3 3
2
3 3
6 6 6
7
6
8 8 8 8 8
0
1
2
3
4
5
6
7
8
9
10
Boy Girl Class 2 Class 3 Total
Baseline Midline Final
30
Figure 5. Mean number of familiar words correctly read (out of 40)
Listening Comprehension (LC)
For this sub-task, the assessor read a passage to the learners, and the learners were 
asked questions about the story they listened to. The mean score for boys was 3.3 and 
for girls 3.2, and this difference is not statistically significant. Listening comprehension 
scores have increased quite dramatically since the start of the program. 
Figure 6 Mean Listening Comprehension Score (out of 4)
12
9 9 9
11
23
21 20
24
22
31 30
28
33
31
0
5
10
15
20
25
30
35
40
Boy Girl Class 2 Class 3 Total
Baseline Midline Final
1.0 1.0 1.0 1.0 1.0
2.0
1.9 1.8
2.1
2.0
3.3 3.2 3.2
3.4 3.3
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Boy Girl Class 2 Class 3 Total
Baseline Midline Final
31
To summarize, pupils performed remarkably better on all sub-tests at the end of the 
project. The reading comprehension target of 25% of boys and girls reading with 
comprehension at the end of grade 2 was surpassed with 57 percent of girls and 61 
percent of boys reading with comprehension. While there was a difference between 
boys and girls in terms of reading performance, the difference wasn’t statistically 
significant in class 2. The difference between boys and girls do appear to be widening 
as they progress in school – by class 3, the gap between the performance of boys and 
girls had increased, and the difference was statistically significant. In the following 
subsections, we consider what has changed in instruction and the teaching and 
learning environment that might help explain the improvements we see in reading 
outcomes. 
MGD.1.1 Improve quality of instruction
Improving the quality of literacy instruction is a major part of the implementation 
strategy of the MGD III program. The program aims to develop the skills of teachers in 
several ways including sponsoring teachers to acquire their teaching certificate from 
Northern Polytechnic; providing in-service training in Diagnostic Teaching Methods 
(DTM) designed by the by International Literacy Association (ILA); and coaching of 
teachers in literacy instruction by TALLE. 
MGD 1.1.1 More consistent teacher attendance
For teaching and learning to take place at a minimum, pupils and teachers must be 
present in school. Almost all schools (97 percent) had a book for recording teacher 
attendance. The teacher attendance rate on the day of the survey, based on the 
attendance register, was 92 percent overall (95 percent for men and 88 percent for 
women). This was up from the attendance rate at the baseline survey (see Figure 7). 
32
Figure 7. Teacher attendance rate on the day of the survey.
The increased attendance rate may reflect teachers’ satisfaction with the MGD III
program. As shown in Figure 8, most teachers are very satisfied with all aspects of the 
program, with the highest level of satisfaction being with the school meals, reading 
clubs and provision of TLMs and the lowest being with the construction of school 
buildings and WASH facilities and SILC support. In areas where teachers said they were 
not satisfied with any aspect of the program, it was usually because they did not have 
those elements in their schools or communities. For example, they might not have SILC 
groups formed or their school buildings had not been rehabilitated or they needed 
more classroom buildings than they had. It is also possible that teachers and head 
teachers knew the evaluation was taking place, and therefore tried to be present. 
However, on-going monitoring data also suggests that teacher attendance is relatively 
high. 
87% 83%
87% 90% 88% 90%
95%
88%
92%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Male Female Total
Baseline Interim Final
33
Figure 8. Level of Teachers’ Satisfaction with Various elements of the MGD III program (% of teachers 
VERY satisfied)
MGD 1.1.2 Better access to school supplies and materials
To be effective, teachers need materials to help them teach. At the final evaluation, 
classrooms were print-richer with a higher percentage of classrooms featuring 
children’s work, vocabulary words or alphabet strips, and WASH posters on the wall 
(Figure 9). 
Figure 9. Availability of teaching and learning materials and supplies
51
55
60
66
67
74
76
86
87
94
0 20 40 60 80 100
Construction of buildings and WASH facilities
SILC Support
Furniture
Life Skills
Advocacy
In Service Training
Distance Education Programme
Provision of TLM
Reading Clubs
 Food supplies/School Meals
Percent (%) of Teachers
33
20
17 14
62
49
61
38
55
75
70
53
0
10
20
30
40
50
60
70
80
Teacher Furniture Pupil's work on wall Vocabulary words or
alphabet strip on wall
WASH Poster
Percent (%) of observed classrooms
Baseline Midline Final
34
The seating arrangement in schools had improved since the baseline, when 9 percent 
of schools had no seating for students (Figure 10). At midline and final, 71 percent of 
classrooms had students seated either 2 or 3 to a desk, which is adequate seating; 28 
percent or 28-29 percent of classrooms had more than 3 children sharing desk. Most 
of the classrooms (70%) where more than three children were sharing a desk/bench 
were in Mongo and Neini chiefdoms. The situation had improved since the baseline, 
when 48 percent of classrooms did not have adequate seating.
Figure 10. Availability of desks and benches in observed classrooms
MGD 1.1.3 Improved literacy instructional and learning materials 
This year, the MGD III project delivered over 470,000 texts and other teaching and 
learning materials (TLMs) to schools, surpassing its life-of-project (LOP) target of 
270,000. These TLMs included slates, chalk, exercise books, and reader and textbooks 
for the four core subjects (Maths, English, Social Studies, Science). All head teachers 
reported receiving some TLMs from the MGD III project. The availability and use of 
textbooks in the observed classrooms are shown in Figure 11 below. The percentage 
of observed classrooms with no texts being used dropped from 25 percent at baseline 
to 3 percent at the end of the project, which is good progress. However, in at least a 
third of classrooms, only the teacher was using a textbook even though all head 
teachers reported that their schools had received texts from CRS. This means that even 
though schools had the textbooks, they were not being used in the classroom. 
8 9 14
41
62 57
39
28 28
9
1 0
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Baseline Midline Final
There are no desks/benches
More than 3 children share a
desk/bench
Three children share a desk/bench
Two children share a desk/bench
Each child has own desk/bench
35
Enumerators reported that in some schools, the textbooks are kept in stores or head 
teacher offices, with limited access to children. In other schools with no office, the 
head teachers keep the TLMs in their homes, which also makes it inaccessible for 
children’s use during class time. 
Figure 11. Availability of textbooks or readers in observed classrooms
MGD 1.1.4 Increased skills and knowledge of teachers
During this final evaluation, enumerators observed and interviewed 164 teachers in 
their classrooms. The teaching workforce is mostly male (87 percent) and only 38 
percent of them had a teaching certificate. Ninety-five percent were teaching classes 2 
to 4, while 5 percent taught classes 5 to 6. 
Table 4 Summary of Teacher Characteristics (N=164) (Percentage)
Teacher Characteristics Final Baseline
% Male 87 92
% Has Teaching Certificate 38 31
Highest Qualification if no Teaching Certificate
8 11
18
11 11
25 27
11
19
31 53
32
25
14
3
0
10
20
30
40
50
60
70
80
90
100
Baseline Midline Final
Percent (%) of observed classroms
There are no books or readers
By the teacher only
By the children, 3 or more
By the children, shared by two
By the children, one each
36
Teacher Characteristics Final Baseline
% BECE 38 21
% WASSCE 53 44
% O'LEVEL 7 27
% Other 2 7
Class level
% Class 2-4 95% 100
% Class 5-6 5%
Teacher training 
The MGD III project invested significant resources in improving the quality of 
instruction through training and coaching of teachers. Monitoring data showed that 
701 teachers were trained in Year 2 and 975 teachers in Year 3 of the project, 
surpassing the LOP target of 701. These teachers were trained in Diagnostic Teaching 
Methods (DTM) for literacy instruction. In addition to the DTM training, eligible 
teachers were enrolled in a Distance Education Programme (DEP), led by Northern 
Polytechnic. The DEP is a one-year program, at the end of which enrolees take a 
national examination. Those who pass the exam receive a teaching certificate (TC￾Lower) from the National Council of Technical and Vocational Awards (NCTVA). Project 
records showed that there were 530 teachers trained or certified as of Year 3, 
surpassing the LOP target of 450. The results of the 2017 NCTVA exam was released in 
July 2018, and 60 percent of MGD-III supported teachers passed and should be 
receiving their certificates. 13 percent will need to resit one or two subjects, while 26 
percent failed. 
As Table 5 shows, 90 percent of teachers in the final evaluation sample reported 
having attended training on DTM and 52 percent were participating in the DEP. Seventy 
percent of teachers received training on life skills instruction as part of the MGD III
program. These training rates were much higher than the training rates reported 
during the baseline. 
Table 5 Teacher Training Profile
Training Final Baseline
% Diagnostic Training Method (DTM) 90 28
% Distance Education Program 52 20
% Life Skills Training 70 40
37
Coaching and Mentoring
In addition to the training, MGD III also hired 20 literacy coaches and 5 coach 
supervisors, who were responsible for training teachers and providing classroom-based 
coaching and support. Each literacy coach covered 8-11 schools in a geographic cluster 
and spent at least 3 weeks per month in the schools giving direct support to teachers. 
During these coaching visits, they observed teachers in their classrooms and provided 
feedback on their teaching. Where needed, they also provided demonstration lessons 
to help struggling teachers. Ordinarily, MEST supervisors are supposed to be doing this 
task, but there are not enough of the supervisors and their resources to visit schools 
are limited. 
Figure 12 shows that teachers received visits from coaches, head teachers, and MEST 
supervisors, but the frequency of these visits differed. 98 percent of teachers reported 
that they were visited by a coach in the month prior to the survey, 95 percent reported 
visits by the head teacher during the year, while 51 percent reported visits by a MEST 
supervisor. The results show that MEST supervisors still struggle to visit school, but the 
coach visits make up for their absence. 
Figure 12. Supervisory visits to teachers in the month and year prior to the survey
96 93
56
98 95
51
0
10
20
30
40
50
60
70
80
90
100
Coach
(past month)
Head teacher
(during year)
MEST supervisor
 (during year)
Percent (%) of teachers
Midline Final
38
Observation of Teaching Skills
The skills and knowledge of teachers were assessed using teacher surveys and 
teaching observations. The rubric used for the teaching observation and teacher’s 
knowledge of pedagogy combined tools developed by International Literacy
Association (ILA) and MEST. Teachers’ competencies were assessed in 2 broad areas as 
outlined below:
1. The teacher uses a variety of pupil (formative) assessment techniques during the 
lesson
2. The teacher demonstrates good instructional practice (explores prior 
knowledge, uses lesson plans, a variety of teaching methods)
Teacher performance was scored using the guidelines shown in Table 6. 
Table 6 Scoring guidelines for teacher observations
1 Not yet 
Started
2 Beginning 3 Proficient 4 Excellent
There is no evidence 
of desired behavior. 
The teacher needs 
significant support to 
develop practice. 
The behavior is 
attempted, but not 
consistent. The 
teacher needs 
ongoing support to 
develop practice
The behavior is 
acceptable and 
somewhat consistent 
and could be used as 
a model for others. 
The behavior is 
consistent and 
exemplary. The 
teacher could teach 
others to develop this 
behavior. 
Figure 13 below shows how teachers performed in Area 1, and across all areas, 
teachers show marked improvement over time. For example, at the final evaluation 
73 percent of teachers were judged to be proficient in lesson planning, compared to 
46 percent at midline and 22 percent at baseline. 
39
Figure 13 Area 1: Teachers Demonstrate good instructional practice
Area 2 focused on teachers’ proficiency in using a variety of pupil assessment 
techniques during the lesson, which is one of the foundation skills of the DTM training. 
Figure 14 shows that 84 percent of teachers were proficient in checking for pupil
understanding at final, compared to 54 percent at midline and 27 percent at the 
baseline. Similar improvements are seen in other areas such as adjusting practice 
based on the assessed need of students and setting clear objectives. 
42
13
4
53
10
2
31
12
2
36
41
23
39
17
48
40
20
18
42
59
47
44
63
18
37
58
4 4
14 8
18
3
11
20
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
BL ML Final BL ML Final BL ML Final
Evidence of lesson plans Teacher explores prior
learning
Teacher uses a variety of
teaching methods
Not Started Beginning Proficient Excellent
40
Figure 14. Area 2 Teacher uses a variety of pupil assessment techniques
The final practice area (Area 3) observed was how well the teacher engaged the pupils 
during the lesson. Here again, we see marked improvement in proficiency in these 
skills over time. 86 percent of teachers were reported to be proficient in getting pupils 
to participate and stay engaged in the final evaluation, compared to 66 percent at 
midline and only 37 percent in the baseline (Figure 15). Similarly, 78 percent of 
teachers got pupils to read, write, and speak during the lesson as compared to only 36 
percent in the baseline. 
25
4 1
28
4 1
39
13
4
48
41
14
52
38
20
32
34
17
23
46
57
19
49
54
21
37
56
4 8
27
2 8
25
8 15 24
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
BL ML Final BL ML Final BL ML Final
Checks for understanding Adjust practice when
needed
Lesson objectives are
clear
Not Started Beginning Proficient Excellent
41
Figure 15. Area 3 Teacher engages pupils throughout the lesson
Across all 8 teacher behaviors highlighted above, at the final evaluation, only 4 percent 
of teachers did not show proficiency in any area, and 46 percent of teachers were 
proficient in all eight teaching areas assessed (Figure 16). This is in sharp contrast to 
the baseline evaluation where 40 percent were not proficient in any areas, and only 8 
percent were proficient in all areas. Whilst the project did not meet the LOP target of 
100 percent of teachers demonstrating the use of new and quality teaching technique, 
96 percent is very close and quite remarkable. 
16
2 1
26
2 1
46
33
13
37
33
21
32
58
56
28
58
57
5 8
30
8 8
21
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
BL ML Final BL ML Final
Pupil's participation and
 attentiveness
Pupils read, write and speak during
lesson
Not Started Beginning Proficient Excellent
42
Figure 16. Teacher proficiency across all 8 teaching practice areas
Teacher Motivation
To better understand what inspires teachers in their jobs, we asked teachers what 
aspects of their job they found motivating. When presented with the job aspect, 
teachers had to rank it from 1 to 5, with 1 being NOT motivating to 5 being highly 
motivating. Figure 17 reveals that most teachers are highly motivated by the 
interpersonal relationships they have with colleagues, head teachers, and students. 
The most demotivating aspects of their jobs relate to salaries and benefits, MEST 
policies, job security and working conditions. The latter is not surprising since most
schools are not approved, and therefore teachers are not receiving salaries and/or 
benefits from MEST. 
40
14
9 7 5 6 8
3
8
4 3 4
7 7 9 12 9
46
0
10
20
30
40
50
60
70
80
90
100
0 1 2 3 4 5 6 7 8
Percent (%) of teachers
Number of areas of teaching proficiency
Baseline Final
43
Figure 17. Aspects of their jobs that teachers find “motivating” or “highly motivating”
MGD 1.1.5 Increase knowledge and skills of administrators
Head teachers are school administrators who manage the daily operation of the school 
and serve as a link between the school and community. The MGD III project trained the 
head teacher and deputy head teacher from each of the 192 schools on DTM, 
instructional leadership, teacher management, and the management of school records. 
All head teachers surveyed (100 percent) reported attendance of training provided by 
MGD III. In terms of the benefits perceived, most reported that the training helped 
them improve their teaching skills and their supervision and support of other teachers. 
Other benefits mentioned included improvements in school management, lesson 
planning, teacher observation and supervision, record keeping, and building 
relationships with teachers. As discussed above, 95 percent of teachers reported that 
they received a supervisory classroom visit from their head teacher during the school 
year, and 92 percent of these reported more than two visits. Not only have head 
23
38
39
45
49
59
60
63
69
71
71
71
72
74
77
80
82
83
0 10 20 30 40 50 60 70 80 90 100
Salary and benefit
MEST policies
Job security
Working conditions
Status of teaching profession
Work of teaching
Personal life
Sense of achievement
Potential for professional Growth
Potential for advancement
Responsibility
Sense of accountability
Evaluation
Recognition
Relationships with students
Supervision by superiors
Relationship with head teacher
Relationships with other teachers
Percent of teachers (%)
44
teachers increased their knowledge, but they are also putting that knowledge into 
practice. The MGD III tracks, as a key indicator, the number of administrators who 
demonstrate at least one new administration skill. Monitoring data suggests that this 
LOP target of 192 was surpassed by the end of the project. 
MGD 1.2 Improve pupil attentiveness
Attentiveness was defined as pupils asking questions, actively participating in lessons, 
and following instructions without distraction. Enumerators assessed the level of 
student attentiveness and categorized classrooms based on their observation: little 
evidence (less than a third of students were attentive), moderate evidence (about half 
the class was attentive), or extensive evidence (more than a half of the class was 
attentive). Pupils observed during the final evaluation were more attentive than in 
previous evaluations. For example, 54 percent of classrooms showed extensive 
evidence of students asking questions or seeking help compared to only 6 percent at 
midline and 13 percent at baseline. The pattern is similar to the other areas studied. 
That said, in about 40 percent of classrooms, there was still little evidence of attentive
students (in terms of asking questions, participating in lessons, or working without 
distraction); instead, students mostly followed the teacher’s instructions. 
On average, across all behaviors, 64 percent of classrooms in the final evaluation 
showed moderate or extensive evidence of student attentiveness, and this is reported 
in the key indicator table. This is improved from 34 percent at midline and 42 percent 
at baseline. The key indicator of student attentiveness, with LOP of 80 percent, is the 
percent of pupils in targeted schools who are identified as attentive or very attentive. 
The measure used in the evaluation uses the classroom and not the student as the unit 
of analysis.
45
Figure 18. Percentage of classrooms that had (little/moderate/extensive) evidence of student 
attentive behaviors
MGD 1.2.1 Reduced short-term hunger
All schools were serving meals to children daily. At the time of the final survey, 98 
percent of children reported that they had eaten a meal in school, compared to 89 
percent at midline (Figure 19). 92 percent of kids reported that they were not at all 
hungry, again an improvement from the midline. The 8 percent of learners who 
reported being somewhat hungry or very hungry were either fasting or said they did 
not receive enough food. On this indicator, the project met and surpassed its LOP 
target, which was 10 percent of children indicating they were hungry or very hungry. 
40
60 59
41
25
46 43
32 33
20 20
31
5
34
28
12
51
31
9
47 47
30
56
50
54
6 13
48
25 24
48
20 19
50
23 19
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Final Mid Base Final Mid Base Final Mid Base Final Mid Base
Ask questions or seek
help
Participate in the
lesson
Listen and work
without Distraction
Follow instructions
without distraction
Little Evidence Moderate Evidence Extensive Evidence
46
Figure 19 Hunger status of children
Schools varied in serving meals with 33 percent of children reporting meal times in the 
morning (before 11:30 a.m.) and 67 percent reported having meals in the afternoon. 
MGD 1.3 Improve pupil attendance
According to monitoring data from CRS, pupil attendance has increased significantly 
from the baseline. In 2016, about two-thirds of students were reported to be attending 
schools regularly - defined as students whose attendance rate is above 80 percent. A 
year later, 100 percent of students were reported to be attending school regularly. At 
the final evaluation, 89 percent of students were deemed to be regularly attending 
school, which is less than the midline, but much better than in 2016. During the day of 
the survey, the attendance registers showed that attendance rate at 86 percent, and 
there was no difference between attendance rate for boys and girls. The attendance on
the day of survey (86%) is what we would expect from the monitoring data, which 
suggests that 89 percent of students attended school regularly. 
MGD 1.3.3 Improved school infrastructure
89
72
28
0
98 98
92
5
2
98
0
10
20
30
40
50
60
70
80
90
100
Ate meal in
school
Not at all
hungry
Somewhat
hungry
Very hungry Given food
every day
Percent of chidren (%)
Midline
Final
47
According to project monitoring reports, the MGD III project has constructed or 
rehabilitated up to 182 school facilities, including 69 latrines, 37 food stores, 24 
boreholes, and 52 classroom blocks. They have done this with the support of 
communities who contributed labor and materials to the construction totaling almost 
$140,000. The MGD III project has met its target in terms of construction of latrines 
(69 of 62), food stores (37 of 37), and classroom blocks (52 of 52). Only in the 
provision of a safe water source did the project not meet its indicator (24 of 30 
completed)
The final evaluation assessed the status of school infrastructure in the visited schools. 
Figure 20 shows that the school infrastructure is solid – 88 percent had concrete 
floors, 95 percent had concrete or mud polished wall, and 92 percent had a corrugated 
metal roof. 
Figure 20. Status of School Infrastructure
Despite progress in the provision of latrines and boreholes, there are still many 
schools without access to water, sanitation, and hygiene facilities. Only 53 percent of 
8
92
3
18
77
0
12
88
3
97
1
15
75
8
17
83
0 20 40 60 80 100
Other
No roof
Corrugated metal
Other
Mud polished
Concrete polished
Mud unpolished
Other
Earth
Concrete
Roof Wall Floor
Percent (%) of Schools
Baseline
Final
48
schools had a (potentially) safe source of water within the school compound, and for 
most, it was a hand pump well. Furthermore, of those with water facilities in the 
school, 41 percent were not functioning (Figure 21). In most cases, the water had dried 
up or the hand pump was broken. 
Latrine facilities have improved, with 95 percent of observed schools having a latrine, 
with separate ones for boys and girls. All latrines had a hand washing facility in or 
close by, and 88 percent of schools had soap and water at those hand washing 
facilities. 
Figure 21. Status of water and toilet facilities in observed schools
MGD 1.3.4 Increased pupil enrollment 
Pupil enrollment is tracked via project monitoring. Trends in enrolment over the course 
of the MGD III project is shown in Table 7. As of June 2018, there were 32,053 children 
enrolled in MGD III-supported schools, of which 49 percent were girls. The enrolment 
decreased slightly (-1.4%) since its highest point at the midline. The project staff 
attributes the reduction in enrolment to insecurities associated with the 2018 
presidential and parliamentary elections, which took place between March and April 
2018. It could also be due to normal transferring of children and families to other 
52
61 65
72
50
63
72
77
47
53
59
95
72
100
88
0
10
20
30
40
50
60
70
80
90
100
Safe water
in
compound
Working
water
source
Latrine Latrine
Clean
Hand
Washing
Facility
Soap and
Water
Percent(%) of schools
Baseline Midline Final
49
areas. The project has met and surpassed its LOP target (15,186 girls; 15,733 boys) for 
student enrolment.
Table 7 Trends in Enrollment, 2016-2018
2016 2017 2018
Female 13,715 (48%) 15,797 (49%) 15,564 (47%)
Male 14,748 16,702 16,489
Total 28,463 32,499 32,053
Factors affecting reading performance
Overall, we can conclude that the MGD III has achieved its strategic objective on 
improved literacy. The data show remarkable increases in the percentages of children 
who can read, and the project targets were exceeded. Logistic regression analysis was 
conducted to understand factors that contributed to whether a child can read based on 
individual (gender, age, member of reading club), school characteristics (whether the 
school is approved, and percent of teachers qualified), and chiefdom characteristics. 
The result is presented in Table 8. 
Table 8. Factors that predict whether a student can read and understand grade-level text
Logistic regression Number of obs 951
LR chi2(10) 238.23
Prob > chi2 0.000
Log likelihood = -494.82326 Pseudo R2 0.194
Odds Ratio Std. Err. Z P>z [95% Conf. 
Interval]
Male* 1.554 0.243 2.82 0.005 1.144 2.111
Class 3* 1.564 0.261 2.68 0.007 1.127 2.169
Age 6-11 yrs 0.954 0.231 -0.19 0.846 0.593 1.534
Member of Reading Club* 3.179 0.574 6.4 0.000 2.231 4.529
School is approved by MEST 0.828 0.157 -1 0.319 0.571 1.200
Percent of qualified Teachers 1.402 0.488 0.97 0.332 0.709 2.774
Chiefdom (base: Neini)
Mongo* 1.815 0.400 2.71 0.007 1.179 2.795
Dembelia Sinkunia* 7.094 2.112 6.58 0.000 3.958 12.714
Neya* 2.252 0.481 3.8 0.000 1.482 3.424
Sulima* 18.405 8.238 6.51 0.000 7.655 44.249
50
_cons 0.268 0.087 -4.06 0.000 0.142 0.506
Notes * Statistically significant at the .05 level. N= 951; Adjusted R2 = 0.194
The logistic analysis presents the odds ratios, and we can derive the following from the 
table:
• The odds that a student can read is predicted by their gender, class level, 
membership in a reading club, and the chiefdom in which they live. 
• The odds of being able to read is 1.6 times more for a boy than a girl
• The odds of being able to read is 1.6 times more for a pupil in class 3 than in 
class 2
• The odds of being able to read is 3.2 times more for members of reading clubs 
than non-members
• A child attending school in Neini is the least likely to be able to read and Sulima 
is the most likely. Compared to students attending school in Neini, children in 
Mongo are 1.8 times more likely to read; Neya 2.2 times more; Dembelia 
Sinkunia 7.1 times more; and Sulima 18.4 times more. 
While the above analysis does not provide evidence of causation, the model is robust 
enough to inform our understanding of what predicts that a child in the MGD targeted 
schools will be able to read. One area that could use further exploration is trying to 
understand the differences between chiefdoms in terms of progress made by pupils.
From discussions with field staff, differential efforts by literacy coaches in supporting 
teachers to improve reading instruction may play a role in the differences amongst 
schools and chiefdoms. 
MGD S.O.2. Increased use of good health and dietary practices
Under this strategic objective, MGD III trained the school communities on good 
practices related to health and dietary practices. Students and teachers were trained on 
good health and hygiene practices, and cooks were trained on techniques for safe food 
preparation and storage. The project was also to provide better access to clean water 
through the development of wells. Finally, the project collaborated with partners such 
as Helen Keller International and UNICEF in the provision of deworming services to 
schools. Below, we consider what effect these activities had in the increasing use of 
good health and hygiene practices
51
MGD 2.1 Improved knowledge of health and hygiene practices
Pupils were asked about their knowledge of good health and hygiene, and their 
responses are shown in Figure 22. The most mentioned practices at the final 
evaluation were: washing hands with soap and water before eating (76 percent) and 
washing hands with soap and water after using the toilet (75 percent). The least 
mentioned practices were keeping nails clean (no mention); putting trash into bins (22
percent) and keeping toilets clean (22 percent). Pupils fared better in the final than in 
the midline evaluation: at midline, 12 percent of pupils could name 6 of 10 practices, 
compared to 20 percent of pupils at final. However, 20 percent is still a very low 
percentage, especially since this question was about knowledge and not about 
behavior. 
Figure 22 Pupils’ knowledge of good health and hygiene
0
22
22
23
23
38
45
46
64
68
75
76
21
10
15
10
25
37
28
23
69
49
64
56
0 10 20 30 40 50 60 70 80 90 100
Cut and keep nails clean
Put trash into a bin
Keep toilets clean and provided with soap/ash
Hair braiding
Eat good food
Wear clean clothes
Brush teeth twice a day
Drink safe, treated water
Wash our bodies daily
Keep environment clean and safe
Wash hands with soap and water after using toilet
Wash hands with soap and water before eating
% of pupils 
Midline Final
52
MGD 2.2. Increased knowledge of safe food preparation and storage practices
The team surveyed cooks to determine their knowledge and skills in safe food 
preparation and storage practices. A total of 163 cooks were interviewed, and of those, 
99 percent (all but 2) reported having been trained on safe food preparation and 98 
percent (all but 4) were trained on food storage practices and child health and 
nutrition. Those who had not been trained were new cooks. Those who attended the 
training reported having benefitted from the training in several ways, including that it 
made them better cooks and that it helped them improve hygiene practices. 
Cooks were asked to list good practices in safe food preparation and storage practices, 
and these were checked against the main messages delivered in the training delivered 
by CRS. Their responses are shown in Figure 23. The most commonly mentioned 
practices were: washing hands with clean water and soap (90 percent), washing food 
before cooking (87 percent), washing utensils and dishes with soap and water and 
drying them (86 percent). The least mentioned practices were: kitchen should be free 
from animals (19 percent); using tongs or spoons (and not bare hands) to handle 
cooked food (49 percent) and wearing of kitchen aprons (58 percent). Cooks’ 
knowledge of good hygiene practices show marked improvement at final as 82 percent
of cooks could recall at least 5 of 10 practices, compared to 46 percent at midline, and 
60 percent at the baseline. The LOP target of 75 percent was achieved by the project. 
53
Figure 23. Cook’s knowledge of safe food preparation and storage practices
When asked about challenges encountered as a cook, the main challenges identified 
were: lack of salary, lack of utensils, lack of condiments, and lack of water. Some cooks 
also mentioned having to fetch firewood, too much smoke in the kitchen, and being 
disparaged by other members of the community as challenges. 
The status of kitchens had improved since the midline evaluation (see Figure 24). 
Almost all schools were equipped with a kitchen that had shelves, racks, handwashing 
facilities, aprons, and soap and water. 
19
49
58
60
61
61
62
85
86
87
90
43
21
46
61
64
46
79
79
77
79
0 10 20 30 40 50 60 70 80 90 100
Kitchen should be free from animals
Use tongs or spoons to handle cooked food
Wear kitchen apron or apparel
Put cleaned utensils on a platform
Store the food in a clean place
Cover the food after dishing
Storage should be free from flies
Sweep the kitchen or where food is prepared
Wash utensils and dishes with clean water
and soap and dry them
Wash the food items before cooking
Wash hands with clean water and soap
Percent (%) of Cooks Interviewed
Baseline Midline Final
54
Figure 24. Status of Kitchens in Schools
All schools had a storage facility: 90 percent had ventilated blocks with mesh, 95 
percent had a metal door with locks, and 95 percent had a concrete wall (Figure 25). 
The status of storage facilities had improved greatly over the year since the midline 
survey.
97 97 97 97 98
91 90
39
74
37
87
35
0
10
20
30
40
50
60
70
80
90
100
Has Kitchen Shelves Rack/Pallet Handwashing
facility
Aprons
available
Soap
available and
seen
Percent (%) of Kitchens
Final Midline
55
Figure 25. Status of Storage Facilities
MGD 2.3 Increased knowledge of nutrition
Pupils were not asked about their knowledge of nutrition, but they were asked about 
the types of foods they ate on the day before the survey. As Figure 26 shows, almost 
all pupils reported eating grains (rice, cassava, etc.); 71 percent ate fruits, and 69
percent ate vegetables. Sixty percent of pupils reported eating a minimum acceptable 
diet that included at least 4 of the 7 identified food groups, which is the highest across 
all the evaluation. 
100
90
95 95
90
20
72
0
10
20
30
40
50
60
70
80
90
100
Has Store
Room
Ventilated
Blocks
with
mesh
Metal
door with
locks
Concrete
Wall
Percent (%) of kitchens
Final Midline
56
Figure 26. Food groups contained in pupils’ diet on the day prior to the survey
MGD 2.4 Increased access to clean water and sanitation services 
As mentioned above in Section MGD 1.3.3, only 53 percent of schools have access to a 
safe water source within the school compound, and of those, only 59 percent were 
working. This means less than a third of schools had access to a working source of 
safe water within the schools. 
Access to latrines have improved with 95 percent of schools having a working latrine, 
and the project surpassed its LOP target for the number of latrines built. Availability of 
soap for handwashing is much better as 91 percent of kitchens were observed to have 
soap and water available compared to only 13 percent at midline; in addition, 88 
percent of latrines had soap and water available compared to only 18 percent at 
midline. 
9
35
52
59
69
71
95
4
16
29
31
54
57
94
5
23
44
71
69
54
99
0 20 40 60 80 100
Eggs
Dairy
Legumes and Nuts
Meat or Fish
Vegetables
Fruits
Grains
% of pupils
Baseline
Midline
Final
57
MGD 2.5. Increased access to preventative health services
The GOSL, with support from its donor partners, provide preventative health services, 
including deworming and preventative health services to school children. The Ministry 
of Health and Sanitation carries out the deworming exercise twice in a school year. CRS 
gives collaborative support to the process in terms of sensitization to communities and 
logistical support to enable medical staff to reach MGD-supported schools. About half 
of schools received including deworming services during the 2017/18 school year. 
MGD 2.6. Increased access to requisite food preparation and storage tools and equipment
Table 9 shows the availability of food preparation and serving utensils across the 
various schools. There are large differences amongst schools in terms of the 
quantities of kitchen equipment and serving utensils available. For example, while 
some schools had no spoons, another had 930. Per project staff, CRS does not provide 
utensils and it is up to the communities to provide the necessary utensils. Clearly, 
some communities struggle with this, and during interviews with cooks, the lack of 
utensils and kitchen equipment came up as one of the challenges faced. 
Table 9 Availability of kitchen equipment and Utensils
Utensils
Final Midline
Avg. Min Max Avg. Min Max
No. Big Pots 2 1 6 2 0 3
Big Bowls for cooked food 1 0 8 1 0 4
Big bowls for sauce 1 0 6 0 0 8
Wooden spoon 4 0 50 2 0 30
Scooping spoon 3 0 6 3 0 55
Serving Plates 41 0 343 29 0 250
Spoons 60 0 930 25 0 250
Buckets 3 0 8 2 0 10
Towels 3 0 8 2 0 6
Cups 34 0 500 24 0 250
Knives 1 0 4 0 0 3
Mortar 1 0 2 0 0 1
Mortar pestle 1 0 4 0 0 2
In some schools, students had to share utensils or take turns eating because there 
were not enough utensils to go around. This slowed down the lunch process and 
caused some children not to have enough food. 
58
Progress on Key Performance Indicators
As part of the project monitoring of outputs and outcomes, the MGD III project has 
selected several indicators that will be used to monitor progress. During the baseline 
survey in 2016, information was collected on the starting point of several of these 
indicators and these are shown in Table 10. The values of these indicators were 
updated in June 2018 with data from CRS monitoring and the final evaluation. 
Significant progress has been made on many of the key indicators, especially on the 
literacy measures. The percentage of children who can read and understand grade￾level text quadrupled since the baseline, and the project has exceeded its ‘Life of 
Project’ (LOP) targets. 
59
Table 10. Key Performance Indicators: Midline, Baseline and End of Project Targets
Indicator
Baseline 
(2016)
Midline 
(2017)
Final 
(2018)
Life of Project 
(LOP) Target Comments
Girls Boys Girls Boys Girls Boys Girls Boys
Percent of pupils who, by the 
end of two grades of 
schooling, demonstrate that 
they can read and understand 
the meaning of grade-level text 
(girls/boys)
8.3% 7.9% 32% 39% 56% 61% 25% 25%
Percent of class 2 pupils 
who scored at least 80 
percent in a reading 
comprehension test
Percent of pupils in target 
schools who are identified as 
attentive or very attentive 
during class/instruction 
(girls/boys)
42% 34% 64% 80%
This is the percent of 
classrooms in which pupils 
were attentive or 
moderately attentive. 
Updated baseline and 
midline figures
Percent of pupils in target 
schools who indicate that they 
are hungry or very hungry 
during the school days 
(boys/girls)
92% 95% 26% 30% 8% 8% 10% 10%
This question was only 
asked from pupils who had 
been provided a meal
Percent of school-age children 
receiving a minimum 
acceptable diet (boys/girls)
55% 52% 28% 27% 63% 58% 100% 100%
Percent of school-aged 
children who report 
consuming at least 4 out of 
7 food groups on the day 
prior to the survey 
Percent of pupils in target 
schools who achieve a passing 
score on a test of good health 
and hygiene practices 
(boys/girls) – 6 out of 10 
practices
55% 44% 11% 13% 19% 21% 70% 70%
Percent of pupils who could 
name at least 6 of 10 good 
hygiene practices. 
Percent of food preparers at 
target schools who achieve a 
passing score on a test of safe 
food preparation and storage 
71% 46% 82% 75%
Percent of cooks who could 
name at least 5 of 10 safe 
food preparation practices
60
Indicator
Baseline 
(2016)
Midline 
(2017)
Final 
(2018)
Life of Project 
(LOP) Target Comments
Girls Boys Girls Boys Girls Boys Girls Boys
Percent of participants 
demonstrating at least 3 out of 
5 social/life skills
 78% 97% 95% CRS Indicator Performance 
Tracking Table
Percent of teachers in target 
schools who demonstrate the 
use of new and quality 
teaching techniques or tools 
because of USDA assistance 
60% 79% 96% 100%
Percent of teachers that 
demonstrated proficiency 
in at least one new 
teaching technique 
Percent of pupils in target 
schools who attend school at 
least 80% of scheduled school 
days per school year
67% 66% 96% 96% 89% 89% 100% 100% CRS Indicator Performance 
Tracking Table
Number of teachers in target 
schools who demonstrate the 
use of new and quality 
teaching techniques or tools
0 701 975 701 CRS Indicator Performance 
Tracking Table
Number of teachers trained or 
certified as a result of USDA 
assistance (MGD I-6)
0 296 530 450 CRS Indicator Performance 
Tracking Table
Number of school 
administrators (head teachers) 
in target schools who 
demonstrate the use of new 
teaching technique or tool 
0 192 384 192 CRS Indicator Performance 
Tracking Table
Number of SILC groups sharing 
out and starting a new cycle 0 47 58 75 CRS Indicator Performance 
Tracking Table
Percent of Parent-Teacher 
Associations or similar “school” 
governance structures 
contributing to their school as 
a result of USDA assistance
 98% 100% 100% CRS Indicator Performance 
Tracking Table
Number of WASH Clubs formed 0 55 192 192 CRS Indicator Performance 
Tracking Table
61
Indicator
Baseline 
(2016)
Midline 
(2017)
Final 
(2018)
Life of Project 
(LOP) Target Comments
Girls Boys Girls Boys Girls Boys Girls Boys
Number of pupils receiving 
deworming medication(s) 28,463 32,042 30,919 CRS Indicator Performance 
Tracking Table
Number of schools using an 
improved water source (MGD I￾19)
0 0 24 30 CRS Indicator Performance 
Tracking
Number of latrines constructed 
or rehabilitated with USDA 
assistance (MGD Indicator 7)
0 52 69 62
CRS Indicator Performance 
Tracking
Number of kitchens or 
storerooms constructed or 
rehabilitated with USDA 
assistance (MGD Indicator 7)
0 34 37 37 CRS Indicator Performance 
Tracking Table 
Number of individuals 
benefiting directly from USDA 
funded intervention 
(male/female)
35,008 39,105 39,128 37,766 CRS Indicator Performance 
Tracking Table
Number of individuals 
benefiting indirectly from 
USDA funded intervention 
(MGD Indicator 26)
42,476 49,011 49,018 40,473 CRS Indicator Performance 
Tracking Table
Number of social assistance 
beneficiaries participating in 
productive safety nets with 
USDA assistance (male/female) 
(MGD I-17)
35,310 39,076 35,350 37,737 CRS Indicator Performance 
Tracking Table
Number of daily school meals 
provided to school-age 
children because of USDA 
assistance (MGD I-15)
0 3,883,887 8,164,834
(cumulative) 12,585,158 CRS Indicator Performance 
Tracking Table
Number of pupils regularly 
(80%) attending USDA 
supported school 
(male/female)
9,169 9,776 15,647 16,541 13,136 14,063 15,156 15,686 CRS Indicator Performance 
Tracking Table
62
Indicator
Baseline 
(2016)
Midline 
(2017)
Final 
(2018)
Life of Project 
(LOP) Target Comments
Girls Boys Girls Boys Girls Boys Girls Boys
Number of textbooks and 
other teaching and learning 
materials provided because of 
USDA assistance
0 181,317 471,235 290,000 CRS Indicator Performance 
Tracking Table
Number of schools with 
improved early grade literacy 
instructional materials
0 192 192 192 CRS Indicator Performance 
Tracking Table
Number of community 
meetings held (CRS I-33) 0 57 218 52 CRS Indicator Performance 
Tracking Table
Number of literacy coaches 
trained in Diagnostic Teaching 
Methods (CRS I-6)
0 25 25 25 CRS Indicator Performance 
Tracking Table
Number of teachers trained in 
Diagnostic Teaching Methods 
because of USDA assistance 
(male/female) (CRS I-7)
0 701 975 701 CRS Indicator Performance 
Tracking Table
Number of school buildings or 
classrooms constructed or 
rehabilitated because of USDA 
assistance. (MGD I-7)
0 19 52 52 CRS Indicator Performance 
Tracking Table
Number of classrooms in 
project schools receiving new 
school furniture (CRS I-18)
0 145 241 240 CRS Indicator Performance 
Tracking Table
Number of pupils enrolled in 
school receiving USDA 
assistance (male/ female)
13,715 14,748 15,797 16,702 15,564 16,489 15,186 15,733 CRS Indicator Performance 
Tracking Table
Note: Green-shaded cells show that Life of Project targets was exceeded
63
RELEVANCE: TO WHAT EXTENT DOES THE PROJECT CONFORM TO THE 
NEEDS AND PRIORITIES OF TARGET GROUPS AND THE POLICIES OF THE 
COUNTRY AND DONOR?
The MGD III project, to a very large extent, conforms to the needs and priorities of the 
country, communities and USDA. At the policy level, providing school meals for all 
children, improving WASH facilities, and improving learning are all priorities of the 
GOSL as outlined in the Education Sector Plan (ESP) 2018-2020. While there has been a 
change in government and administration since the ESP was prepared, we have no 
reason to believe that the new government will abandon these policy priorities. 
Are the activities and outputs of the project consistent with the overall goals and the 
attainment of its objective?
The MGD III project conforms to the objectives of the donor, USDA, whose McGovern￾Dole program seeks to reduce hunger and promote literacy and primary education. 
This evaluation has shown that the MGD III goals and objectives have been achieved in 
the targeted schools in Koinadugu – more children are reading, staying in school, and 
have reduced hunger.
Do the program meet community and government expectations? 
The interviews and focus group discussions at the community and schools also confirm 
that the MGD III project meets the needs of the communities. Community members 
mentioned that they were involved in the discussions about the design of the project, 
and that they are frequently consulted about the project activities. They believe that 
the MGD III project tries to address the educational needs of the communities such as 
the need for qualified teachers, safe drinking water, safe learning environments, 
ensuring their children are learning, and encouraging regular attendance of teachers. 
Although some community members feel that CRS should be doing more, the current 
activities do conform the needs and priorities of communities.
Are stakeholders (school management committee, parents, teachers, local authorities) 
satisfied with their participation in the program? Why or why not?
Various stakeholders were largely satisfied with the program. CRS field staff are 
satisfied with the program because they can see the impact it is making in the lives of 
64
children and communities. Learners are very satisfied with the meals they get during 
school hours. 
Teachers expressed high-level of satisfaction with all aspects of the MGD III project, 
but they were most pleased with the school meals, reading clubs, provision of TLMs, 
and the teacher training programs. When they expressed dissatisfaction, it was 
because certain aspects of the program were not available in their schools or 
communities such as SILC or the life skills program or for aspects that were beyond the 
project’s control such as the fact that they do not receive salaries. 
Head teachers also expressed satisfaction with the MGD III project especially with the 
areas related to advocacy, provision of TLMs, in-service training, and school meals. 
They were most likely to express dissatisfaction with the construction activities and the 
provision of school furniture because they needed more furniture to meet the growing 
enrolments and WASH facilities had not been upgraded. 
Parents and communities also expressed high levels of satisfaction with their 
participation in the MGD III program. They appreciated the fact that their children were 
fed and that they were learning to read and write. Because the needs of communities 
are large, many communities would like the MGD III to do more – provide more school 
buildings, WASH facilities, provide condiments, etc. 
How well does the project complement and link to activities of other donors at the local 
level?
The MGD III activities were the largest program active in these chiefdoms in Koinadugu. 
Nonetheless, the project did coordinate with other partners by attending education 
sector coordination meetings at the national and district level with other partners 
supporting education. 
EFFICIENCY: HOW COULD INTEGRATION OF THE TWO STRATEGIC 
OBJECTIVES BE ENHANCED OR FURTHER LEVERAGED TO DEEPEN 
POSITIVE PROGRAM RESULTS? 
65
How well have the two strategic objectives been integrated to deepen positive program 
results?
Schools play a critical role in promoting health and safety of young children and 
helping them to establish good health practices for a lifetime. Integrating health and 
education program within schools is an effective and efficient way to improve the 
overall well-being of children in the community. The Ebola crisis showed that whole 
communities could be at risk from poor health and hygiene practices. Children who are 
healthy are more prepared to learn as they attend school more regularly. 
The MGD III project supports the healthy development of children in many ways 
including: providing nutritional meals, improving on WASH facilities, coordinating with 
other programs that provide deworming, and training pupils and teachers on good 
WASH and nutrition practices. 
The health outcomes, while showing some improvement, did not meet the expected 
targets. For example, only 20 percent of students could name at least 6 good hygiene 
practices. On the other hand, food preparers are more knowledgeable of safe practices 
for handling food, although many claims to lack the wherewithal (utensils, water, etc.) 
to put into practice. 
SUSTAINABILITY: TO WHAT EXTENT WILL THE BENEFITS OF THE 
PROJECT CONTINUE AFTER THE END OF THE PROJECT LIFE? 
What activities and/or outcomes (both expected and unexpected) of the program are 
likely to be sustained? What evidence is there to suggest this?
As mentioned earlier in the report, the new government has made the provision of 
free, quality basic education its flagship project starting September 2018. While the 
details of the implementation are unclear, approved government and government￾assisted schools will receive support to ensure that tuition is free including free 
textbooks in the core subjects. The GOSL will also implement a national school feeding 
program, initially targeting vulnerable communities in every district. CRS has been 
working with the MEST in developing the national school feeding implementation 
66
protocol. Approved government-assisted schools will benefit from some of these 
interventions. 
If teachers trained by MGD III stay in the schools, then they are likely to continue with 
improved teaching practices. However, teachers who have been trained and certified 
are unlikely to stay in a school if they don’t receive a salary. There is a risk that 
teachers will leave for other schools where they can be on the payroll. 
When asked about what activities they might be able to continue after the project 
comes to an end, communities highlighted the following: 
• continue to enforce bye-laws around school attendance
• SILC activities 
• School gardens
• Activities of the mothers’ groups
What are the major factors that can influence the achievement or nonachievement of the 
sustainability of the project?
One of the major factors that will influence the achievement or nonachievement of 
sustainability is whether the school will be approved by the government to receive 
government assistance. While communities have expressed interest in continuing to 
support the schools, additional resources from the government will be needed. Schools 
that are government-assisted are more likely to receive salaries for teachers, TLMs, 
professional development for teachers, and school meals. 
It will also be important that MEST at the district level have the capacity to undertake 
their roles and responsibilities, which include monitoring and supervising schools and 
teachers. For now, they are only able to visit some of these schools with support from 
MGD III. Literacy coaches have been instrumental in providing much-needed support to 
teachers and with the cessation of the project, MEST officials and head teachers will 
need to be capacitated to provide this support. Head teachers or deputy heads should 
be trained on providing coaching support, and for MEST officials, they will need 
support in also developing the logistical capacity to get to schools. 
67
Finally, CRS will have to develop a solid exit strategy that helps communities off MGD 
III support onto government support. After so many years of MGD III support, 
communities have come to rely on the project for supporting education. During this 
fourth year, the project must work actively with communities to seek other sources of 
support. 
How do the government’s capacities, policies, procedures, and priorities contribute to 
sustainability?
As mentioned, the GOSL has committed to providing free, quality education to all
students in government and government-assisted schools. These policies are aimed to 
remove barriers to accessing education, especially for the most marginalized. The 
government’s plan, while not fully funded, is being supported by donors and external 
partners. The GOSL also will be implementing a national school feeding program, and 
CRS with support from the MGD III has been instrumental in helping them develop their 
strategy. The successful implementation of these policies will contribute towards 
sustaining some of the activities and outcomes produced by this project
What strategies should be used to obtain long-lasting support from communities and
local/central administration that goes beyond the time of the project
The ongoing support provided by coaches have been key to realizing the improved 
outcomes. The literacy coaches are employees of CRS, and if the program ends, their 
support will cease. The likely officials to do the work of the coaches are the MEST 
supervisors and inspectors; however, they are currently not able to provide the level of 
support the CRS coaches provide, due to lack of resources. The MGD III project should 
work closely with MEST at district level on planning to take on this task. This means 
not only providing technical support but working with them on budgeting and 
prioritizing their work to ensure that this support continues. The MEST already see the 
benefits of the coaching support to teachers. At the school level, head teachers can 
also provide peer-based coaching to teachers. 
During the FGDs, the community members were optimistic that they would be able to 
continue providing some of the benefits that the MGD III program provided. For 
example, they suggested that they would develop school farms or gardens, enforce 
68
bye-laws to ensure parents send their children to school, mothers clubs will continue 
to monitor attendance, and parents can continue to reinforce good hygiene practices. 
However, it should be noted that even now, communities struggle to provide even 
condiments and serving utensils to the schools. 
As mentioned in previous reports, obtaining government approval of schools is the 
most important strategy for long-term success. The project has made significant 
progress in getting schools to apply for approval, but only 32 percent of the schools in 
this sample were approved. Of those that were not approved, 41 percent had applied 
for approval and 59 percent had not. Their reasons for not applying included: not 
knowing or understanding the procedures, the head teacher being new to the school, 
the school not meeting the eligibility criteria such as having trained teachers and 
waiting on someone from the Ministry to inspect the school. 
Of the schools who did apply for approval, 71% (or 10) applied in 2017, reflecting that 
CRS did help schools with their approval process. However, more can be done 
especially for schools who said they don’t know the process and others who said they 
don’t meet the criteria. 
IMPACT: WHAT WERE THE POSITIVE OR NEGATIVE EFFECTS, BOTH 
INTENDED AND UNINTENDED, OF THE PROJECT? 
What were the intended and unintended positive and negative effects of the intervention 
on children, communities, and institutions?
The most remarkable achievement of the MGD III is that reading skills of children have 
increased, surpassing the targets that were set in terms of the percentage of children 
who could read with understanding at the end of grade 2. The project has also 
improved the teaching skills of teachers in the communities, ensured that learners and 
teachers are not hungry, provided teaching and learning materials for schools, and 
through the SILCs provided a way for parents to save and access financial resources to 
improve the well-being of their family. These have all been positive effects, for which 
all stakeholders are grateful. 
69
This project has also contributed to developing the capacity of MEST at the national 
level and the district level through the provision of technical and financial assistance. 
The secondment of a technical adviser at the School Feeding Secretariat at MEST and 
opportunities provided for MEST officials to meet with and learn from other countries’ 
experiences with school feeding have had a positive effect on the development of 
MEST’s own school feeding program. 
Negative effects of the intervention are minimal. One potential negative effect is that 
communities have become dependent on the MGD III for their support, thereby 
reducing their agency to do more for themselves. From the FGDs, communities 
continue to look to CRS to do more for their schools and to expand to even more 
areas. This means communities have also contributed significantly to the program 
through providing their labor and in-kind donations to schools in support of the school 
feeding and construction activities.
Another unintended and potentially negative effect is the fact that some children are 
denied food in school because their parents cannot afford to contribute towards the 
cost of condiments. In some communities, parents are asked to contribute up to Le 
5,000 (less than US$1) a month, and while this amount is small, it is possible that 
some parents may still not be able to afford it. School communities should ensure that 
the learners are not penalized because their parents cannot afford this monetary 
contribution. 
To what extent can identified changes be attributed to the intervention?
Because the evaluation did not use a control group or randomized control, the 
identified challenges cannot be concretely attributed to the intervention. However, 
comparing the performance of children in targeted schools over time is much better 
than the average performance on reading by learners nationally. 
CONCLUSION & 
RECOMMENDATIONS 
CONCLUSION 
70
The results from the MGD III Final Evaluation shows that the project met its first 
strategic objective of improving the literacy skills of learners in targeted schools in 
Koinadugu district. The life of project target was that 25 percent of boys and girls, by 
the end of two grades of schooling, would be able to read and understand the meaning 
of grade-level text. The final evaluation found that 56 percent of girls and 61 percent 
of boys could read grade-level text with understanding. This is remarkable progress 
given that at the baseline evaluation, conducted two years earlier, only 8 percent of 
boys and girls could read. The project met most of the intermediate results based on 
the theory of change including improved teacher attendance, better access to TLMs, 
increased skills and knowledge of teachers, and increased skills and knowledge of 
teachers. 
The activities around strategic objective 2 were also largely completed although there 
were some gaps. The project surpassed its LOP targets for the construction of latrines 
but did not meet its targets for the construction of boreholes. Many schools still do not 
have access to safe drinking water on the premises. The cooks demonstrated improved 
knowledge of safe practices for preparing and storing food, but many schools still 
don’t have the requisite utensils or equipment for preparing and storing food. 
Whilst the overall goal of increasing reading skills was met, it is impossible to say 
whether the theory of change as outlined by CRS holds. This is because it is difficult to 
isolate the effects of the various components or to know what combination of activities 
resulted in the improved performance. Also, because the evaluation did not include 
control schools, one cannot conclusively say that the changes in literacy are due to the 
MGD III intervention.
RECOMMENDATIONS 
Improved Literacy Skills 
• Gender differences in reading: there are still some gender differences in 
reading, with boys performing better than girls. The project should promote 
gender equality by ensuring that there is no discrimination in terms of gender in 
teaching practices, curriculum materials, or school culture. Gender 
71
responsiveness should be an integral part of all training. The teaching staff is 
overwhelmingly male, and some affirmative action may be needed to encourage 
more women to enter teaching as that has been shown to have positive effect on 
the learning outcome for girls. 
• Use of teaching and learning materials: in at least a third of classrooms, only 
the teacher had and used a textbook. The project team should continue to 
encourage teachers to distribute textbooks in the classrooms for children to 
use. Use of textbooks should be an integral part of teacher training and 
coaching. 
• School approval: the MGD III has made good progress in terms of getting 
schools to submit an application for approval from the GOSL. However, there are 
still too many schools that are not approved. It is critical that all schools submit 
their application, even as they work towards meeting the eligibility criteria. 
• Community engagement: engage parents and communities in support reading 
(and not just the feeding program). Parents and caregivers, who can, should 
read to their children. Those who can't, should encourage their children to read 
to them, tell them stories, or encourage their children to join a reading club. 
• Reading clubs: encourage children to join reading clubs as children who are 
members of a reading club tend to perform better. This is likely because these 
clubs provide children with an opportunity to practice their reading and 
introduce them to different kinds of reading materials. For children who cannot 
join reading clubs, they should be encouraged to take some reading materials 
home. 
Improved health and hygiene practices
• Intensify efforts to complete the construction of wells and reduce delays in the 
process. Access to safe water is needed. 
• Care should be taken while constructing wells to ensure that it is done at the 
right time of the year. Many schools had wells that had dried up at the time of 
the evaluation, which sometimes suggests that the well was dug at the wrong 
time, and not when the water table is at its lowest. 
• Work with communities to ensure that the needed cooking and eating utensils 
are provided.
• Ensure that children from the poorest families are not penalized because their 
parents cannot afford to pay the cost of condiments. 
• Better measures are needed for some of the indicators – for example, asking 
children about the foods they ate the previous day as a proxy for increasing 
knowledge of nutrition is not a very good measure as children don’t usually 
have control over what food they eat in these contexts. 
72
Other recommendations
• Develop an exit and sustainability plan with communities, local government and 
MEST
• The role of literacy coaches has proved to be instrumental in providing much￾needed support for teachers. The sustainability strategy should identify who in 
the school community (head teacher, deputy head teacher, another teacher) can 
provide peer coaching to other teachers. 
• Evaluation of any future MDG project should include an experimental or quasi￾experimental design with a control group to improve the project’s ability to 
make causal claims
• Simplify the theory of change for this intervention. The current theory of change 
is complex and difficult to test
• Continue to work with the current government to design and test the national 
school feeding program and gain commitments that they will include the MGD III 
schools in the national school feeding programs when CRS phases out. 
• Encourage cross-chiefdom learning as there are differences in performance on 
both objectives across the various chiefdoms. For example, some communities 
have strategies for ensuring utensils are provided for children and cooks, while 
others do not. This provides an opportunity for learning across chiefdoms that 
could be facilitated by CRS. 
73
REFERENCES
Montrose International. (2014). Consultancy for the Design and Conduct of National 
Early Grade Assessments in Literacy and Numeracy (EGRA/EGMA) for Primary 
School Pupils in Sierra Leone.
Statistics Sierra Leone. (2016). 2016. Freetown: Statistics Sierra Leone. Retrieved 10 21, 
2017, from https://www.statistics.sl/wp-content/uploads/2017/01/final￾results_-2015_population_and_housing_census.pdf
The Economist Intelligence Unit. (2017, October 21). Sierra Leone Food Security Index. 
Retrieved from 
http://foodsecurityindex.eiu.com/Country/Details#Sierra%20Leone
United Nations Development Programme UNDP. (2016). Human Development Report 
2016: Human Development for Everyone. New York: UNDP. Retrieved October 
21, 2017, from 
http://hdr.undp.org/sites/default/files/2016_human_development_report.pdf
74
ANNEXES 
The following annexes are included in a separate document. 
A. Scope of Work
B. Field data collection instruments
C. Responses to recommendations from the baseline survey
D. Evaluation Questions
75