August 2017
This publication was produced at the request of the United States Department of Agriculture. It was 
prepared independently on behalf of Catholic Relief Services by Bidemi Carrol and Mohamed Moigua of Converge SL.
Midline Survey Report: Phase III of the McGovern-Dole 
Food for Education Program in Sierra Leone
i
August 27, 2017
DISCLAIMER 
The author’s views expressed in this publication do not necessarily reflect the views of 
the United States Department of Agriculture or the United States Government. 
Accessibility Note: An accessible version of this report can be made available upon 
request by emailing FAS.monitoring.evaluation@usda.gov. 
Program: McGovern-Dole International Food for Education and Child Nutrition
Agreement Number: FFE-636-2015/018-00
Funding Year: Fiscal Year 2015
Project Duration: 2015-2018
Implemented by: Catholic Relief Services
Evaluation Authored by: Converge SL
Bidemi Carrol
Mohamed Moigua
iii
Midline Survey Report: Phase III of the Food for Education Program in Sierra Leone .........................1
Contents..................................................................................................................iii
Table of Figures .......................................................................................................iv
List of Tables ............................................................................................................ v
Acronyms ................................................................................................................ vi
Executive Summary...................................................................................................2
Background ...........................................................................................................2
Purpose .................................................................................................................2
Methodology..........................................................................................................2
Findings and Conclusion........................................................................................3
Recommendations .................................................................................................6
Introduction..............................................................................................................9
Country Context ....................................................................................................9
Project Background..............................................................................................11
Intervention Logic................................................................................................13
Review Purpose & Questions ...................................................................................15
Evaluation Purpose ..............................................................................................15
Review Questions.................................................................................................15
Evaluation methods.................................................................................................15
Sampling and Sample Size ...................................................................................16
Data Quality, Management, and Analysis..............................................................20
Findings..................................................................................................................21
EFFECTIVENESS: To what extent has the project has achieved (or has made 
progress towards) its objectives (or intermediate results)? ...................................21
iv
RELEVANCE: To what extent does the project conform to the needs and priorities 
of target groups and the policies of the country and donor? ................................58
EFFICIENCY: How could integration of the two strategic objectives be enhanced or 
further leveraged to deepen positive program results? .........................................58
SUSTAINABILITY: To what extent will the benefits of the project continue after the 
end of the project life? .........................................................................................59
Conclusions & Recommendations............................................................................60
Improved Literacy Skills .......................................................................................60
Improved health and hygiene practices ................................................................61
Other recommendations ......................................................................................62
Annexes .................................................................................................................63
Figure 1. Results Framework for the MGD III Program.................................................13
Figure 2. Percentage of boys and girls who can read and understand grade level text 
(Interim and Baseline).................................................................................................24
Figure 3. Mean Number of letters identified correctly (out of 26)................................25
Figure 4. Mean scores in Phonemic Awareness sub tasks (out of 10), Midline and 
Baseline .....................................................................................................................26
Figure 5. Mean number of familiar words correctly read (out of 40)............................27
Figure 6. Teacher attendance rate on the day of the survey. .......................................28
Figure 7. Availability of teaching and learning materials and supplies.........................29
Figure 8. Availability of desks and benches in observed classrooms ...........................30
Figure 9. Availability of textbooks or readers in observed classrooms ........................31
Figure 10. Supervisory visits to teachers in the month prior to the survey ..................34
Figure 11 Area 1: Teachers Demonstrate good instructional practice..........................35
Figure 12. Area 2 Teacher uses a variety of pupil assessment techniques...................37
Figure 13. Area 3 Teacher engages pupils throughout the lesson..............................38
Figure 14 Teachers’ self-assessment of knowledge and skills in various teaching 
techniques .................................................................................................................39
v
Figure 15. Aspects of their jobs that teachers find “highly motivating” .......................40
Figure 16 Percentage of pupils showing attentive behaviors in class...........................41
Figure 17 Hunger status of children ...........................................................................42
Figure 18 Status of School Infrastructure....................................................................43
Figure 19 Status of water and toilet facilities in observed schools...............................44
Figure 20 Pupils’ knowledge of good health and hygiene ...........................................47
Figure 21 Cook’s knowledge of safe food preparation and storage practices ..............48
Figure 22. Status of Kitchens in Schools .....................................................................49
Figure 23 Food groups contained in pupils’ diet on the day prior to the survey ..........50
Table 1. Number of sample pupils and schools allocated by chiefdom........................18
Table 2. Response Rates for different units of analysis ...............................................19
Table 3. Percentage (%) of pupils who answered comprehension questions by class and 
gender .......................................................................................................................23
Table 4 Summary of Teacher Characteristics ..............................................................32
Table 5 Scoring guidelines for teacher observations ...................................................34
Table 6. Results from Logistic Regression showing factors that predict whether a 
student can read and understand grade level text ......................................................45
Table 7 Availability of kitchen equipment and Utensils ...............................................51
Table 8 Key Performance Indicators: Midline, Baseline and End of Project Targets .....53
vi
CRS Catholic Relief Services
CSR Country Status Report (on Education)
CTA Community Teacher Association
DEO District Education Office
DTM Diagnostic Teaching Methods
FWR Familiar Word Reading
FFE Food for Education
GOSL Government of Sierra Leone
ILA International Literacy Association
M&E Monitoring and Evaluation
MC Mother’s Club
MEST Ministry of Education, Science and Technology
MGD McGovern-Dole
MICS Multiple Indicator Cluster Survey
NP Northern Polytechnic
PA Phonemic Awareness
PQTR Pupil Qualified Teacher Ratio
PTR Pupil-Teacher Ratio
SCR School Census Report
SILC Savings and Internal Lending Committee
SMC School Management Committee
TALLE The Association of Language and Literacy Educators
TLM Teaching and Learning Materials
USDA United States Department of Agriculture
WAEC West African Examination Council
WASSCE West African Senior Secondary Certificate Examination
WFP World Food Programme
1
2
BACKGROUND 
This report presents the findings of the midterm review of Catholic Relief Services’ 
(CRS) implementation of Phase III of the McGovern-Dole Food for Education (MGD III) in 
Sierra Leone. CRS has been implementing this project, which is funded by the United 
States Department of Agriculture (USDA), in selected chiefdoms in Koinadugu district 
since 2008. The aim of the McGovern-Dole Program is to reduce hunger and improve 
literacy and primary education, especially for girls. The MGD III project in Sierra Leone 
has two overarching goals: (1) to improve the literacy skills of children in the 192 
schools targeted by the program and (2) to improve health and hygiene practices in 
targeted school communities. It is a 3-year project and this review comes at the end of 
the second year. The project aims to achieve these goals by providing school meals, 
teacher training, improving water and sanitation environments, and improving skills 
and knowledge of various beneficiaries. For Sierra Leone, which is one of the poorest 
countries in the world that is also recovering from a recent Ebola epidemic, this 
partnership with CRS/USDA has been constructive. 
PURPOSE
The purpose of this midterm review was two-fold: (1) to assess progress the MGD III 
project has made towards achieving its intended results, based on the project’s 
effectiveness, relevance, efficiency and sustainability; and (2) to recommend actions for 
improving the implementation during the final year of the project. 
METHODOLOGY
The midterm review used a mixed-method approach, which combined the use of 
qualitative and quantitative data. Quantitative data was collected from pupil surveys, 
reading assessments, classroom observations, and school observation checklists. The 
qualitative data came from semi-structured interviews with key informants and focus 
group discussions. The team interviewed head teachers, government officials, CRS 
3
staff, and staff of other partner organizations and held focus group discussions with 
community members. 
A two-stage cluster sampling approach was used to select schools and pupils within 
schools to get a representative sample. Sample sizes were chosen to get statistically 
significant insights at a 95 percent confidence level. The data collection took place 
between May 30th and June 9th, 2017, one year after the baseline data collection. 
FINDINGS AND CONCLUSION
MGD III is a noteworthy project aimed at engaging teachers, head teachers, and 
communities to improve the literacy skills and health and hygiene of young children. It 
is noteworthy because of its long-term commitment to Koinadugu district, one of the 
poorest and most food-insecure districts, in Sierra Leone. The most important findings 
from the midterm review are provided below.
Effectiveness
The project has made good progress towards achieving its strategic objectives. The 
gains achieved in the first objective, related to improving literacy skills, are particularly 
impressive. The percentage of class 2 students who could read and understand grade 
level text increased by a factor of 5, from 8 percent at baseline to 39 percent at the 
interim review. If this rate of improvement continues into the final year, the project will 
surpass its target of 40 percent of students being able to read after 2 years of 
schooling. These improvements appear to be because of improvements in teacher 
knowledge and skills, brought about by an intensive focus on teacher training and in￾classroom coaching. 
The theory of change hypothesized that improved quality of literacy instruction, 
combined with improved attentiveness, and improved student attendance will improve 
the literacy skills of pupils. The data collected confirmed improvements in all three 
areas since the baseline: the percentage of students regularly attending school 
increased from 67 percent at baseline to 100 percent at midline; student attentiveness 
improved from 58 percent at baseline to 66 percent at midline; and the quality of 
4
literacy instruction also improved in many areas. Given these findings, we can conclude 
that the activities around teacher training and coaching, provision of school meals, and 
provision of literacy materials have been especially effective in improving the quality of 
literacy. The evaluation also concludes that given the marked improvement in scores, 
the additional resources committed to supporting teachers such as coaching, have 
contributed to the successes achieved. 
Nonetheless, there is still much to do to improve this objective. The data shows that 
girls are lagging boys in terms of literacy, and that this difference is statistically 
significant. Care must be taken to ensure that gender issues are considered in the 
teaching and learning experience to ensure that teachers pay attention to the needs of 
girls in the classroom. Also, while the quality of teaching instruction has improved, 
many teachers are still not proficient in some basic pedagogy skills and will need 
additional support to improve.
Whereas the progress made in literacy is clear and undisputable, progress under the 
second objective related to improved health and hygiene practices is not. The theory 
of change suggested increased knowledge of health and hygiene practices, nutrition, 
and safe food preparation practices, and increased access to clean water and sanitation 
facilities, preventative health services, and food preparation and storage tools would 
lead to increased use of good health and dietary practices. 
The review shows that knowledge of health and hygiene practices and safe food 
preparation and storage practices declined from the baseline, and it is unclear why this 
happened. One possible reason is that about 14 percent of the cooks in the sample 
had not received the training on safe food preparation and storage because they were 
new or not selected to attend the training. The review also found that because of the 
delay in construction of wells and latrines, access to clean water and sanitation 
facilities has not improved since the baseline. The implementation of the project 
activities in this area needs to be improved if the MGD III is to meet its end of project 
targets. 
5
Relevance
The project is highly relevant and various beneficiaries (teachers, head teachers, and 
students) and stakeholders (Ministry of Education officials, community leaders, and 
partners) consider it a priority. MGD III addresses crucial needs of the targeted 
communities such as improved learning, access to safe drinking water, and nutritious 
meals to students. The project is also in line with the goals of the Ministry of 
Education, which include improving foundational literacy skills of all children and 
improving WASH facilities in schools. Another very important aspect of the program is 
that is has been responsive to changes in the context and has used lessons learned 
from experience implementing the MGD projects over the years to adapt the design of 
the program to make it more relevant to communities. For example, because of the 
experience of the Ebola outbreak, the Phase III project design included more focus on 
health and hygiene practices to help prevent future outbreaks of diseases. 
Efficiency
Integrating health and education within schools is an efficient strategy for improving 
the overall well-being of children in the community. Based on the overall review 
findings and analysis, CRS and its partners have done a good job in implementing this 
project.
Nonetheless, some key informants indicated some shortcomings and bottlenecks in 
executing the project. These included delays in the construction and rehabilitation of 
WASH facilities (latrines and wells) and in distributing certain teaching and learning 
materials. Informants also mentioned logistical challenges related to transportation 
and cash transfers, which lead to delays in implementation. 
Sustainability
Sustainability is an important feature of McGovern-Dole programs worldwide, and for 
USDA this means that schools served by MGD III can continue the activities on their 
own or with support from other sources such as the government or local community 
partners after USDA’s support ends.
CRS took several steps to ensure sustainability of the MGD III project, including 
aligning the project objectives to the goals of the government, developing the capacity 
6
of various stakeholders, and helping schools through the formal approval process that 
qualifies them to receive government support. 
During the focus group discussions, communities expressed a willingness to continue 
the activities after the project ends, including providing school meals, maintaining 
school infrastructure, and continuing to send their children to school. While their 
willingness is encouraging, even now communities struggle with providing basic 
condiments and utensils to support the CRS school feeding program. It is unlikely that 
they will continue with many of the MGD III activities.
For government-approved schools, it is possible that the Ministry of Education, Science 
and Technology (MEST) and its partners will continue some of these activities. MEST is 
now trying to implement a National School Feeding program and teachers in MEST￾approved schools are eligible to receive salaries and on-going professional 
development activities. However, over 75 percent of MGD III schools in this sample 
remain unapproved, and therefore ineligible for support. A key strategy for 
sustainability will be to intensify advocacy efforts to get these schools approved. 
RECOMMENDATIONS
The recommendations in this section are derived from the findings and suggest ways 
to strengthen key areas of the project to improve the chances of meeting or surpassing 
the end of project targets. The main recommendations are as follows:
Improved Literacy Skills 
• Teacher training: while teacher practice has improved overall, there are still 
many individuals who have not mastered basic skills. The project should find a 
way of identifying these teachers and providing them with targeted support and 
coaches can play a role in this.
• Use of teaching and learning materials: while the project has distributed 
thousands of textbooks, these are not being used effectively in classrooms. In 
most classrooms, only the teacher has a text. The review recommends a small
qualitative study on the use of TLMs and the findings from such a study can be 
7
used to design a training for teachers and head teachers on the use and 
management of textbooks in their lessons.
• Class participation: students, especially girls, should be encouraged to speak 
more and ask questions in class. 
• Community engagement: engage parents and communities in supporting
reading (and not just the feeding program). There are several ways that they can 
do this including reading to their children, telling stories, and providing space 
and time for children to read after school. 
• Girls and gender: given the findings that girls are underperforming boys, issues 
of gender in teaching practice and reading should be explicitly addressed.
Ensure that teachers are encouraging and paying attention to both boys and 
girls, that materials are gender-sensitive, and that training focuses on this issue. 
This is especially important given that the 90 percent of the teachers are male. 
Improved health and hygiene practices
• Reinforce key hygiene messages throughout the curriculum and in extra￾curricular activities.
• The project should have a plan for training new cooks who enter the project 
midway and others who were not selected for the training. This might involve 
those who were trained passing on the information to others or MGD staff doing 
cluster trainings during the year.
• Schools should have supplies that enable children to practice what they have 
learned about good hygiene practices. For example, handwashing stations with 
water and soap should be present in all schools and kitchens and there should 
be bins for trash. 
• Keeping school facilities clean should be a communal responsibility, and CRS 
should work with schools to develop strategies to maintain good hygiene 
practices. Only about 57 percent of the schools in the sample have a WASH club.
• Minimum standards for food preparation and storage tools and equipment 
should be developed together with the communities, and MGD III should ensure 
that schools meet the standard
• Over 40 percent of schools have not received the required preventative health 
services (e.g. deworming and vitamin A supplementation). MGD III does not 
itself provide these services, but it should advocate for supported schools with 
the appropriate agencies. 
• Provide reading materials for children that reinforce health messages in a fun 
and engaging way
8
• Prioritize infrastructure and supplies that impact student safety and well-being. 
These include schools that have no working safe source of water, no working 
toilet facilities, no hand-washing stations, and no roof. Soap (or other cleaning 
agents) and water should be mandatory in all school toilets and kitchen. 
• Intensify efforts to complete the construction of wells and latrines
Other recommendations
• Develop a sustainability plan with communities, local government and MEST to 
prepare for the end of the MGD III program.
• School approval: work closely with schools to ensure that schools meet the 
minimum standards to apply for approval and that all of them apply. There are 
still head teachers reporting that they do not know or understand the process 
for applying for school approval.
• Collect further monitoring data on the coaching process that includes records of 
the number of times teachers are visited. If a structured observation tool is 
used, then that data should be analysed with a view to identifying areas for 
further refresher training.
9
COUNTRY CONTEXT
Sierra Leone is one of the poorest countries in the world, ranking 179 out of 188 
countries in the 2016 UNDP Human Development Index (United Nations Development 
Programme UNDP, 2016). It is also one of the most food insecure countries in the world, 
ranking 109 out of 113 countries on the 2016 Global Food Security Index from the 
Economic Intelligence Unit (The Economist Intelligence Unit, 2017). Part of the reason 
for the poor social and economic conditions is that the country was engulfed in a civil 
war lasting over a decade that officially ended in 2001. Unfortunately, in 2014, even as 
Sierra Leone was trying to rebuild its social and economic systems, the country was hit 
by the largest Ebola outbreak the world has ever seen. The first confirmed case was in 
February 2014, and by December 2015 the country had recorded an estimated 8,700 
confirmed cases and over 3,500 confirmed deaths. The government took stringent 
actions to try to curb the disease, including delaying the reopening of schools for almost 
7 months. The Ebola epidemic had a devastating effect on the economy, which is still 
being felt today, and the country remains very fragile.
Education performance is low and progress in achieving learning outcomes has been 
slow. A national assessment of reading in the early grades showed that many children 
could not read simple words or comprehend simple passages after three years of 
schooling (Montrose International, 2014). In addition, while remarkable progress had 
been made in enrolments at the primary level, the 2015 Population and Housing Census 
showed that 35 percent of children of primary-school age were not attending school
(Statistics Sierra Leone, 2016). Literacy rates are also among the lowest in the world, and 
the same census data showed that just 51 percent of the population over aged 10 was 
literate, with the literacy rate for women a mere 41 percent. 
In response to these challenges, the United States Department of Agriculture (USDA) 
has made Sierra Leone one of its priority countries for the McGovern-Dole Food for 
Education (MGD) Programs. The MGD program helps support education, child 
development and food security in low-income, food-deficit countries around the globe. 
10
Its key objective is to reduce hunger and improve literacy and primary education, 
especially for girls, by providing school meals, teacher training, and related support.
11
PROJECT BACKGROUND 
Catholic Relief Services (CRS) has been implementing the McGovern-Dole Food for 
Education (MGD) Program in Koinadugu District in Sierra Leone since 2008. Koinadugu 
was chosen because of the district’s food insecurity status, high malnutrition rates 
amongst children under age five, and below average education performance. The MGD
program has been implemented in three distinct phases.
Phase 1: The first phase of the MGD program (MGD I) ran from 2008 to 2012 in four 
chiefdoms in Koinadugu - Sulima, Mongo, Neini, and Neya. These chiefdoms were 
chosen because they were the most marginalized in the district. Between 2008 and 
2012, the MGD I program distributed almost 1,500 metric tons of food, corresponding 
to 5,780,201 meals served to 18,610 pupils. The project also included take-home 
rations for girls in upper primary from which over 5,000 girls benefitted. In addition to 
food aid, the MGD I project trained school management committees (SMCs) and 
improved on school infrastructure. The MGD I project also distributed teaching and 
learning materials (TLMs) and furniture to schools. 
Phase II: The second phase of the program (MGD II) began in September 2012 and 
ended in January 2016. In Phase II the program expanded to include a fifth chiefdom 
(Dembelia Sinkunia) and to include an additional 75 schools from existing program 
chiefdoms. Phase II also included additional activities such as in-service teacher 
training on Diagnostic Teaching Methods (DTM) to improve literacy instruction and 
established Savings and Internal Lending Committees (SILC) to help strengthen the 
financial status of households. The food aid component continued in Phase II using 
the same two modalities: (1) two in-school meals per day; and (2) take-home rations for 
girls in upper primary who maintain at least an 85 percent attendance rate. The Ebola 
outbreak created a public health emergency that challenged the implementation of 
MGD II activities and diminished its potential impact. 
Phase III: The third phase (MGD III), which is the focus of this interim (midline) review 
is being implemented in the same five districts as MGD II and will run from December 
2015 to September 2018. Most of the activities remain the same as in MGD II, but 
there are important differences. First, MGD III provides one school meal daily, whereas 
12
MGD II provided two. The project also switched from bulgur to fortified rice as the 
staple food. These changes were made to align the school meals component more 
closely with the Government of Sierra Leone’s (GOSL) planned school feeding program. 
Second, MGD III intensified the attention paid to the literacy component of the program 
by hiring 25 literacy coaches who are responsible for 8-12 schools each, and who 
provide professional development support (training, coaching, mentoring) to teachers 
of the early grades. Thirdly, in response to the Ebola crises, MGD III includes a second 
strategic objective focused on improving health and dietary practices of learners and 
the wider school communities. 
13
INTERVENTION LOGIC
The MGD program is a school feeding program with two main goals: (1) improved
literacy of school-aged children and (II) improved health and dietary practices of pupils 
and communities. Figure 1 illustrates the intervention logic of the MGD III program. 
14
Under goal 1 (improved literacy), there are three sub-objectives (improved quality of 
literacy instruction, improved pupil attentiveness, and improved pupil attendance) and 
multiple activities (e.g. teacher training, school feeding, provision of teaching and 
learning materials). Under goal 2 (increased use of good health and dietary practices), 
there are six sub-objectives; three are related to increased knowledge of good health 
and hygiene practices, safe food preparation, and nutrition, and three are related to
increased access to clean water, preventative health services, and food preparation and 
storage tools. CRS summarizes the theory of change of the MGD III project as follows:
In the implementation of the MGD III, CRS partners with local and international 
organizations such as the International Literacy Association (ILA), The Association of 
Language and Literacy Educators (TALLE), Northern Polytechnic (NP), and Caritas. ILA 
provides training in diagnostic teaching methods (DTM) for teachers and supervisors, 
TALLE supports the coaching on literacy instruction, Caritas provides Life Skills Training
and implements the WASH activities, and NP provides teacher certification through a 
distance education program. Throughout the implementation, all parties focus on the 
IF the quality of literacy instruction is improved AND children,
having received school meals, have their short-term hunger 
reduced AND their attentiveness and attendance improved
THEN the literacy of school-aged children in the project area 
will be improved.
Also, IF pupils learn about improved health and hygiene practices
at school AND cooks learn about safe food preparation and 
storage practices AND schools have improved access to clean 
water and sanitation sources AND pupils have increased access 
and knowledge to preventative health interventions AND food 
preparers have increased access to requisite food preparation and 
storage facilities, tools and equipment the use of health and 
dietary practices will improve.
15
development of capacity in schools and communities and national and local government 
agencies and departments. 
EVALUATION PURPOSE
The purpose of the interim review was to highlight progress being made by the MGD III
project towards meeting its objectives. The review was to assess the MGD III in terms 
of its effectiveness, relevance, efficiency, and sustainability and to recommend ways to
improve the project.
REVIEW QUESTIONS
The main questions for the interim review, which followed the DAC evaluation criteria, 
were:
• EFFECTIVENESS: To what extent has the project has achieved (or has made 
progress towards) its objectives (or intermediate results)? 
• RELEVANCE: To what extent does the project conform to the needs and priorities 
of target groups and the policies of the country and donor? 
• EFFICIENCY: How could integration of the two strategic objectives be enhanced 
or further leveraged to deepen positive program results? 
• SUSTAINABILITY: To what extent will the benefits of the project continue after 
the end of the project life? 
The midterm evaluation adopted a mix of both quantitative and qualitative research 
methods. The quantitative survey design consisted of pupil survey, reading 
assessment, cook assessment, teacher and classroom observation and school 
observation checklist. The qualitative component had two parts: in-depth interviews 
with key informants (head teachers, project staff, MEST officials, and partner 
organizations) and focus group discussions with community members. Insights from 
16
the qualitative interviews and discussions complemented data obtained through the 
surveys and allowed researchers to examine certain topics in more depth.
The interim review did not include a comparison group because it was difficult to find 
schools that were disadvantaged as much or in the same ways as the targeted MGD III 
schools. Because the available communities didn't provide a good comparison, the 
ethical issues of continuing to visit schools who did not receive the same resources 
while taking information from them became problematic. Therefore, management 
decided to use a pre/post design rather than a comparison group design. 
SAMPLING AND SAMPLE SIZE
A two-stage cluster sampling approach was employed to select pupils, teachers and 
cooks. At the first stage, schools were randomly selected as clusters and then pupils, 
cooks, and teachers within schools were selected at the second stage. The advantages 
of cluster sampling are that it is more economical, time-efficient, and easier to manage 
than a simple random survey. The disadvantage of this sampling technique is that 
individuals (teachers or pupils) within a cluster (school) tend to have similar 
characteristics, and there is a chance that they may not be representative of the overall 
population. 
Sample Size 
Pupils, teachers and food preparers were the sampling units of the quantitative survey. 
The sample sizes for the sampling units were computed using four (4) key indicators. 
MGD III baseline results for these indicators were used for computing sample sizes for 
the sampling units. 
Pupils
The formula below was used to compute the sample size for pupils:
where,
• n is the required sample size (i.e. number of pupils to be surveyed)
• 4 is the factor required to achieve 95% level of confidence;
 
 
2
arg
14 2.1
M in of error
deeffpp
n


17
• p is the value of the key indicator; in this case, the ‘percent of pupils identified as attentive during 
classroom activities (boy & girls) (0.58)
1;
• deff (design effect) is the factor by which the precision of the estimates of the key indicators 
decrease due to the sampling design relative to a survey that uses simple random sampling (2)2;
• 1.2 is the factor necessary to raise the sample size by 20% for the expected non-response3
. This 
assumes 80% response rate;
• The margin of error (± 5 percentage points) i.e. Alpha α=0.05.
Using the formula, the sample size for pupils was estimated at 900. The decision was 
made to select 15 children per school across grades 2-6 as too many learners from one 
school would reduce the variability in our sample, which meant that we needed to have 
60 schools in our sample. 
The schools were the primary sampling unit and were targeted for the following 
surveys:
• Pupils survey,
• Reading assessment,
• School observation checklist, 
• Key informant interviews (with head teachers, SMCs, CTA & Mothers club)
• Teacher and classroom observation. 
The surveys used are include in full in Annex B and, except for a few questions, they 
are the same as those used during the baseline. This makes it possible to compare the 
responses at midline to those at baseline. 
Table 1 shows the number of schools and pupils to be sampled by chiefdom.
 
1 MGD III baseline result.
2 The design effect, estimated at 2, is chosen because of the large cluster sample size (i.e. 15 pupils per school).
3 The 20% non-response is assumed to increase the sample size sufficiently large to be able to detect an effect on 
the following groups: meals only, meals + THR, meals + SILC and nothing.
18
Chiefdom
No. of 
MGD 
Schools
Proportion 
of schools
Number of 
schools to 
sample
Adjusted 
number 
of schools 
to sample
Number of 
pupils to 
sample
Dembelia Sinkunia 18 0.094 5.6 6 90
Mongo 43 0.224 13.4 13 195
Neya 37 0.193 11.5 12 180
Neini 65 0.338 20.3 20 300
Sulima 29 0.151 9.1 9 135
Total 192 1 59.9 60 900
Pupils were chosen in each school by a simple random (ballot) method across the five 
grades (2 to 6) for the pupils’ survey; 3 pupils in each grade yielded 15 pupils per 
school. 
The reading assessment test was administered to only Grades 2 and 3 pupils. The 
target number of pupils was 480 for each of class 2 and 3 for a total of 960 pupils. 
This translated to 16 pupils from each of the 60 sample schools. 
Teachers
Simple random sampling method was used to select teachers in each school. Three (3) 
teachers were selected in each survey school – yielding teachers – for the teacher 
and classroom observation. The instruction to enumerators was to observe teachers of 
classes 2-4.
Cooks/Food Preparers
Based on the sample size calculations, we selected three (3) cooks per school for 
interview, yielding a maximum of 180 cooks for the survey.
FGDs were held in all survey clusters/school communities with community 
stakeholders including women and men from SILC and non-SILC households. 
Discussions were held separately for men and women and each FGD had between 7 
and 12 participants. Enumerators were instructed to select a wide range of ordinary 
community members as well as community leaders (chiefs, women’s leaders, and 
headmen). Community members who had already been interviewed because of their 
19
membership in SMCs and Mothers Groups were not included to increase the diversity 
of opinions.
Table 2 gives the response rates for schools (head teachers), teachers cooks, and
pupils. Response rates were high and ranged from 88 percent to 100 percent.
 
Group Target 
number
Number of 
responses
Response 
rate
Schools 60 60 100%
Teachers 180 158 88%
Pupils 900 888 99%
Cooks 180 160 89%
Recruitment and Training of Enumerators
Thirty-six (36) enumerators and five supervisors were recruited for the data collection. 
The supervisors and most of the enumerators had participated in the baseline 
evaluation of the MGD III and previous evaluations of MGD II, and as such had 
experience with the tools and methodology. All enumerators had at least a post￾secondary qualification and many had participated in other surveys and/or had taught 
in schools. 
The supervisors and enumerators attended a 9-day training on the administration of 
the evaluation tools/questionnaires and general survey protocols to equip them to
collect appropriate and quality data for analysis. For this survey, quantitative data 
collection was done using mobile devices (iPad minis), and the enumerators had 
additional training on collecting data using mobile devices. Training also included role￾play to deepen practice on the administration of the tools and translation of keywords 
into the lingua franca (Krio) to help in the focus group discussions and pupil 
interviews. 
Another focus of the training was on ethical issues in conducting surveys with children 
and vulnerable populations. The training included the importance of consent and 
protecting the rights and dignity of all those interviewed. This meant that each child 
20
had to agree to proceed with the survey and we did not take personally identifiable 
information (e.g. names) from them. We also asked that each child be interviewed in a 
public place, but away from their teacher and others who could hear their answers. 
Finally, if it appeared children could not do a subtest, the enumerators were trained to 
stop that subtest and move to something else. 
Furthermore, at the end of the training, arrangements were made for enumerators and 
supervisors to practice administering the tools in nearby schools and communities. 
This allowed for data collectors to practice administering the tools in ‘real-life’ 
situations and gave an opportunity to test the appropriateness of the questions. After 
the pretesting, final revisions were made to the tools and rendered on the tablets 
before the start of the field research. 
DATA QUALITY, MANAGEMENT, AND ANALYSIS
The first step in ensuring that the evaluation produced good quality data was to 
develop well-defined questions that helped to capture accurate and relevant data for 
analysis. The evaluation tools/questionnaires had been reviewed multiple times since 
many of them were used in previous evaluations. The new questions added also went 
through multiple reviews. In addition, the questionnaires and surveys were piloted in 
the field before adopted and administered to collect the data for the evaluation. During 
the administration of the survey, supervisors visited enumerators in their various sites 
to ensure that data collection was proceeding as planned and to provide quality 
assurance to the evaluation process. The use of mobile devices to collect data helped 
to produce good quality quantitative data. 
Quantitative data collected in the iPads was exported to Excel and then into SPSS 
software package in which further cleaning of data was carried out. Errors and 
mismatches due to inconsistencies during data collection were checked and corrected; 
this enhanced logical consistency and quality of data. The cleaned data was then 
exported to Stata 14 (a statistical software) for analysis. All programs created to 
21
analyze the data and produce tables have been saved so that the findings can be 
reproduced by other researchers.
This section summarizes the major findings from the midline review organized by the 
evaluation questions. The data used in this analysis were collected between May 30th
and June 9th, about a month before the end of the school year. 
EFFECTIVENESS: TO WHAT EXTENT HAS THE PROJECT HAS ACHIEVED 
(OR HAS MADE PROGRESS TOWARDS) ITS OBJECTIVES (OR INTERMEDIATE 
RESULTS)?
The MGD III project has two main strategic objectives and numerous sub-objectives. 
Below the findings are organized based on these objectives and sub-objectives. Where 
possible, we have reported the starting point of the indicators, as measured during the 
baseline, the status during this interim review, and the final target at project end. 
MGD S.O.1 Improved Literacy of School-aged children
One of the main strategic objectives (SO) of MGD III is to improve the literacy skills of 
children in the program. USDA’s standard indicator to measure literacy skills is: the 
“percentage of children, who at the end of two grades of schooling, demonstrate that 
they can read and understand grade level text.” Although the children in class 2 were 
the population of interest, children in class 3 were also sampled because previous 
assessments had shown that children in class 2 had minimal reading skills and scored 
zero in several of the reading subtests. 
The literacy assessment tool used was adapted from a nationally developed and 
validated test developed by Dr. Johanna Kuyvenhoven, a literacy educator from Calvin 
College. As a consultant with UNICEF, the assessment tool was used to assess children 
all over Sierra Leone. The skills assessed are like other tests of early grade reading –
phonemic awareness, letter identification, familiar word reading, and reading and 
22
comprehension. The main difference is that these subtests are not timed. The same 
assessment tool was used in the baseline and midline, but the order of some of the 
words and letters was changed.
Learners were given a short, grade-appropriate story to read aloud after which the 
enumerators asked five comprehension questions. Only learners who could read the 
first sentence of the text correctly were asked questions. The number of questions 
asked of each pupil depended on how far in the story each child had read. Children 
were asked only those questions that could be answered from the text that they read. 
Four questions in this subtask were literal and could be answered directly from 
information provided in the story. The last question was an inferential question and 
required pupils to combine information from the story with their background 
knowledge to derive a correct answer. A learner is said to be able to “read and 
understand grade level text” if they answer at least 4 of 5 questions correct on the 
reading comprehension test. 
In total 951 pupils (481 in class 2 and 470 in class 3) took the reading assessment, of 
which 56 percent were boys. Given high-level of pupil absenteeism on the day of 
survey, it is possible that our sample was biased if it turns out that pupils who were 
absent are different from those who were not. For example, if mostly lower-performing 
pupils were absent then our mean estimates would be higher than they are in the 
population. 
Table 3 shows the percentage of boys and girls who attained a particular score on the 
reading comprehension. About a third of class 2 pupils scored zero on the 
comprehension test (40 percent girls and 29 percent boys). In class 3, 21 percent 
scored zero (20 percent of boys and 22 percent girls) on the comprehension test. A 
pupil could get a zero score if they couldn’t read the first line of the text and were 
therefore not asked any of the comprehension questions or if they did attempt the 
questions, but didn’t get any of them right. 
23
Score on 
comprehension
(out of 5)
Class 2 Class 3
Boy Girl Total Boy Girl Total
0 28.6 39.7 33.7 20.3 22.1 21.1
1 8.0 3.2 5.8 3.0 5.4 4.0
2 6.9 7.3 7.1 11.7 9.8 10.9
3 17.6 17.4 17.5 20.7 25.5 22.8
4 17.6 16.9 17.3 21.8 16.7 19.6
5 21.4 15.5 18.7 22.6 20.6 21.7
Based on the indicator definition, the percentage of children who can read and 
understand grade level is depicted in Figure 2 below. The table shows remarkable 
achievement over the year since the baseline was done. At interim review 39 percent of 
boys and 32 percent of girls could read and understand grade level text at the end of 
class 2, compared with only 8 percent during the baseline, an increase of a factor of 4.4
This is quite a dramatic change, and later in this report we investigate the changes in 
instructional practice and coaching that may have contributed to this.
This level of performance in the midline review shows that the project has exceeded its 
end of project target of 30 percent of children being able to read grade level text. 
However, is also important to note that while reading improved for both boys and girls, 
girls performed worse than boys in the reading comprehension test, and that 
difference appears to be statistically significant.
 
4 This was a different cohort of pupil, but they received the same test as was administered during the baseline. 
24
In addition to the reading comprehension sub-test, we also assessed learner’s progress 
in several reading skills such as alphabet naming, phonemic awareness, phonics, and 
listening comprehension. 
Alphabet Naming
In this task, children were presented with a random list of the 26 letters of the 
alphabet (in mixed upper and lower case) and were asked to name the letters. This was 
the simplest task for the learners to perform. There was no difference in the scores of
boys and girls.
The average number of letters identified correctly was 23 in this review improved from 
19 during the baseline (Figure 3). Slightly more than half of the pupils (53 percent) 
could identify all 26 letters of the alphabet by the time of the interim review, which is 
better than at baseline. However, after two years of schooling, it is expected that all 
pupils should be able to identify all the letters of the alphabet. 
38.9
32.4
44.4
37.3
7.9 8.2
15.1 12.4
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Boy Girl Boy Girl
Class 2 Class 3
% of learners
Interim Baseline
25
Phonemic Awareness (PA)
There were two types of PA tasks. In the first, the learner was given pictures of 10 
common objects, told the names of the object, and asked to say the initial sound. In 
the second PA task, the learner listened to a set of words read aloud and they had to 
identify the word that begins with a different sound. For example, they would hear the 
words “moon ball mouse” and would have to identify “ball” as the word with a different 
beginning sound. 
Figure 4 compares the mean scores in the PA subtests from baseline to midline and 
shows strong improvement over the year. Learners scored slightly better on the first 
subtask than on the second, possibly because they were more familiar with the task.
There was no difference in the scores of boys and girls in the PA tasks.
23 23
22
24
23
20
18
17
21
19
0
5
10
15
20
25
Boy Girl Class 2 Class 3 Total
Midline Baseline
26
Familiar Word Reading 
In this sub-task, learners were presented with a list of 40 familiar words and were 
asked to read the words. The test was discontinued if the pupil couldn’t read any of 
the first ten words. Learners in class 2 and class 3 had a different set of words, but 
with some overlap. The number of words read correctly doubled from the baseline to 
the interim review, from 11 to 22, as shown in Figure 5. Boys performed better than 
girls, and as expected, pupils in class 3 performed better than those in class 2. The 
difference in both cases is statistically significant. The percentage of pupils who scored 
zero on this sub-task also reduced dramatically from 42 percent during the baseline to 
17 percent a year later. 
6 6
5 5
3 3 3
2
0
1
2
3
4
5
6
7
8
9
10
Boy Girl Boy Girl
From Pictures From Text
Midline Baseline
27
Listening Comprehension (LC)
For this sub-task, the assessor read a passage to the pupils and pupils then had to 
answer a few questions about the story they were read. The mean score for boys was
2.02 and for girls 1.85, and this difference while small is statistically significant. 
Sixteen percent of learners scored zero on this sub-test during the midline, compared 
to 34 percent during the baseline.
To summarize, pupils performed remarkably better on all sub-tests at the interim 
review compared to the baseline. The starting point was low, but the progress is still 
remarkable. That said, only a third of pupils can read and understand grade level text
and girls are performing worse than boys. Also, because we do not have a control 
group, we cannot clearly attribute all the change we see to the FFEIII project. 
MGD.1.1 Improve quality of instruction
Improving the quality of literacy instruction is a major part of the implementation 
strategy of the MGD III program. The program aims to develop the skills of teachers in 
several ways including sponsoring teachers to acquire their teaching certificate from 
Northern Polytechnic; providing in-service training in Diagnostic Teaching Methods 
23
21 20
24
22
12
9 9 9
11
0
5
10
15
20
25
30
35
40
Boy Girl Class 2 Class 3 Total
Average number of words identified correctly 
(out of 40)
Midline Baseline
28
(DTM) designed by the by International Literacy Association (ILA); and coaching of 
teachers in literacy instruction by TALLE. 
MGD 1.1.1 More consistent teacher attendance
For teaching and learning to take place, at a minimum pupils and teachers must be 
present in school. Almost all schools (97 percent) had a book for recording teacher 
attendance. The teacher attendance rate on the day of the survey, based on attendance 
register, was 90 percent overall, 90% for male and 88% for females. This was up from 
the attendance rate at the baseline survey (see Figure 6). 
MGD 1.1.2 Better access to school supplies and materials
To be effective, teachers need materials to help them teach. The MGD III project has 
provided teaching and learning materials to targeted schools to improve the learning 
materials. During the classroom observation, enumerators recorded the evidence of 
the availability of various materials in the classroom. At baseline, 90 percent of 
classrooms had a blackboard, but only a third had any teacher furniture and even less 
had evidence of materials (children's work, posters etc.) on the wall of the classrooms, 
which may be because schools don't have the stationery to make these. At the time of 
90% 88% 90% 87% 83% 87%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Male Female Total
Interim Baseline
29
the current interim review, a smaller percentage of schools were observed to have 
blackboards (75 percent), but a larger percentage of schools were observed to have all 
other supplies (see Figure 7). With regards to the chalk boards it is unclear why they 
seem to be fewer in place as 196 were distributed since the baseline and others were 
distributed in MGD II. It is possible that our sample had more schools in which the 
chalkboards were delivered in MGD II, and these have become damaged or stolen. 
The seating arrangement in schools appear to have improved with fewer schools in 
which more than 3 children share a desk or in which there were no desks. (Figure 8).
CRS delivered benches and desks made to accommodate three learners (3-seater 
benches). We expect the seating arrangement to improve over the next year as CRS has 
not yet reached its target for distributing school furniture. MGD III plans to deliver 
furniture to 240 classrooms over the life of the project (150 in year 1 and 90 in year 2), 
but as of June 2017, only 145 classrooms (60 percent) had received furniture. 
75
62
49
61
38
90
33
20 17 14
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Chalkboard Teacher
Furniture
Pupil's work on
wall
wordchart WASH Poster
Percent (%) of observed classrooms
Midline Baseline
30
MGD 1.1.3 Improved literacy instructional and learning materials 
Books and other reading materials are essential for children to learn; as such, the MGD 
III project distributed teaching and learning materials (TLMs) to all schools at the start 
of the 2016-2017 school year. The distributed TLMs included 9,185 slates, 5,000 
laminated number card sets, 5,000 laminated alphabet card sets, 140,938 exercise 
books, 5,000 boxes of chalk, 27,453 report cards and 60,000 supplementary readers. 
Because textbooks arrived late, their distribution started in May and will continue until 
August 2017. However, the schools should have had textbooks inventory from MGD II. 
The availability and use of textbooks in the observed classrooms, at baseline and 
midline, are shown in Figure 9 below. The percentage of observed classrooms with no 
texts being used dropped from 25 percent to 14 percent, which is good. However, in 
most classrooms (53 percent), only the teacher was using a textbook. Combining these 
two facts, the percentage of classrooms where no pupils were using textbook (either 
because there were none or only the teacher was using) was higher during the midline 
review (67 percent) than the baseline (56 percent). Despite the low use of textbooks in 
observed classrooms, 97 percent of head teachers (all, but 2) interviewed 
acknowledged receiving textbooks from CRS. Unfortunately, it is common practice for 
schools in Sierra Leone to keep the books in head teacher’s office rather than use in 
1
9
62
28
1
3
8
41 39
9
0
10
20
30
40
50
60
70
80
90
100
Each child has
own
desk/bench
Two children
share a
desk/bench
Three children
share a
desk/bench
More than 3
children share a
desk/bench
There are no
desks/benches
Percent (%) of observed classrooms
Midline Baseline
31
the classrooms. Training for head teachers and teachers should include a session on 
the management and use of textbooks in the classroom.
In terms of the availability of other teaching and learning materials for pupils, about 75 
percent of them had exercise books and pencils, 25 percent had alphabet cards
(compared to 10 percent at baseline), and 11-12 percent had slates and chalk
(unchanged from baseline).
In addition to reading materials provided for school, MGD III also provided reading 
materials for use in reading clubs where children come together to read after school 
hours. According to monitoring records, 152 reading clubs have been established, and 
92 percent of schools in the sample reported having a reading club. Participation in 
reading clubs are voluntary, and 64 percent of children in classes 2 and 3 were 
members of reading clubs. The books used in the reading clubs were provided through 
a partnership with the International Book Bank. 
MGD 1.1.4 Increased skills and knowledge of teachers
11 11 11
53
14
8
11
25
31
25
0
10
20
30
40
50
60
70
80
90
100
By the children,
one each
By the children,
shared by two
By the children,
3 or more
By the teacher
only
There are no
books or readers
Percent (%) of observed classroms
Midline Baseline
32
During this interim study, enumerators observed and interviewed 158 teachers in their 
classrooms. The teaching workforce is overwhelmingly male (90 percent), which may 
have an adverse effect in the learning outcomes of female pupils, who performed 
statistically worse than boys in the reading comprehension indicator. With regards to 
training and qualifications, 24 percent of the teachers interviewed had a teaching 
certificate; of those without a certificate most had at least a senior secondary 
qualification (WASSCE or O ’Levels). The percentage of teachers with a teaching 
certificate appears to have gone down since the baseline, and this could be because of 
the instance of the sample or because teachers with certificates are leaving these 
schools. Some key informants mentioned teacher attrition as a challenge. 
Teacher Demo Midline Baseline
Number Interviewed 158 118
% Male 90 92
% Has Teaching Certificate 24 32
Highest Qualification, if no 
Certificate
% BECE 32 20
% WASSCE 57 44
% O'LEVEL 12 7
% Other 6
Training
% DTM 84 28
% DE 59 20
Teacher training 
The MGD III project invested significant resources in improving the quality of 
instruction through training and coaching of teachers. As Table 4 shows, 84 percent of 
teachers reported having attended training on Diagnostic Teaching Methods (DTM) and
59 percent were participating in the distance education (DE) course offered by 
Northern Polytechnic, which will lead to a teaching certificate on successfully passing 
an exam. These participation rates were significantly higher than the participation 
levels reported during the baseline. 
33
All teachers are eligible to attend DTM training, which focused on classroom 
management, lesson plans, and child-centred methodology. The distance education 
training is for uncertified teachers who meet the selection criteria and pass the 
college’s entrance exams. At the completion of DE program, teachers who pass the 
terminal exams will receive a teaching certificate. Chiefdoms with lower percentages of 
certified teachers were prioritized for participation in the DE program. There are 
currently 296 teachers enrolled in the distance education program. 
Starting with MGD III, teachers of classes 1-3 received training on early grade reading 
instruction, focusing on specific methodologies for teaching pupils in Class 1-3, use of 
early grade instructional materials, and developing locally-made TLMs.
Coaching and Mentoring
In addition to all the training, MGD III also hired 20 literacy coaches and 5 coach 
supervisors, who were responsible for training teachers and providing classroom-based 
coaching and support between the trainings. Each literacy coach covers 8-11 schools in 
a geographic cluster and spends at least 3 weeks per month in the field giving direct 
support to their designated schools. On average, they should visit each school at least 
three times per month. During these coaching visits, they observe teachers in their 
classrooms and provide feedback on their teaching. Where needed, they also provide 
demonstration lessons to help teachers who are struggling in certain areas. In MGD II, 
MEST supervisors were expected to play these roles, but due to other commitments 
and logistical duties they don’t always have time to visit schools. 
Figure 10 shows that teachers do indeed get support from coaches, head teachers and 
MEST supervisors. 96 percent of teachers reported that were visited by a coach in the 
month prior to the survey, 93 percent reported visits by head teacher, while 56 percent 
reported visits by a MEST supervisor. 
34
In the next section, we report on the findings from the teacher and classroom 
observation and compare teacher practice between the baseline and midline. 
Observation of Teaching Skills
The skills and knowledge of teachers were assessed using teacher surveys and 
teaching observations. The rubric used for the teaching observation and teacher’s 
knowledge of pedagogy combined tools developed by International Literacy
Association (ILA) and MEST. Teacher competencies were assessed in 2 broad areas as 
outlined below:
1. The teacher uses a variety of pupil (formative) assessment techniques during the 
lesson
2. The teacher demonstrates good instructional practice (explores prior 
knowledge, uses lesson plans, variety of teaching methods)
Teacher performance was scored using the guidelines shown in Table 5. 
1 Not yet 
Started
2 Beginning 3 Proficient 4 Excellent
There is no 
evidence of desired 
behavior. The 
The behavior is 
attempted, but not 
consistent. The 
The behavior is 
acceptable and 
somewhat 
The behavior is 
consistent and 
exemplary. The 
0
10
20
30
40
50
60
70
80
90
100
Coach Head Teacher MEST Supervisor
Percent (%) of teachers
35
teacher needs 
significant support 
to develop practice. 
teacher needs 
ongoing support to 
develop practice
consistent and 
could be used as a 
model for others. 
teacher could teach 
others to develop 
this behavior. 
Figure 11 below shows how teachers performed in Area 1 during midline and baseline 
and across all areas, teachers performed better at the midline, which is what is 
expected. 46 percent of teachers were proficient in lesson planning, 52 percent 
explored prior knowledge of pupils at the start of a lesson, and 48 percent of teachers 
showed proficiency with using a variety of teaching methods (e.g. a mix of group work 
pair work, and whole class instruction). These percentages were more than double 
those at baseline. Furthermore, at the midline, there was only a small percentage of 
teachers (less than 13 percent) that had not started practicing these teaching methods.
Further statistical analysis showed that the variables most likely to predict proficiency 
in the above skills were if the teacher already had a teaching certificate or if they 
13
42
10
42
12
31
41
36
39
36
40
48
42
18
44 37
18
4 4 8 11
3
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Midline Baseline Midline Baseline Midline Baseline
Evidence of lesson plans Teacher explores prior
learning
Teacher uses a variety of
teaching methods
Not Started Beginning Proficient Excellent
36
reported having been visited by MEST supervisors during the year.
5 However, teachers 
who already had a teaching certificate (whether from MGD or not) were significantly 
more proficient in their practice than those who did not. Teachers who reported being 
visited by a MEST supervisor were observed to be more proficient in all three tasks 
above. We did not see any effect of coach visits because there was no variation in the 
sample as almost all teachers were visited by a coach. 
Area 2 focused on teachers’ proficiency in using a variety of pupil assessment 
techniques during the lesson, which is one of the foundation skills of the DTM training. 
Figure 12 shows that 54 percent of teachers were proficient in checking for pupil
understanding compared to 27 percent at the baseline; 57 percent showed proficiency 
in adjusting their lesson based on pupil responses, and 52 percent had clearly 
identified lesson objectives. The share of teachers displaying proficiency in these areas 
were more than double that of the baseline.
 
5 Chi-Squared test, with p=0.05
37
As in area 1, logistics regression reveal that the odds of being proficient in any of the 
above skills is significantly higher for teachers with a teaching certificate and teachers 
who were visited by a MEST supervisor during the year.6
The final practice area (Area 3) observed was how well the teacher engaged the pupils 
during the lesson. Here again, we see marked improvement in proficiency in these 
skills from the baseline to the midline. Sixty-six percent of teachers were reported to 
be proficient in getting pupils to participate and stay engaged in the midline review, 
compared to only 37 percent in the baseline (Figure 13). Similarly, 66 percent of 
teachers got pupils to read, write, and speak during the lesson as compared to only 36 
percent in the baseline. Statistical analysis showed that female teachers are more likely
than male teachers to be proficient in this skill area; similarly, teachers who were 
visited by MEST during the year were more likely to be proficient than those were not. 
 
6
Logistic regression (p=0.05), and controlling for sex, teacher certificate, whether MEST visited, and whether they 
attended training
4
25
4
28
13
39
41
48
38
52
34
32
46
23
49
19
37
21
8
4 8
2
15 8
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Midline Baseline Midline Baseline Midline Baseline
Checks for
understanding
Adjust practice when
needed
Lesson objectives are
clear
Not Started Beginning Proficient Excellent
38
Self-reporting of knowledge of skills related to reading instruction
In addition to the observation of teacher practice, teachers had the opportunity to self￾assess their knowledge and skills in areas related to literacy instruction. They could 
rate their abilities and confidence on a scale from 1-4 with 1 being “I know nothing 
about this”; 2 – “I know about this, but I do not know how to use it”; 3 – “I know about 
this and have some confidence in my abilities in this area; and 4 – “I have excellent 
knowledge and skill in this area”. The results are shown in Figure 14 for the baseline 
and midline assessments. 
Compared to the baseline, more teachers expressed confidence in their ability to use 
the various teaching techniques at baseline and less teachers reported not knowing 
anything about the technique. For example, at baseline 42 percent of teachers said 
they knew nothing about “Word Recognition and Phonics”, whereas only 18 percent of 
teachers reported that at midline (Figure 14). Similarly, 53 percent of teachers reported 
being confident in comprehension teaching strategies compared to 25 percent at 
baseline. 
2
16
2
26
33
46
33
37
58
32
58
28
8 5 8 8
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Midline Baseline Midline Baseline
Pupil's participation and
attentiveness
Pupils read, write and speak during
lesson
Not Started Beginning Proficient Excellent
39
Teacher Motivation
To better understand what inspires teachers in their jobs, we asked teachers what 
aspects of their job they found motivating. When presented with the job aspect, 
teachers had to rank it from 1 to 5, with 1 being NOT motivating to 5 being highly 
motivating. Figure 15 reveals that most teachers are highly motivated by their 
18
42
20
54
11
47
11
48
7
40
8
45
11
46
21
15
52
22
57
46
18
39
14
45
23
45
24
50
22
41
21
47
22
30
32
15
18
7
4
1
4
1
8
2
8
1
6
3
6
1
12
2
4
4
2
4
3
32
39
37
32
37
28
37
27
37
36
44
32
30
30
44
49
32
56
34
0 10 20 30 40 50 60 70 80 90 100
Mid
Base
Mid
Base
Mid
Base
Mid
Base
Mid
Base
Mid
Base
Mid
Base
Mid
Mid
Base
Mid
Base
Word Recog. Fluency Vocabulary
Comprehensi
on Assessment
Eff.Questioni
ng Motivation
Indep
enden
t Grouping Adapting
Know Nothing Know about, but can't use skill Confident using Excellent skills
40
interpersonal relationships they have with head teachers, students and colleagues. 
However, very few teachers are motivated by job aspects related to remuneration, 
conditions of service, professional status, or MEST policies. The latter is not surprising 
since many teachers are volunteer teachers, who do not receive government salaries. 
While the MGD III project cannot address these issues, they can affect some by 
continuing to recognize teachers for their hard work and effort. 
MGD 1.1.5 Increase knowledge and skills of administrators
Ninety-one percent of head teachers (all but 5) reported having attended training on 
DTM. In terms of the benefits perceived, most reported that the training helped them 
improve their teaching and the teaching of other teachers. Other benefits mentioned 
included improvements in school management, lesson planning, teacher observation 
and supervision, record keeping, and building relationships with teachers. 
10
15
16
22
26
29
29
30
34
42
42
43
44
57
61
63
63
67
0 10 20 30 40 50 60 70 80
Salary and benefits
Working Conditions
MEST Policies
Job Security
Sense of achievement
Professional Status
Factors in Personal Life
Work itself
Responsibility
Potential for Advancement
Sense of accountability
Proessional Growth
Evaluation
Recognition
Relationship with colleagues
Relationship with students
Supervision by superiors
Relationship with Head Teacher
Percent (%) of teachers
Factors that "highly motivate" teachers
41
MGD 1.2 Improve pupil attentiveness
During the classroom observation, enumerators recorded what children were doing 
during the class time. Attentiveness was defined as pupils asking questions, actively 
participating in lessons, and following instructions without distraction. Figure 16
shows whether the observation saw little evidence (less than a third of class); moderate 
evidence (about half the class); or extensive evidence (more than a half of class) of 
attentive behaviors. Between the baseline and midline, pupils improved on their class 
participation and following instructions without distraction. For example, at midline 75 
percent of classrooms showed at least moderate evidence of class participation 
compared to only 54 percent at midline. But there was no improvement in pupils
asking questions and seeking help or with listening and working without distraction. 
The least practiced behavior was asking questions or seeking help from teacher.
Teachers still need to do more to encourage pupils to ask questions when they are 
struggling with concepts for real learning to take place.
MGD 1.2.1 Reduced short-term hunger
At the time of the baseline, the school feeding had yet to start in the schools as none 
of the pupils interviewed reported receiving meals in school. At the midline survey, 98 
percent of pupils interviewed reported receiving meals daily at school, but only 89 
percent had had a meal at the time of the interview. Schools varied in when they serve 
60 59
25
46
32 33
20
31
34 28
51
31
47 47
56
50
6
13
25 24 20 19 23 19
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Midline Baseline Midline Baseline Midline Baseline Midline Baseline
Ask questions or
seek help
Participate in the
lesson
Listen and work
without
Distraction
Follow
instructions
without
distraction
Percent (%) of classrooms
Little Evidence Moderate Evidence Extensive Evidence
42
meals with 52 percent of children reporting meal times in the morning and 45 percent 
reported having meals in the afternoon. Of those who had eaten, 72 percent reported 
not hungry, while 28 percent reported they were still somewhat hungry. 
MGD 1.3 Improve pupil attendance
According to monitoring data from CRS, pupil attendance has increased significantly 
from the baseline. At the baseline about two-thirds of students were reported to be 
attending schools regularly, which is defined as students whose attendance rate is 
above 80 percent. A year later, 100 percent of students were reported to be attending 
school regularly. 
MGD 1.3.3 Improved school infrastructure
Figure 18 shows the status of the school infrastructure at midline and baseline. Most 
schools are one building structures. The majority have corrugated iron roofs (93 
percent), concrete polished walls (72 percent); and concrete floors (82%). The 
infrastructure remains relatively unchanged from the baseline. The project target is to 
rehabilitate 52 school buildings, and 19 buildings had been rehabilitated by end of 
August 2017. There were two schools that had no roofs, and these should be 
prioritized for renovation as this could hamper children’s safety and ability to learn 
during extreme weather. 
89
72
28
0
98
0
10
20
30
40
50
60
70
80
90
100
Ate meal in
school
Not at all
hungry
Somewhat
hungry
(not
enough
food)
Very
hungry
Given food
every day
Percent of chidren (%)
43
Unlike the school buildings, the status of water and sanitation facilities (WASH) was 
quite poor. Only 50 percent of schools had a (potentially) safe source of water, and for 
almost all, it was a hand pump well. Furthermore, of those with water wells, 37 percent
were not functioning (Figure 19). Less than half of schools had a hand washing facility 
at or near the toilet. The status of WASH facilities in the schools have remained 
relatively unchanged since the baseline despite this being a priority for the project.
Monitoring data and interviews with project personnel revealed that there are delays in 
the completion of WASH projects. Out of 62 planned toilets, 52 had been completed by 
end of August, and the remaining 10 are scheduled to be completed by the end of 
September. Caritas has installed 191 tippy taps (hand washing facilities) in 55 schools 
where school WASH Clubs have been formed. The project target is 80 schools. Twenty￾five schools were slated to have new wells, but none of these projects have been 
completed. The project need to intensify its efforts in this area.
93
5
2
72
15
2
10
1
82
17
1
97
3
75
15
2
8
83
17
0 20 40 60 80 100
Corrugated metal (Zinc)
No roof
Thatch
Concrete polished wall
Mud polished
Concrete unpolished wall
Thatch
Mud unpolished
Other
Concrete floor
Earth floor
Other
Roof Wall Floor
% of schools
Baseline
Midline
44
Toilet facilities also need attention. Of the schools observed, only 65 percent had a 
functioning toilet and only 72 percent of those toilets were clean (Figure 19). 
Furthermore, only 18 percent of toilets had a handwashing station with water and soap 
available. 
MGD 1.3.4 Increased pupil enrollment 
Pupil enrollment is tracked via project monitoring. As of June 2016, there were 28,463
pupils enrolled in project schools of which 48 percent were girls. A year later, in June 
2017, school enrolment was 32,499 reflecting a 14 percent increase from the previous 
year. Enrolment of girls increase by 15 percent and in 2017 they made up 49 percent 
of enrolment. 
Factors affecting reading performance
Overall, we can conclude that the MGD III has made progress towards the achievement 
of the strategic objective on improved literacy. The data shows remarkable increases in 
the percentages of children who can read, albeit from a very low base. Logistic 
regression analysis was conducted to predict whether a student will be able to read 
based on individual (gender, age, member of reading club) and school characteristics
(whether the school is approved and percent of teachers qualified). The result is 
50
63
72
77
47
52
61 65
72
0
10
20
30
40
50
60
70
80
90
100
Safe
water
Working
water
source
Working
Toilet
Toilet
clean
Toilet
Hand
Washing
Facility
Percent(%) of schools
Midline Baseline
45
presented in Table 6 and the dependent variable is whether the student can read and 
understand grade-level text (YES=1 if they got 4 of 5 comprehension questions correct, 
0 otherwise). 
Logistic regression 
 Num. of observations = 950
LR chi2(9) = 161.68
Log likelihood = -552.87652 Prob > chi2 = 0.0000
Pseudo R2 = 0.1276
Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]
Male 1.474* 0.217 2.630 0.009 1.104 1.968
Age 6-11 yrs. 0.748 0.130 -1.670 0.094 0.533 1.051
Member of Reading Club 1.470* 0.238 2.380 0.017 1.071 2.018
School is approved by MEST 3.136* 0.692 5.180 0.000 2.035 4.833
Percent of qualified teachers 0.150* 0.056 -5.090 0.000 0.072 0.311
Chiefdom
Mongo 4.806* 1.300 5.800 0.000 2.828 8.168
Neini 5.974* 1.542 6.930 0.000 3.602 9.906
Neya 0.637 0.188 -1.530 0.127 0.358 1.136
Dembelia 1.256 0.398 0.720 0.472 0.675 2.337
_cons 0.245 0.074 -4.640 0.000 0.136 0.444
The logistic analysis presents the odds ratios, and we can derive the following from the 
table:
• The odds that a student can read is statistically significantly impacted by their 
gender, membership in a reading club, attending an approved school, the share 
of qualified teachers in the school, and the chiefdom in which they live.
• The odds of being able to read for boys is 1.5 times that of girls
• The odds of being able to read for a child who is in a reading club is 1.5 times 
that of non-members
• The odds of being able to read is 3 times more for a student attending a school 
approved by MEST
46
• The odds of being able to read is less if you are attending a school with a higher 
percentage of qualified teachers
• The odds of being able to read for students living in Mongo is 4.8 times more 
than Sulima and for children living in Neini the odds are almost 6 times more
than a child in Sulima.
• The age variable is not statistically significant
While the above analysis does not provide evidence of causation, the model is robust 
enough to inform our understanding of what predicts if a child will be able to read. The 
result that is most confounding is that related to the negative effect of the share of 
qualified teachers. One possible explanation is that despite the larger share of 
qualified teachers in a school, the qualified teachers are not usually assigned to teach 
the lower grades. This combined with the fact that teachers selected to participate in 
the distance education program come from schools with a lower percentage of 
qualified teachers (based on the project’s selection criteria) would explain this result. It 
could also mean that there is a problem with this variable. This issue warrants further 
investigation, but removing this variable does not change the influence of the other 
variables in the equation. 
MGD S.O.2 Increased use of good health and dietary practices
Under this strategic objective, MGD III trained the school communities on good 
practices related to health and dietary practices. Students and teachers were trained on 
good health and hygiene practices and cooks were trained on techniques for safe food 
preparation and storage. The project was also to provide better access to clean water 
through the development of wells. Finally, the project collaborated with partners such 
as Helen Keller International and UNICEF in the provision of deworming services to 
schools. Below we consider what effect these activities had in the increasing the use of 
good health and hygiene practices
MGD 2.1 Improved knowledge of health and hygiene practices
Pupils were asked about their knowledge of good health and hygiene, and their 
responses are shown in Figure 20. The most mentioned practices at the midline review 
were: washing hands with soap and water (79 percent) and washing daily (69 percent).
The least mentioned practices were putting trash into bins (10 percent); hair braiding 
(10 percent) and keeping toilets clean (15 percent). Pupils fared much worse during the 
47
midline than in the baseline in recalling their knowledge of basic hygiene: at midline 
only 12 percent of pupils could name 6 of 10 practices, compared to 49 percent of 
pupils at baseline. This drop in scores is unexpected and may be a problem with the 
data collection or coding.
MGD 2.2 Increased knowledge of safe food preparation and storage practices
The team surveyed cooks to determine their knowledge and skills in safe food 
preparation and storage practices. A total of 160 cooks were interviewed (about 3 per 
school), and of those 86 percent reported having been trained on safe food 
preparation and storage practices. Those who were not trained were either new cooks 
to the school, were busy at the time of training, or were not selected to attend the 
training. 
They were then asked to list examples of good practices in safe food preparation and 
storage practices, and these were checked against main messages delivered in the 
training delivered by CRS. Their responses are shown in Figure 21. The most commonly 
mentioned practices were: washing food before cooking (83 percent), washing hands 
with clean water and soap (77 percent), washing utensils and dishes with soap and 
27
29
36
44
45
51
66
75
77
79
25
10
10
15
21
23
28
49
37
78
69
0 10 20 30 40 50 60 70 80 90 100
Eat good food
Put trash into a bin
Hair braiding
Keep toilets clean and provided with soap/ash
Cut and keep nails clean
Drink safe water that has been treated and…
Brush teeth twice a day
Keep environment clean and safe
Wear clean clothes
Wash hands with soap and water before…
Wash our bodies daily
% of children
Midline
Baseline
48
water and drying them (73 percent). The least mentioned practices were: using tongs 
or spoons (and not bare hands) to handle cooked food (27 percent) and putting clean 
utensils on a platform (29 percent). Forty-six percent of cooks could recall at least 5 of 
10 safe food preparation and storage behaviors as compared to 60 percent at the 
baseline. 
When asked about challenges encountered as a cook, the main challenges identified 
were: lack of payment, lack of utensils, and lack of water. A few cooks mentioned lack 
of condiments, lack of a suitable kitchen, needing to take care of younger children, and 
lack of community support as other challenges. The project expects the cooks to 
volunteer their time and for communities to provide utensils and condiments to the 
schools. 
46
21
61
46
43
64
79
79
79
77
27
29
42
42
44
52
59
61
73
77
83
0 20 40 60 80 100
Use tongs or spoons to handle cooked food
Put cleaned utensils on a platform
Wear kitchen apron or apparel
Store the food in a clean place
Storage should be free from flies
Kitchen should be free from animals
Cover the food after dishing
Sweep the kitchen or where food is
prepared
Wash utensils and dishes with clean water
and soap and dry them
Wash hands with clean water and soap
Wash the food items before cooking
Percent (%) of Cooks Interviewed
Midline Baseline
49
During the school observation, enumerators checked the kitchen and food storage 
facilities and reported on these. Ninety-percent of schools had a kitchen (see Figure 
22). Of the schools with kitchens, 39 percent had shelves, 37 percent had a 
handwashing facility, and 35 percent had observed soap. It is important that WASH 
situation be improved. Most of the schools had aprons and rack/pallets. 
A similar 90 percent of the schools observed had a storage facility. Of those with 
storage facilities, 20 percent had ventilated blocks with mesh, 70 percent without 
mesh and 10 percent had no ventilated blocks. Seventy-two percent of store rooms 
were secured by a metal door with locks, 17 percent by a metal door without a lock, 
and 11 percent had no metal door.
MGD 2.3 Increased knowledge of nutrition
Pupils were not asked about their knowledge of nutrition, but they were asked about 
the types of foods they ate on the day before the survey. As Figure 23 shows, almost 
all children ate some form of grains (rice, cassava, etc.); 31 percent ate meat or fish; 
and 54 percent ate vegetables. Compared to the baseline, fewer pupils reported eating 
90
39
74
37
87
35
0
10
20
30
40
50
60
70
80
90
100
Has Kitchen Shelves Rack/Pallet Handwashing
facility
Aprons
available
Soap
available and
seen
Percent (%) of schools
50
all different food groups, save for fruits. Twenty-seven percent of pupils had a 
minimum acceptable diet that included at least 4 of the 7 identified food groups, which 
is much lower than the forty-two percent at baseline. This result is not trustworthy and 
may have happened because of difficulties with understanding the question, not 
remembering the foods consumed, or enumerators wrongly classifying the food. Most 
pupils reported receiving the CRS meal, which consists of lentils, vegetable oils, and 
rice. This should mean that at least three of the below food groups (legumes, grains, 
vegetables) should be near 100 percent.
MGD 2.4 Increased access to clean water and sanitation services 
As mentioned above in Section MGD 1.3.3, 48 percent of schools do not have access to 
a safe water source and 35 percent do not have access to a toilet. Soap is especially 
4
16
29
57
54
31
94
5
23
44
54
69
71
99
0 10 20 30 40 50 60 70 80 90 100
Eggs
Dairy
Legumes and Nuts
Fruits
Vegetables
Meat or Fish
Grains
% of Pupils
Baseline Midline
51
rare both in the kitchen and toilet facilities. In only 8 schools (13 percent) did the 
enumerator see a soap in the kitchen area.
MGD 2.5 Increased access to preventative health services
The GOSL, with support from its donor partners, provide preventative health services, 
including deworming and preventative health services to school children. The Ministry 
of Health and Sanitation carries out the deworming exercise twice in a school year. CRS 
gives collaborative support to the process in terms of sensitization to communities and 
logistical support to enable medical staff to reach FFE-supported schools. However, 43 
percent of head teachers reported that their school did not receive deworming services
during the current academic year. 
MGD 2.6 Increased access to requisite food preparation and storage tools and equipment
Cooks were asked about the numbers of food preparation and storage tools and 
equipment they had in their school. Their responses are summarized in Table 7. There 
are large differences amongst schools in terms of the quantities of kitchen equipment 
and serving utensils available. The baseline report recommended that CRS establish 
minimum standards for kitchen equipment and utensils and to ensure that all schools 
meet the minimum standards. However, per project staff, CRS does not provide 
utensils and it is up to the communities to provide the necessary utensils. During 
interviews with cooks, the lack of utensils and kitchen equipment came up as one of 
the challenges faced. 
Baseline Midline
Avg. Min Max Avg. Min Max
No. Big Pots 1 0 2 2 0 3
Big Bowls for cooked food 0 0 4 1 0 4
Big bowls for sauce 2 0 80 0 0 8
Wooden spoon 1 0 15 2 0 30
Scooping spoon 5 0 98 3 0 55
Serving Plates 29 0 160 29 0 250
Spoons 26 0 135 25 0 250
Buckets 1 0 5 2 0 10
Towels 3 0 80 2 0 6
52
Baseline Midline
Avg. Min Max Avg. Min Max
Cups 25 0 130 24 0 250
Knives 0 0 15 0 0 3
Mortar 0 0 1 0 0 1
Mortar pestle 0 0 0 0 2
Progress on Key Performance Indicators
As part of the project monitoring of outputs and outcomes, the MGD III project has 
selected several indicators that will be used to monitor progress. During the baseline 
survey in 2016, information was collected on the starting point of several of these 
indicators and these are shown in Table 8. The values of these indicators were updated 
in June 2017 with data from CRS monitoring and the midline survey.
Significant progress has been made on many of the key indicators, especially on the 
literacy measures. The percentage of children who can read and understand grade 
level text quadrupled since the baseline, and the project is close to meeting its end of 
project targets. 
53
)
Percent of pupils who, by the end of 
two grades of schooling, demonstrate 
that they can read and understand 
the meaning of grade level text 
(girls/boys)
8.3% 7.9% 32% 39% 30% 30%
Percent of class 2 pupils 
who scored at least 80 
percent in a reading 
comprehension test
Percent of pupils in target schools 
who are identified as attentive or very 
attentive during class/instruction 
(girls/boys)
58% 66% 80%
This is the percent of 
classrooms in which pupils 
were attentive or moderately 
attentive
Percent of pupils in target schools 
who indicate that they are hungry or 
very hungry during the school days 
(boys/girls)
92% 95% 26% 30% 0% 0%
At midline, this question 
was only asked of pupils 
who had been provided a 
meal
Percent of school
-age children 
receiving a minimum acceptable diet 
(boys/girls)
55% 52% 28% 27% 100% 100%
Percent of school
-aged 
children who report 
consuming at least 4 out of 
7 food groups on the day 
prior to the survey, as 
defined by the MGD 
Indicator Handbook. This
indicator is unreliable as we 
would expect different 
results given the school 
meal
Percent of pupils in target schools 
who achieve a passing score on a test 
of good health and hygiene practices 
(boys/girls) 
– 6 out of 10 practices
55% 44% 11% 13% 70% 70%
Percent of pupils who could 
name at least 6 of 10 good 
hygiene practices. Unclear 
whether this is a problem in 
the data or reflects the 
actual situation. 
Percent of food preparers at target 
schools who achieve a passing score 
on a test of safe food preparation and 
storage 
71% 46% 75%
Percent of cooks who could 
name at least 5 of 10 safe 
food preparation practices
54
)
Percent of participants demonstrating 
at least 3 out of 5 social/life skills 78% 80% CRS Indicator Performance 
Tracking Table
Percent of teachers in target schools 
who demonstrate use of new and 
quality teaching techniques or tools 
because of USDA assistance 
27% 54% 100%
Percent of teachers 
observed that demonstrated 
proficiency in practice of 
checking for pupils 
understanding during the 
lesson, a foundation of DTM
Percent of pupils in target schools 
who attend school at least 80% of 
scheduled school days per school 
year
67% 66% 100% 100% 100% 100% CRS Indicator Performance 
Tracking Table
Percent of administrators (head 
teachers) who demonstrate at least 
one new administration skill
72% 98%
Percent of head teachers 
who keep CRS attendance 
register
Number of teachers in target schools 
who demonstrate use of new and 
quality teaching techniques or tools 
as a result of USDA assistance
0 701 701 CRS Indicator Performance 
Tracking Table
Number of teachers trained or 
certified as a result of USDA 
assistance (MGD I-6)
0 296 300 CRS Indicator Performance 
Tracking Table
Number of school administrators and 
officials (head teachers) in target 
schools who demonstrate use of new 
teaching technique or tool as a result 
of USDA assistance
0 192 192 CRS Indicator Performance 
Tracking Table
55
)
Number of SILC groups sharing out 
and starting a new cycle 0 42 60 CRS Indicator Performance 
Tracking Table
Percent of Parent-Teacher 
Associations or similar “school” 
governance structures contributing to 
their school as a result of USDA 
assistance
98% 100% CRS Indicator Performance 
Tracking Table
Number of WASH Clubs formed 0 55 130 CRS Indicator Performance 
Tracking Table
Number of pupils receiving 
deworming medication(s) 28,463 32,042 30,919 CRS Indicator Performance 
Tracking Table
Number/% of schools using an 
improved water source (MGD I-19) 0 0 15 There is a delay in the 
construction of wells
Number of latrines constructed or 
rehabilitated with USDA assistance 
(MGD Indicator 7)
0 52 62
CRS Indicator Performance 
Tracking Table 
Construction of latrines is at 
70% completion. Should be 
100% completion by 
October 1. 
Number of kitchens or storerooms 
constructed or rehabilitated with 
USDA assistance (MGD Indicator 7)
0 34 37
CRS Indicator Performance 
Tracking Table 
Number of individuals benefiting 
directly from USDA funded 
intervention (male/female)
35,008 39,105 36,577 CRS Indicator Performance 
Tracking Table
Number of individuals benefiting 
indirectly from USDA funded 
intervention (MGD Indicator 26)
42,476 49,011 38,392 CRS Indicator Performance 
Tracking Table
56
)
Number of social assistance 
beneficiaries participating in 
productive safety nets with USDA 
assistance (male/female) (MGD I-17)
35,310 39,076 36,548 CRS Indicator Performance 
Tracking Table
Number of daily school meals 
provided to school age children as a 
result of USDA assistance (MGD I-15)
0 3,883,887 5,351,299 CRS Indicator Performance 
Tracking Table
Number of pupils regularly (80%) 
attending USDA supported school 
(male/female)
9,169 9,776 15,647 16,541 14,573 15,083 CRS Indicator Performance 
Tracking Table
Number of text books and other 
teaching and learning materials 
provided as a result of USDA 
assistance
0 181,317 107,333 CRS Indicator Performance 
Tracking Table
Number of schools with improved 
early grade literacy instructional 
materials
0 192 192 CRS Indicator Performance 
Tracking Table
Number of community meetings held 
(CRS I-33) 0 57 52 CRS Indicator Performance 
Tracking Table
Number of literacy coaches trained in 
Diagnostic Teaching Methods as a 
result of USDA assistance (CRS I-6)
0 25 25 CRS Indicator Performance 
Tracking Table
Number of teachers trained in 
Diagnostic Teaching Methods as a 
result of USDA assistance 
(male/female) (CRS I-7)
0 701 701 CRS Indicator Performance 
Tracking Table
57
)
Number of school buildings or 
classrooms constructed or 
rehabilitated as a result of USDA 
assistance. (MGD I-7)
0 19 52 CRS Indicator Performance
Tracking Table
Number of classrooms in project 
schools receiving new school 
furniture (CRS I-18)
0 145 240 CRS Indicator Performance 
Tracking Table
Number of pupils enrolled in school 
receiving USDA assistance (male/ 
female)
13,715 14,748 15,797 16,702 14,602 15,128 CRS Indicator Performance 
Tracking Table
58
RELEVANCE: TO WHAT EXTENT DOES THE PROJECT CONFORM TO THE 
NEEDS AND PRIORITIES OF TARGET GROUPS AND THE POLICIES OF THE 
COUNTRY AND DONOR?
The MGD III project, to a very large extent, conforms to the needs and priorities of the 
country, communities and USDA. At the policy level, providing school meals for all 
children, improving WASH facilities, and improving learning are all priorities of the 
GOSL and were part of the Government’s Post-Ebola recovery program. The MGD III 
project therefore contributes to this national endeavor.
The interviews and focus group discussions at the community and schools also confirm 
that the MGD III project meets the needs of the communities. School feeding, provision 
of WASH facilities, and the training of teachers and cooks were very beneficial to the 
communities. 
EFFICIENCY: HOW COULD INTEGRATION OF THE TWO STRATEGIC 
OBJECTIVES BE ENHANCED OR FURTHER LEVERAGED TO DEEPEN 
POSITIVE PROGRAM RESULTS? 
Integrating health and education program within schools is an effective and efficient 
way to improve the overall well-being of children in the community. School-aged 
children are at risk of ill-health throughout their school years, which will impact on 
their ability to attend school regularly and on their academic achievement. 
The MGD III project supports the healthy development of children in many ways 
including: providing nutritional meals, improving on WASH facilities, coordinating with 
other programs that provide deworming and Vitamin A supplementation, and training 
pupils and teachers on good WASH practices. 
While these are all good activities, it is clear from the responses from interviews and 
FGDs that the needs in this area are large especially with regards to the provision of 
clean water and toilet facilities. Most schools still lack access to potable water and 
soap in schools is a rarity. CRS can further deepen its work with head teachers and 
other partners to help improve the WASH facilities.
59
Another way to integrate the two programs is to develop reading materials for pupils 
that incorporate key health messages that the project wants to promote in a fun and 
engaging way. A final recommendation from key stakeholders is to engage whole 
communities in this process so that pupils are encouraged to practice good health and 
hygiene outside of the school.
SUSTAINABILITY: TO WHAT EXTENT WILL THE BENEFITS OF THE 
PROJECT CONTINUE AFTER THE END OF THE PROJECT LIFE? 
During the FGDs, the communities remain optimistic that they will be able to continue 
with various aspects of the MGD III project even after the project ends, including 
feeding, maintaining of school infrastructure, and continuing to send their children to
schools. They suggest numerous strategies through which they would be able to do 
this including: establishing school farms/gardens, encouraging parents to make 
financial contributions, using SILC proceeds to continue to support the school, 
embarking on self-help projects, and obeying the laws to send their children to school. 
While the optimism is encouraging, it is unclear whether they will be able to do all of 
this without the support of MEST and other partners. The communities are poor, and 
they struggle to provide even the condiments necessary to prepare the meals. It is 
possible that the MEST will be able to expand its National School Feeding programme 
to include Koinadugu district, but the funding for that programme is also uncertain. 
Some recommendations from key stakeholders interviewed were that CRS should 
develop and begin an exit and sustainability plan.7 We also recommend intensifying 
efforts to receive MEST approval for schools. Over 75 percent of schools in this sample 
are unapproved. Of the unapproved schools, only 40 percent had even applied for
approval. The reasons given for not applying included: not knowing or understanding 
 
7 One respondent mentioned learning from the CAUSE Canada exit strategy. This is another organization that 
provided school meals in some schools in Koinadugu
60
the procedures, the head teacher being new to the school, and the school not meeting 
the minimum standards. These are all things that are actionable, and since CRS now 
has an Advocacy Manager, it is recommended that the field agents, with support of the 
Advocacy Manager, work with struggling schools to help them prepare their 
applications. If there are schools who do not meet the minimum standards, then CRS 
could target their resources to help these schools meet the minimum standards rather 
than spreading them thin across many schools. Schools that are approved are much 
more likely to get resources (financial, trained teachers, teaching and learning 
materials, etc.) from the government. 
The overall conclusion is that MGD III has made good progress especially in strategic 
objective 1. We see marked increase in percentages of children who can read, which we 
haven’t seen in earlier evaluations. These improvements were likely influenced by the 
increased resources dedicated to the literacy aspects of the program, most notably, the 
introduction of literacy coaches. On the other hand, progress in strategic objective 2 
was limited. There were delays in the provision of improved WASH infrastructure 
(latrines, wells, and handwashing stations) and knowledge of good hygiene and safe 
food preparation practices appear diminished. Below are recommendations to 
strengthen implementation during the final year of the project.
IMPROVED LITERACY SKILLS 
• Teacher training: while teacher practice has improved overall, there are still 
many individuals who have not mastered basic skills. The project should find a 
way of identifying these teachers and providing them with targeted support. 
• Use of teaching and learning materials: while the project has distributed 
thousands of textbooks, these are not being used effectively in classrooms. In 
most classrooms, only the teacher has a text. Carry out a quick study on the use 
of TLMs and provide explicit instructions to teachers on how to use these 
textbooks in their lessons.
• Class participation: students, especially girls, should be encouraged to speak 
more and ask questions in class. 
61
• School approval: work closely with schools and MEST to ensure that schools 
receive approval by MEST. There are still head teachers reporting that they do 
not know or understand the process for applying for school approval.
• Community engagement: engage parents and communities in support reading 
(and not just the feeding program). There are several ways that they can do this 
including reading to their children, telling stories, and providing space and time 
for children to read after school. 
• Girls and gender: given the findings that girls are underperforming boys, issues 
of gender in teaching practice and reading should be explicitly addressed.
Ensure that teachers are encouraging and paying attention to both boys and 
girls, that materials are gender-sensitive, and that training focuses on this issue. 
This is especially important given that the 90 percent of the teachers are male. 
IMPROVED HEALTH AND HYGIENE PRACTICES
• Reinforce key hygiene messages throughout the curriculum and in extra￾curricular activities
• The project should have a plan for training new cooks who enter the project 
midway and others who were not selected for the training. This might involve 
those who were trained passing on the information to others or MGD staff doing 
cluster trainings during the year.
• Schools should have supplies that enable children to practice what they have 
learned about good hygiene practices. For example, handwashing stations with 
water and soap should be present in all schools and kitchens and there should 
be bins for trash. 
• Keeping school facilities clean should be a communal responsibility, and CRS 
should work with schools to develop strategies to maintain good hygiene 
practices. Only about 57 percent of the schools in the sample have a WASH club.
• Minimum standards for food preparation and storage tools and equipment 
should be developed together with the communities, and MGD III should ensure 
that schools meet the standard
• Over 40 percent of schools have not received the required preventative health 
services (e.g. deworming and vitamin A supplementation). MGD III does not 
itself provide these services, but it should advocate for supported schools with 
the appropriate agencies. 
• Provide reading materials for children that reinforce health messages in a fun 
and engaging way
• Prioritize infrastructure and supplies that impact student safety and well-being. 
These include schools that have no working safe source of water, no working 
62
toilet facilities, no hand-washing stations, and no roof. Soap (or other cleaning 
agents) and water should be mandatory in all school toilets and kitchen. 
• Intensify efforts to complete the construction of wells and latrines
OTHER RECOMMENDATIONS
• Develop a sustainability plan with communities, local government and MEST to 
prepare for the end of the MGD III program.
• Collect further monitoring data on the coaching process that includes records of 
the number of times teachers are visited. If a structured observation tool is 
used, then that data should be analysed with a view to identifying areas for 
further refresher training. 
63
Montrose International. (2014). Consultancy for the Design and Conduct of National 
Early Grade Assessments in Literacy and Numeracy (EGRA/EGMA) for Primary 
School Pupils in Sierra Leone.
Statistics Sierra Leone. (2016). 2016. Freetown: Statistics Sierra Leone. Retrieved 10 21, 
2017, from https://www.statistics.sl/wp-content/uploads/2017/01/final￾results_-2015_population_and_housing_census.pdf
The Economist Intelligence Unit. (2017, October 21). Sierra Leone Food Security Index.
Retrieved from 
http://foodsecurityindex.eiu.com/Country/Details#Sierra%20Leone
United Nations Development Programme UNDP. (2016). Human Development Report 
2016: Human Development for Everyone. New York: UNDP. Retrieved October 
21, 2017, from 
http://hdr.undp.org/sites/default/files/2016_human_development_report.pdf
64
The following annexes are included in a separate document. 
A. Scope of Work
B. Field data collection instruments
C. Responses to recommendations from the baseline survey