Classroom for students who are deaf or hard of hearing. Credit: Whole-of-Project Performance Evaluation of the Reading for Success Project, 2019
FINAL PERFORMANCE EVALUATION 
OF THE NATIONAL PROGRAM FOR 
READING – MOROCCO
APRIL 2023
This publication was produced at the request of the United States Agency for International 
Development. It was prepared independently by Mark Lynd (Team Lead), Drew Schmenner, Fiona 
Eichinger, and Erica Wang, School-to-School International.
1
FINAL PERFORMANCE
EVALUATION OF THE
NATIONAL PROGRAM
FOR READING –
MOROCCO
FINAL REPORT
TASKING NO. USAID, 7200AA19M00017
Published: April 2023
DISCLAIMER
The author’s views expressed in this publication do not necessarily reflect the views of the United States 
Agency for International Development or the United States Government.
2
ABSTRACT
This report evaluates the performance of USAID/Morocco’s National Program for Reading (NPR) 
project. It evaluates the extent to which NPR achieved its activity targets and outcomes and built on the 
findings from the previous whole-of-project Reading for Success (RFS) evaluation. Research questions 
focus on six topics: learning outcomes; capacity building in assessment data and use of data; scaling; 
parental and community engagement; private sector engagement; and equity. Evaluators from School-to￾School International completed an initial desk review of NPR documents, took part in an evaluation 
design workshop, analyzed relevant NPR quantitative data, and conducted primary qualitative data 
collection, including key informant interviews and focus group discussions in November and December 
2022. Evaluators found that NPR fostered a fruitful relationship between partners and positively changed 
classrooms, with various factors associated with improved learning outcomes; encountered notable 
obstacles during implementation, including high teacher turnover rates and the COVID-19 pandemic, 
which negatively impacted students’ well-being and socioemotional skills and was associated with 
learning loss that possibly had a disproportionate impact on younger students; conducted a successful 
capacity-building program on assessment data and use of data, with room for increased use and sharing 
of EGRA results; resulted in the Ministry of National Education, Preschool, and Sports providing a solid 
foundation for scaling of the new curriculum, with the pressing need for intensive teacher training and 
coordinated monitoring; increased parental support for children’s reading, with parents eager to be 
more involved due to limited engagement; collaborated with the private sector, including the 
development of supplementary reading materials and improved coordination with publishers, who 
created enhanced materials yet need urging to continue revisions; and helped all groups learn equitably, 
according to perceptions, with concerns about needing to better reach students with special needs.
3
TABLE OF CONTENTS
Abstract 2
List of Tables 5
List of Figures 5
Acronyms 6
1. Executive Summary 7
1.1 Evaluation Purpose 7
1.2 Project Background 7
1.3 Evaluation Questions 7
1.4 Methodology 8
1.5 Findings, Conclusions, and Recommendations 8
1.5.1 EQ1: Learning Outcomes 8
1.5.2 EQ2: Capacity Building in Assessment Data and Use of Data 9
1.5.3 EQ3: Scaling 10
1.5.4 EQ4: Parental and Community Engagement 10
1.5.5 EQ5: Private Sector Engagement 11
1.5.6 EQ6: Equity 11
2. Evaluation Purpose and Questions 13
3. Project Background 14
4. Evaluation Methods & Limitations 19
4.1 Tool Design and Review Process 20
4.2 Sampling 21
4.3 Respondents 22
4.4 Research Tools 24
4.4.1 Key Informant Interviews (KIIs) 24
4.4.2 Focus Group Discussions (FGDs) 24
4.4.3 Secondary Data Sources 24
4.5 Data Collection, Analysis, and Validation 25
4.5.1 Data Collector Training 25
4.5.2 Qualitative Fieldwork 25
4.5.3 Data Analysis 26
4.5.4 Data Validation 28
4.6 Limitations 28
5. Findings, Conclusions, and Recommendations 30
5.1 Learning Outcomes 30
5.1.1 Findings 30
4
5.1.2 Conclusions 34
5.1.3 Recommendations 35
5.2 Capacity Building in Assessment Data and Use of Data 36
5.2.1 Findings 36
5.2.2 Conclusions 39
5.2.3 Recommendations 39
5.3 Scaling 40
5.3.1 Findings 40
5.3.2 Conclusions 42
5.3.3 Recommendations 42
5.4 Parental and Community Engagement 43
5.4.1 Findings 43
5.4.2 Conclusions 44
5.4.3 Recommendations 45
5.5 Private Sector Engagement 45
5.5.1 Findings 46
5.5.2 Conclusions 48
5.5.3 Recommendations 49
5.5.6. Equity 49
5.6.1 Findings 49
5.6.2 Conclusions 54
5.6.3 Recommendations 55
Annexes 56
Annex I: Evaluation Statement of Work 57
Annex II: NPR Performance Indicators and Annual Targets 66
Annex III: Disclosure of Conflicts of Interest 69
Annex IV: Summary of Evaluation Team Members 70
Annex V: Sources of Information 73
Annex VI: Consent Forms 77
Annex VII: Data Collection Instruments 85
Annex VIII: Getting to Answers Matrix 147
Annex IX: Protocol for USAID Attendance at Parent FGDs 162
Annex X: Summary of Analyses of Secondary Data 164
Annex XI: Early Grade Reading Assessment (EGRA) Matrix 180
Annex XII: Findings, Conclusions, and Recommendations Matrix 189
Annex XIII: Data Validation Workshop Discussion Notes 199
5
LIST OF TABLES
Table 1: Main NPR Performance Indicators and Annual Targets.........................................................................17
Table 2: Schedule for NPR Piloting and Scale Up ....................................................................................................17
Table 3: Respondents by Targeted Number, Tool, Subgroup, and Sampling or Selection Criteria ............22
Table 4: Tools and Variables Included in NPR Final Evaluation Secondary EGRA Analysis..........................27
Table 5: EGRA Results in ORF by Grade and Gender...........................................................................................32
Table 6: Correlations between strong performance on ORF in EGRA and contextual factors by grade, 
sex and location ...............................................................................................................................................................52
Table 7: Correlations between strong reading performance in ORF in SRM study and contextual factors 
by grade, sex, and location ............................................................................................................................................53
Table 8: Correlations between strong reading performance in reading comprehension in Arablc language 
curriculum effectiveness study and contextual factors by grade, sex, and location ........................................53
Table 9: Evaluation Matrix .............................................................................................................................................61
Table 10: Primary Data Collection Tools and Sample Targets.............................................................................62
Table 11: NPR Performance Indicators and Annual Targets.................................................................................66
Table 12: KIIs and FGDs Conducted in the NPR Final Evaluation.......................................................................73
LIST OF FIGURES
Figure 1: RFS-NPR Results Framework......................................................................................................................16
Figure 2: Triangulation Using Parallel Combinations for Mixed-Methods Evaluation .....................................20
Figure 3: Has this school receives SRMs from the MOE?......................................................................................34
Figure 4: RFS-NPR Results Framework......................................................................................................................57
Figure 5: Has this school received SRMs from the MOE? .................................................................................. 166
Figure 6: Why should you teach letter/syllable sounds to children? They need to sound out (decode) in 
order to read................................................................................................................................................................. 167
Figure 7: Have you ever received training (in person or remotely) on how to teach reading?................ 167
Figure 8: Have you organized outreach to help parents to use SRMs or 3asafeer at home?.................... 169
Figure 9: Can teachers access 3asafeer at the school with school equipment?............................................. 169
Figure 10: May grade 2 and 4 students borrow books to take home to use?............................................... 170
Figure 11: Are SRMs appropriate for the reading skill level in your class?..................................................... 170
Figure 12: Most students are reading the text quietly (classroom observation)........................................... 173
Figure 13: Girls and boys were asked questions equally (classroom observation)....................................... 174
Figure 14: Girls and boys were given equal access to learning materials (classroom observation) ......... 174
Figure 15: To what extent are you able to stick to your weekly schedule (teacher interview)............... 175
Figure 16: Have you been able to cover the Arabic curricula for your grade? (teacher interview)......... 175
Figure 17: Teacher guides students to read individually and in a quiet voice (classroom observation).. 176
Figure 18: Most students are deducing relationships between sentences (classroom observation)........ 176
Figure 19: Most students are deducing relationships between sentences (classroom observation)........ 177
Figure 20: Teacher taught lesson in standard Arabic (classroom observation)............................................. 177
Figure 21: Girls and boys were given equal opportunities to participate in activities (classroom 
observation)................................................................................................................................................................... 178
Figure 22: How often do you teach reading? (teacher interview).................................................................... 178
Figure 23: Are you able to complete the reading lesson plan in the time allotted for reading? (teacher 
interview)........................................................................................................................................................................ 179
6
ACRONYMS
ADS Automated Directives System
AREF Académie Régionale de l’Éducation et de la Formation 
(Regional Academy for Education and Training)
CNEE Centre National de l’Evaluation et des Examens
(National Center for Evaluation and Examinations)
CNIPE Centre National des Innovations Pédagogiques et de l’Expérimentation 
(National Center for Pedagogical Innovations and Experimentation)
CRMEF Centre Régional des Métiers de l’Éducation et de la Formation 
(Regional Teacher Training Center)
DEC Development Experience Clearinghouse
EGRA Early Grade Reading Assessment
FA Formative Assessment
FGD Focus Group Discussion
FY Fiscal Year
KII Key Informant Interview
MoE Ministry of National Education, Preschool, and Sports
MOOC Massive Open Online Course
MSL Moroccan Sign Language
NPR National Program for Reading
RFS Reading for Success
SRM Supplementary Reading Materials
SSE Small-Scale Experimentation
STS School-to-School International 
USAID U.S. Agency for International Development
7
1. EXECUTIVE SUMMARY
1.1 EVALUATION PURPOSE
The performance evaluation of the National Program for Reading (NPR) project will assist the United 
States Agency for International Development’s Mission in Morocco (USAID/Morocco) in gathering 
information for two purposes—to evaluate the extent to which NPR has achieved its activity targets and 
outcomes; and build on the findings from the previous whole-of-project Reading for Success (RFS) 
evaluation conducted in 2019 that were related to project implementation, sustainability, and the role of 
the Ministry of National Education, Preschool, and Sports (MoE).
1.2 PROJECT BACKGROUND
In 2017, the Reading for Success-National Program for Reading (RFS-NPR) was awarded to Creative 
Associates. A $19.3M activity, NPR was a follow-on project to USAID/Morocco’s Reading for 
Success/Small-Scale Experimentation (SSE) Activity, which ended in 2018. NPR ended in October 2022 
and built upon SSE’s use of a pilot Arabic reading curriculum for grades 1 through 4 and included a 
strong set of reading supplementary and instructional materials. NPR’s overarching goal was to help 
“students in target regions read fluently and with comprehension by the end of the second school year.” 
To achieve this goal, NPR managed the following three components, which were linked to the 
intermediate results outlined in the activity framework: 1) Improved reading and writing materials used 
in primary schools; 2) Improved reading and writing instruction in primary schools; and 3) Learning and 
assessment system improved.
1.3 EVALUATION QUESTIONS
Six evaluation questions and related sub-questions guided the NPR evaluation. The research team 
finalized the questions with input from USAID/Morocco, MoE officials, and implementing partners. The 
first and sixth evaluation questions include analysis of findings by gender and data disaggregated by 
gender.
1. Learning Outcomes: How has the new NPR reading curriculum and teacher instruction 
package affected learning outcomes in target schools? What role has the COVID-19 pandemic 
played in learning outcomes?
a. Delivery: How well did NPR deliver on the development and testing of the revised 
reading curriculum and teacher instruction interventions?
b. Capacity Building: To what extent was NPR successful in strengthening the MoE’s 
ability to develop and deliver an education reform in primary grades?
2. Capacity Building in Assessment Data and Use of Data: How well did NPR support the 
MoE’s ability to use Early Grade Reading Assessment (EGRA) data and classroom formative 
assessments to support adaptive management and improved learning outcomes at both the 
classroom and central level? How well was the capacity-building conducted with the MoE’s 
examination and assessment departments in terms of their ability to conduct EGRAs? In the case 
8
the MoE uses EGRA data to support adaptive management and improve learning outcomes, how 
will this data be used, and how could it be facilitated?
3. Scaling: How effectively did the MoE use and scale up the new curricula introduced by NPR? 
What are the issues around managing the curricula effectively at a national scale?
4. Parental and Community Engagement: What are the best practices and lessons learned 
from NPR for engaging communities and parents in reading programs?
5. Private Sector Engagement: What are lessons learned from NPR related to private sector 
engagement in reading programs?
6. Equity: What are lessons learned from NPR related to education access, quality, or results 
among urban vs. rural, boy vs. girl students, or geographic regions? How will these lessons 
learned inform future education programming and investment decisions?
1.4 METHODOLOGY
For this study, evaluators used a mixed-methods approach, including both qualitative and secondary 
quantitative analysis. During the design phase, evaluators mapped each item from each tool to relevant 
evaluation questions to ensure that sufficient information was elicited from all appropriate respondents. 
Data collection for the evaluation’s qualitative phase began with personnel training in Rabat from 
November 7–12, 2022, followed by four weeks of data collection across six regions and 12 provinces in 
Morocco. Data were gathered from a variety of NPR stakeholder respondent groups to gain an 
understanding of multiple perspectives across a variety of levels—national, regional, provincial, and 
school/community—and to enable triangulation of data. Qualitative data collection methods included 
key informant interviews (KIIs) and focus group discussions (FGDs). 
To conduct secondary analysis, the team gathered NPR project data, including Early Grade Reading 
Assessment (EGRA) results and data from a supplementary reading materials (SRM) study and an Arabic 
language curriculum effectiveness study. Triangulation used qualitative and quantitative methods to gain a 
more holistic understanding of NPR and mitigate potential bias in data analysis. 
1.5 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 
1.5.1 EQ1: Learning Outcomes
Findings
Many respondents described how, thanks to the introduction of a curriculum with a learner-centered 
approach, students enjoyed reading more and improved in other aspects of Arabic, including speaking, 
writing, and listening. Obstacles that respondents reported impeding NPR implementation included 
COVID, which not only affected learning outcomes but also students’ well-being and socio-emotional 
skills, as well as high teacher turnover rates. While grade 1–3 students’ ORF scores generally peaked at 
baseline, declined at midline, and partially recovered at endline, grade 4 students’ ORF scores increased 
both from baseline to midline and midline to endline. These declines in ORF scores from baseline to 
midline illustrate COVID-19’s association with learning loss. The ORF results highlight another notable 
trend—girls generally outperformed boys. In addition, other secondary analyses of NPR data conducted 
as part of this evaluation highlighted various factors associated with improved reading outcomes.
When asked about NPR’s impact on the MoE’s ability to develop and deliver education reform, 
stakeholders characterized the MoE’s relationship with partners as collaborative and participatory, with 
9
the MoE empowered to sustain the new curriculum, learning materials, and massive open online course 
(MOOC) content by being involved in all stages of their production and rollout.
Conclusions
Key NPR funders and implementers developed a fruitful working relationship with the central MoE and 
helped them develop the technical capacity needed to develop, adapt, and sustain the new curriculum. 
Their work culminated in a project that has positively changed classrooms, based on respondents’ 
descriptions of how teachers and students have engaged with the new curriculum and related activities 
in experimental schools. The COVID-19 pandemic and hybrid schooling had an effect on learning loss, 
however, possibly with a disproportionate impact on younger students; in contrast, grade 4 students’ 
ORF scores increased from baseline to endline, possibly due to longer exposure to NPR. Teacher 
turnover was also perceived as negatively impacting learning outcomes. Secondary analyses of data from 
EGRA, SRM, and Arabic language curriculum effectiveness studies found that stronger reading 
performance correlated with several variables. 
Recommendations
1. Continue to research NPR cohorts to understand the project’s cumulative impact.
2. Determine strategies to reduce teacher turnover.
3. Apply a learner-centered approach to other aspects of school curricula.
4. Document internal best practices of NPR partnership.
1.5.2 EQ2: Capacity Building in Assessment Data and Use of Data
Findings
NPR organized a series of capacity-building workshops for MoE personnel on evaluation that were 
viewed as successful, though some respondents raised concerns about possibility of applying results in 
the field. Although implementers, donors, and central MoE respondents made decisions to varying 
degrees based on data, the extent to which EGRA results and other data were shared with others was 
limited. The majority of respondents reported not having received the formative assessment (FA) toolkit 
developed to address the needs of struggling learners. Of the few teachers who were trained and 
received print copies, some gave positive feedback about the FA toolkit, but other teachers who had 
received training in FA found it difficult to implement.
Conclusions
Although the assessment capacity building program was viewed as successful, some respondents raised 
concerns about possibility of applying results in the field. In addition, data sharing has been limited, 
making it difficult to use data to improve practice, and as noted by senior MoE officials, the use of EGRA 
and other results remained limited. Still, on numerous occasions, NPR, including the MoE, practiced 
adaptive management. As for the FA toolkit, few respondents said they had received it or knew much 
about its contents. Those who reported having it found it difficult to implement.
Recommendations
1. Wherever possible, conduct capacity-building activities in person.
2. Continue to build and sustain the central MoE’s capacity in data use and assessment, including 
support to schedule periodic EGRAs as part of the official school calendar and to develop a 
guide about managing a reform agenda and implementing international research.
3. Design training to address the needs of evaluators, pedagogical experts, and administrators, and 
articulate roles in data collection, analysis, interpretation, and use for each of these.
4. Focus assessment training on the application of results with an eye to improving materials and 
classroom practices.
5. Standardize data sharing protocols—for example, the use of a publicly accessible platform.
10
6. Amplify efforts to distribute and train education support personnel and teachers in the use of 
FA materials and methods.
1.5.3 EQ3: Scaling
Findings
After NPR introduced the new curriculum and materials in its experimental schools, the MoE scaled up 
the program to all schools nationwide. Project stakeholders noted how NPR helped the MoE gain the 
technical knowledge by producing and distributing the curriculum, materials, and MOOC content 
nationwide; anticipating pushback on the new curriculum and, over time, building buy-in from regions 
and provinces; and assisting with decision-making concerning design changes. All these efforts had 
respondents confident that the MoE was in the driver’s seat to continue scaling of the curriculum after 
NPR closed in 2022. Despite the MoE being in command, respondents explained it was premature to 
pass any definitive judgment on the effectiveness of scaling, primarily due to training programs still being 
rolled out across the country. The process of training teachers on the new curriculum nationwide 
remains ongoing and varied, based on inspectors’ responses. 
Conclusions
The MoE provided a solid foundation for schools nationwide to adopt the new Arabic curriculum 
introduced by NPR, demonstrated effective management by distributing the curriculum and other 
materials, and displayed the technical expertise it developed by producing the curriculum and materials. 
The most pressing issue around managing the curricula at scale concerns the need for intensive teacher 
training and coordinated monitoring—not only of these training initiatives but also of classroom teaching 
practices. 
Recommendations
1. Institutionalize the effective practices adopted as part of NPR scaling, so future scaling efforts for 
new curricula do not have to reinvent the wheel.
2. Equip and empower school directors and/or inspectors to conduct classroom monitoring and 
ensure teachers are applying what they learned from MOOCs.
3. Devise a national strategic plan to assess the national rollout of training initiatives and ensure 
inspectors implement the training toolkit with fidelity.
1.5.4 EQ4: Parental and Community Engagement
Findings
Despite being asked how NPR engaged with communities and parents, most respondents talked about 
general activities involving parents or difficulties in engaging parents and communities in children’s 
reading. While respondents credited the project’s SRM activities with increasing parental engagement, 
most parents described a lack of engagement with their children’s school. Respondents also frequently 
shared perceptions that parents have difficulty in supporting their children’s learning due to being 
uninformed or unable to follow the guidance or communicate with their children.
Conclusions
Increased parental support for children’s reading and the SRM activity were strengths of NPR’s parental 
engagement, with SRM study data showing how boys and rural students benefited most when outreach 
was conducted, as well as all Grade 2 students benefiting when school directors checked in with 
parents. Further, when school directors reached out to parents, some reading scores were higher. 
Despite parents and communities not being sufficiently engaged, they showed eagerness to be more 
involved. Still, many respondents viewed parents as unable to support their children’s reading.
11
Recommendations
1. Conduct campaigns to support and motivate students at home.
2. Establish a system of communication between teachers and parents.
3. Implement strategies that help parents overcome challenges in supporting their children’s 
reading, as well as increasing their awareness about the general importance of reading. 
1.5.5 EQ5: Private Sector Engagement
Findings
NPR engaged with the private sector for two major initiatives—producing new student textbooks and 
teacher guides with publishing companies and creating SRMs with 3asafeer, a Dubai-based educational 
technology company. Despite the enhanced coordination between the MoE and publishers, which led to 
improved materials, respondents reported that the publishers could improve the quality of textbooks 
and teacher guides. The relationship between 3asafeer, the MoE, and NPR was constructive and 
successful, according to respondents, with all parties sharing the same goal of wanting to provide 
students with enjoyable reading materials. Although the SRM activity was largely successful with 
distributing and using print books, respondents explained that some teachers, students, and families had 
difficulty accessing and using the 3asafeer digital platform. Secondary data analyses indicated that grade 4 
students who had access to the 3asafeer digital platform, as well as who read stories from the library, 
had better reading outcomes. 
Conclusions
Although improved collaboration between the MoE and textbook publishers—including training and 
better communication—enhanced the process of producing learning materials, the MoE needs to 
continue to urge private textbook publishers to revise materials to align fully with the new curriculum. 
While NPR’s collaboration with 3asafeer highlights the value of engaging with partners outside of 
Morocco to access low-cost educational resources, the lack of engagement with the 3asafeer digital 
platform underscores the obstacles to implementing an EdTech intervention in Morocco. The benefits of 
increasing access are apparent, as secondary analyses suggest that having access to 3asafeer may make a 
difference in reading performance.
Recommendations
1. Strategize how to continue to engage with publishers and raise their technical capacity to revise 
and improve learning materials to address acknowledged gaps.
2. Provide teachers with paper copies of their guides.
3. Explore additional ways that the MoE could partner with the private sector, especially internet 
providers, and expand access to materials.
4. To investigate low usage rates of 3asafeer, conduct a case study in experimental schools to 
determine how to overcome obstacles that led to that.
1.5.6 EQ6: Equity
Findings
Respondents indicated that NPR ensured its design, materials, and practices were equitable from the 
outset, and many respondents said that the new curriculum helps all groups learn equitably. As for 
gender difference, many respondents said there was none; however, in several cases, they noted that 
girls perform better than boys (but did not attribute this disparity to NPR). According to EGRA results, 
girls performed better than boys. The SRM study data indicated that some practices helped boys and 
girls differently. Of all subgroups, respondents most frequently mentioned students with special needs. 
To be sure, NPR made numerous efforts to support students who were deaf or hard of hearing, 
including developing distance and hybrid modalities during the pandemic. Yet, respondents also made it 
clear that students with special needs were sometimes ignored.
12
Conclusions
Respondents perceived that the new curriculum helps all groups learn equitably—girls and boys, children 
from different language groups, children from urban and rural areas, and children with disabilities—and 
that girls are better performers than boys. Of all subgroups included in secondary data analyses, boys 
and rural students were found to benefit from contextual factors the most. Of all subgroups, 
respondents most frequently expressed concerns about students with special needs. 
Recommendations
1. Conduct more research to determine how different groups interact with new materials and 
respond to new strategies introduced in the new curriculum.
2. Determine feasible initiatives to address issues that students with special needs face. Possible 
ideas include, whenever possible and feasible, reserving a separate class for students with special 
needs; providing specialists to support children with special needs, especially for more severe 
cases; and providing a curriculum with its own schedule, including materials (“kit”) tailored to 
the circumstances of students with different kinds of special needs.
13
2. EVALUATION PURPOSE AND 
QUESTIONS
The performance evaluation of the National Program for Reading (NPR) project will assist the United 
States Agency for International Development’s Mission in Morocco (USAID/Morocco) in gathering 
information for two purposes:
• To evaluate the extent to which NPR has achieved its activity targets and outcomes; and
• To build on the findings from the previous whole-of-project Reading for Success (RFS) 
evaluation conducted in 2019 that were related to project implementation, sustainability, and 
the role of the Ministry of National Education, Preschool, and Sports (MoE).
The evaluation results will be shared with the USAID/Morocco Education Office in Rabat, as well as 
potentially broader USAID audiences, including USAID/Washington and other missions, to highlight best 
practices and lessons learned and inform upper management of the results achieved by USAID/Morocco 
in basic education and early grade reading. In addition, relevant results will be shared with Government 
of Morocco counterparts and other donors to further promote improving basic education in Morocco 
through enhanced reading instruction in early grades.
Six evaluation questions and their related sub-questions guided the NPR evaluation. The research team 
finalized these questions in collaboration with USAID/Morocco, MoE officials, and implementing 
partners. The first and sixth evaluation questions include analysis of findings by gender and data 
disaggregated by gender. 
1. Learning Outcomes: How has the new NPR reading curriculum and teacher instruction 
package affected learning outcomes in target schools? What role has the COVID-19 pandemic 
played in learning outcomes?
a. Delivery: How well did NPR deliver on the development and testing of the revised 
reading curriculum and teacher instruction interventions?
b. Capacity Building: To what extent was NPR successful in strengthening the MoE’s 
ability to develop and deliver an education reform in primary grades?
2. Capacity Building in Assessment Data and Use of Data: How well did NPR support the 
MoE’s ability to use Early Grade Reading Assessment (EGRA) data and classroom formative 
assessments to support adaptive management and improved learning outcomes at both the 
classroom and central levels? How well was the capacity-building conducted with the MoE’s 
examination and assessment departments in terms of their ability to conduct EGRAs? In the case 
the MoE uses EGRA data to support adaptive management and improve learning outcomes, how 
will this data be used, and how could it be facilitated?
3. Scaling: How effectively did the MoE use and scale up the new curricula introduced by NPR? 
What are the issues around managing the curricula effectively at a national scale?
4. Parental and Community Engagement: What are the best practices and lessons learned 
from NPR for engaging communities and parents in reading programs?
5. Private Sector Engagement: What are lessons learned from NPR related to private sector 
14
engagement in reading programs?
6. Equity: What are lessons learned from NPR related to education access, quality, or results 
among urban vs. rural, boy vs. girl students, or geographic regions? How will these lessons 
learned inform future education programming and investment decisions?
3. PROJECT BACKGROUND
USAID/Morocco conceived the Reading For Success project to address Development Objective 3 (DO): 
Enhanced educational development for children at the primary level. This objective was detailed in the 
Mission’s Country Development Cooperation Strategy released in 2013.1 Citing persistent inequities in 
education and lagging literacy levels, USAID/Morocco shifted its focus toward primary education after 
targeting other educational domains—including middle schools, teacher training institutes, and job 
training for disadvantaged youth—during the preceding ten years. As USAID/Morocco laid out in its 
CDCS, the foundational skill of literacy provides a springboard for students’ future success in the 
classroom, workforce, and broader civil society. The CDCS emphasized the importance of synergies 
with the Moroccan government’s education agenda, including the comprehensive reform initiative Vision 
2030.2
As a whole, RFS aimed to address lagging reading outcomes among Moroccan primary-grade students. 
While reading achievement has improved in recent years, it remains low. Moroccan students’ reading 
scores on the Progress in International Reading Literacy Study (PIRLS) were statistically significantly 
higher in 2016 than in 2011, showing improvement, but Morocco ranked 48th out of 50 countries in 
2016 and scored well below international benchmarks.3 In addition to PIRLS results, prior research 
supported by USAID underscores the opportunity for improvement. In 2011, an Early Grade Reading 
Assessment (EGRA) administered in the Doukkala Abda region revealed that only one out of three 
grade 2 and 3 students could read a second-grade level text well enough to comprehend it.4
Additionally, 33 percent of grade 2 students and 17 percent of grade 3 students could not read any 
words of a Modern Standard Arabic (MSA) text. Factors cited as contributing to poor performance on 
the EGRA included a lack of teacher preparation and reading materials, as well as a mismatch between 
the language of instruction at schools and the predominant languages are spoken at home.
RFS was developed in two phases. The first phase focused on building evidence for implementing early￾grade reading activities through research. It also featured a series of workshops and meetings with the 
MoE and peers from other countries in the region. All the knowledge and evidence gained during this 
first phase informed the second phase of activity design and implementation, including RFS-Small Scale 
Experimentation (RFS-SSE), RFS-NPR, and RFS-Improving Deaf Children’s Reading Through Technology 
(RFS-IDCRT).5 The RFS-Human and Institutional Capacity Development (RFS-HICD) assessment of the 
MoE was conducted during the initial research phase. 
1 USAID-Morocco, USAID-Morocco Country Development Cooperation Strategy—2013-2017, (2013).
2 Kingdom of Morocco Ministry of National Education, Vocational Training, Higher Education, and Scientific Research, Vision 2030, accessed 
March 2023, https://www.men.gov.ma/Fr/Documents/ Vision_ strateg_CSEF16004fr.pdf
3 Mullis, I.V.S., Martin, M.O., Foy, P., & Hooper, M. PIRLS 2016 International Results in Reading. (Chestnut Hill, Mass.: Trends in International 
Mathematics and Science Study & PIRLS International Study Center, Lynch School of Education, Boston College, and International Association 
for the Evaluation of Educational Achievement, 2017).
4 RTI International, Student Performance in Reading and Mathematics, Pedagogic Practice, and School Management in Doukkala Abda, Morocco, (2012).
5 NORC at the University of Chicago, Desk Review for the Whole-of-Project Performance Evaluation of the Reading for Success Project. 
(2019). https://pdf.usaid.gov/pdf_docs/PA00W4KT.pdf
15
USAID/Morocco’s early grade reading research began in 2011 with the EGRA administered in the 
Doukkala Abda region. It then commissioned a series of studies in 2013 on early grade reading in 
Morocco, including analyses of curricula and texts used in formal and non-formal education; pre-service 
teacher training; teachers’ attitudes, beliefs, and practices toward teaching reading in Arabic; and the 
textbook sector. The research concluded in 2015 with a peer review of the MoE’s Arabic curriculum 
and the RFS-HICD activity.
In December 2013, a series of workshops and meetings kicked off with a workshop led by All Children 
Reading for participants from the region. Country teams convened in Rabat to discuss the theory and 
practice of early-grade reading and sketch out initial ideas for reform in Arabic reading instruction. 
USAID/Morocco subsequently organized two workshops on reading instruction in Arabic in November 
2014 for MoE staff—one for high-level officials and the second for a larger group of practitioners. 
USAID/Morocco also facilitated a series of visits and discussions between the MoE and their 
counterparts in Egypt. This allowed Moroccan officials to learn about the Arabic reading reforms that 
had been implemented in Egypt.
The RFS’s second phase of design and implementation started with the launch of the RFS-SSE activity in 
2015. RFS-SSE aimed to improve early-grade reading outcomes by developing and piloting new Arabic 
reading lessons and materials for grades 1 and 2 and implementing summer enrichment reading 
programs. The activity’s lessons and materials ushered in a new phonics-based pedagogical approach to 
teaching reading in Morocco, representing a shift from the historically dominant whole language method.6
The MoE selected four regions for piloting the new lessons and materials—Sous Massa, Rabat-Kenitra, 
Orient, and Fes-Meknes—based on geographic, cultural, and socioeconomic factors. The RFS-SSE 
activity closed in March 2018.7
RFS-NPR built on RFS-SSE’s work at a greater scale. In 2017, the fully funded $19,272,436.00 follow-on 
project was awarded to Creative Associates. The project ran from May 19, 2017, to October 18, 2022,
under award number AID-608-C-17-00001. Ending in October 2022, NPR built upon RFS-SSE’s use of a 
pilot Arabic reading curriculum for grades 1 through 4 and included a robust set of supplementary 
reading and instructional materials. NPR’s overarching goal was to help “students in target regions read 
fluently and with comprehension by the end of the second school year.” To achieve this goal, NPR 
managed the following three components, which were linked to the intermediate results outlined in the 
activity framework:
1) Improved reading and writing materials used in primary schools;
2) Improved reading and writing instruction in primary schools; and
3) Learning and assessment system improved.
The NPR results framework showing each of the intermediate results (IRs), sub-intermediate results 
(SIRs), and linkages is shown graphically in Figure 1. NPR revised this framework in 2019 to reflect 
changes in its scope; discontinued SIRs are marked in red, revised ones in yellow, and new ones in 
green. 
Centered on parental and community engagement through the establishment of reading enrichment 
activities, the third IR was removed from NPR’s scope of work in 2019 due to the MoE’s request for 
NPR to support the development of a new Arabic reading curriculum and materials for grades 5 and 6, 
6 RTI International and Al Akhawayn University, Research on Reading in Morocco: Analysis of the National Education Curriculum and 
Textbooks—Final Report (Component 1)—Part 1 (Curriculum Analysis), (2015), p. 18.
7 Chemonics International Inc., Reading for Success-Small Scale Experimentation 2015-2018. (2018). https://chemonics.com/wp￾content/uploads/2018/07/Morocco_RFS-SSE_Final_Report_v2.pdf.
16
as reflected in SIR 1.5 and 1.7, respectively. Prior to the third IR being removed, NPR supported the 
implementation of children’s summer reading programs by awarding grants to three Moroccan non￾governmental organizations which implemented the activities. After NPR modified its results framework, 
it shifted its focus from parental engagement to its SRM activity.
Figure 1: RFS-NPR Results Framework
The project’s main indicators linked to the results framework are shown in Table 1, as well as the 
project’s 5-year targets and cumulative actuals as reported in NPR’s end-of-project final report. NPR’s 
complete list of indicators is available in Annex II.
17
Table 1: Main NPR Performance Indicators and Annual Targets
# Performance Indicator 5-Year 
Targets
Cumulative 
Actuals Against 
5-Year Targets
1. Percentage of learners who demonstrate reading 
fluency and comprehension of grade level text at 
the end of grade 2 (ES. 1-1)
6% increase 
over baseline 
(Baseline: 
38.7%)
36.0%
3. Number of learners reached in reading programs 
at the primary level (ES. 1-3, ES. 1-5)
4,432,229 4,552,752
(2,185,309 girls and 
2,367,443 boys)
5. Number of primary or secondary textbooks and 
other teaching and learning materials provided 
with USG assistance (ES. 1-49)
4,556,369 9,510,321
12. Number of education administrators and officials 
who complete professional development activities 
with USG assistance (ES. 1-12)
1,639 2,409
13. Number of primary school educators who 
complete professional development activities on 
implementing evidence-based reading instruction 
with USG assistance (ES. 1-6, ES. 1-7)
93,393 120,790 (66,434 
women and 54,356 
men)
25. Number of primary school classrooms that 
receive a complete set of essential reading 
instructional materials with USG assistance (ES. 1-
11)
138,814 134,825 (55,279 
urban, 79,546 
rural)
26. Number of public and private schools receiving 
USG assistance (ES. 1-50)
24,358 24,790 (6,913 
urban, 17,877 
rural)
A number of implementation milestones regarding NPR’s curriculum development and textbook 
production, teacher and inspector training, distance learning content, supplementary reading materials, 
MSL video lessons, the formative assessment toolkit, and MOE capacity building can be identified. 
NPR initially planned to develop, validate, and scale new Arabic language curriculum and materials for 
grades 1 to 4. It followed the schedule in Table 2 for piloting them in the project’s 90 experimental 
schools and then scaling up nationwide.
Table 2: Schedule for NPR Piloting and Scale Up
Phase Description 2017–18 2018–19 2019–20
Experimentation 90 experimental schools 
in 4 regions G1-2 G3-4 None
Scaling Nationwide expansion G1 G2 G1-4
In June 2019, NPR expanded its scope to grades 5 and 6 as well. NPR and the MoE validated the revised 
grade 5 and 6 curriculum as a part of a final review and validation of the entire grade 1 to 6 curriculum 
in May and June 2021. 
18
To complement its curriculum and materials development, NPR supported the MoE in training activities 
for inspectors and teachers. Teachers and principals at NPR’s 90 experimental schools and the 
inspectors overseeing them received various trainings on specific topics during the 2018-19, 2019-20, 
and 2020-21 school years. 
NPR also collaborated with the MOE’s National Center for Pedagogical Innovations and 
Experimentation (CNIPE, or Centre National des Innovations Pédagogiques et de l’Expérimentation) to 
develop a series of MOOCs to train teachers nationwide on the new Arabic language curriculum. NPR’s 
efforts on MOOC development began in 2019 by drafting a framework with the MoE and conducting 
initial training workshops. In 2020, NPR formalized an agreement with World Learning to provide 
technical support and training for MOOC development. Work on MOOC content ramped up as the 
MoE officially launched the new Arabic language curriculum at the start of the 2021-22 school year. In 
the summer of 2021, the MoE piloted MOOC II and III courses nationwide with 345 teachers after 
receiving technical assistance from NPR on the pilot design. NPR also introduced MOOC II and III 
courses to teachers and principals in its 90 experimental schools, with an initial training in September 
2021, followed by refresher trainings in December 2021 and February 2022. The MOOC I course on 
the new curriculum was finalized in the spring of 2022.
The supplementary reading material (SRM) activity began in November 2019 when NPR and MoE 
counterparts reviewed, selected, and validated 100 stories from the digital library of 3asafeer, an 
educational technology company based in Dubai. After editing the stories, NPR printed 23,950 books in 
early 2020 and delivered them to 63 experimental schools by the start of the 2020-21 school year. NPR 
also enabled students and teachers at these schools to access the books digitally through the 3asafeer 
platform. NPR distributed SRMs to the remaining 26 experimental schools at the start of the 2021-22 
school year.
In response to the COVID-19 pandemic and national school closures, the MoE recorded and broadcast 
lessons to students across the country. In May 2020, the MoE asked NPR to assist in adapting these 
lessons for students who are deaf or hard of hearing with MSL interpretation. NPR worked with several 
program coordinators, two interpreters, and technical staff to produce the videos for grades 1 to 6 for a 
variety of subjects, including Arabic, math, French, and other grades 1 to 6. After filming lessons from 
the former grade 6 curriculum in May 2020, the team then recorded lessons sequentially, starting with 
grade 1 from July to November 2020 and concluding with grade 6 in September and October 2021. 
Schools began reopening to in-person learning in October 2021. 
NPR also developed and finalized a Formative Assessment (FA) toolkit, including an FA guide, student 
remediation guide, and a teacher’s guide to unit tests for grades 1 to 4. After NPR incorporated 
feedback from inspectors and teacher trainers in its experimental regions, the MoE validated the guide in 
March 2021. This was used for a training on FA for grade 1 to 3 teachers in NPR’s experimental schools 
that month and a national one-day training on FA for 104 inspectors in July. Following a revalidation 
workshop with regional inspectors in early 2022, NPR submitted the completed FA toolkit to the MoE 
and printed 200 copies for distribution to AREFs between April and June 2022.
Over the course of the project, NPR also organized capacity-building workshops for MoE personnel 
between 2019 and 2022. Topics included national evaluation and assessment design and planning; 
benchmarking; and EGRA data analysis and interpretation, among others. 
19
4. EVALUATION METHODS & 
LIMITATIONS
To answer the six primary NPR performance evaluation questions and related sub-questions, the 
evaluators utilized a mixed-methods research approach, including both qualitative analysis and secondary 
quantitative analysis. 
The NPR performance evaluation took place from August 12, 2022 to April 14, 2023. The evaluation 
team consisted of Dr. Mark Lynd and Dr. Carol da Silva as Team Leaders and Education Evaluation 
Experts. Technical Management was provided by Matthew Murray and Fiona Eichinger, with additional 
support from Qualitative Analyst & Technical Writer Drew Schmenner, Senior Statistician Dr. Michel 
Rousseau, Gender Lead and Senior Analyst Dr. Melanie Phillips, Program Coordinator Erica Wang, and 
Editor and Graphic Designer Laura Zasoski. 
This evaluation began with a literature review to provide the context, background, and understanding 
needed for effective qualitative fieldwork. The literature review supported the design of more in-depth 
and nuanced subsequent evaluation components. 
During the design phase, evaluators clearly mapped each item on each tool to relevant evaluation 
questions and sub-questions in Excel. This ensured that sufficient information was elicited from all 
appropriate respondents to answer the evaluation questions. These detailed tool maps were used to 
produce versions of each tool in Word that were structured for appropriate interviews and discussion 
flow with respondents. 
The evaluation team conducted data collector training in November 2022, followed by qualitative 
fieldwork over approximately four weeks in November and December 2022 in collaboration with 
contracted data collectors. Data was gathered from a variety of NPR stakeholder respondent groups to 
gain an understanding of multiple perspectives and experiences across a variety of levels—national, 
regional, provincial, and school/community—and to enable triangulation of data and findings. The 
qualitative data collection methods included semi-structured key informant interviews (KIIs) and focus 
group discussions (FGDs). 
In addition, the team gathered existing NPR project data as possible, including Early Grade Reading 
Assessment (EGRA) results, to conduct secondary analysis. Triangulation – an approach described in 
Figure 2 – used qualitative and quantitative methods to gain a more holistic understanding of NPR and 
mitigate potential bias in data analysis. 
Data was analyzed from December 2022 to March 2023, culminating in a Data Validation Workshop 
with key stakeholders on March 8, 2023 in Rabat. 
20
Figure 2: Triangulation Using Parallel Combinations for Mixed-Methods Evaluation
The qualitative and quantitative analyses proposed in this design report will employ a “parallel combinations” analytical approach, 
illustrated below, through which each method is applied separately from the other and integrated or triangulated following independent 
analysis:
As appropriate, a “sequential combinations” approach will also be employed, enabling findings from qualitative fieldwork to inform the 
design and implementation of secondary quantitative analysis. To support this purpose, qualitative data collection tools will be structured 
so that the research team can capture information from key stakeholders on relevant and available secondary data sources that could 
further shed light on the evaluation question(s) being examined.
4.1 TOOL DESIGN AND REVIEW PROCESS
This evaluation began with a literature review to provide the context, background, and understanding 
needed for effective qualitative fieldwork. The literature review supported the design of more in-depth 
and nuanced subsequent evaluation components. The evaluation team studied existing documents from 
USAID/Morocco, NPR, and the MoE, as well as external documents pertaining to early-grade reading in 
Morocco. This review provided an overview of education and early-grade reading in Morocco, including 
past activities by USAID and the MoE. It also spoke to the research that informed the NPR design and 
the NPR survey, including the overall results framework, project indicators, and related activities.
During the design phase, evaluators clearly mapped each item on each tool in Excel to relevant 
evaluation questions and sub-questions to ensure sufficient information was elicited from all appropriate 
respondents to answer the evaluation questions. These detailed tool maps were used to produce 
versions of each tool in Word that were structured for appropriate interviews and discussion flow with 
respondents. This matrix can be referenced in Annex VIII.
Each tool was aligned with one specific type of respondent: 
USAID/Morocco officials;
• NPR implementers and subcontractors;
• MoE officials; 
• Non-MoE government officials; 
• Moroccan Sign Language activity participants;
• Private sector personnel; and
• School-level respondents, including school directors, teachers, and parents.
21
While the same tool was utilized for subgroups within a respondent type, additional sub-sections or 
targeted items were incorporated for specific subgroups, where relevant. For example, MoE officials at 
different levels had specific sections of the core MoE tool removed based on their level of involvement 
in the project. Similarly, implementing partner staff were asked different sets of questions depending on 
the activity or activities in which they participated. In addition, respondents in experimental and scale-up 
regions were asked different sets of questions that aligned with the exposure they would have had to 
NPR. The tools for regional and provincial MoE officials and school-level respondents, including school 
directors, teachers, and parents, varied slightly between experimental and scale-up regions because 
respondents in scale-up areas were assumed not to have as much background knowledge about NPR. 
The evaluation team hosted a planning and tools pre-testing workshop in October 2022 in Rabat to 
build consensus and buy-in around the evaluation design. Discussion topics included the evaluation 
questions, sample of respondents, and the use of results. As a result of the workshop, the sample size 
was increased from eight to 12 schools. During subsequent consultations with central MoE officials, it 
was recommended that certain non-MoE officials and certain directorates within the MoE be dropped 
from the sample due to limited involvement with NPR, including one respondent each from the Ministry 
of Solidarity, Women, Family, and Social Development; the Ministry of Health; the MoE’s Directorate of 
Cooperation and Teaching in Private Schools; and the MoE’s Directorate of Strategy, Statistics, and 
Planning. Instead, the evaluation team aimed to conduct interviews with individuals involved with 
different project activities, including the development of supplementary reading materials, the 
production of textbooks, and the communication of MoE initiatives. Tools were also revised and refined 
based on the discussions and participants’ perspectives on local contexts. The workshop included key 
representatives from the MoE, USAID/Morocco, and NPR implementing partners. 
To facilitate time management, each finalized tool marked priority questions that data collectors first 
asked respondents, followed by any additional questions that enumerators could pose within the time 
limit set for each KII (60 minutes) and FGD (90 minutes). The final data collection instruments can be 
viewed in Annex VII.
The tools included informed consent scripts and forms, which can be referenced in Annex VI. Data 
collectors followed clearly defined protocols for respondents’ participation in the evaluation, including 
an outline of risks and benefits, the right to decline participation, and confidentiality. Institutional Review 
Board (IRB) approval was obtained from the Moroccan Association for Research and Ethics prior to 
beginning this fieldwork. 
4.2 SAMPLING
This qualitative-based evaluation used purposive sampling. Respondents participating in this evaluation 
included school-level respondents; MoE and non-MoE government officials at the provincial, regional, 
and national levels; personnel of USAID/Morocco and NPR implementing partners; and personnel from 
other NPR partner organizations.
Data were collected in the same six regions as the whole-of-project performance evaluation of the RFS 
project in 2019—the four regions with NPR experimental schools and two regions in which the Ministry 
of National Education, Preschool, and Sports (MoE) scaled up the new curricula and materials that NPR 
introduced. The same provinces in each region were also selected. Since only one province per scale-up 
region was selected for the whole-of-project performance evaluation in 2019, an additional province per 
scale-up region was selected to represent one urban and one rural province in each region. 
22
Evaluators visited two schools apiece in each region, equating to one per selected province and totaling 
12 schools overall. Evaluators utilized existing secondary data from NPR to purposively select the 
schools to ensure varied representation of a range of school characteristics such as urbanicity,8 the 
predominant mother language of students,9 school performance,10 school population size, and 
socioeconomic status. The qualitative data gathered from school-based sources does not aim for 
representativeness or generalizability, as a quantitative study would. Experimental schools were selected 
based on NPR EGRA data to include schools with both low and high EGRA score averages. Urbanicity 
and the percentage of students who spoke Darija at home were included as additional criteria to ensure 
that both urban and rural schools were represented, as well as schools in which the majority of students 
and less than the majority of students spoke Darija at home. As EGRA data was not available for scale￾up schools, it was not possible to select schools based on low and high average scores. STS’s Field 
Manager selected these schools with input from appropriate regional MoE officials. Urbanicity was the 
primary factor in scale-up school selection to ensure classrooms in rural and urban areas were both 
visited. 
4.3 RESPONDENTS
Enumerators generally targeted the same set of respondents at the school, provincial, and regional 
levels, as shown in Table 3. At each of the 12 schools, enumerators aimed to conduct separate focus 
group discussions (FGDs) with parents and teachers, as well as a key informant interview (KII) with the 
school director. In addition, certain MoE officials were targeted for KIIs in each of the 12 provinces, 
including the provincial director and an inspector, and in each of the six regions, including the AREF 
director and a provincial Arabic language teacher trainer. Enumerators also aimed to conduct KIIs with 
at least two CMREF directors. 
At the level of the national MoE, enumerators planned to conduct KIIs with officials in the Curriculum 
Directorate, Centre National de l’Evaluation et des Examens (CNEE), Centre National des Innovations 
Pédagogiques et de l’Expérimentation (CNIPE), and individuals involved with different project activities, 
including the development of supplementary reading materials, production of textbooks, and 
communication of MoE initiatives. Enumerators also aimed to conduct KIIs with two interpreters 
involved with NPR’s Moroccan Sign Language (MSL) activity and several individuals in the private 
textbook industry who collaborated with NPR and the MoE on the development of new materials.
STS also planned to conduct some KIIs remotely with respondents, including officials from USAID, 
Creative Associates International, and 3asafeer, an educational technology company that provided 
supplementary reading materials to NPR.
Table 3: Respondents by Targeted Number, Tool, Subgroup, and Sampling or Selection Criteria
# Respondents 
(N) Tool Subgroups Sampling/Selection Criteria
1
USAID Morocco 
Officials 
(at least 2)
USAID KII 
Guide N/A
At least 2 KIIs with 
USAID/Morocco Education 
officials
8 For example, the demographic criteria may distinguish between urban, peri-urban, rural, and deep rural schools. 9 Such as Darija, the Moroccan Arabic dialect, or Amazigh languages, such as Tarifit, Tashelhit and Central Atlas Tamazight.
10 Ideally, “high performing” and “low performing” schools may be determined by existing EMIS, EGRA data, or other RFS-SSE or RFS-NPR 
project data and/or experience.
23
# Respondents 
(N) Tool Subgroups Sampling/Selection Criteria
2
USAID 
Implementing 
Partners (2)
USAID 
Implementin
g Partners
KII Guide
N/A Two KIIs with senior NPR staff 
MoE Central Office 
Directors 
Four KIIs with Central Office 
Directors, including officials from 
CNIPE, CNEE, and the Curricula 
Directorate 
AREF/Regional 
Directors
Six KIIs with AREF regional 
directors and their assistants
3 MoE government 
officials (42)
MoE Official 
KII Guide Provincial Directors 12 KIIs with provincial directors
Inspectors/Provincial 
Coordinators
12 KIIs with inspectors/provincial 
coordinators
CMREF Directors Two KIIs with CMREF directors 
Province-level Arabic 
teacher trainers from 
CMREF
Six KIIs with provincial-level 
Arabic teacher trainers
4 Non-MoE 
stakeholders (2)
MSL Activity 
participants
Representatives of 
marginalized groups
Two KIIs with respondents 
identified by MoE or 
USAID/Morocco who 
participated in the efforts to 
provide MSL interpretation for 
distance-learning video lessons
5 Private sector
Private 
Sector Textbook publishers Two KIIs with relevant personnel 
identified by MoE or USAID
personnel (3) Personnel 
KII Guide 3asafeer (educational 
technology company)
One KII with relevant personnel 
identified by MoE or USAID
6 School directors 
(12)
School 
Director
KII Guide 
N/A 12 KIIs with primary school 
directors in NPR schools
7 Teachers (min. 48; 
max. 96)11
Teacher 
FGD Guide N/A
12 FGDs with primary school 
teachers (grade 1-6) in NPR 
schools
11 This table assumes four to eight teachers per FGD. While each FGD with teachers will aim to include eight participants, the total number of 
teachers who participate depends on the number of teachers available within each sampled primary school and targeted grade level. For 
example, if a school only includes one teacher (or section) for each targeted grade level (i.e., grades 1–6), the maximum number of teachers 
available to participate in the FGD would be six. In cases where more than eight grade 1–6 teachers are available, the criteria for selecting 
teacher participants prioritized including one teacher from each targeted grade level as well as gender balance or participants (i.e., the inclusion 
of an equal number of male and female teachers if available). If needed, additional selection criteria may be added, such as prioritizing teachers 
who have been at the school for at least four years, as they would have had greater exposure to the RFS activity.
24
# Respondents 
(N) Tool Subgroups Sampling/Selection Criteria
8
Parents 
(min. 72; max. 
120)12
Parent FGD 
Guide N/A
12 FGDs with parents of primary 
school students (grades 1-6) who 
are active in school management 
committees, parent 
organizations, or other school 
groups
4.4 RESEARCH TOOLS
4.4.1 Key Informant Interviews (KIIs)
The evaluators aimed to conduct 63 KIIs, including four KIIs with USAID/Morocco and its implementing 
partners, 47 KIIs with Moroccan government officials and other stakeholders, and 12 KIIs with school 
directors. Additional details on respondent types and subgroups anticipated for participation within each 
of the broader categories—as well as associated targets—are detailed in Table 3. The total number and 
types of KIIs conducted by the conclusion of data collection are detailed in Annex V. 
4.4.2 Focus Group Discussions (FGDs)
A total of 24 FGDs were aimed to be conducted across twelve school visits. As noted in Table 3, the 
evaluation team was to complete 12 FGDs with primary school teachers (one FGD per school visit) and 
12 FGDs with parents of primary school students (one FGD per school visit). FGDs aimed for gender￾balanced participation where possible. For example, half of the parents of students in early primary 
grades invited to participate in FGDs were fathers or male guardians, and half were mothers or female 
guardians.13 The total number and types of FGDs conducted by the conclusion of data collection are 
detailed in Annex V.
4.4.3 Secondary Data Sources
For secondary quantitative data analysis, the evaluation team worked with USAID/Morocco, 
implementing partners, and MoE to secure the following datasets:
• School-, teacher-, and pupil-level data for the NPR baseline study conducted in May 2018, the 
midline conducted in 2021, and the endline administered in 2022; and
• NPR datasets, including midline data collected for the positive/negative deviance school study, 
Arabic-language curriculum effectiveness study, and research on students’ well-being and 
learning loss due to the COVID-19 pandemic.
The requested data was de-identified at the individual level but identified at the school level to facilitate 
meaningful analysis. 
12 This table assumes six to 10 parents per FGD. While each FGD with parents will aim to include ten participants total, the evaluators 
recognize it may be difficult to identify, recruitment and mobilize 10 parents at each site at the designated time and date. In recognition of this 
constraint, and standard practice on ideal FGD group size, a minimum of six parents per FGD has been established.
13 Only one parent or guardian is expected per family.
25
4.5 DATA COLLECTION, ANALYSIS, AND VALIDATION
4.5.1 Data Collector Training
Data collectors participated in a training in Morocco from November 7–12, 2022, before data collection 
began. During the training, participants were introduced to the context of the NPR final performance 
evaluation and reviewed data collection tools question by question to ensure they understood the 
content and purpose of each. Participants also received a handbook describing the procedures for 
facilitating FGDs, conducting KIIs, and capturing full, detailed, and accurate notes. Additionally, 
participants conducted a piloting session at nearby schools. 
Data collectors were divided into teams, with each team consisting of two locally recruited Arabic- and 
French-speaking speaking data collectors. Each data collection team had one woman and one man 
researcher. Each team was led by an STS home office staff proficient in French and English who 
facilitated the training and observed data collectors during practice sessions. When data collection 
commenced, this team lead facilitated daily debriefing sessions and data monitoring to provide quality 
assurance. One team of STS evaluators was also formed to conduct remote interviews. During 
fieldwork, the data collection teams communicated regularly with one another to ensure consistency of 
procedures and conduct remote debriefing sessions together. The composition of the teams ensured 
the collection of data in Arabic, French, English, or Darija languages. 
4.5.2 Qualitative Fieldwork
MoE approval was obtained prior to data collection. Evaluators submitted a list of proposed schools and 
respondents to USAID/Morocco. Through the Mission’s existing relationship with MoE, the Mission 
requested approval to collect data from the MoE prior to data collection. 
Data collection for the qualitative portion of the evaluation began with personnel training in Rabat, 
Morocco, from November 7–12, 2022. This was directly followed by approximately four weeks of data 
collection across six regions and 12 provinces in Morocco. 
Data collection was largely successful, and most scheduled KIIs or FGDs were conducted. However, 
two FGDs could not be executed for external reasons. One parent FGD could not be conducted 
because the sampled school does not have a parent association, which is typical for other schools, as it 
is a satellite school. Since a member of the parent association usually arranges meetings with parents, 
this barrier impeded gathering respondents. Additionally, parents worked and lived at far distance from 
the school. Secondly, one teacher FGD could not be conducted due to a teacher strike and subsequent 
boycotting of all activities not directly related to work. However, such instances were accounted for 
when the school sample was increased before the start of data collection. 
In addition, STS had requested that a regional official select which schools be visited in one specific 
region, as there was no EGRA data available for this particular region to guide selection by STS. In 
addition, one replacement school was selected, as the originally selected school was not a primary 
school. 
In advance of each FGD or KII, the team lead identified an interviewer and a notetaker. The interviewer 
led the interview or discussion while digitally recording it. The notetaker kept detailed notes during the 
FGD or KII.14 Permission to audio-record was requested as part of the consent process prior to the 
start of the KIIs. In cases where a KII or FGD participant declines permission to be audio-recorded, the 
14 Handwritten notes were in any language, but typed notes were in either English or French. For example, if an interview was conducted in 
Arabic, the interview was recorded to supplement the interviewer’s field notes (to be compiled at the end of the interview). The majority of 
interviews were conducted in Arabic, enabling one team member to conduct the interview while the other took notes.
26
team respected their wishes and proceeded with the interview without the audio-recording while taking 
note of the refusal to be recorded. Field notes were typed up as soon as possible to capture details 
from the interview while it was still fresh. The team lead reviewed these electronic notes for quality and 
completeness.
The evaluation team conducted research in accordance with international norms on the ethical conduct 
of research with human subjects, including obtaining consent from respondents.15 Qualitative data, 
including audio recordings, were stored on a secured, password-protected server. Once field notes 
were saved electronically, the audio files were also uploaded to a secure, password-protected server 
and deleted from the audio-recording device. 
4.5.3 Data Analysis
Qualitative Analysis
The teams recorded and took handwritten notes during FGDs and KIIs to capture qualitative data and 
then used these to finalize typed, expanded field notes in French. Qualitative data—including audio 
recordings—were stored on a secure, password-protected server. Finalized debriefing and field notes 
were imported into NVivo, a qualitative data analysis software package, to systemically code and analyze 
them and identify salient themes across a range of respondents that answered each evaluation question. 
The qualitative data analysis methodology incorporated an iterative approach and included thematic 
content analysis of narrative data to identify and validate emerging themes. A codebook was developed 
at two stages—when the evaluation design was finalized and during data analysis when additional themes 
emerged. Responses from different individuals and respondent groups were compared to identify 
discrepancies, if any existed, and to offer explanations for such discrepancies. The relationships between 
responses were also examined to learn about linkages across themes. Multiple analysts working in a 
single software package ensured that data, codes, and findings were stored in a common location. 
Secondary Quantitative Analysis 
Data collected during NPR were analyzed to supplement the qualitative data collected in November and 
December 2022. Due to time constraints, not all datasets provided by NPR could be analyzed for this 
evaluation. Therefore, three datasets were selected for secondary analyses based on their relevance to 
this evaluation—the EGRA baseline and endline data, the Arabic language curriculum effectiveness study, 
and the SRM study. For each of these studies, salient variables were selected to determine the extent to 
which they correlated with students’ reading scores. All results cited in this report were found to be 
statistically significant at the p < 0.05 level or lower. Comprehensive tables from the analyses explained 
in this section can be found in Annex X.
Baseline and Endline EGRA
For this analysis, NPR EGRA baseline and endline data were used from a total of 586 schools and 5,826 
students in grades 1–4. Ten variables from the EGRA and additional surveys—including the teacher 
questionnaire, classroom observation form, and school inventory form—were selected for analysis, as
shown in Table 4, and frequencies were run. 
15 The consent procedures clearly explained the purpose, use, treatment, and destruction plan for any audio recorded data so that respondents 
could make an informed decision.
27
Table 4: Tools and Variables Included in NPR Final Evaluation Secondary EGRA Analysis
TOOL VARIABLES
Teacher 
questionnaire
Do you teach reading every day?
Why should you teach letter/syllable sounds to children? Answer: They need to 
sound out (decode) in order to read.
Do you have access to supplementary materials?
Have you ever received training (in person or remotely) on how to teach 
reading?
Classroom 
observation
Teacher has his/her own copy of the teacher guide
All students have textbook or printed materials
Are there other student reading materials in the classroom?
School 
inventory
Has this school received supplementary reading materials (SRM) from the MOE 
for…
SRMs are stored in classrooms
Teachers have access to SRMs
To compute change scores between baseline and endline, all scores from three EGRA subtasks—
phonemic awareness, syllable reading, and oral reading fluency (ORF)—were calculated at the school 
level by averaging the results of all the students of the same grade level in the same school. Then 
changes in average scores were computed for each grade by subtracting the baseline average subtask 
scores from the endline average subtask scores of each of the three subtasks. Average change scores of 
all schools combined were analyzed by each of the ten selected variables, and tests for significance of 
difference were run between schools answering “yes” and schools answering “no.” Finally, average 
change scores were calculated by the type of school—full and partial treatment (experimental and 
nonexperimental)—and then regression analysis was run for each type of school in relation to each of 
the variables. The differences between these changes—called an interaction effect in statistics—were 
then tested for statistical significance for each variable at each grade level. Note that where statistically 
significant differences were identified, results can be generalized to the larger populations from which 
these samples of schools were drawn with a high level of confidence.
SRM Study
Data from these analyses were drawn from a study conducted by NPR during the EGRA post-test from 
December 2020 to May 2021 in a sample of 50 schools—25 experimental and 25 comparison—with 
grade 2 and 4 students and teachers. The same items included in the EGRA analysis described above 
were used in this analysis. Data were collected using a student questionnaire, teacher questionnaire, and 
school director/school inventory questionnaire. For the analysis, individual student EGRA ORF endline 
scores16 were calculated, then multi-level regressions were run to test for significance of difference 
between respondents answering “yes” and respondents answering “no” on a range of items. Results 
were disaggregated by grade, gender, and location (urban/rural).
Arabic Language Curriculum Effectiveness Study
Student-level scores from an NPR-administered assessment of reading comprehension in grades 5 and 6 
in May 2021 were analyzed in relation to questions asked in a teacher questionnaire and classroom 
observation, both administered by NPR during the program. Items were selected for analysis based on 
16 This is in contrast to the previous analysis, where EGRA scores were averaged by school, then compared from baseline to endline.
28
their relevance to the NPR evaluation questions. Results were disaggregated by gender and urbanicity. 
Tests for significance of difference were run between “yes” and “no” answers by grade – all students 
combined – as well as by grade disaggregated by gender and urbanicity. Interaction effects were also 
calculated by comparing differences between “yes” and “no” answers for each subgroup.
4.5.4 Data Validation
Following data collection, coding, and analysis, the evaluation team shared initial findings remotely with 
USAID and incorporated feedback into the draft of the report. After this draft report is submitted, the 
evaluation team shared and validated the evaluation findings with Mission staff, implementing partners, 
and MoE officials in Rabat. This in-person workshop also provided an opportunity to refine 
recommendations further to help ensure that they are specific and actionable. STS submitted the final 
evaluation report to USAID following written feedback from USAID and validation of findings at the 
workshop. STS completed a final, remote presentation with Mission staff, implementing partners, and 
MoE officials.
4.6 LIMITATIONS
During daily debriefing sessions, the data collector teams shared challenges they encountered in the 
field. STS also acknowledges the limitations of this study. 
• General qualitative approach of the evaluation: Although the general qualitative approach 
of this evaluation provides a nuanced, in-depth, complex picture of the NPR’s impact from a 
diverse set of respondents, the lack of primary quantitative tools to complement these data 
make it impossible to generalize any findings to geographical areas or populations as a whole. 
• Selection of scale-up schools: Schools in the four experimental regions were selected based 
on the highest and lowest EGRA average scores. Urbanicity and percentage of students who 
spoke Darija at home were additional selection criteria. EGRA data was unavailable for the two 
scale-up regions, and thus school selection could not be based on high and low average scores. 
Instead, the STS Field Manager selected these with input from appropriate regional MoE officials, 
including an inspector in Marrakesh-Safi and the AREF director in Drâa-Tafilalet. To limit bias, 
one rural province and one urban province were selected for each scale-up region. However, 
the inability to select scale-up schools based on EGRA data as was done with experimental 
schools remains a limitation of the study. 
• Teacher strike: A national teacher strike prevented one team of enumerators from 
conducting one teacher FGD. 
• Extra attendees: At one parent FGD, the school director remained present for almost half of 
the discussion and provided some answers during the interview. The presence of the school 
director may have biased parent answers. Data collectors discussed this issue with their STS 
team lead and subsequently handled such situations by politely requesting that only parents or 
teachers be present in their respective FGDs. A reminder regarding this was additionally sent to 
all teams to prevent further instances. 
• Participation of USAID staff in two parent FGDs: A USAID representative attended two 
parent FGDs. In some circumstances, the presence of donors or program managers in an FGD 
can constitute conflicts of interest and bias the responses of the participants—either positively 
(e.g., to receive additional support) or negatively (e.g., feel threatened that the donor will discuss 
findings with the government). However, given the nature of this evaluation (it is unlikely that 
many teachers or parents have a deep understanding of USAID’s role in the reform) and the low 
likelihood of bias (since teachers and parents are more likely to associate the reform with the 
MoE, who will not be present in the FGDs), STS considered the possibility of conflicts of 
29
interest and bias minimal. Furthermore, given the important role this evaluation will play in 
USAID’s opportunity to learn directly from beneficiaries, as well as in future USAID program 
design, STS considered USAID’s presence at FGDs a worthwhile risk. Nevertheless, in order to 
avoid possible conflicts of interest and bias, STS proposed, and USAID abided by, the protocol 
listed in Annex IX.
• Absence of an FGD Guide in Darija: The KII and FGD guides were made available to data 
collectors in both Arabic and French. During the training week, data collectors practiced 
conducting FGDs in Darija, as some parents may prefer communicating in this dialect. However, 
it is a known limitation that data collectors were sometimes required to conduct simultaneous 
translation from Arabic to Darija, which could result in variable quality of communication from 
one question to another, depending on the complexity of the question. 
• Respondents’ language of preference: During the course of a KII on the first day of data 
collection, enumerators noticed that the respondent seemed to be more comfortable answering 
questions in French than in Arabic. STS team leads discussed a solution with data collectors that 
day for KIIs at the higher levels of the MoE: asking the respondent for approval to record and, if 
granted, then asking for the language of preference. This addressed the issue of language 
preference at the start of the interview and allowed the respondent to answer the complete set 
of questions in their preferred language. 
• Lack of knowledge about NPR for some respondents: Some regional and provincial MoE 
respondents did not feel comfortable answering questions about NPR because they did not 
know much about it. This hesitation resulted in these respondents either asking for a 
department or division head to take their place or join them.
• Respondents declining to record: In some instances, respondents declined to be recorded, 
either due to their limited knowledge about NPR or hesitation to speak frankly while recorded. 
This may have affected the level of detail in notes, as data collectors could not listen to a 
recording after the interview to expand upon notes. These respondents are not identified in this 
memo, as doing so would violate the promise in the consent form that respondents will face no 
repercussions for declining to be recorded. 
Limitations in the secondary quantitative analysis of EGRA data are also recognized:
• Disparate reading proficiency among different grades: Results in this report sometimes 
show weaker results for students in Grades 1 and 2. This is likely so because measurements of 
fluency capture skills that students develop as they become proficient readers. To capture 
learning in Grades 1 and 2, analyses of student ability with syllable identification or phonemic 
awareness should be conducted. 
• School-level analysis: Analyses discussed in this report were conducted at the school level 
and, as such, had less statistical power than other analyses – for example, ones conducted at the 
student level. 
• Global pandemic: COVID undoubtedly affected overall outcomes and should be considered 
when concluding the role played by individual variables in student performance.
• Lack of a control group: Though this report discusses full and partial treatment schools, the 
NPR design did not include a control group – that is, schools randomly assigned as treatment or 
control. Therefore, reasons for change scores cannot be directly attributed to NPR.
30
5. FINDINGS, CONCLUSIONS, AND 
RECOMMENDATIONS
5.1 LEARNING OUTCOMES
Qualitative and quantitative data were used to answer evaluation questions related to learning 
outcomes. While primary qualitative data were collected from respondents in KIIs and FGDs to learn 
about their perspectives and experiences in response to these learning outcomes, secondary 
quantitative data were also analyzed from extant NPR datasets. Results of the primary qualitative data 
analyses are presented first, followed by a review of ORF results on the NPR baseline, midline, and 
endline EGRAs and additional findings from secondary data analyses.
5.1.1 Findings
Positive Changes Related to New Curriculum Amid Notable Obstacles
Respondents’ descriptions of how teachers and students have engaged with the new 
curriculum and related activities in experimental schools show how NPR has positively 
changed classrooms. “In the experimental schools, we saw how teachers were able to receive and 
implement the new pedagogy with passion and great fidelity,” an implementer said. “They loved the 
approaches and found them simple and found they had an impact on the children.”
These anecdotes brought to life the improvements in the curriculum that reviewers outlined in the 
USAID-commissioned report Morocco Curriculum Review and Capacity Building Activity, including a 
learner-centered approach and a focus on all four parts of literacy acquisition (listening, speaking, 
reading, and writing). Respondents noted how the new curriculum has helped students enjoy 
reading more. “The syllabic reading method is very important,” a teacher said in an FGD. “It has 
contributed a lot to the improvement of students’ reading skills. Now the student learns while having 
fun because using this method is like playing.” Respondents noted how students had improved with 
other aspects of Arabic, including speaking, writing, and listening. A provincial official explained this 
development at length:
In the past, the student could not build a useful sentence, but with this project, the level of the 
Arabic language has improved significantly, and the student has become able to speak fluently, 
and what is important is the love of language and reading for the student. Now, when the 
student takes a story, he knows how to read it, and the reading is not only inside the classes, 
but even outside the class, and that's the beauty of it.
Respondents also singled out the SRM activity as impactful. A regional MoE official said, “In addition, 
enrichment reading has also helped to strengthen the personality of the students and thus made them 
free, independent, [and] amused by new methods and loving the Arabic language.”
Stakeholders characterized the partnership that brought about these changes in school and classroom as 
collaborative and participatory, based on respondents’ description of their working relationship. The 
changes that teachers and school directors in experimental schools witnessed firsthand positively 
influenced the MoE’s outlook on the project. “There is a very strong buy-in and ownership of the 
program by MOE because the field—teachers, principals, school-level staff—sells it [and] spoke very 
31
highly of it. They all want to capitalize on NPR for future programs,” a USAID official said. Not only have 
schools observed changes in students’ attitudes toward reading, but implementers have seen how their 
partners at the MoE have evolved. “When the project ends, we leave behind a developed curriculum,” 
an implementer said, “but also teams who are equipped for this work.” 
In addition to the curriculum, the MoE is poised to carry on other innovative initiatives started under 
NPR, notably the comprehensive massive open online course (MOOC) content developed to train 
teachers at a distance on the new curriculum. “There is a lot of innovation, from capacity building of 
teachers, inspectors, and ministry officials to the assessments, to the MOOCs,” a USAID official said. 
“MOOCs were an important innovation within this work.” NPR collaborated with members of CNIPE 
to develop three MOOC courses, including Introduction to Teaching Arabic Language, Components of 
the Arabic Language Curriculum, and The Unit Project in the Arabic Language Curriculum. “MOOCs 
were part of the digital step of NPR,” a USAID official said. “It has been reformulated and we have seen 
its importance.”
Respondents noted how these positive changes related to the new curriculum occurred despite notable 
obstacles facing NPR, primarily the COVID-19 pandemic. Respondents described how the pandemic not 
only impacted student performance but other factors. Students in rural areas had more difficulty 
than their urban counterparts in accessing distance learning, according to respondents, and the 
pandemic also forced NPR to shift its training of teachers and inspectors from in-person to virtual. “I 
think if NPR was to go the way we started it, without COVID, we would have a better impact,” an 
implementer said. “But we had to do things from a distance and from afar. It impacted on teacher 
training and the delivery of the capacity building.”
COVID also affected students’ well-being and socio-emotional skills. “You can say that COVID affected 
students more psychologically than cognitively,” a parent said in an FGD. Some families witnessed how 
the pandemic affected their children differently. “I have two children—a boy and a girl—and the boy was 
happy to go back to school,” a parent said, “but the girl didn’t want to go back there because she was 
used to isolation.” Another parent described how his daughter was in first grade during the pandemic 
and told him that she learned nothing that year. Still, she continued to second grade: “Therefore, she 
encountered a lot of difficulties in second grade and could not understand quite a few lessons. Despite 
my encouragement, she was afraid and told me her classmates could read and write, but she had lots of 
difficulties understanding.”
Respondents often cited another factor that impeded NPR progress—the high turnover rate for 
teachers. A USAID official explained how many young teachers are hired and assigned at first to semi￾urban and rural regions. These teachers fulfill their duties as an obligation but desire to transfer to urban 
schools, which they consider better. “That turnover is always there,” the USAID official said. “The 
project has invested a lot in the capacity building of the teachers. But then, when you want to assess 
their capacity, you find new teachers who are new to the approach itself. This is typical of what would 
have happened in other regions in the country.” A regional MoE official added, “This is one of the most 
significant challenges faced by NPR, as the movement of teachers and their transfer from experimental 
schools to other schools has clearly affected the pace of implementation and support.” The amount of 
turnover was apparent during teacher FGDs. While most teachers were aware of NPR at five of seven 
experimental schools sampled for FGDs, most teachers at two schools had not. At one of these two 
schools, only one of eight teachers in the FGD had participated in an NPR training.
Labor struggles were also cited as an obstacle to NPR. Respondents described how a teacher and 
inspector strikes impeded training plans during the project. For instance, when NPR scaled the 
grade 1 and 2 curriculum nationwide during the 2018-19 school year, an inspector strike prevented 
32
teachers from being trained, and during the 2021-22 school year, contractual teachers went on strike 
for 45 days.
Trends in EGRA ORF Results
While grade 1–3 students’ ORF scores generally peaked at baseline, declined at midline, 
and partially recovered at endline, grade 4 students’ ORF scores increased both from 
baseline to midline and midline to endline. The highest of the three ORF scores for both boys and 
girls by grade from the three EGRAs is displayed in green in Table 5, while the lowest of the three is 
displayed in red. In grades 1–3, students generally scored highest on ORF at baseline in 2018 and lowest 
at the midline in 2021. In most cases, the ORF scores partially recovered by endline; however, grade 1 
boys and grade 3 girls had slightly higher ORF scores at the endline than at baseline. In contrast, grade 4 
students’ ORF scores increased both from baseline to midline, and midline to endline.17
The declines in ORF scores from baseline to midline illustrate COVID-19’s association with learning 
loss. While the baseline was conducted before the pandemic began in 2018, the midline was 
administered during the pandemic in 2021. In addition, the declines from baseline to midline were more 
sizeable for younger students in grades 1 and 2, perhaps illustrating the disproportionate impact of the 
pandemic and hybrid schools on them.
In KIIs, respondents cited the gains from the midline to endline as positive signs of NPR’s 
impact, especially considering the project’s obstacles. A USAID official cited the obstacles that hindered 
students’ ability to engage with teachers, including COVID and teacher strikes. “All these factors would 
not support student gains,” the USAID official said. “But the assessment went to the positive side, so 
that means that NPR … was integrated and had a positive impact on student performance.”
The ORF results highlight another notable trend—girls generally outperformed boys. At baseline, girls’ 
ORF scores were statistically significantly higher than boys’ in all grades, while at the endline they 
remained statistically significantly higher in grades 2–4, with boys only narrowing the gap with girls in 
grade 1. This trend aligns with respondents’ perceptions detailed later in this report that girls tend to 
learn and perform better than boys.
Table 5: EGRA Results in ORF by Grade and Gender
Asterisks indicate statistically significant differences between girls’ and boys’ scores by grade
For each sex and grade, green indicates a high score, red indicates a low score
Grade Sex Baseline 
(2018)
Midline 
(2021)
Endline 
(2022)
Gain: 
Baseline to 
Endline
Difference in 
gains: Girls 
vs boys
Grade 1 Boys 10.78 7.22 11.2 0.42 2.37*
Girls 14.35* 8.48 12.4 -1.95
Grade 2 Boys 17.62 14.98 17.6 -0.02 1.91
Girls 22.03* 18.93 20.1* -1.93
Grade 3 Boys 33.07 30.54 31.4 -1.67 -1.9
Girls 37.07* 35.51 37.3* 0.23
17 Note that these increases were not tested for statistical significance.
33
Grade Sex Baseline 
(2018)
Midline 
(2021)
Endline 
(2022)
Gain: 
Baseline to 
Endline
Difference in 
gains: Girls 
vs boys
Grade 4 Boys 30.70 32.33 36.1 5.4 -0.19
Girls 34.51* 37.54 40.1* 5.59
*=p<.05 or lower
Additional Findings from Secondary Data Analyses
This section presents the results of analyses of data from three studies conducted during NPR, as noted 
earlier in the data analysis section—the baseline and endline EGRA, the SRM study, and the Arabic 
language curriculum effectiveness study. Improvements in student scores on EGRA from baseline to 
endline are referred to as “gain scores.” Note that this analysis refers to “stronger performance” as 
larger gain scores (from the EGRA study) or endline scores (from the SRM and Arabic language 
curriculum effectiveness studies). Due to the likely impact of COVID, the analysis also considers 
smaller declines relative to other groups as signs of stronger performance as well. Note that all 
results refer to combined full treatment and partial treatment schools unless otherwise indicated. All 
results are statistically significant at the p < 0.05 level or below unless otherwise stated. For additional 
information on analytical methods, see the methodology section earlier in the report and Annex X.
Teaching reading every day correlated with stronger performance on multiple EGRA 
subtasks in grade 1. Gain scores were larger in grade 1 in ORF and syllable reading, and declines in 
phonemic awareness were smaller when teachers reported they taught reading every day compared to 
the scores of students whose teachers did not report teaching reading every day.
Teachers receiving training in reading and the use of SRMs correlated with stronger EGRA 
performance in grades 2–4. Gain scores were greater in grades 2 and 3 in ORF and in grade 3 in 
syllable reading, when teachers reported being trained in reading, compared to students whose teachers 
had not been trained. Gain scores were greater in grade 4 in ORF when teachers reported having 
participated in “all” or “most” of training in the use of SRMs. (See also section 6. Equity for additional 
differences found by sex and location.)
Possession of a teacher’s guide correlated with stronger EGRA performance in grades 1-3, 
as well as stronger comprehension scores in grades 5 and 6. Declines in phonemic awareness 
scores in grade 1 were smaller when teachers were observed with a copy of their guide. Declines in 
ORF scores were smaller in grade 2 and gains were greater in grade 3 when teachers had a copy of their 
teachers’ guide compared to students of teachers in those grades who did not have a guide. In grades 2 
and 3, interaction effects were found in which students whose teachers had their copy of a teachers’ 
guide showed positive gains in syllable reading (1.82 syllable gains in grade 2 and 3.01 in grade 3) while 
students in those grades saw substantial declines (-15.72 syllables in grade 2 and -8.29 syllables in grade 
3) when their teachers did not have a guide. In grade 4, gain scores were greater in syllable reading 
when teachers reported having a guide. Oddly, ORF gain scores were smaller in grade 4 when teachers 
had a guide than when they did not, yet syllable reading gain scores were larger when teachers had a 
guide. In addition, higher scores were also found in the Arabic language curriculum effectiveness study 
amongst students in grades 5 and 6 whose teachers had their guides.
The presence and use of SRMs correlated with stronger reading performance in grades 1–
4. Gain scores were greater in grade 1 in ORF when teachers said they had access to SRMs. 
34
Disaggregated by school type, interaction effect analyses found that full-treatment (experimental) 
schools benefited more than partial-treatment (non-experimental) schools at all grade levels when they 
had received SRMs from the MoE, with differences of 11 CWPM in grade 3 and 17 CWPM in grade 4 in 
favor of full-treatment schools: 
Figure 3: Has this school receives SRMs from the MOE?
CWPM when School Director answered yes no, by school type
Having SRMs also correlated with stronger syllable reading scores. When teachers in grades 
1–4 in full-treatment schools reported having received SRMs from the MoE, their students read 
substantially more syllables than students where teachers reported not receiving SRMs, with differences 
as great as approximately 38 syllables in grade 1, 20 in grade 2, 9 in Grade 3, and 8 in grade 4—in favor 
of students whose teachers reported having received materials. 
Access to 3asafeer, as well as reading 3asafeeer materials, correlated with stronger reading 
performance in grades 2 and 4. Students in grade 2 scored higher on ORF at the endline when they 
had accessed the 3asafeer digital library, when they read stories from the library, and when their 
teachers accessed 3asafeer at the school with school equipment. 
Introducing lessons and prediction activities correlated with stronger reading performance 
in grade 6. Stronger scores were found in grade 6 when teachers guided students to predict content 
and when they introduced lessons by explaining what students would learn.
5.1.2 Conclusions
The COVID-19 pandemic and hybrid schooling had an effect on learning loss, possibly with 
a disproportionate impact on younger students; in contrast, grade 4 students’ ORF scores 
increased from baseline to endline, possibly due to longer exposure to NPR. Morocco is no 
0.95
-1.36 -21.83
-0.76
-30
-20
-10
0
10
no yes
G1
partial full
-0.38
-0.69
-11.5
-1.18
-15
-10
-5
0
no yes
G2
partial full
-3.47
-2
-19.99
1.78
-30
-20
-10
0
10
no yes
G3
partial full
6.65
7.38
-7.39
4.2
-10
-5
0
5
10
no yes
G4
partial full
35
stranger to learning loss from the pandemic, which affected children’s learning worldwide. It is notable, 
however, that EGRA data show how scores decreased more for children in early primary grades. It may 
be worthwhile to track these trends to see how the learning loss may evolve over time as these children 
progress through primary school. Unlike ORF scores in grades 1–3, which declined or remained 
relatively flat from baseline to endline, grade 4 student ORF rates increased from baseline to endline. 
These gains may be associated with oIder students’ longer exposure to NPR interventions. 
Teacher turnover was perceived as also negatively impacting learning outcomes. In addition 
to the pandemic, many respondents cited teacher turnover as an obstacle, and the dearth of teachers at 
two experimental schools sampled for FGDs illustrated this issue. It is important to note, however, that 
EGRA data did not account for rates of teacher turnover in assessing students, so it may be worthwhile 
to find a concrete of measuring its impact quantitatively. 
NPR has positively changed classrooms. Although students were not interviewed as part of this 
evaluation, it was clear from teachers, parents, and education officials how the new curriculum had 
enlivened classrooms with its learner-centered approach. 
Key NPR funders and implementers developed a fruitful working relationship with the 
central MoE and helped them develop the technical capacity needed to develop, adapt, and 
sustain the new curriculum, learning materials, and MOOCs. In KIIs, partners did not report 
any contentious issues or air any grievances about collaboration during NPR. This strong working 
relationship bodes well for future partnerships between the MoE and USAID-funded efforts.
Secondary analyses of data from EGRA, SRM, and Arabic language curriculum 
effectiveness studies found that stronger reading performance correlated with several 
variables, including teaching reading every day, teachers receiving training in reading instruction and the 
use of SRMs, possession of a teacher’s guide, the presence of SRMs in schools, access to and reading 
3asafeer materials, and including lesson introduction and prediction activities in instruction.
5.1.3 Recommendations
Continue to research NPR cohorts to understand the project’s cumulative impact. Possible 
research topics include how each NPR cohort performs over time, particularly to see if the trends since 
the COVID-19 pandemic continue, as well as how teacher training, especially when done as planned, 
may lead to improved learning outcomes. 
Determine strategies to reduce teacher turnover. It is important to note that teacher turnover 
is not an issue that only affected experimental schools. It is a nationwide impediment to improving 
learning outcomes, especially if training programs during scale-up are initially inconsistently implemented. 
The MoE and partners should strategize how to mitigate its impact as much as possible, including how to 
use MOOCs as a fallback option to enable teachers to continue training.
Apply a learner-centered approach to other aspects of school curricula. One of the notable 
successes of the new Arabic language curriculum is how students have engaged more enthusiastically 
with the material. While it is not beyond the scope of this evaluation to know what the status of 
curricula reform in other subjects may be in Morocco, it would be worthwhile for the MoE to explore 
expanding this approach to other subjects if it has not already done so.
Document internal best practices of NPR partnership. It is possible that elements of the working 
relationship between the MoE and new partners for future USAID-funded projects may not follow the 
same framework as NPR’s. Due to the lack of issues in collaboration during the life of NPR, it would be 
36
beneficial to document the aspects that worked best to institutionalize them for future education 
projects.
5.2 CAPACITY BUILDING IN ASSESSMENT DATA AND USE OF DATA
Respondents shared various views on capacity building in assessment data, as well as the MoE’s use of 
data, including the design and effectiveness of the NPR assessment capacity building program, the extent 
to which data have been used to facilitate adaptive management, and associated needs for support to 
strengthen this capacity; and the effectiveness of the rollout of NPR’s formative assessment materials 
and methods.
5.2.1 Findings
Assessment capacity building program: Design and effectiveness 
Starting in 2019, NPR organized a series of capacity-building workshops for MoE personnel on 
evaluation. The general topics included assessment design; item development; developing a research 
agenda; benchmarking; data analysis, interpretation, and use; and test equating. Respondents involved 
with this series—including the donor, NPR implementers, and workshop participants—were asked if 
NPR should do anything differently if it were going to plan this series of capacity-building workshops 
again, and if so, what specifically should be done differently.
Most respondents reported that the design of the capacity-building series worked well. A central 
MoE official noted that the approach was good—that training used to be viewed as weak, but now it has 
become a strong element of the new curriculum. A USAID respondent noted that the assessment 
capacity building program “was the right approach, right topics, and the right time. We saw fidelity.” 
Another USAID respondent noted that the same approach to training would be used again in the NPR 
follow-on project, Bridge to Middle School, and expanded to other subjects such as science. An 
implementer noted that one of the best workshops was data analysis, during which MOE officials 
reviewed baseline and midline EGRA data. 
Although NPR invited a variety of participants from all 12 regions of the country to participate in the 
workshops, including teachers, inspectors, and MoE officials, some respondents suggested that the 
limited number of invites did not meet the demand for such capacity-building activities. A USAID 
respondent noted how implementers ended up expanding the capacity building to include not 
only members of CNEE but also inspectors and CRMER personnel. “Evaluators can learn how 
to produce mathematic data on what works, while pedagogical experts would learn how to interpret 
results and consider strategies to address gaps revealed by the research,” the USAID respondent said. 
Some CNEE and regional MoE officials mentioned the importance of decentralizing training to empower 
more people in the educational system. “We didn’t receive training as the inspectors did; we only 
handled logistics,” an AREF official said. “In order to get information, we have to contact the inspectors 
directly.” 
Adaptive management 
The evaluation also sought to understand whether, over time, the MoE had come to use data to 
facilitate decision-making about program design or implementation—an element of adaptive 
management—as well as to understand the MoE’s need for additional support in this area. To find this 
out, respondents were asked why capacity-building activities had or had not improved the MoE’s 
institutional ability to conduct EGRAs and use the results for informing implementation and if the MoE 
used—or had any plans to use—EGRA results to make any changes to the Arabic language education 
reform, and if so, how.
37
In response to these questions, some interviewees in experimental regions confirmed that EGRAs had 
been conducted in their regions and schools, and that workshop participants had learned how to 
design EGRAs and analyze their data. 
Yet, to what extent the MoE examined EGRA data and made decisions to modify the Arabic curriculum 
reform design and implementation was unclear. One project implementer suggested this process would 
happen in the future, saying, “I believe the MoE will use feedback from EGRA results again when 
reviewing the elementary school curriculum.” Another implementer felt that EGRA data had been used 
to make decisions about national exams at the end of Grades 2, 4, and 6. These decisions and intentions 
need to be corroborated. Another implementer indicated: “I think the MoE will analyze those results in 
the future to inform decisions. It’s the Moroccan government strategy.” 
Some respondents indicated that the MoE had made several decisions based on data that 
informed the curriculum design. One central MoE official stated that EGRA results had informed 
the curriculum design in the beginning, then helped the MoE monitor and measure student learning. And 
thanks to these results, he noted, EGRA had “improved our work.” One implementer reported that the 
midline EGRA results created a “feedback loop”—a discussion in which the MOE saw the gap (largely 
due to COVID) and, according to another implementer, “wanted to look at remediation”:
The MoE started to work with a method called Teaching at the Right Level (TaRL), then asked 
NPR to work on a remediation toolkit, so NPR worked on assessment and remediation. This 
conversation evolved into the decision to develop a formative assessment guide for teachers.18
Although implementers, donors, and central MoE respondents made decisions to varying degrees based 
on data, the extent to which the MoE was ready to use EGRA or other results remained limited. Said 
one central ministry official, “Our personnel was trained in data analyses, but we didn’t have time to use 
the results of their analysis.” According to MoE officials at the regional and provincial level, as well as 
inspectors and school-level personnel, the sharing and application of EGRA results and other data was 
limited. Multiple regional and provincial MoE officials, as well as school directors, noted that they had 
not received data or results. In the few instances that respondents reported learning about EGRA 
results, they said they had only heard about them verbally. Said one respondent from a regional office, 
“Our superiors told us to consult the ministry to have more information regarding EGRA and its use.” 
An inspector shared a similar story: “We collected the data but didn’t draw the conclusions ourselves. 
The results are at the curriculum directorate.” A school director added, “We haven’t received a single 
report, nor conclusions, related to the tests done in our schools.” One regional officer lamented the 
fact that he had to get all his data from inspectors and that his inability to analyze it himself constituted 
an impediment to improving practice:
The data we receive from inspectors are very weak. We don’t have any idea if students’ reading 
is improving or not or how they are learning…. Inspectors’ reports are very short and don’t 
provide enough information to analyze results. We’d like to see, for example, that in school A, 
students in class E aren’t learning to read, so we can provide additional teacher training.
In the absence of any information on the results of EGRAs conducted in their schools, some regional 
officials reported that the only indicator they used to understand student reading performance was the 
level of student participation in regional and national reading competitions.
To be sure, numerous actors played a role in collecting and using data. Some school directors reported 
using EGRA in their schools once or twice a year. One provincial director from an experimental region 
18 Paraphrase of notes from a respondent.
38
said, “Based on EGRA, there were recommendations that included the involvement of teachers and 
organization of workshops for inspectors, who plan meetings with their teachers.” An inspector 
described a similar pattern. “The inspector general at the provincial level takes results from EGRA, 
extracts and analyses them,” he said. “We ask him to give us the results concerning reading; then 
recommendations are made concerning the intervention.” Whether the EGRAs in these examples refer 
to NPR summative measures (baseline, midline, endline) or EGRA used for formative purposes is not 
clear.
NPR used data other than EGRA results to facilitate decision-making about program design or 
implementation, including the introduction of the assessment capacity building workshop and printing 
supplementary reading materials instead of only providing them digitally. One implementer explained 
that the initial NPR proposal did not included the assessment capacity building plans, “but we delivered 
the capacity building workshop based on the needs of the assessment crisis.” NPR also changed course 
when implementing its SRM activity with 3asafeer. “When we tried to introduce the reading enrichment 
program with 3asafeer, many students who lived in remote areas could not access the platform because 
of the challenges of connection,” an implementer said. “And you really see some impact. That’s why we 
went for printing and were able to compensate that by sending printed materials to these areas where 
kids had difficult access to the platforms.” 
FA toolkit & methods
As noted above, midline EGRA results spurred the MoE to seek ways to address the needs of struggling 
learners, resulting in a number of initiatives such as TaRL and the development of a formative 
assessment toolkit, which consisted of a formative assessment (FA) guide, a student remediation guide, 
and a teacher’s guide to unit tests for grades 1 to 4. The intention was to train the inspectors, who, in 
turn, would train the teachers in experimental schools, then make an electronic version of the toolkit 
available to teachers nationwide. To learn about the rollout of this toolkit, respondents were asked if 
they had received it and, if so, used it; to what extent they found the toolkit valuable; and to what 
extent the toolkit was scaled nationwide. 
Toolkit distribution and use
When asked if they had received the toolkit, most teachers were either unaware of the NPR formative 
assessment materials or had heard of them but were not familiar with them. It should be noted that 
responses were difficult to interpret for two reasons. First, sometimes respondents used the word 
“guides,” so they may have felt the question referred to teachers’ guides, not the FA toolkit. Second, 
some respondents indicated they had received print versions of the FA toolkit, while others reported 
accessing the digital version. Some made no indication in which format they had received the toolkit. 
Therefore, when they said they had not received the toolkit, this distinction presumably meant they had 
not received the print version and may not know they could access the digital version.
Given these caveats, the majority of respondents reported not having received either the print or 
electronic version of the FA toolkit. In one instance, a regional official indicated he had received the 
print version of FA toolkits and that they had been distributed to teachers, but school directors and 
teachers in the schools from that region included in the evaluation sample said they had not received 
them. Only four respondents indicated they had “received” the toolkit (i.e., had access to it) digitally—a 
region MoE official and an inspector, and two teachers from scaling zones. One teacher from a scaling 
zone said, “As part of my self-instructional program, I downloaded a digital version, so it wasn’t official 
or considered part of my official work. I use it implicitly in my work.”
For teachers who were familiar with the guides, researchers asked about their experience with their 
use. Four teachers responded that they had received training but implementing the materials was 
39
difficult, without specifying the nature of the difficulty. Another teacher simply said they were difficult to 
use. One teacher said he/she was faced with time constraints and only used the FA materials during the 
fifth week of each term (the assessment week). And one teacher said they were useful but that the 
number of copies was insufficient.
MoE perceptions of FA toolkit
When asked about their perceptions of the FA toolkit materials, most respondents indicated that they 
were not yet familiar with them, as noted above. Still, on the other hand, teachers in both experimental 
and scale-up schools noted that they had received training in FA but found it difficult to implement. One 
teacher noted: “The timing is tight, so I use it the fifth week of each unit” (that is, the week dedicated to 
assessing the previous unit, or four weeks). Another noted: “I improvise one week in each unit,” 
suggesting that incorporating FA tasks into one’s teaching can be challenging and that fifth-week 
assessments can be difficult, so the teacher improvises. In addition, one teacher in an experimental 
school noted that the FA toolkit helped in lesson planning and time management. 
Despite the general lack of awareness of the FA toolkit, the central MoE seemed eager to expand 
its use. A USAID respondent noted that FA was being incorporated into other subjects, saying, “I guess 
they are satisfied with where it is now and want to give it time to sink in. But the assessment approach 
is being incorporated into the other subjects (biology, geography, etc.).”
5.2.2 Conclusions
The assessment capacity building program was viewed as successful, though some 
respondents raised concerns about possibility of applying results in the field. The program’s 
effective design is best illustrated by the fact that it is being used as the blueprint for the follow-on 
Bridge to Middle School project. Still, as noted by multiple respondents, the program could be more 
effective if it included more actors in regional and provincial offices.
On numerous occasions, NPR, including the MoE, practiced adaptive management. 
However, as noted by senior MoE officials, the use of EGRA and other results remained limited. NPR 
changed course in numerous ways—including printing SRMs instead of relying on the 3asafeer digital 
platform, introducing capacity building on assessment practices as part of the NPR project design, and 
adding FA support midway through the project. However, as senior MoE officials noted, it will take time 
for the reform to roll out fully, including using EGRA and classroom formative assessment data to 
improve learning outcomes. 
Data sharing has been limited, making it difficult to use data to improve practice. Multiple 
regional and provincial officials reported that they had not received any EGRA data. One regional MoE 
official’s laments about not receiving data or being able to analyze them himself underscore how more 
decentralized training and data-sharing efforts could enable more actors to improve educational 
practices. 
Most respondents reported that they had not received the FA toolkit and knew little about 
its contents. When respondents did have access, they said they found it difficult to 
implement. It is evident from the lack of knowledge of the FA toolkit that its usage is very limited.
5.2.3 Recommendations
Wherever possible, conduct capacity-building activities in person. One respondent noted that 
the series worked better in person and that this would be true at both central and AREF levels. 
Furthermore, skills acquired at a distance should be reinforced in person.
40
Continue to build and sustain the central MoE’s capacity in data use and assessment, 
including support to schedule periodic EGRAs as part of the official school calendar and develop a guide 
about managing a reform agenda and implementing international research.
Design training to address the needs of evaluators, pedagogical experts, and administrators 
and articulate roles in data collection, analysis, interpretation, and use for each of these 
roles. One respondent noted that in the beginning, the program focused on CNEE’s needs. In time, 
program implementers realized that capacity building should be provided for evaluators as well as 
pedagogical experts. Respondents revealed additional needs to expand training to other education 
actors. For example, a provincial MoE official reported that in these workshops, he had learned about 
the administrative part of the evaluation, but not the pedagogical aspects. Inspectors could particularly 
benefit from capacity building in results analysis and interpretation because they are the only MoE 
officials who regularly visit schools and classrooms and interact with teachers and school directors.
Focus assessment training on the application of results with an eye to improving materials 
and classroom practices. An NPR program implementer noted that the assessment capacity-building 
series was ambitious and that more support is needed to deepen participants’ understanding of the 
material covered in these workshops. Respondents at the regional and provincial levels also noted the 
need for additional training in research design and EGRA administration. Many respondents emphasized 
the importance of training in data use—that is, how to apply findings to practice—specifically, how 
findings can inform teacher training at the CRMEFs and in the provinces by inspectors, teacher practice, 
curricular gaps, or needed improvements for instructional and supplementary materials. Importantly, 
one provincial director expressed a need to learn how to apply results while sharing best practices 
acquired by teachers—echoing project implementers’ recommendation to focus on how to interpret 
data to enhance classroom practices.
Standardize data sharing protocols—for example, the use of a publicly accessible platform. 
With respondents detailing how data sharing has been limited, it is clear MoE officials have an appetite 
for accessing data and using results to inform their decision-making. Creating a publicly accessible 
platform to share data would facilitate this desire.
Amplify efforts to distribute and train education support personnel and teachers in the use 
of FA materials and methods. Respondent’s lack of knowledge and application of FA materials and 
methods necessitates a more robust approach to training and distributing the FA toolkit, especially if the 
central MoE is eager to expand its scope.
5.3 SCALING
The MoE scaled up the curriculum nationally that it developed in partnership with NPR in three stages 
throughout the 5-year project. First, it introduced the grade 1 and 2 curriculum in classrooms 
nationwide in the fall of 2018, followed by the grade 3 and 4 curriculum one year later. After the MoE 
added revision of grade 5 and 6 curriculum to NPR’s scope in mid-2019, it expanded the new curriculum 
across the country for the 2020-21 school year. Lastly, the MoE completed a final validation of the 
entire suite of the revised grade 1–6 curriculum in mid-2021 and made an official decree about its 
adoption. 
5.3.1 Findings
Major project stakeholders noted how NPR helped MoE gain the technical knowledge needed 
to scale up the curriculum, materials, and MOOC content from the project’s experimental 
schools to all schools nationwide. A USAID official said, “The capacity building that was provided to 
41
the MoE actors will help the ministry to lead that effort.” An implementer added, “NPR didn’t only work 
with the MoE to make a new curriculum document but also to provide the tools and work closely with 
the ministry to develop the tools to deliver the curriculum.” These tools include technical guides on 
topics such as SRMs, formative assessment, differentiated learning, and other topics, which the MoE has 
fully adopted and validated. “This would have never existed before,” the implementer said.
In addition, the MoE asked for NPR’s help to develop an in-service training toolkit as one of the 
project’s last deliverables to help standardize training nationwide, and the MoE has devoted funding to 
this training program. Implementers and the MoE got feedback from inspectors and trainers from the 
four experimental regions to finalize the toolkit. “It was very important to do that,” an implementer 
said. “The pedagogy and structure of the training itself need to take into consideration the innovations 
under NPR.” 
Respondents noted how MOOCs are a key component of the in-service training toolkit because they 
offer teachers another avenue for accessing validated training content, especially when inspectors may 
meet with teachers only once or twice during the school year. “We don’t have many inspectors,” an 
implementer said. “The ratio of inspectors to teachers is very small. We needed to find different ways 
to introduce the new curriculum via the MOOCs and accommodate the ratio by making the MOOCs an 
essential part of the teacher’s training.” 
NPR stakeholders and the MoE not only solicited input and feedback from regional and provincial 
officials when developing the in-service training toolkit but also for all other major components of the 
curriculum. This process was proactive and deliberate, respondents said, because, without buy-in from 
regions all across Morocco, regional officials may have pushed back against the first major curriculum 
reform in nearly 20 years. “Speaking of challenges, one of the important challenges is that you had to 
‘sell’ your project to 12 regions,” an implementer said. “When you go to scale, that was a challenge 
because many of the people in these scale-up regions were not part of developing the curriculum. We 
overcame this by using a sharing approach.” At each stage of the national scale-up—from expanding the 
grade 1 and 2 curriculum in 2018 to the national validation of the entire curriculum for grades 1 to 6 in 
2021—NPR shared the draft curriculum with regional officials and organized national workshops to 
gather their feedback and incorporate it in revisions. “This helped us overcome the issue that we had to 
go to scale right after [piloting],” an implementer said.
All these efforts had respondents confident that the MoE was in the driver’s seat to continue 
scaling the curriculum after the project closed in 2022. “The most important thing is that the 
project is sustained after its life cycle,” a USAID official said. “Right now, it is owned by the ministry.” An 
implementer added, “The ministry has already been doing the work. The ministry has been considering 
this as their own project.”
Despite the MoE firmly being in command, respondents explained that it was premature to pass any 
definitive judgment on the effectiveness of scaling up the new curriculum, primarily due to 
training programs still being rolled out across the country. USAID: “[It’s] too soon to say that the 
experience has matured in terms of the whole primary education curriculum because it is not the same 
level of resources and training provided,” a USAID respondent said. “We need more time to see more 
results. That is education everywhere; you will not see fruits in one year.” An implementer added, “I 
think we need time to make sure that the training is scaled up. We know the textbooks and guides are 
distributed in the schools, but we are not sure about the training themselves.”
The process of training teachers on the new curriculum nationwide remains ongoing and 
varied, based on responses from inspectors. For example, one inspector in an experimental province 
42
said that training only began in November after logistical and technical challenges delayed plans for 
training earlier in 2022. Another inspector in a different experimental province estimated that only 20 
percent of teachers had been trained on the new curriculum. By contrast, an inspector in a scale-up 
province seemed to have trained many teachers, citing specific difficulties teachers were having with the 
new curriculum and how he was working with them to address the issues. 
Regardless of the variance in training from province to province, it is self-evident that the training and 
support teachers in experimental schools received from project resources were more 
comprehensive than what teachers elsewhere have received from MoE resources. A USAID 
respondent explained how the difference results partly from the cascade model the MoE must use. 
“Even if the project was training master trainers, when you go through layers of the training, the quality 
of the training is lost,” the official said, “because the training is not coming from the source, it is coming 
from the master trainer training the inspectors and then the inspectors training teachers within their 
regions.” One inspector from an experimental region who was highly involved with NPR activities in 
those schools was especially vocal about this difference, even insisting after the interview on the 
importance of training teachers in non-experimental schools after he had discussed it multiple times 
during the KII. The inspector said, “The training in experimental schools was responsible, but as for the 
non-experimental schools, it was quick, and it wasn’t generalized; only some benefited from it.”
NPR and the MoE have attempted to address these gaps through the development of MOOC content in 
their in-service training toolkit. Still, regional and provincial MoE officials described how teachers’ uptake 
might depend on their comfort with using technology, as well as their access to it. “We are dealing with 
a target audience of different ages and abilities,” an inspector in an experimental region said about 
teachers. “I am talking about technical abilities. The path to learning available on the platform requires 
technical skills, and this technical adaptation is not accessible to everyone.” Despite respondents 
reporting that teachers have signed up for MOOCs, they said teachers’ varied experience has led to 
differing results in the classroom. “The extent to which distance learning course objectives were 
achieved may require further study and analysis, given face-to-face training differs from distance 
learning,” an inspector from an experimental region said.
5.3.2 Conclusions
The MoE provided a solid foundation for schools nationwide to adopt the new Arabic 
curriculum introduced by NPR. It demonstrated effective management by distributing the 
curriculum and other materials and displayed the technical expertise it developed through 
the production of the curriculum and materials. The MoE effectively managed the scale-up of the 
new curriculum by seeking input from all 12 regions and ultimately getting buy-in from officials across 
the country and then illustrated its technical expertise by developing and validating the in-service training 
toolkit and other guides and then making them available to all educators online.
The most pressing issue around managing the curricula at scale concerns the need for 
intensive teacher training and coordinated monitoring—not only of these training 
initiatives but also of classroom teaching practices. As respondents noted, the scale-up of the 
curriculum is still ongoing as inspectors train teachers about the new curriculum and teachers access 
MOOC content. It is important not only to monitor the status of training initiatives nationwide but also 
how teachers are implementing the new curriculum in their classrooms, especially from online content.
5.3.3 Recommendations
Institutionalize the effective practices adopted as part of NPR scaling, so future scaling 
efforts for new curricula do not have to reinvent the wheel. The process that NPR and the MoE 
followed to expand the curriculum nationwide, especially seeking input from educators in all 12 regions, 
43
should be replicated for future Directorate of Curricula initiatives due to its effectiveness in mobilizing 
widespread support.
Equip and empower school directors and/or inspectors to conduct classroom monitoring 
and ensure teachers are applying what they have learned from MOOCs. One school director 
in a KII requested a tool he could use to monitor his teachers to ensure they were implementing the 
new curriculum as desired. This request underscores the importance of providing ways for inspectors 
and school directors to monitor teachers effectively, especially because some teachers are accessing 
MOOC content independently. Such tools would be especially useful for school directors because 
inspectors’ visits to each school are few and far between.
Devise a national strategic plan to assess the national rollout of training initiatives and 
ensure inspectors are implementing the training toolkit with fidelity. The central MoE should 
be more proactive in determining if inspectors implement the in-service training toolkit as intended. It is 
insufficient to receive reports from regional educational offices about the number of teachers being 
trained. Information needs to be gathered about the quality of training so that officials can address any 
major issues.
5.4 PARENTAL AND COMMUNITY ENGAGEMENT
Although some respondents in experimental regions did discuss the NPR-related training that parents 
received about 3asafeer—with many noting that accessing the platform was difficult due to limited 
connectivity in rural areas—or described how parental engagement increased due to NPR as a whole, 
most respondents talked about activities to involve parents more generally or difficulties in engaging 
parents and communities in children’s reading. 
5.4.1 Findings
Parental engagement and NPR
Respondents described how the project’s SRM activities encouraged more parental 
engagement in their children’s reading. Multiple school directors in experimental schools said that 
the SRM activity had met their expectations, with more books being distributed to students. A school 
director from a scale-up school said, “I think there is a positive change in terms of parental support so 
that some come to ask for story titles to borrow. Thus, we have proposed a list of stories to parents.” 
A school director at an experimental school reported that engagement had improved despite the level 
being generally insufficient among parents. 
Still, overall, parents who participated in FGDs mainly described limited engagement with their 
children’s school. Nearly all said they had not been invited to the school for orientations about 
3asafeer, SRMs, or any other activities. One parent detailed how parents were only invited to school at 
the start of the year to register for the parents’ association. Other parents explained how school 
directors and teachers had limited, indirect communication with them. “There are activities, but we are 
informed by our children,” a parent said, “so if our children did not tell us, we would not be aware of 
them.” Another parent at a scale-up school explained parents received a message from the director 
during a parents’ association meeting about a program encouraging reading, but no other details. 
These responses mirror NPR’s data collected during an endline study of its SRM activity in experimental 
schools. Only 40 percent of teachers and 35 percent of school directors reported providing support 
activities to parents to help them use SRMs or 3asafeer at home. Further, when teachers were asked 
how they monitored whether parents were providing support at home for SRMs and 3asafeer, 57 
44
percent said they checked with students, while 29 percent said they did no monitoring. Only 5 percent 
of teachers said they held individual meetings with parents.
Non-NPR-related engagement activities
Some respondents described engagement initiatives not connected to NPR, including top-down 
activities from the MoE and bottom-up ones from schools locally. Some inspectors, for example, noted 
that guidance had been issued concerning the roles of parents’ associations. Other inspectors indicated 
that their offices gave directives to schools to organize school-based awareness-raising activities to 
orient teachers and school directors on the importance of involving parents in their children’s schooling. 
As for localized strategies, one inspector in a scale-up region described his approach to informing 
parents about exam results, perhaps conflating NPR with the exams system more generally. The 
inspector explained how each school was required to organize four meetings with parents. “During 
these meetings, teachers talk with the parents about their children’s performance and give them 
strategies for improvement,” he said. In many other instances, respondents described activities they led 
in their schools and communities to promote reading. However, it was not evident that these activities 
were initiated by, or directly related to, NPR. These activities included meetings to promote 
rehabilitation and use of school libraries, the creation of parent groups on WhatsApp or Facebook, the 
promotion of reading competitions, school open houses, and school ceremonies to honor student 
achievement.
Difficulties in engaging parents
The most frequent response to this set of questions came from a range of respondent types, including 
inspectors, school directors, teachers, and parents themselves—parents have numerous difficulties, in 
general, supporting reading and their children’s education. 
Respondents cited multiple reasons for these difficulties. Some inspectors said they do not believe 
parents have the capacity to support their children. “Most of the time,” an inspector said, “parents are 
incapable of following the advice given to them.” Another inspector in an experimental region explained 
how some schools in rural and urban areas had organized activities with parents to raise awareness 
about the importance of having their children read 30 minutes a day. “These communicated suggestions 
did not receive good feedback from parents from rural areas because most of them are ignorant.” 
Other respondents cited other difficulties, including parents being busy with work or their inability to 
communicate with their children. 
Additional Findings from Secondary Data Analyses
Analysis of data from the SRM study showed that grade 2 students’ endline ORF scores were 
significantly higher when school directors conducted monthly check-ins with parents on the use of 
student SRMs. Additional correlations were found between stronger reading performance and parental 
involvement by subgroup (by sex and location)—see Section 6. Equity.
5.4.2 Conclusions
Increased parental support for children’s reading and the SRM activity were strengths of 
NPR’s parental engagement. By increasing student access to reading materials, the project’s SRM 
activity did lead to increased parental support for reading. Still, parents have untapped potential to help 
their children, as illustrated by data from NPR’s evaluation of its SRM program that revealed that only 40 
percent of teachers and 35 percent of school directors reported providing support activities to parents 
to help them use SRMs or 3asafeer at home.
45
Despite parents and communities not being sufficiently engaged, parents and communities 
showed eagerness to be more involved. While evidence was limited of wide-scale NPR parental 
support activities or specific NPR strategies that appeared to work, respondents frequently referenced 
local initiatives, indicating demand for parental involvement. In addition, schools and communities 
exhibited the capacity and motivation to organize activities to increase parental involvement in their 
children’s reading and education in general. Respondents also shared the lively tradition of sending child 
candidates to reading competitions, showing intense pride in knowing their children had been invited to 
these competitions. These examples point to a foundation of interest and capacity to involve parents 
and communities more systematically in supporting early-grade reading. This eagerness must be 
harnessed—as well as appropriate strategies devised—to optimize parental and community involvement 
regardless of location, occupation, or income.
Many respondents viewed parents as unable to support their children’s reading. The deep￾seated belief that parents lack the capacity to support their children’s reading are an obstacle to 
increasing efforts and activities for the community and parental engagement. Respondents shared 
multiple reasons why parents do not have the skills to engage with their children, emphasizing how it 
may be difficult to convince certain officials that parents are eager to help their children at home. 
When school directors did reach out to parents, some reading scores were found to be 
higher. In particular, grade 2 students’ endline EGRA scores were significantly higher when School 
Directors conducted monthly check-ins with parents on using student SRMs (see also Section 6. Equity 
below for additional evidence of stronger performance linked to engaging with parents).
5.4.3 Recommendations
Conduct campaigns to support and motivate students at home. Students and schools benefited 
from the SRM program, but the potential remains for the growth of reading activities outside the 
classroom. Stakeholders could design and launch campaigns to promote such activities.
Establish a system of communication between teachers and parents. Although parents and 
communities said they were eager to get more involved in their children’s education, they shared that 
schools and teachers did not reach out to them often. Schools should establish a more formal channel of 
communication between teachers and parents.
Implement strategies that help parents overcome challenges in supporting their children’s 
reading, as well as increasing their awareness about the general importance of reading.
Multiple types of respondents, including parents themselves, described the challenges that parents and 
communities face in helping their children learn to read. Therefore, it is essential to understand these 
challenges more deeply and then design effective parental and community engagement interventions that 
address them. In addition, programs to improve parental engagement should include components to 
build parents’ awareness about the importance of reading.
5.5 PRIVATE SECTOR ENGAGEMENT
Respondents discussed how NPR engaged with the private sector for two major project initiatives—the 
production of new student textbooks and teacher guides with private publishing companies and the 
creation of SRMs with 3asafeer, a Dubai-based educational technology company. Implementers and the 
MoE trained private publishers on the new components of the Arabic language curriculum prior to 
textbook production and then provided multiple rounds of feedback on their drafts of materials. NPR 
collaborated with 3asafeer to develop an SRM activity in its experimental schools, with 3asafeer 
providing 100 stories that the MoE adapted to the Moroccan context and then printed and distributed 
46
to experimental schools, followed by scaling up the SRM activity to schools nationwide. 3asafeer also 
provided access to its digital platform to students and teachers in experimental schools.
5.5.1 Findings
Private textbook publishing
During the production of learning materials, NPR played an intermediary role in soliciting and 
providing feedback between the MoE and publishers. According to respondents and NPR 
materials, the MoE and publishers had never engaged in such communication, as well as training on the 
new curriculum. One respondent closely involved in implementation reported, “The ministry will push 
the publishers, which is something that never happened before. They worked closely with them to 
provide clear training to publishers on the teaching and learning materials.” The textbook publishers 
noted how communication between them and the MoE in official NPR communications material had 
improved. The article outlined how inspectors had provided feedback on materials, as well as teachers 
in experimental schools. Mrs. Fatima Houssaini said in the article, “We appreciate the unprecedented 
support provided by NPR to our work. Receiving such detailed feedback is not only beneficial for 
Moroccan students but pushes us forward and gives us motivation to always do better.”
The back-and-forth between the MoE and publishers included multiple rounds of revision. For instance, 
NPR made multiple revisions of seven packages of grade 3 and 4 student books and teacher’s manuals in 
2020: “This labor-intensive effort, which NPR’s team completed through multiple reviews of the 
materials, came at the urgent and repeated requests of the Ministry and constituted a major 
contribution of NPR’s effort to ensure the quality desired in the new textbooks.”19
Respondents from textbook publishers voiced their appreciation of the new curriculum and 
pride in their own work, but they also shared some areas for improvement with training and 
feedback. Respondents from one publisher in an FGD said they attended some NPR training but didn’t 
believe they were helpful enough, stating, “In our opinion, these meetings were insufficient.” In an FGD 
with respondents from another publishing company, two people described how the feedback provided 
by MoE officials was contradictory. One respondent said, “Sometimes we find contradictions in the 
remarks; that is, more than one person revises the content, and sometimes there is no coordination 
between those people at the ministry who do the revisions.”
Despite the enhanced communication between the MoE and publishers, respondents reported that the 
publishers still could greatly improve the overall quality of the textbooks and teacher 
guides. A respondent closely involved in implementation stated, “[We] need to ensure they know the 
vision of the curriculum, the scientific background for adopting it, and best practices in teaching and 
international standards.” One respondent explained the need for improvements at the micro and macro 
levels, “You can have a great methodology in the curriculum document, but how can we reflect this in 
the textbook? Maybe they need international support to see how they can use different fonts, 
bookbinding, type of paper that is good for children’s eyes, etc. But above all, the need for improvement 
is technical expertise.” Other respondents noted the deficiencies of materials in KIIs and FGDs. Multiple 
teachers cited the limited number of printed teacher guides, with many having to rely on digital copies, 
as well as the quality of paper, which many teachers said was not durable.
These quality issues were raised as well in the USAID-commissioned Morocco Curriculum Review and 
Capacity Building Activity Final Report published in June 2021. The report cited two “areas of concern” 
primarily linked to the textbooks and teacher guides produced by the private companies, not the 
curriculum documents designed by the MoE. The first area of concern noted in the report was the 
19 Page 8 of the NPR FY20, Q3 quarterly report. The revisions are detailed in this section of the report.
47
“absence of an overall curriculum framework that informs the design, implementation, 
and evaluation of textbook content and activities.” In addition, the team of reviewers 
“uncovered that while the Curriculum Directorate advocates for a theoretical framework for 
curriculum design and instruction, such a framework is largely absent in teacher guides and student 
textbooks.” The second area of concern relates to the “overall quality of textbooks and teacher guides. 
While there are many best-practice innovations in the revised curriculum … there remain concerns 
about how these innovations are enacted in the textbooks and teacher guides.”
These gaps were also mentioned in the NPR article about the improved working relationship between 
textbook publishers and the MoE. The article states, “Given that the MoE delegates the production of 
instructional materials to private sector publishers, it is particularly challenging to ensure that the 
publishers develop materials that accurately reflect the curriculum improvements and updates.”
MoE respondents were also aware of these gaps and shared how they have tried to address them, but 
to no avail. “[Some editors] are still clinging to the textbooks they have produced, and they have 
problems grasping the innovations of the new curriculum,” a central MoE respondent said. “To 
overcome these obstacles … I contacted the publishers to reach out to young people capable of taking 
over publishing; however, this request did not get a response from any of the publishers.”
3asafeer and SRMs
According to respondents, the relationship between 3asafeer, the MoE, and NPR was 
constructive and successful. All parties shared the same goal of wanting to provide students with 
enjoyable reading material from 3asafeer that engages them and teaches “children to look at the world 
as scholars and scientists,” as one respondent described it, which differs from the majority of children’s 
books in the Arab world that “are not interesting … [and] tell you to listen to mom and dad.” An 
implementer said, “It went beyond what we expected. The CEO was interested in cultivating 
relationships with the MoE and accessing the Moroccan market.” One central MoE respondent 
corroborated the fruitful working relationship, noting there were no obstacles collaborating with 
3asafeer. The respondent had one regret, however—not taking advantage of the data 3asafeer provided 
about student use of the digital reading platform due to how busy everyone was with project 
implementation.
Although the SRM activity was largely successful with distributing and using print books, as previously 
detailed in EQ1, respondents explained that some teachers, students, and families had difficulty 
accessing and using the 3asafeer digital platform. Multiple respondents in the MoE at regional and 
provincial levels said they believed that some people had trouble, particularly in rural areas, accessing 
the digital platform due to a lack of access to devices such as smartphones and tablets, as well as poor 
Internet connectivity. One provincial MoE respondent discussed how officials participated in remote 
training on accessing the platform and enabling teachers to distribute free online access codes to 
students. However, the official said, “But all the problems we encountered in this process were that the 
students living in rural areas had many problems connecting to the internet, and therefore those who 
benefitted from 3asafeer were students in urban areas.”
NPR project data, however, presented a more nuanced picture. As part of the SRM intervention, NPR 
provided access to 3asafeer’s digital platform to students in sampled experimental schools. A 
comparably low number of rural and urban students used the platform. According to an NPR study on 
its SRM intervention, only about one-fourth of students—24.2 percent—reported they had read stories 
from the platform and could offer an example of a story they had read, including 22.6 percent in rural 
areas and 26.4 percent in urban areas. Students’ preference for books printed on paper may be another 
reason why students may not have engaged with the platform. In the NPR SRM study, students who said 
they used the platform were asked if they preferred print or digital medium for reading, and nearly three 
48
times as many said they preferred the print (62 percent) to digital (21 percent). In addition, lack of 
access to devices was only one reason students reported not using 3asafeer, according to the SRM 
study. While 25 percent of students said they didn’t use 3asafeer said they didn’t due to lack of a phone 
or computer, 29 percent said they didn’t have an access code, and 22 percent said they didn’t know how 
to use it.
Additional Findings from Secondary Data Analyses
Analyses of data from the SRM study found stronger reading performance—in this case, higher gains in 
CWPM—correlated with grade 2 students having access to and using the 3asafeer platform—see 
sections 1. Learning Outcomes and 6. Equity.
5.5.2 Conclusions
Improved coordination between the MoE and textbook publishers—including training and 
better communication—enhanced the learning material production process. It is evident 
from qualitative data and secondary sources that NPR marked an important step forward in the working 
relationship between the MoE and publishers. It is especially notable that a publisher noted the project’s 
“unprecedented support” in the USAID article and that publishers in FGDs voiced support for the new 
curriculum. 
Although private textbook publishers made progress during NPR, the MoE needs to 
continue to urge them to revise materials to align fully with the new curriculum. The gaps 
between the curriculum document and the textbooks and teacher guides stand out as one of the most 
apparent obstacles to realizing the full potential of the new Arabic curriculum reform, as noted by 
respondents in KIIs and Morocco Curriculum Review and Capacity Building Activity Final Report. The MoE’s 
outsourcing of textbook production to the private industry means that the MoE’s development of 
technical expertise during NPR is diminished if its vision is not reflected in the materials distributed to 
classrooms. The central MoE’s request to publishers to engage younger people in production was a 
notable first step in addressing this key issue. Still, due to the industry’s unresponsiveness, new strategies 
for engagement and more rounds of training and revisions are likely necessary.
NPR’s collaboration with 3asafeer highlights the value of engaging with partners outside of 
Morocco to access low-cost educational resources. It is difficult to picture how NPR’s SRM 
activity would have been possible without collaborating with a company with access to a trove of 
engaging stories that needed minimal effort to adapt to the Moroccan context. In other words, to have 
produced dozens of stories from scratch in Morocco—let alone develop an online portal for them—
would likely have been beyond the scope of NPR. This success with SRMs contrasts with the issues in 
producing learning materials in Morocco.
The lack of engagement with the 3asafeer digital platform underscores the obstacles to 
implementing an EdTech intervention in Morocco. Although the lack of access to devices as well 
as the internet curtailed the use of the 3asafeer digital platform in NPR experimental schools, other 
factors also likely played a role, including students’ preference for print books, lack of capacity in how to 
use an online portal like 3asafeer, and logistical problems in promoting its use (e.g., not distributing 
access codes). It is important for implementers to consider these other factors when planning such an 
intervention, especially in rural areas where teachers, students, and parents may lack access to devices 
and the internet and have limited exposure to these types of interventions. 
Secondary analyses suggest that having access to 3asafeeer may make a difference in 
reading performance. When grade 2 students had access to 3asafeer and used the materials, their 
reading performance was stronger than if they did not (see also sections 1. Learning Outcomes above and 
49
6. Equity below for additional findings concerning correlations between 3asafeer and reading 
performance).
5.5.3 Recommendations
Strategize how to continue to engage with publishers and raise their technical capacity to 
revise and improve learning materials to address acknowledged gaps. The MoE and 
stakeholders should brainstorm how to approach publishers with the directive to incorporate the 
curriculum’s innovations more fully into materials, perhaps through highlighting specific sections of 
materials that need to be updated or bringing in international experts for more extensive training and 
feedback. 
Provide teachers with paper copies of their guides. Teachers in multiple FGDs requested paper 
copies of teacher guides, echoing the same request that one implementer recalled hearing from teachers 
in the field. This respondent noted that private publishers might not have the financial incentive to print 
paper copies. If that is true, then partners must determine how it would be possible to get more paper 
copies in the hands of teachers.
Explore additional ways that the MoE could partner with the private sector, especially 
internet providers, and expand access to materials. The MoE should look into ways the private 
sector could help more learners access reading materials, as well as more ministry officials access and 
download the numerous guides and materials it produced as part of NPR, possibly through leveraging 
different platforms. It should also consider how to make 3asafeer available to all children, as well as 
provide access to other platforms and reading content offline.
To investigate low usage rates of 3asafeer, conduct a case study in experimental schools to 
determine how to overcome obstacles that led to that. NPR’s evaluation of the SRM activity 
revealed low usage of the 3asafeer digital platform as well as multiple difficulties students faced in using 
the platform. This NPR evaluation did not recommend measures to improve the low usage rates or 
address the challenges that teachers and teachers faced. A targeted case study could determine what 
underlying issues should be addressed and how to do so.
5.5.6. EQUITY
Respondents tended to answer questions related to equity in two ways. While some talked about 
efforts, NPR made to ensure the needs of children from different backgrounds were addressed—
including by region or province, urbanicity, language (Arabic vs. Amazigh), or gender, as well as students 
with disabilities (hereafter called “children with special needs” for the phrase frequently heard during 
this evaluation, “enfants à besoins spécifiques”—others talked about how children did or did not 
experience the new curriculum differently. As for steps taken to address any differences in how the 
curriculum was received, no respondents indicated that steps had been taken to address situations in 
which they perceived groups had received the curriculum differently. The following sections present 
what respondents did say.
5.6.1 Findings
Efforts made by NPR to address the needs of children from different backgrounds
Respondents from USAID, the central MoE, and implementers indicated that NPR ensured the 
project’s design, materials, and practices were equitable from the outset. The process began 
with a USAID-commissioned study that examined existing materials and the extent to which they 
respected children of different identities. During NPR implementation, stakeholders constantly made 
efforts to ensure that all groups were included in materials and work practices, including women in 
50
teacher training and all geographic regions on NPR implementation staff. According to an implementer, 
materials were designed to reflect different students’ identities. For example, to combat 
stereotypes, materials included girls interested in space exploration and engineering, as well as boys who 
liked to read or were introverted. An implementer also noted that NPR materials featured varied 
geographic areas, with pictures from both urban and rural areas as well as references to Moroccans 
living abroad in Europe and the U.S. A teacher also mentioned the use of Berber names in the materials 
as an effort to create a level playing field for them on and Arabs. One regional MoE official noted that 
the curriculum had content for students with various types of disabilities, “which was very positive.”
How students responded to the new curriculum
Many respondents said that the new curriculum helps all groups learn in an equitable manner.
According to a range of respondents, including a CRMEF trainer, an inspector, a teacher, a school 
director, and parents from several regions, speakers of Amazigh and Arabic speakers alike benefited and 
performed equally. Said one parent, “My Amazigh-speaking son now speaks Arabic better than 
Amazigh.” 
In some instances, respondents said that some students benefited more than others. Some said 
that rural students benefited less, in some cases, because of limited internet accessibility (especially less 
access to the 3asafeer platform), but in others, due to relative distances and poverty. One provincial 
director said, “Students in the mountains and foothills don’t get the resources they need.” Interestingly, 
one regional officer said the opposite—that rural students seemed to be benefiting more because of 
supplementary reading materials they received. Note that it is not clear how rural students may have 
benefited more from the distribution of SRMs than their urban counterparts. A central MoE official 
concurred:
Coverage in the rural areas constituted a gap (“c’etait une evasion”). Creating relevant local 
stories, preparation of relevant teaching tools, and presenting local vocabulary in classrooms –
these methods were used in some well-resourced urban schools but were absent in other rural 
schools.
Respondents also frequently revealed their beliefs about differences among students in the responses. 
One linguistic belief that respondents shared was that children who speak other languages at home, 
including Amazigh, learn Arabic better than children who speak Arabic at home. In reference to Amazigh 
speakers, a respondent suggested they learn Arabic better than native speakers due to the fact that they 
try harder because it is a second language for them.
Several respondents put forward another belief—that some instructional methods promote inclusivity. 
For example, an implementer said that “the 5-part approach—phonemic awareness, alphabetic principle, 
fluency, vocab, and reading comprehension—helps learners in all languages, including second languages.” 
Other examples of pedagogical approaches that foster inclusion cited by respondents included the use of 
differentiated instruction, having students learn in groups, and project-based learning. 
As for gender differences, many respondents said no significant differences were apparent. However, 
some respondents noted that girls might perform better than boys in some cases (but did not attribute 
this disparity to NPR). Some respondents, including a provincial director, a school director, and a 
teacher, observed that girls’ test scores were better than boys’, and some stated that girls learn better 
in their early years, with one saying boys catch up later. Reasons for boys’ weaker performance included 
life situations, such as the need to work in agriculture, but more often, respondents suggested girls are 
more predisposed to learning. One teacher said, “Girls are more interested in learning.” Another 
51
teacher noted, “Girls deepen their learning and sacrifice to read and study at home, while boys look for 
games to play.”
Of all subgroups, respondents most frequently expressed concern about students with special 
needs. “Differences exist, whether they be socio-cultural, linguistic, or ethnic,” an inspector said. 
“But there’s not a significant difference between children in these groups—except in the case of people 
with special needs. This is the one category where children struggle with learning.”
To be sure, NPR made numerous efforts to support students who were deaf or hard of 
hearing. An implementer noted that students with special needs were reflected in NPR’s illustrations 
and texts, and another implementer noted how NPR developed hybrid modalities during COVID so 
students who are deaf or hard of hearing could follow instructional programs on TV:
In a rapid but interactive process, we managed to have all the new courses of the Moroccan 
curricula for all subject areas in Grades 1-6 (not only Arabic) recorded and translated into sign 
language. We provided these courses on Moroccan TVs for Moroccan students and provided 
copies through the ministry to the schools.
A provincial director said that students, parents, and the MoE appreciated those efforts. According to 
one implementer, the hybrid lessons helped students who are deaf or hard of hearing catch up with 
other students. 
NPR had some influence over field practices concerning children with special needs as well. 
An inspector said, “We advise teachers to use ‘differentiated pedagogy’ and not ‘differentiated 
evaluation.’ This is strongly endorsed by NPR, especially for children with special needs.” Other local 
practices sometimes reflected the strategy promoted by NPR, though whether NPR was the motivation 
behind these strategies is unclear. A provincial official said, “We use inclusive practices with special 
needs students and have had some success.” Another said, “In one of our schools, students with special 
needs were all integrated into the program.” An inspector from a scaling region said, “We’re preparing 
all our schools to be inclusive for all students regardless of disability, and we’re developing continuous 
assessment tools for this.” Unfortunately, some respondents spoke of practices, that were at best 
unhelpful for children with disabilities, such as the case where an MoE official in a scaling region said, 
“Students with disabilities have different exams, usually by going to a lower class.”
Yet respondents also made it clear that students with special needs were sometimes ignored.
One inspector said that, in some instances, teachers simply did not adapt their lessons. For some 
respondents, the idea of integrating students with special needs into mainstream classes was downright 
problematic. An inspector said that integrating children with special needs “slowed the progress of the 
entire class.” Several teachers spoke of the difficulty of teaching large classes while also trying to engage 
with children with special needs. Even when they got integrated, another teacher noted, they get 
marginalized.
One of the most common comments from respondents was that the new curriculum did not 
sufficiently address children with disabilities. Some respondents cited a lack of tools, training, and 
personnel. A teacher in a scale-up school said it was left up to the teacher. Administrators spoke of the 
lack of qualified personnel in the classroom or to assist students with disabilities with the online 
program. Many teachers mentioned about the lack of training—for example, knowing how to integrate 
students or distinguish between autism and other developmental disabilities. One parent shared in an 
FGD, “Some students have mobility issues, others communication issues. Teachers do their best, but 
there are big differences in types of disabilities, and between mild and severe autism—we need a 
52
specialist for this.” An inspector noted that the new curriculum accounted for the needs of children who 
are deaf or hard of hearing but not children in other categories. A CMREF trainer noted that while the 
new curriculum addressing the needs of students with mild disabilities, it did not address the needs of 
the more severely disabled.
Other respondents cited the uninformed but unsurprising notion that the problems related to students 
with special needs do not exist or that there are no children with special needs in their schools. 
Numerous respondents said that all children are the same, that all children’s needs are the same, or in 
some instances, that “there are no children with special needs in our school.”
Additional Findings from Secondary Data Analyses
Factors related to SRMs, parent outreach, teacher characteristics, and teaching and 
learning practices correlated with stronger performance,20 though they varied by grade, 
gender, and location. These factors were found by analyzing three NPR studies—the EGRA, the SRM 
study, and the Arabic language curriculum effectiveness study—as shown in Table 6, Table 7, and Table 
8.21 In two instances, members of all subgroups—girls, boys, urban and rural students—equally 
benefited from teaching practices—when teachers asked girls and boys questions equally and when 
teachers gave girls and boys equal opportunities. Yet, in many other cases, correlational and interaction 
effect analyses found that girls and boys benefited differently, as did students from rural and urban 
schools. Boys and rural students were found to benefit in the greatest number of instances. 
Table 6: Correlations between strong performance on ORF in EGRA and contextual factors 
by grade, sex and location
Variable Comparison 
type
G1 G2 G3 G4
SRMs
School has received SRMs Interaction full full full full
Teachers had access to SRMs Gain all
Teacher characteristics
Teachers have a teacher’s guide22 Change all all
Their teachers had received training in teaching 
reading23 Change all all
Their teachers had received training in teaching 
reading24 Interaction rural
Teaching/learning practices
Teachers expressed importance of decoding25 Interaction rural
Note: “All” refers to all student groups combined (school type, gender, and location), and “full” refers to a full treatment school.
20 Strong performance includes “gain” (higher gain scores than comparison group), “endline” (higher endline scores than comparison group), 
and “interaction” (greater benefit when variable was more present for one group than another). All differences are statistically significant at the 
.05 level or below.
21 All references to subgroups indicate statistically significant relationships found between that variable and reading performance at the 
p < 0.05 level.
22 According to EGRA data, syllable reading scores in grades 2, 3 and 4 were also higher when teachers had copies of their own teacher’s guide. 23 According to EGRA data, students scored lower in G2 but higher in G3 when their teachers said they had received training. 24 According to EGRA data, students scored lower in G2 but higher in G3 when their teachers said they had received training. 25 According to EGRA data, syllable reading scores in grade 2 were also higher when teachers reported the importance of teaching decoding.
53
Table 7: Correlations between strong reading performance in ORF in SRM study and contextual factors by 
grade, sex, and location
Variable Comparison 
type
G2 G4
SRMs
SRMs are appropriate for reading levels Interaction boys, rural
Teachers could access 3asafeer Interaction rural rural
Students have access to Asafeer Digital Library Endline all
Students were able to borrow books to take home 
and use Interaction boys girls
Parent outreach
Schools organized outreach with parents to use 
SRMs and 3asafeer at home Interaction rural, boys
Schools checked in with parents on student use of 
SRMs Endline all
Teacher characteristics
Their teachers had received training in teaching 
reading26 Endline all
Note: “All” refers to all student groups combined (school type, gender, and location).
Table 8: Correlations between strong reading performance in reading comprehension in Arablc language 
curriculum effectiveness study and contextual factors by grade, sex, and location
Variable Comparison 
type
G5 G6
Teacher characteristics
Teachers have a teacher’s guide27 Endline all all
Teachers reported Darija as their mother tongue Endline all
Teaching/learning practices
Teachers taught in standard Arabic Endline all
Teachers expressed importance of decoding28 Interaction rural rural
Teachers completed their reading lesson plan in the 
time allotted Interaction rural
Teachers stuck to their weekly reading schedule Interaction girls
Teachers taught reading every day29,30 Interaction urban
Teachers were able to cover the Arabic curriculum Interaction girls
26 According to EGRA data, students scored lower in G2 but higher in G3 when their teachers said they had received training. 27 According to EGRA data, syllable reading scores in grades 2, 3 and 4 were also higher when teachers had copies of their own teacher’s guide. 28 According to EGRA data, syllable reading scores in grade 2 were also higher when teachers reported the importance of teaching decoding. 29 According to EGRA data, syllable reading scores in grade 1 were also higher when teachers taught reading every day. 30 In a contrary finding, the Arabic language curricium effectiveness Study data also showed that grade 6 students scored higher when their 
teachers taught reading 4-5 times per week or less (compared to those who said they taught reading every day)
54
Variable Comparison 
type
G5 G6
Teachers introduced lesson by explaining what 
students will learn Endline all
Teachers guided students to predict text content 
based on image Endline all
Students deduced relationships between sentences Interaction rural rural
Students devised possible interpretations Endline
Teachers guided students to read individually in a 
quiet voice Interaction rural
Students read quietly Interaction boys
Teachers asked girls and boys questions equally Endline all
Girls and boys were asked questions equally Interaction boys
Girls and boys were given equal opportunities to 
participate in activities and when teachers taught 
reading every day 
Endline all
Interaction all urban
Girls and boys were given equal access to learning 
materials Interaction boys
Teachers were observed giving girls and boys equal 
opportunities Endline all
Note: “All” refers to all student groups combined (school type, gender, and location).
5.6.2 Conclusions
Respondents perceived that the new curriculum helps all groups learn in an equitable 
matter—girls and boys, children from different language groups, children from urban and 
rural areas, and children with disabilities. Some respondents also detailed how they thought the 
new curriculum itself and instructional methods encourage inclusivity in classrooms. Importantly, while 
effective use of these strategies arguably improve conditions for children of different backgrounds and 
needs, they do not by themselves constitute a sufficient response to addressing the needs of different 
students, especially members of historically disadvantaged groups.
Respondents perceived that girls are better performers than boys. Although some 
respondents said no gender differences were evident with the new curriculum, others said they believed 
that girls inherently perform better than boys. 
Respondents most frequently expressed concerns about students with special needs in all 
subgroups. All types of respondents brought up students with special needs and the difficulties they 
believed this subgroup has with learning, highlighting the opportunity to target these students with 
future interventions.
55
Of all subgroups included in these analyses, boys and rural students were found to benefit 
from contextual factors most, suggesting that NPR inputs and associated behaviors in these areas 
may have contributed to stronger reading performance.
5.6.3 Recommendations
More research is needed to determine how different groups interact with new materials 
and respond to the new strategies introduced in the new curriculum. For example, some 
respondents perceived that girls are more inclined than boys to learn, while others said children who 
speak another language at home learn Arabic better. These beliefs need to be either confirmed or 
dispelled with thoughtful research studies.
Determine feasible initiatives to address the issues that students with special needs face in 
classrooms. Possible ideas include:
• Whenever possible and feasible, reserve a separate class for students with special needs.
• Provide specialists to support children with special needs, especially for more severe cases.
• Provide a curriculum with its own schedule, including materials (“kit”) tailored to the 
circumstances of students with different kinds of special needs.
56
ANNEXES
57
ANNEX I: EVALUATION STATEMENT OF WORK
Statement of Work
Background and Purpose
The Reading for Success-National Program for Reading (RFS-NPR) was awarded to Creative Associates 
in 2017 and ends in June 2022. A $19.3M activity, NPR is a follow-on to USAID/Morocco’s Reading for 
Success- Small-Scale Experimentation (SSE) Activity which ended in 2018. NPR builds upon the use of a 
pilot Arabic reading curriculum for grades 1-4 and includes a strong set of reading supplementary and 
instructional materials. The overarching goal of RFS-NPR is to help “Students in target regions read fluently 
and with comprehension by the end of the second school year.” To achieve this goal, NPR manages the 
following four components which are linked to the intermediate results outlined in the activity 
framework:
• Improved reading and writing materials used in primary schools;
• Improved reading and writing instruction in primary schools;
• Reading and writing enrichment programs expanded; and
• Learning and assessment system improved.
The RFS-NPR theory of change showing each of the intermediate and sub-intermediate results and the 
linkages is shown graphically in Figure 4.
Figure 4: RFS-NPR Results Framework
A midterm evaluation was conducted in 2019 of the Whole of Project, Reading for Success which 
included the National Program for Reading1. It is expected that this evaluation will build upon the 
findings from the previous evaluation and assess how well the activity has achieved its end of activity 
targets and outcomes.
58
1 Whole-of-Project Performance Evaluation of the Reading For Success project – Morocco (usaid.gov)
The priorities and questions for this evaluation will build on findings from the previous evaluation related 
to project implementation, the role of the Ministry of Education (MOE), and sustainability.
Evaluation Purpose
This evaluation will focus on the outcomes of NPR and examine the sustainability of its work in terms of 
the MOE’s efforts to take NPR components to national scale. It is expected that the evaluation will take 
a participatory approach involving USAID, the MOE, and Creative Associates to identify critical learning 
elements of this evaluation and to share and validate findings and recommendations with all 
stakeholders. The following evaluation questions have been reviewed by both USAID/Morocco and 
Creative Associates. It is expected that the winner of this solicitation will confer with the Ministry of 
Education on the scope of this evaluation and take into account their learning interests in this evaluation.
Evaluation Questions
This evaluation will examine the effectiveness of NPR’s approach in terms of its overall outcome, based 
on its results framework. The questions focus on approaches and outcomes as well as sustainability of 
the activity’s products within the MOE’s service delivery. The following are the guiding questions for this 
evaluation:
1. How has the new NPR reading curriculum and teacher instruction package affected learning 
outcomes in target schools? What role has the COVID-19 pandemic played in learning 
outcomes? More specifically:
a) How well did NPR deliver on the development and testing of the revised reading curriculum 
and teacher instruction interventions?
b) To what extent was NPR successful in strengthening the Ministry’s ability to develop and 
deliver an education reform in primary grades?
2. How well did NPR support the MOE’s ability to use Early Grade Reading Assessment (EGRA) 
data and classroom formative assessments to support adaptive management and improved 
learning outcomes at both the classroom and central level? How well was the capacity-building 
conducted with the MOE examination and assessment departments in terms of their ability to 
conduct EGRAs? In the case the MoE is using EGRA data to support adaptive management and 
improve learning outcomes, how will this data be used and how could it be facilitated?
3. How effectively did the MOE use and scale-up the new curricula introduced by NPR? What are 
the issues around managing the curricula effectively at a national scale?
4. What are best practices and lessons learned from NPR for engaging communities and parents in 
reading programs?
5. What are lessons learned from NPR related to private sector engagement in reading programs?
6. What are lessons learned from NPR related to education access, quality, or results among urban 
vs rural, male vs. female students, and/or geographic regions? How will these lessons learned 
inform future education programming and investment decisions?
Evaluation Approach
To the extent that it is practical, evaluators should triangulate findings over multiple sources and 
methods. Data sources and methods should be appropriate to the country context and the practicalities 
of physically visiting beneficiary communities. Methods may include:
• A review of relevant project and secondary documents;
59
• Analysis of secondary data sources (particularly EGRA data) relevant to the evaluation; analysis 
of the different assessment studies conducted by NPR team;
• Interviews with key informants at the central, district, and community levels who are directly 
involved in the activity;
• Focus groups or group interviews with a sample of community-level beneficiaries including 
parents, students, and teachers.
The evaluation will employ rigorous methods to analyze qualitative and quantitative data. In general, 
quantitative data will describe project outputs, and quantifiable outcomes. Qualitative data will provide 
evidence of the outcomes associated with project interventions that are not easily quantified. The 
evaluation team will use rigorous content or thematic analyses and coding techniques to identify key 
themes in the qualitative data and use quantitative analysis (e.g., descriptive statistics, cross-tabulation, or 
regression) methods to report results and identify patterns and correlations in the quantitative and 
qualitative data.
Fieldwork should employ culturally appropriate methods. The analysis should disaggregate results by sex 
whenever applicable and draw out sex-specific conclusions and recommendations. For example, 
capturing differences, if any, between men and women’s participation, perceptions, and experiences.
The evaluation design should account for the changing restrictions related to COVID-19. The safety of 
team members, implementing partner staff, informants, and community members is paramount. 
Expatriate team members should plan to travel to Morocco and work closely with local team members 
to conduct in-person fieldwork.
USAID recommends the evaluation team refer to this tool (Assessing the Quality of Education Evaluations 
Tool) as they prepare and conduct this evaluation.
Methodology
STS will design and conduct a non-experimental, mixed-methods performance and process evaluation. 
This evaluation will examine the effectiveness of NPR’s approach in terms of overall outcomes based on 
its results framework. The evaluation questions focus on approaches and outcomes as well as 
sustainability of the activity’s products within the MOE’s service delivery.
STS will begin with a literature review that builds upon the 2019 Whole-of-Project evaluation literature 
review, which will provide the context, background, and understanding needed for effective qualitative 
fieldwork as well as support the design of more in-depth and nuanced subsequent evaluation 
components. The evaluation team will study existing documents from USAID/Morocco and NPR 
implementing partners, as well as external documents pertaining to early grade reading in Morocco and 
those from Morocco’s MOE. The review will provide an overview of education and early grade reading 
in Morocco as well as the research that informed the design of NPR. The review will incorporate NPR’s 
results framework and project indicators and will describe cross-cutting issues of inclusion and disability, 
gender, sustainability, lessons learned, and challenges. This literature review will inform the design of a 
detailed draft concept note that describes the methodology and steps of the proposed evaluation.
The evaluation team will then host a five-day evaluation design and tools pre-testing workshop in Rabat 
to ground the evaluation in the local context and realities, refine the evaluation questions, build 
consensus and buy-in on the evaluation approach, and refine and pre-test primary data collection tools. 
The evaluation design and tools pre-testing workshop will include key representatives from the MOE 
and USAID/Morocco, and NPR implementing partners. Using the updated questions and sub-questions, 
participants will be asked to complete a stakeholder mapping exercise to identify the most appropriate 
60
data sources and methods for answering the each of the proposed evaluation questions. This mapping 
will be used to drive the final sample for data collection.
In order to answer the six primary evaluation questions and related sub-questions, the evaluators will 
primarily use a qualitative research approach. The team will conduct fieldwork over the course of 
approximately three weeks, collecting data from a variety of RFS stakeholder respondent groups in 
order to gain understanding of multiple perspectives and experiences across a variety of levels (national, 
regional, provincial, and school/community) and to enable triangulation of data and findings.
Triangulation will also include using both qualitative and quantitative methods described below to gain a 
more holistic and deeper understanding of the RFS project as well as mitigate potential bias in the 
evaluation data and findings.
Qualitative data collection methods such as semi-structured key information interviews (KIIs) will 
capture a variety of views, experiences, and perspectives from key stakeholders at national, regional, 
provincial, and school levels. In addition, focus group discussions (FGDs) with beneficiaries at the school 
and community levels will be conducted through school/learning site visits across multiple regions and 
provinces. Where possible, existing NPR project data (including EGRA results and other monitoring and 
evaluation data) as well as administrative and/or performance data from Morocco’s National Education 
Management Information System (EMIS)—L’évaluation des Acquis des Élèves et du Système 
d’Information (MASSAR)—will be analyzed to supplement qualitative findings.
The evaluators will integrate a gender-sensitive evaluation approach, drawing on resources such as 
USAID’s How-To Notes on Engendering Evaluations on Engendering Evaluation and Gender Integration 
in Education Programming and USAID’s Gender-sensitive Evaluation: Best and Promising Practices for 
Engendering Evaluation. This approach will include ensuring all data are disaggregated and reported by 
sex where appropriate, engaging female and male researchers, and striving for gender balance among 
respondents.1 Gender frameworks will be considered in the analyses as well, such examining how the 
interventions may have affected boys and girls differently.
STS will also incorporate social inclusion in the evaluation approach, including consideration of 
geographical or regional variations, ethnicity, language, socio-economic status, disability status, and age in 
the design, sampling, and analytic approaches. Geographic and regional variations will inform the 
evaluation’s stratified sampling approach across provinces. These areas also include variations in language 
groups, including both Arabic and Amazigh languages. The range of respondents, including high-level 
government officials and teachers and parents from purposively selected rural disadvantaged areas, 
incorporates an array of socioeconomic statuses.
The following evaluation matrix presents an illustrative summary mapping of evaluation questions to data 
sources, to be further detailed, refined, and finalized in collaboration with the MOE and USAID and 
during the evaluation design and tools pretesting workshop.
61
Table 9: Evaluation Matrix
Evaluation Questions Data Sources
1. How has the new NPR reading curriculum and teacher 
instruction package affected learning outcomes in target schools? 
What role has the COVID-19 pandemic played in learning 
outcomes?
More specifically:
How well did NPR deliver on the development and testing of the 
revised reading curriculum and teacher instruction interventions?
To what extent was NPR successful in strengthening the Ministry’s 
ability to develop and deliver an education reform in primary 
grades?
Secondary data analysis
KIIs with school, provincial, 
regional, and national level 
actors, USAID and IPs
FGDs with teachers
2. How well did NPR support the MOE’s ability to use Early Grade
ReadingAssessment (EGRA) data and classroom formative
assessments to support adaptive management and improved
learning outcomes at both the classroom and central level? How 
well
was the capacity-building conducted with the MOE examination 
and
assessment departments in terms of their ability to conduct 
EGRAs?
In the case the MoE is using EGRA data to support adaptive
management and improve learning outcomes,
how will this data be used and how could it be facilitated?
Secondary data analysis
KIIs with school, provincial,
regional, and national level 
actors,
USAID and IPs
FGDs with teachers
3. How effectively did the MOE use and scale-up the new curricula 
introduced by NPR? What are the issues around managing the 
curricula effectively at a national scale?
Secondary data analysis
KIIs with provincial, regional, 
and national level actors, USAID 
and IPs
4. What are best practices and lessons learned from NPR for 
engaging
communities and parents in reading programs?
Secondary data analysis
KIIs with provincial, regional, 
and national level actors, USAID 
and IPs FGDs with teachers and 
parents/community members
5. What are lessons learned from NPR related to private sector 
engagement in reading programs?
Secondary data analysis
KIIs with provincial, regional, 
and national level actors, USAID 
and IPs
62
Evaluation Questions Data Sources
6. What are lessons learned from NPR related to education 
access, quality, or results among urban vs rural, male vs. female 
students, and/or geographic regions? How will these lessons 
learned inform future education programming and investment 
decisions?
Secondary data analysis
KIIs with school, provincial, 
regional, and national level 
actors, marginalized groups, 
USAID and IPs
1 For example, focus group discussions with parents will aim for equal participation between 
men and women.
Primary respondents sampled for this evaluation will include school-level respondents; MOE and non￾MOE government officials at the provincial, regional, and national levels; personnel of USAID/Morocco 
and NPR project staff. Evaluators will collect data from respondents from selected provinces across all 
regions of the country. Evaluators aim to visit 6 schools – one school in 6 regions of the country. The 
regions will be purposively selected to represent a range of regional and school characteristics (e.g., 
geographic location in Morocco, demographic and socio-economic characteristics, urbanicity, 
predominant mother language of school, school performance, and school population size).
STS will refine the qualitative tools developed under the Whole-of-Project RFS evaluation to align with 
the primary evaluation questions of interests for the NPR Final Evaluation. During the design phase, STS 
will clearly map in Excel each item on each tool to relevant evaluation questions and sub-questions to 
ensure that sufficient information is elicited from all appropriate respondents to answer the evaluation 
questions. These detailed tool maps will be used to produce versions of each tool in Word that are 
structured for appropriate interview and discussion flow with respondents.
Each tool will be aligned with one specific type of respondent, including:
• USAID/Morocco officials
• NPR implementing partners
• MOE officials
• Marginalized groups
• Non-MOE government officials; and
• School-level respondents, including school directors, teachers, and parents
While the same tool will be utilized for subgroups within a respondent type, additional sub-sections or 
targeted items may be incorporated for specific subgroups within the main tool, where relevant. In the 
table below, STS has provided an illustrative list of primary data collection tools and sample targets for 
each respondent type.
Table 10: Primary Data Collection Tools and Sample Targets
Primary Data Collection Tools Sample Targets
Key Informant Interviews
KII – Provincial Director 12 KIIs (2 per selected region)
KII – Inspector/Provincial Coordinator 12 KIIs (2 per selected region)
KII – Provincial Arabic Teacher Trainer 6 KIIs (1 per selected region)
KII – MOE Regional Director 6 KIIs (1 per selected region)
63
Primary Data Collection Tools Sample Targets
KII – School Director 6 KIIs (1 per selected region)
KIIs – Regional Directors of Teacher Training Centers 2 KIIs overall
KIIs – Marginalized Groups 2 KIIs overall
KIIs – National MOE 6 KIIs overall
KIIs – Ministries of Health & Solidarity 2 KIIs
KIIs – USAID/Morocco and Implementing Partners 2 KIIs
KIIs – Implementing Partners 2 KIIs
TOTAL KIIs 58 KIIs
Focus Group Discussions
FGD – Teachers 6 KIIs (1 per selected region)
FGDs – Parents/Community Members 6 KIIs (1 per selected region)
TOTAL FGDs 12 FGDs
Data collectors will be selected according to multiple criteria, including prior relevant qualitative data 
collection experience and fluency in both Arabic and French. STS envisions most KIIs and FGDs will be 
conducted in Arabic or Amazigh languages, as appropriate, and will instruct data collectors to type their 
detailed notes in French. STS will draw from its existing pool of skilled data collectors in Morocco from 
the RFS Whole of Project evaluation. Data will be analyzed in French by French-speaking STS qualitative 
researchers.
Analysis Plan
Qualitative Coding & Analysis
Preliminary data capture and coding will begin during field work with the team debriefing sessions that 
take place each night within each of the data collection teams. In addition, data collection teams will join 
a group call twice per week during data collection. These debriefing sessions and associated notes will 
provide a way to document teams’ preliminary findings in real time during data collection. These notes 
will inform the out-briefing session with USAID and supplement the finalized field notes.
Finalized debriefing and field notes will be imported into NVivo 12, a qualitative data analysis software 
package. The qualitative data analysis methodology will incorporate an iterative approach and include 
thematic content analysis of narrative data to identify and validate emerging themes. A codebook will be 
developed at two stages—when the evaluation design is finalized, and during data analysis when 
additional themes emerge. Each night, the team lead will review KII and FGD notes in English or French 
taken by data collectors to ensure that notes are complete and sufficiently detailed. Notes will be 
uploaded nightly for further review by remote staff to track against the data collection schedule. After 
the coding process generates meaningful patterns from the field notes, analysts will use these codes to 
perform second-order analyses. These analyses will examine coded data in NVivo 12 to identify salient 
themes across a range of respondents that answer each evaluation question. Responses from different 
individuals and respondent groups will be compared to identify discrepancies, if any exist, and to offer 
explanations for such discrepancies. The relationships between responses will also be examined to learn 
about linkages across themes. Hypotheses that explain findings will be tested by examining codes for 
disconfirming patterns or alternative explanation. Multiple analysts working in a single software package 
will ensure that data, codes, and analysis findings are stored in a common, shared location for reference 
during reporting.
Quantitative Analysis
64
Quantitative data analysis will be conducted using the Stata/SE 17 statistical software package. Data will 
be imported and reviewed to ensure completeness and consistency in variable naming and coding, 
particularly when multiple datasets are merged. All relevant variables will be examined for missingness 
and outliers prior to analysis and appropriate strategies will be employed to address any observed 
irregularities (e.g., robustness checks and multiple imputation). Multiple regression analysis will be the 
primary analytical approach, allowing for statistical cross-group comparisons (e.g., pre/post, 
treatment/comparison) while controlling for confounders and/or increasing statistical precision of the 
estimates. Where appropriate, basic descriptive and summary statistics and cross-tabulations will also be 
presented.
The evaluation team will use qualitative field work as an opportunity to collect reliable information on 
the availability and relevance of further secondary quantitative data sources—including internal M&E 
data from implementing partners—that could support supplementary analysis and further triangulate the 
evaluation questions.
Qualitative and quantitative findings will be integrated while the final report is written. For relevant 
research questions, secondary quantitative analysis will be utilized to contextualize, supplement, and/or 
triangulate the qualitative findings. If divergent findings emerge during analysis, the evaluators will re￾examine the raw qualitative data through a more in-depth content analysis to determine if any other 
factors or issues were missed when data was first being organized in order to gain a deeper 
understanding of the data. In addition, disaggregated data may be reanalyzed by characteristics of 
respondents – such as locale of the respondent, respondent type, etc. - to help provide additional 
explanations for the variances the findings.
Reporting
STS understands that the primary audience of the full evaluation report is the USAID/Morocco 
Education Office in Rabat. The full evaluation report will adhere to the USAID Office of Education’s 
reporting templates and USAID guidance for evaluation reports.
The data and major findings will be summarized in the conclusion section of the report. This section will 
include a discussion of lessons learned and provide specific recommendations for the implementation of 
future activities, particularly related to sustainability considerations. Recommendations will be action￾oriented, practical, and specific, as well as note which actors or stakeholders are best situated to 
implement or take up the recommendations.
After the draft report is submitted, the evaluation team will share and validate the evaluation findings in 
a remote session with Mission staff and implementing partners. This workshop will also provide an 
opportunity to further refine recommendations to help ensure that they are specific and actionable. STS 
will submit the final evaluation report to USAID following written feedback from USAID and validation 
of findings at the workshop. Following USAID acceptance of the draft evaluation report, STS will 
conduct a workshop in Morocco in Arabic with Mission staff, implementing partners, and MoE officials 
to present the final findings.
The draft and final evaluation reports will include executive summaries in English. The evaluation team 
will also translate the final executive summary into Arabic. The primary audience of the executive 
summary is the Moroccan MOE and related stakeholders in the education system. Clearly articulated 
recommendations for appropriate audiences will be included in the translated executive summary.
STS will work with MSI to prepare the final report (e.g., 508 compliance and formatting) for submission 
to the Development Experience Clearinghouse (DEC).
65
Deliverables
Required deliverables for this assignment must include:
• Evaluation work plan: Following an initial discussion with USAID, MOE, and Creative 
Associates, the contractor will submit a detailed work plan outlining the evaluation purpose, 
questions, methods, staffing, schedule, and data collection tools. The evaluation will not begin 
until USAID approves the work plan.
• Completed Data: All data collected will be the property of USAID and will be submitted in 
the required format. Data may also be required to be inserted on DDL.
• Findings presentation/debriefs: After completing the fieldwork and analysis, the evaluation 
team will present initial findings, conclusions, and recommendations to USAID. Following this 
presentation, the contractor will make a separate presentation to the MOE in Arabic or provide 
translation during the presentation.
• Draft report: The evaluation team will submit a draft report that adheres to USAID guidelines 
for evaluation reports.
• Final report: The evaluation team will incorporate USAID feedback to the draft report and 
submit a final report with an executive summary in Arabic.
• Development Experience Clearinghouse (DEC) version of final report: MSI will work 
with the contractor to prepare the final report (e.g., 508 compliance and formatting) for 
submission to the DEC.
The Mission may also request additional deliverables including:
• Interim briefings during fieldwork; and
• A short (one or two page) evaluation brief summarizing key findings, conclusions, lessons, and 
recommendations in an easily digestible format using infographics.
Activities and Estimated Schedule
The schedule in the table below shows the estimated duration of each phase of the evaluation. Final due 
dates for each deliverable shall be agreed upon by mutual consent of both parties within the period of 
this contract.
Estimated 
Duration Task
4 weeks Preparation (literature review, initial consultation with USAID, MOE and the 
implementing partner, and initial design)
2 weeks Team planning workshop (refine/finalize design, plan data collection, develop and 
pretest data collection instruments)
4 weeks Fieldwork (conducting interviews, focus groups and collecting secondary data)
3 weeks Analysis (quantitative and qualitative analysis; findings)
2 days Debrief presentation (preparation and presentation)
6 weeks Writing (draft and final report)
66
ANNEX II: NPR PERFORMANCE INDICATORS AND ANNUAL TARGETS
NPR’s complete set of performance indicators are included in Table 11. The annual targets that NPR 
included in the project’s quarterly reports during FY2022 are listed as well. No FY2022 results were 
available because the evaluation team did not receive any final results from the project.
Table 11: NPR Performance Indicators and Annual Targets
# Performance Indicator Annual Target 
(10/2021–09/2022)
Goal: Students in target regions read fluently and with 
comprehension by the end of the second year of school
1. Percentage of learners who demonstrate reading fluency and 
comprehension of grade level text at the end of grade 2 (ES. 1-1)
35% (41% for girls 
and 29% for boys)
2. Percentage of learners at the end of grade 2 with zero scores on 
oral reading fluency assessments
N/A
3. Number of learners reached in reading programs at the primary 
level (ES. 1-3, ES. 1-5)
4,432,229
(2,117,180 girls and 
2,315,049 boys)
IR 1: Improved reading and writing materials used in primary 
schools
4. Percentage of teachers using materials, teacher guides, textbooks, 
and supplemental reading materials correctly (per criteria)
N/A
5. Number of primary or secondary textbooks and other teaching 
and learning materials provided with USG assistance (ES. 1-49)
4,556,369
SIR 1.1: Support for new Arabic language curriculum development 
for grades 1-4
6a. Number of grade-level EGR/W curricula with scope-and-sequence 
developed for grades 1-4
N/A
SIR 1.2: Develop a sequenced plan to experiment the new 
curriculum in target schools
7. Plan for experimentation, validation, and scale-up operational N/A
SIR 1.3: Support the MOE to improve the quality of Arabic 
language materials for grades 1-4
8a. Number of sets of textbooks, teacher guides, and teaching￾learning materials specifications developed for grades 1-4
N/A
SIR 1.4: Support the development of supplementary reading 
materials for reading and writing instruction and practice in Arabic 
in grades 1-4
9. Number of grade-level sets of supplemental reading materials 
developed for grades 1-4
N/A
SIR 1.5: Support for new Arabic language curriculum development 
for grades 5-6
6b. Number of grade-level EGR/W curricula with scope-and-sequence 
developed for grades 5-6
N/A
SIR 1.6: Finalize primary school curricular guidance (national)
6c. Number of grade-level EGR/W curricula with scope-and-sequence 
developed for grades 1-6
N/A
67
# Performance Indicator Annual Target 
(10/2021–09/2022)
SIR 1.7: Support the MOE to improve the quality of Arabic 
language materials for grades 5-6
8b. Number of sets of textbooks, teacher guides, and teaching￾learning materials specifications developed for grades 5-6
N/A
IR 2: Improved reading and writing instruction in primary schools
10. Percentage of teachers meeting or surpassing minimum 
performance standards for EGR/W instruction (per criteria)
N/A
SIR 2.1: Provide support to the MOE for training of grade 1-4 
teachers in best pedagogical practices
11. Number of teacher professional development delivery models 
developed for grades 1-4 in-service teacher training
N/A
SIR 2.2: Support in-service TOT/teacher training for EGR/W in 
Arabic for grades 1-6
12. Number of education administrators and officials who complete 
professional development activities with USG assistance (ES. 1-12)
398
13. Number of primary school educators who complete professional 
development activities on implementing evidence-based reading 
instruction with USG assistance (ES. 1-6, ES. 1-7)
93,393
SIR 2.3: Support training for support personnel on skills for 
improving teachers' early grade reading instructional practice
14. Number of coaching packages developed N/A
SIR 2.4: Support pre-service teacher training for early grade 
reading in Arabic for grades 1-6
15. Number of pre-service training packages developed for grades 1-6 N/A
SIR 2.5: Train teachers using supplementary materials for reading 
and writing in classrooms grades 1-4
13. Number of primary school educators who complete professional 
development activities on implementing evidence-based reading 
instruction with USG assistance (ES. 1-6, ES. 1-7)
N/A
IR 4: Improved learning and assessment systems
20. Number of policies, programs, and procedures changed based on 
evidence and data from feedback loops
N/A
SIR 4.1: Support student reading and writing assessments for 
grades 1-4 at defined intervals
21. Number of research and assessment studies conducted N/A
SIR 4.2: Use student learning assessment results to improve the 
model, including the trainers, teacher and student materials
22. Number of EGR/W material sets revised based on analysis of 
EGR/W assessments
N/A
SIR 4.3: Develop formative and summative assessment instruments 
for teachers' use
23. Number of formative and summative assessment instrument 
packages for teacher use developed, refined, and approved
N/A
SIR 4.4: Develop MOE capacity for measurement and evaluation of 
improved reading outcomes
68
# Performance Indicator Annual Target 
(10/2021–09/2022)
25. Number of primary school classrooms that receive a complete set 
of essential reading instructional materials with USG assistance 
(ES. 1-11)
121,446
26. Number of public and private schools receiving USG assistance 
(ES. 1-50)
24,358
Moroccan Sign Language Activities
27. Number of education institutions or organizations supported with 
preparedness, prevention and/or response interventions (pre￾primary to higher education/formal or non-formal)
N/A
28. Number of primary-level courses developed and recorded for deaf 
and hard-hearing students
N/A
29. Number of courses broadcasted on Moroccan TV channels N/A
69
ANNEX III: DISCLOSURE OF CONFLICTS OF INTEREST 
DISCLO SURE O F ANY CONFLICTS O F INTEREST 
Memorandum Regarding Organizational Conflict of Interest for School-to-School International (STS) 
School-to-School International (STS) does not foresee an Organizational Conflict of Interest (OIC) 
resulting from STS’s role as a subcontractor on the Morocco NPR Final Evaluation 2022 – Final, as 
outlined by the terms in Contract Information Bulletin 99-17 (CIB). According to the CIB, high potential 
for OIC occurs in the following scenarios:31
a) Where a firm that designs a USAID activity under contract with USAID wants to be eligible for 
the competition to implement the activity;
b) Where a firm that evaluates an activity or contractor under contract with USAID wants to 
provide services that are requested as a result of the evaluation; and
c) Where a firm that audits USAID contractors under contract with USAID seeks to do consulting 
work under contract with USAID (sometimes in competition with the firms audited).
None of the above-mentioned scenarios outlined in the CIB apply to STS’s subcontractor role on the 
Morocco NPR Final Evaluation 2022 - Final. STS acted strictly in an evaluation role having no role in 
project implementation. STS’s role included conducting Key Informant Interviews and Focus Group 
Discussions, as well as analyzing secondary quantitative data. As part of this work, STS designed data 
collection tools, facilitated enumerator training, and completed data analysis. 
31 https://www.usaid.gov/sites/default/files/documents/1868/cib9917.pdf
70
ANNEX IV: SUMMARY OF EVALUATION TEAM MEMBERS
The team contributing to this evaluation offer rich and diverse experience and expertise to ensure high￾quality and cost-effective support to the evaluation. The evaluators have deep experience in mixed￾methods evaluations of early grade reading programs, including in Morocco and with the RFS project 
specifically. 
Senior Technical Manager, Matthew Murray, will provide technical and quality oversight to the 
evaluation, including design workshop planning, design report development, and reviewing key 
deliverables. Mr. Murray is a program manager and evaluator with over twelve years of experience in 
monitoring, evaluation, and learning in development settings. He has managed education and early 
childhood development projects funded by USAID, FCDO, the World Bank, and education agencies in 
sub-Saharan Africa, the Middle East and North Africa, and Central Asia, ranging in size from $550,000 to 
$10 million. He has managed and conducted qualitative and mixed-methods study design, data collection 
and analysis for performance and impact evaluations. He managed the 2019–20 USAID RFS performance 
evaluation in Morocco, which drew on 87 key informant interviews (KIIs) and 20 focus group 
discussions (FGDs) to inform USAID’s Country Development Cooperation Strategy. Mr. Murray led 
remote training sessions, oversaw data collection, and analyzed data in NVivo in both English and 
French. He holds an M.A. in International Affairs from George Washington University and studied 
Arabic in Morocco.
Technical Manager, Fiona Eichinger, will serve as a primary point of contact for MSI and USAID 
throughout the contract. She will support the design of the evaluation and will lead the programmatic, 
budgetary, logistical, and contractual activities and ensure all deliverables are high-quality and on time. As 
an experienced manager in the field of education, curriculum development, monitoring, and evaluation 
since 2016, Ms. Eichinger has gathered country experience in Malawi, Morocco, Togo, the Philippines, 
Nepal, Malta, Spain, France, and Germany. Ms. Eichinger holds a master’s degree in international 
relations specializing in development and humanitarian assistance from Syracuse University with an 
exchange semester at Sciences Po in Paris, France. She is fluent in German, professionally proficient in 
Spanish, and studying Arabic.
Evaluation Team Leader, Carol da Silva, Ph.D., will provide oversight and guidance to the 
technical design of the evaluation, tool development, data analysis, and report writing to ensure overall 
technical quality. Dr. da Silva is STS’s Senior Director of Impact and Learning and leads STS’s Monitoring, 
Evaluation, Research, and Learning Department. She is an experienced literacy expert and mixed￾methods researcher with over 12 years of experience working in education and international 
development, with a focus on both classroom- and community-based interventions to improve children’s 
reading. She has served as a researcher and technical advisor for multilateral, bilateral and international 
non-governmental organizations, as well as a faculty member at higher education institutions teaching 
international education. As a technical expert in language and literacy, Dr. da Silva has led or contributed 
to literacy programming and monitoring and evaluation activities for organizations ranging from Save the 
Children, FHI360, and RTI International, among others. As an original co-author of Save the Children’s 
award-winning Literacy Boost intervention, Dr. da Silva played a key role in the program’s teacher 
training and community engagement toolkits, including leading trainings on community and parent 
engagement in children’s reading. Dr. da Silva holds an Ed.D. in Education Policy, Leadership and 
Instructional Practice and an Ed.M. in International Education from Harvard University.
Evaluation Team Leader and Education Evaluation Expert, Mark Lynd, Ph.D., will be 
responsible for supervising the quality of the evaluation design and execution, including qualitative data 
collection and analysis, identifying and disseminating findings, lessons learned, and evidence from 
71
evaluations. Dr. Lynd is STS’s President and Co-Founder and serves as a technical advisor for 15 
projects and 30 staff in over 20 countries. He is a proficient French speaker and provided support to the
2019–20 USAID RFS performance evaluation in Morocco. Dr. Lynd holds master’s and doctoral degrees 
from the University of Massachusetts, Amherst.
Qualitative Analyst & Technical Writer, Drew Schmenner will support the evaluation design 
and implementation and lead in qualitative analysis and reporting of qualitative findings. Mr. Schmenner is 
a Senior Data and Technical Writing Associate at STS. For nearly a decade, he has supported STS-led 
evaluations of early grade reading programs with a variety of monitoring, evaluation, and learning skills, 
including playing key roles in qualitative data analysis, report writing for mixed-method studies, data 
cleaning, and providing technical and logistical support for data collection efforts in the field. Mr. 
Schmenner brings key experience contributing technical expertise in qualitative analysis and writing for 
the performance evaluation of four activities under USAID’s RFS project in Morocco. For the Morocco 
RFS evaluation, he wrote the initial desk review, helped to analyze qualitative data in NVivo, and served 
as one of the lead writers of the final report. He holds an M.A. in International Studies from the 
University of San Francisco and is proficient in French.
Senior Statistician, Michel Rousseau, Ph.D., will lead in quantitative data analysis and support in￾person workshops with the MoE. Dr. Rousseau is a skilled statistician with a decade of experience as a 
professor of education, evaluation, and psychometrics at the University of Quebec in Canada. He has 
provided technical expertise to numerous USAID-funded EGRAs in the development of their sample 
size, methodology, data analysis, and reporting. Dr. Rousseau has previously worked closely with the 
MoE in Morocco to deliver capacity-strengthening workshops under the RFS-SSE activity. Dr. Rousseau 
holds a Ph.D. in Measurement and Evaluation and an M.A. in Measurement from the University of Laval 
in Canada. He is a native French speaker and fluent in English.
Gender Lead & Sr. Analyst, Melanie Phillips, Ph.D., will lead in aspects of the evaluation design, 
implementation, analysis, and reporting related to gender and will provide support to quantitative 
analysis. Dr. Phillips is a skilled researcher who uses a combination of empirical methods, including 
surveys, experiments, and in-depth fieldwork. She has studied the gender dynamics of women’s political 
representation in African countries and has taught graduate-level courses in data analysis and gender and 
international human rights. Dr. Phillips brings in-depth skills in quantitative data analysis and experience 
in all phases of the research process. She holds a Ph.D. from the University of California, Berkeley, in 
political science. 
Program Coordinator, Erica Wang, will support the Technical Manager in programmatic, 
budgetary, logistic, and contractual tasks. Ms. Wang has assisted in the administration and management 
of education projects funded by DFID and USAID, including projects in Uganda, Djibouti, Nepal, Malawi, 
Rwanda, and Ethiopia. She has experience in remotely monitoring data collections for early grade 
reading assessments (EGRAs), programming various assessments in Tangerine, KoBo Toolbox, and 
SurveyCTO, and tool and training adaptations. She has provided critical financial, managerial, and 
administrative support to projects ranging in size from $300,000 to $1.2 million. Ms. Wang is 
professionally proficient in French and holds bachelor’s degrees in government and politics, French 
language and literature, and piano performance from the University of Maryland, College Park. 
Editor & Graphic Designer, Laura Zasoski will provide professional editing services to all of STS’s 
major deliverables as well as graphic design and data visualization support to results summaries and 
presentations. Ms. Zasoski has more than ten years of experience working with education and non￾profits and specializing in editorial oversight, content development, and strategic messaging. She has 
overseen the structure, formatting, and editing of technical and research reports for projects funded by 
72
FCDO, USAID, USDA, the U.S. Department of Education, and domestic and international education 
agencies. She combines her skills in Microsoft Office and the Adobe Creative Suite to create data 
visualizations to supplement STS’s technical reports. She holds a master’s in education policy from the 
University of Maryland, College Park.
73
ANNEX V: SOURCES OF INFORMATION
Primary Data Sources:
All respondents received and signed consent forms regarding their participation. 
Table 12: KIIs and FGDs Conducted in the NPR Final Evaluation
RESPONDENT (N) SUBGROUPS TARGET 
(N)
CONDUCTED 
(N)
NOTES
USAID Morocco 
officials 
N/A At least 2 2 KIIs with 
education staff 
Upon consultation with the 
two USAID officials, it was 
not deemed necessary to 
conduct additional 
interviews.
Implementing partners N/A 2 2 KIIs with 
senior NPR staff 
MoE government 
officials 
MoE Central 
Office 
Officials
4 5 KIIs with 
central office 
respondents 
(Directorate of 
Curricula, 
CNIPE, CNEE, 
Division Pôle 
Manuels 
Scolaires, 
Division de la 
Promotion des 
Technologies 
Educatives, des 
Ressources 
Multimédias et 
des Techniques 
de 
Communication)
AREF 
regional 
directors and 
their 
assistants
6 6
Provincial 
directors
12 12
Inspectors 12 12
74
RESPONDENT (N) SUBGROUPS TARGET 
(N)
CONDUCTED 
(N)
NOTES
CMREF 
directors
2 5 This number was increased 
from 2 to 5 given availability 
in the data collection 
schedule and opportunity 
for richer data. 
Provincial￾level Arabic 
teacher 
trainers from 
CMREF
6 6
School-level 
respondents 
School 
directors
12 12 One school director was 
interviewed remotely.
Teachers 
(four to eight 
per school)
12 11 One teacher FGD was not 
conducted.
Parents (six 
to 10 per 
school)
12 11 One parent FGD was not 
conducted.
MSL Activity 
participants 
Two 
interpreters 
who played 
key roles in 
NPR’s MSL 
activity
2 2
Private sector 
stakeholders
Relevant 
personnel of 
3asafeer 
(educational 
tech 
company)
1 1 
Textbook 
publishers
2 2 Group KIIs were separately 
conducted with two 
publishing companies. 
Secondary Data Sources: 
• School-, teacher-, and pupil-level data for the NPR baseline study conducted in May 2018, the 
midline conducted in 2021, and the endline administered in 2022; 
• NPR datasets, including midline data collected for the positive/negative deviance school study, 
Arabic-language curriculum effectiveness study, and research on students’ well-being and 
learning loss due to the COVID-19 pandemic; and
• Sources listed below to conduct the literature review.
75
Creative Associates International, Inc., USAID Reading for Success-National Program for Reading Arabic 
Language Curricula Grades 5 and 6 Study: Findings Report, (2021).
—, USAID Reading for Success-National Program for Reading Monitoring, Evaluation, and Learning Plan 
(MELP)—Draft for Discussion, (2020).
—, USAID Reading for Success-National Program for Reading Policy Linking Report: Setting Benchmarks 
for Grades 1 to 4, (2021).
—, USAID Reading for Success-National Program for Reading Quarterly Report 3, Fiscal Year 2022, 
(2022).
—, USAID Reading for Success-National Program for Reading Quarterly Report 2, Fiscal Year 2022, 
(2022).
—, USAID Reading for Success-National Program for Reading Quarterly Report 1, Fiscal Year 2022, 
(2022).
—, USAID Reading for Success-National Program for Reading Fiscal Year 2021 Annual Report, (2021).
—, USAID Reading for Success-National Program for Reading Quarterly Report 3, Fiscal Year 2021, 
(2021).
—, USAID Reading for Success-National Program for Reading Quarterly Report 2, Fiscal Year 2021, 
(2021).
—, USAID Reading for Success-National Program for Reading Quarterly Report 1, Fiscal Year 2021, 
(2021).
—, USAID Reading for Success-National Program for Reading Fiscal Year 2020 Annual Report, (2020).
—, USAID Reading for Success-National Program for Reading Quarterly Report 3, Fiscal Year 2020, 
(2020).
—, USAID Reading for Success-National Program for Reading Quarterly Report 2, Fiscal Year 2020, 
(2020).
—, USAID Reading for Success-National Program for Reading Quarterly Report 1, Fiscal Year 2020, 
(2020).
—, USAID Reading for Success-National Program for Reading Fiscal Year 2019 Annual Report, (2019).
—, USAID Reading for Success-National Program for Reading Quarterly Report 3, Fiscal Year 2019, 
(2019).
—, USAID Reading for Success-National Program for Reading Quarterly Report 1, Fiscal Year 2019, 
(2019).
—, USAID Reading for Success-National Program for Reading Fiscal Year 2018 Annual Report, (2018).
—, USAID Reading for Success-National Program for Reading Quarterly Report 3, Fiscal Year 2018, 
(2018).
—, USAID Reading for Success-National Program for Reading Quarterly Report 2, Fiscal Year 2018, 
(2018).
—, USAID Reading for Success-National Program for Reading Quarterly Report 1, Fiscal Year 2018, 
(2018).
—, USAID Reading for Success-National Program for Reading School Deviance Study: Results, (2021).
—, USAID Reading for Success-National Program for Reading School Effectiveness Study: Baseline 
Report 2018, (2018).
—, USAID Reading for Success-National Program for Reading Student Learning and Remediation: 
Endline Results, (2021).
—, USAID Reading for Success-National Program for Reading Supplementary Reading Materials: Endline 
Results, (2021).
Social Impact, Inc. Morocco Curriculum Review and Capacity Building Activity (CCA) Final Report: June 
2021, (2021).
—. Morocco E-Takwine Learning Management System (LMS) Study: Final Report, (2022).
TALM/MRA Mobilising for Rights Associates. USAID/Morocco Gender Analysis Final Report: November 
2020, (2020).
76
USAID/Morocco, USAID/Morocco Country Development Cooperation Strategy—2020-2025, (2020).
—, Reading for Success-National Program for Reading Request for Task Order Proposal no, SOL-608-
16-000006: Section C—Description/Specifications/Statement of Work, (2016).
77
ANNEX VI: CONSENT FORMS
Consent Form for Parent Focus Group Discussions
USAID National Program for Reading (NPR) Performance Evaluation
School-to-School International
A. Form
This form is called a Consent Form. It will give you information about the study so you can make an 
informed decision about participation in this research. If you decide to participate, you will be asked to 
sign this form and you will be given a copy for your records.
B. Purpose
USAID/Morocco and the Ministry of National Education, Preschool, and Sports (MoE) have been active 
in promoting quality early grade reading education in Morocco for many years. We are working with 
School-to-School International to conduct a study to better understand how USAID and the MOE have 
been supporting children’s education at the primary level in Morocco. The staff at your children’s school 
identified you as a potential participant for this discussion because you have children studying in the 
grade levels the project is working in. We are particularly interested in hearing about your experience 
and understanding of MOE and USAID supported activities taking place at your child’s primary school 
that focus on improving children's reading skills and increasing community support for reading. 
However, we cannot guarantee any additional aid, services, or program action will take place as a result 
of your participation in this focus group discussion. We also cannot offer you any compensation for your 
participation. I hope you will participate since your views are very important to understanding how we 
can improve the education and lives of children in Morocco. There are no foreseeable risks involved in 
participating in this study.
C. Procedures
The focus group discussion should take between 1 – 1.5 hours. You do not need to answer all the 
questions in the discussion if you do not want to. If a question makes you feel uncomfortable or you do 
not know the answer, it is ok to tell the Interviewer that you do not want to answer the question. You 
can also withdraw from the focus group discussion at any time without any penalty. 
D. Confidentiality
Your answers will be private. To better keep track of all the information provided today, and to help the 
interviewer focus on facilitating this discussion, we will take notes on paper and would like to tape 
record this discussion so we can listen to it later. No one except those involved in the research study 
will have access to the discussion notes or tape recordings. We will only record the discussion on our 
audio-recorder with your permission. Nobody will be able to directly link your responses to your name. 
Your responses are confidential and your answers will remain anonymous. You will be given a number 
and your name will be kept in a confidential file at our main offices. Your name will not be associated 
with any of the data or reporting. 
Participants will be asked not to use any names during the focus group discussion. Please be advised that 
although the researchers will take every precaution to maintain confidentiality of the data, the nature of 
focus groups prevents the researchers from guaranteeing confidentiality. The researchers would like to 
remind participants to respect the privacy of your fellow participants and not repeat what is said in the 
focus group to others. The evaluation results will be shared with Government of Morocco counterparts 
and other donors to further promote improving basic education in Morocco through enhanced reading 
78
instruction in early grades. In addition, results will be shared with USAID audiences to highlight best 
practices and lessons learned, and the results achieved by USAID/Morocco in basic education and early 
grade reading.
E. Persons to contact 
You can ask the Facilitator any questions that you have at any time. You may also contact the people in 
charge of this study. 
Contact in Morocco: .
F. Agreement
Do you have any questions?
[Interviewer: Answer the participant’s questions about the interview before proceeding to the 
next question.]
You have read and/or had read to you the explanation of this focus group discussion, you have been 
given a copy of the consent form, a chance to ask questions, and you know that you can refuse to 
participate. I am going to ask for your consent to participate in this discussion. By saying yes, you agree 
to participate in the focus group. By saying no, you decline to participate. Do you agree to take part in 
the interview? 
[Interviewer: Confirm verbal consent has been obtained from the participant.]
G. Interviewer Statement of Confirming Voluntary Verbal Consent Obtained from all 
Focus Group Participants
[To be completed after consent has been obtained verbally from all participants and prior to beginning the focus 
group discussion.]
By signing below, I affirm that I have read this document to the all focus group participants and each 
participant has provided verbal consent. To the best of my knowledge, the participants understand the 
details contained in this form and have each been given a copy. I have explained to the participants the 
focus group discussion purpose and procedures and we have discussed all the risks that are involved. I 
have answered questions to the best of my ability that the participants had.
_________________________ ____________________ __________
Signature of Interviewer Print Name Date
79
Consent Form for Teacher Focus Group Discussions
USAID National Program for Reading (NPR) Performance Evaluation
School-to-School International
A. Form
This form is called a Consent Form. It will give you information about the study so you can make an 
informed decision about participation in this research. If you decide to participate, you will be asked to 
sign this form and you will be given a copy for your records.
B. Purpose
USAID/Morocco and the Ministry of National Education, Preschool, and Sports (MoE) have been active 
in promoting quality early grade reading education in Morocco for many years. We are working with 
School-to-School International to conduct an external performance evaluation of USAID’s education 
work under the National Program for Reading (NPR) Project. The objective of NPR is to support the 
improvement of instruction and reading and writing materials in primary schools, as well as learning and 
assessment systems. We are gathering data to better understand to what extent NPR has achieved its 
objectives and to inform the implementation of current and future activities in Morocco. You have been 
identified as a potential participant for this discussion because you are teaching in the grade levels that 
NPR supports. We would like to ask you some questions about your experience and engagement with 
NPR, including the new Arabic language curriculum, textbooks, supplementary reading materials, and 
other project activities. However, we cannot guarantee any additional aid, services, or program action 
will take place as a result of your participation in this focus group discussion. We also cannot offer you 
any compensation for your participation. I hope you will participate since your views are very important 
to understanding how we can improve the education and lives of children in Morocco. There are no 
foreseeable risks involved in participating in this study.
C. Procedures
The focus group discussion should take between 1 – 1.5 hours. During this discussion, we will ask you 
to reflect on relevant on your understanding and experience of NPR project activities, implementation, 
and lessons learned. You do not need to answer all the questions in the discussion if you do not want 
to. If a question makes you feel uncomfortable or you do not know the answer, it is ok to tell the 
Interviewer that you do not want to answer the question. You can also withdraw from the focus group 
discussion at any time without any penalty. 
D. Confidentiality
Your answers will be private. To better keep track of all the information provided today, and to help the 
interviewer focus on facilitating this discussion, we will take notes on paper and would like to tape 
record this discussion so we can listen to it later. No one except those involved in the research study 
will have access to the discussion notes or tape recordings. We will only record the discussion on our 
audio-recorder with your permission. Nobody will be able to directly link your responses to your name. 
Your responses are confidential and your answers will remain anonymous. You will be given a number 
and your name kept in a confidential file at our main offices. Your name will not be associated with any 
of the data or reporting.
Participants will be asked not to use any names during the focus group discussion. Please be advised that 
although the researchers will take every precaution to maintain confidentiality of the data, the nature of 
focus groups prevents the researchers from guaranteeing confidentiality. The researchers would like to 
remind participants to respect the privacy of your fellow participants and not repeat what is said in the 
focus group to others. The evaluation results will be shared with Government of Morocco counterparts 
80
and other donors to further promote improving basic education in Morocco through enhanced reading 
instruction in early grades. In addition, results will be shared with USAID audiences to highlight best 
practices and lessons learned, and the results achieved by USAID/Morocco in basic education and early 
grade reading.
E. Persons to contact 
You can ask the Facilitator any questions that you have at any time. You may also contact the people in 
charge of this study. 
Contact in Morocco: .
F. Agreement
Do you have any questions?
[Interviewer: Answer the participant’s questions about the focus group discussion before 
proceeding to the next question.]
You have read and/or had read to you the explanation of this focus group discussion, you have been 
given a copy of the consent form, a chance to ask questions, and you know that you can refuse to 
participate. I am going to ask for your consent to participate in this discussion. By saying yes, you agree 
to participate in the focus group. By saying no, you decline to participate. Do you agree to take part in 
the interview? 
[Interviewer: Confirm verbal consent has been obtained from the participant.]
G. Interviewer Statement of Confirming Voluntary Consent Obtained from all Focus 
Group Participants 
By signing below I indicate that the all focus group participants have read and, to the best of my 
knowledge, understand the details contained in this document and have each been given a copy. I have 
explained to the participants the focus group discussion purpose and procedures and we have discussed 
all the risks that are involved. I have answered questions to the best of my ability that the participants 
had.
_________________________ ____________________ __________
Signature of Interviewer Print Name Date
81
Consent Form for Key Informant Interviews
USAID National Program for Reading (NPR) Performance Evaluation
School-to-School International
This Consent Form gives you information about our research so you can make an informed decision 
about participating. If you decide to participate, you will be asked to sign this form and you will be given 
a copy for your records.
A. Purpose
USAID/Morocco and the Ministry of National Education, Preschool, and Sports (MoE) have been active 
in promoting quality early grade reading education in Morocco for many years. We are working School￾to-School international to conduct an external performance evaluation of USAID’s education work 
under the National Program for Reading (NPR) Project. The objective of NPR is to support the 
improvement of instruction and reading and writing materials in primary schools, as well as learning and 
assessment systems. We are gathering data to better understand to what extent NPR has achieved its 
objectives and to inform the implementation of current and future activities in Morocco. You have been 
identified as a project key stakeholder. We would like to ask you some questions about your work with 
NPR, including its impact on learning outcomes; the MoE’s efforts to scale project interventions and use 
assessment data; and lessons learned from the project’s engagement with parents and communities, 
collaboration with the private sector, and implementation and reception from different beneficiaries. 
However, we cannot guarantee any additional aid, services, or program action will take place as a result 
of your participation in this interview. We also cannot offer you any compensation for your 
participation. I hope you will participate as your views are very important to understanding how we can 
improve the education and lives of children in Morocco. There are no foreseeable risks involved in 
participating in this study.
B. Procedures
The interview should take no more than one hour. During this discussion, we will ask you to 
reflect on your understanding and experience of NPR education activity, implementation, and lessons 
learned. You do not need to answer all the questions in the interview if you do not want to.
If a question makes you feel uncomfortable or you do not know the answer, it is okay to tell the 
Interviewer that you do not want to answer the question. You can also stop the Interview at any 
time without any penalty.
C. Confidentiality
Please be assured that your response will remain confidential and private. To better keep 
track of all the information provided today, and to help the interviewer focus on facilitating this 
discussion, we will take notes on paper and we would also like to tape record this discussion so we can 
listen to it later, if we have your permission. No one except those involved in the research study 
will have access to the interview notes or tape recordings. Your name will never be used in 
connection with any of the information you tell without your permission. The evaluation 
results will be shared with USAID audiences, Government of Morocco counterparts and other donors 
to highlight best practices and lessons learned, and the results achieved by NPR.
D. Persons to contact 
You can ask the Interviewer any questions that you have at any time. You may also contact the people in 
charge of this study. Our contact in Morocco is . 
F. Participant Statement of Voluntary Consent
82
When signing this form, I am agreeing to voluntarily participate in this interview. I have had a 
chance to read this consent form, and it was explained to me in a language which I use. I have 
had the opportunity to ask questions and have received satisfactory answers. I have been 
informed that I can withdraw at any time. A copy of this signed Informed Consent Form has been 
given to me.
________________________ ____________________ __________
Participant Signature Print Name Date
G. Interviewer Statement of Confirming Voluntary Consent Obtained
By signing below, I indicate that the participant has read and, to the best of my knowledge, understands 
the details contained in this document and has been given a copy. I have explained to the participant the 
interview purpose and procedures and we have discussed all the risks that are involved. I have answered 
questions to the best of my ability that the participant had.
_________________________ ____________________ __________
Signature of Interviewer Print Name Date
83
Consent Form for Key Informant Interviews (Remote)
USAID National Program for Reading (NPR) Performance Evaluation
School-to-School International
This Consent Form gives you information about our research so you can make an informed decision 
about participating. If you decide to participate, you will be asked to sign this form and you will be given 
a copy for your records.
A. Purpose
USAID/Morocco and the Ministry of National Education, Preschool, and Sports (MoE) have been active 
in promoting quality early grade reading education in Morocco for many years. We are working with 
School-to-School international to conduct an external performance evaluation of USAID’s education 
work under the National Program for Reading (NPR) Project. The objective of NPR is to support the 
improvement of instruction and reading and writing materials in primary schools, as well as learning and 
assessment systems. We are gathering data to better understand to what extent NPR has achieved its 
objectives and to inform the implementation of current and future activities in Morocco. You have been 
identified as a project key stakeholder. We would like to ask you some questions about your work with 
NPR, including its impact on learning outcomes; the MoE’s efforts to scale project interventions and use 
assessment data; and lessons learned from the project’s engagement with parents and communities, 
collaboration with the private sector, and implementation and reception from different beneficiaries. 
However, we cannot guarantee any additional aid, services, or program action will take place as a result 
of your participation in this interview. We also cannot offer you any compensation for your 
participation. I hope you will participate as your views are very important to understanding how we can 
improve the education and lives of children in Morocco. There are no foreseeable risks involved in 
participating in this study.
B. Procedures
The interview should take no more than one hour. During this discussion, we will ask you to 
reflect on your understanding and experience of NPR education activity outcomes, implementation, and 
lessons learned. You do not need to answer all the questions in the interview if you do not 
want to. If a question makes you feel uncomfortable or you do not know the answer, it is okay to tell 
the Interviewer that you do not want to answer the question. You can also stop the Interview at 
any time without any penalty.
C. Confidentiality
Please be assured that your response will remain confidential and private. To better keep 
track of all the information provided today, and to help the interviewer focus on facilitating this 
discussion, we will take notes on paper and we would also like to tape record this discussion so we can 
listen to it later, if we have your permission. No one except those involved in the research study 
will have access to the interview notes or tape recordings. Your name will never be used in 
connection with any of the information you tell without your permission. The evaluation 
results will be shared with USAID audiences, Government of Morocco counterparts and other donors 
to highlight best practices and lessons learned, and the results achieved by NPR.
D. Persons to contact 
You can ask the Interviewer any questions that you have at any time. You may also contact the people in 
charge of this study. Our contact in for this study is .
F. Participant Statement of Voluntary Consent
84
When signing this form, I am agreeing to voluntarily participate in this interview. I have had a 
chance to read this consent form, and it was explained to me in a language which I use. I have 
had the opportunity to ask questions and have received satisfactory answers. I have been 
informed that I can withdraw at any time. A copy of this signed Informed Consent Form has been 
given to me.
________________________ ____________________ __________
Participant Signature Print Name Date
G. Interviewer Statement of Confirming Voluntary Consent Obtained
By signing below, I indicate that the participant has read and, to the best of my knowledge, understands 
the details contained in this document and has been given a copy. I have explained to the participant the 
interview purpose and procedures and we have discussed all the risks that are involved. I have answered 
questions to the best of my ability that the participant had.
_________________________ ____________________ __________
Signature of Interviewer Print Name Date
85
ANNEX VII: DATA COLLECTION INSTRUMENTS
Focus Group Discussion Guide: Parents
Qualitative Instruments: USAID NPR Performance Evaluation
A note about this tool:
Purpose: This focus group discussion (FGD) guide will enable you to gather information from parents 
to get an overview about their involvement with the NPR project. The objective is to gain insights and 
understanding of their perception of NPR activities, project outcomes, relevance and effectiveness of 
specific activities, inclusion, and lessons learned.
Recommended sources: Information should be collected through eight FGDs with parents of primary 
school students (grades 1-6) who are active in school management committees, parent organizations, or 
other school groups, including a cross-section of parents of high-performing and low-performing 
students. Up to 10 parents should participate in each FGD. FGDs will aim for gender-balanced 
participation where possible; for example, half of the parents of students in early primary grades 
participating in FGDs will be fathers/male guardians and half will be mothers/female guardians. Only one 
parent or guardian is expected per family.
Background
A. Respondent Type:_____________________________________________________
B. School Name:________________________________________________________
C. Province: __________________________________________________________
D. Region: ____________________________________________________________
E. Facilitator name: ____________________________________________________
F. Note taker name: ____________________________________________________
G. Date: ______________________________________________________________
H. Start Time: _________________________________________________________
I. Introduce team members and obtain consent: ___ YES ____ NO 
J. State purpose of evaluation: ___ YES ____ NO
Introduction and Consent 
[NOTE TO INTERVIEWER: please review and complete the FGD consent form with the participants 
individually. Once agreement to participate voluntarily is obtained from all participants, please continue. 
If a participant does NOT agree to take part in the study, thank them for their time, and make a note 
they did not want to participate. While reviewing and completing the consent form, make sure to also 
ask for permission to record the interview on our audio-recorder; if all participants say YES please 
continue; if any participant says NO, acknowledge that you will not record the conversation and 
proceed without turning on the audio-recording.]
**IMPORTANT NOTE**:
Only begin audio-recording AFTER consent to participate and be audio-recorded has been received from ALL 
participants. If consent is not obtained from all participants, do not audio-record the discussion. Indicate in notes 
if discussion was not audio-recorded. 
1. The MoE has worked with USAID to develop a new Arabic language curriculum in grades 1 to 6. 
This curriculum was piloted in select schools and then scaled nationwide in 2021. The program 
consisted of a) curriculum materials; b) new textbooks; c) access to the MOOC content; d) 
86
supplemental reading materials (books and teachers’ guides); e) any other training activities. Are 
you aware of this program? 
a. If yes, have you seen any of the new materials (textbooks and supplemental reading 
materials)? 
i. If yes, do you have thee mateirls at home? Which ones?
1. If yes, have you used the materials at all with your child/ren at home? 
How? 
2. If yes, do you find them easy to use? Why/why not? 
3. If yes, are they enjoyable for your children? 
b. If yes, are the new materials (textbooks and supplemental reading materials) helping 
your child/ren learn how to read?
2. How has COVID impacted your children's education? Have you observed any differences in 
their attitude about school since schools reopened after the pandemic? If yes, what are they?
Now I’d like to ask about activities for engaging community and parental support for reading. 
3. Have teachers or school directors organized any outreach activities to help you with the use of 
supplementary reading materials and the 3asafeer digital library at home? (NOTE: Only ask 
about the digital library in experimental schools.) 
a. If yes, what type of guidance have they provided to you? 
b. If yes, did you follow their guidance? 
c. If yes, did your children read the materials? 
4. Has the MoE or your school conducted any other activities to engage you as parents or the 
community as a whole? 
a. If yes, what were the activities?
b. If yes, are you more engaged with your children’s education due to the activities?
c. If yes, is the community more engaged with your children’s education due to the 
activities?
5. [NOTE: If they have said earlier that the new curriculum has been introduced in this school] Do 
you think the new way of teaching reading, including the new textbooks and supplementary 
reading materials, helps girls and boys learn equally well? 
a. How can this new approach to teaching reading be improved to better meet the needs 
of both girls and boys?
6. Have you seen any notable differences in how the new curriculum has been received by 
students whose mother tongue is Arabic and those whose mother tongue is Amazigh? [NOTE: 
If respondent notes differences, follow up if the curriculum and materials meet the needs of one 
87
group more than another.] 
a. If yes, please describe these differences. 
b. If yes, have there been any steps taken to address them?
7. Have you seen any notable differences in how the new curriculum has been received by children 
with disabilities, or their teachers or parents/guardians? 
a. If yes, please describe these differences. 
b. If yes, have there been any steps taken to address them?
8. Overall, have there been any unexpected outcomes or results (positive or negative) with this 
new approach to learning how to read? Please detail.
Closing
1. Those are all of my questions. Are there any additional comments that you would 
like to share with us?
2. Thank you very much for taking the time to answer our questions. Do you have any 
questions for our team?
Summary and Conclusion
Thank you for your time. Your help in this research is very important. As I mentioned, 
the results of the report will be used by USAID Morocco and the Ministry of National 
Education, Preschool, and Sports (MoE) to understand how the NPR project activities 
are going and are being implemented. The final results of our research project will be 
published in a report in the coming months. These results will be communicated back to 
USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
POST-FOCUS GROUP NOTES:
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
88
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
________________________________________________________
89
Focus Group Discussion Guide: Teachers - Experimental Schools
A note about this tool:
Purpose: This focus group discussion (FGD) guide will enable you to gather information from teachers 
to get an overview about their involvement with the NPR project. The objective is to gain insights and 
understanding of NPR project outcomes, project implementation, inclusion, and lessons learned. 
Recommended sources: Information should be collected through eight FGDs with primary school 
teachers (grade 1-6) in NPR pilot schools. Up to 10 teachers should participate in each FGD. FGDs will 
aim for gender-balanced participation where possible; for example, half of the teachers participating in 
FGDs will be male and half will be female.
Qualitative Instruments: USAID NPR Performance Evaluation
Background
A. Respondent Type:_____________________________________________________
B. School Name:________________________________________________________
C. Province: __________________________________________________________
D. Region: ____________________________________________________________
E. Facilitator name: ____________________________________________________
F. Note taker name: ____________________________________________________
G. Date: ______________________________________________________________
H. Start Time: _________________________________________________________
I. Introduce team members and obtain consent: ___ YES ____ NO
J. State purpose of evaluation: ___ YES ____ NO
Introduction and Consent 
[NOTE TO INTERVIEWER: please review and complete the FGD consent form with the 
participants. Once agreement to participate voluntarily is obtained from all participants, please 
continue. If a participant does NOT agree to take part in the study, thank them for their time, and 
make a note they did not want to participate. While reviewing and completing the consent form, 
make sure to also ask for permission to record the interview on our audio-recorder; if all 
participants say YES please continue; if any participants say NO, acknowledge that you will not 
record the conversation and proceed without turning on the audio-recording.]
**IMPORTANT NOTE**:
Only begin audio-recording AFTER consent to participate and be audio-recorded has been received from ALL 
participants. If consent is not obtained from all participants, do not audio-record the discussion. Indicate in 
notes if discussion was not audio-recorded.
1. The MoE developed and rolled out an early grade reading curriculum and teacher 
instruction package called NPR in 90 experimental schools. As you probably know, 
the program consisted of a) curriculum materials; b) new textbooks; c) access to 
the MOOC content; d) supplemental reading materials (books and teachers’ 
guides); e) capacity building on measurement and evaluation; f) any other training 
activities. Were each of these activities and materials delivered at your school? 
90
2. Do you all have the teacher’s guide? Does anyone here not have it? 
a. If you have the guide, how frequently do you use the teacher’s guide? 
b. If you have the guide, is it easy to use? Why/why not? 
i. If it's easy to use, in what ways does it help you teach reading? 
c. If you do not have it, what materials do you use to prepare your lessons for class? 
3. Have you received the new textbooks?
a. If yes, do they help your students learn how to read? Why/why not?"
4. What have been NPR's most important and valuable contributions to improving 
reading for learners in your school? Why are these contributions the most valuable?
5. Within the different elements of NPR, what has met your expectations? Why? What has not 
met your expectations? Why, and how would you suggest these shortcomings be addressed?
6. Has COVID specifically affected the rollout of the curriculum reform? How?
7. As part of NPR, teachers were trained in experimental schools in differentiated 
instruction, use of visual aids, and school-based teacher support and follow-up 
(TSF). Did you participate in any of these trainings? If so, did you benefit from 
them? How? How would you rate them—excellent, good, fair, poor? Did they 
improve your skills in teaching reading and/or your director's skills in supporting 
you? If yes, how?
8. NPR developed a series of distance-learning courses—or massive open online 
courses (MOOC)—to train teachers on the new curriculum. Did any of you enroll in 
these courses? If so, did you benefit from them? How? How would you rate them—
excellent, good, fair, poor? If so, did the MOOCs improve your ability to teach 
reading with your (special population)? In what ways? 
9. In 2020, NPR printed and distributed supplementary reading materials and teachers’ guides in 
experimental schools, as well as provided access to digital reading materials. Can children access 
these materials in your classroom? If so, how much time per week do your students use the 
SRMs? Have SRMs helped to improve early grade reading skills in your school? In what ways? 
What did they do well? What would you like to see done differently?
10. Are you aware of EGRA—the Early Grade Reading Assessment—a tool for measuring children's 
reading ability in the early grades? If so, has EGRA been used to assess children's reading ability 
in your classroom? If yes, do you know the results? If yes, what have you done with the results?
11. NPR created a formative assessment toolkit, including a formative assessment 
guide, student remediation guide, and a teacher’s guide to unit tests for grades 1 to 
4. Have you received any of these materials? If so, how often do you use them? Do 
you think they helped your learners learn to read better? How?
12. What additional support could the MoE provide to better help you implement the 
new Arabic language curriculum?
Now I’d like to ask about NPR’s activities for engaging community and parental support for reading. 
91
13. Have you organized any outreach activities to help parents with the use of supplementary 
reading materials and the 3asafeer digital library at home? If yes, what type of guidance have you 
provided to parents? Did the parents follow the guidance? Did students read the materials? 
14. Have you been involved with any other activities as part of NPR to increase community and 
parental support for reading? If so, what, and did anything change regarding community and 
parental support due to the activities?
15. Have you seen any notable differences in how the new curriculum has been received 
by children from different backgrounds – e.g., from certain regions/provinces, 
Arabic vs Amazigh, boys vs girls, children with disabilities, urban vs. rural? If yes, 
please describe these differences. Have there been any steps taken to address 
them?
16. Overall, have there been any unexpected outcomes or results (positive or negative) 
with NPR? Please detail.
Closing
1. Are there any additional comments that you would like to share with us? 
2. Thank you very much for taking the time to answer our questions. Do you have any 
questions for our team?
Summary and Conclusion
Thank you for your time. Your help in this research is very important. As I mentioned, 
the results of the report will be used by USAID Morocco and the Ministry of National 
Education, Preschool, and Sports (MoE) to understand how the NPR project activities 
are going and are being implemented. The final results of our research project will be 
published in a report in the coming months. These results will be communicated back to 
USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
POST-FOCUS GROUP NOTES:
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
92
______________________________________________________________________
______________________________________________________________________
________________________________________________
93
Focus Group Discussion Guide: Teachers - Scale-Up Schools
Qualitative Instruments: USAID NPR Performance Evaluation
A note about this tool:
Purpose: This focus group discussion (FGD) guide will enable you to gather information from teachers 
to get an overview about their involvement with the NPR project. The objective is to gain insights and 
understanding of NPR project outcomes, project implementation, inclusion, and lessons learned. 
Recommended sources: Information should be collected through eight FGDs with primary school 
teachers (grade 1-6) in NPR pilot schools. Up to 10 teachers should participate in each FGD. FGDs will 
aim for gender-balanced participation where possible; for example, half of the teachers participating in 
FGDs will be male and half will be female.
Background
A. Respondent Type:_____________________________________________________
B. School Name:________________________________________________________
C. Province: __________________________________________________________
D. Region: ____________________________________________________________
E. Facilitator name: ____________________________________________________
F. Note taker name: ____________________________________________________
G. Date: ______________________________________________________________
H. Start Time: _________________________________________________________
I. Introduce team members and obtain consent: ___ YES ____ NO
J. State purpose of evaluation: ___ YES ____ NO
Introduction and Consent 
[NOTE TO INTERVIEWER: please review and complete the FGD consent form with the 
participants. Once agreement to participate voluntarily is obtained from all participants, please 
continue. If a participant does NOT agree to take part in the study, thank them for their time, and 
make a note they did not want to participate. While reviewing and completing the consent form, 
make sure to also ask for permission to record the interview on our audio-recorder; if all 
participants say YES please continue; if any participants say NO, acknowledge that you will not 
record the conversation and proceed without turning on the audio-recording.]
**IMPORTANT NOTE**:
Only begin audio-recording AFTER consent to participate and be audio-recorded has been received from ALL 
participants. If consent is not obtained from all participants, do not audio-record the discussion. Indicate in 
notes if discussion was not audio-recorded.
1. The MoE has worked with USAID to develop a new Arabic language curriculum in 
grades 1 to 6. This curriculum was piloted in select schools and then scaled 
nationwide in 2021. The program consisted of a) curriculum materials; b) new 
textbooks; c) access to the MOOC content; d) supplemental reading materials 
(books and teachers’ guides); e) any other training activities. Are you aware of this 
program? If so, were each of these activities and materials delivered at your school? 
94
2. Do you all have the teacher’s guide? Does anyone here not have it? 
a. If you have the guide, how frequently do you use it? 
b. If you have the guide, is it easy to use? Why/why not? 
i. If it's easy to use, in what ways does it help you teach reading? 
c. If you do not have it, what materials do you use to prepare your lessons for class?
3. Do the new textbooks help your students learn how to read? Why/why not?
4. [Ask only if new curriculum is being implemented at the school] What have been 
the new curriculum's most important and valuable contributions to improving 
reading for learners in your school? Why are these contributions the most valuable?
5. [Ask only if new curriculum is being implemented at the school] Within the different elements 
of the new curriculum, what has met your expectations? Why? What has not met your 
expectations? Why, and how would you suggest these shortcomings be addressed?
6. [Ask only if new curriculum is being implemented at the school] Has COVID specifically affected 
the rollout of the curriculum reform? How?
7. Have you received any training from inspectors or anyone else on the new 
curriculum? 
a. If so, when, and on what topics? 
b. If so, did you benefit from them? How? 
c. Do you think the training covered all you need to know in order to adopt the 
new curriculum and use the new materials at your school? Why/why not? 
i. If no, what additional training do you need?
8. The MoE developed a series of distance-learning courses—or massive open online 
courses (MOOC)—to train teachers on the new curriculum. Are you aware of these 
courses? 
a. If yes, have you enrolled in these courses? 
b. If so, did you benefit from them? How? 
c. How would you rate them—excellent, good, fair, poor? 
d. If so, have the MOOCs improved your ability to teach reading with your 
students (example, students with special needs, marginalized groups, etc.)? 
In what ways? 
9. In 2022, the MoE printed and distributed supplementary reading materials and teachers’ guides 
nationwide. Can children access these materials in your classroom? 
a. If so, how much time per week do your students use the SRMs? 
b. Have the SRMs helped to improve early grade reading skills in your school? In what 
95
ways? 
c. What did they do well? 
d. What would you like to see done differently?
10. The MoE created a formative assessment toolkit, including a formative assessment 
guide, student remediation guide, and a teacher’s guide to unit tests for grades 1 to 
4. Have you received these materials? If so, how often do you use them? If so, do 
you think they helped your learners learn to read better? How?
11. What additional support could the MoE provide to better help you implement the 
new Arabic language curriculum?
Now I’d like to ask about activities for engaging community and parental support for reading. 
12. Have you organized any outreach activities to help parents with the use of supplementary 
reading materials at home? If yes, what type of guidance have you provided to parents? Did the 
parents follow the guidance? Did students read the materials?
13. Have you been involved with any other activities as part of the introduction of the new 
curriculum to increase community and parental support for reading? If so, what, and did anything 
change regarding community and parental support due to the activities?
14. Have you seen any notable differences in how the new curriculum has been received 
by children from different backgrounds – e.g., from certain regions/provinces, 
Arabic vs Amazigh, boys vs girls, children with disabilities, urban vs. rural? If yes, 
please describe these differences. Have there been any steps taken to address 
them?
15. Overall, have there been any unexpected outcomes or results (positive or negative) 
with the new curriculum? Please detail.
Closing
1. Are there any additional comments that you would like to share with us? 
2. Thank you very much for taking the time to answer our questions. Do you have any 
questions for our team?
Summary and Conclusion
Thank you for your time. Your help in this research is very important. As I mentioned, 
the results of the report will be used by USAID Morocco and the Ministry of National 
Education, Preschool, and Sports (MoE) to understand how the NPR project activities 
are going and are being implemented. The final results of our research project will be 
published in a report in the coming months. These results will be communicated back to 
USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
96
POST-FOCUS GROUP NOTES:
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
________________________________________________________________
97
Key Informant Interview Guide: Implementers
A note about this tool:
Purpose: This focus group discussion (FGD) guide will enable you to gather information from teachers 
to get an overview about their involvement with the NPR project. The objective is to gain insights and 
understanding of NPR project outcomes, project implementation, inclusion, and lessons learned. 
Recommended sources: Information should be collected through eight FGDs with primary school 
teachers (grade 1-6) in NPR pilot schools. Up to 10 teachers should participate in each FGD. FGDs will 
aim for gender-balanced participation where possible; for example, half of the teachers participating in 
FGDs will be male and half will be female.
Qualitative Instruments: USAID NPR Performance Evaluation
Background
A. Respondent type:_____________________________________________________
B. Respondent role/position:_______________________________________
C. Respondent Organization: _______________________________________
D. Gender: _______________________________________
E. Age: _______________________________________
F. Interviewer name: 
____________________________________________________________
G. Note taker name: 
_____________________________________________________________
H. Date: 
_____________________________________________________________________
__
I. Start Time: 
__________________________________________________________________
J. Introduce team members and obtain consent: ___ YES ____ NO
K. State purpose of evaluation: ___ YES ____ NO
Introduction and Consent 
[NOTE TO INTERVIEWER: Please review and complete the KII Consent form with the participant. If 
agreement to participate voluntarily is obtained, please continue. If participant does NOT agree to take 
part in the study, thank them for their time, make a note they did not want to participate. Make sure to 
also ask for permission to record the interview on our audio-recorder; if participant says YES please 
continue; if participant says NO, acknowledge that you will not record the conversation and proceed 
without turning on the audio-recording.]
**IMPORTANT NOTE**:
Begin audio-recording AFTER consent to participate and be audio-recorded has been received.
98
1. In your view, has NPR helped to increase the number of primary school children 
reading fluently and with comprehension skills in NPR’s experimental schools? If 
yes, which aspects of NPR contributed to these improvements? What have been 
NPR’s most important and valuable contributions to improving children's reading in 
its experimental schools? Why do you consider these the most important or 
valuable contributions?
2. NPR included two broad elements: curriculum and materials development and 
teacher instructional training. Has either of these elements met your expectations? 
If yes, in what ways? What has not met your expectations? Why, and how would you 
suggest these shortcomings be addressed?
3. What kinds of internal, NPR-related concerns or obstacles, if any, have impeded 
implementation of these activities? How has COVID specifically impeded these 
activities? Besides COVID, have any other external, contextual challenges impeded 
NPR? If so, how?
4. How well did NPR deliver on the creation and rollout of the curriculum in its 
experimental schools? What has impeded creation and rollout? Has the new 
curriculum addressed issues with the former curriculum?
5. NPR trained teachers in its experimental schools in differentiated instruction, use of visual aids, 
and school-based teacher support and follow-up (TSF). Have these interventions improved 
teacher instruction in NPR’s experimental schools? If yes, in what ways? What has impeded 
teacher instruction interventions?
6. NPR developed a series of distance-learning courses—or massive open online courses 
(MOOC)— to train teachers on the new curriculum. Have the MOOCs improved teachers’ 
instruction? If yes, in what ways? What has impeded the development of MOOCs and their use 
by teachers?
7. In 2020, NPR printed and distributed supplementary reading materials and teachers’ guides in 
experimental schools (show examples of materials). Has the supplemental reading materials 
program helped to improve children’s reading skills in experimental schools? If yes, in what 
ways? What has impeded implementation of the supplemental reading materials program?
8. Is the MoE’s ability to develop and deliver education reform in primary grades 
stronger after its participation in NPR? If yes, in what ways, and what role did NPR 
play in improving the MoE’s ability? Are there any areas or ways in which the MoE’s 
ability to develop and deliver education reform could still be improved?
I’d also like to ask you about NPR’s activities to develop the MoE’s capacity for 
measurement and evaluation of improved reading outcomes. 
9. Starting in 2019, NPR organized a series of capacity building-workshops for MoE 
personnel on evaluation. The general topics included assessment design; item 
development; developing a research agenda; benchmarking; data analysis, 
interpretation, and use; and test equating. If NPR was going to plan this series of 
capacity-building workshops again, should NPR do anything differently? If so, what 
and why? [PROBE: Different topics, format, time, materials, etc.]
10. Have these activities improved the MoE’s institutional ability to conduct EGRAs and 
use the data for informing implementation? Why or why not?
99
11. Has the MoE used—or have any plans to use—any EGRA data to make any changes 
to the design of the Arabic language education reform (either curriculum, 
materials, teacher training, data use, etc.)? If so, how? 
12. NPR created a formative assessment toolkit, including a formative assessment 
guide, student remediation guide, and a teacher’s guide to unit tests for grades 1 to 
4. To what extent did MoE personnel find these materials valuable? To what extent 
did they use them? To what extent have the formative assessment materials been 
scaled nationwide? 
a. Has the MoE used—or have any plans to use—any formative assessment data 
to make any changes to the design of the Arabic language education reform 
(either curriculum, materials, teacher training, data use, etc.)? If so, how? 
13. Does the MoE need any support to use data to support adaptive management and 
improve learning outcomes in the future? If so, what kind of support?
Now I’d like to ask about the launch of the new curriculum nationally. Before the start of 
the 2021-22 school year, the MoE announced that schools were required to use the new 
Arabic language curriculum introduced by NPR, and 8,500 copies of the curriculum were 
printed and distributed to schools nationwide. 
14. What did the MoE do to support the launch of the new curriculum and materials 
nationwide at the start of the 2021-22 school year? Did it provide enough support to 
regional and provincial education offices in order for schools nationwide to 
implement the curriculum and use the new materials as designed? Why/why not?
15. In February 2022, the MoE urgently requested that NPR develop a toolkit to help 
inspectors provide in-service training to teachers about the new Arabic language 
curriculum. What prompted this request? Do you know if inspectors have been 
using this toolkit since NPR submitted it to the MoE? If yes, has the inspectors’ 
training led to schools nationwide implementing the new curriculum and using the 
new materials as designed? Why/why not? 
16. To your knowledge, are teachers nationwide implementing the new curriculum and materials? If 
so, what proportion of teachers are implementing it—all, most, some, or few? [Ask if previous 
response is most, some, or few] Why aren't all schools implementing it? What additional 
support could the MoE provide to ensure all schools are able to implement the new Arabic 
language curriculum?
17. What did the MoE do well with scaling the new curriculum nationally? What did it not do well? 
What was the biggest obstacle the MoE had to overcome in scaling the curriculum, and how did 
it address it?
Now I’d like to ask about NPR’s activities for engaging community and parental support for reading. 
18. What kind of outreach activities did NPR train teachers and school directors to organize to help 
parents with the use of supplementary reading materials and the 3asafeer digital library at home? 
To your knowledge, how many teachers and school directors organized these activities—all, 
most, some, or few? Did parents follow the guidance? Did students read the materials?
19. What other activities did NPR implement to increase community and parental support for 
reading? Did anything change regarding community and parental support due to the activities?
100
Now I’d like to ask about NPR’s engagement with the private sector in reading programs. 
20. NPR and the MoE worked with the private textbook industry during the 
development of the new curriculum. What were the textbook industry’s strengths 
and weaknesses in this partnership? Were there any obstacles NPR had to 
overcome? If so, what did NPR try to do to overcome them?
21. NPR and the MoE worked with the company 3asafeer during the development of 
the supplemental reading materials program. What were 3asafeer’s strengths and 
weaknesses in this partnership? Were there any obstacles NPR had to overcome? If 
so, what did NPR try to do to overcome them? 
Now I’d like to ask about NPR’s implementation and reception in urban and rural areas, different 
regions of the country, and with girls and boys.
22. Have you seen any notable differences in how the new curriculum has been received 
by children from different backgrounds – e.g., from certain regions/provinces, 
Arabic vs Amazigh, boys vs girls, children with disabilities, urban vs. rural? If yes, 
please describe these differences. Have there been any steps taken to address 
them?
23. Overall, have there been any unexpected outcomes or results (positive or negative) 
with NPR? Please detail.
CLOSING
1. Those are all of my questions. Are there any additional comments that you would like to 
share with us?
2. Thank you very much for taking the time to answer our questions. Do you have any 
questions for our team?
SUMMARY AND CONCLUSION
Thank you for your time. Your help in this research is very important. As I mentioned, 
the results of the report will be used by USAID Morocco and the Ministry of National 
Education, Preschool, and Sports (MoE) to understand how the NPR project activities 
are going and are being implemented. The final results of our research project will be 
published in a report in the coming months. These results will be communicated back 
to USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
POST-KII NOTES:
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
101
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
102
Key Informant Interview Guide: Ministry of Education Communications 
Officer
About this tool:
Purpose: This interview guide will enable you to gather information from key informants in the Ministry 
of Education (MOE) to gain insight into their involvement in the NPR program. This is to gain insight and 
understanding of their engagement with NPR activities and implementing partners, with the new Arabic 
language curricula introduced under the NPR program, the inclusion of various stakeholders, and the 
sustainability of approaches.
Recommended Sources: Information should be gathered from a wide range of provincial, regional, 
and national MOE officials. 
The provincial KIIs will be conducted in twelve provinces within the same six regions as the schools, i.e. 
the four regions where NPR experimental schools are located and the two regions where the new 
reading curriculum and materials have been scaled. In each of the twelve provinces, two MoE officials 
will be interviewed—the provincial director and the inspector/provincial coordinator. In addition, six 
provincial Arabic teacher trainers from the sampled provinces will be interviewed.
At the regional level, interviewees will include the MoE regional director in each of the six regions 
visited, as well as two regional directors of teacher training centers from the sampled regions. 
National-level interviews will be conducted with six officials from the MoE. Within the MoE, 
interviewees may include the same six directors of divisions who were interviewed in 2019: the 
Directorate of Strategy, Statistics, and Planning; the Communication Directorate; the Curriculum 
Directorate; the National Center for Pedagogical Innovations and Experimentation (CNIPE); the 
National Center for Evaluation and Examinations (CNEE); and the Directorate of Cooperation and 
Teaching in Private Schools. 
If more than one official is interested and available to participate at the designated date, time and 
location, the KII may be administered as a group interview but the potential impact of power dynamics 
(such as age, gender, and level of authority in the organization or ministry) within the interview must be 
taken into consideration and mitigated. 
Qualitative Instruments: Performance Evaluation for the USAID NPR Program
Context
A. Respondent's role/position: 
_____________________________________________________
B. Participating Department or Organization: 
______________________________________________
C. Region: __________________________________________
D. Province: ________________________________________
E. Sex: _______________________________________
F. Age: _______________________________________
G. Name of the interviewer: 
103
___________________________________________________________
H. Name of 
Notetaker:__________________________________________________________
I. Date: 
_______________________________________________________________________
J. Start time: 
__________________________________________________________________
K. Introduction of team members and obtaining consent: ___ YES____ NO
L. Indicate the purpose of the evaluation: ___ YES ____ NO
Introduction and Consent 
[NOTE TO INTERVIEWER: Please review and complete the CIS consent form with each participant. If 
you have obtained voluntary agreement to participate from each individual, please proceed. If a 
participant does not agree to take part in the study, thank them for their time and please note their 
wish not to participate. Also be sure to ask permission to record the interview on your audio device. If 
the participant answers yes, please continue. If the participant answers NO, agree not to record the 
conversation and continue without engaging your audio recorder].
**IMPORTANT NOTE**:
Do not begin recording until AFTER you have obtained consent to participate and record.
1. As you know, the Ministry of Education has been working with USAID to develop a 
curriculum reform - a new program for teaching Arabic in grades 1-6. This 
curriculum was piloted in selected "experimental schools" and then expanded to 
the entire country in 2021. The program included: a) teaching materials; b) new 
textbooks; c) a training toolkit for inspectors; d) access to MOOC content; e) 
additional reading materials (books and teacher guides); f) MSL video content; g) 
capacity building in measurement and assessment; h) any other training activities. 
Are you aware of this program? 
a. If so, what role did you play in this program?
2. The development and implementation of the new program lasted from 2017 to 
2022. During this time, have you noticed any changes in the perceptions of different 
populations (Ministry staff, teachers, parents) about the program?
3. During this same time period, were there changing communication strategies on 
the part of the Department on reform with these populations? 
a. If so, what changes in strategy have been made? 
b. Were these different strategies effective? Please explain.
4. To your knowledge, has the NPR program (implementation of the new curriculum) 
had an impact on:
a. Teaching Arabic reading in Morocco? Please explain.
b. Students' reading skills? Please explain.
104
c. Attitudes about the new curriculum or approach to teaching? Please explain.
5. Have you seen incidents where the new curriculum has had a different impact?
a. For girls versus boys?
b. For students from advantaged groups compared to their peers (e.g., special 
needs students, minority language groups, disadvantaged regions, etc.)?
CLOSING
1. We have come to the end of my questions. Do you have any additional comments?
2. Thank you very much for taking the time to answer our questions. Do you have any 
questions for our team?
A. SUMMARY AND CONCLUSION
Thank you for your time. Your help with this research is critical. As I mentioned, the 
results of the report will be used by USAID in Morocco and the MOE to understand 
how the RFS project activities are and will be implemented. The final results of our 
research under the project will be published in a report in the coming months. These 
results will be shared with USAID and the Ministry.
End time: ___________________________
Total time: ________Time _________Minutes
NOTES FOLLOWING KII :
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
105
Key Informant Interview Guide: Ministry of Education Official - Central
A note about this tool:
Purpose: This interview guide will enable you to gather information from the Ministry of National 
Education, Preschool, and Sports (MoE) key informants to get an overview about their involvement in 
the NPR project. The objective is to gain insights and understanding of their engagement with the NPR 
activities and implementers, the new Arabic language curricula introduced as part of NPR, inclusion, and 
sustainability.
Recommended sources: Information should be collected through with a variety of MoE officials at 
the provincial, regional, and national levels. 
The provincial KIIs will be conducted in twelve provinces within the same six regions as the schools, i.e. 
the four regions where NPR experimental schools are located and the two regions where the new 
reading curriculum and materials have been scaled. In each of the twelve provinces, two MoE officials 
will be interviewed—the provincial director and the inspector/provincial coordinator. In addition, six 
provincial Arabic teacher trainers from the sampled provinces will be interviewed.
At the regional level, interviewees will include the MoE regional director in each of the six regions 
visited, as well as two regional directors of teacher training centers from the sampled regions. 
National-level interviews will be conducted with six officials from the MoE. Within the MoE, 
interviewees may include the same six directors of divisions who were interviewed in 2019: the 
Directorate of Strategy, Statistics, and Planning; the Communication Directorate; the Curriculum 
Directorate; the National Center for Pedagogical Innovations and Experimentation (CNIPE); the 
National Center for Evaluation and Examinations (CNEE); and the Directorate of Cooperation and 
Teaching in Private Schools. 
If more than one official is interested and available to participate at the designated date, time and 
location, the KII may be administered as a group interview but the potential impact of power dynamics 
(such as age, gender, and level of authority in the organization or ministry) within the interview must be 
taken into consideration and mitigated. 
Qualitative Instruments: USAID NPR Performance Evaluation
Background
A. Respondent type:_____________________________________________________
B. Respondent role/position:_______________________________________
C. Respondent Ministry or Organization: _______________________________________
D. Region: __________________________________________
E. Province: ________________________________________
F. Gender: _______________________________________
G. Age: _______________________________________
H. Interviewer name: 
____________________________________________________________
106
I. Note taker name: 
_____________________________________________________________
J. Date: 
_______________________________________________________________________
K. Start Time: 
__________________________________________________________________
L. Introduce team members and obtain consent: ___ YES ____ NO
M. State purpose of evaluation: ___ YES ____ NO
Introduction and Consent 
[NOTE TO INTERVIEWER: Please review and complete the KII Consent form with the participant. If 
agreement to participate voluntarily is obtained, please continue. If participant does NOT agree to take 
part in the study, thank them for their time, make a note they did not want to participate. Make sure to 
also ask for permission to record the interview on our audio-recorder; if participant says YES please 
continue; if participant says NO, acknowledge that you will not record the conversation and proceed 
without turning on the audio-recording.]
**IMPORTANT NOTE**:
Begin audio-recording AFTER consent to participate and be audio-recorded has been received.
1. In your view, has NPR helped to increase the number of primary school children 
reading fluently and with comprehension skills in NPR’s experimental schools? If 
yes, which aspects of NPR contributed to these improvements? What have been 
NPR’s most important and valuable contributions to improving children's reading in 
its experimental schools? Why do you consider these the most important or 
valuable contributions?
2. NPR included two broad elements: curriculum and materials development and 
teacher instructional training. Has either of these elements met your expectations? 
If yes, in what ways? What has not met your expectations? Why, and how would you 
suggest these shortcomings be addressed?
3. What kinds of internal, NPR-related concerns or obstacles, if any, have impeded 
implementation of these activities? How has COVID specifically impeded these 
activities? Besides COVID, have any other external, contextual challenges impeded 
NPR? If so, how?
4. How well did NPR deliver on the creation and rollout of the curriculum in its 
experimental schools? What has impeded creation and rollout? Has the new 
curriculum addressed issues with the former curriculum?
5. NPR trained teachers in its experimental schools in differentiated instruction, use of visual aids, 
and school-based teacher support and follow-up (TSF). Have these interventions improved 
teacher instruction in NPR’s experimental schools? If yes, in what ways? What has impeded 
teacher instruction interventions?
6. NPR developed a series of distance-learning courses—or massive open online courses 
(MOOC)— to train teachers on the new curriculum. To what extent have teachers enrolled in 
these courses? Have the teachers used what they learned to improve their instruction? If yes, 
107
how? What has impeded the development of MOOCs?
7. In 2020, NPR printed and distributed supplementary reading materials and teachers’ guides in 
experimental schools (show examples of materials) and then distributed them nationwide in 
2022. Has the supplemental reading materials program helped to improve children’s reading 
skills? If yes, in what ways? What has impeded implementation of the supplemental reading 
materials program?
8. Do you feel your unit is more able to develop and deliver education reform in 
primary grades after the development of the new curriculum? If yes, in what ways? 
What role have NPR partners (including Creative and USAID) played in improving 
your office's ability? Are there any areas or ways in which your ability to develop 
and deliver education reform could still be improved?
I’d also like to ask you about NPR’s activities to develop the MoE’s capacity for 
measurement and evaluation of improved reading outcomes. Starting in 2019, NPR 
organized a series of capacity building-workshops for MoE personnel on evaluation. The 
general topics included assessment design; item development; developing a research 
agenda; benchmarking; data analysis, interpretation, and use; and test equating. 
9. Did you participate in any of these workshops? If yes, if NPR conducts similar 
capacity-building workshops in the future on benchmarking and measurement, 
should anything be done differently? If so, what and why? [PROBE: Different topics, 
format, time, materials, etc.]
10. If yes (participated), have these activities improved your unit's ability to conduct 
EGRAs and use the data for informing implementation? Why or why not?
11. To your knowledge, has your unit used—or have any plans to use—any EGRA data 
to make any changes to the design of the Arabic language education reform (either 
curriculum, materials, teacher training, data use, etc.)? If so, how? 
12. [NOTE: Show the FA toolkit to the respondent.] NPR created a formative 
assessment toolkit, including a formative assessment guide, student remediation 
guide, and a teacher’s guide to unit tests for grades 1 to 4. Have you received any 
FA data from schools nationwide? If so, have you used this data at all to make 
changes to the design of the Arabic language education reform (either curriculum, 
materials, teacher training, data use, etc.)? If so, how?
13. Does your unit need any support in the future to use data to make any changes to 
the design and/or management [NOTE TO REVIEWER: "design and management" 
is how we're referring to adaptive management] of the Arabic language reform 
(either curriculum, materials, teacher training, data use, etc.)? If yes, what kind of 
support?
Now I’d like to ask about the launch of the new curriculum nationally. Before the start of 
the 2021-22 school year, the MoE announced that schools were required to use the new 
Arabic language curriculum introduced by NPR, and 8,500 copies of the curriculum were 
printed and distributed to schools nationwide.
14. What did your unit do to support the launch of the new curriculum and materials 
nationwide at the start of the 2021-22 school year? Did you provide enough support 
to regional and provincial education offices in order for schools nationwide to 
108
implement the curriculum and use the new materials as designed? Why/why not?
15. [ASK ONLY OF CURRICULA DIRECTORATE] In February 2022, the MoE 
Curricula Directorate requested that NPR develop a toolkit to help inspectors 
provide in-service training to teachers about the new Arabic language curriculum. 
What prompted this request? Do you know if inspectors were trained and 
supported on how to use the toolkit? Have they been using this toolkit? If yes, has 
the inspectors’ training led to schools nationwide implementing the new curriculum 
and using the new materials as designed? Why/why not? 
16. To your knowledge, are teachers nationwide implementing or not the new curriculum and 
materials? If so, what proportion of teachers are implementing it—all, most, some, or few? [Ask 
if previous response is most, some, or few] Why aren't all schools implementing it? What 
additional support could the MoE provide to ensure all schools are able to implement the new 
Arabic language curriculum?
17. What did your unit do well with scaling the new curriculum nationally? What did it not do well? 
What was the biggest obstacle your unit had to overcome in scaling the curriculum, and how did 
it address it?
Now I’d like to ask about NPR’s activities for engaging community and parental support for reading. 
18. What kind of outreach activities did NPR train teachers and school directors to organize to help 
parents with the use of supplementary reading materials and the 3asafeer digital library at home? 
To your knowledge, how many teachers and school directors organized these activities—all, 
most, some, or few? Did parents follow the guidance? Did students read the materials? 
19. What other activities did NPR implement to increase community and parental support for 
reading? Did anything change regarding community and parental support due to the activities? 
Now I’d like to ask about NPR’s engagement with the private sector in reading programs. 
20. [ASK ONLY OF CURRICULA DIRECTORATE] NPR and the MoE worked with the 
private textbook industry during the development of the new curriculum. What 
were the textbook industry’s strengths and weaknesses in this partnership? Were 
there any obstacles NPR had to overcome? If so, what did NPR try to do to 
overcome them? 
21. [ASK ONLY OF CURRICULA DIRECTORATE] NPR and the MoE worked with the 
company 3asafeer during the development of the supplemental reading materials 
program. What were 3asafeer’s strengths and weaknesses in this partnership? Were 
there any obstacles NPR had to overcome? If so, what did NPR try to do to 
overcome them? 
Now I’d like to ask about NPR’s implementation and reception in urban and rural areas, different 
regions of the country, and with girls and boys.
22. Have you seen any notable differences in how the new curriculum has been received 
by children from different backgrounds – e.g., from certain regions/provinces, 
Arabic vs Amazigh, boys vs girls, children with disabilities, urban vs. rural? If yes, 
please describe these differences. Have there been any steps taken to address 
them?
109
23. Overall, have there been any unexpected outcomes or results (positive or negative) 
with NPR? Please detail.
CLOSING
1. Those are all of my questions. Are there any additional comments that you would 
like to share with us?
2. Thank you very much for taking the time to answer our questions. Do you have any 
questions for our team?
B. SUMMARY AND CONCLUSION
Thank you for your time. Your help in this research is very important. As I mentioned, 
the results of the report will be used by USAID Morocco and the MoE to understand 
how the NPR project activities are going and are being implemented. The final results 
of our research project will be published in a report in the coming months. These 
results will be communicated back to USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
POST-KII NOTES:
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
110
Key Informant Interview Guide: Ministry of Education Official -
Experimental Regions
A note about this tool:
Purpose: This interview guide will enable you to gather information from the Ministry of National 
Education, Preschool, and Sports (MoE) key informants to get an overview about their involvement in 
the NPR project. The objective is to gain insights and understanding of their engagement with the NPR 
activities and implementers, the new Arabic language curricula introduced as part of NPR, inclusion, and 
sustainability.
Recommended sources: Information should be collected through with a variety of MoE officials at 
the provincial, regional, and national levels. 
The provincial KIIs will be conducted in twelve provinces within the same six regions as the schools, i.e. 
the four regions where NPR experimental schools are located and the two regions where the new 
reading curriculum and materials have been scaled. In each of the twelve provinces, two MoE officials 
will be interviewed—the provincial director and the inspector/provincial coordinator. In addition, six 
provincial Arabic teacher trainers from the sampled provinces will be interviewed.
At the regional level, interviewees will include the MoE regional director in each of the six regions 
visited, as well as two regional directors of teacher training centers from the sampled regions. 
National-level interviews will be conducted with six officials from the MoE. Within the MoE, 
interviewees may include the same six directors of divisions who were interviewed in 2019: the 
Directorate of Strategy, Statistics, and Planning; the Communication Directorate; the Curriculum 
Directorate; the National Center for Pedagogical Innovations and Experimentation (CNIPE); the 
National Center for Evaluation and Examinations (CNEE); and the Directorate of Cooperation and 
Teaching in Private Schools. 
If more than one official is interested and available to participate at the designated date, time and 
location, the KII may be administered as a group interview but the potential impact of power dynamics 
(such as age, gender, and level of authority in the organization or ministry) within the interview must be 
taken into consideration and mitigated. 
Qualitative Instruments: USAID NPR Performance Evaluation
Background
A. Respondent type:_____________________________________________________
B. Respondent role/position:_______________________________________
C. Respondent Ministry or Organization: _______________________________________
D. Region: __________________________________________
E. Province: ________________________________________
F. Gender: _______________________________________
G. Age: _______________________________________
H. Interviewer name: 
____________________________________________________________
111
I. Note taker name: 
_____________________________________________________________
J. Date: 
_______________________________________________________________________
K. Start Time: 
__________________________________________________________________
L. Introduce team members and obtain consent: ___ YES ____ NO
M. State purpose of evaluation: ___ YES ____ NO
Introduction and Consent 
[NOTE TO INTERVIEWER: Please review and complete the KII Consent form with the participant. If 
agreement to participate voluntarily is obtained, please continue. If participant does NOT agree to take 
part in the study, thank them for their time, make a note they did not want to participate. Make sure to 
also ask for permission to record the interview on our audio-recorder; if participant says YES please 
continue; if participant says NO, acknowledge that you will not record the conversation and proceed 
without turning on the audio-recording.]
**IMPORTANT NOTE**:
Begin audio-recording AFTER consent to participate and be audio-recorded has been received.
1. In your view, has NPR helped to increase the number of primary school children 
reading fluently and with comprehension skills in NPR’s experimental schools? If 
yes, which aspects of NPR contributed to these improvements?
a. What have been NPR’s most important and valuable contributions to improving 
children’s reading in its experimental schools? Why do you consider these the most 
important or valuable contributions?
2. Within the different elements of NPR, what has met your expectations? Why? What has not 
met your expectations? Why, and how would you suggest these shortcomings be addressed?
3. What kinds of internal, NPR-related concerns or obstacles, if any, have impeded 
implementation of these activities? How has COVID specifically impeded these 
activities? Besides COVID, have any other external, contextual challenges impeded 
NPR? If so, how?
4. How well did NPR deliver on the creation and rollout of the curriculum in its 
experimental schools? What has impeded creation and rollout? Has the new 
curriculum addressed issues with the former curriculum?
5. NPR trained teachers in its experimental schools in differentiated instruction, use of visual aids, 
and school-based teacher support and follow-up (TSF). Have these interventions improved 
teacher instruction in NPR’s experimental schools? If so, in what ways? What has impeded 
teacher instruction interventions?
6. NPR developed a series of distance-learning courses—or massive open online courses 
(MOOC)— to train teachers on the new curriculum. To what extent have teachers enrolled in 
these courses? Have the teachers used what they learned to improve their instruction? If yes, 
how? What has impeded the development of MOOCs?
112
7. In 2020, NPR printed and distributed supplementary reading materials and teachers’ guides in 
experimental schools (show examples of materials) and then distributed them nationwide in 
2022. NPR also asked schools to dedicate 30 minutes per week for students to use them. Have 
all schools dedicated 30 minutes per week to the activity? If not, why do you think schools have 
not dedicated that time to the activity? Have these materials helped to improve children’s early 
grade reading skills? If yes, in what ways?
8. Is your regional/provincial office/inspectorate/CMREF more able to develop and 
deliver education reform in primary grades after the development of the new 
curriculum? If yes, in what ways, and what role has the MoE played in improving 
your regional/provincial office/inspectorate/CMREF’s ability? Are there any areas or 
ways in which your ability to develop and deliver education reform could still be 
improved?
I’d also like to ask you about NPR’s activities to develop the MoE’s capacity for 
measurement and evaluation of improved reading outcomes. 
9. Starting in 2019, NPR organized a series of capacity building-workshops for MoE 
personnel on evaluation. The general topics included assessment design; item 
development; developing a research agenda; benchmarking; data analysis, 
interpretation, and use; and test equating. Did you participate in any of these 
workshops? If yes, if NPR conducts similar capacity-building workshops in the future 
on benchmarking and measurement, should anything be done differently? If so, 
what and why? [PROBE: Different topics, format, time, materials, etc.]
10. If yes (participated), did these activities improve your unit’s ability to conduct 
EGRAs and use the data for informing implementation? Why or why not?
11. To your knowledge, have you used—or have any plans to use—any EGRA data to 
make any changes to the design of the Arabic language education reform (either 
curriculum, materials, teacher training, data use, etc.)? If so, how? 
12. [NOTE: Show the FA toolkit to the respondent.] NPR created a formative 
assessment toolkit, including a formative assessment guide, student remediation 
guide, and a teacher’s guide to unit tests for grades 1 to 4. Have you received any 
FA data from schools in your region/province? If so, have you used this data at all to 
make changes to the implementation of the Arabic language education reform 
(either curriculum, materials, teacher training, data use, etc.)? If so, how?
13. Does your unit need any support in the future to use data to make any changes to 
the design and/or management of the Arabic language reform (either curriculum, 
materials, teacher training, data use, etc.)? If yes, what kind of support?
Now I’d like to ask about the launch of the new curriculum nationally (beyond the 
experimental schools). Before the start of the 2021-22 school year, the MoE announced 
that schools were required to use the new Arabic language curriculum introduced by NPR, 
and 8,500 copies of the curriculum were printed and distributed to schools nationwide. 
14. To your knowledge, did the MoE do anything to support the launch of the new 
curriculum and materials nationwide at the start of the 2021-22 school year? Did it 
provide enough support to your region/province in order for schools in your 
region/province to implement the curriculum and use the new materials as 
113
designed? Why/why not?
15. Has anyone in your office received training or support to use the new Arabic language 
curriculum, including a training toolkit to help inspectors provide in-service training to teachers? 
If yes, what kind of training or support was received, and has it been received throughout your 
region/province? If yes, have those who have been trained been able to implement what they 
were trained on? Are the issues with this work? If yes, what are they?
16. Are teachers implementing the new curriculum and materials in your region/province/CMREF? If 
so, what proportion of teachers are implementing it—all, most, some, or few? [Ask if previous 
response is most, some, or few] Why aren’t all schools implementing it? What additional 
support could the MoE provide to ensure all schools are able to implement the new Arabic 
language curriculum?
17. What did the MoE do well with scaling the new curriculum nationally? What did it 
not do well? What was the biggest obstacle the MoE had to overcome in scaling the 
curriculum, and how did it address it?
Now I’d like to ask about NPR’s activities for engaging community and parental support for reading. 
18. Have you been involved with training teachers and school directors on outreach activities to 
organize with parents to help them with the use of supplementary reading materials and the 
3asafeer digital library at home? If so, what are the outreach activities? To your knowledge, how 
many teachers and school directors organized these activities—all, most, some, or few? Did 
parents follow the guidance? Did students read the materials?
19. As part of NPR, have there any other activities or programs in your province/region designed to 
engage parents or community members in their children’s reading? If so, what, and did anything 
change regarding community and parental support due to the activities? 
Now I’d like to ask about NPR’s engagement with the private sector in reading programs. 
20. NPR and the MoE worked with the private textbook industry during the development of the 
new curriculum. Are you aware of readers produced by these companies? If so, what did they 
do or not do well? What would you like to see done differently?
21. NPR and the MoE worked with 3asafeer during the development of the new supplemental 
reading materials. Are you aware of supplemental reading materials produced by 3asafeer? If so, 
what did they do or not do well? What would you like to see done differently?
Now I’d like to ask about NPR’s implementation and reception in urban and rural areas, different 
regions of the country, and with girls and boys.
22. Have you seen any notable differences in how the new curriculum has been received 
by children from different backgrounds – e.g., from certain regions/provinces, 
Arabic vs Amazigh, boys vs girls, children with disabilities, urban vs. rural? If yes, 
please describe these differences. Have there been any steps taken to address 
them?
23. Overall, have there been any unexpected outcomes or results (positive or negative) 
with NPR? Please detail.
114
C. CLOSING
1. Those are all of my questions. Are there any additional comments that you would 
like to share with us?
2. Thank you very much for taking the time to answer our questions. Do you have any 
questions for our team?
D. SUMMARY AND CONCLUSION
Thank you for your time. Your help in this research is very important. As I mentioned, 
the results of the report will be used by USAID Morocco and the MoE to understand 
how the NPR project activities are going and are being implemented. The final results 
of our research project will be published in a report in the coming months. These 
results will be communicated back to USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
POST-KII NOTES:
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
115
Key Informant Interview Guide: Ministry of Education Official -
Experimental Regions
A note about this tool:
Purpose: This interview guide will enable you to gather information from the Ministry of National 
Education, Preschool, and Sports (MoE) key informants to get an overview about their involvement in 
the NPR project. The objective is to gain insights and understanding of their engagement with the NPR 
activities and implementers, the new Arabic language curricula introduced as part of NPR, inclusion, and 
sustainability.
Recommended sources: Information should be collected through with a variety of MoE officials at 
the provincial, regional, and national levels. 
The provincial KIIs will be conducted in twelve provinces within the same six regions as the schools, i.e. 
the four regions where NPR experimental schools are located and the two regions where the new 
reading curriculum and materials have been scaled. In each of the twelve provinces, two MoE officials 
will be interviewed—the provincial director and the inspector/provincial coordinator. In addition, six 
provincial Arabic teacher trainers from the sampled provinces will be interviewed.
At the regional level, interviewees will include the MoE regional director in each of the six regions 
visited, as well as two regional directors of teacher training centers from the sampled regions. 
National-level interviews will be conducted with six officials from the MoE. Within the MoE, 
interviewees may include the same six directors of divisions who were interviewed in 2019: the 
Directorate of Strategy, Statistics, and Planning; the Communication Directorate; the Curriculum 
Directorate; the National Center for Pedagogical Innovations and Experimentation (CNIPE); the 
National Center for Evaluation and Examinations (CNEE); and the Directorate of Cooperation and 
Teaching in Private Schools. 
If more than one official is interested and available to participate at the designated date, time and 
location, the KII may be administered as a group interview but the potential impact of power dynamics 
(such as age, gender, and level of authority in the organization or ministry) within the interview must be 
taken into consideration and mitigated. 
Qualitative Instruments: USAID NPR Performance Evaluation
Background
A. Respondent type:_____________________________________________________
B. Respondent role/position:_______________________________________
C. Respondent Ministry or Organization: _______________________________________
D. Region: __________________________________________
E. Province: ________________________________________
F. Gender: _______________________________________
G. Age: _______________________________________
H. Interviewer name: 
____________________________________________________________
116
I. Note taker name: 
_____________________________________________________________
J. Date: 
_______________________________________________________________________
K. Start Time: 
__________________________________________________________________
L. Introduce team members and obtain consent: ___ YES ____ NO
M. State purpose of evaluation: ___ YES ____ NO
Introduction and Consent 
[NOTE TO INTERVIEWER: Please review and complete the KII Consent form with the participant. If 
agreement to participate voluntarily is obtained, please continue. If participant does NOT agree to take 
part in the study, thank them for their time, make a note they did not want to participate. Make sure to 
also ask for permission to record the interview on our audio-recorder; if participant says YES please 
continue; if participant says NO, acknowledge that you will not record the conversation and proceed 
without turning on the audio-recording.]
**IMPORTANT NOTE**:
Begin audio-recording AFTER consent to participate and be audio-recorded has been received.
1. In your view, has NPR helped to increase the number of primary school children 
reading fluently and with comprehension skills in NPR’s experimental schools? If 
yes, which aspects of NPR contributed to these improvements?
a. What have been NPR’s most important and valuable contributions to 
improving children’s reading in its experimental schools? Why do you 
consider these the most important or valuable contributions?
2. Within the different elements of NPR, what has met your expectations? Why? What has not 
met your expectations? Why, and how would you suggest these shortcomings be addressed?
3. What kinds of internal, NPR-related concerns or obstacles, if any, have impeded implementation 
of these activities? How has COVID specifically impeded these activities? Besides COVID, have 
any other external, contextual challenges impeded NPR? If so, how?
4. How well did NPR deliver on the creation and rollout of the curriculum in its experimental 
schools? What has impeded creation and rollout? Has the new curriculum addressed issues with 
the former curriculum?
5. NPR trained teachers in its experimental schools in differentiated instruction, use of 
visual aids, and school-based teacher support and follow-up (TSF). Have these 
interventions improved teacher instruction in NPR’s experimental schools? If so, in 
what ways? What has impeded teacher instruction interventions?
6. NPR developed a series of distance-learning courses—or massive open online 
courses (MOOC)— to train teachers on the new curriculum. To what extent have 
teachers enrolled in these courses? Have the teachers used what they learned to 
improve their instruction? If yes, how? What has impeded the development of 
MOOCs?
117
7. In 2020, NPR printed and distributed supplementary reading materials and 
teachers’ guides in experimental schools (show examples of materials) and then 
distributed them nationwide in 2022. NPR also asked schools to dedicate 30 
minutes per week for students to use them. Have all schools dedicated 30 minutes 
per week to the activity? If not, why do you think schools have not dedicated that 
time to the activity? Have these materials helped to improve children’s early grade 
reading skills? If yes, in what ways?
8. Is your regional/provincial office/inspectorate/CMREF more able to develop and deliver 
education reform in primary grades after the development of the new curriculum? If yes, in what 
ways, and what role has the MoE played in improving your regional/provincial 
office/inspectorate/CMREF’s ability? Are there any areas or ways in which your ability to 
develop and deliver education reform could still be improved?
I’d also like to ask you about NPR’s activities to develop the MoE’s capacity for measurement and 
evaluation of improved reading outcomes. 
9. Starting in 2019, NPR organized a series of capacity building-workshops for MoE personnel on 
evaluation. The general topics included assessment design; item development; developing a 
research agenda; benchmarking; data analysis, interpretation, and use; and test equating. Did you 
participate in any of these workshops? If yes, if NPR conducts similar capacity-building 
workshops in the future on benchmarking and measurement, should anything be done 
differently? If so, what and why? [PROBE: Different topics, format, time, materials, etc.]
10. If yes (participated), did these activities improve your unit’s ability to conduct EGRAs and use 
the data for informing implementation? Why or why not?
11. To your knowledge, have you used—or have any plans to use—any EGRA data to make any 
changes to the design of the Arabic language education reform (either curriculum, materials, 
teacher training, data use, etc.)? If so, how? 
12. [NOTE: Show the FA toolkit to the respondent.] NPR created a formative assessment toolkit, 
including a formative assessment guide, student remediation guide, and a teacher’s guide to unit 
tests for grades 1 to 4. Have you received any FA data from schools in your region/province? If 
so, have you used this data at all to make changes to the implementation of the Arabic language 
education reform (either curriculum, materials, teacher training, data use, etc.)? If so, how?
13. Does your unit need any support in the future to use data to make any changes to the design 
and/or management of the Arabic language reform (either curriculum, materials, teacher 
training, data use, etc.)? If yes, what kind of support?
Now I’d like to ask about the launch of the new curriculum nationally (beyond the 
experimental schools). Before the start of the 2021-22 school year, the MoE announced 
that schools were required to use the new Arabic language curriculum introduced by NPR, 
and 8,500 copies of the curriculum were printed and distributed to schools nationwide. 
14. To your knowledge, did the MoE do anything to support the launch of the new curriculum and 
materials nationwide at the start of the 2021-22 school year? Did it provide enough support to 
your region/province in order for schools in your region/province to implement the curriculum 
and use the new materials as designed? Why/why not?
15. Has anyone in your office received training or support to use the new Arabic language 
118
curriculum, including a training toolkit to help inspectors provide in-service training to teachers? 
If yes, what kind of training or support was received, and has it been received throughout your 
region/province? If yes, have those who have been trained been able to implement what they 
were trained on? Are the issues with this work? If yes, what are they?
16. Are teachers implementing the new curriculum and materials in your 
region/province/CMREF? If so, what proportion of teachers are implementing it—
all, most, some, or few? [Ask if previous response is most, some, or few] Why aren’t 
all schools implementing it? What additional support could the MoE provide to 
ensure all schools are able to implement the new Arabic language curriculum?
17. What did the MoE do well with scaling the new curriculum nationally? What did it not do well? 
What was the biggest obstacle the MoE had to overcome in scaling the curriculum, and how did 
it address it?
Now I’d like to ask about NPR’s activities for engaging community and parental support 
for reading. 
18. Have you been involved with training teachers and school directors on outreach 
activities to organize with parents to help them with the use of supplementary 
reading materials and the 3asafeer digital library at home? If so, what are the 
outreach activities? To your knowledge, how many teachers and school directors 
organized these activities—all, most, some, or few? Did parents follow the guidance? 
Did students read the materials?
19. As part of NPR, have there any other activities or programs in your province/region 
designed to engage parents or community members in their children’s reading? If 
so, what, and did anything change regarding community and parental support due 
to the activities? 
Now I’d like to ask about NPR’s engagement with the private sector in reading programs. 
20. NPR and the MoE worked with the private textbook industry during the development of the 
new curriculum. Are you aware of readers produced by these companies? If so, what did they 
do or not do well? What would you like to see done differently?
21. NPR and the MoE worked with 3asafeer during the development of the new supplemental 
reading materials. Are you aware of supplemental reading materials produced by 3asafeer? If so, 
what did they do or not do well? What would you like to see done differently?
Now I’d like to ask about NPR’s implementation and reception in urban and rural areas, different 
regions of the country, and with girls and boys.
22. Have you seen any notable differences in how the new curriculum has been received 
by children from different backgrounds – e.g., from certain regions/provinces, 
Arabic vs Amazigh, boys vs girls, children with disabilities, urban vs. rural? If yes, 
please describe these differences. Have there been any steps taken to address 
them?
23. Overall, have there been any unexpected outcomes or results (positive or negative) 
with NPR? Please detail.
119
E. CLOSING
1. Those are all of my questions. Are there any additional comments that you would 
like to share with us?
2. Thank you very much for taking the time to answer our questions. Do you have any 
questions for our team?
F. SUMMARY AND CONCLUSION
Thank you for your time. Your help in this research is very important. As I mentioned, 
the results of the report will be used by USAID Morocco and the MoE to understand 
how the NPR project activities are going and are being implemented. The final results 
of our research project will be published in a report in the coming months. These 
results will be communicated back to USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
POST-KII NOTES:
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
120
Key Informant Interview Guide: Ministry of Education Official - Scale￾Up Regions
A note about this tool:
Purpose: This interview guide will enable you to gather information from the Ministry of National 
Education, Preschool, and Sports (MoE) key informants to get an overview about their involvement in 
the NPR project. The objective is to gain insights and understanding of their engagement with the NPR 
activities and implementers, the new Arabic language curricula introduced as part of NPR, inclusion, and 
sustainability.
Recommended sources: Information should be collected through with a variety of MoE officials at 
the provincial, regional, and national levels. 
The provincial KIIs will be conducted in twelve provinces within the same six regions as the schools, i.e. 
the four regions where NPR experimental schools are located and the two regions where the new 
reading curriculum and materials have been scaled. In each of the twelve provinces, two MoE officials 
will be interviewed—the provincial director and the inspector/provincial coordinator. In addition, six 
provincial Arabic teacher trainers from the sampled provinces will be interviewed.
At the regional level, interviewees will include the MoE regional director in each of the six regions 
visited, as well as two regional directors of teacher training centers from the sampled regions. 
National-level interviews will be conducted with six officials from the MoE. Within the MoE, 
interviewees may include the same six directors of divisions who were interviewed in 2019: the 
Directorate of Strategy, Statistics, and Planning; the Communication Directorate; the Curriculum 
Directorate; the National Center for Pedagogical Innovations and Experimentation (CNIPE); the 
National Center for Evaluation and Examinations (CNEE); and the Directorate of Cooperation and 
Teaching in Private Schools. 
If more than one official is interested and available to participate at the designated date, time and 
location, the KII may be administered as a group interview but the potential impact of power dynamics 
(such as age, gender, and level of authority in the organization or ministry) within the interview must be 
taken into consideration and mitigated. 
Qualitative Instruments: USAID NPR Performance Evaluation
Background
A. Respondent type:_____________________________________________________
B. Respondent role/position:_______________________________________
C. Respondent Ministry or Organization: _______________________________________
D. Region: __________________________________________
E. Province: ________________________________________
F. Gender: _______________________________________
G. Age: _______________________________________
H. Interviewer name: 
____________________________________________________________
121
I. Note taker name: 
_____________________________________________________________
J. Date: 
_____________________________________________________________________
__
K. Start Time: 
__________________________________________________________________
L. Introduce team members and obtain consent: ___ YES ____ NO
M. State purpose of evaluation: ___ YES ____ NO
Introduction and Consent 
[NOTE TO INTERVIEWER: Please review and complete the KII Consent form with the participant. If 
agreement to participate voluntarily is obtained, please continue. If participant does NOT agree to take 
part in the study, thank them for their time, make a note they did not want to participate. Make sure to 
also ask for permission to record the interview on our audio-recorder; if participant says YES please 
continue; if participant says NO, acknowledge that you will not record the conversation and proceed 
without turning on the audio-recording.]
**IMPORTANT NOTE**:
Begin audio-recording AFTER consent to participate and be audio-recorded has been received.
1. The MoE has worked with USAID to develop a new Arabic language curriculum in 
grades 1 to 6. This curriculum was piloted in select schools and then scaled nationwide 
in 2021. The program consisted of a) curriculum materials; b) new textbooks; c) 
training toolkit for inspectors; d) access to the MOOC content; e) supplemental 
reading materials (books and teachers’ guides); f) MSL video content; g) capacity 
building on measurement and evaluation; h) any other training activities. Are you 
aware of this program? 
2. If yes, to any of the above, what was done? If yes, did these activities help to increase the number 
of primary school children reading fluently and with comprehension skills? Please describe why.
3. Within the different elements of the new curriculum, what has met your expectations? 
Why? What has not met your expectations? Why, and how would you suggest these 
shortcomings be addressed?
4. What kinds of internal, NPR-related concerns or obstacles, if any, have impeded implementation of 
these activities? How has COVID specifically impeded these activities? Besides COVID, have any 
other external, contextual challenges impeded NPR? If so, how?
5. As part of the new curriculum development (NPR?), some MoE inspectors attended 
national workshops to finalize and validate the new curriculum. Did you participate in 
any of these workshops? If so, did they improve your understanding about the new 
curriculum and your role in implementing it? If yes, how?
6. The MoE developed a series of distance-learning courses—or massive open online courses 
(MOOC)— to train teachers on the new curriculum. To what extent have teachers enrolled in 
these courses? Have the teachers used what they learned to improve their instruction? If yes, how? 
122
What has impeded the development of MOOCs?
7. In 2022, the MoE printed and distributed supplementary reading materials and teachers’ guides to 
schools nationwide (show examples of materials) and asked schools to dedicate 30 minutes per 
week for students to use them. Did all schools in your region/province receive these materials? 
Have all schools dedicated 30 minutes per week to the activity? If not, why do you think schools 
have not dedicated that time to the activity? Have these new materials helped to improve early 
grade reading skills? If yes, in what ways?
8. Is your regional/provincial office/inspectorate/CMREF more able to develop and 
deliver education reform in primary grades after the development of the new 
curriculum? If yes, in what ways, and what role has the MoE played in improving your 
regional/provincial office/inspectorate/CMREF's ability? Are there any areas or ways in 
which your ability to develop and deliver education reform could still be improved?
9. [NOTE: Show the FA toolkit to the respondent.] As part of the new curriculum, the 
MoE created a formative assessment toolkit, including a formative assessment guide, 
student remediation guide, and a teacher’s guide to unit tests for grades 1 to 4. Have 
you received these materials? If yes, have you received any FA data from schools in 
your region/province? If so, have you used this data at all to make changes to the 
implementation of the Arabic language education reform (either curriculum, 
materials, teacher training, data use, etc.)? If so, how? "
Now I’d like to ask about the launch of the new curriculum nationally. 
10. Before the start of the 2021-22 school year, the MoE announced that schools were 
required to use the new Arabic language curriculum introduced by NPR, and 8,500 
copies of the curriculum were printed and distributed to schools nationwide. To your 
knowledge, did the MoE do anything to support the launch of the new curriculum and 
materials nationwide at the start of the 2021-22 school year? Did it provide enough 
support to your unit in order for schools in your region/province to implement the 
curriculum and use the new materials as designed? Why/why not?
11. Has anyone in your office received training or support to use the new Arabic language 
curriculum, including a training toolkit to help inspectors provide in-service training to 
teachers? If yes, what kind of training or support was received, and has it been 
received throughout your region/province? If yes, have those who have been trained 
been able to implement what they were trained on? What are the issues with this 
work?
12. Are teachers implementing the new curriculum and materials in your 
region/province/CMREF? If so, what proportion of teachers are implementing it—all, 
most, some, or few? [Ask if previous response is most, some, or few] Why aren't all 
schools implementing it? What additional support could the MoE provide to ensure all 
schools are able to implement the new Arabic language curriculum?
Now I’d like to ask about activities for engaging community and parental support for reading.
13. Have you been involved with training teachers and school directors on outreach activities to 
organize with parents to help them with the use of supplementary reading materials at home? If so, 
what are the outreach activities? To your knowledge, how many teachers and school directors 
organized these activities—all, most, some, or few? Did parents follow the guidance? Did students 
123
read the materials?
14. As part of the new curriculum, has the MoE conducted any other activities in your province/region 
designed to engage parents or community members in their children’s reading? If so, what, and did 
anything change regarding community and parental support due to the activities? 
Now I’d like to ask about the MoE’s engagement with the private sector in reading programs. 
15. NPR and the MoE worked with the private textbook industry during the development of the new 
curriculum. Are you aware of readers produced by these companies? If so, what were the strengths 
and weaknesses of the textbooks? What would you like to see done differently in the textbooks?
16. NPR and the MoE worked with 3asafeer during the development of the new supplemental reading 
materials. Are you aware of supplemental reading materials produced by 3asafeer? If so, what were 
the strengths and weaknesses of the supplemental reading materials? What would you like to see 
done differently in the reading materials?
Now I’d like to ask about NPR’s implementation and reception in urban and rural areas, different 
regions of the country, and with girls and boys.
17. Have you seen any notable differences in how the new curriculum has been received 
by children from different backgrounds – e.g., from certain regions/provinces, Arabic 
vs Amazigh, boys vs girls, children with disabilities, urban vs. rural? If yes, please 
describe these differences. Have there been any steps taken to address them?
18. Overall, have there been any unexpected outcomes or results (positive or negative) 
with the new curriculum? Please detail.
G. CLOSING
1. Those are all of my questions. Are there any additional comments that you would 
like to share with us?
2. Thank you very much for taking the time to answer our questions. Do you have any 
questions for our team?
H. SUMMARY AND CONCLUSION
Thank you for your time. Your help in this research is very important. As I mentioned, 
the results of the report will be used by USAID Morocco and the MoE to understand 
how the NPR project activities are going and are being implemented. The final results 
of our research project will be published in a report in the coming months. These 
results will be communicated back to USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
POST-KII NOTES:
Please comment on
124
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
125
Key Informant Interview Guide: MSL Intervention Respondents
A note about this tool:
Purpose: This interview guide will enable you to gather information from Moroccan Sign Language 
(MSL) intervention key informants to get an overview about their involvement with the NPR project. 
The objective is to gain insights and understanding of their engagement with NPR, inclusion, and lessons 
learned.
Recommended sources: Information should be collected through interviews with two individuals 
who participated in the efforts to provide MSL interpretation for distance-learning video lessons.
If more than one individual is interested and available to participate at the designated date, time and 
location, the KII may be administered as a group interview but the potential impact of power dynamics 
(such as age, gender, and level of authority in the organization or ministry) within the interview must be 
taken into consideration and mitigated.
Qualitative Instruments: USAID NPR Performance Evaluation
Background
A. Respondent type:_____________________________________________________
B. Respondent role/position:_______________________________________
C. Respondent Ministry or Organization: _______________________________________
D. Gender: _______________________________________
E. Age: _______________________________________
F. Interviewer name: 
____________________________________________________________
G. Note taker name: 
_____________________________________________________________
H. Date: 
_____________________________________________________________________
_
I. Start Time: 
__________________________________________________________________
J. Introduce team members and obtain consent: ___ YES ____ NO
K. State purpose of evaluation: ___ YES ____ NO
[NOTE TO INTERVIEWER: Please review and complete the KII Consent form with the participant. If 
agreement to participate voluntarily is obtained, please continue. If participant does NOT agree to take 
part in the study, thank them for their time, make a note they did not want to participate. Make sure to 
also ask for permission to record the interview on our audio-recorder; if participant says YES please 
continue; if participant says NO, acknowledge that you will not record the conversation and proceed 
without turning on the audio-recording.]
**IMPORTANT NOTE**:
Begin audio-recording AFTER consent to participate and be audio-recorded has been received.
1. Had you had any involvement with NPR prior to this MSL activity? Had NPR or the MoE 
expressed any interest in working with your group/organization prior to the pandemic?
2. Can you describe your role with the production of the MSL video lessons for NPR during the 
COVID-19 pandemic?
126
3. Did the activity help improve the reading and MSL skills of children who are deaf or hard of 
hearing? If yes, in what ways?
4. What were the activity’s most important and valuable contributions to improving children’s 
reading and MSL skills? Why do you consider these the most important or valuable 
contributions?
5. Within the different elements of NPR, what has met your expectations? Why? What has not 
met your expectations? Why, and how would you suggest these shortcomings be addressed?
6. What kinds of internal, NPR-related concerns or obstacles, if any, have impeded implementation 
of these activities? Besides COVID, have any other external, contextual challenges impeded 
NPR? If so, how?
7. According to NPR project reports in 2020 and 2021, the MoE had technical difficulties 
transferring the video lessons to their online distance learning platform. To your knowledge, 
have the videos been transferred online since 2021? If the videos were transferred, have 
students been accessing and using the video content? Do you have knowledge of how many 
students have accessed the platform and how they are using it? If the videos were not 
transferred, do you know why the MoE is still having difficulties?
8. To our knowledge, the MSL activity ended in October 2021 when schools reopened for in￾person learning for the start of the 2021-22 school year. Is that correct? Have you done any 
work with NPR or the MoE since the activity closed with children who are deaf or hard of 
hearing?
9. Has the MSL activity had any impact since it closed in October 2021? Why/why not? If it hasn’t 
had any more impact, do you think the activity should continue to be utilized by the MoE, or 
should it focus on other interventions targeted at children who are deaf and hard of hearing? If 
yes, what do you think needs to happen for the activity to make an impact with children who 
are deaf or hard of hearing?
10. What were the biggest successes of your work with NPR, including USAID and the MoE? What 
were the biggest challenges in working with the MoE and NPR, and how could project 
implementers and the MoE address them in future projects?
CLOSING
1. Those are all of my questions. Are there any additional comments that you would 
like to share with us?
2. Thank you very much for taking the time to answer our questions. Do you have any 
questions for our team?
SUMMARY AND CONCLUSION
Thank you for your time. Your help in this research is very important. As I mentioned, 
the results of the report will be used by USAID Morocco and the Ministry of National 
127
Education, Preschool, and Sports (MoE) to understand how the NPR project activities 
are going and are being implemented. The final results of our research project will be 
published in a report in the coming months. These results will be communicated back 
to USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
POST-KII NOTES:
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
128
Key Informant Interview Guide: Private Sector - Relevant Personnel 
from 3asafeer
A note about this tool:
Purpose: This interview guide will enable you to gather information from private sector key informants 
to get an overview about their involvement with the NPR project. The objective is to gain insights and 
understanding of their engagement with NPR, inclusion, and lessons learned.
Recommended sources: Information should be collected through interviews with individuals who 
made key contributions to NPR, namely two representatives of private textbook publishers and one 
representative of the Dubai-based educational technology company, 3asafeer, which helped to support 
the development of supplementary reading materials.
If more than one relevant private sector informant is interested and available to participate at the 
designated date, time and location, the KII may be administered as a group interview but the potential 
impact of power dynamics (such as age, gender, and level of authority in the organization or ministry) 
within the interview must be taken into consideration and mitigated.
Qualitative Instruments: USAID NPR Performance Evaluation
Background
A. Respondent type:_____________________________________________________
B. Respondent role/position:_______________________________________
C. Respondent Ministry or Organization: _______________________________________
D. Gender: _______________________________________
E. Age: _______________________________________
F. Interviewer name: ___________________________________________________________
G. Note taker name: ___________________________________________________________
H. Date: ____________________________________________________________________
I. Start Time: ________________________________________________________________
J. Introduce team members and obtain consent: ___ YES ____ NO
K. State purpose of evaluation: ___ YES ____ NO
[NOTE TO INTERVIEWER: Please review and complete the KII Consent form with the participant. If 
agreement to participate voluntarily is obtained, please continue. If participant does NOT agree to take 
part in the study, thank them for their time, make a note they did not want to participate. Make sure to 
also ask for permission to record the interview on our audio-recorder; if participant says YES please 
continue; if participant says NO, acknowledge that you will not record the conversation and proceed 
without turning on the audio-recording.]
**IMPORTANT NOTE**:
Begin audio-recording AFTER consent to participate and be audio-recorded has been received.
1. How many years have you worked in the field of educational technology? When do you create 
3asafeer, and what kind of partnerships have you developed since the company’s founding?
129
2. When did NPR first contact you about collaborating with the project? Prior to your involvement 
with NPR, had you ever worked on a project funded by USAID? If no, what made your 
collaboration with USAID, NPR, and the Moroccan MoE unique to other partnerships you have 
had?
I want to ask you about the initial drafting of your MOU with the MoE. 
3. Did those discussions about the MOU start in early 2020? What were the key responsibilities 
outlined for 3asafeer and the MoE in the MOU? Did you encounter any challenges in drafting the 
MOU? If so, what were they, and how did you address them? 
4. In the spring of 2021, the MOE extended its MOU with your company from May 2021 to 
October 2022. Besides the expansion of the number of titles on the platform, were there any 
additions made to the MOU? If so, what were they?
5. You participated in some virtual meetings in June 2021 about the platform and new titles with 
Moroccan inspectors and teachers. What was your role during these meetings? Do you think 
the meetings fulfilled their objectives? Why/why not?
6. Did you participate in any other trainings/meetings with inspectors and/or teachers during your 
involvement with the project? If so, what were they, and do you think the trainings fulfilled their 
objectives? Why/why not?
7. Did any inspectors/teachers raise challenges they had with the digital platform with you during 
these meetings? If so, what were they, and did you do anything to address their challenges? If so, 
what did you do? 
8. NPR conducted a study of its supplemental reading program during the 2020-21 school year. 
Did the project share its findings with you? If so, what did you think of the findings, considering 
your work with the platform elsewhere? Since the project expanded access 3assafeer to 
additional schools for the 2021-22 school year, do you have any additional data about 3asafeer’s 
reach and impact?
9. Did the MOU with the MOE end in October as originally planned? If yes, does the MOE plan to 
renew its MOU with you? Why/why not?
10. What is the future of the 3asafeer platform and its use in Morocco?
11. What were the biggest successes of your work with NPR, including USAID and the MoE? What 
were the biggest challenges in working with the MoE and NPR, and how could project 
implementers and the MoE address them in future projects? 
CLOSING
1. Those are all of my questions. Are there any additional comments that you would 
like to share with us?
2. Thank you very much for taking the time to answer our questions. Do you have any 
questions for our team?
SUMMARY AND CONCLUSION
Thank you for your time. Your help in this research is very important. As I mentioned, 
130
the results of the report will be used by USAID Morocco and the Ministry of National 
Education, Preschool, and Sports (MoE) to understand how the NPR project activities 
are going and are being implemented. The final results of our research project will be 
published in a report in the coming months. These results will be communicated back 
to USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
POST-KII NOTES:
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
131
Key Informant Interview Guide: Private Sector - Textbook Publishers
A note about this tool:
Purpose: This interview guide will enable you to gather information from private sector key informants 
to get an overview about their involvement with the NPR project. The objective is to gain insights and 
understanding of their engagement with NPR, inclusion, and lessons learned.
Recommended sources: Information should be collected through interviews with individuals who 
made key contributions to NPR, namely two representatives of private textbook publishers and one 
representative of the Dubai-based educational technology company, 3asafeer, which helped to support 
the development of supplementary reading materials.
If more than one relevant private sector informant is interested and available to participate at the 
designated date, time and location, the KII may be administered as a group interview but the potential 
impact of power dynamics (such as age, gender, and level of authority in the organization or ministry) 
within the interview must be taken into consideration and mitigated.
Qualitative Instruments: USAID NPR Performance Evaluation
Background
A. Respondent type:_____________________________________________________
B. Respondent role/position:_______________________________________
C. Respondent Ministry or Organization: _______________________________________
D. Gender: _______________________________________
E. Age: _______________________________________
F. Interviewer name: ___________________________________________________________
G. Note taker name: ___________________________________________________________
H. Date: ____________________________________________________________________
I. Start Time: ________________________________________________________________
J. Introduce team members and obtain consent: ___ YES ____ NO
K. State purpose of evaluation: ___ YES ____ NO
[NOTE TO INTERVIEWER: Please review and complete the KII Consent form with the participant. If 
agreement to participate voluntarily is obtained, please continue. If participant does NOT agree to take 
part in the study, thank them for their time, make a note they did not want to participate. Make sure to 
also ask for permission to record the interview on our audio-recorder; if participant says YES please 
continue; if participant says NO, acknowledge that you will not record the conversation and proceed 
without turning on the audio-recording.]
**IMPORTANT NOTE**:
Begin audio-recording AFTER consent to participate and be audio-recorded has been received.
1. How long have you worked in the textbook industry? What has your role(s) been? 
What work have you specifically done with literacy and reading materials?
132
2. How long did you work with the NPR project? What has your role(s) been? What 
materials have you developed as part of NPR, and for what grades?
3. Can you describe the start of the materials creation process from when you started 
on drafting the materials to delivery of the first draft of these materials?
4. Did you attend any trainings conducted by NPR on the new curriculum? If so, did 
the training adequately prepare you to deliver the first draft? Why/why not?
5. If you attended trainings conducted by NPR, did the trainings cover any materials 
on best practices in textbook creation and production? If so, did you learn anything 
new that you incorporated into the creation of the new materials? If so, what were 
they?
6. Can you describe the process of receiving feedback on your initial draft and then 
revising the materials? How many rounds of revisions did your materials go 
through? Who provided this feedback?
7. NPR and the MoE solicited feedback from inspectors and teachers on textbooks and 
teachers’ guides and conducted meetings with publishers to share this feedback. Did 
you attend any of these meetings? If so, did these sessions provide adequate 
feedback to revise the materials? Why/why not?
8. If yes, had you ever received feedback directly from inspectors and teachers before 
as you did during NPR? If not, what did you think of the process? How did this 
process compare to the creation of other learning materials in the past?
9. Were you involved at all in the printing and distribution of new materials? If so, did you 
encounter any unique challenges in the printing and distribution? If so, what were they, and what 
lessons did you learn from them?
10. Having worked with the new Arabic language curriculum, what do you view as its 
strengths in comparison to former iterations? Are there any gaps that still exist or 
areas that could be strengthened?
11. Do you think innovations in the new curriculum have been consistently 
incorporated into all the materials (not just the ones that your company produced)? 
Why/why not?
12. Have you been involved at all with the rollout of the new curriculum in schools 
nationwide? If so, what was your role? What were the strong points of the rollout, 
and what were the challenges?
13. Have you worked on any new materials projects since working on NPR? If so, have you applied 
any of the knowledge or experience you gained with NPR to the new project? If so, what have 
you applied, and why?
14. What were the biggest successes of your work with NPR, including USAID and the 
MoE? What were the biggest challenges in working with the MoE and NPR, and how 
could project implementers and the MoE address them in future projects?
133
CLOSING
1. Those are all of my questions. Are there any additional comments that you would 
like to share with us?
2. Thank you very much for taking the time to answer our questions. Do you have any 
questions for our team?
SUMMARY AND CONCLUSION
Thank you for your time. Your help in this research is very important. As I mentioned, 
the results of the report will be used by USAID Morocco and the Ministry of National 
Education, Preschool, and Sports (MoE) to understand how the NPR project activities 
are going and are being implemented. The final results of our research project will be 
published in a report in the coming months. These results will be communicated back 
to USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
POST-KII NOTES:
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
134
Key Informant Interview Guide: School Director - Experimental 
Schools
A note about this tool:
Purpose: This interview guide will enable you to gather information from School Director key 
informants to get an overview about their involvement with the NPR project. The objective is to gain 
insights and understanding of general perceptions of quality of education, learning outcomes, NPR 
project implementation, inclusion, and lessons learned.
Recommended sources: Information should be collected through interviews with School Directors 
who are actively working on early grade reading and/or inclusive education. Eight Primary School 
Directors will be interviewed from eight provinces in Morocco—one school apiece in the four regions 
where NPR experimental schools are located and two schools apiece in two regions where the new 
reading curriculum and materials have been scaled. If more than one official is interested and available to 
participate at the designated date, time and location, the KII may be administered as a group interview 
but the potential impact of power dynamics (such as age, gender, and level of authority in the 
organization or ministry) within the interview must be taken into consideration and mitigated.
Qualitative Instruments: USAID NPR Performance Evaluation
Introduction
A. Respondent type:_____________________________________________________
B. School Name:_______________________________________
C. Province: _______________________________________
D. Region: _______________________________________
E. Gender: _______________________________________
F. Age: __________________________________________
G. Interviewer name: 
______________________________________________________________
H. Note taker name: 
_____________________________________________________________
I. Date: 
_____________________________________________________________________
__
J. Start Time: 
__________________________________________________________________
K. Introduction of team member and consent: ___ YES ____ NO
L. State purpose of evaluation: ___ YES ____ NO
Introduction and Consent 
[NOTE TO INTERVIEWER: Please review and complete the KII Consent form with the participant. If 
agreement to participate voluntarily is obtained, please continue. If participant does NOT agree to take 
135
part in the study, thank them for their time, make a note they did not want to participate. Make sure to 
also ask for permission to record the interview on our audio-recorder; if participant says YES please 
continue; if participant says NO, acknowledge that you will not record the conversation and proceed 
without turning on the audio-recording.]
**IMPORTANT NOTE**:
Begin audio-recording AFTER consent to participate and be audio-recorded has been received.
1. The MoE developed and rolled out an early grade reading curriculum and teacher 
instruction package called NPR in 90 experimental schools. As you probably know, 
the program consisted of a) curriculum materials; b) new textbooks; c) access to 
the MOOC content; d) supplemental reading materials (books and teachers’ 
guides); e) capacity building on measurement and evaluation; f) any other training 
activities. Have you adopted this program in your school? 
a. If yes, which of these activities and materials have you adopted? 
b. If yes, which activities or materials made the greatest impact on teachers' 
instructional ability?
c. If yes, which activities or materials made the greatest impact on children's 
reading skills?
d. What have been NPR's most important and valuable contributions to 
improving reading for learners in your school? Why are these contributions 
the most valuable?
2. Within the different elements of NPR, what has met your expectations? Why? 
What has not met your expectations? Why, and how would you suggest these 
shortcomings be addressed?
3. Has COVID specifically affected the rollout of the curriculum reform? How?
4. As part of NPR, teachers were trained in experimental schools in differentiated instruction, use 
of visual aids, and school-based teacher support and follow-up (TSF). Did you and/or teachers in 
your school participate in any of these trainings? 
a. If yes, did you benefit from them? How? 
i. How would you rate them—excellent, good, fair, or poor? 
b. If yes, do you think they improved teachers' skills in teaching reading and/or your skills 
in supporting teachers? How? 
c. If yes, do you think these trainings improved instruction for all groups of students? 
d. If not, which group was left out, and why?
5. NPR developed a series of distance-learning courses—or massive open online courses 
(MOOC)—to train teachers on the new curriculum. Do you know if any of your teachers 
enrolled in these courses? 
a. If yes, do you feel the MOOCs have been effective in improving teachers’ ability to teach 
136
reading with your (special population)? In what ways? 
6. In 2020, NPR printed and distributed supplementary reading materials and 
teachers’ guides in experimental schools. Can children access these materials in 
your classroom? 
a. If yes, how much time per week do your students use the SRMs? 
b. If yes, have the SRMs have helped to improve early grade reading skills in 
your school? If yes, in what ways? 
c. What would you like to see done differently?
7. Are you aware of EGRA—the Early Grade Reading Assessment—a tool for 
measuring children's reading ability in the early grades? 
a. If yes, has EGRA been used to assess children's reading ability in your school? 
1. If yes, do you know the results? If yes, have you been able to 
use the results? How? 
8. NPR created a formative assessment toolkit, including a formative assessment guide, student 
remediation guide, and a teacher’s guide to unit tests for grades 1 to 4. Did your school receive 
these materials? 
a. If yes, do you think they helped your learners learn to read better? How?
9. What additional support could the MoE provide to better help your teachers implement the 
new Arabic language curriculum?
Now I’d like to ask about NPR’s activities for engaging community and parental support 
for reading. 
10. Have you organized any outreach activities to help parents with the use of 
supplementary reading materials and the 3asafeer digital library at home? 
a. If yes, what type of guidance have you provided to parents? 
b. Did parents follow the guidance? Did students read the materials?
11. Have you been involved with any other activities as part of NPR to increase 
community and parental support for reading? If yes, what, and did anything change 
regarding community and parental support due to the activities? 
12. Have you seen any notable differences in how the new curriculum has been received 
by children from different backgrounds – e.g., from certain regions/provinces, 
Arabic vs Amazigh, boys vs girls, children with disabilities, urban vs. rural? If yes, 
please describe these differences. Have there been any steps taken to address 
them?
13. Overall, have there been any unexpected outcomes or results (positive or negative) 
with NPR? Please detail.
SUMMARY AND CONCLUSION
137
Thank you for your time. Your help in this research is very important. As I mentioned, 
the results of the report will be used by USAID Morocco and the Ministry of National 
Education, Preschool, and Sports (MoE) to understand how the NPR project activities 
are going and are being implemented. The final results of our research project will be 
published in a report in the coming months. These results will be communicated back 
to USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
POST-KII NOTES:
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
138
Key Informant Interview Guide: School Director - Scale-Up Schools
A note about this tool:
Purpose: This interview guide will enable you to gather information from School Director key 
informants to get an overview about their involvement with the NPR project. The objective is to gain 
insights and understanding of general perceptions of quality of education, learning outcomes, NPR 
project implementation, inclusion, and lessons learned.
Recommended sources: Information should be collected through interviews with School Directors 
who are actively working on early grade reading and/or inclusive education. Eight Primary School 
Directors will be interviewed from eight provinces in Morocco—one school apiece in the four regions 
where NPR experimental schools are located and two schools apiece in two regions where the new 
reading curriculum and materials have been scaled. If more than one official is interested and available to 
participate at the designated date, time and location, the KII may be administered as a group interview 
but the potential impact of power dynamics (such as age, gender, and level of authority in the 
organization or ministry) within the interview must be taken into consideration and mitigated.
Qualitative Instruments: USAID NPR Performance Evaluation
Introduction
A. Respondent type:_____________________________________________________
B. School Name:_______________________________________
C. Province: _______________________________________
D. Region: _______________________________________
E. Gender: _______________________________________
F. Age: __________________________________________
G. Interviewer name: _______________________________________________________
H. Note taker name: ________________________________________________________
I. Date: _________________________________________________________________
J. Start Time: _____________________________________________________________
K. Introduction of team member and consent: ___ YES ____ NO
L. State purpose of evaluation: ___ YES ____ NO
139
Introduction and Consent 
[NOTE TO INTERVIEWER: Please review and complete the KII Consent form with the participant. If 
agreement to participate voluntarily is obtained, please continue. If participant does NOT agree to take 
part in the study, thank them for their time, make a note they did not want to participate. Make sure to 
also ask for permission to record the interview on our audio-recorder; if participant says YES please 
continue; if participant says NO, acknowledge that you will not record the conversation and proceed 
without turning on the audio-recording.]
**IMPORTANT NOTE**:
Begin audio-recording AFTER consent to participate and be audio-recorded has been received.
1. The MoE has worked with USAID to develop a new Arabic language curriculum in 
grades 1 to 6. This curriculum was piloted in select schools and then scaled 
nationwide in 2021. The program consisted of a) curriculum materials; b) new 
textbooks; c) access to the MOOC content; d) supplemental reading materials 
(books and teachers’ guides); e) any other training activities. Are you aware of this 
program? If so, were each of these activities and materials delivered at your school? 
If so, which activities or materials made the greatest impact on children's reading 
skills and teachers' instructional ability? 
2. [Ask only if new curriculum is being implemented at the school] What have been 
the new curriculum's most important and valuable contributions to improving 
reading for learners in your school? Why are these contributions the most valuable?
3. [Ask only if new curriculum is being implemented at the school] Within the 
different elements of the new curriculum, what has met your expectations? Why? 
What has not met your expectations? Why, and how would you suggest these 
shortcomings be addressed?
4. [Ask only if new curriculum is being implemented at the school] Has COVID 
specifically affected the rollout of the curriculum reform? How?
5. Have you received any training from inspectors or anyone else on the new curriculum? If so, 
when, and on what topics? If so, did you benefit from them? How? How would you rate them—
excellent, good, fair, or poor? Do you think the training covered all you need to know in order 
to adopt the new curriculum and use the new materials at your school? Why/why not? If no, 
what additional training do you need?
6. [Ask if director mentioned he/she was aware of MOOC program] The MoE developed a series 
of distance-learning courses—or massive open online courses (MOOC)— to train teachers on 
the new curriculum. Do you know if any of your teachers enrolled in these courses? If so, do 
you feel the MOOCs have been effective in improving teachers’ ability to teach reading with 
your (special population)? In what ways? 
7. [Ask if director mentioned he/she was aware of SRM program] In 2022, the MoE 
printed and distributed supplementary reading materials and teachers’ guides 
nationwide. Can children access these materials in your classroom? If so, how much 
time per week do your students use the SRMs? Have the SRMs helped to improve 
early grade reading skills in your school? In what ways? If yes, what did they do well? 
What would you like to see done differently?
8. The MoE created a formative assessment toolkit, including a formative assessment guide, 
140
student remediation guide, and a teacher’s guide to unit tests for grades 1 to 4. Did your school 
receive these materials? If so, do you think they helped your learners learn to read better? 
How?
9. What additional support could the MoE provide to better help your teachers implement the 
new Arabic language curriculum?
Now I’d like to ask about activities for engaging community and parental support for 
reading. 
10. Have you organized any outreach activities to help parents with the use of 
supplementary reading materials at home? If yes, what type of guidance have you 
provided to parents? Did the parents follow the guidance? Did students read the 
materials?
11. Have you been involved with any other activities as part of the introduction of the 
new curriculum to increase community and parental support for reading? If so, 
what, and did anything change regarding community and parental support due to 
the activities?
12. Have you seen any notable differences in how the new curriculum has been received 
by children from different backgrounds – e.g., from certain regions/provinces, 
Arabic vs Amazigh, boys vs girls, children with disabilities, urban vs. rural? If yes, 
please describe these differences. Have there been any steps taken to address 
them?
13. Overall, have there been any unexpected outcomes or results (positive or negative) 
with the new curriculum? Please detail.
SUMMARY AND CONCLUSION
Thank you for your time. Your help in this research is very important. As I mentioned, 
the results of the report will be used by USAID Morocco and the Ministry of National 
Education, Preschool, and Sports (MoE) to understand how the NPR project activities 
are going and are being implemented. The final results of our research project will be 
published in a report in the coming months. These results will be communicated back 
to USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
POST-KII NOTES:
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
_____________________________________________________________________________
141
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
142
Key Informant Interview Guide: USAID/Morocco
A note about this tool:
Purpose: This interview guide will enable you to gather information from USAID/Morocco key 
informants to get an overview about their involvement with the NPR project. The objective is to gain 
insights and understanding of their engagement with NPR, inclusion, and lessons learned.
Recommended sources: Information should be collected through interviews with at least three 
USAID/Morocco members of the Education Office—the Director, the Education Specialist, and the 
Contracting Officer’s Representative.
If more than one relevant private sector informant is interested and available to participate at the 
designated date, time and location, the KII may be administered as a group interview but the potential 
impact of power dynamics (such as age, gender, and level of authority in the organization or ministry) 
within the interview must be taken into consideration and mitigated.
Qualitative Instruments: USAID NPR Performance Evaluation
Background
A. Respondent type:_____________________________________________________
B. Respondent role/position:_______________________________________
C. Respondent Ministry or Organization: _______________________________________
D. Gender: _______________________________________
E. Age: _______________________________________
F. Interviewer name: ___________________________________________________________
G. Note taker name: ___________________________________________________________
H. Date: ____________________________________________________________________
I. Start Time: ________________________________________________________________
J. Introduce team members and obtain consent: ___ YES ____ NO
K. State purpose of evaluation: ___ YES ____ NO
143
[NOTE TO INTERVIEWER: Please review and complete the KII Consent form with the participant. If 
agreement to participate voluntarily is obtained, please continue. If participant does NOT agree to take 
part in the study, thank them for their time, make a note they did not want to participate. Make sure to 
also ask for permission to record the interview on our audio-recorder; if participant says YES please 
continue; if participant says NO, acknowledge that you will not record the conversation and proceed 
without turning on the audio-recording.]
**IMPORTANT NOTE**:
Begin audio-recording AFTER consent to participate and be audio-recorded has been received.
1. In your view, has NPR helped to increase the number of primary school children 
reading fluently and with comprehension skills in NPR’s experimental schools? If 
yes, which aspects of NPR contributed to these improvements?
a. What have been NPR’s most important and valuable contributions to 
improving children's reading in its experimental schools? Why do you 
consider these the most important or valuable contributions?
2. NPR included two broad elements: curriculum and materials development and 
teacher instructional training. Has either of these elements met your expectations? 
If yes, in what ways? What has not met your expectations? Why, and how would you 
suggest these shortcomings be addressed?
3. What kinds of internal, NPR-related concerns or obstacles, if any, have impeded 
implementation of these activities? How has COVID specifically impeded these 
activities? Besides COVID, have any other external, contextual challenges impeded 
NPR? If so, how?
4. How well did NPR deliver on the creation and rollout of the curriculum in its 
experimental schools? What has impeded creation and rollout? Has the new 
curriculum addressed issues with the former curriculum? If so, which ones?
5. NPR trained teachers in its experimental schools in differentiated instruction, use of visual aids, 
and school-based teacher support and follow-up (TSF). Have these interventions improved 
teacher instruction in NPR’s experimental schools? If yes, which interventions? In what ways? 
What has impeded teacher instruction interventions?
6. NPR developed a series of distance-learning courses—or massive open online courses 
(MOOC)— to train teachers on the new curriculum. Have the MOOCs improved teachers’ 
instruction? If yes, in what ways? What has impeded the development of MOOCs and their use 
by teachers?
7. In 2020, NPR printed and distributed supplementary reading materials and teachers’ guides in 
experimental schools (show examples of materials). Has the supplemental reading materials 
program helped to improve children’s reading skills in experimental schools? If yes, in what 
ways? What has impeded implementation of the supplemental reading materials program?
8. Is the MoE’s ability to develop and deliver education reform in primary grades 
stronger after its participation in NPR? If yes, in what ways, and what role did NPR 
play in improving the MoE’s ability? Are there any areas or ways in which the MoE’s 
ability to develop and deliver education reform could still be improved?
144
I’d also like to ask you about NPR’s activities to develop the MoE’s capacity for 
measurement and evaluation of improved reading outcomes. 
9. Starting in 2019, NPR organized a series of capacity building-workshops for MoE 
personnel on evaluation. The general topics included assessment design; item 
development; developing a research agenda; benchmarking; data analysis, 
interpretation, and use; and test equating. If NPR was going to plan this series of 
capacity-building workshops again, should NPR do anything differently? If so, what 
and why? [PROBE: Different topics, format, time, materials, etc.]
10. Have these activities improved the MoE’s institutional ability to conduct EGRAs and 
use the data for informing implementation? Why or why not?
11. Has the MoE used—or does it have any plans to use—EGRA data to make any 
changes to the design of the Arabic language education reform (either curriculum, 
materials, teacher training, data use, etc.)? If so, how? 
12. NPR created a formative assessment toolkit, including a formative assessment 
guide, student remediation guide, and a teacher’s guide to unit tests for grades 1 to 
4. To what extent did MoE personnel find these materials valuable? To what extent 
did they use them? To what extent have the formative assessment materials been 
scaled nationwide? If yes, to what extent?
a. Has the MoE used—or does it have any plans to use—formative assessment 
data to make any changes to the design of the Arabic language education 
reform (either curriculum, materials, teacher training, data use, etc.)? If so, 
how? 
13. Does the MoE need any support to use data to support adaptive management and 
improve learning outcomes in the future? If so, what kind of support?
Now I’d like to ask about the launch of the new curriculum nationally. Before the start of 
the 2021-22 school year, the MoE announced that schools were required to use the new 
Arabic language curriculum introduced by NPR, and 8,500 copies of the curriculum were 
printed and distributed to schools nationwide. 
14. What did the MoE do to support the launch of the new curriculum and materials 
nationwide at the start of the 2021-22 school year? Did it provide enough support to 
regional and provincial education offices in order for schools nationwide to 
implement the curriculum and use the new materials as designed? Why/why not?
15. In February 2022, the MoE urgently requested that NPR develop a toolkit to help 
inspectors provide in-service training to teachers about the new Arabic language 
curriculum. What prompted this request? Do you know if inspectors have been 
using this toolkit since NPR submitted it to the MoE? If yes, has the inspectors’ 
training led to schools nationwide implementing the new curriculum and using the 
new materials as designed? Why/why not? 
16. To your knowledge, are teachers nationwide implementing the new curriculum and materials? If 
so, what proportion of teachers are implementing it—all, most, some, or few? [Ask if previous 
response is most, some, or few] Why aren't all schools implementing it? What additional 
support could the MoE provide to ensure all schools are able to implement the new Arabic 
language curriculum?
145
17. What did the MoE do well with scaling the new curriculum nationally? What did it not do well? 
What was the biggest obstacle the MoE had to overcome in scaling the curriculum, and how did 
it address it?
Now I’d like to ask about NPR’s activities for engaging community and parental support for reading. 
18. What kind of outreach activities did NPR train teachers and school directors to organize to help 
parents with the use of supplementary reading materials and the 3asafeer digital library at home? 
To your knowledge, how many teachers and school directors organized these activities—all, 
most, some, or few? Did parents follow the guidance? Did students read the materials? 
19. What other activities did NPR implement to increase community and parental support for 
reading? Did anything change regarding community and parental support due to the activities? 
Now I’d like to ask about NPR’s engagement with the private sector in reading 
programs. 
20. NPR and the MoE worked with the private textbook industry during the 
development of the new curriculum. What were the textbook industry’s strengths 
and weaknesses in this partnership? Were there any obstacles NPR had to 
overcome? If so, what did NPR try to do to overcome them? 
21. NPR and the MoE worked with the company 3asafeer during the development of 
the supplemental reading materials program. What were 3asafeer’s strengths and 
weaknesses in this partnership? Were there any obstacles NPR had to overcome? If 
so, what did NPR try to do to overcome them? 
Now I’d like to ask about NPR’s implementation and reception in urban and rural areas, different 
regions of the country, and with girls and boys.
22. Have you seen any notable differences in how the new curriculum has been received 
by children from different backgrounds – e.g., from certain regions/provinces, 
Arabic vs Amazigh, boys vs girls, children with disabilities, urban vs. rural? If yes, 
please describe these differences. Have there been any steps taken to address 
them?
23. Overall, have there been any unexpected outcomes or results (positive or negative) 
with NPR? Please detail.
CLOSING
1. Those are all of my questions. Are there any additional comments that you would 
like to share with us?
2. Thank you very much for taking the time to answer our questions. Do you have any 
questions for our team?
SUMMARY AND CONCLUSION
Thank you for your time. Your help in this research is very important. As I mentioned, 
the results of the report will be used by USAID Morocco and the Ministry of National 
Education, Preschool, and Sports (MoE) to understand how the NPR project activities 
are going and are being implemented. The final results of our research project will be 
published in a report in the coming months. These results will be communicated back 
146
to USAID and the MoE.
End Time: ___________________________
Total length: ________Hours _________Minutes
POST-KII NOTES:
Please comment on
● Any factors that may have affected the truthfulness of the responses given and the willingness of the 
interview subject to participate, 
● If more than one respondent participated, the different perspectives that emerged through 
disagreements in the interviews,
● Any additional insights or comments that should be noted.
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
147
ANNEX VIII: GETTING TO ANSWERS MATRIX 
Cells beginning with an asterisk (also highlighted in blue) represent prioritized questions.
1 2 3a 3b 3c 3d 3e 3f 4 5a 5b 6a 6b 7a 7b 8
Tool 
Name
USAID KII 
Guide
USAID 
Implementing 
Partners KII 
Guide
MoE Official KII 
Guide/Central 
MoE
MoE Official KII 
Guide/
Communicatio
ns Officer
MoE Official KII 
Guide/
Experimental 
Regions
MoE Official KII 
Guide/
Experimental 
Regions -
Inspectors Only
MoE Official KII 
Guide/Scale￾up Regions
MoE Official KII 
Guide/Scale￾up Regions -
Inspectors Only
MSL 
Intervention KII 
Guide
Private Sector 
Personnel KII 
Guide
Private Sector 
Personnel KII 
Guide
School 
Director KII 
Guide/Experim
ental Schools
School 
Director KII 
Guide/Scale-up 
Schools
Teacher FGD 
Guide / 
Experimental 
Schools
Teacher FGD 
Guide / Scale￾Up Schools
Parent FGD 
Guide
Respondents 
(N) 2 2 4 1 19 8 10 4 2 2 1 8 4 8 4 12 Subgroups
N/A N/A MoE Central 
Office Directors 
MoE Central 
Office 
Communication 
Officer
AREF/Regional 
Directors 
Provincial 
Directors/Provinci
al Coordinators 
CMREF Directors 
Province-level 
Arabic teacher 
trainers from 
CMREF 
Inspectors AREF/Regional 
Directors 
Provincial 
Directors/Provinci
al Coordinators 
CMREF Directors 
Province-level 
Arabic teacher 
trainers from 
CMREF 
Inspectors Representatives 
involved with the 
MSL internvention
Textbook 
publishers
3asafeer 
(educational 
technology 
company)
N/A N/A N/A N/A N/A
Sampling /Selection Criteria
At least two KIIs 
with 
USAID/Morocco 
Education officials
One KII with 
COP or DCOP 
and one KII with 
NPR senior staff
4 KIIs with Central 
Office Directors, 
including officials 
from CNIPE, 
CNEE, and the 
Curricula 
Directorate 
1 KII with a 
Communication 
Officer
4 AREF Regional 
Directors
8 Provincial 
Directors/Provinci
al Coordinators
4 Provincial Arabic 
Language Teacher 
Trainers
3 CMREF 
Directors 
8 Inspectors 2 AREF Regional 
Directors
4 Provincial 
Directors/Provinci
al Coordinators
2 Provincial Arabic 
Language Teacher 
Trainers
2 CMREF
Directors
4 Inspectors Two KIIs with 
respondents 
identified by MoE 
or 
USAID/Morocco 
who participated 
in the efforts to 
provide MSL 
interpretation for 
distance-learning 
video lessons
Two KIIs with 
relevant personnel 
identified by MoE 
or USAID
One KII with 
relevant personnel 
identified by MoE 
or USAID
8 KIIs with 
primary school 
directors in NPR 
schools
4 KIIs with 
primary school 
directors in scale￾up schools
8 FGDs with 
primary school 
teachers (grade 1-
6) in NPR schools
4 FGDs with 
primary school 
teachers (grade 1-
6) in scale-up 
schools
12 FGDs with 
parents of primary 
school students 
(grades 1-6) who 
are active in 
school 
management 
committees, 
parent 
organizations, or 
other school 
groups
EQs1. LEARNING OUTCOMES. How has the new NPR reading curriculum and teacher instruction package affected learning outcomes in target schools? What 
role has the COVID-19 pandemic played in learning outcomes? More 
specifically: *In your view, has 
NPR helped to 
increase the 
number of 
primary school 
children reading 
fluently and with 
comprehension 
skills in NPR’s 
experimental 
schools? If yes, 
which aspects of 
NPR contributed 
to these 
improvements?
*In your view, has 
NPR helped to 
increase the 
number of 
primary school 
children reading 
fluently and with 
comprehension 
skills in NPR’s 
experimental 
schools? If yes, 
which aspects of 
NPR contributed 
to these 
improvements?
*In your view, has 
NPR helped to 
increase the 
number of 
primary school 
children reading 
fluently and with 
comprehension 
skills in NPR’s 
experimental 
schools? If yes, 
which aspects of 
NPR contributed 
to these 
improvements? 
*As you know, the 
Ministry of 
Education has 
been working with 
USAID to develop 
a curriculum 
reform - a new 
program for 
teaching Arabic in 
grades 1-6. This 
curriculum was 
piloted in selected 
"experimental 
schools" and then 
expanded to the 
entire country in 
2021. The 
program included: 
a) teaching 
materials; b) new 
textbooks; c) a 
training toolkit for 
inspectors; d) 
access to MOOC 
*In your view, has 
NPR helped to 
increase the 
number of 
primary school 
children reading 
fluently and with 
comprehension 
skills in NPR’s 
experimental 
schools? If yes, 
which aspects of 
NPR contributed 
to these 
improvements?
*In your view, has 
NPR helped to 
increase the 
number of 
primary school 
children reading 
fluently and with 
comprehension 
skills in NPR’s 
experimental 
schools? If yes, 
which aspects of 
NPR contributed 
to these 
improvements?
*The MoE has 
worked with 
USAID to develop 
a new Arabic 
language 
curriculum in 
grades 1 to 6. This 
curriculum was 
piloted in select 
schools and then
scaled nationwide 
in 2021. The 
program consisted 
of a) curriculum 
materials; b) new 
textbooks; c) 
training toolkit for 
inspectors; d) 
access to the 
MOOC content; 
e) supplemental 
reading materials 
(books and 
teachers’ guides); f) 
*The MoE has 
worked with 
USAID to develop 
a new Arabic 
language 
curriculum in 
grades 1 to 6. This 
curriculum was 
piloted in select 
schools and then 
scaled nationwide 
in 2021. The 
program consisted 
of a) curriculum 
materials; b) new 
textbooks; c) 
training toolkit for 
inspectors; d) 
access to the 
MOOC content; 
e) supplemental 
reading materials 
(books and 
teachers’ guides); f) 
*The MoE 
developed and 
rolled out an early 
grade reading 
curriculum and 
teacher instruction 
package called 
NPR in 90 
experimental 
schools. As you 
probably know, 
the program 
consisted of a) 
curriculum 
materials; b) new 
textbooks; c) 
access to the 
MOOC content; 
d) supplemental 
reading materials 
(books and 
teachers’ guides); 
e) capacity building 
on measurement 
*The MoE has 
worked with 
USAID to develop
a new Arabic 
language 
curriculum in 
grades 1 to 6. This 
curriculum was 
piloted in select 
schools and then 
scaled nationwide 
in 2021. The 
program consisted 
of a) curriculum 
materials; b) new 
textbooks; c) 
access to the 
MOOC content; 
d) supplemental 
readingmaterials 
(books and 
teachers’ guides); 
e) any other 
training activities. 
*The MoE 
developed and 
rolled out an early 
grade reading 
curriculum and 
teacher instruction 
package called 
NPR in 90 
experimental 
schools. As you 
probably know, 
the program 
consisted of a) 
curriculum 
materials; b) new 
textbooks; c) 
access to the 
MOOC content; 
d) supplemental 
reading materials 
(books and 
teachers’ guides); 
e) capacity building 
on measurement 
*The MoE has 
worked with 
USAID to develop 
a new Arabic 
language 
curriculum in 
grades 1 to 6. This 
curriculum was 
piloted in select 
schools and then 
scaled nationwide 
in 2021. The 
program consisted 
of a) curriculum 
materials; b) new 
textbooks; c) 
access to the 
MOOC content; 
d) supplemental 
reading materials 
(books and 
teachers’ guides); 
e) any other 
training activities. 
*The MoE has 
worked with 
USAID to 
develop a new 
Arabic language 
curriculum in 
grades 1 to 6. This 
curriculum was 
piloted in select 
schools and then 
scaled nationwide 
in 2021. The 
program 
consisted of a) 
curriculum 
materials; b) new 
textbooks; c) 
access to the 
MOOC content; 
d) supplemental 
reading materials 
(books and 
teachers’ guides); 
e) any other 
148
1 2 3a 3b 3c 3d 3e 3f 4 5a 5b 6a 6b 7a 7b 8
content; e) 
additional reading 
materials (books 
and teacher 
guides); f) MSL 
video content; g) 
capacity building in 
measurement and 
assessment; h) any 
other training 
activities. Are you 
aware of this 
program? 
MSL video 
content; g) 
capacity building 
on measurement 
and evaluation; h) 
any other training 
activities. Are you 
aware of this 
program? 
MSL video 
content; g) 
capacity building 
on measurement 
and evaluation; h) 
any other training 
activities. Are you 
aware of this 
program? 
and evaluation; f) 
any other training 
activities. Were 
each of these 
activities and 
materials delivered 
at your school? If 
so, which activities 
or materials made 
the greatest 
impact on 
children's reading 
skills and teachers' 
instructional ability?
Are you aware of 
this program? If so, 
were each of 
these activities and 
materials delivered 
at your school? If 
so, which activities 
or materials made 
the greatest 
impact on 
children's reading 
skills and teachers' 
instructional ability? 
and evaluation; f) 
any other training 
activities. Were 
each of these 
activities and 
materials delivered 
at your school?
Do you all have 
the teacher’s 
guide? Does 
anyone here not 
have it? If you do 
not have it, what 
materials do you 
use to prepare 
your lessons for 
class? How 
frequently do you 
use the teacher's 
guide? Is it easy to 
use? Why/why 
not? If it's easy to 
use, in what ways 
does it help you 
teach reading? Do 
the new 
textbooks help 
your students 
learn how to read? 
Why/why not?
Are you aware of 
this program? If so, 
were each of 
these activities and 
materials delivered 
at your school? 
Do you all have 
the teacher’s 
guide? Does 
anyone here not 
have it? If you do 
not have it, what 
materials do you 
use to prepare 
your lessons for 
class? How 
frequently do you 
use the teacher's 
guide? Is it easy to 
use? Why/why 
not? If it's easy to 
use, in what ways 
does it help you 
teach reading? Do 
the new 
textbooks help 
your students 
learn how to read? 
Why/why not?
training activities. 
Are you aware of 
this program? 
If so, have you 
seen any of the 
new materials 
(textbooks and 
supplemental 
reading materials)? 
If yes, have you 
used the materials 
at all with your 
child/ren at home? 
How? Do you 
find them easy to 
use? Why/why 
not? Are they 
enjoyable for your 
children? Are the 
new materials 
(textbooks and 
supplemental 
reading materials) 
helping your 
child/ren learn 
how to read?
*What have been 
NPR’s most 
important and 
valuable 
contributions to 
improving 
children's reading 
in its experimental 
schools? Why do 
you consider 
these the most 
important or 
valuable 
contributions?
*What have been 
NPR’s most 
important and 
valuable 
contributions to 
improving 
children's reading 
in its experimental 
schools? Why do 
you consider 
these the most 
important or 
valuable 
contributions?
*What have been 
NPR’s most 
important and 
valuable 
contributions to 
improving 
children's reading 
in its experimental 
schools? Why do 
you consider 
these the most 
important or 
valuable 
contributions?
*If so, what role 
did you play in this 
program?
What have been 
NPR’s most 
important and 
valuable 
contributions to 
improving 
children’s reading 
in its experimental 
schools? Why do 
you consider 
these the most 
important or 
valuable 
contributions?
*What have been 
NPR’s most 
important and 
valuable 
contributions to 
improving 
children’s reading 
in its experimental 
schools? Why do 
you consider 
these the most 
important or 
valuable 
contributions?
If yes, to any of the 
above, what was 
done? If yes, did 
these activities help 
to increase the 
number of 
primary school 
children reading 
fluently and with 
comprehension 
skills? Please 
describe why.
*If yes, to any of 
the above, what 
was done? If yes, 
did these activities 
help to increase 
the number of 
primary school 
children reading 
fluently and with 
comprehension 
skills? Please 
describe why.
*What have been 
NPR's most 
important and 
valuable 
contributions to 
improving reading 
for learners in 
your school? Why 
are these 
contributions the 
most valuable?
*[Ask only if new 
curriculum is being 
implemented at 
the school] What 
have been the 
new curriculum's 
most important 
and valuable 
contributions to 
improving reading 
for learners in 
your school? Why 
are these 
contributions the
most valuable?
*What have been 
NPR's most 
important and 
valuable 
contributions to 
improving reading 
for learners in 
your school? Why 
are these 
contributions the 
most valuable?
*[Ask only if new 
curriculum is being 
implemented at 
the school] What 
have been the 
new curriculum's 
most important 
and valuable 
contributions to 
improving reading 
for learners in 
your school? Why 
are these 
contributions the 
most valuable?
N/A
*NPR included 
two broad 
elements: 
curriculum and 
materials 
development and 
teacher 
instructional 
training. Has either 
of these elements 
met your 
expectations? If 
yes, in what ways? 
What has not met 
your expectations? 
Why, and how 
would you suggest 
these 
*NPR included 
two broad 
elements: 
curriculum and 
materials 
development and 
teacher 
instructional 
training. Has either 
of these elements 
met your 
expectations? If 
yes, in what ways? 
What has not met 
your expectations? 
Why, and how 
would you suggest 
these 
*NPR included 
two broad 
elements: 
curriculum and 
materials 
development and 
teacher 
instructional 
training. Has either 
of these elements 
met your 
expectations? If 
yes, in what ways? 
What has not met 
your expectations? 
Why, and how 
would you suggest 
these 
*To your 
knowledge, has 
the NPR program 
(implementation 
of the new 
curriculum) had an 
impact on:
a. Teaching Arabic 
reading in 
Morocco? Please 
explain.
b. Students' 
reading skills? 
Please explain.
c. Attitudes about 
the new 
curriculum or 
approach to 
*Within the 
different elements 
of NPR, what has 
met your 
expectations? 
Why? What has 
not met your 
expectations? 
Why, and how 
would you suggest 
these 
shortcomings be 
addressed?
Within the 
different elements 
of NPR, what has 
met your 
expectations? 
Why? What has 
not met your 
expectations? 
Why, and how 
would you suggest 
these 
shortcomings be 
addressed?
*Within the 
different elements 
of the new 
curriculum, what 
has met your 
expectations? 
Why? What has 
not met your 
expectations? 
Why, and how 
would you suggest 
these 
shortcomings be 
addressed?
Within the 
different elements 
of the new 
curriculum, what 
has met your 
expectations? 
Why? What has 
not met your 
expectations? 
Why, and how 
would you suggest 
these 
shortcomings be 
addressed?
*Within the 
different elements 
of NPR, what has 
met your 
expectations? 
Why? What has 
not met your 
expectations? 
Why, and how 
would you suggest 
these 
shortcomings be 
addressed?
*[Ask only if new 
curriculum is being 
implemented at 
the school] Within 
the different 
elements of the 
new curriculum, 
what has met 
your expectations? 
Why? What has 
not met your 
expectations? 
Why, and how 
would you suggest 
these 
shortcomings be 
addressed?
Within the 
different elements 
of NPR, what has 
met your 
expectations? 
Why? What has 
not met your 
expectations? 
Why, and how 
would you suggest 
these 
shortcomings be 
addressed?
[Ask only if new 
curriculum is being 
implemented at 
the school] Within 
the different 
elements of the 
new curriculum, 
what has met 
your expectations? 
Why? What has 
not met your 
expectations? 
Why, and how 
would you suggest 
these 
shortcomings be 
addressed?
N/A
149
1 2 3a 3b 3c 3d 3e 3f 4 5a 5b 6a 6b 7a 7b 8
shortcomings be 
addressed?
shortcomings be 
addressed?
shortcomings be 
addressed?
teaching? Please 
explain.
*What kinds of 
internal, NPR￾related concerns 
or obstacles, if any, 
have impeded 
implementation of 
these activities? 
How has COVID 
specifically 
impeded these 
activities? Besides 
COVID, have any 
other external, 
contextual 
challenges 
impeded NPR? If 
so, how?
*What kinds of 
internal, NPR￾related concerns 
or obstacles, if any, 
have impeded
implementation of 
these activities? 
How has COVID 
specifically 
impeded these 
activities? Besides 
COVID, have any 
other external, 
contextual 
challenges 
impeded NPR? If 
so, how?
*What kinds of 
internal, NPR￾related concerns 
or obstacles, if any, 
have impeded 
implementation of 
these activities? 
How has COVID 
specifically 
impeded these 
activities? Besides 
COVID, have any 
other external, 
contextual 
challenges 
impeded NPR? If 
so, how?
*What kinds of 
internal, NPR￾related concerns 
or obstacles, if any, 
have impeded 
implementation of 
these activities? 
How has COVID 
specifically 
impeded these 
activities? Besides 
COVID, have any 
other external, 
contextual 
challenges 
impeded NPR? If 
so, how?
What kinds of 
internal, NPR￾related concerns 
or obstacles, if any, 
have impeded 
implementation of 
these activities? 
How has COVID 
specifically 
impeded these 
activities? Besides 
COVID, have any 
other external, 
contextual 
challenges 
impeded NPR? If 
so, how?
What kinds of 
internal, NPR￾related concerns 
or obstacles, if any, 
have impeded 
implementation of 
these activities? 
How has COVID 
specifically 
impeded these 
activities? Besides 
COVID, have any 
other external, 
contextual 
challenges 
impeded NPR? If 
so, how?
What kinds of 
internal, NPR￾related concerns 
or obstacles, if any, 
have impeded 
implementation of 
these activities? 
How has COVID 
specifically 
impeded these 
activities? Besides 
COVID, have any 
other external, 
contextual 
challenges 
impeded NPR? If 
so, how?
*Has COVID 
specifically affected 
the rollout of the 
curriculum 
reform? How?
*[Ask only if new 
curriculum is being 
implemented at 
the school] Has 
COVID specifically 
affected the rollout 
of the curriculum 
reform? How?
Has COVID 
specifically affected 
the rollout of the 
curriculum 
reform? How?
[Ask only if new 
curriculum is being 
implemented at 
the school] Has 
COVID specifically 
affected the rollout 
of the curriculum 
reform? How?
*How has 
COVID impacted 
your children's 
education? Have 
you observed any 
differences in their 
attitude about 
school since 
schools reopened 
after the 
pandemic? If yes, 
what are they?
2. NPR DELIVERY. How well did NPR deliver 
on the development and testing of the revised 
reading curriculum and teacher instruction 
interventions? *How well did 
NPR deliver on 
the creation and 
rollout of the 
curriculum in its 
experimental 
schools? What has 
impeded creation 
and rollout? Has 
the new 
curriculum 
addressed issues 
with the former 
curriculum? If so, 
which ones?
*How well did 
NPR deliver on 
the creation and 
rollout of the 
curriculum in its 
experimental 
schools? What has 
impeded creation 
and rollout? Has 
the new 
curriculum 
addressed issues 
with the former 
curriculum?
*How well did 
NPR deliver on 
the creation and 
rollout of the 
curriculum in its 
experimental 
schools? What has 
impeded creation 
and rollout? Has 
the new 
curriculum 
addressed issues 
with the former 
curriculum?
*How well did 
NPR deliver on 
the creation and 
rollout of the 
curriculum in its 
experimental 
schools? What has 
impeded creation 
and rollout? Has 
the new 
curriculum 
addressed issues 
with the former 
curriculum?
How well did 
NPR deliver on 
the creation and 
rollout of the 
curriculum in its 
experimental 
schools? What has 
impeded creation 
and rollout? Has 
the new 
curriculum 
addressed issues 
with the former 
curriculum?
N/A N/A N/A N/A N/A N/A N/A
NPR trained 
teachers in its 
experimental 
schools in 
differentiated 
instruction, use of 
visual aids, and 
school-based 
teacher support 
and follow-up 
(TSF). Have these 
interventions 
improved teacher 
instruction in 
NPR’s 
experimental 
schools? If yes, 
which 
interventions? In 
what ways? What 
has impeded 
teacher instruction 
interventions?
NPR trained 
teachers in its 
experimental 
schools in 
differentiated 
instruction, use of 
visual aids, and 
school-based 
teacher support 
and follow-up 
(TSF). Have these 
interventions 
improved teacher 
instruction in 
NPR’s 
experimental 
schools? If yes, in 
what ways? What 
has impeded 
teacher instruction 
interventions?
NPR trained 
teachers in its 
experimental 
schools in 
differentiated 
instruction, use of 
visual aids, and 
school-based 
teacher support 
and follow-up 
(TSF). Have these 
interventions 
improved teacher 
instruction in 
NPR’s 
experimental 
schools? If yes, in 
what ways? What 
has impeded 
teacher instruction 
interventions?
NPR trained 
teachers in its 
experimental 
schools in 
differentiated 
instruction, use of 
visual aids, and 
school-based 
teacher support 
and follow-up 
(TSF). Have these 
interventions 
improved teacher 
instruction in 
NPR’s 
experimental 
schools? If so, in 
what ways? What 
has impeded 
teacher instruction 
interventions?
*NPR trained 
teachers in its 
experimental 
schools in 
differentiated 
instruction, use of 
visual aids, and 
school-based 
teacher support 
and follow-up 
(TSF). Have these 
interventions 
improved teacher 
instruction in 
NPR’s 
experimental 
schools? If so, in 
what ways? What 
has impeded 
teacher instruction 
interventions?
*As part of the 
new curriculum 
development 
(NPR?), some 
MoE inspectors 
attended national 
workshops to 
finalize and validate 
the new 
curriculum. Did 
you participate in 
any of these
workshops? If so, 
did they improve 
your 
understanding 
about the new 
curriculum and 
your role in 
implementing it? If 
yes, how?
As part of the new 
curriculum 
development 
(NPR?), some 
MoE inspectors 
attended national 
workshops to 
finalize and validate 
the new 
curriculum. Did 
you participate in 
any of these 
workshops? If so, 
did they improve 
your 
understanding 
about the new 
curriculum and 
your role in 
implementing it? If 
yes, how?
As part of NPR, 
teachers were 
trained in 
experimental 
schools in 
differentiated 
instruction, use of 
visual aids, and 
school-based 
teacher support 
and follow-up 
(TSF). Did you 
and/or teachers in 
your school 
participate in any 
of these trainings? 
If so, did you 
benefit from 
them? How? How 
would you rate 
them—excellent, 
good, fair, or not 
good? If so, do you 
think they 
improved 
teachers' skills in 
teaching reading 
and/or your skills 
in supporting 
teachers? How? 
Have you received 
any training from 
inspectors or 
anyone else on the 
new curriculum? If 
so, when, and on 
what topics? If so, 
did you benefit 
from them? How? 
How would you 
rate them—
excellent, good, 
fair, or not good? 
Do you think the 
training covered all 
youneed to know 
in order to adopt 
the new 
curriculum and 
use the new 
materials at your 
school? Why/why 
not? If no, what 
additional training 
do you need?
*As part of NPR, 
teachers were 
trained in 
experimental 
schools in 
differentiated 
instruction, use of 
visual aids, and 
school-based 
teacher support 
and follow-up 
(TSF). Did you 
participate in any 
of these trainings? 
If so, did you 
benefit from 
them? How? How 
would you rate 
them—excellent, 
good, fair, not 
good? Did they 
improve your skills 
in teaching reading 
and/or your 
director's skills in 
supporting you? If 
yes, how?
*Have you 
received any 
training from 
inspectors or 
anyone else on the 
new curriculum? If 
so, when, and on 
what topics? If so, 
did you benefit 
from them? How? 
How would you 
rate them—
excellent, good, 
fair, not good? Do 
you think the 
training covered all 
you need to know 
in order to adopt 
the new 
curriculum and 
use the new 
materials at your 
school? Why/why 
not? If no, what 
additional training 
do you need?
N/A
150
1 2 3a 3b 3c 3d 3e 3f 4 5a 5b 6a 6b 7a 7b 8
Do you think 
these trainings 
improved 
instruction for all 
groups of 
students? If not, 
which group was 
left out, and why?
NPR developed a 
series of distance￾learning courses—
or massive open 
online courses 
(MOOC)—to 
train teachers on 
the new 
curriculum. Have 
the MOOCs 
improved 
teachers’ 
instruction? If yes, 
in what ways? 
What has 
impeded the 
development of 
MOOCs and 
their use by 
teachers?
NPR developed a 
series of distance￾learning courses—
or massive open 
online courses 
(MOOC)—to 
train teachers on 
the new 
curriculum. Have 
the MOOCs 
improved 
teachers’ 
instruction? If yes, 
in what ways? 
What has 
impeded the 
development of 
MOOCs and 
their use by 
teachers?
NPR developed a 
series of distance￾learning courses—
or massive open 
online courses 
(MOOC)—to 
train teachers on 
the new 
curriculum. To 
what extent have 
teachers enrolled 
in these courses? 
Have the teachers 
used what they 
learned to 
improve their 
instruction? If yes, 
how? What has 
impeded the 
development of 
MOOCs?
NPR developed a 
series of distance￾learning courses—
or massive open 
online courses 
(MOOC)—to 
train teachers on 
the new 
curriculum. To 
what extent have 
teachers enrolled 
in these courses? 
Have the teachers 
used what they 
learned to 
improve their 
instruction? If yes, 
how? What has 
impeded the 
development of 
MOOCs?
*NPR developed a 
series of distance￾learning courses—
or massive open 
online courses 
(MOOC)—to 
train teachers on 
the new 
curriculum. To 
what extent have 
teachers enrolled 
in these courses? 
Have the teachers 
used what they 
learned to 
improve their 
instruction? If yes, 
how? What has 
impeded the 
development of 
MOOCs?
The MoE 
developed a series 
of distance￾learning courses—
or massive open 
online courses 
(MOOC)—to 
train teachers on 
the new 
curriculum. To 
what extent have 
teachers enrolled 
in these courses? 
Have the teachers 
used what they 
learned to 
improve their 
instruction? If yes, 
how? What has 
impeded the 
development of 
MOOCs?
*The MoE 
developed a series 
of distance￾learning courses—
or massive open 
online courses 
(MOOC)—to 
train teachers on 
the new 
curriculum. To 
what extent have 
teachers enrolled 
in these courses? 
Have the teachers 
used what they 
learned to 
improve their 
instruction? If yes, 
how? What has 
impeded the 
development of 
MOOCs?
NPR developed a 
series of distance￾learning courses—
or massive open 
online courses 
(MOOC)—to 
train teachers on 
the new 
curriculum. Do 
you know if any of 
your teachers 
enrolled in these
courses? If so, do 
you feel the 
MOOCs have 
been effective in 
improving 
teachers’ ability to 
teach reading with 
your (special 
population)? In 
what ways? 
[Ask if director 
mentioned he/she 
was aware of 
MOOC program] 
The MoE 
developed a series 
of distance￾learning courses—
or massive open 
online courses 
(MOOC)—to 
train teachers on 
the new 
curriculum. Do 
you know if any of 
your teachers 
enrolled in these 
courses? If so, do 
you feel the 
MOOCs have 
been effective in 
improving 
teachers’ ability to 
teach reading with 
your (special 
population)? In 
what ways? 
*NPR developed a 
series of distance￾learning courses—
or massive open 
online courses 
(MOOC)—to 
train teachers on 
the new 
curriculum. Did 
any of you enroll 
in these courses? If 
so, did you benefit 
from them? How?
How would you 
rate them—
excellent, good, 
fair, not good? If 
so, did the 
MOOCs improve 
your ability to 
teach reading with 
your (special 
population)? In 
what ways? 
*The MoE 
developed a series 
of distance￾learning courses—
or massive open 
online courses 
(MOOC)—to 
train teachers on 
the new 
curriculum. Are 
you aware of 
these courses? If 
yes, have you 
enrolled in these 
courses? If so, did 
you benefit from 
them? How? How 
would you rate 
them—excellent, 
good, fair, not 
good? If so, have 
the MOOCs 
improved your 
ability to teach 
reading with your 
(special 
population)? In 
what ways? 
N/A
In 2020, NPR 
printed and 
distributed 
supplementary 
reading materials 
and teachers’ 
guides in 
experimental 
schools (show 
examples of 
materials). Has the 
supplemental 
reading materials 
program helped to 
improve children’s 
reading skills in 
experimental 
schools? If yes, in 
what ways? What 
has impeded 
implementation of 
the supplemental 
reading materials 
program?
In 2020, NPR 
printed and 
distributed 
supplementary 
reading materials
and teachers’ 
guides in 
experimental 
schools (show 
examples of 
materials). Has the 
supplemental 
reading materials 
program helped to 
improve children’s 
reading skills in 
experimental 
schools? If yes, in 
what ways? What 
has impeded 
implementation of 
the supplemental 
reading materials 
program?
In 2020, NPR 
printed and 
distributed 
supplementary 
reading materials 
and teachers’ 
guides in 
experimental 
schools (show 
examples of 
materials) and 
then distributed 
them nationwide 
in 2022. Has the 
supplemental 
reading materials 
program helped 
to improve 
children’s reading 
skills? If yes, in what 
ways? What has 
impeded 
implementation of 
the supplemental 
reading materials 
program?
In 2020, NPR 
printed and 
distributed 
supplementary 
reading materials 
and teachers’ 
guides in 
experimental 
schools (show 
examples of 
materials) and 
then distributed 
them nationwide 
in 2022. NPR also 
asked schools to 
dedicate 30 
minutes per week 
for students to use 
them. Have all 
schools dedicated 
30 minutes per 
week to the 
activity? If not, why 
do you think 
schools have not 
dedicated that 
time to the 
activity? Have 
these materials 
*In 2020, NPR 
printed and 
distributed 
supplementary 
reading materials 
and teachers’ 
guidesin 
experimental 
schools (show 
examples of 
materials) and 
then distributed 
them nationwide 
in 2022. NPR also 
asked schools to 
dedicate 30 
minutes per week 
for students to use 
them. Have all 
schools dedicated 
30 minutes per 
week to the 
activity? If not, why
do you think 
schools have not 
dedicated that 
time to the 
activity? Have 
these materials 
In 2022, the MoE 
printed and 
distributed 
supplementary 
reading materials 
and teachers’ 
guides to schools 
nationwide (show 
examples of 
materials) and 
asked schools to 
dedicate 30 
minutes per week 
for students to use 
them. Did all 
schools in your 
region/province 
receive these 
materials? Have all 
schools dedicated 
30 minutes per 
week to the 
activity? If not, why 
do you think 
schools have not 
dedicated that 
time to the 
activity? Have 
these new 
*In 2022, the MoE 
printed and 
distributed 
supplementary 
reading materials 
and teachers’ 
guides to schools 
nationwide (show 
examples of 
materials) and 
asked schools to 
dedicate 30 
minutes per week 
for students to use 
them. Did all 
schools in your 
region/province 
receive these 
materials? Have all 
schools dedicated 
30 minutes per 
week to the 
activity? If not, why 
do you think 
schools have not 
dedicated that 
time to the 
activity? Have 
these new 
*In 2020, NPR 
printed and 
distributed 
supplementary 
reading materials 
and teachers’ 
guides in 
experimental 
schools. Can 
children access 
these materials in 
your classroom? If 
so, how much 
time per week do 
your students use 
the SRMs? Have 
the SRMs have 
helped to improve 
early grade reading 
skills in your 
school? If yes, in 
what ways? What 
did they do well? 
What would you 
like to see done 
differently?
*[Ask if director 
mentioned he/she 
was aware of SRM 
program] In 2022, 
the MoE printed 
and distributed 
supplementary 
reading materials 
and teachers’ 
guides nationwide. 
Canchildren 
access these 
materials in your 
classroom? If so, 
how much time 
per week do your 
students use the 
SRMs? Have the 
SRMs helped to 
improve early 
grade reading skills 
in your school? In 
what ways? If yes, 
what did they do 
well? What would 
you like to see 
done differently?
In 2020, NPR 
printed and 
distributed 
supplementary 
reading materials 
and teachers’ 
guides in 
experimental 
schools, as well as 
provided access to 
digital reading 
materials. Can 
children access 
these materials in 
your classroom? If 
so, how much 
time per week do 
your students use 
the SRMs? Have 
SRMs helped to 
improve early 
grade reading skills 
in your school? In 
what ways? What 
did they do well? 
What would you 
like to see done 
differently?
In 2022, the MoE 
printed and 
distributed 
supplementary 
reading materials 
and teachers’ 
guides nationwide. 
Can children 
access these 
materials in your 
classroom? If so, 
how much time 
per week do your 
students use the 
SRMs? Have the 
SRMs helped to 
improve early 
grade reading skills 
in your school? In 
what ways? What 
did they do well? 
What would you 
like to see done 
differently?
N/A
151
1
2 3a 3b 3c 3d 3e 3f
4 5a 5b 6a 6b 7a 7b
8
helped to improve 
children’s early 
grade reading 
skills? If yes, in what 
ways?
helped to improve 
children’s early 
grade reading 
skills? If yes, in what 
ways?
materials helped 
to improve early 
grade reading 
skills? If yes, in what 
ways?
materials helped 
to improve early 
grade reading 
skills? If yes, in what 
ways?
3. MOE CAPACITY STRENGTHENED. To what extent was NPR successful in 
strengthening the Ministry’ s ability to develop and deliver an education reform in 
primary grades?
*Is the MoE’s 
ability to develop 
and deliver 
education reform 
in primary grades 
stronger after its 
participation in 
NPR? If yes, in 
what ways, and 
what role did NPR 
play in improving 
the MoE’s ability? 
Are there any 
areas or ways in 
which the MoE’s 
ability to develop 
and deliver 
education reform 
could still be 
improved?
*Is the MoE’s 
ability to develop 
and deliver 
education reform 
in primary grades 
stronger after its 
participation in 
NPR? If yes, in 
what ways, and 
what role did NPR 
play in improving 
the MoE’s ability? 
Are there any 
areas or ways in 
which the MoE’s 
ability to develop 
and deliver 
education reform 
could still be 
improved?
*Do you feel your 
unit is more able 
to develop and 
deliver education 
reform in primary 
grades after the 
development of 
the new 
curriculum? If yes, 
in what ways? 
What role have 
NPR partners 
(including Creative 
and USAID) 
played in 
improving your 
office's ability? Are 
there any areas or 
ways in which 
your ability to 
develop and 
deliver education 
reform could still 
be improved?
*Is your
regional/provincial 
office/inspectorate/
CMREF more 
able to develop 
and deliver 
education reform 
in primary grades 
after the 
development of 
the new 
curriculum? If yes, 
in what ways, and 
what role has the 
MoE played in 
improving your 
regional/provincial 
office/inspectorate/
CMREF’s ability? 
Are there any 
areas or ways in 
which your ability 
to develop and 
deliver education 
reform could still 
be improved?
Is your 
regional/provincial 
office/inspectorate/
CMREF more 
able to develop 
and deliver 
education reform 
in primary grades 
after the 
development of 
the new 
curriculum? If yes, 
in what ways, and 
what role has the 
MoE played in 
improving your 
regional/provincial 
office/inspectorate/
CMREF’s ability? 
Are there any 
areas or ways in 
which your ability 
to develop and deliver education 
reform could still 
be improved?
*Is your 
regional/provincial 
office/inspectorate/
CMREF more 
able to develop 
and deliver 
education reform 
in primary grades 
after the 
development of 
the new 
curriculum? If yes, 
in what ways, and 
what role hasthe 
MoE played in 
improving your 
regional/provincial 
office/inspectorate/
CMREF's ability? 
Are there any 
areas or ways in 
which your ability 
to develop and 
deliver education 
reform could still 
be improved?
Is your 
regional/provincial 
office/inspectorate/
CMREF more 
able to develop 
and deliver 
education reform 
in primary grades 
after the 
development of 
the new 
curriculum? If yes, 
in what ways, and 
what role has the 
MoE played in 
improving your 
regional/provincial 
office/inspectorate/
CMREF's ability? 
Are there any 
areas or ways in 
which your ability 
to develop and 
deliver education 
reform could still 
be improved?
N/A N/A N/A N/A N/A
*I’d also like to ask 
you about NPR’s 
activities to 
develop the MoE’s 
capacity for 
measurement and 
evaluation of 
improved reading 
outcomes. 
*I’d also like to ask 
you about NPR’s 
activities to 
develop the MoE’s 
capacity for 
measurement and 
evaluation of 
improved reading 
outcomes. 
*I’d also like to ask 
you about NPR’s 
activities to 
develop the MoE’s 
capacity for 
measurement and 
evaluation of 
improved reading 
outcomes. 
*I’d also like to ask 
you about NPR’s 
activities to 
develop the MoE’s 
capacity for 
measurement and 
evaluation of 
improved reading 
outcomes. 
I’d also like to ask 
you about NPR’s 
activities to 
develop the MoE’s 
capacity for 
measurement and 
evaluation of 
improved reading 
outcomes. 
N/A [NOTE: This 
question does not 
apply because we 
assume no MoE 
officials from non
-
intervention 
regions 
participated in any 
capacity-building 
activities.]
N/A [NOTE: This 
question does not 
apply because we 
assume no MoE 
officials from non
-
intervention 
regions 
participated in any 
capacity-building 
activities.]
N/A N/A N/A N/A N/A
4. ASSESSMENT & DATA CAPACITY & USE. How well did NPR support 
the MOE’ s ability to use Early Grade Reading Assessment (EGRA) data 
and classroom formative assessments to support adaptive management and 
improved learning outcomes at both the classroom and central level? How 
ll h i b ildi d d i h h MOE i i d *Starting in 2019, 
NPR organized a 
series of capacity 
building￾workshops for 
MoE personnel on 
evaluation. The 
general topics 
included 
assessment design; 
item development; 
developing a 
research agenda; 
benchmarking; 
data analysis, 
interpretation, and 
use; and test 
equating. If NPR 
was going to plan 
this series of 
capacity-building 
workshops again, 
should NPR do 
*Starting in 2019, 
NPR organized a 
series of capacity 
building￾workshops for 
MoE personnel on 
evaluation. The 
general topics 
included 
assessment design; 
item development; 
developing a 
research agenda; 
benchmarking; 
data analysis, 
interpretation, and 
use; and test 
equating. If NPR 
was going to plan 
this series of 
capacity-building 
workshops again, 
should NPR do 
*Starting in 2019, 
NPR organized a 
series of capacity 
building￾workshops for 
MoE personnel on 
evaluation. The 
general topics 
included 
assessment design; 
item development; 
developing a 
research agenda; 
benchmarking; 
data analysis, 
interpretation, and 
use; and test 
equating. Did you 
participate in any 
of these 
workshops? If yes, 
if NPR conducts 
similar capacity-
*Starting in 2019, 
NPR organized a 
series of capacity 
building￾workshops for 
MoE personnel on 
evaluation. The 
general topics 
included 
assessment design; 
item development; 
developing a 
research agenda; 
benchmarking; 
data analysis, 
interpretation, and 
use; and test 
equating. Did you 
participate in any 
of these 
workshops? If yes, 
if NPR conducts 
similar capacity￾Starting in 2019, 
NPR organized a series of capacity 
building￾workshops for 
MoE personnel on 
evaluation. The 
general topics 
included 
assessment design; 
item development; 
developing a 
research agenda; 
benchmarking; 
data analysis, 
interpretation, and 
use; and test 
equating. Did you 
participate in any 
of these 
workshops? If yes, 
if NPR conducts 
similar capacity￾N/A [NOTE: This 
question does not 
apply because we 
assume no MoE 
officials from non
-
intervention 
regions 
participated in any 
capacity-building 
activities.]
N/A [NOTE: This 
question does not 
apply because we 
assume no MoE 
officials from non
-
intervention 
regions 
participated in any 
capacity-building 
activities.]
N/A N/A N/A N/A N/A
152
1
2 3a 3b 3c 3d 3e 3f
4 5a 5b 6a 6b 7a 7b
8
anything 
differently? If so, 
what and why? 
[PROBE: Different 
topics, format, 
time, materials, 
etc.]
anything 
differently? If so, 
what and why? 
[PROBE: Different 
topics, format, 
time, materials, 
etc.]
building 
workshops in the 
future on 
benchmarking and 
measurement, 
should anything be 
done differently? If 
so, what and why? 
[PROBE: Different 
topics, format, 
time, materials, 
etc.]
building 
workshops in the 
future on 
benchmarking and 
measurement, 
should anything be 
done differently? If 
so, what and why? 
[PROBE: Different 
topics, format, 
time, materials, 
etc.]
building 
workshops in the 
future on 
benchmarking and 
measurement, 
should anything be 
done differently? If 
so, what and why? 
[PROBE: Different 
topics, format, 
time, materials, 
etc.]
*Have these 
activities improved 
the MoE’s 
institutional ability 
to conduct 
EGRAs and use 
the data for 
informing 
implementation? 
Why or why not?
*Have these 
activities improved 
the MoE’s 
institutional ability 
to conduct 
EGRAs and use 
the data for 
informing 
implementation? 
Why or why not?
*If yes 
(participated), have 
these activities 
improved your 
unit's ability to 
conduct EGRAs 
and use the data 
for informing 
implementation? 
Why or why not?
*If yes 
(participated), did 
these activities 
improve your 
unit’s ability to 
conduct EGRAs 
and use the data 
for informing 
implementation? 
Why or why not?
If yes (participated), 
did these activities 
improve your 
unit’s ability to 
conduct EGRAs 
and use the data 
for informing 
implementation? 
Why or why not?
N/A [NOTE: This 
question does not 
apply because we 
assume no MoE 
officials from non
-
intervention 
regions 
participated in any 
capacity-building 
activities.]
N/A [NOTE: This question does not 
apply because we 
assume no MoE 
officials from non
-
intervention 
regions 
participated in any 
capacity-building 
activities.]
*Are you aware of 
EGRA—the Early 
Grade Reading 
Assessment—a 
tool for measuring 
children's reading 
ability in the early 
grades? If so, has 
EGRA been used 
to assess children's 
reading ability in 
your school? If yes, 
do you know the 
results? Have you 
been able to use 
the results? How? 
N/A Are you aware of 
EGRA—the Early 
Grade Reading 
Assessment—a 
tool for measuring children's reading 
ability in the early 
grades? If so, has 
EGRA been used 
to assess children's 
reading ability in 
your classroom? If 
yes, do you know 
the results? If yes, 
what have you 
done with the 
results?
N/A N/A
*Has the MoE 
used—or does it 
have any plans to 
use—EGRA data 
to make any 
changes to the 
design of the 
Arabic language 
education reform 
(either curriculum, 
materials, teacher 
training, data use, 
etc.)? If so, how? 
*Has the MoE 
used—or have 
any plans to use—
any EGRA data to 
make any changes 
to the design of 
the Arabic 
language education 
reform (either 
curriculum, 
materials, teacher 
training, data use, 
etc.)? If so, how? 
*To your 
knowledge, has 
your unit used—
or have any plans 
to use—any 
EGRA data to 
make any changes 
to the design of 
the Arabic 
language education 
reform (either 
curriculum, 
materials, teacher 
training, data use, 
etc.)? If so, how? 
*To your 
knowledge, have 
you used—or 
have any plans to 
use—any EGRA 
data to make any 
changes to the 
design of the 
Arabic language 
education reform 
(either curriculum, 
materials, teacher 
training, data use, 
etc.)? If so, how? 
To your 
knowledge, have 
you used—or 
have any plans to 
use—any EGRA 
data to make any 
changes to the 
design of the 
Arabic language 
education reform 
(either curriculum, 
materials, teacher 
training, data use, 
etc.)? If so, how? 
N/A [NOTE: This 
question does not 
apply because we 
assume no MoE 
officials from non
-
intervention 
regions 
participated in any 
capacity-building 
activities.]
N/A [NOTE: This question does not 
apply because we 
assume no MoE 
officials from non
-
intervention 
regions 
participated in any 
capacity-building 
activities.]
N/A N/A N/A N/A N/A
*NPR created a 
formative 
assessment toolkit, 
including a 
formative 
assessment guide, 
student 
remediation guide, 
and a teacher’s 
guide to unit tests 
for grades 1 to 4. 
To what extent 
did MoE 
personnel find 
these materials 
valuable? To what 
extent did they 
use them? To 
what extent have 
the formative 
*NPR created a 
formative 
assessment toolkit, 
including a 
formative 
assessment guide, 
student 
remediation guide, 
and a teacher’s 
guide to unit tests 
for grades 1 to 4. 
To what extent 
did MoE 
personnel find 
these materials 
valuable? To what 
extent did they 
use them? To 
what extent have 
the formative 
*[NOTE: Show 
the FA toolkit to 
the respondent.] 
NPR created a 
formative 
assessment toolkit, 
including a 
formative 
assessment guide, 
student 
remediation guide, 
and a teacher’s 
guide to unit tests 
for grades 1 to 4. 
*[NOTE: Show 
the FA toolkit to 
the respondent.] 
NPR created a 
formative 
assessment toolkit, including a 
formative 
assessment guide, 
student 
remediation guide, 
and a teacher’s 
guide to unit tests 
for grades 1 to 4. 
[NOTE: Show the 
FA toolkit to the 
respondent.] NPR 
created a 
formative 
assessment toolkit, 
including a 
formative 
assessment guide, 
student 
remediation guide, 
and a teacher’s 
guide to unit tests 
for grades 1 to 4. 
[NOTE: Show the 
FA toolkit to the 
respondent.] As 
part of the new 
curriculum, the 
MoE created a 
formative 
assessment toolkit, 
including a 
formative 
assessment guide, 
student 
remediation guide, 
and a teacher’s 
guide to unit tests 
for grades 1 to 4.
[NOTE: Show the 
FA toolkit to the 
respondent.] As 
part of the new 
curriculum, the 
MoE created a 
formative 
assessment toolkit, 
including a 
formative 
assessment guide, 
student 
remediation guide, 
and a teacher’s 
guide to unit tests 
for grades 1 to 4.
N/A N/A N/A N/A N/A
153
1
2 3a 3b 3c 3d 3e 3f
4 5a 5b 6a 6b 7a 7b
8
assessment 
materials been 
scaled nationwide? 
If yes, to what 
extent?
assessment 
materials been 
scaled nationwide? 
*Has the MoE 
used—or does it 
have any plans to 
use—formative 
assessment data to 
make any changes 
to the design of 
the Arabic 
language education 
reform (either 
curriculum, 
materials, teacher 
training, data use, 
etc.)? If so, how? 
*Has the MoE 
used—or have any plans to use—
any formative 
assessment data to 
make any changes 
to the design of 
the Arabic 
language education 
reform (either 
curriculum, 
materials, teacher 
training, data use, 
etc.)? If so, how? 
*Have you 
received any FA 
data from schools 
nationwide? If so, 
have you used this 
data at all to make 
changes to the 
design of the 
Arabic language 
education reform 
(either curriculum, 
materials, teacher 
training, data use, 
etc.)? If so, how?
*Have you 
received any FA 
data from schools 
in your 
region/province? If 
so, have you used 
this data at all to 
make changes to 
the 
implementation of 
the Arabic 
language education 
reform (either 
curriculum, 
materials, teacher 
training, data use, 
etc.)? If so, how?
Have you received 
any FA data from 
schools in your 
region/province? If 
so, have you used 
this data at all to 
make changes to 
the 
implementation of 
the Arabic 
language education 
reform (either 
curriculum, 
materials, teacher 
training, data use, 
etc.)? If so, how?
Have you 
received these 
materials? If yes, 
have you received 
any FA data from 
schools in your 
region/province? If 
so, have you used 
this data at all to 
make changes to 
the 
implementation of 
the Arabic 
language education 
reform (either 
curriculum, 
materials, teacher 
training, data use, 
etc.)? If so, how?
Have you 
received these 
materials? If yes, 
have you received 
any FA data from 
schools in your 
region/province? If 
so, have you used 
this data at all to 
make changes to 
the 
implementation of 
the Arabic 
language education 
reform (either 
curriculum, 
materials, teacher 
training, data use, 
etc.)? If so, how?
NPR created a 
formative 
assessment toolkit, 
including a 
formative 
assessment guide, 
student 
remediation guide, 
and a teacher’s 
guide to unit tests 
for grades 1 to 4. 
Did your school 
receive these 
materials?If so, do 
you think they 
helped your 
learners learn to 
read better? How?
The MoE created 
a formative 
assessment toolkit, 
including a 
formative 
assessment guide, 
student 
remediation guide, 
and a teacher’s 
guide to unit tests 
for grades 1 to 4. 
Did your school 
receive these 
materials? If so, do 
you think they 
helped your 
learners learn to 
read better? How?
*NPR created a 
formative 
assessment toolkit, 
including a 
formative 
assessment guide, 
student 
remediation guide, 
and a teacher’s 
guide to unit tests 
for grades 1 to 4. 
Have you received 
any of these 
materials? If so, 
how often do you 
use them? Do you 
think they helped 
your learners learn 
to read better? 
How?
*The MoE created 
a formative 
assessment toolkit, 
including a 
formative 
assessment guide, 
student 
remediation guide, 
and a teacher’s 
guide to unit tests 
for grades 1 to 4. 
Have you received 
these materials? If 
so, how often do 
you use them? If 
so, do you think 
they helped your 
learners learn to 
read better? How?
N/A
*Does the MoE 
need any support 
to use data to 
support adaptive 
management and 
improve learning 
outcomes in the 
future? If so, what 
kind of support?
*Does the MoE 
need any support 
to use data to 
support adaptive 
management and 
improve learning 
outcomes in the 
future? If so, what 
kind of support?
*Does your unit 
need any support 
in the future to 
use data to make 
any changes to the 
design and/or 
management 
[NOTE TO 
REVEIWER: 
"design and 
management" is 
how we're 
referring to 
adaptative 
management] of 
the Arabic 
language reform 
(either curriculum, 
materials, teacher 
training, data use, 
etc.)? If yes, what 
kind of support?
*Does your unit 
need any support 
in the future to 
use data to make 
any changes to the 
design and/or 
management of 
the Arabic 
language reform 
(either curriculum, 
materials, teacher 
training, data use, 
etc.)? If yes, what 
kind of support?
Does your unit 
need any support 
in the future to 
use data to make 
any changes to the 
design and/or 
management of 
the Arabic 
language reform 
(either curriculum, 
materials, teacher 
training, data use, 
etc.)? If yes, what 
kind of support?
N/A [NOTE: This 
question does not 
apply because we 
assume no MoE 
officials from non
-
intervention 
regions 
participated in any 
capacity-building 
activities.]
N/A [NOTE: This 
question does not 
apply because we 
assume no MoE 
officials from non
-
intervention 
regions 
participated in any 
capacity-building 
activities.]
N/A N/A N/A N/A N/A
5. SCALING. How effectively did the MOE use and 
scale up the new curricula introduced by NPR? What 
are the issues around managing the curricula effectively 
at a national scale?
*Now I’d like to 
ask about the 
launch of the new 
curriculum 
nationally. Before 
the start of the 
2021
-22 school 
year, the MoE 
announced that 
schools were 
required to use 
the new Arabic 
language 
curriculum 
introduced by 
NPR, and 8,500 
copies of the 
*Now I’d like to 
ask about the 
launch of the new 
curriculum 
nationally. Before 
the start of the 
2021
-22 school 
year, the MoE 
announced that 
schools were 
required to use 
the new Arabic 
language 
curriculum 
introduced by 
NPR, and 8,500 
copies of the 
*Now I’d like to 
ask aboutthe 
launch of the new 
curriculum 
nationally. Before 
the start of the 
2021
-22 school 
year, the MoE 
announced that 
schools were 
required to use 
the new Arabic 
language 
curriculum 
introduced by 
NPR, and 8,500 
copies of the 
*Now I’d like to 
ask about the 
launch of the new 
curriculum 
nationally (beyond 
the experimental 
schools). Before 
the start of the 
2021
-22 school 
year, the MoE 
announced that 
schools were 
required to use 
the new Arabic 
language 
curriculum 
introduced by 
*Now I’d like to 
ask about the 
launch of the new 
curriculum 
nationally (beyond 
the experimental 
schools). Before 
the start of the 
2021
-22 school 
year, the MoE 
announced that 
schools were 
required to use 
the new Arabic 
language 
curriculum 
introduced by 
*Now I’d like to 
ask about the 
launch of the new 
curriculum 
nationally. Before 
the start of the 
2021
-22 school 
year, the MoE 
announced that 
schools were 
required to use 
the new Arabic 
language 
curriculum 
introduced by 
NPR, and 8,500 
copies of the 
*Now I’d like to 
ask about the 
launch of the new 
curriculum 
nationally. Before 
the start of the 
2021
-22 school 
year, the MoE 
announced that 
schools were 
required to use 
the new Arabic 
language 
curriculum 
introduced by 
NPR, and 8,500 
copies of the 
N/A N/A N/A N/A N/A
154
1
2 3a 3b 3c 3d 3e 3f
4 5a 5b 6a 6b 7a 7b
8
curriculumwere 
printed and 
distributed to 
schools 
nationwide. What 
did the MoE do to 
support the launch 
of the new 
curriculum and 
materials 
nationwide at the 
start of the 2021
-
22 school year? 
Did it provide 
enough support to 
regional and 
provincial 
education offices in 
order for schools 
nationwide to 
implement the 
curriculum and 
use the new 
materials as 
designed? 
Why/why not?
curriculum were 
printed and 
distributed to 
schools 
nationwide. What 
did the MoE do to 
support the launch 
of the new 
curriculum and 
materials 
nationwideat the 
start of the 2021
-
22 school year? 
Did it provide 
enough support to 
regional and 
provincial 
education offices in 
order for schools 
nationwide to 
implement the 
curriculum and 
use the new 
materials as 
designed? 
Why/why not? 
curriculum were 
printed and 
distributed to 
schools 
nationwide. What 
did your unit do to 
support the launch 
of the new 
curriculum and 
materials 
nationwide at the 
start of the 2021
-
22 school year? 
Did you provide 
enough support to 
regional and 
provincial 
education offices in 
order for schools
nationwide to 
implement the 
curriculum and 
use the new 
materials as 
designed? 
Why/why not?
NPR, and 8,500 
copies of the 
curriculum were 
printed and 
distributed to 
schools 
nationwide. To 
your knowledge, 
did the MoE do 
anything to 
support the launch 
of the new 
curriculum and 
materials 
nationwide at the 
start of the 2021
-
22 school year? 
Did it provide 
enough support to 
your 
region/province in 
order for schools 
in your 
region/province to 
implement the 
curriculum and 
use the new 
materials as 
designed? 
Why/why not?
NPR, and 8,500 
copies of the 
curriculum were 
printed and 
distributed to 
schools 
nationwide. To 
your knowledge, 
did the MoE do 
anything to 
support the launch 
of the new 
curriculum and 
materials 
nationwide at the 
start of the 2021
-
22 school year? 
Did it provide 
enough support to 
your 
region/province in 
order for schools 
in your 
region/province to 
implement the 
curriculum and 
use the new 
materials as 
designed? 
Why/why not?
curriculum were 
printed and 
distributed to 
schools 
nationwide. To 
your knowledge, 
did the MoE do 
anything to 
support the launch 
of the new 
curriculum and 
materials 
nationwide at the 
start of the 2021
-
22 school year? 
Did it provide 
enough support to 
your unit in order 
for schools in your 
region/province to 
implement the 
curriculum and 
use the new 
materials as 
designed? 
Why/why not?
curriculum were 
printed and 
distributed to 
schools 
nationwide. To 
your knowledge, 
did the MoE do 
anything to 
support the launch 
of the new 
curriculum and 
materials 
nationwide at the 
start of the 2021
-
22 school year? 
Did it provide 
enough support to 
your unit in order 
for schools in your 
region/province to 
implement the 
curriculum and 
use the new 
materials as 
designed? 
Why/why not?
*In February 2022, 
the MoE urgently 
requested that 
NPR develop a 
toolkit to help 
inspectors provide 
in
-service training 
to teachers about 
the new Arabic 
language 
curriculum. What 
prompted this 
request? Do you 
know if inspectors 
have been using 
this toolkit since 
NPR submitted it 
to the MoE? If yes, 
has the inspectors’ 
training led to 
schools 
nationwide 
implementing the 
new curriculum 
and using the new 
materials as 
designed? 
Why/why not? 
*In February 2022, 
the MoE urgently 
requested that 
NPR develop a 
toolkit to help 
inspectors provide 
in
-service training 
to teachers about 
the new Arabic 
language 
curriculum. What 
prompted this 
request? Do you 
know if inspectors 
have been using 
this toolkit since 
NPR submitted it 
to the MoE? If yes, 
has the inspectors’ 
training led to 
schools 
nationwide 
implementing the 
new curriculum 
and using the new 
materials as 
designed? 
Why/why not? 
*[ASK ONLY 
OF CURRICULA 
DIRECTORATE] 
In February 2022, 
the MoE Curricula 
Directorate 
requested that 
NPR develop a 
toolkit to help 
inspectorsprovide 
in
-service training 
to teachers about 
the new Arabic 
language 
curriculum. What 
prompted this 
request? Do you 
know if inspectors 
were trained and 
supported on 
how to use the 
toolkit? Have they 
been using this 
toolkit? If yes, has 
the inspectors’ 
training led to 
schools 
nationwide 
implementing the 
new curriculum 
and using the new 
materials as 
designed? 
Why/why not? 
Has anyone in 
your office 
received training 
or support to use 
the new Arabic 
language 
curriculum, 
including a training 
toolkit to help
inspectors provide 
in
-service training 
to teachers? If yes, 
what kind of 
training or support 
was received, and 
has it been 
received 
throughout your 
region/province? If 
yes, have those 
who have been 
trained been able 
to implement 
what they were 
trained on?Are 
the issues with this 
work? If yes, what 
are they?
Has anyone in 
your office 
received training 
or support to use 
the new Arabic 
language 
curriculum, 
including a training 
toolkit to help 
inspectors provide 
in
-service training 
to teachers? If yes, 
what kind of 
training or support 
was received, and 
has it been 
received 
throughout your 
region/province? If 
yes, have those 
who have been 
trained been able 
to implement 
what they were 
trained on? Are 
the issues with this 
work? If yes, what 
are they?
*Has anyone in 
your office 
received training 
or support to use 
the new Arabic 
language 
curriculum, 
including a training 
toolkit to help 
inspectors provide 
in
-service training 
to teachers? If yes, 
what kind of 
training or support 
was received, and 
has it been 
received 
throughout your 
region/province? If 
yes, have those 
who have been 
trained been able 
to implement 
what they were 
trained on? What 
are the issues with 
this work?
*Has anyone in 
your office 
received training 
or support to use 
the new Arabic 
language 
curriculum, 
including a training 
toolkit to help 
inspectors provide 
in
-service training 
to teachers? If yes, 
what kind of 
training or support 
was received, and 
has it been 
received 
throughout your 
region/province? If 
yes, have those 
who have been 
trained been able 
to implement 
what they were 
trained on? What 
are the issues with 
this work?
N/A N/A N/A N/A N/A
155
1 2 3a 3b 3c 3d 3e 3f 4 5a 5b 6a 6b 7a 7b 8
To your 
knowledge, are 
teachers 
nationwide 
implementing the 
new curriculum 
and materials? If so, 
what proportion 
of teachers are 
implementing it—
all, most, some, or 
few? [Ask if 
previous response 
is most, some, or 
few] Why aren't 
all schools 
implementing it? 
What additional 
support could the 
MoE provide to 
ensure all schools 
are able to 
implement the 
new Arabic 
language 
curriculum?
To your 
knowledge, are 
teachers 
nationwide 
implementing the 
new curriculum 
and materials? If so, 
what proportion 
of teachers are 
implementing it—
all, most, some, or 
few? [Ask if 
previous response 
is most, some, or 
few] Why aren't 
all schools 
implementing it? 
What additional 
support could the 
MoE provide to 
ensure all schools 
are able to 
implement the 
new Arabic 
language 
curriculum?
To your 
knowledge, are 
teachers 
nationwide 
implementing or 
not the new 
curriculum and 
materials? If so, 
what proportion 
of teachers are 
implementing it—
all, most, some, or 
few? [Ask if 
previous response 
is most, some, or 
few] Why aren't 
all schools 
implementing it? 
What additional 
support could the 
MoE provide to 
ensure all schools 
are able to 
implement the 
new Arabic 
language 
curriculum?
Are teachers 
implementing the 
new curriculum 
and materials in 
your 
region/province/C
MREF? If so, what 
proportion of 
teachers are 
implementing it—
all, most, some, or 
few? [Ask if 
previous response 
is most, some, or 
few] Why aren’t 
all schools 
implementing it? 
What additional 
support could the 
MoE provide to 
ensure all schools 
are able to 
implement the 
new Arabic 
language 
curriculum?
*Are teachers 
implementing the 
new curriculum 
and materials in 
your 
region/province/C
MREF? If so, what 
proportion of
teachers are 
implementing it—
all, most, some, or 
few? [Ask if 
previous response 
is most, some, or 
few] Why aren’t 
all schools 
implementing it? 
What additional 
support could the 
MoE provide to 
ensure all schools 
are able to 
implement the 
new Arabic 
language 
curriculum?
*Are teachers 
implementing the 
new curriculum 
and materials in 
your 
region/province/C
MREF? If so, what 
proportion of 
teachers are 
implementing it—
all, most, some, or 
few? [Ask if 
previous response 
is most, some, or 
few] Why aren't 
all schools
implementing it? 
What additional 
support could the 
MoE provide to 
ensure all schools 
are able to 
implement the 
new Arabic 
language 
curriculum?
*Are teachers 
implementing the 
new curriculum 
and materials in 
your 
region/province/C
MREF? If so, what 
proportion of 
teachers are 
implementing it—
all, most, some, or 
few? [Ask if 
previous response 
is most, some, or 
few] Why aren't 
all schools 
implementing it? 
What additional 
support could the 
MoE provide to 
ensure all schools 
are able to 
implement the 
new Arabic 
language 
curriculum?
Covered above Covered above N/A N/A N/A
What did the 
MoE do well with 
scaling the new 
curriculum 
nationally? What 
did it not do well? 
What was the 
biggest obstacle 
the MoE had to 
overcome in 
scaling the 
curriculum, and 
how did it address 
it?
What did the 
MoE do well with 
scaling the new 
curriculum 
nationally? What 
did it not do well? 
What was the 
biggest obstacle 
the MoE had to 
overcome in 
scaling the 
curriculum, and 
how did it address 
it?
What did your 
unit do well with 
scaling the new 
curriculum 
nationally? What 
did it not do well? 
What was the 
biggest obstacle 
your unit had to 
overcome in 
scaling the 
curriculum, and 
how did it address 
it?
*What did the 
MoE do well with 
scaling the new 
curriculum 
nationally? What 
did it not do well? 
What was the 
biggest obstacle 
the MoE had to 
overcome in 
scaling the 
curriculum, and 
how did it address 
it?
What did the 
MoE do well with 
scaling the new 
curriculum 
nationally? What 
did it not do well? 
What was the 
biggest obstacle 
the MoE had to 
overcome in 
scaling the 
curriculum, and 
how did it address 
it?
N/A N/A What additional 
support could the 
ministry provide 
to better help 
your teachers 
implement the 
new Arabic 
language 
curriculum?
N/A *What additional 
support could the 
ministry provide 
to better help you 
implement the 
new Arabic 
language 
curriculum?
*What additional 
support could the 
ministry provide 
to better help you 
implement the 
new Arabic 
language 
curriculum?
N/A
6. LESSONS: ENGAGING COMMUNITIES & 
PARENTS. What are best practices and lessons 
learned from NPR for engaging communities and 
parents in reading programs? Now I’d like to ask 
about NPR’s 
activities for 
engaging 
community and 
parental support 
for reading. 
Now I’d like to ask 
about NPR’s 
activities for 
engaging 
community and 
parental support 
for reading. 
Now I’d like to ask 
about NPR’s 
activities for 
engaging 
community and 
parental support 
for reading. 
*The 
development and 
implementation of 
the new program 
lasted from 2017 
to 2022. During 
this time, have you 
noticed any 
changes in the 
perceptions of 
different 
populations 
(Ministry staff, 
teachers, parents) 
about the 
program?
Now I’d like to ask 
about NPR’s 
activities for 
engaging 
community and 
parental support 
for reading. 
*Now I’d like to 
ask about NPR’s 
activities for 
engaging 
community and 
parental support 
for reading. 
Now I’d like to ask 
about activities for 
engaging 
community and 
parental support 
for reading. 
*Now I’d like to 
ask about activities 
for engaging 
community and 
parental support 
for reading. 
*Now I’d like to 
ask about NPR’s 
activities for 
engaging 
community and 
parental support 
for reading. 
*Now I’d like to 
ask about activities 
for engaging 
community and 
parental support 
for reading. 
Now I’d like to ask 
about NPR’s 
activities for 
engaging 
community and 
parental support 
for reading. 
Now I’d like to ask 
about activities for 
engaging 
community and 
parental support 
for reading. 
*Now I’d like to 
ask about activities 
for engaging 
community and 
parental support 
for reading. 
What kind of 
outreach activities 
did NPR train 
teachers and 
school directors 
to organize to help 
parents with the 
What kind of 
outreach activities 
did NPR train 
teachers and 
school directors 
to organize to help 
parents with the 
What kind of 
outreach activities 
did NPR train 
teachers and 
school directors 
to organize to help 
parents with the 
*During this same 
time period, were 
there changing 
communication 
strategies on the 
part of the 
Department on 
Have you been 
involved with 
training teachers 
and school 
directors on 
outreach activities 
to organize with 
*Have you been 
involved with 
training teachers 
and school 
directors on 
outreach activities 
to organize with 
Have you been 
involved with 
training teachers 
and school 
directors on 
outreach activities 
to organize with 
*Have you been 
involved with 
training teachers 
and school 
directors on 
outreach activities 
to organize with 
*Have you 
organized any 
outreach activities 
to help parents 
with the use of 
supplementary 
reading materials 
*Have you 
organized any 
outreach activities 
to help parents 
with the use of 
supplementary 
reading materials 
Have you 
organized any 
outreach activities 
to help parents 
with the use of 
supplementary 
reading materials 
Have you 
organized any 
outreach activities 
to help parents 
with the use of 
supplementary 
reading materials 
*Have teachers or 
school directors 
organized any 
outreach activities 
to help you with 
the use of 
supplementary 
156
1 2 3a 3b 3c 3d 3e 3f 4 5a 5b 6a 6b 7a 7b 8
use of 
supplementary 
reading materials 
and the 3asafeer 
digital library at 
home? To your 
knowledge, how 
many teachers and 
school directors 
organized these 
activities—all, 
most, some, or 
few? Did parents 
follow the 
guidance? Did 
students read the 
materials? 
use of 
supplementary 
reading materials 
and the 3asafeer 
digital library at 
home? To your 
knowledge, how 
many teachers and 
school directors 
organized these 
activities—all, 
most, some, or 
few? Did parents 
follow the 
guidance? Did 
students read the 
materials?
use of 
supplementary 
reading materials 
and the 3asafeer 
digital library at 
home? To your 
knowledge, how 
many teachers and 
school directors 
organized these 
activities—all, 
most, some, or 
few? Did parents 
follow the 
guidance? Did 
students read the 
materials? 
reform with these 
populations? 
a. If so, what 
changes in strategy 
have been made? 
b. Were these 
different strategies 
effective? Please 
explain.
parents to help 
them with the use 
of supplementary 
readingmaterials 
and the 3asafeer 
digital library at 
home? If so, what 
are the outreach 
activities? To your 
knowledge, how 
many teachers and 
school directors 
organized these 
activities—all, 
most, some, or 
few? Did parents 
follow the 
guidance? Did 
students read the 
materials?
parents to help 
them with the use 
of supplementary 
reading materials 
and the 3asafeer 
digital library at 
home? If so, what 
are the outreach 
activities? To your 
knowledge, how 
many teachers and 
school directors 
organized these 
activities—all, 
most, some, or 
few? Did parents 
follow the 
guidance? Did 
students read the 
materials?
parents to help 
them with the use 
of supplementary 
reading materials 
at home? If so, 
what are the 
outreach activities? 
To your 
knowledge, how 
many teachers and 
school directors 
organized these 
activities—all, 
most, some, or 
few? Did parents 
follow the 
guidance? Did 
students read the 
materials?
parents to help 
them with the use 
of supplementary 
reading materials 
at home? If so, 
what are the 
outreach activities? 
To your 
knowledge, how 
many teachers and 
school directors 
organized these 
activities—all, 
most, some, or 
few? Did parents 
follow the 
guidance? Did 
students read the 
materials?
and the 3asafeer 
digital library at 
home? If yes, what 
type of guidance 
have you provided 
to parents? Did 
parents follow the 
guidance? Did 
students read the 
materials?
at home? If yes, 
what type of 
guidance have you 
provided to 
parents? Did the 
parents follow the 
guidance? Did 
students read the 
materials? 
and the 3asafeer 
digital library at 
home? If yes, what 
type of guidance 
have you provided 
to parents? Did 
the parents follow 
the guidance? Did 
students read the 
materials? 
at home? If yes, 
what type of 
guidance have you 
provided to 
parents? Did the 
parents follow the 
guidance? Did 
students read the 
materials?
reading materials 
and the 3asafeer 
digital library at 
home? (NOTE: 
Only ask about 
the digital library in 
experimental 
schools.) If yes, 
what type of 
guidance have 
they provided to 
you? Did you 
follow their 
guidance? Did 
your children read 
the materials? 
What other 
activities did NPR 
implement to 
increase 
community and 
parental support 
for reading? Did 
anything change 
regarding 
community and 
parental support 
due to the 
activities? 
What other 
activities did NPR 
implement to 
increase 
community and 
parental support 
for reading? Did 
anything change 
regarding 
community and 
parental support 
due to the 
activities?
What other 
activities did NPR 
implement to 
increase 
community and 
parental support 
for reading? Did 
anything change 
regarding 
community and 
parental support 
due to the 
activities? 
As part of NPR, 
have there any 
other activities or 
programs in your 
province/region 
designed to 
engage parents or 
community 
members in their 
children’s reading? 
If so, what, and did 
anything change 
regarding 
community and 
parental support 
due to the 
activities? 
*As part of NPR, 
have there any 
other activities or 
programs in your 
province/region 
designed to 
engage parents or 
community 
members in their 
children’s reading? 
If so, what, and did 
anything change 
regarding 
community and 
parental support 
due to the 
activities? 
As part of the new 
curriculum, has 
the MoE 
conducted any 
other activities in 
your 
province/region 
designed to 
engage parents or 
community 
members in their 
children’s reading? 
If so, what, and did 
anything change 
regarding 
community and 
parental support 
due to the 
activities? 
*As part of the 
new curriculum, 
has the MoE 
conducted any 
other activities in 
your 
province/region 
designed to 
engage parents or 
community 
members in their 
children’s reading? 
If so, what, and did 
anything change 
regarding 
community and 
parental support 
due to the 
activities? 
*Have you been 
involved with any 
other activities as 
part of NPR to 
increase 
community and 
parental support 
for reading? If so, 
what, and did 
anything change 
regarding 
community and 
parental support 
due to the 
activities? 
*Have you been 
involved with any 
other activities as 
part of the 
introduction of the 
new curriculum to 
increase 
community and 
parental support 
for reading? If so, 
what, and did 
anything change 
regarding 
community and 
parental support 
due to the 
activities?
Have you been 
involved with any 
other activities as 
part of NPR to 
increase 
community and 
parental support 
for reading? If so, 
what, and did 
anything change 
regarding 
community and 
parental support 
due to the 
activities? 
Have you been 
involved with any 
other activities as 
part of the 
introduction of the 
new curriculum to 
increase 
community and 
parental support 
for reading? If so, 
what, and did 
anything change 
regarding 
community and 
parental support 
due to the 
activities?
*Has the MoE or 
your school 
conducted any 
other activities to 
engage you as 
parents or the 
community as a 
whole? If so, what, 
and are you and 
the community 
more engaged 
with your 
children’s 
education due to 
the activities?
*Now I’d like to 
ask about NPR’s 
engagement with 
the private sector 
in reading 
programs. 
*Now I’d like to 
ask about NPR’s 
engagement with 
the private sector 
in reading 
programs. 
*Now I’d like to 
ask about NPR’s 
engagement with 
the private sector 
in reading 
programs. 
Now I’d like to ask 
about NPR’s 
engagement with 
the private sector 
in reading 
programs. 
Now I’d like to ask 
about NPR’s 
engagement with 
the private sector 
in reading 
programs. 
Now I’d like to ask 
about the MoE’s 
engagement with 
the private sector 
in reading 
programs. 
Now I’d like to ask 
about the MoE’s 
engagement with 
the private sector 
in reading 
programs. 
N/A N/A N/A N/A N/A
7. LESSONS: PRIVATE SECTOR ENGAGEMENT. What 
are lessons learned from NPR related to private sector 
engagement in reading programs?
*NPR and the 
MoE worked with 
the private 
textbook industry 
during the 
development of 
the new 
curriculum. What 
were the 
textbook 
industry’s 
strengths and 
weaknesses in this 
partnership? 
Were there any 
obstacles NPR 
had to overcome? 
If so, what did 
*NPR and the 
MoE worked with 
the private 
textbook industry 
during the 
development of 
the new 
curriculum. What 
were the 
textbook 
industry’s 
strengths and 
weaknesses in this 
partnership? 
Were there any 
obstacles NPR 
had to overcome? 
Ifso, what did 
*NPR and the 
MoE worked with 
the private 
textbook industry 
during the 
development of 
the new 
curriculum. What 
were the 
textbook 
industry’s 
strengths and 
weaknesses in this 
partnership? 
Were there any 
obstacles NPR 
had to overcome? 
If so, what did 
NPR and the MoE 
worked with the 
private textbook 
industry during the 
development of 
the new 
curriculum. Are 
you aware of 
readers produced 
by these 
companies? If so, 
what did they do 
ornot do well? 
What would you 
like to see done 
differently?
NPR and the MoE 
worked with the 
private textbook 
industry during the 
development of 
the new 
curriculum. Are 
you aware of 
readers produced 
by these 
companies? If so, 
what did they do 
or not do well?
What would you 
like to see done 
differently?
NPR and the MoE 
worked with the 
private textbook 
industry during the 
development of 
the new 
curriculum. Are 
you aware of 
readers produced 
by these 
companies? If so, 
what were the 
strengths and 
weaknesses of the
textbooks? What 
would you like to 
see done 
differently in the 
textbooks?
NPR and the MoE 
worked with the 
private textbook 
industry during the 
development of 
the new 
curriculum. Are 
you aware of 
readers produced 
by these 
companies? If so, 
what were the 
strengths and 
weaknesses of the 
textbooks? What 
would you like to 
see done 
differently in the 
textbooks?
N/A N/A N/A N/A N/A
157
1 2 3a 3b 3c 3d 3e 3f 4 5a 5b 6a 6b 7a 7b 8
NPR try todo to 
overcome them? 
NPR try to do to 
overcome them?
NPR try to do to 
overcome them? 
*NPR and the 
MoE worked with 
the company 
3asafeer during the 
development of 
the supplemental 
reading materials 
program. What 
were 3asafeer’s 
strengths and 
weaknesses in this 
partnership? 
Were there any 
obstacles NPR 
had to overcome? 
If so, what did 
NPR try to do to 
overcome them? 
*NPR and the 
MoE worked with 
the company 
3asafeer during the 
development of 
the supplemental 
reading materials 
program. What 
were 3asafeer’s 
strengths and 
weaknesses in this 
partnership? 
Were there any 
obstacles NPR 
had to overcome? 
If so, what did 
NPR try to do to 
overcome them? 
*NPR and the 
MoE worked with 
the company 
3asafeer during 
the development 
of the 
supplemental 
reading materials 
program. What 
were 3asafeer’s 
strengths and 
weaknesses in this 
partnership? 
Were there any 
obstacles NPR 
had to overcome? 
If so, what did 
NPR try to do to 
overcome them? 
NPR and the MoE 
worked with 
3asafeer during 
the development 
of the new 
supplemental 
reading materials. 
Are you aware of 
supplemental 
reading materials 
produced by 
3asafeer? If so, 
what did they do 
or not do well? 
What would you 
like to see done 
differently?
NPR and the MoE 
worked with 
3asafeer during 
the development 
of the new 
supplemental 
reading materials. 
Are you aware of 
supplemental 
reading materials 
produced by 
3asafeer? If so, 
what did they do 
or not do well? 
What would you
like to see done 
differently?
NPR and the MoE 
worked with 
3asafeer during 
the development 
of the new 
supplemental 
reading materials. 
Are you aware of 
supplemental 
reading materials 
produced by 
3asafeer? If so, 
what were the 
strengths and 
weaknesses of the
supplemental 
reading materials? 
What would you 
like to see done 
differently in the 
reading materials?
NPR and the MoE 
worked with 
3asafeer during 
the development 
of the new 
supplemental 
reading materials. 
Are you aware of 
supplemental 
reading materials 
produced by 
3asafeer? If so, 
what were the 
strengths and 
weaknesses of the 
supplemental 
reading materials? 
What would you 
like to see done 
differently in the 
reading materials?
N/A N/A N/A N/A N/A
8. LESSONS: EQUITABLE RESULTS . What are lessons learned 
from NPR related to education access, quality, or results among 
urban vs rural, boy vs. girl students, and/or geographic regions? How 
will these lessons learned inform future education programming and 
i d ii ? *Have you seen 
any notable 
differences in how 
the new 
curriculum has 
been received by 
children from 
different 
backgrounds – e.g., 
from certain 
regions/provinces, 
Arabic vs Amazigh, 
boys vs girls, 
children with 
disabilities, urban 
vs. rural? If yes, 
please describe 
these differences. 
Have there been 
any steps taken to 
address them?
*Have you seen 
any notable 
differences in how 
the new 
curriculum has 
been received by 
children from 
different 
backgrounds – e.g., 
from certain 
regions/provinces, 
Arabic vs Amazigh, 
boys vs girls, 
children with 
disabilities, urban 
vs. rural? If yes, 
please describe 
these differences. 
Have there been 
any steps taken to 
address them?
*Have you seen 
any notable 
differences in how 
the new 
curriculum has 
been received by 
children from 
different 
backgrounds – e.g., 
from certain 
regions/provinces, 
Arabic vs Amazigh, 
boys vs girls, 
children with 
disabilities, urban 
vs. rural? If yes, 
please describe 
these differences. 
Have there been 
any steps taken to 
address them?
*Have you seen 
incidents where 
the new 
curriculum has 
had a different 
impact?
a. For girls versus 
boys?
b. For students 
from advantaged 
groups compared 
to their peers (e.g., 
special needs 
students, minority 
language groups, 
disadvantaged 
regions, etc.)?
*Have you seen 
any notable 
differences in how 
the new 
curriculum has 
been received by 
children from 
different 
backgrounds – e.g., 
from certain 
regions/provinces, 
Arabic vs Amazigh, 
boys vs girls, 
children with 
disabilities, urban 
vs. rural? If yes, 
please describe 
these differences. 
Have there been 
any steps taken to 
address them?
*Have you seen 
any notable 
differences in how 
the new 
curriculum has 
been received by 
children from 
different 
backgrounds – e.g., 
from certain 
regions/provinces, 
Arabic vs Amazigh, 
boys vs girls, 
children with 
disabilities, urban 
vs. rural? If yes, 
please describe 
these differences. 
Have there been 
any steps taken to 
address them?
*Have you seen 
any notable 
differences in how 
the new 
curriculum has 
been received by 
children from 
different 
backgrounds – e.g., 
from certain 
regions/provinces, 
Arabic vs Amazigh, 
boys vs girls, 
children with 
disabilities, urban 
vs. rural? If yes, 
please describe 
these differences. 
Have there been 
any steps taken to 
address them?
*Have you seen 
any notable 
differences in how 
the new 
curriculum has 
been received by 
children from 
different 
backgrounds – e.g., 
from certain 
regions/provinces, 
Arabic vs Amazigh, 
boys vs girls, 
children with 
disabilities, urban 
vs. rural? If yes, 
please describe 
these differences. 
Have there been 
any steps taken to
address them?
*Have you seen 
any notable 
differences in how 
the new 
curriculum has 
been received by 
children from 
different 
backgrounds – e.g., 
from certain 
regions/provinces, 
Arabic vs Amazigh, 
boys vs girls, 
children with 
disabilities, urban 
vs. rural? If yes, 
please describe 
these differences. 
Have there been 
any steps taken to 
address them?
*Have you seen 
any notable 
differences in how 
the new 
curriculum has 
been received by 
children from 
different 
backgrounds – e.g., 
from certain 
regions/provinces, 
Arabic vs Amazigh, 
boys vs girls, 
children with 
disabilities, urban 
vs. rural? If yes, 
please describe 
these differences. 
Have there been 
any steps taken to 
address them?
*Have you seen 
any notable 
differences in how 
the new 
curriculum has 
been received by 
children from 
different 
backgrounds – e.g., 
from certain 
regions/provinces, 
Arabic vs Amazigh, 
boys vs girls, 
children with 
disabilities, urban 
vs. rural? If yes, 
please describe 
these differences. 
Have there been 
any steps taken to 
address them?
*Have you seen 
any notable 
differences in how 
the new 
curriculum has 
been received by 
children from 
different 
backgrounds – e.g., 
from certain 
regions/provinces, 
Arabic vs Amazigh, 
boys vs girls, 
children with 
disabilities, urban 
vs. rural? If yes, 
please describe 
these differences. 
Have there been 
any steps taken to 
address them?
*Have you seen 
any notable 
differences in how 
the new 
curriculum has 
been received by 
children from 
different 
backgrounds –
e.g., from certain 
regions/provinces, 
Arabic vs 
Amazigh, boys vs 
girls, children with 
disabilities, urban 
vs. rural? If yes, 
please describe 
these differences. 
Have there been 
any steps taken to 
address them?
*Had you had any 
involvement with 
NPR prior to this 
MSL activity? Had 
NPR or the MoE 
expressed any 
interest in working 
with your 
group/organization 
prior to the 
pandemic?
*How long have 
you worked in the 
textbook industry? 
What has your 
role(s) been? 
What work have 
you specifically 
done with literacy 
and reading 
materials?
*How many years 
have you worked 
in the field of 
educational 
technology? 
When do you 
create 3asafeer, 
and what kind of 
partnerships have 
you developed 
since the 
company’s 
founding?
7. LESSONS: 
PRIVATE SECTOR 
ENGAGEMENT. 
What are lessons 
l d f NPR 
*Can you describe 
your role with the 
production of the 
MSL video lessons 
for NPR during 
the COVID-19 
pandemic?
*How long did 
you work with the 
NPR project? 
What has your 
role(s) been? 
What materials 
have you 
*When did NPR 
first contact you 
about 
collaborating with 
the project? Prior 
to your 
involvement with 
158
1
2 3a 3b 3c 3d 3e 3f
4 5a 5b 6a 6b 7a 7b
8
developed as part 
of NPR, and for 
what grades?
NPR, had you
ever worked on a 
project funded by 
USAID? If no, 
what made your 
collaboration with 
USAID, NPR, and 
the Moroccan 
MoE unique to 
other partnerships 
you have had?
*Did the activity 
help improve the 
reading and MSL 
skills of children 
who are deaf or 
hard of hearing? If 
yes, in what ways?
*Can you describe 
the start of the 
materials creation 
process from 
when you started 
on drafting the 
materials to 
delivery of the first 
draft of these 
materials?
*I want to ask you 
about the initial 
drafting of your 
MOU with the 
MoE. Did those 
discussions about 
the MOU start in 
early 2020? What 
were the key 
responsibilities 
outlined for 
3asafeer and the 
MoE in the MOU? 
Did you 
encounter any 
challenges in 
drafting the MOU? 
If so, what were 
they, and how did 
you address them? 
*What were the 
activity’s most 
important and 
valuable 
contributions to 
improving 
children’s reading 
and MSL skills? 
Why do you 
consider these the 
most important or 
valuable 
contributions?
*Did you attend 
any trainings 
conducted by 
NPR on the new 
curriculum? If so, 
did the training 
adequately 
prepare you to 
deliver the first 
draft? Why/why 
not?
*In the spring of 
2021, the MOE 
extended its 
MOU with your 
company from 
May 2021 to
October 2022. 
Besides the 
expansion of the 
number of titles 
on the platform, 
were there any 
additions made to 
the MOU? If so, 
what were they?
*Within the 
different elements 
of NPR, what has 
met your 
expectations? 
Why? What has 
not met your 
expectations? 
Why, and how 
would you suggest 
these 
shortcomings be 
addressed?
*If you attended 
trainings 
conducted by 
NPR, did the 
trainings cover any 
materials on best 
practices in 
textbook creation 
and production? If 
so, did you learn 
anything new that 
you incorporated 
into the creation 
of the new 
materials? If so, 
what were they?
*You participated 
in some virtual 
meetings in June 
2021 about the 
platform and new 
titles with 
Moroccan 
inspectors and 
teachers. What 
was your role 
during these 
meetings? Do you 
think the meetings 
fulfilled their 
objectives? 
Why/why not?
*What kinds of 
internal, NPR-
*Can you describe 
the process of 
*Did you 
participate in any 
159
1
2 3a 3b 3c 3d 3e 3f
4 5a 5b 6a 6b 7a 7b
8
related concerns 
or obstacles, if any, 
have impeded 
implementation of 
these activities? 
Besides COVID, 
have any other 
external, 
contextual 
challenges 
impeded NPR? If 
so, how?
receiving feedback 
on your initial draft 
and then revising 
the materials? 
How many 
rounds of 
revisions did your 
materials go 
through? Who 
provided this 
feedback?
other 
trainings/meetings 
with inspectors 
and/or teachers 
during your 
involvement with 
the project? If so, 
what were they, 
and do you think 
the trainings 
fulfilled their 
objectives? 
Why/why not?
*According to 
NPR project 
reports in 2020 
and 2021, the 
MoE had technical 
difficulties 
transferring the 
video lessons to 
their online 
distance learning 
platform. To your 
knowledge, have 
the videos been 
transferred online 
since 2021? If the 
videos were 
transferred, have 
students been 
accessing and using 
the video content? 
Do you have 
knowledge of how 
many students 
have accessed the 
platform and how 
they are using it? If 
the videos were 
not transferred, 
do you know why 
the MoE is still 
having difficulties?
*NPR and the 
MoE solicited 
feedback from 
inspectors and 
teachers on 
textbooks and 
teachers’ guides 
and conducted 
meetings with 
publishers to share 
this feedback. Did 
you attend any of 
these meetings? If 
so, did these 
sessions provide 
adequate feedback 
to revise the 
materials? 
Why/why not?
*Did any 
inspectors/teacher
s raise challenges 
they had with the 
digital platform 
with you during 
these meetings? If 
so, what were 
they, and did you 
do anything to 
address their 
challenges? If so, 
what did you do? 
*To our 
knowledge, the 
MSL activity ended 
in October 2021 
when schools 
reopened for in
-
person learning for 
the start of the 
2021
-22 school 
year. Is that 
correct? Have you 
done any work 
with NPR or the 
MoE since the 
activity closed with 
children who are 
deaf or hard of 
hearing?
*If yes, had you 
ever received 
feedback directly 
from inspectors 
and teachers 
before as you did 
during NPR? If not, 
what did you think 
of the process? 
How did this 
process compare 
to the creation of 
other learning 
materials in the 
past?
*NPR conducted 
a study of its 
supplemental 
reading program 
during the 2020
-
21 school year. 
Did the project 
share its findings 
with you? If so, 
what did you think 
of the findings, 
considering your 
work with the 
platform 
elsewhere? Since 
the project 
expanded access 
3assafeer to 
additional schools 
for the 2021
-22 
school year, do 
160
1
2 3a 3b 3c 3d 3e 3f
4 5a 5b 6a 6b 7a 7b
8
you have any 
additional data 
about 3asafeer’s 
reach and impact?
*Has the MSL 
activity had any 
impact since it 
closed in October 
2021? Why/why 
not? If it hasn’t had 
any more impact, 
do you think the 
activity should 
continue to be 
utilized by the 
MoE, or should it 
focus on other 
interventions 
targeted at 
children who are 
deaf and hard of 
hearing? If yes, 
what do you think 
needs to happen 
for the activity to 
make an impact 
with children who 
are deaf or hard of 
hearing?
Were you 
involved at all in 
the printing and 
distribution of new 
materials? If so, did 
you encounter any 
unique challenges 
in the printing and 
distribution? If so, 
what were they, 
and what lessons 
did you learn from 
them?
*Did the MOU 
with the MOE 
end in October as 
originally planned? 
If yes, does the 
MOE plan to 
renew its MOU 
with you? 
Why/why not?
*What were the 
biggest successes 
of your work with 
NPR, including 
USAID and the 
MoE? What were 
the biggest 
challenges in 
working with the 
MoE and NPR, 
and how could 
project 
implementers and 
the MoE address 
them in future 
projects?
*Having worked 
with the new 
Arabic language 
curriculum, what 
do you view as its 
strengths in 
comparison to 
former iterations? 
Are there any gaps 
that still exist or 
areas that could be 
strengthened?
*What is the 
future of the 
3asafeer platform 
and its use in 
Morocco?
*Do you think 
innovations in the 
new curriculum 
have been 
consistently 
incorporated into 
all the materials 
(not just the ones 
that your 
company 
produced)? 
Why/why not?
*What were the 
biggest successes 
of your work with 
NPR, including 
USAID and the 
MoE? What were 
the biggest 
challenges in 
working with the 
MoE and NPR, 
and how could 
project 
implementers and 
the MoE address 
them in future 
projects? 
*Have you been 
involved at all with 
the rollout of the 
161
1
2 3a 3b 3c 3d 3e 3f
4 5a 5b 6a 6b 7a 7b
8
new curriculum in 
schools 
nationwide? If so, 
what was your 
role? What were 
the strong points 
of the rollout, and 
what were the 
challenges?
Have you worked 
on any new 
materials projects 
since working on 
NPR? If so, have 
you applied any of 
the knowledge or 
experience you 
gained with NPR 
to the new 
project? If so, what 
have you applied, 
and why? *What were the 
biggest successes 
of your work with 
NPR, including 
USAID and the 
MoE? What were 
the biggest 
challenges in 
working with the 
MoE and NPR, 
and how could 
project 
implementers and 
the MoE address 
them in future 
projects?
162
ANNEX IX: PROTOCOL FOR USAID ATTENDANCE AT PARENT FGDS
PROTOCOL FOR USAID ATTENDANCE
AT FGDS WITH PARENTS
NPR EVALUATION
SCHOOL-TO-SCHOOL INTERNATIONAL
Management Systems International (MSI) recently contracted School-to-School International (STS) to 
conduct a final evaluation of the National Program for Reading (NPR) funded by USAID/Morocco. NPR 
provided support to the Moroccan Ministry of National Education, Preschool, and Sports (MoE) from 
2017 to 2022 to develop, test, and initiate the scale-up of the Arabic language curriculum reform in 
Grades 1-6.
The evaluation design consists of Key Informant Interviews with USAID, implementers, education 
officials, decision makers, and administrators, as well as Focus Group Discussions (FGDs) with teachers 
and parents, both from regions that benefited directly from NPR (experimental regions) and regions 
where NPR has been scaled by the MoE (nonexperimental regions). The results of the evaluation will 
inform the Moroccan MoE’s continued improvement and scaling of the reform as it expands into junior 
secondary. Importantly, the evaluation will be an opportunity for the MoE and USAID to learn about 
stakeholders’ experiences with the reform. Lessons learned from the evaluation will inform future 
education support programming decisions made by USAID/Morocco. 
USAID has requested permission to attend some parent FGDs. In some circumstances, the presence of 
donors or program managers in an FGD can constitute conflicts of interest and bias the responses of 
the participants – either positively (e.g., to receive additional support) or negatively (e.g., feel threatened 
that the donor will discuss findings with the government). However, given the nature of this evaluation 
(it is unlikely that many teachers or parents have a deep understanding of USAID’s role in the reform) 
and the low likelihood of bias (since teachers and parents more likely associate the reform with the 
MoE, who will not be present in the FGDs), STS considers the possibility of conflicts of interest and bias 
minimal. Furthermore, given the important role this evaluation will play in USAID’s opportunity to learn 
directly from beneficiaries, as well as in future USAID program design, STS considers USAID’s presence 
at FGDs a worthwhile risk. Nevertheless, in order to avoid possible conflicts of interest and bias, STS 
proposes the following protocols:
1. STS will consult directly with USAID to share its schedule of parent FGDs in order to 
determine USAID’s availability and to review possibilities that USAID presence at these specific 
FGDs may pose a conflict of interest. In particular, STS will discuss with USAID which schools 
may have an existing history or direct experience with USAID, in order to select schools with a 
lower risk of bias. 
2. USAID will communicate the decision for USAID to attend parent FGDs to the MoE.
3. During the FGDs,
163
a. Evaluators will introduce the USAID participant(s) by sharing their name, title, and a 
brief explanation of the purpose of their attendance. 
b. USAID observers will ensure a minimal presence (e.g., sitting behind the parents).
c. USAID will ensure a neutral presence by refraining from engaging (e.g., asking questions, 
interjecting, nodding, frowning).
4. Once the FGD has been completed, USAID may choose to ask questions or have a discussion 
with parents. This follow-up conversation will not be audio recorded or captured in the FGD 
notes. Instead, separate confidential notes may be kept on this post-FGD conversation. Any 
information shared in these post-FGD discussions will be kept separate from the final evaluation 
report. 
5. As always, participants will be informed that everything they say will be kept confidential and 
that their answers will remain anonymous. Their names will not be associated with any of the 
data or reporting. Attendees of the FGD will be reminded to respect the privacy of other 
participants and to not repeat to others what was said in the FGD.
6. In the final evaluation report, USAID’s presence in select parent FGDs will be noted. 
164
ANNEX X: SUMMARY OF ANALYSES OF SECONDARY DATA
Memo: Summary of analyses of secondary data, NPR Final Evaluation 
School-to-School International – 23 February 2023
INTRODUCTION
For the NPR program evaluation, two types of data were included: primary data, collected through Key 
Informant Interviews and Focus Group Discussions in November and December 2022, and secondary 
data, consisting of existing program reports (quarterly and annual reports and reports from studies 
conducted during NPR) as well as data sets from some of these studies, including the EGRA 
baseline/midline/endline, the Arabic language curriculum effectiveness Study, the School Deviance Study, 
the Student Remediation Study, the Summer Reading Program Study, and the Supplementary Materials 
Study.
Due to the time required to gain access to these data sets, not all could be analyzed for the NPR 
evaluation. Three sets were therefore selected for secondary analyses based on their relevance to the 
NPR evaluation questions: the EGRA baseline and endline data, the Arabic language curriculum 
effectiveness Study, and the Supplementary Materials Study. For each of these studies, salient variables 
were selected to determine the extent to which they correlated with students’ reading scores. This 
memo summarizes the results of these three analyses. Fuller descriptions of these studies, analytical 
methods used, and results summary tables and output tables may be provided upon request. 
All results cited in this memo were found to be statistically significant at the 0.05 level or lower.
Key takeaways from EGRA analyses
• From baseline to endline, increases in Oral Reading Fluency (ORF) scores were found 
in grade 4 only. Grade 4 students showed gains of 5.3 CWPM from baseline to endline while 
grade 1-3 students saw declines in ORF scores. (Note that these differences were not tested for 
statistical significance.)
• Having a copy of a teacher’s guide correlated with better several results, including 
phonemic awareness G1), syllable reading (G2, G3, G4) and ORF (G2, G3) – though correlated with 
weaker ORF scores in grade 4.
• Teaching reading every day was associated with better results, including greater gains in 
grade 1 in syllable reading and ORF and smaller declines in phonemic awareness in grade 1.
• Having received SRMs correlated with big differences in ORF scores, especially in full 
treatment schools of up to 21 more CWPM compared to full treatment schools that had not 
received them.
• Teachers receiving training in how to reach reading appeared to benefit rural students 
in grade 3 more than their urban counterparts.
RESULTS OF EGRA BASELINE/ENDLINE ANALYSES
Overview of Analysis
Data for this analysis were drawn from baseline and endline EGRA scores from a total of 586 schools -
284 partial-treatment and 302 full-treatment schools – and a total of 5,826 students in grades 1-4. The 
tools and items used were as follows:
165
Teacher questionnaire
• Teach reading every day
• Why should you teach letter/syllable sounds to children? 
• Do you have access to supplementary materials?
• Have you ever received training (in person or remotely) on how to teach reading?
Classroom observation
• Teacher has his/her own copy of the teacher’s guide
• All students have textbook or printed materials
• Are there other student reading materials in the classroom
School inventory (Interview with School Director)
• Has this school received supplementary reading materials (SRM) from the MOE?
• SRMs are stored in classrooms
• Teachers have access to SRMs
To compute change scores between baseline and endline, all scores from three subtasks – phonemic 
awareness, syllable reading, and ORF - were calculated for each grade (grades 1-4) at the school level by 
averaging results of all the students of the same grade level in the same school. Then gain scores 
(changes in average scores) were computed for each grade by subtracting the baseline average subtask 
scores from the endline average subtask scores of each of the three subtasks.
Change scores were calculated for all schools combined as well as by school type, gender, and 
urbanicity. Tests for the significance of difference were run between schools answering “yes” and 
schools answering “no” by variable, school type, and subgroup– e.g., the difference between yes and no 
for girls vs. the difference between yes and no for boys. The differences between these changes –
“interaction effects” – were then tested for statistical significance for each variable at each grade level. 
Because schools were randomly selected, results can be generalized to the larger sample frame from 
which the schools were selected with a high degree of confidence. 
Results
NB: In many cases, little improvement in reading scores was found – likely due, in part, to the effects of 
the pandemic. Therefore, this analysis interpreted results as beneficial both when gains were greater and 
when declines were smaller. 
• Overall: From baseline to endline, ORF scores (the number of words correctly read per 
minute) increased overall, school types and grades combined, by 0.8 words. Learners in grades 
1-3 posted declines in ORF scores; only grade 4 students showed an increase – of 5.3 Correct 
Words Per Minute (CWPM) – from baseline to endline. Note that tests were not run to test 
for the statistical significance of the changes described in this bullet; for the following analyses, all 
results were found to be statistically significant at the .05 level or below.
• Phonemic awareness by variable: PA scores tended to decline from baseline to endline 
(note: PA was only tracked for grades 1 and 2). Declines were smaller in grade 1 when teachers 
taught reading every day and when they had copies of their own teachers’ guide. 
• Syllable reading by variable: SR scores were higher when teachers taught reading every day 
(G1), when teachers reported why they teach decoding (G2), and when teachers had copies of 
their own teacher’s guide (G2, G3, G4). Having received training correlated with varying SR 
results, with students scoring lower in G2 but higher in G3.
• ORF scores by variable: ORF scores were higher when teachers taught reading every day 
(G1), had access to supplementary materials (G1), and teachers had received training in reading 
166
(G2, G3). Having a copy of the teachers’ guide correlated with favorable results for students in 
grades 2 and 3 but weaker results in grade 4. On the question Has this school received 
supplementary reading materials (SRMs) from the MOE? having received SRMs made a bigger 
difference in full-treatment schools compared to partial-treatment schools. The following figures 
show the interaction effect, where statistically significant differences were found between full 
and partial-treatment outcomes on this question in all four grade levels, with large differences 
found in favor of full-treatment schools (e.g., 21 more cwpm when teachers reported receiving 
SRMS) - much higher differences than in partial-treatment schools. Note that a greater 
proportion of full-treatment schools received SRMs than partial-treatment, which may have 
resulted in limited variance in partial-treatment school data and thus more modest results: 
Figure 5: Has this school received SRMs from the MOE?
CWPM when School Director answered yes no, by school type
ORF scores by gender: No statistically significant differences were found by gender.
ORF scores by location: Of the ten variables, statistically significant differences were only found in 
two. In the first, a slight difference was found between rural and urban students in grade 2 whose 
teachers reported the importance of decoding. When rural teachers said decoding was important, their 
students’ scores improved from baseline to endline by 4.26 cwpm more than when rural students’ 
teachers did not report this. In contrast, urban students’ scores improved slightly less – by 3.57 cwpm –
when their teachers reported the importance of decoding vs. when they did not. The difference 
between these two “slopes” was statistically significant.
0.95
-1.36 -21.83
-0.76
-30
-20
-10
0
10
no yes
G1
partial full
-0.38
-0.69
-11.5
-1.18
-15
-10
-5
0
no yes
G2
partial full
-3.47
-2
-19.99
1.78
-30
-20
-10
0
10
no yes
G3
partial full
6.65
7.38
-7.39
4.2
-10
-5
0
5
10
no yes
G4
partial full
167
Figure 6: Why should you teach letter/syllable sounds to children? They need to sound out (decode) in order to 
read
CWPM when teachers answered yes or no, by student location, grade 2
The other variable for which statistically significant differences were found concerned training in reading, 
where for rural students, the difference between teachers saying no and yes was an additional 7.74 
cwpm while for urban students, the difference between teachers saying no and yes was 1.55 cwpm 
fewer: 
Figure 7: Have you ever received training (in person or remotely) on how to teach reading?
CWPM when teachers answered yes or no, by student location, grade 3
-3.67
0.59
-0.64
2.93
-4
-3
-2
-1
0
1
2
3
4
no yes
rural urban
-4.09
3.65 3.35
1.8
-5
-4
-3
-2
-1
0
1
2
3
4
5
no yes
rural urban
168
RESULTS OF SRM STUDY ANALYSES
Key takeaways from SRM data analyses
• Students scored higher on ORF when:
o Their teachers reported participating in “most” or “all” of training in the use of SRMs (grade 4)
o School Directors conducted monthly check-ins with parents on the use of student SRMs (grade 2)
o Learners had access to 3asafeer Digital Library and when they read stories from the library (grade 4)
• Boys benefitted more than girls (as reflected in ORF scores) when:
o They were able to borrow books to take home and use (grade 2)
o Schools organized outreach with parents to use SRMs and 3asafeer at home 
o SRMs are appropriate for reading levels (grade 2)
• Girls benefitted more than boys (as reflected in ORF scores) when:
o They were able to borrow books to take home and use (grade 4)
• Rural students benefitted more than urban students (as reflected in ORF scores) when:
o Schools organized outreach with parents to use SRMs and 3asafeer at home 
o Teachers could access 3asafeer (grades 2 & 4)
o SRMs are appropriate for reading levels (grade 2)
Overview of Analysis
Data from these analyses were drawn from a study conducted by RFS-NPR during the EGRA post-test 
from December 2020 to May 2021 in a sample of 50 schools – 25 experimental and 25 comparison –
including 459 students in grade 2 and 459 students in grade 4 (918 total) and 21 teachers in grade 2 and 
21 teachers in grade 4 (42 total), and their school directors (21 total). The same items included in the 
EGRA analysis described above were used in this analysis. Data were collected using the following tools: 
• Student questionnaire
• Teacher questionnaire
• School Director questionnaire/School Inventory questionnaire
For the analysis, individual student EGRA ORF endline scores32 were calculated, then multi-level 
regressions were run to test for significance of the difference between respondents answering “yes” and 
respondents answering “no” on a range of items. Results were disaggregated by grade, gender and 
location (urban/rural).
Results
• When G4 teachers reported having participated in “all” or “most” trainings in using SRMs, their 
learners read 3.71 more CWPM at the endline.
• When School Directors were asked Have you organized outreach to help parents to use 
SRMs or 3asafeer at home? boys’ scores were higher than girls’ when the response was 
“yes” and rural students scored higher than urban students when the response was “yes.”
32 This is in contrast to the previous analysis, where EGRA scores were averaged by school, then compared from baseline to endline.
169
Figure 8: Have you organized outreach to help parents to use SRMs or 3asafeer at home?
CWPM when School Director answered yes or no, by student gender and location, grade 2
• All Grade 2 students benefitted when School Directors checked in with parents: 
When School Directors reported conducting monthly check-ins with parents on student use of 
SRMs, grade 2 students read 2.24 more CWPM.
• Engagement with the 3asafeer digital platform was associated with greater learning 
overall. G4 students who had access to Asafeer Digital Library scored 4.49 CWPM more than 
students who did not. G4 students who read stories from the library scored 6.07 CWPM more.
• When their teachers could access 3asafeer, rural students benefitted more than 
urban students in grades 2 and 4. When teachers had access to 3asafeer, rural students 
scored 5 CWPM higher than urban students in grade 2 and 6 CWPM more in grade 4.
Figure 9: Can teachers access 3asafeer at the school with school equipment?
CWPM when School Director replied yes or no, by location and grade
• Borrowing books benefit boys more in grade 2, and girls more than boys in grade 4.
Being able to borrow books benefitted boys more than girls in grade 2; in grade 4, the opposite 
pattern was found: 
26.43 25.3
18.69 23.56
0
5
10
15
20
25
30
no yes
girl boy
20.96
28
24.28
23.03
0
5
10
15
20
25
30
no yes
rural urban
19.79
28.6
23.47
23.69
0
5
10
15
20
25
30
35
no yes
Grade 2
rural urban
59.53
64.7
66.29
58.49
54
56
58
60
62
64
66
68
no yes
Grade 4
rural urban
170
Figure 10: May grade 2 and 4 students borrow books to take home to use?
CWPM when teachers replied yes or no, by gender and grade
• When SRMs are appropriate for reading levels, boys and rural students fared better 
in grade 2. Having appropriate SRMs was more beneficial for boys than girls and more 
beneficial for rural students than urban ones in grade 2: 
Figure 11: Are SRMs appropriate for the reading skill level in your class?
Cwpm when teachers responded yes or no, by gender and location, grade 2
29.59 25.42
16.76 20.74
0
20
40
no yes
G2
girl boy
59.45
68
56.06 53.13
0
50
100
no yes
G4
girl boy
28.51 26.84
16.7
22.26
0
5
10
15
20
25
30
no yes
girl boy
15.93
25.22 25.62
23.45
0
5
10
15
20
25
30
no yes
rural urban
171
RESULTS OF ARABIC LANGUAGE CURRICULUM EFFECTIVENESS STUDY 
ANALYSES
Key takeaways from Arabic Effectiveness data analyses
• Students scored higher overall on the Arabic Effectiveness comprehension endline –
urban & rural, girls & boys combined - when:
o Teachers used selected methods, including guiding their students to predict content (G6), 
introducing lessons by explaining what students will learn (G6), asking girls & boys questions equally 
(G5), and giving girls and boys equal opportunities (G6)
o Teachers had their own Teacher’s Guides (G5, G6)
o Teachers spoke Darija as their maternal language (G5)
o Teachers were able to stick to their weekly schedule (G5)
o Students devised possible interpretations (G6)
• Boys benefitted more than girls (as reflected in comprehension scores) when:
o Students read quietly (G6)
o Girls and boys were asked questions equally (G6)
o Girls and boys were given equal access to learning materials (G5)
• Girls benefitted more than boys (as reflected in comprehension scores) when:
o Teachers stuck to their weekly reading schedule (G6)
o Teachers were able to cover the curriculum (G6)
• Rural students benefitted more than urban students (as reflected in comprehension 
scores) when:
o Teachers guided students to read individually in a quiet voice
o Students deduced relationships between sentences (G6)
o Teachers completed their reading lesson plan in the time allotted (G6)
o Teachers taught in standard Arabic (G5, G6)
• Urban students benefitted more than rural students (as reflected in comprehension 
scores) when:
o Girls and boys were given equal opportunities to participate in activities (G6)
o Teachers taught reading every day (G6)
Overview of Analysis
Student-level scores from an NPR-administered reading comprehension assessment in grades 5 and 6 in 
May 2021 were analyzed in relation to questions asked in a teacher questionnaire and classroom 
observation administered by NPR during the program. The following items were selected for analysis 
based on their relevance to the NPR evaluation questions:
Teacher questionnaire
• Q6: What is your maternal language?
• Q20: How often do you teach reading?
• Q21: To what extent are you able to stick to your weekly schedule?
• Q23: Are you able to complete the reading lesson plan in the time allotted for reading?
• Q38: What is the earliest grade to expect a child to read Arabic proficiently?
172
• Q43: How often do you read stories from books or tell stories to them?
• Q54: How often do you use an active learning technique to teach
• Q81: Do you have access to supplementary reading materials?
• Q93: Received training on how to teach reading using phonics?
• Q124: Have you been able to cover the Arabic curricula for your grade?
• Q124: Have you been able to cover the Arabic curricula for your grade?
• Q135: What help you received with COVID-related school issues?
• Q126: My students have experienced learning loss due to school closures.
• Q128: My students have been able to catch up to grade-level performance. 
Classroom observation
● Q23: Teacher has his/her own copy of the Teacher Guide
● Q26: How many students have pencil or pen?
● Q38: Teacher guides students to predict the content of the text based on the images.
● Q39: Teacher guides students to listen carefully while teacher reads the text.
● Q40: Teacher guides students to read individually and in a quiet voice (independent reading).
● Q41: Teacher guides students to search for new vocabulary meanings and use them.
● Q42: Teacher guides students to use word map, multiple meaning, or word structure.
● Q44: Teacher guides students to make predictions about text content based on title.
● Q46: Most students are listening to the teacher read out loud.
● Q47: Most students are reading alternately following the example of the teacher.
● Q48: Most students are reading the text.
● Q51: Most students are explaining their answers in sentences of the text.
● Q52: Most students are deducing relationships between sentences.
● Q53: Most students are extracting the general structure of the text.
● Q54: Most students are devising possible interpretations.
● Q55: Teacher introduces the lesson by explaining what students will learn.
● Q58: Teacher reads aloud to students.
● Q63: Teacher gives classwork for students to practice.
● Q64: Teacher concludes the lesson with summary of what was learned.
● Q90: Teacher taught the lesson in standard Arabic.
● Q96: 
o Girls and boys were asked questions equally
o Girls and boys were given equal access to learning materials
o Girls and boys were given equal opportunities to participate in activities
Results were disaggregated by gender and urbanicity. Tests for significance of difference were run 
between “yes” and “no” answers by grade – all students combined – as well as by grade disaggregated by 
gender and urbanicity. Interaction effects were also calculated by comparing differences between “yes” 
and “no” answers for each subgroup – e.g., the difference between yes and no for girls vs. the difference 
between yes and no for boys. Where statistically significant differences are identified, results can be 
generalized with a high level of confidence to in larger populations from which these samples of schools 
were drawn. All results presented in this report are statistically significant at the .05 level or below 
unless otherwise stated.
Results
Overall: Students scored significantly higher overall – girls and boys combined, urban and rural 
combined - when the following was reported from classroom observations:
173
By grade
Grade 5
● Teachers asked girls and boys questions equally
● Teachers gave girls and boys equal opportunities to participate in activities 
● Teachers reported Darija as their maternal language compared to Arabic
● Teachers said they were able to stick to their weekly schedule
Grade 6
● Teachers guided students to predict the content of the text based on the image 
● Teachers introduced the lesson by explaining what students will learn 
● Teachers taught reading 4-5 times per week or less (compared to those who said they taught 
reading every day)
● Students devised possible interpretations
● Teachers taught the lesson in standard Arabic
Both grades 5 and 6: Teachers had their own copy of the Teacher’s Guide
Gender
When grade 6 students were observed reading texts quietly, boys appeared to benefit more than girls, 
with a difference of almost two additional points when observed reading quietly compared to half a 
point less for girls when they were observed reading quietly: 
Figure 12: Most students are reading the text quietly (classroom observation)
Comprehension scores when students were observed reading quietly, by gender, grade 6
When grade 6 girls and boys were asked questions equally, boys seemed to benefit more than girls, 
scoring one point higher when students were asked questions equally in contrast to girls, who scored 
one point lower:
20.18
19.74
17.76
19.46
16.5
17
17.5
18
18.5
19
19.5
20
20.5
no yes
girl boy
174
Figure 13: Girls and boys were asked questions equally (classroom observation)
Comprehension scores by gender, grade 6
When grade 5 girls and boys were given equal access to learning materials, boys benefitted more than 
girls, scoring over one point higher compared to girls, who when materials were equally shared, scored 
slightly lower than when they were not: 
Figure 14: Girls and boys were given equal access to learning materials (classroom observation)
Comprehension scores by gender, grade 5
When teachers said they were always able to stick to their weekly reading schedule, grade 6 girls 
benefitted while grade 6 boys showed lower scores: 
20.51
19.52
17.94
18.93
16
17
18
19
20
21
no yes
girl boy
14.02 13.87
12.34
13.59
11.5
12
12.5
13
13.5
14
14.5
no yes
girl boy
175
Figure 15: To what extent are you able to stick to your weekly schedule (teacher interview)
Comprehension scores by gender, grade 6
When grade 6 teachers said they were able to cover the Arabic curriculum for their grade, girls’ 
comprehension scores were two points higher than if teachers said no. In contrast, boys’ scores were 
lower if teachers said they were able to cover the curriculum. 
Figure 16: Have you been able to cover the Arabic curricula for your grade? (teacher interview)
Comprehension scores by gender, grade 6
Location
When teachers were observed guiding students to read individually in a quiet voice, rural students in 
grade 5 benefitted by over two points. In contrast, this behavior appeared to have little effect on urban 
grade 5 students. 
19.39
20.59
18.41
17.71
16
17
18
19
20
21
somewhat, never always
girls boys
18.15
20.12
19.21
17.98
16.5
17
17.5
18
18.5
19
19.5
20
20.5
no yes
girls boys
176
Figure 17: Teacher guides students to read individually and in a quiet voice (classroom observation)
Comprehension scores by location, grade 6
In grade 5, deducing relationships between sentences appeared to have no impact on the reading ability 
of urban students. In contrast, for rural students, this practice was associated with almost three more 
points in comprehension.
Figure 18: Most students are deducing relationships between sentences (classroom observation)
Comprehension scores by location, grade 5
However, rural students seemed to benefit more, scoring five points higher on the comprehension test 
when this behavior was observed. In contrast, urban students’ performance was only higher by about 
one point.
12.26
14.51
13.48 13.4
11
11.5
12
12.5
13
13.5
14
14.5
15
no yes
rural urban
11.62
14.36
13.42 13.49
0
2
4
6
8
10
12
14
16
no yes
rural urban
177
Figure 19: Most students are deducing relationships between sentences (classroom observation)
Comprehension scores by location, grade 6
Rural students seemed to benefit more in grades 5 and 6 than their urban counterparts when their 
teachers taught using standard Arabic:
Figure 20: Teacher taught lesson in standard Arabic (classroom observation)
Comprehension scores by location and grade
When girls and boys in grade 6 were given equal opportunities to participate in activities, overall, they 
scored almost one more point, but disaggregating by location, differences were substantial: urban 
students in grade 6 scored almost three points higher when girls and boys had equal opportunities, while 
rural students scored two points lower when given equal opportunities:
14.29
19.21
19.01 19.82
0
5
10
15
20
25
no yes
rural urban
10.45
14.04
13.86
13.31
0
5
10
15
no yes
G5
rural urban
14.96 19.58
19.17
19.74
0
5
10
15
20
25
no yes
G6
rural urban
178
Figure 21: Girls and boys were given equal opportunities to participate in activities (classroom observation)
Comprehension scores by location, grade 6
Grade 6 urban students scored half a point higher when their teachers reported teaching reading every 
day; however, grade 6 rural students scored three and a half points lower when their teachers re[prted 
teaching reading every day.
Figure 22: How often do you teach reading? (teacher interview)
Comprehension scores by location, grade 6
Grade 6 rural students benefitted more than their urban counterparts when their teachers reported 
being able to complete their reading lesson plan in the time allotted,
19.89
17.74 17.94
20.76
16
17
18
19
20
21
no yes
rural urban
19.27
15.57 19.19
19.72
0
5
10
15
20
25
4-5 days a week or less every day
rural urban
179
Figure 23: Are you able to complete the reading lesson plan in the time allotted for reading? (teacher 
interview)
Comprehension scores by location, grade 6
14.56
18.71
19.06 19.76
0
5
10
15
20
25
no yes
rural urban
180
ANNEX XI: EARLY GRADE READING ASSESSMENT (EGRA) MATRIX 
Early Grade Reading Assessment (EGRA) Matrix
National Program for Reading (NPR) Final Evaluation
All findings significant at the .05 level or less unless otherwise stated.
Evaluation 
Question
Data 
Source
Subtask Finding Conclusion Recommendation
EQ1 
Learning 
Outcomes
NPR 
baseline 
and
endline
Oral 
Reading 
Frequency 
(ORF)
Where Grade 1 
teachers reported 
they taught reading 
every day, ORF 
scores increased by 
1.52 correct words 
per minute (cwpm) 
from baseline to 
endline overall, full 
and partial schools 
combined.
C.1. ORF gains were 
greater in Grade 1 
when teachers 
reported that they 
taught reading every 
day, full and partial 
schools combined. 
Teaching reading 
every day also 
correlated with 
smaller declines in 
phonemic awareness 
(see below). 
Correlations were 
not found for higher 
grade levels, 
suggesting that 
teaching reading 
every day may be 
particularly crucial in 
the earliest grades.
R.1. Investigate why 
teaching reading every 
day correlated with 
stronger phonemic 
awareness and ORF 
outcomes in Grade 1 
but did not correlate 
with stronger scores 
in Grades 2 through 4. 
181
Evaluation 
Question
Data 
Source
Subtask Finding Conclusion Recommendation
Where Grade 1 
teachers said they 
had access to 
supplementary 
materials, ORF 
scores increased 
overall, full and partial 
treatment schools 
combined, with 
students of teachers 
reporting “yes” gaining 
0.27 cwpm, a 
differences of almost 3 
cwpm compared to 
schools reporting no, 
whose scores declined 
by 2.69 cwpm.
Examined by school 
type, gains were most 
favorable for Grade 3 
and Grade 4 learners
in full treatment 
schools whose 
teachers reported 
having access to 
supplementary 
materials, with 
differences of 10.96 
CWPM in favor of 
Grade 3 students and 
10.03 CWPM in favor 
of Grade 4 students in 
full treatment schools 
whose teachers 
reported having access 
to these materials. 
Paradoxically, students 
in partial treatment 
schools fared better 
when teachers 
reported not having 
access to SRMs.
C.2. Having access
to SRMs (according 
to teacher reports) 
correlated 
significantly with 
greater gains in ORF 
in full treatment 
schools in Grades 3 
and 4 and with 
syllable sound 
identification results 
in Grades 1 through 
4. 
C.3. The reception
of SRMs from the 
MoE correlated with 
smaller losses and 
greater gains in ORF 
scores in Grades 1-4 
in full treatment 
schools.
C.4. The 
preponderance of 
favorable results 
found on the ORF 
subtask, as well as on 
the syllable sound 
identification subtask 
in Grades 1 through 
4, when SRMs were 
present, suggests a 
strong relationship 
between SRMs and 
learning in these two 
skill areas. The 
presence of SRMs 
correlated with 
smaller declines in 
ORF scores in Grade 
1, but not in gains –
R.2. Given the 
frequency with which 
the availability of 
SRMs correlated with 
stronger ORF and 
syllable sound 
identification scores, 
examine the extent to 
which SRMs were 
available, and ways in 
which they were used, 
to strengthen ORF 
and syllable sound 
identification skills. 
182
Evaluation 
Question
Data 
Source
Subtask Finding Conclusion Recommendation
Where teachers were
found to have 
access to SRMs in 
the school inventory, 
Grade 1 learners in 
full treatment schools 
showed ORF a decline 
of .78 CWPM, though 
this decline was 14.23 
CWPM in favor of 
students where SRMs 
had been found in 
comparison to full 
treatment schools 
where they were not.
an unsurprising 
finding since 
phonemic awareness 
is essentially a 
prereading skill so 
would be less likely 
to be influenced by 
print materials than 
ORF or syllable 
sound identification.
Where it was 
reported that schools 
had received 
supplementary 
reading materials 
(SRMs) from the 
MOE, learners in full 
treatment schools that 
received SRMs posted 
the smallest losses and 
the greatest gains. In 
schools where SRMs 
had been received, 
differences in scores 
ranged from 21 
CWPM in Grade 1, 10 
in Grade 2, 22 in 
Grade 3 and 12 in 
Grade 4 in favor of 
students in those 
schools over ones 
where SRMs had not 
been received. 
Differences were 
greater in full 
treatment schools 
than in partial 
treatment schools.
183
Evaluation 
Question
Data 
Source
Subtask Finding Conclusion Recommendation
When asked “Have 
you ever received 
training (in person 
or remotely) on 
how to teach 
reading?” in full and 
partial schools 
combined, gains were 
greater in Grade 2 and 
Grade 3 where 
teachers reported 
having received 
training. By school 
type, Grade 2 students 
in full treatment 
schools whose 
teachers reported 
having been trained 
gained 0.31 CWPM –
a 2.87 CWPM 
difference in favor of 
students whose 
teachers had been 
trained over ones who 
said “no.” 
C.5. ORF gains were 
greater in Grades 2 
and 3, and syllable 
sound identification 
scores were stronger 
in Grade 3, when 
teachers said they 
had received training 
in reading. Similar 
correlations were 
not found between 
phonemic awareness 
scores and training.
R.3. Given the positive 
correlations found 
between receiving 
training in reading and 
stronger ORF and 
syllable sound 
identification scores, 
conditions under 
which training was 
received – e.g., 
amount, frequency, 
adaptation to grade 
levels - should be 
examined. Possible 
reasons higher 
phonemic awareness 
scores did not
correlate with 
receiving training 
should also be 
explored.
184
Evaluation 
Question
Data 
Source
Subtask Finding Conclusion Recommendation
Where teachers were 
observed with a copy 
of a teacher’s 
guide, results were 
mixed in favor of 
Grade 2 and 3 
students whose 
teachers had a guide, 
and in favor of Grade 
4 students whose 
teachers did not have 
a guide. Where 
teachers were 
observed with a guide, 
Grade 2 ORF scores 
declined by 0.44 
CWPM – a difference 
of approximately 6.5 
CWPM compared to a 
7.07 CWPM decline 
for students whose 
teachers did not have 
a guide. In Grade 3, 
students whose 
teacher had a guide 
gained 0.49 CWPM 
compared to a decline 
of 5.44 CWPM where 
they did not have a 
guide – a difference of 
5 CWPM in favor of 
students whose 
teachers had a guide. 
Paradoxically, in 
Grade 4, ORF scores 
increased by 2.98 
CWPM from baseline 
to endline when 
teachers had a guide, 
but increased more -
by 8.02 CWPM -
when they did not.
C.6. Teachers’ access 
to teacher’s guides 
correlated with ORF 
gains in G2 and G3, 
with syllable sound 
identification in 
Grades 2, 3, and 4, 
and with smaller 
declines in phonemic 
awareness scores in 
Grade 1 (see below). 
R.4. Given the 
correlations found 
between having a 
teacher’s guide and 
higher ORF scores in 
Grades 2 and 3, and 
lower declines in 
phonemic awareness 
in Grade 1, examine 
how teachers use 
these guides in Grades 
1 to 3 to build these 
skills, and ways to 
reinforce these 
practices. At the same 
time, examine reasons 
why having a teacher’s 
guide did not 
correlate with 
stronger scores in 
Grade 4.
185
Evaluation 
Question
Data 
Source
Subtask Finding Conclusion Recommendation
Syllable 
Sound 
Identificatio
n
Scores for grade 1 
students on syllable 
sound identification
decreased by 31.5 
percent (from 41.0 to 
28.1 syllables 
identified) from 
baseline to midline, 
but increased slightly 
for grade 4 students 
(3.9 percent) 
(differences not tested 
for significance of 
difference)
C.7. Declines from 
baseline to midline, 
which was conducted 
a year after the 
pandemic began, 
were more sizeable 
for younger students. 
The pandemic may 
have had a 
disproportionate 
impact on younger 
students.
R.5. Track these 
trends to see how 
learning loss may 
evolve over time as 
children – especially 
ones in the lower 
grades during the 
pandemic - progress 
through primary 
school.
Where teachers 
reported having 
been trained in 
reading, Grade 3 
students fared better 
than those whose 
teachers said “no,” 
gaining 6.00 CWPM 
from baseline to 
endline compared to a 
1.47 CWPM decline 
for students whose 
teachers said “no” – a 
7.5 difference in favor 
of students whose 
teachers had been 
trained. Oddly, the 
reverse was true in 
Grade 2, where 
students of teachers 
reporting not having 
received training 
gained (1.66 CWPM) 
whereas scores for 
students whose 
teachers reported 
they had received 
training declined by 
0.53 syllables correctly 
identified.
See C.5. See R.3.
186
Evaluation 
Question
Data 
Source
Subtask Finding Conclusion Recommendation
In Grades 2, 3 and 4, 
syllable identification 
gains were greater by 
17 syllables in Grade 
2, 11 syllables in 
Grade 3, and 1 syllable 
in Grade 4 in favor of 
students whose 
teachers had their 
own copy of a 
teachers’ guide, all 
schools combined. 
Paradoxically, having a 
teacher’s guide was 
associated with 
greater gains in partial 
than in full treatment 
schools.
See C.6. See R.4.
In full treatment 
schools, students of 
teachers reporting 
having access to 
supplementary 
materials – showed 
gains of 3.10 syllable 
sounds identified in 
Grade 3 and 9.08 in 
Grade 4 –22 syllables 
more than students 
whose teachers 
reported “no” in 
Grade 3, and 17 
syllables more than 
students in Grade 4 
whose teachers 
reported “no.” 
See C.2. See R.2.
187
Evaluation 
Question
Data 
Source
Subtask Finding Conclusion Recommendation
When teachers in full 
treatment Schools 
reported having 
received SRMs 
from the MoE, their 
students in Grades 1 
through 4 read 
substantially more 
syllables that students 
where teachers 
reported not receiving 
SRMs, with differences 
as great as 
approximately 38 
syllables in Grade 1, 
20 in Grade 2, 9 in 
Grade 3 and 8 in 
Grade 4, in favor of 
students whose 
teachers reported 
having received 
materials. In partial 
treatment schools, 
results were mixed. 
Phonemic 
Awareness
Declines in phonemic 
awareness scores 
were smaller (0.05 
fewer sounds 
identified) when 
teachers said they 
taught reading 
every day compared 
to 0.06 fewer sounds 
where teachers did 
not teach reading 
every day, all schools 
combined. Students 
fared better in partial 
treatment schools 
where teachers taught 
reading every day, 
with change scores 
remaining the same 
compared to declines 
found in full treatment 
schools.
See C.1. See R.1.
188
Evaluation 
Question
Data 
Source
Subtask Finding Conclusion Recommendation
When teachers 
were observed with 
a copy of their own 
guide, G1 students 
saw smaller declines in 
phonemic awareness 
scores (.05 fewer 
sounds identified) than 
when teachers were 
not observed with 
their own guide (.14 
fewer sounds 
identified)
See C.6. See R.4.
189
ANNEX XII: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX
Findings, Conclusions, Recommendations (FCR) Matrix
National Program for Reading (NPR) Final Evaluation
Evaluation 
Question Finding Conclusion Recommendation
EQ1 Learning 
Outcomes
While grade 1–3 students’ 
ORF scores generally 
peaked at baseline, 
declined at midline, and 
partially recovered at 
endline, grade 4 students’ 
ORF scores increased 
both from baseline to 
midline and midline to 
endline. These declines in 
ORF scores from baseline 
to midline illustrate 
COVID-19’s association 
with learning loss. The 
ORF results highlight 
another notable trend—
girls generally 
outperformed boys.
Students in rural areas had 
more difficulty than their 
urban counterparts in 
accessing distance 
learning, according to 
respondents
COVID also affected 
students’ well-being and 
socioemotional skills
The COVID-19 pandemic 
and hybrid schooling had 
an effect on learning loss, 
possibly with a 
disproportionate impact 
on younger students; in 
contrast, grade 4 students’ 
ORF scores increased 
from baseline to endline, 
possibly due to longer 
exposure to NPR. 
Secondary analyses of data 
from EGRA, SRM and 
Arabic language 
curriculum effectiveness 
studies found that 
stronger reading 
performance correlated 
with several variables.
Continue to research 
NPR cohorts to 
understand the project’s 
cumulative impact.
Teachers had a high 
turnover rate, which 
impeded NPR progress
Teacher turnover was 
also perceived as 
negatively impacted 
learning outcomes
Determine strategies to 
reduce teacher turnover.
190
Evaluation 
Question Finding Conclusion Recommendation
The Morocco Curriculum 
Review and Capacity 
Building Activity fostered 
a learner-centered 
approach and a focus on 
all four parts of literacy 
acquisition (listening, 
speaking, reading, and 
writing)
Students enjoyed reading 
more and improved in 
other aspects of Arabic, 
including speaking, writing, 
and listening
NPR has positively 
changed classrooms.
Apply learner-centered 
approach to other aspects 
of school curricula.
The partnership between 
the central MoE and key 
NPR stakeholders was 
collaborative and 
participatory, based on 
respondents’ 
characterization of their 
working relationship
NPR stakeholders 
empowered the MoE to 
sustain the new 
curriculum, learning 
materials, and massive 
open online course 
(MOOC) content by 
involving them in all stages 
of their production 
Key NPR funders and 
implementers developed a 
fruitful working 
relationship with the 
central MoE and helped 
them develop the 
technical capacity needed 
to develop, adapt, and 
sustain the new 
curriculum.
Document internal best 
practices of NPR 
partnership
191
Evaluation 
Question Finding Conclusion Recommendation
EQ2
Assessment & 
Capacity 
Building to 
Support Data 
Use
Most respondents 
reported the assessment 
capacity building series 
was well-designed
Some respondents 
suggested training a larger 
range of actors to use 
results effectively
The assessment capacity 
building program was 
successful, though some 
respondents raised 
concerns about 
possibilities of applying 
results in the field.
Wherever possible, 
conduct capacity building 
activities in person
Design training to address 
needs of evaluators, 
pedagogical experts, and 
administrators; articulate 
data collection, analysis, 
interpretation, and use 
roles for each of these
Continue to build and 
sustain the central MoE’s 
capacity in data use and 
assessment, including 
support to schedule 
periodic EGRAs as part of 
the official school calendar 
and to develop a guide 
about managing a reform 
agenda and implementing 
international research.
NPR made use of adaptive 
management: 
Introducing the 
assessment capacity 
building workshop
Printing materials for rural 
students with limited IT 
access
Adding MSL to televised 
lessons
Adding formative 
assessment & remediation 
support upon seeing 
results of midline
On numerous occasions, 
NPR practiced adaptive 
management. However, 
the extent to which the 
MoE drove these 
adaptations is unclear.
Focus assessment training 
on the application of 
results with an eye to 
improving materials and 
classroom practices
192
Evaluation 
Question Finding Conclusion Recommendation
When asked what data 
had been shared with 
them, most respondents 
(regional and provincial 
directors, CMREF 
directors & trainers, 
inspectors, school 
directors, and teachers in 
experimental and scaling 
zones): “We have not 
received data or results” 
Roles concerning data 
collection, analysis, and 
use are unclear
Data sharing has been 
limited, making it difficult 
to use data to improve 
practice.
Standardize data sharing 
protocols; for example, 
the use of a publicly 
accessible platform
Most respondents 
indicated they had not 
received the FA toolkit, 
either print or electronic 
versions
Of the few teachers who 
were trained and received 
print copies, some gave 
positive feedback about 
the FA toolkit
Some teachers who had 
received training found it 
difficult to implement
Most respondents 
reported that they had 
not received the 
formative assessment (FA) 
toolkit and knew little 
about its contents. When 
they did have access, 
respondents found it was 
difficult to implement.
Amplify efforts to 
distribute and train 
education support 
personnel and teachers in 
the use of FA materials 
and methods
193
Evaluation 
Question Finding Conclusion Recommendation
EQ3
Curricula at a 
National Scale
The MoE displayed its 
ability to use and scale the 
new curricula by:
Producing and distributing 
the curriculum and 
materials nationwide, 
including supplementary 
reading materials, online 
resources, and MOOC 
content
Anticipating pushback on 
the new curriculum and, 
over time, building buy-in 
from regions and 
provinces, leading to the 
widespread adoption of 
new phonetic approach
Assisting with decision 
making concerning design 
changes (see previous 
slides)
The MoE provided a solid 
foundation for nationwide 
adoption of the new 
Arabic curriculum.
Institutionalize the 
effective practices 
adopted as part of NPR 
scaling so future scaling 
efforts for new curricula 
do not have to reinvent 
the wheel.
Primary NPR 
stakeholders—including 
the MoE, NPR, and 
USAID—are aware of 
these needs
The process of training on 
the new curriculum 
nationwide remains 
ongoing and varied
The most pressing issue 
around managing the 
curricula at scale concerns 
the need for intensive 
teacher training and 
coordinated monitoring—
not only of training 
initiatives but also of 
classroom teaching 
practices.
Equip and empower 
school directors and/or 
inspectors to conduct 
classroom monitoring and 
ensure teachers are 
applying what they learned 
from MOOCs
Devise a national strategic 
plan to assess the national 
rollout of training 
initiatives and ensure 
inspectors are 
implementing the training 
toolkit with fidelity
194
Evaluation 
Question Finding Conclusion Recommendation
EQ4
Community & 
Parental 
Engagement
The use of story books 
(SRMs) was seen as 
positive and parental 
support increased because 
of NPR
What worked: Story 
books and parental 
support
According to the SRM 
study data, boys and rural 
students benefited most 
when outreach was 
conducted. In addition, all 
Grade 2 students 
benefited when school 
directors checked in with 
parents.
Parents are not invited, 
communication is 
insufficient, and 
expectations are unclear. 
What did NOT work: 
Parents & community are 
not sufficiently engaged
Conduct campaigns for 
parents to support and 
motivate students at 
home
Establish system of 
communication between 
teachers and parents
Respondent perceptions 
of parents include: 
Parents cannot follow 
guidance
Parents are uninformed
Knowledge level of 
parents is low
Families are affected by 
poverty
Parents are busy with 
work
Parents are unable to 
communicate with kids
Challenge: Many view 
parents as unable to 
support their children in 
reading. 
Implement strategies that 
help parents overcome 
challenges they have in 
supporting their children’s 
reading, as well as 
increasing their awareness 
about the general 
importance of reading. 
195
Evaluation 
Question Finding Conclusion Recommendation
EQ5
Private Sector 
Engagement
The MoE conducted 
trainings and shared 
feedback from educators 
on draft materials
NPR played an 
intermediary role in 
soliciting and providing 
feedback between the 
MoE and publishers
Publishers noted their 
appreciation for the MoE 
feedback on materials 
MoE collaborated with 
textbook publishers 
during NPR more than it 
had in the past, resulting 
in improvements to the 
materials production 
process and the materials 
themselves.
Strategize how to 
continue to engage with 
publishers and raise their 
technical capacity to 
revise and improve 
learning materials to 
address acknowledged 
gaps 
Respondents in the MoE, 
NPR, and USAID are 
aware of the gaps 
between the new 
curriculum, student 
textbooks, and teacher 
guides
These concerns were 
raised in a USAID￾commissioned curriculum 
report published in 2021
The gaps include those at 
a macro level—how the 
curriculum is reflected in 
materials—and a micro 
level—quality of the paper 
in materials
Although private textbook 
publishers made progress 
during NPR, they need 
continued urging to revise 
materials to align fully 
with the new curriculum.
196
Evaluation 
Question Finding Conclusion Recommendation
The relationship between 
3asafeer and NPR was 
constructive and 
successful according to all 
respondents involved
An underutilized aspect of 
the collaboration was the 
data that 3asafeer 
provided on students’ use 
of the digital platform
The SRM study data 
indicated that grade 4 
students who had access 
to 3asafeer Digital Library 
read 4.49 CWPM more
and that G4 students who 
read stories from the 
library read 6.07 CWPM 
more.
NPR’s collaboration with 
3asafeer highlights the 
value of engaging with 
partners outside of 
Morocco to access low￾cost educational 
resources.
Secondary analyses 
suggest that having access 
to 3asafeer may make a 
difference in reading 
performance.
Explore additional ways 
that the MoE could 
partner with the private 
sector, especially internet 
providers, and expand 
access to materials.
MoE respondents believe 
the lack of engagement 
was focused in rural areas 
and due to lack of internet 
or difficulty accessing 
devices
NPR data revealed a more 
nuanced picture, as 
student use of 3asafeer 
was comparably low in 
rural and urban areas 
(22.6% and 26.4%, 
respectively)
The lack of engagement 
with the 3asafeer digital 
platform underscores the 
obstacles of implementing 
an EdTech intervention in 
Morocco.
Provide teachers with 
paper copies of their 
guides (digital forms are 
not enough) 
Conduct a case study in 
experimental schools to 
determine how to 
overcome obstacles that 
led to low usage rates of 
the 3asafeer digital 
platform
197
Evaluation 
Question Finding Conclusion Recommendation
EQ6
Education 
Access, 
Quality, and 
Results 
Across 
Populations 
NPR ensured the design, 
materials, and practices 
were equitable
Curriculum in practice 
supports children equally
Some aspects benefited 
rural students more, some 
urban more
New program better than 
old
The new curriculum helps 
all groups learn in an 
equitable manner: girls 
and boys, children from 
different language groups, 
children from urban and 
rural areas, and children 
with disabilities.
Of all subgroups included 
in these analyses, boys and 
rural students were found 
to benefit from contextual 
factors most.
Observation: Girls learn, 
perform better than boys
Girls’ scores were better 
than boys’ (Prov Dir exp, 
School Director exp, 
teacher exp)
Girls learn better in early 
years, then boys catch up 
(Implementer)
Reasons
Boys work in agriculture 
(School Director exp)
Girls are more interested 
in learning (teacher exp)
Girls deepen their 
learning and sacrifice to 
read and study at home 
while boys look for games 
to play (teacher exp)
Girls performed better 
than boys (though didn’t 
attribute this to NPR)
198
Evaluation 
Question Finding Conclusion Recommendation
NPR made numerous 
efforts to support 
students who were deaf 
and hard of hearing
Distance and hybrid 
modalities were 
developed during COVID 
New curriculum doesn’t 
sufficiently address 
children with disabilities 
Lack of tools, training, and 
personnel
Need to tailor curriculum 
to types of disabilities 
Need to address 
disabilities other than deaf 
and hard of hearing
Respondents most 
frequently expressed 
concerns about students 
with special needs, of all 
subgroups.
Whenever possible and 
feasible, reserve a 
separate class for students 
with special needs
Provide specialists to 
support children with 
special needs, especially 
for more severe cases
Provide a curriculum with 
its own schedule, including 
materials (“kit”) tailored 
to the circumstances of 
students with different 
kinds of special needs
More research is needed 
to determine how 
different groups interact 
with new materials and 
respond to the new 
strategies introduced in 
the new curriculum
199
ANNEX XIII: DATA VALIDATION WORKSHOP DISCUSSION NOTES
National Program for Reading (NPR) Final Evaluation
Data Validation Workshop
Notes from Group Discussion
Evaluation Question 1 – Learning Outcomes
Results: there was a big decline in results because of Covid 19. It had a big impact on learning outcomes. 
Other variables to this decline may include reluctance and resistance of certain actors in the regions 
(inspectors, but also teachers who haven’t been involved since the beginning of the training). In addition, 
teacher training coincided with some events — like inspector strikes.
Regarding the effectiveness of the instruments used — It is clear that as soon as Covid was over, the 
decline was curbed and a positive impact on learning outcome appeared. One explanation might be that 
whenever there is a new project or whenever there is no sufficient appropriation of the program by 
actors, we might see certain weaknesses or drawbacks. In other words, there should be stability and 
continuity for the results to be sustained and increasing. For example, 4th graders performed well 
because of the stability. As has been stated in the results, 4th grade results are more visible as students 
have spent more time in the program intervention. It is recommended that the MoE assess and study 
the results in future years to track trends. We can see a trend from baseline to midline, then midline to 
endline. Will this trend from midline to endline continue? The MoE should keep studying this. If the 
teacher training went as planned, will the results be better? More work should be continued. 
Regarding teacher turnover (leaving schools), we can see that only 1 out of 8 teachers in experimental 
schools (trained under NPR) continued to work in their same school. Can we really see the impact of 
the program in this situation? What training model can we talk about if the teachers are not retained in 
the schools? One of the tools that could be used to address that turnover is the use of MOOCs, to 
allow teachers to perfect their training even if they move from an experimental school to a pilot school. 
The training from which pilot teachers benefited was stronger than that from which non-pilot teachers 
benefitted. The results witnessed only happened in pilot schools. Teacher turnover can have an impact 
in the experimental period. We need to recover that in the scale up period. 
Reiterated is the fact that the results could only be seen in very long periods. Any change at its beginning 
might be met with certain aspects of resistance. 
The approach is a radical change in the relationship between the reader and what is read (the text). It is 
historically thought that in Arab countries one may read or learn by heart without understanding what 
you are reading (this remains to prove). The last reform of reading curriculum in Morocco was 20 years 
ago or so. So the way of teaching is outdated. The teachers of today were the students of yesterday. 
Remains to check whether resistance to change is more important in the new teachers too. To 
summarize the important points: (1) understanding what we are reading and (2) resistance of older 
teachers to the new curriculum. Both can impact the outcome of the evaluation.
Evaluation Question 2 – Assessment & Capacity Building to Support Data Use
Many workshops were organized from the inception of the program to display the program to 
inspectors and people specialized in guidance. We tested the program’s instruments with real students 
in real schools. There were other workshops related to assessments, establishing benchmarks and item 
design. One important workshop was about results analysis and interpretation. This was very important 
200
for people who work at the central level and some teacher trainers and inspectors from various regions. 
We need to scale this up to reach more people in the regions in order to make profit from the results. 
We talked about difficulties in the regions. Some people tend to not to be fully involved. This implication 
fades from the central units from the ministry to the academies, provinces, and especially the schools. In 
the region, people concerned with assessment (regional and provincial exam centers) and those involved 
with pedagogical affairs (staff and inspectors) should be involved in this project. Inspectors are the only 
people who get into the schools and classrooms. They should be involved. 
The instruments were uploaded to the ministry’s website / portal, and those who are concerned with 
the project downloaded the materials — hopefully all of them. Those who don’t know about the project 
are not concerned with it. There are no sufficient printed copies of the guide available for everybody. 
The only printed copies were sent to the schools such as the guide. The teachers were also advised to 
download a copy of the guide for free. 
Covid 19 has constrained in person trainings. This gave rise to negative impact – there weren’t enough 
sessions to enable or empower those actors. Covid left its impact. Those workshops should not be 
limited only to the central administration, but also the regions and provinces. 
Regarding capacity improvement for the MoE — there is a recommendation for the MoE to make 
partnerships with private sector, especially internet providers, to grant access for learners, and 
employees of the ministry to access different platforms to download the guides. 
It might be good for future programs to have a guide developed for how the MoE could manage reform 
agenda or implementation of international research. Having a guide would be a sustainable reference.
Evaluation Question 3 – Curricula at a National Scale 
From 2020-2021, two years of covid have determined, in a way, the results. Especially regarding 
inspectors not being able to attend the conferences and workshops. People were not encouraged to 
organize activities in person to avoid infection. The idea of training and the lack of enough training was 
reiterated in various observations. 
The production of support materials required great efforts and contributed to the results we have. If we 
didn’t have that, the results may have been worse. 
We should measure the impact in a direct, immediate way. To measure its real impact of the project, 
there is a need for more time ( program stability) . In the future, are there any projects in the pipeline 
for the sustainability of the measure of the students? There is a recommendation that measuring 
students should be permanent at different phases of the year. This is the business of the MoE. They have 
to draw on the conclusions and think about the future and install a system of evaluation that includes 
EGRA in its system of assessment. 
Evaluation Question 4 – Community & Parental Engagement
Some insisted that the program did not include clearly involving parents or society as an objective. But I 
think there was involvement by the parents through the short stories. Parents came to the schools and 
asked “why do you ask our children to read these stories?” And some parents went to the schools and 
asked them to give more books to their children, which helped break barriers between parents and the 
school. In the initial program maybe there wasn’t an initial idea of including parents wasn’t apparent 
enough in the objective. 
201
Also, extensive reading and supplementary reading have been encouraged by the MoE. One 
recommendation — if there is any other program, the idea of sensitizing parents to the importance of 
reading should be included. 
Parental engagement was a component of the project. There has been a shift in the program to use 
SRMs for more global parental engagement. We were about to prepare a teacher’s guide about parental 
engagement. Result 3 focused on parental engagement. In summer 2019, the program (the MoE through
associations) implemented the summer reading program to support kids. There was a question of 
whether the MoE would be able to sustain this or not. There was a shift in the focus from community 
engagement — this is why there were no target activities for parents beyond SRMs in NPR. 
We should say that some people view some parents in a particular way, rather than making a value 
judgement saying that all parents are the same. 
Evaluation Question 5 – Private Sector Engagement 
The general conclusion is that it was a very good initiative to have publishers come to the MoE and 
express their needs. There is a tendency on the part of the MoE regarding professionalization of this 
sector, so that things would be legal and equitable for everyone. There is an appreciation of 3asafeer and 
recommendation to continue using this and make it accessible to all the children. We can add other 
platforms to it, and more content to be used offline and by all children. There is a need to build on 
other partnerships with donors and private sector. This was the case with 3asafeer - it was built upon 
the memorandum of understanding. MoE should make these partnerships available to future programs 
to leverage that resource. 
Evaluation Question 6 – Education Access, Quality, and Results by Population 
One remark: “The new program is better than the previous one.”—This should be reformulated. This is 
a value judgement! We are comparing the efficacy. We are not comparing programs. (Correction from 
STS: This is a quote from one of the respondents. We will put it in quotations to make this clear.)
The curriculum's development and improvement is a continuum. There are certain variables that change 
the results. We talked about the supportive documents. Another important factor is management of the 
behavior of learners. There is a close relationship between managing the difficult behavior of learners 
and results. A study from 1992 says that 75% of students with learning disabilities also have difficult 
behaviors. 
It has been suggested that there are many variables in addition to learners that might explain 
performance (Teachers, principals, school environment, parents, peers, etc.). 
Over 80% of students with behavior difficulties also have learning difficulties. Students during Covid saw 
negative results. Future training should keep in mind this issue. 
Regarding the outcome of girls vs boys and rural vs urban, we have always found that urban student are 
excelling more than in rural areas, and girls more than boys.
STS Response: On the question of mental health and well-being: being at school is not only academics. It 
means being at ease and at peace with oneself. There is a non-academic aspect of schooling.
U.S. Agency for International Development
1300 Pennsylvania Avenue, NW
Washington, DC 20523