This publication was produced for review by the United States Agency for International Development. It was prepared by Dr. Karla Giuliano Sarr, Lisa Slifer-Mbacke, Gaelle Simon, Aissatou Mbaye and Abdoulaye Tambandia for USAID/Senegal Monitoring and Evaluation Project. NIGER EDUCATION COMMUNITY STRENGTHENING (NECS) PROGRAM Final Performance Evaluation AUGUST 2018 XXX FOR MEP i | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV CONTENTS ACKNOWLEDGMENTS.....................................................................................................................III ACRONYMS & ABBREVIATIONS................................................................................................... IV EXECUTIVE SUMMARY.......................................................................................................................1 EVALUATION PURPOSE AND QUESTIONS......................................................................................... 1 ACTIVITY BACKGROUND ......................................................................................................................... 1 EVALUATION DESIGN, METHODS AND LIMITATIONS.................................................................. 1 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS .............................................................. 2 EVALUATION PURPOSE AND EVALUATION QUESTIONS ..................................................9 EVALUATION QUESTIONS......................................................................................................................... 9 ACTIVITY BACKGROUND................................................................................................................9 IMPLEMENTATION HISTORY – AN OVERVIEW...............................................................................11 EVALUATION METHODS AND LIMITATIONS........................................................................ 11 DATA COLLECTION METHODS............................................................................................................12 SAMPLING.......................................................................................................................................................12 DATA ANALYSIS...........................................................................................................................................13 LIMITATIONS .................................................................................................................................................13 FINDINGS AND CONCLUSIONS................................................................................................. 14 EVALUATION QUESTION 1.....................................................................................................................14 EVALUATION QUESTION 2.....................................................................................................................18 EVALUATION QUESTION 3.....................................................................................................................24 EVALUATION QUESTION 4.....................................................................................................................29 EVALUATION QUESTION 5.....................................................................................................................35 RECOMMENDATIONS..................................................................................................................... 39 USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | ii ANNEXES ANNEX I: WORK PLAN .................................................................................................................. 43 ANNEX II: EVALUATION METHODS & LIMITATIONS ....................................................... 104 ANNEX III: FIELDWORK DATA COLLECTION SCHEDULE ............................................. 111 ANNEX IV: LIST OF KEY INFORMANT INTERVIEW AND FOCUS GROUP DISCUSSION PARTICIPANTS ................................................................................................................................. 112 ANNEX V: NUMBER OF INTERVIEWS BY TYPE OF RESPONDENT............................... 113 ANNEX VI: LIST OF PEOPLE INTERVIEWED .......................................................................... 114 ANNEX VII: NECS ACTIVITY TIMELINE................................................................................... 117 ANNEX VIII: LIST OF DOCUMENTS REVIEWED................................................................... 119 ANNEX IX: REFLEXION SUR LA MISE A L’ECHELLE DE L’ASL/ SYNTHESE DE L’ACTIVITE ............................................................................................................................................................... 121 ANNEX X: EVALUATION TOOLS ............................................................................................. 132 ANNEX XI: TABLE DES PERFORMANCES DES ELEVES DESAGREGEES PAR SEXE.... 137 ANNEX XII: NECS THEORY OF CHANGE ............................................................................. 138 ANNEX XIII: COMPARISON OF READING APPROACHES IN NIGER........................... 138 ANNEX XIV: AVAILABLE PERFORMANCE RESULTS ON READING PROGRAMS IN NIGER ............................................................................................................................................................... 138 ANNEX XIV: FINDINGS AND CONCLUSIONS TABLE – PRELIMINARY RESULTS ... 138 ANNEX XVI: MATRICE DES RECOMMANDATIONS ET ACTIONS A MENER............ 171 ANNEX XVII: CODE OF CONDUCT........................................................................................ 138 iii | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV ACKNOWLEDGMENTS Working closely with the various stakeholders to develop this final evaluation report has been an honor and a privilege. By nature, evaluation is a delicate and sensitive endeavor and, nonetheless, stakeholders received the evaluation team in a welcoming and collaborative manner throughout the process. The team specifically wishes to thank the communities, school personnel and, of course, students for their openness and feedback about the NECS activity. MEP representatives at the departmental and regional levels were also accommodating and collegial. Niamey-level MEP representatives graciously accommodated multiple visits from evaluators and participated actively in multiple feedback sessions in a short span of time. Lastly, Monsieur Yahouza Ibrahim, the MEP Secretary General, USAID Niger staff, Ahmed Tijiane Diallo and Aboubacar Kossomi, and the NECS team, specifically, Ali Amadou, Romain Babagbeto, Idrissa Ibrah Waziri, Ousseini Manzo, Hadiza Mamadou, Tcha Berei and particularly, Ousmane Brah Mamane, deserve special recognition for facilitating the evaluation process. We sincerely hope that the information presented here contributes to supporting reading acquisition for Nigerien children. Much thanks to you all and best wishes for next steps! ~ The Evaluation Team USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | iv ACRONYMS & ABBREVIATIONS AeA Aide et Action / Aid and Action AME Association des Mères d’Elèves / (School) Mothers’ Associations AMEP Activity Monitoring and Evaluation Plan ANED Association Nigerienne pour l’education et le developpement / Nigerien Association for Education and Development AOR Agreement Officer’s Representative APE Association de Parents d’Elèves / Parent Teachers’ Associations (PTA) APS Apprentissage par la situation / Situational Learning Approach ARL Apprentissage Rapide de la Lecture / Accelerated Learning Reading Approach ASL Apprentissage Systématique de la Lecture / Systemmatic Reading Approach CDGES Comité de gestion décentralisée des établissements scolaires / Community School Management Committees CDPLN Direction des Curricula et de la Promotion des Langues Nationale (CDPLN)/ Directorate for Curricula and the Promotion of National Languages CE1 Cours Elémentaire 1ere année / Third grade CE2 Cours Elémentaire 2ieme année./ Fourth grade CGPE Comité de Gestion des Points d’Eau / Water Point Management Committee CI Cours d’Initiation / 1st grade CM1 Cours Moyen 1ere année / 5th grade CM2 Cours Moyen 2eme année / 6th grade COP Chief of Party CP Cours préparatoire / 2nd grade Concern Concern Worldwide Niger DAGE Direction de l’Appui à la Gestion de l’Ecole / Directorate for School Management DCOP Deputy Chief of Party DDEP Direction départementale de l’éducation primaire / Departmental Primary Education Direction DECP Direction de l’Enseignement du Cycle Primaire / Directorate of Primary School Teaching DEP Direction des Etudes et Programmation / Directorate of Studies and Planning v | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV DESASO Direction de l’Evaluation, du suivi des acquis scolaires et de l’orientation (DESASO) / Directorate of Evaluation and monitoring of school performance DESQ Direction de l’Evaluation et du Suivi de la Qualité / Directorate of Monitoring and Evaluation and Quality Assurance DFIC Direction de la Formation Initiale et Continue / Directorate of Teacher Training DGCGDES Direction Générale des comités de gestion décentralisée des établissements scolaires / Upper Directorate of Community School Management Committees DGEB Direction Générale de l’Education de Base / Upper Directorate of Basic Education DPAFA Direction des programmes d'alphabétisation et de formation des adultes / Direcotrate of Literacy and Adult Training Programs DPSF Direction de la Promotion de la Scolarisation des Filles / Direction for Promotion of Girls’ Education DQA Data Quality Assessment DREP Directions régionales de l’Education Primaire / Regional Primary Education Directorates EGR Early Grade Reading EGRA Early Grade Reading Assessment EQ Evaluation Questions FCR Findings, Conclusions and Recommendations FGD Focus Group Discussion IDAENF Inspection Départementale de l’Alphabétisation et de l’Education Non Formelle / Departmental Inspectorate for Adult Literacy and Non-Formal Education IEP Inspection de l’Enseignement Primaire / Primary Education Inspection IMAGINE Improve the Education of Girls in Niger IP Implementing Partner IPTT Indicator of Performance Tracking Table ITT Indicator Tracking Table KIIs Key Informant Interviews M&E Monitoring and Evaluation MCC Millennium Challenge Corporation MEP Monitoring and Evaluation Project USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | vi MEP/A/PLN/EC Ministère de l’Enseignement Primaire, de l’Alphabétisation, de la Promotion des Langues Nationales et de l’Education Civique / Ministry of Primary Education, Adult Literacy and the Promotion of National Languages and Civics Education MSI Management Systems International NECS Niger Education and Community Strengthening (Project) NGO Non-Governmental Organization PDDE Programme Decennal de Developpement de l'Education / Decennial Education Development Program Plan Plan International POC Point of Contact SCOFI Scolarisation des Filles / Girls’ Schooling SRO Sahel Regional Office STS School-to-School International TCP Treshold Country Program USAID United States Agency for International Development VIE Volontaires de l’Intégration Educative / Volunteers for Educational Induction 1 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV EXECUTIVE SUMMARY EVALUATION PURPOSE AND QUESTIONS The purpose of this evaluation is to determine the progress made to date in achieving the stated objectives of the Niger Education and Community Strengthening (NECS) activity. The evaluation also assessed the potential for scaling up of components within NECS, with particular focus on the NECS reading curriculum. The evaluation’s findings will be used to support future design of USAID education interventions in Niger. The evaluation questions are: 1) To what extent has NECS achieved its objectives in relation to targets? 2) How did the NECS “Systematic Reading Approach” (ASL) approach enhance reading outcomes in NECS schools? 3) In terms of scalability, what are the comparative advantages of the ASL approach as compared to the other reading approaches in Niger? 4) What types of changes are visible in NECS schools and communities? 5) To what extent did the project successfully target girls? ACTIVITY BACKGROUND The 2012-2018 NECS activity leverages evidence-based reading, engages communities, and encourages families to keep their children in school. NECS’ goal is improved educational opportunities available to children and strengthened links between communities and state structures. This goal is to be achieved through two interdependent strategic objectives: SO1: Increased student grade reading achievement by improving instruction in primary schools and promoting a culture of reading in communities; and SO2: Increased access to quality education in schools, especially for girls. Plan International (Plan) is NECS’ implementing partner. NECS uses an innovative “Systematic Reading Approach” (Approche systématique de la lecture, ASL) to improve reading outcomes for girls and boys across 160 schools in Niger. NECS was originally launched with Millennium Challenge Corporation (MCC) Threshold Country funding and managed by the USAID/Niamey Office with oversight from the USAID/West Africa Regional Mission in Ghana. In 2015, oversight of the portfolio was transferred to USAID/Sahel Regional Office in Dakar. SRO extended the activity in June 2016 for an additional two years, bringing the total value of the NECS activity to $11,599,656.00 over a 6-year period. EVALUATION DESIGN, METHODS AND LIMITATIONS The evaluation team conducted an extensive desk review of all background documents including NECS formal reports as well as Ministry of Primary Education (MEP) assessments and reporting on current primary education reforms. The evaluation team also reviewed quantitative data related to all NECS indicators as well as student assessment data. In terms of fieldwork, the evaluation team used a purposeful sample of NECS schools, selected based on region, type of interventions received by NECS, and language of instruction. The sample included five schools in each of the NECS four regions: Maradi, Tahoua, Tillaberi and Zinder. The team conducted key informant interviews (KII) with national and local level education authorities and officials as well as focus group discussions (FGD) with community leaders and community members and classroom observations of reading lessons. The team also conducted a brief non-diagnostic reading exercise with students. In total, the team held 174 KIIs, 70 FGDs, and 20 USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 2 exercises with 692 participants during the evaluation fieldwork which took place between May 25 and June 21, 2018. Data analysis included thematic and content analysis, with categories developed inductively. In addition, the team used NVivo qualitative analysis software to review and code interviews at the central level as well as KIIs with teachers and school directors. The evaluation team used data triangulation to strengthen findings or identify areas of divergence. Evaluation limitations included the challenge of identifying and locating the numerous USAID and Plan Niger and Plan USA staff as well as the various subcontractors that intervened throughout the span of NECS activities. The evaluation team relied on the MSI home office and Dakar office to track down the various personnel involved in the project over its 6-year life span and to coordinate phone interviews. A significant challenge was the lack of valid student testing data in order to determine student progress over the life of the project. The team spent extensive time with Plan Niger and Plan USA reviewing data sets and discussing the collection process, timing and validity of the data in order to ascertain relevance. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS FINDINGS AND CONCLUSIONS Evaluation Question 1: To what extent has NECS achieved its objectives in relation to targets? In answering this question, the evaluation team first clarified the project’s objectives as they changed over the six years of implementation to incorporate increased focus on improving early grade reading outcomes. At the outset of the project, the objective was to “support primary school education for girls” focusing on enrolling more girls and improving the overall school environment. As the project developed and the 2011 USAID Education Policy took hold, improved reading instruction became one of the two strategic objectives for the project. By the 2016 extension, the strategic objective related to reading became the first objective using USAID standardized reading measurement indicators. The evaluation team found that the NECS activity has met the majority of its objectives, making remarkable progress in its last year, 2017-2018. According to project data, overall, NECS has reached 7 of its 10 targets as of June 2018. In particular, the achievement of community-related indicators, improving community-school relationships and strengthening community support for education are among the project’s strongest achievements. NECS far surpassed its output targets in terms of number of teaching and learning materials provided and teachers trained. Of the three targets not reached, NECS missed two indicators relating to girls’ and student enrollment in school by narrow margins. Analysis of the NECS M&E system also identified areas needing strengthening. However, NECS did not achieve the higher-level outcome indicator target of reading improvement. The team concluded that the target of 50% of second grade students reading at fluent levels (40 words per minute) is not reasonable given the low-levels of student reading capacity in Niger and the typical level of reading improvements seen in other similar countries. Further, the data collected by the project through an early grade reading assessment (EGRA) and reviewed by the evaluation team are problematic in terms of their validity. Sub-contractors always led the development of the reading approach for NECS. Over the life of the project, three different sub-contractors oversaw the reading approach, each with a different style and method. Changes to sub-contractors were made due to administrative 3 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV reasons, and in the case of VIE, due to dissatisfaction with the ARL approach. The current sub￾contractor, School-to-School International (STS) was not directly involved in the 2015 Mathematica baseline EGRA collection and indicates that the most recent EGRA data collected in May 2018, intended to measure improvement, cannot be considered valid. In addition, changes to the sampling frame prevent comparison of results. Niger poses an extremely challenging learning environment and evaluation participants were quick to identify key contributing factors: high student and teacher absenteeism, low teacher capacity with heavy reliance upon contractual teachers, and recurrent teacher strikes. The multiple changes in reading subcontractors resulted in delays to effectively launching the reading curriculum. The first full school year of NECS class cohorts were established during the 2017-2018 school year, five years following the activity’s launch. The evaluation team found that there was an overall lack of strong reading expertise capacity within Plan USA, Plan Niger and NECS as well as limited understanding of reading assessment tools and approaches. Evaluation Question 2: How did the NECS “Systematic Reading Approach” (ASL) approach enhance reading outcomes in NECS schools? The Systematic Reading Approach (ASL) is a new reading methodology in Niger that integrates several basic reading skills: sound-letter correspondence, decoding, oral and written comprehension and the “whole language” method for transitioning into French. Despite the complicated operating environment for schools in Niger and the implementation challenges that NECS encountered throughout its life cycle, reports from stakeholders consistently point to ASL improving students’ reading abilities. Unfortunately, the lack of rigorous reading performance data weakens arguments for ASL’s success in enhancing reading outcomes. The evaluation team’s review of ASL materials as well as interviews with teachers and school directors show that the approach has been effective in allowing teachers to understand and implement ASL’s step￾by-step approach to reading. While some teachers and school directors expressed the view that NECS ongoing technical assistance is overwhelming, most respondents felt that such support provides value￾added to the system. The evaluation team found among the schools and communities visited that ASL’s use of national languages attracts children to reading, motivates many teachers and creates a bridge for parents to participate more actively in their children’s education. One challenge of ASL instruction identified by the team is not unique to ASL but is part of the larger bilingual curriculum reform underway in Niger: teacher placement based on mother tongue langue skills. Some schools visited identified teachers that were not fluent in the local mother tongue language and therefore was not able to instruct the ASL curriculum. The team also found that a more important detriment to reading outcomes is the transition to French at the end of grade 2 as compared to exit programs in other countries which occur later in grade 3. This language transition in grade 2 is also not aligned with the current MEP bilingual curriculum reform in which students begin to read and write in French in grade 2 with French becoming the language of instruction in grade 5. Another drawback to ASL reading outcomes as identified by the variety of national-level education stakeholders interviewed, was the fact that NECS did not strongly articulate or clearly communicate its reading approach across all implementers and stakeholders. This resulted in differences in understanding and support for the project across MEP directorates, especially regarding curricular elements like choice USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 4 of scripted lessons and length and content of teaching and learning materials (TLM), as well as views on how NECS aligns with the current bilingual curriculum reform being scaled up in the 2018-2019 school year. Evaluation Question 3: In terms of scalability, what are the comparative advantages of the ASL approach as compared to the other reading approaches in Niger? Similar to most developing countries, Niger implements several new approaches to reading in order to improve students’ skills. Among them are ASL (NECS), ARL (ELAN) and APS (the current MEP bilingual curriculum reform). Interviews with stakeholders revealed a sense of competition amongst implementers to develop the most effective reading approach in order gain Ministry approval for country-wide application. The evaluation team reviewed the current approaches and found that the ASL reading program is the most comprehensive reading program currently being implemented in classrooms in Niger in terms of its step-by-step teaching/learning process, its targeting of five core reading skills plus listening comprehension and writing skills, and inclusion of a scope and sequence. In terms of comparative advantages as a reading approach, stakeholders generally recognized that ASL elicits greater interest among students and communities in reading and that students learn to read more quickly with ASL. The majority of stakeholders find the approach to be credible and well-adapted to Niger’s realities. In addition, stakeholders felt that opportunities for scalability are strengthened given the Minister’s interest in ASL, the initiative currently underway to identify best practices and steps for integrating these practices into the larger MEP system. At the same time, many central-level MEP officials and NECS staff recognized that political will is a critical condition for successful scaling-up. Indeed, the three topics that generated the most discussion amongst participants during the ASL Scalability Work Session were political will, funding and the lack of rigorous performance results to justify scaling up of any one approach or certain of its components. Another element that currently complicates the scaling up of NECS is that while NECS has developed strong collaboration with MEP offices and officials from the central to the school level, the project still has access to external funding, has its own staff and is not solely dependent upon the Ministry. Without this support, ASL scaling up would become expensive and difficult. Evaluation Question 4: What types of changes are visible in NECS schools and communities? The evaluation team observed that NECS schools and communities stand out for their engagement and implementation of activities that support reading and student enrollment. While challenges remain within certain communities, most schools demonstrate positive change in levels of engagement and the existence of school reading support organizations. Findings indicate that NECS has also served to strengthen the linkage between schools and communities through its programming. The evaluation team found that use of local languages within instruction, provision of trainings and technical support to teachers and mentoring seem to have most benefitted students. At the same time, parents have benefitted from adult literacy programming, continued engagement and reinvigorated school support organizations. Findings demonstrate in particular that public reading sessions motivate and engage school-level and community stakeholders and, most importantly, children and their parents. 5 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Positive effects may have been even stronger had adult literacy activities continued beyond 2014. Additionally, the mentoring program holds much promise for supporting at-risk students, but creative thinking is needed to ensure its success. Communities’ efforts to support schools are noteworthy: community organizations visited demonstrate improved management and planning relative to supporting school needs, particularly around the construction and upkeep of school infrastructures. Financial and in-kind contributions are significant, despite the low-resource environment that characterizes many of Niger’s communities. Evaluation Question 5: To what extent did the project successfully target girls? NECS implemented gender initiatives throughout project life which have had a steady and positive effect on gender-related outcomes. Initiatives include gender trainings and awareness-raising, award ceremonies for the best female students and the project’s Code of Conduct which sets standards within schools for equal and supportive environments for girl students and female teachers. The initiatives reached 48% of girls’ enrollment in target school, just short of 50% targeted. However, this is higher than the national average of 46% girls’ enrollment. Despite the project’s efforts, girls’ enrollment and retention as well as their role in student leadership remain serious challenges in Niger. Classroom observations of grades 1 and 2 within the sampled NECS schools confirm that while teachers regularly recognize female students and that girls participate actively in reading lessons, within the 20 schools visited by the evaluation team, all of the “class leaders” were boys. The majority (83%) of the “assistant class leaders” were girls. School-wide, NECS has focused on developing School Governments within NECS schools. Within the 20 schools visited, girls’ participation within school governments outnumbered boys’ participation (55% overall). At the same time, girls held an inferior number of leadership positions (36% of presidencies; 46% of prime ministries) of positions recorded. While the Code of Conduct has been largely positive, the evaluation team learned of and observed cases of misguided application that warrant renewed attention to community engagement and clear communication with school and regional actors on the purpose of the code. Although the MEP’s interest in scaling up the Code of Conduct is encouraging, it must recognize that continued and consistent mobilization and awareness-raising efforts are necessary to achieve success within NECS schools. RECOMMENDATIONS The evaluation team worked with stakeholders in Niger to validate initial findings. Based on these findings, the evaluation team facilitated a recommendations workshop hosted by the MEP with the participation of USAID/Niamey and NECS staff. The following recommendations were developed from this workshop and are divided into two sections: the first focuses on immediate actions that should be taken now to support improved management and project outcomes and the second suggests actions for future similar USAID education interventions in Niger. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 6 Immediate Considerations: FOR NECS 1. Before printing additional documents, further strengthen ASL TLMs by addressing typos and misalignment of references within materials through a formal copyediting process. NECS Education Coordinators already possess documentation of teacher feedback that will support this activity. 2. Continue to offer training and technical support to NECS teachers in ASL methodology. Identify and address gaps as indicated by a small number of teachers and school directors. Make adjustments to schedules and duration to further strengthen trainings. 3. Closely monitor community engagement in schooling and where needed, reinvigorate community awareness-raising and support efforts. Essential activities include post-literacy efforts, public reading sessions, effective management and reporting of community organizations and women’s involvement. While effects of NECS community mobilization efforts observed at the community level have been impressive, sustainability is questionable and may need reinforcement in the short-term. 4. Reinforce gender-equity training and awareness raising to include clear roles for men and boys in supporting girls’ schooling. 5. Clarify roles and expectations for school governments to improve their functioning. 6. Investigate implementation of Code of Conduct signing at the school level to assure that signing is voluntary and transparent. Should inconsistencies be sustained, identify a plan of action to clarify expectations and process with school, regional and central level stakeholders. Continue community advocacy work in areas where pushback is most prevalent, especially outreach with religious and community leaders. FOR NECS WITH MEP 1. Develop strategy and guidance for teachers in bilingual schools to implement ASL techniques alongside APS. Monitor teacher and student experience closely with deliberate feedback loops to inform possible scale up of ASL components alongside APS within other Reform schools. 2. Develop and communicate clear technical assistance strategy to teachers and school directors. Work with inspectors, pedagogical advisors, departmental and regional directors on providing constructive feedback so that teachers feel more accompanied and less surveilled in their teaching practice. 3. Identify new strategies to strengthen and reinvigorate the mentoring program maintaining volunteer status. Investigate opportunities to work with retirees and provide symbolic and in￾kind recognition. Expand framing of mentoring activity to support linking students with successful female role models. FOR NECS WITH PLAN INTERNATIONAL 1. Develop an action plan to strengthen NECS M&E systems moving forward. Elements of the action plan should include: 1) training and ongoing support to school and regional level actors in monitoring efforts as well as to assure fidelity of implementation of activities, 2) how to share and centralize monitoring information within the NECS activity, 3) whether to strengthen current excel-based practices for information management or to reactivate/develop a user￾friendly database that can be supported by in-country staff, 4) professional development opportunities for NECS staff members to enhance understanding of strong program monitoring practices and how learning initiatives may be integrated into daily routines. 7 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV 2. Allow adequate time for planning and implementation work with STS to clarify EGRA 2019 process so that sample allows for comparison and to ensure credible results. 3. Reconsider how to push back transition to French until 3rd grade (CE1) in order to take full advantage of grade 2 for reading instruction within a familiar language following evidence-based practice. This may require providing support to CE1 teachers for transition. 4. Redefine and articulate reading approaches employed within ASL to bolster and reinforce advocacy efforts within the Ministry. 5. Engage MEP officials in information and capacity building sessions on evidence-based reading approaches to develop reading expertise amongst MEP officials and increase project buy-in, especially around critical milestones in reading acquisition and appropriate application of scripted lessons. FOR NECS WITH PLAN INTERNATIONAL, USAID/NIAMEY AND SRO 1. Revisit AMEP to revise Indicator 1 on student performance so that reading performance target is more appropriate to Nigerien context. FOR PLAN INTERNATIONAL USA WITH PLAN NIGER 1. Identify M&E support individuals who will provide regular technical assistance to NECS M&E staff. While M&E technical assistance has figured among support available to the project, the consistency of this support has been erratic, particularly within the last year of the project. FOR MEP 1. Thoroughly review the benchmark identified for EGRA scores – follow a rigorous, participatory process that takes into account current levels of student performance as well as linguistic complexities. 2. Provide adequate support and follow-up to the language mapping processes indicated by the Direction of Curriculum and Promotion of National Languages. Recent developments related to the Senegal Reading for All or Lecture pour tous (LPT) activity may provide some guidance. 3. Stakeholders urgently need guidance from MEP to identify best directions for educational programming and expenditures focused on providing explicit and systematic reading instruction. Nigerien children and parents deserve stable, well-articulated and well-advocated educational strategies. 4. Consider ASL’s comparative advantage to other existing reading programs in addressing the five￾critical evidence-based reading components, providing a scope and sequence, systematic instruction, engaging teaching practices and existent TLMs that may complement and fill in gaps within delivery within Reform schools. Supplement Reform strategies with these central ASL components. 5. Work closely with NECS team in expanded application of the Code of Conduct to ensure appropriate and transparent application. Consider ways to implement gender training in order to assure fidelity of implementation. 6. MEP Regional officials (IEPS, DDEPs, DREPs, SCOFI representatives and pedagogical advisors) should continue to work closely with community organizations, particularly Mothers’ Associations in order to sustain girls’ schooling recruitment and maintenance efforts. FOR PLAN INTERNATIONAL/NECS WITH MEP 1. Reexamine assessment options in order to identify the most appropriate tool for Nigerien primary school students. While EGRA may still be an option, the project should also closely USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 8 follow progress on the other earlier reading acquisition EGRA sub-tasks: orientation to print, letter name identification and listening comprehension. Doing so may help address floor effects and provide more motivating results. 2. Plan for and undertake a rigorous, participatory process to identify the most appropriate benchmarks for EGRA sub-task scores. Engage appropriate reading technical experts to assure quality. This process needs to take into account current levels of student performance as well as linguistic complexities. Such a process may also greatly benefit MEP as once such benchmarks are established they can be adopted for use with all current early grade reading programs. 3. Develop an in-country technical advisor group made up of MEP officials, implementing partners and donors to share best practices in EGRA and other educational assessments that focus on reading. Doing may address a serious gap in knowledge of sound assessment practices and support more credible measurement outcomes. FOR USAID/NIAMEY 1. Accompany the MEP in identifying appropriate, valid and relevant EGRA benchmarks to support eventual credible and externally valid performance results. Given capacity constraints, external reading experts to MEP will likely need to be engaged. FOR USAID/NIAMEY WITH NECS AND MEP 1. Develop working group to better understand and address concerns for bringing ASL to scale around political buy-in and resource identification. Based on on-going consultations develop a participatory scaling up action plan. Elaborate advocacy and capacity building initiatives to support a potential scaling up process. LONG-TERM CONSIDERATIONS FOR USAID: FOR USAID/NIAMEY AND SRO 1. Develop a new solicitation process for the next phase of early grade reading support to ensure adequate reading technical expertise throughout the life of the project. Awardees will need to demonstrate reading specialist expertise at the headquarters, country and project level in order to ensure regular and adequate oversight of reading initiatives. Next phases will need to include clear philosophical and evidence-based technical grounding of the identified reading approach. FOR USAID/NIAMEY 1. Based on espoused desires to increase involvement within the Northern Tillabery region where insecurity and displacement present serious concerns, build in socio-emotional learning (SEL) components within educational programming. SEL can be integrated within the reading curriculum and does not necessarily require more time with students. Training and SEL support for teachers and school personnel are also necessary programming elements. Review INEE Guidance Note on Pyschosocial Support. IRC’s Healing Classroom project in Diffa may also offer some insights into programming options. 2. Further investigate results and cost-effectiveness of alternative models to formal schooling initiatives like Save the Children’s Literacy Boost and IRC’s Healing Classrooms model. 9 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV EVALUATION PURPOSE AND EVALUATION QUESTIONS This report presents the findings, conclusions and recommendations of an external final evaluation of the Niger Education and Community Strengthening (NECS) program. As commissioned by the USAID/Senegal Sahel Regional Office (SRO), the purpose of this evaluation is to: 1) Assess the progress of the contractor to completion in terms of achievements, management processes and challenges in meeting the stated objectives of the cooperative agreement and sustainability of achievements. 2) Make recommendations for scaling-up of best practices for the Government of Niger and USAID. 3) Make recommendations for the design of similar activities in the future. The audience for this evaluation is USAID/Niger and the USAID/Senegal SRO and Niger’s Ministry of Primary Education (MEP). The USAID/Washington/E3/Education Bureau may also be interested in the results of the evaluation as they relate to their Education Policy’s improved reading goal, as well as to expanding access to education in crisis and conflict-affected contexts. Current implementing partners will likely find interest in this report, as well as all stakeholders involved in improving both the access and quality of formal education programs in complex environments. The report comprises five parts. This section introduces the NECS program, the purpose of the evaluation and guiding evaluation questions. Section 2 provides an overview of the NECS activity. Section 3 reviews the methods used in the evaluation, the sample and data collection activities. Section 4 addresses findings and conclusions for each of the five evaluation questions. Section 5 presents the evaluation team’s recommendations that emerge from the findings, as through various participatory activities. EVALUATION QUESTIONS USAID established five questions to guide this final evaluation’s review of the NECS program’s performance in relation to multiple activity components: 1) To what extent has NECS achieved its objectives in relation to its targets? 2) How did the NECS “Apprentissage systemmatique de la lecture (ASL)” approach enhance reading outcomes in NECS Schools? 3) In terms of scalability, what are the comparative advantages of the ASL approach as compared to the other reading approaches in Niger? 4) What types of changes are visible in NECS schools and communities? 5) To what extent did the project successfully target girls? ACTIVITY BACKGROUND The 2012-2018 funded Niger Education and Community Strengthening (NECS) Program leverages evidence-based reading practices, engages communities, and encourages families to enroll and keep their children in school. The goal of NECS is improved educational opportunities available to children and USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 10 strengthened links between communities and state structures. This goal is achieved through two interdependent strategic objectives. 1 TABLE 1: NECS STRATEGIC OBJECTIVES SO1: Increased student grade reading achievement by improving instruction in primary schools and promoting a culture of reading in communities SO2: Increase access to quality education in schools, especially for girls IR1.1: Improved reading instruction in primary schools through the development and delivery of the ASL curriculum by school directors, teachers and pedagogical advisors IR2.1: An improved school environment through monitoring and support from Pedagogical Advisors, School Management Committee Focal Points and Girls’ Education Focal Points IR1.2: Culture of reading is promoted in communities through monitoring and support from Adult Literacy Inspections and School Management Committee Focal Points. For this phase of the project, this will move toward supporting these inspection authorities to monitor and promote community reading activities. IR2.2: Increased community participation to support education through monitoring and support from Pedagogical Advisors and School Management Committee Focal Points. This activity builds on past gains achieved by USAID investments in education in Niger and an MEP-led initiative to promote reading in four local languages across Niger. Under a sole-sourced contract, Plan International (Plan) is NECS’s implementing partner, along with consortium member Aide et Action. NECS operates in seven regions and 160 schools (including 12 bilingual schools)2 and their communities. FIGURE 1: NECS INTERVENTION AREAS 1 In addition to Table 1, see Annex XII for the NECS theory of change. 2 With the NECS+ extension, the original 150 NECS schools evolved to include 12 bilingual schools that are part of Niger’s curricular reform project. With two schools in Diffa that closed due to insecurity, there are 160 schools at the end of project. 11 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV IMPLEMENTATION HISTORY – AN OVERVIEW The original NECS activity was developed as part of a Millennium Challenge Corporation (MCC) Threshold Country Program (TCP) in 2008 with the objective of increasing girls’ enrollment and retention in primary school. The TCP was managed by the USAID/West Africa Regional Mission in Accra, Ghana. The project was suspended due to the political situation in Niger in December 2009. The reopening of the project occurred in June 2012 with a new start date for the NECS cooperative agreement with USAID from June 15, 2012, through June 30, 2016. In the interim, USAID launched its Global Education Strategy in 2011, which emphasized improving reading skills of primary school children. In the relaunch of the NECS activity, implementers defined the two strategic objectives above to guide the four-year intervention, namely increased access to quality education (Strategic Objective 1) and increased student grade reading achievement (Strategic Objective 2). Strategic Objective 1 (SO1) activities included improving physical and social environment for students and increased community support for education. SO2 activities included improving reading instruction in primary schools and promoting a culture of reading in communities. To meet these objectives, NECS implements activities focused at the school, community and ministerial levels (see Box 1). BOX 1: KEY NECS ACTIVITIES Action Plans Adult Literacy Advocacy AME/APE/CGPE ARL/ASL Award Ceremonies Awareness Raising CGDES Code of Conduct Deworming Make-up Classes Mentoring Public Reading Sessions Reading Contests School Governments Supporting Community Organizations Teaching & Learning Materials Trainings Village Libraries On June 30, 2016, a modification to the cooperative agreement extended the project by two years, from June 30, 2016, to June 30, 2018, and added USD $3,982,451 to the agreement. The project’s extension period, called NECS+, shifted from the West Africa Regional Mission in Ghana to the USAID/Senegal Sahel Regional Office (SRO) in Dakar. The Activity Monitoring and Evaluation Plan (AMEP) submitted in November 2016 reduced the performance indicators and revised them to include eight new standard indicators related to the USAID 2011 Education Policy. The AMEP retained the two NECS SOs but switched their order, with SO1 now focused on improving student grade reading and SO2 aiming to improve access to quality education. (See Annex VI for a detailed timeline of program developments.) EVALUATION METHODS AND LIMITATIONS A team of five people worked collaboratively to produce this evaluation. They included three international experts, one M&E associate and a Nigerien evaluator: USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 12 • Lisa Slifer-Mbacke, currently MEP COP and specialist in the area of education, provided methodological orientation and continuous backstopping and oversight; • Karla Giuliano Sarr, independent consultant and education expert, served as Team Leader; • Gaëlle Simon, MSI Technical Manager, served as the Reading Expert for the team and focused on the comparative advantage of the NECS reading approach • Aissatou Mbaye Sy, MEP M&E Manager, served as overall Task Manager • Abdoulaye Tambandia, independent consultant, based in Niger, provided local expertise, overseeing sampling choices, and identifying local contractors to serve as data collectors Due to security concerns and the high volume of interviews with education stakeholders in Niamey the Team Leader and Reading Specialist remained in the capital while the MEP M&E Associate and the Nigerien Evaluator led data collection activities at the regional level. Data collection took place between May 25th and June 21st with intensive school-level data collection occurring during a fifteen day period between May 29th and June 9th. Additional interviews continued through July 3rd. DATA COLLECTION METHODS This performance evaluation utilized document review and both qualitative and quantitative methods. (See Annex for more details and copies of tools.) • Document Review: Review of documents provided an understanding of activity objectives and implementation characteristics, contexts, and challenges, and helped to identify and describe achieved activity results. Close review of curriculum documents provided insights into reading methodologies and approaches. • Qualitative methods: The evaluation team conducted key informant interviews with relevant individuals at funding agencies (USAID and MCC), MEP officials, and staff of Plan and activity partners as well as other reading program implementers. At the school-level, focus groups were organized with various community organizations as well as community leaders. Both KIIs and FGDs followed a semi-structured approach where tools served as guides for discussion. The team also conducted classroom observations of reading lessons within 19 of 20 schools visited as well as a brief non￾diagnostic reading exercise with groups of 6 students (3 girls, 3 boys) in all 20 schools. In total, the team held 174 KIIs, 70 FGDs, and 20 exercises with 692 participants. (See Annex VI for a detailed table of evaluation participants.) In addition, the team employed a number of participatory methods during meetings with key stakeholders, including MSI’s Scalability Checklist. • Quantitative methods: Although the team attempted to use existing data to conduct a comparative analysis of reading fluency from the start of the activity to the end of the program, this comparison was not possible. The team did explore the number of targets identified in the activity’s AMEP to ascertain differentiation between targets and actuals. SAMPLING Purposeful sampling methods guided the qualitative data sample choices. Quantitative data derive from previous assessments and evaluations. The qualitative data include interviews with key stakeholders as well as visits to 20 of the 160 NECS+ schools. Criteria for selection included region, type of school (hard, soft and bilingual) and language of instruction. The denomination “hard” indicates that these schools 13 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV benefitted from the IMAGINE project as well as NECS. “Soft” indicates only the presence of the NECS activity. The sample identifies 5 schools in each of the four regions (Maradi, Tahoua, Tillaberi and Zinder) also within reasonable distance from Niamey as necessary because of security concerns. Of the schools in the sample, 9 are hard, 7 are soft and 4 are bilingual. In terms of language of instruction, the majority (12) of schools offer instruction in Haussa, 5 in Zarma, 2 in Kanuri and 1 in Fulfulde. (See Annex II for more detail.) DATA ANALYSIS Data analysis drew on findings organized by evaluation question, facilitated by standardized data collection instruments (see Annex X). Qualitative data collected via the key informant interviews and FGDs were analyzed using thematic and content analysis, with categories developed inductively. Analysis of all data collection activities began in the field, as team identified interesting or significant points and held team debriefings to determine the efficacy and consistency of the data collection. A local transcribing agency worked with recordings of a select number of interviews to develop transcriptions. Preliminary in-country data analysis used a findings and conclusions rubric to analyze the data and capture emerging themes and/or patterns. In order to achieve an even deeper level of analysis, NVivo qualitative analysis software was used to systematically review and code interviews at the central level as well as KIIs with teachers and school directors. Detailed descriptive and analytical notes and the Preliminary Findings and Conclusions Table served as the basis for analysis for remaining interviews. The evaluation team used data triangulation, the use of two or more data sources, to strengthen findings or identify areas of divergence. Throughout the analysis, visual depictions are included where possible to enhance understanding. The team also produced a chronograph to better understand and convey the NECS activity’s evolution. LIMITATIONS The evaluation team encountered limitations due to 1) the project’s complicated life cycle and staff turnover, 2) the impossibility of comparing existent quantitative data and 3) a compressed timeframe. During its lifetime, project partners rotated and many individuals contributing to the project changed positions. The evaluation team traced the history of positions and held multiple interviews in order to access the most relevant project information. Secondly, while the original workplan anticipated comparing data across various early grade reading assessments, in-depth investigation revealed inconsistent samples that made it impossible to perform a trends analysis. Moreover, there was often misalignment between assessments and program implementation and significant data validity concerns existed for both the 2017 and 2018 EGRAs further limiting opportunities for comparison. (See Annex XIV for a table that explains the major findings and limitations for the major relevant assessments.) Lastly, the period of data collection aligned with the end of the school year and the beginning of the raining season, resulting in a reduced presence of students at sample schools. In order to address this limitation, two teams conducted school level data collection simultaneously and used creative methods to simulate regular classroom environments. Data collection also took place during the month of Ramadan and two national holidays interrupted the data collection phase. In-country presentations on preliminary data also needed to be moved forward to accommodate stakeholders’ schedules further compressing the preliminary data analysis period. The high quality of initial analysis was confirmed through spot-checking and formal evaluation and structural coding techniques during the remote data analysis phase and writing phases. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 14 FINDINGS AND CONCLUSIONS EVALUATION QUESTION 1 EQ1: To what extent has NECS achieved its objectives in relation to its targets? FINDINGS Responses to this first evaluation question principally rely on project data, as well as interviews and supplementary documents that allow for triangulation. This section also accounts for the project’s historical evolution. (See detailed timeline in Annex VI). HISTORY OF PROJECT OBJECTIVES AND INDICATORS As the project evolved, emphasis on reading increased. At the beginning of the Improving the Education of girls in Niger (IMAGINE) project, the program objective was to “support primary school education for girls” (MCC/USAID Strategic Objective Grant Agreement [SOAG]), p. 1). The USAID Reading Strategy took effect in 2011, a year prior to the relaunch of NECS activities and the establishment of its two strategic objectives. The indicator revision process took place in preparation for NECS+ and the November 2016 AMEP demonstrated a reordering of strategic objectives so that the reading-related objective became SO1. Emails provided by the USAID SRO M&E specialist demonstrate engagement among program stakeholders during this process. Decreasing the indicators from 38 to 103 was an effort to reduce the burden on NECS staff and Plan International agreed with the suggestion to focus on USAID standardized indicators, as substantiated by an interview with the NECS M&E specialist. As a result, seven of the 10 indicators directly addressed education. Plan International remained the prime contractor from IMAGINE through NECS+. No data indicate considerations of an open call for proposals for an organization with reading-specific expertise. ACHIEVEMENT OF TARGETS According to project data, NECS+ has reached seven of its 10 targets as of the end of the second year of the NECS+ project extension (June 2018; see table 2). In particular, the achievement of community￾related indicators (7, 9 and 10) confirm findings from field-level interviews. In terms of improving community-school relationships and strengthening community support for education, participants were generally equivocal in identifying related major achievements of the NECS activity. Of the three targets that were not reached, two indicators (2 and 8) missed by narrow margins. Participants at all levels frequently identified socio-economic constraints as persistent obstacles to achieving these access-related indicators, including leaving school due to agricultural and nomadic practices, family migrations and social norms concerning girls’ activities. Although the rate of girls’ attendance (48 percent) is below the project target, it exceeds the national average of 46 percent (MEP Annual Statistics, 2015-2016). NECS 3 The NECS phase (2012-2016) featured 34 indicators and four countering violent extremism (CVE) indicators. Of these, six performance indicators were linked to the two strategic objectives, 28 indicators focused on activities (water points, mentoring, support for community structures, adult literacy). Seven indicators addressed reading and two focused on the distribution of materials. Among the revised 10 NECS+ indicators, six address reading, one addresses school access for girls, one addresses community reading practices, one addresses the functionality of supporting school structures and one addresses parent and community engagement. (AMEP, November 2016; midterm evaluation, 2015). 15 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV surpassed the fifth target by more than threefold (314 percent of target), a U.S. Government (USG) standard indicator to record the number of teaching and learning materials provided through government assistance. The production of additional materials (School-to-School International [STS] decodables and the alphabet posters) to the June 2018 data collection accounts for this increase. 4 The remaining target (Indicator 1) that was not attained is the USG standard indicator for reading fluency and the only outcome indicator for reading performance. The target identified that on an early grade reading assessment (EGRA), “at least 50 percent of CP5 (grade 2) students 1) read a text of 40 1-3 syllable words within a minute without making more than three mistakes, and 2) correctly respond to two out of three simple questions about the text6.” Findings indicate that several factors have influenced the lack of EGRA results. First, the benchmark itself is questionable, given that its establishment did not follow a typical rigorous and participatory process (STS EGRA 2017 report). In an interview, a USAID Washington Literacy and Reading Team staff member also questioned the accuracy of this fluency benchmark. Mathematica identified 50 words per minute as its benchmark and Concern Worldwide Niger used 45 correct words per minute for its EGRA conducted in 2017. Secondly, NECS EGRA results are problematic. Only one child (0.2 percent) met the benchmark during the 2017 EGRA and complications undermine the reliability of data collected (STS 2017 EGRA Report). Some confusion exists over responsibilities for the EGRA 2017 and STS finally supported the EGRA application from a distance (Interview with M&E specialist; interview with STS). STS directly oversaw the EGRA 2018 application and while the report had not been released at the time of writing, correspondence with STS reveals additional concerns about the most recent data collection due to leakage (STS statement for MSI). Comparative reviews of other reading assessments in Niger also indicate low scores where available (see Annex XIV), including some zero scores for reading performance, indicating floor effects (PASEC2014, Mathematica 2015, Mathematica, 2017). The PASEC2014 data shows that less than 10 percent of CP students in Niger are above the minimal threshold for a composite score of oral comprehension, decoding and written comprehension. Baseline data for NECS (Mathematica, 2015) collected in the fall of 2013 found that “reading skills for all four languages, however, were very low and nearly indistinguishable from 0” (p. 100). An interview with the former USAID reading goal lead suggested that early childhood assessments may be more appropriate and they would focus on other skills, including concepts of print7 and early EGRA. Evaluation participants identified challenges to improved student performance in reading, including environmental and program-level factors. Environmental factors include high student and teacher 4 The NECS M&E specialist said this number accounts for 7,719 CI teaching and learning materials in year 1, 5,241 CP materials as well as replacement CI materials in year 2; and supplementary 16,210 student decodables books and 377 alphabet posters, a total of 29,547 teaching and learning materials (TLMs). This change was made during the USAID SRO M&E specialist’s data quality assessment visit in June 2018. 5 CP refers to Cours préparatoire (preparatory class) and is the equivalent of first grade. 6 Source: USAID, MEP, MCC. NECS (n.d.). Exhibit 2: Objectifs de la lecture et grille de progression CI-CP. 7 Concepts of print refers to skills such as knowing the directionality of print (in English in French: left to right, then up to down), purpose of punctuation, correspondence of illustrations with textual meaning, and the notion of letters and words. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 16 absences (Mathematica 2015, STS 2017 EGRA report), strong presence of contractual teachers, 8 recurrent teacher strikes, and the 2017 contractual teacher assessment that led to multiple teacher dismissals. 9 Delayed program rollout, with the first full CI/CP NECS cohort established during the 2017- 2018 school year, also complicates reading results (Interview, NECS interim chief of party [COP]; indicator tracking tables 2017 and 2018; several interviews with school directors). “NECS, through its intervention, has allowed for great collaboration between community organizations and the school. Members of the community visit us at least three times a week. NECS arrival within our community has awakened these organizations and they now know what’s expected of them. Parents are now motivated to build temporary school classrooms, to develop actions plans and to prepare for the beginning of the school year in advance. They built three teacher’s housing units. They’ve already achieved 96 percent of their action plan. … They are certainly motivated.” — School director, Tillabery TABLE 2: PROJECT TARGET ACHIEVEMENT # INDICATOR TARGET LOP ACTUAL (JUNE 2017) % LOP TARGET (JUNE 2017) LOP ACTUAL (JUNE 2018) % LOP TARGET (JUNE 2018) SO1 1 % learners: reading fluency/grade-level text comprehension (CP) 50% of students 0% 0% TBD TBD 2 # of learners in primary schools 42,500 students 39,790 95% 41,229 97% 3 # of learners reached in reading programs 15,100 students 14,746 98% 15,141 100% 4 # of primary school educators who complete professional development activities on implementing reading instruction 468 teachers 81 17% 522 112% 5 # of primary textbooks and other teaching and learning materials provided with USG assistance 9,400 TLMs 7,719 82% 29,547 314% 6 # primary school classrooms that receive a complete set of essential reading instructional materials 353 classrooms 163 46% 378 107% 8 According to the MEP 2011 National Assessment, recruitment of contractual teachers took place on a massive scale, increasing from 17 percent in 2000-2001 to 82 percent in 2010 (p. 18). 9 In 2017, the MEP organized a country-wide assessment of contractual teachers that required teachers to answer typical basic education content questions, i.e., content that their students were expected to master. Results were devastating for many teachers, and for the system, resulting in a large number of teachers, including NECS teachers to lose their jobs. Those with low scores but not low enough to end their contracts, underwent remediation courses in order to remain within the system. The exact number of teachers who had their contracts terminated was not available despite the evaluation team’s efforts. 17 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV TABLE 2: PROJECT TARGET ACHIEVEMENT # INDICATOR TARGET LOP ACTUAL (JUNE 2017) % LOP TARGET (JUNE 2017) LOP ACTUAL (JUNE 2018) % LOP TARGET (JUNE 2018) 7 % of active reading communities (village library, action plan – reading integrated, 4 public reading sessions) 90% of communities 63% 70% 99% 110% SO2 8 Proportion of girls enrolled in the NECS+ project intervention schools in grade 1 50% of schools 46% 92% 48% 96% 9 % of schools meeting the NECS minimum standard 90% of schools 31% 35% 100% 111% 10 # of parent-teacher associations (PTAs) or community governance structures engaged in primary or secondary education supported with USG assistance 140 PTAs 71 51% 148 106% REVIEW OF THE MONITORING AND EVALUATION (M&E) SYSTEM Discussions with participants revealed both strengths and weaknesses within the NECS M&E system. Weaknesses include the lack of a comprehensive activity-level database and significant turnover in the M&E specialist position10 (interview, M&E specialist; focus group discussion [FGD] NECS M&E specialist, NECS interim COP). An absence of professional development also seems to be an issue, with little recent support from Plan Niger, Plan USA and USAID11 (interview, M&E Plan Niger; NECS M&E specialist; NECS finance director). Multiple stakeholders, including NECS staff (FGD) and a former AeA Education coordinator identified the reliance on school and regional-level data collectors as both a strength and a weakness of the M&E system. This dependence on state-supported stakeholders12 is seen as promoting ownership of project activities; it increased during the NECS extension, which saw the phasing-out of nearly all of the NECS education coordinators, who had been responsible for coordinating monitoring responsibilities (interview with community mobilization specialist; interview with Aide et Action education coordinator; interview with NECS M&E specialist). A challenge that the NECS team overcame in the last year of the extension was assuring regular reporting from community organizations (2018 Indicators Tracking Table (ITT)), often complicated by weak literacy. Nonetheless, school-level discussions reveal a level of dissatisfaction with regular monitoring demands from about a third of teachers, and directors espoused 10 The M&E specialist at the time of the evaluation was the fourth person in that position since the beginning of NECS in 2012. He had originally worked with the project as an associate and returned to the project as M&E specialist in February 2017. 11 After data collection, the USAID SRO M&E specialist spent time with the team at the end of June for an annual data quality assurance (DQA) visit. 12 This system involves monthly reports from school management committees contact persons (CGDES), pedagogical advisors, and quarterly visits from inspectors. School directors submit annual reports (Interview, NECS Community Mobilization specialist; Interview, M&E Plan Niger; interview, Aide et Action education coordinator). USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 18 dissatisfaction and feelings of constraint by the NECS monitoring system. Other teachers see reporting as a regular responsibility. “They [NECS] are bothersome. Every week they ask us to send our weekly lesson plan. The project doesn’t trust us. They catch you for the smallest things ‘Last week, weren’t you on lesson 18? Why haven’t you finished it yet?’” — School director, Zinder CONCLUSIONS Based on the strategic objectives and target results, the NECS activity has met the majority of its objectives, making remarkable progress in its last year. This late achievement is also highly problematic after six years of program implementation and only finally establishing a program cohort in 2017-2018. Due largely to the establishment of the 2011 USAID Education Policy with a focus on reading, the project evolved from an access-focused to a reading-focused program. Despite the technical change in direction, there was no review of whether or not a new solicitation was warranted. A severe lack of results for Indicator 1 (reading fluency) remains significant, despite the cost of EGRAs. While low scores are not surprising, given other reading scores in similar neighboring countries (e.g., Mali and Senegal) and the complexity of the Nigerien context, the absence of reliable comparative data across all of the assessments suggests a grave misunderstanding of assessment requirements and perhaps mismanagement. The identified benchmark requires revisions to be more appropriate for Nigerien contexts. Moreover, floor effects of fluency sub-tasks results suggest the EGRA tool and, in particular, the emphasis on fluency measures may not be appropriate for discerning student performance progress. By June 2018, the NECS activity seems to have overcome many of the weaknesses within its M&E system. Nonetheless, present M&E configurations largely support reporting rather than learning and evaluative thinking, the latter of which is critically important in complex environments. While NECS is moving away from education coordinator roles and passing responsibilities to MEP officials, which is a laudable approach for a program nearing its end, regional and school-level actors are not fully ready to take on this responsibility. Doing so may further reduce the quality of monitoring data and promote project burnout among school directors. In an extension, the program should reconsider how to provide adequate communication and technical support for school and regional-level monitoring. EVALUATION QUESTION 2 EQ2: How did the NECS “ASL” approach enhance reading outcomes in NECS Schools? This section addresses how the NECS program may have contributed to improving students’ abilities in reading at NECS schools. The evaluation team drew extensively from school-level data, as well as interviews with regional and central-level MEP officials, project staff and document reviews to answer this question. 19 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS OVERVIEW OF ASL The Apprentissage systemmatique de lecture (ASL) is a new methodology in Niger that integrates several basic reading skills: sound-letter correspondence, decoding, oral and written comprehension and the “whole language” method for transitioning into French. ASL establishes a scope and sequence for reading in grades 1 and 2 and prescribes a systematic, step-by-step process through scripted lessons. In addition, instruction incorporates innovative and playful activities13. In first grade (CI), learning begins with the alphabet and the method stresses pre-reading activities. In second grade (CP), learning begins with a text and focuses on reading comprehension, understanding a text that’s read aloud and mini￾assessments at the end of each session. ASL implements four local languages of instruction. Students transition from the local language to French at the end of CP (between lessons 75 and 120). An interview with NECS senior staff indicate that while this was originally planned for year 3, the decision was made to begin with French in year 2 due to absence in materials for CP. READING PERFORMANCE In the absence of reliable EGRA fluency results, the evaluation sought to understand how ASL may have enhanced other reading outcomes. Findings from the 2015 comparative study of innovative projects in Niger demonstrate that NECS schools had the highest results in terms of letter recognition14 (see Annex XII). The Mathematica 2016 impact evaluation also shows gains in letter recognition, although those levels are still low. As part of the final evaluation, the team devised a simple, non-diagnostic exercise for CI and CP students that it applied in groups of six students per school. The exercise, conducted with 120 students (60 girls and 60 boys), showed that between half and three-quarters of students were able to read letters and syllables, with the same tendencies for girls and boys. At the same time, less than half of students were able to read words and simple sentences. Classroom observations also demonstrated that in all classrooms, students actively read from the board without prompting. Testimonies from all stakeholders at the school, regional and central levels indicate that the NECS program is making strides in reading gains. Almost all teachers and school directors, pedagogical advisors, inspectors and departmental as well as regional directors interviewed identified that students are more enthusiastic for reading and agreed that students are learning to read better and more quickly with ASL. At a handful of schools, including a bilingual school, teachers and directors insisted that ASL is the best current approach within the Nigerien school systems, an observation supported by discussions with the majority of individuals tasked with teacher support (inspectors, Regional Primary Education Directorates (DREPs ; Directeurs régionales de l’éducation primaire) and Departmental Directors of Primary Education Direction (DDEP ; Directeurs départementaux de l’éducation primaire). Discussions 13 Activities include the alphabet song, “duck lips” (where a speaker enunciates a word while firmly holding one’s lips closed with both hands, allowing the speaker to “feel” the syllables), stamping (banging one’s fist on the blackboard to identify syllables, name box (randomly drawing the names of students on slips of paper from a box), syllable squares (a grid on the board in which to divide up the syllables of a word) and check-in signs (thumbs up, down, etc. to indicate non-verbal responses). 14 Scores: NECS: 61.7 percent; ELAN: 56.3 percent; CONCERN: 47.9 percent; traditional schools: 21.8 percent; bilingual traditional schools: 21.6 percent. A limitation of this study is that ASL had been present in the school for only two months prior to data collection. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 20 with stakeholders at nearly half of the schools that the team visited identified the alphabet song, “duck lips” and stamping methods as highly effective, a point of agreement for many central-level actors as well. Additionally, conversations with NECS and Plan Niger leadership, as well as USAID, revealed that NECS’s demonstrations at both a UNICEF forum on best practices and the March 2018 NECS Steering Group meeting led to the minister’s enhanced interest in scaling up certain NECS initiatives, including ASL, in all MEP schools. 15 “Already, it’s a bilingual school, although ASL has distinctive characteristics. Its methodology stimulates learning, such as the alphabet song, sounds and syllables, words. This is what has led to a good number of students reading.” — Teacher, bilingual school COMPONENTS OF ASL APPROACH AND SUPPORT ASL TEACHING AND LEARNING MATERIALS NECS has developed and provided teaching and learning materials (see narrative in the EQ1 section) that included the CI and CP student books and teachers’ guides, as well as a new CP decodables book for students to take home. Nearly half of school-level stakeholders commented on the importance of these materials to improve the teaching and learning environment within schools. Some stakeholders, particularly teachers, school directors, Primary Education Inspectors (Inspecteurs de l’enseignement primaire, IEPs) and members of the ASL committee said the content was contextualized to children’s known home and school environments. At the same time, several findings point to deficiencies in TLMs. Despite the guides being developed with ministry personnel and approved by MEP, the most repeated complaint from all levels of stakeholders is that the CP teachers’ guide is too thick, at 471 pages for the Hausa version. Teachers and directors at half of the schools visited also shared this view, with a school director in Maradi saying that the guide “is pretty frightening.” NECS has taken this feedback into account and was in the process of reducing the size of both CI and CP materials at the time of data collection. A review of the revised CP draft teachers’ guide reveals that a reduced number of pages, but the font is also smaller and several helpful textboxes have been removed. Other limitations of TLMs include that Plan USA, some NECS staff, a school director and several MEP and USAID officials identified a lack of images in the CI books. However, the former and current USAID/Washington reading leads both argued that contact with text was more important and said that due to space constraints on the page, images would reduce contact with text. These comments also align with Readsters’ interview data, but STS curriculum developers did not fully agree, so CP materials include illustrations. Some teachers and school directors identified typographical errors in TLMs, as well as misalignment in references between resources. Teachers and directors at more than a quarter of schools visited expressed that lessons and texts are too long for students. Similarly, directors and 15 Interviews with individuals including NECS staff, Plan Niger country representative and the MEP secretary general and document review show that other initiatives include the mentoring program, code of conduct, gender-sensitive monitoring tools and community engagement practices (minutes from March 2018 Steering Group meeting). 21 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV teachers expressed concerns that certain passages or words in local languages were not comprehensible within all regions. “ASL’s strengths are that kids read and understand what they are reading, for instance, CP students can read posters in the village. … At this time, I don’t think there’s a better learning approach than ASL. With ASL, CI/CP students read fluently and expressively. They read texts that CM16 students can’t. They have a reading book at home and are supported by literate parents.” — Interview, school director, Maradi) SCRIPTED LESSONS Interviews with various stakeholders at the school, regional and central levels revealed a debate concerning the beneficial or constraining nature of scripted lessons. NECS activity designers, namely reading subcontractors Readsters and STS that scripted lessons were a deliberate and strategic choice for teachers with little to no formal training. The former USAID reading specialist underlined that scripted lessons remain a best practice. Similarly, the International Rescue Committee (IRC) Healing Classrooms program in Niger17 also uses scripted lessons but calls them “guided lessons” and highlights the various choices teachers have in lesson delivery (source: interviews). Additionally, according to FGDs with Readsters the ASL committee, in an environment where teachers may not be familiar with reading and writing in local languages and where standardization of languages across various regional registries continues, a scripted lesson in local languages can be useful and contribute to language standardization. Both Readsters and STS experts agreed that once teachers had mastered the routine of delivering the reading lesson, they would no longer need to depend so strongly on the scripts. “The teacher delivers the lessons and the students understand it very well. It’s a great collaboration. You can’t go wrong teaching a lesson because you have the guide in your hands.” — FGD with teachers, Zinder On the other hand, many participants, including some school directors, regional education authorities and central MEP officials, see the scripted lessons negatively, saying that they dull teachers’ creativity and negate the importance of proper lesson preparation. Teacher preparation of lessons is an expectation of a good teacher within the Nigerien education system (interview with Aide et Action (AeA) education coordinator). Several participants mistakenly argued that scripted lessons free teachers from lesson preparation; most notably, these included a few school directors, pedagogical advisors and at least one MEP official serving on the ASL committee. At the central level, at least three high-level officials in the MEP spoke strongly against the scripted nature of NECS lessons; one of them described the guide as “controlling.” A pedagogical advisor in Tahoua said the scripts make teachers “lazy.” At the school level, several teachers and directors also commented negatively. One teacher in the Maradi Region complained that he felt like a “prisoner” of the ASL teacher guide. 16 CM stands for Cours Moyen, or the sub-section within the primary education structure that equate to 5th and 6th grades. 17 The IRC Healing Classrooms program began in 2017 and is an afterschool program for 2nd through 4th graders focusing on reading and math as well as social-emotional learning. The program includes teacher training as well as other components. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 22 “The teacher comes, picks up the script, ‘Today, we’re going to have such and such a lesson,’ and he starts to read. He is looking at it and needs to do so until the end of the lesson. It’s not within our teaching philosophy. Our teaching beliefs require that we prepare our lessons.” — Technical advisor to the minister USE OF NATIONAL LANGUAGES While ASL does not require using a local language for instruction, the way that ASL has been implemented within Niger marries ASL with the use of local languages as media of instruction. Several comments related to students’ reading abilities involved the use of local languages. ASL also aligns with the MEP Curriculum Reform initiative that aims to transform traditional schools into bilingual schools country-wide by 2024. Within the 20 schools visited, interviews with 5 directors and with 10 teachers (or sets of teachers, depending upon the school) across all four regions sampled identified the use of local languages as promoting student understanding, retention of material and rapidity of acquiring literacy. One interviewee identified that children are more confident in class, while another described teaching and learning in local languages as comparable to creating a family atmosphere (“en situation de famille”). Nearly half of all teachers were satisfied that NECS allowed them to learn to read and write in their own languages, and some identified this as a motivating factor. At the same time, significant systemic challenges emerged from assigning teachers to areas where the teacher’s language and the local language did not match. This was the case for at least two teachers the evaluation team observed. In one school in Zinder, teachers recounted that the choice of Kanuri as the language of instruction has pushed some families to move their children elsewhere and has led to poor community participation. An interview with the MEP director of curriculum and the promotion of national languages also identified this as an issue for the State’s bilingual education reform and indicated that a mapping study has been commissioned for summer 2018. TEACHER TRAINING AND TECHNICAL ASSISTANCE Discussions with NECS implementers confirmed that training for teachers and school directors follows a cascade training model (training of trainers), allowing for language-specific sessions and involving various stakeholders. Education coordinators, the ASL committee and reading experts (Readsters/STS) train inspectors and pedagogical advisors. The inspectors and pedagogical advisors, with the support of the education coordinators and ASL committee members, then train school directors and teachers. Many pedagogical advisors indicated that the presence of a reading expert was important to the overall quality of the training. A couple of MEP officials shared during interviews that teachers were satisfied with training and ongoing technical assistance that NECS provides and that these components are strengths of the NECS activity. The NECS trainings focus specifically on the skills and content needed to support reading instruction. In almost half of the schools visited, teachers or directors identified that NECS trainings allow for teachers to improve their skills and performance, which may positive affect students’ skills. Regular technical assistance was also a positive attribute of NECS for some, and participants noted that ongoing contact with pedagogical directors allowed teachers to learn new strategies and motivated them to improve their performance. NECS funding allows regional educational personnel to visit schools, whereas typically, resources for fuel are lacking. A handful of teachers and school directors also identified teacher learning circles and NECS-supported teacher exchanges as great professional development 23 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV opportunities. While discussions with NECS staff emphasized the existence of alternative systems for teachers who missed formal trainings, interviews with teachers in Tillabery and a director in Zinder indicated that some individuals still did not benefit from trainings. In terms of sought improvements, exchanges inform that a few teachers and some pedagogical advisors find the length of training insufficient. 18 At a quarter of schools sampled, teachers and school directors identified trainings to be either delayed or scheduled at inconvenient times. They suggested that trainings take place during school vacations rather than at the beginning of the school year. Perhaps more importantly, while the majority of participants overwhelmingly identified positive attributes of NECS trainings and ongoing technical assistance, a couple of voices identified frustration with “nonstop monitoring” and burdensome reporting requirements. A senior MEP official identified the need for monitoring instruments to be streamlined. In addition, teachers and school directors at two of the four bilingual schools identified feeling overwhelmed in teaching both ASL and the Reform school’s Aprentissage par la situation (APS, Situational learning) approaches. They suggested more guidance on how to integrate them within their teaching. CHALLENGES WITH PROGRAM IMPLEMENTATION NECS began in 2012, but TLMs for both CI and CP became available to teachers in 2017-2018 that. The first cohort of students experienced ASL in both CI and CP during the 2017-2018 school year. Interviews with NECS, Plan International and USAID personnel reveal several reasons for delays, including three changes in reading subcontractors and new methodologies: Apprentissage rapide en lecture (ARL) with Voluntaires de l’intégration educative (Volunteers for Educational Induction; VIE), ASL for CI with Readsters, ASL for CP with STS. Gaps occurred between reading subcontractors from January through November 2014 and May through November 2016. Significant turnover also occurred at NECS, Plan USA and USAID. In addition to the Niger education system’s challenges, such as frequent teacher strikes, teachers with weak qualifications and a high level of student and teacher mobility, interviews and document review indicate that internal factors related to program management complicated NECS implementation throughout its life cycle. As responses to EQ1 demonstrate, the emphasis on reading evolved from 2012 and, according to conversations with Plan USA, Plan originally submitted its proposal based on its girls’ education and community work expertise. Plan relied on the expertise of its reading subcontractors. Data indicate that there was no consistent reading support for the program and no reading specialist available to provide technical support and oversight of the program outside of the reading subcontractors, i.e., within NECS, Plan Niger and Plan USA, as well as within USAID/Niger, USAID/SRO and USAID/Washington. While the program benefited from regular educational support, this support came from generalists rather than reading specialists. Education generalists certainly offer much to the project but typically have a broad knowledge of educational issues rather than the in-depth knowledge of reading acquisition, approaches and challenges that reading experts possess. 18 In the past, teacher training lasted three days, but it has been extended to five days. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 24 CONCLUSIONS Despite the complicated operating environment for schools in Niger and the implementation challenges that NECS encountered throughout its life cycle, reports from stakeholders consistently point to ASL improving students’ reading abilities. The lack of rigorous and recognizable reading performance data weakens arguments for ASL’s success in enhancing reading outcomes. Support for teachers, teacher training and the development of teaching and learning materials have been largely successful in allowing teachers to understand and implement ASL’s step-by-step approach to reading. While some teachers and school directors find ongoing technical assistance to be overwhelming, such support provides value added to the system. Ongoing refinement and clarifying roles and expectations may reinforce the practice. The systematic nature of the ASL method is a fundamental element of good and effective reading instruction. Key aspects include the scope and sequence of content, the step-by-step process and regular student assessment at the end of each lesson ASL’s use of national languages attracts children to reading, motivates many teachers and creates a bridge for parents to participate more in their children’s education. Nonetheless, major challenges related to matching teachers’ mastered language with the language they teach may be a risk for ASL’s effectiveness but is also a system-wide problem. Perhaps more importantly, the transition to French at the end of year 2 is early in comparison to early exit programs in other countries. Even in the curriculum reform schools in Niger, students begin with French as the language of instruction in year 3. The lack of reading expertise at both Plan International and USAID available to support the project and provide technical supervision seem to have contributed to the limitations in the implementation, daily management and supervision of ASL activities. The reading approach that NECS developed was not strongly articulated or communicated. This has led to inconsistencies between CI and CP materials, as well as differences in opinion on the quality of texts developed and factors such as the benefits and correct application of scripted lessons, the appropriate length of teachers’ guides and the appropriate level of illustrations within TLMs. In the end, key decision￾makers and implementers from Readsters and STS, along with USAID/SRO, USAID/Niger, Plan USA, Plan Niger and NECS were not all on the same page, making it impossible to clearly communicate the purpose and underlying philosophy of ASL to MEP counterparts, inhibiting buy-in among key stakeholders. EVALUATION QUESTION 3 EQ3: In terms of scalability, what are the comparative advantages of the ASL approach as compared to the other reading approaches in Niger? To answer this question, the evaluation team drew from key informant interviews (KIIs), specifically with individuals from MEP, NECS, Readsters and STS, as well as other reading program implementers in Niger. Close document analysis of teaching and learning materials also provided critical insights. Given MSI’s extensive experience investigating scaling-up practices and offering trainings for practitioners, the evaluation team facilitated an interactive discussion to explore a possible scaling-up of ASL using the MSI Scaling-Up Checklist (see Annex IX for the checklist and a briefing document from the work session.) 25 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV This section provides a brief overview of reading approaches within Niger, an in-depth comparison of these approaches and a synthesis of key stakeholder perspectives on scaling up ASL. Identification of other reading approaches relies on stakeholder recommendations, namely USAID/Niger, NECS staff and MEP officials. Based on these recommendations, the evaluation team focused primarily on comparing the various NECS approaches with two other approaches: 1) the APS approach in use within the MEP’s curriculum reform bilingual schools and, 2) the International Organization of La Francophonie’s Ecoles et languages nationales en Afrique (ELAN) initiative. 19 The team also spoke with implementers at Save the Children and the IRC, which both operate programs in Niger. After analysis, program models differed too greatly and are not included here. FINDINGS Throughout Niger, the Ministry of Primary Education and a handful of non-governmental organizations are implementing several new approaches to reading as a way to improve students’ reading skills. Among them are ASL (NECS), ELAN and APS (i.e., la Réforme). Interviews with stakeholders revealed a sense of competition amongst implementers to develop the reading approach that will be the most effective and thereby, adopted by the Ministry for country-wide application. OVERVIEW OF APPROACHES These innovative programs all began as of 2013 and have slight variations in implementation. All but the Reform strategy operate within traditional Nigerien primary schools, while the Reform schools, by nature, create a new type of school that offers bilingual instruction. They also operate on different scales, with ELAN operating the fewest schools (10 in total) and the Reform operating with the most schools (500 schools and, beginning with the 2018-2019 school year, extending to 5,000). In addition, ELAN schools work selectively with teachers who already have Hausa language training. The other approaches work with teachers who already have been assigned to the schools irrelevant of local language abilities or training. (See Annex XII for additional comparison of implementation elements). INCONSISTENCIES WITHIN THE NECS READING APPROACHES A clear definition of the ASL approach is necessary to respond to this evaluation question. However, three changes of reading subcontractors characterized the evolution of the NECS reading approach. The first NECS approach was the Approche rapide de lecture (ARL), which took effect in schools in 2013. ARL was originally a strategy for out-of-school adolescents and did not last the full school year. Discussions with MEP officials demonstrated a high level of dissatisfaction with this approach. Next, Readsters￾developed ASL and CI materials entered classrooms first in March 2015. ASL was originally conceived as an “explicit structured systematic early reading instruction that is research based and generally considered for K-3 or struggling readers of any age” (source: interview with Readsters). By design, the emphasis is on decoding and spelling patterns rather than meaning patterns looking to establish automaticity of decoding with students who likely have limited access to reading materials at home. Next, STS colleagues developed CP materials with a commitment to continue with the ASL method. STS curriculum experts expressed criticism for the CI approach as too phonics-based, preferring a hybrid 19 Concern is not mentioned here because it uses the same reading approach as ELAN (ARL). USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 26 between phonics and a whole-language approach and an overall more child-friendly approach. Discussions with MEP and some USAID/Niger officials also reveal similar concerns for wanting a child￾friendly book with illustrations. CI materials begin with structured study of the alphabet and CP materials begin with studying texts. COMPARISON OF READING APPROACHES Drawing on current research suggesting that comprehensive reading programs balance or integrate a phonics-based approach and a whole-language approach to reading instruction, 20 the evaluation team investigated the extent to which existing reading programs integrate systematic and explicit instruction of five core reading skills: 1. Phonemic awareness is the ability to hear and manipulate sound in words. 2. Phonics, or the Alphabetic Principle, is the ability to associate sounds with letters and use these sounds to read words. 3. Reading fluency is the ability to read words in isolation or in connected text with automaticity, accuracy and expression. 4. Vocabulary development is the ability to understand (receptive) and use (expressive) words to acquire and convey meaning. 5. Reading comprehension is the ability of readers to extract meaning from text. In addition to these five core reading skills, young readers should develop listening comprehension skills and writing skills. Table 3 provides a comparison of the reading programs currently in use in Niger. It includes contextual elements (type of school, language of reading instruction, grade levels), evidence of systematic and explicit instruction of the five core reading skills, as well as listening comprehension and writing skills, as discussed above, and other variables that are generally accepted as important elements of a comprehensive reading program. TABLE 3 CORE READING SKILLS COMPARISON ACROSS READING PROGRAMS READING PROGRAM ASL ASL ARL (VIE) APS ELAN Type of School Bilingual + Traditional Bilingual + Traditional Traditional Bilingual Traditional Languages Local Local Local Local + French Local Grade Levels CI CP CI CI – CM2 CI - CP Phonemic Awareness • • • • • Phonics/Decoding • • • • • 20 A phonics-based approach focuses on word identification and decoding words, while a whole-language approach focuses on comprehension and meaning. As such, in the lower grades, effective reading instruction includes targeted teaching of phonemic awareness, phonics or decoding, fluency, vocabulary and reading comprehension strategies. 27 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV TABLE 3 CORE READING SKILLS COMPARISON ACROSS READING PROGRAMS Reading Fluency • • • • • Vocabulary • • • Reading Comprehension • • Listening Comprehension • • • • Writing/Spelling • • • Leveled or Decodable Texts Embedded Assessment • • Suggested Scope and Sequence • • • • Based on a review of curricular materials as well as interview data, the table depicts that all innovative approaches target letter recognition, phonemic awareness, phonics/decoding and reading fluency skills. A difference among programs is that ASL begins with teaching sound-letter correspondence and decoding to read words and sentences and, finally, reach comprehension. APS and ELAN, on the other hand, start by reading sentences and words to discover sounds, letters and syllables. Additionally, ASL features a systematic standardized step-by-step teaching/learning process, whereas the exact process APS and ELAN teachers follow varies among teachers. As indicated in the response to EQ1, of all of these approaches, only Concern is able to demonstrate positive EGRA results. COMPARATIVE STRENGTHS AND WEAKNESSES OF ASL FOR SCALING UP In discussing possibilities for scaling up ASL nationally, there was general recognition that ASL elicits interest among students and communities in reading and that students can learn to read more quickly with ASL. The majority of stakeholders find the approach to be credible, well-regarded and well-adapted to Niger’s realities. In addition, the minister’s interest in ASL and the progress underway to identify best practices and how to successfully integrate them into the larger MEP system was another point that several participants indicated as a good condition for scaling up. Inclusion of ASL within the Minister’s request for a stocktaking report provides additional evidence of the Minster’s interest in the reading approach. ASL’s inspiration for the country-wide current Emergency Program (Programme d’urgence) targeting reading and math performance within at least one region similarly suggests possibilities for scaling-up. At the same time, many central-level MEP officials and NECS staff recognized that to meet scaling-up conditions, political will is necessary. Indeed, three topics generated the most discussion among participants during the ASL Scalability Work Session: political will, funding and the lack of rigorous performance results needed to justify scaling up one project or some of its components. In addition, an element that currently complicates the scaling up of NECS is that while NECS has developed strong collaboration with MEP offices and officials from the central level to the school level, the project still enjoys external funds, operates its own staff and is not solely dependent upon the ministry. These USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 28 factors would make scaling up expensive and difficult. Stakeholders also identified important concerns regarding how ASL might be integrated without hampering other existing reading programs. CONCLUSIONS The current environment of reading programs operating in Niger is highly competitive. Programs need guidance urgently from the ministry on the reading programs it wishes to support moving forward. The current situation of multiple implementing partners running experimental programs is unsustainable. The NECS reading project employed three approaches during its lifecycle: ARL with VIE, ASL (CI) with Readsters and, finally, ASL (CP) with STS. While the latter two approaches have more in common than not, a distinction exists between the two philosophically, mimicking the phonics versus whole-language debate that continues to rage in international reading circles. Given Plan and NECS staff’s lack of understanding of these differences, they have gone unchecked during implementation. A clear articulation of the NECS reading approach would be critical for a possible extension and even more important for scaling up. Of the reading programs the evaluation team reviewed, the ASL reading program in CP stands out as the most comprehensive reading program currently being implemented in classrooms in Niger. It targets all five core reading skills, plus listening comprehension and writing skills, includes regular assessments of student performance in reading and a scope and sequence which provides information on the “systematic” design of the program, its organization and logical sequence. Moreover, given the components of other reading programs, ASL could serve as a supplement to the APS approach within Reform schools. ASL’s clear systematic instructions for teachers would offer value added to the current primary education system. Scaling up is a complicated process and few scaling-up initiatives actually achieve their goals. To improve chances for a successful scale-up, the MEP will need to address the various weaknesses identified during this evaluation, including solid documentation of program results, identifying necessary political will and resources for scaling up. Until these conditions exist, scaling up will be difficult. In addition to NECS ASL, USAID may wish to investigate the Save the Children Literacy Boost initiative in Niger along, with the IRC Healing Classrooms project in Diffa. Both offer promising models that encourage teachers and engage the community. While full analysis was beyond the scope of this report, it’s likely that both Save and the IRC incorporate many of the key reading skills identified above within their programming. 29 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV EVALUATION QUESTION 4 EQ4: What types of changes are visible in NECS schools and communities? Responses to this evaluation question rely upon data collected through interviews, school and classroom observation, and document review. In addition, findings complement those related to both evaluation questions 2 and 5. FINDINGS CHANGES AT THE SCHOOL LEVEL This section builds on findings under EQ2 that focus on heightened interest in reading among students, and the development and distribution of TLMs (including posters). These findings identified improved relationships among key stakeholders. First, with NECS support, regional MEP representatives are able to conduct school visits, something that had not been possible before. The visits tend to be frequent and often take place in tandem with a variety of MEP representatives. While MEP representatives find the visits to be positive, as indicated in the discussion under EQ2, some school-level actors feel surveilled and that the regular monitoring and technical assistance has a negative impact. Secondly, school, regional and central-level stakeholders said the relationship between students and teachers has improved through NECS initiatives. Classroom observations indicate a trusting and dynamic relationship between teachers and students for under half of lessons attended. SCHOOL GOVERNMENTS Another visible change associated with the NECS activity is the establishment and reinforcement of school governments21 in 158 schools. 22 Based on interviews, the majority of MEP inspectors and school directors recognize that school governments function better with NECS’ support. These same stakeholders identified many school governments’ active participation in school management, the development of action plans and the organization of school celebrations and sports events as important activities. At the same time, school visits and focus groups with school government ministers demonstrated great variation in the efficacy of school governments from one school to the next. It seems that many are still not aware of expectations. For instance, at one school in Tahoua, the school government began during the 2017-2018 school year and students were unsure of their responsibilities. Similarly, discussions demonstrated that few governments benefit from intended teacher oversight, although teachers may feel differently. 21 The school government represents a group of students, both boys and girls, from grades 1 through 6 who hold various ministerial positions. Positions may include president, prime minister, education minister, water minister, labor minister, justice minister, financial minister, advisor, etc. School government responsibilities may include the development and implementation of action plans, monitoring student attendance, monitoring school discipline, organizing sketches to raise awareness of pressing issues and activities related to hygiene in both the school and the classroom. 22 Source: Presentation for the March 2018 Steering Group meeting. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 30 CHANGES AT THE COMMUNITY LEVEL Findings in this section focus on how NECS activities have affected community organizations, and parental and community attitudes and behaviors toward education. ENHANCING SCHOOL SUPPORT ORGANIZATIONS According to Plan Niger and NECS staff, the project sought to galvanize and reinvigorate community support structures to promote schooling. The emphasis on community mobilization was stronger during the first NECS phase, with greater focus on ASL during the extension phase (2016-2018). As of March 2018, the project had worked with 474 community organizations (PTAs, mothers’ associations and community school management committees [CGDES]). 23 “Support structures have never been so active!“ — School director, Zinder Discussions with senior MEP officials responsible for school management committees, as well as counterparts at the department-level and several CGDES presidents, confirmed the general functionality of CGDES within NECS schools. According to interviews, teachers and school management committees see the development and sustainability of action plans as innovative. Interviews with teachers, school management committees and inspectors identified CGDES integrated action plans that facilitate coordination among community organizations as very successful. Project data also noted that the average number of activities that school management committees execute at NECS schools is more than twice the average rate at non-NECS schools. 24 (See Figure 3.) Similarly, in interviews and discussions with inspectors, teachers, school directors, community members and MEP central officials, respondents speak to a remarkable change in women’s engagement in school-related meetings, even if it’s a minor shift. Interviews with community organizations, confirmed by teachers and school officials, revealed that many community support organizations meet more regularly and produce written minutes from meetings. At the same time, some MEP department-level representatives and inspectors observe certain weaknesses within school-related community organizations, due largely to misunderstandings of their roles and responsibilities, as well as limitations in producing written minutes from meetings. The high mobility of community members remains another challenge. “One of the weaknesses that I’ve noted during my supervision visits is that NECS should place more emphasis on training members of school support organizations, make them even more aware…As long as the structures themselves cannot write the agenda or produce minutes from their meetings, it will be difficult to really speak of any change.” — DDEP, Tahoua 23 Source: Presentation for the March 2018 Steering Group Meeting. 24 In 2015-2016: NECS school CGDES averaged 10.89 activities per year, twice the national average of 5.4. In 2016-2017, NECS schools averaged 13 activities compared to the national average of five, or 2.6 times greater. Source: Bilan des PAAQ à travers les PFD/CGDES et ANASAC (2016, 2017) 31 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV COMMUNITY PARTICIPATION AT SCHOOLS Analysis of data demonstrates that the various school-support community organizations contribute to NECS schools in multiple ways. Many are active and work closely with school directors and teachers. Interviews with these organizations and with school directors identified that relationships between the school and community have improved since NECS began its activities. While analysis indicates that community engagement in schools varies significantly from one locality to another, field visits indicated a reasonable level of community participation at almost all of the schools involved in the project. One school is an outlier, however. As indicated in the discussion under EQ2, community members at one school in Zinder are opposed to the selection of Kanuri as the language of instruction and have distanced themselves from the school. According to a senior MEP official, positive examples of community support for schooling include a school in Dosso, where groups of men from the community have organized to use their carts to fetch water for the school, freeing students (girls in particular) from the task. Similarly, a mothers’ association in Tahoua delivers water to the school for students on hot days. Several other focus group discussions with community organizations and interviews indicated that community organizations often oversee school maintenance and clean-up. A few mothers’ associations work with students to clean latrines, for example, and some school management committees clean schoolyards regularly. Communities also contribute to improved enrollment and school retention through several initiatives. Almost all stakeholders informed the evaluation team of the important role that school mothers’ associations and mother educators [les mères éducatrices] have played in organizing pre-enrollment campaigns, including door-to-door canvassing to discuss the importance of school enrollment with fellow mothers. School management committees, village chiefs and others also seem to play a key role in recruitment efforts. According to school directors, community school management committees (CGDES) and the department-level MEP representatives for the Upper Directorate for Community School Management Committees (DGCGDES), communities make substantial financial contributions to support NECS schools. They argue that these efforts are due in part to NECS awareness-raising activities. Many parents and organizations also identified in-kind contributions that parents make at the end of the harvest, coinciding with the beginning of the school year. External data support these claims, as the average funding figures for NECS school management committees out-fundraised other schools nationally by over 1.5 times in 2015-2016 and more than twofold in 2016-201725 (see Figure 3). “It’s always the NECS schools that have a better financial engagement during our department-level school management committee meetings.” — Pedagogical advisor, Maradi 25 2015 -2016 (NECS : 273,441 vs. 170,508 – national average) and two times (346,000 vs. 166,108, 2016-2017): Source: Bilan des PAAQ à travers les PFD/CGDES et ANASAC (2016, 2017) USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 32 FIGURE 2: COMPARISON OF ACTIVITIES EXECUTED26; FIGURE 3: COMPARISON OF AVERAGE FUNDRAISING Another way that many communities participate in their children’s education is through constructing buildings for the schools, most frequently classrooms and teachers’ residencies. As an illustration, the PTA and school management committees built five classrooms for a school in Tahoua while community members built six classrooms and perform regular upkeep for a school in Maradi. At five of 20 schools visited, school support organizations have constructed teacher housing. A senior MEP official indicated that building lodging for teachers helps communities ensure a teacher’s continued presence and contributions to the community. The evaluation team also noted nearly all visited schools had enclosures put in place by CGDES or school governments. In addition, discussions with school personnel, regional MEP officials and central-level MEP officials frequently mentioned that many NECS schools are able to prepare before the school year starts through constructing makeshift classrooms, cleaning schoolyards, etc. This advanced and organized preparation allows schools to begin classes on schedule. PROMOTING A LITERATE ENVIRONMENT Adult literacy efforts and related programming, specifically public reading sessions, as well as the production of written materials and public announcements are positive visible changes that adult literacy inspectors (IDAENF), school directors, department inspectors, IEPs and community members identify. Two adult literacy campaigns took place between 2012 and 2014, ending due to the first reading subcontractor’s mismanagement of funds. In the absence of formal literacy initiatives, many communities have continued to actively participate in “post-literacy” initiatives. MEP officials responsible for adult literacy spoke highly of NECS’s ability to support these initiatives. Visits and discussions within communities sampled confirm the existence of village libraries since 2014. As a result, some parents are borrowing books for their own reading as well as to read with children. Adult literacy inspectors commented that usage of village libraries varies widely among and within communities, and that women may borrow more than men. Participants indicate that community members produce written documents that are published and distributed. In addition, the majority of evaluation participants identified that 26 Source: Bilan des PAAQ à travers les PFD/CGDES et ANASAC (2016, 2017). 33 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV community members demonstrate great interest in public reading sessions. Nonetheless, data also indicate that the success of the public reading sessions varied from community to community. “It’s the fundamental problem in Niger. … Generally, there isn’t any support or technical supervision provided by communities for what happens at school. … Through our joint monitoring missions and exchanges with communities or, better yet, with parents at the NECS schools, we’ve noticed a difference. … There, school is beginning to interest communities.” — DREP, Tahoua PARENTS’ INCREASED ENGAGEMENT IN CHILDREN’S LEARNING Findings from data address ways that parents are more able to participate in their children’s learning through initiatives such as adult literacy opportunities and the NECS’s reading curriculum using local languages. During a focus group, the ASL committee recognized that NECS has provided a resource for parents that was not previously present. Because of these changes, almost all of the school directors and teachers at NECS schools, along with the department-level school management committee (CGDES) point people, recognize that community engagement has improved since the beginning of the project. The majority of IEPs and pedagogical advisors recognize ASL as a “link” between the school and home. The effects of parents’ heightened engagement have led to them monitoring their children’s schoolwork and having a greater presence within the school. “Parents take more and more of an interest in school. … They see that their children are more comfortable and that they read at home, and that does them good.” — IEP, Tahoua MENTORING PROGRAM As of March 2018, 3,036 students — 51 percent girls, and 20 percent of total NECS students — participated in the mentoring program. The program also trained 262 teachers (40 percent female) and 564 community members (43 percent female), a total of 826 individuals, in mentoring. 27 Although mentors receive training and ongoing support, by design, mentors do not receive financial remuneration. A number of stakeholders, including nearly half of school-level interviewees, identified the mentoring program as a positive change within communities. Inspectors, school directors, teachers, pedagogical advisors and MEP central-level officials all identified that, as a result of mentoring, at-risk students improve their school performance. Some teachers, directors and NECS staff explained that the mentoring allowed some children who had left school to continue their studies. At the same time, the mentoring program encountered significant difficulties and as a few participants indicated, it “slowed down” when a number of mentors resigned when they realized they would not be paid. Various NECS staff indicated having made clear that mentorship was voluntary but it does not seem to have been understood by all mentors. Moreover, almost all of the DDEPs, IEPs, pedagogical advisors and central-level MEP officials indicated a strong desire for NECS to identify a solution. Another challenge seems to be that economic and agricultural demands limit the abilities for many 27 Source: NECS PowerPoint for March 2018 Steering Group meeting. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 34 mentors to be present during the entire school year. A senior MEP official suggested working with retirees as a possible solution. Another MEP official offered other symbolic and non-monetary suggestions, such as public recognition within the community, for instance, during school ceremonies, or for the community to provide in-kind contributions, such as tending their fields or providing agricultural products to the mentors. “In my class, three students had gotten into the habit of not concentrating well during lessons, but thanks to their mentors, we learned that these kids spent a lot of their time in houses in the village with TVs and that they returned home late at night. It was the mentor who spoke with these children’s parents to obtain this information and to insist [that] parents … no longer allow them to do so.” — Teacher, Zinder CONCLUSIONS Community-level efforts to improve community awareness and support for education have largely been successful for NECS throughout the program’s six years of functioning. While results are uneven and challenges remain within certain communities, positive changes at the school level that support both a more active and engaged reading experience as well as enhanced engagement of school support structures are evident at the majority of schools visited. It is likely that these changes may also exist throughout the remainder of NECS schools. Indeed, findings indicate that NECS has strengthened the linkage between school and community through its programming. The use of local languages within instruction, provision of trainings and technical support to teachers and mentoring seem to have most benefited students. At the same time, parents have benefited from adult literacy programming and continued engagement as well as reinvigorated school support organizations. Findings demonstrate that public reading sessions motivate and engage school-level and community stakeholders and, most importantly, children and their parents. Positive effects may have been even stronger if adult literacy activities had continued beyond 2014, Community organizations within many of the villages visited demonstrate improved management and planning relative to supporting school needs, particularly around the construction and upkeep of school infrastructures. Financial and in-kind contributions are significant, especially given the low-resource environment that characterizes many of Niger’s communities. Communities’ efforts are significant and merit further encouragement. NECS has succeeded in creating a budding platform for women’s voices and engagement, particularly around recruitment and, to a lesser extent, literacy and book lending practices. Consideration of how to sustain and amplify such activities would continue to benefit participating communities. The mentoring program holds much potential for supporting students with socio-economic and academic challenges, but it has encountered considerable obstacles, including the high mobility of many individuals and entrenched systematic expectations for remuneration that are symptomatic of the weaknesses of development initiatives. Creative thinking is required to overcome these obstacles. Suggestions within findings, such as in-kind and symbolic recognition and greater utilization of retirees, may be worthwhile alternatives. Additionally, the mentoring program represents an opportunity for NECS to more purposefully promote positive gender role models and visioning for girls in particular. 35 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV EVALUATION QUESTION 5 EQ5: To what extent did the project successfully target girls? FINDINGS Three activities make up NECS’s gender-equity focused initiatives: gender-equity trainings and awareness-raising, promoting the adoption of a code of conduct among teachers and school directors within all NECS schools and creating gender-sensitive monitoring instruments. Responses to this evaluation question rely on document review as well as interviews and school/classroom observations. “Children’s education, especially for young girls, is much improved here. We go door-to-door and we share messages during ceremonies in order to advise parents. This had a great effect. … Parents understand the usefulness of schooling better and the importance of keeping kids at school, or even the negative effects of early marriage.” — Girls’ Schooling Representative (Scolarisation des filles, SCOFI/Tahoua) GENDER TRAININGS NECS gender trainings and awareness raising sessions involved teachers and community members including PTAs, mother’s associations, school management committees (CGDES) and SCOFI representatives and inspectorates. In total, 854 teachers and supervisors, 374 of whom are women (44 percent), were trained in different gender-related modules. Review of program documents illustrate that gender-equity modules include discussions of the difference between “sex” and “gender,” reviewing and problematizing gender stereotypes and related social norms and exploring the consequences of gender within school and society. NECS also worked closely with the Directorate for Promoting Girls’ Schooling (DPSF) and the Teacher Training Directorate (DFIC) to include gender equity items within teacher monitoring instruments and to develop a new gender-sensitive observation tool. These tools were piloted in the Madua Region. NECS targeted particular school-level practices, including: • Ending gender-based seating in class; • Ensuring equitable attention to both girl and boy students by teachers within class; and • Equitable sharing of school chores by gender. SUCCESSES OF GENDER INITIATIVES Interviews with teachers and school directors illustrate a sense that girls’ enrollment in NECS schools has been increasing since the NECS activity. As indicated within the response to EQ1, NECS statistics demonstrate a 48 percent girls’ enrollment rate in CI in 2017-2018. Data collected from the 20 schools visited as part of the final evaluation show a range in school-wide girls’ enrollment of 31 to 52 percent, with CI in NECS schools ranging from 34 to 65 percent (19 schools) and 34 to 65 percent in CP (19 schools). The sample average of girls’ enrolment for CI classes was 49 percent, and 41 percent for CP. Data also showed that CI girls outnumbered CI boys at nine of 19 schools. Field observations and interviews within the eight “hard” schools within the evaluation sample indicate that improved amenities, such as functioning latrines, seem to contribute to improved recruitment. Nearly half of USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 36 interviews with teachers and/or school directors identified awareness-raising and other activities of mother’s associations as highly effective. When asked to comment on the successes of NECS gender initiatives, notably, SCOFI representatives, the majority of school directors and school management committee (CGDES) secretaries identified increased recognition among communities of the importance of girls’ schooling as indicative of NECS’s major contribution to gender equity. Increased awareness of gender issues was also noted at the school level. The mentoring program and the ceremony for the best female student also represent activities that seem to have greatly contributed to reaching targeted girls. “Today, the community is now conscious that if it needs a female health agent, like midwives, for example, that it should allow girls to pursue their studies which will benefit her, her family and her community.” — FGD with school management committee (CGDES), Tahoua The evaluation team observed reading lessons in 19 of 20 classrooms (eight CI and 11 CP). Teams identified that girls and boys sat together in 13 classrooms while six classrooms maintained gender￾separate seating. 28 Table 4 depicts that girls surpassed boys in terms of active recognition and participation although the presence of observers in the classroom may have increased these tendencies. TABLE 4: CLASSROOM INTERACTIONS BY GENDER ACTIVITY GIRLS BOYS Directing questions at students 56% 44% Students responding to open questions 56% 44% Students going to the blackboard 58% 32% Teachers reviewing student work 58% 32% On the other hand, data analysis also shows that within the 20 schools visited, all of the “class leaders” were boys. Not one girl held this position within the classrooms visited. On the other hand, the majority (83 percent) of the “assistant class leaders” were girls. CODE OF CONDUCT The code of conduct is a two-page document with five chapters and 26 articles dedicated to increasing teacher accountability for gender equity within the classroom and within the community. Through project reports and interviews with project staff, it is clear that the code of conduct is one of the key gender strategies employed by NECS. While gender equity clauses are the focus of the document, a number of articles address general equity and inclusion items as well as teachers’ active participation within the community engagement. The code of conduct was developed in consultation with teachers’ unions, civil society groups and with the active participation of MEP officials, in particular, the director of the Directorate for Promoting Girls’ Schooling. Once teachers have accepted the code of conduct, the 28 The six classes were in Tillabery (one), Maradi (two) and Zinder (three). 37 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV signing takes place in public with an audience from the community and witnesses. According to NECS staff, the project has achieved a 98 percent signing rate. Interviews with stakeholders from the school to the central level identified great enthusiasm for the code of conduct. The code of conduct is one of the five points that the ministry has included for a stocktaking report and it is being considered for scaling up across the country. As indicated in the narrative in response to EQ3, communities also described a high level of satisfaction for the code. According to the DDEPs, IEPs and the pedagogical advisors, adopting the code of conduct has led to greater respect for gender equity within all of school activities within NECS schools. Pedagogical advisors reported seeing strong gender-equity sensitive practices in NECS classrooms. In addition, school directors and teachers noted the effects of implementing the code of conduct detailed in Figure 4. FIGURE 4: CODE OF CONDUCT EFFECTS AT SCHOOLS, REPORTED BY TEACHERS & DIRECTORS In addition, observations of schools visited demonstrated that within the 20 school governments, girls’ representation ranged from 29 percent to 86 percent with an average of 55 percent. Of positions recorded, girls held 36 percent of presidencies, 46 percent of prime minister positions and 38 percent of girls’ schooling minister positions. Many of the girls’ schooling ministers spoke of their role in encouraging girls within the community to attend school, particularly through house visits. Discussions with SCOFI representatives and school personnel confirmed these efforts. CHALLENGES WITH CODE OF CONDUCT APPLICATION Interviews with teachers and school directors illustrate some challenges inherent within NECS’s approach. A teacher in Zinder, for example, was not aware of the code of conduct, while two other teachers, one in Maradi and one in Zinder, indicated that they signed the code without knowing its content. A third teacher, in Maradi, was not aware of the code, although the evaluation team saw the teacher’s name and signature on the document. Another director shared that teachers are not comfortable with the code of conduct, while two other directors expressed feeling like the code has been forced on teachers. One teacher added that they signed just to avoid being sanctioned. A director in Zinder expressed misgivings, saying that teachers had signed the code but that it conflicts with religious beliefs about gender equality. Same chances for girls and boys Regular girls’ participation in class Avoid gender discrimination Mixed-gender rows Sports for both girls and boys Equality Working with both Good functioning of school Equal representation in school government Putting girls first Live and work together in harmony USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 38 “There needs to be another way to have people sign it because right now it seems as if it’s forced on teachers.” — Director, Tillabery REMAINING CHALLENGES TO GIRLS’ SCHOOLING While interviews with the majority of pedagogical advisors, school directors, SCOFI representatives and community members identify progress with girls’ enrollment, they also identify that girls’ attendance and retention remain acute challenges. According to stakeholders, poverty, exodus, needs for tending the fields and early marriage are the main factors that determine if a girl will continue attending school. A small proportion of school mothers’ associations focus groups acknowledged continued hesitation within communities. In addition, discussions with various stakeholders demonstrated that NECS-encouraged mixed-gender seating represents a risk for girls leaving school. Stakeholders explained their hypothesis that because some parents didn’t agree with this set-up, they might take their girls out of school. “In the village, even if it’s a bilingual school, we haven’t really noticed real changes in girls’ enrollment despite all of the NECS awareness-raising activities. … There are lot of school-age girls still in the village. … You’ll see a parent with six girls who chooses to send two or three and keeps the others at home for domestic work. … There are a lot of girls who stay at home. … In addition, there is also a lot of relocation that takes place even in the middle of the school year.” — Teacher, Bilingual School CONCLUSIONS Strong gender initiatives have had a steady and positive presence throughout the implementation of the NECS activity. Initiatives include gender trainings and awareness-raising, award ceremonies for the best female students and, of course, the code of conduct. The latter, in particular, takes advantage of community’s collective consciousness and mutual support to strengthen accountability for gender equity. Classroom observations within the sampled NECS schools confirm that teachers regularly recognize female students and that girls participate actively in reading lessons. An area of growth remains supporting girls in taking on leadership positions. NECS has mobilized key actors to support girls’ schooling efforts, including school governments, regional and department-level MEP representatives, pedagogical advisors, teachers and school directors, and community organizations — specifically school mothers’ associations. While the code of conduct has been largely positive, enough cases of misguided application exist to warrant some concern and renewed attention to community engagement and clear communication with school and regional actors to avoid misuse. Although the MEP’s interest in scaling up the code of conduct is encouraging, MEP must recognize the great mobilization and awareness-raising efforts that were necessary to achieve success within NECS schools. In addition to promoting awareness for girls’ schooling and their active engagement, NECS has an opportunity to more purposefully engage boys and men in examining their roles in supporting gender equity and keeping them engaged in schooling initiatives, in general. 39 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Girls’ enrollment may have seen some progress under NECS, but this evaluation did not perform a longitudinal quantitative analysis. Retention remains a serious challenge to the Nigerien school system, largely due to persistent socio-economic and cultural barriers. RECOMMENDATIONS The following list of recommendations takes into account the short-term need for guidance on a possible extension of the NECS activity and the long-term needs of similar early grade reading programs in Niger. Recommendations build on findings and conclusions from above as well as results from the participatory Recommendations Workshop that took place in Niamey with MEP, USAID and NECS stakeholders in June 2018 (see annex XVI for full list). NECS 1. Before printing additional documents, further strengthen ASL TLMs by addressing typos and misalignment of references within materials through a formal copyediting process. NECS Education Coordinators already possess documentation of teacher feedback that will support this activity. 2. Continue to offer training and technical support to NECS teachers in ASL methodology. Identify and address gaps as indicated by a small number of teachers and school directors. Make adjustments to schedules and duration to further strengthen trainings. 3. Closely monitor community engagement in schooling and where needed, reinvigorate community awareness-raising and support efforts. Essential activities include post-literacy efforts, public reading sessions, effective management and reporting of community organizations and women’s involvement. While effects of NECS community mobilization efforts observed at the community level have been impressive, sustainability is questionable and may need reinforcement in the short-term. 4. Reinforce gender-equity training and awareness raising to include clear roles for men and boys in supporting girls’ schooling. 5. Clarify roles and expectations for school governments to improve their functioning. 6. Investigate implementation of Code of Conduct signing at the school level to assure that signing is voluntary and transparent. Should inconsistencies be sustained, identify a plan of action to clarify expectations and process with school, regional and central level stakeholders. Continue community advocacy work in areas where pushback is most prevalent, especially outreach with religious and community leaders. NECS WITH MEP: 1. Develop strategy and guidance for teachers in bilingual schools to implement ASL techniques alongside APS. Monitor teacher and student experience closely with deliberate feedback loops to inform possible scale up of ASL components alongside APS within other Reform schools. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 40 2. Develop and communicate clear technical assistance strategy to teachers and school directors. Work with inspectors, pedagogical advisors, departmental and regional directors on providing constructive feedback so that teachers feel more accompanied and less surveilled in their teaching practice. 3. Identify new strategies to strengthen and reinvigorate the mentoring program maintaining volunteer status. Investigate opportunities to work with retirees and provide symbolic and in￾kind recognition. Expand framing of mentoring activity to support linking students with successful female role models. NECS WITH PLAN INTERNATIONAL: 1. Develop an action plan to strengthen NECS M&E systems moving forward. Elements of the action plan should include: 1) training and ongoing support to school and regional level actors in monitoring efforts as well as to assure fidelity of implementation of activities, 2) how to share and centralize monitoring information within the NECS activity, 3) whether to strengthen current excel-based practices for information management or to reactivate/develop a user￾friendly database that can be supported by in-country staff, 4) professional development opportunities for NECS staff members to enhance understanding of strong program monitoring practices and how learning initiatives may be integrated into daily routines. 2. Allowing adequate time for planning and implementation work with STS to clarify EGRA 2019 process so that sample allows for comparison and to ensure credible results. 3. Reconsider how to push back transition to French until 3rd grade (CE1) in order to take full advantage of grade 2 for reading instruction within a familiar language following evidence-based practice. This may require providing support to CE1 teachers for transition. 4. Redefine and articulate reading approaches employed within ASL to bolster and reinforce advocacy efforts within the Ministry. 5. Engage MEP officials in information and capacity building sessions on evidence-based reading approaches to develop reading expertise amongst MEP officials and increase project buy-in, especially around critical milestones in reading acquisition and appropriate application of scripted lessons. NECS WITH PLAN INTERNATIONAL, USAID NIGER AND USAID SRO 1. Revisit AMEP to revise Indicator 1 on student performance so that reading performance target is more appropriate to Nigerien context. PLAN INTERNATIONAL (USA) WITH PLAN NIGER: 1. Identify M&E support individuals who will provide regular technical assistance to NECS M&E staff. While M&E technical assistance has figured among support available to the project, the consistency of this support has been erratic, particularly within the last year of the project. 41 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV MINISTRY OF PRIMARY EDUCATION (MEP): 1. Thoroughly review the benchmark identified for EGRA scores – follow a rigorous, participatory process that takes into account current levels of student performance as well as linguistic complexities. 2. Provide adequate support and follow-up to the Language Mapping processes indicated by the Direction of Curriculum and Promotion of National Languages. Recent developments related to the Senegal Lecture pour tous (LPT) may provide some guidance. PLAN INTERNATIONAL/NECS ACTIVITY WITH THE MINISTRY OF PRIMARY EDUCATION: 1. Reexamine assessment options in order to identify the most appropriate tool for Nigerien primary school students. While EGRA may still be an option, the project should also closely follow progress on the other earlier reading acquisition EGRA sub-tasks: orientation to print, letter name identification and listening comprehension. Doing so may help address floor effects and provide more motivating results. 2. Plan for and undertake a rigorous, participatory process to identify the most appropriate benchmarks for EGRA sub-task scores. Engage appropriate reading technical experts to assure quality. This process needs to take into account current levels of student performance as well as linguistic complexities. Such a process may also greatly benefit MEP as once such benchmarks are established they can be adopted for use with all current early grade reading programs. 3. Develop an in-country technical advisor group made up of MEP officials, implementing partners and donors to share best practices in EGRA and other educational assessments that focus on reading. Doing may address a serious gap in knowledge of sound assessment practices and support more credible measurement outcomes. USAID NIGER 1. Accompany the MEP in identifying appropriate, valid and relevant EGRA benchmarks to support eventual credible and externally valid performance results. Given capacity constraints, external reading experts to MEP will likely need to be engaged. USAID NIGER WITH NECS AND MEP 1. Develop working group to better understand and address concerns for bringing ASL to scale around political buy-in and resource identification. Based on on-going consultations develop a participatory scaling up action plan. Elaborate advocacy and capacity building initiatives to support a potential scaling up process. MEP 1. Stakeholders urgently need guidance from MEP to identify best directions for educational programming and expenditures focused on providing explicit and systematic reading instruction. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 42 Nigerien children and parents deserve stable, well-articulated and well-advocated educational strategies. 2. Consider ASL’s comparative advantage to other existing reading programs in addressing the five critical evidence-based reading components, providing a scope and sequence, systematic instruction, engaging teaching practices and existent TLMs that may complement and fill in gaps within delivery within Reform schools. Supplement Reform strategies with these central ASL components. 3. Work closely with NECS team in expanded application of the Code of Conduct to ensure appropriate and transparent application. Consider ways to implement gender training in order to assure fidelity of implementation. 4. MEP Regional officials (IEPS, DDEPs, DREPs, SCOFI representatives and pedagogical advisors) should continue to work closely with community organizations, particularly Mothers’ Associations in order to sustain girls’ schooling recruitment and maintenance efforts. LONG-TERM CONSIDERATIONS FOR USAID USAID NIGER & SRO 1. Develop a new solicitation process for the next phase of early grade reading support to ensure adequate reading technical expertise throughout the life of the project. Awardees will need to demonstrate reading specialist expertise at the headquarters, country and project level in order to ensure regular and adequate oversight of reading initiatives. Next phases will need to include clear philosophical and evidence-based technical grounding of the identified reading approach. USAID NIGER 1. Based on espoused desires to increase involvement within the Northern Tillabery region where insecurity and displacement present serious concerns, build in socio-emotional learning (SEL) components within educational programming. SEL can be integrated within the reading curriculum and does not necessarily require more time with students. Training and SEL support for teachers and school personnel are also necessary programming elements. Review INEE Guidance Note on Pyschosocial Support. IRC’s Healing Classroom project in Diffa may also offer some insights into programming options. 2. Further investigate results and cost-effectiveness of alternative models to formal schooling initiatives like Save the Children’s Literacy Boost and IRC’s Healing Classrooms model. 43 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV ANNEX I: WORK PLAN NIGER EDUCATION AND COMMUNITY STRENGTHENING PROGRAM EVALUATION WORK PLAN APRIL 2018 This publication was produced for review by the United States Agency for International Development. It was prepared by Management Systems International, a Tetra Tech Company, for the USAID/Senegal Mission-Wide Monitoring and Evaluation Project. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 44 Niger Education and Community Strengthening (NECS) Final Performance Evaluation WORK PLAN Contracted under AID-685-C-15-00003 USAID Senegal Monitoring and Evaluation Project 45 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV CONTENTS ACRONYMS 1. INTRODUCTION 2. ACTIVITY DESCRIPTION 3. DEVELOPMENT HYPOTHESIS AND IMPLEMENTATION 4. EVALUATION QUESTIONS 5. DATA COLLECTION METHODS AND SOURCES 5.1 STRENGTHS AND LIMITATIONS OF DATA COLLECTION METHODS. 5.2 GETTING TO ANSWERS MATRIX 6. SAMPLING 7. DATA ANALYSIS 8. FIELD WORK PLAN 9. ROLES AND RESPONSIBILITIES ANNEX I: GETTING TO ANSWERS MATRIX ANNEX II: INTERVIEW TOOLS ANNEX III: DETAILED FIELDWORK SCHEDULE USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 46 ACRONYMS AME Association des Mères d’Elèves ANED Association Nigerienne pour l’education et le developpement AOR Agreement Officer’s Representative APE Association de Parents d’Elèves CDGES Coordination Décentralisée d’Etablissements Scolaires CE1 Cours Elémentaire 1ere année CE2 Cours Elémentaire 2ieme année CGPE Comité de Gestion des Points d’Eau CI Cours d’Initiation CM1 Cours Moyen 1ere année CM2 Cours Moyen 2eme année COP Chief of Party DAGE Direction de l’Appui à la Gestion de l’Ecole DCOP Deputy Chief of Party DCRE Direction des Curricula et de la Réforme de l’Enseignement DDEP Departmental Primary Education Direction DECP Direction de l’Enseignement du Cycle Primaire DEP Direction des Etudes et Programmation DESASO Direction de l’Evaluation, du suivi des acquis scolaires et de l’orientation (DESASO) DESQ Direction de l’Evaluation et du Suivi e la Qualité DFIC Direction de la Formation Initiale et Continue DGEB Direction Générale de l’Education de Base DGEP Direction of Reform, Promotion of National Languages 47 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV DGPLN Direction Générale de la Promotion des Langues Nationales DPSF Direction de la Promotion de la Scolarisation des Filles DPSF Direction for Promotion of Girls’ Education DREP Directions régionales de l’Education Primaire EQ Evaluation Questions FCR Findings, Conclusions and Recommendations FGD Focus Group Discussion IDAENF Inspection Départementale de l’Alphabétisation et de l’Education Non Formelle IP Implementing Partner KIIs Key Informant Interviews M&E Monitoring and Evaluation MCC Millennium Challenge Corporation MEP Monitoring and Evaluation Project MEP/A/PLN/EC Ministère de l’Enseignement Primaire, de l’Alphabétisation, de la Promotion des Langues Nationales et de l’Education Civique MSI Management Systems International NECS Niger Education and Community Strengthening NGO Non Governmental Organization Plan Plan International POC Point of Contact SCOFI Scolarisation des Filles SRO Sahel Regional Office STS School-to-School International USAID United States Agency for International Development USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 48 VIE Volontaires de l’Intégration Educative 49 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV 1. INTRODUCTION USAID SOW Manager Ahmed Tidiane Diallo Activity COR/AOR Babacar Kossomi MEP SOW Manager Aissatou Mbaye Sy Activity Title Niger Education and Community Strengthening Program (NECS) Activity Period June 15, 2012-June 30, 2018 Award/Contract # AID-624-A-12-00004 Funding $11,599,656.00 Implementing Organization Plan International Geographic Coverage Niger Task Final Performance Evaluation Task Start and End Dates April-August, 2018 The USAID Sahel Regional Office (SRO) is requesting this final performance evaluation of the Niger Education and Community Strengthening (NECS) activity. The purpose of this evaluation is to: 4) Assess the progress of the contractor to completion in terms of achievements, management processes, and challenges in meeting the stated objectives of the cooperative agreement, and sustainability of achievements. 5) Make recommendations for scaling-up of best practices, for the Government of Niger and USAID; 6) Make recommendations for the new design of similar activities in the future. The audience for this evaluation is USAID/Niger and the USAID/Senegal Sahel Regional Office (SRO) and the Ministry of Primary Education (MEP). USAID/Washington/E3/Education Bureau may also be interested in the results of the evaluation as they relate to their Education Policy improved reading goal. The evaluation will be led by a four-person team: one international team leader, one international reading expert, one MEP M&E Associate and one Niger evaluator. The team leader will be Karla Giuliano Sarr, independent consultant and specialist in the area of education. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 50 She will provide methodological orientation to the team and lead interviews, data analysis and report writing. Gaëlle Simon will serve as the Reading Expert for the team and will focus on the comparative advantage of the NECS reading approach as compared to the other current education quality programs. Dr. Sarr will be supported by Aissatou Mbaye Sy, MEP M&E Associate. In Niger, the team will work with Abdoulaye Tambandia who will serve as the local evaluator overseeing the focus group discussions with NECS communities as well as key informant interviews with education inspectors, local authorities and traditional leaders working with NECS. He will work with the team leader to make sure that the focus group guide is accurately translated into local language and oversee the transcription and translation of all focus groups and interviews. 2. ACTIVITY DESCRIPTION The 2012-2018 funded NECS leverages evidence-based reading, engages communities, and encourages families to keep their children in school. The NECS Activity’s goal is improved educational opportunities available to children and strengthened links between communities and state structures. This goal is achieved through two interdependent strategic objectives: SO1: Increased student grade reading achievement by improving instruction in primary schools and promoting a culture of reading in communities; and SO2: Increased access to quality education in schools, especially for girls. This activity builds on past gains achieved by USAID investments in education in Niger and a Niger Ministry of Education (MOE)-led initiative to promote reading in four local languages across Niger. Plan International (Plan) is NECS implementing partner. NECS activities have directly contributed to improved reading performance, enhanced quality learning environments, mobilization of communities around reading, and reduced gender barriers to educational access. NECS uses an innovative “Systemic Reading Approach” (ASL) that leads to critical reading outcomes for boys and girls across 160 schools in Niger. A 2015 MOE-led assessment showed NECS’ reading approach to be the best among reading programs currently being implemented in Niger. 3. DEVELOPMENT HYPOTHESIS AND IMPLEMENTATION The original NECS activity was developed as part of an Millennium Challenge Corporation (MCC) Threshold Country Program (TCP) in 2008 with the objective of increasing girls’ enrollment and retention in primary school. The TCP was managed by the USAID/West Africa Regional Mission in Accra, Ghana. The project was suspended due to the political situation in Niger in December 2009. The re-opening of the project occurred in June 2012 with a new start date for the NECS co-operative agreement with USAID beginning June 15, 2012 until June 30, 2016. In the interim, USAID launched its Global Education Strategy in 2011 which placed primary focus on improving reading skills of primary school children. In the re-launch of the NECS activity, two strategic objectives were defined for the four-year intervention: increased access to quality education (strategic objective 1) and increased student grade reading achievement (strategic objective 2). Strategic Objective 1 (SO1) activities included improving physical and social environment for students and increased community support for education. 51 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV SO2 activities included improving reading instruction in primary schools and promoting a culture of reading in communities. On June 30, 2016, a modification was made to the co-operative agreement extending the project by two years from June 30, 2016 to June 30, 2018 and adding an additional $3,982,451 dollars to the agreement. The extension period of NECS, called NECS+ was shifted to the USAID Sahel Regional Office within the USAID/Senegal Mission in Dakar. The Activity Monitoring and Evaluation Plan (AMEP) submitted in November 2016 reduced the number of performance indicators and added eight new standard indicators related to the USAID 2011 Education Policy. The AMEP retained two SOs but switched their numbers with SO1 now focused on improving student grade reading and SO2 improving access to quality education. The AMEP identified four results areas and noted that each represented a key component of school and community strengthening, and how the project works within and engages the school, community, and decentralized education authorities primarily at the inspection level: IR1.1: Improved reading instruction in primary schools through the development and delivery of the ASL curriculum by school directors, teachers, and pedagogical advisors IR1.2: Culture of reading is promoted in communities through monitoring and support from Adult Literacy Inspections and School Management Committee Focal Points. For this phase of the project, this will move toward supporting these inspection authorities to monitor and promote community reading activities. IR2.1: An improved school environment through monitoring and support from Pedagogical Advisors, School Management Committee Focal Points, and Girls’ Education Focal Points IR2.2: Increased community participation to support education through monitoring and support from Pedagogical Advisors, School Management Committee Focal Points. NECS+’s theory of change is that if these four key result areas are supported and maintained for a specific school through sustainable institutional changes made by key decentralized educational authorities, then: • Strengthen school directors, teachers, and Parent/Teachers Associations to provide greater support to schools • Community actors are mobilized around education, both by supporting school development plans and promoting reading culture for children in their community • Through greater awareness and engagement, more children attend school, especially girls • Improved reading skills at targeted schools through the use of national language instruction. 4. EVALUATION QUESTIONS USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 52 The evaluation will answer the following questions: 1) To what extent has NECS achieved its objectives in relation to its targets? 2) How did the NECS “ASL” approach enhance reading outcomes in NECS Schools? 3) In terms of scalability, what are the comparative advantages of the ASL approach as compared to the other reading approaches in Niger? 4) What types of changes are visible in NECS schools and communities? 5) To what extent did the project successfully target girls? 5. DATA COLLECTION METHODS AND SOURCES This performance evaluation will engage in desk review, qualitative and quantitative methods to assemble data required for arriving at findings and conclusions. ● Desk Review: the evaluation team will review and analyze activity design and implementation documents, which add up to hundreds of pages. This review will arrive at an understanding of the activity objectives and implementation characteristics, contexts, and challenges, and will identify and describe achieved activity results. ● Qualitative methods: the evaluation team will conduct key informant interviews with USAID/Niger, host country government officials, and senior staff of Plan both in Niger and in their home office as well as activity partners such as Aide et Action International and School to School (STS). Focus groups will be organized with community leaders and group interviews will be conducted with school staff and education inspectors. To the extent possible, the evaluation team will conduct structured classroom observation. ● Quantitative methods: The team will use data from the Mathematica baseline evaluation in 2008 and 2016 as well as the NECS 2017 EGRA assessment to conduct a comparative analysis of reading fluency from the start of the activity to today. Data regarding girls’ enrollment figures will also be analyzed from the inception of the activity from 2008 until now, looking at any noted differences between NECS and NECS+. The team will also explore the number of targets achieved in the activity’s AMELP and identify level of differentiation between targets and actuals. 5.1 STRENGTHS AND LIMITATIONS OF DATA COLLECTION METHODS. Data collection methods are driven by the five evaluation questions and comprise a mixed￾methods approach. Given the large number of schools and the wide variety of regions covered in NECS, it will not be possible to visit a representative sample of schools. As detailed in the sampling section below, the team will focus on schools within four regions and in close proximity to the capital due to security concerns. Because the sample is not representative, it will not be possible to generalize findings from the sample to the larger population of schools. Nonetheless, results from the evaluation will provide a thick qualitative description of the 53 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV current status of NECS activities and sentiments regarding those activities across a spectrum of stakeholders. In addition, the utilization of both quantitative and qualitative data aligns with a mixed methods approach that is able to take advantage of broad trends that quantitative data is able to provide alongside in-depth descriptions characteristic of qualitative data. Triangulating across multiple stakeholders, data sources and collection methods will reinforce the rigor of the evaluation and conclusions drawn from it. Additionally, the evaluation takes place at the end of the schoolyear, making it difficult to conduct comprehensive classroom observations and creating some uncertainty of finding school-based actors within the schools. For this reason, two teams will collect data in the field while the Team Leader remains in Niamey to work with MEN and other organization counterparts. In this manner, the data collection period is compressed. Additional complications also derive from the data collection falling within the Muslim month of Ramadan and the Eid-al￾Fitr holiday that will take place in mid-June and likely cause some disruption to evaluation plans. 5.2 GETTING TO ANSWERS MATRIX The identification of potential data collection methods and sources appropriate for each Evaluation Question (EQ) are presented in a completed “Getting to Answers” matrix is included as an annex to this workplan. 6. SAMPLING Purposeful sampling methods will guide the qualitative data sample choices. Quantitative data derives from previous assessments and evaluations. The qualitative data includes interviews with key stakeholders in unique positions (such as the Plan COP and numerous GON officials) as well as visits to a number of NECS+ schools. The sample includes 20 of the 162 current NECS+ schools. Criteria for selection included region, type of school (hard, soft and bilingual) as well as being sure to include schools from all four languages represented within NECS+ schools. The denomination “hard” indicates that these schools benefitted from the IMAGINE project as well as NECS. “Soft” indicates only the presence of the NECS activity. The sample identifies 5 schools in each of the four regions (Maradi, Tahoua, Tillaberi and Zinder) also within reasonable distance from Niamey as necessary because of security concerns. Of the schools in the sample, 9 are hard, 7 are soft and 4 are bilingual, noting that bilingual schools did not benefit from the IMAGINE project and are therefore also soft schools. In terms of language of instruction, the majority (12) of schools offer instruction in Haussa, 5 in Zarma, 2 in Kanuri and 1 in Fulfulde. The table below provides detailed information about the schools within the sample. No. Nom du village Type de Site (X) Region Langue(s) Hard Soft Biling. 1 Kanjiwa X Tahoua Haussa 2 Guidan Ider X Tahoua Haussa 3 Bazaga Bilingue X Tahoua Haussa USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 54 4 Maïkourou X Tahoua Haussa 5 Boudé X Tahoua Haussa 6 Kannia X Tillaberi Zarma 7 Nobba X Tillaberi Zarma 8 Zarmey bilingue X Tillaberi Zarma 9 Dargol Quartier X Tillaberi Zarma 10 Banizoumbou 1 X Tillaberi Zarma 11 Maiguigé Karfi X Maradi Haussa 12 Tsararaou Allah Sarki X Maradi Haussa 13 Saber X Maradi Haussa 14 Baban Anné X Maradi Haussa 15 Tatsegouma Saboua X Maradi Haussa 16 Dougueré Mai Gao X Zinder Fulfulde 17 Chago X Zinder Kanuri 18 Farayé X Zinder Haussa 19 Bande bilingue X Zinder Haussa 20 Soubdou Centre X Zinder Kanuri 7. DATA ANALYSIS Data analysis will draw on findings organized by evaluation question, facilitated by standardized data collection instruments attached. Qualitative data collected via the key informant interviews will be analyzed using thematic and content analysis, with categories developed inductively. At the end of each day, team members will type up notes from the interviews, noting at the top of each interview guide any interesting or significant points, and having a round-up meeting to determine the efficacy and consistency of the data collection. Recording of interviews will be provided to the MEP team in Dakar for transcriptions and translations, as necessary. These will be used to enhance the analysis from the written notes and refine their analysis. The evaluation team will note patterns and develop themes from the interviews. The team will design an analysis rubric to analyze the data and capture these 55 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV emerging themes and/or patterns. Once the team has completed the thematic analysis, it will analyze key findings and themes using content analysis. Content analysis will help the team go beyond descriptions of changes in practices and attitude to identify the most salient characteristics of these changes, as a result of NECS’ interventions. The evaluation team will use data triangulation, the use of two or more data sources, to strengthen findings or identify areas of divergence. In order to illuminate the validity of findings, percentages of respondents (for example, 60%, or 6/10 respondents) will be included in the narrative. Master analytical graphs showing descriptive statistics (frequencies and frequency distributions) will also provide more visual reinforcement to the narrative analysis of findings. One other analytical tool that has proven useful in similar management and performance reviews is a chronograph. A chronograph will be developed showing the evolution of the activity from 2008 to today and pinpoint major shifts and/or evolutions in the activity. The purpose of this will be to show the operational context over time at various levels: general operating environment, project management, and specific performance benchmarks. These might include political unrest, teacher strikes, staffing gaps/shifts, and objective/target shifts and can thus provide very useful overviews of challenges and advantages that offer additional insight into an activity’s implementation. 8. FIELD WORK PLAN The timeline below provides an overview of how time will be spent in-country and when final deliverables will be provided. The field work schedule is included in Annex III. There are two calendars: one for school visits and the other for appointments within the capital. Dates Tasks/Deliverables March 19-April 27 Recruitment of team and focus group moderators/interviewers April 23- May 4 Desk review and completing answering questions with secondary data April 23-May 11 Development of work plan, field itinerary and tools May 14 Submit work plan to USAID for approval May 7-18 Prepare appointments and field schedule May 20 Travel to Niamey May 21 Team Coordination Meeting and initial meeting with USAID/Niamey staff May 22 Interviews with USAID/Niamey staff USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 56 Dates Tasks/Deliverables May 23-24 Group interview with Plan Niamey senior activity staff, individual interviews with senior staff, Ministry of Education. Gender and other Niamey stakeholders May 25-26 Train enumerators May 27 Depart for regions May 28-June 7 Regional interviews/focus groups May 28-29 Continue interviews in Niamey June 2 Arrival of Reading Expert to Niamey June 4-7 |Reading Expert joins team in the field June 4-7 Additional interviews with Ministry of Education. Gender and other Niamey stakeholders June 8 Return to Niamey June 11-12 Interviews with other quality education projects in Niamey June 13-18 Data analysis June 14 Departure from Niamey (Reading Expert) June 19 Data walk with USAID (findings and initial presentation) June 21 Presentation of findings with Plan and MEP June 22-23 Development of findings, conclusions and recommendations table June 25 Validation of findings, conclusions and recommendations by USAID June 26 Recommendations workshop with MEP, Plan and USAID June 27 Departure from Niamey (Education Expert) July 2-13 Development of draft report July 18 Submission of draft report 57 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Dates Tasks/Deliverables August 1 Feedback from USAID on report August 2-10 Revisions to report August 20 Submission of final report August 31 USAID approval of report USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 58 9. ROLES AND RESPONSIBILITIES The evaluation team has four principals: the team leader, the reading expert one local evaluator, and a MEP evaluation associate. The team leader has the overall responsibility for the evaluation report, but other roles and responsibilities divide up along these lines, per the following table. This does not include the donor’s responsibilities with respect to review. Table II: Roles and Responsibilities Roles & Responsibilities Team Leader Nigerien Evaluator Reading Expert M&E Associate MEP Staff Dakar/DC Workplan X X X Desk Review X X X Data collection instruments X X X X Schedule & appointments X X Logistics X X Interviews w GoN, IPs donors and Plan X X X X Focus groups and community interviews X X X Data entry X X X X X Data analysis X X X X X Draft FC table + presentation X X X X Recommendations workshop X X X Draft report X X X Quality review X Edit report X 59 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Prepare report for USAID review X Revise report X Edit revised report X USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 60 ANNEX I: GETTING TO ANSWERS MATRIX EVALUATION QUESTIONS TYPE OF ANSWER NEEDED DATA COLLECTION METHODS DATA SOURCE(S) SAMPLING OR SELECTION CRITERIA 1) To what extent has NECS achieved its objectives in relation to its targets? Descriptive (including chronograph; Comparative planned/actual achievements and narrative) Document review Key informant interviews (KIIs) Plan Niamey, Washington and field-based staff, USAID interviews semi￾structured interviews (may be easier to do this in small groups) School Visit Checklist - NECS M&E plans; quarterly and annual reports Plan 2016 baseline report developed by STS - Mathematica Study Report - additional USAID, Plan reports (trip reports, internal auto-evaluations, etc.) - KIIs (MoE, DPSF, DECP, DESQ, DESASO, DDEP, DGEP, Education donors and partners) - School tour; Observation Purposeful sample 2) How did the NECS “ASL” approach enhance reading Descriptive and Content analysis Document review of Plan/STS descriptions and review of training and classroom modules. - Plan and STS reports - KIIs with Plan/STS reading staff and M&E staff Purposeful sampling - CI & CP Teachers 61 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV outcomes in NECS Schools? Review of auto-evaluation EGRAs and EGRA results since introduction of ASL in 2015. KIIs and Group interviews at community/school level - KIIs and Group interviews with teachers, head teachers and supervisors - Focus groups with students (CI/CP; School governance committee) -Focus groups with community - Classroom Observation - Students – Gouvernement Scolaire; 6 CI or CP students per school (separated by gender): 6 (random selection) - AME/APE; CGDES; membres communautaires 3) In terms of scalability, what are the comparative advantages of the ASL approach as compared to the other reading approaches in Niger? Descriptive Comparative (by activity component and approaches) KIIs Document review Scalability Checklist analysis of the activity - KIIs MoE, DAGE, DECP, DESQ, DESASO, DDEP, DGEP, USAID, Plan, donors in education and technical education quality projects) Purposeful sample 4) What types of changes are visible in NECS schools and communities? Descriptive analysis, observation Focus groups; group interviews and KIIs - KIIs (education inspectors, Plan staff in Niamey and field) - FGDs and KIIs with School Management Committees, mothers’ associations, parents, students. SCOFIs, Purposeful sample - CI & CP Teachers Students – Gouvernement Scolaire - AME/APE; CGDES; membres communautaires USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 62 teachers, and head teachers, students - Tour of the school - Classroom Observation 5) To what extent did the project successfully target girls? Descriptive analysis and comparative analysis (targets versus actuals) Document review KIIs Focus groups and group interviews - KIIs (MoE, DPSF, DCRE) education inspectors, Plan staff in Niamey and field) - School Management Committees, mothers’ associations, SCOFIs, students, parents, teachers, and head teachers - Classroom Observation Purposeful sample - CI & CP Teachers Students – Gouvernement Scolaire Parents – non-leaders, male and female together 63 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV ANNEX II: INTERVIEW TOOLS List of Tools on the following pages - Guide – Reading Program Organizations - Checklist – Completed interviews list - Guide – Regional education inspectors, Conseillers Pedagogiques - Checklist – School Visit - Guide - Head Teachers - Guide - Teachers (CI & CP) - Classroom Observation (CI, CP) - Guide – Student reading discussion and activity (CI, CP) - Guide - Student Government Committees - Guide – Community members - Guide – Community leaders (AME, APE; CGDES) - Guide Staff NECS/Aide et Action - Guide – USAID/MCC - USAID/SRO M&E - Guide Niveau Central – MOE< MEP, MEP/A/PLN/EC & Directions - Guide Other Technical Organizations (Reading Program) USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 64 EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) Consentement Principes de l’enquête / Le consentement verbal : Projets/Ministère/Acteurs institutionelles/Directeurs/Enseignants Bonjour, le MSI (Management Systems International) effectue l’étude d’évaluation du projet NECS pour le compte de l’USAID. La finalité est de collecter des informations afin de mieux comprendre les résultats du projet et de formuler des recommandations pour de futurs programmes. Nous aimerions avoir votre aide dans ce domaine. Est-ce que vous voulez participer dans cette étude ? [Attendez la réponse de la personne. Si la personne semble à l’aise, passez directement au consentement verbal.] Les résultats de l'enquête seront publiés sous la forme du texte narratif. Etant donné votre poste, il peut avoir des instances où il sera possible de vous identifier. Comme vous savez, le rapport final sera partagé avec l’USAID et le Ministère de l’Education National dans le but de renforcer l'implémentation des programmes. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer l'entretien. En fin de l’étude, l’enregistrement sera envoyé au niveau de l’USAID pour documenter l’activité. Votre nom ne sera pas lié à l’enregistrement. Mais si vous ne voulez pas participer, il n’y aura pas de problème. Une fois que nous commençons, si vous préférez ne pas répondre à une question, vous pouvez nous le dire. Est-ce que vous nous donnez votre consentement ? [Avez-vous obtenu le consentement verbal? ☐ ] Communautés Bonjour. Nous sommes avec le MSI (Management Systems International) et nous cherchons à appuyer l’état américain à mieux comprendre les effets des projets. Nous sommes là pour vous écouter. Votre école et votre communauté ont reçu des fonds de l’état américain pour améliorer la réussite des enfants dans le cadre du projet NECS. Votre établissement fait partie des établissements qui ont été choisi à participer dans cette enquête. La finalité est de collecter des informations afin de mieux comprendre les résultats du projet et de formuler des recommandations pour de futurs programmes. Vos expériences du projet et vos perspectives sont très importants. Est-ce que vous voulez participer dans cette étude ? [Attendez la réponse.] 65 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Votre nom ne sera pas mentionné dans le rapport fait sur cette enquête. Les résultats de l'enquête seront publiés sous la forme du texte narratif sans identification des individus. Le rapport final sera partagé avec l’USAID et le Ministère de l’Education National dans le but de renforcer l'implémentation des programmes. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer l'entretien. En fin de l’étude, l’enregistrement sera envoyé au niveau de l’USAID pour documenter l’activité. Votre nom ne sera pas lié à l’enregistrement. Mais si vous ne voulez pas participer, il n’y aura pas de problème. Aussi, une fois que nous commençons, si vous préférez ne pas répondre à une question, vous pouvez nous le dire. Est-ce que vous nous donnez votre consentement ? [Avez-vous obtenu le consentement verbal? ☐ ] Les enfants Bonjour. Nous sommes des amis de votre Maître/Maîtresse. Nous travaillons avec un projet qui aide l’école. Nous aimerons discuter avec vous et faire des exercices. Est-ce que vous voulez discuter avec nous ? Si non, ce n’est pas grave. [Attendez la réponse.] Notre discussion prendra environs 30 minutes. Nous n’allons pas partager vos réponses avec ni le Directeur ni les enseignants. Est-ce que vous nous donnez votre accord ? [Avez-vous obtenu le consentement verbal? ☐ ] USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 66 Liste de Contrôle des Entretiens Effectués Nom du Village : Région : Date(s) de visite : Equipe : Activités achevées _____ Visite de courtoisie – Autorités administratives Nom de l’autorité : Fonction : _____ Visite de courtoisie – Chef de village Nom : _________________________________ _____ KII avec DREP/DDEP ; Nom de l’Inspecteur : _________________________________ _____ KII avec IEP ; Nom de l’Inspecteur : _________________________________ _____ Focus groupe avec CP ; Noms des Conseillers : _________________________________ _____ KII avec IAENF/IDAENF ; Nom de l’Inspecteur : ________________________________ _____ Visiter l’école _____ KII avec le Directeur; Nom: ______________________________ _____ Focus groupe – Enseignants Nbre. de participants : ______ Hommes: ______ / Femmes : _______ _____ Observation de cours de lecture CI ou CP _____ Focus groupe – Elèves CI Nbre. de participants : ______ Garçons: ______ / Filles : _______ _____ Focus groupe – Elèves CP Nbre. de participants : ______ Garçons: ______ / Filles : _______ _____ Focus groupe – Elèves Gouv. Scolaire Nbre. de participants : ______ Garçons: ______ / Filles : _______ _____ Focus groupe – CGDES Nbre. de participants : ______ Hommes: ______ / Femmes : _______ 67 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV _____ Focus groupe – Leaders communautaires Nbre. de participants : ______ (APE, AME, CGPE) Hommes: ______ / Femmes : _______ _____ Focus groupe – Communauté Nbre. de participants : ______ Hommes: ______ / Femmes : _______ USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 68 EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) Acteurs ciblés DREP/DDEP, IEP, IAENF/IDAENF, CP (Inspection Départementale de l’Alphabétisation et de l’Education Non Formelle) Activités (réussites/défis/contraintes) 1. Depuis quand occupez-vous ce poste ? Quelles sont les principales activités de votre institution ? 2. Pouvez-vous nous expliquer le rôle de votre institution dans les interventions du projet NECS? 3. Quelles sont les principales activités déroulées par USAID/NECS dans votre zone d’intervention ? 4. Qu’avez-vous particulièrement retenu des activités du projet NECS ? 5. De votre point de vue, quels ont été les activités à succès du projet ? Qu’est-ce qui a favorisé cela ? 6. De votre point de vue, quelles ont été les activités qui ont enregistré les plus faibles résultats ? Qu’est-ce qui explique cette situation ? 7. Avec l’intervention de NECS au sein des écoles et des communautés, quels types de changements pouvez-vous partager avec nous ? Lecture [En cas d’exemple, noter le nom de l’école ou du village cité.] 1. De votre expérience, comment appréciez-vous l’approche d’apprentissage de la lecture (ASL) déroulée par NECS dans les établissements ? Comment cela vous a-t-il aidé dans la mise en œuvre des cours, surtout en ce qui concerne l’enseignement de la lecture ? 2. Quelle comparaison pouvez-vous faire entre l’ASL et les autres approches en lecture au Niger (avantages et inconvénients) ? 3. Quels sont les atouts et limites que vous avez relevé dans l’approche ASL ? Quelles sont les suggestions pour l’amélioration de l’ASL ? 4. Pensez-vous que le projet NECS a contribué à rehausser les compétences des enseignants dans l’enseignement de la lecture aux élèves de CI et CP ? Si oui, comment ? Donner des exemples. 5. Pensez-vous que cette nouvelle approche d’enseignement de la lecture prôné par NECS a pu aider les élèves à avoir de meilleures performances en lecture ? Si oui, comment et dans quelle mesure (notes des élèves, assiduité à l’école) ? Formations Plan d’action Code de conduite Mentorat Alphabétisation ASL CGPE CGDES La bibliothèque Lecture publique Séances lectures Gouvernement scolaire Concours de lecture Cours de rattrapage Déparasitage Sensibilisation 69 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV 6. Avez-vous eu échos de changements de comportement ou d’attitudes vis-à-vis de la lecture chez les élèves grâce au NECS ? Si oui, quel genre de changements ? (Participation en classe, concentration sur les enseignements du maître, intérêt pour la lecture) Communauté 1. Comment appréciez-vous la participation de la communauté dans les activités de NECS ? Qu’en est-il de cette participation communautaire dans la gestion des écoles que vous couvrez ? (Entretien des infrastructures, plans d’actions, contributions financières) 2. Avez-vous noté des changements dans la motivation de la communauté autour de l’éducation en général depuis l’arrivée du projet dans le village? Si oui, quel genre de changements ? Donner des exemples. 3. Qu’en est-il de la motivation de la communauté pour la lecture et la scolarisation des filles depuis que le projet NECS est présent dans les villages ? Donner des exemples. 4. Avez-vous remarqué une évolution des relations entre les écoles et les communautés depuis l’arrivée de NECS dans le village ? Si oui, comment cette évolution se manifeste-t-elle? 5. Pouvez-vous nous donner des exemples précis ? Genre 1. USAID/NECS avait pour objectif de promouvoir la scolarisation et le maintien des filles à l’école, quelles observations pouvez-vous faire sur cela ? 2. Avez-vous noté des changements dans le comportement ou attitudes par rapport au « Genre » et à l’Equité » grâce au activités de NECS ? Quels types de changements et chez quels acteurs (élèves, enseignants, parents, etc.) ? 3. Connaissez-vous le Code de Conduite développé par NECS ? Quelles sont vos appréciations concernant ce code par rapport aux changements qu’ils pourrait induire chez les enseignants ? Pouvez-vous partager des exemples précis ? 4. Quels sont les atouts et limites de ce code de conduite ? Suggestions 1. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être ? 2. S’il y’a une chose que vous aurez aimé changer du projet NECS qu’est-ce ce que cela pourrait être ? 3. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir? USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 70 FICHE – VISITE ECOLE Date(s) de visite : Membres de l’équipe d’évaluation : Nom de l’école: Région : Nom du directeur : Date de prise de poste : Structures Financements des infrastructures : ☐ IMAGINE ☐ Etat ☐ Communauté ☐ Autre: _____________ Existence de latrines ☐ Oui ☐ Non Si oui, Latrines séparées ☐ Oui ☐ Non Nbre. de latrines : ______ (filles) ______ (garçons) Latrines sont utilisables ? ☐ Oui ☐ Non Si non, Raison : _______________ Point d’eau Corps d’enseignants Existence point d’eau ☐ Oui ☐ Non Si oui : Construit par IMAGINE/NECS : ☐ Oui ☐ Non Qui est-ce qui fournit l’eau à l’école ? ☐ Elèves ☐ Communauté Qui fait le maintien de ce point d’eau ? Le nettoyage ? S’il y a problème ? Total d’Enseignants : ______ Nbre. Femmes : ________ Nbre. Hommes : ________ Population des élèves Effectif total : ________ G : ________ F : _________ Total CI : ___________ G : ________ F : _________ Total CP: ___________ G : ________ F : _________ Total CE1 : ___________ G : ________ F : _________ Total CE2 : ___________ G : ________ F : _________ Total CM1 : ___________G : ________ F : ________ 71 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Total CM2 : ___________G : ________ F : ________ Documents obtenus (X) ________ Photo de Plan d’action de l’école ________ Photo de Code de conduite genre Observations – Tour de l’école [Remarquer la répartition des tâches, l’état des bâtiments] USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 72 EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS Acteurs ciblés : Directeurs d’écoles Activités (réussites/défis/contraintes) Depuis quand êtes-vous directeur de cette école ? Depuis quand le projet USAID/NECS mène des activités dans cette école ? 1. Quelles sont les principales activités déroulées par USAID/NECS au sein de votre « établissement ? 2. Dans quelles activités de NECS avez-vous pris part ? 3. Qu’avez-vous principalement retenu de la mise en œuvre des activités du projet NECS en vous référant à son historique (de 2012 à aujourd’hui) ? 4. De votre point de vue, quels ont été les activités à succès du projet ? Qu’est-ce qui a favorisé cela ? 5. De votre point de vue, quelles ont été les activités qui ont enregistré les plus faibles résultats/performances ? Qu’est-ce qui explique cette situation ? 6. Avec l’intervention de NECS au sein de l’école, quels types de changements pouvez-vous partager avec nous ? Lecture 1. De votre expérience, comment appréciez-vous l’approche d’apprentissage de la lecture déroulée par NECS dans votre établissement (ASL) ? Comment cela vous a-t-il aidé dans la mise en œuvre des cours, surtout en ce qui concerne l’enseignement de la lecture ? 2. Quels sont les atouts et limites que vous avez relevé dans l’approche ASL ? Quelles sont les suggestions pour son amélioration ? 3. Quelles sont les différentes approches en lecture que vous connaissez en dehors de l’ASL ? Quelle comparaison pouvez-vous faire entre ces différentes approches ? 4. Est-ce que le projet NECS a contribué à rehausser les compétences des enseignants dans l’enseignement de la lecture aux élèves de CI et CP ? Si oui, comment ? Donner des exemples. 5. Pensez-vous que cette nouvelle approche d’enseignement de la lecture prôné par NECS a pu aider les élèves à avoir de meilleures performances en lecture ? Si oui, comment et dans quelle mesure (notes des élèves, assiduité à l’école) ? 6. Remarquez-vous des changements dans le comportement ou les attitudes vis-à-vis de la lecture chez les élèves grâce au NECS ? Si oui, lesquels ? (Participation en classe, concentration sur les enseignements, intérêt pour la lecture) Formations Plan d’action Code de conduite Mentorat Alphabétisation ASL CGPE CGDES La bibliothèque Lecture publique Séances lectures Gouvernement scolaire Concours de lecture Cours de rattrapage Déparasitage Sensibilisation 73 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Communauté 1. Comment appréciez-vous la participation de la communauté dans les activités de NECS ? Qu’en est-il de cette participation communautaire dans la gestion de l’école ? (Entretien des infrastructures, plans d’actions, contributions financières) 2. Voyez-vous des changements dans la motivation de la communauté autour de l’éducation en général depuis l’arrivée du projet dans le village? Si oui, quel genre de changements ? Donner des exemples. 3. Qu’en est-il de la motivation de la communauté pour la lecture et la scolarisation des filles depuis que le projet NECS est dans le village ? Donner des exemples. 4. Avez-vous remarqué une évolution des relations entre l’école et la communauté depuis l’arrivée de NECS dans le village ? Si oui, comment cette évolution se manifeste-t-elle? Genre 1. USAID/NECS avait pour objectif de promouvoir la scolarisation et le maintien des filles à l’école, quelles observations pouvez-vous faire sur cela ? 2. Remarquez-vous des changements dans le comportement ou les attitudes par rapport au « Genre » et à l’Equité » grâce au activités de NECS ? Quels types de changements et chez quels acteurs (élèves, les enseignants, les parents, vous-même) ? 3. Connaissez-vous le Code de Conduite développé par NECS ? Quelles sont vos appréciations concernant ce code par rapport aux changements induits ? Quels sont ses atouts et limites ? Suggestions 1. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être ? 2. S’il y’a une chose que vous aurez aimé changer du projet NECS qu’est-ce ce que cela pourrait être ? 3. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir ? USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 74 EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS Acteurs ciblés : Enseignants CI et CP Activités (réussites/défis/contraintes) 1. Depuis quand enseignez-vous dans cette école ? Depuis quand le projet USAID/NECS mène des activités dans cette école ? 2. Quelles sont les principales activités déroulées par USAID/NECS au sein de cet établissement ? 3. Quelles sont les activités de NECS qui concernent directement les enseignants ? 4. De toutes les activités de NECS, laquelle vous a le plus marqué en tant qu’enseignant et pourquoi ? 5. De votre point de vue, quels ont été les activités à succès du projet ? Qu’est-ce qui a favorisé cela ? 6. De votre point de vue, quelles ont été les activités qui ont enregistré les plus faibles résultats ? Qu’est-ce qui explique cette situation ? 7. Avec l’intervention de NECS au sein de l’école, quels types de changements pouvez-vous partager avec nous aujourd’hui ? Lecture 1. Comment appréciez-vous l’approche d’apprentissage de la lecture déroulée par NECS dans votre établissement (ASL) ? Comment cela vous a-t-il aidé dans les leçons, surtout en ce qui concerne l’enseignement de la lecture ? 2. Quels sont les atouts et limites que vous avez relevé dans l’approche ASL ? Quelles sont les suggestions pour son amélioration ? 3. Est-ce que le projet NECS a contribué à rehausser vos compétences en tant qu’enseignants dans l’enseignement de la lecture aux élèves de CI et CP ? Si oui, comment ? Donner des exemples. 4. Pensez-vous que cette nouvelle approche ASL prôné par NECS a pu aider les élèves à avoir de meilleures performances en lecture ? Si oui, comment et dans quelle mesure (notes des élèves, assiduité à l’école, etc.) ? 5. Remarquez-vous des changements dans le comportement ou les attitudes des élèves vis-à-vis de la lecture grâce à l’ASL en particulier et au NECS en général ? Si oui, lesquels ? (Participation en classe, concentration sur les enseignements, intérêt pour la lecture) Communauté Formations Plan d’action Code de conduite Mentorat Alphabétisation ASL CGPE CGDES La bibliothèque Lecture publique Séances lectures Gouvernement scolaire Concours de lecture Cours de rattrapage Déparasitage Sensibilisation 75 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV 1. Comment appréciez-vous la participation de la communauté dans les activités de NECS ? Qu’en est-il de cette participation communautaire dans la gestion de l’école ? (Entretien des infrastructures, plans d’actions, contributions financières) 2. Voyez-vous des changements dans la motivation de la communauté autour de l’éducation en général depuis l’arrivée du projet dans le village? Si oui, quel genre de changements ? Donner des exemples. 3. Qu’en est-il de la motivation de la communauté pour la lecture et la scolarisation des filles depuis que le projet NECS est dans le village ? Donner des exemples. 4. Avez-vous remarqué une évolution des relations entre l’école et la communauté depuis l’arrivée de NECS dans le village ? Si oui, comment cette évolution se manifeste-t-elle? Genre 1. USAID/NECS avait pour objectif de promouvoir la scolarisation et le maintien des filles à l’école, quelles observations pouvez-vous faire sur cela ? 2. Remarquez-vous des changements dans le comportement ou les attitudes par rapport au « Genre » et à l’Equité » grâce au activités de NECS ? Quels types de changements et chez quels acteurs (élèves, les enseignants, les parents, vous-même) ? 3. Connaissez-vous le Code de Conduite développé par NECS ? Quelles sont vos appréciations concernant ce code par rapport aux changements induits ? Quels sont ses atouts et limites ? Suggestions 1. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être ? 2. S’il y’a une chose que vous aurez aimé changer du projet NECS qu’est-ce ce que cela pourrait être ? 3. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir ? USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 76 Guide d’observation à l'école Informations globales: Nom de l'enseignant: Date: Niveau: ☐ CI ☐ CP Titre de leçon observée: Effectif de la classe: _______ Nombre de filles: _______ Nombre de garçons: _______ Nombre d'élèves présents: _______ No. de filles présentes: ______ No. de garçons présents:______ Disposition de la classe: Assez de tables-bancs pour tous? ☐Oui ☐ Non Garçons et filles s’assoient ensemble ? ☐ Oui ☐ Non Combien de filles sans place? ________ Combien de garçons sans place? _______ La lecture : Utilisation du guide Enseignant NECS : ☐ Oui ☐Non Nbre. d’élèves avec livre NECS : _____ Déroulement de la leçon : ☐ L’enseignant lit ☐ Identification des lettres par les enfants ☐ Lecture par les enfants ☐ Récitation ☐ Chant ☐ Autre : _____________ La participation des élèves : Mettre un « X » à chaque fois qu’un élève agit. Fille Garçon Total L'enseignant interroge l'élève L'élève prend la parole L'élève va au tableau 77 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV L'enseignant regarde le travail de l'élève La discipline des élèves: Noter chaque fois que l'enseignant(e) discipline l'un des élèves. En utilisant la grille suivant, noter la nature de l'interaction et si l'élève corrigé est garçon ou fille. Sexe de l'élève Conduite sanctionnée Nature de la discipline Fille Garçon Parler avec l'élève Humiliation verbale Suspendre la récréation Correction corporelle Tâches administratives et d'entretien: Responsable de classe: ☐ Fille ☐ Garçon Adjoint: ☐ Fille ☐ Garçon Que font les élèves pour aider l'enseignant(e)? Mettre un « X » pour chaque élève selon la colonne qui convient (fille ou garçon). Tâche Filles Garçons Effacer le tableau Distribuer les cahiers Balayer la salle de classe Apporter de l'eau Autre : _____________________________ Autre : _____________________________ Remarque : USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 78 EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group) ☐ CI ☐ CP Sélection: Se servir de la liste nominative de la classe pour choisir de façon aléatoire de 6 élèves (3 filles et 3 garçons). Emplacement : Faire assoir les élèves en ordre en formant un ‘’U’’ si possible en classe. - Inviter chaque élève à se présenter en indiquant son nom. - Etablir une relation de confiance par des jeux, chants etc. Consignes A Noter durée Thème Modérateur Réponses des Elèves Filles Réponses des Elèves Garçons 5mn Petite introduction Demander aux élèves ce qu’ils aimeront devenir quand ils sont grands. 1) _______________________ 2) _______________________ 3) _______________________ 1) _______________________ 2) _______________________ 3) _______________________ 79 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Les perceptions sur la lecture Pourquoi selon vous il est important d’apprendre à Lire ? 1) _______________________ 2) _______________________ 3) _______________________ 1) _______________________ 2) _______________________ 3) _______________________ Consignes A Noter durée Thème Modérateur Réponses des Elèves Filles Réponses des Elèves Garçons 5 mn Familiarité avec matériel de lecture • Qui est-ce qui a une histoire à nous raconter ? • [A poser à 2 enfants] Quel est ton histoire préféré lue à l’école ? Nombre : ___________ Réponses : Nombre : ___________ Réponses : [Présente la copie de l’histoire. Demander] • Qui connait cette histoire ? Nombre : ___________ Nombre : ___________ [Présente plusieurs livres dont ceux du (niveau CI-CP) et demande aux élèves :] • Qui peut identifier les livres dans lesquels ils ont lu cette histoire ? Observations : USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 80 5mn Habitude de lecture à la maison • Qui a des livres à la maison ? Nombre : ___________ Nombre : ___________ • C’est quel genre de livre ? Note les réponses. Note les réponses. • D'habitude, qui lit avec toi à la maison? Note les réponses. Note les réponses. • En quelle langue? Note les réponses. Note les réponses. • Est-ce que vous lisez ensemble chaque jour? (Lever le doigt.) Nombre de « oui »: ___________ Nombre de « oui » : ___________ Compétences Exercices [Prendre note. Compter les bonnes réponses parmi toutes les réponses.] 15 mn Décodage - Faire tirer au sort les lettres d’alphabet par les 6 élèves et leur demander de lire l’un après l’autre Nmb. Trouvé : __________ Nmb. Non-trouvés : __________ Nmb. Trouvé : __________ Nmb. Non-trouvés : __________ - Faire tirer des syllabes. Nmb. Trouvé : __________ Nmb. Non-trouvés : __________ Nmb. Trouvé : __________ Nmb. Non-trouvés : __________ - Faire tirer des mots à tour de rôle et demander à chacun de lire. Nmb. Trouvé : __________ Nmb. Non-trouvés : __________ Nmb. Trouvé : __________ Nmb. Non-trouvés : __________ 81 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Demander à plusieurs élèves à tour de rôle : - Qui peut lire cette phrase pour moi ? (choisir des phrases courtes) ? Nmb. Trouvé : __________ Nmb. Non-trouvés : __________ Nmb. Trouvé : __________ Nmb. Non-trouvés : __________ USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 82 EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) Acteurs ciblés : Staff NECS/Aide-et Action A. Historique 1. Depuis quand êtes-vous dans le projet ? • Pouvez-vous brièvement nous parler de votre expérience avec NECS ? 2. On a noté que les objectifs stratégiques de NECS ont évolué et ont fait l’objet de plusieurs modifications, pouvez-vous nous en retracer l’historique ? • Qu’est-ce qui a conduit à ces types de changements et comment se sont-ils traduits dans les performances de NECS ? 3. Qu’avez-vous principalement retenu de la mise en œuvre des activités du projet NECS en vous référant à son historique (de 2012 à aujourd’hui) ? B. Réussites/Défis 1. De votre point de vue, quels ont été les activités à succès du projet ? • Qu’est-ce qui a favorisé cela ? • Selon vous quel est l’activité que NECS a le mieux réussi durant ces six (06) dernières années de mise en œuvre ? • Qu’est-ce qui avait favorisé cela (facteurs explicatifs)? • Comment visiblement cela s’est traduit au niveau des écoles et communautés concernées ? 2. De votre point de vue, quelles ont été les activités qui ont enregistré les plus faibles résultats/performances ? Qu’est-ce qui explique cette situation ? • Quels ont été les principales contraintes/défis qui ont conduit à ces faibles performances ? • Quelles ont été les conséquences au niveau des écoles et communautés ? • Quelles ont été les stratégies mises en œuvre par le projet pour contourner ces défis ? C. Lecture 1. Quelles sont les expériences que vous pouvez partager avec nous concernant la composante « Lecture » du projet ? (Outils pédagogiques et démarches/méthodes; formations) • Régions • Hard/Soft/Billingues • Langues 83 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV 2. Quelles sont les grandes leçons que vous en avez tiré ? 3. Globalement, quelle appréciation faites-vous de l’approche ASL ? 4. Quelles ont été ses répercussions sur les performances en lecture des élèves ainsi que les compétences pédagogiques des enseignants? 5. Quelle comparaison faites-vous entre l’approche ASL et les autres approches en lecture mises en œuvre au Niger ? Avantages/Inconvénients de chaque approche ? D. Communauté 1. Quelles appréciations faites-vous du niveau de motivation/engagement des communautés par rapport à l’éducation et à la lecture telle qu’abordé par NECS ? (L’installation d’une culture de lecture) 2. Quels sont les changements majeurs notés dans les relations entre les communautés et le corps enseignant grâce l’intervention de NECS ? Quels ont été les principaux défis dans l’amélioration de ces relations ? E. Genre 1. Qu’avez-vous particulièrement retenu des activités visant la promotion de la scolarisation des filles ? 2. Quels ont été les grandes réussites du projet en matière de genre ? 3. Quels ont été les défis majeurs durant la mise en œuvre des activités relatives au genre ? F. Suggestions 1. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être? 2. S’il y’a une chose que vous aurez aimé changer ou améliorer du projet NECS qu’est-ce ce que cela pourrait être ? 3. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir (dans le cadre d’une mise à l’échelle par exemple) ? USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 84 EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) Acteurs ciblés: USAID/NECS AOR, Education Officer, MCC/Director of Threshold Historique 1. Quelle est votre rôle au sein du projet NECS ? 2. Pouvez-vous nous rappeler l’historique/le processus qui a conduit au financement du projet NECS ? 3. On a noté que les objectifs stratégiques de NECS ont évolué et ont fait l’objet de modifications, qu’est-ce qui a conduit à ces types de changements ? Réussites/Défis 1. Selon vous quel est l’objectif que le NECS a le mieux atteint à ce jour ? Qu’est-ce qui avait facilité la réalisation de l’atteinte de cet objectif ? Remarquez-vous des différences entres les écoles hard/soft/bilingues ? 2. De votre point de vue, quel est l’objectif de NECS qui enregistre le moins de résultats à ce jour ? Quelle en est l’explication selon vous ? (hard/soft/bilingues) 3. Quels ont été les principales contraintes/difficultés observées dans la mise en œuvre des différentes activités de NECS ? Comment ces difficultés ont-elles été contournées durant les années de mise en œuvre ? 4. Quels ont été les principaux défis auxquels [l’USAID/MCC] a été confronté ? Qu’est-ce qui a été fait pour les contourner et comment cela s’est traduit au fur et à mesure dans l’atteinte des résultats du projet ? Lecture 1. Quelle appréciation faites-vous de l’approche ASL sur les performances en lecture des élèves ainsi que les compétences éducatives des enseignants (outils pédagogiques et démarches/méthodes; formations) ? 2. Comment comparez-vous l’approche ASL avec les autres approches en lecture déroulées au Niger ? Avantages/Inconvénients ? Communauté 1. Quelles appréciations faites-vous du niveau de motivation des communautés par rapport à l’éducation et à la lecture telle qu’abordé par NECS ? (L’installation d’une culture de lecture) 2. Quels sont les changements majeurs notés dans les relations entre les autorités éducatives au niveau décentralisés et les communautés avec l’intervention de NECS ? 85 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Genre 1. Par rapport à l’objectif de promouvoir la scolarisation des filles, comment voyez-vous les points forts du NECS ? Quelles sont les points faibles ? Recommandations 1. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être? 2. S’il y’a une chose que vous aurez aimé changer ou améliorer du projet NECS qu’est-ce ce que cela pourrait être ? 3. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir (dans le cadre d’une mise à l’échelle par exemple) ? USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 86 EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group) Acteurs ciblés : Gouvernement scolaire (GS) [Discuter avec les enfants pour les mettre à l’aise. Jouer. Taquiner.] 1. Pouvez-vous vous présentez (âge, classe, poste) 2. Qu’est-ce que vous faites comme activités ? Pourriez-vous décrire les activités du GS ? 3. Depuis combien de temps participez-vous au GS ? 4. Combien de filles et combien de garçons y’a-t-il dans le GS ? 5. A quelle périodicité se tiennent les réunions du GS ? Avez-vous participé à la dernière réunion ? 6. Participez-vous dans d’autres activités de NECS ? Si oui, lesquelles ? 7. Quels sont les types de problèmes que gèrent le GS ? Formations Plan d’action Code de conduite Mentorat Alphabétisation ASL CGPE CGDES La bibliothèque Lecture publique Séances lectures Gouvernement scolaire Concours de lecture Cours de rattrapage Sensibilisations Déparasitage 87 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) Communautés (AME, APE, CGPE, CGDES) Présentation Demander à chacun de se présenter (nom, poste occupé dans son organisation). [Noter combien de femmes, combien d’hommes.] Quelles sont les instances qui composent le bureau ? 1. A quelle fréquence tenez-vous les réunions ? De quand date la dernière réunion ? 2. Quelles sont les activités que vous menez en dehors des réunions ? Activités/Partenariat avec NECS 1. En tant qu’organisation communautaire, quelles sont les principales activités déroulées dans le cadre du projet NECS? • Pouvez-vous nous expliquer le déroulement de ces activités? • Quelle était votre participation (niveau d’implication de votre organisation) ? • Quelles ont été les activités à succès du projet dans votre village ? Qu’est-ce qui explique cela ? • Quelles ont été les activités qui n’ont pas bien marché dans votre village ? Qu’est-ce qui explique cela? Initiatives Locales pour l’éducation 1. Quelles ont été les initiatives prises par votre organisation pour la gestion de l’école ? (entretien, maintenance des infrastructures, constructions) 2. En tant qu’organisation, est-ce que vous faites des contributions financières au niveau de l’école ? 3. Quelle a été la contribution de votre organisation en faveur de la scolarisation des filles ? (Perceptions communautaires – maintien des filles à l’école) 4. Quel genre d’actions avez-vous mené pour renforcer la promotion de la lecture dans le village ? (Parents / élèves) Effets du NECS (Lecture, Equité/Genre, Scolarisation des filles) 1. Quels sont les changements (importants de votre point de vue) que vous avez relevés depuis l’arrivée du projet NECS au sein de votre communauté ? Formations Plan d’action Code de conduite Mentorat Alphabétisation ASL CGPE CGDES La bibliothèque Lecture publique Séances lectures Gouvernement scolaire Concours de lecture Cours de rattrapage Déparasitage USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 88 2. Avec l’arrivée de NECS comment les liens entre l’école et la communauté ont été renforcé ? Depuis l’arrivée de NECS comment participez-vous dans l’éducation de vos enfants ? Recommandations 1. Quelles sont vos suggestions pour la prochaine phase de NECS s’il y en a ? (A conserver ? A améliorer ? A supprimer ?) 89 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) Communautés (Membres de la communauté) Présentation Demander à chacun de se présenter (nom, activité dans la communauté). Demander « Qui sont ceux qui ont des enfants à l’école ? » [Ils lèvent le doigt. Noter le nombre de personnes.] Activités 1. En tant que membre de la communauté, quelles sont les activités déroulées par le projet NECS dans votre village ? • Pouvez-vous nous expliquer le déroulement de ces activités? • Quelle était votre participation (niveau d’implication de la communauté) ? • Quelles ont été les activités à succès du projet dans votre village ? Qu’est-ce qui explique cela ? • Quelles ont été les activités qui n’ont pas bien marché dans votre village ? Qu’est-ce qui explique cela? Effets du NECS (Lecture, Equité/Genre, Scolarisation des filles) 2. Quels sont les changements (importants de votre point de vue) que vous avez relevés depuis l’arrivée du projet NECS ? 3. Avec l’arrivée de NECS comment les liens entre l’école et la communauté ont été renforcé ? • Comment participez-vous dans l’éducation de vos enfants ? 4. Que pensez-vous de la présence des filles à l’école ? • Est-ce que ceci a toujours été votre position avant l’arrivée de NECS ? Si non, qu’est-ce qui a favorisé cette prise de conscience ? • Quelle importance voyez-vous dans les études que vos filles mènent à l’école ? • Est-ce que la plupart des filles en âge d’aller à l’école y sont? 5. Selon vous, quelle est l’importance de la lecture pour vos enfants ? 6. Croyez-vous que les études au-delà de l’école primaire sont importantes pour vos enfants ? Pour quoi ? Est-ce que c’est différent pour la fille, comparé au garçon ? Formations/sensibilisations Programme de mentorat Alphabétisation Comité de gestion Comité de gestion de point d’eau La bibliothèque Groupe de lecture Séances lectures Concours lecture USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 90 7. Comment voyez-vous aujourd’hui la scolarisation et le maintien des filles au sein des écoles ? Recommandations 8. Quelles sont vos suggestions pour la prochaine phase de NECS s’il y en a ? (A conserver ? A améliorer ? A supprimer ?) 91 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) Acteurs ciblés: USAID/NECS AOR, Education Officer, MCC/Director of Threshold Historique 1. Quelle est votre rôle au sein du projet NECS ? 2. Pouvez-vous nous rappeler l’historique/le processus qui a conduit au financement du projet NECS ? 3. On a noté que les objectifs stratégiques de NECS ont évolué et ont fait l’objet de modifications, qu’est-ce qui a conduit à ces types de changements ? Réussites/Défis 5. Selon vous quel est l’objectif que le NECS a le mieux atteint à ce jour ? Qu’est-ce qui avait facilité la réalisation de l’atteinte de cet objectif ? Remarquez-vous des différences entres les écoles hard/soft/bilingues ? 6. De votre point de vue, quel est l’objectif de NECS qui enregistre le moins de résultats à ce jour ? Quelle en est l’explication selon vous ? (hard/soft/bilingues) 7. Quels ont été les principales contraintes/difficultés observées dans la mise en œuvre des différentes activités de NECS ? Comment ces difficultés ont-elles été contournées durant les années de mise en œuvre ? 8. Quels ont été les principaux défis auxquels [l’USAID/MCC] a été confronté ? Qu’est-ce qui a été fait pour les contourner et comment cela s’est traduit au fur et à mesure dans l’atteinte des résultats du projet ? Lecture 3. Quelle appréciation faites-vous de l’approche ASL sur les performances en lecture des élèves ainsi que les compétences éducatives des enseignants (outils pédagogiques et démarches/méthodes; formations) ? 4. Comment comparez-vous l’approche ASL avec les autres approches en lecture déroulées au Niger ? Avantages/Inconvénients ? Communauté 3. Quelles appréciations faites-vous du niveau de motivation des communautés par rapport à l’éducation et à la lecture telle qu’abordé par NECS ? (L’installation d’une culture de lecture) 4. Quels sont les changements majeurs notés dans les relations entre les autorités éducatives au niveau décentralisés et les communautés avec l’intervention de NECS ? USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 92 Genre 2. Par rapport à l’objectif de promouvoir la scolarisation des filles, comment voyez-vous les points forts du NECS ? Quelles sont les points faibles ? Recommandations 4. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être? 5. S’il y’a une chose que vous aurez aimé changer ou améliorer du projet NECS qu’est-ce ce que cela pourrait être ? 6. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir (dans le cadre d’une mise à l’échelle par exemple) ? 93 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) Autres partenaires techniques et financiers (Concern, Save the Children, JICA, World Bank, Swiss Cooperation, UNICEF, UNESCO and OXFAM) 1. Quel est l’expérience de votre organisation dans le déroulement des projets de lecture ? 2. Pouvez-vous nous décrire brièvement vos principales approches en matière de « Lecture » au Niger ? Qu’identifiez-vous comme les points forts et les points faibles de ces approches ? 3. Comment percevez-vous le contexte nigérien par rapport aux programmes et initiatives dans le secteur de l’éducation ? Pour les programmes de lecture, en particulier ? 4. Connaissez-vous l’approche ASL ? Quelles sont les appréciations que vous pouvez faire aujourd’hui de l’approche ASL ? Comment comparez-vous l’approche ASL avec les approches utilisée par votre organisation ? Par rapport aux approches en matière de lecture en générale ? 5. En ce qui concerne la mise à l’échelle des projets de lecture, que voyez-vous comme atout ? Comme défis ? 6. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir ? En particulier, dans le cadre d’une mise à l’échelle ? USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 94 ANNEX III: DETAILED FIELDWORK SCHEDULE Part I. NECS FINAL PERFORMANCE EVALUATION - FIELD WORK SCHEDULE (Niamey) Dates Tasks/ Interviews Tim es Contact Person Title Address E-mail Phone number May 21 Bescaye Diop Regional M&E Specialist USAID/Senegal/S RO bdiop@usaid.gov May 25 USAID Niamey/SRO Ahmed Tidiane Diallo Evaluation POC USAID Niger Mission tadiallo@usaid.gov Garett Harries Office Chief USAID/Niger Mission gharries@usaid.gov Aboubacar Kossomi AOR USAID/Niger Mission akossomi@usaid.gov Hadjara Moussa USAID/Niger Mission mhadjara@usaid.gov Jennifer Swift￾Morgan 95 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Nana Serwah Amoako AOR, USAID Accra Regional Office Cornelia Tremann Yves Korea Former AOR May 26-27 Weekend – review/refine data collection instruments/train enumerators/FG moderators May 28 Plan International 9.00 Chief of Party 10.00 M&E Specialist 11.00 Reading Specialist Plan International Washington USAID Washington Reading Team/Specialist Aide et Action Internationale 12.00 Tcha Berei Country Director USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 96 Aide et Action Internationale Moussa Daoda, Tando Ibrahim, Omar Coordinateurs d’éducation School-to-School International (STS) 15.00 May 29 Ministry of Primary Education 9.00 Direction de l’Appui à la Gestion de l’Ecole (DAGE) Direction des Curricula et de la Réforme de l’Enseignement (DCRE) Direction de l’Enseignement du Cycle Primaire (DECP) Direction of Quality Monitoring and Evaluation (DESQ) Direction de l’Evaluation, du suivi des 97 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV acquis scolaires et de l’orientation (DESASO) Departmental Primary Education Direction (DDEP) Direction of Reform, Promotion of National Languages (DGEP) Direction for Promotion of Girls’ Education (DPSF) Regional Primary Education Direction (DREP) May 30 Travel to Regions May 30 ELAN 10.00 CONCERN PTFs May 31 June 1 USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 98 June 2- 3 WEEKEND June 4 Arrival of Reading Expert, Gaelle Simon June 5 Reading Expert travels to field to join teams June 6 June 7- 8 WEEKEND – review data/transcripts June 9 Return to Niamey June 11-14 Data analysis/preperation of FC matrix and presentation June 15 Data walk with USAID Niamey 8.30 99 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV June 18 Presentation of findings - Plan and MEP Recommendations Workshop June 20 Departure from Niamey 103 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Part II: School Visit Schedule 5/26/2018 5/27/2018 5/28/2018 5/29/2018 5/30/2018 5/31/2018 6/1/2018 6/2/2018 6/3/2018 6/4/2018 6/5/2018 6/6/2018 6/7/2018 6/8/2018 6/9/2018 Madaoua/ourno Madaou/Bangui Koni /Allela Malbaza/malbaza Gothéye Dargol Balley/Tagazar Balley/TagazarFinligué/filingue Finligué/filingue Boudé Kannia Maïkourou Nobba Kanjiwa Kodey Bazaga Bilingue Zarmey bilingue Briefing Voyage Guidan Ider Dargol Quartier retour à Niamey Farayé Bande Bilingue Zinder-Maradi Chago Tsararaou Allah Sarki Dougueré Mai Gao Saber Soubdou Centre Maiguigé Karfi Baban Anné Madarounfa Ecole quartier gouré/gouré gouré/gouré gouré/gouré Magaria /Bande Magaria /Bandé Tessaoua/Tessaoua ourafne/ourafant Tessaoua/Tessaoua Aguie/Aguie Madarounfa /Madarounfa Retour sur Niamey Ecole Hard Ecole Soft Ecole Bilingue Légende: USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 104 ANNEX II: EVALUATION METHODS & LIMITATIONS A team of five people worked collaboratively to produce this evaluation. They included three international experts, one M&E associate and a Nigerien evaluator. Lisa Slifer-Mbacke, currently MEP COP and specialist in the area of education, provided methodological orientation prior to data collection, and backstopping throughout the data collection, analysis and report writing phases. Karla Giuliano Sarr, independent consultant and education expert, served as Team Leader and led the in-country field work and preliminary analysis phase, formal data analysis and report writing. Gaëlle Simon, MSI Technical Manager, served as the Reading Expert for the team and focused on the comparative advantage of the NECS reading approach as compared to the other current education quality programs. Aissatou Mbaye Sy, MEP M&E Manager, served as overall Task Manager for this evaluation. She participated in field research, supported scheduling of field activities and as Task Manager, was responsible for ensuring that all deliverable deadlines were met by the team. From Niger, Abdoulaye Tambandia, independent consultant, provided local expertise, overseeing sampling choices, identifying local contractors to serve as data collectors and provide transcription and offering local insights throughout the process. Due to security concerns, but notably, the high volume of interviews with education stakeholders in Niamey, including the MEP, USAID, Plan and other education providers, the Team Leader and Reading Specialist remained in Niamey while the MEP M&E Associate and the Nigerien Evaluator led data collection activities at the school-level outside of Niamey. The bulk of data collection took place between May 25th and June 21st with intensive school-level data collection occurring during a fifteen day period between May 29th and June 9th. Additional reinforcement interviews, as well as phone interviews with key stakeholders whose schedules had not allowed for earlier contact continued through July 3rd. DATA COLLECTION METHODS This performance evaluation utilized document review, and both qualitative and quantitative methods to assemble data required for arriving at findings and conclusions. The five evaluation questions drove methodological decisions. (Copies of tools are available in annexes X). • Document Review: the evaluation team reviewed and analyzed activity design and implementation documents, adding up to hundreds of pages. In addition, the evaluation team collected relevant program documents and analyzed them throughout the data collection and analysis phases. Review of documents provided an understanding of activity objectives and implementation characteristics, contexts, and challenges, and helped to identify and describe achieved activity results. Close review of curriculum documents provided insights into reading methodologies and approaches. • Qualitative methods: the evaluation team conducted key informant interviews with USAID/Niger, USAID/SRO and USAID/Washington, MCC as well as host country MEP officials, and senior staff of Plan both in Niger and in their home office, activity partners such as Aide et Action, Readsters, and School to School (STS). In addition, the team met with other reading program implementers (ELAN, Concern, Save the Children and IRC). Focus groups were organized with various community organizations as well as community leaders. In total, the team held 174 KIIs and 70 FGDs with 692 participants. Both KIIs and FGDs followed a semi-structured approach where tools served as guides for discussion. The team also conducted classroom observations of reading lessons within 19 of 20 schools visited as well as a brief non-diagnostic reading exercise with groups of 6 students (3 girls, 3 boys) in all 20 schools. See Table 1 for additional information. 105 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV  Participatory methods: In addition, the team held several meetings with key stakeholders that allowed for more participatory data review and analysis. These activities included a working session focused on “Scaling-Up ASL” with key MEP and NECS stakeholders based on the MSI Scalability Checklist (see Annex IX), a Data Walk with USAID, Presentations of Preliminary Findings and Conclusions, and a Recommendations workshop. Each of these sessions allowed for member checking as well as additional data collection. • Quantitative methods: Although the team attempted to use data from the Mathematica baseline evaluation in 2008 and 2016 as well as the NECS 2017 and 2018 EGRA assessments to conduct a comparative analysis of reading fluency from the start of the activity to the end of the program, this comparison was not possible (see Limitations below). The team did explore the number of targets achieved in the activity’s AMELP and identified level of differentiation between targets and actuals. Table 1: Interview & Participant Summary Table Stakeholder Type KIIs FGDs Reading exercice Totals Males Females Totals MEP 22 0 0 22 18 4 22 USAID - Niger 4 0 0 2 3 1 4 USAID - Washington 2 0 0 2 0 2 2 USAID - SRO 2 0 0 2 1 1 2 MCC 1 0 0 1 0 1 1 Other NGOs operating reading program 6 0 0 6 3 3 6 Plan (Niger & USA) 2 0 0 2 2 0 2 NECS Reading Sub￾contractors 4 0 0 4 1 3 4 NECS activity 14 2 0 16 13 0 13 CGDES 0 14 0 14 51 15 66 Community leaders 5 0 0 5 5 0 5 Community members (not leaders) 0 29 0 29 109 86 195 DREP/ DDEP 12 0 0 12 12 0 12 School Directors 17 0 0 17 16 1 17 Teachers 27 5 0 32 17 17 34 USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 106 Inspectors (IEP/IAENF) 21 0 0 21 21 0 21 Pedagogical Advisor (CP) 13 0 0 13 13 0 13 SCOFI 14 0 0 14 1 13 14 Point Focal CGDES 7 0 0 7 6 1 7 Students - School Government 0 20 0 20 58 74 132 Students (CI or CP) 0 0 20 20 60 60 120 Totals 174 70 20 262 411 282 692 The identification of potential data collection methods and sources appropriate for each evaluation question is presented in Table 2 below. Table 2: Data Collection Methods & Sources EVALUATION QUESTIONS DATA COLLECTION METHODS DATA SOURCE(S) 6) To what extent has NECS achieved its objectives in relation to its targets? Document review Key informant interviews (KIIs) Plan Niamey, Washington and field-based staff, USAID interviews semi-structured interviews (may be easier to do this in small groups) School Visit Checklist - NECS M&E plans; quarterly and annual reports Plan 2016 baseline report developed by STS - Mathematica Study Report - additional USAID, Plan reports (trip reports, internal auto-evaluations, etc.) - KIIs (MoE, DPSF, DECP, DESQ, DESASO, DDEP, DGEP, Education donors and partners) - School tour; Observation 7) How did the NECS “ASL” approach enhance reading outcomes in NECS Schools? Document review of Plan/STS descriptions and review of training and classroom modules. Review of auto-evaluation EGRAs and EGRA results since introduction of ASL in 2015. KIIs and Group interviews at community/school level - Plan and STS reports - KIIs with Plan/STS reading staff and M&E staff - KIIs and Group interviews with teachers, head teachers and supervisors - Focus groups with students (CI/CP; School governance committee) -Focus groups with community - Classroom Observation 107 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV 8) In terms of scalability, what are the comparative advantages of the ASL approach as compared to the other reading approaches in Niger? KIIs Document review Scalability Checklist analysis of the activity - KIIs MoE, DAGE, DECP, DESQ, DESASO, DDEP, DGEP, USAID, Plan, donors in education and technical education quality projects) 9) What types of changes are visible in NECS schools and communities? Focus groups; group interviews and KIIs - KIIs (education inspectors, Plan staff in Niamey and field) - FGDs and KIIs with School Management Committees, mothers’ associations, parents, students. SCOFIs, teachers, and head teachers, students - Tour of the school - Classroom Observation 10) To what extent did the project successfully target girls? Document review KIIs Focus groups and group interviews - KIIs (MoE, DPSF, DCRE) education inspectors, Plan staff in Niamey and field) - School Management Committees, mothers’ associations, SCOFIs, students, parents, teachers, and head teachers - Classroom Observation SAMPLING Purposeful sampling methods guided the qualitative data sample choices. Quantitative data derives from previous assessments and evaluations. The qualitative data includes interviews with key stakeholders in unique positions (such as the Plan COP and numerous GON officials) as well as visits to a number of NECS+ schools. The sample includes 20 of the 160 current NECS+ schools. Criteria for selection included region, type of school (hard, soft and bilingual) as well as being sure to include schools from all four languages represented within NECS+ schools. The denomination “hard” indicates that these schools benefitted from the IMAGINE project as well as NECS. “Soft” indicates only the presence of the NECS activity. The sample identifies 5 schools in each of the four regions (Maradi, Tahoua, Tillaberi and Zinder) also within reasonable distance from Niamey as necessary because of security concerns. Of the schools in the sample, 9 are hard, 7 are soft and 4 are bilingual, noting that bilingual schools did not benefit from the IMAGINE project and are therefore also soft schools. In terms of language of instruction, the majority (12) of schools offer instruction in Haussa, 5 in Zarma, 2 in Kanuri and 1 in Fulfulde. The table below provides detailed information about the schools within the sample. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 108 Table 3: Sample School Characteristics No. Nom du village Type de Site (X) Region Langue(s) Hard Soft Biling. 1 Kanjiwa X Tahoua Haussa 2 Guidan Ider X Tahoua Haussa 3 Bazaga Bilingue X Tahoua Haussa 4 Maïkourou X Tahoua Haussa 5 Boudé X Tahoua Haussa 6 Kannia X Tillaberi Zarma 7 Nobba X Tillaberi Zarma 8 Zarmey bilingue X Tillaberi Zarma 9 Dargol Quartier X Tillaberi Zarma 10 Banizoumbou 1 X Tillaberi Zarma 11 Maiguigé Karfi X Maradi Haussa 12 Tsararaou Allah Sarki X Maradi Haussa 13 Saber X Maradi Haussa 14 Baban Anné X Maradi Haussa 15 Tatsegouma Saboua X Maradi Haussa 16 Dougueré Mai Gao X Zinder Fulfulde 17 Chago X Zinder Kanuri 18 Farayé X Zinder Haussa 19 Bande bilingue X Zinder Haussa 20 Soubdou Centre X Zinder Kanuri DATA ANALYSIS Data analysis drew on findings organized by evaluation question, facilitated by standardized data collection instruments attached. Qualitative data collected via the key informant interviews and FGDs were analyzed using thematic and content analysis, with categories developed inductively. Analysis of all data collection activities began in the field, as team members typed up interviews, noting at the top 109 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV of each interview guide any interesting or significant points, and having a round-up meeting to determine the efficacy and consistency of the data collection. A local transcribing agency worked with recordings of a select number of interviews to develop transcriptions. Preliminary in-country data analysis used a findings and conclusions rubric to analyze the data and capture emerging themes and/or patterns. Once the team had completed the thematic analysis, it analyzed key findings and themes using content analysis. Content analysis helped the team go beyond descriptions of changes in practices and attitudes to identify the most salient characteristics of these changes, as a result of NECS’ interventions. In order to achieve an even deeper level of analysis, the team leader used NVivo qualitative analysis software to systematically review and code interviews at the central level as well as KIIs with teachers and school directors. Detailed descriptive and analytical notes and the Preliminary Findings and Conclusions Table served as the basis for analysis for remaining interviews. The evaluation team used data triangulation, the use of two or more data sources, to strengthen findings or identify areas of divergence. In order to illuminate the validity of findings, a Likert scale indication (a few, some, many, the majority) is used to identify scope and where possible, percentages of respondents (for example, 60%, or 6/10 respondents) will be included in the narrative. Wherever appropriate, analytical graphs showing descriptive statistics (frequencies and frequency distributions) and other visual depictions provide more visual reinforcement to the narrative analysis of findings. A chronograph is an analytical tool that has proven useful in similar management and performance reviews. The team developed a chronograph to better understand the evolution of the activity from 2008 to today and pinpoint major shifts and/or evolutions in the activity. The purpose of this will be to show the operational context over time at various levels: general operating environment, project management, and specific performance benchmarks. These might include political unrest, teacher strikes, staffing gaps/shifts, and objective/target shifts and can thus provide very useful overviews of challenges and advantages that offer additional insight into an activity’s implementation. LIMITATIONS COMPLICATED PROJECT LIFE CYCLE AND STAFF TURNOVER The NECS activity involved a high number of program activities as well as stakeholder groups (MEP officials at the school, regional and central levels; Plan Niger; NECS; USAID Niger/SRO/Washington, etc.). In addition, between 2012 and 2018, many members of personnel changed positions. These factors combined led to the necessity of tracing the history of certain positions in order to identify most relevant stakeholders for interviews. In some cases, multiple people holding the same position were interviewed. Also, while the evaluation team used multiple attempts to try to reach individuals no longer associated with the project, in some cases, there was no response and the evaluation team compiled information available from accessible stakeholders. In addition, within some communities, multiple donors and projects were present making it difficult at times to isolate NECS activities and outcomes. IMPOSSIBILITY OF COMPARING QUANTITATIVE PERFORMANCE DATA While the original workplan anticipated comparing data across various early grade reading assessments, upon more in-depth investigation it became clear that comparison would be impossible. Reasons for this inability included changes in samples as well as how the timing of assessments coincided with program roll-out and the effects that could actually be tested. Without consistent samples, it was not possible to perform a trends analysis. Moreover, significant data validity concerns USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 110 existed for both the 2017 and 2018 EGRAs further limiting opportunities for comparison. The evaluation team developed tables (see Annex XIV) in order to better explain the major findings and limitations for the major relevant assessments. COMPRESSED TIMEFRAME Time constraints were a challenge for this evaluation. Data collection occurred towards the end of the school year in Niger, and after the rains had begun. These conditions resulted in some students already joining their families in agricultural work rather than attending school. In order to address this limitation, two teams conducted school-level data collection at the same time. When necessary, the team used creative methods to convene students and to simulate a regular classroom environment to the extent possible. Unforeseen transportation problems also required extending school level field work by two days. In addition, the data collection phase took place during the Muslim month of Ramadan. Two official religious holidays (Lailat al-Qad and Eid-al-Fitr) fell during the in-country data collection and preliminary analysis phases. Lastly, presentations of preliminary results to key stakeholders took place a few days earlier than expected due to meetings in Niamey requiring the attendance of MEP officials. These adjustments resulted in preliminary analyses taking place during a compressed timeframe. The team implemented measures to allow for efficient and intensive preliminary data analysis. The high quality of that initial analysis was confirmed by spot-checks and formal evaluation and structural coding techniques during the out-of￾country data analysis phase. 111 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV ANNEX III: FIELDWORK DATA COLLECTION SCHEDULE NO. NOM DU VILLAGE TYPE DE SITE (X) REGION LANGUE(S) DATES HARD SOFT BILING. 1 Bazaga Bilingue X Tahoua Haussa 29/05/2018 2 Kanjiwa X Tahoua Haussa 30/05/2018 3 Boudé X Tahoua Haussa 31/05/2018 4 Maïkourou X Tahoua Haussa 01/06/2018 5 Guidan Ider X Tahoua Haussa 02/06/2018 RETOUR SUR NIAMEY 04/06/2018 6 Dargol Quartier X Tillabery Zarma 05/06/2018 7 Nobba X Tillabery Zarma 06/06/2018 8 Kannia X Tillabery Zarma 07/06/2018 9 Banizoumbou 1 X Tillabery Zarma 08/06/2018 10 Zarmey bilingue X Tillabery Zarma 09/06/2018 11 Baban Anné X Maradi Haussa 29/05/2018 12 Tsararaou Allah Sarki X Maradi Haussa 30/05/2018 13 Maiguigé Karfi X Maradi Haussa 31/05/2018 14 Saber X Maradi Haussa 01/06/2018 15 Tatsegouma X Maradi Haussa 02/06/2018 DEBUT COLLECTE ZINDER 04/06/2018 16 Chago X Zinder Kanuri 05/06/2018 17 Soubdou Centre X Zinder Kanuri 06/06/2018 18 Dougueré Mai Gao X Zinder Fulfulde 07/06/2018 19 Farayé X Zinder Haussa 08/06/2018 20 Bande bilingue X Zinder Haussa 09/06/2018 USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 112 ANNEX IV: LIST OF KEY INFORMANT INTERVIEW AND FOCUS GROUP DISCUSSION PARTICIPANTS STAKEHOLDER TYPE KIIS FGDs READING EXERCISE TOTALS MEP 22 0 0 22 USAID – Niger 4 0 0 2 USAID – Washington 2 0 0 2 USAID – SRO 2 0 0 2 Other NGO operating reading program 6 0 0 6 Plan 2 0 0 2 NECS Reading Sub-contractor 4 0 0 4 NECS activity 14 2 0 16 MCC 1 0 0 1 CGDES 0 14 0 14 Community leaders 5 0 0 5 Community members 0 29 0 29 DREP/ DDEP 12 0 0 12 Head Teacher 17 0 0 17 Teachers 27 5 0 32 Inspectors (IEP/IAENF) 21 0 0 21 Pedagogical Advisor (CP) 13 0 0 13 SCOFI 14 0 0 14 PF CGDES 7 0 0 7 Students - School Government 0 20 0 20 Students (CI or CP) 0 0 20 20 TOTALS 174 70 20 262 113 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV ANNEX V: NUMBER OF INTERVIEWS BY TYPE OF RESPONDENT (With gender disaggregation) STAKEHOLDER TYPE MALES FEMALES TOTALS MEP 18 4 22 USAID – Niger 3 1 4 USAID – Washington 0 2 2 USAID – SRO 1 1 2 Other NGO operating reading program 3 3 6 Plan 2 0 2 NECS Reading Sub-contractor 1 3 4 NECS activity 13 0 13 MCC 0 1 1 CGDES 51 15 66 Community leaders 5 0 5 Community members 109 86 195 DREP/ DDEP 12 0 12 Head Teacher 16 1 17 Teachers 17 17 34 Inspectors (IEP/IAENF) 21 0 21 Pedagogical Advisor (CP) 13 0 13 SCOFI 1 13 14 PF CGDES 6 1 7 Students - School Government 58 74 132 Students (CI or CP) 60 60 120 TOTALS 411 282 692 USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 114 ANNEX VI: LIST OF PEOPLE INTERVIEWED NAME AFFLIATION TITLE Tcha Berei Aide et Action Country Director Bouri Aide et Action Missing Ahmu Naya Aide et Action Coordinateur parraignage afrique Markele Garouza Aide et Action Supervision Projet PASEDU (UNICEF) Moussa Daouda Aide et Action Coordinateur education NECS M. Issoufa HALIDOU MOUSSA Concern Worldwide Education Program Manager – Responsible Programme Education Ali Ibrahim DDEP inspecteur Ismael Keita Direction de l'evaluation et des concours professionels Evaluateur Hassane Harouna Direction de l'evaluation et des concours professionels Chef de Division sur les acquis scolaires Hassane Djibo Adamou Direction de l'Evaluation scolaireet des Concours Professionels Directeur Zara Bakingue Direction de la formation initiale et continue (DFIC) Directrice Laouali Mariam Chipkaou Direction de la Promotion de la Scolariations des filles (DPSF) Directrice Hassan Amadou Soumana Direction de la Promotion de la Scolarisation des Filles (DPSF) Point Focal Chérif Ali Abdoul Karim Direction des Curricula de la Promotion des Langues Nationales Point focal ELAN Roua Boukar Koura Direction es Curricula et de la Promotion des Langues Nationale (CDPLN) Directrice Rougga Himadou Direction generale de l'alphabetisation et de l'enseignement non-formel Directeur General Jafarou Saley Direction Generale de la Scolarisation Directeur Général Tarihou Elhadji Moussa Division Gestion Decentralisee de l'Ecole (DGDE) Directeur Mahaman Mani DPAFA Point Focal NECS Benjamin Bach IRC Niger Education Coordinator Kiruba Murugaiah IRC USA To follow Kimberly Smith IRC USA To follow 115 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV NAME AFFLIATION TITLE MALLAM GARBA Maman MEN Conseiller Technique (depuis 16 mois) Carolyn Perrin Millennium Challenge Corporation Associate Director, Monitoring & Evaluation Division Yahouza Ibrahim Ministere de l'Education Primaire Secretaire Generale Amadou Ali NECS COP - last day Thursday 5/31/18 Ousmane Brah Mamane NECS Specialiste Suivi Evaluation NECS Romain Babagbeto NECS Specialiste Education / COP Interim (from 6/1) Idrissa Ibrah Waduri NECS Specialiste Moblisation Sociale NECS Ousseini Manzo NECS Directeur de Finances Ousmane Manzo NECS Directeur de Finances Hamani Ali NECS Coordinateur d'education Johnson Bien-Aime PLAN International Representant Resident Sarah Havekost Plan International Senior Program Manager Barira Magagi Plan/Niger Specialiste en éducation Ali Maidouka Plan/Niger Spécialiste suive evaluation passe-plus Erambel Iba Abdoulaye Promotion de la Qualité et de la Normalisation (DGPQ/N) Directeur Général Linda Farrell Readsters Founder, Business Partner Michael Hunter Readsters Founder, Business Partner Bescaye Diop Sahel Regional Program Office, USAID (Senegal) Regional Monitoring and Evaluation Specialist Alhadji Amadou Save the Children Senior Regional Education Advisor, West and Central Africa Lisa Easterbrooks STS Senior Program Manager Mary Denauw STS Education, Training and Literacy Expert Cornelia Treeman USAID - SRO Regional Knowledge Management & Learning Specialist Jennifer Swift Morgan USAID (Prev.) AOR Rebecca Rhodes USAID Washington Senior Education Advisor; Team Lead, Reading and Literacy Team (Acting) Penelope Bender USAID Washington (Former) Former Lead, Early Grade Readings (Goal One), USAID Washington - now Head, Global Education Services, BurdaEducation USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 116 NAME AFFLIATION TITLE Kossomi Aboubacar USAID/Niger AOR Ahmed Tidiane Diallo USAID/Niger Education Officer Garrett Harries USAID/Niger Democracy, Gouvernance and Countering Violent Extremistm Officer 117 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV ANNEX VII: NECS ACTIVITY TIMELINE USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 118 119 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV ANNEX VIII: LIST OF DOCUMENTS REVIEWED 1 ARRETE N°184 SG création mission organisation et fonctionnement du comité national d’organisation du concours de lecture (CNOCL) 2 Niger Education and Community Strengthening Project (NECS), PERFORMANCE MANAGEMENT PLAN (PMP), Revised July 10, 2015 3 Niger Education and Community Strengthening Project Extension (NECS+), ACTIVITY MONITORING & EVALUATION PLAN (AMEP), Revised for NECS+, November 11, 2016 4 Réunion du comité de pilotage du projet NECS+ [ppt], March 2018 5 Plan de suivi et évaluation du projet NECS, version française, 11 novembre 2016/AMEP 6 March 2018 Steering Group Minutes 7 L’Atelier National de Suivi/accompagnement des Activités des CGDES et FCC (ANASAC ) Dosso les 08 et 09 Août 2017_ Synthèse des Bilans PAAQ et PA/FCC 2016-2017 8 Quelques statistiques relatives à la participation communautaire dans la zone d'intervention du projet NECS et au niveau National_ Global Analyse mobilisation communautaire a travers les Plan d'actions (An 2, 3, 4 et 4) 9 Diagnostic organisationnel des AME/dynamisation 2013 10 Diagnostic organisationnel des APE/dynamisation 2013 11 Diagnostic organisationnel des CGDES/dynamisation 2013 12 Synthèse Globale de l’évaluation des structures partenaires de NECS 13 Module Formation des mentors Décembre 2016 14 Types d'appui des mentors à l'endroit des enfants en difficultés scolaires Décembre 2016 15 Module de formation sur le processus de mise en place démocratique des structures (CGDES, APE et AME) Mai 2013 16 Guide sur la mobilisation des communautés dans la réalisation et la gestion des infrastructures/équipements scolaires Mai 2013 17 Exercice sur le genre 18 Fiche de suivi de la mise en œuvre du code de conduite des enseignants 19 MEP: Grille d’appréciation d’une leçon 20 MEP: Evaluation rapide des structures 21 Plan International Education Capacity Statement (Draft) 22 Reports listed within reading outcomes table 23 Concern: Early Grade Reading Assessments and the Bilingual Pilot – Tahoua, Niger 24 IRC 3EA Niger Impact Report 25 Mathematica NECS Baseline report 26 Mathematica NECS EGRA Round 1 USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 120 27 Mathematica NECS Final report 28 PASEC 2014 Niger Rapport Finale 29 MEP: Mission d’appui à la DES/CP (Ex- DESAS/O) pour la réalisation d'une évaluation des impacts de la réforme du curriculum sur les acquis des élèves du Cl (Tome 2) 30 OIF: Rapport intermédiaire de recherche à destination de l’Organisation Internationale de la Francophonie 2013- 2015 31 STS Memorandum regarding NECS_2017 and 2018 EGRA Data Quality 32 March 2018 Steering Group Minutes 33 Réunion du comité de pilotage du projet NECS+ [PPT], March 2018 34 Code de bonne conduit 35 Exercice sur le genre 36 Fiche de suivi de la mise en œuvre du code de conduite des enseignants 37 MEP: Grille d’appréciation d’une leçon 38 MEP: Evaluation rapide des structures 39 Plan International Education Capacity Statement (Draft) 40 Reports listed within reading outcomes table 41 Concern: Early Grade Reading Assessments and the Bilingual Pilot – Tahoua, Niger 42 March 2018 Steering Group Minutes 43 Réunion du comité de pilotage du projet NECS+ [PPT], March 2018 44 Synthèse des Activités FCC et CGDES_2015-2016 45 Rapports trimestriels et annuels des activités du projet (NECS Q4 24Sept_2013_Annual ; NECS_Annual - Quarterly Report April-June 2017 FINAL ; NECS+PY5 Annual Workplan Narrative_20 Sept 16; QR 4 NECS April to June 2015 Annual; QR4 & PY4 NECS QR) 46 Scaling Up — From Vision to Large-Scale Change/Tools and Techniques for Practitioners December 2012 47 Millions Learning, faire prendre de l’essor à l’éducation dans les pays en développement, résumé analytique 48 Statistiques de l'éducation de base et alphabétisation, annuaire 2016-2017 121 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV ANNEX IX: REFLEXION SUR LA MISE A L’ECHELLE DE L’ASL/ SYNTHESE DE L’ACTIVITE Lundi 17 Juin 2018/Salle de réunion MEP L’une des questions d’évaluation du projet NECS portait sur la possibilité de procéder à la mise à l'échelle de l’ASL. À cette fin, une séance de réflexion a été menée le lundi 18 juin 2018 avec les acteurs du Ministère de l’Enseignement Primaire, de l’Alphabétisation, de la promotion des langues nationales et de l’Education Civique ainsi que les acteurs du NECS. Grâce à une longue expérience de Management Systems International (MSI) dans l’appui aux organisations dans la mise à l'échelle, l’équipe d’évaluation, en présence du représentant de l’USAID/Niger a eu dérouler pendant 3 heures différentes activités. En effet, avec l’un des produits de cette expérience de MSI qui est une boite à outils comprenant différentes rubriques. Une participation de qualité caractérisait la séance. Les participants se sont démontrés comme experts en ASL et ont fait des commentaires pertinents et productifs. Déroulement : Après la présentation des différents participants, l’équipe a procédé à l’identification des opportunités et préoccupations des uns et des autres en ce qui concerne l’ASL. Chacun, sur la base de ses expériences et connaissances avec NECS a pu avancer ses opinions dans ce sens (voir tableau ci￾dessous plus de détails). Le paquet de 28 rubriques constitue un exercice qui aide l’utilisateur à mieux cerner les forces, les lacunes et les prochaines étapes pour la mise à l'échelle. Ainsi, les participants ont été divisés en 5 groupes de travail et chaque groupe avait en charge un paquet de rubriques sur lesquels il devait fournir leur position (avis positif, négatif ou neutre sur les facilités/difficultés que poseraient une mise à l’échelle de l’ASL). Après cette séquence de travaux de groupe, les participants ont, en plénière, rempli la Grille d’évaluation de la mise à l’échelle de manière consensuelle et ceci sur la base de connaissances empiriques ou scientifiques. PARTICIPANTS OPPORTUNITÉS PRÉOCCUPATIONS N°1 • Possibilité de pouvoir apprendre aux enfants à lire très rapidement • Démarche à capitaliser pour le MEP • Par rapport au guide de l’enseignant on sait que le Niger est un pays où les ressources sont très rares • Préoccupation pour assurer une multiplication d’un guide qui fait presque 500 pages N°2 • Opportunité unique pour le MEP d’apporter une critique de fond sur l’ASL… • Notre souhait c’est d’apporter quelque chose au système éducatif nigérien et avec l’ASL c’est possible • Sinon j’ai été à l’intérieur du pays j’ai vu que les acteurs locaux comme les IEP et DDEPs apprécient bien l’approche Ma préoccupation actuelle c’est la capacité du projet de continuer le travail compte tenu de la déperdition du staff que nous sommes en train de connaître… USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 122 PARTICIPANTS OPPORTUNITÉS PRÉOCCUPATIONS N°3 Efficacité de l’ASL pour apprendre aux élèves à lire • Ma préoccupation c’est par rapport au guide parce qu’au début on nous a dit que c’est un guide avec une démarche scriptée… • Compte tenu du niveau des enseignements il fallait tout donner et c’est d’ailleurs ce que l’experte en lecture avait souligné… • Ma préoccupation c’est aussi le suivi des enseignants sur le terrain ; ce qui va engendrer d’autres couts ; • Comme c’est l’entrée par les langues, il faut assurer une bonne assise de la langue au niveau des enseignants… N°4 • Préoccupation avec l’empiétement de l’approche ASL avec d’autres approches (comment utiliser l’ASL sans piétiner d’autres approches qui sont explorées dans les écoles ou sans porter préjudice aux autres approches parce qu’on sait que l’ASL prend le temps intégral de lecture avec ses 90mns par jour) • Est-ce qu’il sera possible d’intégrer l’ASL dans ce qui existe déjà (avec ses séances de lecture intégrale). On va essayer de croiser d’autres expérimentations, donc comment l’utiliser avec ces approches déjà sur le terrain. N°5 • ASL a suscité un engouement chez les élèves ; les enfants ont le plaisir d’aller à l’école et d’apprendre, ce qui est une grande opportunité; • Comment amener les décideurs à accepter la mise à l’échelle de l’ASL « Tout le problème est là » N°6 • Le ministère a fait une évaluation qui a montré le niveau très bas des enseignants ce qui montre des preuves sur leurs capacités limitées à mener des leçons et le fait d’avoir des leçons complétement scriptées va permettre aux enseignants de disposer de leurs leçons déjà bien préparer et bien détaillées. Le volume du guide de l’ASL n’est un problème. Il répond en fait à qui il est destiné. • Il ne faudra surtout pas que l’enseignant pense qu’il n’a rien à préparer parce que tout simplement il a un guide scripté avec lui • Attirer l’attention des enseignants pour qu’ils lisent et préparent les leçons avant de venir en classe • « Il faut être professionnel pour lire des leçons en langues nationales) • Dernière préoccupation c’est le coût de la reproduction du matériel pédagogique avec le document qui est volumineux. N°7 • En 2015 il y’a eu une évaluation qui classait NECS en bonne position pour ce qui est de l’approche de la lecture ce qui montre l’opportunité que NECS dispose parce parmi 3 ou 4 approches il était au premier rang. • Aujourd’hui on ne peut plus continuer à pratiquer cette discipline qui est la lecture de la même manière. Il y’a aussi le bilinguisme qui est là et mérite réflexions sur les méthodes à appliquer. • Pour la question du volume du guide pour moi ce n’est pas un problème quand on parle de reproduction parce que ce n’est pas pour les élèves mais 123 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV PARTICIPANTS OPPORTUNITÉS PRÉOCCUPATIONS pour les enseignants. Donc c’est bien d’avoir ce guide vu le niveau de nos mauvais enseignants. N°8 • L’opportunité que je vois dans la mise à l’échelle de l’ASL c’est d’abord :la disponibilité du Ministère à travers la présence du SG, la disponibilité des ressources (USAID) • Les élèves apprennent à lire avec enthousiasme . • Attention de ne pas donner du tout cuit à l’enseignant. Il faut le former pour qu’il puisse bien faire son travail. • Financement en lien avec le volet suivi des enseignants sur le terrain( conseiller, inspecteurs, SCOFI) N°9 • L’approche ASL aide l’apprenant à décoder (les élèves arrivent à lire) N°10 • Les uns et les autres se sont largement penchés là-dessus : • La participation n’est pas assez représentative pour parler de la mise à l’échelle de l’ASL il faut que les cadres du Ministère soient tous présents. Il n’y a pas beaucoup de gens pour parler objectivement de l’ASL ici. Dans cette réunion plus de 4/5 des personnes ici pressentes sont acquises pour la cause de l’ASL. • Critique sur la méthodologie parce qu’elle aurait voulu avoir des résultats sur la pertinence de l’ASL avant que les participants ne se prononcent sur cela. Ici il y’a très peu de gens qui peuvent décider d’une mise à l’échelle. N°11 • la question qui préoccupe notre ministère c’est la question de la lecture et je n’ai pas de doutes sur l’ASL parce que j’ai vu les élèves de NECS qui ont été amené au forum…. • Quand on parle de mise à l’échelle il ne faut pas perdre de vue que nous nous sommes déjà engagés dans une réforme…On ne peut pas faire une juxtaposition des choses mais plutôt penser à comment intégrer l’ASL à ce qui se fait déjà… • Si l’enseignant a de l’ingéniosité il peut puiser dans toutes les approches et à tous les supports pour préparer sa leçon de lecture…Il faut voir ici avec l’ALS qu’est-ce qu’il faut prendre comme bonnes pratiques et les intégrer à l’élan de réforme déjà engagé… le passage à l’échelle de l’ASL doit tenir compte de tout ce que nous faisons au niveau de la réforme. • Mais pour l’enseignant il faudra le former, le suivre et lui donner des supports qui lui permettront d’être autonome dans ce qu’il fait dans sa salle de classe. N°12 • Le ministère s’est lancé dans un processus de réforme donc il est ouvert à toute bonne pratique…. Il faut capitaliser les bonnes pratiques…Identifier les besoins de l’enfant qui ne sait pas lire et adapter les supports… • La mobilisation des ressources il y’a aussi la mise en place d’un fond commun pour le secteur de l’éducation qui est une bonne brèche où les partenaires pourront agir. • La seule préoccupation c’est au niveau des ressources financières et leur disponibilité mais heureusement il y’a une opportunité avec les fonds communs. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 124 PARTICIPANTS OPPORTUNITÉS PRÉOCCUPATIONS • N°13 • Au Niger, 90% des enfants n’arrivent pas à lire donc l’ASL est une opportunité qui d’offre des innovations très claires en lecture qui ont été exploré. Donc l’ASL répond vraiment à un besoin. • Elle permet surtout l’entrée par la langue et crée un engouement aux niveaux des communautés. • En termes de préoccupations il faut noter qu’il y’a 4 approches, donc au￾delà de la réforme qui est en cours il faut faire une évaluation voir quoi prendre de qui et avoir une seule approche pour tout le système éducatif… • Par ailleurs pour le financement il y’a aussi au-delà de la reproduction des manuels et guides, le suivi et le fonctionnement du dispositif de suivi avec les conseillers pédagogiques N°14 • Les élèves arrivent à lire avec l’ASL qui a des avantages irréfutables. • Il y’a des difficultés avec la mise en œuvre des réformes que nous sommes en train de faire maintenant avec la question des moyens et la question de la reproduction des manuels N°15 • Possibilité d’avoir une entrée par les langues telle que la réforme l’a amorcé • L’engouement des élèves et des parents pour la lecture; • Avoir des enseignants de qualité pour enseigner en langues N°16 • Engouement au niveau des communautés avec une interaction entre les écoles et les parents d’élèves qui n’est pas un résultat définitif mais plutôt un acquis ; • La formation des enseignants qui est un élément sur lequel il va falloir investir N°17 L’USAID (finance NECS en tenant de compte des initiatives du MEP) inscrit parfaitement en droite ligne avec le PSEF et les réformes engagées par le MEP. Toutes ces initiatives de lecture ne peuvent pas se juxtaposer, le MEP doit décider et choisir…Il n’y a pas un choix de programme clairement fait par le MEP parce il y’a combien d’initiatives pour un seul système ? Soucions-nous de la qualité de l’éducation de l’enfant nigérien en utilisant l’entrée par les langues. Moins gaspiller les ressources avec toutes ces approches pour différents partenaires…L’entrée par la langue a été clairement choisie par le MEP… Tous les pays émergeant se sont développés grâce à l’éducation en utilisant la langue locale comme langue d’apprentissage. Le Niger aussi doit faire autant. N°18 • Il y a eu une évaluation en 2015 réalisé par le ministère qui place l’ASL en première position. • Ce n’était pas une évaluation n’était pas une évaluation en tant que telles parce que en 2015 ce n’étaient pas toutes les écoles qui étaient concernées. C’était sur une période on a fait l’évaluation. L’école n’était pas aux mêmes niveaux de progression dans les programmes. Et il n’y avait pas l’école de la reforme.et L’ASL était à ces débuts. • Même si c’est sur une semaine que l’évaluation a été faite et que toute toutes les approches ont été touchées alors cela est valable. • Je pense que c’est insuffisant pour généraliser. • Si on compare l’ASL à l’approche globale ce n’est pas correcte. l’entrée par les langues ne peut pas être comparée avec l’entrée par le français.( suivre la discussion à la 51mn). Conclusions Point forts • Performances de modèle • Priorité politiques et le besoin Points faibles • Le financement • Manque de preuves solides • Les évidences Participant N°1 • Regarder ce qui est pertinent dans l’ASL pour voir ce qu’il faut prendre et mettre ensemble avec les éléments de la réforme. Sinon ça sera de bouleverser beaucoup de chose. Participant N°2 • Il y a 12 écoles de la réforme avec NECS on attend les résultats de l’évaluation EGRA pour voir l’impact de l’ASL 125 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV PARTICIPANTS OPPORTUNITÉS PRÉOCCUPATIONS Participant N°3 • Il faut une étude qui va matérialiser les constats. • La mise à l’échelle est possible mais il faut l’engagement de tous (politique et bailleur de font) en faisant valoir les preuves • Montrer qu’il y a des choses positives, même s’il faut les présences de plusieurs cadres du MEP • Amener à la réunion les acteurs de terrain ,pas seulement les cadres du MEP GRILLE D’EVALUATION DE LA MISE A ECHELLE CARACTÉRISTIQUE DU MODÈLE A LA MISE EN ECHELLE EST PLUS FACILE B LA MISE EN ECHELLE EST PLUS DIFFICILE C ☺✓ ✓ ✓ Est-ce que le modèle est crédible ? 1 Preuves solides Peu ou pas de preuves solides 2 Evaluation externe et indépendante Pas d’évaluation externe et indépendante 3 Il y a des preuves que le modèle marche dans différents contextes sociaux Il n’y a pas de preuves que le modèle marche dans différents contextes sociaux 4 Le modèle a le support d’individus et institutions éminents Le modèle n’a pas le support des individus et les institutions éminents Les résultats du modèle peuvent￾ils être observés ? 5 L’impact est très visible à l‘œil nu ; tangible Impact peu visible ; difficile à communiquer au public 6 Clairement associé à l’intervention Pas clairement associé à l’intervention 7 Témoignages et documents existent avec un attrait émotionnel certain Peu ou pas d’évidence, peu d’attrait émotionnel Quelle est la pertinence du modèle ? 8 Résout objectivement un problème significatif et persistant Résout un problème qui affecte peu de personnes ou d’impact limité 9 Résout un problème qui est actuellement l’une des priorités politiques Résout un problème qui n’est pas une priorité politique 10 Répond à un besoin fortement ressenti par les bénéficiaires potentiels Répond à un besoin peu ressenti par les bénéficiaires potentiels Le modèle offre-t￾il des avantages relatifs par rapport à la pratique actuelle ? 11 Les solutions actuelles pour cette question sont jugées insuffisantes Les solutions actuelles sont jugées adéquates 12 Efficacité supérieure par rapport aux solutions actuelles Peu ou pas de preuve objective de la supériorité par rapport aux solutions actuelles 13 Efficacité supérieure par rapport aux autres modèles novateurs mis en place L’efficacité supérieure par rapport aux autres modèles novateurs n’est pas établie Est-il facile d’adopter et de transférer le modèle ? 14 Mise en œuvre au sein des systèmes présents, de l'infrastructure et des ressources humaines Nécessite des systèmes, infrastructures, ou ressources humaines nouveaux et/ou supplémentaires 15 Contient des éléments qui s’ajoutent facilement aux systèmes présents Est un paquet complet ou exhaustif de plusieurs composants USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 126 GRILLE D’EVALUATION DE LA MISE A ECHELLE 16 Proche des pratiques et comportements actuels de la population ciblée Grand départ par rapport aux pratiques et comportements actuels de la population ciblée 17 Proche des pratiques actuelles et de la culture de l'organisation (s) qui est en train d’adopter le modèle Grand départ par rapport aux pratiques et culture de l'organisation (s) qui est en train d’adopter le modèle 18 L’accord de seulement quelques décideurs est nécessaire pour l'adoption du modèle L’accord de nombreux décideurs est nécessaire pour l'adoption du modèle 19 Le modèle a un nombre limité de contrôle et de supervision Le modèle a un nombre élevé de contrôle et de supervision 20 Efficacité démontrée dans divers contextes organisationnels Efficacité démontrée que dans un contexte organisationnel 21 Peu de sophistication technique des composantes et activités du modèle Grande sophistication technique des composantes et activités du modèle 22 L’innovation majeure est une technologie claire et facile à reproduire, par exemple, le vaccin La mise au point du modèle n'est pas une technologie facile à reproduire 23 Peu de complexité ; simple avec peu de composants qui s’ajoutent facilement aux systèmes existants Complexité élevée avec de nombreux éléments ensemble intégré 24 Demande peu de supervision et de surveillance La réalisation demande beaucoup de supervision et de surveillance Le modèle peut-il être testé ? 25 Peut être testé par l’usager sur une échelle limitée Ne peut être testé par l’usager sans être adopté à grande échelle Y-a-t-il une source de financement durable ? 26 Supérieure coût-efficacité démontré par rapport aux solutions existantes ou autres clairement établie Peu de preuves de supériorité en termes de coût-efficacité 27 Nécessite un petit engagement de fonds à l'échelle Demande un grand engagement de fonds à l’échelle 28 Le modèle a son propre financement interne (par exemple, les frais d'utilisation) ou de la dotation Pas de financement interne ; le modèle dépend de source de financement externe Nombre total de contrôles 127 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Activité de réflexion sur la mise à l’échelle En ce qui concerne le processus de la mise à l’échelle de l'ASL, par rapport aux critères des rubriques, où se trouvent des points d'accord d'opinion entre les participants? Où se trouvent les points de divergence ? CARACTÉRISTIQUES DU MODÈLE ITEMS D’APPRÉCIATIONS POINTS DE CONVERGENCE POINTS DE DIVERGENCE Est-ce que le modèle est crédible ? 1 X : Avec l’absence de preuves solides, une partie des participants avaient jugé difficile d’apprécier la crédibilité du modèle sur cette base. Même si au départ (en travaux de groupe) ils avaient jugé que le modèle avait des preuves solides qui renforcent son crédibilité, au moment de la plénière, le reste des participants est revenu sur l’absence de données EGRA ou encore d’évaluation quantitative ayant produits des statistiques fiables. D’autres qui ont mis en avant l’étude comparative menée par le MEP entre les différentes approches en lecture développés au Niger pour dire que l’ASL a des preuves qui permettent de dire que c’est une démarche crédible. Les participants n’étaient pas d’accord que l’évaluation de 2015 pouvait être utilisé comme référence pour apprécier le niveau de solidité du modèle. Ce que quelques participants ont refusé disant que cette évaluation ne concernait pas toutes les écoles et c’était juste pour une période (seulement 3 mois d’activités…Les écoles des différentes approches n’étaient pas au même niveau). Et par conséquent elle ne pouvait être considérée comme une référence pour dire que l’ASL est une approche crédible avec des preuves solides. Le rapport de suivi des encadreurs pédagogiques a aussi été brandi par une partie des participants comme moyen pour apprécier l’ASL et son crédibilité. D’autres ont aussi soulevé qu’il n’était pas faisable de comparer l’approche globale avec l’ASL parce que les langues d’enseignements sont différentes. 2 X Des doutes ont été émises sur la nature externe et indépendante d’une évaluation rigoureuse de l’ASL 3 X Absence d’informations claires et avérées sur les possibilités de pouvoir utiliser le modèle dans différents contextes. Pas de connaissances sur les performances de l’ASL dans d’autres pays. 4 X Les résultats du modèle peuvent-ils être observés ? 5 X 6 X 7 X Quelle est la pertinence du modèle ? 8 X 9 X 10 X USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 128 CARACTÉRISTIQUES DU MODÈLE ITEMS D’APPRÉCIATIONS POINTS DE CONVERGENCE POINTS DE DIVERGENCE Le modèle offre-t-il des avantages relatifs par rapport à la pratique actuelle ? 11 X X Avec l’absence de preuves scientifiques quelques participants ont souhaité que la neutralité soit choisie à ce niveau pour être plus fidèle à la réalité 12 X X C’est le même constat qui a été fait à ce niveau en jugeant également que NECS ou ASL n’ont pas assez de preuves pour démontrer la supériorité de la démarche sur les autres approches comme APS ou encore ce qu’ELAN ou CONCERN font. 13 X : Selon la plupart des participants il n’est pas établi que l’ASL est d’une efficacité supérieure par rapport aux autres approches vu qu’il n’existe pas une évaluation complète sur les autres approches comme APS ou encore ce qu’ELAN ou CONCERN. Par exemple, il y’a le rapport EGRA qui est toujours en cours de rédaction pour démontrer sur le plan scientifique que les enfants apprennent à lire rapidement avec l’ASL. La question de l’évaluation comparative de 2015. Jusque-là ce ne sont que des témoignages qui sont là. Est-il facile d’adopter et de transférer le modèle ? 14 X 15 X 16 X 17 X : Selon certains, Il n’y a pas eu un document de capitalisation permettant de mieux apprécier. Toutefois, d’autres participants ont également dit que l’entrée par les langues telle que retenue dans l’ASL est le fondement même de la réforme de la politique éducative entrepris par le ministère. Plusieurs participants ont mis en avant la difficulté pour eux de dire que l’ASL est proche des pratiques actuelles et de la culture du MEP au cas il souhaiterait adopter le modèle. Il n’y a pas eu un document de capitalisation permettant de mieux apprécier. Toutefois, d’autres participants ont également dit que l’entrée par les langues telle que retenue dans l’ASL est le fondement même de la réforme de la politique éducative entrepris par le ministère. Mais une partie a aussi reconnu qu’il y’a des éléments nouveaux apportés par l’ASL. 18 X Tous les participants n’étaient pas totalement convaincus que seulement l’accord de quelques décideurs est nécessaire pour l'adoption du modèle ASL. D’après quasiment tous les participants il y’a une incertitude qui fait qu’ils préfèrent être neutres. 19 X 20 X : Les participants ne sont pas tous convaincus de l’efficacité de l’ASL et de la manière dont cela a été démontré comme modèle pouvant être appliqué dans divers contextes organisationnels. 129 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV CARACTÉRISTIQUES DU MODÈLE ITEMS D’APPRÉCIATIONS POINTS DE CONVERGENCE POINTS DE DIVERGENCE 21 X Même s’il y’a les méthodes avec bec de canard, estampillage, compréhension orale, etc., avec l’absence d’un document qui capitalise l’ASL du début à la fin c’est-à-dire depuis sa conception jusqu’à sa mise en œuvre, les participants ont préféré être neutres sur ce point. Le niveau de qualification de la sophistication technique des composantes et activités du modèle n’est pas démontré à 100% selon eux. 22 X 23 X 24 X Le modèle peut-il être testé ? 25 X Y-a-t-il une source de financement durable ? 26 X Un peu de divergence sur qui doit répondre à cette question (des financiers ou l’USAID) même si l’évaluation d’impact menée par Mathematica est là pour faire une appréciation… Absence d’éléments pour bien apprécier le niveau d’efficacité avec les aspects de coût-efficacité. 27 X 28 X USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 130 Images de la séance 131 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Completed MSI Scalability Checklist for ASL Developed during Work Session USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 132 ANNEX X: EVALUATION TOOLS EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) USAID/NECS AOR, Education Officer, MCC/Director of Threshold Programs 1. Pouvez-vous nous rappeler l’historique/le processus qui a conduit au financement du projet NECS ? 2. Quels ont été les principaux défis auxquels l’USAID a été confronté ? Qu’est-ce qui a été fait pour les contourner et comment cela s’est traduit au fur et à mesure dans l’atteinte des résultats du projet ? 3. On a noté que les objectifs stratégiques de NECS ont évolué et ont fait l’objet de modifications, qu’est-ce qui a conduit à ces types de changements ? 4. Selon vous quel est l’objectif que le NECS a le mieux atteint à ce jour ? Qu’est-ce qui avait facilité la réalisation de l’atteinte de cet objectif ? 5. De votre point de vue, quel est l’objectif de NECS qui enregistre le moins de résultats à ce jour ? Quelle en est l’explication selon vous ? 6. Quels ont été les principales contraintes/difficultés observées dans la mise en œuvre des différentes activités de NECS ? Comment ces difficultés ont-elles été contournées durant les années de mise en œuvre ? 7. Quelle appréciation faites-vous de l’approche ASL sur les performances en lecture des élèves ainsi que les compétences éducatives des enseignants (outils pédagogiques et démarches/méthodes) ? 8. Quelles appréciations faites-vous du niveau de motivation des communautés par rapport à l’éducation et à la lecture telle qu’abordé par NECS ? 9. Quels sont les changements majeurs notés dans les relations entre les autorités éducatives au niveau décentralisés et les communautés avec l’intervention de NECS ? 10. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être ? 11. S’il y’a une chose que vous aurez aimé changer ou améliorer du projet NECS qu’est-ce ce que cela pourrait être ? 12. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir (dans le cadre d’une mise à l’échelle par exemple) ? 133 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) Equipe du projet NECS (COP, spécialiste en éducation/lecture) et autres sous￾contractants (STS, VIE, Readsters, AeA) 1. Pouvez-vous nous rappeler l’historique/le processus qui a conduit au financement du projet NECS ? 2. Qu’avez-vous principalement retenu de la mise en œuvre des activités du projet NECS en vous référant à son historique (de 2012 à aujourd’hui) ? 3. Quels ont été les principaux défis auxquels le staff de NECS a été confronté ? Qu’est-ce qui a été fait pour les contourner et comment cela s’est traduit au fur et à mesure dans l’atteinte des résultats du projet ? 4. On a noté que les objectifs stratégiques de NECS ont évolué et ont fait l’objet de modifications, qu’est-ce qui a conduit à ces types de changements et comment se sont-ils traduits à votre niveau ? 5. Selon vous quel est l’objectif que le NECS a le mieux atteint durant sa mise en œuvre ? Qu’est-ce qui avait facilité la réalisation de l’atteinte de cet objectif ? 6. Quel est l’objectif ou les objectifs de NECS qui enregistre le moins de résultats à ce jour ? Qu’est-ce qui pourrait expliquer cette situation ? 7. Quels sont les changements majeurs notés dans les relations entre les autorités éducatives au niveau décentralisés et les communautés avec l’intervention de NECS ? Quels ont été les principaux défis dans l’amélioration de ces relations ? 8. Quels ont été les principales contraintes/difficultés observées dans la mise en œuvre des différentes activités de NECS au niveau des écoles, notamment l’ASL ? Comment ces difficultés ont-elles été contournées ? 9. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être ? 10. S’il y’a une chose que vous aurez aimé changer ou améliorer du projet NECS qu’est-ce ce que cela pourrait être ? 11. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir (dans le cadre d’une mise à l’échelle par exemple) ? USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 134 EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) DREP/DDEP, IEP, IAENF/IDAENF, CP (Inspection Départementale de l’Alphabétisation et de l’Education Non Formelle) Activités (réussites/défis/contraintes) 1. Depuis quand occupez-vous ce poste ? Quelles sont les principales activités de votre institution ? 2. Pouvez-vous nous expliquer le rôle de votre institution dans les interventions du projet NECS? 3. Quelles sont les principales activités déroulées par USAID/NECS dans votre zone d’intervention ? 4. Qu’avez-vous particulièrement retenu des activités du projet NECS ? 5. De votre point de vue, quels ont été les activités à succès du projet ? Qu’est-ce qui a favorisé cela ? 6. De votre point de vue, quelles ont été les activités qui ont enregistré les plus faibles résultats ? Qu’est-ce qui explique cette situation ? 7. Avec l’intervention de NECS au sein des écoles et des communautés, quels types de changements pouvez-vous partager avec nous ? Lecture [En cas d’exemple, noter le nom de l’école ou du village cité.] 1. De votre expérience, comment appréciez-vous l’approche d’apprentissage de la lecture (ASL) déroulée par NECS dans les établissements ? Comment cela vous a-t-il aidé dans la mise en œuvre des cours, surtout en ce qui concerne l’enseignement de la lecture ? 2. Quelle comparaison pouvez-vous faire entre l’ASL et les autres approches en lecture au Niger (avantages et inconvénients) ? 3. Quels sont les atouts et limites que vous avez relevé dans l’approche ASL ? Quelles sont les suggestions pour l’amélioration de l’ASL ? 4. Pensez-vous que le projet NECS a contribué à rehausser les compétences des enseignants dans l’enseignement de la lecture aux élèves de CI et CP ? Si oui, comment ? Donner des exemples. 5. Pensez-vous que cette nouvelle approche d’enseignement de la lecture prôné par NECS a pu aider les élèves à avoir de meilleures performances en lecture ? Si oui, comment et dans quelle mesure (notes des élèves, assiduité à l’école) ? 6. Avez-vous eu échos de changements de comportement ou d’attitudes vis-à-vis de la lecture chez les élèves grâce au NECS ? Si oui, quel genre de changements ? (Participation en classe, concentration sur les enseignements du maître, intérêt pour la lecture) Formations Plan d’action Code de conduite Mentorat Alphabétisation ASL CGPE CGDES La bibliothèque Lecture publique Séances lectures Gouvernement scolaire Concours de lecture Cours de rattrapage Déparasitage Sensibilisation 135 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Communauté 1. Comment appréciez-vous la participation de la communauté dans les activités de NECS ? Qu’en est-il de cette participation communautaire dans la gestion des écoles que vous couvrez ? (Entretien des infrastructures, plans d’actions, contributions financières) 2. Avez-vous noté des changements dans la motivation de la communauté autour de l’éducation en général depuis l’arrivée du projet dans le village? Si oui, quel genre de changements ? Donner des exemples. 3. Qu’en est-il de la motivation de la communauté pour la lecture et la scolarisation des filles depuis que le projet NECS est présent dans les villages ? Donner des exemples. 4. Avez-vous remarqué une évolution des relations entre les écoles et les communautés depuis l’arrivée de NECS dans le village ? Si oui, comment cette évolution se manifeste-t-elle? 5. Pouvez-vous nous donner des exemples précis ? Genre 1. USAID/NECS avait pour objectif de promouvoir la scolarisation et le maintien des filles à l’école, quelles observations pouvez-vous faire sur cela ? 2. Avez-vous noté des changements dans le comportement ou attitudes par rapport au « Genre » et à l’Equité » grâce au activités de NECS ? Quels types de changements et chez quels acteurs (élèves, enseignants, parents, etc.) ? 3. Connaissez-vous le Code de Conduite développé par NECS ? Quelles sont vos appréciations concernant ce code par rapport aux changements qu’ils pourrait induire chez les enseignants ? Pouvez-vous partager des exemples précis ? 4. Quels sont les atouts et limites de ce code de conduite ? Suggestions 1. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être ? 2. S’il y’a une chose que vous aurez aimé changer du projet NECS qu’est-ce ce que cela pourrait être ? 3. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir ? USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 136 EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) Niveau central : MOE, MEP, MEP/A/PLN/EC, Direction de l’enseignement primaire, Gestion Décentralisée des écoles, Direction d’Appui de gestion des écoles (DAGE), Direction de la promotion de la scolarisation de filles 1. Pouvez-vous nous expliquer le rôle de votre institution durant les interventions du projet NECS ? 2. Qu’avez-vous principalement retenu de la mise en œuvre des activités du projet NECS en vous référant à son historique (de 2012 à aujourd’hui) ? 3. De votre point de vue quel est l’objectif que le NECS a le mieux atteint durant sa mise en œuvre (de 2012 à aujourd’hui) ? Qu’est-ce qui avait facilité la réalisation de l’atteinte de cet objectif d’après votre expérience ? 4. A votre avis, quel est l’objectif ou les objectifs de NECS qui enregistre le moins de résultats à ce jour ? Qu’est-ce qui pourrait expliquer cette situation ? 5. D’après votre expérience quelles ont été les principales contraintes dont le projet NECS a été confronté dans son exécution ? 6. Quelles appréciations faites-vous aujourd’hui des effets du projet NECS dans les performances en lecture des élèves et dans les compétences des enseignants par rapport à la lecture ? 7. Qu’en est-il des effets sur la mobilisation/engagement des communautés autour de l’éducation et plus particulièrement de la lecture ? 8. Quelle appréciation faites-vous de l’approche ASL déroulé par NECS sur les performances en lecture des élèves (curriculum, outils pédagogiques et démarches/méthodes) ? 9. Quels ont été selon vous les principaux défis et challenges auxquels le comité ASL a été confronté durant ses interventions dans les écoles NECS et comment les ont- ils contournés ? Quels sont selon vous les principaux points forts et points faibles de l’ASL ? 10. Comment pensez-vous que l’évolution des capacités en lecture des élèves, grâce à l’ASL dans les langues locales pourrait avoir des incidences ou prolongements sur la langue française ? 11. Quelles sont les innovations apportées par NECS dans les aspects relatifs au « genre » et comment appréciez-vous les résultats obtenus dans ce sens ? 12. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être ? 13. S’il y’a une chose que vous aurez aimé changer du projet NECS qu’est-ce ce que cela pourrait être ? 14. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir (dans le cadre d’une mise à l’échelle par exemple) ? 137 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS Directeurs d’écoles Activités (réussites/défis/contraintes) Depuis quand êtes-vous directeur de cette école ? Depuis quand le projet USAID/NECS mène des activités dans cette école ? 8. Quelles sont les principales activités déroulées par USAID/NECS au sein de votre « établissement ? 9. Dans quelles activités de NECS avez-vous pris part ? 10. Qu’avez-vous principalement retenu de la mise en œuvre des activités du projet NECS en vous référant à son historique (de 2012 à aujourd’hui) ? 11. De votre point de vue, quels ont été les activités à succès du projet ? Qu’est-ce qui a favorisé cela ? 12. De votre point de vue, quelles ont été les activités qui ont enregistré les plus faibles résultats/performances ? Qu’est-ce qui explique cette situation ? 13. Avec l’intervention de NECS au sein de l’école, quels types de changements pouvez-vous partager avec nous ? Lecture 7. De votre expérience, comment appréciez-vous l’approche d’apprentissage de la lecture déroulée par NECS dans votre établissement (ASL) ? Comment cela vous a-t-il aidé dans la mise en œuvre des cours, surtout en ce qui concerne l’enseignement de la lecture ? 8. Quels sont les atouts et limites que vous avez relevé dans l’approche ASL ? Quelles sont les suggestions pour son amélioration ? 9. Quelles sont les différentes approches en lecture que vous connaissez en dehors de l’ASL ? Quelle comparaison pouvez-vous faire entre ces différentes approches ? 10. Est-ce que le projet NECS a contribué à rehausser les compétences des enseignants dans l’enseignement de la lecture aux élèves de CI et CP ? Si oui, comment ? Donner des exemples. 11. Pensez-vous que cette nouvelle approche d’enseignement de la lecture prôné par NECS a pu aider les élèves à avoir de meilleures performances en lecture ? Si oui, comment et dans quelle mesure (notes des élèves, assiduité à l’école) ? 12. Remarquez-vous des changements dans le comportement ou les attitudes vis-à-vis de la lecture chez les élèves grâce au NECS ? Si oui, lesquels ? (Participation en classe, concentration sur les enseignements, intérêt pour la lecture) Formations Plan d’action Code de conduite Mentorat Alphabétisation ASL CGPE CGDES La bibliothèque Lecture publique Séances lectures Gouvernement scolaire Concours de lecture Cours de rattrapage Déparasitage Sensibilisation USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 138 Communauté 6. Comment appréciez-vous la participation de la communauté dans les activités de NECS ? Qu’en est-il de cette participation communautaire dans la gestion de l’école ? (Entretien des infrastructures, plans d’actions, contributions financières) 7. Voyez-vous des changements dans la motivation de la communauté autour de l’éducation en général depuis l’arrivée du projet dans le village? Si oui, quel genre de changements ? Donner des exemples. 8. Qu’en est-il de la motivation de la communauté pour la lecture et la scolarisation des filles depuis que le projet NECS est dans le village ? Donner des exemples. 9. Avez-vous remarqué une évolution des relations entre l’école et la communauté depuis l’arrivée de NECS dans le village ? Si oui, comment cette évolution se manifeste-t-elle? Genre 5. USAID/NECS avait pour objectif de promouvoir la scolarisation et le maintien des filles à l’école, quelles observations pouvez-vous faire sur cela ? 6. Remarquez-vous des changements dans le comportement ou les attitudes par rapport au « Genre » et à l’Equité » grâce au activités de NECS ? Quels types de changements et chez quels acteurs (élèves, les enseignants, les parents, vous-même) ? 7. Connaissez-vous le Code de Conduite développé par NECS ? Quelles sont vos appréciations concernant ce code par rapport aux changements induits ? Quels sont ses atouts et limites ? Suggestions 4. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être ? 5. S’il y’a une chose que vous aurez aimé changer du projet NECS qu’est-ce ce que cela pourrait être ? 6. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir ? 139 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS Enseignants CI et CP Activités (réussites/défis/contraintes) 14. Depuis quand enseignez-vous dans cette école ? Depuis quand le projet USAID/NECS mène des activités dans cette école ? 15. Quelles sont les principales activités déroulées par USAID/NECS au sein de cet établissement ? 16. Quelles sont les activités de NECS qui concernent directement les enseignants ? 17. De toutes les activités de NECS, laquelle vous a le plus marqué en tant qu’enseignant et pourquoi ? 18. De votre point de vue, quels ont été les activités à succès du projet ? Qu’est-ce qui a favorisé cela ? 19. De votre point de vue, quelles ont été les activités qui ont enregistré les plus faibles résultats ? Qu’est-ce qui explique cette situation ? 20. Avec l’intervention de NECS au sein de l’école, quels types de changements pouvez-vous partager avec nous aujourd’hui ? Lecture 13. Comment appréciez-vous l’approche d’apprentissage de la lecture déroulée par NECS dans votre établissement (ASL) ? Comment cela vous a-t-il aidé dans les leçons, surtout en ce qui concerne l’enseignement de la lecture ? 14. Quels sont les atouts et limites que vous avez relevé dans l’approche ASL ? Quelles sont les suggestions pour son amélioration ? 15. Est-ce que le projet NECS a contribué à rehausser vos compétences en tant qu’enseignants dans l’enseignement de la lecture aux élèves de CI et CP ? Si oui, comment ? Donner des exemples. 16. Pensez-vous que cette nouvelle approche ASL prôné par NECS a pu aider les élèves à avoir de meilleures performances en lecture ? Si oui, comment et dans quelle mesure (notes des élèves, assiduité à l’école, etc.) ? 17. Remarquez-vous des changements dans le comportement ou les attitudes des élèves vis-à-vis de la lecture grâce à l’ASL en particulier et au NECS en général ? Si oui, lesquels ? (Participation en classe, concentration sur les enseignements, intérêt pour la lecture) Communauté 10. Comment appréciez-vous la participation de la communauté dans les activités de NECS ? Qu’en est-il de cette participation communautaire dans la gestion de l’école ? (Entretien des infrastructures, plans d’actions, contributions financières) Formations Plan d’action Code de conduite Mentorat Alphabétisation ASL CGPE CGDES La bibliothèque Lecture publique Séances lectures Gouvernement scolaire Concours de lecture Cours de rattrapage Déparasitage Sensibilisation USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 140 11. Voyez-vous des changements dans la motivation de la communauté autour de l’éducation en général depuis l’arrivée du projet dans le village? Si oui, quel genre de changements ? Donner des exemples. 12. Qu’en est-il de la motivation de la communauté pour la lecture et la scolarisation des filles depuis que le projet NECS est dans le village ? Donner des exemples. 13. Avez-vous remarqué une évolution des relations entre l’école et la communauté depuis l’arrivée de NECS dans le village ? Si oui, comment cette évolution se manifeste-t-elle? Genre 8. USAID/NECS avait pour objectif de promouvoir la scolarisation et le maintien des filles à l’école, quelles observations pouvez-vous faire sur cela ? 9. Remarquez-vous des changements dans le comportement ou les attitudes par rapport au « Genre » et à l’Equité » grâce au activités de NECS ? Quels types de changements et chez quels acteurs (élèves, les enseignants, les parents, vous-même) ? 10. Connaissez-vous le Code de Conduite développé par NECS ? Quelles sont vos appréciations concernant ce code par rapport aux changements induits ? Quels sont ses atouts et limites ? Suggestions 7. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être ? 8. S’il y’a une chose que vous aurez aimé changer du projet NECS qu’est-ce ce que cela pourrait être ? 9. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir ? 141 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) Communautés (AME, APE, CGPE) 1. En tant que membre de la communauté, quelles sont les principales activités déroulées par le projet NECS avec vous ? 2. Qu’avez-vous principalement retenu de la mise en œuvre de ces activités du projet NECS en vous référant à son historique (de 2012 à aujourd’hui) ? 3. Quels sont les changements majeurs (importants de votre point de vue) que vous avez relevé depuis l’arrivée du projet NECS au sein de votre communauté ? 4. Quelles sont les formations/activités de sensibilisation reçues de la part du projet et quels ont été leurs effets sur votre motivation à participer au système éducatif local ? Quels sont les thèmes généralement abordé par les agents du projet ? 5. Avec l’arrivée de NECS comment les liens entre l’école et la communauté d’une part et d’autre part les liens entre la communauté et les structures éducatives de l’Etat au niveau local ont été renforcé ? 6. Comment voyez-vous aujourd’hui l’intégration et la rétention des filles au sein des écoles ? 7. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être ? 8. S’il y’a une chose que vous aurez aimé changer du projet NECS qu’est-ce ce que cela pourrait être ? 9. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir (dans le cadre d’une mise à l’échelle par exemple) ? Formations Plan d’action Code de conduite Mentorat Alphabétisation ASL CGPE CGDES La bibliothèque Lecture publique Séances lectures Gouvernement scolaire Concours de lecture Cours de rattrapage Déparasitage Sensibilisation USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 142 EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group/KII) Autres partenaires techniques et financiers (Concern, ELAN, IRC, etc.) 1. Pouvez-vous nous décrire brièvement vos principales approches en matière de « Lecture » au Niger ? 2. Quelles sont les principaux enjeux auxquels vous êtes confronté dans le déroulement de vos activités ? 3. Quelles sont les stratégies généralement déployées pour y faire face ? 4. Comment percevez-vous le contexte nigérien par rapport aux programmes et initiatives dans le secteur de l’éducation ? 5. Quelles sont appréciations que vous pouvez faire aujourd’hui de l’initiative NECS par rapport à ses objectifs sur les performances en lecture des élèves et la motivation/engagement communautaire autour de l’éducation ? 6. S’il y’a une chose que vous aurez aimé conserver du projet NECS qu’est-ce que cela pourrait être ? 7. Quelles sont vos principales suggestions pour une meilleure fonctionnalité d’une initiative comme NECS à l’avenir (dans le cadre d’une mise à l’échelle par exemple) ? 143 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV EVALUATION FINALE DES PERFORMANCES DE NECS GUIDE DES DISCUSSIONS (Focus Group) Gouvernement scolaire (GS) [Discuter avec les enfants pour les mettre à l’aise. Jouer. Taquiner.] Pouvez-vous vous présentez (âge, classe, poste) 1. Qu’est-ce que vous faites comme activités ? Pourriez-vous décrire les activités du GS ? 2. Depuis combien de temps participez-vous au GS ? 3. Combien de filles et combien de garçons y’a-t-il dans le GS ? 4. A quelle périodicité se tiennent les réunions du GS ? Avez￾vous participé à la dernière réunion ? 5. Participez-vous dans d’autres activités de NECS ? Si oui, lesquelles ? 6. Quels sont les types de problèmes que gèrent le GS ? Formations Plan d’action Code de conduite Mentorat Alphabétisation ASL CGPE CGDES La bibliothèque Lecture publique Séances lectures Gouvernement scolaire Concours de lecture Cours de rattrapage Sensibilisations Déparasitage 137 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV ANNEX XI: TABLE DES PERFORMANCES DES ELEVES DESAGREGEES PAR SEXE (T = Trouvé ; NT = Non Trouvé) NIVEAU IDENTIFICATION SYLLABES IDENTIFICATION MOTS LECTURE PHRASES T - FILLES NT - FILLES T - GARÇONS NT - GARÇONS T - FILLES NT - FILLES T - GARÇONS NT - GARÇONS T - FILLES NT - FILLES T - GARÇONS NT - GARÇONS CI 1 2 3 0 2 1 2 1 1 2 2 1 CP 1 2 3 0 0 3 2 1 0 3 1 2 CI 3 0 3 0 2 1 1 2 2 1 1 2 CP 2 1 3 0 2 1 1 2 2 1 3 0 CP 3 0 3 0 3 0 3 0 3 0 3 0 CI 3 0 1 2 2 1 2 1 3 0 2 1 CP 3 0 3 0 2 1 2 1 2 1 2 1 CI 2 1 3 0 2 1 3 0 2 1 3 0 CI 3 0 1 2 2 1 1 2 2 1 1 2 CP 3 0 3 0 1 2 3 0 3 0 3 0 CI 1 2 1 2 2 1 0 3 1 2 0 3 CP 1 2 2 1 1 2 2 1 0 3 2 1 CI 1 2 2 1 0 3 2 1 0 3 3 0 CP 1 2 1 2 0 3 0 3 0 3 0 3 CI 1 2 1 2 1 2 1 2 1 2 1 2 CP 1 2 1 2 0 3 0 3 1 2 0 3 CP 1 2 2 1 2 1 2 1 0 3 2 1 CI 1 2 0 3 1 2 0 3 1 2 1 2 CP 3 0 2 1 3 0 2 1 3 0 2 1 CI 1 2 2 1 1 2 1 2 2 1 2 1 USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 138 Totaux 36 24 40 20 29 31 30 30 29 31 34 26 USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 138 ANNEX XII: NECS THEORY OF CHANGE (Source: AMEP, November 2016, p. 2) USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 138 ANNEX XIII: COMPARISON OF READING APPROACHES IN NIGER READING PROGRAM NECS REFORM ELAN CONCERN IRC* APPROACH ARL (VIE) ASL (CI) APS – no particular approach ARL ARL Explicit, systematic, guided lessons YEAR BEGAN 2013 2015 (CI); 2017 (CP) 2015 2013 2014 2017 GRADE LEVELS IMPLEMENTED CI-CP (2 grades) CI-CP (2 grades) CI-CM1 (5 grades) Ci-CE (3 grades) CP – CE2 (3 grades) TYPE OF SCHOOL Traditional Bilingual + Traditional Bilingual Traditional Traditional Traditional # SCHOOLS 12 Bilingual; 148 Traditional 500 (5000 as of fall 2017) 10 71 29 LOCATIONS 5 Regions (Diffa, Tahoua, Tillabery, Maradi, Zinder) All 8 Regions Niamey, Maradi Diffa, Tillabery (previously Tahoua) Diffa LANGUAGES Local Local + French Local French SPECIFIC LOCAL LANGUAGES Fulfulde, Hausa, Kanuri, Zarma Fulfulde, Hausa, Kanuri, Zarma Hausa, French Hausa, French N/A TRAINING 5 days 10-15 days (reduced from original 45) 20 days (10 days pre￾service; 2 5- day in) 25 days (5 days bilingual program; 10 days ARL); 10 day in) 2 days literacy (part of a larger training) USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 138 ANNEX XIV: AVAILABLE PERFORMANCE RESULTS ON READING PROGRAMS IN NIGER RECENT ASSESSMENTS OF 1ST AND 2ND GRADE READING IN NIGER, COMPARATIVE TABLE (PART I) Object of Study/Program PASEC2014 Teaching and learning innovations in Niger (ELAN, NECS, CONCERN, Bilingual schools) NECS Impact Evaluation (Baseline) Mathematica EGRA NECS Impact Evaluation (End￾line) Date of data collection 2014 May 2015 October-November 2013 May 2014 May 2016 Publishing date 2016 Summer 2015 April 2015 April 2015 July 2017 Author CONFEMEN Direction de l’évaluation du suivi des acquis scolaires et de l’orientation (MEP/A/PLN/EC) Mathematica Mathematica Mathematica Sample Stratified sample; CP: 86 schools, 813 students, 6 regions Stratified sample; 551 schools; 6 regions; 521 CI students (207 – traditional bilingual schools; 40 – CONCERN, 139 traditional schools, 40 – ELAN, 95 – NECS); 295 CP students (traditional bilingual schools, traditional schools, ELAN) Clustered random assignment (2 rounds): NECS-only and control villages; includes 149 NECS intervention villages in the impact evaluation sample who were enrolled in grade 1 or grade 2 during the previous school year 1,007 students in grades 1 and 2 from 27 randomly selected NECS intervention schools Clustered random assignment (2 rounds): NECS-only and control villages; includes 142 NECS intervention villages in the impact evaluation sample Reading Task(s) Oral comprehension, decoding, written comprehension Letter identification (CI) Minor adaptions to EGRA (2009): oral language, letter recognition, word reading, oral reading fluency, and reading comprehension; plus receptive and expressive oral vocabulary skills Based on EGRA (2009): letter identification, familiar word reading, invented word reading, oral reading fluency of grade 2– level text, and reading comprehension. Minor adaptions to EGRA (2009): oral language, letter recognition, word reading, oral reading fluency, and reading comprehension; plus receptive and expressive oral vocabulary skills Language(s) French National languages; French Hausa, Zarma, Kanuri and another local language Hausa, Zarma, Kanuri and another local language Hausa, Zarma, ‘Kanuri and another local language 139 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV RECENT ASSESSMENTS OF 1ST AND 2ND GRADE READING IN NIGER, COMPARATIVE TABLE (PART I) Major findings (relevant to CI & CP) 9.8% of students above threshold (under 10%) (composite score) Average scores – Reading letters: NECS: 61.7%; ELAN: 56.3%; CONCERN: 47.9%; traditional schools: 21.8%; bilingual traditional schools: 21.6%. Threshold analysis: Above minimal threshold: NECS: 72.6%; ELAN: 57.5%; CONCERN: 47.5%; traditional schools: 19.5%; traditional bilingual schools: 18.3% “Most children possess strong oral language skills as measured by the first three subtasks— receptive oral language, expressive oral language, and listening comprehension. Reading skills for all four languages, however, were very low and nearly indistinguishable from 0.” (p. 100); Identified strong floor effects in reading tasks. NECS-only schools have significantly lower rates of school enrollment the previous year 2012-2013 (53.8% compared to 58.8%; significant at the 5 percent level); NECS-only school children are more likely to report absences of more than two consecutive weeks (52% compared to 48.3%; significant at the 10 percent level). Scores are highest for letter identification, though they are still low. Depending on the language, children are able to identify an average of between three and nine letters per minute out of a possible 100. Scores in letter identification for grades 1 and 2 are comparable. This may reflect the effect of the NECS intervention, which had been implemented in grade 1 for four months at the time of data collection. The NECS intervention had not begun in second grade yet by the 2013– 2014 school year. In both grades, scores are much lower for reading skills other than letter identification. Students in all languages identified, on average, less than one familiar word per minute (out of a possible 50). Mean scores for invented word reading, oral reading fluency, and reading comprehension are near or equal to zero. We find no significant, consistent differences in scores between students of different grades, genders, NECS intervention groups, or regions. There are strong floor effects in these data in all languages and both grades.” (p. xi) The NECS only: a 0.15 standard deviation positive impact on normalized local￾language test scores: NECS & IMAGINE: a 0.21 standard deviation positive impact on normalized local-language test scores (p. xviii) EGRA results show low levels of reading proficiency and strong floor effects in letter identification, familiar word reading, oral reading fluency, and reading comprehension subtasks. Letter identification subtask than in the previous studies. There are fewer floor effects for the oral language skills, which include receptive vocabulary, expressive vocabulary, and oral comprehension. Increases over wave 1 scores for letter recognition (new average of around 8.1 to 9.9 letters per minute; previous was less than one letter). Across all languages, mean scores are close to zero for reading comprehension. Of 50 words per minute, on average, children are able to read 1 to 3 wpm, with two children able to read all 50 wmp (one in Hausa, one in Zarma). (p. F.4) Limitations (relevant to NECS) ASL began in March 2015 (only 2 months prior to assessment). Does not include invented word reading; Cannot compare EGRA results across languages Cannot compare EGRA results across languages Cannot compare EGRA results across languages USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 140 RECENT ASSESSMENTS OF 1ST AND 2ND GRADE READING IN NIGER, COMPARATIVE TABLE (PART I) Notes Other tasks tested but NECS only tested on letter recognition If demonstrates any change due to NECS intervention, is related to ARL - At time of data collection, “grade 1 students had four months of the NECS early grade reading curriculum and grade 2 students had not had any of the early grade reading curriculum.” ASL is introduced in March 2015. At time of data collection, all CI students had been exposed to ASL and CP students had continued to use CI ASL materials in CP. No ASL CP materials had yet been introduced. RECENT ASSESSMENTS OF 1ST AND 2ND GRADE READING IN NIGER, COMPARATIVE TABLE (PART II) Object of Study/Program NECS EGRA 2017 NECS EGRA 2018 CONCERN: Early Grade Reading Assessments and the Bilingual Pilot - Tahoua, Niger ELAN Réforme – Final Evaluation (Rapport aide-memoire) Date of data collection May 2017 May 2018 Baseline: November 2013; Endline: May 2017 2013-2015 Unknown Publishing date June 2018 Not yet published (expected: August 2018) 2017 December 2014 September 2017 Author Plan/STS Plan/STS Concern OIF /Université de Nantes MEP/A/PLN/EC Sample Stratified sample based on language; 53 NECS schools; 962 students – new sample drawn Stratified sample based on lessons completed. All 14 schools in phase 1 in Tahoua; in 2013 sample aimed to be representative of each gender in each grade; in 2017 sample aimed to be representative of each grade only N/A – see limitations below 775 students (100 of which were from traditional schools) Reading Task(s) Adapted from Mathematica’s EGRA: letter sound identification, familiar word reading, nonword reading, oral passage Adapted from Mathematica’s EGRA: letter sound identification, familiar word reading, nonword reading, oral passage Letter sound knowledge, familiar word reading, invented word reading, oral reading fluency, reading comprehension N/A Letter identification, syllable recognition, simple word and phrase reading 141 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV RECENT ASSESSMENTS OF 1ST AND 2ND GRADE READING IN NIGER, COMPARATIVE TABLE (PART II) reading and reading comprehension reading and reading comprehension Language(s) Zarma, Hausa, Kanuri, and Fulfulde Zarma, Hausa, Kanuri, and Fulfulde Hausa and French Hausa and French National languages Major findings (relevant to CI & CP) Only 1 Zarma-instructed student (female) meets benchmark of reading 40 wpm with less than 3 mistakes; Across 3 languages, boys’ and girls’ scores were statistically similar for all subtasks. Letter sound identification mean scores were the highest for all languages, ranging from 4.35 to 18.59 for CI and from 10.66 to 21.61 out of 50. Other reading tasks scores were quite lower with a high number of zero results, particularly for reading comprehension. Findings not yet released. After three years of schooling, only 5% of CE1 students reach the threshold of 45cwpm in Hausa, and only 2% in French, while only twenty￾two percent (22%) reach the comprehension threshold of 80% and only 12% for French. In addition, over a quarter of CE1 students are unable to read a single word in either language at the end of the model. Overall significant reduction in zero scores across subtasks. Reduction in number of CP and CE1 students who could not read a single word from nearly 100% to between 25- 34%. Between baseline and endline, CP scores increase in Hausa by 11 cwpm, and in French by 7 cwpm. N/A Results within Reform schools are better than within traditional schools. [Statistics not available in this document – no other documents accessible to evaluation team.] Limitations (relevant to NECS) The EGRAs were developed prior to the release of the EGRA Toolkit 2.0. The tools are missing listening comprehension, one of the four core tasks currently required of all EGRAs. There is no baseline for comparison. Benchmark development did not adhere to international standards. Small sample size results in a high margin of error. Reliability of findings is weakened by “The fact that pedagogical advisors were using the same EGRA test in schools is evidence of test leakage and prevents comparison of these results to baseline for evaluative purposes. Once test leakage has occurred, conclusions from the results cannot be used to answer the research questions of the evaluation study.” (STS Memorandum: NECS+ 2017 and 2018 EGRA Data Quality; June, 2018) Reduction in sample size between endline and baseline. Change in timing definitions for sub-tasks between applications. Due to data entry problems, the first data collection was not analyzed and the second data collection did not happen due to financial difficulties. Because this is a multi￾country study, Niger still figures in report, though there is no data. There are no other known evaluations of the ELAN program in Niger. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 142 RECENT ASSESSMENTS OF 1ST AND 2ND GRADE READING IN NIGER, COMPARATIVE TABLE (PART II) inconsistencies in timing and not following EGRA protocols. Notes CP materials implemented in November 2017 – results can only reflect exposure to CI TLMs Full evaluation report was not available; current report identified programmatic weaknesses and strengths as well as a list of recommendations USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 138 ANNEX XIV: FINDINGS AND CONCLUSIONS TABLE – PRELIMINARY RESULTS FINDINGS SOURCES CONCLUSIONS EQ 1. TO WHAT EXTENT HAS NECS ACHIEVED ITS OBJECTIVES IN RELATION TO ITS TARGETS? C1.1. The objectives for NECS have evolved between IMAGINE, NECS and NECS+. At the beginning of the IMAGINE project, the program objective was to “support primary school education for girls” (SOAG, p. 1). Over time, the objectives focus more specifically on reading. This is clear from the fact that the order of the strategic objectives was switched between NECS and NECS+, more specifically, that the strategic objective linked to reading became the first strategic objective. AMEP Revised Version, November 11, 2016 ; Mid-term evaluation; SOAG MCC Threshold Program (March 17, 2018) 1. NECS project indicators and targets evolved towards a greater emphasis on reading. Throughout the project’s life cycle and responding to the reordering of strategic outcomes, the focus on reading was reinforced within the indicators (C1.1, C1.2,) 2. Despite improvements within the M&E system during the NECS+ phase, weaknesses within the M&E system remain. (C1.3, C1.4, C1.5, C1.6, C1.7, C1.8, C1.9, C1.10, C1.11, C1.12) 3. Numerous testimonies of NECS successes in improving reading were present at both the central and regional levels. Nonetheless, these observations are not founded on solid performance-based evidence. (C1.13, C1.14, C1.15, C1.16, C1.17, C1.18, C1.19, C1.20, C1.21) 4. The project suffered from deficiencies in the ability to manage an EGRA process that incorporates consistent data collection practices and produces valid data. (Plan US, STS, NECS) (C1.5, C1.18, C1.21) 5. Several factors make it impossible to evaluate improvements in student grade reading achievements that may be due to the NECS project, including the significant delay in the development of reading materials and the lack of valid comparable EGRA scores. In addition, 2016- 2017 academic year disturbances and irregularities in EGRA application decrease the possibilities of demonstrating the effects of the NECS progress in a rigorous manner. As a C1.2. The evolution of the indicators demonstrates a greater focus on reading. During the NECS phase (2012-2016), there were 34 indicators, with 6 performance indicators linked to the two strategic objectives, 28 indicators focusing on activities (water points, mentoring, support for community structures, adult literacy). Of these, 7 indicators addressed reading as well as two others that focused on the distribution of materials. The AMEP for NECS+ reduced the number of indicators to 10 in order to focus on 6 USAID Standard indicators and to have a more manageable number of indicators. Among the 10 NECS+ indicators, 6 address reading, 1 addresses school access for girls, 1 addresses community reading practices, 1 addresses the functionality of supporting school structures, and 1 addresses parent and community engagement. AMEP Revised Version, November 11, 2016 ; Mid-term evaluation; KII M&E Specialist C1.3. The project’s monitoring and evaluation has experienced some challenges, such as changes in M&E Specialists at Plan International (USA) as well as NECS, a high number of indicators (38) for the NECS phase (reduced to 10 for NECS+). The mid-term evaluation that took place in July 2015 identified a number of weaknesses within the M&E system. The NECS M&E Specialist expressed feeling very little support after a certain M&E colleague at Plan USA left. One of the persistent constraints seems to be the need to collect data for certain indicators bi-annually. Nonetheless, the Monitoring, Evaluation, Research & Learning (MERL) Advisor at SRO Dakar identified having accompanied the NECS team in identifying indicators as well as the DQA annual visits. The MERL advisor is preparing for a trip to Niamey at the end of June for a Data Quality Assessment. FGD NECS Staff ; Mid-term evaluation; AMEP Revised Version, November 11, 2016 ; ITT Document ; KII MERL Advisor, SRO Dakar C1.4. Lack of continuity in M&E Specialists (high turnover) led to weaknesses in M&E system’s efficacy as well as application. FGD NECS Staff, KII M&E Specialist ; Document review: 139 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS annual reports, Mid-term evaluation report consequence, IR 1.1 has not been met. (C1.16, C1,18, C1.21, C1.22, CI.23, C1.26, C1.28) 6. The NECS project provided teaching and learning materials for first and second grade and distributed them to schools in a way that exceeded established targets. The same is true for teacher training. In regards to books, difficult conditions and having to replace some of the first grade books brings into question the durability of materials. (C.1.27, C1.29, C1.30, C1.31, C132, C1.33, C1.34, C1.35) 7. After 5 years of operation, it’s only in 2017-2018 that 2nd grade (CP) materials reach teachers and there’s finally a project cohort. This delay reduces the possibility of the new teaching method improving reading instruction. (C.1.22, C1.29) 8. The NECS project met the target number of students enrolled in project schools and the number of 1st (CI)/2nd graders (CP) directly reached by the project. However, interruptions such as teacher strikes and students leaving after the first rainfall are identified as obstacles to quality education. (C.1.23, C1.25, C1.26) 9. At the end of the project, 5 of 10 targets have been met and 5 remain at risk. There is misalignment between indicator targets and actual project performance (over-estimation – Indicator 8 related to girls’ access or under-estimation – Indicators 5 and 6 related to materials distribution). (C1.,16, C1.24, C1.25, C1.27, C1.29, C1.34, C1.36, C1.38, C1.40, C1.41) 10. Given the lack of credible reading performance results (Indicator #1), it will be very difficult to defend the successes of the project. Even though observations show that communities are increasingly involved in schools, all of the targets related to community involvement in education C1.5. Weaknesses in technical M&E support available to NECS from Plan International (USA))/Plan Niger and SRO/USAID. FGD NECS Staff, KII M&E Specialist ; Document review: annual reports, Mid-term evaluation report C1.6. Persistent difficulties with data collection frequency for indicators 7, 8, 9 and 10 identified in the AMEP as bi-annual. FGD NECS Staff, KII M&E Specialist, Document review: annual reports : ITT Tracking Table C1.7. Weaknesses in M&E results framework don’t allow for continuity given turnover of M&E Specialists. FGD NECS Staff, KII M&E Specialist ; Document review: ITT Tracking Table C1.8. NECS M&E Specialist identifies a lack of regular documentation in the monitoring of community activities and, as a result, the inability to meet certain targets. FGD NECS Staff, KII M&E Specialist ; Document review: ITT Tracking Table C1.9. Lack of M&E database that would allow for activity-level information to be centralized. FGD NECS Staff, KII M&E Specialist ; Document review: annual reports, Mid-term evaluation report C1.10. NECS M&E personnel are not sufficiently supported. Trainings, technical support or coaching are not available. FGD M&E Specialist and Interim COP; KII Spécialiste mobilisation sociale C1.11. Weaknesses in monitoring system coupled with lack of M&E expertise at the field level (negative effects on information quality, level of detail of data collected). FGD NECS Staff, KII M&E Specialist C1.12. The NECS M&E Specialist identified the lack of regular documentation from communities as an obstacle to meeting certain targets. Discussions with community members as well as school directors revealed these same challenges that are linked to low literacy levels among community members. Document review– Indicator Tracking Table 2018 ; KII M&E Specialist SO1: Increased student grade reading achievement by improving instruction in primary schools and promoting a culture of reading in communities IR1.1: Improved reading instruction in primary schools through the development and delivery of the ASL curriculum by school directors, teachers, and pedagogical advisors USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 140 FINDINGS SOURCES CONCLUSIONS IR1.2: Culture of reading is promoted in communities through monitoring and support from Adult Literacy Inspections and School Management Committee Focal Points. For this phase of the project, this will move toward supporting these inspection authorities to monitor and promote community reading activities. are at risk. At the same time, Indicators 7, 9 and 10 are complex indicators with multiple criteria. In addition, given that the formal adult literacy phase of the project ended in 2014, it is unreasonable that relevant community organizations are able to develop meeting minutes as proof of having met. (C1.16, C1.18, C1.19, C1.21, C1.36, C1.37, C1.40, C1.41, C1.42) C1.13. The NECS+ project’s strengths were recognized when it participated in the « forum nationale sur l’éducation. » Because this forum was reserved for programs with “good practices,” its participation indicates central stakeholders’ belief in the strengths of NECS. During the presentation, NECS brought in a teacher to do a demonstration with students. Several upper-level officials in the MEP directorates in Niamey, as well as NECS staff, mentioned the significance of this activity. KII M&E Specialist, NECS+ Quarterly report Jan-March 2018 ; KIIs COP, NECS Staff ; DG… C1.14. Diverse actors unanimously recognized the strengths of NECS’ work to reinforce teaching capacities in reading, including the ASL Committee, the Direction du Curricula, la Direction de la Formation initiale et continue, NECS staff, Aide-et-Action and USAID. Program elements they identified include that students learn to read within a short time period, a systematic and interesting methodology, and the supports provided. Scalability Work Session C1.15. NECS followed this same process during a Steering Group (Comité de Pilotage) meeting. After the demonstration, the Minister identified a series of steps to follow to capitalize on good practices. Among other very positive appraisals of the project, it is noted in the meeting minutes that “With unanimity everyone recognized the efficacy of the (ASL) reading approach that has strongly contributed to improving reading performance of schoolchildren.” Steering Committee Minutes 2018; KII Sec. Générale du MEP ; KII DG de la qualité et de la normalisation C1.16. Indicator #1: Percent of learners who demonstrate reading fluency and comprehension of grade level text at the end of grade 2 with USG assistance (USAID Standard Indicator ES.1-1). Targets set prior to 2017 EGRA and approved by the Ministry: At least 50% of CP (grade 2) students: • Read a text of 40 1-3 syllable words within a minute without making more than 3 mistakes • Correctly respond to 2 out of 3 simple questions about the text (if the student cannot correctly read the text, the text will be read aloud prior to asking the questions). For the 2017 EGRA, only one student (female) within the sample succeeded in meeting this threshold (0.26% of the sample), a percentage far below the target of 50%. Document review: (Benchmarks_Objectifs de la lecture et grille de progression CI_CP) ; EGRA 2017 Report ; KII NECS M&E Specialist 141 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS C1.17. Some expressed questioning the relevance of this wpm threshold given the situation in Niger and the skill level of students and teachers. There was some mention that the ministry was questioning the threshold as too ambitious. The USAID Washington Senior Education Advisor also found that the threshold was high as after two years of instruction it is reasonable that children are able to read between 14 and 25 wpm in a language that they know. KII M&E NECS Specialist ; KII Senior Education Advisor, USAID Washington C1.18. The 2017 EGRA data collection encountered many difficulties including confusion between Plan and STS about leading the assessment, problems following application protocols, for example, timing irregularities. KII NECS M&E Specialist; FGD STS C1.19. The 2017 EGRA report indicates that the 2017 assessment will not serve as a baseline because students already benefitted from NECS activities. EGRA 2017 Report C1.20. STS is currently analyzing the 2018 EGRA data. The report is expected at the end of June. KII Plan Program Manager ; FDG STS; KII NECS Education Specialist C1.21. Meetings with many of the NECS staff, the USAID Education Specialist and the interview with STS revealed that there is a good chance that the 2018 EGRA data will not be reliable due to contamination (test leakage). With the intention to better prepare students for the assessment, and with the support of the project, Pedagogical Advisors used the 2017 EGRA test as a regular mini￾assessment exercise during school visits. FGD STS ; KII M&E Specialist ; KII USAID Education Officer ; KII NECS Education Specialist; KII Plan USA C1.22. First grade (CI) materials were introduced late in the project cycle, in March 2015. The second grade (CP) materials were introduced in October/November 2017. The project’s only cohort emerged during the 2017-2018 school year, that is, students benefitting from ASL in first grade (2016-2017) and in second grade (2017-2018). Document review: annual reports, FGD Readsters, FGD NECS staff ; KII USAID AOR ; KII USAID Education Officer ; KII Country Director, Plan Niger; FGD Staff Aide et Action C1.23. There were multiple interruptions in the 2016-2017 school year, especially due to teacher strikes. As an example, in Dosso there were 65 declared strike days. There were fewer strikes in 2017-2018. KII DG de la qualité et normalisation, KII M&E Specialist, FGD staff NECS : document review : Impact de grèves dans les écoles NECS_Cas de Dosso_2016- 2017 (NECS) USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 142 FINDINGS SOURCES CONCLUSIONS C1.24. Indicator #2 : Number of learners in primary schools and/or equivalent non-school based settings reached with USG education assistance (USAID Standard Indicator ES1.3): As of March, 2018, there were 42,738 students (target met by 101%). Document review– Indicator Tracking Table 2018 ; KII M&E Specialist C1.25. Indicator #3 : Number of learners reached in reading programs/interventions at the primary level (USAID Standard Indicator ES.1-5): As of March 2018, the project exceeded the target (15,100) as there were 15,690 first grade (CI) and second grade (CP) students within the NECS schools (104%, target met). Document review– Indicator Tracking Table 2018 ; KII M&E Specialist C1.26. Data collection at the end of the school year was difficult in all four regions due to student absences. In many of the villages, parents had taken students to the fields after the first rain. Document review– Indicator Tracking Table 2018 ; KII M&E Specialist C1.27. Indicator #4 : Number of primary school educators who complete professional development activities on implementing evidence-based reading instruction with USG assistance (USAID Standard Indicator ES.1-9): As of March 2018, the project had trained 522 teachers (including head teachers), of which there were 195 women and 327 men. The project exceeded the target of 468, a performance of 112%. The number of teachers trained in 2016-2017 was much lower (17% of target). School visits; Document review– Indicator Tracking Table 2018 ; KII M&E Specialist C1.28. Teachers’ skill level is a significant obstacle to teaching quality in Niger. As illustration, the 2017 assessment of contract teachers led to a large portion of teachers’ contracts being cancelled, including NECS teachers. Of the 65 NECS teachers that participated in the assessment, 35 teachers had their contracts cancelled. Another 26 teachers were identified for participation in remediation classes. FGD NECS Staff, KII NECS COP, Education specialist, KII DG de la qualité et la normalisation; Document review : annual report 2017 ; C1.29. Indicator #5 : Number of primary or secondary textbooks and other teaching and learning materials (TLM) provided with USG assistance (USAID Standard Indicator ES.1-12): As of March 2018, the target of 9,400 materials had been exceeded as the project provided 12,960 teaching and learning materials to project beneficiaries (138% of target). These materials include teacher’s kits given to head teachers and 1st (CI)/2nd grade (CP) teachers (teacher’s guide, student book, and student reader), 1 student book per 2 students and for 2nd grade, decodables for children. Document review– Indicator Tracking Table 2018 ; KII M&E Specialist 143 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS C1.30. During the assessment activity with children that took place in schools as part of this evaluation, many children identified the decodable as a book that they had at home. Document review– Indicator Tracking Table 2018 ; KII M&E Specialist C1.31. Classroom visits and observations confirmed the existence of materials even through many schools had already stopped classes after the first rain. Student exercise C1.32. There were delays in the revision and printing of 2nd grade (CP) materials. This delay has been resolved. At the moment of data collection for the final evaluation, the books had been distributed. Grade 1 and 2 lesson observations C1.33. The number of student books in classrooms often exceeds the target of 1 book per 2 children. Grade 1 and 2 lesson observations; KIIs teachers; KII Inspecteur Niamey 1/Ancien DG Scolarisation C1.34. Indicator #6 : Number of primary school classrooms that receive a complete set of essential reading instructional materials with USG assistance (USAID Standard Indicator ES.1-13): As of March 2018, the target had been met and exceeded (434 kits over 353 expected, a rate of 123%). During Q1, the project distributed 155 2nd grade (CP) kits and 279 kits in Q2, the latter including replacement materials for 1st grade (CI). The replacements were in response to teachers identifying issues with deteriorating books due to inadequate conditions in make-shift classrooms. FGD STS ; KII NECS Education Specialist; KII M&E Specialist ; Grade 1 and 2 lesson observations; KIIs teachers C1.35. During interviews and focus groups, over a quarter of teachers spoke about the delicate nature of the books. This was reinforced by the project replacing first grade books in March 2018. Difficult conditions explain book deterioration including the makeshift classrooms and harsh sun as well as children using the books. Document review– Indicator Tracking Table 2018, quarterly reports ; KII M&E Specialist C1.36. Indicator #7: Percentage of active reading communities: This indicator combines three criteria: 1) existence of a village library, 2) existence of an action plan that includes reading activities, 3) implementation of 4 public reading sessions per year. Despite this indicator being identified for biannual data collection, the data collection for this indicator occurs at the end of the year, estimated for the end of June 2018. The percentage attained in 2016- 2017 was 63%, very low compared to the 90% target. KII/FGD teachers ; document review : Observations des enseignants des manuels USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 144 FINDINGS SOURCES CONCLUSIONS C1.37. Field visit observations showed that the majority of villages had libraries. Even more visible, teams noted that in almost all communities CGDES action plans regrouped the action plans of other community associations. Even though stakeholders often cited public reading sessions, it was not possible to identify their regularity. FGD CGDES, FGD Communities, KIIs school directors SO2: Increase access to quality education in schools, especially for girls IR2.1: An improved school environment through monitoring and support from Pedagogical Advisors, School Management Committee Focal Points, and Girls’ Education Focal Points IR2.2: Increased community participation to support education through monitoring and support from Pedagogical Advisors, School Management Committee Focal Points. C1.38. Indicator #8 : Proportion of girls enrolled in the NECS+ project intervention schools in grade 1: As of March 2018, the percentage of girls enrolled in NECS schools was 48%, just under the target of 50%. According to an interview with the M&E Specialist, progress towards 50% has slowed down. Document review– Indicator Tracking Table 2018; KII M&E Specialist C1.39. Schools visited within the sample revealed a lower percentage of approximately 44% out of total school population. School observations, KIIs with IEPs, SCOFIs, teachers and school directors, School Mothers’ Associations, PTAs, Communities C1.40. Indicator #9 : Percentage of schools meeting the NECS minimum standard: This indicator is based on 4 criteria: 1) a functional mentoring system, 2) CGDES active involvement (80% of action plans have been implemented and they meet 7 times per year), 3) adoption of code of conduct, and 4) a functional school government. At the time of the evaluation, no data had been collected for this indicator even though it’s identified for biannual data collection. In June 2017, 31% of communities had met the target. In 2018, this target is at risk. One of the principle factors for this week percentage may be linked to the requirement of needing to produce meeting minutes. Document review– Indicator Tracking Table 2018; KII M&E Specialist C1.41. Indicator #10 : Number of parent teacher associations (PTAs) or community governance structures engaged in primary or secondary education supported with USG assistance: This indicator regroups 3 criteria: 1) active involvement of CGDES (# of meetings), 2) rate of implementation of activities (80%), 3) involvement of contact persons (les points focaux développement). Document review– Indicator Tracking Table 2018; KII M&E Specialist 145 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS At the time of the evaluation, no data had been collected for this indicator even though it’s identified for biannual data collection. As of June 2017, 51% of communities had met these criteria. Again, one of the factors that may explain this low rate may be the need to produce meeting minutes. C1.42. Lack of program monitoring data for certain indicators related to community activities (7, 9 and 10) may be due to low community literacy levels. KII M&E Specialist; Document review– Indicator Tracking Table 2018 EQ 2. HOW DID THE NECS “ASL” APPROACH ENHANCE READING OUTCOMES IN NECS SCHOOLS? IMPLEMENTATION OF “ASL” APPROACH C2.1. Several changes involving new sub-contractors and new methodologies characterize the implementation of the NECS reading strategy: ARL with VIE, ASL for grade 1 (CI) with Readsters, ASL for grade 2 (CP) with STS. Document review; KIIs Education specialist NECS, AOR USAID, Education Officer USAID, FGD Readsters, ASL Committee 1. Participant testimonies from parents, teachers and school directors identify increased demand for as well as improvements in students’ reading skills. The near inexistence of scientific and credible evidence hinders demonstration of these gains. 2. Even though the NECS project began in June 2012, ASL was introduced late due to a number of management constraints and limitations. This late introduction impacted the production of project results. (C2.1, C2.2, C2.3, C2.4). 3. The lack of reading expertise at both Plan and USAID available to support the project and provide technical supervision seem to have contributed to the limitations in the implementation, daily management and supervision of ASL activities. (C2.5, C2.6). 4. The partnership framework that NECS established with the Ministry has strengthened collaboration. This relationship includes both the central level and regional levels. There was also a clear evolution in the Ministry’s involvement during ASL implementation (teacher training, ASL support, and student assessment) in partnership with regional education stakeholders. (C2.7, C2.8, C2.9). C2.2. There were moments when a reading sub-contractor was missing from the project : between the end of VIE’s contract and the recruitment of Readsters (January – November 2014), and between the end of Readster’s contract and the recruitment of STS (May – November 2016) Document review: quarterly and annual reports ; KII/FGDs NECS Education specialist, Former NECS COP; NECS staff, AOR USAID, Education Officer USAID, Plan International (USA) C2.3. It’s only during the 2017-2018 school year that ASL was available for grade 2 (CP) due to delays and gaps in grade 2 implementation. KIIs NECS Education specialist; Document review : quarterly and annual reports) C2.4. Data missing on the impact of ASL in second grade (CP) due to poor alignment between student assessments and ASL implementation. Document review (2015 report on innovations - MEP); KII AOR USAID, NECS Education specialist C2.5. There was an absence of a reading specialist at NECS, Plan Niger, Plan USAID, USAID/Niger as well as USAID/Washington’s general lack of recognition of the education components of which did not allow for adequate support for implementing partners as well as quality control for developed reading methodologies and TLMs. KIIs AOR USAID, Education specialist NECS, Education Officer USAID, Plan Niger, Plan International Washington C2.6. In addition to staff turnover at USAID as well as Plan Niger, NECS and Plan USAID, it was in 2013 that USAID Niger began to provide technical supervision for the project with the arrival of a new AOR with education expertise. In addition, USAID Niger KIIs AOR USAID, Education Officer USAID, Plan Washington USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 146 FINDINGS SOURCES CONCLUSIONS established a position for an Education Officer in 2015. Nonetheless, neither of these individuals were reading specialists. NECS also experienced a prolonged gap in leadership after the departure of the first COP. In regards to reading technical support, no reading specialist was available at NECS, Plan USA nor Plan Niger. The project did not benefit from specific reading support from USAID West Africa nor USAID Senegal (SRO). The USAID Washington office provided some technical knowledge at the beginning of the project and when there were difficulties with reading sub-contractors, a rarity for USAID Washington. During the life cycle of the project, stakeholders relied upon the reading expertise provided by sub-contractors (VIE, Readsters, then STS). There were also gaps between sub-contractor contracts. C2.7. Stakeholders from both central and regional levels identify a thoughtful and rich collaboration with the project (teacher training, ASL supervision, student assessment). KII/FGDs avec DG Qualité et normalisation MEP, Directeur de l’évaluation scolaire MEP, DREP, DDEP, IEP, Pedagogical Advisors, FGD NECS staff, Former NECS COP C2.8. In response to concerns expressed by the Ministry regarding NECS reading program implementation (2012), NECS created a joint committee with the Ministry (2013) that would review the reading program. The ASL Committee evolved in its production of TLMs. Ministry officials, NECS, USAID Niger, Plan Niger/USA unanimously recognize that since STS joined the project the ASL Committee works closely with the NECS project. Document review (quarterly reports), FGD ASL Committee, KIIs Former AOR, Plan International USA, Former NECS COP, NECS Education specialist C2.9. A Steering Committee, a Technical Support Committee as well as focus people were put into place to support NECS within the various directorates of the MEP. Many of these stakeholders shared positive experiences of a fruitful collaboration with the NECS project. KIIs SG de l’enseignement primaire, DG de la Promotion de la Qualité et de normalisation, KII Former NECS COP, FGD NECS staff STUDENT READING PERFORMANCE C2.10. EGRA results analysis (Mathematica 2016, STS 2017, STS 2018) – to be confirmed – at the moment that the table was developed, results were not yet accessible. EGRA (Mathematica 2016, STS 2017, STS 2018) 5. Despite the overwhelming applause for ASL reading instruction, field observations show that a good number of first (CI) and 2nd grade (CP) students still have difficulties reading (C2.11, C2.12, C2.13, C2.14). C2.11. The exercise conducted with 120 students (60 girls and 60 boys) showed that between more than half and less than three quarters of students were able to read letters and syllables. Tendencies were the same for girls and boys. Exercise with 120 1st and 2nd grade students from 20 sample schools within the regions of Tahoua, Tillabéry, Maradi, Zinder. 147 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS C2.12. At the same time, the test conducted with 120 students (60 girls and 60 boys) showed that less than half of students are able to read words and simple sentences. Exercise with 120 1st and 2nd grade students from 20 sample schools within the regions of Tahoua, Tillabéry, Maradi, Zinder. 6. Low teacher qualifications in Niger may affect the implementation of ASL lessons within the classroom and thus, student performance (C.2.15). C2.13. With the exception of one school director, individuals tasked with teacher support (school directors, inspectors, DDEP, DREP) unanimously spoke about how students at NECS schools read better than students at les « écoles traditionnelles. » KII/FGDs with teachers, school directors, IEPs, DDEP, DREP, community members C2.14. With the exception of one school director, all participants within the four regions included in the sample affirmed that CI and CP students are able to read letters, syllables, words and sentences. KII/FGDs with teachers, pedagogical advisors ; ASL Committee, Former NECS COP, NECS Education Specialist, Secrétaire Général MEP C2.15. Several stakeholders identified teacher’s low capacity as a major challenge to attaining good results. As an example, during the 2017 contractual teacher assessment, of the 61 NECS teachers concerned, 35 had their contracts cancelled and 26 were chosen for remediation. KII/FGDs with le Directeur de la qualité MEP, Pedagogical Advisors, Département de l’évaluation du MEP ; Document review ASL METHODOLOGY C2.16. ASL is a new methodology in Niger that integrates several basic reading skills: sound-letter correspondence, decoding, oral and written comprehension and the “whole language” method for transitioning into French. KII/FGDs with ASL Committee, STS Reading Specialist ; Readsters ; Document review: 2nd grade teachers’ guide 7. Thanks to the specificity of ASL structure and program elements, stakeholders generally observe that children have improved their reading skills (C2.16, C2.17, C2.18). 8. Despite the well-thought out nature and application of the ASL methodology, certain stakeholders identify some limitations, especially, the lack of a gradual introduction of French in 2nd grade (CP). (C2.19) 9. Using scripted lessons is a great way to provide teachers with ongoing professional development and to standardization the application of the ASL method. Nonetheless, the scripted format of lessons is limited as they don’t allow teachers to personalize the reading lessons. This aspect of the C2.17. The ASL approach incorporates the introduction of innovative and playful reading instruction activities such as the alphabet song, duck lips, stamping, name box, syllable squares, and check-in signs (thumbs-up, etc.). CI/CP Reading lesson observations (Zarmey bilingue/ Tillabery ; Tsararou Allah Sarki/Maradi) ; KII/FGDs with Readsters, NECS Education specialist, ASL Committee, AOR, Education Officer, USAID, teachers, pedagogical advisors ; KII DG Qualité et normalisation C2.18. The ASL approach defines a scope and sequence for reading learning content which may be seen as value-added. KIIs AOR, NECS Education Specialist, STS Reading Specialist ; Document review: CI and CP teachers’ guides USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 148 FINDINGS SOURCES CONCLUSIONS C2.19. The transition week to French is too abrupt in CP and doesn’t seem to be a gradual process. « Better start with images. The ASL transition should happen by associating images with words in French rather than all of a sudden an entire sentence in French” (CP Teacher, Tillabery) KII enseignante du CP Zarmey Bilingue ; Document review: 2nd grade teachers’ guide teacher’s guides has led to teachers not preparing their lessons in advance. (C2.20, C2.20, C2.22, C2.23, C2.24, C2.25, C2.26) 10. ASL’s use of national languages attracts children to reading. Nonetheless, major challenges related to matching teachers’ mastered language with the language they teach may be a risk for ASL’s effectiveness. (C2.28, C2.29) 11. The systematic nature of the ASL method is a fundamental element of good and effective reading instruction. Key aspects include the scope and sequence of content, the step-by-step process and regular student assessment at the end of each lesson. (C2.18, C2.21, C2.28). C2.20. According to members of the ASL Committee, the fully scripted lessons allow for lessons to be implemented in a standardized way. FGDs ASL Committee C2.21. The steps recommended by the ASL approach in the teacher’s guide promote highly structured content. “The teaching and learning approach is easy to implement. It’s the same number of steps. There are four in each lesson. It’s the same fun set-up…The directions are very specific.” (FGD, ASL Committee) KIIs ASL Committee ; IEP ; DDEP ; Pedagogical Advisors C2.22. Many stakeholders commented that the scripted lessons strengthen the skills of teachers that may not have the necessary qualifications. It’s in-service training. KIIs ASL Committee, CONCERN, Directeur de la qualité MEP, IEP, DDEP, DREP, Pedagogical Advisors C2.23. Several stakeholders also shared the idea that scripted lessons limit teachers’ autonomy and creativity. KIIs ASL Committee, CONCERN, Directeur de la qualité MEP, IEP, DDEP, DREP, Pedagogical Advisors C2.24. Other stakeholders, such as some school directors, pedagogical advisors and certain ASL Committee members think that scripted lessons make teachers’ work easier to the extent that teachers no longer have to prepare lesson plans. KIIs ASL Committee, Pedagogical Advisors, school directors C2.25. At the same time, other school directors, inspectors and central level officials think that teachers should always prepare plans for reading lessons even if they are scripted. “The guide makes teachers lazy and doesn’t push them to prepare their reading lessons” (Pedagogical Advisor, Tahoua). KIIs Directeur de la qualité MEP, IEP, Pedagogical Advisors, school directors C2.26. In first grade (CI), learning begins with the alphabet and the method stresses pre-reading activities. In second grade (CP), learning begins with a text and focuses on reading comprehension of a text, on understanding a text read aloud, and mini￾assessments at the end of each session. Document review: CI/CP teachers’ guides ; KII NECS Education Specialist 149 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS C2.27. The use of national languages in teaching and learning for 1st and 2nd grades (CI/CP) is seen as a very effective way to get kids interested in reading. KII/FGDs teachers, IEP, Pedagogical Advisors, DDEP, DREP, school directors, community members, ASL Committee, C2.28. Certain teachers work in a language that they have not mastered (for example: a Zarma teacher who delivers lessons in Hausa in the region of Tahoua). Reading class observations ; KII Directrice de la réforme MEP TEACHING AND LEARNING MATERIALS (TLMS) C2.29. Students are motivated by illustrations that depict their environment and their daily realities. KII/FGDs teachers, school directors 12. Despite accounting for students’ lived realities through illustrations and the relevant content of teaching and learning materials (teachers’ guides, student book), which was recognized by many stakeholders from both the central and regional level, findings identify a number of perceived inadequacies such as a lack of images in the 1st grade (CI) book and errors in both 1st grade (CI) and 2nd grade (CP) materials. (C2.31, C2.32, C2.33, C2.36). 13. The lack of a clear understanding between the users and designers of the teaching and learning materials has led to diverse points of view on the materials’ level of detail. Certain stakeholders subscribe to a reductionist view of the guide and see only its thickness rather than its useful and informative nature. This difference in opinion may negatively affect how the ASL guide is used in classrooms. (C2.34, C2.35) 14. Language variation and the need for further standardization of national languages remains a challenge. At the same time, teaching within a local language from the same guide and in different regions of the country may support the standardization process. (C2.36) C2.30. Some stakeholders find the content to be contextualized – the child sees in the book what s/he sees at home. “The illustrations are by a local artist.” (STS) KII/FGDs teachers, STS, school directors, IEP, ASL Committee C2.31. Some central level stakeholders as well as the STS team, some members of the ASL Committee and Plan USA records identify the lack of images in the CI student book as a major weakness. The contents of the first grade (CI) teachers’ guide is less attractive. KII/FGDs with ASL Committee, STS ; KII Plan International ; Document review (Synthesis of teacher comments) C2.32. In the majority of interviews with various stakeholders (Ministry, teachers, inspectors, directors, NECS staff) the large volume of the CP teachers guide (471 pages – Hausa version, for example) was mentioned. Almost all of the teachers interviewed said they found it difficult to use during lessons. KII/FGDs teachers, FGD NECS staff ; Document review (Synthesis of teacher comments) C2.33. In addition to the DG de la scolarisation, the majority of teachers shared that lessons are too long and that it’s difficult to finish them within the prescribed time period. KII/FGDs teachers, Directeur de la qualité MEP, NECS ; Document review (Synthesis of teacher comments) C2.34. The guides did not undergo a quality control process prior to distribution. Several errors were identified (typos, lack of alignment with the student book, some incoherence in the sequence of lessons, repetition of texts, etc.) KII/FGDs teachers, NECS Education Specialist ; Document review (Synthesis of teacher comments) C2.35. Several stakeholders brought up the need for language standardization. Teachers and Pedagogical advisors identified the issue with language variation between geographic areas. Certain words and sentences within the teacher’s guide and student book KII/FGDs with ASL Committee, enseignant, CP ; Document review USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 150 FINDINGS SOURCES CONCLUSIONS are not understood in the same way across different communities. At the same time, some stakeholders see the ASL scripted lessons as a way to standardize language. (Premières observations du terrain sur le guide de l’enseignant de CP) STUDENT ENTHUSIASM FOR READING C2.36. All of the regional level stakeholders (teachers, pedagogical advisors, inspectors, DDEP and DREP) attest that ASL provides children with «a love » for reading. 1st (CI) and 2nd (CP) grade students try to read, to decode what they find outside of school (public announcements, parents’ literacy materials). “Children have a love for reading. Often even when walking, if they see some bits of paper they pick them up to read them or they play around with reading billboards or other signs.” (FGCCP Maradi) KII/FGDs with school directors, teachers, Pedagogical Advisors, les IEP, les DDEP et les DREP 14. In general, various activities held with participants revealed that ASL has generated a passion for reading among students. Classroom observations showed how children in both 1st (CI) and 2nd (CP) grades were enthusiastic to participate in reading exercises. (C2.37, C2.38, C2.39) C2.37. The majority of stakeholders explain that the ASL reading instruction motivates students because they gain the ability to read and to understand what they are reading. “Students are able to understand what they read, for instance, they recognize that their relative used that word yesterday.” (Teacher, Tahoua) KII/FGDs with teachers, school directors, pedagogical advisors ; FGD ASL Committee ; KII DG Qualité et normalisation ; FGD NECS staff C2.38. Students read the writing on the board without the teacher asking them. Reading becomes more and more spontaneous for certain students. “I see progress each time I assess my students. The kids themselves want to be assessed.” (FGD, CI & CP Teachers) CI/CP Reading lesson observations ; Student exercise ; KII/FGD with teachers TEACHER TRAINING AND SUPPORT C2.39. Participants identified a number of the training’s strengths: the availability of teaching and learning materials (teachers), the existence of a reading expert and the training content (Pedagogical advisors). KII/FGDs with teachers, pedagogical advisors 15. In-service teacher training is one of the contributing factors to improving teacher performance in implementing ASL as well as the improvement in student skills. Nonetheless, findings revealed inadequacies such as when training was scheduled and its short duration. (C2.40, C2.41, C2.42, C2.43, C244) C2.40. Many teacher recognized that the ASL training helped them to improve their language skills. “Thanks to the training, I learned to understand Hausa better and it’s become easier for me to teach” according to a teacher in Zinder. KII/FGDs with teachers 151 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS C2.41. Teacher training used to last 3 days and has been extended to 5 days. Including schools directors in the trainings promotes improved teacher supervision and continuity in the case of teacher absence. Some teachers and pedagogical advisors find the number of days dedicated to the ASL teaching strategy to be insufficient. KII NECS Education Specialist; KII/FGD teachers, Pedagogical Advisors 16. Despite the obvious technical value added, NECS efforts to provide ongoing support to teachers has been seen by some as a constraining process for teachers. (C2.45, C2.46, C2.47, C2.48, C2.49) C2.42. Training follows a cascade training model (training of trainers) so that information can be shared by language and so as to involve various stakeholders. Education coordinators along with the ASL Committee and reading experts (Readsters/STS) train inspectors and pedagogical advisors. The inspectors and pedagogical advisors, with the support of the education coordinators and ASL Committee members, then train and support school directors and teachers. KII Former NECS COP C2.43. The majority of teachers interviewed said that project poorly chose when to hold trainings. They preferred that it take place during vacation time rather than at the beginning of the school year. KII/FGDs with teachers C2.44. A small number of teachers admitted that they felt overburdened by the ASL reading program which has very clear steps to follow from the beginning to the end. It’s possible that other teachers share this sentiment as well. KII/FGDs with teachers C2.45. Less than half of teachers expressed that the amount of support was rather too intense with a lot of people involved and at various levels. Teachers felt that the NECS project lacked confidence in their abilities. “The [NECS} are bothersome. Every week they ask us to send our weekly lesson plan. The project doesn’t trust us. They catch you for the smallest things ‘Last week, weren’t you on lesson 18? Why haven’t you finished it yet?’” KII/FGDs with teachers ; KII Directors C2.46. At least two schools, teachers complained of having to send in their assessment results to inspections instead of having the Pedagogical Advisor come to them as had been planned. KII/FGDs with teachers C2.47. Other teachers recognized the added-value of the support provided and the usefulness of follow-ups. One teacher even maintained that, “thanks to this support, I improved my understanding of the language and my desire to teach.” (Zinder) KIIS with teachers USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 152 FINDINGS SOURCES CONCLUSIONS C2.48. A small number of teachers clearly criticized NECS for not providing compensation and allowances for their tasks. KIIS with teachers EQ. 3. IN TERMS OF SCALABILITY, WHAT ARE THE COMPARATIVE ADVANTAGES OF THE ASL APPROACH AS COMPARED TO THE OTHER READING APPROACHES IN NIGER? C3.1. Throughout Niger several competing innovative approaches are being implemented to improve students’ reading skills. Among them are ASL (NECS), ELAN and APS (within bilingual schools, i.e., la Réforme). Document review; KIIs with NECS MEP representatives; Scalability Work Session 1. There are significant differences within the reading methodologies that are being implemented in Niger. While ASL uses an explicit methodology that guides teachers step by step throughout the teaching of a lesson, APS and ELAN are based on competencies but without clear instructions for teachers as to how to teach the lesson. (C3.1., C3.2., C3.3.) 2. APS, ELAN and ASL all target the same reading competencies: learning letters, recognizing syllables, and reading words and sentences, yet they invert the process. APS and ELAN begin from sentences and then move towards coding, while ASL begins with decoding and then moves towards reading sentences. In addition, ASL systematically practices vocabulary and comprehension development with students. (C.3.3., C.3.4., C.3.5, C.3.6.) 3. The MEP should consider integrating ASL methods within APS and ELAN as they all target the same learning competencies/skills. The Programme d’urgence in the Tahoua region provides a good example of how ASL might be integrated. ASL offers value added in providing teachers with an explicit teaching and learning method as well as in addressing vocabulary development and reading comprehension. (C.3.3., C.3.4., C.3.5, C.3.6., C.3.13) 4. Convincing evidence on the effectiveness of the various innovative reading programs is not yet C3.2. These innovative programs are using different approaches and methodologies. The bilingual schools use l’APS (Approche par la situation) based on students’ experiences. ELAN is implementing an accelerated approach to reading and writing within multilingual contexts; the approach is known as « L’apprentissage rapide de la lecture » [an Accelerated Reading Approach] even though it is being used within formal classes throughout the entire year. Concern is also implementing ELAN’s ARL method. NECS using l’ASL (apprentissage systématique de la lecture) [Systemmatic Reading Approach] that corresponds with an understanding of reading as a linguistic code activity. Document review; KIIS with NECS, STS, MEP Representatives ; KII Concern ; Scalability Work Session C3.3. ASL, APS and ELAN all target the same competencies. Even so, the ASL teacher follows a systematic standardized step-by-step teaching/learning process in order to meet these reading competencies whereas the APS and ELAN teachers target reading competencies without a specific process (the exact process varies from one teacher to the next). Document review; KIIS with NECS, STS, and MEP representatives C3.4. NECS’ ASL is based on teaching sound-letter correspondence and decoding in order to read words, sentences and finally reach comprehension. APS and ELAN, on the other hand, start by reading sentences and words in order to discover sounds, letters and syllables. Document review; KIIS with NECS, STS, and MEP representatives C3.5. All of the innovative reading approaches in Niger include the teaching of letters and words, and thus, decoding. Document review; KIIS with NECS, STS, and MEP representatives C3.6. NECS’ ASL approach incorporates more sessions focused specifically on vocabulary development, comprehension questions Document review; KIIS with NECS, STS 153 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS about a text read as well as a read-aloud text, and integrated student performance assessments. available. Some studies have attempted to understand the effects of various programs on student performance but the misalignment between the studies’ timeline and the implementation of various projects limits conclusions that may be drawn from these studies. (C3.7., C3.8., C.3.9.) 5. Although ASL has been well received within the regions visited, as well as at the departmental and school level, its implementation largely occurs outside of governmental structures, making it expensive and difficult for a potential scaling up unless the implementation model is revisited. (C3.10. C3.11., C3.12. ; C3.15) 6. In terms of scaling up ASL, even without solid evidence, the ASL methodology is seen as effective and relevant. Notwithstanding, several critical challenges need resolution, such as the Ministry’s decision, appropriation of funds, and incorporating good ASL practices within the Reform program. (C3.14, C3.15). C3.7. In 2015, the Ministry performed a comparative study on innovative reading programs and showed that 1st graders (CI) in NECS schools performed better at reading letters than 1st graders (CI) in the bilingual, ELAN and Concern schools. Document review: Rapport de l’évaluation des innovations pédagogiques conduites au Niger ; KIIs with la Direction de l’évaluation du MEP, NECS Education Specialist C3.8. Due to the cumulative delays in implementing ASL, the 2015 comparative report only took into account the task of 1st graders (CI) reading letters within the NECS schools. The study could address only 3 months of ASL implementation. Document review: Rapport de l’évaluation des innovations pédagogiques conduites au Niger ; KIIs with la Direction de l’évaluation du MEP C3.9. Convincing evidence on the effectiveness of the various innovative reading programs is not yet available. In the case of NECS, April 2018 data are the first that assess ASL in second grade (CP). For ELAN, the implementation zone is very small and does not allow for generalization. In regards to APS, a study focusing on grade 1 (CI) and grade 2 (CP) reading outcomes has not yet been conducted. Document review; KIIs with NECS et MEP representatives C3.10. APS and ELAN are institutionalized within the MEP and are implemented by MEP entities. Document review; KIIs with NECS et MEP representatives C3.11. ASL, implemented by NECS, operates largely outside of MEP, for instance, Pedagogical Advisors and Inspectors receive allowances for gas, per diem, hotel stays. KII/FGDs with NECS, MEP Representatives, Pedagogical Advisors, IEPs, DDEPs et DREPs C3.12. Preliminary results from the scalability checklist discussed during an ASL work session show that stakeholders find the ASL approach to be credible, well regarded and well adapted to Niger’s realities. KII/FGDs with NECS, MEP representatives, Pedagogical Advisors, IEPs, DDEPs and DREPs, teachers C3.13. The Tahoua Region’s DREP was inspired by his region’s experience with ASL and declared a « programme d’urgence » [an emergency program] in the region in order to improve students’ reading performance in 3rd thru 6th grades (CE to CM2) using several elements of the ASL method (ducklips, stamping, etc.). In the end, the Ministry adopted this program as a national remediation program, even though they left teachers the choice of methods used. KIIS with Tahoua DREP, Maradi DDEP, Maradi Pedagogical Advisor, Zinder DREP, the Sécretaire Générale of MEP, le DG de la qualité et la normalisation USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 154 FINDINGS SOURCES CONCLUSIONS C3.14. In regards to a potential scaling up of ASL, during the work session with representatives from the Directions de l’IFIC [Pre￾Service and In-Service Teaching Training] and Curricula, ASL Committee, Aide-et Action, NECS, USAID and the SG, participants agreed on a number of opportunities that scaling up ASL might offer to the education system: 1) teaching children to read within a short time period; 2) scripted guides that take into account teachers’ low qualifications, 3) ASL responds to the needs of the population and aligns with current policies (PSSEB). ASL Scalability Work Session C3.15. Nonetheless, this same group identified a number of concerns related to potentially scaling up ASL: S1) lack of solid evidence, 2) need to mobilize funds, especially for thick teachers’ guides, 3) issue of how to integrate ASL without replacing other existing reading initiatives, 4) need for open communication between Ministry officials, NECS management and the funder, 5) urgent need for the Ministry’s decision as to which good practices will be chosen in order to end speculation. ASL Scalability Work Session EQ. 4. WHAT TYPES OF CHANGES ARE VISIBLE IN NECS SCHOOLS AND COMMUNITIES? CHANGES VISIBLE AT SCHOOLS C4.1. The majority of primary education inspectors and school directors recognize improved functionality of school governments since the beginning of NECS. These participants underline an improved dynamic within the group due to capacity building sessions dedicated to trainings on how to develop action plans. According to them, the school governments now actively participate in school management through organization inter-grade sports competitions, school celebrations and in fundraising. « When the teacher is absent, the school government ministers take care of their classmates. They also are responsible for school cleanliness, for instance, the Hygiene Minister manages cleaning the courtyard and bathrooms. They also manage safety in taking care of conflicts that arise when students fight among themselves.” (School Director/Tahoua) KIIS with PFD/CGDES, NECS staff, IEPs, school directors and teachers, Pedagogical Advisors, FGD with School governments 1. In general, analysis of both field visit observations and interviews at the central and regional levels reveal positive changes related to NECS activities after 6 years of implementation within the majority of schools and communities. At the school level, various changes have been identified and they manifest themselves in various ways often dependent upon the category of stakeholder. Introducing ASL within traditional and bilingual schools has led to new experiences for NECS schools. In addition to more general changes at the school, children and even teachers have discovered a more adequate reading learning environment. (C4. 5 ; C4. 6 ; C4. 7 ; C4. 8 ; C4. 9) 2. In regards to capacity building for student governments, it’s undeniable that NECS has planted seeds of change. At the same time, the original concept of teachers providing regular guidance to school governments has not been C4.2. While in the field, it was noted, that not all NECS schools’ school governments function at the same level. The majority of the school governments visited are still somewhat reticent and are still getting used to their roles/responsibilities as they were newly created. Example: the Boudé school government began this school year (2017- Field observations, FGDs with school governments in all 4 regions of the sample 155 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS 2018) and children are not yet very clear about what is expected of them. systematically followed, and this has reduced their effectiveness. (C 4.1 ; C4.2 ; C4.3) 3. The NECS project has strengthened relationships between educational representatives of MEP at the regional level, notably located at the inspectorates, through the initiation of a technical assistance and monitoring system. This has led to a coming together of primary school support institutions and schools. At the same time, it bears mention that some teachers perceive these closer ties as increasing pressure on teachers. In addition, this bringing together of regional and school-level actors also entails strengthened solidarity among teachers through mini-CAPEDs (teacher discussion groups) dedicated to helping teachers support one another in implementing ASL. (C4. 4 ; C. 12) 4. Capacity strengthening activities and NECS awareness raising initiatives for community structures that support schooling have led to a new direction in school engagement. Communities are more interested in supporting schools with some of the most compelling illustrations being how organizations are able to better plan and implement their infrastructure preparations for the new school year. In addition, these organizations manage conflicts and lead pre￾enrollment and girls’/boys’ schooling campaigns. In addition, there seems to be greater women’s involvement within school-related community structures. (C4. 9 ; C4.10 ; C4.11 ; C4.16 ; C4.17 ; C4.18 ; C 4.23 ; C 4.24 ; C 4. 25 ; C 4. 26 ; C 4. 32 ; C 4. 33) 5. A new environment has developed in classrooms due to the signing and adoption of the Code of Good Conduct by teachers and school directors. At the same time, discussions identified that not all signatories were aware of the code’s contents, even if they publicly declared their commitment. Interviews conducted at the regional level also revealed that NECS sensitizations with organizations such as the School Mothers’ Associations (AME), and in return, the awareness C4.3. It was originally conceived that teachers would provide guidance to school governments. Yet, at the school level, the team observed that the majority of school governments are left to their own devices, even though only a small minority of teachers recognized this. “We always ask the ministers to stop and see us at our homes to speak with us about their student government roles but they never do…” (Teacher/Tillabery) Field observations, FGD with school governments, KIIs with teachers and School directors C4.4. There were perceptible positive changes in the relationship between inspectorates and schools with a greater frequency and a diversification in joint supervision, technical support and monitoring visits that bring together the departmental inspectors, pedagogical advisors, CGDES Focal points and SCOFI (both monthly and quarterly). “If NECS has succeeded in its activities, it’s due, in part, to their strong presence at the school level through our various missions.” (DDEP/Zinder) This new presence has reinforced the educational authorities within the schools, however, this is not perceived as positively by the majority of teachers who say they feel they are being watched on a daily basis. “During the same month, we may be visited by the Pedagogical Advisor who comes on a monitoring mission and who also has to do classroom observations or his mini-assessments, and then also have a visit from the Inspector or maybe even a visit from the DDEP.” (Teacher/Tahoua) KIIS with DREPs, DDEPs, IEPs, Pedagogical Advisors, SCOFIs, PF Départemental CGDES, IAENF, Directors and teachers C4.5. The majority of children encountered during field visits (classroom observations in 1st (CI) and 2nd (CP) grades) displayed a great motivation to read and to explore the written word when they see it in their immediate surroundings. In addition to observations, the near totality of teachers and school directors said that these important changes began with the implementation of ASL and public reading sessions (take place in the school courtyard). “Before they used to really love gym class activities, but now they prefer reading to that…Once you mention reading you see the joy on their faces…Some even want to keep reading until evening time… “ (Teacher/Tahoua) KIIS with school directors, teachers, NECS staff, IEPs, Pedagogical advisors. Document review: Project activity reports C4.6. The availability of teaching and learning materials within the classrooms represents a change in the school setting. In almost all of the schools visited, there was one book available for every two Reading class observations, KII with teachers and with school directors, FGD with NECS Staff, Education specialist USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 156 FINDINGS SOURCES CONCLUSIONS students as well as a teacher guide, which is also a new teaching assets now available in the classroom. raising the AMEs did within communities, has led to improvements in the number of girls’ attending schools during the enrollment period. (C4.13 ; C4.14 ; C4.15) 6. There were differences in levels of community commitment and participation depending upon the activity. Findings showed that there were differences between men and women’s participation in adult literacy activities. There were also differences noted between communities with NECS schools. These same differences between organizations also related to their understanding of their roles and responsibilities as well as limitations in writing up minutes from meetings. (C 4. 19 ; C 4. 20 ; C 4. 22 ; C 4. 25 ; C 4. 27) 7. With the establishment of village reading clubs as well as organized public reading sessions, NECS has been able to initiate new community involvement focusing on reading that favors the development of a literate environment for children. Indeed, there have been changes in the sorts of traditional activities held within communities. (C 4. 5 ; C 4. 20 ; C 4. 21 ; C 4. 28) 8. ASL’s use of local languages within schools alongside the beginning of adult literacy classes for parents within the community, has led to new relationships between parents and students. (C 4. 31) 9. Discussions with all education stakeholders at the central and regional levels, as well as the implementing partners unanimously recognized the relevance and the usefulness of the mentoring program. It’s clear that mentoring may be an excellent strategy for supporting students with socio-economic and school challenges. C4.7. Almost all of the classrooms visited have posters on the walls that display the alphabet in local languages as well as provide decorative illustrations. “The posters and the colors have really attracted students…I have to say that they didn’t know that before… “ (School Director, Tahoua) KII with teachers, school directors, Pedagogical Advisors, IEPs C4.8. With NECS and specifically, with ASL, there has been a “regaining of trust between teachers and students” as the Tahoua DREP explained. He shared that in many cases, the project has been able to help teachers to “communicate better with students.” Reading class observations, however, indicated that it’s only a minority of teachers that are able to achieve this level of trust with their students. Reading class observations, KIIs with Tahoua DREP, School directors and teachers, Pedagogical advisors C4.9. Teams observed the presence of school enclosures made of hedges put into place by community support organizations (generally by the School Management Committee – CGDES – or school governments) in almost all the schools visited. According to the majority of inspectors and School Management Committee Department level representatives, this change is due to NECS awareness raising activities with organizations and community members. “Before, our school was a thruway…All the animals and townspeople would walk through the school courtyard in order to go into the village or to the fields…Now, thanks to the efforts of these organizations, the school is more or less protected.” (Community member, Zinder) FGD with community members, IEPs, PFD/CGDES, NECS staff, School Management committees (CGDES) C4.10. At all of the schools visited, the evaluation team observed straw hut buildings constructed by community organizations for the purpose of providing remaining classrooms. At 5 of the 20 schools within the sample, these same organizations had also constructed mud brick lodgings for teachers. In addition, interviews with School Management Committees (CGDES) and community members indicated that NECS awareness-raising sessions had made organizations aware of the importance of these straw huts. School observations, FGD with Community members, PFD/CGDES, School Management committees (CGDES), Teachers, School directors C4.11. At both the local (IEP, PFD/CGDES, school directors) as well as at the central level, participants shared that they had gained awareness of the importance of advanced preparations for the opening of the school year (building make-shift classrooms, cleaning school courtyards, etc.). Focus groups with communities also demonstrated how community FGD with Community members, PFD/CGDES, School Management committees (CGDES), Teachers, School directors 157 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS members prepare to welcome teachers and students in improved conditions. “It’s the PTA that manages the cleanliness of the school. For instance, each year starting on September 15th, they hold a preparatory meeting to clean the courtyard for the opening of the school year.” (School director, Tahoua) Nonetheless, because of misunderstandings between the project and mentors, the activity did not deliver the long-term expected results. Several mentors leaving the program didn’t allow for continuous support to selected mentorees. (C 4. 29 ; C 4. 30) C4.12. NECS put into place Mini-Teacher Instruction Groups (CAPED) that occur at the school level that promote « assistance between teachers” with the guidance of inspectors (les IEPs) or Pedagogical Advisors about various aspects of teaching ASL lessons. KIIS with DDEPs, IEPs, Pedagogical Advisors, teachers and with school directors C4.13. Introducing the Code of Conduct at NECS Schools has led to changes in where students sit within the classroom. Classroom observations demonstrated that girls and boys sit side-by-side, a practice that did not happen prior to NECS, according to the majority of school directors and teachers interviewed. The Code of Conduct and its implementation led to schools replacing single-sex rows of students with mixed rows. “The Code of Conduct is an asset for both students and teachers as it has improved learning conditions.” (DDEP/Konni) Reading class observations, KIIs with NECS staff, Pedagogical Advisors, IEPs, School directors, teachers C4.14. Even though the Code of Conduct seems to favor “visible” changes at the school/classroom level at NECS schools, analysis showed that some teachers signed the document without having read it or without being entirely in agreement with its articles. Example: A teacher in a bilingual school in Tahoua was not able to respond to questions about the Code of Good Conduct without reading the document. Another teacher in Zinder said that he took only a minute to review the document, sign it and then hand it in to the School director… Reading class observations, KIIs with teachers and with school directors C4.15. Almost half of the student body at all 20 schools visited was made up of girls. Out of 10 students, 4 were girls, or 44% of the total sample of schools. This demonstrates a fairly good recruitment of female students at the beginning of the school year. At the same time, all of the IEPs, SCOFs, school directors and teachers agreed that there have been real positive changes in enrolling girls, due in large part to the awareness raising efforts of the School Mothers’ Associations (AME), PTAs and School Management Committees (CGDES) with the encouragement of NECS. School observations, KIIs with IEPs, SCOFIs, teachers and with school directors, School Mothers’ Associations (AME), PTAs, Community members USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 158 FINDINGS SOURCES CONCLUSIONS C4.16. In all of the schools visited, evaluation teams identified general action plans with School Directors that centralize the action plans of all of the other organizations. According to interviews with teachers and School Management Committees, this hadn’t been the case prior to the project. Inspectors also touched on this point during interviews and how they find it to be one of the most successful activities undertaken by NECS. School observations, KIIs with PFD/CGDES, School directors and teachers CHANGES VISIBLE WITHIN COMMUNITIES C4.17. Even though all of the stakeholders at the regional level (DREP), departmental level (DDEP) and a minority of IEPs present at the community-level recognize generally that one of the fundamental challenges to education in Niger is the lack of community engagement in schooling, field visits demonstrated that there is a reasonable level of community participation at the schools involved in the project. “Even if in our approach, we have always begun by working with the traditional authorities, we’ve noted and we recognize that up until now, communities are not yet prioritizing school. It’s the fundamental problem in Niger…Generally, there isn’t any support or technical supervision provided by communities for what happens at school…Through our joint monitoring missions and exchanges with communities or better yet, with parents at the NECS schools, we’ve noticed a difference…There, school is beginning to interest communities.” (DREP, Tahoua) KIIs with DREPs, DDEPs, IEPs, School directors and teachers, FGDs with community members C4.18. Almost all of the school directors and teachers at NECS schools, along with the Department level School Management Committee (CGDES) point people, recognize that there’s improved community engagement since the beginning of the project. They also note parents’ heightened interest for their children’s studies through ASL and accompanying adult literacy efforts. “Parents take more and more of an interest in school…They see that their children are more comfortable and that they read at home and that does them good. “ (IEP, Tahoua) KIIS with PFD/CGDES, IEPs, DDEPs, School directors and teachers C4.19. Field observations demonstrated, however, that community engagement varies in intensity from one place to another. For instance, teams remarked that turnout for focus groups was not the same between one school and another, something that participants noted as well. “In Souboudo it’s difficult to gather people for a meeting but in other villages, we’re really satisfied by the community’s level of engagement.” (DDEP/Gouré/Zinder). Another example is a school in Tillabery, where the school director mentioned that there re conflicts within the village to the extent that the School observations, KIIs with DREPs, DDEPs, PFD/CGDES, School directors and teachers, FGDs with PTAs, Mothers’ Associations (AME), School Management committees (CGDES), Community members, parents 159 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS community cannot come together within a single organization. According to him, the village functions as two factions and complicates large gatherings, like general assemblies or confirming action plans. He would have also preferred that NECS work directly with the community rather than passing through the school. C4.20. The existence of village library trunks within communities (since 2014) has changed the quality of life for children and parents and has led to an evolution towards a literate environment for children. At the same time, communities and adult literacy inspectors have different opinions about the frequency with which communities use these books and library trunks. KIIS with IAENFs, community members, School directors ; KII with Former NECS COP ; Document review (Project quarterly and annual reports) C4.21. Public reading sessions with parents, as well as the production of written materials, and also reading clubs and public announcements are also visible changes according to Adult literacy inspectors (IAENF), school directors, department inspectors, IEPs and community members. All of these stakeholders spoke highly about these changes as one of the most positive aspects of NECS’ reading activity. KIIS with IAENFs, DDEPs, IEPs, FGDs with community members, School directors C4.22. Participants observed differences between men and women’s frequencies of book borrowing. According to Adult literacy inspectors, women are more actively involved in literacy efforts than men, especially in participating in public presentations as part of reading clubs. Nonetheless, during the exercise conducted with grade 1 (CI) and grade 2 (CP) students, when asked about who reads with them at home, many identified that they read mainly with their older brothers and sisters who are in middle school. “There are more women who borrow books and read them with their children – they read them with their children at home and that’s very motivating…Things are really starting to change.” (IAENF/Tillabery) KIIS with IAENFs, FGD with Community members C4.23. According to school directors, School Management Committees (CGDES) and the Department-level Point of contact for School Management Committees (CGDES), participants observe a good level of financial engagement within communities in order to respond to school problems. According to these same sources, this is due, in part, to awareness raising activities that NECS conducted with school￾related community structures. Another aspect that was revealed, for example, is that parents make in-kind contributions at the end of the KIIS with IAENFs, PFD/CGDES, School directors, FGD with Community members ; Document review (Rapport de synthèse des évaluations des structures) USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 160 FINDINGS SOURCES CONCLUSIONS harvest coinciding with the beginning of the year in order to take care of eventual needs arising at the school. “It’s always the NECS schools that have a better financial engagement during our departmental School Management Committee meetings [Fédérations Communales des CGDES].” (Pedagogical Advisor, Maradi) C4.24. Parents demonstrate improved participation in school-related organizations’ general assembly meetings. According to a good number of School Management Committee (CGDES) presidents and Department-level Points of contact for School Management Committees (CGDES), this is a major change that speaks to the new dimension of community school management. KIIS with PFDs/CGDES, School directors, FGDs with Community members C4.25. Even if it’s a minor tendency within interviews, findings show that discussions with inspectors, school directors and community members speak to a remarkable change in women’s engagement in school￾related meetings. Moreover, some Points of contact for School Management Committees (CGDES) agreed with this observation making comparisons between NECS communities and those without NECS schools. "Since NECS came to our village, the community is engaged in school activities, especially women who have stepped forward in regards to all of the awareness raising activities…There are many more of them present at meetings.” (Teacher/Maradi) KIIs with PFDs/CGDES, School directors, FGDs with Community members C4.26. Participatory school management organizations (PTAs, Mother’s Associations (AME), School Management Committees (CGDES) are active and work closely with school directors and teachers. Interviews with these organizations as well as school directors identified that relationships between the school and community have improved since NECS began its activities. “At the Satchi school, the School Management Committee (CGDES), School Mothers’ Associations (AME) and the village chief continue to make other parents aware of their children’s attendance and they go to peoples’ houses to provide support.” (DDEP/Tahoua) KII with DDEP, School directors and teachers, FGD with PTAs, School Mothers’ Associations (AME), School Management committees (CGDES) C4.27. Even though the Points of contact for School Management Committees (CGDES) generally identified a strong level of community engagement as one of the remarkable changes at the community level, a good number of the DDEPs and IEPs continue to see certain weaknesses within school-related community organizations, due largely to misunderstandings of their roles and responsibilities as well as limitations in producing written minutes from meetings. KII with DDEP, PFD/CGDES, IEP, FGD with School Mothers’ Associations (AME), PTAs, School Management committees (CGDES), NECS M&E Specialist 161 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS “One of the weaknesses that I’ve noted during my supervision visits is that NECS should place more emphasis on training members of school support organizations, make them even more aware…As long as the structures themselves cannot write the agenda or produce minutes from their meetings, it will be difficult to really speak of any change.” (DDEP, Tahoua) C4.28. All of the departmental inspectors, the IEPS, school directors, pedagogical advisors, and teachers recognized that the “Public Reading” activity is a great example of change at the community level. Moreover, during FGDs with community member, they widely identified this activity as the one that made the greatest impact on them. “Because of the public reading sessions, many parents and former students have decided to return to school…And thanks to that, recruiting 1st graders (CI) is much easier since NECS began its awareness raising activities.” (FGD with community members, Maradi) KII with DDEPs, Pedagogical Advisors, PFD/CGDES, IEPs, school directors, FGDs with community members C4.29. IEPs, school directors, teachers and pedagogical advisors all identified at-risk students improving their school performance as the greatest change due to the mentoring program, which they noted benefitted from strong community commitment. Indeed, it’s community members who volunteer to be mentors in order to support these students with difficulties in school. “In my class, three students had gotten into the habit of not concentrating well during lessons but thanks to their mentors, we learned that these kids spent a lot of their time in houses in the village with TVs and that they returned home late at night. It was the mentor who spoke with these children’s parents to obtain this information and to insist with parents to no longer allow them to do so.” (Teacher, Zinder) KIIS with IEPs, Pedagogical Advisors, School directors and teachers, FGDs with community members and mentors ; FGD with Aide et Action ; Document review: NECS project activity reports ; NECS Mid-term evaluation report C4.30. Notwithstanding, the mentoring program slowed down in its activities because a number of mentors resigned when they weren’t paid for their tasks. Moreover, almost all of the DDEPs, IEPs and Pedagogical Advisors pleased so that the project would plan to identify other motivations or distinctions for mentors. For this reason, interviewees identified the mentoring program as the activity that succeeded the least. “Mentors abandoned the program along the way but it’s an activity that has a lot of promise if they’re able to find people who can be engaged from the beginning to the end.” (Pedagogical Advisor, Tahoua) KIIS with IEPs, Pedagogical Advisors, School directors et teachers, FGDs with community members and mentors, KII with l’Inspecteur de Niamey 1 (Ancien Directeur de la scolarisation) ; FGD with Aide et Action ; Document review (NECS project activity reports) ; NECS Mid-term evaluation report USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 162 FINDINGS SOURCES CONCLUSIONS During interviews at the central level, this same idea arose in almost all discussions. Inspectors and directors working within the MEP clearly identified their desire to see some sort of support provided to mentors either from the project or from the community. C4.31. During interviews, the majority of IEPs and pedagogical advisors recognized how ASL “serves as a link between the school and home because it takes place in local languages and is accessible to parents due to their literacy.” This change was made possible through ASL and adult literacy initiatives. During almost all interviews, from the central to the local level, interviewees continually identified as a major change how through NECS, schools became “the village’s school” rather than the “school in the village.” KIIS with IEPs, Pedagogical Advisors, School directors and teachers, FGDs with community members, NECS staff, Education Officer USAID/Niger C4.32. Through focus groups with community members, PTAs, School Mothers’ Associations (AME), School Management Committees (CGDES) as well as interviews with school directors, teachers and inspectors, the majority of participants argued that school support organizations play a central role in managing conflicts as well as addressing children’s school attendance. “The biggest change in the community that I’ve noticed is parents becoming conscience of the importance of respecting school time so that children succeed in their studies.” (Pedagogical Advisor, Zinder) FGDs with community members, avec PTAs, School Mothers’ Associations (AME), School Management committees (CGDES); KIIs with Inspectors, Pedagogical Advisors, School Directors and teachers C4.33. Through focus groups with Mother’s Associations (AME) and with Points of contact for School Management Committees (CGDES) [PFD/CGDES] as well as with school directors and teachers, stakeholders identified positive changes due to NECS awareness raising about girls’ and boys’ school enrollment. Almost all stakeholders informed the evaluation team of the important role that School Mothers’ Associations and Mother educators [les mères éducatrices] have played in organizing pre-enrollment campaigns, including door-to￾door canvasing to discuss the importance of school enrollment with fellow mothers. Example: In Sabar, before June 15th of the current academic year (2017-2018), the school director had already recruited 1st grade (CI) students for the following year. (School Director, Maradi) FGDs with community members, PTA, School Mothers’ Associations (AME), School Management committees (CGDES); KIIs with Inspectors, Pedagogical Advisors, School directors and teachers 163 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS EQ. 5. TO WHAT EXTENT DID THE PROJECT SUCCESSFULLY TARGET GIRLS? C5.1. Since 2013, NECS has organized more than ??? training and awareness raising sessions focusing on gender with various school and community actors. These sessions allowed teachers and community members, including PTAs, Mother’s Associations and School Management Committees (CGDES) to clarify their understanding of major gender concepts, particularly within the Nigerien context. In total, 854 teachers and supervisors, of whom 374 are women, were trained in different gender-related modules. In parallel, the SCOFI point people for the department-level inspectorates (DDEP) and community-level inspectorates (IEP) also received training about gender issues in order to be able to lead the same training with the Mother’s Associations (AME). “After trainings received the project, we are going to work with the School Mothers’ Associations through information and awareness raising sessions about gender…They will then represent us in the field…In turn, they will work with other mothers to raise consciousness about the important of schooling for girls.” (KII with SCOFI/Tahoua) Document review : 2013, 2014, 2015 Quarterly and annual reports ; KIIs with teachers, school directors, SCOFIs ; FGDs with School Mothers’ Associations (AME) 1. Despite changes to project objectives between NECS and NECS+, findings show that gender has always been at the heart of project activities within both communities and schools. A series of trainings, implemented using a cascade (Training of trainers) approach, were led with SCOFI representatives, Pedagogical advisors, school directors and teachers. Generally, stakeholders have gained increased familiarity with important gender and equity concepts. The last step in the training process involved School Mothers’ Associations (AME) which aims to increase their engagement and promoting girls’ schooling within their respective communities, even though the situation may often be discouraging. They have a pretty good grasp of their roles and responsibilities towards other women. While it’s one thing to convince an organized group of women who are already involved with a governmental structure it’s another to convince an entire community already steeped in its socio￾cultural and religious realities. (C5. 1 ; C5. 4 ; C5. 5 ; C5. 6 ; C5. 7 ; C5. 8 ; C5. 9 ; C5. 10) 2. IMAGINE’s heritage has clearly contributed to improving girls’ enrollment and girls’ attendance at “hard” NECS schools. Even if it’s not an exclusive factor of girls’ enrollment for 1st grade (CI), it’s undeniable that “Hard” schools offer an environment with better amenities for girls, especially with separate latrines, compared to what is available in the majority of “soft” schools. Based on observations and data analysis, a preliminary conclusion of this evaluation is that school infrastructures that are designed to account for girls’ particular needs may provide a way to attract and retain girls at school. (C5. 2) 3. Among the results that attest to NECS’ successful targeting of girls’ education, school governments play a particular role. Girls and boys work side￾by-side within this body which may help them to C5.2. Field observations and interviews within 8 “Hard” schools within the sample allowed for the observation that amenities (functioning latrines for girls) did, to a certain extent, contribute to improving girls’ enrollment during recruiting. Almost all of the school directors of “Hard” schools that completed the « school observation checklist » and held interviews with the evaluation team shared the same opinion. Analysis of data from school observation checklists, of which 9 were “hard” schools, show that 46% of students were girls, so above the average of 44% of girls found amongst all 20 schools. School observations ; KIIs with teachers, school directors, SCOFIs ; FGDs with School Mothers’ Associations (AME) C5.3. Gender issues are also visible in girls’ participation within school government management where girls occupy ministerial positions related to SCOFI or hygiene. Moreover, it was noted that almost all of the school governments reflect gender parity with 3 girls/3 boys. In the majority of schools visited, teachers, school directors, as well as pedagogical directors reported that it’s girls who ensure proper management of school activities more than boys. “In one school in the department of Magéria, it was reported that girls lead all the publically organized school activities and we see a real harmony between them and boys…” (Pedagogical Advisor, Zinder) KIIS with SCOFIs, teachers, school directors, Pedagogical Advisors, FGDs with School Government officials USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 164 FINDINGS SOURCES CONCLUSIONS “It’s been observed that there’s been an increase in girls’ school attendance thanks to house visits by the Girls’ Schooling Minister along with the school director…which is an achievement of the school government.” (KII with SCOFI/Tillabery) develop a sense of equity very early. The integration of girls within this governing body from a young age may help to change their way of seeing relationships with boys and, in the long term, may lead to a new generation with new mentalities. (C5. 3) 4. Beyond these more positive aspects, there are stakeholders that sound the alarm about risks and factors that weigh against gender objectives, specifically girls’ retention at school. Even if there is equity in girls’ access, the issue of retention and its multiple contributing factors remains. This situation provides pause for main stakeholders to ask questions about the real achievements of the project related to gender, especially when in certain communities, not all school-age girls systematically attend school. Few communities are able to openly recognize this black spot in gender sensitization outcomes within the community. (C5. 9 ; C5. 10 ; C5. 11 ; C5. 18) 5. In terms of noted findings, reservations about gender-related accomplishments are warranted. Within some communities, the fragility of gender outcomes is very perceptible. Based on the reservations of certain stakeholders, it’s easy to understand that parents who feel distracted from their beliefs and convictions might pull away and take their children out of school because it doesn’t really align with the practices found within community life. (C5. 21; C5. 22) 6. In general, the code of conduct that lays out gender-related rules at school has been well received by both central and regional stakeholders. Teachers and directors, who are the main focus of the code, expressed a great commitment to the central ideas within the document. Overall, exchanges with stakeholders at the central level (DFIC, DPSF) as well as the decentralized level (DDEPS, IEPs, SCOFIs, Pedagogical Advisors) show that the gender C5.4. According to teachers, school directors, SCOFI representatives and community members, most notably School Mothers’ Associations (AME) members, they are noticing a change in parents’ mentalities more and more. According to them, parents are allowing themselves to be convinced and to accept enrolling their girls in school thanks to NECS sensitizations. The majority of them agreed that certain parents are determined to see their children, and their daughters in particular, study beyond primary school. “Parents of children living in the countryside don’t want to enroll their girls in school but rather their boys. It’s with the advent of NECS that they understand the usefulness of girls attending schools…Men die and leave their families…A girl who has been to school will be much more useful.” (School director, Maradi) KIIS with IEPs, SCOFIs, teachers, school directors, Pedagogical Advisors, FGDs with community members, School Mothers’ Associations (AME) C5.5. Almost all of the SCOFI representatives and school directors mentioned that there’s a clear difference between parents who now have a heightened awareness and those who haven’t participated in the awareness-raising sessions. This testifies to the strong results of the series of awareness-raising sessions that NECS has put into place with the SCOFI representatives within the IEPs. One of the SCOFI representatives in Tahoua said that even if there’s a small extent to which the communities and organizations have mentioned it, the skits have helped raise awareness for girls’ schooling and have contributed to increasing consciousness amongst parents. “Children’s education, especially for young girls, is much improved here. We go door-to-door and we share messages during ceremonies in order to advise parents. This had had a great effect…Parents understand the usefulness of schooling better and the importance of keeping kids at school, or even the negative effects of early marriage.” (SCOFI/Tahoua) KIIS with IEPs, SCOFIs, teachers, school directors, Pedagogical Advisors, FGDs with community members, School Mothers’ Associations (AME) C5.6. SCOFI representatives have noted changes in School Mothers’ Associations at NECS schools through various monitoring/supervisory visit. They have observed that there is increased recognition of school’s importance for girls. In addition, the majority of school directors and School Management Committee (CGDES) secretaries general also brought up this point as indicative of the project’s major contribution to gender equity. They also spoke about seeing the “beginning of a mindset change” for parents. KIIS with IEPs, SCOFIs, teachers, school directors, CP, FGDs with School Management committees (CGDES), community members, School Mothers’ Associations (AME), NECS staff 165 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS “Today, the community is now conscious that if it needs a female health agent, like midwives, for example, that it should allow girls to pursue their studies which will benefit her, her family and her community.” (FGD with School Management Committee (CGDES), Tahoua. “There has really been some progression…During our first visit, we had to go looking for people in the village so that they come meet with us. But, during our second visit, people were waiting for us at the school.” (KII with SCOFI representative, Tahoua) equity tools that NECS has developed have been well appropriated. An instrument like the code of conduct has revolutionized the school environment within NECS schools and led teachers and directors to agree to new inclinations to improve class practices in order to better integrate girls. Findings show that, to a good extent, teachers implement many of these practices. (C5. 12 ; C5. 13 ; C5. 14; C5. 15 ; C5. 16 ; C5. 17 ; C5. 19 ; C5. 20) 7. A review of quarterly and annual project reports and discussions with central and local level stakeholders reveal that many project activities have attempted to reach a large number of girls. This is the case with the mentoring program and the awards ceremony for the best female student. The project also developed many tools that aim to help better integrate girls into teachers’ classroom teaching and learning practices. These two activities, if well-structured and well￾supported, may allow the project to reach a good number of at-risk and vulnerable girls as well as provide a good source of motivation for them. (C5. 23) C5.7. According to School Mothers’ Associations (AME), School Management Committees (CGDES) and school directors as well as SCOFI representatives, the effects of the community awareness-raising sessions have garnered a new perspective of women, a way of seeing women, among the population. “Now women are more involved in decision-making. Prejudices the community had towards women have decreased due to awareness-raising by community organizations.” (School Director, Tahoua) KIIS with IEPs, SCOFIs, teachers, school directors, Pedagogical Advisors, FGDs with School Management committees (CGDES), community members, School Mothers’ Association (AME) C5.8. School directors and teachers identified a strong level of engagement by mother educators and argued that it’s due to the leadership of School Mothers’ Association (AME) presidents that there has been a good level of girls’ enrollment at the beginning of the pre￾enrollment period. According to the majority of them, as well as SCOFI representatives and the IEPs, and even some of the DDEPs, who also shared this view, the School Mothers’ Associations have used multiple channels to raise awareness among other mothers (religious and family ceremonies, door-to-door canvasing, etc.) “Ever since the gender sensitization, we observe that women recognize the importance of educating girls…They have even stopped making them work during the school year…There really has been a change since girls and boys do the same activities at home as well as at school.” (FGD with teachers, Maradi) “It’s too bad that the former School Mothers’ Association president isn’t here. She left her position at the beginning of the year because she now lives in Niamey…She was very active and effective in mobilizing community members for discussion sessions.” (KII, School director, Tillabery) KIIS with IEPs, SCOFI, school directors, CP, FGDs with School Management committees (CGDES), community members, School Mothers’ Associations (AME), PTA C5.9. However, during focus groups, even if it was a small proportion of School Mothers’ Associations that admitted it, certain mother educators recognized that they often faced hesitation within the community as well as cases of school-leaving that did not make their pre-enrollment work any easier. “There are challenges to first grade (CI) student recruitment, especially among nomadic parents who continue to resist sending their children to school…we have FGDs with School Management committees (CGDES), community members, School Mothers’ Associations (AME), PTA USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 166 FINDINGS SOURCES CONCLUSIONS tried multiple times to recruit 1st graders (CI) among the Peuhls and Tuaregs but without much success.” (FGD, PTA & School Mothers’ Association, Tahoua) C5.10. Field observations showed that there is a fairly decent amount of girls attending schools. Based on school enrollment statistics for the current school year provided by school directors, of 10 students, 4 are girls, or about 44% of the total school population within the schools sampled. School observations ; KIIs with School directors C5.11. Overall, interviews with the majority of pedagogical advisors, school directors, SCOFI representatives and community members, identify that girls’ access has reached its apex. Nonetheless, discontinuities are evident in terms of school completion and girls’ retention at school. At the same time, while statistics show satisfactory results in terms of girls’ enrollment, girls’ attendance and retention remain acute challenges. According to stakeholders, poverty, exodus, needs for tending the fields, and early marriage are the main factors that determine if a girl is going to continue attending school. Even if all of the focus groups with community members identified an increased consciousness about the importance of girls’ education, inspectors and directors expressed a different opinion. “In the village, even if it’s a bilingual school we haven’t really noticed real changes in girls’ enrollment despite all of the NECS awareness-raising activities…There are lot of school-age girls still in the village…You’ll see a parent with six girls who chooses to send two or three and keeps the others at home for domestic work…There are a lot of girls who stay at home…In addition, there is also a lot of relocation that takes place even in the middle of the school year… “ (CP Teacher, Tillabery) KIIs with DDEPs, IEPs, SCOFIs, School directors, Pedagogical Advisors, teachers C5.12. Promoting gender equity at school and within communities through gender-sensitive teaching and learning techniques has been a really important aspect of the NECS project. Through project reports and interviews with project staff, it is clear that the code of conduct is one of the key gender strategies employed by NECS. Signed by nearly 98% of stakeholders (specifically teachers and school directors), the content of the code of conduct (5 chapters and 26 articles) requires a major change in the gender situation at school. Through this document, teachers are called to treat girls and boys in the same way within the classroom. FGD with NECS staff, KII with Former NECS COP; School observations ; KIIs with School directors and teachers, Document review (Project quarterly and annual reports) 167 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS ”The code of conduct hasn’t gone beyond the school. It has brought about change within teachers’ activities. Before, they thought that it was simply a rule but now they know that it helps them to respect gender equity between girls and boys throughout school activities.” (Pedagogical Advisor, Maradi) C5.13. An interview with the NECS Point person at the Direction de la promotion de la scolarisation de la fille revealed that they have appropriated this instrument as an institution working on gender issues within the Nigerien school system. In the field, with the exception of 2 SCOFI representatives, all of the others expressed working closely with NECS school communities in order to ensure progress towards gender equity. Their perspective identifies an evolution within communities in regards to gender equity. “In NECS villages, thanks to awareness-raising, the issue of gender equity has been integrated…Social constructs and habits have been set aside, and today, it’s expected that boys as well as girls do the same chores. » (PF/SCOFI-Zinder) FGD with NECS staff ; KII with Former NECS COP; School observations ; KII with the Point Focal NECS of the DPSF ; KIIs with School directors and teachers ; Document review: Project quarterly and annual reports C5.14. Analysis of interviews with pedagogical advisors, whose role is to support gender equity within the classroom, indicated that teachers are implementing a good number of the code’s provisions within the classroom. Even if they recognized, to a certain extent, some hesitancy at the beginning, they unanimously stated that they have observed a good level of implementation of the code’s principles. These same observations arose in conversations with the IEPs, the DDEPs and school directors. “In the NECS code of conduct, there’s an article that I can’t recall the number of, but it addresses equality between men and women. For me, this attacks religious beliefs and that’s not appropriate…But generally it’s good because it promotes good behavior.” (School director, Tahoua) FGD with NECS staff ; KII with Former NECS COP, KII with Pedagogical Advisors ; School observations ; KIIs with School directors teachers ; Document review (Project quarterly and annual reports ; Note synthétique sur l’équité et le genre/Mai 2016) C5.15. According to the DDEPs, IEP, as well as the Pedagogical Advisors, teachers and school directors adopting the code of conduct has led to greater respect for gender equity within all of school activities within NECS schools. According to them, thanks to their supervisory visits, their oversight and support, they have been able to see changes brought in by this code. Both girls and boys now perform the same chores such as “erasing the board,” “fetching water,” “cleaning the classrooms,” and “raising the flag.” FGD with le NECS staff, KII with Former NECS COP; School observations ; KIIs with School directors and teachers, Document review: Project quarterly and annual reports) C5.16. Classroom observations of reading classes in 1st and 2nd grade (CI/CP) showed that during the 19 observations conducted, teachers demonstrated some of the techniques established within the teachers’ guide. For example, within 13 of the 19 classes observed, students no School observations (CI/CP classes) ; FGD with community members ; KII with Pedagogical USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 168 FINDINGS SOURCES CONCLUSIONS longer sat according to gender. Within classes, there was no longer a row for girls and a row for boys. Students sat side-by-side on the ground or in their school benches. Notwithstanding, discussions with various stakeholders demonstrated that this NECS classroom set-up represents a risk for girls leaving school. Stakeholders explained their hypothesis that because some parents weren’t in agreement with this set-up they might take their girls out of school. Advisors, teachers and with school directors. C5.17. During these same observations of 1st/2nd graders (CI/CP), teams observed a few teachers using the “name box” to randomly call upon students, a practice that NECS recommended. At the same time, teams observed that even without using this technique, teachers directed questions to girls as well as boys in a more or less equitable way. Pedagogical advisors also unanimously confirmed during interviews that teachers treated girls and boys the same during their assessments and classroom observations. It’s worth mentioning that based on the team’s observations, the name box is not always easy to implement in large classes or when there are great differences between the number of boys and girls in the class. Observations of all 20 schools in the sample (CI/CP classes); KIIs with teachers and with school directors; KIIs with Pedagogical Advisors C5.18. The « Gender sensitivity rubric » «[« Grille sensible au genre »], that was piloted in Madoua and whose scaling up to other NECS schools was planned for the NECS+ extension (2016-2018), was not mentioned by any department-level education officials during interviews. While quarterly and annual reports speak about advocating for its adoption within the MEP in collaboration with the Pre-Service and In-Service Teacher Training Directorate (DFIC), in the field, no stakeholder mentioned this tool. As a reminder, this rubric is a monitoring tool that should be used by Pedagogical Advisors to verify that teachers are integrating gender equity practices in the classroom. KII with Former NECS COP, FGD with NECS Staff, KII with NECS Education Specialist; KII with Pedagogical Advisors 169 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV FINDINGS SOURCES CONCLUSIONS C5.19. Nevertheless, classroom observations showed that during a reading lesson within visited that : - In regards to teachers directing questions at students, 56 % of the time, questions were directed at girls, and boys 44% of the time; - In regards to children responding to open questions, the same trend was observed (56% of the time, it was girls, and 44% of the time, boys) ; - For students going to the blackboard, girls went 58% of instances observed, and boys 32% of the time. - When teachers reviewed students’ work, observation data shows that teachers looked at girls’ work 58% of the time and boys 32% of the time. Reading lesson observations – in 19 of the schools (CI/CP classes) ; KIIs with CI/CP teachers C5.20. On a different note, the classroom observation tool sought to understand if more boys than girls held the role of “class leader” [« responsable de classe »]. Data analysis shows that within the 20 schools visited, all of the school leaders were boys. Not one girl held this position within the classrooms visited. On the other hand, in terms of the “assistant class leader” [« adjoint au responsable de classe »], the majority (83%) were girls. Reading lesson observations – in 19 of the schools (CI/CP classes) ; KII with CI/CP teachers C5.21. During interviews, even though it was to a small extent, certain communities in Tahoua or even Tillabery, recognized several causes linked to a low rate of girls’ enrollment that persist. For example, certain communities admitted that it was due to « some parents’ way of thinking, early marriage, prejudices against school, girls’ contributions to housework or small business or exodus [largely rural but could also be emigration, or more generally, families moving], which is a very common phenomenon that causes parents to leave with their children.” A good number of DDEPs, DREPs, IEPs, Pedagogical Advisors, school directors, teachers and some of the SCOFI representatives agreed with these points. Stakeholders generally recognized the set of limitations that hamper reaching the objective linked to girls’ access. FGDs with School Management Committees (CGDES), community members, School Mothers’ Associations (AME), PTA ; KII with DREPs, DDEPs, IEPs, Pedagogical Advisors, Teachers and with school directors C5.22. In regards to girls’ retention, interviews showed that there is still a lot that needs to be done. Overall, all of the DDEPs and SCOFIs spoke of major challenges that arise due to multiple reasons. According to them, sometimes, due to poverty, parents pull their girls out of school due to a combination of factors generally linked to the context and existing realities. They indicated that these realities differ from one area to another. FGDs with School Management Committees (CGDES), community members, School Mothers’ Associations (AME), PTA ; KII with DREPs, DDEPs, IEPs, Pedagogical Advisors, Teachers and with school directors USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 170 FINDINGS SOURCES CONCLUSIONS “In Zarma and Hausa societies, early marriage is the key factor that prevents girls’ access and retention at school…While there have been improvements with NECS, there is still a lot to do in this direction…These are still critical considerations for girls that need to be addressed within these societies.” (IEP, Tahoua) C5.23. The mentoring program and the ceremony for the best female student also represent activities that seem to have highly contributed to reaching targeted girls. Since 2013, project activity reports have included numbers that speak to an evolution in student enrollment in these activities. The majority of interviews also revealed that stakeholders such as the IEPs, Pedagogical advisors, SCOFI representatives, school directors, teachers and community organizations, also see these activities as having been instrumental in addressing girls’ access. “Mentoring helped an at-risk girl in my class. I would always see that she didn’t try very hard in class…It’s when she became a mentoree, with the help of her mentor, that we were better able to understand her constraints…In the end, she spent better days at school.” (Teacher, Tahoua) Document review (2013, 2014, 2015 Project Activity Reports) ; KIIs with DDEPs, IEPs, Pedagogical Advisors, School directors and teachers, FGDs with community members, PTAs, School Mothers’ Associations (AME), School management committee (CGDES) 171 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV ANNEX XVI: MATRICE DES RECOMMANDATIONS ET ACTIONS A MENER N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 QUESTION D’EVALUATION N°1 : DANS QUELLE MESURE NECS A-T-ELLE ATTEINT SES OBJECTIFS PAR RAPPORT A SES CIBLES 1 Manque de constance dans les ressources humaines dédiées au S&E du projet contribuant à créer des failles dans l’efficacité du système suivi-évaluation et dans son application Tenir compte du profil requis dans le recrutement du personnel Le projet a connu 4 spécialistes M&E en 6 ans de mise en œuvre PlN/COP/NECS USAID/ Niger (Spécialiste en éducation) SG/MEP Continue X Créer une motivation du personnel afin d’assurer le maintien et la stabilité des ressources humaines Associer le MEP au processus de recrutement des staff 2 Faiblesses dans l’appui apporté par le dispositif d’appui technique en S&E disponible de la part de Plan International/Plan Niger et SRO/USAID Renforcer le dispositif d’appui technique au niveau de Plan International pour accompagner le personnel de S&E du projet NECS Plan US/PIN Responsable MERL de PIN Courant 2018 X 3 Persistance des contraintes relatives à la fréquence de collecte des données de suivi pour les indicateurs n°7, 8, 9 et 10 désignés dans le Plan de S&E comme semi-annuels. Revoir la fréquence de collecte des données et adapter les indicateurs concernés comme annuels et non semi-annuels Il s’agit de 2 collecte en début et en fin d’année scolaire Plan US COP NECS Courant 2018 X 4 Faiblesses notées au niveau du système de S&E dont la structuration ne permet pas d’assurer une continuité entre les M&E spécialistes du projet Améliorer la dynamique de centralisation des informations en agissant sur certaines parties du dispositif de suivi et évaluation (consolidation des données de suivi, le suivi interne) Renforcer les compétences en suivi du staff local (les CP, les PFD CGDES, les SCOFI, les inspecteurs Les cadres de concertation et d’échanges ont permis de ne pas trop sentir ce gap dans le suivi ; Problème de continuité qui se pose pour avoir une mémoire institutionnelle du projet ; le projet avait des difficultés pour trouver des staff M&E compétents (4 M&E depuis 2012) ; le système est bien conçu et les outils sont PIN/COP NECS/ Responsable S&E NECS COP NECS Octobre 2018 X USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 172 N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 IEP/IAENF) avec les Coordonnateurs Education Ajouter un assistant S&E pour renforcer le staff au niveau de NECS Harmoniser le système de reporting au niveau des acteurs locaux disponibles pour permettre à tout M&E de mobiliser les informations ; Trop de pression pour les staff M&E recrutés 5 Le Spécialiste en suivi￾évaluation NECS identifie le manque de documentation régulière pour le suivi au niveau des communautés et par conséquent l’atteinte de certaines cibles Sensibiliser et outiller (fiches) la communauté dans la documentation des activités basées sur les évidences pour améliorer la performance du projet Insuffisance dans la documentation Spécialiste S&E/NECS COP/NECS Octobre 2018 X 6 Absence de formations pour le personnel de NECS sur l’EGRA et ses pré-requis en matière de collecte de données valides Pas de recommandations Voir le constat 10 Rejection du constat par l’ensemble des acteurs dans le petit groupe 7 Absence d’une base de données pouvant permettre une centralisation des informations de toutes les composantes du projet Avoir une base de données centrale qui sera manipulable du staff et qui va regrouper l’ensemble des informations pour tous les autres indicateurs du projet Il existe une base de données avec NECS+ (indicateurs contractuels) mais qui est juste focus en tant que maquette Excel sur les indicateurs de l’extension PLAN USA Spécialiste S&E/NECS Octobre 2018 X 8 Le personnel de S&E de NECS n’est pas assez soutenu et ne bénéficie pas de formations ou encore d’appui technique ou coaching Renforcer le dispositif d’appui technique au niveau de Plan International pour accompagner le personnel de S&E du projet NECS Plan US/PIN Responsable MERL de PIN Courant 2018 X 9 Faiblesses au niveau du dispositif de suivi avec une absence d’expertise en S&E présents sur le terrain (effets négatifs sur la qualité des Renforcer les compétences en suivi du staff local (les CP, les PFD CGDES, les SCOFI, les inspecteurs IEP/IAENF) avec les Coordonnateurs Education PIN/COP NECS/ Responsable S&E NECS COP NECS Octobre 2018 X 173 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 informations et sur le niveau détails des données collectées) Harmoniser le système de reporting au niveau des acteurs locaux 10 Insuffisance dans les capacités à gérer un EGRA avec des données valides et d’assurer une harmonisation des pratiques de collecte (Plan US, STS, NECS) Avoir un spécialiste au niveau de Plan US qui pourrait avoir un regard sur les processus et produits EGRA Elaborer de nouveaux outils pour les évaluations EGRA de 2019 Continuer à intégrer certaines variables dans l’échantillonnage des écoles (niveau de progression dans les leçons ASL et la langue d’apprentissage) PLAN US/STS COP NECS Octobre 2018 X 11 5 indicateurs sur les 10 de NECS + n’ont pas atteint du fait de la gestion du projet (aspects composites avec plusieurs critères à renseigner pour un seul indicateur, Faire des choix stratégiques en concertation avec le MEP Débat – est-ce que c’est la gestion ? Plutôt un problème stratégique du choix (de sous￾contractant). COP/NECS DGPQN/MEP Continue X 12 Absence de données de suivi pour alimenter certains indicateurs qui relèvent des activités communautaires (7, 9 et 10) du fait d’un faible niveau d’alphabétisation des membres de la communauté Finaliser la collecte et le traitement des données des indicateurs 7,9 et 10 Les données non￾disponibles au moment de l’évaluation seront disponibles et traitées en fin juin 2018 Spécialiste en S&E/NECS COP/NECS 15 Juillet 2018 X Renforcer le niveau d’alphabétisation des membres des structures participatives DPAFA/MEP DGAENF Janvier 2019 X 13 Inadéquation entre les cibles des indicateurs et la faisabilité en termes de performances (sur-estimation/Indicateurn°8 sur l’accès des filles à l’école ou sous-estimation/Indicateurs n°5 et n°6 sur la distribution des manuels). Adapter les cibles des indicateurs au contexte et aux réalités locales Surproduction de manuels due à plusieurs phases de productions (2 livres par enfant) et changements d’approches (ARL vers ASL) Reprise de la production pour les livres abimés (2016-2017) Responsable S&E/NECS DES/MEP Continue X Définir des éléments d’explication/justification par rapport à la non atteinte de ces indicateurs afin d’orienter la décision Continue X USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 174 N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 QUESTION D’EVALUATION N° 2. COMMENT L'APPROCHE « ASL » DE NECS A-T-ELLE AMELIORE LES RESULTATS EN MATIERE DE LECTURE DANS LES ECOLES CIBLEES PAR L’ACTIVITE ? 1 Moments d’absence de sous￾contractant pour la lecture qui a ralenti le projet dans l’atteinte des objectifs de lecture (départ de VIE et recrutement de Readsters/janvier au novembre 2014 ; départ de Readsters et recrutement de STS/mai 2016 au novembre 2016) Stabiliser les structures/cabinets d’appui pour éviter le ralentissement dans la mise en œuvre du projet et assurer la continuité en agissant sur les clauses de prestations Plan US COP NECS /PIN Continue X 2 Données insuffisantes sur l’impact de l’ASL au CP dû au mauvais alignement entre les évaluations de performance des élèves et la mise en œuvre de l’ASL Accélérer l’analyse et la mise à disposition des résultats de l’évaluation EGRA pour apprécier de l’impact de l’ASL Plan US COP NECS Continue X Envisager un deuxième EGRA à la prochaine année scolaire (2018- 2019) 3 Absence de spécialiste en lecture à NECS, Plan Niger, Plan USA, USAID/Niger n’a pas permis un suivi/soutien adéquat aux partenaires de mise en œuvre ou un contrôle de qualité de la méthodologie et des support pédagogiques développés. Recruter d’un Spécialiste en lecture par Plan/NECS Plan US/ USAID/Washing ton COP NECS USAID/ Niger (Spécialiste en Education) USAID/ Washington (Equipe lecture) Faire recourt aux Spécialistes en Continue X lecture disponibles au niveau de l’USAID/Washington pour accompagner le partenaire de mise en œuvre et aussi faire le suivi contrôle de qualité 4 Faible niveau de certains enseignants identifié comme défi majeur pour l’obtention des bons résultats au niveau des élèves. Définir et respecter les profils requis dans le recrutement des enseignants et poursuivre le renforcement de leurs capacités DRH/MEP Direction de l’Evaluation Scolaire et du Suivi Continue X 175 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 5 La semaine de convergence vers le français est trop « brusque » au CP et ne semble pas avoir suivi une démarche progressive. Revoir la semaine de convergence dans sa conception Besoin de clarifier si le constat concerne la conception ou la mise en œuvre La recommandation concerne la mise en œuvre Organiser des CAPED et mini COP/NECS DCPLN CAPED et encourager l’encadrement de proximité pour davantage aider les enseignants à mieux assimiler l’approche. 6 Incompréhension et absence de clarification dans la communication autour des leçons scriptées qui pour certains servent au renforcement des capacités des enseignants qui n’ont pas toujours le niveau requis (formation continue); alors que pour d’autres les leçons scriptées limitent l’autonomie et la créativité des enseignants (absence de préparation des leçons). Organiser des foras de réflexion avec les différents acteurs Le processus d’adaptation est en cours pour la réduction du guide qui n’est plus exclusivement basée sur les leçons scriptees mais aussi sur des leçons modèles qui donnera plus de marge, plus de créativité à l’enseignant COP/NECS DFIC/DCPLN/M EP Au courant de 2018 X 7 Enseignants affectés dans des écoles NECS travaillant dans une langue qu’ils ne maitrisent pas (ex. enseignant Zarma dispensant des leçons en Haussa) Redéployer les enseignants sur la base de la langue d’enseignement et de la langue parlée Potentiel transfert de compétences au niveau des communes IEPs/MEP DCPLN/MEP Rentrée scolaire 2018/ 2019 X 8 Identification d’un manque d’images dans le livret du CI qui est considéré comme n’ayant pas un contenu attrayant. Un constat qui rejoint un débat mondial sur l’inclusion des images dans les livres de lecture pour les débutants. Illustrer davantage les livrets du CI avec des couleurs (1 à 2) Considérer le cahier des charges pédagogiques ainsi que les coûts que cela impliquent ; c’est une décision à prendre avec le MEP ; selon la disponibilité des ressources COP/NECS DCPLN/MEP Rentrée scolaire 2018/ 2019 X USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 176 N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 9 Guide du maitre du CP déclaré comme étant trop volumineux (471 pages pour le Hausa, par exemple). Quasiment tous les enseignants interrogés ont dit d’avoir du mal à l’utiliser durant les leçons. Aérer les illustrations et la mise en forme pour que cela soit plus lisible Processus de réduction et mise en page est en cours de finalisation Spécialiste en Education/NECS COP/NECS En cours X 10 Leçons ASL appréciées comme étant trop longues et très difficile à terminer dans les délais prescrits. Apprécier et éventuellement alléger les étapes qui ne sont pas importantes COP/NECS DCPLN/MEP Continue X Mettre des séquences des leçons dans le guide selon le temps imparti 11 Absence d’une revue du guide par un éditeur pour un processus de contrôle de la qualité avant la distribution. Plusieurs erreurs ont été recensées (de frappe, manque d’harmonisation avec le livret de l’élève, quelques incohérences dans la progression des leçons, répétition de passage, etc.) Consacrer plus de temps dans la relecture et la correction des documents dans le cadre du processus d’élaboration Dans le processus de réduction du volume des guides ces préoccupations ont été prise en compte COP/NECS DCPLN/MEP Continue X Contractualiser avec un éditeur pour assurer le suivi de la qualité (de la relecture à la mise en page) 12 Absence d’une standardisation des langues causant un problème de variation d’une zone géographique à une autre. Certains mots et phrases dans les guides et les livrets des élèves ne sont pas compris de la même manière dans toutes les communautés. Harmoniser les terminologies en tolérant les variances linguistiques sur le terrain et sensibiliser les acteurs lors des formations C’est très difficile de trouver le juste milieu avec les problèmes de dialecte ; Lexiques existant au niveau des écoles NECS COP/NECS DCPLN/MEP Continue X Intégrer cette dimension linguistique au niveau de la formation des enseignants Utiliser aussi les lexiques thématiques et les dictionnaires bilingues qui existent au Niger 177 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 13 Durée de la formation/initiation à l’enseignement ASL jugée comme insuffisante par les enseignants et directeurs d’écoles (5 jours pour la langue, 3 jours pour la méthodologie) pour les nouveaux ; Pour les recyclages, la formation dure 3 jours. Prévoir plus de temps pour les formations, par exemple 7 jours pour la langue et 5 jours pour l’approche ASL COP/NECS DCPLN/MEP Continue X Mettre en place un comité technique pour la détermination de la durée adéquate de la formation 14 Le moment des formations est jugé mal-choisie par le projet (début d’année scolaire généralement) Organiser les formations pendant les vacances autant que faire se peut Une planification antérieure avait prévu une formation en Aout ou Septembre mais des perturbations n’ont pas permis d’exécuter les dispositions prises COP/NECS DCPLN/MEP Continue X Rappeler au MEP la question des affectations afin de toucher 15 Sentiment de surcharge de travail et une intensité du suivi mené par les autorités éducatives de la part des enseignants (hebdomadaire, mensuel, trimestriel) Maintenir le dispositif et la fréquence de suivi et dans la mesure du possible et sensibiliser les enseignants sur les objectifs de la démarche IEPs, CP et les Coordonnateurs régionaux de NECS COP/NECS Continue X Améliorer le contenu des missions de suivi menées par les conseillers pédagogiques EQ. 3. DANS UNE PERSPECTIVE DE MISE A L’ECHELLE, QUELS SONT LES AVANTAGES COMPARATIFS DE L'APPROCHE ASL PAR RAPPORT AUX AUTRES APPROCHES DE LECTURE AU NIGER ? 1 Plusieurs innovations sont mises en œuvre à travers le Niger pour améliorer le niveau des élèves en lecture. Parmi ces efforts : l’ASL avec NECS, ELAN, et l’APS dans les écoles bilingues dites de la réforme. Evaluer l’ensemble des innovations en cours pour aider le MEP en vue d’une prise de décision DSE/MEP PIN Janvier 2019 X USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 178 N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 2 Des données probantes sur l’efficacité des innovations en lecture ne sont pas encore disponibles. Pour NECS, les données collectées en avril 2018 sont les premières qui évaluent l’ASL au CP. Pour ELAN, le champ d’action du programme est restreint et ne permet pas une généralisation. Pour l’APS, une étude de la performance des élèves au CI et CP en lecture n’a pas encore eu lieu. Evaluer l’ensemble des innovations en cours pour aider le MEP en vue d’une prise de décision DES/MEP PIN Janvier 2019 X 3 L’ASL, mise en œuvre par NECS, fonctionne largement en dehors des structures du MEP, par exemple, les conseillers pédagogiques et les inspecteurs reçoivent des prises en charge (carburant, per diem, nuitées). Pas de recommandations formulées Débat sur le constat : Pas un problème d’institutionnalisation plutôt d’ancrage dans le système. Cas de réussite du projet IFADEM dont la mise à l’échelle constitue un cas de succès. 4 Manque de preuves solides, scientifiques et reconnues pour démontrer la pertinence ou la plus-value de l’ASL. Organiser une autre évaluation scientifique et valide pour évaluer la performance des élèves Discussion sur les EGRAs COP/NECS DES/MEP Mai 2019 X 5 La mobilisation des fonds pour la reproduction du matériel pédagogique de l’ASL dans le contexte nigérien de « rareté des ressources » Intégrer les besoins dans le dispositif d’acquisition du MEP pour les matériels pédagogiques DCPLN/MEP DGPQN Rentrée scolaire 2019 X 6 La juxtaposition des initiatives en lecture sur le terrain et la question de l’intégration de l’ASL dans l’APS Evaluer l’ensemble des innovations en cours pour aider le MEP en vue d’une prise de décisions DES/MEP PIN Janvier 2019 X 179 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 7 Insuffisance dans la communication menée entre les cadres du Ministère, les cadres du projet NECS et l’USAID; Dynamiser le cadre existant et attribuer un cahier de charge précis assortis d’un dispositif de S&E SG/MEP COP/NECS Continue X 8 Urgence d’une prise de décision de la part du Ministère pour le choix des bonnes pratiques afin mettre fin à la spéculation Evaluer l’ensemble des innovations en cours pour aider le MEP en vue d’une prise de décisions Initiative en cours avec les ateliers de capitalisation des bonnes pratiques) par le MEP DES/MEP PIN Janvier 2019 X 9 Quelques préoccupations par rapport au suivi des enseignants sur le terrain et les moyens que cela nécessite Tenir en compte des moyens disponibles pour prendre en charge le suivi des encadreurs Les décideurs prennent en compte la nécessité d’avoir des moyens nécessaires pour assurer ce suivi DGPQN/ MEP DCPLNL/MEP Hypothès e X 10 Limites liées à la maîtrise de langue nationale comme langue d’enseignement par les enseignants Assurer des formations régulières à tous les enseignants Une seule formation ne suffit pas pour la maîtrise de la langue DCPLN, DFIC/MEP DFIC X 11 Difficultés pour amener les décideurs à intégrer la mise à l’échelle de l’ASL dans le cadre de la réforme du curriculum Evaluer l’ensemble des innovations en cours pour disposer des preuves solides pour influencer prise de décision DCPLN, DFIC/MEP DFIC X Poursuivre les pratiques en cours (atelier de capitalisation, forum) pour aider à la prise de décision 12 Difficultés relevées par rapport à l’intégration de l’ASL dans les formations des enseignants au niveau des écoles normales Pas de recommandation USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 180 N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 EQ. 4. QUELS SONT LES TYPES DE CHANGEMENTS VISIBLES DANS LES ECOLES ET COMMUNAUTES CIBLEES PAR NECS ? 1 Les gouvernements scolaires des écoles NECS visitées n’ont pas le même niveau de fonctionnalité. La plupart d’entre eux restent encore timides dans leur dynamisme et dans la connaissance de leurs rôles/responsabilités du fait de leur récente création. Redynamiser les gouvernements scolaires à travers une meilleure conscientisation des membres sur leurs rôles Les moments de suivi et d’observation des GS ne collent pas généralement avec le déroulement de leurs activités Enseignants et directeurs d’écoles Conseillers Pédagogiques /MEP Continue X 2 Dans le dispositif des gouvernements scolaires, il était prévu que les enseignants assurent leur encadrement. Mais sur le terrain, il est observé que la plupart d’entre eux sont laissés à leur propre sort, même si c’est une minorité parmi les enseignants qui l’a reconnu. Renforcer le dispositif d’encadrement des GS en y associant les directeurs d’écoles et les parents Les parents et le personnel enseignant doivent s’investir davantage Enseignants et directeurs d’écoles Conseillers Pédagogiques/M EP Continue X Sensibiliser les enseignants pour une meilleure appropriation de leur part vis-à-vis des GS 3 Seulement une minorité des enseignants observés sont parvenus à installer un climat de confiance avec leurs élèves. Former les directeurs d’écoles sur le leadership pédagogique et administratif Conseillers pédagogiques IEPs Continue X Veiller à ce que les directeurs assurent leur rôle de suivi et d’encadrement envers les enseignants Intégrer dans les missions de suivi et CAPED des conseillers pédagogiques des points de discussions avec les enseignants sur la relation élèves-maîtres 4 Identification des structures qui ne sont pas encore Renforcer les capacités des structures communautaires en Le format des PVs demandés au niveau PFDs/CGDES Continue X 181 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 autonomes (production des Plans d’actions, PVs de réunions, initiatives locales…) élaboration de plans d’actions, PVs et autres document communautaire doit être revu et adapté aux capacités des communautés Directeurs d’écoles Veiller à ce que les directeurs assurent mieux leur rôle d’encadreur des structures communautaires d’appui à l’école 5 Absence d’un ancrage profond des résultats de NECS au niveau communautaire : des changements plus « superficiels » que « profonds » Définir des stratégies qui vont permettre de poursuivre et d’accompagner les changements déclenchés au niveau des communautés DPC/SS/EE/MEP DGS/MEP Janvier 2019 X 6 Au niveau communautaire, des changements de pratiques limités par des réalités et croyances endogènes Produire des messages plus adaptés aux contextes et dont le contenu est axé sur les croyances socioculturelles Une zone peut-être très orientée par les opinions d’un leader religieux alors qu’une autre zone peut être gagnée par des pratiques commerciales ou migratoires DPC/SS/EE/MEP DGS/MEP Janvier 2019 X Concevoir des messages de sensibilisation en tenant des particularités de chaque zone 7 Une adhésion des communautés se faisant à des intensités différentes d’un endroit à un autre Définir des stratégies de mobilisation communautaire en fonction des réalités du milieu visé DPC/SS/EE/MEP DGS/MEP Janvier 2019 X 8 Fonctionnalité et l’utilisation des livrets des bibliothèques￾caisse a été apprécié de différentes manières avec des fréquences de prêts des livrets Réfléchir et changer le contenu des livrets pour qu’ils soient plus attrayant pour les hommes (toucher aux centres d’intérêts des hommes) Il faut trouver de meilleurs moments pour avoir les hommes dans les clubs de lecture (le soir par exemple) DPAFA/DES/ DGAENF MEP DGS/MEP Janvier 2019 X USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 182 N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 différentes entre les hommes et les femmes. Sensibiliser les hommes sur l’importance de la lecture et sur la fréquentation des bibliothèques villageoises 9 Faiblesse des structures communautaires d’appui à l’école à travers la méconnaissance de leurs rôles et responsabilités Systématiser des sessions de recyclage annuel avec une stratégie efficiente en collaboration avec les points focaux CGDES Les structures sont changeantes ; les acteurs ne sont pas les mêmes (le projet a fait des formations depuis 06 ans) DPC/SS/EE/MEP DGS/MEP Janvier 2019 X Renforcer le suivi des structures en revoyant le contenu des messages 10 Moment de ralentissement avec le mentorat du fait d’une incompréhension et d’un malentendu entre le projet et les mentors (démission et du départ de plusieurs mentors avec l’absence de rémunérations pour leurs tâches). Mettre en place d’un système de motivation et de reconnaissance endogène (même au niveau des maires) des mentors C’est la reconnaissance de leurs efforts qui est plus demandée par les mentors que des rémunérations financières DPC/SS/EE/MEP DGS/MEP Janvier 2019 X EQ. 5. DANS QUELLE MESURE LE PROJET A-T-IL REUSSI A ATTEINDRE LES FILLES DANS LE CIBLAGE ? 1 Quelques réticences au niveau de certains membres des communautés et des cas d’exode qui ne facilitent pas le travail relatif au pré￾recrutement que font les AMEs Impliquer toutes les parties prenantes lors des pré-recrutement y compris les leaders d’opinion chefs traditionnels, leaders religieux au niveau communautaire CGDES PFDs/CGDES Continue X Modifier le timing du pré￾recrutement dans les plans d’actions et les mener pendant les vacances 183 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 Faire des sensibilisations au cas par cas pour s’assurer qu’on est en face des personnes influentes au niveau des communautés 2 Des causes associées aux faibles taux de scolarisation des jeunes filles qui continuent de perdurer (exode, choix stratégiques des parents, locaux inadéquats, pauvreté, mariages précoces, petits commerces, mentalités des parents, préjugés sur l’école) Mener d’autres formes de sensibilisations plus concrètes par exemple aller vers une mobilisation des femmes qui sont des modèles de réussites DPC + DPSF/MEP DGS + DPQN/MEP Continue X Développer des mesures incitatives à plusieurs niveaux telles que les AGRs pour les parents Mettre en place un système de concours entre les villages d’une même commune pour gratifier et reconnaître les villages les plus performant 3 Le pari de la fréquentation, du maintien et de l’achèvement des filles pose encore problème au niveau des communautés Intensifier l’alphabétisation des femmes et leur motivation DPSF/MEP DGS + DPQN/MEP Octobre 2018 X Renforcer les échanges d’expériences entre les acteurs de sensibilisation au niveau des communautés Mettre en place un système d’émulation des filles au niveau local 4 Des enseignants qui ont signé le Code de bonne conduite sans l’avoir lu ou sans être parfaitement en accord avec les articles qui le constituent Organiser des journées d’informations/d’imprégnation sur le code de bonne conduite à l’intention des enseignants Conseillers pédagogiques IEPs Continue X Insister au niveau des conseillers pédagogiques pour une application USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 184 N° CONSTATS RECOMMANDATIONS COMMENTAIRES RESPONSABLE DE L’ACTION RESPONSABLE DU SUIVI DATE LIMITE NIVEAU DE PRIORITÉ 1 2 3 rigoureuse et correcte du dispositif du Code de bonne conduite 5 « Grille sensible au genre », dont l’expérimentation a été annoncé au niveau de Madaoua et dont la mise à l’échelle était prévue durant la phase d’extension NECS + (2016- 2018) n’est pas encore généralisée Clarifier la situation par rapport à la généralisation et à son envergure (champ d’actions) En réalité il ne s’agissait pas de généraliser au niveau national mais plutôt uniquement dans la DDEP de Madaoua (IEP Ourno). Dans le Code de conduite il était prévu d’avoir cette grille comme outil de vérification/évaluation sur les pratiques des enseignants en matière de genre en classe DFIC/MEP DSE/MEP Octobre 2018 X Faire le point sur les acquis/réussites de l’expérimentation faite à Madaoua Envisager une généralisation de la Grille dans toutes les écoles NECS 6 100% des responsables étaient des garçons. Aucune fille n’a occupé le poste de responsable de toutes les classes visitées. Inciter les enseignants à confier aux filles de plus de responsabilités SCOFI DPSF/MEP Continue X Renforcer les connaissances des enseignants sur les questions du genre Motiver les filles pour qu’elles s’intéressent à la gestion et à se porter volontaires pour les postes de responsabilité (lutter contre l’autocensure des filles) USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 138 ANNEX XVII: CODE OF CONDUCT Project Niger Education and Community Strengthening (NECS) Commune de :………………………………………………………………………………………………………… IEP de : …………………………………………………………………………………………………………………….. Ecole de :…………………………………………………………………………………………………………….. Nom et prénom de l’enseignant-e :……………………………………………………………………… CODE DE BONNE CONDUITE DE L’ENSEIGNANT-e SUR LE GENRE 139 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV PREAMBULE Dans le cadre de l’appui au gouvernement nigérien pour la mise en œuvre de sa politique éducative, le projet Niger Education and Community Strengthening (NECS) a identifié l’équité genre comme un important levier sur lequel il faut agir pour rehausser la scolarisation des enfants particulièrement celles des filles. Ainsi, il accompagne plusieurs activités de promotion de la scolarisation de la jeune fille tant au niveau de l’environnement scolaire qu’à l’intérieur des classes. A ce titre, le projet a renforcé les capacités des enseignants et des encadreurs pédagogiques pour une meilleure prise en compte du genre lors de l’animation des cours et de la gestion des classes. Cela renforcera sans nul doute la motivation des filles à venir régulièrement à l’école, à mieux apprendre et à y rester jusqu’au moins la fin du cycle de base. C’est pourquoi le projet NECS a appuyé les services étatiques, la société civile active en éducation et les enseignants à proposer ce code de conduite qui est un ensemble d’engagements que l’enseignant s’engage librement à respecter afin de promouvoir le genre en classe et hors de la classe. Chapitre I : Du principe de l’égalité des sexes face à l’éducation Article 1 : je reconnais que l’éducation est un droit humain de base mondialement reconnu pour tout le monde (sans distinction de sexe). Article 2 : j’accepte la politique éducative prônée par les autorités qui consacrent la gratuité et l’obligation scolaire des enfants (filles et garçons) jusqu’à l’âge de 16 ans. Article 3 : j’adhère à l’idée que les filles comme les garçons ont droit à une éducation de qualité à laquelle ils doivent accéder de manière équitable. Chapitre II : Des principes pédagogiques en faveur du genre Article 4 : dans mes activités pédagogiques, je ne manifesterai ni de la haine, ni de la complaisance basée sur le sexe. Article 5 : je m’engage à : • bannir tous les obstacles socio-culturels et les mauvaises interprétations des préceptes religieux liés au genre pour que tous (filles et garçons) puissent apprendre ; USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 140 • respecter les différences individuelles des élèves et reconnaitre que le genre, ainsi que l’âge, l’origine ethnique, la langue, le handicap et la religion font tous partie de l’identité de l’apprenant. Article 6 : je reconnais que les filles comme les garçons peuvent avoir les mêmes capacités intellectuelles et physiques. De ce fait : • il n’y a pas lieu pour moi de sous-estimer les filles et surestimer les garçons ou vice-versa ; • je proposerai les mêmes types d’activités à tous mes élèves sans distinction ; • dans les apprentissages, je mettrai en activité aussi bien les filles que les garçons, les forts comme les faibles ; • je différencierai mon enseignement pour qu’il soit accessible à tous (filles comme garçons et personnes vivant avec le handicap) en tenant compte des élèves forts comme des élèves faibles). Article 7 : j’adopterai une attitude impartiale et objective lors des évaluations en ne fondant mes appréciations et mes notes que sur les qualités des productions et les efforts de chacun et de chacune de mes élèves. Chapitre III : Des principes de promotion du genre dans l’environnement scolaire Article 8 : je m’engage à rendre l’environnement scolaire plus attrayant pour mettre les élèves (garçons et filles) dans un cadre d’apprentissage favorable. Article 9 : je veillerai à ce que, dans l’espace classe, les tables soient mixtes (filles et garçons ; élèves forts et élèves faibles). Article 10 : je m’engage à constituer des groupes de travail mixtes pour toutes les activités intellectuelles et physiques. Article 11 : je ferai dans ma classe des tableaux de service mixtes (balayage et nettoyage du tableau). Article 12 : je favoriserai lors des apprentissages les interactions dans les différents sens sans distinction de sexe. Article 13 : je m’engage à bannir toute forme de violence basée sur le genre. Article 14 : je ferai des sensibilisations sur le genre en milieu scolaire. 141 | NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION USAID.GOV Article 15 : je serai disponible aux différentes sollicitations d’ordre pédagogique de mes élèves sans distinction de sexe, ni de provenance sociale. Article 16 : je m’engage à protéger les élèves contre toute situation qui pourrait porter préjudice à leur fréquentation scolaire Chapitre IV : Des principes de promotion du genre à travers le comportement des enseignants dans la communauté Article 17 : je me comporterai comme modèle à suivre dans le domaine du genre à l’école (avec mes élèves et mes collègues) et au village (avec la communauté). Article 18 : je m’engage à me comporter décemment sur le plan vestimentaire. Article 19 : je bannirai de mon vocabulaire tout propos tendant à faire une discrimination basée sur le genre. Article 20 : je m’engage à accompagner les structures participatives sur les questions liées au genre. Article 21 : j’agirai promptement par la sensibilisation pour corriger tout comportement sexiste qui a tendance à s’enraciner dans la vie communautaire. Article 22 : je m’investirai dans toutes les activités de la communauté villageoise (travail collectif, mariage, décès, baptême). Article 23 : je m’engage à utiliser toutes les voies pour informer les autorités sur les problèmes qui dégradent la scolarisation en général et celle des filles en particulier. Article 24 : je m’engage à être aux côtés des parents pour accompagner les élèves sans discrimination aucune. Chapitre V : Des dispositions générales Article 25 : Ce code de conduite est conçu sous forme d’engagement contenant des chapitres et des articles. Il entre en vigueur à partir de sa date de signature et sera diffusé et publié partout où besoin sera. Article 26 : Compte tenu de tout ce qui a été dit, Je prends résolument et solennellement cet engagement devant la communauté, devant mes collègues et devant mes élèves. USAID.GOV NIGER EDUCATION COMMUNITY STRENGTHENING PROGRAM: FINAL PERFORMANCE EVALUATION | 142 .. Le./ / 20 Signature de L’Enseignant TEMOINS Le Directeur de l’école Le président du CGDES Le Conseiller Pédagogique L’Inspecteur