OUR SISTERS READ MIDTERM PERFORMANCE EVALUATION REPORT JULY 2016 This publication was produced for review by the United States Agency for International Development. It was prepared by USAID/Senegal Monitoring and Evaluation Project. The report was written by Gaëlle Simon with support from Abdourahim Gaye and Aïssatou Mbaye. OUR SISTERS READ MIDTERM PERFORMANCE EVALUATION USAID/SENEGAL Contracted under AID-685-C-15-00003 USAID/Senegal Mission-Wide Monitoring and Evaluation Project (MEP) DISCLAIMER This evaluation is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents are the sole responsibility of Management Systems International and do not necessarily reflect the views of USAID or the United States Government. Our Sisters Read Midterm Performance Evaluation Report iii CONTENTS Acknowledgments.........................................................................................................................iv Acronyms .................................................................................................................................. v Executive Summary...................................................................................................................... 1 Introduction and Project Background ..................................................................................................................................... 1 Evaluation Purpose and Questions........................................................................................................................................... 1 Evaluation Design, Methods and Limitations.......................................................................................................................... 1 Findings & Conclusions................................................................................................................................................................ 2 Recommendations ........................................................................................................................................................................ 5 Introduction and Project Background......................................................................................... 7 Evaluation Purpose & Evaluation Questions .............................................................................. 8 Evaluation Purpose ....................................................................................................................................................................... 8 Evaluation Questions.................................................................................................................................................................... 8 Evaluation Design, Methodology & Limitations ......................................................................... 9 Data Collection Methods and Sources..................................................................................................................................10 Data Analysis Methods..............................................................................................................................................................11 Limitations.....................................................................................................................................................................................12 Findings & Conclusions ............................................................................................................... 13 Recommendations ...................................................................................................................... 32 Annex 1: Evaluation Statement of Work ................................................................................. 34 Annexe 1.1: Calendrier d’Execution ......................................................................................................................................48 Annexe 1.2: Matrice de la Revue Documentaire................................................................................................................49 Annexe I.3: Matrice des Resultats, Conclusions et Recommandations ........................................................................50 Annex 2: Evaluation Work Plan................................................................................................ 51 Annex 2.I: Getting to Answers Matrix ..................................................................................................................................63 Annex 2.2: Evaluation Timeline ...............................................................................................................................................67 Annex 2.3: Roles and Responsibilities ...................................................................................................................................68 Annex 3: Interview Guides ......................................................................................................... 69 Annex 4: School Level Data for Sampling ................................................................................ 74 Annex 5: Sources of Information............................................................................................... 76 Our Sisters Read Midterm Performance Evaluation Report iv ACKNOWLEDGMENTS The evaluation team wishes to thank all those who contributed to this performance evaluation, including: USAID/Senegal, who provided funding and technical support; the MEP team in Dakar; Women’s Global Education Project (WGEP) employees; and Femmes Plus team members. We would also like to thank all participants in the districts of Sokone and Toubacouta who met with the evaluation team to share their experiences with and appreciation for the Our Sisters Read initiative. The evaluation team would like to further thank the students, teachers, head teachers, parents and community members who readily volunteered their time. Our Sisters Read Midterm Performance Evaluation Report v ACRONYMS AME Association des mères d’élèves (Association of Student’s Mothers) APE Association des parents d’élèves (Students’ Parents Association) AOR Agreement Officer Representative CDCS Country Development Cooperation Strategy CGE Comité de gestion d’école (School Management Committee) CM Cours Moyens (Grades 5 and 6) CODEC Collectif des Directeurs d’Ecole (Collective of School Directors) DQA Data Quality Assessment DEE Direction de l’éducation élémentaire (Bureau of Primary Education) DO Development Objective EQ Evaluation Question FP Femmes Plus GOS Government of Senegal ICT Information and Communications Technology IEF Inspecteur d’éducation et de formation (Inspector for Education and Training) IR Intermediate Result KGBX Keur Gui Bok Xol KII Key Informant Interview MEN Ministère de l'Education Nationale (Ministry of Education) MEP Monitoring and Evaluation Project M&E Monitoring and Evaluation MSI Management Systems International OCA Organizational Capacity Assessment OSR Our Sisters Read PAQUET Programme d’Amélioration de la Qualité, de l’Equité et de la Transparence (Quality, Fairness, and Transparency Improvement Program) PDEF Programme décennal de l’éducation et de la formation (Yearly Program of Education and Training) SNERS Système national d’évaluation des rendements scolaires (National School Performance Evaluation System) USAID United States Agency for International Development UNESCO United Nations Educational, Scientific and Cultural Organization WGEP Women’s Global Education Project Our Sisters Read Midterm Performance Evaluation Report 1 EXECUTIVE SUMMARY Introduction and Project Background The Our Sisters Read (OSR) activity was awarded to Women’s Global Education Project (WGEP), a US based organization, under the USAID Development Grants Program in September 20131. This four-year, $1,700,904 activity seeks to improve reading skills for primary school children and to create lifetime￾learners, with an emphasis on promoting girls’ literacy. WGEP works with a local partner organization, Femmes Plus (FP), to implement interventions in Senegal’s Fatick region, and more specifically in the district of Foundioungne. OSR engages and supports primary-aged school children (Grades 1 to 6), their families, and their communities to create and strengthen a culture of reading while reducing the gender gap in reading and education. Implemented in 52 schools in Sokone and Toubacouta, OSR is targeting 6,800 students in primary grades, of which at least 50 percent are girls. The overarching goal for OSR is “to create strong readers and lifetime learners among rural Senegalese primary school students, especially girls”. The attainment of this goal is dependent on the achievement of the three objectives focused on 1) creating access to mobile libraries and communication centers; 2) increasing motivation to read through contests, scholarships and book clubs and 3) increase community and parent support toward reading. OSR is part of the USAID/Senegal’s Education Intermediary Result 1: better educated youth, which supports Development Objective (DO) 3: more effective citizen participation in the management of public affairs at the national and local levels. Evaluation Purpose and Questions USAID/Senegal initiated a midterm performance evaluation of OSR to understand the implementation of interventions and the activity’s progress towards the attainment of its stated objectives. This performance evaluation addresses the following three global evaluation questions (EQ): • EQ 1: How well has the theory of change been validated through the activity’s implementation? • EQ 2: To what degree has the implementing partner (including the subcontractor) achieved the activity’s objectives? • EQ 3: How has the capacity of the implementing partner (including the subcontractor) changed over the course of the activity? What gaps remain? Evaluation Design, Methods and Limitations A team of three conducted the performance evaluation. Team members included a senior national education specialist, the monitoring and evaluation (M&E) associate for the MEP project, and a lead evaluator from the Management Systems International (MSI) education practice area. The field work in Sokone and Toubacouta, lasted ten days, from 18 to 29 June 2016. This evaluation uses a purposive sample to collect data in schools and communities. In total, evaluators chose 8 schools for this evaluation, including four schools in Toubacouta and four schools in Sokone. 1 As of 2015, localworks replaces the Development Grants Program and is referred to as the Small Grants Program. https://www.usaid.gov/partnership-opportunities/ngo/development-grants-program Our Sisters Read Midterm Performance Evaluation Report 2 Data Collection Methods and Sources This performance evaluation relies on qualitative methods with some quantitative analysis of student reading performance data. The evaluation team did not collect primary assessment data or other types of quantitative data. All quantitative data used for this evaluation were obtained from secondary sources. To evaluate the activity’s performance in achieving its objectives and measure its influence on beneficiaries’ attitudes towards and practices in reading, the team conducted a total of 56 interviews with beneficiaries and stakeholders. Evaluators used the following data collection methods: • Desk review of activity documents; • Group interviews with male and female students in Grades 5 and 6 (CM1 and CM2), teachers, and community members. Evaluators convened meetings with village chiefs, religious leaders, members of school management committees (CGE) and parent associations (AME) and parents; • Individual interviews with nine school directors and the mayors of Sokone, Toubacouta and Karang; • Key informant interviews (KII) with the management teams at WGEP, Femmes Plus and USAID, with the education inspector for Sokone and Toubacouta and the director for primary education (DEE) at the ministry of education (MEN); and • Analysis of student test scores was conducted using activity data. Data Analysis Methods Data analysis draws on findings organized by evaluation questions. The team analyzed qualitative data collected from the individual and group interviews for theme and content. The team collected additional quantitative data from activity documents, e.g. reports. Student performance data provided by the implementers was analyzed using simple univariate descriptive statistics. Quantitative data analysis was led by the MEP statistician using SPSS statistical package. Limitations Limitations of this evaluation include inconsistencies in indicator data from activity documents, the validity and reliability of student performance data, a lack of observation data of teacher classroom practices, and a lack of organizational capacity assessment of finance and administrative audit for WGEP by USAID which made it difficult to answer portions of question 3 related to changes in institutional capacity. Findings & Conclusions The following section provides a synthesis of the evaluation results. It presents findings and conclusions for each of the evaluation questions followed by recommendations. EQ 1: How well has the theory of change been validated through the activity’s implementation? The theory of change that underpins the results framework for OSR posits that a literate environment, where students are encouraged to read and where the community promotes and supports literacy and schooling, will improve overall student performance in schools, and particularly in reading. To determine whether that theory of change has been validated through the implementation of the OSR activity, the Our Sisters Read Midterm Performance Evaluation Report 3 evaluation team assessed the degree to which the activity outcomes and intermediate results were achieved. The OSR results framework and three objectives are outlined below. Figure 1: OSR Results Framework OSR has motivated students to read and has mobilized parents and community leaders to support education. However, the evaluation team found no substantiated evidence to prove that students’ gains in reading are attributable to the intervention, nor that teachers have improved their instructional literacy practices since being trained by OSR. The theory of change for OSR was not validated through its implementation in part because the school￾based outcomes were not achieved. Though successfully implemented, the community-based interventions are not sufficient to develop foundational reading skills in primary-aged students. EQ 2: To what degree has the implementing partner (including the subcontractor) achieved the activity’s objectives? Findings The results of OSR student reading test in reading out loud and reading comprehension at pre- and post￾test stages show improvement in performance. The test scores are presented below. In all grades, students performed better at post-test than at pre-test in reading out loud, while in Grades 2, 4 and 6, students achieved lower mean scores in reading comprehension at post-test than at pre-test. However, there is limited evidence to suggest that these gains in reading out loud and in reading comprehension are attributable to OSR. The level of student motivation to read is clearly seen in the activity’s project sites visited. According to WGEP and FP, the annual reading contests, in which all 52 schools participate, are motivating factors for students, schools and communities. Teachers and community members also mentioned that the scholarships are a source of motivation for the students. For the 2015-2016 school year, 139 scholarships were awarded to primary school students and 41 scholarships were awarded to middle school students according to the year 3 quarter 2 OSR report. The community support for reading is clearly present in the communities visited. All community members interviewed complimented the activity’s community mobilization strategy. In particular, respondents reported that the adult literacy classes were well attended by women. In its Year 3, Quarter 2 report, OSR notes that adult literacy classes were offered in 36 sites during the second quarter. This number is up 28 sites the year before and is elevated due to requests from community members. Our Sisters Read Midterm Performance Evaluation Report 4 Conclusions OSR’s most accomplished intervention is community mobilization. Parents and community leaders regularly attend sensitization meetings and women are enthusiastic about the adult literacy classes. OSR has reached 8,077 unique students, 4,176 of whom are girls, according to the year 3 quarter 2 OSR report. OSR has surpassed its goal to provide access to Information and Communications Technology (ICT) centers and mobile libraries to 6,800 students in the Sokone and Toubacouta areas. The OSR reading activities, as well as the annual reading contest and the scholarship packages, are motivating factors for students to read. However, the proportion of students visiting the ICT centers and borrowing books from the physical libraries remains low. Approximately 14 percent of students in the catchment area of the Sokone center visited the center in June, while 37 percent of students in the catchment area for the center in Toubacouta visited the center in May2. While parents and teachers believe that student reading has improved, the results from OSR’s testing process are mixed. The OSR student performance data indicates that students achieved some gains in reading in all grades. In Grades 2, 4 and 6, however, students achieved lower scores in reading comprehension at post-test than at pre-test. However, given the concerns about the validity and reliability of the student performance data, the evaluation team cannot make a determination as to whether the observed gains are due to the impact of the OSR interventions or to the test’s construction. EQ 3: How has the capacity of the implementing partner (including the subcontractor) changed over the course of the activity? What gaps remain? Findings USAID issued The Our Sisters Read contract to Women’s Global Education Project as part of the Agency’s Development Grants Program. A core objective of the program is for USAID Missions to provide partner organizations "access to technical assistance to improve their planning and managements systems and other competencies needed to successfully implement USAID-funded activities"3. However, the evaluation team did not find any mention of plans or funds geared towards the organizational capacity development of WGEP or FP in activity documents. In October 2015, USAID/OFM conducted the first OCA for FP. It noted that approximately 30 percent of the recommendations from the first assessment in May 2014 had been completed by then. Important gaps remained, particularly in terms of human resources policies and manuals, filing systems and financial management. USAID/OFM noted that FP and the activity relied heavily on its director and that FP did not have a qualified financial manager at the time. USAID conducted a data quality assessment (DQA) in March 2016, which cited several concerns with the monitoring data as reported by WGEP and FP. These concerns include instances of double counting of beneficiaries, the absence of supporting documents, inadequate data collection forms, the lack of a manual for data collection and management, and a lack of data quality assurance procedures. The FP director explained that FP is working to put in place systems to improve the capacity of its members to better manage programs. Her priorities for FP include the establishment of a board of directors, registration as a non-governmental organization (NGO), training in fundraising and making the 2 Please see Annex IV for figures used for these calculations. These data were provided by OSR to inform the sampling for this evaluation. In Sokone, 295 students out of 2146 in the catchment area visited the center in June. In Toubacouta, 199 students out of 528 in the catchment area visited the center in May. 3 http://apply07.grants.gov/apply/opportunities/instructions/oppM-OAA-GRO-ALPS-09-001-LOCAL-cfda98.001-cidM-OAA￾GRO-ALPS-09-001-LOCAL-instructions.pdf Our Sisters Read Midterm Performance Evaluation Report 5 organization more visible. For WGEP, capacity-building of FP involves ensuring that the OSR interventions are sustainable. WGEP explained that during the remaining year of the activity, they will work with FP to turn the ICT centers into community centers where resources, such as the internet, can be used to generate revenue and cover part of the operating expenses. Conclusions There is an important difference between WGEP and FP in their vision and long-term plans for FP. While WGEP seems to be mainly focused on planning for the sustainability of OSR interventions, the FP director is looking to strengthen the organization's technical, financial and management systems for its long-term health. This disparity in vision has consequences on organizational capacity-building plans. WGEP and FP rely heavily on locally recruited staff, interns and volunteers to implement the OSR activity. However, the pool of available and qualified candidates is very restricted which negatively affects the implementation of interventions and organizational development, particular in regards to strong financial and data management systems. WGEP has not received sufficient and timely guidance on how to manage USAID contracts. For this new grantee, the lack of early guidance has meant that they have had to learn USAID rules and regulations as they were implementing the activity. This led to miscommunication, frustrations and implementation delays. Recommendations The following recommendations are based on the findings and conclusions presented in this report and are organized for the consideration of key stakeholders. WGEP 1. WGEP and USAID/Senegal should develop an institutional capacity development plan with a timeline, targets and allocated budget for Femmes Plus. 2. WGEP and Femmes Plus should hire a qualified financial manager from outside the immediate Sokone and Toubacouta areas to respond to immediate needs and to train a locally hired manager who will eventually take on financial management responsibilities for the OSR activity and for Femmes Plus in the long-term. 3. Similarly, a qualified monitoring and evaluation officer should be recruited to help set up systems for appropriate data collection, verification and management. 4. OSR should review the planning of school visits by the mobile libraries to prioritize schools that are furthest from the centers. 5. Though intended to be used as extracurricular activities, OSR should continue to implement the reading activities in remote villages during the summer vacations. 6. OSR, and future similar activities, should provide initial training and continuous coaching, or teacher professional development, to all teachers within a school through the Collectif des Directeurs d’Ecole, or CODEC and “cellule d’animation pédagogique”. 7. OSR should continue to provide adult literacy classes as a key motivator. 8. OSR should replicate, in communities that solicit it, the tutoring models that some communities have initiated. FP Our Sisters Read Midterm Performance Evaluation Report 6 9. Femmes Plus should focus on sustainable organizational capacity development including the retention of a qualified finance manager and deputy director. USAID 10. OSR’s community mobilization strategy should serve as a model for other activities that have a community-based component. 11. USAID/Senegal should provide early guidance on available programmatic and management resources to new grantees, whether located in the US or in Senegal. 12. USAID/Senegal should conduct an initial OCA of first time local grantees within the first six months of a contract. Our Sisters Read Midterm Performance Evaluation Report 7 INTRODUCTION AND PROJECT BACKGROUND In Senegal, compulsory education lasts ten years for children ages seven to sixteen. With the implementation of the education and training reform program, PDEF,4 from 2000 to 2010, primary school enrollment rates increased to 82 percent gross enrollment in 2010. However, since then, educational statistics in Senegal have decreased or remained stagnant, losing gains achieved under the PDEF. According to UNESCO5, the gross enrollment rate in 2014 was 81 percent; and while access to education did increase with PDEF, educational achievement remained low, with only 61 percent of students reaching the last year of primary education in 2011. According to national educational statistics in Senegal (SNERS), slightly more than half of students in grades two and four had acquired foundational reading skills in 20106, while 70 percent of youth ages fifteen to twenty-four were literate.7 To address these concerns, the Government of Senegal (GOS) launched le Programme d’Amélioration de la Qualité, de l’Equité et de la Transparence (PAQUET), the 2013-2025 national education policy that seeks to achieve universal primary education and improve educational quality. The Our Sisters Read (OSR) activity was awarded to Women’s Global Education Project (WGEP), a US based organization, under the USAID Development Grants Program in September 2013.8 This four-year, $1.7 million activity seeks to improve reading skills for primary school children and to create lifetime learners with an emphasis on promoting girls’ literacy. WGEP works with a local partner organization, Femmes Plus (FP), to implement interventions in Foundiounge district (Fatick region). The activity is implemented under Development Objective 3 (DO3) of USAID/Senegal’s Country Development Cooperation Strategy (CDCS) and as part of its support to the GOS and PAQUET9. OSR engages and supports primary-aged school children, (grades one to six) their families and communities to create and strengthen a culture of reading while reducing the gender gap in reading and education. Implemented in 52 schools, 28 in Sokone and 24 in Toubacouta, OSR is targeting 6,800 students in primary grades, of which at least 50 percent are girls. The overarching goal for OSR is to create strong readers and lifetime learners among rural Senegalese primary school students, especially girls. Attaining this goal depends on achieving three intermediate results (IRs): • IR 1: Build foundational reading skills for 6,800 rural primary school students (at least 50 percent female) by creating access to technology centers, mobile labs and trained literacy professionals by 2017; • IR 2: Encourage and motivate students to read through innovative resources and incentives, reaching students at participating schools each year through contests, scholarships, mentorships and book clubs; and • IR 3: Build communities of parents and leaders that support young girls and boys to increase their reading and technological literacy. This multipronged approach seeks to contribute to an enduring shift in attitudes towards literacy and girls’ education, which will translate into higher achievement levels in reading for primary-aged students. Activity interventions, including reading contests, scholarships, book clubs, and mentoring are designed to motivate and generate enthusiasm among young readers, supplementing school curricula and filling in gaps where the quality of education is lacking. The physical resources (information & communication technology (ICT) 4 Programme décennal de l’éducation et de la formation (PDEF) 5 http://www.uis.unesco.org/DataCentre/Pages/country-profile.aspx?code=SEN 6 http://www.gouv.sn/IMG/pdf/PAQUETEF.pdf 7 UNESCO 2015. 8 As of 2015, localworks replaces the Development Grants Program and is referred to as the Small Grants Program. https://www.usaid.gov/partnership-opportunities/ngo/development-grants-program 9 Programme d’Amélioration de la Qualité, de l’Equité et de la Transparence (PAQUET) Our Sisters Read Midterm Performance Evaluation Report 8 centers, libraries and mobile labs) serve as both locations for interventions and community anchors. Community outreach interventions that support adult literacy and parental and community involvement address underlying attitudes towards literacy and promote its importance, particularly for girls. Figure 2 illustrates the theory of change for the OSR activity, as understood by the evaluation team: Figure 2: Our Sisters Read Theory of Change EVALUATION PURPOSE & EVALUATION QUESTIONS Evaluation Purpose After nearly three years of implementation, USAID/Senegal initiated a midterm performance evaluation of OSR to understand the implementation of interventions and the activity’s progress towards the attainment of its stated objectives. This performance evaluation will generate data on implementation successes and challenges, how stakeholders and beneficiaries perceive and value the activity, and whether the activity is on target to achieve its goals and objectives. The findings and conclusions will provide recommendations for midcourse corrections and insights and lessons learned will inform the upcoming basic education activity funded by USAID/Senegal. Evaluation Questions The evaluation questions (EQ) addressed by this performance evaluation follow three global questions with a series of sub-questions outlined below: 1. How well has the theory of change been validated through the activity’s implementation? a. Have activity interventions, such as access to resources, extra-curricular activities, book clubs, competitions, scholarships, etc., motivated students to read? b. Has teacher training improved classroom practices? c. Have activity interventions, such as adult literacy, community meetings, mentoring by teachers, etc., increased community mobilization? Which interventions were most effective? d. What is the level of community engagement in this activity? Why? Our Sisters Read Midterm Performance Evaluation Report 9 2. To what degree has the implementing partner (including the subcontractor) achieved the activity’s objectives? a. Was the activity implemented according to the design? Has the implementation of the activity followed the original design? b. What internal or external factors (including within USAID) have had the biggest impact on the implementation of the activity? Which factors may continue to impact the activity and which ones seem to have had short-term/specific impact? c. How did the implementing partners and the USAID education team address challenges during the implementation of the activity and what are some lessons learned? 3. How has the capacity of the implementing partner (including the subcontractor) changed over the course of the activity? What gaps remain? EVALUATION DESIGN, METHODOLOGY & LIMITATIONS This performance evaluation was conducted by a team of three people, including a senior national education specialist, an M&E associate for the MEP project, and an evaluation team leader from the MSI education practice area. The fieldwork in the Foundiougne district (specifically Sokone and Toubacouta) lasted ten days from 18 to 29 June 2016. This evaluation used a purposive sample to collect data in schools and communities. Evaluators conducted an initial stratification of the 52 intervention schools according to urban and rural locations. For the purposes of this evaluation, the evaluation team considered urban schools to be those within the towns of Sokone and Toubacouta. Schools that are outside of the towns were classified as rural schools. Of the 52 OSR interventions schools, three in Toubacouta and eight in Sokone were classified as urban schools. The distinction between urban and rural schools for sampling purposes was important given that the urban schools, particularly those in Sokone, are typically larger than village schools, have more than one class per grade, and have easier access to the activity’s ICT centers. Second, the evaluation team considered school accessibility to ICT centers. The final sample includes at least one urban and one rural school with access to ICT centers in Sokone and in Toubacouta. Third, the team considered school size and distances from the main towns. Schools that were too isolated were removed from the sample to allow the team to maximize the number of schools and communities that could be visited daily. The team also included an isolated school at the border with The Gambia because the geopolitical environment differs from that of Sokone and Toubacouta. Finally, Femme Plus suggested schools where the activity was well implemented and also schools where there had been challenges. In total, eight schools were chosen for this evaluation with four schools in Toubacouta, of which one is urban, and four schools in Sokone, of which one is urban. Figure 3 above depicts the geographical spread Figure 3: Distribution of Sampled Schools Our Sisters Read Midterm Performance Evaluation Report 10 of the sampled schools in Sokone (Danthakoune, Diossong, KGBX, and Ndiokholokho) and in Toubacouta (Karang Socé, Médina Sangako, Sandicoly, and Toubacouta 1). Data Collection Methods and Sources This performance evaluation relies on qualitative methods with some quantitative analysis of student reading performance using data provided by OSR. The evaluation team did not collect primary assessment data or other types of quantitative data. All quantitative data used for this evaluation were obtained from secondary sources. To evaluate the activity’s performance in achieving its objectives and measure its influence on beneficiary attitudes towards and practices in reading, the team conducted a total of 56 interviews with beneficiaries and stakeholders. The following data collection methods were used during this evaluation: Desk Review The evaluation team conducted a desk review of activity documents to develop an understanding of the activity’s theory of change and implementation characteristics, contexts, and challenges and to assess the level of achievement against targets. Group and Individual Interviews The team conducted group interviews in schools with boy and girl students in grades five and six (CM1 and CM2), teachers and community members. Individual and group interviews were held with village chiefs, religious leaders, members of school management committees (CGEs) and parent associations such as associations des mères d’élèves (AMEs) and associations des parents d’élèves (APEs) and parents to discuss OSR community mobilization activities as well as students’ reading behaviors at home and in schools. Teachers and students provided information on their level of satisfaction with reading activities in schools and ICT centers. Activity staff discussed the implementation of activities in schools, ICT centers, and in communities. The team conducted 18 group interviews with community members of which 39 teachers and 78 students participated in group interviews. Female students were interviewed separately from boys to facilitate open discussion. Nine school directors10 were individually interviewed in order to discuss teacher practices for the teaching of reading, student performance in reading and parental engagement. The mayors of Sokone, Toubacouta and Karang discussed collaboration with and support for the activity as well as their appreciation for and perceptions of the activity interventions. Lastly, the team conducted KIIs with the management teams at WGEP; Femmes Plus; USAID; the education inspector for Sokone and Toubacouta; and the DEE at the MEN. Analysis of program data The team also analyzed student test scores in reading using data provided by the activity. The activity conducted the post-test in June 2016 and it was implemented in 12 of the 24 schools in Toubacouta that had participated in the baseline in November 2015. The evaluation team analyzed baseline and post-test data from 11 of these schools to assess student performance (1 school in Toubacouta did not participate in the pre-test). 10 The evaluation team visited 8 target schools but interviewed one additional school director in the FP office Our Sisters Read Midterm Performance Evaluation Report 11 The “Getting to Answers Matrix” for this evaluation is included in the evaluation work plan annexed to this report (Annex II). Information sources, number of respondents and data collection methods for each evaluation question are summarized in Table 1 below: Table 1: Data Source, Number of People Consulted, Data Collection Method Evaluation question (EQ) topic Students Teacher s School Directors Community DEE + Inspector Mayors OSR staff WGEP + FP USAI D (78) (41) (9) (87) (2) (5) (11) (3) (3) Group Group Individual Group KII Individual Group KII KII 1. Validation of theory of change X X X X X X X 1a. Student motivation X X X X X 1b. Classroom practices X X X X 1c. Community mobilization X X X X X X X 1d. Community engagement X X X X X X 2. Achievement of objectives X X X 2a. Implementation and design X X 2b. Internal and External factors X X X 2c. Addressing challenges X X X X X X 3. Organizational capacity X X The list of project documents reviewed and a complete list of respondents are available in Annex V: Sources of Information. Data Analysis Methods Qualitative Data Analysis Data analysis draws on findings organized by evaluation questions. Qualitative data collected from the individual and group interviews were analyzed using thematic and content analysis. At the end of each data collection day, team members typed up interview notes, noting at the top of each interview guide any interesting or significant findings as well as common and emerging themes. Team members also noted areas to be pursued in subsequent interviews to triangulate findings from interviews or project documentation. The evaluation team recorded interviews and sent them to the MEP team in Dakar every day for transcriptions and translations, as necessary. Quantitative Data Analysis The team utilized secondary quantitative data collected from activity documents, such as quarterly and annual reports as well as a baseline student testing report and post-test scores. Student performance data provided by the implementers were analyzed using simple univariate descriptive statistics. The MEP data analyst led quantitative data analysis using the SPSS statistical package. Our Sisters Read Midterm Performance Evaluation Report 12 Limitations Limitations of this evaluation include: • Inconsistencies in indicator data from project documents • Validity and reliability of student performance data • Lack of observation data of teacher classroom practices • Lack of organizational capacity assessment or finance and administrative audit for WGEP The evaluation team’s review of the project documents revealed inconsistencies in the indicator data reported. Of particular concern were instances of possible double-counting of student and parent participants in activity interventions. The DQA that USAID/Senegal conducted in March 2016 for OSR also highlighted those concerns. The quarterly report for quarter two of year three (January to March 2016) of the activity indicates corrections to previously reported data. However, it remains unclear which corrective measures were implemented in the activity’s M&E systems and processes to improve the quality of the data collected and reported. Students in seven schools in Sokone were tested in reading at the end of the 2014-2015 school year. However, given that these schools had already received OSR interventions for one to two years, these data cannot serve as a baseline to measure activity impact on student reading performance in the Sokone region. In November 2015, OSR conducted a reading test with students in the 24 Toubacouta schools that were just joining the activity to collect baseline data. A follow-up test was implemented in June 2016 in 12 Toubacouta and 12 Sokone schools chosen randomly. According to WGEP, the baseline data from the Toubacouta schools could serve as a baseline for Sokone schools as well. However, given the differences between the two regions, e.g. number of large urban schools in Sokone, and the inability to statistically measure equivalency of the two regions at baseline, the evaluation team has recommended against using the Toubacouta baseline for Sokone. Additionally, the evaluation team was unable to determine the validity of the tests, particularly as it relates to the difficulty level of the content (e.g. whether the content is too easy or too difficult for the particular grade level). In addition, no information is available to confirm that that the tests at baseline and follow￾up are of the same difficulty level. As such, it is difficult to judge whether differences in student scores are due to performance levels or are a factor of the test’s construction. While the evaluation question 1b seeks to understand classroom practices for the teaching of reading, the evaluation team was unable to conduct classroom observations. The evaluation team’s findings are thus solely based on the information provided by respondents. An organizational capacity assessment of the prime contractor, WGEP, was not conducted so limited information is available regarding its capacity to implement such activities. For this reason, the response to question number three, for the primary contractor, was limited to interviews and a desk review of project documents. Our Sisters Read Midterm Performance Evaluation Report 13 FINDINGS & CONCLUSIONS The report presents findings and conclusions for each of the evaluation questions followed by overall recommendations. EQ 1: How well has the theory of change been validated through the activity’s implementation? The theory of change that underpins the results framework for OSR posits that a literate environment, where students are encouraged to read and where communities promote and support literacy and schooling, will improve overall student performance in schools, and particularly in reading. To determine whether OSR activity implementation has validated that theory of change, the evaluation team assessed the degree to which the activity outcomes and intermediate results were achieved. For that purpose, the findings and conclusions from the following evaluation questions and sub-questions informed the conclusions presented in Table 2 below: • EQ 1a: Have activity interventions, such as access to resources, extra-curricular activities, book clubs, competitions, scholarships, etc., motivated students to read? • EQ 1b: Has teacher training improved classroom practices? • EQ 1c: Have activity interventions, such as adult literacy, community meetings, mentoring by teachers, etc., increased community mobilization? Which interventions were most effective? • EQ 1d: What is the level of community engagement in this activity? Why? • EQ 2: To what degree has the implementing partner (including the subcontractor) achieved the activity’s objectives? Findings As a community based activity, OSR worked closely with traditional community-based authorities such as village chiefs and imams as their main entry point into communities. However, they had some challenges initially in being able to work local education authorities which resulted in limitations to accessing schools. While OSR now has signed an agreement with the local education inspectors, they have not had the full and open access to schools as they would have wished. OSR’s reading modules are typically used only after school, twice a week when the OSR outreach staff visit the schools. This has limited OSR’s ability to influence in-classroom instruction and time allocated to reading activities. OSR increased student access to reading materials through their mobile library and ICT centers. This access, along with student reading competitions, tutoring, and limited access to tablets with reading materials, helped to increase student interest in reading. Parents and community members viewed student scholarships targeting at risk (potential drop out) students as consistently effective, and parents appreciated the one-on-one tutoring time that teachers provided to students. Parents also mentioned literacy classes as a highly motivating factor in their own participation in OSR. The majority of teachers and school directors interviewed expressed the view that students were improving their reading skills. OSR conducted tests which show limited evidence of improvement in reading comprehension, while there is some evidence of improvement in reading out loud. Our Sisters Read Midterm Performance Evaluation Report 14 Table 2: Theory of Change and Results Framework Goal: To create strong readers and lifetime learners among rural Senegalese primary school students, especially girls Intermediate Results Outcomes Findings and Conclusions Build foundational reading skills for 6,800 rural primary school students (at least 50 percent female) by creating access to technology centers, mobile labs and trained literacy professionals by 2017 Increased access to reading materials through ICT centers, libraries and mobile labs OSR improved access to reading materials for students but there is no evidence to demonstrate improvement in reading skills as a result of OSR interventions Improved instructional literacy practices Innovative reading activities were introduced, but there is mixed evidence as to whether teachers implement them in their classrooms. There is no evidence that they led to improved instructional practices in the classroom. Encourage and motivate students to read through innovative resources and incentives, reaching students at participating schools each year through contests, scholarships, mentorships and book clubs Activities motivate primary school students to read OSR activities have increased student motivation to read. Increased opportunities for students to attend school Scholarships support retention of students in schools. Build communities of parents and leaders that support young girls and boys to increase their reading and technological literacy Increased support among parents and community leaders for literacy generated through a comprehensive campaign to promote literacy OSR strengthened parental and community support for education and reading. Conclusions OSR motivated students to read and mobilized parents and community leaders to support education. However, the evaluation team did not find evidence that attributed students’ gains in reading to the intervention, or evidence that teachers improved their instructional literacy practices since receiving OSR training. OSR was limited in its ability to intervene in classrooms and therefore had a limited effect on improving instructional practices in the classroom. The project’s implementation did not validate OSR’s the theory of change, in part because the school￾based outcomes, in terms of improved teaching practices in the classroom, were not achieved. Though successfully implemented, the community-based interventions are not sufficient to develop foundational reading skills in primary-aged students. Our Sisters Read Midterm Performance Evaluation Report 15 EQ 1a: Have activity interventions, such as access to resources, extra￾curricular activities, book clubs, competitions, scholarships, etc., motivated students to read? Findings According to the school directors interviewed, students are reading more and better since the start of OSR activities. The inspector for Sokone and Toubacouta noted that students are more motivated to read due to the growth of environments that are conducive to reading. Teachers interviewed provided examples of reading practices students have started displaying since the beginning of the OSR activities. These included reading street signs, picking up pieces of paper on the school ground to read, and reading the writings on clothes and fabrics. School directors, teachers and the inspector interviewed explained that the OSR interventions are particularly motivating for students because they are engaging and provide students an opportunity to express themselves. When interviewed by the evaluation team, students in grades 5 and 6 enthusiastically shared their favorite OSR reading activity, such as “Langa Burri11” and comics, and listed book titles they had read and enjoyed, such as “La Grande Forêt du Saloum”, “Sidi et Rama” and “Boubou Golo”. Students were particularly excited about the tablets. In fact, all respondents cited the tablets as the most effective OSR reading activity, and requested that more tablets be made available to the schools on a regular basis. The FP director and WGEP noted that the tablets were included as an activity to further motivate students and parents to read, but that the intent was never to provide one tablet per child (the ICT coordinator takes 25 tablets to schools to use with students in the classroom during specific appointed times). WGEP and the FP director also pointed to the participation of schools, students, parents and community leaders at the annual reading competition as evidence of beneficiaries’ motivation. Teachers at Keur Gui Bok Xol (KBGX), for example, mentioned that the competitions were a great motivating factor for parents and students who are publicly recognized for their success in reading. The evaluation team reviewed activity documents which indicate that students are participating in activity interventions. In June 2016, 475 students reportedly visited the centers, representing approximately 18 percent of the 2,674 students in the catchment areas. The July 1, 2016 OSR biweekly report states that in the month of June, 295 students (including 192 girls) visited the newly opened Sokone ICT center who “voluntarily visited the center to participate in the reading activities of Sac à Mots, Albums, Livre Ami and Fresque”. The report also notes that 196 students (127 girls) used the computer room during the same period. OSR also reported that 3,613 students (1,817 girls) had participated in reading activities in the 13 schools visited by the outreach officers (encadreurs) in May 2016. Conclusions While it is beyond scope of this evaluation to measure the motivation level of students to read, the evaluation team noted the great enthusiasm for OSR activities. The novelty and interactive nature of the 11 An activity which requires students to reconstitute a story using missing and out of order words. Figure 4: Student Work from the OSR Reading Activity "Calendrier Personnalisé" in Diossong Our Sisters Read Midterm Performance Evaluation Report 16 interventions make them very attractive to students and to parents. In particular, even though the tablets were used in each school only once during the school year, they clearly served as a highly motivational tool with students. OSR has been successful in providing students with opportunities and materials to read. However, it remains unclear how much of that enthusiasm is sustained and acted upon outside of OSR interventions. Though some of the interviewed students mentioned that their parents buy them books, other respondents did not corroborate these statements. Furthermore, few students in the centers’ catchment areas are actually visiting the centers. EQ 1b: Has teacher training improved classroom practices? Findings OSR developed 18 reading activities, each adapted to the three grade levels (CI/CP, CE1/CE2, CM1/CM2). According to the November 2015 biweekly report, the inspectorate validated these during a three-day workshop, which the OSR literacy expert led and attended by four inspectors, 56 teachers and the OSR staff. As USAID explained, and given the restrictions on access to schools, these reading activities are regarded as extra-curricular activities and it is not clear that they are used during the day as part of the standard curriculum. Per the theory of change for the OSR activity presented in Table 2, a critical activity and output of OSR is the training of teachers in reading methodologies to improve student reading skills (objective 1 in the results framework). OSR trained teachers in 6 of the 18 reading activities according to school directors and teachers. Respondents at OSR and in schools confirmed the participation of school directors and some teachers at these training sessions. The FP director noted that in one third of the schools, only the school director received training, while in the other schools, the school director and a deputy were trained. At all eight of the schools visited, teachers and school directors confirmed that one or two teachers had been trained alongside the school director. The FP director explained that, due to budgetary limitations, the activity utilized a “training of trainers” approach, where those trained were then tasked with replicating the training in their schools using a cascade model. While OSR’s training was viewed favorably by the inspector, school directors and teachers all recommended that OSR provide training to all teachers within a school, not only to a select few. The FP director recognized that given the traditional approach to teaching and working with students, it is difficult for those who were not directly OSR trained to engage with its play-based interventions. According to the indicator table in the year 3 quarter 2 report, OSR has trained 314 teachers and school directors in Sokone and Toubacouta (see Table 3 below). “The students are much more motivated because they have before them the literate environment that is needed, and these are the books.” − Moustapaha Diouf, Education inspector for Sokone and Toubacouta Our Sisters Read Midterm Performance Evaluation Report 17 Table 3: Number of teachers trained Year 1 Year 2 Year 3 Total SOKONE TOUBACOUTA TOTAL SOKONE TOUBACOUTA TOTAL SOKONE TOUBACOUTA TOTAL M F M F M F M F M F M F 37 42 0 0 79 98 52 0 0 150 0 0 61 12 73 302 The interviewed school directors, teachers and the inspector expressed that there was congruence between some of the reading activities and the national curriculum. The most often cited example of that is the letter (la lettre) reading activity. When asked about the integration of the OSR activities into the daily classroom schedule, respondents provided mixed answers. For some teachers and school directors, the activities are easily implemented during school hours, while for others the current school program is overloaded and does not allow for additional activities. Nevertheless, the teachers interviewed did confirm that the activities were implemented with the students, either during the school day or after school hours. When probed further about the implementation of the reading activities, respondents again provided mixed responses. Throughout the evaluation interviews, some teachers and school directors were adamant that the teachers themselves led the reading activities, with some support from the OSR outreach officers, while others explained that the OSR outreach officers led all of the reading activities with the students. For his part, the inspector for Sokone and Toubacouta argued that the use of the outreach officers by OSR undermines the teachers’ ability to fully utilize the reading activities since the officers are the ones who implement them. Finally, the interviewed school directors and inspector pointed to the lack of OSR school and classroom monitoring as a factor in the inconsistent application of the reading activities by teachers. They expressed that teachers needed to be motivated to implement the reading activities through proper training and regular monitoring. They have suggested using the School Directors’ Collective (CODEC) structure and “cellule d’animation pédagogique” (teacher training units) for training and monitoring of targeted teachers. Conclusions While stakeholders appreciated the OSR activities, the evaluation team was unable to assess the level of utilization of the reading activities by the teachers to determine any possible influence on classroom practices. Though some of the reading activities are aligned with the curriculum, a lack of training on how to integrate the reading activities into the normal school day has reinforced the notion that they are mainly extra￾curricular. The OSR training did not focus on pedagogy or methods for the teaching of reading. Transferring active teaching methods, which may be integrated into the reading activities, but which are not made explicit to the teachers, is a difficult task that is unlikely to happen in the absence of continuous guidance and coaching. Our Sisters Read Midterm Performance Evaluation Report 18 EQ 1c: Have activity interventions, such as adult literacy, community meetings, mentoring by teachers, etc., increased community mobilization? Which interventions were most effective? Findings When the evaluation team requested a meeting with eight community members in Diossong and Médina Sangako, over 30 people came to each meeting to express their appreciation for OSR. Throughout the evaluation interviews, community leaders and parents expressed great enthusiasm for OSR. All community members interviewed stated that they had participated in meetings with OSR and supported their efforts. According to project records, 1,387 people (800 women, 587 men) attended five community meetings in the second quarter of year 3. WGEP and community members, including the mayors, attributed the success of the community mobilization strategy to the fact that OSR started by first engaging with key community leaders such as the village chiefs, imams and women’s associations. The FP director explained that OSR deliberately targeted leaders, traditional chiefs, and religious leaders, to solicit their support prior to engaging with the community. Community leaders served as entry points in all OSR intervention villages. WGEP and the FP director agree that OSR’s ability to gain access to these leaders was in large part due to the fact that FP and its director are themselves community members who are very well respected and valued. Community members interviewed cited the literacy classes as the most impactful OSR community intervention. There is consensus among the respondents, including school directors and teachers, that the literacy classes have helped women learn numbers and letters, knowledge they now use in both their income-generating activities and with their mobile phones. School directors, teachers and community members interviewed also claimed that mothers are now better equipped to support their children with their lessons. Several teachers explained that parents are now ensuring that students attend the reading activities and tutoring classes. They also pointed out that mothers are now able to understand students’ marks and monitor their performance. Community members also mentioned that they have participated in the community meetings organized by OSR. They explained that these meetings are meant to sensitize the community about subjects such as the importance of education and reading, the importance of keeping girls in school and reduction of early marriages. Conclusions OSR strengthened parent and community support for reading and literacy, further evidenced by the high participation rate of community members in the meetings with the evaluation team. Activity interventions have successfully mobilized community members, particularly women, who are now using basic literacy skills in commerce. The most successful strategies employed by WGEP and FP for community mobilization are: 1) engaging key community leaders prior to launching interventions, 2) focusing on women to garner support for education and 3) providing adult literacy classes. “Truly they [OSR] are present in the community and what they have gained [from the project] they have given back to the people, they have shared it with the community.” − Abdoul Aziz Dème, Imam Sokone Our Sisters Read Midterm Performance Evaluation Report 19 EQ 1d: What is the level of community engagement in this activity? Why? Findings Community engagement in the OSR activity is documented in reports via the participation rate of community members in OSR interventions such as community meetings, adult literacy, and use of the ICT centers. In the July 1 biweekly report, OSR notes that 150 community members participated in a meeting at the Sokone center during which they discussed potential loss of student reading skills over the summer, as well as OSR summer reading interventions. From its outset, the activity entered into communities through traditional, community-based authorities, such as the village chiefs and imams. This approach resulted in a wide range of support from various sectors of the community. In addition, each community has a designated liaison with OSR. This person is a volunteer who is responsible for organizing the community meetings and adult literacy classes. Formal political support is clear from both Mayors in Toubacouta and Sokone. The mayor of Toubacouta agreed to a long-term lease with OSR for a public building that could accommodate the ICT center. According to WGEP and FP, the mayor is paying the water and electricity bills for the center. Community members interviewed stated that they conduct door-to-door campaigns to promote the enrollment of students in Grade 1. School directors and teachers interviewed confirmed that there was an increase in the enrollment of students in Grade 1. In interviews with the inspector for Sokone and Toubacouta, the school director at Niokholokho, the director of the CODEC, and the FP director, all named Médina Sangako as an example of community organization to improve students’ reading performance. According to the school director at Médina Sangako, following a zero passing rate in the Grade 6 exams last year, the women in the community reached out to the FP director for help in improving their students’ school performance. The women organized 13 tutoring clubs and a calendar that identifies the students to be tutored, the tutor (an older student), and the location. Parents committed to contributing 1,000CFA per week and OSR lent solar lanterns. According to the teachers at Médina Sangako, though the tutoring is organized and done by the community, they also participate to evaluate each month what progress has been made. According to the school director, this year, during the departmental preparation tests for the Grade 6 exams, 12 to 15 students earned passing marks. Conclusions The majority of communities that participate in the OSR activity are mobilized and engaged to support education in their communities. Community members participate in large numbers in community meetings designed to address specific themes such as the importance of girls’ education. They also attend literacy classes, which are led by the community liaison. Of their own initiative, communities have also organized school enrolment campaigns and tutoring clubs in support of struggling students. EQ 2: To what degree has the implementing partner (including the subcontractor) achieved the activity’s objectives? Findings The program description included in the original cooperative agreement for OSR notes that the activity will, “create strong readers and lifetime learners among rural Senegalese primary school students, Our Sisters Read Midterm Performance Evaluation Report 20 especially girls.12” WGEP and FP proposed to reach this goal by achieving three objectives, shown in Figure 5 below. Figure 5: OSR Results Framework Objective 1: Build foundational reading skills for 6,800 rural primary school students (at least 50% female) by creating access to technology centers, mobile labs and trained literacy professionals by 2017 WGEP provided the evaluation team the raw data sets for the reading tests conducted in November 2015 (pre-test) in 24 schools in Toubacouta and in June 2016 (post-test). Twenty-four schools were selected in Sokone (12) and in Toubacouta (12) for the post-test. In order to assess gains in reading performance, the evaluation team matched the Toubacouta schools that participated in the post-test to their pre-test, baseline data. In total, 11 schools in Toubacouta have pre- and post-test data. One school in Toubacouta, Sadio Sala, was selected for the post-test but did not participate in the pre-test and was thus eliminated from the analysis. Since the activity did not collect pre-test data for the Sokone schools, the post-test data was not used in the analysis of student gains. The mean scores for the reading out loud (oralisation) and reading comprehension subtasks per grade at pre- and post-test are presented in Table 4 and in Figures 6 and 7 below. In all grades, students performed better at post-test than at pre-test in reading aloud. However, in Grades 2, 4 and 6, students achieved lower mean scores in reading comprehension at post-test than at pre-test. 12 OSR Results Framework Our Sisters Read Midterm Performance Evaluation Report 21 Table 4: Student Reading Mean Scores per Grade Grade Pre-test Post-test Difference Grade 1 (CI) Reading 1.03 9.47 8.44 Comprehension 1.89 3.88 1.99 Grade 2 (CP) Reading 4.68 9.38 4.70 Comprehension 8.08 7.89 -0.19 Grade 3 (CE1) Reading 3.73 5.12 1.39 Comprehension 2.51 3.92 1.41 Grade 4 (CE2) Reading 4.18 5.96 1.78 Comprehension 5.51 3.66 -1.85 Grade 5 (CM1) Reading 3.38 5.25 1.86 Comprehension 2.25 7.30 5.06 Grade 6 (CM2) Reading 4.37 6.22 1.85 Comprehension 7.07 3.95 -3.12 In the Table 5 below the mean scores for the reading out loud subtask are presented by gender and grade. Boys performed better than girls on pre- and post-tests in Grades 3 and 6, while girls performed better than boys in Grades 1, 2 and 4. In Grade 5, boys (1.94) achieved higher gains than girls (1.78) from pre￾test to post-test. Note that the differences in performance between boys and girls are not statistically significant. Figure 6: Mean Scores in Reading Out Loud Figure 7: Mean Scores in Reading Comprehension Our Sisters Read Midterm Performance Evaluation Report 22 Table 5: Mean Scores in Reading by Gender and Grade Grade Boy Girl Difference Grade 1 (CI) Pre-test 0.87 1.17 0.30 Post-test 9.16 9.76 0.60 Grade 2 (CP) Pre-test 4.48 4.93 0.45 Post-test 8.66 10.11 1.45 Grade 3 (CE1) Pre-test 3.89 3.58 -0.31 Post-test 5.26 4.99 -0.27 Grade 4 (CE2) Pre-test 4.17 4.20 0.03 Post-test 5.72 6.23 0.51 Grade 5 (CM1) Pre-test 3.34 3.43 0.09 Post-test 5.28 5.21 -0.06 Grade 6 (CM2) Pre-test 4.63 4.09 -0.54 Post-test 6.43 6.01 -0.42 In Table 6 (below), the mean scores for the reading comprehension subtask are presented by gender and grade. Boys performed better than girls at pre-test and post-test in Grades 1, 4 and 5, while girls performed better than boys in Grade 2. In Grade 3, girls (1.67) achieved higher gains than boys (1.13) from pre-test to post-test. In Grade 6, both boys and girls achieved lower average scores at post-test than at pre-test. Table 6: Mean Scores in Reading Comprehension by Gender and Grade Grade Boy Girl Difference Grade 1 (CI) Pre-test 1.96 1.83 -0.12 Post-test 4.04 3.73 -0.31 Grade 2 (CP) Pre-test 7.44 8.73 1.29 Post-test 7.45 8.28 0.82 Grade 3 (CE1) Pre-test 2.62 2.41 -0.21 Post-test 3.75 4.08 0.33 Grade 4 (CE2) Pre-test 5.68 5.32 -0.37 Post-test 3.71 3.60 -0.11 Grade 5 (CM1) Pre-test 2.25 2.24 -0.01 Post-test 7.37 7.23 -0.14 Grade 6 (CM2) Pre-test 7.26 6.86 -0.40 Post-test 3.85 4.05 0.20 The annual report for OSR for year 2 notes that 4,145 students, of which 2,119 were girls, received the OSR reading interventions in the target schools in Sokone. At the time of this evaluation, the cumulative total of students receiving OSR reading interventions in schools in both Sokone and Toubacouta between January 2016, when interventions started in Toubacouta schools, and June 2016 was not available. However, in the June 1, 2016 biweekly report, OSR reports that 3,613 students (1,796 girls) in both areas participated in the reading activities during the month of May. Our Sisters Read Midterm Performance Evaluation Report 23 According to OSR’s July 1 2016 biweekly report, 295 students had voluntarily visited the Sokone center since it opened in May 2016. In Toubacouta, 199 students had visited the center in May 2016, according to the June 1, 2016 biweekly report for OSR. The July 1, 2016 biweekly report also notes that 191 students had borrowed books from the Sokone and Toubacouta centers during the month of June. The two mobile libraries are also functional. One services the 28 intervention schools in Sokone while the other services the 24 intervention schools in Toubacouta. In the June 1 biweekly report, OSR reported that the mobile libraries had visited six schools during the month May and that students had borrowed a total of 668 books from the mobile libraries. Figure 813 below provides a summary of the number of schools visited by the mobile libraries between January and May 2016 and the number of books borrowed by the students. The number of schools visited is indicated by the blue line and right (secondary) axis. Figure 8: Mobile Libraries Visits to Schools With OSR activity implementation in the 24 Toubacouta schools, the OSR outreach team has grown to eight, with four officers in Sokone and four officers in Toubacouta. The outreach officers are tasked with implementing the reading interventions in the 52 OSR target schools with teachers and students in Grades 1 to 6. In the June 1, 2016 biweekly report, OSR reported that their outreach officers had visited 13 schools during the second half of May. According the outreach officers interviewed, the OSR literacy expert trained them and school directors and teachers on the OSR reading activities. However, both the inspectors for Sokone and Toubacouta, and the FP director noted during the interviews that the outreach officers were not sufficiently trained in pedagogy. Objective 2: Encourage and motivate students to read through innovative resources and incentives, reaching students at participating schools each year through contests, scholarships, mentorships and book clubs According to WGEP and FP, the annual reading contests, in which all 52 schools participate, are motivating factors for students, schools and communities. The teachers at KGBX and Danthakoune, as well as the 13 The complete data disaggregated by gender are missing for the month of February and thus only includes the number of girls for half the month. The data for April were not disaggregated by gender. 0 2 4 6 8 10 12 14 16 18 20 January February March April May 0 200 400 600 800 1000 1200 1400 1600 1800 2000 Number of schools Number of books Books Borrowed Books Borrowed by Girls Schools Visited Our Sisters Read Midterm Performance Evaluation Report 24 school directors in Médina Sangako and Toubacouta, echoed those sentiments noting that the reading contests create excitement in the students and pique the interest of parents and communities. The teachers and community members interviewed mentioned that the scholarships are also a source of motivation for the students. For the 2015-2016 school year, 139 scholarships were awarded to primary school students and 41 scholarships were awarded to middle school students according to the year 3 quarter 2 OSR report. The scholarships consist of school supplies, remedial reading tutoring and summer reading tutoring. Objective 3: Build communities of parents and leaders that support young girls and boys to increase their reading and technological literacy All community members interviewed complimented the activity’s community mobilization strategy. In particular, respondents reported that the adult literacy classes were well attended by women. In its year 3 second quarter report, OSR reports that the adult literacy classes were offered in 36 sites during the second quarter, up from 28 sites the year before, at the request of community members. The high prevalence of the women in the classes is due to cultural traditions that separate men and women in activities. Conclusions The student performance data indicates that students achieved some gains in reading in all grades. In Grades 2, 4 and 6, students achieved lower scores in reading comprehension at post-test than at pre-test. Boys and girls achieved very similar scores at pre-test and at post-test in both reading out loud and in reading comprehension. Given the concerns about the validity and reliability of the student performance data, the evaluation team cannot make a determination as to whether the observed gains are due to the impact of the OSR interventions or to natural growth (the expectation that students would demonstrate gains independently of the interventions). OSR has surpassed it goal to provide access to ICT centers and mobile libraries to 6,800 students in the Sokone and Toubacouta areas. In the year 3 quarter 2 report, OSR reports that it has reached 8,077 unique students, 4,176 of whom are girls. However, based on the current training model for teachers and outreach officers, the activity is not on track to provide trained literacy professionals by 2017. While the activity has provided training to some teachers and to its outreach officers, the training has focused on how to execute a select few OSR reading activities and did not include training on literacy teaching methods and pedagogy. Community mobilization is the most accomplished intervention by OSR. Parents and community leaders regularly attend sensitization meetings and women are enthusiastic about the adult literacy classes. EQ 2a: Was the activity implemented according to the design? Has the implementation of the activity followed the original design? Findings To reach the three activity objectives, the grantees proposed a series of interventions that "will engage and support primary-aged school children, their families and communities to create and strengthen a culture of reading while reducing the gender gap in reading and education14". In the original design, WGEP and FP proposed to work in 60 schools and communities. At the time of this evaluation, the activity is 14 OSR 2015 Annual Report Our Sisters Read Midterm Performance Evaluation Report 25 undergoing implementation in 52 schools and communities, per the 2014 Performance Monitoring Plan (PMP). Activity documents, site visits and interviews with beneficiaries confirm that the majority of proposed activities in the original design were implemented. These include literacy classes and community meetings, functioning libraries and ICT centers in Sokone and in Toubacouta, two mobile libraries, solar lanterns, reading activities led by teachers and outreach officers, annual reading competitions and scholarship packages for students in elementary and middle schools. Several planned interventions could not be verified because of a lack of available data. These include the training of teachers in reading methodologies and the use of ICT in their classrooms, the book clubs and book days, book friends/mentors, and parent workshops. According to the FP director and WGEP, one of the biggest variations from the original activity design is the access to the targeted schools. School directors confirmed that the OSR activity can visit schools twice a week, on Tuesdays and Thursdays after school hours, and not on a daily basis as would have been optimal. According to WGEP, they would have preferred to visit the schools "whenever we wanted", but instead they had to review their plans once the inspectorate limited the availability of the schools and teachers. Another variation from the original design occurred with the sequence and schedule for the implementation of the physical and mobile libraries. Activity documents and respondents confirm that the library and ICT centers opened in 2016, and not in 2013 as originally planned. Similarly, one mobile library was to begin visits to villages in quarter four of 2013, concurrently or soon after the opening of the physical libraries. According to the WGEP, the delays with the centers provided an opportunity to rethink the design and to begin lending books in the villages with the mobile libraries, prior to the opening of the physical libraries. The original design proposes tutoring support by older students to primary age students who require extra help. Throughout the evaluation interviews, it became quickly evident that there was some confusion about the implementation and organization of the tutoring program. Respondents used the French terms for tutoring (tutorat) and remedial support (encadrement) interchangeably. The FP director stated that remedial classes are provided to the 139 primary school scholarship recipients only. Scholarship recipients are chosen by the school director from students who are most at risk of dropping out and only include students in Grades 1 and 2 (CI and CP). However, the OSR year 3 quarter 2 report states that 680 students participated in remedial classes at a rate of 10 students per class. While the data is inconsistent regarding student participation in the remedial classes, the FP director explained that teachers are paid 25,000 CFA to lead these remedial classes. Tutoring, on the other hand, refers to support for struggling students by the community and/or older students with no remuneration. The evaluation team met with one high school graduate who tutored a young girl in Sokone. The tutor explained that the FP director, with whom she had a previous connection from the WGEP/FP project "Nos soeurs à l'école" (Our Sisters to School), had recruited her. Community members at Médina Sangako explained that they had reached out to the FP director to help them organize a tutoring program when the school achieved a zero passing rate at the Grade 6 exams last year. Whether through tutoring or remedial classes, community members, school directors, teachers and the tutor interviewed testified that students who received the extra support demonstrated improved performance in reading and in other subjects. But while the extra support provided seems to have yielded positive results, the FP director expressed concerns that the activity is relying on paid teachers. In her opinion, once funding ceases, teachers will stop providing the extra support. She has suggested that the OSR activity implement more of the community-led tutoring model of support and discontinue the Our Sisters Read Midterm Performance Evaluation Report 26 remedial classes in 2017. She believes the tutoring model is more sustainable but recognizes that tutors would need training in pedagogy, which they currently are not receiving from the activity. Though not included in the original design, FP has helped women who participate in its literacy classes to access microfinance loans from local institutions such as Crédit Mutuel. Community members interviewed and the FP director provided multiple examples of women who were able to receive small loans to finance small businesses. Conclusions The majority of the interventions from the original activity design were implemented as proposed. However, there were some delays to the implementation schedule, particularly with the mobile libraries and ICT centers, and some appropriate modifications to the proposed approaches. Most critically, the unexpected restricted access to the intervention schools has limited the activity's ability to influence classroom practices and to provide regular opportunities for students in remote villages to engage in meaningful reading activities. Additionally, changing the approach for extra support to struggling students from a volunteer and community-based endeavor to one led by paid teachers seems to diverge from WGEP's and FP's core approaches to development, which it has as a core tenet of community engagement. EQ 2b: What internal or external factors (including within USAID) have had the biggest impact on the implementation of the activity? Which factors may continue to impact the activity and which ones seem to have had short-term/specific impact? Findings Interviewees cited several factors as having had an important impact on the implementation of the activity. These factors influenced the timeline of the implementation and the approaches to the interventions. Table 7 below summarizes the identified external and internal factors that have had an impact and that may continue to have an impact on the implementation of the OSR activity. Table 7: Factors that Impact Implementation Collaboration with the education authorities As mentioned above, one of the biggest changes to the design of the activity resulted from the limits imposed initially by the education inspectorate. According to WGEP, it was important that project implementers respected local politics and protocols at the launch of the activity and throughout the implementation of the interventions. This included engaging village leaders and the education inspectors prior to engaging communities and schools. However, according to the FP director, the inspectorate would not initially allow an outside consultant to train teachers to put into practice new, non-validated, External Factors Internal Factors Factors that continue to impact implementation 1. Collaboration with the education inspectorate 2. Buy-in from village leaders 3. Support to women/mothers 1. Recognized and valued local leadership 2. Limited human resources 3. Knowledge of USAID rules and regulations Factors that have short￾term/specific impact on the implementation 1. Teacher strikes Our Sisters Read Midterm Performance Evaluation Report 27 reading activities in the schools. The activity eventually signed an agreement with the inspectorate in 2014. Inspectors co-trained teachers and school directors on the new and validated reading activities and set the school visit schedule which limits the OSR staff to two days a week in all schools. Given the localized nature of the activity, there was limited interaction with the National Ministry of Education (MEN). Buy-in from village leaders All community groups interviewed pointed to the fact that the activity began its implementation by engaging first with village leaders. According to the FP director, the OSR team started the implementation process by organizing meetings with village leaders, imams and marabouts prior to interventions. Facilitating partners organized initial meetings to introduce the activity and solicit support. Once village leaders were on board, community meetings, literacy classes and school-based activities took place. WGEP explained that while the time spent gaining buy-in from the village chiefs was difficult at times, it was crucial to ultimately be able to navigate local politics. Support to women/mothers Respondents explained that the support provided by the activity to women has resulted in improved literacy skills, which women are now using to manipulate mobile phones and to make calculations for commerce. In Médina Sangako, community members interviewed explained, “women are the engine for development”; they manage and initiate all activities. Teachers in Karang Socé further explained that the literacy classes are beneficial because once a mother is literate the entire community is literate. These teachers and the FP director explained that mothers are the main caregivers to children; they spend the most amount of time with them and are thus better placed to provide support at home and to encourage reading. Teacher strikes The IEF inspector for Sokone and Toubacouta cites the teacher strikes as one of the major factors to have had an impact on the implementation of the activity. The teacher strikes affected all public OSR schools and lasted for a total of 48 schools days. According to WGEP, the FP director and the school directors interviewed, the teacher strikes impeded implementation for several months and teachers were not able to put into practice the reading activities as planned. Recognized and valued local leadership The mayors, school directors and community members interviewed noted that the FP director was instrumental in the implementation of the OSR activity. Also, WGEP explained that the FP director enjoys a “nice reputation” within the community. The mayor for Sokone said that he was very impressed with the OSR activity and as such had named the FP director as a special counsel to the municipality for girls’ education. Limited human resources WGEP has provided several training sessions to the OSR staff, including trainings for the librarians at Alliance Française in Dakar and training for the outreach officers on the reading activities. However, the FP director noted that additional training was needed for the OSR team to engage properly with students and community members while the inspector mentioned that the OSR outreach officers lacked pedagogical training. USAID/OFM conducted an organizational capacity assessment (OCA) of FP, which highlighted the need for a qualified financial manager for FP as well as a deputy to the director. When “Community mobilization is the approach for this activity, meaning that it started with basic local structures…. Everyone, everyone is engaged.” − Abdoulaye Mbaye, Former Mayor and Parent Our Sisters Read Midterm Performance Evaluation Report 28 asked about staffing needs, the FP director mentioned that OSR activities are difficult to implement in her absence since no other staff member has similar administrative or technical knowledge and authority. Knowledge of USAID rules and regulations WGEP and the FP director expressed some frustrations with the lack of guidance from USAID, particularly in the first year of the activity. The FP director explained that the OSR team received very limited training on USAID financial procedures, and WGEP noted that they “learned by doing”. The implementers mentioned the tax exemption status and the problems with advances and reconciliations as examples of areas where there was a lack of understanding, which provoked delays in the implementation of the activity. Conclusions The implementers were able to capitalize on internal factors, such as the reputation of the FP director, to gain access to and buy-in from key local stakeholders at the onset of the activity. Engaging the community leaders and women early on served as an important springboard for launching community-based activities and for garnering support for other aspects of the activity. Though OSR relies heavily on community mobilization to achieve its results, the initial difficult collaboration with the inspectorate at the start of the activity illustrates well as to why community-based activities which seek to intervene in schools must also engage very early on with the proper education authorities. Working on issues of in-classroom changes to teaching approaches also require interface with the MEN. The activity’s relationship with the local education authorities, community leaders, and women continues to have an impact on its implementation as these actors decide who may participate and to what degree. The inspectorate has the authority to limit or expand access to schools. Its decisions influence greatly how often the activity interacts with school directors, teachers and students, and thus its ability to improve classroom practices and student reading performance. Mothers, on the other hand, organize their children’s day and decide on the amount of time that can be devoted to reading outside of school instead of working in the fields or in the homes. Capitalizing on the reputation of the FP director, their experience in the area, and their understanding of local protocols, WGEP and FP deliberately planned a very successful community outreach program. However, the implementation of the OSR activity continues to be heavily dependent on the FP director even as it expands to new villages and incorporates new activities. This overreliance on the director limits her ability to interact with and be responsive to the communities, and to provide needed training to team members and to provide quality assurance for the interventions. EQ 2c: How did the implementing partners and the USAID education team address challenges during the implementation of the activity and what are some lessons learned? Findings Implementers and USAID staff mentioned several challenges met during implementation of the OSR activity during the interviews with the evaluation team. The biggest challenges, their resolutions, and the lessons learned are presented below. Our Sisters Read Midterm Performance Evaluation Report 29 1. Access to schools Challenge: The inspectorate initially refused to allow teachers and school directors to participate in the training sessions organized by the activity. The inspectorate also refused to allow the activity to implement interventions in the schools. Resolution: According to the FP director, the inspector strike provided the opening the activity needed to access schools. During the strike, the activity reached out to school directors with whom the FP had previously established a working relationship through its “Nos soeurs à l’école” activity. The school directors responded positively, which gave OSR an opportunity to implement the reading activities with teachers and students in the schools. The implementation in these initial schools provided the activity the data needed for its negotiations with the inspectorate. Ultimately, the inspectorate, impressed by the testimony of the participating schools and by the student results, agreed to the OSR training plan and to the implementation of the reading activities in the schools. Lesson Learned: Community engagement programs centered on education need to engage local education authorities early in the planning stages. 2. Training of teachers Challenge: The activity budget did not permit for all teachers to be trained directly by the activity’s literacy expert in the reading activities. Resolution: Implementers used a cascade model. The OSR literacy expert trained school directors, the inspectors, and one or two teachers in each school. Trained teams were then tasked with replicating the training for other colleagues at the schools. According to the inspector, school directors, and teachers interviewed, the training model was not implemented as planned. In some schools, teachers explained that their colleagues were debriefed on the training received, while in others colleagues were provided one day of training. Schools directors and teachers also mentioned that some colleagues who had not participated in the training were reluctant to implement the reading activities, as they did not feel it was applicable to them. Lesson Learned: For this activity, the cascade model can be used, but should involve formal training to all teachers in a given school to ensure quality, consistency and buy-in from the entire school team. 3. Construction of the ICT centers Challenge: WGEP encountered challenges during the procurement of land and the construction of the ICT centers. According to WGEP, the first identified parcel of land provided by the Mayor of Sokone wasn’t large enough for their purposes and the purchase of the second parcel fell through. Once the procurement was completed on another parcel, the construction of the building took longer than planned because of the contractor’s poor performance. Resolution: According to activity documents, WGEP imposed fees on the contractor for every day the construction was delayed beyond the original timeline. In Toubacouta, the activity negotiated with the mayor a long-term lease on an available building. The municipality in Toubacouta is funding the costs of water and electricity at that center. Lesson Learned: There needs to be a clear legal and political strategy for the procurement of land by implementers. New implementing partners with no construction experience require additional support from USAID. An experienced manager should manage constructions locally. 4. USAID rules and regulations Challenge: According to WGEP and FP, there were some implementation delays as they waited for the approval of WGEP’s tax-exempt status from the Government of Senegal. In addition, there was Our Sisters Read Midterm Performance Evaluation Report 30 misunderstandings around the process for advances and reconciliations, which resulted in USAID denying additional requests for funds by WGEP. Resolution: The current Agreement Officer Representative (AOR) shared the USAID tax exemption documents with WGEP and FP, which allowed them to clear from customs equipment and materials for the ICT centers and libraries. Following guidance from USAID/OFM, WGEP was able to reconcile outstanding advances and to receive additional funds to continue implementation. Lesson Learned: New implementing partners require additional orientation and continuous support to help them navigate USAID rules, regulations and resources. EQ 3: How has the capacity of the implementing partner (including the subcontractor) changed over the course of the activity? What gaps remain? Findings The Our Sisters Read contract was issued to Women’s Global Education Project as part of the Development Grants Program at USAID. The program, now called the Small Grants Program, seeks to establish partnerships with small organizations that have no, or very little, direct working experience with the agency. A core objective of the program is for USAID Missions to provide partner organizations "access to technical assistance to improve their planning and managements systems and other competencies needed to successfully implement USAID-funded activities"15. USAID Missions are thus tasked with providing capacity-building support to enhance implementing partners' organizational and technical capacities. WGEP expressed an expectation of capacity building through this activity. However, they felt that they had to "learn by doing" since they have received little capacity building support on how to manage USAID grants. Interviewees presented two situations as illustrations of this lack of guidance. Both WGEP and USAID/OFM cited initial difficulties with the management of advances and reconciliations, which created large balances and raised concerns about the burn rate and budgeting for the activity. In addition, WGEP and FP were not aware that they could use the USAID blanket tax exemption for procurements. They only came to understand this in December 2015 after having initiated a lengthy process to obtain tax-exempt status from the GOS. WGEP explained that, during the first year of the activity, they had been assigned four different agreement officer representatives (AORs), which may explain the gaps in guidance. Upon review of the agreement between USAID and WGEP and other activity documents, including budgets and annual work plans, the evaluation team did not find any mention of plans or funds to support the organizational capacity development of WGEP or FP. When questioned about capacity building as part of OSR, interviewees focused on the support provided to FP. WGEP explained that capacity building is an integral part of their approach. WGEP and the FP director provided multiple examples of instances of training provided to FP and the OSR local staff. Several training sessions geared towards improving activity implementation were provided by WGEP, including training on literacy activities for the outreach officers and community liaisons (relais), planning and 15 http://apply07.grants.gov/apply/opportunities/instructions/oppM-OAA-GRO-ALPS-09-001-LOCAL-cfda98.001-cidM-OAA￾GRO-ALPS-09-001-LOCAL-instructions.pdf “The award that we received was part of the Development Grants Program…it is specifically geared towards small organizations…like ours and I would submit that sometimes that is lost sight of from the USAID side.” − LILY MESSIH, WGEP Our Sisters Read Midterm Performance Evaluation Report 31 scheduling of interventions, using spreadsheets, data monitoring and reporting, and understanding budget line items, amongst others. Also, interviewees mentioned USAID-led training sessions for FP on financial management. USAID/OFM traveled to Sokone in May 2014 to conduct the first OCA for FP. However, upon realizing that basic systems and processes for financial and administrative management were missing, the USAID team provided instead a training which included recommendations for organizational and management improvements. In October 2015, USAID/OFM returned to Sokone and conducted the first OCA for FP. USAID/OFM noted that approximately 30 percent of the recommendations from May 2014 had been completed by then. Important gaps remained, particularly in terms of human resources policies and manuals, filing systems and financial management. USAID/OFM noted that FP and the activity relied heavily on its director and that FP did not have a qualified financial manager at the time. According to WGEP and the FP director, it has been difficult to find a qualified financial manager from the Sokone area who understands USAID regulations. At the time of this evaluation, WGEP and FP were recruiting the fourth financial manager for OSR. In addition to the recruitment of the financial manager, WGEP cited the development of a procurement policy, human resources policies and manuals and the implementation of systems as actions that have been taken in response to the 2015 OCA16. FP and WGEP also noted that the OSR staff has increased to 15 in 2016 which has required, and continues to require, training and capacity building. As noted in the limitations section of this report, the evaluation team discovered several gaps in the data provided for this activity. For example, several fields were left incomplete in the data sets received with the student performance data. Beneficiary data, such as visits to centers, are not always disaggregated by gender. The DQA conducted by USAID in March 2016 also cited several concerns with the monitoring data as reported by WGEP and FP. These concerns include instances of double counting of beneficiaries, the absence of supporting documents, inadequate data collection forms, a lack of manual for data collection and management, and a lack of data quality assurance procedures. For the FP director there remains important training needs for the OSR staff in communication for behavior change, remediation and pedagogy. The FP director noted that she has included in her reports to WGEP that she and the OSR team are in need of training in management, financial management and budgeting. The FP director explained that her long-term plans for FP are ambitious. With the training and guidance received from USAID/OFM, she explained that FP is working to put in place systems to improve the capacity of its members to better manage programs. Her priorities for FP include the establishment of a board of directors, registration as a non-governmental organization (NGO), training in fundraising and making the organization more visible. For WGEP, capacity building of FP involves ensuring that the OSR interventions are sustainable. WGEP believes that hiring staff locally and providing them with skills training will ensure sustainability as well as lend legitimacy to the activity. WGEP also explained that during the remaining year of the activity, they will work with FP to turn the ICT centers into community centers where resources, such as the internet, can be used to generate revenue and cover part of the operating expenses. Conclusions Through the implementation of OSR, Femmes Plus (FP) has slowly gained skills and knowledge to improve the implementation of its most important program, OSR. Yet, capacity building of an organization generally 16 USAID/OFM is scheduled to return to Sokone for a follow-up on the OCA on July 18, 2016. Our Sisters Read Midterm Performance Evaluation Report 32 aims to improve its ability to fulfill its mission and sustain itself in the future. Capacity building should ideally strengthen an organization's management policies and systems, and not merely its programs. There is an important disparity between WGEP and FP in their vision and long-term plans for FP. While WGEP seems to be mainly focused on planning for the sustainability of OSR interventions, the FP director is looking to strengthen the organization's technical, financial and management systems for its long-term health. Capacity building is not understood and thought of in the same manner by these two partner organizations. One partner, WGEP, is focused on strengthening the technical and programmatic capacity of FP to consolidate, and potentially expand to new geographic areas, gains made under OSR. The other partner, FP, is looking to capitalize on the technical and organizational training received as part of OSR to improve its ability to effectively deliver its mission at the conclusion of OSR and over time. This disparity in vision has consequences on capacity building plans. As noted by the FP director and USAID/OFM, OSR and FP rely heavily on their local director. This overreliance on the FP director limits the implementation of OSR interventions and is detrimental to future programmatic plans. The FP director is highly regarded in the community, by WGEP and USAID. She is responsive to communities but expressed frustration at her inability to work more closely with the Toubacouta center or to visit all of the intervention schools. And while it is commendable that the OSR team has increased in size and that training continues to be provided to improve work products, none of the new hires are skilled enough to provide the type of programmatic and management leadership that is required for a growing organization. The FP director remains the sole technical and management leader in the Sokone office. In keeping with their approach to development, WGEP and FP rely heavily on locally recruited staff, interns and volunteers to implement the OSR activity. However, the pool of available and qualified candidates is very restricted which negatively affects the implementation of interventions and organizational development, particular in regards to strong financial and data management systems. WGEP has not received sufficient and timely guidance on how to manage USAID contracts. For this new grantee, the lack of early guidance meant that they had to learn USAID rules and regulations as they were implementing the activity, which led to miscommunication, frustrations and implementation delays. RECOMMENDATIONS The following recommendations are based on the findings and conclusions presented in this report and are organized for the consideration of key stakeholders. WGEP 1. Education projects, regardless of size and strategy, should coordinate early and closely with the Ministry of Education, at the central, regional and district levels, around curriculum reforms related to reading. 2. WGEP, FP and USAID/Senegal should develop an institutional capacity development plan with a timeline, targets and allocated budget. 3. WGEP and Femmes Plus should hire a qualified financial manager from outside the immediate Sokone and Toubacouta areas. A qualified financial manager will allow Femmes Plus and WGEP to establish proper procedures and systems per the OCA recommendations and action plan. The financial manager can be recruited on a temporary basis to respond to immediate needs and to train a locally hired manager who will eventually take on financial management responsibilities for the OSR activity and for Femmes Plus in the long-term. Our Sisters Read Midterm Performance Evaluation Report 33 4. Similarly, WGEP and Femmes Plus should hire a qualified monitoring and evaluation officer to help set up systems for appropriate data collection, verification and management. As WGEP is currently recruiting a new Field Coordinator, it should consider a candidate with strong M&E experience. 5. As the FP director noted, the schools with easier access to the ICT centers are “spoiled”. OSR should review the planning of school visits by the mobile libraries to prioritize schools that are furthest from the centers. A calendar of visits for schools closer to the centers can be established to ensure that these students regularly visit the centers to borrow books and use other library resources. 6. Though intended as extracurricular activities, OSR should continue to implement the reading activities in remote villages during summer vacations. As much as possible, the mobile library, with the tablets, should visit communities that do not have access to the ICT centers to capitalize on the motivation to read that has been developed in students and reinforce the skills learned throughout the year. 7. For this activity, given the size of the intervention schools and the lack of qualified master trainers, direct training should be provided to all targeted teachers. 8. OSR, and future similar activities, should provide initial training and continuous coaching, or teacher professional development, to all teachers within a school through the CODEC and “cellule d’animation pédagogique”. 9. OSR should produce a short addendum to the teacher guide that provides tips for the integration of the activities throughout the school day and for the transfer of methods to other subjects or lessons. 10. OSR should continue to engage the inspectorate in various aspects of the activity to ensure good coordination with the schools. Delivering training and monitoring services through the CODEC necessarily engages local education officials and increases their stake in its success. 11. OSR should continue to provide adult literacy classes as a key motivator. 12. OSR should replicate, in communities that solicit it, the tutoring models that some communities have initiated. FP 13. Femmes Plus should focus on sustainable organizational capacity development including the retention of a qualified finance manager and deputy director. USAID 14. The community mobilization strategy employed by OSR should serve as a model to other activities that have a community-based component. 15. USAID should provide early guidance on available programmatic and management resources to new grantees, whether located in the US or in Senegal. 16. USAID/Senegal should conduct an initial organizational capacity assessment of first time local grantees within the first six months of a contract. Our Sisters Read Midterm Performance Evaluation Report 34 ANNEX 1: EVALUATION STATEMENT OF WORK ÉVALUATION DES PERFORMANCES DE L’ACTIVITÉ «OUR SISTERS READ» CADRE DE TRAVAIL APRIL 25, 2016 This publication was produced for review by the United States Agency for International Development. It was prepared by Management Systems International, a Tetra Tech Company, for the USAID/Senegal Mission-Wide Monitoring and Evaluation Project. Our Sisters Read Midterm Performance Evaluation Report 35 ÉVALUATION DES PERFORMANCES DE L’ACTIVITÉ « OUR SISTERS READ » CADRE DE TRAVAIL Contracted under AID-685-C-15-00003 USAID Senegal Mission Wide Monitoring and Evaluation Project DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 36 ACRONYMES CDCS Country Development Cooperation Strategy CDT Cadre De Travail COR Contracting Officer’s Representative CV Curriculum Vitae DEC Development Experience Clearinghouse DO Development Objective EGRA Early Grade Reading Assessment IA Inspection d’Académie IEF Inspection de l'Education et de la Formation INEADE Institut national d'Etude et d'Action pour le Développement de l'Education IP Implementation Partners IR Intermediate Result LOE Level Of Effort M&E Monitoring and Evaluation MEN Ministère de l’Education Nationale MEP Monitoring and Evaluation Project ONG Organisation Non Gouvernementale OS Objectif spécifique OSR Our Sisters Read PALME Partenariat pour l'Amélioration de la Lecture et des Mathématiques dans PAQUET Programme d'Amélioration de la Qualité, de l'Equité et de la Transparence PDEF Programme décennal de l'éducation et de la formation PMP Performance Monitoring Plan S&E Suivi & Evaluation SOW Statement of Work TAP Taux d’achèvement du primaire TPM Technical Planning Meeting UNICEF Fonds des Nations unies pour l'enfance USA United States of America USAID United States Agency International for Development WGEP Women’s Global Education Project L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 37 1. INFORMATIONS DE BASE Manager du CDT à l’USAID Badara Sarr, EDU AOR/COR de l’ACTIVITE17 Badara Sarr, EDU Manager du CDT à MEP Moussa Sy, Spécialiste Sénior en Evaluation INTITULE DE L’ACTIVITE Our Sisters Read Date de début et de fin de l’activité Du 9 septembre 2013 au 8 septembre 2017 Subvention/Contrat CA-AID-685-A-13-00003 Montant du financement $1,700,904 Partenaire d’exécution Women’s Global Education Project/Femmes Plus Zone d’intervention Fatick Période à évaluer Du 9 septembre 2013 au présent Date du début et de fin de l’évaluation Du 12 mai 2016 au 29 juillet 2016 2. CONTEXTE DE L’ACTIVITE18 La scolarisation et le maintien des enfants à l’école jusqu’à l’âge de 16 ans, comme le prévoit la législation sénégalaise, reste un défi majeur au Sénégal. Les privations aussi bien en matière d’accès que de maintien des enfants à l’école sont encore réelles, quoique à des proportions variables selon les régions. Au Sénégal, beaucoup d’enfants commencent leur scolarité sans jamais achever le cycle élémentaire. Si dans le domaine de l’accès, la plupart des facteurs qui entravaient l’amélioration des niveaux d’accès ont été bien adressés par les politiques mises en œuvre par l’Etat et ses partenaires, des défis importants restent à être relevés, non seulement pour scolariser les enfants issus des familles les plus pauvres, mais également et surtout pour le maintien à l’école de ces élèves une fois scolarisés, pour leur assurer un parcours scolaire suffisamment long leur permettant d’acquérir les connaissances de base essentielles. En effet, la mise en œuvre du Programme décennal de l'éducation et de la formation (PDEF) a permis d’atteindre, entre autres, des résultats significatifs dans le relèvement du taux d’accès dans l'enseignement primaire. Ce programme a aussi permis l’amélioration de l’indice de parité filles/garçons en faveur des filles, avec une évolution fulgurante à partir de 2000 aboutissant à une inversion de la tendance en 2010. Toutefois, même si l’accès à la scolarité a été considérablement amélioré au Sénégal durant cette période, il reste encore une proportion importante de ces enfants qui éprouvent d’importantes difficultés à s’y maintenir. Au problème de l’accès, s’est donc substitué celui du maintien à l’école, notamment celui des filles. Les déperditions scolaires dont les fortes traductions restent les niveaux élevés d’abandons et de 17 Le concept « Activité » correspond ici à projet. Il devra être entendu dans ce sens dans toutes ses utilisations dans ce document. 18 Ibidem, correspond au sens classique de projet. L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 38 redoublements, sont encore des réalités qui caractérisent le parcours scolaire d’une partie importante d’élèves, en particulier les jeunes filles. En effet, au Sénégal, beaucoup d’enfants commencent leur scolarité sans jamais l’achever. Parmi les différents facteurs qui entravaient la performance scolaire et la rétention figure en bonne place la question de la qualité des apprentissages et les niveaux d’acquisitions des élèves dont la plupart sont liés à des problèmes d’accès aux intrants pédagogiques dus à la pauvreté. Compte tenu du fait que la déperdition scolaire touche davantage les filles que les garçons, notamment dans la transition moyen-secondaire, le Sénégal et plusieurs de ses partenaires ont accordé une attention particulière à la question du maintien des filles dont les taux d’achèvement du primaire (TAP) sont encore très faibles comparés à ceux des garçons. Plus nombreuses que les garçons au départ grâce aux efforts spectaculaires enregistrés dans l’inversion de l’indice de parité en leur faveur, les filles restent encore désavantagées dans le maintien et les performances scolaires. Elles sont plus nombreuses que les garçons à abandonner et à décrocher de l’école au fur et à mesure que l’on progresse du primaire au secondaire. Pour pallier à cette forte déperdition scolaire relevant d’une multitude de facteurs, aussi bien scolaires qu’extra-scolaires, le Sénégal a institué l’obligation ou le droit pour tout enfant de bénéficier d’une scolarité de dix (10) ans au moins (entre les âges de 6 à 16 ans) avec la loi 2004-37. Cependant, en dépit de cette disposition juridique qui visait à réduire la déperdition scolaire et d’assurer un parcours scolaire minimum de dix ans à tous les enfants, les taux d’achèvement du primaire et de rétention du système s’avèrent encore faibles. Dans ses efforts de soutien à l’Etat du Sénégal dans sa politique éducative, pour qui, la problématique de la qualité est au cœur de ses préoccupations (PAQUET 2013-2015) et en appui à l’initiative « Women’s Global Education Project », l’USAID, à travers « Our Sisters Read » qui intègre parfaitement l’objectif de développement 3 (DO319), a financé cette activité d’une durée de quatre ans, dans la région de Fatick, plus particulièrement dans la circonscription de l’ Inspection de l'Education et de la Formation (IEF)20 de Foundioune. Le but de cette activité est d’améliorer l’intérêt pour la lecture et les capacités en lecture des élèves des écoles primaires en milieu rural, en particulier les filles. L’atteinte de cet objectif passera par la réalisation des trois objectifs intermédiaires ci-après : • OS1 : bâtir des capacités fondamentales en lecture de 6.800 élèves des écoles primaires en milieu rural, avec au moins 50% de filles en leurs offrant l’accès à des centres technologiques, des laboratoires mobiles et à travers la formation des enseignants d’ici 2017 ; • OS2 : encourager et motiver les élèves à la lecture à travers la mise à disposition de ressources novatrices et incitatives qui atteignent les élèves dans les écoles par le biais de concours, de bourses et des clubs de lecture ; • OS3 : bâtir des communautés de parents et de leaders qui soutiennent les élèves (garçons et filles) dans leurs efforts d’amélioration de leurs capacités en lecture et d’utilisation des ressources technologiques d’apprentissage. D’un budget de 1,700,000 US dollars, l’activité Our Sisters Read (OSR) a été mise en œuvre en partenariat avec l’ONG locale « Femmes Plus ». Les tâches principales exécutées dans ce cadre sont, entre autres : • Formation des enseignants, • Activités pédagogiques (séances de lecture, clubs de lecture, jeux pédagogiques), • Activités incitatives (concours, bourses), • Mentorat et encadrement des élèves par les enseignants et • Alphabétisation des adultes. 19 CDCS 20 IEF : Inspection de l’Education et de la Formation L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 39 3. HYPOTHESE DE DEVELOPPEMENT ET SA MISE EN ŒUVRE L’objectif final de cette activité est d’améliorer les aptitudes en lecture des élèves, notamment des filles dans les écoles primaires rurales du département de Foundiougne (région de Fatick). Différentes études ont établi la relation entre la déperdition scolaire (échecs scolaires, abandons, etc.) avec la baisse de la qualité des apprentissages dans le système éducatif. Plusieurs initiatives ont été menées par des institutions comme l’Institut national d'Etude et d'Action pour le Développement de l'Education (INEADE) et divers autres projets pour montrer la faiblesse des niveaux des aptitudes, particulièrement en lecture et en mathématiques. Les échecs scolaires consécutifs à cette baisse des performances entrainent une démotivation aussi bien chez les élèves que chez les parents dans le maintien des enfants à l’école, en particulier pour les filles. SYNTHESE DU CADRE LOGIQUE DE L’ACTIVITE But de l’activité : Améliorer les performances en lecture des élèves notamment des filles des écoles primaires rurales. RI 1: Bâtir les capacités fondamentales en lecture de 6,800 élèves du cycle primaire en milieu rural, avec au moins 50% de filles, en offrant l’accès à des centres de technologies, des laboratoires mobiles et la formation des enseignants d’ici 2017. Augmenter l’accès aux matériels de lecture à travers les centres technologiques, les librairies et les laboratoires mobiles. Améliorer les pratiques de l’enseignement de l’alphabétisation. RI 2 : Encourager et motiver les élèves à la lecture à travers la mise à disposition de ressources novatrices et incitatives qui atteignent les élèves dans les écoles par le biais de concours, de bourses et des clubs de lecture. Motiver les élèves à la lecture Accroitre les opportunités de rétention des élèves à l’école RI 3 : Bâtir des communautés de parents et de leaders qui soutiennent les élèves (filles et garçons) dans leurs efforts d’améliorations de leurs capacités en lecture et d’utilisation des ressources technologiques d’apprentissage Accroitre le soutien des communautés (parents et leaders) à travers une campagne globale de promotion de l’alphabétisation. La logique d’intervention de l’activité OSR est basée sur l’hypothèse selon laquelle une amélioration des capacités en lecture des élèves permettrait non seulement de renforcer les aptitudes des enfants, mais également de soutenir de meilleures performances scolaires. La réduction des échecs qui en résulterait participerait à créer un environnement social, familial et communautaire soutenant les études des enfants, plus particulièrement pour les filles qui sont les plus exposées à l’abandon scolaire au profit des activités économiques, les migrations de travail et les tâches domestiques. L’atteinte des objectifs spécifiques déclinés plus haut (cf. contexte) permettrait d’améliorer les acquisitions fondamentales des élèves et leurs performances scolaires dans le domaine de la lecture. Les activités mises en œuvre à cette fin comprennent des sessions de formation au profit des instituteurs, des activités de lecture en dehors des classes (activités extra-scolaires, activités ludiques, jeux pédagogiques, etc.), des sessions d’alphabétisation d’adultes et bien d’autres prestations organisées par le laboratoire mobile, une mise à disposition de manuels et de clubs de lecture pour les élèves. Parallèlement à ces activités pédagogiques, le projet a mis en place diverses autres activités incitatives destinées à susciter et à soutenir l’intérêt et la motivation des enfants à participer dans les activités liées à l’amélioration de la lecture : concours, bourses, etc. L’approche L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 40 développée ainsi que les différentes activités déroulées ont volontairement visé une implication active et soutenue des enfants (élèves des deux sexes), les enfants du primaire, leurs familles et les communautés pour promouvoir la culture de la lecture en vue d’améliorer les performances et ainsi de réduire les inégalités de genre en matière d’éducation. Toutefois, il est à noter que des retards et changements importants ont été enregistrés pendant la mise en œuvre du projet : i) Retard dans la construction et la mise à disposition du centre de l’information et communication technologique entrainant une suspension des activités associées à la disponibilité de cette infrastructure ; ii) Suspension du projet PALME dont les évaluations standardisées devaient être utilisées dans la détermination de la situation de référence du projet (baseline). 4. SOURCES D’INFORMATION EXISTANTES Les sources d’information qui pourront être utilisées dans le cadre de cette évaluation, sans se limiter à cette liste, sont déclinées ci-après,: N° Titre du document Dates 1 WGEP- Our Sisters Read, Performance Monitoring Plan 2 WGEP - Our Sisters Read, Branding and Marking Plan 3 WGEP- Our Sisters Read, Annual Implementation Plan, Oct 1, 2013- Sept 30, 2014 4 WGEP- Our Sisters Read, Year 1 First Quarterly Report, Sept 9, 2013- Dec 31, 2014 5 WGEP- Our Sisters Read, Year 1 Second Quarterly Report, Jan 1, 2014- March 31, 2014 6 WGEP- Our Sisters Read, Year 1 Third Quarterly Report, April 9, 2014 - Jun 30, 2014 7 WGEP- Our Sisters Read Annual Report- Fourth Quarter Oct 1, 2013- Sept 2014 8 WGEP - Our Sisters Read, Annual Performance Indicator Report, Oct 1, 2013- Sept 30, 2014 9 WGEP- Our Sisters Read Annual Implementation Plan, Oct 1, 2014- Sept 30, 2015 10 WGEP- Our Sister Read, Year 2 Annual Report- Fourth Quarter Oct 1, 2014- Sept 30, 2015 11 WGEP- Our Sisters Read Annual Implementation Plan, Oct 1, 2015- Sept 30, 2016 12 WGEP- Our Sisters Read Performance Indicators Review Oct 2015 13 Evaluation de la qualité des apprentissages, Jangandoo, IFAN 2014 14 PASEC (2007). Evaluation du PASEC, CONFEMEN, Dakar 2007 15 PASEC 2014- Performances des systèmes éducatifs en Afrique Subsaharienne francophone : Compétences et facteurs de réussite au primaire. PASEC, CONFEMEN, Dakar. 2015 16 Etat des lieux de l’éducation de base au Sénégal, MEN 2014 L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 41 5. BUT, AUDIENCES ET UTILISATIONS PREVUES DES RESULTATS Le but de cette évaluation est de déterminer le niveau d’exécution actuel de l’activité, y compris les réussites et défis identifiés durant sa mise en œuvre. Elle revêt deux dimensions complémentaires : sommative et formative. Caractère sommatif de l’évaluation : les résultats de cette évaluation permettront à l’USAID de disposer de données actuelles et fiables sur le niveau des tâches associées à cette activité, tout autant que les résultats intermédiaires atteints et/ou en voie de l’être. Les changements utiles pourront, sur cette base, être apportés dans le but d’atteindre les résultats escomptés ; Caractère formatif de l’évaluation à mi-parcours : les résultats de l’évaluation à mi-parcours permettront un apprentissage à partir de la mise en œuvre opérationnelle de l’activité. Dans cette perspective, les bonnes pratiques ainsi que leçons apprises seront utilisées dans une logique d’apprentissage pour informer la mise en œuvre ultérieure. Enfin, dans une perspective plus large, les résultats de l’évaluation pourront être utilisés par le MEN et tout autre partenaire technique dans le cadre du développement d’activités et de programmes futurs, particulièrement dans le domaine de l’amélioration des capacités de lecture des élèves. 6. QUESTIONS DE L’ÉVALUATION La question principale de l’évaluation est de déterminer le niveau d’amélioration des aptitudes en lecture des élèves des écoles primaires couvertes par l’activité « Our Sisters Read » ? Les questions spécifiques sont les suivantes : 1. Quel était le niveau de précision de la théorie du changement qui sous-tend la conception et la mise en œuvre de l’activité ? a. Les mesures prévues par l’activité (par exemple l’accès aux ressources, clubs de livres, concours, bourses, etc.) ont-elles conduit à motiver les élèves ? b. Les mesures prévues par l’activité (par exemple l'alphabétisation des adultes, le mentorat par des enseignants, etc.) ont-elles renforcé la mobilisation communautaire ? c. Les interventions prevues par l'activite (par exemple, l'alphabetisation des adultes, les rencontres communautaires. le mentorat par des enseignants, etc.) ont-elles renforce Ia mobilisation communautaire ? Quelles sont celles qui ont ete les plus reussies ? d. Quel a ete le degre d'adhesion de Ia communaute a ce projet ? Pourquoi ? 2. Dans quelle mesure le partenaire d'exécution (y compris le principal et le secondaire) a-t-il atteint les objectifs poursuivis par l’activité ? a. Le niveau d'execution de l'activite correspond-elle ace qui etait prevu ? b. Quels sont les facteurs internes ou externes (y compris au sein de l'USAID) qui ont le plus grand effet sur la mise en œuvre de l’activité ? Quels sont ces facteurs qui pourraient être rencontrés à nouveau et ceux qui semblent aberrants ? L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 42 c. Comment le partenaire d’exécution et l’équipe Education a-t-il relevé les défis rencontrés dans la mise en œuvre de l’activité et quelles sont les leçons à en tirer ? 3. Comment les capacités du partenaire d'exécution (y compris le partenaire principal et le secondaire) ont-elles changé au cours de la mise en œuvre de l’activité ? Quelle sont les lacunes qui subsistent ? 7. DESAGREGATION SELON LE GENRE ET EFFETS DIFFERENTIELS L’un des objectifs assignés à l’activité et d’atteindre au moins 50% de filles parmi les bénéficiaires ayant amélioré leurs aptitudes en lecture. Les données collectées dans cette évaluation seront donc nécessairement désagrégées par sexe pour apprécier le niveau de couverture des filles certes, mais également dans la perspective d’analyser plus finement les effets différentiels selon le genre sur différentes dimensions (amélioration des aptitudes de lecture, performances scolaires, participation aux concours organisés, etc.) pour les garçons et pour les filles. 8. MODELE ET METHODOLOGIE DE L’EVALUATION La mise en œuvre de cette évaluation sera basée sur une démarche participative. Elle impliquera toutes les parties prenantes de l’activité « Our Sisters Read » : autant ses bénéficiaires directs (élèves) et communautés que les bénéficiaires indirects et acteurs. La démarche méthodologique sera axée sur quatre (4) étapes principales : (i) revue et analyse de la documentation, (ii) conception de la matrice de l’évaluation par le biais de la Matrice des réponses et des outils de collecte (guides), (iii) enquête sur le terrain et (iv) analyse des données et production du rapport d’évaluation. Revue et analyse des documents La revue de toute la documentation élaborée dans le cadre de l’activité sera effectuée (proposal de l’activité, rapports annuels, rapports trimestriels, plans de travail, rapports de suivi des activités, PMP, etc.). L’analyse de cette documentation permettra une appréciation préliminaire des réalisations effectuées (nombre d’élèves touchés par l’activité, manuels distribués, nombre de formations en alphabétisation pour les adultes, nombre d’écoles et d’élèves concernés par l’activité, scores des élèves issus des tests de lecture, etc.). Elle est essentielle pour le cadrage de la mission ainsi que pour le traitement initial des données. Elle sera complétée par la revue de la documentation disponible sur le sujet : documents de politique du Gouvernement, travaux et études antérieures menées sur la même thématique, etc. Les données quantitatives et statistiques de l’activité, ainsi que toutes autres données secondaires pertinentes seront exploitées en vue d’une analyse quantitative. Conception de la matrice « Matrice des réponses » et élaboration des outils de collecte Après l’exploitation initiale de la documentation, l’équipe d’évaluation produira la matrice d’évaluation qui déclinera les processus de collecte de l’information sur la base des questions de recherche principales. L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 43 Les cibles auprès desquelles ces informations seront collectées seront également identifiées et précisées. Pour chaque catégorie de cible, des outils appropriés seront développés pour satisfaire aux besoins spécifiques d’information. Les outils de collecte seront conçus en fonction de la cible, mais également selon la technique de collecte utilisée (entretien semi-structuré, entretien approfondi, focus-groups, récits de cas, etc.). Un pré-test des outils sera organisé dans le souci de vérifier qu’ils généreront des réponses aux questions d’évaluations. De ce point de vue, la rencontre de planification technique prévue avec l’USAID, MEP/Sénégal et les consultants permettra de passer en revue l’ensemble des outils élaborés ainsi que leur conformité aux attentes des parties prenantes. Enquête sur le terrain Les données primaires de l’évaluation seront collectées directement sur le terrain par l’équipe d’évaluation. Elle se fera aussi bien à Dakar (Ministère, USAID, UNICEF, etc.) que sur le terrain au niveau de la zone d’intervention de l’activité où toutes les parties prenantes et bénéficiaires seront rencontrées et interviewées. Les outils de collecte élaborés, partagés et validés serviront à la conduite des enquêtes sur le terrain auprès des différentes cibles. Les interviews (semi-structurés, approfondis et focus-groups) seront conduites pour recueillir les opinions et appréciations des parties prenantes, tout autant que pour apprécier les effets et changements induits par les activités chez les élèves et dans leurs communautés respectives. L’incidence des interventions sur la qualité de la lecture des élèves ainsi que l’intérêt accordé par les parents seront également appréhendées. Les acteurs de l’éducation (enseignants, directeurs d’écoles et inspecteurs de l’éducation de de la formation au niveau des IEF et IA) seront également interviewés. Des entretiens de groupes seront organisés avec les membres de la communauté, les élèves et enseignants intervenant dans les écoles couvertes par l’activité. Au-delà des entretiens qualitatifs, les données empiriques permettant de fournir les preuves des changements pourront être collectées au niveau des établissements visités si elles sont disponibles auprès des enseignants et/ou directeurs d’écoles. L’expert spécialiste en lecture veillera à fournir une appréciation objective et documentée des ressources et données disponibles sur le terrain. 9. ANALYSE DES DONNEES La matrice des réponses une fois élaborée par l’équipe, fournira plus de détails sur les outils d’analyse de données qui seront pertinents. L’utilisation de cet outil va assurer que les réponses apportées dans cette évaluation seront triangulées sur la base de diverses techniques et sources de collecte. Pour le traitement des données qualitatives qui seront produites par les entretiens et focus-groups, les procédés de l’analyse thématique seront utilisés. L’ensemble des entretiens et des focus group seront transcrits, saisis et recodés. Leur analyse thématique procédera par une identification des contenus structurants en lien avec les questions principales de recherche. Pour ce qui est des données quantitatives et autres statistiques produites dans le cadre de l’activité, leur analyse permettra de présenter aussi bien les tendances globales observées que les situations spécifiques aux garçons et aux filles. Au besoin, d’autres critères de désagrégation pourront également être utilisés pour affiner les analyses et offrir davantage de détails dans la présentation des résultats. L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 44 10. FORCES ET LIMITES DES APPROCHES UTILISEES La disponibilité des documents élaborés par l’activité est une réelle opportunité pour l’évaluation. Ils permettent de suivre les réalisations à partir du plan de mise en œuvre, des fiches de suivis, des indicateurs du plan de suivi renseignés par l’équipe de WGEP et son partenaire « Femmes Plus ». En revanche, l’une des limites majeures à signaler pour cette évaluation est l’absence de données de référence dont la production était assujettie à la disponibilité des évaluations standardisées de l’activité Partenariat pour l'Amélioration de la Lecture et des Mathématiques dans (PALME). La suspension prématurée de cette activité a privé « Our Sisters Read » de disposer d’une baseline indispensable pour une appréciation correcte de sa performance. De plus, la non utilisation d’une évaluation effective des apprentissages (EGRA) dans cette évaluation à mi-parcours constitue une autre limite majeure d’appréciation des progrès engendrés dans les capacités de lecture des élèves. Cependant, l’utilisation de techniques variées dans les procédures de collecte des données ainsi que la triangulation des sources d’information (pluralité et diversité des cibles des enquêtes) constitue des forces qui permettent de réduire de manière très significative les biais liés au recours principal aux techniques qualitatives. 11. LIVRABLES DE L’ÉVALUATION Les livrables attendus sont les suivants : • Le cadre méthodologique (démarche, outils de collecte, méthodes de traitement et d’analyse) ; • Briefing et exposé des aspects méthodologiques à l’équipe Education de l’USAID ; • La base des données collectées auprès des élèves, des acteurs éducatifs (directeurs, enseignants, et inspecteurs des IA et IF) et des communautés – y compris la base des données brutes avec le dictionnaire des données et les notes des membres de l’équipe (questionnaires remplis, éléments sonores des interviews, fiche de dépouillement.) • Le rapport préliminaire ; • Débriefing et présentation des résultats, conclusions et recommandations à l’intention de chaque partie prenante (USAID, partenaires de l’activité, communautés, écoles, IA, IF) (en français et anglais) ; • Le rapport final de l’évaluation (en français et anglais) ; • Compte rendu de la rencontre de partage (à développer dans le rapport final) ; • Le rapport de la rencontre de partage qui sera annexé au rapport final de l’évaluation. 12. COMPOSITION DE L’ÉQUIPE L’équipe d’évaluation sera composée d’un chef d’équipe internationale un spécialiste en évaluation d’Education et un associé en S&E du MEP qui va appuyer leur travail. Mme Gaëlle Simon sera la chef d’équipe de la mission d’évaluation en tant qu’experte en lecture avec plus de douze (12) années d’expérience dans les évaluations de l’apprentissage et l’amélioration de la qualité des prestations de services d’Education. Mme Simon a une grande expérience nationale et internationale dans la conduite d’évaluation de niveau de lecture. Elle sera soutenue par un expert de haut niveau en développement L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 45 communautaire dans le domaine de l’éducation qui se chargera d’analyser les forces et faiblesses des stratégies mises en œuvre pour susciter une dynamique sociale en faveur de la lecture. MEP Sénégal est à la recherche de ce profil important dont la mission au sein de l’équipe sera de mettre en œuvre le volet communautaire de l’évaluation. Tous les membres de l’équipe fourniront une déclaration signée attestant qu’il n’y a aucun conflit d’intérêt ou de décrire un conflit d’intérêt existant. Dates Tâches/Livrables Niveaux d’intervention (en jours) Chef d’équipe – Gaelle Simon Expert en développement communautaire Total 1-10 Juin, 2016 Revue des rapports et développement du plan de travail et matrices des réponses 4 3 7 11 Juin 2016 Voyage international 1 1 13 Juin 2016 Réunion de planification technique inclure briefing avec USAID 1 1 2 14 Juin, 2016 USAID revue du plan de travail et des outlils 1 1 2 15-24 Juin 2016 Travail de terrain à Fatick 9 9 18 25-29 Juin 2016 Compilation des résultats, conclusions et recommandations 4 2 6 30 Juin 2016 Présentation du cadre de résultats, Conclusions et recommandations à USAID 1 1 2 1 Juillet 2016Voyage international 1 1 4 Juillet-15 Juillet 2016 Analyse des données et rédaction du rapport 6 5 11 19 Juillet, 2016 Soumission du rapport draft a USAID 22 Juillet, 2016 Atelier avec le MEN, IPs et parties prennantes du projet 1 1 9 Aout 2016 Finalisation du rapport 5 3 8 Niveau estimé total estimé de l'effort (LOE) 33 26 59 13. PARTICIPATION DE L’USAID ET DE SES PARTENAIRES L’équipe de l’USAID sera impliquée tout au long du processus de conception et de mise en œuvre de cette évaluation. Leur contribution sera sollicitée à toutes les étapes, en particulier dans la conception et design de l’étude pour une prise en compte correcte des besoins et attentes de l’USAID. Au-delà de cette contribution initiale de l’équipe de l’USAID, des entretiens seront réalisés avec les responsables impliquées dans ce projet à l’USAID selon leur disponibilité. En outre, les partenaires de l’USAID sur cette activité, L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 46 en l’occurrence WGEP et l’ONG locale Femmes Plus seront impliqués dans le processus de l’évaluation, y compris dans la collecte des données sur le terrain. Le MEN participera également dans le cadrage initial de l’évaluation et dans le partage des résultats. Ces différents moments seront mis à profit pour recueillir les inputs et commentaires des différentes parties dans la conception et la mise en œuvre de l’évaluation. Une réunion initiale avec l’équipe USAID est prévu le 13 et 14 juin 2016 et le débriefing des résultats préliminaires pour le 30 juin 2016. La participation de l’USAID, WGEP/Femmes Plus et du MEN est essentielle à ces deux activités, d’où un besoin de leur disponibilité à ces deux dates. 14. PLANIFICATION ET LOGISTIQUES La programmation de cette évaluation à mi-parcours de l’activité Our Sisters Read et les aspects logistiques seront conjointement assurés, durant tout le processus, par les équipes de l’USAID (bureau Education) et MEP/Sénégal. Les consultants retenus auront un appui technique et managérial assuré par l’équipe du MEP/Sénégal. MEP travaillera directement avec les partenaires, notamment Femmes Plus, pour organiser les rencontres et les activités de terrain. L’USAID planifiera les réunions avec les autorités présentes sur le terrain en collaboration avec MEP/Sénégal. Des échanges seront aussi effectués avec WGEP pour le cadrage des visites de terrain. 15. EXIGENCES POUR LE REPORTING Tous les livrables seront soumis en français et en anglais. Les principales exigences du reporting concernent, au-delà des précisions linguistiques, le nombre de pages du rapport final. Il s’agit d’un rapport final de trente-cinq pages (35 pages), les annexes non-inclus, qui sera soumis à l’USAID/Sénégal. Les annexes comprendront entre autres, le Cadre de Travail de l’évaluation, une explication détaillée de la méthodologie, le rapport de l’atelier de présentation des résultats, conclusions et recommandations ainsi que les outils de collecte développés. Le canevas de l’USAID pour les évaluations servira de modèle de base pour la rédaction du rapport (voir le modèle en annexe). Tous les livrables produits par l’équipe d’évaluation notamment les rapports seront soumis à l’appréciation de l’USAID DEC, du MEP/Sénégal et des partenaires établis à Washington (USAID et WGEP). Il faut ajouter à ces dispositions prévues pour le reporting, que les rapports (préliminaire comme final) seront disponibles en versions anglaises et françaises. L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 47 16. DOCUMENTS DE REFERENCES ANNEXES X Calendrier d’exécution X Matrice de la revue documentaire X Matrice des résultats, conclusions et recommandations X Budget X CV Déclaration de conflit d’intérêt X Politique d’évaluation de l’USAID X Structure du rapport d’évaluation 17. AUTORISATIONS The undersigned hereby authorizes the following items (checked below) for the Statement of Work (SOW) described above: X Completion of the SOW, as described above; X SOW staffing, as described above; X Concurrence with Contracting Officer’s Travel Approval for the Consultant(s), requested above (if received prior to review). [COR to either sign below or indicate approval in a return email] __________________________________________ ________________ Contracting Officer’s Representative (COR) Date Roy Geiser, or designate L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 48 ANNEXE 1.1: CALENDRIER D’EXECUTION Our Sisters Read Evaluation/Timeline Mai Juin Juillet Etapes Semaine 1 2 3 4 1 2 3 4 1 2 3 4 Revue des rapports Voyage international Réunion de planification technique Briefing initial avec l’équipe d’Education de l’USAID Développement des outils, des itinéraires, de la matrice des réponses ♦ Travail de terrain à Fatick Compilation des résultats, conclusions et recommandations Présentation du cadre de résultats, conclusions et recommandations 30 Voyage international Analyse des données et rédaction du rapport Atelier avec le MEN, IPs et parties prennantes du projet 22 Finalisation du rapport ♦ ♦ = livrables L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 49 ANNEXE 1.2 : MATRICE DE LA REVUE DOCUMENTAIRE L’EVALUATION DU ACTIVITÉ “OUR SISTERS READ” : CADRE DE TRAVAIL 50 ANNEXE I.3 : MATRICE DES RESULTATS, CONCLUSIONS ET RECOMMANDATIONS Question de l’Evaluation Résultats Conclusions Recommandations 1 2 3 4 5 6 7 8 9 10 Our Sisters Read Performance Evaluation Report 51 ANNEX 2: EVALUATION WORK PLAN WORK PLAN AND METHODOLOGY FOR THE PERFORMANCE EVALUATION OF THE « OUR SISTERS READ» ACTIVITY 20 JUNE 2016 This publication was produced for review by the United States Agency for International Development. It was prepared by Management Systems International, a Tetra Tech Company, for the USAID/Senegal Mission-Wide Monitoring and Evaluation Project. Our Sisters Read Performance Evaluation Report 52 WORK PLAN AND METHODOLOGY FOR THE PERFORMANCE EVALUATION OF THE « OUR SISTERS READ» PROGRAM Contracted under AID-685-C-15-00003 USAID/Senegal Mission-Wide Monitoring and Evaluation Project DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Our Sisters Read Performance Evaluation Report 53 Table of Contents A. INTRODUCTION................................................................................................................ 54 B. EVALUATION QUESTIONS............................................................................................. 55 C. DATA COLLECTION METHODS AND SOURCES ....................................................... 57 D. SAMPLING........................................................................................................................... 59 E. DATA ANALYSIS................................................................................................................ 60 F. FIELD WORK PLAN........................................................................................................... 60 ANNEXE I: GETTING TO ANSWERS MATRIX.................................................................... 63 ANNEXE II: EVALUATION TIMELINE................................................................................... 67 ANNEXE III: ROLES AND RESPONSIBILITIES..................................................................... 68 Our Sisters Read Performance Evaluation Report 54 WORK PLAN AND METHODOLOGY FOR THE PERFORMANCE EVALUATION OF THE « OUR SISTERS READ» ACTIVITY A.INTRODUCTION The Our Sisters Read (OSR) activity seeks to improve reading competency for primary school children and to create lifetime-learners, with an emphasis on promoting girls’ literacy. Women’s Global Education Project (WGEP) works with its partner organization, Femmes Plus, to implement in the Fatick region, and more specifically in the district of Foundiounge this USD $1,700,904 four-year activity funded by the United States Agency for International Development (USAID) Senegal under Development Objective 3 (DO3) of the Country Development Cooperation Strategy (CDCS), as part of its support to the Government of Senegal and the 2013-2025 national education policy (PAQUET)21. OSR engages and supports primary-aged school children, their families and communities to create and strengthen a culture of reading while reducing the gender gap in reading and education. Implemented in 52 schools, 28 in Sokone and 24 in Toubacouta, OSR is targeting 6,800 students in primary grades, of which at least 50% are girls. The overarching goal for OSR is “to create strong readers and lifetime learners among rural Senegalese primary school students, especially girls”. The achievement of three intermediate results will contribute to the attainment of this goal. They are as follows: • IR 1: Build foundational reading skills for rural primary school students by creating access to technology centers, mobile labs and trained literacy professionals; • IR 2: Encourage and motivate students to read through the use of innovative resources and incentives; and • IR 3: Build communities of parents and leaders that support young girls and boys to increase their reading and technological literacy. Program interventions, including reading contests, scholarships, book clubs and mentoring, are designed to motivate and generate enthusiasm among young readers, supplementing school curriculum and filling in gaps where the quality of education is lacking. The physical resources (computer labs, libraries and mobile labs) serve as locations and distribution points for interventions and serve as community anchors. Community outreach interventions that support adult literacy and parental and community involvement address underlying attitudes towards literacy and promote its importance, particularly for girls. This multipronged approach seeks to ultimately contribute to an enduring shift in attitudes towards literacy and girls’ education, which will translate into higher achievement levels in reading for primary-aged students. 21 Programme d’Amélioration de la Qualité, de l’Equité et de la Transparence (PAQUET) Our Sisters Read Performance Evaluation Report 55 The Our Sisters Read activity works in collaboration with public sector officials particularly in the Ministry of National Education (MEN) to ensure that interventions are coordinated with Ministry priorities. OSR is seen as a complement to the MEN’s reading and performance based results in primary schools. The theory of change that underpins the results framework for OSR posits that a literate environment, where students are encouraged to read and where the community promotes and supports literacy and schooling, will improve overall student performance in schools, and particularly in reading. Table 8: “Our Sisters Read” Results Framework Activity Goal: Create strong readers and lifetime learners among rural Senegalese primary school students, especially girls IR1: Build foundational reading skills for 6,800 rural primary school students (at least 50% female) by creating access to technology centers, mobile labs and trained literacy professionals by 2017 Outcome 1.1. Increased access to reading materials through ICT centers, libraries and mobile labs Outcome 1.2: Improved instructional literacy practices IR2: Encourage and motivate students to read through innovative resources and incentives, reaching students at participating schools each year through contests, scholarships, mentorships and book clubs Outcome 2.1: Activities motivate primary school students to read Outcome 2.2: Increased opportunities for students to attend school IR3: Build communities of parents and leaders that support young girls and boys to increase their reading and technological literacy Outcome 3.1: Increased support among parents and community leaders for literacy generated through A comprehensive campaign to promote literacy Midway through the activity’s period of performance, USAID/Senegal initiated a performance evaluation to understand the implementation of interventions and the activity’s progress towards the attainment of its stated objectives. This performance evaluation will generate data on implementation successes and challenges, on how stakeholders and beneficiaries perceive and value the activity and on whether the activity is on target to achieve its goal and objectives, and will provide recommendations for midcourse corrections, as needed. B.EVALUATION QUESTIONS This evaluation seeks to understand students’ performance levels in reading in the OSR intervention schools, with particular interest given to the community mobilization component of the activity. The following are the three evaluation questions (EQs) USAID/Senegal has asked that this evaluation address. They comprise the central task before the evaluation team and will serve as the main structure for reporting study findings, conclusions and recommendations: • EQ 1: How well has the theory of change been validated through the activity’s implementation? • EQ 2: To what degree has the implementing partner (including the subcontractor) achieved the activity’s objectives? • EQ 3: How has the capacity of the implementing partner (including the subcontractor) changed over the course of the activity? What gaps remain? Our Sisters Read Performance Evaluation Report 56 This evaluation will generate and present empirical data based on high quality, replicable social science methods. Conclusions and recommendations derived from the study’s findings will be stated clearly and separately, with a special emphasis on the degree to which recommendations are actionable, practical and directed to specific parties. The evaluation team will undertake semi-structured interviews with a wide range of stakeholders and beneficiaries. The interviews will provide the opportunity for the team to gather participants’ perceptions of and appreciation for the activity. The interviews will be conducted with individuals or in small groups. The evaluation team will respond to EQ 1 by addressing four sub-questions: a. Have activity interventions, such as access to resources, extra-curricular activities, book clubs, competitions, scholarships, etc., motivated students to read? The evaluation team will visit a total of 7 schools in Sokone and Toubacouta. The team will conduct small group interviews with groups of boy and groups of girl students in CM1 and CM2 grades targeted by the activity. The evaluation will focus on student practices and attitudes towards reading with particular attention given to student use of the ICT centers and mobile libraries. Semi-structured interviews with teachers will provide additional information on teachers’ observations of students’ habits and interest in reading, while interviews with parents will provide information on students’ reading habits at home. To understand the motivation and performance of younger students in CI, CP, CE1, CE2, we will include targeted questions in the teacher and parent interview guides. b. Has teacher training improved classroom practices? Semi-structured interviews with teachers, school directors and school inspectors will generate data on classroom practices during reading lessons. Though a lack of observational baseline data precludes the evaluation team from comparing observed practices before and after activity interventions, the team will seek to understand if and how teachers, school directors and school inspectors identify the influence of the activity within the classrooms. The team will examine if teachers have incorporated in their practice the teaching methods suggested by the activity through training, demonstrations and guides. c. Have activity interventions, such as adult literacy, community meetings, mentoring by teachers, etc., increased community mobilization? Which interventions were most effective? As stated previously, this evaluation has a particular interest in understanding if and how community engagement can promote achievement in reading in schools. A central tenet of the OSR activity is that increased literacy skills for community members will boost the value ascribed to literacy, which in turn will motivate families to maintain students in schools longer, thus leading to improved reading scores and overall improved school performance. Through small group interviews with a variety of community members, including parents, and particularly mothers, and individual interviews with religious and community leaders, this evaluation will focus on how community members perceive and value reading for themselves and for children. The evaluation will collect data on community members’ reading habits that have emerged as a result of the activity and on practices they employ that promote and support reading for children. Additionally, the team will collect data on interventions community members have found to be the most engaging. The interview guides for community members will include questions about their preference for “Our Sister Read” interventions and will seek to understand why community members have found them to be particularly effective. Preferred interventions may include those specifically targeting parents and other communities, e.g. community meetings, and those targeting teachers and students but which have had an effect on community members’ reading habits, e.g. lanterns given to children to read at home. d. What is the level of community engagement in this activity? Why? Qualitative and quantitate data will serve to answer EQ 1d in regards to the level of engagement of the community with the “Our Sisters Read” activity. Qualitative data gathered will be used to understand why Our Sisters Read Performance Evaluation Report 57 and how community members participate in activity interventions such as reading classes and how this involvement has promoted a culture of reading. Respondents will be asked to discuss interventions they felt were most useful for improving their literacy skills and for creating a literate community. Quantitative data collected by the activity about attendance in community meetings and reading classes, for example, will be used to confirm or disconfirm the preferences articulated by community members. The evaluation team will respond to EQ 2 by addressing three sub-questions: a. Was the activity implemented according to the design? Has the implementation of the activity followed the original design? b. What internal or external factors (including within USAID) have had the biggest impact on the implementation of the activity? Which factors may continue to impact the activity and which ones seem to have had short-term/specific impact? c. How did the implementing partners and the USAID education team address challenges during the implementation of the activity and what are some lessons learned? The evaluation team will conduct extensive desk review of activity documents and will interview the WGEP, FP and USAID to better understand decisions regarding activity implementation and their alignment with the activity design. The desk review and interviews will generate data about field challenges, unrealized critical assumptions, actors and factors that influenced the implementation of the activity. Mayors and inspectors will also be asked to provide insight about the implementation of the activity. To address EQ 3, the evaluation team will conduct interviews with the WGEP, FP and USAID. The team will review activity documents for a description of efforts geared towards capacity building. The team will also review the report from the 2015 Organization Capacity Assessment (OCA) conducted for Femme Plus by USAID as well as the report from the 2014 training provided by the Office of Financial Management (OFM). It is our understanding that a capacity building plan was not included in the budget for this activity nor were funds budgeted specifically for that purpose. And while it is outside the scope of this evaluation to conduct a follow-up OCA, the team will collect information from the implementing partners and from USAID to understand perceived human and technical resources required to implement an activity of this scope and current human and technical resources available to the implementers. The evaluation will also capture Femmes Plus' impressions of how they have developed since the start of the activity and how it is moving forward on the recommendations from the training and OCA. C.DATA COLLECTION METHODS AND SOURCES This performance evaluation will rely on qualitative methods with some quantitative analysis of student reading performance using data provided by OSR. The evaluation team will not collect primary assessment data or other types of quantitative data. All quantitative data used for this evaluation will be obtained from secondary sources, including potentially the Ministry of Education. This evaluation will examine the activity’s performance in achieving its objectives and will measure its influence on beneficiary attitudes towards and practices in reading. The following data collection methods will be used during this evaluation: • Desk Review: the evaluation team will review and analyze activity design and implementation documents. This review has several purposes. First, the team will develop an understanding of the activity theory of change and implementation characteristics, contexts, and challenges. Second, the team will identify and describe achieved activity results. Our Sisters Read Performance Evaluation Report 58 • Qualitative methods: the evaluation team will conduct semi-structured interviews with a wide range of beneficiaries and stakeholders. The team will organize individual and group interviews to gather information on beneficiaries’ experiences with, appreciation for and perceptions of the activity interventions. • Quantitative data analysis: the evaluation team will not collect primary quantitative data. However, when available and pertinent, the team will analyze secondary data provided by the activity, USAID and/or MEN. Quantitative data may include student scores on reading assessments or other school achievement tests and participation rates of beneficiaries in key activity interventions such as attendance in community meetings. The identification of potential data collection methods and sources appropriate for each EQ is presented in Table 9 below. A completed “Getting to Answers” matrix is included in Annex I. Table 9: Data Collection Methods and Sources Evaluation Question Data Collection Method Data Sources EQ 1: How well has the theory of change been validated through the activity’s implementation? - Review of activity documents & reports - Semi-structured interviews -Activity documentation & reports -Implementing partner staff EQ 1a: Have activity interventions, such as access to resources, extra￾curricular activities, book clubs, competitions, scholarships, etc., motivated students to read? - Review of activity documents & reports - Semi-structured interviews - Group interviews - Data mining -Activity documentation & reports -Activity assessment data set - Project beneficiaries: students, teachers, community members (parents, religious leaders, village chief, local associations), school directors -Implementing partner staff -Departmental education stakeholders (e.g. inspectors) - Standardized assessment scores from government tests EQ 1b: Has teacher training improved classroom practices? - Review of activity documents & reports - Semi-structured interviews - Group interviews -Activity documentation & reports - Project beneficiaries: students, teachers, community members (parents), school directors -Implementing partner staff -Departmental education stakeholders (e.g. inspectors) EQ 1c: Have activity interventions, such as adult literacy, community meetings, mentoring by teachers, etc., increased community mobilization? Which interventions were most effective? - Review of activity documents & reports - Semi-structured interviews - Group interviews -Activity documentation & reports - Project beneficiaries: students, teachers, community members (parents, religious leaders, village chief, local associations), school directors -Implementing partner staff EQ 1d: What is the level of community engagement in this activity? Why? - Review of activity documents & reports - Semi-structured interviews - Group interviews -Activity documentation & reports - Project beneficiaries: community members (parents, religious leaders, village chief, local associations) -Implementing partner staff Our Sisters Read Performance Evaluation Report 59 Evaluation Question Data Collection Method Data Sources EQ 2: To what degree has the implementing partner (including the subcontractor) achieved the activity’s objectives? - Review of activity documents & reports - Semi-structured interviews -Activity documentation & reports -Implementing partner staff - USAID EQ 2a: Was the activity implemented according to the design? Has the implementation of the activity followed the original design? - Review of activity documents & reports - Semi-structured interviews -Activity documentation & reports -Implementing partner staff - USAID staff EQ 2b: What internal or external factors (including within USAID) have had the biggest impact on the implementation of the activity? Which factors may continue to impact the activity and which ones seem to have had short￾term/specific impact? - Review of activity documents & reports - Semi-structured interviews -Activity documentation & reports -Implementing partner staff - USAID staff -Departmental education stakeholders (e.g. inspectors) - Local authorities (e.g. mayors) EQ 2c: How did the implementing partner and the USAID education team address challenges during the implementation of the activity and what are some lessons learned? - Review of activity documents & reports - Semi-structured interviews -Activity documentation & reports -Implementing partner staff - USAID staff EQ 3: How has the capacity of the implementing partner (including the subcontractor) changed over the course of the activity? What gaps remain? - Review of activity documents & reports - Semi-structured interviews -Activity documentation & reports -Implementing partner staff - USAID staff D.SAMPLING This evaluation will use a purposive sample to collect data in schools and communities. An initial stratification of the 52 intervention schools was conducted according to urban and rural locations. For the purposes of this evaluation, we consider urban schools to be those within the towns of Sokone and Toubacouta. Village schools are classified as rural schools. Of the 52 interventions schools, three in Toubacouta and eight in Sokone are considered to be urban. Second, the team considered whether the schools had easy access to an ICT center. The sample includes at least one urban school and one rural school with access to ICT centers. Third, the team considered school size and distances from the main town. Schools that were too isolated were removed from the sample to allow the team to maximize the number of visits and interviews that can be conducted daily. Finally, the team conferred with Femme Plus to ensure that schools that have responded positively to the activity and those that have not also been included. Initially, the team had considered using the baseline reading scores from the project for additional stratification. Unfortunately, reading scores are not available for the schools in Sokone. For Toubacouta, the reading scores were used to verify that schools with high, medium and low performance had in fact been selected for the sample. Our Sisters Read Performance Evaluation Report 60 In total, 7 schools were chosen for this evaluation with three schools in Toubacouta, of which one is urban, and four schools in Sokone, of which one is urban. In each school, the evaluation team will conduct interviews with: • The school director • Teachers • Students – gender specific groups of students from CM1 and CM2 • Community members including individual interviews with the mayor, the village chief and religious leader and group interviews with parents and local associations. Throughout the implementation of the study, the team will consider other respondents who may provide additional information in response to the three evaluation questions. E. DATA ANALYSIS Data analysis will draw on findings organized by EQ. Qualitative data collected via the semi-structured individual and group interviews will be analyzed using thematic and content analysis. At the end of each day, team members will type up notes from the interviews, noting at the top of each interview guide any interesting or significant findings. Recordings of the interviews will be sent to the MSI team in Dakar for transcriptions and translations, as necessary. It is expected that transcriptions of interviews will be available within 1 or 2 days and sent back to the field team for review and analysis. The evaluation team will note patterns and develop themes from the interviews. The team will design a rubric to analyze the data and capture these emerging themes. Once the team has completed the thematic analysis, it will analyze key findings and themes using content analysis. Content analysis will help the team go beyond descriptions of changes in practices and attitude to identify the most salient characteristics of these changes. Quantitative data and statistics collected from activity documentation, e.g. reports, data sets, or government sources, will be analyzed using simple descriptive statistics such as frequency distribution. Quantitative data analysis will be led by the Senegal Mission-Wide Monitoring and Evaluation Project statisticians using SPSS statistical package. Finally, the evaluation team will use data triangulation, the use of two or more data sources, to confirm or disconfirm findings. F. FIELD WORK PLAN WGEP and Femmes Plus is collecting data on students reading levels in all 52 project schools in June 2016. The OSR implementers are using tools developed for each grade level to compare baseline and follow-up results in 12 schools in Toubacouta and 12 schools in Sokone. The evaluation team will analyze the assessment data to determine if student reading levels have improved since the baseline. The work plan below ensures that the draft report will be submitted to USAID/Senegal on 19 July 2016, and the final report on 9 August 2016. The tight timeline for this study does not allow for delays. National school examinations are scheduled for 21 and 22 June, which reduces the number of days available for the team to conduct school visits. Additionally, schools are unavailable to the team on 20 and 23 of June due to teacher travel in preparation for the exams. As a result, the team will have 4 days to visit schools, Our Sisters Read Performance Evaluation Report 61 including a Saturday. On days when schools are inaccessible, the team will collect data from the implementing partner and from community members. A timeline of activities for this evaluation is included in Annex II. Day Tasks Tuesday 14 June Team Planning Meeting Wednesday 15 June Draft work plan and tools for presentation to USAID and travel logistics Thursday 16 June Presentation of work plan and tools to USAID at 15:00 Friday 17 June Finalize work plan, tools Saturday 18 June Travel to Fatick Initial meeting with the “Inspecteur Départemental de l’éducation et de la formation” in Foundiougne Arrival in Toubacouta Sunday 19 June Evaluation team preparation for field work Monday 20 June Meeting and interviews with Femme Plus in Sokone Visits to and interview with temporary ICT center and mobile library in Sokone Interview with community in Sokone: • 1 individual interview with leaders, e.g. mayor Tuesday 21 June Interview with community in Sokone – Sokone 2 and Keur Yoro Diop • 2 individual interviews with leaders, e.g. religious leader, community leader • 1 group interview, e.g. parents, associations… Wednesday 22 June Visits to and interview with ICT center and mobile library in Toubacouta Interview with community in Toubacouta: • 1 individual interview with leaders, e.g. mayor • 1 group interview, e.g. association Thursday 23 June Interview with community in Toubacouta - Simong Diene and Bani • 2 individual interviews with leaders, e.g. religious leader, community leader • 1 group interview, e.g. parents, associations… Friday 24 June School and community interviews in Sokone - Diossong Saturday 25 June School and community interviews in Ndiokholokho (Sokone) and Sandicoly (Toubacouta) Sunday 26 June Return to Dakar (Gaëlle only) Monday 27 June School and community interviews in Sokone – Dantakhoune and KBGX Tuesday 28 June School and community interviews in Toubacouta - Karang Socé and Toubacouta 1 Wednesday 29 June Interviews with IF/IA Briefing with the “Inspecteur Academic” in Fatick Return to Dakar Thursday 30 June Meeting with MEN/DE and USAID Draft presentation of initial findings, conclusion and recommendations Friday 1 July Presentation of initial findings, conclusion and recommendations to USAID Our Sisters Read Performance Evaluation Report 62 Day Tasks Tuesday 5 July to Monday 18 July Data analysis and reporting of findings, conclusion and recommendations Tuesday 19 July Submission of draft evaluation report to USAID Friday 22 July Validation workshop with USAID, MEN and other partners and stakeholders Monday 25 July to Monday 8 August Review and finalization of evaluation report Tuesday 9 August Submission of final evaluation report Our Sisters Read Performance Evaluation Report 63 ANNEX 2.I: GETTING TO ANSWERS MATRIX N° Questions d’Evaluation Type de réponse/ Evidences Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entretiens avec des informateurs clés, Mini-enquête22 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Fréquence des Distributions, Analyses thématique, Cross￾Tabulations, Analyse du Contenu Source(s) de données Méthode 1 Quel était le niveau de précision de la théorie du changement qui sous-tend la conception et la mise en œuvre de l’activité ? • Anecdotes • Explication • Documents du projet : rapports trimestriels, contrat, plan de suivi et d’évaluation, rapport bihebdomadaire • Femmes Plus/WGEP • Entretien semi￾directif isolé • Visite des lieux d’intervention • Analyse thématique • Analyse du contenu 1a Les interventions prévues par l’activité (par exemple l’accès aux ressources extra scolaires, clubs de livres, concours, bourses, etc.) ont-elles conduit à motiver les élèves ? • Anecdotes • Statistiques (résultats des tests en lecture) • Comparaison23 • Explication24 • Documents du projet : rapports trimestriels, contrat, plan de suivi et d’évaluation, rapport bihebdomadaire • Base de données des scores en lecture (si disponible) • Directeurs d’école • Enseignants • Élèves • Membres de la communauté • Femmes Plus/WGEP • Entretien semi￾directif isolé • Entretien semi￾directif de groupe • Visite des lieux d’intervention • Exploration des données (résultats des tests en lecture) Echantillonnage des écoles par choix raisonné : • Urbain ou rurale • Centre TIC ou Sans centre TIC • Effectif total (grande ou petite) • Proche ou éloignée (frontalière) • Analyse thématique • Analyse du contenu • Analyse descriptive • Fréquence des distributions pour l’analyse de résultats en lecture • Analyse univariée pour résultats en lecture 22 Les données issues des évaluations sont des livrables et les méthodes doivent indiquer comment les données seront collectées, par exemple pour les focus groups USAID requièrent une transcription. 23 Comparaison – avec les données de base/initiales, les plans/cibles or autres standards ou normes 24 Explication – pour les questions qui cherchent à comprendre “pourquoi” et pour l’attribution d’un effet à une intervention spécifique (causalité) Our Sisters Read Performance Evaluation Report 64 N° Questions d’Evaluation Type de réponse/ Evidences Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entretiens avec des informateurs clés, Mini-enquête22 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Fréquence des Distributions, Analyses thématique, Cross￾Tabulations, Analyse du Contenu Source(s) de données Méthode 1b La formation des enseignants a-t-elle amélioré les pratiques de classe ? • Anecdotes • Comparaison • Explication • Documents du projet : rapports trimestriels, contrat, plan de suivi et d’évaluation, rapport bihebdomadaire • Directeurs d’école • Inspecteurs • Enseignants • Élèves • Femmes Plus/WGEP • Entretien semi￾directif isolé • Entretien semi￾directif de groupe • Visite des lieux d’intervention Echantillonnage des écoles par choix raisonné • Analyse thématique • Analyse du contenu 1c Les interventions prévues par l’activité (par exemple l'alphabétisation des adultes, les rencontres communautaires, le mentorat par des enseignants, etc.) ont-elles renforcé la mobilisation communautaire ? Quelles sont celles qui ont été les plus réussies ? • Anecdotes • Comparaison • Explication • Documents du projet : rapports trimestriels, contrat, plan de suivi et d’évaluation, rapport bihebdomadaire • Directeurs d’école • Enseignants • Élèves • Membres de la communauté • Femmes Plus/WGEP • Entretien semi￾directif isolé • Entretien semi￾directif de groupe • Visite des lieux d’intervention Echantillonnage des écoles par choix raisonné • Analyse thématique • Analyse du contenu 1d Quel a été le degré d’adhésion de la communauté à ce projet? Pourquoi ? • Anecdotes • Comparaison • Explication • Documents du projet : rapports trimestriels, contrat, plan de suivi et d’évaluation, rapport bihebdomadaire • Membres de la communauté • Femmes Plus/WGE • Entretien semi￾directif isolé • Entretien semi￾directif de groupe • Visite des lieux d’intervention Echantillonnage des écoles par choix raisonné • Analyse thématique • Analyse du contenu Our Sisters Read Performance Evaluation Report 65 N° Questions d’Evaluation Type de réponse/ Evidences Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entretiens avec des informateurs clés, Mini-enquête22 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Fréquence des Distributions, Analyses thématique, Cross￾Tabulations, Analyse du Contenu Source(s) de données Méthode 2 Dans quelle mesure le partenaire d'exécution (y compris le sous contractant) a-t-il atteint les objectifs poursuivis par l’activité ? • Anecdotes • Explication • Documents du projet : rapports trimestriels, contrat, plan de suivi et d’évaluation, rapport bihebdomadaire • Femmes Plus/WGEP • USAID • Entretien semi￾directif isolé • Visite des lieux d’intervention 2a Le niveau d’exécution de l’activité correspond-elle à ce qui était prévu ? • Anecdotes • Explication • Documents du projet : rapports trimestriels, contrat, plan de suivi et d’évaluation, rapport bihebdomadaire • Femmes Plus/WGEP • USAID • Entretien semi￾directif isolé • Visite des lieux d’intervention 2b Quels sont les facteurs internes ou externes (y compris au sein de l'USAID) qui ont le plus grand effet sur la mise en œuvre de l’activité ? Quels sont ces facteurs qui pourraient être rencontrés à nouveau et ceux qui semblent ponctuels ? • Anecdotes • Explication • Documents du projet : rapports trimestriels, contrat, plan de suivi et d’évaluation, rapport bihebdomadaire • Femmes Plus/WGEP • USAID • Inspecteurs • Entretien semi￾directif isolé • Visite des lieux d’intervention • Analyse thématique • Analyse du contenu 2c Comment le partenaire d’exécution et l’équipe Education a-t-il relevé les défis rencontrés dans la mise en œuvre de l’activité et quelles sont les leçons à en tirer ? • Anecdotes • Explication • Documents du projet : rapports trimestriels, contrat, plan de suivi et d’évaluation, rapport bihebdomadaire • Femmes Plus/WGEP • USAID • Entretien semi￾directif isolé • Visite des lieux d’intervention Our Sisters Read Performance Evaluation Report 66 N° Questions d’Evaluation Type de réponse/ Evidences Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entretiens avec des informateurs clés, Mini-enquête22 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Fréquence des Distributions, Analyses thématique, Cross￾Tabulations, Analyse du Contenu Source(s) de données Méthode 3 Comment les capacités du partenaire d’exécution (y compris le sous contractant) ont elles changé au cours de la mise en œuvre de l’activité ? Quelles sont les lacunes qui subsistent ? • Anecdotes • Explication • Documents du projet : rapports trimestriels, contrat, plan de suivi et d’évaluation, rapport bihebdomadaire • Femmes Plus/WGEP • USAID • Entretien semi￾directif isolé • Visite des lieux d’intervention Our Sisters Read Performance Evaluation Report 67 ANNEX 2.2: EVALUATION TIMELINE Tasks Timeframe 12 – 18 June 19 - 25 June 26 June – 2 July 3 – 9 July 10 – 16 July 17 – 23 July 24 – 30 July 31 July – 5 Aug 6 – 12 Aug Team planning meeting X Tools development, field planning, completion of Getting to answers matrix X Initial briefing with USAID X Interviews in Sokone and Toubacouta with beneficiaries and stakeholders X X Draft findings, conclusion and recommendations X Presentation of initial findings to USAID 1 Data analysis and draft evaluation report X X Submit draft evaluation report to USAID 19 Validation workshop with USAID, MEN, WGEP/FP and other partners 22 Revision and finalization of evaluation report following comments from USAID and results of validation workshop X X Submit final evaluation report to USAID 9 Our Sisters Read Performance Evaluation Report 68 ANNEX 2.3: ROLES AND RESPONSIBILITIES Person / Role Responsibilities Gaëlle Simon Lead evaluator (International consultant) Lead evaluator in charge of: • Tools development • Field work supervision • Quality assurance • Analysis of findings, conclusions and recommendations • Development of the evaluation report Abdourahim Gaye National Education Specialist (Local consultant) • Support development of work plan • Lead planning of field work • Liaise with education officials for approvals and meetings • Support the development of tools • Conduct interviews during field work • Support data analysis and reporting • Lead findings validation workshop Aïssatou Mbaye MEP M&E Associate • Document review and synthesis • Planning and organizing field work • Support the development of tools • Conduct interviews during field work • Support data analysis Souleymane Barry MEP Research Officer and Task Manager fr the OSR evaluation • Support to the design of tools • Support to the field work • Quantitative data analysis Transcriptions and translations • Transcribe recorded interviews • Translate interviews as required Moussa SY MEP Senior Evaluation Specialist • Support design of the tools, methodology and data analysis • Quality assurance Lisa Slifer-Mbacke MEP COP • Communication with USAID • Management of evaluation process • Quality assurance Our Sisters Read Performance Evaluation Report 69 ANNEX 3: INTERVIEW GUIDES Introduction • (Présentation personnelle et institutionnelle) Mon nom est ______________ et je travaille pour le compte de MSI, qui est contracté par l'USAID pour cette évaluation. Merci d'avoir pris le temps de me rencontrer. • (But de la visite) Je suis ici dans le cadre de l’évaluation des performances du projet « Nos Sœurs Lisent ». • (Temps de discussion) Nous avons [45 minutes - 1 heure] à passer ensemble pour discuter du projet. Êtes-vous disposé à répondre à quelques questions pour cette durée ? • (Consentement) Cet entretien est entièrement volontaire et vous êtes libre de ne pas participer. Si vous acceptez de participer, vous pouvez choisir d'arrêter à tout moment ou de sauter des questions auxquelles vous ne voulez pas répondre. Vos réponses et votre participation à cet entretien sont entièrement confidentielles. Nous ne partagerons aucune information qui vous identifie avec quiconque en dehors de l'équipe d’évaluation. • (Liberté d'arrêter) S'il vous plaît, cet entretien peut à tout moment être arrêté pour poser des questions au sujet de ce consentement ou toute autre chose. Ai-je votre consentement pour commencer? • (Transition) Avez-vous des questions à me poser avant de commencer ? (Noter les questions) Communauté Guide d’Entretien Semi-dirigé Mobilisation communautaire 1. Pour vous, en quoi consiste la mobilisation communautaire? A quoi sert-elle ? Est-elle importante ? Pourquoi ou pourquoi pas ? 2. A quelles activités de mobilisation communautaire participez-vous? o Si les répondants ne mentionnent pas les activités de NSL, demandez-leur: avez-vous participé à des cours d’alphabétisation de l’activité NSL? Si oui, selon vous quel est son utilité ? Pouvez-vous donner des exemples de ce que l’alphabétisation vous permet de faire et de mieux comprendre ? o Demandez aussi pour leur participation aux rencontres de communauté (community engagement meetings). Selon vous, quel est l’utilité de ces rencontres ? Qu’appréciez-vous le plus ? Comment amélioriez-vous ces rencontres ? Comment utilisez-vous les informations reçues ? o Demandez aussi pour leur expérience avec le mentorat avec les enseignants. 3. Quelles sont les activités de mobilisation communautaire les plus réussies ? Pourquoi ? Pouvez-vous donner des exemples des changements dans vos habitudes, pratiques, attitudes ou connaissances liés à votre participation dans ces activités ? 4. Comment les communautés se sont mobilisées pour le relèvement de l’accès et de la rétention dans les écoles ? Comment ses activités de mobilisation influencent l’accroissement des performances scolaires, et plus particulièrement en lecture ? Pratiques et attitude des membres de la communauté vis-à-vis de la lecture 5. Selon vous, quelle est l’importance de la lecture pour les adultes? Pour les jeunes? 6. Quelles initiatives avez-vous prises pour encourager les enfants à s’intéresser à la lecture (même en dehors de l’école) ? 7. Etes-vous suffisamment équipés pour aider vos enfants à améliorer leur performance en lecture ? o Si oui, comment le faites-vous ? Quels effets votre soutien a sur vos enfants ? Our Sisters Read Performance Evaluation Report 70 o Si non, comment l’activité NSL pourrait vous aider dans ce sens ? Que vous faudrait-il pour pouvoir aider vos enfants dans leur performance en lecture ? Motivation des élèves 8. Depuis l’avènement du projet NSL que constatez-vous chez vos enfants en terme de leur pratique en et intérêts pour la lecture? Demandez de donner des exemples précis de changements d’habitudes, de pratiques ou d’attitudes. 9. Quelles sont les activités de vos enfants en dehors de l’école ? Pouvez-vous nous donner des exemples précis? 10. Avez-vous remarqué une amélioration dans la performance de vos enfants à l’école, et plus particulièrement en lecture ? o Si oui, comment expliquez-vous cette amélioration ? Qu’est-ce qui a contribué à cette amélioration ? o Si non, comment le projet NSL pourrait intervenir ? 11. Avez-vous d’autres informations sur l’activité NSL à partager ? Enseignants Guide d’Entretien Semi-dirigé Formation des enseignants 1. Avez-vous reçu une formation à l’enseignement de la lecture de l’activité « Nos Sœurs Lisent » ? o Si oui, quelle est la nature de la formation que vous avez reçue ? Qui a animé la session de formation ? Qu’avez-vous le plus apprécié de cette formation ? Pourquoi ? Pouvez-vous me donner des exemples de nouvelles pratiques apprises lors de la formation et que vous utilisez en salle de classe ? [Si l’enseignant mentionne un matériel spécifique, demandez-lui de vous le montrer si possible]. o Si non, avez-vous entendu parler de l’activité « Nos Sœurs Lisent » ? De qui ? Que savez-vous de cette activité ? Si l’enseignant n’a pas reçu de formation et ne connais pas l’activité NSL, arrêtez l’entretien et remerciez l’enseignant pour sa participation. Notez que l’entretien a été arrêté. Amélioration des pratiques de classe 2. Comparé à votre pratique antérieure, en quoi vos leçons de lecture se sont bonifiées ? [Demandez de donner un ou plusieurs exemples précis : planification des leçons, matériels utilisés, méthode d’enseignement, organisation et gestion de la salle de classe, évaluations des performances des élèves….] Motivation et performance des élèves 3. Avez-vous remarquez une amélioration des performances des élèves en lecture depuis que vous utilisez les méthodologies et le matériel de l’activité NSL ? o Si oui, comment ? Demandez des précisions sur les méthodologies et matériels qui semblent les plus efficaces. o Si non, pourquoi? 4. Avec votre nouvelle pratique comment réagissent vos élèves lors des leçons de lecture? [Si nécessaire, ajoutez sont-ils plus engagés dans les leçons ? Participent-ils plus ? Sont-ils plus ou moins souvent absents?....] 5. Quelles pratiques liées à l’activité NSL et visant directement les élèves ont amélioré leur performance en ou leur intérêt pour la lecture? Pourquoi? Comment? o Si aucune ne sont effectives, pourquoi ? Qu’est-ce qui a échoué? Qu’est-ce qui manque? 6. Comment les élèves manifestent ils leur goût à la lecture? A quelles activités de lecture participent-ils le plus ? [Si l’enseignant ne mentionne pas les interventions de NSL, demandez s’ils connaissent les TIC, clubs, etc. Si oui, demandez s’ils savent si leurs élèves y participent.] 7. Qu’est-ce qui motive (ou ne motive pas) les élèves pour les activités de lecture ? Mobilisation communautaire Our Sisters Read Performance Evaluation Report 71 8. Quelles sont les initiatives prises par les parents ou les organisations communautaires pour encourager les enfants à s’intéresser à la lecture ? (Même en dehors de l’école) 9. Quelles sont vos appréciations sur les interventions de l’activité NSL destinées à la communauté (alphabétisation, mentorat…) ? Avez-vous déjà mené des activités de mentorat à l’endroit des élèves? Si oui, comment l’appréciez-vous ? Si non, pourquoi ? Avez-vous des recommandations dans ce sens ? 10. Avez-vous des autres informations sur l’activité NSL à partager ? Élèves Guide d’Entretien Semi-dirigé Pratiques et attitudes des élèves vis-à-vis de la lecture 1. Aimez-vous la lecture? o Si oui, pourquoi ? Pouvez-vous partager avec nous vos habitudes en lecture [livres que vous aimez lire ? dernier livre lu ? Livre préféré?] o Si non, pourquoi pas? 2. Lisez-vous des livres en dehors de l’école ? à la maison, dans les clubs de lecture, à la bibliothèque - quelle bibliothèque 3. A quelle fréquence lisez-vous ? le soir, les week-ends, pendant les vacances scolaires, jamais Fréquentation et participation aux activités du projet 4. Quelles sont les activités de lecture auxquelles vous avez participé ? [Si les élèves ne mentionnent pas les activités clés de NSL, demandez-leur] : o Fréquentez-vous la bibliothèque mobile ? Empruntez-vous des livres de la bibliothèque ? o Faites-vous partie d’un club de lecture ? Lequel ? Quelles sont les activités du club ? Qu’est-ce qui vous plaît le plus dans le club ? Quels sont les jours de fréquentation du club? o Est-ce que vous fréquentez le centre ICT ? Si oui pourquoi ? Si non pourquoi pas ? Qu’est-ce que vous aimez le plus dans les activités du centre ? Amélioration des pratiques de classe 5. Faites-vous souvent en classe des exercices se rapportant à la lecture ? Quels exercices de lecture faites-vous en classe ? Parmi les exercices de lecture, le quel aimez-vous le plus pourquoi ? Mobilisation communautaire 6. Est-ce que vos parents, ou sœurs et frères, vous encouragent-ils à lire ? Si oui, comment vous encouragent-t-ils [achats de livres, temps pour la bibliothèque] ? 7. Pendant les vacances qui viennent, quelles sont les activités que vous comptez faire ? Qu’est-ce que vos parents aimeraient que vous fassiez ? [Lire/Réviser/Cours de vacance ? Travailler aux champs ? Aller faire du commerce ou travailler à Dakar] Directeurs d’école et Inspecteurs Guide d’Entretien Semi-dirigé Formation des enseignants 1. Avez-vous participé à des sessions de formation dans le cadre de l’activité NSL ? Si oui, lesquelles ? Quel était votre rôle ? Quelle est votre appréciation de ces formations ? Comment peuvent-elles être améliorées? 2. Quelle stratégie de formation a été utilisée (formation directe, en cascade, à distance…) ? Si la formation a été en cascade qui ont été les formateurs des relais ? Qui ont été les relais auprès des enseignants ? Quel était le contenu de cette formation ? 3. À votre avis, est-ce que ces formations répondent aux besoins immédiats des enseignants et des élèves ? Our Sisters Read Performance Evaluation Report 72 o Si oui, quels sont ces besoins et comment les formations y répondent-elles ? Demandez des exemples précis. o Si non, quels ont les besoins immédiats des enseignants et des élèves et comment pensez-vous que l’activité NSL, à travers ces formations, pourrait y répondre ? 4. Comment ces formations pourraient-être améliorées ? Amélioration des pratiques de classe 5. Comment appréciez-vous les pratiques de classe des enseignants impliqués dans le projet NSL ? Demandez des exemples précis. 6. Comparé aux pratiques antérieures de ces enseignants, en quoi les leçons de lecture se sont bonifiées? Demandez des exemples précis de changements de pratique. 7. Quels aspects de l’activité NSL pensez-vous ont eu le plus grand impact sur les pratiques des enseignants en salle de classe et pendant les leçons de lecture ? Comment? Pourquoi? Motivation et performance des élèves 8. Dans vos visites de classe, lors des leçons de lecture, comment agissent/réagissent les élèves? Sont-ils impliqués dans les leçons ? Participent-ils activement ? Demandez des exemples précis. 9. Basé sur vos observations, comment a évolué durant l’année scolaire le niveau de performances des élèves en lecture? Est-ce qu’il y a une amélioration des performances des élèves depuis le début des interventions de l’activité NSL ? o Si oui, dans quel domaine/matière ? Comment expliquez-vous cette amélioration ? Demandez des exemples précis. Est-ce que le répondant est capable de lier les améliorations de performances à une intervention spécifique de NSL ? Facteurs qui ont influencé la mise en œuvre de l’activité 10. Comment appréciez-vous la mise en œuvre de l’activité NSL ? o Qu’est-ce qui a bien réussi ? Pourquoi? o Qu’est-ce qui n’a pas bien réussi ? Pourquoi ? Quelles suggestions feriez-vous pour améliorer la mise en œuvre de NSL ? Si le répondant souligne seulement des facteurs internes à NSL, posez des questions sur l’influence de facteurs externes tels que les autres projets en éducation, les directives du MEN, les grèves scolaires, etc. 11. Quelles leçons peut-on tirer de la mise en œuvre de cette activité ? 12. Avez-vous des autres informations sur l’activité NSL à partager ? Partenaire d’exécution Guide d’Entretien Semi-dirigé Mobilisation communautaire 1. Pouvez-vous nous décrire votre approche/votre stratégie pour la mobilisation communautaire ? (Quels étaient les cibles, objectifs…) Please describe the community mobilisation strategy used by the project? What were the objectives and targets? 2. Sur la totalité des interventions de l’activité lesquelles sont les plus efficaces/réussies pour la mobilisation communautaire? Pourquoi? Comment? Which activities were the most effective? Why ? Facteurs qui ont influencé la mise en œuvre de l’activité 3. Qu’est-ce qui a le plus facilité la mise en œuvre de l’activité ? (Si nécessaire, demandez des précisions par intervention, ex. centre TIC, les bibliothèques mobiles, alphabétisation, formation, tablettes, visite d’écoles/suivi, réunion communautaire….) What were some key factors or actors that supported the implementation of the project? How? Our Sisters Read Performance Evaluation Report 73 4. A quel niveau se situent les difficultés pour la mise en œuvre de l’activité? Comment les avez-vous contournées? What were some difficulties encountered during the implementation of the project? How were they solved? 5. À votre avis quelles difficultés pourraient resurgir ? Pourquoi? In your opinion which factors or actors will continue to be influential? Why? How? Capacités organisationnelles 6. Êtes-vous au courant des interventions dans le cadre de NSL pour développer la capacité institutionnelle de FP ? Si oui, quelles interventions ? Dans quels domaines (technique, suivi et évaluation, financier, ressources humaines, gestion/administration) ? What were some activities, implemented as part of OSR, designed for capacity building of FP? 7. Quels sont les besoins majeurs de FP en matière de renforcement des capacités ? What are some critical needs of FP in terms of capacity building? 8. Avez-vous remarqué un renforcement des capacités de FP durant NSL ? Si oui, citez des changements spécifiques en capacités (amélioration des ressources humaines, amélioration des systèmes financiers, etc.) Has there been capacity building of FP and WGEP throughout the implementation of the project? If so, can you give examples? 9. Quel est votre plan sur le long-terme pour la gestion durable de NSL ? What are the long-term plans for FP and WGEP? Partenaire d’exécution – La mairie Guide d’Entretien Semi-dirigé 1. Quelle appréciation avez-vous pour l’activité NSL? 2. Comment percevez-vous l’engagement de la communauté par rapport à cette activité ? 3. Comment soutenez-vous la mise en œuvre de l’activité NSL? 4. Quelles recommandations avez-vous pour l’activité NSL? 5. Avez-vous d’autres informations sur l’activité NSL à partager ? Our Sisters Read Performance Evaluation Report 74 ANNEX 4: SCHOOL LEVEL DATA FOR SAMPLING ECOLES DANS LE SOKONE Caractéristiques Urbaine s Rurale s Effectif Global Nombr e de Filles Possède un Centre (ICT) Distance de Sokone 1. Sokone 1 X 312 158 X 1.5 km/5 mn 2. Sokone 2 X 520 261 X 1 km/5 mn 3. Sokone 3 X 500 273 X 1 km/5 mn 4. Sokone 4 X 491 273 NON 500 m/5 mn 5. Sokone 5 X 173 80 NON 2 km/5 mn 6. Sokone 6 X 333 173 NON 2 km/ 5mn 7. Mbouloum X 220 110 X 3 km/ 10mn 8. Ndoffane M.D. X 55 28 NON 3 km/ 15 mn 9. Limane X 46 26 NON 4 km/ 25 mn 10. Badoudou X 97 49 NON 4 km/ 25 mn 11. Diaglé X 183 76 NON 2 km/ 10 mn 12. KGBX X X 1 km / 5 mn 13. EPST X 280 154 X 1.5 km/ 5 mn 14. Ndiambang X 83 42 NON 4 km/ 25 mn 15. Keur Yoro Diop X 69 44 NON 7 km/ 30mn 16. Diamaguene X 73 36 NON 2.5 km/ 15mn 17. Ndiaw Sérère X 57 30 NON 5 km/ 25 mn 18. NdraméSadiabou X 59 42 NON 7 km/ 35mn 19. Dantakhoune X 163 66 X 7 km/ 20 mn 20. Ndiokholokho X 93 42 NON 5 km/ 15 mn 21. Dagadiawdine X 93 48 NON 4 km/ 15 mn 22. Diossong X 151 95 X 6 km/ 30mn 23. Kébé Coudé X 106 64 NON 9 km/ 35mn 24. Bambougar Mal. X 104 59 NON 5 km/ 25 mn 25. Bambougar Mas. X 59 34 NON 7 km/ 30 mn 26. Bambougar El X 42 26 NON 7 km/ 30 mn 27. NdiogoneTallène X 160 82 NON 5 km/ 20 mn 28. NdiafféNdiaffé X 148 81 NON 7 km 15mn Our Sisters Read Performance Evaluation Report 75 ECOLES DANS LE TOUBACOUTA Caractéristiques Urbaines Rurales Effectif Global Nombre de Filles Possède un Centre (ICT) Distance de Toubacouta 1. Senghor × 59 29 NON 14KM/45MIN 2. Simong Diène × 68 32 × 10KM/30MIN 3. Keur Mama Lamine × 180 90 NON 12KM/20MIN 4. NAT × 113 67 NON 15KM/30MIN 5. KeurMalick Fadé × 34 16 NON 13KM/25MIN 6. Keur Aliou Guèye × 97 37 NON 7KM/15MIN 7. Toubacouta 1 × 396 218 × 1KM/3MIN 8. Toubacouta 2 × 192 87 NON 2KM/5MIN 9. Bany × 132 60 × 2MK/10MIN 10. DagaBabou × 137 69 NON 3KM/10MIN 11. Dassilamé Sérère × 146 68 NON 4KM/13MIN 12. Némabah × 160 93 NON 5KM/15MIN 13. Sandicoly × 151 92 NON 9KM/15MIN 14. Batamar × 57 35 NON 13KM/45MIN 15. Sadio Sala × 83 38 NON 11KM/30MIN 16. Médina sangako × 287 153 NON 4KM/10MIN 17. SanthieBerra × 67 34 NON 13KM/20MIN 18. Bamako × 105 58 NON 1KM/5MIN 19. Mansarinko × 72 34 NON 36KM/50MIN 20. Karang 3 × 396 291 NON 28KM/30MIN 21. KarangSocé × 131 56 NON 30KM/40MIN 22. Némanding × 134 77 NON 11KM/15MIN 23. Ndoumboudji × 227 113 NON 7KM/13MIN 24. Médina Santhie × 65 33 NON 13KM/20MIN Our Sisters Read Performance Evaluation Report 76 ANNEX 5: SOURCES OF INFORMATION Documents Reviewed Documents Dates USAID/Senegal - Cooperative Agreement # AID-685-A-13-00003 Sep 09, 2013 USAID/Senegal – Modification of Assistance # AID-685-A-13-00003 Oct 14, 2015 WGEP- Our Sisters Read, Performance Monitoring Plan WGEP - Our Sisters Read, Branding and Marking Plan WGEP- Our Sisters Read, Annual Implementation Plan, Year 1 0ct 01, 2013 - Sep 30, 2014 WGEP- Our Sisters Read, Year 1 First Quarterly Report Sep 09, 2013 - Dec 31, 2013 WGEP- Our Sisters Read, Year 1 Second Quarterly Report Jan 01, 2014 - Mar 31, 2014 WGEP- Our Sisters Read, Year 1 Third Quarterly Report Apr 01, 2014 - Jun 30, 2014 WGEP- Our Sisters Read Annual Report, Year 1 - Fourth Quarter 0ct 01, 2013 - Sept 30, 2014 WGEP - Our Sisters Read, Annual Performance Indicator Report 0ct 01, 2013 - Sept 30, 2014 WGEP – Our Sisters Read – Bi-weekly reports Dec 01, 2013 - July 15, 2014 WGEP- Our Sisters Read Annual Implementation Plan, Year 2 0ct 01, 2014- Sept 30, 2015 WGEP- Our Sisters Read, Year 2 First Quarterly Report Oct 01, 2014 – Dec 31, 2014 WGEP- Our Sisters Read, Year 2 Second Quarterly Report Jan 01, 2015 – Mar 31, 2015 WGEP- Our Sisters Read, Year 2 Third Quarterly Report Apr 01, 2015 - Jun 30, 2015 WGEP- Our Sisters Read Annual Report, Year 2 - Fourth Quarter Oct 01, 2014- Sept 30, 2015 WGEP – Our Sisters Read – Bi-weekly reports Oct 15, 2014 – Dec 15, 2014 WGEP – Our Sisters Read – Bi-weekly reports Jan 15, 2015 – Apr 01, 2015 WGEP – Our Sisters Read – Bi-weekly reports Jul 15, 2015 – Aug 01, 2015 WGEP- Our Sisters Read Annual Implementation Plan, Year 3 0ct 01, 2015- Sept 30, 2016 WGEP- Our Sisters Read, Year 3 First Quarterly Report Oct 01, 2015 – Dec 31, 2015 WGEP- Our Sisters Read, Year 3 First Quarter Monitoring Report Oct 01, 2015 – Dec 31, 2015 WGEP- Our Sisters Read, Year 3 Second Quarterly Report Jan 01, 2016 – Mar 31, 2016 WGEP – Our Sisters Read – Bi-weekly reports Sep 15, 2015 WGEP – Our Sisters Read – Bi-weekly reports Oct 15, 2015 – Jan 01, 2016 WGEP – Our Sisters Read – Bi-weekly reports Feb 01, 2016 – Jun 30, 2016 WGEP- Our Sisters Read Performance Indicators Review Oct 2015 WGEP- Our Sisters Read Performance Indicators Sheets 2015 - 2016 USAID/Senegal -Our Sisters Read - Data Quality Assessment Report March 8 to 10, 2016 WGEP - Budget – Standard Form 424A 2013 WGEP – Annual work plan budget – Year 1, Year 2, Year 3 WGEP – Plan Suivi Environnement Nov 13, 2015 WGEP – Plan Suivi Environnement Dec 11, 2015 WGEP – Student test scores in reading (24 Toubacouta schools) Dec 2015 WGEP – Student test scores in reading (24 schools in Toubacouta and Sokone) June 2016 Evaluation de la qualité des apprentissages, Jangandoo, IFAN 2014 PASEC (2007). Evaluation du PASEC, CONFEMEN, Dakar 2007 PASEC 2014- Performances des systèmes éducatifs en Afrique Subsaharienne francophone : Compétences et facteurs de réussite au primaire. PASEC, CONFEMEN, Dakar. 2015 Etat des lieux de l’éducation de base au Sénégal, MEN 2014 Our Sisters Read Performance Evaluation Report 77 Respondents # Date Respondent Interview Title 1 20/06/2016 Adjaratou Senghor Individual Présidente de l’Association Femme Plus 2 20/06/2016 Mbathio Diouf Group Encadreurs en Lecture Sokone Alioune Badara Gueye Encadreurs en Lecture Sokone Guillaume Diouf Encadreurs en Lecture Sokone Ndiatté Sene Formatrice en Informatique Sokone Dulfé Serrano Bibliothécaire Sokone 3 20/06/2016 Abdoulaye Mbaye Individual Personne ressource et parent d’élève 4 20/06/2016 Moustapha Guèye Individual Maire de Sokone 5 20/06/2016 Ndèye Khady Dème Group Trésorière Femme Plus Fatoumata Dème Membre Association Femme Plus Diamal Sarr Membre Association Femme Plus Sokhna Amy Ndiaye Membre Association Femme Plus Mada Senghor Membre Association Femme Plus 6 21/06/2016 Pape Seydou Ndianko Individual Maire de Toubacouta 7 21/06/2016 Demba Bâ Individual Chef du village de Keur Yoro Diop 8 21/06/2016 Momodou Sow Individual Imam Keur Yoro Diop 9 21/06/2016 El Hadji Amadou Bâ Group Trésorier CGE École De Keur Yoro Diop Abdou Bâ Président APE de Keur Yoro Diop 10 21/06/2016 Aïssata Bâ Group Présidente AME de Keur Yoro Diop Sira Bâ Membre AME de Keur Yoro Diop Aïssata Sow Membre AME de Keur Yoro Diop 11 21/06/2016 Mame Maty Ndiaye Group Présidente De Sop Sa Dieukeur De Sokone Adama Ndiaye Membre De Sop Sa Dieukeur De Sokone Fatou Deme Membre De Sop Sa Dieukeur De Sokone 12 21/06/2016 Abdoul Aziz Deme Individual Imam à Sokone 13 21/06/2016 Omar Cissé Group Relais à Sokone Bocar Bâ Relais à Keur Yoro Diop Demba Guèye Relais à Senghor 14 22/06/2016 Khady Gnima Sarr Group Relais pour Banny Bourama Ndour Parent d’élève Banny Sanoussi Diamé Parent d’élève Banny Ansou Diamé Parent d’élève Banny Alim Sonko Trésorier CGE Ecole Banny Abdou Diamé Imam du village Banny Youssoupha Sarr Parent d’élève Banny Ibrahima Diamé Parent d’élève Banny Momodou Diouf Parent d’élève Banny Momodou Sarr Parent d’élève Banny Sana Diatta Parent d’élève Banny Seyni Mané Parent d’élève Banny 15 22/06/2016 Abdoulaye Bodian Group Radio Niombato FM Toubacouta Moussa Mané Radio Niombato FM Toubacouta 16 22/06/2016 Rokhaya Diouf Group Réseau des Femmes Toubacouta Aminata Senghor Réseau des Femmes Toubacouta Yandé Ndao Réseau des Femmes Toubacouta Mariama Diamé Réseau des Femmes Toubacouta 17 22/06/2016 Moustapha Senghor Group Encadreur ICT Toubacouta Our Sisters Read Performance Evaluation Report 78 # Date Respondent Interview Title Diarraye Ndiaye Encadreur Toubacouta Mariama Seydi Encadreur Toubacouta Aminata Thior Encadreur Toubacouta Saphiatou Senghor Bibliothécaire Toubacouta Moustapha Fall Formateur en Informatique Toubacouta 18 23/06/2016 Seynabou Sarr Group Association Badjenou Gokh Simong Diène Jacqueline Kama Association de Simong Diene Marie Sarr Parent d’élève Simong Diène Diaffe Sarr Chef De Village Simong Diène Boundaw Diouf Parent d’élève Simong Diène Thierno Sarr Représentant de l’Imam Simong Diène Mr Niang Directeur CGE Ecole Simong Diène Diadjé Sarr APE Simong Diène 19 23/06/2016 Abdoulaye Sarr Group Président CGE Toubacouta Souleymane Sylla Trésorier APE Toubacouta Boubacar Senghor Président APE Toubacouta Ndèye Fatou Seydi Relais Toubacouta Dominique Dione Directeur Ecole Primaire de Toubacouta Bacary Sarr Association Villageoise De Toubacouta 20 23/06/2016 Mr Cheikh Faty Group Directeur Ecole Médina Sangako Momodou Séne Président CODEC 21 24/06/2016 Khady Ndong Individual Stagiaire et tutrice à Femme Plus 22 24/06/2016 El Hadji Wade Individual Directeur Ecole Diossong 23 24/06/2016 Ousmane Dia Group APE Diossong Mouhamadou Lamine Cissé Chef de village de Diossong Amy Collé Ndiaye Relais Diossong Aliou Dieng CGE Diossong Awa Sylla AME Diossong Aïssatou Niass AME Diossong Ousmane Cissé APE Diossong Mariama Lô AME Diossong Amy Cissé APE Diossong Babacar Cissé Association Diossong 24 24/06/2016 Aziz Lam Group Enseignant CI Diossong Pape Amadou Seck Enseignant CP Diossong Monsieur Wade Enseignant CE1 Diossong Madame Fatou Sarr Enseignant CE2 Diossong Papa Amath Diouf Enseignant CM1 Diossong Fall Mame Ngoné Diéne Enseignant CM2 Diossong 25 24/06/2016 08 Elèves Filles Group Diossong 26 24/06/2016 08 Elèves Garçons Group Diossong 27 24/06/2016 Nouha Sonko Group CI CP Français Médina Sangako Momodou Fall Maître Arabe Médina Sangako Sekou Sarr CM1 et CM2 Français Médina Sangako Maimouna Diedhiou CE1 et CE2 Français Médina Sangako Alassane Senghor CE1 et CE2 Arabe Médina Sangako Mohamed Lamine Diouf CM1 et CM2 Arabe Médina Sangako Momodou Sarr CI CP Arabe Médina Sangako 28 24/06/2016 Aliou Senghor Group Chef de village Médina Sangako Guèye Fall Chef de village Médina Sangako Abdoul Khamid Fall Encadreur Arabe Médina Sangako Mouhamed Senghor Encadreur Français Médina Sangako Our Sisters Read Performance Evaluation Report 79 # Date Respondent Interview Title Momodou Senghor Encadreur Français Médina Sangako Mba Senghor Encadreur Arabe Médina Sangako Amsatou Diame Encadreur Français Médina Sangako Awa Thior Présidente AME Médina Sangako 29 24/06/2016 08 Elèves Filles Group Médina Sangako 30 24/06/2016 08 Elèves Garçons Group Médina Sangako 31 25/06/2016 Moustapaha Diouf Individual Inspecteur District de Toubaouta-Sokone 32 25/06/2016 Coumba Sarr Individual Directrice Ecole Primaire de Sandicoly 33 25/06/2016 Lamine Diouf Group Enseignant CM1 Sandicoly Sakou Sene Enseignant CI Sandicoly Madieyna Diouf Enseignant CM2 Sandicoly Ousseynou Sarr Enseignant CE2 Sandicoly 34 25/06/2016 11 Elèves Filles Group Sandicoly 35 25/06/2016 Yama Thiam Group Parent d’élève Sandicoly Fatma Ndong Parent d’élève Sandicoly Djibril Diouf Conseiller Municipal Sandicoly Sidi Senghor CGE Sandicoly Sophie Cissé Parent d’élève Sandicoly 36 25/06/2016 06 Elèves Garçons Group Niokholokho 37 25/06/2016 Souhaïbou Diemé Individual Directeur École Primaire de Niokholokho 38 25/06/2016 Moussa Badji Group Enseignant CI CP Niokholokho Cheikhou Mboup Enseignant CM1 CM2 Niokholokho 39 27/06/2016 Mame Coumba Ndianko Group Enseignant CM1 Ecole Keur Gui Bok Xol Hamady Diallo Enseignant CE2 Ecole Keur Gui Bok Xol Ndèye Souadou Ndong Enseignant CP Ecole Keur Gui Bok Xol 40 27/06/2016 Ibrahima Sall Individual Directeur Ecole Keur Gui Bok Xol 41 27/06/2016 Salimatou Diallo Group Parent d’élève Dantakhoune Gallo Wagué Présidente CGE Dantakhoune Ramatoulaye Marrone Parent d’élève Dantakhoune Khady Senghor Parent d’élève Dantakhoune Marie Faye Parent d’élève Dantakhoune Marie Diouf Parent d’élève Dantakhoune Adam Diouf Parent d’élève Dantakhoune Madjiguéne Faye Parent d’élève Dantakhoune Yacine Diallo Parent d’élève Dantakhoune Khady Senghor Parent d’élève Dantakhoune Sadio Sidibé Parent d’élève Dantakhoune Haby Mariko Parent d’élève Dantakhoune Saly Doumbaya Parent d’élève Dantakhoune Hibou Diouf APE et Chef de Village Dantakhoune Mariama Sarr Relais Dantakhoune 42 27/06/2016 09 Elèves Filles Group Dantakhoune 43 27/06/2016 Amath Thiam Group Enseignant CE1 Dantakhoune Pierre Ndong Enseignant CP Dantakhoune Khar Guissé Enseignant CM2 Dantakhoune Amadou Diouf Enseignant CM1 Dantakhoune Lamine Seydi Enseignant CI Dantakhoune 44 27/06/2016 Lamine Seydi Individual Directeur Ecole Primaire De Dantakhoune 45 27/06/2016 Amy Maglio Group Executive Director WGEP Lily Messih Program Manager WGEP 46 28/06/2016 Dioguéne Gomis Group Maire de la commune de Karang Socé Yacine Badiane 1ère Adjointe au Maire Karang Socé Our Sisters Read Performance Evaluation Report 80 # Date Respondent Interview Title Fatoumata Badiane Ndiaye Assistante du Maire Karang Socé 47 28/06/2016 Daba Ndour Group Parent d’élève Karang Socé Gorgui Séne Président APE Karang Socé Diatou Diouf Parent d’élève Karang Socé Yahya Sarr Chef de village Karang Socé Saliou Ndour Président CGE Karang Socé Coumba Diouf Parent d’élève Karang Socé Elisabeth Diouf Parent d’élève Karang Socé Aissatou Diouf Trésorière CGE et relais Karang Socé 48 28/06/2016 Cherif Diop Group Enseignant CE1 Karang Socé Oumy Diatta Enseignant CE2 Karang Socé Abdoulaye Diouf Enseignant CM2 Karang Socé Talla Dieng Maître d’Arabe Karang Socé Alassane Ndiaye Enseignant CI Karang Socé Ansoumana Bassene Enseignant CM1 Karang Socé Dramé Sow Enseignant CP Karang Socé 49 28/06/2016 07 Elèves Garçons Group Karang Socé 50 28/06/2016 Sory Traoré Individual Directeur Ecole Primaire Karang Socé 51 28/06/2016 Bacary Demba Group Enseignant CM2 Toubacouta 1 Syaka Sané Enseignant CI Toubacouta 1 Oumar Diamé Sarr Maître d’Arabe Toubacouta 1 Ngoné Niang Enseignante CE2 Toubacouta 1 Kekoaye Demba Enseignant CM1 Toubacouta 1 52 28/06/2016 06 Filles + 07 Garçons Group Toubacouta 1 53 28/06/2016 Dominique Dione Individual Directeur École Primaire Toubacouta 1 54 29/06/2016 Mme Khady Diop Mbodji Individual Directrice DEE 55 30/06/2016 Ndeye Doufi Diagne Group USAID/Senegal, Voucher Examiner Francis Carayol USAID/Senegal, Financial Analyst 56 01/07/2016 Badara Sarr Individual USAID/Senegal, Education Specialist Our Sisters Read Performance Evaluation Report 81 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov