AUGUST 2016 This publication was produced for review by the United States Agency for International Development. It was prepared by Dr. Karla Giuliano Sarr and Groupe d’Etudes de Recherche et d’Appui au Développement for USAID/Senegal Monitoring and Evaluation Project. PERFORMANCE EVALUATION OF THE FIXED￾AMOUNT REIMBURSEMENT AGREEMENT (FARA) RURAL AND URBAN SCHOOL CONSTRUCTION AND WATER POINT INSTALLATION ACTIVITY PERFORMANCE EVALUATION OF THE FIXED AMOUNT REIMBURSEMENT AGREEMENT (FARA) RURAL AND URBAN SCHOOL CONSTRUCTION AND WATER POINT INSTALLATION ACTIVITY PERFORMANCE EVALUATION USAID/Senegal Contracted under AID-685-C-15-00003 USAID Senegal Mission-Wide Monitoring and Evaluation Project DISCLAIMER This evaluation is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents are the sole responsibility of Management Systems International, a Tetra Tech company, and do not necessarily reflect the views of USAID or the United States Government. i CONTENTS Acknowledgments..........................................................................................................................ii Cover Photo.................................................................................................................................................................ii Acronyms iii Executive Summary...................................................................................................................... 1 Evaluation Purpose and Evaluation Questions.................................................................................................... 1 Project Background ................................................................................................................................................... 1 Evaluation Design, Methods and Limitations....................................................................................................... 2 Findings and Conclusions......................................................................................................................................... 2 Recommendations...................................................................................................................................................... 4 Evaluation Purpose & Evaluation Questions .............................................................................. 1 Evaluation Purpose..................................................................................................................................................... 1 Evaluation Questions................................................................................................................................................. 1 Activity Background...................................................................................................................... 2 Evaluation Methods and Limitations........................................................................................... 4 Data Collection Methods and Source................................................................................................................... 4 Data Analysis Methods.............................................................................................................................................. 5 Limitations.................................................................................................................................................................... 6 Findings, Conclusions and Recommendations ........................................................................... 7 Question 1................................................................................................................................................................... 7 Question 2.................................................................................................................................................................14 Question 3.................................................................................................................................................................19 Question 4.................................................................................................................................................................24 Question 5.................................................................................................................................................................32 Recommendations....................................................................................................................................................36 Annex I: Evaluation Statement of Work .................................................................................. 39 Annex II: Evaluation Methods and Limitations ........................................................................ 66 Data Collection Methods and Source.................................................................................................................66 Sampling Strategy......................................................................................................................................................69 Limitations..................................................................................................................................................................70 Data Analysis Methods............................................................................................................................................71 Annex III: Data Collection Instruments.................................................................................... 72 Annex IV: Sources of Information........................................................................................... 105 Annex V: Disclosure of Any Conflicts of Interest................................................................... 107 Annex VI: Figure: Student Population Change at Constructed Schools ............................. 109 ii ACKNOWLEDGMENTS The authors wish to thank Alhassane Ndiaye for his dedication to the FARA school construction activity, as well as his support throughout the evaluation process. Cover Photo Inaugurated in February, 2016, the Simbandi Balante Middle School in the Sedhiou Region was the last school to be built as part of the school construction activity. The photo shows the administrative building with the USAID plaque. To the left of the building is a plant-based temporary classroom structure required to meet the needs of the large school population. Photo by Karla Giuliano Sarr iii ACRONYMS APE Association des Parents d’Elèves/Parent Teacher Association CGE Comité de Gestion de l’Ecole/School Management Committee CTR Coordinateur technique régionale/Regional Technical Coordinator DAGE Direction de l’Administration Générale et de l’Equipement/General Administration and Equipment Direction DCEF Direction de la Coopération Economique et Financière/Financial and Economic Cooperation Direction DCS Direction des Constructions Scolaires/Education Construction Direction DEMSG Direction de l’Enseignement Moyen Secondaire Général/General Middle Secondary Education Direction DI Direction de l’Investissement/Investment Direction DPRE Direction de la Planification et de la Réforme de l’Education/Planning and Education Reform Direction FARA Fixed Amount Reimbursement Agreement GERAD Groupe d’Etudes de Recherche et d’Appui au Développement G2G Government-to-Government GOS Goverment of Senegal IA Inspection d’Académie/Academic Inspection IEF Inspection de l’Education et de la Formation/Education and Training Inspection MEF Ministère de l’Economie et des Finances/Ministry of Economy and Finances MEN Ministère de l’Education Nationale/Ministry of National Education MEP Senegal USAID Monitoring and Evaluation Program MSI Management Systems International PTA Parent Teacher Association RTC Regional Technical Coordinator SMC School Management Committee SOW Statement of Work USAID U.S. Agency for International Development 1 EXECUTIVE SUMMARY The school construction and water point installation activity underwent a mid-term evaluation in 2012 that concentrated largely on technical areas of structural effectiveness, program operations and quality assurance. The Activity expired in June 2016, so USAID has, in collaboration with the Government of Senegal, commissioned this final evaluation. Evaluation Purpose and Evaluation Questions This evaluation will focus on questions related to access and student retention, as well as gaining a better understanding of evolving institutional capacity and the collaborative mechanisms that guided implementation. The lessons learned from this study will allow the Ministry of Education and USAID to inform future interventions. The primary audiences for the evaluation are USAID and the Government of Senegal. Its results will help the Education Team at USAID/Senegal and USAID/Washington overall, as part of the Agency’s ongoing process to revise its programs that deal with conflict and crisis zones, with the goal of eventually obtaining new agreements with the Government of Senegal that support USAID’s overall global strategy. The evaluation may also help the Government of Senegal adjust its policies on school construction and management of school equipment, as well as access to the education system in general. Finally, the results of the evaluation could provide a reference for all stakeholders (international development agencies, civil society, private sector, government, local collectives, etc.) involved with the creation and management of educational infrastructure in particular and public infrastructure in general. The evaluation findings could also contribute to lessons learned about the current government-to￾government mechanisms and strategies to involve the community in public infrastructure management. Project Background The Fixed Amount Reimbursement Agreement (FARA) school construction activity in Senegal has operated since 2006, supporting the Senegalese Ministry of Education (MEN) and Ministry of Finance (MEF) to build middle schools and water points in 11 regions of the country (Matam, Saint-Louis, Kaffrine, Fatick, Dakar, Thies, Diourbel, Kolda, Tambacounda, Louga and Sedhiou). This USAID program is unique in its focus on school construction and its government-to-government approach. Within this strategy, USAID and the Government of Senegal reached an agreement by which USAID reimburses the Government of Senegal a fixed amount of funds to cover the construction of middle schools, school materials (including desks and books) and associated water sources. Since 2006, the Activity has supported the construction of 46 schools and 30 water sources. In addition, the strategy has inspired the Government of Senegal to develop a parallel program, through which it has constructed additional schools and water sources. Because the Ministry instituted a matching project where the Government of Senegal matched funds for USAID interventions, over time the program simply became known as Matching within the concerned ministries. The funding amount for the activity (2007-2016) was USD $9,707,500. The evaluation team did not receive documents that outlined the development hypothesis, but the evaluation led the team to understand that the creation of permanent physical local middle schools would improve access and the retention of students, particularly girls, by reducing the need to travel and improving the learning environment by providing a safe space for students to learn. 2 Evaluation Design, Methods and Limitations The evaluation team visited half (23) of the schools constructed through the Activity to collect enrollment and retention statistics from 2007 to present. At these schools, the team spoke with principals and conducted observations of the school buildings and water points to gain perspective about the maintenance of the structures. At 12 of the 23 schools, the team facilitated several additional qualitative activities that allowed a deeper understanding of collaboration and community engagement. In addition to visiting the schools, the evaluation team met with key informants at ministries and with USAID/Senegal. This evaluation paid special attention to how these newly constructed middle schools and water sources may respond to the differing needs of girls and boys. To the extent possible, the evaluation team disaggregated data by gender. The evaluation used the following data collection methods below (see Annex II for a more complete discussion of evaluation methods):  Desk review: Identification and review of relevant technical reports and background documents;  Qualitative methods: Key informant interviews, focus groups, structured observation, photos; and  Quantitative methods: Completion of school statistics and school construction summary sheets. Data analysis drew on findings and was guided by the evaluation questions. Qualitative data was analyzed using thematic and content analysis and was supported through the use of NVivo qualitative analysis software. NVivo assisted primarily in isolating data according to themes and sources. The analysis of quantitative data was largely descriptive, focusing on student data and addressing issues of access and retention A number of limitations existed for the evaluation. The timeline was a significant issue in conducting the evaluation. The Koritè (Eid Al Fitr) holiday fell in the middle of data collection, requiring a two-phase field strategy. In addition, school was ending for the summer, so many students and some teachers were unavailable for interviews and focus groups. Project-wise, the school construction activity began in 2007 and evolved greatly during the project cycle. Widespread personnel turnover limited interviews with individuals who could remember the beginning of the project. A lack of background documentation on the developmental hypothesis or results framework for this particular activity further complicated the study. Because USAID prioritized inclusion of schools built in certain conflict-zone areas in the South as part of the sample, the spread of sample schools built across the four phases of the Activity is uneven (most schools in the South were built under phases three and four). This means that the data collected may not be representative of the 23 Activity schools that were not part of the evaluation. Findings and Conclusions Between 2007 and 2016, the school construction activity built and equipped 46 middle school complexes and installed 30 water points throughout 11 regions of Senegal. In all but one case, the schools had already been established and the Activity complexes replaced temporary structures. The school complexes utilized a common design that included an administrative block, four classrooms, one laboratory, one multi-purpose room, separate lavatories for girls and boys, staff toilets, a school guard post and a cement brick-wall enclosure with gate. 3 Overwhelmingly, study participants applauded USAID’s construction of school complexes and water sources. Findings indicate that the Activity has responded to the needs of many school populations by providing high-quality and efficiently constructed physical structures that have greatly improved students’ and school personnel’s safety and comfort. Increased capacity and closer ties to the local community have allowed some schools to reincorporate children who had previously left school. The number of students attending the middle schools involved in the Activity improved greatly when compared with the student population previous to construction. Unfortunately, it is not possible to control for external factors, including the increase in primary school enrollment within the context of Education for All, the commitment by member states of the UN Educational, Scientific and Cultural Organization (UNESCO) to provide quality basic education for all children, youths and adults. While school construction has contributed to improving schooling conditions for many students and teachers, upkeep and maintenance issues pose serious health and safety risks and threaten the sustainability of the constructed school complexes. Bathroom upkeep and maintenance is a significant problem; in six of the 23 schools, these facilities are no longer functional. Many students, even those at schools with functioning bathrooms, are unsatisfied with the conditions and resort to going home, to neighborhood houses or in the bush behind the schools to relieve themselves. In addition, 17 of the 23 schools visited demonstrated a lack of adequate classrooms for the school population. While the cause of the mismatch between school population and the size of the complexes built is unclear, one standard prototype was used in 45 of the 46 schools constructed through this activity. Schools adapted to overpopulation in different ways. Notably, the laboratories at all of the schools and many of the multi￾purpose rooms/computer labs have been converted into regular classrooms. Similarly, eight of the 23 schools continue to function with temporary structures on the school grounds. Complications also include a lack of electricity and inconsistent access to water sources for some of the schools. Although the Activity has improved the capacity of the education system in providing more permanent structures, a more adaptive school complex design might have been more appropriate and even more cost￾effective. USAID viewed the Activity as a unique opportunity for government-to-government assistance that included institutional capacity development for the Government of Senegal. Overall, the Activity proved that the Ministry of Education (MEN) can effectively build middle schools in a timely, cost-effective and quality manner. While MEN efforts have improved capacity to manage the school construction process and oversight as well as understanding of community participation approaches, these are largely individual achievements. No systematic changes were visible and little has been adopted out to the larger MEN systems. For example, the issue of transportation costs and means to provide regular guidance and quality control over the construction of schools still exists. No indication exists that at the end of the Activity, MEN will continue to utilize any of the Activity’s processes or unique components, such as the steering committee. Community mobilization was a key component of the Activity. School and community actors from eight of the 12 schools involved in qualitative data collection activities confirmed that school construction community mobilization meetings took place led by the Direction de l’Enseignement Moyen Secondaire Général/General Middle Secondary Education Direction (DEMSG). Community support for schools included identifying the land for the school, providing water and labor for construction, assuring electricity and connections to water sources and planting trees. In general, however, community involvement during the construction process was limited. Weak relationships and unclear roles between communities and contractors, as well as inflexibility in school design, may have lessened contributions and dampened ownership by communities of the school complexes. Similarly, the short duration of Activity community mobilization efforts and external socioeconomic and cultural factors may further complicate sustained community ownership of schools and participation in 4 building management and maintenance activities. Moreover, in almost all cases, communities had actively advocated for and organized to establish their local schools well before the Activity constructions. Given this notable history of community engagement, it is impossible to discern the extent to which Activity community mobilization efforts may have affected community ownership of schools. In terms of collaboration between official structures, the Activity implicated actors across the central, regional and school/local levels. At the central level, interviews with ministry officials repeatedly identified two strong collaborative mechanisms in particular: the Activity steering committee and joint field visits. While these collaborative activities began with much fervor at the inception of the Activity, they waned significantly towards the end and the level of communication amongst actors is questionable. Similarly, at the local and regional levels, collaboration seems to have been hampered by a lack of information-sharing; some community actors identified great frustration about feeling like bystanders throughout the construction process. The individualized roles of certain actors, particularly the pivotal role of the School Construction Office (DCS) FARA coordinator, may also have weakened collaboration. Additional external factors are noteworthy, including limited parental involvement. At the same time, some notable examples stand out, especially in terms of the roles of certain mayors’ offices in identifying school sites as well as some district councils (conseil départemental) supporting schools through payment of personnel as well as utilities. One school in particular, Hilele Middle School (Sedhiou Region), provides an excellent example of coordination among school and local actors to contribute to the upkeep and operations of the school. While a few schools demonstrated effective management strategies, intensive training and ongoing coaching are needed at the school level as well as with regional actors. In addition, school upkeep as currently organized may follow cultural practices that involve girls more than boys in cleaning activities, reinforcing marginalization. Recommendations Understanding that actors at USAID as well as the Government of Senegal’s Ministries of Education and Finance may consider this report in the elaboration of future school construction programs, the evaluation team offers the recommendations that follow. (For the fully detailed list of recommendations see the full “Findings, Conclusions and Recommendations” section of the evaluation report.): USAID  Continue to support the Government of Senegal in its school construction efforts in response to the evident need, especially of school populations still in temporary structures.  Require trimester reports to be completed by the Government of Senegal with updates from each ministerial office.  Identify an ongoing training and mentorship approach for school administrators, regional school construction representatives and contractors in community engagement and ongoing community mobilization strategies. USAID & Ministry of Education & Ministry of Finance  Identify clear roles, responsibilities and expectations for both preventative and ongoing school maintenance.  Develop explicit language within the Activity regarding institutional capacity building, if this is truly an objective of the G2G work.  Focus on MEN systems and how to improve collaboration and joint monitoring of school construction within a FARA. 5 Ministry of Education  Reinforce the role of the DCS FARA activity coordinator to assure the political authority necessary to convene actors from multiple ministerial offices.  Reinstate the Steering Committee and the use of joint field visits.  Institute regular reporting requirements from each directorate to compile updates and lessons learned on the Activity, perhaps by quarter.  Reinforce the involvement of regional actors in school construction and school maintenance projects.  Adopt a similar high level of quality control and oversight for all MEN school construction, which would require additional funds, manpower and means of transportation for the DCS. Direction des Constructions Scolaires (DCS)  Continue to work with competent and responsible contractors, enforcing technical specifications and further strengthening coordination through the Regional Technical Coordinators.  Emphasize the need for a reliable water source at the time that construction begins.  Identify a school model that can be tailored to the needs of schools and their projected population in the future. Conduct needs assessments prior to construction.  Reinforce lessons learned about equipment procurement and distribution and involve regional actors in the distribution of materials. Direction de l’Enseignement Moyen Secondaire Général (DEMSG)  Strengthen and lengthen community mobilization efforts to respond to the actual needs of communities.  Provide ongoing mentorship to school principals on school maintenance and include related modules in training for new principals.  Provide gender training for school personnel, community members and students to make them aware of how gender stereotypes and roles may inhibit learning, particularly for female students.  Include proper maintenance as a criterion for performance review of principals. School Principals  Foster strong engagements with community members.  Integrate school upkeep into the school’s culture, providing information sessions and ongoing support to students, especially regarding proper bathroom use and care.  Identify an approach to maintaining the school grounds that promotes student ownership while also taking into account gender equity. 1 EVALUATION PURPOSE & EVALUATION QUESTIONS Evaluation Purpose The school construction and water point installation activity underwent a mid-term evaluation in 2012 that concentrated largely on technical areas of structural effectiveness, program operations and quality assurance. The Activity will expire in June 2016, so USAID has, in collaboration with the Government of Senegal, commissioned this final evaluation. This evaluation will focus on questions related to access and student retention, as well as gaining a better understanding of the collaborative mechanisms and evolving institutional capacity that guided implementation. The lessons learned from this study will allow the Ministry of Education (MEN) and USAID to inform future interventions. The primary audience for the evaluation is USAID and the Government of Senegal. The results of the evaluation will help the Education Team at USAID/Senegal and USAID/Washington overall, as part of its ongoing process to revise its programs that deal with conflict and crisis zones, with the goal of eventually obtaining new agreements with the Government of Senegal that support USAID’s overall global strategy. The evaluation may also help the Government of Senegal adjust its policies on school construction and management of school equipment, as well as access to the education system in general. Finally, the results of the evaluation could provide a reference for all stakeholders (international development agencies, civil society, private sector, government, local collectives, etc.) involved with the creation and management of educational infrastructure in particular and public infrastructure in general. The evaluation findings could also contribute to lessons learned about the current government-to￾government (G2G) mechanisms and strategies for involving the community in public infrastructure management. Evaluation Questions This evaluation seeks to understand how the construction and equipment of new middle schools and the establishment of water points may have affected the access and retention of students at the secondary level. USAID established four evaluation questions that will anchor evaluation activities and serve as the main structure for reporting study findings, conclusions and recommendations. The questions are: 1) What evidence is there that the FARA mechanism is pertinent in building the capacity of the education system? 2) Did the use of the FARA mechanism strengthen the capacity of the MEN in the following areas: school construction program design, school construction program planning, procurement, school construction program implementation and monitoring, quality control and maintenance, payment capacity and processes?1 3) In what capacity have community mobilization efforts that were implemented under FARA been favorable to communities benefitting from new infrastructure? What are the strengths and limits of the community ownership approach in terms of managing buildings and infrastructure? 1 This question was added after the initial findings were presented to USAID, so it does not appear in the statement of work (SOW). The question was inserted as Question 2 rather than added at the end to maintain a better flow of the findings and conclusions of the evaluation. 2 4) To what extent has FARA promoted collaboration among stakeholders? 5) To what degree has the FARA activity favored access to and continued attendance at middle schools for boys and girls? What has worked well? What has not worked? ACTIVITY BACKGROUND Education provides an opportunity for individuals and their families to improve their living and working conditions. The Government of Senegal (GOS) recognizes the importance of education as illustrated in its “Lettre de Politique Générale pour le secteur de l’éducation et de la formation” that provides a vision for education from 2012-2025. In this document, the Senegalese Government defines basic education as supporting the first 10 years of schooling, emphasizing the need for children to attend and remain in school for both elementary and middle school. Attention to the middle school level is essential to respond to the increased population of children completing the elementary school cycle within the context of UNESCO’s Education for All. Without sufficient and appropriately located middle schools, even the most successful students may not have the possibility to continue their studies and attain a brighter future. The Fixed Amount Reimbursement Agreement (FARA) school construction activity in Senegal has operated since 2007 to support the Senegalese Ministry of Education (MEN) and Ministry of Finance (MEF) to build middle schools and water sources within 11 regions of the country (Matam, Saint-Louis, Kaffrine, Fatick, Dakar, Thies, Diourbel, Kolda, Tambacounda, Louga and Sedhiou). This USAID program is unique in its focus on school construction and its government-to-government (G2G) approach. Within this strategy, USAID and the GOS reached an agreement by which USAID reimburses a fixed amount of funds to cover the construction of middle schools, furnishing of school materials (including desks and books) and associated water sources. Since 2006, the Activity has supported the construction of 46 schools and 30 water sources. The GOS implemented construction over a four phases (see Figure 1). In addition, the strategy has inspired the Government of Senegal to develop a parallel program, through which it will have constructed an additional schools 46 schools by the end of 2016. Because the Ministry instituted a matching project where the Government of Senegal matched funds for USAID interventions, over time the program simply became known as Matching within the concerned ministries. The funding amount for the activity (2007-2016) was USD $9,707,500. 3 FIGURE 1: MAP OF ACTIVITY SCHOOLS AND THE PHASES OF INTERVENTION Because background documents about the school construction and water point installation activity were limited, the evaluation team needed to focus some initial efforts on understanding the background of the activity. For this reason, this section of the report includes detailed background information. Through interviews with USAID officials, the evaluation team learned that while the school construction activity was managed by USAID/Senegal’s Education Team, its origins stem from a Mission-wide strategy to work directly with host country government and civil society partners. On the tail of the Paris Declaration in 2005 and just prior to the Accra Agenda in 2008, the USAID/Senegal Mission was looking for ways to work directly with government, within its own systems and addressing its own interests. The scope of the Activity with the Ministry of National Education (MEN) was based on a series of lessons learned by USAID/Senegal combined with challenges and needs of the Ministry of Education in 2006. Prior to the development of USAID’s Local Systems Framework paper in 2014 and prior to launch of USAID Forward in 2010, the USAID/Senegal Mission hoped to show a new approach to development work that would be more cost-effective, sustainable and even more efficient than the model of working with international partners. Moreover, as a part of the move toward universal basic education, a diverse group of donors took on primary school classroom construction with limited coordination and control from the MEN. USAID was just completing its girls’ scholarship program2, which highlighted the challenges of retaining girls in schools as they moved from primary to lower secondary levels. The distance to travel to school was a major hindrance to girls’ ability to attend school, unlike boys, who were not perceived to have similar 2 Originally the Education for Democracy and Development initiative (EDDI) from 1998-2003, followed by the Ambassador’s Girls Scholarship Program. 4 safety risks while walking long distances to school. The girls’ scholarship program worked with USAID/Senegal’s middle school support program, Projet d’Appui à l’Enseignement Moyen Secondaire (PAEM) which included among one of its three components, a middle school construction and rehabilitation element. Based on interviews with MEN and USAID officials, the Activity and MEN directly addressed a growing need to address a gap in the development of lower secondary schools that were unable to meet with increased flow of students from primary schools. At the time, few donors were willing to support school construction at the lower secondary level and were interested only in building classrooms attached to primary schools. In initial discussions between USAID and the MEN, the need from the MEN’s perspective to have support for the construction of lower secondary schools was made clear. The MEN also wanted to be able to design a standard school model that follows a holistic model with administrative offices, classrooms, laboratories and separated bathrooms. USAID/Senegal saw the need to work with the GOS directly to improve its ability to manage school construction that could be done in a timely and high-quality manner as directly following the Paris Declaration. USAID/Senegal wanted to structure a FARA that would reimburse the MEN based on the lower secondary schools being completed on time and meeting quality standards. Challenges existed at the outset of the Activity design from the MEN’s perspective in terms of the fact that the MEN would have to use its own funds in the first year to construct schools and that it would not be reimbursed if schools were not constructed in a timely and quality manner. From the MEN’s perspective, a certain level of risk was involved. As part of the management approach, the Education Team asked that a special or “unique” account be established just for the school construction costs through which USAID could directly reimburse costs. The Education Team at the time was led by a former MEN official whom the ministry trusted, and MEN ultimately decided to move forward with this new approach. The evaluation team did not receive documents that outlined the development hypothesis, but the evaluation activities led the team to understand that the creation of permanent physical local middle schools would improve access and the retention of students, particularly girls, by reducing the need to travel and improving the learning environment through provision of a safe space for students to learn. EVALUATION METHODS AND LIMITATIONS During the data collection phase, the Evaluation Team comprised its leader, an international consultant and evaluation specialist, Karla Giuliano Sarr, hired by Senegal MEP; three local research experts; seven supervisors and nine researchers. Except for the international consultant, all team members were external consultants from the local company GERAD (Groupe de recherche et d’appui au développement). Sub-teams composed of an expert, a supervisor and researchers worked together to collect data in various regions simultaneously. Data collection took place between June 29 and July 15. Regional data collection took place in two phases working around the Korité (Eid Al Fitr) holiday (Phase 1: June 29-July 4; Phase 2: July 10-15) with data collection with government officials occurring throughout. In addition, the MEP chief of party conducted supplementary interviews with the Senegalese Government and USAID officials to respond to USAID’s request for an additional evaluation question (#2) following the presentation of preliminary results. These interviews took place between August 2 and August 9, 2016. Data Collection Methods and Source The evaluation used high-quality, replicable social science methods to generate and present empirical data. Findings from the study informed conclusions and subsequently, recommendations which are 5 actionable, practical and directed at specific parties. The evaluation team approached this study using a combination of quantitative and qualitative methods, with greater emphasis on the latter. Methods include the collection of education statistics at the school, local, regional and national levels. Moreover, the team conducted semi-structured interviews and focus groups with a wide range of stakeholders and beneficiaries. These activities for in-depth investigations into the experiences and perspectives of the individuals involved in both the implementation of the activities and those who have benefitted from the new infrastructure. The evaluation team visited half (23) of the schools constructed through the Activity to collect enrollment and retention statistics from 2007 to present. At these schools, the team spoke with the school principal and conducted observations of the school buildings and water points to gain perspective about the maintenance of these structures as well as any changes made subsequent to the preliminary and/or official launch of these schools. At 12 of these 23 schools, the team facilitated several additional qualitative activities that allowed for a deeper understanding of the school’s relationship to the community as well as the collaborations between the various administrating actors. In addition to visiting the schools, the evaluation team met with key informants at the ministerial level and with USAID/Senegal to obtain a fuller perspective on program operations as well as the possible results of the school construction and water point installation activity. This evaluation paid special attention to how these newly constructed middle schools and water sources may respond to the differing needs of girls and boys. To the extent possible, the evaluation team disaggregated data by gender. Annex II outlines the evaluation methods in greater detail, along with the intended data sources, and links the methods and sources with the appropriate research question. The following data collection methods will be used during this evaluation:  Desk review: Identification and review of relevant technical reports and background documents (see Annex 1: SOW Section 4);  Qualitative methods: Key informant interviews (semi-structured), focus groups (semi￾structured), structured observation, taking pictures; and  Quantitative methods: Completion of school statistics sheets, school construction summary sheets. Copies of the tools for data collection are included in the annexes. Because the majority of interviews and focus groups were semi-structured, the tools serve as guides; once in the field, the Evaluation Team adapted tools as required to the needs of participants. This included adjusting the phrasing of questions to make them more accessible, as well as translating question items into local languages. Data Analysis Methods Data analysis drew on findings guided by the evaluation questions. Analysis of qualitative data collected via the semi-structured individual and group interviews used thematic and content analysis. This analysis paid attention to themes and patterns that emerged from the data and coded responses according to those themes. The evaluation team used the NViVo computer programs to aid its analyses. In addition, the evaluation team identified relevant quotes and pictures related to findings. Moreover, analysis paid attention to case studies so as to identify promising practices. Quantitative data analysis was largely descriptive, focusing on student data and addressing issues of access and retention. 6 Limitations A number of limitations exist for the evaluation. The timeline was a significant issue in conducting the evaluation. The Koritè (Eid Al Fitr) holiday fell in the middle of data collection, requiring a two-phase field strategy. In addition, school was ending for the summer, so many students and some teachers were unavailable for interviews and focus groups. With this in mind, the team recruited only ninth grade (3eme) students for focus groups because they were at school preparing for the end-of-cycle exam. Additionally, a misunderstanding of deadlines between MEP and local partner Groupe d’Etudes de Recherche et d’Appui au Développement (GERAD), as well as GERARD’s lack of familiarity with transcriptions and coding methods, led to data processing delays. Project-wise, the school construction activity began in 2007 and evolved greatly during the project cycle. Due to personnel turnover, many people who could remember the original objectives of the activity were unavailable for interview. Compounding this issue was a lack of background documentation on the developmental hypothesis or results framework for this particular activity. Additionally, most people at the local level were unaware of the funding of the activity and unfamiliar with the name “FARA.” Because the Ministry instituted a matching project where the Government of Senegal matched the number of schools constructed through the USAID Activity, over time the program simply became known as Matching within the concerned ministries. There was some related confusion around USAID-funded water points as they were installed at schools most in need, resulting in USAID-funded water points at some Matching schools. This reduced the number of water points included in the evaluation sample. Because USAID prioritized inclusion of schools built in certain conflict-zone areas in the South (Kolda, Sedhiou and Tambacounda) as part of the sample, the spread of sample schools built across the four phases of the Activity is uneven (most schools in the South were built under phases three and four). This means the data collected many not be representative of the other 23 Activity schools that may not be in a conflict area nor built under the latest phase. 7 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS Question 1 What evidence is there that the FARA mechanism is pertinent in building the capacity of the education system? Based on planning discussions that took place before the evaluation, the evaluation team understood this question to investigate the relevance of the FARA mechanism to the education system’s ability to provide educational services. The way the school constructions responded to the needs of the populations in providing safe and permanent structures in more locations throughout the country was a key consideration for this question. While institutional capacity building and systemic change are also related, evaluation question 3 addresses these issues more specifically. Similarly, question 5 examines in detail how the school construction activity may have affected access and continued attendance. Findings According to program background information and discussions with key informants at USAID and the Ministry of Education (Ministère de l’Education National – MEN) and the Ministry of Finance (Ministère de l’économie et des finances – MEF), between 2007 and 2016, the USAID-funded Activity built 46 middle school complexes and 30 water points in 11 regions of Senegal. During the 2015-2016 academic year, the 23 sample schools served 13,122 students (6,731 girls and 6,391 boys) based on statistics provided by school principals. 3 FIGURE 2: MAP OF SAMPLE ACTIVITY SCHOOL POPULATIONS IN 2015-2016 3 School population figures were missing for two schools: Mboss Middle School in the Kaolack Region (all students) and Merina Dakhar in the Thies Region (number of boys). 8 With one exception, the blueprint for the 46 schools built by the FARA government-to-government collaboration included the following components:  Administrative block: principal’s office, teacher’s lounge, school supervisor’s office, staff toilets;  School classrooms: four classrooms, one laboratory, one multi-purpose room (computer lab);  Student lavatory block with separate lavatories for girls and boys;  School guard post; and  Cement brick-wall enclosure and gate. Overwhelmingly, study participants applauded the Activity’s construction of the school complexes and water sources. At the same time, discussions demonstrated a general conflation of the Activity components with the Senegalese Government’s matching program sparked by the USAID initiative, called Matching. This blurring occurs at both local and ministerial levels and was evident at schools and in regional offices where the majority of participants referred to the sample schools as Matching rather than “FARA” Activity schools. Similarly, ministry officials frequently interchanged the terms during interviews identifying that the processes for both USAID-funded and Matching schools were nearly identical. Providing Safe, Permanent Structures and Increasing Instructional Time With one exception, 4 the schools sampled for this evaluation were already officially recognized by the government, possessed an official establishment date and had dedicated personnel and recruitment areas prior to the construction of Activity buildings. In fact, this was the most important criteria required by the Activity for choosing a construction site (DPRE official, interview). Discussions with school personnel and central officials (MEN, MEF and USAID) revealed that prior to the Activity constructions, classes took place in temporary structures that included borrowed elementary school classrooms, warehouses, church buildings, houses and community-built structures. These last structures were often made of plant-based materials or mud bricks that required frequent maintenance and replacement. Through visits to schools, the evaluation team learned that these community-built structures presented students and personnel with serious health and safety hazards in addition to functioning only during part of the school year. For example, in the southern portion of Senegal where rains are particularly heavily, all of the sample schools reported that the Activity constructions have led to an increase in instructional time, sometimes by as much as a couple of months. In addition, where electricity is available, students expressed satisfaction that classes can continue later in the evening. Many schools use Activity classrooms to offer remedial classes and, where electricity is available, middle schoolers and other students in the community often take advantage of the space to study in the evenings. “So the middle school was established in 2006. So from October 2006 to October 2011, we were 100 percent in temporary structures and the middle school was a ghost school. In the rainy season, everything disappeared but near October, November and December, the temporary structures would reappear. So, there was an urgent need to build the school. So that’s how we got the school from 2012 to 2013. ... This allowed us to [maximize] our instructional hours. … It’s really an extraordinary advantage.” — School Principal, Sédhiou Region Reduced Distances to Middle School When asked if the Activity school complex improved the educational system, almost all students, community members and school personnel responded that the Activity allowed students to attend school closer to home. The majority of participants said that before the original founding of these 4 The principal at the Keur Massar Middle School reported that the school’s creation coincided with its construction. 9 schools, only families who had contacts in other towns and money for transportation were able to send their children to middle school. Often, students would stay with a family member or another contact in these towns, sometimes returning home only during major holidays. In some cases, the school was more than 30 kilometers away (for example, Ndiaye Ndioungue Mberess, Saint Louis region). Other middle school children walked or rode their bikes daily to schools that were 3 to 7 kilometers away, often unable to return for lunch and going without food. Conversations with parents demonstrated their pleasure in having their children attend middle school closer to home, which they attributed to the Activity constructions. In particular, having middle schools within the community allows parents greater control of their children’s upbringing. “Since our children are with us, we assist them. We watch after them. They live and eat with us and transportation isn’t a problem. I have to add that having the middle school in our community frees our children from having two different upbringings.” — Parent of Ndiaye Ndiouga Mberess Middle School Student, Saint Louis Region (Focus Group) As mentioned, prior to Activity school construction, data show that all but one of the schools operated using provisional and temporary structures. The section on question 5 on access and continued attendance will further discuss how the majority of students, parents and teachers expressed high levels of dissatisfaction with provisional classroom environments interfering with middle school success. Meeting Students’ Needs “We got these buildings and I feel at ease when I’m studying. …There are other villages that don’t have buildings like these and they hope to have them someday.” — Female Student, Koussy Middle School, Sedhiou Region Focus groups with students demonstrated that the majority of students are satisfied with the Activity school constructions. A common refrain among both female and male students was feeling “at ease” at the schools. Some students mentioned that they are happy to study close to home so they can help their parents. Others mentioned that their school is the envy of other communities. Reasons for satisfaction also include some classrooms being tiled, 5 that bathrooms are available on the school grounds, that classes do not have to stop due to weather and that students have water to drink. Where trees have been planted, several students expressed pleasure in resting in the shade under trees outside of class time. Having electricity is also a major plus for many students where it is available, as it allows for students to make photocopies, conduct experiments in science class and watch educational videos using projectors. Some students who do not have electricity at home commented that they appreciate being able to charge their cellphones at school to use at night as a light source when doing homework. In terms of meeting the needs of school communities, school visits and discussions with local actors also revealed that three school complexes within the Activity final evaluation sample currently serve high school students. Two of the schools within the evaluation sample have been transformed into mixed middle/high schools (Sinthiou Gangué School in the Saint Louis Region and Mboss School in the Kaolack Region) responding to community desires for students to continue their education locally. Many discussions with students and community members at the other Activity middle schools also indicated the need for a local high school. A third school, Bouna Kane School in the Kolda Region, never operated as a middle school as it was immediately turned into a high school to quell community unrest. Discussions with the DCS FARA coordinator confirmed these cases. 5 More tiles were used during the fourth phase than during other phases; this included all classroom spaces as well as verandas. 10 Meeting Teachers’ Needs As a result of the Activity school constructions, the majority of teachers felt that their working conditions have improved significantly. In addition to the well-built classrooms with nice blackboards, the majority of teachers in focus groups expressed their satisfaction with a high-quality teachers’ lounge that is tiled, has good ventilation and, at some schools, also has computers where teachers can do research and prepare their lessons. In addition, many teachers commented that they were pleased to have dedicated lavatories in the administrative building. Previously, teachers would have to relieve themselves in neighborhood houses, similar to the students. Many teachers and administrators commented that as a result of improved conditions, largely due to the school construction activity, teachers are more satisfied with their assignments and turnover at schools has decreased. “There’s a teacher’s lounge, a school supervisor’s office, bathrooms … everything is there so that work goes smoothly. It meets all standards.” — Teacher, Seby Ponty Middle School, Dakar Region Photos 1 & 2. Left: Teachers’ lounge (Simbandi Balante Middle School, Sedhiou Region). Right: Teachers sitting outside administrative building (Zone Sude de Mbour Middle School, Thies Region) Increasing Access to Clean Water According to Activity documents and discussions with GOS and USAID officials, the Activity built 30 water points at middle schools participating in either the USAID-funded or Matching programs. During interviews, school officials and students explained that the water is used for drinking, cooking, cleaning blackboards, light washing after gym class and in bathrooms throughout the school. Having water at the schools also contributes to increased instructional time, as students are not spending time fetching water. Teachers and students can also continue their lessons without needing to wait for water to clean the blackboard. In the case of the Koussy Middle School, the USAID-funded water source irrigates the school’s garden that, in turn, enriches the school food program. At the same time, in schools where water was not available, visits to school sites and discussions with school personnel and students illustrated that a lack of water often weakens the school environment and can lead to severe deterioration of Activity lavatories. Many schools have difficulties maintaining the upkeep of the bathrooms and as a result they are rendered unusable. At the time of evaluation visits, five schools had unusable toilets due to problems with water or upkeep of the facilities. 6 As a result, many students 6 The girls’ bathrooms at Mbao Kamb Middle School (Dakar) were not functioning due to renovations as part of a new project with the French Development Agency (AFD). 11 spoke of the need to go to neighborhood houses to relieve themselves while teachers explained that students, particularly boys, use the fields beyond the school walls. “When there is a water source at the school, students don’t need to go out anymore to fetch water to erase the boards or to drink or to relieve themselves. So, all of this is part of improving quality. They don’t lose time any more.” — Education Inspector, Dakar Region Providing School Equipment Interviews and the completion of checklists at each of the 23 sample schools revealed that the Activity provided schools with equipment and furniture, including classroom desk-benches, teachers’ desks and chairs, administrators’ desks and chairs, cabinets, computers, photocopiers, printers and in some cases books. The majority of evaluation participants frequently referred to the Activity approach as providing a full-package school that was ready to operate. 7 When asked to identify Activity equipment received, in addition to other materials, evaluation participants at the school-level frequently cited photocopiers and, where electricity was present, the majority of schools visited continue to use the original photocopiers provided by USAID. In some cases, administrators had to invest significantly in necessary repairs. Through discussions with the FARA coordinator at the School Construction Office (Direction de constructions scolaires, DCS), the evaluation team learned that they provided computers to schools largely during phase 4. Visits to many phase 4 schools confirmed their receipt of computers. While all schools reported receiving some materials through the Activity, school-level actors indicated that they still needed some materials. For example, at Ndiaye Ndioungue Mberess Middle School, the only phase 1 school where the evaluation team conducted qualitative activities, students, community members and personnel described how they did not receive desk-benches and explained their difficulties in obtaining them through their own resources. Interviews and focus groups revealed that computers were sometimes provided to schools with no electricity; as a result, the computers could not be used. During visits to some schools, the evaluation team observed that the number of certain items exceeded need at the school (for example, the number of metal storage cabinets), while other schools received insufficient numbers of these items. Efforts to discern the distribution of USAID student books and teacher guides were largely unsuccessful. At many schools, respondents indicated either that they did not receive any such materials, or that because classes had concluded, it was impossible to answer the question because books had already been stored for the summer. At almost all schools, personnel and students indicated wishing they had received laboratory equipment. The majority of school principals indicated that they had converted the science laboratories into traditional classrooms due both to limited classrooms and a lack of materials. Unmet Needs Participants at the school level spent significant time discussing needs that the Activity had not fully addressed. One recurring issue is that while bathrooms are present on school grounds, they are no longer functional at six of the 23 schools. Of the 17 schools that reported functioning bathrooms, students and other school actors indicated during interviews that poor conditions prevent some students from using the bathrooms. As a result, many students in Activity complexes resort to going home, to neighborhood houses or in the bush behind the schools to relieve themselves. Reasons for bathroom degradation include cleaning and maintenance problems and insufficient capacity. The facilities have a total of eight or 10 stalls for school populations that range from 240 to 1,480 students, with an average of 600 students (2015-2016 figures). According to interviews with school personnel, students often lack familiarity with the proper use of bathroom fixtures that are significantly different from how 7 In French, this idea was expressed as “un collège clé en main.” 12 students relieve themselves at home, especially in more rural areas. Two schools have closed the bathrooms due to termite and tree root damage that have compromised the foundation. Photos 3–5. Left to right: Bathroom maintenance issues at Sedhiou Middle School III, Sinthiou Gangue School (Saint Louis Region) and Mboss School (Kaolack region). Several students, teachers, school personnel and community members also commented that their schools continue to lack electricity and access to water. While some schools have benefitted from solar panels, the maintenance of solar panels in rural areas was mentioned to be problematic due to a lack of capable solar energy technicians. Without access to water, some students mentioned that they are often late to class after needing to leave the school grounds to relieve themselves because school bathrooms are inadequate. Others shared concerns of not being able to complete the required curriculum because instruction was limited to daylight hours. The lack of electricity is a significant impediment for six of the 23 schools and precludes the use of technological equipment. Only seven of 22 responding schools have functioning computer labs and many participants in rural areas lamented that their children are far behind city children in being able to navigate the Internet. “This village doesn’t exist. We are still in what we call Ancient Africa. No one should have high voltage electrical wires pass in front of the school and still spend 10 years teaching in such conditions as if the world hasn’t changed its teaching methods.” (Community member, Teyel, Kolda Region) Discussions with a variety of evaluation participants and school visits identified that the Activity school design often does not fully address the needs of the school population. At many of the schools visited, the four classrooms and two specialized room design did not correspond with the school population size at the time of construction nor take into account expected population growth. As indicated through discussions with school principals and observations, 17 of the 23 schools visited demonstrated a lack of adequate classrooms for the school population. When asked about the relationship between the capacity of school constructions and the school population, the DCS activity coordinator indicated that there was one school complex prototype implemented across the activity with the exception of the one urban-style school built in Ouakam (Dakar Region). Similarly, the former USAID Education Team leader indicated that the implemented school prototype reflected the original idea to build small rural schools. He commented that once the activity began it was difficult to limit construction to small villages and that the schools complexes were most often built in more populated areas (Interview). 13 Schools adapted to this in different ways. For instance, all of the laboratories and many of the multi￾purpose rooms/computer labs (15 of 228) of the 23 sample schools have been converted into regular classrooms. In other schools, student class sections rotate through classrooms rather than assigning a classroom to each class section9. Eight of the 23 schools continue to function with temporary structures on the school grounds. As illustration, at the newly constructed Simbandi Balante Middle School (Sedhiou Region)10, there are 19 class sections while there are only 5 cement-constructed classrooms resulting in 14 sections residing in temporary structures made out of woven plant materials. At Ngothie Middle School (Kaolack Region), for example, the school continues to use peanut warehouses as additional classrooms. These structures are extremely hot and are located a 20-minute walk away from the main campus, creating safety and supervision challenges. Other persistent difficulties associated with learning and teaching in temporary classrooms include sunlight, rain and wind-borne sand penetrating plant-material based roofs and walls and making students uncomfortable; difficulties reading the chalkboard; flooding; presence of insects, snakes and other vermin; and lack of appropriate ventilation. Photos 6–9. Comparing Activity classroom (left) with additional community-built classrooms (right), Ndiaye Ndioungue Mberess Middle School (Saint Louis Region); Activity classroom with plant-based material constructed temporary classroom at the Simbandi Balante Middle School (Sedhiou Region). 8 Information missing for one of the schools as it was not possible to interview the principal at Boynadji Middle School (Matam Region). 9 The Senegalese school system follows a French-inspired model where students are assigned to a fixed group of students (classe pédagogique). Typically, each of these student sections is assigned a classroom and teachers rotate from classroom to classroom. 10 Students entered classrooms at the beginning of the school year and there was an official inauguration ceremony in February. 14 Also, the layout of buildings often does not account for future additions, as in some cases the administration building is positioned in a way that will not allow a sightline to new structures. At the Hilele Middle School (Kolda Region), for example, there is a problem of space as the school lot is close to a river. Moreover, the case of the Mbao Kamb Middle School (Dakar Region) is particularly noteworthy as the school received funding from the French Development Agency (AFD) for a multi￾story addition that will result in a total of 18 classrooms at the school11. As part of the construction, ADEM will tear down two of the Activity classrooms. Lastly, findings from the school construction activity mid-term evaluation demonstrated that the urban model could be even more cost effective than the rural one-story layout. Comparing the Keur Massar Middle School, a rural model, with the Ouakam Middle School, the mid-term evaluation found that the urban model provides more classrooms (8 compared to 6) at 74% of the cost of the rural model12 (GERAD, 2012, p. 7). Conclusions The Activity has responded to the needs of many school populations by providing high quality and efficiently constructed physical structures that have greatly improved students’ and school personnel’s’ safety and comfort. The Senegalese Government’s voluntary engagement to complement each of the schools through the Matching program is further testament of the relevance of the Activity and, in theory, has doubled its reach. While Activity constructions have contributed to improving schooling conditions for many students and teachers, issues remain at many schools that pose serious health and safety risks and limit many of the Activity’s benefits. Complications include student populations that exceed appropriate classroom capacity and require the persistence of temporary structures as well as a lack of electricity and inconsistent access to water sources. Upkeep and maintenance issues pose serious threats to the sustainability of school complexes. Providing equipment as part of the Activity package was a highlight of the Activity approach for many participants. Inconsistencies in material distribution was a weakness of the approach, however. While the number of phase 1 schools in the sample was limited, there may have been an improvement in the equipment package as Activity phases evolved, particularly in phase 4 with the distribution of computer equipment. Nonetheless, some schools in phase 4 that did not have electricity still received computer equipment in some cases. Although the Activity has improved the capacity of the education system in responding to populations’ need for more permanent structures, a more adaptive school complex design might have been more appropriate and even more cost-effective. Such an approach may have also allowed the education system to account for middle school growth well into the future. Question 2 Did the use of the FARA mechanism strengthen the capacity of the MEN in the following areas?  School construction program design  School construction program planning 11 Through the project Appui pour le developpement de l’enseignement moyen (ADEM). See Dione, Ndong, Diop and Ba (July 22, 2016), Lancement du projet Adem : 17 collèges bientôt construits ou réhabilités. Le Soleil online. Accessed 8/2/2016. http://www.lesoleil.sn/2016-03- 22-23-17-43/item/52726-lancement-du-projet-adem-17-colleges-bientot-construits-ou-rehabilites.html 12 The Ouakam Middle School construction costs amounted to 33,247,270 FCA (approximately $57,000) while the rural design for the Keur Massar school cost 45,004,989 FCFA (approximately $77,000). (See GERAD, 2012, Evaluation à mi-parcours du programme de remboursement forfaitaire (FAR), p. 7). 15  School construction program implementation and monitoring  Quality control and maintenance  Procurement  Payment capacity and processes Findings This evaluation question was added at the request of USAID/Senegal to determine, what, if any, institutional capacity had been built or strengthened by the FARA as a G2G mechanism. In order to answer this, the evaluation team conducted additional interviews with MEN, MEF and USAID/Senegal current and former staff. The first question asked of respondents related to this evaluation question was their understanding of the primary objectives of the school construction activity. Overwhelmingly, the MEN and MEF representatives spoke about increasing the capacity to offer middle school services, particularly offering more local options to communities. Only current and former USAID/Senegal staff commented on improving MEN capacity. The reasons for this were found to be two-fold. The Activity lacked any explicit language in the implementation letters that spoke to increasing the institutional capacity of the MEN. Indeed, at the launch of the first FARA, USAID Forward and the emphasis on G2G approaches had not yet surfaced. While there is more emphasis currently within USAID to partner directly with local partners, government, civil society organizations and private sector, there still remains a lack of clear ADS guidance in terms of approaches for assessing technical management capacity of government institutions. ADS220 (Use and Strengthening of Reliable Partner Government Systems for Implementation of Direct Assistance) focuses primarily on funding mechanism options and the standard tool used by USAID to assess financial and administrative capacity of host country governments. There is in addition a 79-page manual outlining the process for the Public Financial Management Risk Assessment Framework (PFMRAF) that supplements the ADS220 guidance. There is one mention of the need to conduct an assessment of host country government systems related to technical capacity (220.331.a(2)) but there is no guidance on how and no tool or manual provided to support Mission ability to do so. While a PFMRAF was conducted on the MEN, in 2011, four years after the start of the school construction activity, it only assessed the MEN’s financial systems, not institutional technical capacity and personnel power. The second obstacle is, due to the continued use of the FARA since 2007, most current USAID/Senegal education team members and MEN look at education-based indicators to measure the success of the Activity. Indicators around increased schools access and improved retention of students are key indicators to measure the success of the Activity. It is primarily the Office of Financial Management and the Program Office that are interested in the aspect of intuitional capacity development. School Construction Program Design and Planning Most MEN and MEF respondents commented that the Activity design serves as a model for the MEN going forward. However, it was specified that they saw it as a model to roll out to other donors, rather than a model to be used by the MEN itself for how they construct all school buildings. This desire for a similar approach with other donors was mentioned by the Planning Advisor of the DCEF because there is a relationship of “respect” between USAID and the Government of Senegal (GOS); they agreed upon the model and the quality of the schools, MEN builds the schools, checks that the quality adheres to the standards and as long as they do, the reimbursement is made quickly. The freedom to program the money within the context of the school construction activity limits was well liked by all MEN and MEF 16 respondents, noting that previously school construction processes managed by the MEN were often delayed and payments to contractors were not made in a timely manner. However, there seemed to be no indication that this timely payment process would continue outside of the FARA mechanism for other MEN funded school construction. Part of the design of the school construction activity is that once schools reach completion they are signed off by an engineering firm that USAID identified to confirm quality standards in coordination with the DCS. According to USAID/Senegal finance and program staff, USAID/Senegal proposed and funded the Senegalese engineering firm to provide a potential model for the MEN to outsource support from the private sector, given their limited number of engineers working in the DCS. In interviews with the DCS options for filling engineering gaps focus more on the direct hiring of more engineers as MEN staff. In terms of planning, the DPRE was clear that there is still a need for incoming people to learn the necessary planning tools in order to properly account for location and current and future population for middle schools. As will be expanded upon below in other questions, many schools still struggle with a lack of adequate space to accommodate all children who wish to attend the school. The current approach taken by the MEN is to use the standard middle school model and to allow the community to construct temporary classrooms (see questions below for more detailed information). Additionally, during phase four of the school construction activity the MEN decided to double the amount of schools built to a total of 18 schools (from the usual nine per FARA). The attempt to double the number of sites was problematic and led to significant delays, requiring an amendment to push out the completion date of the fourth phase by six months. Additionally, there were issues with reimbursement since the Activity also included a provision for construction of water points, some of which were not completed although the school was finished, causing an interruption in payments to contractors. This expansion, without the required manpower and resources to oversee and quality control the construction sites reflected a limited ability to plan effectively. School Construction Program Implementation and Monitoring The school construction activity was successfully implemented in terms of building the 46 middle schools planned (barring the delay caused by an expansion of scope). Many respondents commented on the utility of having a specific budget line in the Activity budget for school visits. The FARA Activity coordinator at the DCS was seen to visit new schools frequently and commented on how helpful being able to visit schools was in order to monitor progress. However, there was no evidence that this ability existed outside of Activity funded schools. Indeed, the USAID engineer who worked on the school construction activity commented that the issue of ability to travel to construction sites, including the car itself, the cost for travel and lodging were ongoing issues for the MEN and that costs for travel and oversight were not adequately budgeted by the MEN. “Since the beginning of the activity…once we were able to obtain the necessary papers, we would visit the site to assure viability, that it met our criteria…we would start to make visits – us and even the DEMSG, also according to its community mobilization policies made visits, well, if we saw something or if the population told them about a problem at a certain site, they would tell us and as soon as we could we would prepare a visit to go see.” — FARA School Construction Activity Coordinator, DCS, Interview It also should be noted that the coordinator of the FARA school construction program within the DCS was the champion of the project and much of the success of the implementation and monitoring is due to him. This was clearly seen across all interviews with the MEN, MEF and USAID/Senegal staff. 17 “Al Hassane (FARA program director) will leave with so much information and there needs to be someone who works with him NOW to take over. There needs to be a champion for the project.” — Planning Adviser, Education and Training Program Director, Ministry of Economy and Finance Another comment made by respondents was a reduced level of meetings and sharing of progress of Activity schools. In the beginning of the Activity, there was huge enthusiasm to make this new approach working directly with the government successful; to show that this could be an effective model. Joint field visits between the different MEN Directions and MEF Directions were coordinated and joint progress reports written. However, as the school construction activity moved forward and was no longer considered “novel,” the coordination and joint site visits were reduced. The steering committee was seen by respondents as unique to the Activity and as enabling a lighter management requirement than previous projects that typically form a temporary secretariat with government officials who are seconded to the project. However, the committee seemed to meet less frequently toward the latter years of the activity. Additionally, while respondents acknowledged that the DCS was responsible technically for implementation, a few MEN and USAID/Senegal officials expressed the opinion that it was not clear, when it came to management of the overall committee, exactly who was in charge of coordination. Additionally, while community mobilization was mentioned as an improved skill or approach in school construction planning, respondents referred to Projet d’Appui a l’Enseignement Moyen Secondaire (PAEM), another USAID funded project with a middle school construction component that ran semi￾concurrently to the FARA school construction activity. Because of the overlap of the programs (PAEM began in 2003 and completed in 2010) it is not possible to say whether any improvement in community mobilization around school construction was really due to the Activity, or rather learned under PAEM and continued under the Activity. (A more complete discussion of community mobilization efforts follows in evaluation question 3.) Quality Control and Maintenance The school construction activity was unique in that it employed a reimbursement model for schools constructed that met both timely and quality construction standards. This meant that MEN risked not being reimbursed if the independent Senegalese engineering firm did not sign off on a school. According to interviews, the firm was a Senegalese firm contracted by USAID directly. Due to quality control mechanisms in place, MEN only paid a percentage to the contractor until the work they were done was approved, with the understanding that if they were not reimbursed neither would the contractor receive payment. The independent quality check of schools forced a much tighter attention to quality control by both MEN and their contractors to ensure standards were being met to allow for USAID reimbursement. Once schools had received approval, the DAGE and the DI processed the paperwork and the MEN was reimbursed very quickly (typically between 24-48 hours). While non-Activity schools constructed by the MEN also follow a same process of small (20 percent) initial payment if requested by the contractor, and final payment at the delivery of the school, there is less oversight from the MEN on the quality and timeliness of the construction. In addition, there are still issues with timely payment for schools, with funds coming from the central GOS account. While the Activity allowed for tight oversight and quality control of school construction, the evaluation found no evidence of this same level of tight control and oversight being replicated. Maintenance was identified as a chronic issue in the Activity. Even though the implementation letters had a clause requiring an allocation of Activity funds toward ongoing maintenance of schools13, there is 13 Clause 8, Maintenance and Repairs, Implementation Letter No. 685-013-05 for the fourth phase of the Activity. 18 no evidence that ongoing maintenance plans have been developed. The maintenance was seen by many actors as a responsibility of the community, as part of the community mobilization around school construction as well as ongoing maintenance. In an interview with the current USAID/Senegal Education Team Leader and FARA school construction AOR, they discussed the issue of maintenance and that the Director of DCS felt that while the DCS might play a role in preventive maintenance that the remainder would fall under the school and community. “It was clear for the first time in this meeting at what level they weren’t involved in maintenance… This is a weakness in the system, in my opinion.” — USAID Education Team Official, Interview As will be further explored in the next question (#3) maintenance and the effective mobilization of the community to take ownership of the school was only partially successful, with many communities lacking the means to adequately keep up the school or add improvements such as water or electrical access. Maintenance was mentioned as the number one challenge currently facing the MEN’s ability to effectively manage school construction, across all GOS and USAID respondents. Procurement For the most part the procurement procedures for the school construction activity followed the standard process used by the GOS, working through the MEF’s Direction Central des Marchés Publics (DCMP) in coordination with the MEN’s Cellule de Passation des Marchés Publics (CPMP). In terms of the orientation of the contractors selected, additional time was given to explaining to the contractors, the importance of the Activity, the quality standards, oversight and timeliness that would all be required in order for the contractor to receive payment. Both the former USAID Education Team Leader and the FARA activity coordinator at the DCS took time to meet personally with the contractors and explain the importance and also strict regulations of the payment process. Both emphasized the patriotic element of the activity in showing that the GOS is capable of building middle schools in a timely way and with high quality standards. Payment Capacity and Processes The way finances were handled with school construction activity was unique to the project and to the MEF and MEN. According to the DI, while they manage donor projects on a daily basis, the Activity is unique in that it is a system of reimbursement which required the GOS to provide their own funds to the account at the outset. The Activity is also rare in the fact that the MEF directly manages the funds and makes direct payments for all Activity related expenses. In the majority of cases, the DI only processes the bills for donors, verifies the work has been done and then the donor makes the payment to the contractor. Activity funds are maintained in a special account which is separate from the GOS general budget. Most respondents expressed the fact that the initial payment by the GOS into the Activity and the acceptance of reimbursements based on agreed upon quality standards, was a big step and accomplishment for the MEN. It required a certain number of risks and also trust on the part of the government. All respondents from the GOS, both MEN and MEF, expressed satisfaction that Activity funds were held in a unique account as it kept the money “safe” and could not be redirected. The idea of managing a unique account is not a new approach to the MEF, which regularly manages donor special accounts and it does not appear that there is any movement from the MEN side to consider changing this approach to work directly with the general government budget. USAID’s finance officer mentioned that in terms of good practices within strengthening government public financial management systems, using a separate account is a deviation from the best practice of working from a common account, but was selected as a less risky option for working within government systems. However, he noted, it is not considered as a long-term solution. 19 Conclusions The school construction activity lacked any explicit capacity building goals or objectives. The lack of a clear desire by USAID to build MEN capacity meant that capacity building was not focused on, measured, or even recognized as an objective by the end of the project from the side of the Senegalese government. In terms of improved capacity, MEN individuals from the DEMSG have improved capacity to understand the community participation approach but much of that was learned through PAEM prior to the FARA school construction activity. Individuals from the DCS also have improved their capacity to manage the school construction process and oversight. Capacity assessment of government institutions remains a gap within USAID in terms of guidance on how and when to conduct organizational assessments that focus on the requisite technical and managerial capacity to implement an activity. In the case of the school construction activity, capacity￾building can be designed in a traditional sense of looking at gaps in technical capacity – however, this will always result in trained individuals and no improvement in systems. In terms of systematic change, there was none visible. The Activity is operated as a separate project within the MEN and very little has been adopted out to the larger MEN systems. The issue of transportation costs and means to provide regular guidance and quality control over the construction of schools still exists. The funds for the Activity are still maintained in a separate account and not mixed in with the Treasury’s common account for fear of being lost – for lack of a better word. There is still a lack of engineering capacity (not enough engineers) and there does not seem to be a plan to outsource these services. It is still seen as something that needs to be built within the DCS in terms of additional government staff. There is no indication that at the end of the school construction activity any of the processes or unique components, such as the steering committee, will continue to be utilized within the MEN. Overall, the Activity proved that the MEN can effectively build middle schools in a cost effective and quality manner albeit with some delays. However, the overall institutional and systemic capacity of the MEN to implement these kinds of projects outside a FARA framework, has not been proven. Question 3 In what capacity have community mobilization efforts that were implemented under the FARA school construction activity been favorable to communities benefitting from new infrastructure? What are the strengths and limits of the community ownership approach in terms of managing buildings and infrastructure? Findings Interviews with actors at all levels suggested the importance of engaging communities in order to promote ownership of Activity schools and their continued maintenance. For the purposes of this evaluation, community mobilization refers mainly to activities that were directed at non-official actors (parents and other relatives of students, neighbors, etc.). At the same time, there is an obvious overlap with local authorities, such as the mayors and the conseils départementaux (district councils) given their proximity to community members. For this reason, this current question addresses the relationship between these local authorities and community members while the next question (#4) addresses the relationship between these local authorities and their colleagues at a more central level. Due to a lack of background documentation prior to conducting fieldwork, the evaluation first sought to identify the various community mobilization activities that took place under the Activity. 20 Community Mobilization Activities Local, regional and central actors all confirmed that the Direction de l’Enseignement Moyen Secondaire Général (DEMSG) was responsible for community mobilization efforts. Supporting findings above, an official from the DEMSG office clarified that they had received training in community mobilization methods during the USAID-funded EQUIP2 Children’s Sustained Learning Access and Improved Teacher Training Program in Senegal (PAEM Classe) project. 14 “We had experts in community mobilization … people really strong in it. We had inherited this from PAEM. There was a project PAEM Classe that was here and had worked a lot on that. We, well, we had a brief training in it, a brief training at the time in it and there are also experts from this project who had outfitted the DEMSG in terms of these issues.” — DEMSG Official Ensuing discussions with a current DESMG official and the former DEMSG director described community mobilization activities as a three-step process: 1) informational meeting, 2) commitment meeting, and 3) follow-up visit. The first meeting would take place before construction and gather a diverse group of community members ideally including the mayor, village chief, other officials, youth, and women’s groups. The DESMG would ask about the community’s needs for the school and provide information about the Activity helping community members be realistic about possible outcomes. During this meeting, the DESMG would explain its expectation that the community make significant contributions to the successful construction of the school and ask each actor to consider the sort of contribution it might make. He also explained that there are five key areas of support: 1) identifying land, 2) connecting to water and electricity, 3) promoting girls’ education, 4) upkeep and maintenance, and 5) planting trees. Other participants confirmed these areas, including the representative from the DI who had accompanied the DEMSG on certain community mobilization visits. The former director of the DESMG described a typical scenario in which the mayors provide the school with land, the village chief houses members of the contractor team, women provide water for construction and young people plant trees. In addition, local communities would provide local labor either voluntarily or at a lower cost than if workers were brought in from other areas. The former head of the USAID education team offered that the community was envisioned to be the “first line of supervisors.” A DEMSG official further elaborated that communities were expected to welcome construction companies and orient them to the community. They should also assist construction workers in accessing electricity, water and telephone service so as to meet deadlines. According to the DEMSG former director, approximately two months later, there would be a second meeting, in which the DEMSG team asks community members to confirm their intended contribution in front of everyone else and to sign their names. The third visit is a follow-up visit that would occur approximately one year later, during which the DEMSG would meet again with community members and together walk around the school ground to verify that each group had fulfilled its promise for the school. When pressed as to whether or not the DEMSG provided upkeep and maintenance support for schools after one year, he explained that funds were unfortunately not sufficient to accompany schools beyond one year. School and community actors from eight of the 12 schools involved in qualitative data collection activities confirmed that Activity community mobilization meetings took place. The eight schools were across all four phases of Activity construction. It was unclear which of the three meetings had occurred, however. Interview and focus group data from the remaining four schools further indicated some confusion as to whether or not any such meetings took place. Some teachers and PTA/SMC members 14 See USAID (2008). EQUIP2/Children’s Sustained Learning Access and Improved Teacher Training Program in Senegal (USAID/PAEM). Accessed from http://www.equip123.net/webarticles/anmviewer.asp?a=141. 21 indicated during interviews and focus groups that they were not aware of any activities or they spoke vaguely about meetings that were held but they were not certain who organized them. In other instances, personnel had turned over since the earlier phases of the project and current personnel were unable to respond to questions. Communities Supporting Activity Schools Interviews and focus groups, particularly with Parent-Teacher Association/School Management Committee (PTA/SMC) members, provided additional information of how communities participated in school construction and care of the schools once occupied. PTA/SMC members from Teyel told of young people helping to unload construction materials free of charge. At Simbandi Balante, the principal shared how individuals from the community performed digging, masonry work, and offered general construction support. Community members also regularly cleared brush from the school grounds. Community officials within both the Sedhiou III Middle School (Sedhiou Region) and the Merina Dakhar Middle School (Thies Region) communities watched over buildings. Others contributed actively to construction efforts, including supervising how cement bricks were made (interview with PTA/SMC member, Simbandi Balante School, Sedhiou Region). According to interviews with the FARA activity coordinator at the DCS, construction companies also relied upon local communities for essential information as to the location of sand quarries instrumental in forming cement bricks as well as water sources. “Well if a contractor comes with his masons … I am one. If he didn’t choose me to work, well, me I still watched him with, I helped him with … how he mixed [cement mixture]. Did he do it the correct way? I was there and because I worked with him it was no problem to collaborate.” — PTA/SMC Member, Simbandi Balante Middle School, Sedhiou Region Interviews with community members and school-level actors also provided examples of additional support provided for Activity schools after construction. This includes several tree planting projects involving community members, parents, teachers and/or students. In addition, at least five schools were able to connect to electrical and water services through the efforts of community members as well as school personnel. Notably, emigrants from Ndoumga Ourou Alpha (Matam Region) provided the funds necessary to obtain electricity at the Activity middle school, according to discussions with the principal and teachers. In addition, data from focus groups shed light on some examples of community members supporting cleaning activities on school grounds, at Seby Ponty Middle School (Dakar Region) and Hilele Middle School (Kolda Region), in particular. Moreover, at schools where the population exceeded the capacity of Activity buildings, many PTAs and SMCs built complementary temporary structures (either from plant-based materials or cement bricks). When asked the question of how Activity community mobilization efforts led to a great sense of ownership of schools, one community member spoke of an increased understanding of how parents could become involved in school management. “If only in terms of providing direction. Well, FARA has at least provided us with direction about activities that will allow us to improve the teaching quality. First of all, there’s planting trees because they showed us how an environment like this can be really welcoming. And also, in terms of parents, there’s a committee that is in charge of embellishing the school. Another thing is that this project allowed parents to see another management model, that is, that we can be involved in the management of activities, for example, participating in school community activities by supporting their children in their schooling.” — PTA/SMC Member, Seby Ponty Middle School, Dakar Region 22 Photos 10 & 11. Trees planted at Activity schools. (Left: Boynadji Middle School, Matam Region; right: Kebemer 3 Middle School, Louga) Other examples from interviews and focus groups provide insights into additional ways that communities support children’s education. For example, how each year the PTA and SMC at the Samsamba Middle school sends a representative to accompany students sitting for the Brevet Fin d’études moyen (BFEM) exam as students need to travel 11 kilometers to the testing center. The chaperone identifies lodging for students and organizes students’ meals. At the Teyel Middle School (Kolda Region), parents have contributed money and livestock for end-of-year celebrations. “Also, when I do the end-of-year awards ceremony, two times parents provided financial contributions or prizes to reward students…It has even occurred twice at the end of the year that parents contributed rams to parties that teachers were organizing.” — Principal, Teyel Middle School, Kolda Region Limits to the Activity’s Community Mobilization Approach According to discussions with central level actors, notably with the DPRE representative, one of the most crucial criteria for qualifying as a potential school construction site is that the middle school already existed and was at least partially operating within temporary structures. Statistics collected at the middle school levels clarified that all but one of the 23 middle schools included in the sample had been established prior to school construction project activities. Similarly, focus groups and interviews with school principals and particularly community members underlined that communities had organized and often times made financial contributions in order to establish middle schools within their towns and neighborhoods. Many interviews indicated that community actors had approached education inspectors and other authorities with requests to open a middle school. Upon the advice of these authorities, the communities then identified and/or constructed the temporary structures in which schools first operated. Removing schools for which data was uncertain, the range for the number years between school founding and occupation of Activity buildings was one to 10 years with an average of four years. “In the beginning, well before the program, when we wanted to create a middle school. Around 2006, there was strong community mobilization because people were interested. There was enthusiasm and very strong mobilization. We all really chipped in and all that for temporary structures. We had to develop our own resources in order to operate the middle school.” — PTA/SMC Member, Hilele Middle School, Kolda Region The implementation letters between USAID and the Government of Senegal identify the roles and responsibilities of some of the ministry offices. While the letters identify community mobilization as an aspect of constructing the schools, they do not identify any particular structure to carry out activities nor a role for the DESMG. The interview with USAID education team officials confirmed this to be the case and further argued that the activity did not include technical specifications for community 23 mobilization. Reimbursement was not dependent on efforts to engage communities nor any sort of community response. They also concluded that upkeep at schools is of a personal rather than systemic nature and depends uniquely on the management capacity of individual school principals. Discussions with stakeholders at all levels of the program (school/local-level, regional and central) indicated that community levels of participation varied at the time of the construction process. According to the former director of the DESMG, some communities were more successful than others in respecting their engagements. Similarly, the FARA activity coordinator at the DCS identified connections to water as a particularly difficult task for communities and that while many communities committed to bringing running water to the school site, they were often unable and this was often not clear until construction had begun and it was too late to reverse processes. At the Sedhiou Middle School III, the principal and teachers rejoiced at being able to connect to water a year or so after construction but that by that time, the plumbing had not been tested. As a result of numerous leaks, the school administration had to invest significant money so that water fixtures would operate correctly. “There are some communities that respect their commitments. There are others that want to respect their commitments but can’t because of a lack of resources. There are others who respect their commitments with the school but they forget about them.” — Former DESMG Director Discussions with stakeholders revealed that community involvement was also limited after construction. Many principals expressed difficulties getting parents to come to the schools for activities. Some principals mentioned that shortly after construction community members used to clear the ground at the beginning of the year but that now they need to pay people to clear the brush. Community members and other school actors also pointed to schools serving multiple localities and that the majority of parents living outside of the host community did not feel implicated. PTA and SMC actors at several schools repeatedly mentioned the hand-to-mouth existence of many families. In more rural areas, many participants commented that parents were farmers. The former director of the DEMSG mentioned that community mobilization was more difficult in urban communities, particularly because authorities were busier. In addition, principals, teachers and community members at some schools argued that many parents do not really value and care about education. Field discussions also indicated the need for more support and training. A parent at the Seby Ponty Middle School in Dakar identified wanting materials and financial support for school upkeep. Education inspectors from two regions accentuated the need to train community members both in the specifications of constructions as well as school maintenance. An education inspector from Sedhiou suggested future activities involve women’s organizations and that activities apply lessons learned from other projects such as PAEM Classe. Lastly, a key issue in the discussion with USAID and ministry officials following the presentation of preliminary results from the evaluation was that upkeep and maintenance are a serious concern for the sustainability of the construction Activity schools and that there is recognition within the Ministry of Education of needing to reinforce school maintenance efforts. Conclusions Activity community mobilization efforts took place in many communities and sought to engage the population around key issues. Personnel turnover and time passed since earlier efforts may explain the lack of information at some schools. In almost all cases, communities had actively advocated and organized to establish their local schools well before the school construction activity. Given this notable history of community engagement, it is impossible to discern the extent to which Activity community mobilization efforts may have affected community ownership of schools. 24 Community ownership of schools is evidenced through ongoing community support, although support for maintenance varies considerably across schools. The short duration of Activity community mobilization efforts and external socioeconomic and cultural factors may further complicate sustained community ownership of schools and participation in building management and maintenance activities. Question 4 To what extent has FARA promoted collaboration among stakeholders? Findings To answer this question, the evaluation focused both on the various levels of stakeholders as well as the different stages of program implementation. Stakeholders can be classified as acting at three levels: central, regional and local/school level. Note that there is overlap at times between community mobilization and collaboration as some of the actors who work directly with community members are formal structures as well (i.e., conseils départementals and mayors). In light of the complexity of the various institutions involved in the school construction activity, Table 1 provides a description of actors and their role within the Activity. Information from this table draws from resource documents (see footnotes) and interviews and focus groups. The stages of implementation include site identification, construction, and the period following construction when students, school personnel and community members make use of the new Activity school structures. TABLE 1: KEY STAKEHOLDERS OF THE SCHOOL CONSTRUCTION ACTIVITY Institution Key Stakeholders: Description Activity Role Central level: USAID officials, representatives from the Ministry of Education and the Ministry of Finance Ministry of Education (Ministère de l’Education Nationale, MEN) Direction de la Construction Scolaire, (DCS) - Office of School Construction : Lead school construction for the Ministry of Education. Lead coordinating actor. Responsible for school construction activities including recruiting and identifying contractors, supervising worksites, providing equipment, approving invoices and administrative receipts and submitting payment requests. Direction de l’Administration Générale et de l’Equipement, (DAGE) - Office of Administration and Equipment : Lead school construction for the Ministry of Education. Oversight of accounting paperwork submitted by DCS for approval by DI for final payment. Direction de l’Enseignement Moyen Secondaire Général,(DEMSG) - Office of Secondary Education : Office of MEN in charge of secondary education. Oversees all aspects including curriculum, personnel, and overall school operations. Led community mobilization activities. Direction de la Planification et de la Réforme de l’Education (DPRE) – Education Planning and Reform Office : In charge of planning for the Senegalese educational system and education statistics. Assured that site identification confirmed with education planning needs, as well as verifying that property proposed for school construction complies with legal requirements. Ministère de l’Economie et des Finances Direction de l’Investissement (DI) - Office of Investment : Manages contributions from investors. Managed Activity account on a daily basis. Provides approval of payments. 25 Institution Key Stakeholders: Description Activity Role Direction de la Coopération Economique et Financière (DCEF)- Office of Economic and Financial Coorperation Provides overall management of the SOAG between the Government of Senegal and the USAID/Senegal Mission. As part of the SOAG, is responsible for overseeing the FARA mechanisms on a macro level. USAID Education Team: Oversees educational projects in accordance with US Government assistance mandates. Facilitated initial agreements for the Activity. Provide technical oversight necessary to approve reimbursement. Regional Level: education inspectorates (IA and IEF) and regional administrative authorities (governors and prefectures) Ministère de l’Education Nationale Education Inspectors (Inspection d’Académie, IA) 15 : Provide regional oversight for education programs. Representative of MEN at regional level. Direct supervisor of all schools in the district. Among other responsibilities, oversees school construction and rehabilitation projects as well as regular maintenance. Oversees Inspection de l’Education et de la Formation (IEF). Supports RTC. Varies. Education Inspectors (Inspection de l’Education et de la Formation, IEF) : Provides district-level oversight of education programs. Representative of MEN at local level. Plans and manages preventative maintenance activities in concert with local communities. Specifically charged with middle school education. Varies. Regional Technical Coordinator (RTC) (Coordinateur technique regional, CTR) : Technical arm of Inspection d’academie (IA). Regional representation of the Direction des constructions scolaires (DCS)16. Provides local oversight of construction projects. Assists with community mobilization activities. Reports directly to Direction des constructions scolaires (DCS). School/Local Level: middle school administration, students, community members, and local authorities (mayors, Conseil départementale - district council officials, village chiefs) Communauté Principal : Manages school operations. Reports to IEF and IA. Manages maintenance and upkeep of school. During time of construction, serves as a facilitator. District council (Conseil départemental) 17: Created by Decentralization Act III in December 2013, a local council that among other responsibilities, is charged with the outfitting and upkeep of middle schools. Varies. Parent-Teacher Association (PTA) : Group of parents (broadly defined) who support school’s functioning and play a facilitating role towards the community. Liaise between school and community. Support school principal, especially with upkeep and maintenance. 15 See Republique du Sénégal. (2012). Decret No. 2012-1276 relatif à la création des Inspection d’Académie et des Inspections de l’Education et de la Formation. Accessed from http://www.education.gouv.sn/root-fr/upload_pieces/DECRET%202012- 1276%20relatif%20%C3%A0%20la%20cr%C3%A9ation%20des%20%20IA%20et%20IEF.pdf 16 Note that Dakar does not have a RTC because Ministry officials are located there. Also, at the time of the evaluation, the region of Saint Louis did not have a RTC because the former RTC had retired and was not yet replaced. 17 See République du Senegal (2013). Loi n° 2013-10 du 28 décembre 2013 portant Code général des Collectivités locales, Article 312, p. 42. Accessed from http://www.au-senegal.com/IMG/pdf/code_general6119.pdf 26 Institution Key Stakeholders: Description Activity Role School Management Committee (SMC) : Composed of PTA president, school principal, teacher representative and student representative. Support principal and PTA. Supervise construction. Support community mobilization and continued engagement. Collaboration at the Central Level The school construction activity implementation letters stipulate that the Direction de constructions scolaires (DCS) is solely responsible for the construction process working directly with contractors. In addition, interviews revealed that, in practice, the FARA activity coordinator at the DCS coordinates the steering committee and Activity joint field visits while the DCS Director presides over steering committee meetings. While not mentioned explicitly in implementation letters, interviews and focus groups as well as project documents explained that the Direction de l’Enseignement Moyen Secondaire Général (DEMSG) leads community mobilization efforts. The Direction de la Planification et de la Réforme de l’Education (DPRE) is involved in site selection, according to the interview with the DPRE representative to the Activity. The Direction de l’Administration Générale et de l’Equipement, (DAGE) oversees procurement and fiduciary operations according to the implementation letters. Discussions with the DAGE, the Direction de l’Investissement (DI) and the DCS accentuated the DAGE’s role in reviewing accounting paperwork and requesting approval from the DI. An interview with the DI representative for the Activity clarified that the reimbursement account is located with the Ministry of Finance and is operated by the DI. When asked why the DI is involved with this MEN project during an interview, the DI representative for the school construction activity explained that because it is a program with an international donor, the President of Senegal has tasked the DI with creating an account and assuring oversight. He further added that the government-to-government nature of the school construction activity requires DI participation. Interviews with ministry officials repeatedly identified two mechanisms in particular when asked about collaboration: the Activity steering committee and joint field visits. Interviews with stakeholders with greater program longevity, notably from USAID, DCS, and the DEMSG reported that steering committee meetings originally took place monthly and involved the office representative for the Activity but ideally also the office Director. The DCS coordinator expressed that meetings were intended to provide an opportunity for each ministerial office to provide an update of its activities in support of the Activity. The steering committee members would also get together to solve problems and make joint decisions. The Minister of Education had even participated in some earlier meetings. Interviews with representatives from the DAGE and the DI provided examples of topics that were discussed within such meetings such as the program budget and the decision to increase the number of schools for the fourth phase from the original nine to 18 middle schools. Similarly, monthly joint field visits were initiated to allow steering committee members to visit construction sites and to monitor progress and intervene according to their various roles within the field. All discussions with ministry officials as well as USAID education team members revealed that both joint field visits and steering committee meetings became less regular towards the end of the project decreasing from monthly to trimestral meetings. In fact, no participant was able to identify the date of the most recent steering committee meeting, although USAID officials mentioned that a few meetings took place in 2015 due largely to insistence from USAID for the DCS to involve all members. At the same time, it seems that all ministry representatives were not present, so it remains unclear when all committee members last successfully convened. Discussions with USAID officials indicated that there may be an issue of limited buy-in amongst currently involved ministerial officials and perhaps tensions as to which entity should be coordinating the school construction activities, notably, facilitating the steering committee meetings. 27 One of the central issues investigated around this question concerns the roles and responsibilities of individuals and structures. At the DCS, the FARA activity coordinator has held that position since the program began. Discussions with the FARA coordinator himself as well as many other central and regional actors revealed that the FARA coordinator worked largely independently at the DCS. He also went frequently to visit construction sites, often traveling throughout the country at least once a month in addition other joint visits. He further identified the Activity’s inclusion of travel funds as a point of satisfaction in the program. Lastly, the FARA coordinator at the DCS plans to retire in December 2016 and during an interview expressed concerns about his replacement as it seems they have had little involvement with the activity. Local and Regional-Level Collaboration According to discussions with many participants encountered in regional areas, there is a lack of information about the Activity. Several school principals as well as education inspectors (both Inspecteurs d’Academie, IAs and Inspecteurs de l’éducation et de la formation, IEFs) expressed unfamiliarity with USAID’s support for construction of Activity middle schools. A number of stakeholders indicated they did not hold their current positions at the time the schools were built and did not receive related information when they assumed their new roles. At the same time, while some inspectors expressed high levels of satisfaction with their involvement in the Activity, others did not and many of the latter had difficulties answering questions about the Activity during interviews. Inspectors who had been present and wished to be more involved suggested more regular consultations, particularly when choosing school sites, given their real-time local knowledge. “How it [school construction] was done there and what they [Matching program] are doing now? I admit that I don’t have any information which causes dissatisfaction in terms of communication and sharing of information.” (Inspector, Saint Louis Region) When asked about their level of involvement in the school construction activity education inspectors and regional administrators expressed confusion and frustration at times concerning the roles of their structures. For example, discussions with IEF and IA sometimes indicated overlaps in responsibilities and confusion as to which structure should oversee middle schools. At the same time, meetings with district council (Conseil départemental) officials highlighted that these regional headquarters were relatively young, in most cases, less than two years old. These new structures are a result of Decentralization Act III made law in December 2013. While the district councils are responsible for equipping and maintaining middle schools and working closely with other implementing entities, discussions indicated that their involvement with middle schools is still evolving. Nonetheless, discussions with school-level stakeholders and with officials at the district council offices shed light on some ways that they have already provided support to Activity middle schools. Notably, both the Hilele (Kolda Region) and Sedhiou III Middle Schools (Sedhiou Region) receive financial support to pay for school guards. The Activity school at Mboss (Région de Kaolack) received computer equipment. The district councils has also provided the Sedhiou III Middle School with a connection to city water and electricity. Similarly, they pay the Koussy Middle School’s secretary’s salary (Sedhiou Region). Visits to schools and local administrative offices provided numerous examples of the involvement of mayor’s offices in school construction activities, most notably during site selection. Interviews with school principals, community members and education inspectors affirmed the importance of assistance provided by mayors’ offices in identifying and confirming the plot of land dedicated to school construction. In some cases, the mayor’s office was able to identify an adequate plot of land that was unoccupied while in other cases, for example for the Koussy and Sedhiou III Middle Schools, the mayor’s office negotiated the relocation of inhabitants. As indicated above, mayors are also expected to play a 28 significant role with community mobilization efforts yet they are included here in this question because of the official role that they play. Support from mayor’s offices also continues beyond identifying the land for the school. For example, mayors’ offices pay water bills for some schools including Seby Ponty Middle School (Dakar Region), Ndoumga Wouro Alpha Middle School (Matam Region) and Zone Sud de Mbour Middle School (Thies Region) middle schools (interviews with principals). The case of the Koussy Middle School stands out from data collected as an example where challenges remain despite mediation of local authorities. At Koussy, school personnel explained that there is a plot of land between the road and the school and that the owner of the land refuses to accept the mayor’s offer. At certain points during the year, students and school personnel need to walk through tall grass or around crops in order to reach the school entrance. Multi-Level Collaboration Interviews and focus groups provided information on how school sites are chosen and indicate collaboration across actors from the central ministerial level to the communities. Discussions with the DPRE clarified the process of identifying sites. Not only did sites need to have an already existent school at least partly housed in temporary structures, as indicated above, the mayor’s office must be willing to identify an appropriate location and negotiate as necessary. During interviews, DCS officials provided information on the factors involved in choosing a location, noting the school must be close enough to the population to assuage safety concerns, especially for female students. Schools would also ideally be located close to the last electrical post and water source in order to facilitate connection to electricity and running water. However, this was not always possible in rural areas. The same interview offered two examples of when proposed plots of land were refused: in one case, the school would have been located next to a cemetery. In another case, the land was subject to flooding. An education official in Kolda also explained how the town of Velingara was the original site chosen for the current Teyel Middle School, however, when the mayor and community was unable to identify a large enough space in town the project was transferred to Teyel. Discussion with actors also revealed the number of steps involved in site identification, beginning with community mobilization and discussions with local authorities. Subsequently, engineers and technicians needed to approve the site. The next phase after site approval was community mobilization efforts, which were varied according to participants. In addition, all but one community had already approached local authorities and rallied to establish a school prior to identification as a site for Activity school ocnstruction. “During the site identification phase, we do it together, the whole process. When we finish, we have a strong connection.” — DPRE Representative According to discussions with the FARA activity coordinator at the DCS and Regional Technical Coordinators (RTCs) in the field, RTCs are involved in site selection, support DEMSG community mobilization efforts and oversee the technical areas of the school construction process. In addition, discussions with RTCs clarified that they are rarely involved with schools once construction has been completed. In some cases, the evaluation visit was the first time that the RTCs had returned to schools since the official handover of the school. During interviews, some principals indicated that they were not aware of the role of RTCs. Interviews and focus groups with school principals and community members as well as with ministerial actors (DCS and DEMSG notably) also highlighted official school inaugurations as examples of collaboration among multilevel stakeholders. As one education inspector explained, the inauguration provided an occasion for stakeholders from all levels to convene at the construction site and to further 29 raise awareness among beneficiaries of the importance of the schools. Less than half of the 12 schools involved in the qualitative study had inauguration ceremonies and many of the other school principals lamented not having had ceremonies and feeling as if the school had never officially been turned over. “When it was time to inaugurate the school, I would say that none of the actors was asleep. From the central level, students, up to the local level, everyone was gearing up for a ceremony.” — Education Inspector, Sedhiou Region “If there had been an inauguration ceremony for the middle school where they presented the principal and the population with the keys, generally speaking, there would have been collaboration. But the people just came in here behind closed doors. So, it’s difficult to speak about collaboration and up until now, the school still hasn’t been officially turned over.” — Principal, Sedhiou Region At some schools, local actors indicated having little role in the construction process. Some school officials and community members indicated feeling more like observers while others felt entirely shut out of the process. During focus groups at least four of the 12 schools where in-depth qualitative methods were used, community members and principals explained that they did not have opportunities to provide input concerning the school layout or the construction process. At the Merina Dakhar Middle School (Thies Region), the principal emphasized a lack of direct communication between the contractor and the community despite the community’s mobilization for the school in the first place. At Seby Ponty Middle School (Dakar Region) a participant in the focus group with PTA/SMC members said that the school construction concerned only administrators and that communities only observed. Community members from the Teyel Middle School (Kolda Region) said that they did not participate at all in the process while the village chief from Ngothie lamented not being able to provide any sort of technical supervision because he had not been provided the plans for the school. At the Sedhiou Middle School III (Sedhiou Region) the focus group discussion revealed a sense of exclusion from the process. As illustration, community members and teachers at Sedhiou Middle School called attention to the location of the school opening, which opens onto a neighbor’s property when they would have preferred to access the school from another side where there was a road leading to the school. They explained that they had to negotiate with the neighbor in order to allow access and that when adding additional constructions to the school, they needed to break the cement wall enclosure so that construction equipment could enter the complex as it could not maneuver through the actual gate of the school. “Sincerely, I don’t know how to answer your question because the contractors here didn’t have any consideration for others. They went about their work and they left and we didn’t have the right to watch. Even the principal can’t respond to that question because he wasn’t here at the time. They even made a barrier to limit access and we had to negotiate in order to enter the school because they said it hadn’t been inaugurated. I can tell you that the population was excluded during construction.” — PTA/SMC Member, Sedhiou III Middle School, Sedhiou Region At the same time, the evaluation team frequently heard about difficulties that were largely external to the Activity that nonetheless may have made collaboration challenging. For instance, many teachers, principals and PTA representatives commented that parents are often too busy to come to school, often due to socioeconomic conditions. Similarly, others identified cultural beliefs and weak valuation of school as an obstacle to more fruitful collaboration. Many others spoke about the competition between villages and town as to who was responsible for the school and how only host community members often participated in school activities. 30 Post-Construction Maintenance and Repairs As indicated above, the evaluation sought information about collaboration among stakeholders throughout the process, including once schools were fully constructed. How actors continue to care for Activity structures may provide insights into collaboration at the school level. Discussions with the DCS suggested that the district councils are responsible as well as ultimately the schools themselves, thus involving the DEMSG. Discussions with the DEMSG identified school principals as responsible for upkeep. The USAID education team made reference to Maintenance and Repairs clause18 of the amended implementation letter for the fourth phase that requires the government of Senegal to assure the maintenance and repair of Activity schools. When asked about the government’s application of this clause, interviews with DCS officials indicated that they are currently developing training activities to support schools with the upkeep, including provision of cleaning materials and everyday maintenance tools. They intend to launch these activities before the end of the year to correspond with the end of construction of phase four Matching schools. Discussions with the DI representative confirmed that the budget line for maintenance and repairs allocated for the last phase has not been spent. At the school level, observations of school grounds revealed various states of upkeep, frequently identifying serious issues, most notably in bathrooms, some of which were structural and as indicated previously rendered them non-functional. Interviews with principals provided more information. Of the schools where information was available, 12 of 19 schools employ people to clean bathrooms, in all cases women. These women are paid from three sources: school budgets (8 schools), SMC budgets (3 schools), and in the case of the Hilele Middle School a support fund organized by school personnel. In addition, many schools use other strategies including organizing students to sweep classrooms and clear the school grounds and holding schoolwide cleaning days called seet seetal. In many cases, tasks are gender-specific with girls sweeping and cleaning while boys clear brush and take care of the schoolyard. Observations during site visits, however, revealed that in spite of these efforts, many establishments still had difficulties taking adequate care of Activity structures. Hilele Middle School, stands out as one of the few exceptions, however, as interviews and focus groups with school and community actors told a story of ongoing and effective collaboration, where student groups care for the school grounds under the guidance of school supervisors and with parental consent. Observations at the school confirmed the exemplary state of the school. “We told parents that because we don’t have any cleaning staff, because we don’t have a school guard, that we need students to involve themselves directly in the school’s upkeep. So, we’ve created teams of girls and boys in each class to sweep the classrooms. Each week, there is a supervisor who works with students to clean the bathrooms…Students take turns, right, sweeping their classes and this improves the conditions. Each month, well, at least every two months, we hold an external cleaning activity that we call seet seetal in the courtyard and around the school grounds. It’s organized by the students with supervision from the school supervisors.” — Principal, Hilele, Kolda Region 18 Implementation letter No. 685-013-05, Clause 8, p. 4. 31 Photos 12–14. Well-tended bathrooms at the Hilele Middle School (Kolda Region) Conclusions The Activity required collaboration across a number of levels (central, regional and school/local level). Program operations, particularly, financial management, attest to cooperation amongst actors at the central level. Similarly, the steering committee and related joint site visits provided a framework for collaboration. Expectations may have been unrealistic however, and the interest in and ability to coordinate amongst the involved ministerial offices is questionable and waned towards the end of the project. Collaboration supporting the Activity, particular at the local and regional levels, seems to have been hampered by a lack of information sharing. This may have been aggravated by personnel turnover and resulting loss of institutional memory. The individualized roles of certain actors, particularly the pivotal role of the DCS FARA activity coordinator, may also have weakened collaboration. Additional external factors are also noteworthy including limited parental involvement. Involvement of communities during the construction process was limited. Weak relationships and unclear roles between communities and contractors as well as inflexibility in school design may have lessened community contributions and dampened community ownership of Activity schools. School construction activities did little to promote collaboration amongst actors. As a result, confusion and a lack of ownership for post-construction maintenance complicates regular upkeep and repair of schools and may limit the sustainability of the Activity. Although multiple entities are theoretically responsible for upkeep, in practice, decentralization mandates remain ineffective and good maintenance seems largely to depend upon principals’ management capacities. While a few schools demonstrated effective management strategies, intensive training and ongoing coaching are needed at the school level as well as with regional actors. In addition, school upkeep as currently organized may follow cultural practices that involve girls more than boys in cleaning activities reinforcing marginalization. Official school inauguration ceremonies at some Activity schools provided important symbolic opportunities for stakeholder collaboration across levels of the project and to further raise awareness of the importance of the schools and especially, their upkeep. 32 Question 5 To what degree has the FARA Program favored access to and continued attendance at middle schools for boys and girls? What has worked well? What has not worked? Findings Analysis for this question draws on both quantitative and qualitative methods. In accordance with the criteria identified for site selection by the Activity representative at the DPRE, interviews with administrative staff and communities revealed that in all but one case, the schools had already been established prior to Activity constructions. Middle school statistics indicators relevant to access and continued attendance for Activity schools were limited at the ministerial level (DPRE) and unfortunately, data collected directly from middle school level proved to be inconsistent and were not conducive to analysis. Because Activity schools were constructed at various times between 2007 and 2016, the evaluation employed a strategic statistical approach relying upon data from three periods. Even with a reduced number of target data, data were missing for a number of the schools. The three periods analyzed for each school were:  Year after establishment: Academic year after the school’s founding (varies)  Year after entry: Academic year after students entered new Activity buildings (varies)  End-of-project year: Academic year relevant to end of Activity (2014-2015) Interviews and focus group data from actors at all levels (middle school/local, regional, central) provide further evidence and context of ways in which the Activity constructions may have promoted access to secondary schooling as well as continued attendance19. Increased School Enrollment When asked if Activity school constructions favored access, all participants responded positively. At the central level, both the former director of the DEMSG and the Activity representative at the DPRE agreed that school populations had increased as a result of the new school structures. In addition, some principals and teachers shared that the demand of students to attend their schools exceeds their capacity, including at the Hilele Middle School (Kolda Region) and the Sedhiou Middle School III (Sédhiou Region). Official statistics for each school confirm an increase across sample schools. (See Annex VI for information on student population changes by gender across Activity schools according to the three periods identified above.) Statistics for all three target years above were available for only eight of the 23 schools included in this sample20. Given the presence of data for all three indicator years, it is possible to compare data chronologically taking into account enrollment one year after schools were established, enrollment the year after students entered Activity classrooms and enrollment during the 2014-2015 academic year (understood here as the proxy for 2015-2016 as data was not available at the time of analysis). Analysis of school population data confirms an increase in student numbers for all eight schools. Disaggregated by gender, the number of girls enrolled also increased between the year of after school establishment 19 Although ideally this question would have been able to compare data from sample schools with an identified group of control schools, this was beyond the scope of the evaluation design. While the evaluation team had considered comparing Activity schools with statistics from other schools in the same district/region, difficulties collecting statistics, concerns for data reliability and a lack of understanding of differences between schools prohibited such a comparison. 20 The 8 schools are somewhat representative of the larger sample. Half of the schools are rural schools and half are urban. Two of the schools were built in phase 1, four in phase 2 and two in phase 4. Two of the schools have USAID water sources. 33 and the year after construction of Activity complexes. In most cases, this increase is sustained when comparing with school population data for the last year of the project. When looking at the differences between the year after the establishment and the first academic year after students entered the Activity buildings, school populations at these eight middle schools demonstrated an increase ranging from 8% to over four times the population with an average increase of over twice the school population. (See Figures 3 and 4 below). For girls, this increase was even greater. The range of the increase for the percentage of girls at Activity schools varied from more than doubling to increasing more than five times. These figures are even higher when taking into account the most recent school year for which data was available (2014-2015). School populations at these eight middle schools increased in general an average of over four times between the year of school establishment and academic year 2014-2015. Again, for girls, the change was even greater and demonstrated an increase of four and a half times the number of female students at the school between the establishment year and the last year of the school construction activity. While analysis of Activity school enrollment data provides information relevant to student access and continued attendance, baseline and peer school data was absent. This evaluation did not seek to compare Activity schools with regional or national statistics. Similarly, this information does not account for external factors, including the context of Education for All, which favored an increase in elementary school enrollment. FIGURE 3: PERCENTAGE (%) CHANGE BETWEEN YEAR OF ESTABLISHMENT AND OCCUPATION BY GENDER AT ACTIVITY MIDDLE SCHOOLS 83% 26% 40% 157% 70% 151% 78% 25% 58% -6% 58% 110% 79% 93% 95% 0% Urban Urban Urban Rural Rural Rural Urban Rural Mbao Kamb Fatick 3 Zone Sud Mbour Merina Dakhar Ngothie Boynadji Teyel Koussy % change girls % change boys 34 FIGURE 4: PERCENTAGE (%) CHANGE BETWEEN YEAR OF ESTABLISHMENT AND END OF PROJECT BY GENDER AT ACTIVITY MIDDLE SCHOOLS Families and Students Encouraged Qualitative data provided further explanation of the impact that the Activity schools have had on school community members. When asked if the school construction activity had an impact on school access, many participants explained that their middle schools are now closer to their communities. At Ngothie, for example, previous to the establishment of the middle school, students had to walk seven kilometers to the nearest middles school (PTA/SMC focus group). Moreover, prior to the establishment of the middle school at Ndiaye Ndioungue Mberess, the closest middle schools were approximately 30 kilometers away. Comments revealed that closer proximity pleases parents in allowing them greater supervision over their children. Students also indicated that they have more time to study as a result of a shorter commute. In addition, several teachers, community members and students shared that local schools provide a solution to complicated issues that students often faced while living with a guardian. Many principals, teachers and community members also indicated that the establishment of local middle schools allowed them to reintegrate students who had previously left school. At Seby Ponty Middle School (Dakar Region), for example, the school was able to reenroll students who had previously dropped out of school. Similarly, the principal of the Simbandi Balante Middle School explained that with the conflict calming down recently, parents brought back their children back from neighboring Guinea Bissau where they had been going to school so that they could enroll at the newly constructed Activity middle school. At Sedhiou Middle School III, teachers explained how they were able to reintegrate students who had dropped out previously because of a lack of transportation money. Several of the above comments regarding schools being closer address the time between the establishment of the schools and the present. As indicated earlier, in all but one case of the 23 sample schools, the institutions had already been established in temporary structures prior to the construction Activity. The average time between establishment and Activity construction was 4 years. 236% 43% 164% 265% 208% 247% 18% 22% 147% 1% 147% 264% 136% 86% 24% -8% Urban Urban Urban Rural Rural Rural Urban Rural Mbao Kamb Fatick 3 Zone Sud Mbour Merina Dakhar Ngothie Boynadji Teyel Koussy % change girls % change boys 35 According to discussions with students, school personnel and education inspectors, the school environment created by Activity structures promotes enrollment and continued attendance at the middle school level. Teachers at Merina Dakhar Middle School (Thies Region) identified how the Activity complex provides greater security and has a larger capacity, allowing more students to go to school there. Several participants identified that having well-constructed cement structures for students transitioning from elementary schools provides greater motivation than if classes were to take place in temporary structures. When asked about their impressions of the school structure, as indicated in the response to question 1, many students cited high levels of satisfaction in having permanent classrooms with tile floors, water and electricity on the premises, and dedicated bathrooms. The principal and teachers at Teyel Middle School told a story of primary school children competing with one another at the time of the school construction and explaining that if they did not do well, they would not be able to go to the new school. An education inspector in the Kolda Region also shared that students are proud to be at a school with a much improved environment. Similarly, at Activity schools with additional classrooms made of plant-based materials or other less professionally constructed structures, some teachers and principals explained that the older grades occupied the Activity classrooms while younger students occupy the temporary structures. As a result, progressing in classes in order to have class in the Activity classrooms is a source of motivation for students. According to some students and teachers, having electricity at the school is also helpful as it allows students to charge their cellphones that they use at night as flashlights while doing their homework; this helps them keep up with classwork and progress through school. “If there additional buildings were build next to the classrooms, such as bathrooms, an enclosure wall for the athletic field, the school becomes a second home. All of a sudden, the school children feel just as comfortable at school as at home.” — Education Inspector, Kolda Region Discussions also revealed that participants appreciated the Activity’s attention to the specific needs of female students. Many participants including teachers, principals and community members identified the separate girls’ and boys’ bathrooms as a strong point of the Activity approach. For instance, an education inspector from the Saint Louis region argued that separate bathrooms has had an impact on girls’ school attendance. Comments from student focus groups, in particular, contradict these sentiments, however, as a number of students commented on the degraded nature of the bathrooms and how many boys and girls, but particularly girls, continue to use neighborhood houses to meet their needs. Similarly, observations of bathroom areas revealed that bathrooms at 6 of the 23 sample schools are non-functional. “The fact that girls feel comfortable at school is already reassuring. At least in terms of access but also in terms of attendance. Here there are more girls than boys at the elementary and middle school levels. So, building high quality infrastructure and especially bathrooms guarantees girls the possibility to take care of their needs without needing to go someplace else and improves attendance.” — Education Inspector, Matam Region Persisting Challenges “Many have left school because of the cost of school enrollment, lodging, early marriages and unintended pregnancies.” — Female Student, Merina Dakhar Middle School, Thies Region Due to inconsistent data collected from school principals and difficulties obtaining statistics from the Direction de la planification et la reforme de l’éducation (DPRE), only one indicator related to continued attendance was available for quantitative analysis. While ideally analysis would have been drawn from multiple sources, including the school dropout rate, success on the end-of-cycle exam (BFEM) and 36 transition rates to high school, the only related rate available was the repeater rate. Analysis of this rate across schools revealed that repeat rates have increased for the majority of schools across the three years identified (year after establishment, year after entry into Activity buildings and the end-of-project year). Statistical analysis of these rates only demonstrated that the number of girls attending schools in rural zones had a statistically significantly higher probability of repeating. Although comparison of schools with exemplary maintenance, electricity or other variables may also have provided insights into factors promoting lower repeat rates, this information was not conducive to statistical analysis. Discussions at the local and regional levels of how Activity schools may have favored access and continued attendance often included comments about persisting prohibitive factors. Common responses included socio-economic and cultural factors such as limited resources, early marriages, unintended pregnancies, pressure to assist parents, workforce opportunities and difficulties with guardians when residing away from home. Many principals, teachers, students and community members also commented that limited parental involvement and parents’ inability to keep up with their children’s schoolwork often reduces students’ continued attendance. Community members and students also provided examples of how even with schools being closer, many parents still cannot afford enrollment fees nor purchase necessary school supplies. “My brothers can’t come here because there isn’t the money. Dad and Mom, for them to bring all the kids here, to travel 7 kilometers, it’s very difficult.” — Male Student, Sedhiou Middle School III, Sedhiou Region Conclusions The school construction activity created a school environment that has motivated many students to learn and to feel comfortable on the school grounds. Increased capacity and closer ties to the local community have allowed some schools to reincorporate children who had previously left school. Activity beneficiaries, especially at the local and regional levels, feel strongly that Activity construction efforts have led to improved access and continued attendance. Most notably, having schools in close proximity decreases costs and allows parents greater supervision over their children. As a result, parents feel more comfortable sending their children, particularly their daughters, to school. These findings are somewhat unclear however, as almost all schools already existed at the time of Activity constructions. The number of students attending the Activity middle schools improved greatly when compared with the student population previous to construction. Unfortunately, it is not possible to control for external factors, including the effect of Education for All and its push for universal primary education. In addition, a lack of baseline data for all schools as well as an absence of peer-schools limits understanding of the impact that the Activity construction project may have had in terms of access and continued attendance. Analysis of the only indicator related to continued attendance, the repeater rate, shows an increase across the three targeted years of analysis. Several factors may account for these negative results including inadequate school management and increased school populations. Recommendations Understanding that actors at USAID as well as the Ministries of Education and Finance in the Government of Senegal may consider this report in the elaboration of future school construction programs, we offer the following recommendations: 37 USAID  Continue to support the Government of Senegal in its school construction efforts in response to the evident need, especially of school populations still in temporary structures.  Require trimester reports to be completed by the Government of Senegal with updates from each ministerial office. Such reports should be integrated into project milestones. These reports should be disseminated across the ministry offices.  Rely on lessons learned through PAEM Classe project and other activities to identify an ongoing training and mentorship approach for school administrators, regional school construction representatives (RTCs) and contractors in community engagement and ongoing community mobilization strategies. USAID & Ministry of Education & Ministry of Finance  Identify clear roles, responsibilities and expectations for both preventative and ongoing school maintenance. Actors should include institutions already recognized by the Government of Senegal as responsible for school maintenance, including the regional and local education inspectorates (Inspection d’Académie, Inspection de l’éducation et de la formation) as well as the district council (le Conseil départementale). Appropriate dedicated budget line items should be identified and communicated to all involved parties in order to facilitate collaboration.  Develop explicit language within the FARA regarding institutional capacity building, if this is truly an objective of the G2G work. It should stipulate exactly what areas and how each party (both the MEN and USAID) expects the capacity of the various stakeholders to change and the plan for measuring change.  Focus on MEN systems and how to improve collaboration and joint monitoring of school construction within a FARA.  Work closely with MEN to understand how to implement a school maintenance budget for already constructed schools. In support of decentralization efforts, these budgets would need to be developed in concert with school principals, the relevant local structures (education inspectorates, conseils départementales, mayors, etc.) and in a community participatory manner. Ministry of Education  Review coordinating role that DCS FARA activity Coordinator played during process and identify solution so that the coordinating entity of school construction program has the political authority necessary to convene actors from multiple ministerial offices.  Reinstate the Steering Committee and the use of joint field visits. At the same time, such meetings and visits should respect a reasonable timeframe that will allow continued participation at the office Director level. Ongoing team-building activities should be integrated into the work of the Steering Committee.  Institute regular reporting requirements from each directorate in order to compile updates and lessons learned on the FARA activity, perhaps by quarter. This could then set the agenda for the steering committee and help guide joint site visits to schools.  Implement regular monitoring activities to address the implication of actors at all levels, technical capacities, community engagement efforts as well as access and students’ continued attendance.  Institute consistent documentation practices that will help to retain institutional memory despite personnel turnover. 38  Reinforce the involvement of regional actors, including the Inspections d’academie and the Inspections de l’éducation et de la formation, in school construction and school maintenance projects.  Commit to participating in inauguration ceremonies for all schools built.  Adopt a similar high-level of quality-control and oversight for all of MEN school construction, which would require additional funds, manpower and means of transportation for the DCS.  Consider sub-contracting Senegalese engineering firms to support their ability to oversee quality school construction in order to support cost and efficiencies. Direction des Constructions Scolaires (DCS):  Build upon lessons learned, including continuing to work with competent and responsible contractors, enforcing technical specifications, and further strengthening coordination through the Regional Technical Coordinators (RTCs).  Emphasize the need for a reliable water source at the time that construction begins. Only install plumbing in schools with communities that have the capacity to organize for and maintain such a water source. For all other schools, rely upon latrines instead.  Identify a school model that can be tailored to the needs of schools and their projected population in the future. Conduct needs-assessments prior to construction so as to provide the most appropriate model possible for each and every community.  Reinforce lessons learned about equipment procurement and distribution and involve regional actors in the distribution of materials. Have clear needs assessments done prior to material distribution. Direction de l’Enseignement Moyen Secondaire Général (DEMSG)  Working in tight collaboration with the DCS, strengthen and lengthen community mobilization efforts in order to respond to the actual needs of communities, involve communities in each step of the construction process and accompany schools and communities in maintenance.  Provide ongoing mentorship to school principals on school maintenance and include related modules in training for new principals, specifically focusing on student bathrooms. Identify creative ways to engage students, teachers, community and PTA members to take care of their school environment, including competitions between schools with a symbolic reward for the winner.  Provide gender training for school personnel, community members and students to make them aware of how gender stereotypes and roles may inhibit learning, particularly for female students.  Include proper maintenance as a criterion for performance review of principals. School Principals  Foster strong engagements with community members.  Identify an approach to maintaining the school grounds that promotes student ownership while also taking into account gender equity.  Integrate school upkeep into the school’s culture, providing information sessions and ongoing support to students, especially regarding proper bathroom use and care. 39 ANNEX I: EVALUATION STATEMENT OF WORK EVALUATION DU FARA POUR LA CONSTRUCTION DE COLLEGES CADRE DE L’EVALUATION APRIL 1, 2016 This publication was produced for review by the United States Agency for International Development. It was prepared by Management Systems International, a Tetra Tech Company, for the USAID/Senegal Mission-Wide Monitoring and Evaluation Project. 40 EVALUATION DU FARA POUR LA CONSTRUCTION DE COLLEGES CADRE DE L’EVALUATION Contracted under AID-685-C-15-00003 USAID Senegal Mission Wide Monitoring and Evaluation Project DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA xli SOMMAIRE 1. Informations de Base............................................................................................... 43 2. Contexte de l’Activité ............................................................................................. 43 3. Hypotheses de Développement et sa Mise en Oeuvre ........................................ 46 4. Sources d’Informations Existantes......................................................................... 47 5. Buts, Audience et Utilisations Prévues des Résultats de l’Evaluation................ 48 6. Questions de l’Évaluation ....................................................................................... 48 7. Désagrégation selon le Genre et Effets Differentiels ........................................... 48 8. Design de l’Evaluation et Methodologies .............................................................. 49 9. Analyse des Données............................................................................................... 52 10. Forces et Limites de l’Approche d’évaluation ...................................................... 52 11. Livrables de l’Évaluation ......................................................................................... 53 12. Composition de l’Equipe d’Évaluation........................... Error! Bookmark not defined. 13. Participation de l’USAID et Partenaires ............................................................... 55 14. Planification et Logistiques ..................................................................................... 56 15. Exigences pour le Reporting................................................................................... 56 16. Documents de Reference Annexés........................................................................ 56 17. Autorisations............................................................................................................ 56 Annexe I: Calendrier d’Exécution ............................................................................................ 58 Annexe II: Cadre de Résultats, Conclusions et Recommandations Avant la Rédaction du Rapport ............................................................................................. 59 Annexe III: Matrice des Reponses.............................................................................................. 60 Annexe IV : Budget ..................................................................................................................... 61 Annexe V : Curriculum Vitae..................................................................................................... 62 Annexe VI : Déclaration de conflit d’intérêt .................................... Error! Bookmark not defined. Annexe VII: Politique D’Évaluation de l’USAID .............................. Error! Bookmark not defined. Annexe VIII: Structure du Rapport d’Evaluation............................. Error! Bookmark not defined. USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA xlii SIGLES ET ABRÉVIATIONS APE Association des Parents d’Elèves CDCS Country Development Cooperation Strategy CDE Cadre De l’Evaluation CGE Comité de Gestion de l’Ecole COR Contracting Officer’s Representative CV Curriculum Vitae DAGE Direction de l’Administration Générale et de l’Equipement, DCS Direction des Constructions Scolaires DEC Development Experience Clearinghouse DEMSG Direction de l’Enseignement Moyen et Secondaire Général DPRE Direction de la Planification et de la Réforme de l’Education LOE Level of Effort M&E Monitoring and Evaluation MEN Ministère de l’Education Nationale MEP Monitoring and Evaluation Project OCB Organisation Communautaire de Base OMD Objectifs du Millénaire pour le Développement ONG Organisation Non Gouvernementale RGPHAE Recensement Général de la Population, de l’Habitat, de l’Agriculture et de l’Elevage S&E Suivi & Evaluation SOW Statement of Work TBS Taux Brut de Scolarisation TPM Technical Planning Meeting USA United States of America USAID United States Agency International for Development USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 43 1. INFORMATIONS DE BASE Manager du Cadre de l’Evaluation à l’USAID Alpha Ibrahima Bah, EDU Manager de l’activité21 Alpha Ibrahima Bah, EDU Manager du Cadre de l’Evaluation à MEP Sénégal Moussa Sy, Spécialiste Senior en Evaluation Titre de l’Activité Programme de Compensation forfaitaire (FARA) pour la construction, l’équipement et l’approvisionnement en eau de collèges d’enseignement moyen Date de début et de fin de l’Activité 19 janvier 2007 – 30 juin 2016 Accord d’Assistance N° 685-013 Partenaires d’exécution Direction des constructions Scolaires (DCS), Direction de l’Administration Générale et de l’Equipement (DAGE), Direction de l’Enseignement Moyen et Secondaire General (DEMSG), et Direction de la Planification et de la Réforme de l’Education (DPRE)/Ministère de l’Education Nationale Budget de l’Activité 9,707,500 Dollars US Zones d’intervention Nationale Titre de l’évaluation Evaluation de performance des quatre phases du programme FARA Période à évaluer Du 19 janvier 2007 à 2016 Début et fin de l’évaluation 26 Avril – 10 Août 2016 2. CONTEXTE DE L’ACTIVITE L’éducation et la formation constituent un important facteur d’épanouissement personnel et un moyen pertinent pour l’amélioration durable des conditions de travail et d’existence des populations. Conscient de cette situation, le gouvernement du Sénégal s’est engagé de manière résolue dans la promotion de l’éducation et de la formation. Il a ainsi élaboré la « Lettre de Politique Générale pour le secteur de l’éducation et de la formation » couvrant la période 2012-2025 et dont les fondamentaux se trouvent dans la vision suivante : « Un système d’Éducation et de Formation équitable, efficace, efficient, conforme aux exigences du développement économique et social, plus engagé dans la prise en charge des exclus, et fondé sur une gouvernance inclusive, une responsabilisation plus accrue des Collectivités locales et des acteurs à la base ». Plusieurs stratégies sont formulées pour opérationnaliser la lettre sectorielle et renforcer en qualité et en quantité l’offre éducative. Le gouvernement du Sénégal est soutenu dans ses efforts par plusieurs partenaires dont l’Agence des Etats Unis pour le Développement International (USAID) qui met en œuvre plusieurs activités dans ce sens. Le Sénégal a ainsi enregistré des avancées importantes pour le taux brut de scolarisation (TBS) au primaire qui, selon les données du Recensement Général de la Population, de l’Habitat, de l’Agriculture et de l’Elevage (RGPHAE) de 201322, a atteint 80,0% (79,1% pour les garçons et 80,9% chez les filles).Ce niveau élevé du TBS au primaire s’explique par les efforts importants consentis par le gouvernement du Sénégal pour être en phase avec les Objectifs du Millénaire pour le Développement (OMD) dont l’objectif N°2, à l’horizon 2015, était d’assurer 21 Le terme « Activité » se réfère selon la terminologie de l’USAID à « Projet » 22 Agence Nationale de la Statistique et de la Démographie (ANSD). Recensement Général de la Population et de l’Habitat, de l’Agriculture et de l’Elevage (RGPHAE 2013). Chapitre III: alphabétisation, scolarisation, niveau d’instruction et formation professionnelle, Septembre 2014. USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 44 l’éducation pour tous.23 Ces efforts accomplis au niveau du cycle primaire et qui se sont traduits par l’augmentation progressive du TBS ont entrainé un flux d’élèves à inscrire au cycle d’enseignement moyen. Ce qui constitue pour le gouvernement du Sénégal un défi important à relever en termes d’infrastructures d’accueil et de ressources humaines qualifiées pour satisfaire la demande liée à la transition primaire-moyen. Ainsi, le 16 juillet 2006, l’USAID et le gouvernement du Sénégal ont signé l’accord d’objectif stratégique pour une « Jeunesse sénégalaise mieux éduquée ». Cet accord prévoit une compensation plafonnée à un montant fixe (Fixed Amount Reimbursement Agreement - FARA) pour la construction et l’équipement de collèges d’enseignement moyen en zones rurales et urbaines et l’approvisionnement en eau. Il s’agit d’un mécanisme par lequel l’USAID rembourse au gouvernement du Sénégal un montant fixe pour la construction, l’équipement des collèges et la mise en place de points d’eau. La mise en œuvre du FARA est encadrée par des lettres d’exécution signées entre l’USAID et le gouvernement du Sénégal. L’objectif de cet accord est d’augmenter l’accès des élèves au cycle d’enseignement moyen en réduisant notamment les distances entre les collèges et les communautés et en améliorant sensiblement les conditions d’apprentissage. Il s’agit également à travers le FARA de renforcer les capacités du système (dans la planification, la réalisation, et le suivi des infrastructures d’accueil) en vue d’accroitre l’accès à l’enseignement moyen. Le mode opératoire pour la mise en œuvre effective de cet accord est précisé dans des lettres d’exécution signées entre les deux parties. A cet effet, le gouvernement du Sénégal a construit et équipé 46 établissements d’enseignement moyen et a approvisionné 30 collèges en eau. L’USAID, après une inspection satisfaisante des travaux, rembourse les coûts y afférents sur la base d’un montant forfaitaire fixé à l’avance. Le gouvernement du Sénégal, en vertu des dispositions de l’accord le liant à l’USAID, a délégué les pouvoirs au Ministère de l’Education Nationale (MEN), spécifiquement le Directeur de l’Administration Générale et de l’Equipement (DAGE) pour les achats et les opérations fiduciaires et au Directeur des Constructions Scolaires (DCS) pour la construction et l’équipement des salles de classe. La Direction de la Planification et de la Réforme de l’Education (DPRE) et la Direction de l’Enseignement Moyen Secondaire General (DEMSG) sont également dans la planification des activités. Depuis le début de l’Activité, il y a eu quatre Accords de remboursement forfaitaire entre l’USAID et le gouvernement du Sénégal correspondant aux 4 principales phases de l’activité. En novembre 2012, l’activité a fait l’objet d’une évaluation à mi-parcours portant sur la mise en œuvre technique et sur les activités d’inspection, de supervision et de suivi. Cette évaluation qui a couvert la période janvier 2007- Septembre 2012, a relevé pour chacune des deux composantes des atouts et faiblesses et formulé des recommandations générales et spécifiques. En référence aux résultats de cette évaluation et aux autres documents de suivi (rapports d’inspection, procès-verbal de pré-réception des collèges et des points d’eau), l’activité a permis la construction sur financement de l’USAID de 46 collèges équipés et de 30 points d’eau répartis dans 11 régions du Sénégal: Matam, Saint-Louis, Kaffrine, Fatick, Dakar, Thiès, Diourbel, Kolda, Tambacounda, Louga et Sédhiou. 23 La Cible de l’objectif 2 était ainsi formulée : « d’ici à 2015, donner à tous les enfants, garçons et filles, partout dans le monde, les moyens d’achever un cycle complet d’études primaires » USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 45 La carte 1 montre les collèges financés par l‘USAID dans chacune des phases du FARA CARTE 1 : REPARTITION SPATIALE DES COLLEGES SELON LES PHASES USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 46 La carte 2 présente la répartition par région des collèges construits avec le soutien financier de l’USAID. CARTE 2 : REPARTITION SPATIALE DES COLLEGES SELON LES REGIONS L’USAID, en collaboration avec le gouvernement du Sénégal, va conduire une évaluation des performances en vue d’identifier, pour la période du 19 janvier 2007 à présent, les réalisations du projet FARA, ses forces et ses faiblesses afin de permettre au MEN et à l’USAID d’en tirer les meilleures leçons dans les futures interventions. Pour les besoins de cette évaluation, MEP-Sénégal travaillera en étroite collaboration avec l’équipe Education à l’USAID et avec le Ministère de l’éducation nationale et les consultants qui seront contractés pour la mission. 3. HYPOTHESES DE DEVELOPPEMENT ET SA MISE EN OEUVRE Comme déjà indiqué précédemment, le flux d’élèves au primaire pose pour le cycle moyen d’enseignement secondaire de réels défis en termes d’infrastructures d’accueil et de ressources humaines qualifiées. L’Activité FARA s’appuie ainsi sur l’hypothèse selon laquelle la construction et l’équipement de nouveaux collèges, en réduisant les distances entre ceux-ci et les habitations, en assurant l’approvisionnement en eau et en offrant un cadre d’apprentissage plus approprié, va contribuer à l’augmentation des élèves entrants au cycle d’enseignement moyen et au maintien des USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 47 élèves qui s’y trouvent déjà. De ce point de vue, l’évaluation de l’Activité FARA devra analyser dans quelle mesure les infrastructures et équipements réalisés ont augmenté le nombre d’élèves (filles et garçons) au cycle d’enseignement moyen, et/ou aidé au maintien des élèves se trouvant déjà dans les collèges. Cette interrogation a pour objectif de mieux analyser les résultats de l’évaluation à mi￾parcours de FARA qui avaient révélé une réduction des distances séparant les établissements d’enseignement moyen construits aux habitations. En outre, l’Activité FARA part de l’hypothèse que l’implication des populations bénéficiaires dans les activités est un atout pour la maintenance des infrastructures scolaires réalisées. Le FARA comporte ainsi un volet « mobilisation communautaire » davantage réaffirmé dans la lettre d’exécution N° 4 signée en 2012 entre l’USAID et le Gouvernement du Sénégal. Sur la base de cette hypothèse, l’évaluation accordera une importance particulière à la mobilisation communautaire afin de mieux comprendre les forces et limites de la maintenance des ouvrages réalisés dans le cadre du FARA, ainsi que le niveau d’appropriation des équipements par les populations. 4. SOURCES D’INFORMATIONS EXISTANTES Pour les besoins de l’évaluation, les principaux documents suivants sont exploités : Titre du document Dates Lettre d’exécution No. 685-013-04 19 janvier 2007 Lettre d’exécution No 685-013-06 28 octobre 2009 Lettre d’exécution No 685-013-07 1 er avril 2010 Lettre d’exécution No 685-013-05 23 août 2013 Rapports d’inspection des écoles (Seby Ponty, Zac Mbao, Darou Khoudos, Simbandi Balante Mars 2015 Procès-verbal de pré-réception des collègues matching de Simbandi Balante, Sebi Ponty, Zac Mbao, Darou Khoudoss Janvier 2015 Liste des collèges financés par l’USAID Mars 2016 Liste des points d’eau à rembourser par l’USAID Janvier 2016 Rapport d’évaluation à mi-parcours du projet, Novembre 2012 Recensement Général de la Population et de l’Habitat, de l’Agriculture et de l’Elevage (RGPHAE 2013). Chapitre III: alphabétisation, scolarisation, niveau d’instruction et formation professionnelle. Agence Nationale de la Statistique et de la Démographie Septembre 2014 Etat des lieux de l’éducation de base au Sénégal de 2014 Mai 2014 A l’exception de l’état des lieux de l’éducation de base au Sénégal de 2014, tous les autres documents mentionnés ci-dessus ont été consultés pour l’élaboration du présent SOW. Il sera nécessaire d’approfondir leur exploitation et surtout d’identifier des documents complémentaires (notamment les rapports de suivi des performances de l’activité, la carte scolaire de la DPRE) dans le cadre de l’évaluation. USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 48 5. BUTS, AUDIENCE ET UTILISATIONS PREVUES DES RESULTATS DE L’EVALUATION L’objectif de l’évaluation est de générer des informations pertinentes sur les résultats de l’Activité FARA, principalement en termes d’accès au cycle d’enseignement moyen. Le focus sera mis à cet effet sur ses forces et ses faiblesses en rapport avec l’accès et le maintien des élèves au cycle moyen et aussi l’appropriation communautaire en termes de maintenance et de gestion des ouvrages. Sur la base des leçons apprises de la planification, de l’exécution et du suivi des activités du FARA, et au regard des résultats obtenus, l’évaluation va formuler des recommandations stratégiques et opérationnelles ciblant les différents acteurs impliqués. Les résultats de l’évaluation serviront à alimenter les réflexions de l’équipe éducation de l’USAID/Sénégal, élargie à l’USAID/Washington, dans le cadre de la révision en cours de son programme lié à l’accès dans les zones de crises et conflits, et dans la perspective éventuelle de nouveaux accords avec le gouvernement du Sénégal en harmonie avec la Stratégie Globale de l’USAID. Ils pourraient également aider le Gouvernement du Sénégal à réajuster sa politique en matière de construction et de gestion des équipements scolaires et d’accès au système éducatif en général. Enfin, les résultats de l’évaluation pourraient constituer une référence pour tous les acteurs (agences internationales de développement, société civile, secteur privé, gouvernement, collectivités locales, etc.) engagés dans la réalisation et la gestion des infrastructures scolaires en particulier et publiques en général. Ils pourraient également contribuer à la réflexion sur les mécanismes à mettre en place pour une appropriation communautaire des infrastructures publiques. Pour un large partage des résultats de l’évaluation, MEP Sénégal proposera à l’USAID l’organisation d’un atelier réunissant les partenaires clés. 6. QUESTIONS DE L’ÉVALUATION L’évaluation portera en priorité sur les questions suivantes : 6) Quelles sont les évidences de la pertinence de l’activité FARA dans le renforcement des capacités du système éducatif ? 7) Dans quelle mesure l’activité FARA a- t-elle favorisé la collaboration entre les différents acteurs impliqués ? 8) Dans quelle mesure les actions de mobilisation communautaire mises en œuvre dans le cadre du FARA ont-elles favorisé la participation et l’appropriation par les communautés bénéficiaires des ouvrages réalisés ? Quelles sont les forces et les limites de ce mécanisme d’appropriation communautaire par rapport à la maintenance et à la gestion des ouvrages réalisés ? 9) Dans quelle mesure l’Activité FARA a favorisé l’accès et le maintien des filles et des garçons au cycle d’enseignement moyen ? Quelles en sont les leçons apprises ? 7. DESAGREGATION SELON LE GENRE ET EFFETS DIFFERENTIELS L’évaluation prendra en compte la dimension genre tout au long du processus, notamment dans l’élaboration des outils, la composition de l’équipe et dans la collecte et l’analyse des résultats. Ces USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 49 derniers seront désagrégés selon le genre pour mieux prendre en compte les contraintes et opportunités spécifiques aux filles, garçons et aux personnes défavorisées. En effet, il ressort des données empiriques que les longues distances entre les établissements et les lieux de résidence constituent généralement une entrave à l’accès et au maintien des élèves à l’école, surtout les filles. De même, l’absence ou la précarité des infrastructures scolaires, comme des toilettes accessibles, fonctionnelles et séparées selon le sexe, le manque d’eau potable dans les établissements peut constituer une entrave de taille pour les filles. Des contingences socio￾culturelles comme l’excision, les grossesses et le mariage précoce des filles sont souvent en faveur de leur déscolarisation. Il en est de même de la nature élargie des familles qui ont tendance à promouvoir plus l’éducation des garçons que celle des filles. Pour toutes ces raisons, les incidences du projet sur les filles et les garçons seront analysées et feront l’objet de sections dans la présentation du rapport. Une attention sera par ailleurs accordée à l’incidence de l’activité FARA sur l’accès et le maintien des personnes à mobilité réduite (handicapés) dans le système éducatif. En outre, les données seront désagrégées selon le genre en vue d’une meilleure compréhension du niveau d’implication des hommes et des femmes dans les activités de mobilisation sociale en vue d’une appropriation communautaire des ouvrages. Le niveau d’engagement des Associations de Parents d’élèves (APE), des Comité de Gestion de l’Ecole (CGE), des groupements féminins et des autres organisations locales sera étudié en relation avec la mobilisation sociale des acteurs communautaires. Enfin, les résultats seront désagrégés d’un point de vue géographique pour saisir la répartition des activités FARA en milieu urbain et dans les zones rurales. 8. DESIGN DE L’EVALUATION ET METHODOLOGIES L’évaluation sera conduite à travers une combinaison d’approches quantitatives et qualitatives pour une meilleure compréhension des différentes variables étudiées. Pour chacune de ces deux approches, toutes les précautions méthodologiques seront prises pour la validité et la fiabilité des données et la pertinence des conclusions et recommandations de l’évaluation. La démarche quantitative sera privilégiée pour les questions sur les résultats de l’activité, notamment l’accès et la rétention des élèves au cycle d’enseignement moyen. Quant à l’approche qualitative, elle mettra l’accent sur la saisie des perceptions et les visions endogènes des différents acteurs sur les réalisations du projet, la logique des acteurs et les résultats obtenus dans le cadre du FARA. Les acteurs ciblés pour cette approche sont principalement :  des élèves des collèges bénéficiaires ;  les responsables de la Direction des Constructions Scolaires (DCS), de la Direction de l’Enseignement Moyen Secondaire Général (DEMSG), de la Direction de Administration Générale et de l’Equipement (DAGE), de la Direction de la Planification et de la Réforme de l’Education (DPRE),  Le Ministère de l’Economie et des Finances (MEF),  les inspecteurs d’enseignement des localités visitées,  les enseignants et les chefs (principaux) des établissements cibles ;  les membres des Associations des parents d’élève (APE) des collèges visités ;  les membres des Comités de Gestion de l’école (CGE) des collèges visités ;  des personnes-ressources issues des communautés bénéficiaires (leaders religieux et coutumiers, responsables des groupements de femmes, de jeunes et des personnes du troisième âge, etc.) ;  les autorités administratives à l’échelle départementale et communale ;  Les collectivités locales à l’échelle départementale et communale. USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 50 Approche Quantitative La démarche quantitative sera privilégiée pour les aspects ci-après :  L’accès des filles et des garçons au cycle d’enseignement moyen ;  Le maintien des filles et garçons au cycle d’enseignement moyen ;  La transition du Collège au Secondaire pour les filles et garçons ; Ces différents aspects seront analysés de façon longitudinale et en comparaison avec d’autres collèges n’ayant pas bénéficié de l’appui du FARA au sein d’une même Inspection d’Académie (IA). Ils seront également analysés en rapport avec les statistiques au plan national. En se référant aux données existantes au niveau du MEN, la collecte des données quantitatives se fera principalement au moyen d’un questionnaire et ciblera un échantillon suffisamment représentatif des collèges construits entre janvier 2007 et décembre 2015 sur financement de l’USAID. A ce titre, un échantillon de 23 collèges et 15 points d’eau sera constitué représentant, respectivement, la moitié des 46 collèges construits et des 30 points d’eau réalisés. Les 4 collèges construits dans la région de Kolda, 3 dans la région de Sedhiou et 2 dans la région de Tambacounda feront partie de l’échantillon. Le reste des collèges sera choisi de manière aléatoire dans les autres régions d’intervention. L’échantillonnage ciblera tous les niveaux d’intervention de l’Activité FARA (Urbain/rural, Régions/Départements/Communes). Approche Qualitative L’objectif de l’approche qualitative est de comprendre le vécu de l’Activité FARA par les différents acteurs précités et de cerner leurs perceptions sur les conditions d’exécution de l’Accord et les changements constatés. Il est attendu des entretiens qualitatifs feront ressortir des informations substantielles pour une meilleure compréhension des réponses fournies par l’approche quantitative. Les aspects qualitatifs seront collectés au moyen de guides de discussions administrés aux acteurs clés susmentionnés, à travers des interviews semi-structurées et des focus groups pour avoir leurs perceptions sur les modalités de mise en œuvre du FARA et ses résultats.  Au sein de chaque communauté de l’échantillon abritant un collège ou un point d’eau construit dans le cadre du FARA, deux focus groups seront organisés, avec un accent sur les parents dont les enfants ont bénéficié de l’activité, les membres des APEs, et CGEs : o un focus regroupant les hommes; o un autre réunissant les femmes.  A l’échelle des collèges faisant partie de l’échantillon, il sera animé : o Un focus group réunissant les enseignants et le personnel administratif ; o Un focus group avec les élèves de sexe féminin o Un focus group avec les élèves de sexe masculin Les échanges avec les élèves permettront de connaitre leurs points de vue sur les réalisations du FARA et ainsi que leurs perceptions de l’incidence de l’activité FARA sur l’accès et la rétention. Les interviews semi-structurées seront conduites avec les acteurs précités dans chaque localité dont le choix se fera de façon raisonnée en prenant en compte leur connaissance de l’Activité FARA ou leur implication dans sa mise en œuvre. Pour chacun des collèges de l’échantillon, il sera organisé entre 5 et 10 entretiens par catégorie d’acteurs. Au niveau des institutions partenaires (DPRE, DCS, DEMSG, DAGE, MEF), les entretiens seront conduits avec les responsables et avec les personnes faisant office de points focaux de l’Activité FARA. USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 51 Des entretiens approfondis seront également menés avec l’ancien consultant recruté par l’USAID. La démarche méthodologique comprend les principales étapes suivantes :  Analyse de la documentation du projet (lettres d’exécution, rapports d’avancement, etc.). A ce titre, la « Matrice de revue documentaire et d’identification des insuffisances » a été utilisée pour passer en revue les documents et en faire le point.  Rencontres préliminaires avec les acteurs clés du Ministère de l’Education Nationales (MEN) impliqués dans l’exécution et le suivi du Projet afin d’identifier leurs intérêts et motivations pour l’évaluation. Il s’agit principalement de la Direction des Constructions Scolaires (DCS), la Direction de la Planification et de la Réforme de l’Education (DPRE) et de la Direction de l’Enseignement Moyen Secondaire Général (DEMSG).  Etablissement d’un échantillon de 23 collèges et 15 points d’eau.  Dans la composition de l’échantillon, tous les collèges construits dans les régions de Kolda, Sédhiou et Tambacounda seront retenus.  Organisation d’une réunion de planification de l’évaluation au cours de laquelle : o le Cadre de l’évaluation sera présenté et discuté pour son opérationnalisation ; o les standards de qualités du MEP en évaluation seront présentés et discutés ; o le plan de travail pour l’évaluation sera arrêté ; o les aspects logistiques et pratiques nécessaires à la conduite de la mission seront passés en revue ; o les rôles et responsabilités de chaque membre de l’équipe d’évaluation seront définis.  Identification des personnes-ressources pour les entretiens qualitatifs sur la base des rapports exploités et des premiers entretiens sur le terrain. En effet, la technique de l’«effet boule de neige », usitée dans les démarches qualitatives, sera utilisée pour repérer d’autres porteurs d’informations à interviewer. Elle consistera à demander aux acteurs déjà interrogés s’ils peuvent suggérer des personnes à rencontrer pour les besoins de l’évaluation.  Elaboration du questionnaire et des guides de discussion qui seront testés avant leur application à une large échelle ;  Conduite des entretiens avec les différentes parties prenantes ;  Visite des collèges et points d’eau faisant partie de l’échantillon ;  Analyse de données et interprétation des résultats de l’évaluation ;  Synthèse et présentation des résultats, conclusions et recommandations de l’évaluation ;  Rédaction du rapport préliminaire;  Finalisation du rapport de l’évaluation sur la base des observations de MEP Sénégal, de l’USAID/Sénégal, du Ministère de l’Education Nationale (MEN) et de l’atelier de partage des résultats. L’approche méthodologique sera participative et veillera à une triangulation des données pour s’assurer davantage de leur fiabilité. A ce titre, la matrice des réponses (Voir Annexe V), qui est un outil qui permet d’identifier et d’associer à chaque type de données spécifiques collectées une méthode et des techniques d’analyse, sera d’un apport considérable pour préciser les sources de réponse à chacune des principales questions d’évaluation. Il reste entendu que le MEN et l’USAID seront impliqués dans toute la démarche. USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 52 10. ANALYSE DES DONNEES Les données statistiques de l’évaluation couvriront les résultats de l’activité sur l’accès et le maintien des élèves, en tenant compte des nouveaux entrants24. Elles seront traitées à l’aide de Sphinx et SSPS et un dispositif performant sera mis en place en amont de l’enquête (questionnaire opératoire, formation des enquêteurs, sensibilisation des personnes à enquêter, etc.), pendant la phase de terrain (supervision des enquêteurs) et après la collecte des données (apurement et redressement) pour assurer la fiabilité des informations recueillies. Un traitement statistique sera effectué pour une meilleure compréhension des données disponibles à la DPRE sur la carte scolaire et les résultats des élèves. Les données qualitatives feront l’objet d’une analyse de contenu pour identifier dans les propos des personnes-ressources les récurrences en rapport avec les variables étudiées. Par la suite, les thèmes récurrents seront codés et quantifiés à l’aide de Sphinx. Les propos significatifs seront intégrés in extenso dans le rapport d’évaluation pour mieux étayer certains arguments. Pour la fiabilité des données, l’analyse s’appuiera sur le principe de la triangulation des sources d’information. A ce titre, la matrice des réponses sera d’un apport considérable. Avant le début de la rédaction du rapport de l’évaluation, l’équipe fera la synthèse des résultats, conclusions et recommandations, en utilisant la matrice prévue à cet effet. 11. FORCES ET LIMITES DE L’APPROCHE D’EVALUATION L’implication des services du Ministère de l’Education Nationale (DPRE, DAGE, DEMSG et DCS) dans la planification de l’évaluation constitue une des forces de celle-ci. Les discussions préalables avec ces acteurs ont permis de prendre en compte leurs intérêts et priorités dans les TDR de l’évaluation. L’existence du rapport d’évaluation à mi-parcours de 2012 assorti de recommandations constitue une opportunité pour l’évaluation envisagée. Il offre l’occasion de s’interroger sur le niveau de mise en œuvre des recommandations. Des questions spécifiques seront intégrées dans les outils (questionnaires et guides de discussions) pour mieux cerner la prise en compte des recommandations du rapport d’évaluation et leurs incidences dans la suite des activités du FARA. Parmi les limites, on peut citer les suivantes : (1) La diversité des acteurs institutionnels et surtout les changements intervenus à la tête des structures aux différentes phases d’exécution constituent de réels défis pour accéder à la mémoire institutionnelle de l’Activité FARA. Ce qui pourrait compromettre l’accès à certaines informations. Pour juguler cette situation, la mission pourrait être amenée à rencontrer les anciens responsables des structures pour avoir leur vécu de l’Activité FARA. (2) Le manque de données de base qui permettraient de cerner l’accroissement (ou non) de l’accès de manière fiable, en vue d’orienter le choix de recommandations pertinentes. L’influence des résultats de l’évaluation sur la stratégie du gouvernement en matière de construction d’établissements scolaires serait un autre défi. A ce titre, MEP/Sénégal recommandera les actions de partage des résultats de l’évaluation et un dispositif de suivi de l’application des recommandations. 24 Nouvel entrant : au regard des objectifs de la stratégie, un nouvel entrant est considéré comme un apprenant qui accède pour la première fois à la classe d’entrée/au niveau d’éducation ciblée par une activité. Les nouveaux entrants peuvent être identifiés comme des apprenants n’ayant pas redoublé et accédant au niveau/à l’année d’entrée. Si une activité cible le début de l’école primaire, donc le niveau d’entrée est généralement le niveau 1. Si l’activité cible les niveaux/années supérieurs, le niveau/année d’entrée est le premier niveau ciblé par l’activité USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 53 12. LIVRABLES DE L’ÉVALUATION Les principaux livrables de l’évaluation sont : 1) Compte rendu de la réunion de cadrage (In-brief) ; 2) Le cadre méthodologique de la mise en œuvre de l’évaluation incluant les questions clefs, les outils de collecte et d’analyse des données, le plan d’échantillonnage, le calendrier d’exécution de la mission et la composition de l’équipe ; 3) Données collectées (Questionnaires remplis, éléments sonores, transcription des entretiens qualitatifs) ; 4) Compte rendu de la réunion à mi-parcours (Mid-brief) ; 5) La synthèse des résultats, conclusions et recommandations de l’évaluation ; 6) Présentation PowerPoint (en anglais et français) ; 7) Le rapport provisoire de l’évaluation (en anglais et français) ; 8) Le rapport final de l’évaluation (en anglais et français) ; 9) Le rapport de la rencontre de partage des résultats ; 10) Compte rendu de la réunion finale (De-brief) 13. COMPOSITION DE L’EQUIPE D’ÉVALUATION L’évaluation sera conduite par Mme Karla Giuliano Sarr, une experte internationale spécialisée dans les études sur le système éducatif. Mme Sarr qui a une connaissance approfondie de l’Afrique de l’Ouest parle couramment le français et l’anglais. Elle comprend parfaitement la complexité des systèmes éducatifs francophones. Mme Sarr fera office de chef d’équipe et aura en charge la rédaction en anglais du rapport de l’évaluation. Pour mener à bien l’évaluation, un contrat sera signé entre MEP Sénégal et le Groupe d’Etudes, de Recherches et d’Appui au Développement (GERAD)25 et dont la mission sera principalement de travailler avec Mme Sarr dans l’élaboration détaillée du cadre méthodologique de l’évaluation, la définition précise de l’échantillon, la conception des outils de collecte et d’analyse des données, la collecte et l’analyse des résultats et la rédaction du rapport de synthèse des résultats, des conclusions et recommandations. Cette structure devra mettre en place une équipe d’experts appuyés par des enquêteurs expérimentés pour le recueil des données de terrain. MEP-Sénégal apportera son expertise dans toutes les étapes du processus d’évaluation. A cet effet, M. Moussa SY, Spécialiste en évaluation et Mame Aissatou Mbaye, Associée en Suivi-évaluation, participeront aux activités, notamment la revue documentaire, l’élaboration du cadre méthodologique et la collecte et l’analyse des données. De façon spécifique, M. Moussa Sy, veillera à la qualité des outils de collecte des données élaborés par l’équipe, assurera le contrôle de la fiabilité et de la validité des résultats et vérifiera la conformité du rapport final avec le canevas annexé au présent document. Mame Aïssatou Mbaye appuiera l’équipe dans la prise de notes, l’enregistrement audio et la documentation des travaux de terrain. Tous les membres de l’équipe vont fournir une déclaration signée attestant qu’ils ne sont pas en situation de conflit d’intérêt. Le tableau de la page suivante présente les principales activités et le temps d’intervention du Chef d’équipe. 25 Le GERAD a été retenu suite à un appel d’offres restreint ayant ciblé les partenaires d’exécution de MEP/Sénégal USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 54 USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 55 DATES ACTIVITES TEMPS D’INTERVENTION DU CHEF D’EQUIPE 20 mai 2016 Avis d’appel d’offres pour le recrutement de la structure contractante 03 Juin 2016 Date limite pour le dépôt des propositions techniques et financières 06-12 juin 2016 Dépouillement des offres, choix de la structure contractante et signature du contrat de prestation avec la structure retenue 13- 18 Juin 2016 Revue et analyse documentaire 4 19 juin 2016 Arrivée à Dakar du Consultant international 1 20 juin 2016 Rencontre de cadrage avec USAID, MEN 1 21-22-23 Juin 2016 Réunion technique de Planification o Définition de l’échantillon o Élaboration de la méthodologie (approches, outils et manuel d’enquête) o Elaboration du plan de travail 3 24 juin 2016 Réunion de partage et de validation du plan de travail et du cadre méthodologique avec l’équipe Education de l’USAID et MEN 1 25-26 juin 2016 Test et révision des outils et du manuel d’enquête 1 27-28 Juin 2016 Déploiement des équipes (experts, enquêteurs, superviseurs) sur le terrain 1 29 juin – 9 Juillet 2016 Collecte des données de terrain 12 11-15 juillet 2016 Saisie, Analyse et interprétation des résultats 6 18 juillet 2016 Présentation initiale de la synthèse des résultats, conclusions et recommandations de l’évaluation 1 19 juillet 2016 Départ de Dakar du Consultant international 1 20 - 27 Juillet 2016 Rédaction du rapport provisoire 8 1 aout 2016 Soumission du rapport provisoire à l’USAID 2-11 août 2016 Analyse du rapport provisoire par l’USAID 12-18 août 2016 Révision du rapport sur la base des observations formulées et mise en page 3 19 Août 2016 Soumission du rapport final 43 14. PARTICIPATION DE L’USAID ET PARTENAIRES MEP Sénégal travaillera en étroite collaboration avec l’équipe éducation de l’USAID/Sénégal, particulièrement Monsieur Alpha Ibrahima Bah, et le Ministère de l’Education Nationale, notamment la Direction des Constructions Scolaires (DCS), la Direction de la Planification et de la réforme de l’Education (DPRE) et la Direction de l’Enseignement Moyen Secondaire Général (DEMSG). Ils s’impliqueront dans l’analyse du Cadre de l’évaluation, des outils de collecte de données et des USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 56 différents livrables pour en affiner le contenu. Ils contribueront également à la validation du rapport de l’évaluation. 15. PLANIFICATION ET LOGISTIQUES La mission d’évaluation se déroulera sur la période du 26 avril au 10 Août 2016. MEP Sénégal, en étroite collaboration avec l’USAID, prendra toutes les dispositions sur les plans technique, logistique et organisationnel pour une exécution diligente de l’évaluation. La mission prévoit de rencontrer l’USAID le 09 juin pour le cadrage technique de l’évaluation. La présentation des résultats préliminaires, des conclusions et recommandations de l’évaluation est envisagée pour le 6 juillet 2016. Il faut la participation de l’équipe éducation de l’USAID et les directions du MEN pour cette présentation. 16. EXIGENCES POUR LE REPORTING Le rapport final de l’évaluation sera disponible en Français et Anglais et comportera un maximum de 35 pages sans compter les annexes. Il devra respecter le canevas des rapports d’évaluation et l’identité visuelle de l’USAID. Il sera envoyé au Bureau éducation de l’USAID qui se chargera du partage avec le Ministère de l’Education Nationale (MEN). Il sera intégré dans USAID/DEC. MEP Sénégal recommandera à l’USAID les possibilités d’un large partage du rapport, notamment avec les acteurs à l’échelle nationale et internationale travaillant sur la problématique de l’accès à l’éducation. 17. DOCUMENTS DE REFERENCE ANNEXES Verifier si tous les éléments ci-dessous sont en annexe X Calendrier d’exécution X Cadre de résultats, conclusions et recommandations avant la rédaction du rapport X Matrice des réponses Budget X Curriculum Vitae Déclaration de conflit d’intérêt X Politique d’evaluation de l’USAID X Structure du Rapport d’évaluation 18. AUTORISATIONS The undersigned hereby authorizes the following items (checked below) for the Statement of Work (SOW) described above: X Completion of the SOW, as described above; X SOW staffing, as described above; X Concurrence with Contracting Officer’s Travel Approval for the Consultant(s), requested above (if received prior to review). [COR to either sign below or indicate approval in a return email] __________________________________________ ________________ USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 57 Contracting Officer’s Representative (COR) Date Roy Geiser, or designate __________________________________________ ________________ General Development Office Director Date Michelle Barrett, or designate USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 58 ANNEXE I: CALENDRIER D’EXÉCUTION Construction des collèges (FARA) Mai Juin Juillet Aout Calendrier Etapes Semaine 1 2 3 4 1 2 3 4 5 1 2 3 4 5 1 2 3 4 Partage du cadre de travail avec MEN par USAID et MEP Senegal Avis de recrutement de la structure Dépouillement des offres, choix de la structure et signature du contrat Revue et analyse documentaire Arrivée de la consultante internationale Technical Planning Meeting Réunion de validation par USAID des outils et plans de travail Collecte des données de terrain Saisie, Analyse et interprétation des résultats Présentation initiale de la synthèse des résultats, conclusions et recommandations de l’évaluation Retour de la Consultante Internationale Redaction du rapport provisoire Soumission du rapport provisoire Analyse rapport par USAID Révision du rapport Soumission du rapport final USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 59 ANNEXE II: CADRE DE RESULTATS, CONCLUSIONS ET RECOMMANDATIONS AVANT LA REDACTION DU RAPPORT Evaluation Question Findings Conclusions Recommendations 1 2 3 4 5 6 7 8 9 10 USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 60 ANNEXE III: MATRICE DES REPONSES USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 61 ANNEXE IV : BUDGET 62 ANNEXE V : CURRICULUM VITAE Dr. Karla Giuliano Sarr – Team Leader Summary Dr. Sarr has over ten years’ experience working on evaluations and research in West and Central African education systems. Currently, she is conducting a partial baseline study of teachers’ beliefs and attitudes towards mathematics in Ghana as part of a USAID education monitoring and evaluation activity. The study includes both quantitative (descriptive statistics and composite scores) and qualitative (focus groups, interviews, surveys, task activity) data. The baseline report will be used to inform the design of the education pilot project to follow. Prior to her work in Ghana, Dr. Sarr conducted a study for the United Nations Girls Education program in Guinea. The study used both quantitative and qualitative methods, and aimed to provide insights into program effectiveness and its impact on the community. Dr. Sarr has worked and studied in Senegal at various times since 2000, most consequentially between June 2006 and August 2008. Dr. Sarr holds an Ed.D. in Educational Research, Policy and Administration from the University of Massachusetts, Center for International Education. For her dissertation on Integrating Cultures within Formal Schooling: Exploring Opportunities for Cultural Relevancy in Peri-Urban Senegal, she conducted interviews and focus groups in both Wolof and French. Education Ed.D., International Education, University of Massachusetts, Amherst, MA, 2015 M.Ed, International Education, University of Massachusetts, Amherst, MA, 2010 B.S., African Studies, Georgetown University, Washington, D.C., 2002 Experience Evaluations International Qualitative Research Specialist, USAID/Ghana Partnership for Education; School-to￾School International, CA, USA; March 2016 - Present  Conduct data analysis for partial mixed-methods baseline study on teachers’ beliefs and attitudes.  Lead report writing including elaboration of lessons learned.  Oversaw the piloting and finalization of tools and initial data collection with Ghanaian math education expert.  Developed training plan and trained 12 data collectors dispersed to two districts. Consultant, Whole Child Model; School-to-School International, CA, USA; May – Nov 2015  Designed qualitative research study of the impact of Girls Education Project in three schools funded by the United Nations Girls Education Initiative (UNGEI).  Included components evaluating impact of program on students and various stakeholders.  Trained and provided ongoing support to data collection team while in the field.  Integrated quantitative findings into final mixed-methods report in French including lessons learned. 63 Consultant, DYS Professional Development Evaluation of Effective Practice Project; Collaborative for Educational Services, Massachusetts, USA; April – July 2013  Designed small-scale evaluation of professional development practices in juvenile justice education programs.  Implemented data collection through focus groups, observation, interviews and document review.  Analyzed data and synthesized findings via written report and oral presentation.  Led research team and process. Consultant, DYS Initiative; Commonwealth Corporation, Massachusetts, USA; May – Sept 2011  Developed tools, recorded data and produced reports for Education Quality Assurance ongoing evaluation.  Revised and implemented 2011 and 2012 employer surveys, analyzed data and produced final reports.  Authored two literature reviews pertinent to juvenile justice settings on mentoring and transition.  Performed technical assistance to Bridging the Opportunity Gap initiative youth community transition program. Consultant; The Grandmother Project, Senegal; May 2009 – Jan 2010  Designed classroom game to promote discussion of cultural values and intergenerational relationships.  Implemented game during on-site testing period in conjunction with local school personnel and student leaders.  Provided training to teachers and administers.  Evaluated effectiveness of game in promoting discussion of cultural issues and tensions. Teaching and Education Positions Assistant Professor, International Education; SIT Graduate Institute, Vermont, USA; Aug 2015 – Present  Deliver graduate level coursework including Language, Social Justice and Education in Multilingual Contexts; Theory and Practice of International Education; Theory, Practice and Policy in International Education; Advanced Concepts (Evaluation module); and International Education Design and Delivery.  Teach in both on-campus and online programs.  Advise both on-campus students and students in reflective practice practicum phase.  Serve as member of the Institutional Review Board. Study Abroad Advisor – France and West Africa; University of Massachusetts International Programs Office, Massachusetts, USA; Aug 2008 – May 2014  Led office in designing student surveys and performing data analysis to inform programming models.  Designed and delivered intercultural training modules informed by student feedback.  Conceptualized and implemented full-day conference for more than 400 participants with over 30 staff.  Advised students and managed application process for travel abroad to sites in France and West Africa. Teacher Trainer Specialist, Learning Initiatives for Rural Education Project; University of Massachusetts CIE, Massachusetts, USA; Sept 2008 – June 2009  Designed and implemented international training conference in Senegal with the World Bank – May 2009.  Trained 31 teachers, 10 education officers and 30 trainers from West African sub-region. 64  Copy-edited and translated teaching and training documents in English and French. Cultural Exchange Coordinator; Africa Consultants International/Baobag Center, Senegal; June 2006 – Aug 2008  Led department in developing and administering 15-20 client-specific study abroad programs annually.  Managed a cumulative program budget of over $500,000 yearly.  Launched new academic courses requiring the identification and engagement of local partner institutions.  Designed and facilitated cross-cultural training sessions for students and personnel. English Teacher (TEFL) and Community Development Worker; Peace Corps, Gabon; June 2003 – Aug 2005  Taught in two different school systems at middle school grade levels and directed after-school English club.  Provided English classes to 40 national park rangers and 20 guides in inaugural training program.  Co-Authored English Guide and Guards Training Manual of the National Forestry School.  Improved skills and competencies of local counterparts through seminars, team-teaching and collaboration. Community Affairs Specialist; Africa Center for Strategic Studies, Washington, D.C., USA; Jan 2002 – Apr 2003  Supported and executed seminars for international senior military and civilian officials in U.S. and Africa.  Designed and launched Community Program: an alumni network of participants, guest speakers and staff.  Oversaw Africa Center Community events, managed database records and drafted follow-up reports. Countries of Work Experience Senegal, Ghana, Gabon Languages English (native), French (fluent), Wolof (advanced), Spanish (conversational) Publications Sarr, K. G. (2016). Vignette 23: Exquisite care in translating and transcribing. In C. Marshall & G. B. Rossman. Designing Qualitative Research (6th ed.), p. 212. Thousand Oaks, CA: Sage Publications Inc. Sarr, K. G. (2013). “We lost our culture with civilization”: Community perceptions of Indigenous knowledge and education in Senegal. In Benson, C. & Kosonen, K. (Eds.). Language Issues in Comparative Education: Inclusive teaching and learning in non-dominant languages and cultures. Boston, Massachusetts: Sense Publishers. Sarr, K. G., & Mosselson, J. (2010). Issues in teaching refugees in U.S. Schools. In C. Faltis & G. Valdés (Eds.) Education, Immigrant Students, Refugee Students, and English Learners. New York: Teachers College Press. 65 Sarr, K. G. & Lawrence, R. B. (2013). DYS Education Initiative: Supporting Professional Teaching: A Close-Look at Teacher and Instructional Coach Perspectives on Professional Development. Collaborative for Educational Services. Sarr, K. G. (2012). Updated Literature Review Specific to Mentoring for Youth involved in the Juvenile Justice System. Commonwealth Corporation. Sarr, K. G. with Smith, C. (2010). Towards Independence and Self-Sufficiency: Key Components of Employment Transition Programs for Youth Involved in the Juvenile Justice System. Commonwealth Corporation. Learning Initiatives in Rural Education. (May 2009). The Gambia’s Teachers’ Handbook on Multi￾Grade Teaching. Ministry of Basic and Secondary Education in Collaboration with the Learning Initiatives in Rural Education. World Bank Contract #7145279. (Also available in French). Memberships in Professional Societies and Awards American Evaluation Association (AEA) Comparative and International Education Society (CIES) 66 ANNEX II: EVALUATION METHODS AND LIMITATIONS During the data collection phase, the evaluation team was composed of an international consultant Team Leader, Karla Giuliano Sarr, hired by Senegal MEP, 3 local research experts, 7 supervisors and 9 researchers. Except for the international consultant, all other team members were external consultants from the local company GERAD (Groupe de recherche et d’appui au développement). Sub-teams composed of an expert, a supervisor and researchers worked together to collect data in various regions simultaneously. Data collection took place between June 29 and July 15th. Regional data collection took place in two phases working around the Korité (Eid Al Fitr) holiday (phase 1: June 29-July 4; phase 2: July 10-15) with data collection with government officials occurring throughout. In addition, the MEP Chief of Party conducted supplementary interviews with Senegalese government and USAID officials in order to respond to USAID’s request for an additional evaluation question (#3) following the presentation of preliminary results. These interviews took place between August 2 and August 9, 2016. Data Collection Methods and Source The evaluation used high quality, replicable social science methods to generate and present empirical data. Findings from the study informed conclusions and subsequently, recommendations which are actionable, practical and directed at specific parties. The evaluation team approached this study using a combination of quantitative and qualitative methods, with greater emphasis on the latter. Methods include the collection of education statistics at the school, local, regional and national level. Moreover, the team conducted semi￾structured interviews and focus groups with a wide range of stakeholders and beneficiaries. These activities allow for in-depth investigations into the experiences and perspectives of the individuals involved in both the implementation of the activities as well as those who have benefitted from the new infrastructures. The evaluation team visited a sample of half (23) of the schools constructed through FARA to collect enrollment and retention statistics from 2007 to present. At these schools, the team spoke with the school principal conducted observations of the school buildings and water points in order to gain perspective about the maintenance of these structures as well as any changes that may have been made subsequent to the preliminary and/or official receiving of these schools. At 12 of these 23 schools, the team facilitated several additional qualitative activities that allowed for a deeper understanding of the school’s relationship to the community as well as the collaborations between the various administrating actors. In addition to visiting the schools, the evaluation team met with key informants at the ministerial level and with USAID/Senegal in order to obtain a fuller perspective on program operations as well as the possible results of the FARA program. This evaluation paid special attention to how these newly constructed middle schools and water sources may respond to the differing needs of girls and boys. To the extent possible, the evaluation team disaggregated data by gender. The methods annex outlines the evaluation methods in greater detail along with the intended data sources, all the while linking the methods and sources with the appropriate research question. The following data collection methods will be used during this evaluation:  Desk Review: identification and review of relevant technical reports and background documents (see SOW section 4).  Qualitative methods: key informant interviews (semi-structured), focus groups (semi￾structured), structured observation, taking pictures  Quantitative methods: completion of school statistics sheets, school construction summary sheets 67 Copies of the tools for data collection are included in the Annexes. Note that because the majority of interviews and focus groups were semi-structured the tools serve as guides and the evaluation team adapted tools as required to the needs of participants once in the field. This included adjusting the phrasing of questions asked to make them more accessible as well as translating question items into local languages. Document review for this study included a close reading of the following resources: Table 1: Documents Providing Background to this Evaluation Titre du document Dates Lettre d’exécution No. 685-013-04 19 janvier 2007 Lettre d’exécution No 685-013-06 28 octobre 2009 Lettre d’exécution No 685-013-07 1 er avril 2010 Lettre d’exécution No 685-013-05 23 août 2013 Rapports d’inspection des écoles (Seby Ponty, Zac Mbao, Darou Khoudos, Simbandi Balante Mars 2015 Procès-verbal de pré-réception des collègues matching de Simbandi Balante, Sebi Ponty, Zac Mbao, Darou Khoudoss Janvier 2015 Liste des collèges financés par l’USAID Mars 2016 Liste des points d’eau à rembourser par l’USAID Janvier 2016 Rapport d’évaluation à mi-parcours du projet, Novembre 2012 Recensement Général de la Population et de l’Habitat, de l’Agriculture et de l’Elevage (RGPHAE 2013). Chapitre III: alphabétisation, scolarisation, niveau d’instruction et formation professionnelle. Agence Nationale de la Statistique et de la Démographie Septembre 2014 Etat des lieux de l’éducation de base au Sénégal de 2014 Mai 2014 The identification of potential data collection methods and sources appropriate for each evaluation question is presented in Table 2 below. Table 2: Data Collection Methods and Sources Evaluation Question Data Collection Method Data Sources 1. What evidence is there that the FARA mechanism is pertinent in building the capacity of the education system? Key informant interviews - Ministry of Education officials (DCS, DPRE, DEMSG, DAGE) - Ministry of Finance official - Inspectors (IA, IEF) - School principals - USAID/Education Focus group (semi-structured) - PTA Board (Bureau de l’APE) & School Management Committee (CGE) Statistics sheet from 2006 to 2016 -Ministry of Education (MEN) -Inspectorates (IA, IEF) - School principals 68 Evaluation Question Data Collection Method Data Sources FARA Construction Summary Sheet - Principal - Observations at schools by evaluation team and discussions with APE and CGE. 2. To what extent has FARA promoted collaboration among stakeholders? Key informant interviews (semi￾structured) - Ministry of Education officials (DCS, DPRE, DEMSG) - Ministry of Finance official - USAID/Education - Inspectors (IA, IEF) - School principals - Local authorities (Mayors, Prefets,) Focus groups (semi-structured) - Teachers - PTA Board (Bureau de l’APE) & School Management Committee (CGE) Statistics sheet -Ministry of Education -Inspectorates (IA, IEF) - School principals 3. In what capacity have community mobilization efforts that were implemented under FARA been favorable to communities benefitting from new infrastructure? What are the strengths and limits of the community ownership approach in terms of managing buildings and infrastructure? Key informant interviews (semi￾structured) - School principals - Community leaders Focus group (semi-structured) - PTA Board (Bureau de l’APE) & School Management Committee (CGE) - Community leaders (CBOs, ASC; GPF) - Ministry of Education officials (DCS, DPRE, DEMSG, DAGE) - Ministry of Finance official - L’USAID (Education Development Specialist) Statistics sheet -Ministry of Education -Inspectorates (IA, IEF) - School principals FARA Construction Summary Sheet - Principal - Observations at schools by evaluation team 4. To what degree has the FARA Program favored access to and continued attendance at middle schools for boys and girls? What has worked well? What has not worked? Focus groups - Students (girls and boys separately) - Community leaders - PTA Board (Bureau de l’APE) & School Management Committee (CGE) Key informant interviews (semi￾structured) - Inspectors (IA, IEF) - School principals - Ministry of Education officials (DCS, DPRE, DEMSG) - Ministry of Finance official 69 Sampling Strategy The evaluation collected information from half of the 46 schools involved in the FARA Construction Program, ie., 23 schools. The schools were been identified according to several criteria, following a combination of purposeful and random sampling strategies. The first criterion, as indicated by USAID, was to include all of the schools in Kolda and Sedhiou due to these regions’ designation as conflict affected areas and Tambacounda as the latter is considered to be a conflict-prone area. Nine FARA schools exist in these regions, which allowed investigation of 14 middle schools in the remaining eight regions of the country participating in the FARA Construction Program. These remaining 14 schools were identified randomly accounting for a proportional number of schools from each phase of the project, schools in urban as well as rural areas and schools across the 11 regions of Senegal where the FARA program was implemented. Because the evaluation questions seek to understand complex issues such as project relevance, collaboration, community mobilization and access, it was necessary to utilize qualitative methods such as key informant interviews and focus groups. The evaluation team identified 12 schools among the 23 for a more in-depth investigation. The 12 schools make up more than 25% of the 46 schools included in the FARA Construction Program. In choosing the 12 schools, the evaluation team took into account the following: - Inclusion of all schools in Kolda and Sedhiou (7 schools) - Phase of school construction and - Diversity of regional areas. Lastly, the evaluation team paid attention to including a high number of schools where USAID water sources were also constructed. Of the 23 schools identified for the evaluation sample, 10 of them have water sources implemented through the FARA activity. Of these 10 schools, 7 were included in the qualitative component. Table # displays a list of the middle schools identified for this evaluation and their characteristics. Table #: Schools participating in evaluation School USAID Water source Phase Region Evaluation Method Location Bouna Kane II Y 3 Kolda Qualitative Urban Boynadji N 1 Matam Quantitative Rural CEM III- Sedhiou N 3 Sedhiou Qualitative Urban CEMIII Ndiaye Ndiaye N 1 Fatick Quantitative Urban Hilele Y 3 Kolda Qualitative Urban Kebemer III N 4 Louga Quantitative Rural Keur Massar N 3 Dakar Quantitative Urban Koussy Y 4 Sedhiou Qualitative Rural 70 Mbao Kamb N 2 Dakar Quantitative Urban Mboss N 1 Kaffrine Quantitative Rural Merina Dakhar Y 2 Thies Qualitative Rural Ndiaye Ndioungue Mberess N 1 Saint Louis Qualitative Urban Ndindy Y 3 Diourbel Quantitative Rural Ndoga Babacar Y 4 Tambacounda Quantitative Rural Ndoumga Ouro Alpha Y 4 Matam Qualitative Rural Ngothie N 2 Fatick Qualitative Rural San Samba N 4 Sedhiou Qualitative Rural Seby Ponty Y 4 Dakar Qualitative Urban Simbandi Balante Y 4 Sedhiou Qualitative Rural Sinthiou Gangue N 3 Saint Louis Quantitative Rural Tamba Commune I Y 4 Tambacounda Quantitative Urban Teyel (Velingara) N 4 Kolda Qualitative Urban Zone Sud de Mbour N 2 Thies Quantitative Urban Limitations A number of limitations exist for the evaluation. The timeline was a significant issue in conducting the evaluation. The Koritè (Eid Al Fitr) holiday fell in the middle of data collection requiring a two-phase field strategy. In addition, school was ending for the summer, so many students and some teachers were unavailable for interviews and focus groups. With this in mind, we only recruited 3eme students (9th grade) for student focus groups because they were most likely to be at school preparing for the end-of￾cylce exam. Additionally, a misunderstanding of deadlines between MEP and local partner GERAD, as well as GERARD’s lack of familiarity with transcriptions and coding methods led to data processing delays. Project-wise, the FARA school construction project began in 2007 and evolved greatly during the project cycle. Due to personnel turnover many people with recollection of the original objectives of the 71 project were unavailable for interview. This issue was compounded by a lack of background documentation on the developmental hypothesis or results framework for this particular activity. Additionally most people at the local level were unaware of the funding of the project and unfamiliar with the name “FARA.” Because the Ministry instituted a matching project where the Government of Senegal matched USAID interventions and over time, the program simply became known as Matching within the concerned ministries. There was some related confusion around USAID-funded water points as they were installed at schools most in need resulting in USAID-funded water points at some Matching schools. This reduced the number of water points included in the sample. Because USAID prioritized inclusion of schools built in certain conflict-zone areas in the south (Kolda, Sedhiou and Tambacounda) as part of the sample, the spread of sample schools built across the four phases of FARA is uneven (most schools in the south were built under phases three and four). This means that the data collected many not be representative of the other 23 FARA schools that may not be in a conflict area nor built under the latest phase. Copies of the tools for data collection are included in the Annexes. Note that because the majority of interviews and focus groups will be semi-structured the tools serve as guides and the evaluation team adapted tools as required to the needs of participants once in the field. This may include adjusting the phrasing of questions asked to make them more accessible as well as translating question items into local languages. Data Analysis Methods Data analysis drew on findings guided by the evaluation questions. Qualitative data collected via the semi￾structured individual and group interviews was analyzed using thematic and content analysis. This analysis paid attention to themes and patterns that emerged from the data and coded responses according to those themes. The evaluation team used the NViVo computer programs to aid its analyses. In addition, the evaluation team identified relevant quotes and pictures related to findings. Moreover, analysis paid attention to case studies so as to identify promising practices. Quantitative data analysis was largely descriptive, focusing on student data and addressing issues of access and retention. The analysis also employed the Student’s t-test comparison of two means tool. 72 ANNEX III: DATA COLLECTION INSTRUMENTS The tools below can be found within the subsequent pages of this document :  Guide d’entretien – USAID  Guide d’entretien – MEF  Guide d’entretien – MEN : DPRE  Guide d’entretien – MEN : DEMSG  Guide d’entretien – MEN : DCS  Guide d’entretien – MEN : DAGE  Guide d’entretien – Bénéficiaires communautaires  Guide d’entretien – Focus Group Enfants  Guide d’entretien – L’IA et L’IEF  Guide d’entretien – Personnel du CEM  Guide d’entretien d’individus clés (Question #5) – Institutional Capacity Development  Checklist – Qualitative Site  Fiche – Etat du patrimoine  Fiche – Statistiques collège 73 PROJET DE GUIDE D’ENTRETIEN - USAID Principes de l’enquête / Le consentement verbal : Bonjour le GERAD, cabinet de consultance Sénégalais, effectue l’étude d’évaluation du FARA pour la construction de collèges pour le compte de l’USAID. La finalité est de collecter des informations afin de formuler des recommandations en vue de documenter la stratégie à mettre en œuvre pour de futurs programmes. Nous aimerions avoir votre aide dans ce domaine. Est-ce que vous voulez participer dans cette étude ? [Attendez la réponse de la personne. Si la personne semble à l’aise, passez directement au consentement verbal.] Votre nom ne sera pas mentionné dans le rapport fait sur cette enquête. Aussi, les résultats de l'enquête seront publiés sous la forme du texte narratif sans les noms des individus. Néanmoins, étant donné votre poste, il peut avoir des instances où il sera possible de vous identifier. Comme vous savez, le rapport final sera partagé avec l’USAID et le Ministère de l’Education National dans le but de renforcer l'implémentation des programmes. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer l'entretien. En fin de l’étude, l’enregistrement sera envoyé au niveau de l’USAID pour documenter l’activité. Votre nom ne sera pas lié à l’enregistrement. Mais si vous ne voulez pas participer, il n’y aura pas de problème. Une fois que nous commençons, si vous préférez ne pas répondre à une question, vous pouvez nous le dire. Est-ce que vous nous donnez votre consentement ? [Avez-vous obtenu le consentement verbal? ☐ ] Relevant evaluation questions : 1. What evidence is there that the FARA mechanism is pertinent in building the capacity of the education system? 2. To what extent has FARA promoted collaboration among stakeholders? I. Identification Nom et Prénom : Fonction : Durée d’occupation du poste : 74 Interview Themes and Probing Questions Experience with Activity:  How long have you been working with USAID Senegal? With the education team?  What have been your experiences with the FARA activity? (Construction? Equipment? Water sources?)  What has been your role within the FARA activity? How have you been involved? Strengths and weaknesses:  Would you qualify the activity as a success?  What are some of its strengths?  [Recall evaluation question: In what ways might the program have contributed to building the capacity of the educational system in Senegal? What evidence exists to support this opinion?]  What are some of its challenges? Collaboration and Accountability measures:  With which other stakeholders did you collaborate to implement the program? (MEN – which directorates – DCS? DEMGS? DAGE?; MEF?) What was the extent of that collaboration?  What were the procedures set in place for these collaborations?  What sort of records do you have from the activity? What records exist among other stakeholders?  What information did you inherit from your predecessors? Lessons learned – future application?  If the FARA activity continues in Senegal, what sort of revisions would you like to see made to the activity?  Do you think an activity like the FARA Construction project could take place in a different partnering country? Would you recommend it (based on the experience in Senegal)?  What would you consider to be the main lessons learned from this activity?  What do you hope to learn from this evaluation? How can this evaluation be useful for USAID?  What suggestions do you have for us in conducting this evaluation? Schools / Equipment / Water sources 75 PROJET DE GUIDE D’ENTRETIEN A L’INTENTION DE LA DIRECTION DE L’INVESTISSEMENT (MEF) Principes de l’enquête / Le consentement verbal : Bonjour le GERAD, cabinet de consultance Sénégalais, effectue l’étude d’évaluation du FARA pour la construction de collèges pour le compte de l’USAID. La finalité est de collecter des informations afin de formuler des recommandations en vue de documenter la stratégie à mettre en œuvre pour de futurs programmes. Nous aimerions avoir votre aide dans ce domaine. Est-ce que vous voulez participer dans cette étude ? [Attendez la réponse de la personne. Si la personne semble à l’aise, passez directement au consentement verbal.] Votre nom ne sera pas mentionné dans le rapport fait sur cette enquête. Les résultats de l'enquête seront publiés sous la forme du texte narratif sans identification des individus. Le rapport final sera partagé avec l’USAID et le Ministère de l’Education National dans le but de renforcer l'implémentation des programmes. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer l'entretien. En fin de l’étude, l’enregistrement sera envoyé au niveau de l’USAID pour documenter l’activité. Votre nom ne sera pas lié à l’enregistrement. Mais si vous ne voulez pas participer, il n’y aura pas de problème. Une fois que nous commençons, si vous préférez ne pas répondre à une question, vous pouvez nous le dire. Est-ce que vous nous donnez votre consentement ? [Avez-vous obtenu le consentement verbal? ☐ ] I. Identification Nom et Prénom : Fonction : Structure : Durée d’occupation du poste: Question d’introduction : 1. Quel est votre degré de satisfaction par rapport à la mise en œuvre du FARA ? Approfondir :  Quels ont été les points forts de ce programme à votre avis ?  Quels ont été les points faibles de ce programme à votre avis ? Thème 1 : Preuves de la pertinence de l’activité FARA dans le renforcement des capacités du système éducatif 2. Quelles sont vos appréciations sur la pertinence des modalités d’investissements du FARA pour le renforcement des capacités du système éducatif ? Approfondir :  Comment s’est faite votre implication ?  Quel a été votre rôle dans la mise en œuvre du FARA ? Quels sont les mécanismes du FARA?  Quels sont les principaux acquis des mécanismes d’exécution budgétaire ? 76  Avez-vous noté des contraintes dans ce mécanisme budgétaire ?  Quelles sont vos suggestions en vue d’améliorer la contribution du FARA au renforcement des capacités du système éducatif ? 3. Que pensez-vous de la stratégie d’intervention du FARA ? Approfondir :  Quelle est votre appréciation du système de remboursement forfaitaire ?  Si un nouveau programme FARA devait être mis en œuvre, quelles dispositions en matière d’exécution budgétaire suggérez-vous ?  Quelles sont vos recommandations et suggestions pour améliorer la stratégie ? 77 PROJET DE GUIDE D’ENTRETIEN A L’INTENTION DE LA DPRE Principes de l’enquête / Le consentement verbal : Bonjour le GERAD, cabinet de consultance Sénégalais, effectue l’étude d’évaluation du FARA pour la construction de collèges pour le compte de l’USAID. La finalité est de collecter des informations afin de formuler des recommandations en vue de documenter la stratégie à mettre en œuvre pour de futurs programmes. Nous aimerions avoir votre aide dans ce domaine. Est-ce que vous voulez participer dans cette étude ? [Attendez la réponse de la personne. Si la personne semble à l’aise, passez directement au consentement verbal.] Votre nom ne sera pas mentionné dans le rapport fait sur cette enquête. Les résultats de l'enquête seront publiés sous la forme du texte narratif sans identification des individus. Le rapport final sera partagé avec l’USAID et le Ministère de l’Education National dans le but de renforcer l'implémentation des programmes. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer l'entretien. En fin de l’étude, l’enregistrement sera envoyé au niveau de l’USAID pour documenter l’activité. Votre nom ne sera pas lié à l’enregistrement. Mais si vous ne voulez pas participer, il n’y aura pas de problème. Une fois que nous commençons, si vous préférez ne pas répondre à une question, vous pouvez nous le dire. Est-ce que vous nous donnez votre consentement ? [Avez-vous obtenu le consentement verbal? ☐ ] I. Identification Nom et Prénom : Fonction : Structure : Durée d’occupation du poste : Question d’introduction : 1. Quel est votre degré de satisfaction par rapport à la mise en œuvre du FARA ? Approfondir :  Quels ont été les points forts de ce programme à votre avis ?  Quels ont été les points faibles de ce programme à votre avis ? Thème 1 : Preuves de la pertinence de l’activité FARA dans le renforcement des capacités du système éducatif 2. Quelles sont vos appréciations sur la pertinence des investissements et activités du FARA pour le renforcement des capacités du système éducatif ? Approfondir :  Quels sont les principaux acquis à ce niveau ?  Quelles sont les principales contraintes ?  Quelles sont vos suggestions en vue d’améliorer la contribution du FARA au renforcement des capacités du système éducatif ? 78 3. Que pensez-vous de la stratégie d’intervention du FARA ? Approfondir :  Quelle est votre appréciation du système de remboursement forfaitaire ?  Quelle est votre appréciation du niveau d’intégration entre les différentes activités (construction, équipement CEM, mise en place point d’eau, mobilisation communautaire, etc.) ?  Quelles sont vos recommandations et suggestions pour améliorer la stratégie d’intervention du FARA? Thème 2 : Dans quelle mesure l’activité FARA a-t-elle favorisé la collaboration entre les différents acteurs impliqués 4. Quelles sont vos appréciations sur le niveau d’implication de votre structure dans la conception et la mise en œuvre du FARA ? Approfondir :  Comment s’est faite votre implication ?  Quel a été votre rôle dans la mise en œuvre du FARA ?  Comment s’est fait les prises de décisions (par rapport aux choix de localités, le choix d’entrepreneur) ?  Quelles appréciations faites-vous sur la pertinence des localités choisis dans les régions d’intervention du FARA ?  Quels sont les principaux acquis en termes de collaboration entre acteurs du projet (USAID, MEF, DCS, DPRE, DEMSG) ?  Quelles sont les principales contraintes ayant entravé le fonctionnement optimal du cadre partenarial ?  Quelles sont vos suggestions et recommandations en vue de l’amélioration du fonctionnement du cadre partenarial ? Thème 3 : Contribution du FARA à l’accès et au maintien des filles et garçons à l’école 5. Dans quelle mesure le FARA a –t-il favorisé l’accès des filles et des garçons au cycle d’enseignement moyen ? Quels sont les éléments de preuve dont vous disposez pour le confirmer ? 6. Dans quelle mesure le FARA a –t-il favorisé le maintien des filles et des garçons au cycle d’enseignement moyen ? Quels sont les éléments de preuve dont vous disposez pour le confirmer ? 7. Quelles sont vos suggestions en vue de renforcer cette contribution du FARA à l’accès et au maintien des garçons et des filles au cycle moyen ? 79 PROJET DE GUIDE D’ENTRETIEN A L’INTENTION DE LA DEMSG Principes de l’enquête / Le consentement verbal : Bonjour le GERAD, cabinet de consultance Sénégalais, effectue l’étude d’évaluation du FARA pour la construction de collèges pour le compte de l’USAID. La finalité est de collecter des informations afin de formuler des recommandations en vue de documenter la stratégie à mettre en œuvre pour de futurs programmes. Nous aimerions avoir votre aide dans ce domaine. Est-ce que vous voulez participer dans cette étude ? [Attendez la réponse de la personne. Si la personne semble à l’aise, passez directement au consentement verbal.] Votre nom ne sera pas mentionné dans le rapport fait sur cette enquête. Les résultats de l'enquête seront publiés sous la forme du texte narratif sans identification des individus. Le rapport final sera partagé avec l’USAID et le Ministère de l’Education National dans le but de renforcer l'implémentation des programmes. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer l'entretien. En fin de l’étude, l’enregistrement sera envoyé au niveau de l’USAID pour documenter l’activité. Votre nom ne sera pas lié à l’enregistrement. Mais si vous ne voulez pas participer, il n’y aura pas de problème. Une fois que nous commençons, si vous préférez ne pas répondre à une question, vous pouvez nous le dire. Est-ce que vous nous donnez votre consentement ? [Avez-vous obtenu le consentement verbal? ☐ ] I. Identification Nom et Prénom : Fonction : Structure : Durée d’occupation du poste : Question d’introduction : 1. Quel est votre degré de satisfaction par rapport à la mise en œuvre du FARA ? Approfondir :  Quels ont été les points forts de ce programme à votre avis ?  Quels ont été les points faibles de ce programme à votre avis ? Thème 1 : Preuves de la pertinence de l’activité FARA dans le renforcement des capacités du système éducatif 2. Quelles sont vos appréciations sur la pertinence des investissements et activités du FARA pour le renforcement des capacités du système éducatif ? Approfondir :  Quels sont les principaux acquis à ce niveau ?  Quelles sont les principales contraintes ?  Quelles sont vos suggestions en vue d’améliorer la contribution du FARA au renforcement des capacités du système éducatif ? 80 3. Que pensez-vous de la stratégie d’intervention du FARA ? Approfondir :  Quelle est votre appréciation du système de remboursement forfaitaire ?  Quelle est votre appréciation du niveau d’intégration entre les différentes activités (construction, équipement CEM, mise en place point d’eau, mobilisation communautaire, etc.) ?  Quelles sont vos recommandations et suggestions pour améliorer la stratégie d’intervention du FARA? Thème 2 : Dans quelle mesure l’activité FARA a-t-elle favorisé la collaboration entre les différents acteurs impliqués 4. Quelles sont vos appréciations sur le niveau d’implication de votre structure dans la conception et la mise en œuvre du FARA ? Approfondir :  Comment s’est faite votre implication ?  Quel a été votre rôle dans la mise en œuvre du FARA ?  Comment s’est fait les prises de décisions (par rapport aux choix de localités, le choix d’entrepreneur) ?  Quelles appréciations faites-vous sur la pertinence des localités choisis dans les régions d’intervention du FARA ?  Quels sont les principaux acquis en termes de collaboration entre acteurs du projet (USAID, MEF, DCS, DPRE, DEMSG) ?  Quelles sont les principales contraintes ayant entravé le fonctionnement optimal du cadre partenarial ?  Quelles sont vos suggestions et recommandations en vue de l’amélioration du fonctionnement du cadre partenarial ? Thème 3 : Dans quelle mesure les actions de mobilisation communautaire mises en œuvre dans le cadre du FARA ont-elles favorisé la participation et l’appropriation par les communautés bénéficiaires des ouvrages réalisés 5. En quoi ont consisté les activités de mobilisation communautaire dans le cadre du FARA ? Sur quelle période ? 6. Dans quelle mesure ont-elles permis la participation de la communauté dans la gestion, l’entretien et la maintenance des ouvrages réalisés ? Donner des exemples. 7. Dans quelle mesure ont-elles permis leur appropriation par la communauté ? 8. Quelles sont les forces de ce mécanisme d’appropriation communautaire par rapport à la maintenance et à la gestion des ouvrages réalisés ? 81 9. Quelles sont les limites de ce mécanisme d’appropriation communautaire par rapport à la maintenance et à la gestion des ouvrages réalisés ? 10. Quelles sont vos recommandations pour une meilleure participation et appropriation communautaire ? Thème 4 : Contribution du FARA à l’accès et au maintien des filles et garçons à l’école 11. Dans quelle mesure le FARA a –t-il favorisé l’accès des filles et des garçons au cycle d’enseignement moyen ? Quels sont les éléments de preuve dont vous disposez pour le confirmer ? 12. Dans quelle mesure le FARA a –t-il favorisé le maintien des filles et des garçons au cycle d’enseignement moyen ? Quels sont les éléments de preuve dont vous disposez pour le confirmer ? 13. Quelles sont vos suggestions en vue de renforcer cette contribution du FARA à l’accès et au maintien des garçons et des filles au cycle moyen ? 82 PROJET DE GUIDE D’ENTRETIEN A L’INTENTION DCS Principes de l’enquête / Le consentement verbal : Bonjour le GERAD, cabinet de consultance Sénégalais, effectue l’étude d’évaluation du FARA pour la construction de collèges pour le compte de l’USAID. La finalité est de collecter des informations afin de formuler des recommandations en vue de documenter la stratégie à mettre en œuvre pour de futurs programmes. Nous aimerions avoir votre aide dans ce domaine. Est-ce que vous voulez participer dans cette étude ? [Attendez la réponse de la personne. Si la personne semble à l’aise, passez directement au consentement verbal.] Votre nom ne sera pas mentionné dans le rapport fait sur cette enquête. Les résultats de l'enquête seront publiés sous la forme du texte narratif sans identification des individus. Le rapport final sera partagé avec l’USAID et le Ministère de l’Education National dans le but de renforcer l'implémentation des programmes. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer l'entretien. En fin de l’étude, l’enregistrement sera envoyé au niveau de l’USAID pour documenter l’activité. Votre nom ne sera pas lié à l’enregistrement. Mais si vous ne voulez pas participer, il n’y aura pas de problème. Une fois que nous commençons, si vous préférez ne pas répondre à une question, vous pouvez nous le dire. Est-ce que vous nous donnez votre consentement ? [Avez-vous obtenu le consentement verbal? ☐ ] I. Identification Nom et Prénom : Fonction : Structure : Durée d’occupation du poste : Question d’introduction : 1. Quel est votre degré de satisfaction par rapport à la mise en œuvre du FARA ? Approfondir :  Quels ont été les points forts de ce programme à votre avis ?  Quels ont été les points faibles de ce programme à votre avis ? Thème 1 : Preuves de la pertinence de l’activité FARA dans le renforcement des capacités du système éducatif 2. Quelles sont vos appréciations sur la pertinence des investissements et activités du FARA pour le renforcement des capacités du système éducatif ? Approfondir :  Quels sont les principaux acquis à ce niveau ?  Quelles sont les principales contraintes ?  Quelles sont vos suggestions en vue d’améliorer la contribution du FARA au renforcement des capacités du système éducatif ? 83 3. Quelles appréciations faites-vous sur la pertinence des localités choisis dans les régions d’intervention du FARA ? Approfondir :  Comment s’est fait le choix de la localité d’implantation du CEM ?  Quels sont les critères du choix de la localité ? 4. Que pensez-vous de la stratégie d’intervention du FARA ? Approfondir :  Quelle est votre appréciation du système de remboursement forfaitaire ?  Quelle est votre appréciation du niveau d’intégration entre les différentes activités (construction, équipement CEM, mise en place point d’eau, mobilisation communautaire, etc.) ?  Quelles sont vos recommandations et suggestions pour améliorer la stratégie d’intervention du FARA? Thème 2 : Dans quelle mesure l’activité FARA a-t-elle favorisé la collaboration entre les différents acteurs impliqués 5. Quelles sont vos appréciations sur le niveau d’implication des autres structures du MEN dans la conception et la mise en œuvre du FARA ? Approfondir :  Comment s’est faite cette implication ?  Quel a été les rôles respectifs de chaque structure dans la mise en œuvre du FARA ?  Quels sont les principaux acquis en termes de collaboration entre acteurs du projet (USAID, MEF, DCS, DPRE, DEMSG)?  Quelles sont les principales contraintes ayant entravé le fonctionnement optimal de ce cadre partenarial ?  Quelles sont vos suggestions et recommandations en vue de l’amélioration du fonctionnement de ce cadre partenarial ? 84 Thème 3 : Contribution du FARA à l’accès et au maintien des filles et garçons à l’école 6. Dans quelle mesure le FARA a –t-il favorisé l’accès des filles et des garçons au cycle d’enseignement moyen ? Quels sont les éléments de preuve dont vous disposez pour le confirmer ? 7. Dans quelle mesure le FARA a –t-il favorisé le maintien des filles et des garçons au cycle d’enseignement moyen ? Quels sont les éléments de preuve dont vous disposez pour le confirmer ? 8. Quelles sont vos suggestions en vue de renforcer cette contribution du FARA à l’accès et au maintien des garçons et des filles au cycle moyen ? 85 PROJET DE GUIDE D’ENTRETIEN A L’INTENTION DAGE Principes de l’enquête / Le consentement verbal : Bonjour le GERAD, cabinet de consultance Sénégalais, effectue l’étude d’évaluation du FARA pour la construction de collèges pour le compte de l’USAID. La finalité est de collecter des informations afin de formuler des recommandations en vue de documenter la stratégie à mettre en œuvre pour de futurs programmes. Nous aimerions avoir votre aide dans ce domaine. Est-ce que vous voulez participer dans cette étude ? [Attendez la réponse de la personne. Si la personne semble à l’aise, passez directement au consentement verbal.] Votre nom ne sera pas mentionné dans le rapport fait sur cette enquête. Les résultats de l'enquête seront publiés sous la forme du texte narratif sans identification des individus. Le rapport final sera partagé avec l’USAID et le Ministère de l’Education National dans le but de renforcer l'implémentation des programmes. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer l'entretien. En fin de l’étude, l’enregistrement sera envoyé au niveau de l’USAID pour documenter l’activité. Votre nom ne sera pas lié à l’enregistrement. Mais si vous ne voulez pas participer, il n’y aura pas de problème. Une fois que nous commençons, si vous préférez ne pas répondre à une question, vous pouvez nous le dire. Est-ce que vous nous donnez votre consentement ? [Avez-vous obtenu le consentement verbal? ☐ ] I. Identification Nom et Prénom : Fonction : Structure : Durée d’occupation du poste : Question d’introduction : 1. Quel est votre degré de satisfaction par rapport à la mise en œuvre du FARA ? Approfondir :  Quels ont été les points forts de ce programme à votre avis ?  Quels ont été les points faibles de ce programme à votre avis ? 86 Thème 1 : Preuves de la pertinence de l’activité FARA dans le renforcement des capacités du système éducatif 2. Quelles sont vos appréciations sur la pertinence des investissements et activités du FARA pour le renforcement des capacités du système éducatif ? Approfondir :  Quels sont les principaux acquis à ce niveau ?  Quelles sont les principales contraintes ?  Quelles sont vos suggestions en vue d’améliorer la contribution du FARA au renforcement des capacités du système éducatif ? 3. Quelles appréciations faites-vous sur la pertinence des localités choisis dans les régions d’intervention du FARA ? Approfondir :  Comment s’est fait le choix de la localité d’implantation du CEM ?  Quels sont les critères du choix de la localité ? 4. Que pensez-vous de la stratégie d’intervention du FARA ? Approfondir :  Quelle est votre appréciation du système de remboursement forfaitaire ?  Quelle est votre appréciation du niveau d’intégration entre les différentes activités (construction, équipement CEM, mise en place point d’eau, mobilisation communautaire, etc.) ?  Quelles sont vos recommandations et suggestions pour améliorer la stratégie d’intervention du FARA? Thème 2 : Dans quelle mesure l’activité FARA a-t-elle favorisé la collaboration entre les différents acteurs impliqués 5. Quelles sont vos appréciations sur le niveau d’implication des autres structures du MEN dans la conception et la mise en œuvre du FARA ? Approfondir :  Comment s’est faite cette implication ?  Quel a été les rôles respectifs de chaque structure dans la mise en œuvre du FARA ?  Quels sont les principaux acquis en termes de collaboration entre acteurs du projet (USAID, MEF, DCS, DPRE, DEMSG)?  Quelles sont les principales contraintes ayant entravé le fonctionnement optimal de ce cadre partenarial ?  Quelles sont vos suggestions et recommandations en vue de l’amélioration du fonctionnement de ce cadre partenarial ? Thème 3 : Contribution du FARA à l’accès et au maintien des filles et garçons à l’école 87 6. Dans quelle mesure le FARA a –t-il favorisé l’accès des filles et des garçons au cycle d’enseignement moyen ? Quels sont les éléments de preuve dont vous disposez pour le confirmer ? 7. Dans quelle mesure le FARA a –t-il favorisé le maintien des filles et des garçons au cycle d’enseignement moyen ? Quels sont les éléments de preuve dont vous disposez pour le confirmer ? 8. Quelles sont vos suggestions en vue de renforcer cette contribution du FARA à l’accès et au maintien des garçons et des filles au cycle moyen ? 88 GUIDE D’ENTRETIEN : BÉNÉFICIAIRES (APE, CGE, COMMUNAUTES) Principes de l’enquête / Le consentement verbal : Bonjour le GERAD, cabinet de consultance Sénégalais, effectue l’étude d’évaluation du FARA pour la construction de collèges pour le compte de l’USAID. Votre établissement fait partie des établissements qui ont été choisi à participer dans cette enquête. L’étude concerne l’environnement de l’école et non la qualité des enseignements. La finalité est de collecter des informations afin de formuler des recommandations en vue de documenter la stratégie à mettre en œuvre pour de futurs programmes. L’USAID s’intéresse beaucoup à vos perspectives et vos expériences du collège qu’ils ont aidé à réaliser. Nous aimerions avoir votre aide dans ce domaine. Est-ce que vous voulez participer dans cette étude ? [Attendez la réponse de la personne. Si la personne semble à l’aise, passez directement au consentement verbal.] Votre nom ne sera pas mentionné dans le rapport fait sur cette enquête. Les résultats de l'enquête seront publiés sous la forme du texte narratif sans identification des individus. Le rapport final sera partagé avec l’USAID et le Ministère de l’Education National dans le but de renforcer l'implémentation des programmes. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer l'entretien. En fin de l’étude, l’enregistrement sera envoyé au niveau de l’USAID pour documenter l’activité. Votre nom ne sera pas lié à l’enregistrement. Mais si vous ne voulez pas participer, il n’y aura pas de problème. Aussi, une fois que nous commençons, si vous préférez ne pas répondre à une question, vous pouvez nous le dire. Est-ce que vous nous donnez votre consentement ? [Avez-vous obtenu le consentement verbal? ☐ ] Quelles sont les évidences de la pertinence de l’activité FARA dans le renforcement des capacités du système éducatif ?  La construction du collège répond t-elle au besoin d’accroissement de l’accès à l’éducation moyen dans la zone ?  La mise en place du collège découle-t-elle d’un examen des difficultés et des préoccupations des élèves et enseignants de la zone ?  Comment a-t-elle pu contribuer au renforcement du système éducatif ? Dans quelle mesure l’activité FARA a-t-elle favorisé la collaboration entre les différents acteurs impliqués ?  La construction du collège a-t-elle permis une collaboration entre les différents acteurs concernés ?  Quelles sont les incidences de cette collaboration entre acteurs sur le fonctionnement de l’école et l’accroissement de l’accès à l’enseignement moyen ?  Quelles sont les limites de cette collaboration ?  Comment améliorer ou renforcer la collaboration entre acteurs de l’école ? 89 Dans quelle mesure les actions de mobilisation communautaire mises en œuvre dans le cadre du FARA ont-elles favorisé la participation et l’appropriation par les communautés bénéficiaires des ouvrages réalisés ? Quelles sont les forces et les limites de ce mécanisme d’appropriation communautaire par rapport à la maintenance et à la gestion des ouvrages réalisés ?  Comment s’est faite la mobilisation communautaire dans le cadre de la mise en œuvre du FARA ?  La mobilisation communautaire a-t-elle favorisé la participation et l’appropriation des ouvrages réalisés par les communautés bénéficiaires ?  Quelles sont les forces et les limites de ce mécanisme d’appropriation communautaire par rapport à la maintenance et à la gestion des ouvrages réalisés ? Dans quelle mesure l’Activité FARA a favorisé l’accès et le maintien des filles et des garçons au cycle d’enseignement moyen ? Quelles en sont les leçons apprises ?  La réalisation du collège a-t-elle favorisé l’accès et le maintien des filles et des garçons au cycle d’enseignement moyen ? Justifier avec des exemples concrets  Quelles leçons avez-vous appris du Programme FARA par rapport à l’accès et au maintien des enfants au cycle d’enseignement moyen ?  Comment le FARA pourrait-il d’avantage favorisé l’accès et le maintien des enfants au cycle d’enseignement moyen ?  Si un tel projet devait être à nouveau implanté dans votre communauté, quelles dispositions suggéreriez-vous qu’on prenne ? qu’on améliore ? qu’on ne renouvelle pas du tout ? 90 Evaluation de FARA Guide – Focus Groupe des Enfants Principes de l’enquête / Le consentement verbal : Bonjour ! Je m’appelle ____________ et je suis venue pour aider les élevés. Je travaille avec le GERAD, cabinet de consultance Sénégalais. Votre école a été construite avec l’aide de l’état américain à travers une structure qui s’appelle l’USAID. C’était dans le but d’améliorer les conditions d’apprentissage et surtout pour augmenter le nombre d’enfants dans l’enseignement moyen. Nous sommes en train d’effectuer l’étude sur ce programme. Votre établissement fait partie des établissements qui ont été choisi à participer dans cette enquête. L’étude concerne l’environnement de l’école et non la qualité des enseignements. Nous cherchons à collecter des informations afin de formuler des recommandations en vue de documenter la stratégie à mettre en œuvre pour de futurs programmes. L’USAID s’intéresse beaucoup à vos perspectives et vos expériences du collège qu’ils ont aidé à réaliser. Nous aimerions avoir votre aide dans ce domaine. Ceci N’est PAS un examen et vous ne serez PAS notés pour vos réponses. Est-ce que vous voulez participer dans cette étude ? Votre nom ne sera pas mentionné dans le rapport fait sur cette enquête. Les résultats de l'enquête seront publiés sous la forme du texte narratif sans identification des individus. Le rapport final sera partagé avec l’USAID et le Ministère de l’Education National dans le but de renforcer l'implémentation des programmes. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer l'entretien. En fin de l’étude, l’enregistrement sera envoyé au niveau de l’USAID pour documenter l’activité. Votre nom ne sera pas lié à l’enregistrement. Mais si vous ne voulez pas participer, il n’y aura pas de problème. Aussi, une fois que nous commençons, si vous préférez ne pas répondre à une question, vous pouvez nous le dire. Est-ce que vous nous donnez votre consentement ? [Avez-vous obtenu le consentement verbal? ☐ ] Question d’évaluation : Dans quelle mesure l’Activité FARA a favorisé l’accès et le maintien des filles et des garçons au cycle d’enseignement moyen ? Quelles en sont les leçons apprises ? Questions de profil - introduction: 1. Ça fait combien de temps que tu apprends dans ce collège ? o Approfondir :  Si tu allais à un autre collège avant, pourquoi est-ce que tu as changé d’école ? 2. Où est-ce que tu habites ? o Approfondir :  L’école, c’est à combien de kilomètres de chez toi ? Comment est-ce que tu fais pour venir à l’école ? Ca prend combien de temps ? Comment est￾ce que tu fais pour venir à l’école ? 3. Avec qui est-ce que tu habites ? (Chez la maman, le papa, avec un tuteur…) Questions de discussion : 1. Est-ce que vous aimez venir au collège? Pour quoi ? Pourquoi pas ? 91 2. Est-ce que vous vous sentez à l’aise dans ce collège ? Pour quoi ? Pourquoi pas ? (Insister sur les questions de l’environnement - pas les questions liées à la qualité de l’enseignement.) 3. Que pensez-vous de l’environnement de ce collège ? Les bâtiments ? Les toilettes ? Le laboratoire ? La salle d’informatique ? 4. Que pourrait-on faire pour améliorer l’environnement du collège ? 5. S’il y a un point d’eau. Qui utilise le point d’eau ? Ca a que rapport avec l’école ? Est-ce qu’avoir le point d’eau ici vous aide dans les études ? Le fait de ne pas avoir un point d’eau, quel était l’impact sur l’école ? 6. Est-ce que le courant fonctionne au niveau du collège ? Est-ce qu’avoir le courant au niveau du collège vous aide dans les études ? 7. Avez-vous des sœurs ou des frères ou des connaissances qui ne peuvent pas aller au collège ? Pour quoi ? 8. Avez-vous des sœurs ou des frères ou des connaissances qui n’ont pas pu poursuivre leurs études au collège ? Pour quoi ? 92 Evaluation Finale du Projet FARA (Guide d’entretien des individus clés) GUIDE D’ENTRETIEN DES INDIVIDUS CLES –QUESTION #5 Identification : Nom : ……………………………………………………….Titre ………………………………….. Date d'enquête |__|__|/|__|__|/|__|__|__|__| Institution : ……………………………………………………………………… Passation du questionnaire : Heure début : |___|___| H |___|___| mn / Heure fin : |___|___|H |___|___| mn Questions : Q5.01. From your understanding, what were the primary objectives of the FARA school construction work? Basé sur votre compréhension, quels sont les principaux objectifs des travaux de construction d'école du FARA ? ……………………………………………………………………………………………………………….. Probe – any changes in the capacity of the MEN throughout the different phases of the FARA? Please list specific changes Avez-vous constaté des changements dans les capacités du MEN à travers les différentes phases du FARA? Lesquels ? Q5.02. Have there been any changes in the way in which your direction or the other relevant MEN Directions work, since the launch of school construction FARA in 2007? Depuis le lancement de la construction d'école du projet FARA en 2007, y at-il eu des changements dans la façon de travailler au sein de votre Direction ou au niveau des agents des autres Directions concernées ? ……………………………………………………………………………………………………………… ………………………………………………………….,…………………………………………………… … 5.02.01 If yes, what types of changes have occurred? 93 Si oui, quels types de changements ont eu lieu ? 5.02.02 Have there been any procedural changes? Any system change ? Y at-il eu des changements dans les procédures ? D’autres changements dans le système ? 5.03 What has been learned from this process of the FARA school construction? Qu’est-ce qui a été retenu/appris de ce processus de construction d'école de FARA ? 5.04 5.04.01 Comment utiliserez-vous l’experience du FARA dans dans le futur ? ……………………………………………………………………………………………………………… ….. 5.05 How does FARA differ from other school construction process, either through the MEN or with the support of donors? Comment FARA diffère des autres processus de construction d’école, soit au niveau du MEN ou à travers le soutien du bailleur de fonds ? Q5.06. From the MEN’s perspective what types of challenges did the FARA face since its launch in 2007? Du côté du MEN, quels types de défis FARA a dû faire face depuis son lancement dans la première phase en 2007 ? ……………………………………………………………………………………………. ……………………………………………………………………………………………………………… ….…………………………………………………………………………………………………………… ……. Q5.07. How were these challenges addressed ? Comment ces défis ont été contournés/abordés? ……………………………………………………………………………………………………………… ….. Q5.08. Do any challenges still occur ? If yes, please explain Est-ce qu’il y’a des défis qui persistent encore ? Si oui, veuillez expliquer ……………………………………………………………………………………………………………… …. Q5.09. Are there capacities or gaps that you see in the MEN that still require support in terms of school construction planning, management and oversight ? 94 Y at-il des capacités ou des lacunes que vous voyez au niveau du MEN qui nécessitent encore un soutien en termes de plannification, gestion et de suivi des écoles construites ? ………………………………………………………………. ……………………………………………………………………………………………………………… ….. 5.09.01 If yes, please explain, in what way does this impact school construction? Si oui, s'il vous plaît expliquer en quoi elles peuvent avoir des impacts sur la construction d'école ? Nous vous remercions beaucoup de nous avoir accordé de votre temps pour participer à cette séance de discussion ! 95 Grille de participants Elève # Genre (M/F) Age Classe Années dans le collège Habitant avec (Maman, Papa, Grand parents, Tuteur…) Distance entre chez lui et le collège (Km) Moyen de locomotion 1 2 3 4 5 6 7 96 PROJET DE GUIDE D’ENTRETIEN A L’INTENTION DES IA ET DES IEF Principes de l’enquête / Le consentement verbal : Bonjour le GERAD, cabinet de consultance Sénégalais, effectue l’étude d’évaluation du FARA pour la construction de collèges pour le compte de l’USAID. Des collèges dans votre inspectorat font partie des établissements qui ont été choisi à participer dans cette enquête. L’étude concerne l’environnement de l’école et non la qualité des enseignements. La finalité est de collecter des informations afin de formuler des recommandations en vue de documenter la stratégie à mettre en œuvre pour de futurs programmes. L’USAID s’intéresse beaucoup à vos perspectives et vos expériences du collège qu’ils ont aidé à réaliser. Nous aimerions avoir votre aide dans ce domaine. Est-ce que vous voulez participer dans cette étude ? [Attendez la réponse de la personne. Si la personne semble à l’aise, passez directement au consentement verbal.] Votre nom ne sera pas mentionné dans le rapport fait sur cette enquête. Les résultats de l'enquête seront publiés sous la forme du texte narratif sans identification des individus. Le rapport final sera partagé avec l’USAID et le Ministère de l’Education National dans le but de renforcer l'implémentation des programmes. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer l'entretien. En fin de l’étude, l’enregistrement sera envoyé au niveau de l’USAID pour documenter l’activité. Votre nom ne sera pas lié à l’enregistrement. Mais si vous ne voulez pas participer, il n’y aura pas de problème. Une fois que nous commençons, si vous préférez ne pas répondre à une question, vous pouvez nous le dire. Est-ce que vous nous donnez votre consentement ? [Avez-vous obtenu le consentement verbal? ☐ ] I. Identification Nom et Prénom : Fonction : Structure : Durée d’occupation de poste : Question d’introduction : 1. Quel est votre degré de satisfaction par rapport à la mise en œuvre du FARA ? Approfondir :  Quels ont été les points forts de ce programme à votre avis ?  Quels ont été les points faibles de ce programme à votre avis ? Thème 1 : Preuves de la pertinence de l’activité FARA dans le renforcement des capacités du système éducatif 2. Quelles sont vos appréciations sur la pertinence des investissements et activités du FARA pour le renforcement des capacités du système éducatif ? Approfondir :  Quels sont les principaux acquis à ce niveau ?  Quelles sont les principales contraintes ? 97  Quelles sont vos suggestions en vue d’améliorer la contribution du FARA au renforcement des capacités du système éducatif ? 3. Quelles appréciations faites-vous sur la pertinence des localités choisies dans les régions d’intervention du FARA ? Approfondir :  Comment s’est fait le choix de la localité d’implantation du CEM ?  Quels sont les critères du choix de la localité ? 4. Que pensez-vous de la stratégie d’intervention du FARA ? Approfondir :  Quelle est votre appréciation du système de remboursement forfaitaire ?  Quelle est votre appréciation du niveau d’intégration entre les différentes activités (construction, équipement CEM, mise en place point d’eau, mobilisation communautaire, etc.) ?  Quelles sont vos recommandations et suggestions pour améliorer la stratégie d’intervention du FARA? Thème 2 : Dans quelle mesure l’activité FARA a-t-elle favorisé la collaboration entre les différents acteurs impliqués 5. Quelles sont vos appréciations sur le niveau d’implication de votre structure dans la conception et la mise en œuvre du FARA ? Approfondir :  Comment s’est faite votre implication ?  Quel a été votre rôle dans la mise en œuvre du FARA ?  Quels sont les principaux acquis en termes de collaboration entre acteurs du projet (USAID, MEF, DCS, DPRE, DEMSG, IA, IEF) ?  Quelles sont les facteurs ayant contribué à une bonne collaboration ?  Quelles sont les principales contraintes ayant entravé une bonne collaboration ?  Quelles sont vos suggestions et recommandations en vue de l’amélioration de la collaboration ? Thème 3 : Contribution du FARA à l’accès et au maintien des filles et garçons à l’école 6. Dans quelle mesure le FARA a –t-il favorisé l’accès des filles et des garçons au cycle d’enseignement moyen ? Quels sont les éléments de preuve dont vous disposez pour le confirmer ? 7. Dans quelle mesure le FARA a –t-il favorisé le maintien des filles et des garçons au cycle d’enseignement moyen ? Quels sont les éléments de preuve dont vous disposez pour le confirmer ? 98 8. Quelles sont vos suggestions en vue de renforcer cette contribution du FARA à l’accès et au maintien des garçons et des filles au cycle moyen ? 99 GUIDE D’ENTRETIEN : PERSONNEL DU CEM (PRINCIPAL, ENSEIGNANTS) Principes de l’enquête / Le consentement verbal : Bonjour le GERAD, cabinet de consultance Sénégalais, effectue l’étude d’évaluation du FARA pour la construction de collèges pour le compte de l’USAID. Votre établissement fait partie des établissements qui ont été choisi à participer dans cette enquête. L’étude concerne l’environnement de l’école et non la qualité des enseignements. La finalité est de collecter des informations afin de formuler des recommandations en vue de documenter la stratégie à mettre en œuvre pour de futurs programmes. L’USAID s’intéresse beaucoup à vos perspectives et vos expériences du collège qu’ils ont aidé à réaliser. Nous aimerions avoir votre aide dans ce domaine. Est-ce que vous voulez participer dans cette étude ? [Attendez la réponse de la personne. Si la personne semble à l’aise, passez directement au consentement verbal.] Votre nom ne sera pas mentionné dans le rapport fait sur cette enquête. Les résultats de l'enquête seront publiés sous la forme du texte narratif sans identification des individus. Le rapport final sera partagé avec l’USAID et le Ministère de l’Education National dans le but de renforcer l'implémentation des programmes. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer l'entretien. En fin de l’étude, l’enregistrement sera envoyé au niveau de l’USAID pour documenter l’activité. Votre nom ne sera pas lié à l’enregistrement. Mais si vous ne voulez pas participer, il n’y aura pas de problème. Aussi, une fois que nous commençons, si vous préférez ne pas répondre à une question, vous pouvez nous le dire. Est-ce que vous nous donnez votre consentement ? [Avez-vous obtenu le consentement verbal? ☐ ] Quelles sont les évidences de la pertinence de l’activité FARA dans le renforcement des capacités du système éducatif ?  La construction du collège répond-elle au besoin d’accroissement de l’accès à l’éducation moyen dans la zone ?  Avez-vous noté une prise en compte des intérêts spécifiques du personnel enseignant et des élèves dans le processus de mise en œuvre du programme ? (ex : salle d’informatique, toilettes, laboratoires, salle de prof)  Comment le programme a pu contribuer au renforcement du système éducatif ?  Quel a été l’apport du FARA dans la gestion des activités de l’APE ou du CGE ? Dans quelle mesure l’activité FARA a-t-elle favorisé la collaboration entre les différents acteurs impliqués ?  Le mode de gestion du CEM (avec les niveaux d’implication des différents bénéficiaires) répond￾il à vos attentes ?  La construction du collège a-t-elle permis une collaboration entre les différents acteurs concernés ?  Quelle a été la place du personnel du collège dans la mise en place de cette collaboration ?  Quelles sont les incidences de cette collaboration entre acteurs sur le fonctionnement de l’école et l’accroissement de l’accès à l’enseignement moyen ? 100  Quelles sont les forces de cette collaboration ?  Quelles sont les limites de cette collaboration ?  Comment améliorer ou renforcer la collaboration entre acteurs de l’école dans le cadre de projet similaire, notamment dans le domaine de la gestion et de la maintenance ? Dans quelle mesure les actions de mobilisation communautaire mises en œuvre dans le cadre du FARA ont-elles favorisé la participation et l’appropriation par les communautés bénéficiaires des ouvrages réalisés ? Quelles sont les forces et les limites de ce mécanisme d’appropriation communautaire par rapport à la maintenance et à la gestion des ouvrages réalisés ?  Comment appréciez-vous le niveau d’appropriation et d’utilisation des réalisations du programme par le personnel enseignant et les élèves ?  Quelle a été le degré d’implication de la communauté et/ou des bénéficiaires dans la construction, la gestion et l’entretien du collège?  Comment s’est faite la mobilisation communautaire dans le cadre de la mise en œuvre du FARA ?  Les communautés bénéficiaires mettent-elles à profit les investissements réalisés et utilisent-elles les savoir-faire acquis ?  La mobilisation communautaire a-t-elle favorisé la participation et l’appropriation des ouvrages réalisés par les communautés bénéficiaires ?  Quelles sont les forces et les limites de ce mécanisme d’appropriation communautaire par rapport à la maintenance et à la gestion des ouvrages réalisés ? Dans quelle mesure l’Activité FARA a favorisé l’accès et le maintien des filles et des garçons au cycle d’enseignement moyen ? Quelles en sont les leçons apprises ?  La réalisation du collège a-t-elle favorisé l’accès et le maintien des filles et des garçons au cycle d’enseignement moyen ? Justifier avec des exemples concrets ?  Quelles leçons avez-vous appris du Programme FARA par rapport à l’accès et au maintien des enfants au cycle d’enseignement moyen ?  Comment un projet similaire au FARA pourrait-il d’avantage favoriser l’accès et le maintien des enfants au cycle d’enseignement moyen ?  Si un tel projet devait être à nouveau implanté dans votre communauté, quelles dispositions suggéreriez-vous qu’on prenne ? qu’on améliore ? qu’on ne renouvelle pas du tout ? 101 Evaluation Finale FARA List de Contrôle: Visite Site Qualitative Nom du collège : Région : Date(s) de visite : Nom des membres de l’équipe d’évaluation : Activités achevées (X) _____ Visite de courtoisie – Autorités administratives Nom de l’autorité : Fonction : _____ Visite de courtoisie – Autorités locales Nom de l’autorité : Fonction : _____ Entretien avec l’IA ; Nom de l’Inspecteur : __________________________________ _____ Entretien avec l’IEF ; Nom de l’Inspecteur : _________________________________ _____ Entretien avec le principal ; Nom du principal : ______________________________ _____ Remplir la fiche statistique collège _____ Remplir la fiche Etat du patrimoine _____ Focus groupe – Bureau de l’APE & CGE No. de participants : ______ Hommes: ______ / Femmes : _______ _____ Focus groupe – Enseignants No. de participants : ______ Hommes: ______ / Femmes : _______ _____ Focus groupe – Elèves filles No. de participants : ______ _____ Focus groupe – Elèves garçons No. de participants : ______ _____ Focus groupe – Leaders communautaires No. de participants : ______ Hommes: ______ / Femmes : _______ Remarques 102 FICHE – ETAT DU PATRIMOINE Date(s) de visite : Membres de l’équipe d’évaluation : Nom du collège : Région : Nom du principal : Date de prise de poste : No. salles de classe réalisée par l’USAID : No. salles de classe en total : No. classes pédagogiques : No. tables-banc : A part l’Etat du Sénégal avec le remboursement de l’USAID, qui a construit les autres salles de classe ? Matériel de construction : Si un gap entre le nombre de classes pédagogiques et les salles de classe, que fait le collège pour combler ce gap : La zone de polarisation du collège : Etat de bâtiments construits par l’Etat/USAID : Changements apportés depuis la réception provisoire : Effectivité de la réception définitive : Autres constructions de FARA Bloc administratif Oui Non No. de toilettes (administratives) : ______ Salle de professeurs Oui Non Bureau de principal Oui Non Bureau de surveillants Oui Non Bloc hygiénique Oui Non No. de cabines : ______ (filles) ______ (garçons) Est-ce que les toilettes fonctionnent ? Oui Non Raison : Salle d’informatique Oui Non Est-ce que la salle d’informatique fonctionne ? Oui Non Raison : Laboratoire Oui Non Est-ce que le laboratoire fonctionne ? Oui Non Raison : Loge gardien Oui Non 103 Point d’eau Raccordement à l’électricité : Construit par l’Etat/USAID : Oui Non Si pas l’Etat/USAID, qui l’a construit ? Qui paye l’eau ? Qui vient puiser l’eau ? En quoi contribue le point d’eau à l’école ? Qui fait le maintien de ce point d’eau ? Le nettoyage ? S’il y a problème ? Fait par l’Etat/USAID : Oui Non Si pas l’Etat/USAID, qui a fait le raccordement? Qui paye le courant ? En quoi contribue avoir le courant à l’école ? Qui utilise le courant ? Comment faire s’il y a problème ? Equipements fournis par l’USAID/Project FARA (Indiquer le nombre) Administration Ordinateurs – administratives : __________ Bureau: __________ Chaise de bureau : __________ Armoires : __________ Photocopieuse : __________ Observations : Salle de classe Tables-banc : __________ Bureau de prof : __________ Chaise de prof : __________ Ordinateurs – salle d’informatique : __________ Livres – élèves : Livres - Guide d’enseignant : Autre : Observations : Est-ce que la photocopieuse fonctionne ? Oui Non Raison _____________________ Autres Equipements Est-ce que le collège a reçu d’autres équipements ? Oui Non Si oui, la source : Activités achevées (X) ________ Tour du collège ________ Remplir la fiche statistique collège Observations de la visite (générale) USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 104 Fiche – Statistiques collège COLLEGE DE: COMMUNE DE : REGION DE: Années Total éléves Nombre éléves par salle de classe Nombre moyen d'éléves par table-banc Taux de redoublement Taux d'abandon Taux de réussite Interventions d'autres partenaires Nombre garçons Nombre filles Nombre garçons Nombre filles Nombre garçons Nombre filles Nombre garçons Nombre filles Nombre garçons Nombre filles Nom du partenaire Type de contribtion Année de création : Année N+1 : Année N+2 : Année N+3 : Année N+4 : Année N+5 : Année N+6 : Année N+7 : Année N+8 : Année N+9: Année N+10: USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 105 ANNEX IV: SOURCES OF INFORMATION Table #: People interviewed at the regional/local level: Participants Role Femmes Hommes Total School Principal 3 19 22 PTA/SMC Member 5 38 43 Community members/Households 11 24 35 Local authorities 2 24 26 Village chief/Neighborhood chief 0 10 10 District Council (Conseil departementale) 1 6 7 Mayor's Office 1 8 9 Students 73 69 142 Teachers 5 33 38 Administrative authorities (Regional) 2 7 9 Governor 0 1 1 Prefecture 2 6 8 Education Inspection Offices 2 35 37 Inspection d'academie 1 14 15 Inspection de l'education et de la formation 1 21 22 Total 103 249 352 USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 106 Table #: People Interviewed at Central level Nom Poste Role - FARA DCS Al Hassane Ndiaye #1 Chef du projet du FARA; Point focal de la DCS Point focal DPRE Malick Sanokho Chef du bureau carte scolaire Point focal DPRE Amadou Gueye Seye Deputy Director DEMSG Dethier Ba Coordinateur de la cellule suivre et evaluation Point focal DAGE Tamsir Gueye Chef de la division comptabilite et investissement Point focal DAGE Diaga Diagne Directeur DCS Ousmane Diop Directeur DCS Al Hassane Ndiaye #2 Chef du projet du FARA; Point focal de la DCS Point focal DCEF Aminata Fall Conseillère en Planification Chargée de Programmes Secteur Education-Formation DI Alpha Diagne Chef du Bureau Amérique, Asie et ONG Point focal DEMSG Ibrahima Ndour Ancien Directeur USAID Pape Sow Formerly Education Team Leader, USAID Senegal; Currently: USAID Ghana Founder/Visionary USAID Mamadou Niasse Chief Accountant, USAID Abdrahmane Diallo Deputy Director USAID Alpha Ba Education Development Specialist USAID Luann Gronhovd Education Team Leader USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 107 ANNEX V: DISCLOSURE OF ANY CONFLICTS OF INTEREST [The Evaluation Policy requires that evaluation reports include a signed statement by each evaluation team member regarding any conflicts of interest. A suggested format is provided below.] Name Title Organization Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1.Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2.Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3.Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4.Current or previous work experience or seeking employment USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 108 with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5.Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6.Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 109 ANNEX VI: FIGURE: STUDENT POPULATION CHANGE AT CONSTRUCTED SCHOOLS 100 200 300 400 500 600 700 800 900 Establishment/Entry End of project Establishment Entry End of project Entry End of project Establishment Entry End of project Establishment Entry End of project Establishment Entry End of project Establishment Entry End of project Establishment Entry /End of project Establishment Entry End of project Entry End of project Entry End of project Entry End of project Entry End of project Establishment Entry End of project Entry End of project Entry End of project Establishment Entry End of project Urban Urban Rural Urban Urban Rural Rural Rural Rural Rural Rural Urban Urban Urban Urban Rural Rural 1A 2A 1B 1C 1D 2D 1E 1G 2G 1H 1I 2I 1J 2J 1K 2K 3K Students School Descriptions and Year (Year after Establishment/Year after Entry into constructions/Year of End of Project) # Boys # Girls USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 110 Student Totals Repeat Rate College Year Code Urban/ Rural Year Total # Boys # Girls Difference (total pop.) from year of established % Difference (total pop.) from year of established % Diff. (girls - boys) # Boys # Girls Keur Massar1 (Dakar) 2010- 2011 1A Urban Establishment/En try 1,478 712 766 - 8% 2.80% 53.70% 2014- 2015 End of project 1,654 807 847 176 12% 5% 26.50% 24.60% Mbao Kamb (Dakar) 2009- 2010 2A Urban Establishment 449 231 218 - - -6% 24.90% 23.20% 2010- 2011 Entry 764 364 400 315 70% 10% 13.50% 8.80% 2014- 2015 End of project 1,303 570 733 854 190% 29% 30.20% 20.20% Ndindy (Diourbel) 2011- 2012 1B Rural Entry 337 189 148 - - -22% 18.00% 11.50% 2014- 2015 End of project 366 198 168 29 9% -15% 32.30% 26.80% CEM 3 Fatick (Fatick) 2006- 2007 1C Urban Establishment 544 296 248 - - -16% 16.55% 22.18% 2010- 2011 Entry 590 277 313 46 8% 13% 32.85% 22.68% 2014- 2015 End of project 654 299 355 110 20% 19% 36.45% 23.10% Zone Sud Mbour (Thies) 2008- 2009 1D Urban Establishment 467 233 234 - - 0% 19.70% 20.90% 2009- 2010 Entry 695 368 327 41 9% -11% 20.40% 15.30% USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 111 2014- 2015 End of project 1,194 576 618 727 156% 7% 24.70% 26.40% Merina Dakhar (Thies) 2006- 2007 2D Rural Establishment 153 88 65 - - -26% 12.50% 12.30% 2009- 2010 Entry 352 185 167 199 130% -10% 21.10% 21.60% 2014- 2015 End of project 557 320 237 404 264% -26% 23.80% 33.80% Ngothie (Kaolack) 2006- 2007 1E Rural Establishment 284 164 120 - - -27% 11.00% 15.80% 2009- 2010 Entry 498 294 204 214 75% -31% 21.10% 16.20% 2014- 2015 End of project 757 387 370 473 167% -4% 41.10% 31.10% Doumga O.Alpha (Matam) 2005- 2006 1G Rural Establishment 176 91 85 - - -7% 13.20% 15.30% 2014- 2015 Entry /End of project 616 266 350 440 250% 32% 17.70% 18.60% Boynadji (Matam) 2007- 2008 2G Rural Establishment 102 57 45 - - -21% 28.10% 15.60% 2011- 2012 Entry 223 110 113 121 119% 3% 12.70% 12.40% 2014- 2015 End of project 262 106 156 160 157% 47% 40.60% 42.30% Ndiaye N.Mberess (Saint Louis) 2011- 2012 1H Rural Entry 571 268 303 - - 13% 29.10% 31.40% 2014- 2015 End of project 761 334 427 190 33% 28% 29.90% 25.10% Ndoga Babacar (Tamba) 2008- 2009 1I Rural Entry 108 85 23 - - -73% 22.40% 47.80% 2014- 2015 End of project 271 157 114 163 151% -27% 33.80% 36.80% USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 112 Commune I Tamba (Tamba) 2011- 2012 2I Urban Entry 315 175 140 - - -20% 16.60% 26.40% 2014- 2015 End of project 589 300 289 274 87% -4% 18.00% 18.30% Hilèle (Kolda) 2011- 2012 1J Urban Entry 524 236 288 - 22% 36.40% 41.30% 2014- 2015 End of project 664 292 372 140 27% 27% 23.60% 19.40% Teyel (Kolda) 2009- 2010 2J Urban Establishment 214 135 79 - -41% 17.80% 43.00% 2012- 2013 Entry 248 157 91 34 16% -42% 21.70% 27.50% 2014- 2015 End of project 260 167 93 46 21% -44% 0.00% 0.00% CEM 3 Sédhiou (Sédhiou) 2013- 2014 1K Urban Entry 830 469 361 - - -23% 26.40% 25.50% 2014- 2015 End of project 753 387 366 -77 -9% -5% 25.30% 29.00% San Samba (Sédhiou) 2013- 2014 2K Rural Entry 240 179 61 - - -66% 36.30% 36.10% 2014- 2015 End of project 263 161 102 23 10% -37% 31.10% 21.60% Koussy (Sédhiou) 2008- 2009 3K Rural Establishment 458 306 152 - - -50% 19.90% 9.90% 2013- 2014 Entry 497 307 190 39 9% -38% 53.10% 50.50% 2014- 2015 End of project 467 281 186 9 2% -34% 36.70% 42.50% USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 113 Percentage change in school population since the year of establishment Percentage change in girls' population since the year of establishment 70% 190% 8% 20% 9% 156% 130% 264% 75% 167% 119% 157% 16% 21% 9% 2% Entry constructions End of project Entry constructions End of project Entry constructions End of project Entry constructions End of project Entry constructions End of project Entry constructions End of project Entry constructions End of project Entry constructions End of project Mbao Kamb CEM 3 Fatick Zone Sud Mbour Merina Dakhar Ngothie Boynadji Teyel Koussy (urban) (Urban) (Urban) (Rural) (Rural) (Rural) (Urban) (Rural) 83% 236% 26% 43% 40% 164% 157% 265% 70% 208% 151% 247% 78% 18% 25% 22% Entry constructions End of project Entry constructions End of project Entry constructions End of project Entry constructions End of project Entry constructions End of project Entry constructions End of project Entry constructions End of project Entry constructions End of project Mbao Kamb CEM 3 Fatick Zone Sud Mbour Merina Dakhar Ngothie Boynadji Teyel Koussy (Urban) (Urban) (Urban) (Rural) (Rural) (Rural) (Urban) (Rural) USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 114 ANNEXVII: GETTING TO ANSWERS MATRIX N° Questions d’Evaluation Type de réponse/ Evidences nécessaires (Cochez un ou plusieurs, si approprié) Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entrevues avec des informateurs clés, Mini￾enquête26 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Méthodes d’analyse des données Fréquence des Distributions, Analyses des tendances, Analyse Source(s) de données Méthode des Contenus 1 Quelles sont les évidences de la pertinence de l’activité FARA dans le renforcement des capacités du système éducatif ? Oui/Non Les acteurs du MEN (DCS – le Directeur, le Chargé de l’Activité FARA, DPRE – le Directeur, DEMSG – le Directeur, DAGE) Ministère de l’Economie et finance (MEF) – Directeur de l’investissement L’USAID/Education Principal du collège Entrevues d’informateur clé (semi-dirigé) ; Fiche de statistique (nationale) ; Entrevue d’informateur clé (semi-dirigé) Entrevue d’informateur clé (semi-dirigé) 4 1 1 1 12 Analyse des contenus : Identification des thèmes, regroupement des réponses par thème, identification des tendances qui se dégagent pour chaque thème, identification des convergences et des divergences ; identification des citations clés ; Triangulation (points de vue de différents acteurs) X Description Comparaison27 X Explication28 26 Les données issues des évaluations sont des livrables et les méthodes doivent indiquer comment les données seront collectées. Par exemple, pour les focus groups, l’USAID requiert une transcription. 27 Comparaison – avec la situation de référence, programmes/cibles, ou avec d’autres standards ou normes 28 Explication –pour les questions “pourquoi” ou à propos de l’attribution d’un effet à une intervention spécifique (causalité) USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 115 N° Questions d’Evaluation Type de réponse/ Evidences nécessaires (Cochez un ou plusieurs, si approprié) Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entrevues avec des informateurs clés, Mini￾enquête26 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Méthodes d’analyse des données Fréquence des Distributions, Analyses des tendances, Analyse Source(s) de données Méthode des Contenus L’Inspecteur (L’IA) L’Inspecteur (L’IEF) Le bureau de l’APE Le CGE Entrevue d’informateur clé (semi-dirigé) ; Fiche de statistique (collège); Fiche récapitulative Entrevue d’informateur clé (semi-dirigé) ; Fiche de statistiques - région) Entrevue d’informateur clé (semi-dirigé) ; Fiche de statistiques - département Focus groupe (semi-dirigé) Focus groupe (semi-dirigé) 23 23 A varier A varier 12 12 Analyse des contenus ; Traitement statistique – génération des tableaux et des graphiques Analyse des contenus ; Traitement statistique Analyse des contenus ; Traitement statistique Analyse des contenus Analyse des contenus Triangulation USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 116 N° Questions d’Evaluation Type de réponse/ Evidences nécessaires (Cochez un ou plusieurs, si approprié) Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entrevues avec des informateurs clés, Mini￾enquête26 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Méthodes d’analyse des données Fréquence des Distributions, Analyses des tendances, Analyse Source(s) de données Méthode des Contenus 2 Did the use of the FARA mechanism strengthen the capacity of the MEN in the following areas? School construction program design School construction program planning Procurement School construction program implementation and monitoring Quality control and maintenance Payment capacity and processes X X X X Les acteurs du MEN (DCS – le Directer, le Chargé de l’Activité FAR Ministère de l’Economie et finance (MEF Explication Les acteurs du MEN (DCS – le Directeur, le Chargé de l’Activité FARA, DPRE – le Directeur, DEMSG – le Directeur, DAGE￾Directeur, DCEF, DI) Key informant interview Document review – PFMRAF MEN assessment (interview guide) Analyse des contenus; comparison et contrast des information, convergence des reponses. Triangulation entre information du MEN ; USAID et financial risk assessment. 3 Dans quelle mesure les actions de mobilisation communautaire mises en œuvre dans le cadre du FARA ont-elles favorisé la participation et l’appropriation par les communautés bénéficiaires des ouvrages réalisés ? Oui/Non Principal du collège Le bureau de l’APE Le CGE Entrevue d’informateur clé (semi-dirigé); Fiche statistiques ; Fiche récapitulative Focus-groupe (semi-dirigé) Focus-groupe (semi-dirigé) 12 12 12 12 Analyse des contenus ; Traitement statistique Analyse des contenus Analyse des contenus X Description Comparaison Explication USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 117 N° Questions d’Evaluation Type de réponse/ Evidences nécessaires (Cochez un ou plusieurs, si approprié) Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entrevues avec des informateurs clés, Mini￾enquête26 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Méthodes d’analyse des données Fréquence des Distributions, Analyses des tendances, Analyse Source(s) de données Méthode des Contenus Quelles sont les forces et les limites de ce mécanisme d’appropriation communautaire par rapport à la maintenance et à la gestion des ouvrages réalisés ? Leaders communautaires (imams, associations des femmes, associations des jeunes, association villageoises de développement) Les acteurs du MEN (DEMSG – le Directeur, DCS – le Directeur, le Chargé de l’Activité FARA, DPRE – le Directeur, DAGE – le Directeur) Focus-group (semi-dirigé) Entrevues d’informateur clé (semi-dirigé) ; Fiche de statistique (nationale) ; Rencontre d’échange 4 1 Analyse des contenus Triangulation 4 Dans quelle mesure l’activité FARA a-t-elle favorisé la collaboration entre les différents acteurs impliqués ? Oui/Non Les acteurs du MEN (DCS – le Directeur, le Chargé de l’Activité FARA, DPRE – le Directeur, DEMSG – le Directeur, DAGE – le Directeur) Comité de pilotage/Rencontre d’échange Entrevues d’informateur clé (semi-dirigé) ; Fiche de statistique (nationale) Focus-groupe (semi-dirigé) 4 1 1 Analyse des contenus Analyse des contenus USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 118 N° Questions d’Evaluation Type de réponse/ Evidences nécessaires (Cochez un ou plusieurs, si approprié) Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entrevues avec des informateurs clés, Mini￾enquête26 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Méthodes d’analyse des données Fréquence des Distributions, Analyses des tendances, Analyse Source(s) de données Méthode des Contenus Ministère de l’Economie et finance (MEF) L’USAID/Education L’Inspecteur (L’IA) L’Inspecteur (L’IEF) Les autorités administratives (le préfet/-sous-préfet ; Cadre de consultation de développement) Leaders communautaires (imams, associations des femmes, associations des jeunes, association Entrevue d’informateur clé (semi-dirigé) Entrevue d’informateur clé (semi-dirigé) Entrevue d’informateur clé (semi-dirigé) ; Fiche de statistiques - région) Entrevue d’informateur clé (semi-dirigé) ; Fiche de statistiques - département Visite de courtoisie/ Entrevue d’informateur clé (semi-dirigé) Focus-group (semi-dirigé) 1 1 A varier A varier 12 12 Analyse des contenus Analyse des contenus Analyse des contenus ; Traitement statistique Analyse des contenus ; Traitement statistique Analyse des contenus Analyse des contenus USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 119 N° Questions d’Evaluation Type de réponse/ Evidences nécessaires (Cochez un ou plusieurs, si approprié) Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entrevues avec des informateurs clés, Mini￾enquête26 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Méthodes d’analyse des données Fréquence des Distributions, Analyses des tendances, Analyse Source(s) de données Méthode des Contenus villageoises de développement) Les autorités locales (le maire, le Président du département, Chef du village/Chef de quartier) Principal du collège Les enseignants Le bureau de l’APE Le CGE Visite de courtoisie / Entrevue d’informateur clé (semi-dirigé) Entrevue d’informateur clé (semi-dirigé) Focus-groupe (semi-dirigé) Focus-groupe (semi-dirigé) Focus-groupe (semi-dirigé) 12 12 12 (Hommes et femmes - 5 ou 6 personnes) 12 12 Analyse des contenus Analyse des contenus Analyse des contenus Analyse des contenus Analyse des contenus Triangulation 5 Dans quelle mesure l’Activité FARA a favorisé l’accès et le maintien des Oui/Non Elèves (filles, garçons) Focus groupes (filles ; garçons) (semi-dirigé) (de préférence - entretien 24 (12 filles, 12 garçons) Analyse des contenus X Description X Comparaison USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 120 N° Questions d’Evaluation Type de réponse/ Evidences nécessaires (Cochez un ou plusieurs, si approprié) Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entrevues avec des informateurs clés, Mini￾enquête26 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Méthodes d’analyse des données Fréquence des Distributions, Analyses des tendances, Analyse Source(s) de données Méthode des Contenus filles et des garçons au cycle d’enseignement moyen ? Quelles en sont les leçons apprises ? X Explication Leaders communautaires (imams, associations des femmes, associations des jeunes, association villageoises de développement) L’Inspecteur (L’IA) L’Inspecteur (L’IEF) individuel si pas possible de convoquer un groupe d’élèves - -8 élèves (filles et garçons) – 4ème et 3ième en fonction de la disponibilité, mélange élèves habitant loin du collège et près du collège Focus-groupe (semi-dirigé) Entrevue d’informateur clé (semi-dirigé) ; Fiche de statistiques - région Entrevue d’informateur clé (semi-dirigé) ; Fiche de statistiques – département 12 A varier A varier Analyse des contenus Analyse des contenus ; Traitement statistique Analyse des contenus ; Traitement statistique USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 121 N° Questions d’Evaluation Type de réponse/ Evidences nécessaires (Cochez un ou plusieurs, si approprié) Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entrevues avec des informateurs clés, Mini￾enquête26 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Méthodes d’analyse des données Fréquence des Distributions, Analyses des tendances, Analyse Source(s) de données Méthode des Contenus Principal du collège Le bureau de l’APE Le CGE Les acteurs du MEN (DCS – le Directeur, le Chargé de l’Activité FARA, DPRE – le Directeur, DEMSG – le Directeur, DAGE – le Directeur) Entrevue d’informateur clé (semi-dirigé); Fiche statistiques Focus-groupe (semi-dirigé) Focus-groupe (semi-dirigé) Entrevues d’informateur clé (semi-dirigé) ; Fiche de statistique (nationale) 12 23 12 12 4 1 Analyse des contenus ; Traitement statistique Analyse des contenus Analyse des contenus Analyse des contenus Analyse des contenus Analyse des contenus ; Traitement statistique Triangulation USAID/SENEGAL (MEP) - Cadre de travail pour l’évaluation de performance du FARA 122 ANNEX IX: DATA COLLECTION TIMELINE Lundi Mardi Mercredi Jeudi Vendredi Samedi Dimanche 27 juin Formation d’enquêteurs, Testing, Révision d’outils, Elaboration du manuel d’enquête 28 juin Formation d’enquêteurs, Testing, Révision d’outils, Elaboration du manuel d’enquête 29 Juin Déploiement des équipes Collecte (Phase 1 – Les régions hors du sud) 30 Juin Collecte (Phase 1 – Les régions hors du sud) 1 juillet Collecte (Phase 1 – Les régions hors du sud) 2 juillet Collecte (Phase 1 – Les régions hors du sud) 3 juillet Collecte (Phase 1 – Les régions hors du sud) Retour des équipes sur Dakar 4 juillet Collecte (Phase 1 – Les régions hors du sud) Retour des équipes sur Dakar/ Entretiens – MEN, MEF 5 juillet Traitement des premières données 6 juillet La Korité Traitement des premières données 7 juillet Entretiens – L’USAID, MEN, MEF Traitement des premières données 8 juillet Entretiens – L’USAID, MEN, MEF Réunion – Partage d’expériences et analyses préliminaires Traitement des premières données 9 juillet 10 juillet Déploiement des équipes vers le sud Collecte (Phase 2 – Les régions du sud) 11 juillet Collecte (Phase 2 – Les régions du sud) 12 juillet Collecte (Phase 2 – Les régions du sud) 13 juillet Collecte (Phase 2 – Les régions du sud) 14 juillet Collecte (Phase 2 – Les régions du sud) 15 juillet Les équipes reviennent à Dakar 16 juillet Saisie, Analyses préliminaires Préparation de tableau Résultats, Conclusions, Recommandations 17 juillet 123 ANNEX X: EVALUATION TIMELINE This timeline displays the Team Leader schedule for the initial evaluation planning as well as the data collection and initial analysis process leading up through the presentation of preliminary results. DATES ACTIVITES 20 mai 2016 Avis d’appel d’offres pour le recrutement de la structure contractante 03 Juin 2016 Date limite pour le dépôt des propositions techniques et financières 06-12 juin 2016 Dépouillement des offres, choix de la structure contractante et signature du contrat de prestation avec la structure retenue 13- 18 Juin 2016 Revue et analyse documentaire 19 juin 2016 Arrivée à Dakar du Consultant international 20 juin 2016 Rencontre de cadrage avec USAID, MEN 21-22-23 Juin 2016 Réunion technique de Planification o Définition de l’échantillon o Élaboration de la méthodologie (approches, outils et manuel d’enquête) o Elaboration du plan de travail 24 juin 2016 Réunion de partage et de validation du plan de travail et du cadre méthodologique avec l’équipe Education de l’USAID et MEN 27 juin 2016 Réviser le plan de travail ; préparation pour le collecte 28 juin 2016 Test et révision des outils et du manuel d’enquête ; Formation des enquêteurs et superviseurs 29 Juin-1 juillet 2016 Déploiement des équipes (experts, enquêteurs, superviseurs) sur le terrain ; Collecte de données de terrain 2 -8 juillet Traitement des premières données ; Collecte auprès des structures centrale 10 juillet-13 Collecte des données de terrain (2eme phase) 14-15 juillet Collecte des données auprès de structures centrale ; Saisie, analyse et interpétation des résultats ; Séance de travail : Résultats, conclusions et recommandations préliminaires 16-18 juillet 2016 Saisie, Analyse et interprétation des résultats ; Préparation de la présentation à l’USAID et MEN/MEF 19 juillet 2016 Présentation initiale de la synthèse des résultats, conclusions et recommandations de l’évaluation DOCUMENT TITLE i U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov