Madagascar McGovern-Dole International Food for Education and Child Nutrition Mianatra Project Baseline Evaluation January 2023 2 FY 21 Madagascar McGovern-Dole Food for Education Program implemented by Global Communities Baseline Evaluation Program: McGovern-Dole International Food for Education and Child Nutrition Agreement Number: FFE-687-2021-004 Funding Year: Fiscal Year 2022 Project Duration: 2021-2026 (5 years) Implemented by: Global Communities Evaluation Report Authored by: Around the World (ATW) Madagascar Miranto Andrianjavony, Lead Researcher, ATW Michel Rosseau, Technical Advisor/Assessment and Measurement Specialist Alice Michelazzi, Technical Advisor/Evaluation Specialist DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov 3 Contents List of Acronyms................................................................................................................................................7 Executive Summary...........................................................................................................................................8 Project Background and Purpose..................................................................................................................8 Evaluation Design and Methods...................................................................................................................8 Evaluation Limitations...................................................................................................................................9 Findings and Recommendations.................................................................................................................11 1. Introduction and Purpose ...........................................................................................................................17 1.1. Project Context ...............................................................................................................................17 1.2. Project Description..........................................................................................................................19 1.3. Results Framework .........................................................................................................................20 1.4. Purpose of the Evaluation...............................................................................................................20 2. Evaluation Design and Methodology..........................................................................................................21 2.1. Evaluation Questions......................................................................................................................21 2.2. Evaluation Team..............................................................................................................................25 2.3. Evaluation Design............................................................................................................................26 2.3.1. Data Collection Methods........................................................................................................26 2.3.2. Data Collection Procedures ....................................................................................................29 2.4. Sampling..........................................................................................................................................30 2.4.1. Quantitative Sample ...............................................................................................................30 2.4.2. Qualitative Sample..................................................................................................................34 2.5. Confidentiality and Privacy .............................................................................................................35 2.6. Data Collection Quality Control ......................................................................................................36 2.7. Data Analysis Methods ...................................................................................................................36 2.7.1. Quantitative Analysis..............................................................................................................36 2.7.2. Qualitative Analysis.................................................................................................................37 2.8. Evaluation Limitations.....................................................................................................................37 3. Findings.......................................................................................................................................................38 3.1. Characteristics of Sample................................................................................................................39 3.2. Key Project Indicators from PMP....................................................................................................42 3.2.1. Key Results on Improved Literacy of School-Age Children (SO1)...........................................43 3.2.2. Key Results on Use of Health, Nutrition, and Dietary Practices (SO2) ...................................51 3.3. Findings on Baseline Evaluation Questions ....................................................................................56 4 3.3.1. Relevance and Coherence.......................................................................................................56 3.3.2. Effectiveness...........................................................................................................................60 3.3.3. Efficiency .................................................................................................................................62 3.3.4. Sustainability...........................................................................................................................62 4. Conclusions .................................................................................................................................................65 4.1. Improved Literacy of School-Aged Children ...................................................................................65 4.2. Increased Use of Health, Nutrition, and Dietary Practices.............................................................67 4.3. Project Relevance, Coherence, Effectiveness, Efficiency, & Sustainability ....................................67 5. Recommendations......................................................................................................................................68 5.1. Recommendations for Program Strategy and Implementation.....................................................68 5.2. Recommendations for Monitoring and Evaluation ........................................................................69 6. Annexes…………………………………………………………………………………………………………………..Separate Attachment 5 List of Figures Figure 1: Map of Operational Area of the Mianatra Project .......................................................................... 17 Figure 2: Percentage of students reaching Reading Comprehension benchmark per group and gender of the student...................................................................................................................................................... 44 Figure 3: Percentage of students reaching Oral Reading Fluency benchmark per group and gender of the student............................................................................................................................................................ 44 Figure 4: Average fluency scores on EGRA subtask by group......................................................................... 45 Figure 5: Average percent correct scores on EGRA subtask by group ........................................................... 45 Figure 6: Proportion of zero-scores on EGRA subtasks by group................................................................... 46 Figure 7: Percentage of students who are off task per group and gender of the student ............................ 48 Figure 8: Percentage of parents in VSL groups with primary school aged children who report they are financially able to send their child to school by group and gender of the parent ......................................... 49 Figure 9: Percentage of parents that report their child had a health-related absence from school in the prior 2 weeks by group and gender of the parent.......................................................................................... 49 Figure 10: Underlying condition/illness causing student health-related absenteeism in the past two weeks, as reported by parents.................................................................................................................................... 50 Figure 11: Percentage of parents/guardians who can identify three benefits of children completing primary school by group and gender of the parent..................................................................................................... 50 Figure 12: Percentage of parents/guardians reporting each of the benefits of completing primary school by group............................................................................................................................................................... 51 Figure 13: Percentage of parents/guardians who report their child has diarrhea, fever, vomiting, or cough in the past 2 weeks by group and gender of the parent ................................................................................ 52 Figure 14: Percentage of parents reporting each of health problems in the past 2 weeks by group............ 52 Figure 15: Percentage of students who can identify at least three important health/hygiene practices by group and gender of the student.................................................................................................................... 53 Figure 16: Percentage of students reporting each of the health/hygiene practices by group ...................... 53 Figure 17: Percentage of students who can identify at least three of the five critical moments for handwashing by group and gender ................................................................................................................ 54 Figure 18: Percentage of students reporting each of the five critical moments for handwashing by group 55 Figure 19: Percentage of students who can identify at least three important nutrition/dietary practices by group and gender............................................................................................................................................ 55 Figure 20: Percentage of students reporting each nutrition/dietary practices by group .............................. 56 Figure 21: Percentage of grade 4 students in project schools who eat at home........................................... 59 6 List of Tables Table 1: Evaluation limitations and mitigation strategies................................................................................9 Table 2: PMP Indicators baseline value .........................................................................................................11 Table 3: Evaluation questions.........................................................................................................................21 Table 4: PMP Indicators..................................................................................................................................25 Table 5: Methods by respondent and purpose ..............................................................................................26 Table 6: Sample design and margin of error (ME)..........................................................................................31 Table 7: Distribution of control and treatment schools by district................................................................32 Table 8: Sample EGRA.....................................................................................................................................33 Table 9: Sample attentiveness observation and student survey....................................................................33 Table 10: Sample parent survey .....................................................................................................................34 Table 11: Summary of key informant interviews completed .........................................................................34 Table 12: Summary of focus group discussions completed............................................................................35 Table 13: Evaluation limitations and mitigation strategies............................................................................37 Table 14: School characteristics by project and control.................................................................................39 Table 15: Summary statistics for school directors..........................................................................................39 Table 16: Summary statistics for grade 3 students ........................................................................................40 Table 17: Summary statistics for Grade 4 students........................................................................................41 Table 18: Summary statistics for parents .......................................................................................................41 Table 19: Key Mianatra indicators..................................................................................................................42 Table 20: Average scores on EGRA subtasks by gender and group................................................................46 Table 21: Percentages of zero-scores on EGRA subtasks by gender and group ............................................47 Table 22: Student enrollment versus student attendance on day of surveys................................................60 7 List of Acronyms Acronym Full Term AAM Assessor Accuracy Measure ATW Around The World CISCO School district division (Circonscription Scolaire) CRS Catholic Relief Services CWD Children with disabilities DiD Difference in Difference DREN Education regional division (Direction Régionale de l'Education Nationale) EGRA Early Grade Reading Assessment EPP Primary Public School (Ecole Primaire Publique) FEFFI School Management Committees (Farimbon'Ezaka ho Fampandrosoana ny Fampianarana Ifotony) FGD Focus Group Discussion FRAM Parents’ Association (Fikambanan'ny Ray amandrenin'ny Mpianatra) GDP Gross Domestic Product GOM Government of Madagascar KII Key Informant Interview LRP Local and Regional Procurement ME Margin of Error MGD McGovern-Dole Food For Education MEN Ministry of National Education (Le Ministre de l'Éducation Nationale) NSFP National School Feeding Program OECD Organization for Economic Co-operation and Development PMP Performance Monitoring Plan PSE Sectorial plan for education (Plan Sectoriel de l’Education) SOW Statement of Work TOR Terms Of Reference USAID United States Agency for International Development USDA United States Department of Agriculture VSL Village Savings and Loan WASH Water, Sanitation, and Hygiene WFP World Food Programme ZAP Education commune division (Zone Administrative Pédagogique) 8 Executive Summary Project Background and Purpose Global Communities is implementing the first U.S. Department of Agriculture (USDA)-funded McGovern-Dole International Food for Education and Child Nutrition (McGovern-Dole) project in Madagascar, called Mianatra (Malagasy for “Learn”). Global Communities leads a consortium with sub-recipients Catholic Relief Services (CRS), FHI 360, and local partner Malagasy Mahomby, to implement Mianatra and build a five-year alliance with the Government of Madagascar (GOM) to contribute to the achievement of GOM goals of improved student literacy and health and expansion of their National School Feeding Program (NSFP). Mianatra is budgeted at US$25,000,000 and will target 130 schools in the Beloha and Tsihombe districts within the Androy region. Over the life of the project, 90,136 direct beneficiaries, including 64,177 preschool and primary students, will benefit from activities focused on the improvement of students’ literacy, access to food through school feeding, health and nutrition, school water, sanitation, and hygiene (WASH), and supporting systems. Emphasis will be placed on inclusion, equity, and appropriate accommodation to address issues related to gender and children with disabilities (CWD). Mianatra will build clear pathways to sustainable school feeding for lasting impact and benefits of improved education, enrollment, and attendance. This baseline study is the first of three surveys that constitute the overall performance evaluation. Follow-on surveys include the mid-term and endline evaluations which will help to assess overall impact of the project. The purpose of the baseline evaluation is to achieve the following broad objectives: 1) establish baseline indicator values against which to measure progress throughout the life of the project, 2) provide recommendations to maximize the efficiency and effectiveness of project monitoring, 3) refine performance indicator targets and generate contextual data to tailor program activities to school and community needs, 4) identify opportunities and threats to implementation, 5) develop recommendations for strategies to maximize strengths, mitigate challenges, and leverage opportunities, assess beneficiaries’ knowledge and skills to refine capacity building and training plans, and 6) validate and/or refine project strategies and assumptions. The main audience for this evaluation is Global Communities and its implementing partners, including Madagascar’s relevant ministries, CRS, FHI 360, as well as the donor, USDA. Evaluation Design and Methods The evaluation questions are based on the relevant Organization for Economic Co-operation and Development (OECD) evaluation lenses: relevancy, coherence, efficiency, effectiveness, and sustainability) to use in follow-on discussions on how project implementation may be adjusted and improved. Twenty detailed evaluation questions can be found within the main report. The baseline evaluation employs a quasi-experimental approach and benefits from a convergent mixed-methods approach in which qualitative data from key informant interviews (KIIs), focus group discussions (FGDs) and document review as well as quantitative data from Early Grade 9 Reading Assessment (EGRA)1 , classroom observation, students survey and parents survey (for health and nutrition indicators); to respond to the evaluation questions. A total of 90 schools (60 project schools and 30 control schools), selected via two-stage cluster sampling, made up the quantitative sample. Respondents for qualitative interviews were purposefully selected based on the roles of key stakeholders, following guidance indications from Global Communities. In districts where focus groups were conducted, participants (parents of students) were randomly selected based on school enrollment records. A total of 592 grade three students (420 treatment, 172 control) were assessed for literacy using Early Grade Reading Assessment (EGRA); 655 grade four students (467 treatment, 188 control) were observed in the classroom for attentiveness2 using tools adapted from the World Bank Stallings classroom observation method3 and participated in a student survey; 1,378 parents (757 treatment, 621 control) were surveyed on health and nutrition indicators; and a school-level inventory was administered with the school directors in all 90 schools (60 treatment and 30 control). A total of 24 Qualitative data collection activities (eight FGDs and 16 KIIs) were conducted Data collection took place over a period of five weeks in September and October 2022 by an external evaluation team contracted by Global Communities. Qualitative and quantitative data collection occurred simultaneously, though quantitative data collection was more intensive. School-level data collection began on September 13 and continued through October 07, lasting four weeks. The evaluation team began KIIs with stakeholders on September 13 and was able to complete the qualitative part on October 13. Evaluation Limitations The baseline evaluation process encountered several challenges and limitations. Table 1 presents these issues alongside mitigation strategies employed by the evaluation team. Table 1: Evaluation limitations and mitigation strategies Limitation Mitigation Strategy 1 This project used the version of the EGRA tool that was adapted and validated by the MEN, with support from FHI360, in 2020. This work in 2020 resulted in the development and validation of two new subtasks: one on passages and comprehension (listening, oral reading fluency and comprehension) and one on silly sentences (which presents a number of similar sentences, one true and one false, and asks students to determine which is true). As part of the adaptation and validation process in 2020, the new subtasks were piloted by FHI360 in 10 schools. This adapted tool has been adopted by MEN and is currently considered the standard tool for assessing early grade literacy in Madagascar, including periodic assessment by teachers of student literacy. 2 Based on USDA’s standard results framework which links reduced short-term hunger (MGD 1.2.1) with the result Improved Attentiveness (MGD 1.2), the observation tool was used in grade 4 classrooms, since grade 4 students also were selected to participate in the student survey, which assessed self-reported short-term hunger as well as other health and hygiene related knowledge attitudes and practices and to enable analysis of the association between short-term hunger and attentiveness. See the Methods section of the Baseline report for additional detail. 3 The Stallings classroom observation system is a questionnaire and protocol for timed observations that produce quantitative data about interactions of teachers and students in classrooms. It was initially developed to observe basic education classrooms in the United States. Its advantages are that it requires only basic training to administer and produces data with a high degree of reliability between raters. Stallings data is comparable internationally, and it has been validated in a number of developing countries. https://www.worldbank.org/en/programs/sief-trust￾fund/brief/the-stallings-classroom-snapshot 10 Design limitation: The quasi-experimental evaluation design, chosen due to the impossibility of identifying a true control group, does not allow for direct impact measures against a counterfactual Statistical analysis will be used at the midline and endline to control for heterogeneity of the treatment and comparison group using DiD approach with propensity-score matching control. Design limitation: The baseline evaluation assessed grade 3 students at the beginning of the school year as a proxy for grade 2 students at the end of the school year. This might produce a bias in the results as the sample did not include any grade 2 students who did not pass to the next grade at the end of the previous school year. As the focus of the study is to evaluate the impact of the Mianatra project, future rounds of data collection will be carried out at the same time period with grade 3 students. Response bias from stakeholders: The baseline evaluation targets key stakeholders who play an active role in the Mianatra program. Many have an interest in seeing the program succeed, and those most engaged with partners may be positively biased toward its programming, particularly if they are currently receiving financial support. The evaluation team mitigates this bias by stressing confidentiality and the importance of objectivity in improving the program. Additionally, the evaluation team calls upon a diverse group of stakeholders and multiple data sources to triangulate findings to answer the evaluation questions. Selection/sampling bias for students and parents: It is often the case that students who are present at school may represent those that are more committed to schooling as well as students with more support at home, those who live closer to the school, and those who have less additional demands on their time. There may be specific gender-specific factors that influence student attendance (e.g. girls who may carry relatively more burden in household chores and child caretaking, or boys who may be more likely to be involved in informal employment or labor, which can each influence student attendance. Furthermore, as described in the Sampling section, parents surveyed reflected parents available on the day of the survey; this could have resulted in a bias towards parents who live closer to the school (and potentially have greater SES/means to send children to school, involvement and or value in education, and/or other factors.) Finally, the prioritization of mothers over fathers in the parent survey limits the ability to understand the difference between mothers and fathers. Analysis of students sampled pays close attention to gender as well as the socio-economic status to help address these concerns. This limitation can be addressed by using post-stratification weighting to correct for any underrepresentation of the characteristics of the population. While the potential biases in the recruitment of parents for the parent survey were due to survey implementation feasibility, it’s noteworthy that parents overall were oversampled which may mitigate this somewhat; additional findings from the gender assessment may provide the important with additional complementary information around gender roles and differences in knowledge, perceptions, and behaviors between men and women in the project. 11 Selection/Sampling bias: As with most qualitative research, because of purposive sampling, the results are not generalizable. This limitation is a characteristic of a qualitative approach. Analysis accentuates that qualitative findings, while not strictly representative, may be transferable and connect to the experiences of others, thereby providing valuable insights applicable to other stakeholders. Attentiveness evaluation: Gauging student attentiveness required classroom observation by trained enumerators. This approach may cause a distraction for the students in the room. The attendance rate in class during the data collection was very low and there was a fairly low number of students observed. This is a very common issue with classroom observation. The evaluation team mitigates it through triangulation and asking questions about student attention to teachers during FGDs. Findings and Recommendations Key project indicators from the Performance Monitoring Plan The baseline evaluation report provides the baseline value of ten key project indicators collected during baseline as part of the impact evaluation. Literacy levels were low, with only 6% of grade 3 students demonstrating they could read and understand the meaning of grade level text. During observation of Grade 4 classes to assess student attentiveness, 44% of students were off task, meaning that they were observed to be inattentive at least one time during the 30-minute observation period, which consisted of 10 one-minute observations with a 2-minute interval. More than half (54%) of parents were able to identify three benefits of children completing primary school. Though only 28% of parents surveyed reported being current active members of saving and lending groups, of these, the vast majority (84%) declared that they were financially able to send their children to school. Students face many health-related challenges: 22% of parents surveyed reported that their children had a health-related absence from school in the previous two weeks. Furthermore, 73% of parents reported their children had diarrhea, fever, vomiting, or cough in the past 2 weeks. With regards to student knowledge of Health, Nutrition, and Dietary Practices, 30% of students surveyed were able to identify at least three important health/hygiene practices, but fewer students (15%) were able to identify at least three of the five critical moments for handwashing. The percentage of students able to identify at least three important nutrition/dietary practices was also low (16%). The baseline values from the Performance Monitoring Plan (PMP) are provided in Table 2 for the ten key indicators of the project. Table 2: PMP Indicators baseline value Indicators Baseline value MGD SO/RESULTS FRAMEWORK: Improved Literacy of School-Age Children 1. Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text. 6.5% 12 2. Percent of students who are off task (data collected in a single day, at specific intervals). 44.5% 3. Percent of savings and lending group members with primary-school aged children who report they are financially able to send their child to school. 84.0%4 4. Percent of parents that report their child had a health￾related absence from school in the prior 2 weeks. 22.3% 5. Percent of parents/guardians who can identify three benefits of children completing primary school. 53.9% MGD SO2/PROJECT RESULTS FRAMEWORK: Increased Use of Health, Nutrition, and Dietary Practices 6. Percent of parents who report their child has had diarrhea, fever, vomiting, or cough in the past 2 weeks. 73.5% 7. Percent of students in targeted schools who can identify at least three important health/hygiene practices. 30.1% 8. Percent of students in targeted schools who can identify at least three of the five critical moments for handwashing. 15.3% 9. Percent of students in targeted schools who can identify at least three important nutrition/dietary practices. 16.5% LRP SO1/RESULTS FRAMEWORK Improved Effectiveness of Food Assistance through Local and Regional Procurement 10. Percent of students who report liking school meals, using a 5-point facial hedonic scale. 0.0%5 Relevance and Coherence Through its education sector plan which has been in place since 2018, the Government of Madagascar is targeting three areas of education development: access, quality, and governance. The priorities are therefore to increase school enrolment, have better educational outcomes, and decrease the rate of student dropout. Water, sanitation, and hygiene in the public schools are also among the priorities of the Government by installing the program “Ecole amie du WASH” or “WASH friendly school” in collaboration with the Ministry of Public Health. Stakeholders interviewed from the Ministry of National Education (Le Ministre de l'Éducation Nationale, MEN) affirmed that to achieve these objectives, school feeding is at the center of the ministry's activities at the primary school level with the national school feeding program or “Programme Nationale d’Alimentation Scolaire.” They also stated that the McGovern-Dole project helps them partly overcome the lack of financial means which prevents the MEN from establishing a school feeding program in all schools. Key informants further agreed that hunger is one of the major causes of inattentiveness in class, poor school results, and school dropout. This 4 This result is only from the 28% of parents surveyed who reported that they are saving and lending group members. 5 Note that this result of zero is due to no schools currently serving school meals. 13 was further reinforced by survey results from Grade 4 students, with the majority (57%) of Grade 4 students surveyed rarely take or never have breakfast before going to school, though the majority did report eating at home in the evening. In FGDs with parents, respondents most often mentioned the lack of money as the main reason limiting the accessibility of education for school-aged children. Given the project’s focus to work in all communities to establish and support new and existing VSL groups, respondents believed the project will be relevant and responsive to improve the financial capacity of parents. In FGDs with teachers, training is the most expressed need from teachers. Most teachers have never received any training to be able to teach properly. The evaluation team observed in the field that, in many schools, there is only one qualified teacher for many grades, while the others are from parent associations. Effectiveness While the project was designed while COVID-19 related restrictions were still in place, the situation in Madagascar is now normalized and no restrictions are in place regarding school attendance. The project developed a set of activities to be implemented based on the learnings about remote schooling during COVID-19 restrictions and its response, in case of a future lockdown, will be guided by the experience acquired while implementing other McGovern-Dole projects in other countries. The project faces and will need to work to overcome several challenges for overall effectiveness of the program. One significant barrier highlighted during field work relate to the conditions of schools’ infrastructures in the area of intervention. Given the significant issues with water scarcity in the region and its impact on schools, household economic security, and agricultural production, implementers recognized water scarcity as one of the most significant challenges the project faces. Many communes in project districts do not have drinking water and in some places, they cannot even find water. Another considerable challenge is identifying the number of students attending schools within the project implementation area. During field work, the evaluation team found only a fraction of the official number of students in the last school year shared by district officials present at school. Some stakeholders noted the lower levels of existing skills and instructional competencies among teachers, which is most likely explained by the lack of formal education and training among most teachers before teaching, particularly community teachers. Stakeholders also noted challenges around historical misappropriation of project resources that other implementing partners have faced. The project implementer and its partners have a long and established experience in the implementation of McGovern-Dole projects in other countries, and therefore procedures to adapt monitoring tools already tested in other contexts are ongoing and running smoothly. Sustainability The project will focus on creating a lasting impact by strengthening existing systems, structures, and capacities to gradually hand over Mianatra’s integrated school feeding program to the GOM and communities. Through community sensitization and mobilization, the project plans to co￾create a feasible transition strategy and planning and measurement tools with project stakeholders. Local capacity strengthening is embedded in all activities to ensure the mentorship of stakeholders across various ministries and at all levels supports the gradual transition of ownership of key Mianatra interventions to these stakeholders. While Mianatra does not 14 anticipate fully graduating schools within this five-year project, it does seek to assist progression toward graduation. Working in close partnership with the GOM, the project aims to develop and implement a successful and sustainable integrated school feeding model to help inform policy and improve the GOM’s NSFP. Respondents highlighted that the project is working in a drought-prone area and that drought is the biggest threat in the South. Due to food insecurity, there are many other cash transfer and food distribution projects also working in the region. The WASH component of the program plans to rehabilitate existing water infrastructures and install new infrastructures, such as drillings and impluvium (water catchment systems), as much as possible, which should remain beyond the life of the program. Respondents highlighted the importance of the project to engage local entities in the management of this infrastructure and strengthen entities under the Ministry of Water to preserve these infrastructures. Respondents also mentioned that the local climate and water problem could also demotivate farmers. Agents of the Ministry of Agriculture at the district level stated that farmers still face an unfavorable climate (notably contributed to by the prolonged drought); therefore, they are sometimes demotivated to continue farming because planting is often affected by the lack of rain. Project staff affirmed that the project's sustainability plan is complete and considers all the actors who must be engaged to ensure this sustainability. However, the evaluation team identified through the interviews other opportunities that may be at hand and can contribute to sustainability of results. This includes mobilizing groups or associations of local producers to improve their agricultural knowledge and skills and increase their production capacities to supply school feeding. The program plans to involve all key stakeholders (government, teachers, school directors, parents, community groups) to ensure progress toward a suitable national school feeding program. The government is an integral part of the program, from the MEN at the central level to the ministerial structures at the local level. All these actors will be institutionally strengthened during project activities through training, capacity building, and awareness raising. The project plans to co-create a sustainability plan and transition strategy in partnership with project stakeholders at all levels. However, because the project has recently begun, it was noted by respondents that there is not yet a defined and clear commitment made by the local actors for a supporting locally owned and managed school feeding program. Recommendations The recommendations were developed by the Evaluation Team based on the findings and conclusions of the baseline evaluation report to provide the project team with recommended avenues for improvement in the implementation of the project. Once the baseline report is finalized and approved by the donor (USDA), the full results will be shared with the project team and key stakeholders, recommendations fine-tuned as needed for the project and country context, and operationalized, including identifying responsible actors and defining timelines. Recommendations for Program Strategy and Implementation 1. The project must work with relevant stakeholders to confirm the number of students enrolled and actively attending project schools and identify any potential adjustments to project activities. 2. The project should map/assess the number of out of school children in target communities to identify the population of children who are eligible to attend schools but are currently not 15 enrolled and/or attending. The project may also consider collecting feedback from parents and other community stakeholders on barriers to student enrollment and attendance to identify any possible strategies or approaches the project could consider (e.g., sensitization activities for parents and communities, to enroll and/or reintegrate these out of school children into schools). If the project does pursue activities to encourage new enrollment or reintegrate out of school children into schools, the project should develop ways of monitoring and evaluating these efforts and incorporate this into sampling or analysis at midline and endline, to avoid any potential biases that may be introduced by such strategies. 3. The project should consider reviewing project plans and budget allocations and identify/develop school-specific strategies to increase the number of schools with reliable access to water to ensure the proper functioning of the canteen as well as the respect of health and hygiene practices. 4. Given the importance of removing financial barriers for students to attend school and given the relatively low current participation rates of parents in savings and loan groups, reviewing and updating the mapping of existing saving and lending groups in the project area may help to inform project activities to strengthen and/or establish new groups. 5. Prior to establishing any larger school gardens or staple farms, the project should consider updating their mapping exercise and assess potential ways the schools can access bigger plots of land that can realistically be managed and sustained. Given the increased challenges with climate change and water insecurity, this updated mapping may be useful to help the project team to review the current targets related to school gardens and farms and consider a more modest target. 6. Given the number of stakeholders involved in implementing and ensuring longer term sustainability of these efforts, including schools and communities, the GOM at all levels, and other implementing partners, the project should consider additional sensitization and dissemination activities, in addition to initial project launch activities already conducted by the project, to make sure that project strategies, approaches, expected results, and roles and responsibilities are known and understood by all project stakeholders, especially by those in charge of implementing project activities in order to increase awareness and accountability. Given the importance of the sustainability plan in the project and feedback on ensure sustainability and ownership of the activities by the stakeholders, the project should embed discussions about sustainability and co-create a gradual transition strategy at the onset of project activities at all levels (central, local, and school level). Sustainability plans should include defined roles and responsibilities of all actors and the project should ensure they are disseminated to all project stakeholders to ensure awareness, ownership, and accountability to operationalize the plans. Recommendations for Monitoring and Evaluation 1. Ensure that midline and endline studies will be conducted at the same time of the year and with the same grade (students entering grade 3) to ensure comparability of results, especially around potential seasonal variability and impact on school attendance, health related issues, and food security related issues. 2. Ensure consistency of approach in data collection and analysis of survey questions with parents and students in midline and endline, in order to ensure uniformity with baseline analysis; this 16 can be achieved through review and alignment of baseline documentation (e.g. data dictionary, field SOPs, data cleaning notes, analysis scripts, etc.). 3. The evaluation team recommends the project reviews the current indicator and indicator definition for measuring students’ attentiveness in the project’s PMP and consider some of the challenges faced in this baseline evaluation. The evaluation team identified two issues that make this indicator challenging. First, the indicator definition defines students as inattentive if there is only one instance of being off task. The evaluation team questions if one instance is enough to be considered inattentive. Second, at baseline, this indicator was collected when class sizes were small (average of 6-8 students per class). The evaluation team notes that if class sizes are bigger at midline or endline, this may affect results on student attentiveness. The project could consider revising the target or collect additional data through ongoing monitoring and implementation of programmatic activities to capture data on student attentiveness in alternative ways. However, if updates to the indicator approach are made at midline or endline, baseline and any other subsequent values would need to be recalculated (as possible) to ensure consistency over time. The evaluation team encourages the project team to work with USDA to identify how indicators on student attentiveness can be further improved. 4. Given the very low baseline values for literacy (6.5% of students in project schools meeting literacy standards), the project team should review and work with USDA to consider possible revision of targets (currently to increase by 7% by midline and 15% by endline). The evaluation team recommends a more modest target (e.g., increase by 3%) in line with experience in other countries. 17 1.Introduction and Purpose 1.1. Project Context Figure 1: Map of Operational Area of the Mianatra Project Madagascar is one of the world’s poorest countries, with an annual gross domestic product (GDP) per capita of US$515 (the 5th lowest GDP per capita in the world) in 2021. 6 Nearly 78% of the country’s population lives in poverty, 7 with this rate being considerably worse in the south of Madagascar in the Androy region (where 97% live in poverty8 ) where the Global Communities McGovern-Dole program will be implemented (See Figure 1). Household poverty is widespread throughout Madagascar and is cited as the most significant reason for not enrolling or for pulling children out of primary school.9 Climatic shocks also have a major impact on households, particularly erratic rainfall and the increasing severity of recurring droughts, making financial recovery in agriculture-dependent regions difficult.10 In the Deep South, Crisis and Emergency (Integrated Phase Classifications Phase 3 and 4) levels of food insecurity are projected through May 2023, driven by a third consecutive season of drought and below-average harvests.11 Security risks typically present due to increased food insecurity and dependency on external services have further contributed to a rise in security concerns in some areas with theft and land or inter-clan disputes. Theft from food stores/warehouses has been reported, regardless of the source of the goods. Further, high rates of food insecurity and malnutrition have exacerbated the impact of recent health shocks12, including outbreaks of measles, plague, malaria and most recently, COVID-19, which have revealed health system limitations and lack of knowledge on prevention and treatment. Systemic challenges including insufficient government funding, donor dependency, and turnover of administration staff have led to a lack of funding and government support for social programs, 6 World Bank. Indicator: GDP per capita (current US$)-Madagascar. 2021. 7 Government of Madagascar. PNAN III. 2019. 8 Ibid. 9 World Bank. Combined project information documents/integrated safeguards datasheet. 2020. 10 World Food Programme, Extreme weather and climate events heighten humanitarian needs in Madagascar and around the world, 2022. 11 FEWS NET, Madagascar Food Security Outlook, 2022. 12 UNICEF, COVID-19 Situation Report, Madagascar, 2020. 18 such as school feeding programs. As little as 2% of the GDP, or 15% of the national budget, is allotted to the MEN, which is insufficient to revitalize and expand an education system that averages one school to every 990 households.13 Key donors, such as the World Bank, World Food Programme (WFP), UNICEF, and now USDA’s McGovern-Dole program, have stepped in with much needed funding, and all projects are implemented in partnership with the MEN. The GOM’s commitment to education for all is detailed in its Sectoral Plan for Education (PSE) which declares that the MEN will oversee the national education strategy to improve preschool and primary school education, reduce grade repetition rates, extend compulsory basic education through grade nine, and mainstream children with minor to moderate disabilities in the formal education system.14 However, specific challenges remain, such as low enrollment, retention, and attendance rates, exclusion of children with disabilities (CWD), and limited teacher capacity and training contributing to poor student performance and literacy outcomes. In the Deep South, literacy rates remain at 44%, and are even lower at 26% in the Androy region15, compared to the Madagascar average of 72% for individuals aged 15 and older. 16 Twenty seven percent of children ages 5-15 in the Deep South and 48% of children ages 5-15 in the Androy region have not completed primary school, compared to the national rate of 20%.17 Additionally, the majority (76%) of adults in Tsihombe and Beloha have no formal education.18 According to the PSE, the primary school completion rate is 69% nationally, with lower rates in rural areas.19 Comparatively, only 27% of students in the Androy region graduate from primary school.20 Also in the Androy region, the school dropout rate for girls in grade 4 is 38%21 compared to the national dropout rate of 20% primarily due to household poverty, which results in pulling children out of school to reduce expenses and pushing them into labor to increase income.22 Studies on preschool education suggest that early schooling reduces the risk of repetition, promotes higher retention in primary school, and helps children maintain competencies.23, 24 Although preschool enrollment has increased significantly over recent years, nearly 60% of preschool aged children are not enrolled,25 highlighting the vast number of children stripped of their right to education.26 COVID-related restrictions have exacerbated conditions that limit enrollment, increase dropout rates, and have sparked a migration of families to seek better opportunities and food security,27 all disrupting preschool and primary school enrollment, attendance, and performance. 13 Government of Madagascar. Plan Sectoriel de l’Education (2018-2022). June 2017. 14 Government of Madagascar. Plan Sectoriel de l’Education (2018-2022). June 2017. 15 Orange Madagascar. L’Alphabétisation pour rehausser la qualité de vie à Androy. 2022. 16 Government of Madagascar. Plan Sectoriel de l’Education (2018-2022). June 2017. 17 UNICEF. Rapport MICS EAGLE Madagascar. 2022 18 Catholic Relief Services Census, 2020-2021. 19 Government of Madagascar. Plan Sectoriel de l’Education (2018-2022). June 2017. 20 UNICEF. Multiple Indicator Cluster Survey (MICS) 2018. Madagascar Drinking Water, Sanitation, and Hygiene. 2018. 21 World Bank. Combined project information documents/integrated safeguards datasheet (PID-ISDS). 2020. 22 Trading Economics. Madagascar drop-out rate from grade 3 of primary education, female. 2021. 23 Government of Madagascar. PNAN III. 2019. 24 A World Ready to Learn. UNICEF. 2019. 25 UNICEF. 1.3M children are not enrolled in pre-primary education. 2019. 26 UNICEF. Multiple Indicator Cluster Survey (MICS) on Madagascar Drinking Water, Sanitation, and Hygiene. 2018. 27 Africa News. Thousands migrate north to escape hunger and drought. February 2021. 19 Both project districts (Beloha and Tsihombe) each have a Basic Health Center (CSB) and community health workers (810 in Beloha, 1068 in Tsihombe); community health workers are responsible for general awareness raising of health and hygiene in the community as well as mobilization for vaccines, deworming and the use of treated long lasting insecticidal nets for malaria prevention, in both school and community settings. As part of services provided, student health exams are conducted at the beginning and end of each school year to monitor the state of health of the students and their growth; these include services for deworming, student growth monitoring, and micronutrient supplementation (iron/folic acid). 1.2. Project Description Mianatra (Malagasy for “Learn”), the first USDA-funded McGovern-Dole project in Madagascar, aims to build a five-year alliance with the GOM to support improved student literacy and health and expand the GOM’s NSFP. The project is led by Global Communities and implemented through a consortium with sub-recipients CRS, FHI 360, and Malagasy Mahomby (a local partner). Mianatra specifically serves food-insecure regions in the Deep South that are most vulnerable to low school enrollment, poor attendance, high dropout rates, malnutrition, and food insecurity—all of which contribute to some of the poorest education and nutrition outcomes. To address these challenges, Mianatra focuses on expanding the MEN’s efforts through complementary activities and strengthening the capacities needed to support sustainable literacy improvements and school feeding program ownership by the GOM and school communities. With many foundational pieces in place, including GOM policy commitments to an improved early grade reading curriculum and a NSFP, Mianatra is designed to augment, integrate, and coordinate resources in meeting McGovern- Dole program objectives to produce meaningful and lasting results and establish clear pathways for the GOM and communities to take ownership of sustaining the benefits of improved learning and student nutrition through school feeding. From October 2021- September 2026, Global Communities will use U.S. donated commodities and funds provided by USDA to implement a school feeding project in southern Madagascar focused on achieving the following Strategic Objectives and Foundational Results: Strategic Objective 1: Improved literacy of school-aged children: • Improved the quality of literacy instruction through the distribution of school supplies and materials, establishment of libraries, production of books and supplementary reading materials, training teachers, school administrators and school management committees, and the distribution of distanced education materials. • Improved attentiveness and reduce short-term hunger through the provision of school meals, establishing of school gardens, and local and regional procurement (LRP) of commodities for school meals. • Improved student attendance by increasing economic incentives for parents by forming savings and lending groups and increasing community understanding of the importance of education. Strategic Objective 2: Increased Use of Health, Nutrition, and Dietary Practices: 20 • Increase the use of health, nutrition, and dietary practices by providing training on good health, nutrition, and WASH practices, safe food preparation, storage, and commodity management practices, and distribution of deworming medication. • Increase access to clean water and sanitation services through the construction and rehabilitation of block latrines, washrooms, wells, and handwashing stations. Foundational Results: Additionally, the project aims to increase the capacity of government institutions, improve policy and regulatory frameworks, and increase engagement of local organizations and community groups by working with community and government structures, facilitating sustainability planning and measurement, and supporting the GOM to scale and sustain their NSFP. Mianatra is funded with US$25,000,000 and will target 130 schools in the Beloha and Tsihombe districts within the Androy region. Over the life of the project, Mianatra aims to serve 90,136 direct beneficiaries, including 64,177 preschool and primary students, who will benefit from activities focused on the improvement of students’ literacy, access to food through school feeding, WASH, and supporting systems. Emphasis will be placed on gender equity and inclusion, including a focus on CWDs. Mianatra will build clear pathways to sustainable school feeding for lasting impact and benefits of improved education, enrollment, and attendance. 1.3. Results Framework Mianatra’s Theory of Sustainable Change is based on the premise that: • IF the project is linked to established GOM plans from the onset and builds off of existing systems and structures at all levels; and • IF current literacy, health, nutrition, WASH, and food security programs from various technical and funding partners are leveraged (e.g., CRS’ USAID-funded Maharo DFSA program) and strengthened by addressing existing and potential gaps; and • IF capacity of the GOM is strengthened at the national, regional, district, and local levels to effectively support and learn from Mianatra’s integrated school feeding program through participatory evaluation and feedback mechanisms; and • IF stakeholders are empowered from project conception to plan their own sustainability strategies for gradual transition away from reliance on McGovern-Dole assistance for lasting impact; • THEN the literacy of school-aged children will be improved; the use of health, nutrition, and dietary practices will be increased; and the GOM, schools, and communities will be equipped with the capacity, vision, and motivation needed to fully transition away from reliance on USDA and other donor-provided support toward a resilient and truly sustainable NSFP. The full results framework can be found in Annex 1. 1.4. Purpose of the Evaluation This baseline evaluation is the first of three evaluations that constitute the overall performance evaluation. Follow-on surveys include the midline and endline evaluations, which help to answer questions about both performance and impact of the program. The study design and sampling strategies for the three surveys will allow for measuring the impact of the program on different indicators. Differences between proportions and means will be run in a way that allows for the 21 detection of results that are attributable to the project activities. Additionally, the baseline study provides information on relevant contextual factors that may have a bearing on the implementation of activities. Some contextual factors may include governance at the school, community, district, regional, and national levels, perceived nutritional and health needs of the beneficiaries, as well as school management committees’ needs for capacity strengthening. The baseline evaluation study also aims to examine general environmental factors, structural influences, and cultural considerations that may have influences as enablers or barriers to the proposed interventions. The findings at baseline will enable project staff to validate the design of the project and, if needed, adjust interventions and project targets. The study is designed to capture gender and social inclusion dynamics that will be factored into future programming and will be complemented by a gender assessment that will be conducted separately. The baseline evaluation has the following objectives: • Establish baseline indicator values against which to measure progress throughout the life of the project. • Provide recommendations to maximize the efficiency and effectiveness of project monitoring. • Refine performance indicator targets and generate contextual data to tailor program activities to school and community needs. • Identify opportunities and threats to implementation. • Develop recommendations for strategies to maximize strengths, mitigate challenges, and leverage opportunities and assess beneficiaries’ knowledge and skills to refine capacity building and training plans. • Validate and/or refine project strategies and assumptions. The main audience for this evaluation is Global Communities and its implementing partners, including Madagascar’s relevant ministries, CRS, and FHI 360, as well as the donor, USDA. 2. Evaluation Design and Methodology 2.1. Evaluation Questions The evaluation questions and sub-questions are based on the relevant OECD evaluation lenses (relevancy, coherence, efficiency, effectiveness, sustainability, and impact) and findings for these questions may be used in to adjust and improve project implementation. The impact criterion was not included in the baseline evaluation given it will be better suited to include in the midline and endline evaluations. Table 3 provides illustrative questions for the program and design inquiries, based on these OECD categories. Table 3: Evaluation questions OECD Evaluation Categories Project Specific Evaluation questions 22 Relevance and Coherence 1) Is the program relevant to the desired outcomes and objectives of the McGovern-Dole strategy, policy, and plan? • What are the key policies and priorities of the Government of Madagascar around sustainable school feeding, strengthening food production capacity, elementary school education, literacy, and other relevant areas? • What are the key gaps in knowledge and capacity of GOM in these areas? • To what extent are the planned interventions in line with the policies and priorities of the GOM? 2) Is the project’s theory of change realistic, appropriate, and practical to implement? 3) Are there any modifications needed to the project’s design, structure, logic, or management to facilitate the achievement of the desired outcomes and impact? 4) Are the project’s proposed strategies and interventions to increase knowledge and strengthen the capacity of key stakeholders (i.e., GOM, schools, savings groups) appropriate and realistic to achieve the expected results and outcomes? 5) Is the program relevant to address the real, and perceived needs of project beneficiaries (e.g., students, teachers, school administrators, parents/guardians, CWD, and government officials)? How well does the program complement, leverage, and align with other ongoing education, nutrition, and school feeding programs within the country and other GoM plans and priorities? • How do key stakeholders (students, teachers, school administrators, parents/guardians, CWD, government officials) describe priority needs in key areas (e.g., health & nutrition, education/literacy, school feeding, poverty, agriculture, women’s economic and social empowerment, and other development indicators)? How do these vary for women versus men or other sub-groups? • What are the major challenges in achieving progress in these areas? • How do these needs and challenges, directly and indirectly, influence the literacy outcomes in the project area? To what extent do the program objectives and outcomes address the community’s needs? 6) Is the project design suited to improving the level of early grade literacy performance and overcoming the main challenges to improving literacy outcomes? 23 7) Is the program designed to be flexible? For example, can the overall strategy, components, or specific activities be adjusted over time due to changing environments or in response to target population needs? Effectiveness 1) To what extent is the COVID-19 pandemic likely to influence program effectiveness/results and how best can the program address these influences? • What are the influences that COVID (and other relevant external factors) may have on program implementation? How can these impacts be mitigated? 2) What are the main challenges or barriers to achieving desired outcomes across other related project components (i.e., health, nutrition, and dietary practices, strengthened capacity to manage school feeding activities, improved student attendance, and attentiveness)? 3) Is there a clear understanding of roles and responsibilities by all parties involved with implementation and monitoring? • What are the perceived roles and responsibilities of all parties involved in implementation and monitoring? To what extent are these in line with program plans, and how can they be improved/clarified? 4) Are the project’s monitoring and evaluation strategies realistic, appropriate, and practical to implement? • To what extent are the results framework and program approach valid for the local context? Is the theory of change realistic, appropriate, and practical to implement? • What strategies or planned activities seemed effective for meeting the expected results and targets? Which were not? Why? • Does the design, structure, logic, or management need modifications to facilitate the achievement of desired outcomes and impact? • Are the program monitoring and evaluation strategies, including indicators and targets, suitable to monitor and support the implementation of the targeted results? If not, what are the suggested changes? Efficiency 1) How efficient and realistic is the planned allocation of resources (e.g. human resources, time, expertise, funds, etc.) to provide the necessary support and to achieve the broader program objectives? • How efficient is the project planning to engage key actors to finance program activities (e.g. contributions from governments and private entities to support school feeding and purchase education supplies)? 24 Sustainability 1) Is the project’s sustainability plan realistic and appropriate? • What are the main barriers to achieving the sustainability plan of the program? 2) What undeveloped opportunities might be present that can contribute to the sustainability of project benefits and outcomes? • Who needs to be engaged (e.g. private sector, community entities/associations, local government agencies, etc.) to ensure sustainability? 3) Does the project have the means to address threats or gaps (i.e., climate-related shocks, COVID-19) that could impact progress toward sustainability and sustained outcomes/benefits of the project? • What do local organizations and public sector entities need to strengthen to be able to continue implementing activities beyond the end of the program? 4) To what extent have stakeholders and local actors (e.g., GOM at all levels, private sector, community groups, and other organizations implementing school feeding programs) been engaged and committed to ensuring progress toward a sustainable NSFP? • What specific institutional capacity needs (e.g. for key GOM, district, community, and school level stakeholders) should the project address to foster greater host country engagement and accountability? 5) To what extent do stakeholders (e.g. GOM, communities, private sector) provide monetary or in-kind support for school feeding and complementary activities? Are these sustainable inputs? 6) What commitment has been made in support of a gradual transition toward a locally owned and managed school feeding program by the GOM, schools, and communities? 7) Are all the elements of sustainability (i.e. capacity, motivation, resources, linkages) addressed in the project’s sustainability plan? 8) What specific capacity needs for project stakeholders (e.g., GOM, parents, school administrators, school management committees) should the project address to foster greater ownership and accountability to sustain project benefits? To assess the effectiveness of the project over time, the baseline and subsequent rounds of evaluation will assess programmatic key results pathways in the results framework and their overall impact on student educational outcomes (measured through literacy) in project schools and 25 compare levels of change over time. Specifically, the evaluation will assess changes over time in the following intermediate and ultimate outcome areas: 1) How do changes in quality of literacy instruction (MGD 1.1), student attentiveness (MGD 1.2), and health-related absenteeism (MGD 1.3) affect student educational outcomes (specifically related to literacy) (MGD SO1)? 2) How do project school feeding activities (MGD 1.2.1.1 and MGD 1.3.1.1) affect student short￾term hunger (1.2.1) and ultimately influence student attentiveness (MGD 1.2.1)? 3) How does increased knowledge and adoption of health and nutrition practices (MGD SO2, MGD 2.1, and MGD 2.3) among students, parents, and community members affect student health-related absenteeism (MGD 1.3.2)? These questions will be assessed by measuring the PMP indicators listed in Table 4. Table 4: PMP Indicators Indicators MGD SO1/RESULTS FRAMEWORK: Improved Literacy of School-Age Children 1. Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade-level text. 2. Percent of students who are off task (data collected in a single day, at specific intervals). 3. Percent of savings and lending group members with primary-school aged children who report they are financially able to send their child to school. 4. Percent of parents that report their child had a health-related absence from school in the prior 2 weeks. 5. Percent of parents/guardians who can identify three benefits of children completing primary school. MGD SO2/PROJECT RESULTS FRAMEWORK: Increased Use of Health, Nutrition, and Dietary Practices 6. Percent of parents who report their child has had diarrhea, fever, vomiting, or cough in the past 2 weeks. 7. Percent of students in targeted schools who can identify at least three important health/hygiene practices. 8. Percent of students in targeted schools who can identify at least three of the five critical moments for handwashing. 9. Percent of students in targeted schools who can identify at least three important nutrition/dietary practices 2.2. Evaluation Team The baseline evaluation was conducted by an external evaluation team comprised of a local contractor (ATW) and two external technical advisors. The specific roles and responsibilities of the evaluation team included: • The local contractor: In close coordination with the technical advisors, the local contractor lead the overall study design to address the evaluation questions and plan for collecting and analyzing data. • An evaluation specialist technical advisor: This technical advisor provided support for all aspects connected to the training of enumerators and supervision of data collection. This 26 advisor also provided a technical review of documents and processes, data collection tools, qualitative analysis, and reports to ensure rigor and quality. • An assessment and measurement specialist technical advisor: This technical advisor provided support for all the methodological aspects of the baseline study. Specifically, the role of this Advisor was to support the local contractor in the development of the sampling design, survey tools, selection of the comparison schools, quantitative data collection and cleaning, data analysis, and interpretations. More information on the roles and expectations of the local contractor and external technical advisors is provided in the Baseline Evaluation Terms of Reference (see Annex 2). 2.3. Evaluation Design The project evaluation employs a mixed methods approach using a quantitative survey and qualitative research in intervention districts (Tsihombe and Beloha) and control districts (Bekily and Ambovombe) in the southern region of Androy. The qualitative data from key informant interviews (KIIs), focus group discussions (FGDs), and document reviews complement the quantitative survey data—enabling a more in-depth response to the evaluation questions (see Table 3). This evaluation design will allow for impact evaluation (attribution of project activities on key results) from baseline to the midline and endline using Difference in Difference (DiD) analysis techniques. Since DiD relies on changes over time, however, such analysis will not be conducted during this baseline, but data collected in the baseline will be used as part of DiD analysis planned for both the midline and endline evaluation. See Annex 3 for more information on the overall empirical approach to impact identification (DiD) that will be used across baseline, midline, and endline. 2.3.1. Data Collection Methods The baseline evaluation used five principal data collection methods: 1) document review, 2) EGRA 3) quantitative classroom observations, 4) quantitative surveys, and 5) qualitative interviews. Annex 4 provides the evaluation framework matrix detailing the data-collection methods used for each evaluation question while Table 5 outlines the method by respondent and purpose. Each data collection method and tool is described in further detail below. Table 5: Methods by respondent and purpose METHOD RESPONDENT PURPOSE Document Review N/A To validate alignment and linkages to GOM policies, laws, strategies, and existing programs that may impact Mianatra’s operations and implementation. Early Grade Reading Assessment (EGRA) tests Grade 3 students To understand if, at the end of two grades, a student can read grade-level text. Individual interviews use a pre-coded assessment to check the literacy level of a student who has completed two grades of education.28 Classroom observation Grade 4 To assess students’ attentiveness. 28 Given that the data collection was conducted at the beginning of the school year, Grade 3 students are used as a proxy for Grade 2 literacy levels. 27 Structured parent survey Parents, caregivers To assess attitudes, knowledge, and practices of parents on school feeding, education, health, nutrition, WASH, and savings groups. Structured student survey Grade 4 students To gain information on students’ understanding of WASH, hygiene, and nutrition. School inventory School Directors School information on infrastructures, student enrollment, teachers, and school feeding-related information Key Informant interviews MEN (DREN, Chef Cisco), other relevant line Ministries Insights on policy perspective from key Ministry officials to understand the relevance of the program and commitment to sustainability. Key Informant interviews Officials from other associated and relevant Ministries Insights on collaboration and relevance of the program. Key Informant interviews Project partners To understand the program in an in-depth manner including insights on the coordination and facilitation of the partner’s respective component/activity Focus Group Discussions Community members, including those who may participate or benefit from project actions. To explore if the program design responds to the needs of the community and issues of targeting. Document Review The evaluation team conducted a document review that began with files shared by Global Communities. This included reviewing GOM policies, laws, strategies, and existing programs relevant to Mianatra’s operations and implementation to inform the evaluation design and data collection tools. EGRA The baseline evaluation adopted the standard EGRA tool, developed and validated by the GOM through the MEN and with support from FHI 360, to assess student literacy levels. This tool has been developed to be conducted with grade 2 students and assess reading skills such as letter sound, syllable sound, familiar words reading, decoding with invented words reading, fluency with oral reading of a connected text, reading, and listening comprehension. This project used the version of the EGRA tool that was adapted and validated by the MEN, with support from FHI360 in 2020. This work in 2020 resulted in the development and validation of two new subtasks: one on passages and comprehension (listening, oral reading fluency and comprehension) and one on silly sentences (which presents a number of similar sentences, one true and one false, and asks students to determine which is true). As part of the adaptation and validation process in 2020, the new subtasks were piloted by FHI360 in 10 schools. This adapted tool has been adopted by MEN and is currently considered the standard tool for assessing early grade literacy in Madagascar, including periodic assessment by teachers of student literacy. Due to the timing of the baseline survey, which fell at the beginning of the school year, grade 3 students were assessed. Enumerators used a random number generator to select eight (four boys and four girls) grade 3 students per school who have passed the grade 2 examinations and were 28 enrolled in grade 3 at the time of the baseline. The EGRA tool was administered in Standard Malagasy29, in line with MEN policy, and can be found in Annex 5. Classroom Observation To assess student attentiveness, a classroom observation tool, based on the Stalling Classroom Observation tool developed by the World Bank30, was adapted for project use. The Stallings classroom observation system is a questionnaire and protocol for timed observations that produce quantitative data about interactions of teachers and students in classrooms. It was initially developed to observe basic education classrooms in the United States. Its advantages are that it requires only basic training to administer and produces data with a high degree of reliability between raters. Stallings data is comparable internationally, and it has been validated in a number of developing countries. Enumerators followed the same procedure used to conduct a Stalling observation, but when class size was greater than 20 students, observed a sample of students; when the class size was 20 or fewer students, the enumerators observed all students present. Based on USDA’s standard results framework which links reduced short-term hunger (MGD 1.2.1) with the result Improved Attentiveness (MGD 1.2), the observation tool was used in grade 4 classrooms, since grade 4 students also were selected to participate in the student survey, which assessed self-reported short-term hunger as well as other health and hygiene related knowledge attitudes and practices (See description of Student survey below) and to enable analysis of the association between short-term hunger and attentiveness. In each classroom, a representative sample of students was randomly selected in the classroom; when the number of students attending on the survey day was less than the target sample, all attending students were observed. Ten different observations were made in every class observed, at evenly spaced intervals. For each observation, the enumerator observed what the teacher was doing and then observed each of the ten randomly selected students to assess whether they were paying attention or not. The classroom observation tool can be found in Annex 5. Quantitative Surveys The evaluation team developed interview protocols and quantitative surveys based on the evaluation questions. These protocols were further refined with input from Global Communities during the inception phase, based on international standards (e.g. EGRA), lessons learned from implementing evaluations in other MGD programs, and as needed, adaptations to the local context. Surveys were further assessed during pre-testing, which took place on September 09, 2022, in two schools in Ambovombe. Quantitative surveys were then conducted with students, parents (or caregivers), and school directors: Student Survey: The student survey tool on knowledge and practice of hygiene, health, and nutrition was developed using standard approaches for developing data collection instruments that are well aligned with the definition of the key indicators. The survey 29 Malagasy is one of two official languages in Madagascar. There are 12 variants of Malagasy, each of which is considered a distinct languages; the Merina dialect, spoken in Antananarivo, is the basis for Standard Malagasy, the official language used in the schools in Madagascar. The language spoken by inhabitants in the project region (Antandroy people) is Tandroy-Mahafaly Malagasy. Though it shares similarity in lexicon with the Merina dialect (official Malagasy), it is considered a distinct language. 30 For more information, see https://www.worldbank.org/en/programs/sief-trust-fund/brief/the-stallings-classroom￾snapshot 29 targeted Grade 4 students given previous experience in collecting more reliable information from older students Global Communities provided input on the tools based on previous experience in other countries like Tanzania, Guatemala, and Nicaragua. Parent Survey: The parent survey tool focused on the understanding of parents and other caregivers of the importance of education, knowledge of health, hygiene, nutrition, and school absence related to the illness of their child, and participation in savings and lending groups. Global Communities provided input on the survey based on the previous experience from similar evaluations of the McGovern-Dole project in other countries. School Inventory: The school inventory was developed using standard questions and reviewed by Global Communities to help understand the school characteristics such as enrollment of students, number of teachers, and school infrastructure (e.g. latrines, access to water, infrastructure related to storage and preparation of food). All quantitative surveys can be found in Annex 5. Qualitative Interviews FGD and KII guides were developed based on the evaluation questions and tailored for specific groups/ individual respondents based on relevance. The tools were reviewed by Global Communities during the inception phase and can be found in Annex 5. 2.3.2. Data Collection Procedures Data collection took place over a period of five weeks in September and October 2022. During this time, the evaluation team and Global Communities worked closely with the MEN to coordinate communication around and logistics for the data collection exercise with stakeholders at the national, sub-national, and school levels. At the local level, the evaluation team made courtesy visits to the regional and district authorities before starting field activities. Prior to data collection, the evaluation specialist technical advisor led an enumerator training workshop in Ambovombe from September 5 to 9, 2022 (see Annex 6). The workshop was attended by the M&E staff from Global Communities and 24 enumerators and 12 supervisors who had been preselected to participate in the training by the local contractor. The selection of data collectors considered gender, language spoken, previous experience on surveys in the south, and familiarity with technology. The training covered an introduction to the project and the data-collection tools and methods, practical application among trainees, field practice, debriefing and quiz sessions, constructive feedback, and logistics planning. Training also allowed data collectors to become familiar with the two mobile data-collection applications that would be used: Tangerine (for the EGRA) and CSPro (for the quantitative surveys). During the training, two rounds of Assessor Accuracy Measure (AAM) were conducted; EGRA assessment enumerators were then selected from the team of overall enumerators based on the AAM results. Qualitative and quantitative data collection occurred simultaneously, though quantitative data collection was more intensive. The evaluation team began KIIs with stakeholders on September 13 and was able to complete qualitative data collection on October 13. Four interviews were conducted remotely to accommodate scheduling and distance from stakeholders. For the qualitative data, interview audio was recorded and then transcribed; interviewer notes were also 30 taken and used to review and triangulate the recordings. School-level data collection began on September 13 and continued through October 7, lasting four weeks. The quantitative data were collected by trained enumerators through smartphones to ensure data quality and facilitate ongoing data cleaning (see Annex 7) and analysis (see Annex 8). For the quantitative surveys with parents, students, and school directors, CS-PRO was used for mobile data collection. For the EGRA assessments, Tangerine software was used to collect literacy scores; this approach simplifies the preparation and fieldwork involved in literacy assessment and reduces data entry errors. Data collectors were directed to upload data daily to CSPro and Tangerine’s servers (as internet connectivity allowed). 2.4. Sampling This section describes the basic sampling strategy employed by the evaluation team for both the quantitative and qualitative samples. 2.4.1. Quantitative Sample The quantitative sampling design used the three steps stratified cluster random sampling approach. After a list of potential schools in the treatment and control districts was developed, schools were randomly selected using the district as the stratification variable. In addition, at this step, schools in the control district were also stratified to have the same structure as treatment schools where 1) 80% are public schools and 20% are community schools, and 2) 66% have more than 200 students and 33% have 100 to 199 students. At the second step, classrooms were randomly selected from the schools selected in step one (when provided there was more than one class of a relevant grade available in the school at the time of the visit). At the last step, students within the classroom were randomly selected using gender as a stratification variable. The goal of the sample size was first to be sufficient to estimate values for reporting indicators with a 95% confidence interval and, at most, a margin of error of ± 5%. Second, the sample size aims to allow the detection of significant impacts of the program on the indicators during midline and endline evaluations. Detecting such impacts means that changes between baseline and endline should be significantly higher for the treatment group than for the control group. For example, for the first indicator based on the EGRA benchmarks, the program expects an impact of 15% improvement at the endline compared to the baseline. This means that the sample size at baseline should be large enough to allow for the detection of such a change at a statistically significant level. The formula used to determine the total sample size needed for reaching this level of precision is: 𝑛 = 𝑁 ∗ 𝑧 2 ∗ 𝑝𝑞 (𝐸2(𝑁 − 1) + 𝑧 2 ∗ 𝑝𝑞 Where: N = population size (total beneficiaries) Z = confidence level (1.96) E = margin of error (5%) P = the proportion of a population with a particular characteristic (0.5) Q = the proportion of a population not having this characteristic (0.5) 31 Sampling: For computing the expected margin of errors, the evaluation team used the following parameters: • 95% confidence interval • A cluster effect of 0.20 • Expected observed values at baseline (5% for the first indicators and 20% for others) • 10% buffer for non-respond and invalid data The population of treatment schools is composed of all schools in the districts of Beloha and Tsihombe resulting in an overall total of 130 schools. Based on official enrollment numbers that were provided by the MEN for calculating the sufficient sample size for the study, it was understood that 30,293 students are currently enrolled at these 130 schools. Additional sample size information, along with the expected margin of error, is provided in Table 6. Table 6: Sample design and margin of error (ME) Indicators Proposed sample size Estimated Margin of error31 Survey tool Percent of students who, by the end of two grades of primary schooling demonstrate that they can read and understand the meaning of grade level text Treatment: 364 Grade 3 students Control: 182 Grade 3 students ME of 1% EGRA Percent of parents/guardians who can identify three benefits of children completing primary school Treatment: 520 Control: 260 ME of 3.9% Parent questionnaire Percent of parents who report their child has diarrhea, fever, vomiting or cough in the past 2 weeks Treatment: N=520 Control: N=260 ME of 3.9% Parent questionnaire Percent of parents that report their child had a health-related absence from school in the prior 2 weeks Treatment: N=520 Control: N=260 ME of 3.9% Parent questionnaire Percent of students in targeted schools who can identify at least three important health/hygiene practices Treatment: N=520 M= 260 F= 260 Control: N=260 M= 130 F= 130 ME of 3.9% Student questionnaire (grade 4) Percent of students in targeted schools who can identify at least three of the five critical moments for handwashing Student questionnaire (grade 4) Percent of students in targeted schools who can identify at least three important nutrition/dietary practices Student questionnaire (grade 4) 31 With a 95% Confidence Interval 32 Percent of students who are off task Treatment: N = 520 M= 260 F= 260 Control: N=260 M= 130 F= 130 ME of 5.8% Classroom observation tool The baseline was originally designed with two arms: treatment (Beloha and Tsihombe) vs control (Bekily). For the control group arm, since all schools in the Beloha and Tsihombe districts were targeted for project interventions, the comparison schools were targeted from another district in the Androy region (Bekily District) that possesses similar socio-economic characteristics to these two intervention districts including similar levels of income/poverty in the communities and similar levels of ruralness. Furthermore, an additional criterion for selecting the control group schools was to only include schools without a current school feeding program. As described in the below paragraph, an additional district (Ambovombe) was added to the control group to meet the sample size requirements. Based on the power and precision analysis, the initial sampling plan was to select 39 schools: 26 in the treatment districts (13 from each district) and 13 in the control district. During the initial days of baseline data collection at schools, significantly lower than expected school attendance was observed. Specifically, across most schools, and especially in schools in the control district (Bekily), the number of students attending on the day of the survey was significantly lower than expected (often 25% - 35% of expected enrollment) based on official school enrollment data and often less than the targeted number of students to enroll in the baseline. This posed challenges in reaching the required number of students to be enrolled in the survey at each school. To maintain the desired levels of survey power and precision, the evaluation team, in consultation with Global Communities staff, determined that it was necessary to add another district to the control group to expand the number of schools and students that could be sampled for this group, Ambovombe. Given both rural and semi-urban areas in Ambovombe, the selected control schools all came from communities bordering the Bekily district, which are more rural and comparable to the treatment districts, to ensure that the schools had similar characteristics and meet the same sampling criteria for comparison group as Bekily district schools. Given the lower attendance in schools, it was decided to increase the sample of schools to 90 schools: 60 treatment schools (20 in Beloha and 40 in Tsihombe) and 30 control schools (23 in Bekily and 7 in Ambovombe) to achieve the intended power and precision presented. Table 7 shows the distribution of schools across the four districts and two study arms Table 7: Distribution of control and treatment schools by district Project schools Control schools Tsihombe Beloha Bekily Ambovombe 40 20 23 7 Total = 60 Total = 30 EGRA As noted previously, EGRA assessments were conducted with students in grade 3 given that baseline data collection occurred at the beginning of the school year. The target sample was to 33 assess 546 grade 3 students: 364 in project schools and 182 in control schools. Because of the addition of schools in the sampling strategy, EGRA assessments were administered to a total of 592 grade 3 students: 420 in project schools and 172 in control schools. Although students in treatment schools were oversampled and the number of students in control schools was slightly less than anticipated, the margin of error of survey precision was not negatively affected. It should also be noted that even though it was intended to collect data in equal numbers for boys in girls, attendance in the schools were mostly unbalanced between gender (with more females than males attending), which, coupled with the low number of students attending school on survey dates, resulted in more girls than boys in the sample. Table 8 provides the planned and realized EGRA assessment sample. Table 8: Sample EGRA Male Female Total Planned Realized Planned Realized Planned Realized Project schools 182 166 182 259 364 420 Control schools 91 83 91 89 182 172 TOTAL 273 249 273 343 546 592 Attentiveness observation & student survey The target sample was to observe and survey 780 grade 4 students: 520 in project schools and 260 in control schools. However, the attendance rate was low in some schools and the evaluation team did not always meet the expected number of students. As a result, data were collected on 655 grade 4 students: 467 in project schools and 188 in control schools. This slightly affected the margin of error, resulting in a 6.0% margin of error (slightly higher than the originally planned margin of error of 5.8%). Table 9 provides the planned and realized sample for both student observations and surveys. Table 9: Sample attentiveness observation and student survey Male Female Total Planned Realized Planned Realized Planned Realized Project schools 260 186 260 281 520 467 Control schools 130 100 130 88 260 188 TOTAL 390 286 390 369 780 655 Parents and caregivers survey The target sample was to survey 1,040 parents: 520 in project schools and 520 in control schools. Prior to the survey date, the evaluation team data collection supervisor worked with the school directors and student registers to randomly selected parents of students to participate in interviews. Selected parents were then invited to be interviewed on the following day. Ten parents were selected per school in the project schools and 20 parents per school in the control schools. The school director then asked the selected students to inform their parents about the visit from the evaluation team and that their mothers (if available) would need to come to school for an interview. Mothers were prioritized for recruitment for the parent survey given mothers in the project area typically have more involvement and knowledge around issues related to their 34 children’s education and health. However, when mothers were unavailable, fathers were interviewed instead. Table 10 provides the planned and realized sample for parent surveys. Table 10: Sample parent survey Total Planned Realized Project schools 520 757 Control schools 520 621 TOTAL 1,040 1,378 School inventory During baseline, information related to schools’ infrastructures, students’ enrollment, school feeding related activities, and teacher training information were collected through surveys with school directors. The tools were administered in treatment and control schools. All 90 school directors were surveyed: 60 in project schools and 30 in control schools. 2.4.2. Qualitative Sample The qualitative sample was focused on program participants but also included implementers and other stakeholders. Key Informant Interview For key informant interviews, Global Communities provided the evaluation team with a list of intended stakeholders, primarily defined by their role in or relationship to the project. This list included project staff, in-country USG representatives, key Ministry officials relevant to project activities, and project partners (e.g. CRS.) All individuals targeted for KIIs were interviewed. Table 11 describes the number of interviews conducted by type of key informant. Table 11: Summary of key informant interviews completed Organization Title / Department Number of KIIs USAID32 - Senior Education Advisor 1 Global Communities - Chief of Party - Monitoring, Evaluation, Accountability, and Learning - School Feeding Department - Community Mobilization Department - District Coordinator, Tsihombe - District Coordinator, Beloha 6 FHI 360 - Education Technical Advisor 1 32 USAID representative was interviewed as a key USG actor in Madagascar as USDA, the project’s donor, does not currently have any Attachés or staff based in Madagascar. 35 CRS - Deputy Head of Programming - WASH Specialist 2 Ministry of Education - Pedagogical Training Department - Head of Health and School Food Service - Director of the Unit for the Assessment of Academic Achievements - Regional Director of Education 4 Ministry of Water, Sanitation, and Hygiene - Regional Director of Water, Sanitation, and Hygiene 1 Ministry of Agriculture - Regional Director of Agriculture 1 Total 16 Focus Group Discussions FGDs were focused on school staff and members of the community in selected treatment communities. For these FGDs, the evaluation team selected four schools (two in Tsihombe and two in Beloha); within each district one school close to the district center and one school more remote was selected for FGDs. . For focus groups with teachers, all teachers present on that day participated in the FGD. For FGDs with parents, the school director selected parents (convenience sample) to participate. The evaluation team then conducted FGDs in these communities with parents and teachers (see Table 12). Table 12: Summary of focus group discussions completed Category Number of FGDs Location Number of participants School director & Teachers 4 Tsihombe 5 Tsihombe 2 Beloha 4 Beloha 5 Parents of children enrolled in school 2 Tsihombe 8 Beloha 8 Parents of children not enrolled in school 2 Tsihombe 8 Beloha 6 Total 8 46 2.5. Confidentiality and Privacy The evaluation team ensured that provisions were made to protect the confidentiality of baseline participants through a consistently applied informed-consent process. The evaluation team was also attentive to administrative, technical, and physical safeguards to protect the confidentiality of the data during both its storage and disposal. In the written report, the identities of individuals are masked, though the unique positions of some (e.g., leadership in national departments or 36 Mianatra partner organizations) may make some individuals identifiable. This possibility was discussed with interview participants during the informed-consent process. While this baseline evaluation did not meet the formal criteria for human subjects research, the evaluation team also carefully trained enumerators to ensure compliance with the ethical conduct for research involving human subjects, including respect for human dignity, respect for free and informed consent, respect for vulnerable persons, respect for privacy and confidentiality, and respect for justice and inclusiveness, recognizing the potential for harm and maximizing benefits for all involved. Additionally, Global Communities sought permission from the Chief District Officer of Education (CISCO) for the evaluation team to visit the schools, and, in line with national laws in Madagascar, school administrators provided consent for students to be interviewed. 2.6. Data Collection Quality Control Supervisors ensured that interviewers followed survey standard operating procedures and worked to identify and resolve issues that arose in specific sites. They also supervised at least 30% of the surveys carried out by the enumerators. Every day, the evaluation team lead researcher (of the local contractor) hosted a debriefing call with all the data collection supervisors to monitor the progress of data collection and report any problems or incidents. Enumerators were observed during the first day of data collection and remained in regular contact with the local contractor’s head of research, the technical advisors, and the Global Communities team who accompanied data collection throughout the entire two weeks of school￾level surveys. When connectivity and other conditions permitted, data was uploaded daily. The technical advisors reviewed EGRA data daily during initial data collection to ensure the data collected were in line with EGRA administration rules and data collection operating procedures. For qualitative data, the local contractor’s lead researcher ensured data quality by listening to audio recordings from interviews alongside transcripts or notes to ensure the accuracy of transcripts. The local contractor assisted the data collection teams throughout to ensure its smooth operation and to address any issues the teams encountered, especially in terms of managing the schedule for unforeseen circumstances (e.g. unplanned school closures in the absence of school directors). 2.7. Data Analysis Methods The convergent mixed-methods design of the evaluation permitted simultaneous analysis of quantitative and qualitative data. The sections below lay out these distinct processes. 2.7.1. Quantitative Analysis Data analysis was done in STATA 17. Before conducting statistical analysis, the analysis team computed survey weights according to the sample selection process (cluster sample) and applied the survey weights during statistical analysis. See Annex 9 for more detail on how weights were computed. For performing comparisons between subgroups (e.g. control and project; boys and girls), an independent sample t-test was performed with continuous variables and a chi-square test with categorical variables. 37 2.7.2. Qualitative Analysis The qualitative data analysis was done in four steps: 1) data cleaning, 2) reading and developing coding, 3) coding of all the qualitative material and 4) producing results from coding. Interviews were recorded and fully transcribed; however, to facilitate timely review and analysis of discussion results, qualitative interviewers were also asked to develop detailed notes which could be reviewed on an ongoing basis and then used for analysis after being reviewed against the transcripts. Interviews were conducted in Malagasy whenever possible and data analysis was conducted starting from the Malagasy notes. Qualitative codes were developed by the research officer and reviewed by the project manager/research analyst in French. QDA Miner software was used to analyze the qualitative data. Key themes were identified in line with the analysis code book; key relevant quotes included in this baseline report were translated into English by the evaluation team (local contractor). 2.8. Evaluation Limitations The baseline evaluation process encountered several challenges and limitations. Table 13 presents these issues alongside mitigation strategies employed by the evaluation team. Table 13: Evaluation limitations and mitigation strategies Limitation Mitigation Strategy Design limitation: The quasi-experimental evaluation design, chosen due to the impossibility of identifying a true control group, does not allow for direct impact measures against a counterfactual Statistical analysis will be used at the midline and endline to control for heterogeneity of the treatment and comparison group using DiD approach with propensity-score matching control. Design limitation: The baseline evaluation assessed grade 3 students at the beginning of the school year as a proxy for grade 2 students at the end of the school year. This might produce a bias in the results as the sample did not include any grade 2 students who did not pass to the next grade at the end of the previous school year. As the focus of the study is to evaluate the impact of the Mianatra project, future rounds of data collection will be carried out at the same time period with grade 3 students. Response bias from stakeholders: The baseline evaluation targets key stakeholders who play an active role in the Mianatra program. Many have an interest in seeing the program succeed, and those most engaged with partners may be positively biased toward its programming, particularly if they are currently receiving financial support. The evaluation team mitigates this bias by stressing confidentiality and the importance of objectivity in improving the program. Additionally, the evaluation team calls upon a diverse group of stakeholders and multiple data sources to triangulate findings to answer the evaluation questions. Selection/sampling bias for students and parents: It is often the case that students who are present at school may represent those that are more committed to schooling as well as Analysis of students sampled pays close attention to gender as well as the socio￾economic status to help address these concerns. This limitation can be addressed by using post- 38 students with more support at home, those who live closer to the school, and those who have less additional demands on their time. There may be specific gender-specific factors that influence student attendance (e.g. girls who may carry relatively more burden in household chores and child caretaking, or boys who may be more likely to be involved in informal employment or labor, which can each influence student attendance. Furthermore, as described in the Sampling section, parents surveyed reflected parents available on the day of the survey; this could have resulted in a bias towards parents who live closer to the school (and potentially have greater SES/means to send children to school, involvement and or value in education, and/or other factors.) Finally, the prioritization of mothers over fathers in the parent survey limits the ability to understand the difference between mothers and fathers. stratification weighting to correct for any underrepresentation of the characteristics of the population. While the potential biases in the recruitment of parents for the parent survey were due to survey implementation feasibility, it’s noteworthy that parents overall were oversampled which may mitigate this somewhat; additional findings from the gender assessment may provide the important with additional complementary information around gender roles and differences in knowledge, perceptions, and behaviors between men and women in the project. Selection/Sampling bias: As with most qualitative research, because of purposive sampling, the results are not generalizable. This limitation is a characteristic of a qualitative approach. Analysis accentuates that qualitative findings, while not strictly representative, may be transferable and connect to the experiences of others, thereby providing valuable insights applicable to other stakeholders. Attentiveness evaluation: Gauging student attentiveness required classroom observation by trained enumerators. This approach may cause a distraction for the students in the room. The attendance rate in class during the data collection was very low and there was a fairly low number of students observed. This is a very common issue with classroom observation. The evaluation team mitigates it through triangulation and asking questions about student attention to teachers during FGDs. 3.Findings The findings section presents both the baseline indicators of the project and the key results in response to the evaluation questions. First, the characteristics of the sample will be given. Second, the indicators section provides baseline indicator values for the project area versus the control area as well as the differences between boys and girls. Finally, the evaluation question section provides a more comprehensive interpretation including a greater contextualization of the findings that may help explain indicator results. 39 3.1. Characteristics of Sample This section presents the characteristics of the schools for both the project and control schools. All schools visited in both the project and control groups do not currently have any school feeding programs. Only two of the 60 schools visited in the project area have canteen facilities while ten of the 30 control schools visited have canteen facilities (these schools were part of previous school feeding projects implemented in the area but did not have active school feeding activities at the time of this evaluation). Table 14 gives details on the characteristics of the sampled schools. Table 14: School characteristics by project and control Characteristic Project (n=60) Control (n=30) Percentage of public schools 85.0% 86.0% Average number of classrooms per school 2 2.2 Average number of teachers per school 6 4 Percentage of schools with preschool 92.0% 77.0% Percentage of schools with Grade 5 (last grade of primary school) 70.0% 43.0% Average number of enrolled students per school 172 223 Girls ratio in the school 0.44 0.46 Percentage of schools with canteen facility 3.3% 33.3% Percentage of schools with an active school feeding program 0.0% 0.0% With regard to school directors interviewed on school-level characteristics and relevant infrastructure and equipment, the two groups present similar characteristics. Most school directors are male in both project and control schools. More than half have a civil servant contract and more than a third are community teachers, 33 38% in project schools and 30% in control schools. Table 15 shows the details on school directors. Table 15: Summary statistics for school directors Characteristic Project (n=60) Control (n=30) 33 Community teachers are employed and paid by the Parents’ Association in each school and they are normally community members who have not attended any academic training to become teachers. 40 Proportion of school directors who are male 71.6% 80.0% Type of contract: • Civil Servant • Temporary employed (Vacataire) • FRAM (teacher from parents’ association) • Volunteer • Do not know 55.0% 3.3% 38.3% 3.3% 0.0% 56.6% 6.6% 30.0% 3.3% 3.5% Highest level of education completed by school directors: • Formal Education beyond end of high school (Baccalauréat +) • High school (Baccalauréat) • Secondary school (Grade 9, Brevet D’Etudes du Premier Cycle) 1.7% 18.3% 80.0% 3.3% 23.3% 73.3% Regarding grade 3 students who were surveyed for the EGRA, Table 16 shows that there were slightly more students present in the project schools compared to the control schools. The proportion of female students is also higher in the project schools. The average age of students is similar between the project and the control. The GOM school system for grade 1 begins for children aged six which means children are expected to arrive in grade 3 at the age of eight. Table 16 shows that the average age of grade 3 students in both project and control schools is 11. Possible reasons for older students than expected are that students started school late, or they have already repeated grades. Table 16: Summary statistics for grade 3 students Characteristics Project (n=420) Control (n=172) Average number of grade 3 students enrolled per school 26.4 16.3 Average number of grade 3 students present during the visit 7.0 5.7 Proportion of male students 37.8% 47.4% Average age of students 11.7 11.9 Family socioeconomic status: Proportion of students having at home: • Radio 26.4% 8.8% • Television 2.4% 1.2% • Phone 52.4% 57.6% • Bicycle 23.1% 36.0% • Motorcycle 0.7% 1.7% • Water from a pump 0.02% 0% 41 • Electricity 20.8% 39.8% • Latrines 78.9% 72.1% Table 17 shows that the characteristics of grade 4 students are largely similar across project and control schools. At the time of this evaluation, the average age of grade 4 students was nearly 15 in project schools and 13 in control schools. In both project and control schools, the average age of grade 4 students was a bit high considering the GOM school system for grade 1 begins for children aged 6 which means children are expected to arrive in grade 4 at the age of nine. Possible reasons for students being older than expected are that students started school late or have already repeated grades. For the socio-economic indicator, the biggest difference is for the presence of water from a pump where a higher proportion of students in the control school report having this at home. Table 17: Summary statistics for Grade 4 students Characteristic Project (n=467) Control (n=188) Average number of Grade 4 students enrolled per school 15.7 10.3 Average number of Grade 4 students present during the visit 7.8 6.2 Proportion of male students 38.7% 53.8% Average age of students 14.8 13.1 Regarding the parents of students, Table 18 shows that most of the parents surveyed did not attend school (51% did not attend in project and 60% in control group). Similar to what was observed in the grade 4 student sample, more parents in the control group report having water from a pump at home. Table 18: Summary statistics for parents Characteristic Project (n=757) Control (n=621) Proportion of male 21.5% 28.0% Average age of parents 38.3 39.1 Highest level of education of parents: • Bacc + • Lycée (High school) • CEG (Secondary school) • EPP (Primary school) • Did not attend school • Do not know 0.1% 2.6% 11.1% 34.7% 51.2% 0.1% 0.0% 0.8% 6.4% 31.6% 60.9% 0.3% 42 3.2. Key Project Indicators from PMP The baseline evaluation report provides the baseline value of ten key indicators of the project. Table 19 provides an overview of these indicators for the sampled participants in the project schools. Each of these indicators will be discussed further below the table. Table 19: Key Mianatra indicators Indicators Baseline value MGD SO1/RESULTS FRAMEWORK: Improved Literacy of School-Age Children 1. Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text. 6.5% 2. Percent of students who are off task (data collected in a single day, at specific intervals). 44.5% 3. Percent of savings and lending group members with primary-school aged children who report they are financially able to send their child to school. 84.0%34 4. Percent of parents that report their child had a health￾related absence from school in the prior 2 weeks. 22.3% 5. Percent of parents/guardians who can identify three benefits of children completing primary school. 53.9% MGD SO2/PROJECT RESULTS FRAMEWORK: Increased Use of Health, Nutrition, and Dietary Practices 6. Percent of parents who report their child has had diarrhea, fever, vomiting, or cough in the past 2 weeks. 73.5% 7. Percent of students in targeted schools who can identify at least three important health/hygiene practices. 30.1% 8. Percent of students in targeted schools who can identify at least three of the five critical moments for handwashing. 0.8% 9. Percent of students in targeted schools who can identify at least three important nutrition/dietary practices. 16.5% 34 This result is only from the 28% of parents surveyed who reported that they are current saving and lending group members. 43 LRP SO1/RESULTS FRAMEWORK Improved Effectiveness of Food Assistance through Local and Regional Procurement 10. Percent of students who report liking school meals, using a 5-point facial hedonic scale. 0.0%35 3.2.1. Key Results on Improved Literacy of School-Age Children (SO1) This section presents findings for the indicators directly linked to the results for Strategic Objective 1. School feeding programs have a significant, positive effect on school participation (especially for girls) including enrollment, attendance, and retention.36 Mianatra’s design assessment revealed that, without school meals, parents are less motivated to send their children to school. In Androy, the school dropout rate is high37 and contributed to by families’ limited financial capacity—a capacity which has drastically declined due to severe drought and the pandemic impact, leading to early marriage or prostitution for girls and dropping out to work for boys. The project aims to help reduce economic barriers and provide incentives to encourage consistent teacher and student attendance, as well as increase student attentiveness, through the provision of nutritious daily school meals for all preschool and primary students and teachers. Mianatra will also provide training to teachers and reading materials to classrooms to improve the quality of literacy instruction, which aims to help improve the literacy rates of school-age children. Indicators linked to SO1 include: 1. Improved Literacy of School-Aged Children (Percent of students who by the end of two grades of primary schooling demonstrate that they can read and understand the meaning of grade level text) 2. Improved Attentiveness (Percent of students who are off task (data collected in a single day, at specific intervals)) 3. Increased Economic and Cultural Incentives (Percent of savings and lending group members with primary-school aged children who report they are financially able to send their child to school) 4. Reduced Health-Related Absences (Percent of parents that report their child had a health￾related absence from school in the prior 2 weeks) 5. Increased Community Understanding of Benefits of Education (Percent of parents/guardians who can identify three benefits of children completing primary school) Improved Literacy of School-Age Children Literacy of school-age children was measured as the percent of students who, by the end of two grades of primary schooling, demonstrated that they can read and understand the meaning of grade level text. This is a measurement of student reading comprehension. For reading comprehension, the benchmark for Madagascar requires that a student must be able to answer 80% of the reading comprehension questions correctly. Overall, few students (<10%) were assessed to have reached this benchmark in both the project and control schools. Figure 2 shows that 6% and 3% of grade 3 students in project and control schools, respectively, reached the 35 Note that this result of zero is due to no schools currently serving school meals. As such, LRP SO1 is not discussed below. 36 Rassas et al.2018; WFP State of School Feeding 2020; GCNF 37 World Bank. Combined project information documents/integrated safeguards datasheet (PID-ISDS). May 2020. 44 reading comprehension benchmark; this difference between groups is not statistically significant. A higher proportion of girls were able to reach the reading comprehension benchmark than their male counterparts in project schools (9% of girls versus 2% of boys, p<0.001) as well as control schools (5% of girls versus 1% of boys, p<0.006). Figure 2: Percentage of students reaching Reading Comprehension benchmark per group and gender of the student Concerning oral reading fluency, students should be able to correctly read at least 30 words per minute to meet the national benchmark. Overall, students scored slightly higher in oral reading fluency than the reading and comprehension benchmark; however, most students still did not meet the oral reading fluency benchmark. As shown in Figure 3, more grade 3 students in project schools (12.2%) had reached the reading fluency benchmark than grade 3 students in control schools (4.5%, p<0.000). When comparing oral reading fluency by gender, more girls (15%) reached the benchmark than boys (6%, p<0.009) in project schools while, boys (7%) demonstrated higher rates of oral fluency than girls (2%, p<0.021) in control schools. Figure 3: Percentage of students reaching Oral Reading Fluency benchmark per group and gender of the student Figure 4 shows the average fluency scores on EGRA subtasks for each group: • Letter name: the number of correct letter sounds per minute 6.5% 2.1% 9.2% 3.2% 1.3% 4.9% 0% 10% 20% 30% 40% 50% Overall Male Female Overall Male Female Project Control 12.2% 6.3% 15.7% 4.5% 6.8% 2.4% 0% 10% 20% 30% 40% 50% Overall Male Female Overall Male Female Project Control 45 • Syllable: the number of correct syllable sounds per minute • Familiar word: the number of correct familiar words per minute • Reading fluency: the number of correct words per minute while reading connected text Results show that the performance of the students on these subtasks is similar with the only significant difference being the number of correct letters per minute (where the average score for students in the project schools was higher than that in the control group with p=0.036). On average, students in project schools were able to read 11.9 correct words per minute on the reading passage compared with a rate of 10.6 correct words per minute for students in control schools. Figure 4: Average fluency scores on EGRA subtask by group For subtasks where a percent correct score is computed, as shown in Figure 5, the performance of the students in both groups was similar with no statistically significant differences found. Overall, students performed better on the silly sentences subtask in both project and control schools while students could only answer 12.6% of the reading comprehension questions correctly in project schools and 10.5% in control schools. Figure 5: Average percent correct scores on EGRA subtask by group 11.1 16.1 10.8 11.9 8.6 17.2 10.5 10.6 0 5 10 15 20 Letter Name Syllable Familiar Word Reading Fluency Project Control 48.5% 12.6% 20.7% 51.0% 10.5% 25.3% 0% 20% 40% 60% 80% Silly Sentence Reading Comp. Listening Comp. Project Control 46 Regarding the proportion of students who scored zero for the timed subtasks (meaning that the student could not read a single letter, syllable, or word or provide a single correct answer to a comprehension question), the only statistically significant difference between groups on these subtasks was on the listening comprehension subtask where a higher proportion of students with zero-scores was observed in the project schools (p=0.027). The reading comprehension subtask showed the highest percentage of students with zero scores, with 71.5% of the students in project schools and 74.4% in control schools having a zero score. On the other hand, the silly sentences subtask had the lowest percentage of zero scores, with 6.2% in project schools and 3.9% in control schools (see Figure 6). Figure 6: Proportion of zero-scores on EGRA subtasks by group Table 20 presents the average scores on different EGRA subtasks by gender for project and control schools. In projects schools, girls achieved higher scores on all subtasks than boys with all differences being statistically significant. The evaluation team observed fewer differences between gender groups in control schools. In control schools, boys presented higher average scores on syllable and familiar words subtasks while girls outperformed boys only on the reading comprehension subtask. Table 20: Average scores on EGRA subtasks by gender and group. Subtask Project Control Male Female Male Female Letter Name (CLPM) 8.6 12.7* 8.6 8.5 Syllable (CSPM) 13.2 17.9* 19.1* 15.5 12.7% 32.1% 42.2% 6.2% 40.7% 71.5% 52.9% 9.3% 28.2% 34.7% 3.9% 32.6% 74.4% 42.2% 0% 20% 40% 60% 80% 100% Letter Name Syllable Familiar Word Silly Sentence Reading Fluency Reading Comp. Listening Comp. Project Control 47 Familiar Word (CFWPM) 8.8 12.0* 11.9* 9.2 Silly Sentence (% Correct) 44.9% 50.7%* 49.9% 52.0% Reading Fluency (ORF) 9.8 13.2* 11.2 10.2 Reading Comp. (% Correct) 8.8% 14.9%* 8.5% 12.3%* Listening Comp. (% Correct) 17.0% 23.0%* 24.7% 25.9% *Average score is significantly higher (p<0.05) The percentages of students who scored zero (were not able to answer any of the EGRA subtasks on reading or pronunciation correctly) are presented in Table 21 by gender. In the project schools, boys present a significantly higher percentage of zero scores in letter name and silly sentence subtasks while girls show a significantly higher percentage of zero scores in syllable, familiar words, and reading fluency subtasks in the control schools. Table 21: Percentages of zero-scores on EGRA subtasks by gender and group Subtask Project Control Male Female Male Female Letter Name 19.9%* 8.3% 6.2% 12.0% Syllable 35.8% 29.9% 19.5% 36.1%* Familiar Word 44.6% 40.9% 25.3% 43.2%* Silly Sentence 11.5%* 2.9% 6.2% 1.9% Reading Fluency 42.9% 39.3% 26.7% 38.0%* Reading Comprehension 77.1% 68.1% 75.5% 73.5% Listening Comprehension 57.4% 50.3% 47.6%* 37.3% *Percentage is significantly higher (p<0.05) Improved Attentiveness As described in the Methods section, because USDA’s standard results framework links reduced short-term hunger (MGD 1.2.1) with the result Improved Attentiveness (MGD 1.2), the observation tool was used in grade 4 classrooms, since grade 4 students also were selected to participate in the student survey, which assessed self-reported short-term hunger as well as other health and hygiene related knowledge attitudes and practices and to enable analysis of the association between short-term hunger and attentiveness. A student was considered off task if being observed as inattentive at least one time during the 10 observations made for each sampled grade 4 student. Students were considered inattentive if sleeping, looking outside, distracting other students, looking away from the task, playing, or being disciplined by the teacher. 48 Figure 7 shows that 44% of grade 4 students in the project and 55% of grade 4 students in the control groups were inattentive at least one time during these classroom observations. There is no statistically significant difference between students in project and control schools, nor between girls and boys within both project and control schools.38 Figure 7: Percentage of students who are off task per group and gender of the student Increased Economic and Cultural Incentives Though the project intends to support the creation/expansion of village savings and loans (VSL) group to increase household economic resources to support children’s attendance in school, the baseline evaluation assessed current levels of participation in any existing VSL groups 39 (or “voamamy” in the local dialect). Among the parents surveyed, 28% in project schools and 11% in control schools reported that they are active VSL group members. Figure 8 shows that 84% of VSL group members with primary-school aged children in project areas reported they are financially able to send their child to school, which is statistically higher than the 64% of parents who are VSL members in control schools (p<0.001). 38 The evaluation team notes that since a small class may be easier for a teacher to manage and provide students with more individualized focus and attention, findings should be interpreted with caution as the average classroom size during the observation was very low, with an average of seven students in project and six students in control (with some classrooms with as little as 2 students being present on the day of the visit) classrooms. 39 CRS has been supporting Saving and Internal Lending Communities (SILC) in Beloha, Tsihombe and Ampanihy Districts since January 2020 through the Maharo project, a five-year USAID-funded project. These SILCs have been developed as a point of entry to the community. A total of 888 groups have been creased with 18,833 people (11,394 or 60.5% female members); of these, 497 (56.0%) groups are over 12 months old, operating through multiple cycles and implying long-term viability. 44.5% 45.3% 44.0% 55.0% 50.7% 60.0% 0% 20% 40% 60% 80% 100% Overall Male Female Overall Male Female Project Control 49 Figure 8: Percentage of parents in VSL groups with primary school aged children who report they are financially able to send their child to school by group and gender of the parent Reduced Health-Related Absences This project aims to improve students’ attendance and ability to learn through the reduction of health-related absences and increased use of good health, hygiene, nutrition, and dietary practices. In the parent survey, parents were asked whether their children had been absent from school in the past two weeks and whether an illness was the main reason. Figure 9 shows that 22% of parents in project schools reported their child had a health-related absence from school during the two weeks prior to the survey. This was higher than the 11% of health-related absences reported in control schools (p<0.001). Figure 9: Percentage of parents that report their child had a health-related absence from school in the prior 2 weeks by group and gender of the parent According to parents, the illnesses that have most affected children and caused them to miss school are cough, diarrhea, and fever in the project schools while malaria is the most commonly reported in the control schools (see Figure 10). 84.0% 75.5% 86.5% 64.7% 58.3% 68.0% 0% 20% 40% 60% 80% 100% Overall Male Female Overall Male Female Project Control 22.3% ** 12.6% 25.1% 11.3% 9.1% 12.5% 0% 10% 20% 30% 40% 50% Overall Male Female Overall Male Female Project Control 50 53.9% 54.6% 53.7% 28.6% 35.1% 26.2% 0% 20% 40% 60% 80% 100% Overall Male Female Overall Male Female Project Control Figure 10: Underlying condition/illness causing student health-related absenteeism in the past two weeks, as reported by parents Increased Community Understanding of the Benefits of Education Parents were asked an open-ended question to identify the benefits of their child completing school. More than half of the parents surveyed in project schools were able to correctly identify at least three benefits. Parents in project schools were more likely to identify at least three benefits than parents in comparison schools (53.9% in the project versus 28% in the control group, p<0.000). Responses did not differ by the gender of the parent surveyed within project and control schools (see Figure 11). Figure 12 shows the benefits of children completing primary school that were identified by the parents in project and control schools. The most frequent benefits parents in both project and control schools reported for children to complete primary school were: “can succeed in life”, “can get a job”, “acquire knowledge, skills, and ethics”, and “can continue to secondary school” in both project and control schools. Figure 11: Percentage of parents/guardians who can identify three benefits of children completing primary school by group and gender of the parent 67% 15% 6% 6% 2% 0% 0% 0% 2% 2% 0% 44% 37% 26% 14% 5% 1% 1% 0% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% Malaria Cough Diarrhea Fever Vomiting Headache Breathing illness Toothache Sore throat Stomach ache Project Control 51 Figure 12: Percentage of parents/guardians reporting each of the benefits of completing primary school by group 3.2.2. Key Results on Use of Health, Nutrition, and Dietary Practices (SO2) This section presents findings for the indicators directly linked to the use of health, nutrition, and dietary practices (SO2). Without proper nutrition, health conditions and hunger negatively impact children’s cognitive and motor development, attendance and attention, ability to learn, and overall school performance.40 The GOM’s National School Food and Nutrition Plan (PNAN III) recommends the integration of school feeding, nutrition education, health, WASH, and agriculture into primary schools,41 which is supported by evidence that school feeding programs have a greater impact when coupled with complementary services and programs.42 The following indicators are used to assess the communities' and/or students’ knowledge, attitudes, and practices on common health, hygiene, and nutrition: 1. Increased Use of Health, Nutrition, and Dietary Practices (Percent of parents who report their child has had diarrhea, fever, vomiting, or cough in the past 2 weeks) 2. Improved Knowledge of Health and Hygiene Practices (Percent of students in targeted schools who can identify at least three important health/hygiene practices) 3. Improved Knowledge of Health and Hygiene Practices (Percent of students in targeted schools who can identify at least three of the five critical moments for handwashing) 4. Increased Knowledge of Nutrition (Percent of students in targeted schools who can identify at least three important nutrition/dietary practices) Increased Use of Health, Nutrition, and Dietary Practices Parents were asked if their child has had at least one of these four illnesses in the past 2 weeks: diarrhea, fever, vomiting, or cough. High rates of these illnesses were reported in both project 40 Bundy. 2011; Taras. 2005. 41 Government of Madagascar. PNAN III. 2019 42 GCNF. Global Survey Report. 2019. 2.9% 9.6% 10.2% 14.8% 17.8% 18.9% 21.3% 25.5% 33.2% 37.7% 40.8% 47.8% 1.3% 11.3% 6.4% 10.6% 7.6% 20.1% 7.7% 28.8% 24.6% 22.9% 37.5% 31.1% 0% 10% 20% 30% 40% 50% 60% Can speak French Have an ability to express themselves Well educated Capacity building for further development Reduce poverty Caring for parents when they are old Can support themselves Can continue to secondary school (G6 - G9) Complete primary education Can succeed in life Acquire knowledge, skills and ethics Can get good jobs Control Project 52 (73.5%) and control districts (55.7%) by parents (see Figure 13). Parents in project districts were significantly more likely than parents in comparison schools to report that their child had recently had an illness (p<0.000). Figure 13: Percentage of parents/guardians who report their child has diarrhea, fever, vomiting, or cough in the past 2 weeks by group and gender of the parent Cough was the most common illness reported in both project and control groups. Figure 14 shows that the percentage of parents who reported their child had a cough in the past two weeks was 60% in the project group, which is significantly higher than the control group (45%, p<0.000). The second most reported illness was diarrhea, which was higher in the project group (with 27% of the parents reporting this illness) than in the control group (17%, p<0.000). Fever was the second most reported illness in the control group with 31% which was higher than in the project group (23%, p<0.000). 23% of parents in the project group reported their children had experienced vomiting in the past two weeks compared with 18% in the control group (p<0.017). Figure 14: Percentage of parents reporting each of health problems in the past 2 weeks by group 73.5% 73.6% 73.5% 55.7% 52.8% 56.8% 0% 20% 40% 60% 80% 100% Overall Male Female Overall Male Female Project Control 22.9% 23.1% * 26.8% ** 60.8% ** 31.4% ** 17.9% 17.4% 45.3% 0% 20% 40% 60% 80% 100% Fever Vomiting Diarrhea Cough Control Project 53 Improved knowledge of health and hygiene practices Students were asked to identify important health/hygiene practices. Figure 15 shows that 30% of grade 4 students in project schools and 37% in control schools were able to identify at least three important health/hygiene practices; there was no statistically significant difference between project and control schools. In control schools, almost 60% of boys were able to identify at least three important health/hygiene practices, which was higher than girls in control schools (11%, p<0.000). As shown in Figure 16, handwashing was the hygiene practice most reported by the students in both project and control schools. Figure 15: Percentage of students who can identify at least three important health/hygiene practices by group and gender of the student Figure 16: Percentage of students reporting each of the health/hygiene practices by group Improved knowledge of health and hygiene practices Students were asked an open-ended question to identify at least three of the five critical moments for handwashing; the five critical moments for handwashing (per the Performance Monitoring 30.1% 26.6% 32.1% 37.4% 59.1% 11.1% 0% 20% 40% 60% 80% 100% Overall Male Female Overall Male Female Project Control 1.0% 5.4% 25.4% 37.5% 78.4% 8.5% 13.1% 17.0% 41.6% 84.6% 0% 20% 40% 60% 80% 100% Wear clean clothes Take bath Avoid open defecation Use of latrines to defecate Handwashing Control Project 54 Indicator Reference Sheets) are: 1) Before, during, and after preparing food; 2) Before and after eating food; 3) After defecation; 4) After blowing your nose, coughing, or sneezing; and 5) After touching garbage. Note that in the analysis, for critical moments with multiple opportunities (e.g. “before, during, and after preparing food”) responses were coded as correct if they mentioned at least one of the moments. For example, a correct response reflects mentioning at least one of the following “before preparing food”, “during food preparation” and/or “after preparing food.”43 Only 15.3% of grade 4 students in project schools and 8.9% of grade 4 students in control schools were able to identify at least three of the five critical moments for handwashing; this difference was not statistically significant (see Figure 17). “Before and after eating food” is the critical moment that was reported most by the students in both project and control schools (see Figure 18). Figure 17: Percentage of students who can identify at least three of the five critical moments for handwashing by group and gender 43 Note that language for these moments will be updated in the PMP and PIRS 15.3% 18.6% 13.3% 8.9% 9.9% 7.7% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Overall Male Female Overall Male Female Project Control 55 Figure 18: Percentage of students reporting each of the five critical moments for handwashing by group Increased knowledge of nutrition Students were asked to mention important nutrition/dietary practices that they do every day through an open-ended question. Figure 19 shows that 16% of grade 4 students in the project group and 11% of grade 4 students in the control group were able to identify at least three important nutrition/dietary practices. There is no statistically significant difference between students in project and control schools nor between girls and boys within each group. Figure 19: Percentage of students who can identify at least three important nutrition/dietary practices by group and gender 57.5% 76.9% 5.9% 3.2% 25.4% 50.3% 82.4% 3.4% 4.7% 6.4% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% After defecation Before and/or after eating food After touching garbage After blowing your nose, coughing or sneezing Before and/or during and/or after preparing food Control Project 16.5% 18.3% 15.8% 11.8% 18.0% 4.7% 0% 10% 20% 30% 40% 50% Overall Male Female Overall Male Female Project Control 56 As shown in Figure 20, “Eat food that is safe to consume/eat” and “Wash hands before eating” are the nutrition/dietary practices that were most reported by the students in both project and control schools. We can also observe that more grade 4 students in project schools (9%) correctly identified “dietary diversification” than grade 4 students in control schools (1%, p<0.001). Figure 20: Percentage of students reporting each nutrition/dietary practices by group 3.3. Findings on Baseline Evaluation Questions This baseline evaluation of the Mianatra project addresses a series of questions based on five of the OECD evaluation criteria: 1) relevance, 2) coherence, 3) effectiveness, 4) efficiency, and 5) sustainability. In this section, each evaluation question is answered individually, referring to previously mentioned results as necessary. This analysis reflects the mixed-methods approach that guided this impact evaluation, as both quantitative and qualitative data sources inform findings. Where relevant, the analysis explores contradictions and complexities between findings. 3.3.1. Relevance and Coherence According to the OECD, the criterion of relevance addresses the “extent to which the intervention objectives and design respond to beneficiaries’, global, country, and partner/institution needs, policies, and priorities, and continue to do so if circumstances change” and the criterion of coherence addresses the “compatibility of the intervention with other interventions in a country, sector, or institution.” 44 Seven project-specific evaluation sub-questions are discussed in this section: 1. Is the program relevant to the desired outcomes and objectives of the McGovern-Dole strategy, policy, and plan? 44 This definition and subsequent definitions were informed by the following resource: OECD Home: Evaluation Criteria, https://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm 0.0% 8.5% 9.1% 1.2% 65.5% 83.6% 2.5% 3.6% 4.3% 9.7% 61.0% 75.7% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% Importance of iron Importance of fruits and vegetables Importance of vitamin A Dietary diversification Wash hands before eating Eat food that is safe to consume/eat Control Project 57 2. Is the project’s theory of change realistic, appropriate, and practical to implement? 3. Are there any modifications needed to the project’s design, structure, logic, or management to facilitate the achievement of the desired outcomes and impact? 4. Are the project’s proposed strategies and interventions to increase knowledge and strengthen the capacity of key stakeholders appropriate and realistic to achieve the expected results and outcomes? 5. Is the program relevant to address the real, and perceived needs of project beneficiaries? How well does the program complement, leverage, and align with other ongoing education, nutrition, and school feeding programs within the country and other GOM plans and priorities? 6. Is the project design suited to improving the level of early grade literacy performance and overcoming the main challenges to improving literacy outcomes? 7. Is the program designed to be flexible, for example, can the overall strategy, components, or specific activities be adjusted over time due? Through its education sector plan, which has been in place since 2018, the GOM is targeting three areas of education development: access, quality, and governance. The priorities are to increase school enrolment, have better results, and decrease the dropout rate. WASH in public schools is also among the priorities of the GOM by installing the program “Ecole amie du wash” or “Wash friendly school” in collaboration with the Ministry of Public Health. Stakeholders from the MEN who were interviewed affirmed that, to achieve these objectives, school feeding is at the center of the ministry's activities at the primary school level with the NSFP. They also stated that the McGovern-Dole project helps them partly overcome the lack of financial means which prevents the MEN from establishing a school feeding program in all schools. With Mianatra’s main objectives being to improve the quality of literacy instruction, improve attentiveness and student attendance, increase the use of health, nutrition, and dietary practices, and increase access to clean water and sanitation services, project staff noted that the project is aligned with the government priorities because all planned activities are inspired by the policies and priorities of the MEN and planned activities complement ministry programs. Due to the project being in its early stage, KIIs showed that, at this stage, few of the people interviewed are aware of the theory of change of the project. Key informants agreed that hunger is one of the major causes of inattentiveness in class, poor school results, and school dropout. Additionally, the project’s theory of change takes into consideration issues affecting poor educational outcomes by supporting the government’s efforts to improve teacher competency (a high percentage of teachers in project schools are directly hired by the parent associations and have not undergone teacher training). The context in the south of Madagascar (described in detail in section 1.1) is considered by some respondents as an important factor that needs to be taken into consideration while implementing project activities to ensure the success of the project, particularly regarding food and water insecurity and vulnerability to climatic shocks. As mentioned before, stakeholders recognize that Mianatra is inspired by, and aims to align with, the NSFP developed by the MEN. Project stakeholders noted that one of the main objectives of the project is to strengthen the capacity of MEN staff at the district and commune levels to oversee school feeding and complementary activities. As project activities are being initiated, stakeholders recommended the project consider training all Chef ZAPs (commune level education officials) in the two districts of intervention on school feeding and complementary activities supported under Mianatra to ensure activities are aligned across all schools within the target 58 districts. According to the respondents, it is common that Chef ZAPs do not always receive training, supportive monitoring, or accompaniment from other projects or the GOM due to logistical and financial constraints. It has been recommended that Mianatra consider activities to actively involve, train, and support all Chef ZAPs to adequately support their schools and Mianatra project activities. All participants interviewed raised that water is the biggest problem faced by the population living in these areas. It is very difficult for communities to access clean water, though the situation varies from community to community. For example, in some places there is water, but the water is not potable/clean whereas in other communities there is no water at all and community members need to travel or relocate to find water. Lack of water affects children in many areas including nutrition, health, and hygiene and does not allow for an environment conducive to quality education. " The priority need is water. Some [communities] have water, but [it is] difficult to clean or is only available a few times in the year; in some places, there is nothing to do but wait for God to give rain. We will deploy different strategies depending on the situation of each area." (Implementing partner) “We do not have water; sometimes we are even ashamed in front of our students because we teach them some hygiene practices, but we cannot even take care of our bodies.” (Teacher during FGD) According to respondents, the project is adequately designed to respond to the project’s main beneficiaries (i.e. students, parents/guardians, teachers, and school administrators) in the following ways: ➢ Students: Given its focus on school feeding and improved educational instruction, respondents believed the project is designed to meet the needs of the students, especially as many children face food insecurity at home and often do not eat a breakfast meal. Teachers described that students fall asleep in class because they are hungry and have not eaten. “This is a case that we often see; they are just sitting there, and some are sleeping because they are hungry; their heads are elsewhere.” (Teacher during FGD) Figure 21 below shows the percentage of grade 4 students surveyed who eat at home, based on responses from grade 4 students in project schools in the student survey. The majority (57%) rarely take or never have breakfast before going to school, though the majority did report eating at home in the evening. 59 Figure 21: Percentage of grade 4 students in project schools who eat at home Source: Grade 4 students in project school, n= 467 ➢ Parents/caretakers: In FGDs with parents, respondents most often mentioned the lack of money as the main reason limiting the accessibility of education for school-aged children. “I really want my children to go to school and continue as far as possible, but we cannot afford that, we do not have money " (Parents during FGD) According to quantitative data, VSL group membership is a strong factor supporting the ability of parents to send their children to school; however, only 28% of parents surveyed in the project area declared being members of a group. Given the project’s focus to work in all communities to establish and support new and existing VSL groups, respondents believed the project will be relevant and responsive to improve the financial capacity of parents. ➢ Teachers/school administrators: In FGDs with teachers, training is the most expressed need from teachers. Most teachers have never received any training to be able to teach properly. The evaluation team observed in the field that, in many schools, there is only one qualified teacher for many grades, while the others are from parent associations. “I really want to receive training; I want to do well my job. We just attended one day of briefing in the CISCO at the beginning of school year last year and that is all, we have never received any training” (Teacher during FGD) ➢ Government officials: Project staff emphasized that the project will help the government because it supports them in achieving their goals in school feeding and the professional development of teachers. Project staff further reported that the program is designed to be flexible. Project staff noted three conditions that are laid out in the agreement between USDA and Global Communities that cannot be changed: the number of beneficiaries, the quantity of food, and the total amount of funds allocated to the program. As noted previously, during field data collection, the evaluation team documented that the observed number of students in schools visited was significantly less than the anticipated number of students enrolled and attending (based on official school enrollment figures provided by MEN 43% 64% 79% 41% 29% 19% 16% 7% 2% Breakfast Lunch Dinner Proportion of students who eat at home Always Rarely Never 60 for the project implementation area as well as the control area). As described by project staff and other stakeholders, possible reasons for the decrease in student enrollment and the difference between enrollment and students found at schools include an overall decrease in inhabitants (and thus students) in the South due to internal migration; lack of reliability in official figures; and the possibility that some students may not yet have returned to school or new students had not yet been enrolled by their parents (despite the fact that survey teams visited schools at least two to three weeks after the school year had started). One official noted that at the beginning of the school year, it often takes several weeks for both students and staff to return to school. In other qualitative discussions, school staff described internal pressure to inflate enrollment numbers as well as a lack of quality assurance of student enrollment information. Project staff noted the importance of the need to work with USDA, MEN, and other stakeholders to realign and adjust, as necessary, programmatic approaches in line with accurate enrollment figures. Table 22 shows the enrollment differences for grades that were targeted by the survey (grade 3 and grade 4) in the project implementation areas. Table 22: Student enrollment versus student attendance on day of surveys Grade 3 Grade 4 Grade 2 last year (Potentially Grade 3 this year) Grade 3 enrolled this year (from school director) Grade 3 present during the visit Grade 3 last year (Potentially Grade 4 this year) Grade 4 enrolled this year (from school director) Grade 4 presents during the visit Tsihombe n=60 49 29 10 33 17 8 Beloha n=30 37 23 9 25 13 7 Given the significant issues with water scarcity in the region and its impact on schools, household economic security, and agricultural production, implementers recognized water scarcity as one of the most significant challenges the project faces. Implementors further recognized that the project may need to adapt or innovate programmatic approaches and strategies during implementation in light of this scarcity. Additionally, as noted by the respondent below, the project may also need to track changes in the political context in Madagascar and the potential impact of elections as well as potential changes in political leadership on project activities. " The project took into account the potential risks and the modifications to be expected. For example, next year is an election period, we know that changes in political leaders can be a threat to projects, so we are preparing for that" (Project staff) 3.3.2. Effectiveness According to the OECD definition, the criteria of effectiveness addresses “the extent to which the intervention achieved, or is expected to achieve, its objectives, and its results, including any differential results across groups.” The following four research questions fall under these criteria: 1. To what extent is the COVID-19 pandemic likely to influence program effectiveness/results and how best can the program address these influences? 61 2. What are the main challenges or barriers to achieving desired outcomes across other related project components? 3. Is there a clear understanding of roles and responsibilities by all parties involved for implementation and monitoring? 4. Are the project’s monitoring and evaluation strategies realistic, appropriate, and practical to implement? While the project was designed while COVID-19 related restrictions were still in place, the situation in Madagascar is now normalized and no restrictions are in place regarding school attendance. However, the project has developed a set of activities to be put in place in the event of a future lockdown based on learning about remote schooling while COVID-19 restrictions were in place. The project will also be guided by the implementors’ experiences acquired while implementing other McGovern-Dole projects in other countries. Currently, the main barriers highlighted during field work relate to the conditions of schools’ infrastructures in the project area of intervention. First, as previously noted, access to water is the biggest challenge. Many communes in these districts do not have drinking water and in some places, they cannot even find water. Moreover, beans are one of the principal components of the food to be given in the canteens, and cooking beans requires a large quantity of water. In response, Mianatra plans to use different water treatment technologies according to the situation of each implementation area. The accessibility of schools also presents a challenge as project schools are in very remote places. Project activities, such as the transport of food, provision of training, and site visits, may be significantly more difficult (especially during the rainy season) as a result. Another challenge is the level of training and support for teachers as few have had formal academic training to become a teacher. On average, three classes per school are taught by FRAM teachers in the project schools, and 80% of school staff in project schools had not completed their high school education. Even with the project’s focus on teacher training, the limited formal education of teachers may be a barrier to making significant gains in improving the quality of instruction. Another challenge reported is the potential for misuse of resources or food theft, which is unfortunately a frequent situation in Madagascar. Some of the 130 selected schools have already benefited from a school feeding program before but the implementers of the previous program decided to withdraw because they noticed a misappropriation of the food. The project is going to adopt the already established CRS “hot line” through which project stakeholders will be able to quickly report any problems they might face during implementation as a means for combatting potential misuse of project resources and food theft. " This school has already benefited from a canteen from WFP, but they stopped because the former school director misappropriated food" (Parent during FGD) Finally, as noted above, a significant challenge Mianatra faces is determining the actual number of students enrolled in schools. It was observed during the KIIs that all parties involved in the implementation and monitoring of the program understand their respective roles and responsibilities. However, the MEN staff at the local level said that they have not received any documents about the project yet, so their responsibilities are not yet clear to them. 62 " Our roles and responsibilities are not yet clear because there is no document like responsibility charter that has been communicated yet so that we can clearly define our responsibilities" (MEN staff at the local level) The project implementer and its partners have a long and established experience in the implementation of McGovern-Dole projects in other countries, and, therefore, they have procedures for adapting monitoring tools already tested in other contexts that are currently in use. Moreover, the implementation partners have shared that they have ample experience in the implementation of social projects in Madagascar (particularly in the south) and that the monitoring and evaluation strategies employed for this project are not new to them. 3.3.3. Efficiency According to OECD criteria, efficiency is meant to assess “the extent to which the intervention delivers, or is likely to deliver, results in an economic and timely way.” There is one research question under this criterion: 1) How efficient and realistic is the planned allocation of resources to provide the necessary support and to achieve the broader program objectives? Since the project is still in its beginning, it was difficult for baseline interviews and FGD participants to answer the inquiries about efficiency. During interviews with project staff, staff stated that, overall, the human, financial, and material resources all seem to be in place to be able to carry out the planned activities of the project. However, respondents noted that the planned resources allocated to address the water problem are not sufficient and the project might need to allocate additional resources to increase the number of schools with reliable access to water to support project activities and objectives. 3.3.4. Sustainability The project will support schools, communities, and the GOM to sustain the educational and nutrition benefits of Mianatra beyond the life of the project. For this baseline evaluation, eight research questions around sustainability were used to determine the extent to which the project’s sustainability plan is appropriate: 1) Is the project’s sustainability plan realistic and appropriate? 2) What undeveloped opportunities might be at hand that can contribute to the sustainability of project benefits and outcomes? 3) Does the project have the means to address threats or gaps (i.e., climate-related shocks, COVID-19) that could impact progress toward sustainability and sustained outcomes/benefits of the project? 4) To what extent have stakeholders and local actors been engaged and committed to ensuring progress toward a sustainable NSFP? 5) To what extent have stakeholders provided monetary or in-kind support for school feeding and complementary activities, and are these sustainable inputs? 6) What commitment has been made in support of a gradual transition toward a locally owned and managed school feeding program by the GOM, schools, and communities? 7) Are all the elements of sustainability (capacity, motivation, resources, linkages) addressed in the project’s sustainability plan? 8) What are specific capacity needs for project stakeholders that the project should address to foster greater ownership and accountability to sustain project benefits? 63 The project will focus on creating a lasting impact by strengthening existing systems, structures, and capacities to gradually hand over Mianatra’s integrated school feeding program to the GOM and communities. Through community sensitization and mobilization, the project plans to co￾create a feasible transition strategy and planning and measurement tools with project stakeholders. Local capacity strengthening is embedded in all activities to ensure the mentorship of stakeholders across various ministries and at all levels supports the gradual transition of ownership of key Mianatra interventions to these stakeholders. The project’s sustainability plan outlines a process for school graduation including 1) the gradual phase-over of key responsibilities from Mianatra to stakeholders to sustain the benefits of improved education, enrollment, attendance, health, nutrition, and WASH outcomes and 2) the gradual handover of responsibility to GOM and other stakeholders for school meal commodity sourcing, delivery, and provision of quality, nutritious daily school meals after Mianatra-provided commodities cease. While Mianatra does not anticipate fully graduating schools within this five-year project, it does seek to assist progression toward graduation. Working in close partnership with the GOM, the project aims to develop and implement a successful and sustainable integrated school feeding model to help inform policy and improve the GOM’s NSFP. Additionally, the project will advocate for government budgetary expansion to increase the number of schools supported under the NSFP and aims to hand over ten project schools to continue school feeding beyond the life of the project under the GOM’s NSFP. Some key components of the project’s sustainability plan mentioned by respondents included: • School feeding will involve local producers. • Parents will be financially able to support schools, including school canteens, through involvement in saving and lending groups. • Local structures and associations, in particular the FEFFI45 school management committees, will be trained and made aware of the need for good management of school canteens. • MEN officials at district and local levels, school directors, and teachers will be trained to increase their competencies in taking ownership in the continuation of project activities. • The community, including students and parents, will be made aware of behavioral change approaches in various topics (i.e., WASH behaviors, gender equality, commitment to the education of children, financial independence,) and will be oriented towards progressive changes that can lead to sustainable development. Respondents highlighted that the project is working in a drought-prone area and that drought is the biggest threat in the South. Due to food insecurity, there are many other cash transfer and food distribution projects also working in the region. The WASH component of the program plans to rehabilitate existing water infrastructures and install new infrastructures, such as drillings and impluvium (water catchment systems), as much as possible, which should remain beyond the life of the program. Respondents highlighted the importance of the project to engage local entities in the management of this infrastructure and strengthen entities under the Ministry of Water to preserve these infrastructures. Respondents also mentioned that the local climate and water 45 FEFFI was set up by the MEN in 2015 according to decree 707-2015 in all Public Primary Schools to manage activities at the school level, including associations, FRAM, etc. The members of FEFFI are composed of students’ parents, the community (optional), local authorities, teaching and administrative staff, school directors. The FEFFI is a key structure involved in the school feeding program as the GOM plans to have the FEFFRI manage school funds for school feeding and other oversight roles. 64 problem could also demotivate farmers. Agents of the Ministry of Agriculture at the district level stated that farmers still face an unfavorable climate (notably contributed to by the prolonged drought); therefore, they are sometimes demotivated to continue farming because planting is often affected by the lack of rain. According to the results of the interviews, the project will need to communicate to parents and communities early on how they will need to engage in order to continue supporting the school canteens after the project ends. However, it is clear to respondents that a five-year period may not be enough to ensure that canteens will be fully operational and sustainable at the end of the project. Project staff affirmed that the project's sustainability plan is quite complete and considers all the actors who must be engaged to ensure this sustainability. However, the evaluation team identified through the interviews additional opportunities that may contribute to the program’s sustainability as well. For example, respondents stated that the project should focus efforts on mobilizing community groups or associations of local producers to support their growth in skills and ability to increase their production capacities to supply food for school feedings. The program can also consider working with agricultural projects that are implemented in the Androy region. Other local associations, such as women’s associations and student clubs, can also be strengthened or created to reinforce topics promoted by the project (such as raising awareness of women's independence, teaching young girls about menstrual hygiene, and promoting climate and nutrition sensitive agriculture). The program plans to involve all key stakeholders (government, teachers, school directors, parents, and community groups) to ensure progress toward handing over schools to the NSFP. The government is an integral part of the program, from the MEN at the central level to the ministerial structures at the local level. Additionally, the program engages the FEFFI, the school management committee that will be responsible for management of school feeding funds and that will remain even one the Mianatra project ends. All these actors will be institutionally strengthened during project activities through training, capacity building, and awareness raising. The project plans to co-create a sustainability plan and transition strategy in partnership with project stakeholders at all levels. However, because the project has recently begun, it was noted by respondents that there is not yet a defined and clear commitment made by the local actors for a supporting locally owned and managed school feeding program. Throughout the life of the project, Mianatra’s National Program Advisor will serve as a liaison between Mianatra, GOM ministries, and key technical and funding partners to share project success and lessons learned, identify opportunities to leverage programming for greater impact and coordination, and advocate for the integration of successful practices into policy and budget line items. Mianatra will host ministry meetings and workshops and participate in technical working groups to ensure Mianatra’s lessons learned are shared and used to inform and improve GOM policies, plans, and programs (e.g., National School Feeding Law, National School Food and Nutrition Plan). A major objective of the program is to strengthen the capacities of the government to manage and take ownership of program activities. However, the MEN staff raised a point concerning motivation. It was noted that some agents that work on behalf of the government or other local structures are volunteers and do not receive any remuneration. This can be a source of demotivation because people in the same position are paid in other projects. Respondents also stated that while the structures of the ministry are well in place, it is the capacity and the competence of the staff which are not always sufficient. Respondents mentioned that it would be 65 necessary to train all staff at the district and commune levels initially and then conduct an annual refresher course. They must also be made aware of the importance of their work and their contribution to improving the quality of children's education so that they can carry out their work loyally. Training and awareness-raising in ethics can also be useful. The significant challenge for the government is also its financial means. Ministry staff at the district and commune levels need means of travel to be able to visit schools regularly and facilitate their involvement in the implementation of the project. 4. Conclusions 4.1. Improved Literacy of School-Aged Children This project aims to support sustainable literacy improvements and school feeding program ownership by the GOM and school communities. With its strong focus on improving the literacy of school-age children, this baseline study serves as an opportunity to measure the current actual level of reading competency of the students targeted by the project, which will be the comparison point to estimate progress made by the project in this area over time. Student literacy levels were measured as the percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text through the EGRA assessment’s reading comprehension subtask. Overall, the literacy levels of students targeted by the project were very low, with only 6.5% of students in project schools able to reach the minimum level of competency on the reading comprehension subtask. No statistically significant difference in literacy levels between students in project vs control schools was found meaning that even if we observed differences in this sample, we cannot conclude that a difference exists in the minimum level of competency in reading comprehension among students in the project versus control schools. Furthermore, no statistically significant differences were also observed in the average scores of the different EGRA subtasks suggesting that the level of reading competency of the population in treatment and control schools could be considered similar. As expected, reading fluency and reading comprehension subtasks are among the subtask with the highest proportion of students who cannot achieve a single correct answer. Familiar words were also rather difficult for a good proportion of the students, especially in treatment schools where more than 40% of the students were unable to read a single familiar word correctly. Surprisingly, listening comprehension reveals to also be very difficult for a good proportion of the students where more than half of the students in the treatment group were unable to hear a short story and answer a single listening comprehension question correctly. Usually, this is a rather easy task for the students; however, this could be explained by the fact that the language used in the text for the assessment was official Malagasy – the official language of instruction in primary schools in Madagascar – which could be less familiar to students in the population who typically speak a local dialect of Malagasy (Antandroy). In these schools, girls significantly outperformed boys with 9.2% of girls reaching the minimum level of competency compared to 2.1% for the boys as well as on average score of all the EGRA subtasks. This observation is coherent with what was observed in other African countries using similar tools. Mianatra should take this difference into consideration and in the final analysis of 66 the program impact, will have to measure the impact overall but also by gender to evaluate if the impact of the program has any gender differential effect. The project’s theory of change emphasizes the importance of increasing student attentiveness in order to increase student literacy since school attendance is insufficient for students to acquire new reading skills. Students need to be present and active in their learning during classroom time, which may be influenced both by project efforts to reduce hunger during the school day among students through school feeding as well as improving instructional techniques among teachers. In this evaluation, attentiveness is measured by observing students during a classroom period and recording if they were off task during the observation period. At baseline, 44% of Grade 4 students have been observed off-task at least one time during class, and similar levels were found in the control group. One of the principal reasons for inattention noted by teachers was student hunger, specifically that many students sleep in class during the school year because they have not eaten anything at home. The baseline survey with students shows that more than half of students in project schools either never (16%) or rarely (41%) eat breakfast. Attending school on an empty stomach can affect students’ ability to be attentive during class and potentially limit students’ learning. Providing food for a student at school has the potential to improve attentiveness as well as to incentivize students that do not have access to sufficient food at home to attend school. Access to school is another factor limiting school attendance and the development of literacy skills of the students. Baseline study results reveal that just over half of parents in the treatment group can identify three benefits of children completing primary school; parents in control schools were significantly less likely to identify benefits of their children attending school. Money is another barrier for parents to send their kids to schools and some resources are already available in the two targeted districts. Saving and lending groups provide financial resources to support families to send their children to school. While most of the active participants in these groups stated that they have sufficient financial resources to send their children to school, only 28% of the parents in the treatment group are currently active members of a saving and lending group. With the significantly lower than expected attendance of students observed during data collection, it is important to note that the project design (and the supporting evaluation design) is school￾based, and therefore only reflects students actually attending schools. The population of out-of￾school students, students who are eligible to attend school but for whatever reason are not doing so, in the two treatments district is difficult to estimate but the project may consider efforts to strengthen community-level awareness and support on the importance of childhood education so parents may be encouraged or supported to enroll their children in schools. Improving access and participation in saving and lending group presents opportunities to increase household financial security, which can increase opportunities for parents to send their children to school. If project activities do increase access and attendance of children who are currently out-of-school, the potential increase in students attending school may be a positive program outcome but may present a limitation in the evaluation design to detect a positive impact on the level of reading competency of the students. Out of school youth are likely to enter school with relatively lower levels of reading competency and less support at home than their in-school peers but will be included in the treatment group while the population of the control group will remain probably similar. Efforts should be taken to monitor and evaluate any “return to school” activities, and in the midline and endline evaluation, to estimate the proportion of the treatment population that are 67 newly enrolled or “return to school.” This should also be considered in the interpretation of the endline results. Mianatra should also take into consideration the low level of existing skills and instructional competencies among teachers that are reported by some stakeholders participating in the baseline; this is most likely explained by the low levels of formal education and training among most teachers before teaching. This is a potentially important limitation of the project impact; without qualified teachers even if students are attentive during class, they will not successfully develop their literacy skills. Training of those teachers is one of the activities planned by Mianatra, but this training should take into consideration their existing levels of capacity as well as how to best reach and support this specific population of teachers through trainings and other project activities. 4.2. Increased Use of Health, Nutrition, and Dietary Practices The second main objective of this project is to increase the use of health, nutrition, and dietary practices. Baseline results show that approximately one in four students in the treatment group have been absent from school because of a health problem in the past two weeks. In project schools, cough and diarrhea were the main health-related reasons affecting student absenteeism reported by the parents. While malaria, a leading cause of childhood morbidity and mortality in Madagascar, especially in the southeast, was not identified as a health condition affecting student attendance in the treatment group, two-thirds of parents in the control group reported malaria as the most common health-related condition affecting school attendance. This was surprising as malaria is relatively less prevalent in the Androy region. Students show limited knowledge of the different health and nutrition related practices. Only a small percentage of students in the treatment group can identify three critical moments for handwashing but at least more than 50% identify before and/or after eating and after defecation as critical moment, so there is an opportunity to improve students' knowledge in hygiene. Similarly, on questions on identification of important nutrition/dietary practices, most students were not able to provide three correct nutritional practices; however, the proportion of students identifying the importance of eating foods prepared hygienically as well as washing hands before eating was high. While knowledge of handwashing was high, knowledge of other critical opportunities for improved health and hygiene were more limited. It seems that most of the students understand that handwashing is an important practice for health, hygiene, and nutrition; therefore, the program should continue to focus on this practice but also work on the development of the knowledge of students regarding the other important practices. No significant differences between boys and girls have been observed regarding knowledge of health and nutrition practices. The project should make sure that this will be the same at endline, so the program has no differential impact regarding the gender of the students. To study this, additional analysis will be run at endline to compare the change for the boys to the change for the girls to check if there is any difference. 4.3. Project Relevance, Coherence, Effectiveness, Efficiency, & Sustainability Findings show that the Mianatra project is relevant to the context of intervention, aligning fully with the Government of Madagascar’s priorities in education. By using local capacity (for example, trainers will be technical staff from the MEN) the project will contribute to advancing the 68 capacity of local stakeholders. The project responds to some of the biggest constraints as expressed by local stakeholders, supporting education quality, and contributing to reduce hunger. The project sustainability plan is complete and well known by project staff. By engaging local structures throughout the project and for all the activities, the project is focused on creating a lasting impact. While Mianatra does not anticipate fully graduating schools within this five-year project, it does seek to assist progression toward graduation. An important limitation to a feeding program reported by many respondents is access to potable water. According to data collected in this study, access to water is the biggest problem of the population in the South. It is difficult to find clean water and some places do not even have any reliable source of water. Among the 60 project schools that we visited, 2% have access to water at school, 8% can find water near the school, 20% can find water but far from the school, and 70% did not have any source of water. Without clean water, it could be difficult to prepare meals for students at school. This will be a significant challenge the program faces. Also, limitations in access to clean water will mitigate the positive impact of knowledge of correct health and nutrition practices since many of those practices require access to clean water. Finally, the program should also be aware of possible misappropriation of food by school administrations, and close monitoring of food distribution will be required by the project. 5. Recommendations The recommendations were developed by the Evaluation Team based on the findings and conclusions of the baseline evaluation report to provide the project team with recommended avenues for improvement in the implementation of the project. Once the baseline report is finalized and approved by the donor (USDA), the full results will be shared with the project team and key stakeholders, recommendations fine-tuned as needed for the project and country context, and operationalized, including identifying responsible actors and defining timelines. 5.1. Recommendations for Program Strategy and Implementation 1. The project must work with relevant stakeholders to determine and update the number of students enrolled and actively attending project schools and identify any potential adjustments to project activities. 2. The project should map / assess the number of out of school children in target communities to identify the population of children who are eligible to attend schools but are currently not enrolled and/or attending. The project may also consider collecting feedback from parents and other community stakeholders on barriers to student enrollment and attendance to identify any possible strategies or approaches the project could consider (e.g., sensitization activities for parents and communities, to enroll and/or reintegrate these out of school children into schools). If the project does pursue activities to encourage new enrollment or reintegrate out of school children into schools, the project should develop ways of monitoring and evaluating these efforts and incorporate this into sampling or analysis at midline and endline, to avoid any potential biases that may be introduced by such strategies. 3. The project should consider reviewing project plans and budget allocations and identify/develop school-specific strategies to increase the number of schools with reliable access to water to ensure the proper functioning of the canteen as well as the respect of health and hygiene practices. 69 4. Given the importance of removing financial barriers for students to attend school and given the relatively low current participation rates of parents in savings and loan groups, reviewing and updating the mapping of existing saving and lending groups in the project area may help to inform project activities to strengthen and/or establish new groups. 5. Prior to establishing any larger school gardens or staple farms, the project should consider updating their mapping exercise and assess potential ways the schools can access bigger plots of land that can realistically be managed and sustained. Given the increased challenges with climate change and water insecurity, this updated mapping may be useful to help the project team to review the current targets related to school gardens and farms and consider a more modest target. 6. Given the number of stakeholders involved in implementing and ensuring longer term sustainability of these efforts, including schools and communities, the GOM at all levels, and other implementing partners, the project should consider additional sensitization and dissemination activities, in addition to initial project launch activities already conducted by the project, to make sure that project strategies, approaches, expected results, and roles and responsibilities are known and understood by all project stakeholders, especially by those in charge of implementing project activities in order to increase awareness and accountability. Given the importance of the sustainability plan in the project and feedback on ensure sustainability and ownership of the activities by the stakeholders, the project should embed discussions about sustainability and co-create a gradual transition strategy at the onset of project activities at all levels (central, local, and school level). Sustainability plans should include defined roles and responsibilities of all actors and the project should ensure they are disseminated to all project stakeholders to ensure awareness, ownership, and accountability to operationalize the plans. 5.2. Recommendations for Monitoring and Evaluation 1. Ensure that midline and endline studies will be conducted at the same time of the year and with the same grade (students entering grade 3) to ensure comparability of results, especially around potential seasonal variability and impact on school attendance, health related issues, and food security related issues. 2. Ensure consistency of approach in data collection and analysis of survey questions with parents and students in midline and endline, in order to ensure uniformity with baseline analysis; this can be achieved through review and alignment of baseline documentation (e.g. data dictionary, field SOPs, data cleaning notes, analysis scripts, etc.). 3. The evaluation team recommends the project reviews the current indicator and indicator definition for measuring students’ attentiveness in the project’s PMP and consider some of the challenges faced in this baseline evaluation. The evaluation team identified two issues that make this indicator challenging. First, the indicator definition defines students as inattentive if there is only one instance of being off task. The evaluation team questions if one instance is enough to be considered inattentive. Second, at baseline, this indicator was collected when class sizes were small (average of 6-8 students per class). The evaluation team notes that if class sizes are bigger at midline or endline, this may affect results on student attentiveness. The project could consider revising the target or collect additional data through ongoing monitoring and implementation of programmatic activities to capture data on student attentiveness in alternative ways. However, if updates to the indicator approach are made at midline or endline, baseline and any other subsequent values would need to be recalculated (as possible) to ensure 70 consistency over time. The evaluation team encourages the project team to work with USDA to identify how indicators on student attentiveness can be further improved. 4. Given the very low baseline values for literacy (6.5% of students in project schools meeting literacy standards), the project team should review and work with USDA to consider possible revision of targets (currently to increase by 7% by midline and 15% by endline). The evaluation team recommends a more modest target (e.g., increase by 3%) in line with experience in other countries. Annexes Annex 1: Mianatra Results Framework 2 Annex 2: Baseline Evaluation Terms of Reference I. INTRODUCTION A. Global Communities The Cooperative Housing Foundation, doing business as Global Communities, envisions a world of expanded opportunity, where crises give way to resilience, and all people thrive. Global Communities works to build this better world with a spirit of constant innovation and commitment to confront historic barriers to dignity, equity, and inclusion. Global Communities’ mission is to bring together local ingenuity and global insights to save lives, advance equity, and secure strong futures. This is achieved through collaboration with a wide range of public and private sector partners, including communities directly affected by poverty, disasters, and conflict. The following values guide the work that Global Communities does: • Dignity: We respect the dignity and humanity of every individual, and we are committed to equity, inclusion, and justice in everything we do. • Integrity: We consistently hold ourselves and each other to delivering excellence and to the highest professional and ethical standards. • Humility: We are transparent when we make mistakes, trust the wisdom of the communities we serve and seek to preserve the natural environment • Creativity: We are curious, seek to innovate, and embrace a spirit of experimentation • Connection: We actively collaborate with all our stakeholdersto improve understanding of the complex issues and catalyze lasting change. The central goal is to build the capacity of communitiesto direct the development of their own lives and livelihoods. Empowered communities are able to drive change when they recognize everyone’s needs, understand their rights and the natural and market forces that affect them, and are able to take collective action. Once empowered, communities are better able to build constructive relationships with government institutions, the private sector, and civil society in ways that are sustainable. In turn, local government, the private sector, and civil society have a greater ability to address community needs. Global Communitiesis helping communitiesin 35 countriesin Africa, Asia, Europe and the Caucasus, the Middle East & North Africa, and the Americas. For more information visit https://www.globalcommunities.org/. B. Mianatra Program Global Communitiesis implementing the first U.S. Department of Agriculture (USDA)-funded McGovern￾Dole International Food for Education and Child Nutrition (McGovern-Dole) project in Madagascar, called Mianatra (Malagasy for “Learn”). Global Communitiesleads a consortium with sub-recipients Catholic Relief Services(CRS), FHI 360, and local partner Malagasy Mahomby (MM), to implement Mianatra and build a five-year alliance with the Government of Madagascar (GOM) to assist their goals of improved student literacy and health and expansion of their National School Feeding Program (NSFP). Mianatra focuses on food-insecure regions in the Deep South in Madagascar that are most vulnerable to 3 low school enrollment, poor attendance, high dropout rates, malnutrition, and food insecurity leading to among the poorest education and nutrition outcomes. Mianatra’sfocus is on expanding the Ministry of National Education’s(MEN) efforts through complementary activities and strengthening the essential capacities necessary in co-creating the pathwaysto sustainable literacy improvements and school feeding program ownership by the GOM and school communities. With many foundational pieces in place, including GOM policy commitmentsto an improved early grade reading (EGR) curriculum and a NSFP, Mianatra is designed to seize the opportunity to augment, integrate, and coordinate resourcesin meeting McGovern- Dole program objectivesto produce meaningful and lasting results, and establish clear pathwaysfor the GOM and communitiesto take ownership of sustaining the benefits of improved learning and student nutrition through school feeding. Mianatra is budgeted at $25,000,000 USD and will target 130 schools in the Beloha and Tsihombe districts within the Androy region. Over the life of the project, 90,136 direct beneficiaries, including 64,177 preschool and primary students, will benefit from activitiesfocused on the improvement of students’ literacy, access to food through school feeding, health and nutrition, school water, sanitation, and hygiene (WASH), and supporting systems. Emphasis will be placed on inclusion, equity, and appropriate accommodation to address issues related to gender and for CWD. Mianatra will build clear pathwaysto sustainable school feeding for lasting impact and benefits of improved education, enrollment, and attendance. Contextual and Environmental Challenges Mianatra presents a unique operational context that calls for Global Communities and partners to use an adaptive management capacity to respond to and overcome constraintsfor optimal McGovern-Dole program success. Global Communities will draw upon the deep experience of its local partners to help mitigate constraints as well as take advantage of opportunities. Access is challenging in Southern Madagascar outside of large port towns like Tulear. Remote schools are only accessible via dirt roads and flood-prone bridges, slowing supply lines linking rural communitiesto government resources. Householdsfeel the impact from climate-related shocks, particularly erratic rainfall and the increasing severity, frequency, and intensity of droughts. This makes socio-economic recovery for the heavily agriculture-dependent population difficult. The meager resourcesthat a family might have saved up for investment in health, education, and livelihoods expansion is spent on recovery from these shocks, essentially eroding any progress. Mianatra includes activitiessuch as the distribution of tree saplings and a ‘green meals’ approach that seeks to reduce the negative climate and environmental impact of school meal preparations/waste. This in turn is expected to lower the carbon footprint through the use of fuel￾efficientstoves that reduce fuel consumption and emissions up to 50%. The project also encouragestree planting and composting appropriate school waste for garden use. Furthermore, the project leverages systems in the Maharo RFSA1 to improve food security, strengthen savings groups, and share promotional messaging on education. Communities with access to reliable water sources will be identified and supported to practice climate smart agriculture,school based grey water harvesting systems will be put in place to maximize recycling of water used for food preparation and hand washing for watering school gardens, rainwater harvesting and storage systems will be installed where practical, and Global Communities will explore piloting and potentialscaling up of small scale solar powered water desalinization systems for ocean front communities. Additionally, a number of health shocks also exist, including outbreaks of measles, plague, malaria and most recently, COVID-19. Mianatra’s activities 4 include nutrition, health, and WASH messaging and infrastructural interventions aimed at mitigating or reducing severity of health-related shocks. Systemic challenges These include insufficient government funding, donor dependency, and turnover of administration staff which has led to low funding and governmentsupport for social programs,such as school feeding programs. As little as 2% of the GDP or 15% of the national budget is allotted to the MEN, which is insufficient to revitalize and expand an education system that averages one school to every 990 HHs. Key donors, such as the World Bank (WB), World Food Program (WFP), United Nations International Children’s Emergency Fund (UNICEF), and now USDA’s McGovern-Dole program, have stepped in with much needed funding, and all projects are implemented in partnership with the MEN. Donor programs have developed and updated plans and structuresto allow the GOM to expand on and sustain the foundation that programs have built. Mianatra will be a key partner in supporting the new NSFP initiative announced by the President of Madagascar allocating $4.5 Million USD in cash transfersto 2,695 schools to procure foods locally for school meals targeting over 400,000 students. Project Catchment Area Mianatra will target 130 schools in the Beloha and Tsihombe districts within the Androy region. C. Project Objectives Global Communities will, over a period of approximately five years, use the U.S. donated commodities and any funds provided by USDA to implement a school feeding project in southern Madagascarfocused on achieving the following objectives: 1) Improve the quality of literacy instruction through the distribution of school supplies and materials, establishment of libraries, production of books and supplementary reading materials, training teachers, school administrators and school management committees, and the distribution of distanced education materials. 2) Improve attentiveness and reduce short-term hunger through the provision of school meals, establishing of school gardens, and local and regional procurement (LRP) of commodities for school meals. 3) Improve student attendance by increasing economic incentives for parents by forming savings and lending groups and increasing community understanding of the importance of education. 4) Increase the use of health, nutrition, and dietary practices by providing training on good health, nutrition, and WASH practices, safe food preparation, storage, and commodity management practices, and distribution of deworming medication. 5) Increase access to clean water and sanitation services through the construction and rehabilitation of block latrines, washrooms, wells, and handwashing stations; and 6) Increase the capacity of governmentinstitutions, improve policy and regulatory frameworks, and increase engagement of local organizations and community groups by working with community and government structures, facilitating sustainability planning and measurement, and supporting the Government of Madagascar (GOM) to scale and sustain their NSFP. 5 II. TECHNICAL SPECIFICATIONS A. Purpose and Scope of the Baseline Study USDA requests that baseline information be collected by the project within 6 (six) months of the project award date, in order to set accurate and realistic targets and to enable the project to monitor progress and performance throughout the life of the project. Global Communities expectsthis baseline study to be fully integrated in the body of information that the project will use for performance monitoring and evaluation, and for learning. The baseline study will be the first of three surveys that constitute the overall impact evaluation. Follow-on surveysinclude the mid- term and end line evaluations. Sampling strategies between the three surveys will allow for detecting the difference between the proportions and means. Additionally, the baseline study should provide verified information on relevant contextual factors that may have a bearing on the implementation of activities. Some contextual factors may include governance at the school, community, district, regional and national levels, perceived nutritional and health needs of the beneficiaries, as well as school management committees’ needs for capacity building. The study will also examine general environmentalfactors,structural influences, and culturaltaboos/traditionsthat may have influences as enablers or barriers to the proposed interventions. The findings will enable project staff to validate the design of the project and, if needed, adjust interventions. The study should capture gender and social inclusion dynamicsthat will be factored into future programming. The baseline will achieve the following broad objectives: o Establish baseline indicator values against which to measure progressthroughout the life of the project. o Provide recommendationsto maximize efficiency and effectiveness of project monitoring. o Refine performance indicator targets and generate contextual data to tailor program activitiesto school and community needs. o Identify opportunities and threats to implementation. o Develop recommendationsfor strategiesto maximize strengths, mitigate challenges, leverage opportunities, and assess beneficiaries’ knowledge and skills to refine capacity building and training plans. o Validate and/or refine project strategies and assumptions. B. Study Questions The baseline approach will be guided by the relevant Organisation for Economic Co-operation and Development (OECD)2 evaluation lenses (relevancy, coherence, efficiency, effectiveness,sustainability, and impact) to use in follow-on discussions on how project implementation may be adjusted and improved. The 6th criterion-impactis left out of the baseline. The responsesto the questionsshould be based on the information discovered through the investigation process. The Local Contractor will work closely with an international Technical Advisor to develop this portion. Operational and Design Questions 6 Table 1 providesillustrative questionsfor the program and design inquiries. The categories are internationally recognized Table 1: Baseline Questionsto Address Project Operational Design Relevance and Coherence • Is the program relevant to the desired outcomes and objectives of the McGovern- Dole strategy, policy, and plan? Is the project’stheory of change realistic, appropriate, and practical to implement? Are there any modifications needed to the project’s design,structure, logic or management to facilitate the achievement of the desired outcomes and impact? Are the project’s proposed strategies and interventionsto increase knowledge and strengthen capacity of key stakeholders(i.e., GoM, schools, savings groups) appropriate and realistic to achieve the expected results and outcomes? Is the program relevant to address the real, and perceived needs of project beneficiaries(e.g., students, teachers, school administrators, parents/guardians, CWD, government officials)? How well does the program complement, leverage, and align with other ongoing education, nutrition, and school feeding programs within the country and other GoM plans and priorities? Is the project design suited to improving the level of early grade literacy performance and overcoming the main challengesto improving literacy outcomes? Is the program designed to be flexible, for example, can the overall strategy, components, or specific activities be adjusted over time due to changing environment or in response to target population needs? Effectiveness To what extent is the COVID-19 pandemic likely to influence program effectiveness/results and how best can the program address these influences? What are the main challenges or barriers to achieving desired outcomes across other related project components(i.e., health, nutrition, and dietary practices, strengthened capacity to manage school feeding activities, improved student attendance and attentiveness)? Is there a clear understanding of roles and responsibilities by all parties involved for implementation and monitoring? Are the project’s monitoring and evaluation strategiesrealistic, appropriate, and practical to implement? Efficiency How efficient and realistic is the planned allocation of resources(human resources, time, expertise, funds etc.) to provide the necessary support and to achieve the broader program objectives? 7 Sustainability Is the project’ssustainability plan realistic and appropriate? What undeveloped opportunities might be at hand that can contribute to the sustainability of project benefits and outcomes? Does the project have the means to address threats or gaps (i.e., climate-related shocks, COVID-19) that could impact progresstoward sustainability and sustained outcomes/benefits of the project? To what extent have stakeholders and local actors (e.g., GoM at all levels, private sector, community groups, other organizationsimplementing school feeding programs) been engaged and committed to ensuring progresstoward a sustainable NSFP? To what extent have stakeholders(i.e., GoM, communities, private sector) provided monetary or in-kind support for school feeding and complementary activities, and are these sustainable inputs? What commitment has been made in support of a gradual transition toward a locally owned and managed school feeding program by the GoM, schools, and communities? Are all the elements of sustainability (capacity, motivation, resources, linkages) addressed in the project’ssustainability plan? What are specific capacity needs for project stakeholders(e.g., GoM, parents, school administrators,school management committees) that the project should address to foster greater ownership and accountability to sustain project benefits? To what extent is the program design anticipated to have a post implementation impact on the lives of the program participants? Since the purpose of study is establish initial indicator values before the commencement of activities, the contractor(s) will design a rigorousstudy that will provide information which is clear, distinguishable, measurable, and understandable that will form a foundation against which subsequent evaluations will be compared. III. EVALUATIONSTUDY DESIGN/METHODS In line with USDA’s evaluation requirements, the baseline study will employ a quasi-experimental approach, specifically the Non-Equivalent Groups Design (NEGD). This approach will be structured along pre-/post-test lines and will use comparison groups at the baseline, mid-term, and final evaluations. However, assignment to either treatment or comparison group will not be randomized. In other words, the evaluator(s) will not control the assignment to either group through a mechanism of random assignment. Instead, they will be required to select communities and schools that are as similar to the treatment (beneficiary) communities and schools. To enable trend analysis overtime, the same communities and schools will be targeted to participate throughout the life of project (LOP). 8 Treatmentschools will include public schools and their surrounding communitiesin the two districts receiving the project intervention – Beloha and Tsihombe. Treatmentschools will include both schools receiving US donated food commodities and schools benefiting from the LRP commodity stream. Comparison groups of public schools that are not benefitting from the McGovern-Dole program nor the World Food Programme will serve as a proxy for the counterfactual – e.g., without a. school feeding program children do not progressin their educational development. Comparison schools will be randomly selected from comparable districts within the same targeted region of Androy. Comparison schools and communities will be selected based on the level of food insecurity,school enrollment, and school attendance and whether these key characteristics are comparable to the treatment group. Comparison group sample size will be determined by the sample size needed to ensure statistically significant findings. This will allow for a difference-in-difference (DiD) analysis approach that provides an estimation of the causal effect. Mianatra will work closely with key stakeholdersin Madagascar and the Evaluation Team Contractorsto create sampling frameworks. The Evaluation Team Contractors will identify the appropriate DiD approach based on the sampling framework developed/adapted,such as Propensity Score Matching (PSM), stratification, or weighting data. These methods will enable the contractor(s) to measure attribution of program interventionsthrough to the midterm and end line evaluations. In close coordination with the Technical Advisor, the Local Contractor will propose the overall study design with a mixed-method data collection approach (qualitative and quantitative) to address the evaluation questions and plan for collecting and analyzing data. Suggested data collection approaches include a desk review of relevantliterature, quantitative surveys, and qualitative study. This being the first of the three studies, the combined team of evaluators will be expected to design a comprehensive field manual, data collection instruments, and propose a sampling strategy, that will be translated into French, Malagasy, and the local dialect spoken in Androy region. Additionally, the evaluation team will develop and submit a clear data analysis plan for both qualitative and quantitative data. Each evaluation exercise (baseline, midterm, final) will measure literacy among students after two years of primary school using the Early Grade Reading Assessment (EGRA) toolkit, which is consistent with the GoM’s literacy evaluation approach and may be aligned with pilot tools being tested by the MEN. The EGRA is administered verbally by a trained assessor, one-on-one with a child. Additional quantitative tools will capture data on school meals, student and teacher attendance,student attentiveness, and health, WASH, and nutrition practices. Quantitative tools will include surveys among a representative sample of students, parents, and teachers, observational checkliststo track student attentiveness and school infrastructure, and school records to assess attendance. Survey tools and questions will be the same or similar between intervention and comparison areas to maximize comparability. The list of indicatorsto be collected is provided in Table 2 below. Table 2. List of Project Indicatorsto be Collected at Baseline Activity Area Indicators Strategic Objective 1: Improved literacy of school children Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text Percent of students who are off task (data collected in a single day, at specific intervals) Percent of savings and lending group members with primary-school aged children who report they are financially able to send their child to school 9 Percent of parents/guardians who can identify three benefits of children completing primary school Strategic Objective 2: Increased use of health, nutrition and dietary practices Percent of parents who report their child has had diarrhea, fever, vomiting, or cough in the past 2 weeks Percent of parents that report their child had a health-related absence from school in the prior 2 weeks Percent of students in targeted schools who can identify at least three important health/hygiene practices Percent of students in targeted schools who can identify at least three of the five critical moments for handwashing Percent of students in targeted schools who can identify at least three important nutrition/dietary practices A. Sampling Quantitative Sample To reduce logistical challenges, Global Communitiesrecommendsthe use of a multi-stage cluster sampling strategy that will ensure statistical and geographic representativeness, as well as equal inclusion of male and female participants, and to the degree possible, inclusion of CWD. To draw a representative sample, the contractor will use probability-based methods where every unit in the population has known, non-zero probability of selection and where selections are made using well￾established random mechanisms. The overall sample size should be sufficient to detect a certain percentage point difference between baseline and end line values. The following parametersshould be taken into consideration during sample size derivation: 95% confidence interval for one tailed test at 80% power. Additionally, the evaluation team contractorsshould consider an appropriate sample inflate rate of say 10% to account for non- response among the differentsampling units. • Stage one: random selection of clusters (e.g., schools, communities) from all clusters in the sampling frame that will be provided by Global Communities • Stage two: selection of class or household within a school and community respectively • Stage three: random selection of individual respondents The evaluation team contractors will propose a sampling approach based on the above guidance, the sampling frame that will be provided, and the most current Madagascar census information. Below we propose a sample size derivation approach that the evaluation team contractors may consider using: • Involve a multi-stage sampling approach to establish the sample size for schools based on the list of 130 potential treatmentschools. The recommended formulas are p= population size, e = margin of error, and z = the standard deviation. To reduce the risk of bias comparison group determination, it is expected that the evaluation team contractors will make use of available techniquesfor creating a valid comparison group; for example, 10 regression discontinuity design (RDD) and propensity score matching (PSM). The bias of concern here is ‘selection’ bias, or the possibility that those who are eligible or choose to participate in the intervention are systematically different from those who cannot or do not participate, which could potentially lead to observed differences between the two groups being due– in full or in part – to an imperfect match rather than caused by the intervention. Qualitative Sampling The baseline study will involve a qualitative component to inform and contextualize the interpretation of the quantitative findings. As such, the evaluation team contractors will propose in the design: qualitative data collection methods/design,sampling methods(e.g., purposive sampling) including the inclusion/exclusion criteria to be used, corresponding training, data management and analysis, and reporting plan (i.e., data collection, transcriptions, translation, coding, analysis, and quality control/assurance). The evaluation team contractors will design the overall qualitative study approach and will consider a variety of primary data collection methodssuch as semi-structured in-depth interviews, focus group discussions, key informant interviews, direct observation, and/or case studies. As with the quantitative surveys, qualitative sampling should include both individuals who will be directly participating in the intervention and those not specifically targeted by the program. The purpose of interviewing non￾participantsis to understand whether the proposed activities are likely to negatively affect community members who do not participate in the intervention. In addition, the evaluation team should interview USAID, USDA, Mianatra project staff, knowledgeable people from the community, local governmentstaff, community leaders, host government officials, and staff of other agencies and organizations as appropriate. B. Data Collection Methods and Instruments The Local Contractor will propose a detailed methodology and evaluation work plan to carry out the framework suggested by this TOR, which will be reviewed and supported by the Technical Advisor. Table 3 below summarizesthe different data collection methodsto use in the comprehensive evaluation design. 11 METHOD RESPONDENT PURPOSE Document Review N/A To validate alignment and linkagesto GoM policies, laws, and strategies and existing programsthat may impact Mianatra’s operations and implementation. Early Grade Reading Assessment (EGRA) tests. Children at the end of Grade 2 EGRA T2: To understand if at the end of two grades, a student can read grade-level text. Individual interviews use a pre-coded assessment to check the literacy level of a student who has completed two grades of education. Table 3. Methods by Respondent and Purpose Structured Parents, caregivers To assess attitudes, knowledge and practices of parents on school feeding, education, health, nutrition, WASH and savings groups. interviews Fidelity of Teachers To explore and understand what teaching and learning methods are used in the class, the level and type of teacher student engagement, and actual field implementation of the project. implementation tool instead (To be adapted from the MMT project) Structured Students To gain information on students’ understanding of interviews WASH, gardening Key Informant MEN (DREN, Chef Cisco, Chef Zap levels), other relevant line Ministries like MoA, MoH, MAEH Insights on policy perspective from key Ministry officialsto understand the relevance of the program, commitment to sustainability. Interviews Key Informant Officialsfrom other associated and relevant Ministries Insights on collaboration, relevance of the program. Interviews Key Informant Project Implementing To understand the program in an in-depth manner including insights on the coordination, facilitation drawn for their respective component/activity – vital for operationalsuccess. Interviews partners Focus Group Community members, To explore if the program design respondsto the Discussions including those who may needs of the community, issues of targeting. participate or benefit from project actions. 12 C. Enumerator Recruitment and Data Collection Training The Local Contractor will identify and recruit local enumeratorsfor data collection, who should have a bachelor’s degree and experience conducting applied research, surveys, and evaluations. Both female and male enumerators with language skills in French, Malagasy, and necessary local dialects will be included. During the data collection training, supervisors will be identified and assigned this role. The evaluation team contractors will develop a Data Collection Manual for review by Global Communities’ MEL team before training. The local contractor will be responsible for printing all data collection instrumentsfor training, translating into official and local languages and finalization of the instrumentsfollowing the training and field testing, and finally reproducing all instrumentsfor data collection. In the event tablets are used, the Local Contactor will be responsible for all data programming and formatting on the tools. Print copy samples are also required for archival purposes. Training on the adapted EGRA tool will be conducted by Mianatra partner FHI 360 in conjunction with the Technical Advisor. Pilot Survey A pilot survey will be conducted to test all quantitative tools and verify the enumerator capacity to use the tools. The pilot is especially important because the same questions will be used for the subsequent evaluations and must elicit accurate responses. Sufficient time must be allocated for updating all instruments based on the results from the pilot survey. D. Ethical Guidelines The evaluation team must adhere to ethical guidelines as outlined in the American Evaluation Association’s guiding principlesfor evaluatorssummarized below: o Systematic inquiry - must conduct systematic data-based inquiries o Competence - the evaluation team must possess the education, abilities, skills, and experience appropriate to undertake the tasks proposed in these TOR. The evaluation team collectively must demonstrate cultural competence as well. o Integrity/honesty - evaluators must display honesty and integrity in their behavior at individual level and attempt to ensure the same throughout the evaluation process o Respect for people - evaluators will respect the security, dignity, self-worth of respondents, program participants, clients, and other stakeholders. IV. EVALUATIONTEAM CONTRACTORS’ ROLES AND RESPONSIBILITIES A. Evaluation Team Qualifications The overall evaluation team (Local Contractor and Technical Advisor) will mirror the diverse technical program areas of Mianatra program. The complexity of the program requires that team members have broad experience not only in their relevant fields but also the ability to apply their expertise in a multi￾disciplinary environment. Therefore, the Evaluation Team does not necessarily need separate personnel to examine every sector of the program, however, the evaluation team is expected to have within the team expertise in the following areas: • School feeding programming; McGovern-Dole programming a plus 13 Contractor Role Required Capacities Desirable Capacities Local Contractor Draft inception report and present report Develop evaluation design Develop the inquiry tools Obtain all necessary study approvals Support Global Communitiesto obtain GoM/MEN engagement Manage, supervise all data collection Lead data analysis Produce 1st and 2nd drafts of evaluation reports In-depth and educated understanding of country, context, and culture At least 5 years of evaluation experience, MS in statistics, international development, education, or any related background. Demonstrated expertise and experience in the techniques, approaches and methodology related to the collection and analysis of quantitative and qualitative data: sampling techniques, focus groups, surveys, semi-structured. interviews, crossing data e.g., gender, and content analysis French and local language mastery Experience and knowledge in the use of electronic data collection tools in evaluations Possession or ability to hire experienced enumerators that are proficient in local languages Experience in data cleaning, analysis, and interpretation Must be registered to operate in Madagascar Experience in report writing (samples required) English language competency • EGRA Assessment tool (data collection and analysis) preferred • Health and Nutrition • WASH • Commodity management • Capacity building • Gender B. Evaluation Team Composition and Roles Given the importance of the baseline in setting the foundation against which all future studies and will be compared, it is crucial to put together a strong team that possesses all the necessary skillsets. To this end, Global Communitiesis looking to pair a Local Contractor with an experienced Technical Advisor team to get the right mix of skills and contextual competence. Specifically, the Local Contractor team must possess evaluative skills (at least 5 years’ experience) encompassing both qualitative and quantitative data collection expertise, and language proficiency of French, Malagasy and other local dialects spoken in the program areas (Androy region). These Terms of Reference are specific to the role of the Local Contractor. Table 4 below illustratesthe assignment of responsibilities and reciprocalsupportive actions Table 4: Roles and Responsibilities of Contractors 14 Technical Advisor Review inception report Review and approve sampling strategy & overall methodology of the baseline study Review and approve data collection tools. Shadow the training of enumeratorsto provide quality assurance, ensure proper translation and communication processes. Review data analysis and provide feedback Review final evaluation reports and provide feedback 7-10 years of solid experience in evaluating USG- funded program/project(preferably USDA-funded projects) Demonstrated expertise and experience in the techniques, approaches and methodology related to qualitative data: sampling techniques, focus groups, surveys, semi-structured interviews, crossing data, and content analysis PhD preferred in social science (Education, Anthropology, Economics, Sociology). Excellent knowledge and experience in education programs Background in statistics and evaluation methods that use counterfactual and experimental/ quasi￾experimental approach with cohort analysis experience Professional level English and French proficiency Prior experience working in Madagascar is a plus. Knowledge and experience with EGRA T2. Excellent knowledge and experience in education program evaluation including administrating Early Grade Reading Assessment (EGRA) methodologies preferred. C. Roles and Responsibilities: Local Contractor The role of the Local Contractor is to design, develop, manage and execute the baseline study on the ground. The Results Framework is the foundation for the project Theory of Change and point of reference for the evaluation to assess program design and implementation. Specifically, the Local Contractor is expected to lead and manage the following: 1) Develop and Present the Inception Report Before commencing the baseline study, the evaluation team will prepare and submit an inception report detailing the evaluation design protocol to include: o A sampling plan, including the sample design, sampling frame, sample size calculation, level of statistical precision and power, and the respondentselection processes. o Training and field manualsfor the enumerators and field supervisors. o Plans for digital data collection. o Survey questionnaire development, translation, back translation, field testing plan. o Data analysis and management plan including estimation procedures e.g., sample weighting or adjustments and indicator tabulation and subgroup analysis. 15 The role of the Local Contractor will be to draft the inception report, seek input from the Technical Advisor for quality control and compliance before conducting a formal presentation to Global Communitiesfor feedback, negotiations, and eventual approval of the same. For more information on the contents of the inception report, see Table 6. Inception Report Outline below. The Global Communities Madagascar team will brief the evaluation team contractors on the cultural, political, and social context in which the data will be collected to aid them in developing a design that will do no harm. 2) Support Global Communitiesto Obtain Approvals and Endorsements by the MEN The Local Contractor will meet with the MEN and designated key stakeholdersto review and confirm the evaluation design and procedures. This may take the form of a stakeholder engagement workshop prior to launch of survey activities. The Local Contractor will document all approvals and informed consent procedures throughout the evaluation time period. The Local Contractor will also be ultimately responsible for obtaining local IRB approval if necessary. 3) Manage and Supervise All Data Collection The Local Contractor is responsible for all data collection work. This will encompass hiring and training enumeratorsif the Local Contractor does not have the requisite in-house human resources. The Local Contractor will oversee the deployment of the enumerators according to the sampling plan and timeline. All field logisticsfor data collection are the responsibility of the Local Contractor. 4) Lead Data Cleaning, Analysis, Interpretation of Results The Local Contractor is responsible for overseeing data cleaning to produce final analysis files. The Local Contractor will ensure that the six data quality standards are maintained: 1) Validity, 2) Reliability, 3) Precision, 4) Integrity, 5) Timeliness, and 6) Completeness. The Local Contractor is responsible for leading all analysis in line with the agreed analysis plan developed together with the Technical Advisor and Global Communities. The Local Contractorshould not be limited by the data analysis plan that was submitted in the inception plan, should there be emerging correlations or other trends that were not envisaged at that time. In which case, they will seek approval from Global Communitiesto adapt or revise the data analysis plan accordingly. 5) Produce Evaluation Reports The Local Contractor will write the first draft of the baseline report, that they will forward to the Technical Advisor for detailed comment/input. Following revisions, the Local Contractor will submit the report to Global Communities and other approved stakeholdersfor review and feedback. This may take the form of a workshop. The Local Contractor is responsible for documenting all feedback and responses and incorporating adjustments as necessary. D. Level of Effort (LOE) and Timeline An illustrative timeline should be developed based on the average LOE outlined below. 16 Activity/Deliverable # of Days Proposed Timeline Kick-off meeting 1 day May 2022 Documentation review and synthesis 5 days Inception report: Literature review, evaluation design including data collection. and analysis methodology; draft sampling strategy, and intended respondents/key informants; quality assurance plan; draft evaluation schedule; and draft data collection tools) (up to) 10 days for approval Inception Report/Presentation 1 day Development and finalization of survey protocol, methodology and sampling plan. 5 days Recruitment of enumerators 10 days Finalization and validation of instruments(questionnaires, checklists, observation forms, Interview guides, group discussion guides, etc.) 5 days MEN stakeholders’ buy-in workshop 1 day Training of enumerators 3 days Pre-test of survey instruments 2 days June 2022 Finalization of survey instruments. 2 days Data collection (travel to the field) (up to) 15 days Quality control and data analysis 5 days De-briefing on preliminary findings 1 day Draft report 1 5 days July 2022 Draft report 2 2 days Validation workshop (all stakeholders) 1 day Revise and submit report 3 to Global Communities & USDA for feedback 1 day Revisions to final report 5 days Submission of final report -- TOTAL (Up to) 80 days Table 5. Expected Deliverables and Estimated Days 17 E. Relationships & Communications The evaluation team contractorswill work underthe guidance of Mianatra Chief of Party, in coordination with the MEL unit and technical teams both in country and Global Communities HQ. A Point of Contact will be identified within Global Communities as the first line of communication. The Local Contractor and the Technical Advisor will establish a schedule of communications between themselves and hold weekly check-ins with Global Communities. These may include phone calls, emails, text messages, Skype/Zoom calls, or face-to-face meetingsfor effective coordination between both parties and to ensure that potential issues are resolved in a timely manner. Any anticipated changes to the plan during the evaluation must be submitted in writing to Global Communitiesfor approval. V. REPORTS A. Inception Report The report should be organized according to the outline suggested in Table 6 below with a matching PowerPoint presentation. Table 6: Inception Report Outline Report Length Maximum of 16 pages, excluding the Table of Contents, Acronym List, and Annexes and should be written in English; Times New Roman font size 12. Executive Summary: Include an Executive Summary that provides a brief overview of the evaluation purpose, project background, evaluation questions, methods, findings, and conclusions. Chapter 1. Project Background (to be replicated text from TOR) Chapter 2. Documentation review and synthesis Chapter 3. Work Plan: Literature review, evaluation design including data collection. and analysis methodology; draft sampling strategy, and intended respondents/key informants; quality aChssaurptaernce4. Splaamn;plin drafgt e: Sites valuati selecti on sochneprodule; cedur andesdraft data collection tools) Chapter 5. Logistics: Transport, communicationsschedule (call trees), etc. Chapter 6. Proposed Changes or Adjustments Chapter 7. Updated Calendar/Timeline Chapter 8: Quality Assurance Plan Annexes B. Baseline Study Report All final versions of international food assistance evaluation reports will be made publicly available. Evaluatorsshall provide a copy of the evaluation reports that is free of personally identifiable information (PII) and proprietary information. Final versions of evaluation reports ready for publication should be accessible to persons with disabilities. For guidance on creating documents accessible to persons with disabilities, please see the following resources: https://www.section508.gov/create/documents https://www.section508.gov/create/pdfs 18 The report should be organized according to the outline below. A PowerPoint presentation should follow the outline in Table 7 below. Table 7: Baseline Report Outline Report Length Maximum of 50-60 pages, excluding the Table of Contents, Acronym List, and Annexes and should be written in Eng Illuslitsrhat;iTviemRep es Newort ORuomtlinean font size 12. Acknowledgement Table of Contents Table of Exhibits Acronym List Executive Summary (in English and French) Chapter 1. Evaluation Purpose and Research Questions Chapter 2. Project Background Chapter 3. Evaluation Method 3.1 Methodologies 3.2 Sampling Framework 3.3 Data Sources and Data Collection Methods 3.4 Field Work 3.5 Analysis Plan 3.6 Strengths and Limitations Chapter 4. Findings 4.1 Baseline performance indicator table. Chapter 5. Conclusion and Recommendations 5.1 Summary of Key Findings 5.2 Lessons Learned 5.3 Recommendations References Annexes Conflict of interest statement(s)signed by the evaluator(s) disclosing any real or perceived conflicts of interest. Executive Summary Include an Executive Summary that provides a brief overview of the evaluation purpose, project background, evaluation questions, methods, findings, and conclusions. Questions Address all evaluation questionsin the TOR. Methods • Explain evaluation methodology in detail. • Disclose evaluation limitations(e.g., selection bias, recall bias, etc.). NOTE: A summary of methodology can be included in the body of the report, with the full description provided as an annex. Findings 19 Milestone Payment Inception report and inception report presentation, including survey instruments 20 % of the total Completion of Data Collection and Field Work; Debriefing 30 % of the total Draft Evaluation Report Submission 25% of the total Final report submitted and approved; all data handed over to Global Communities 25% of the total • • • Tables with baseline results, disaggregated by gender, department, and grade. Brief description of each table, including any context or explanation needed to help the reader in interpreting and understanding. Detailed description of findingsfor methodologicalresearch questions and key indicators as described in the TOR. Recommendations • • Support recommendations with specific findings. Provide recommendationsthat are action-oriented, practical, and specific. Annexes Include the following annexes, at a minimum: • TOR • All evaluation tools (questionnaires, checklists, discussion guides, surveys, etc.). • All evaluation tools and data collection tools (questionnaires, checklists, discussion guides, surveys, etc.). • Only if applicable, include as an annex Statement(s) of Differencesregarding any significant unresolved differences of opinion on the part of donors, implementers, and/or members of the evaluation. • Clean Data Sets • Final Presentation Materials C. Cost Proposal Offerors must submit a detailed, itemized budget in the Annex A Budget template provided. D. Method of Payment All deliverables must be approved by Global Communities. Payment will be based on the following milestones: Table 8: Payment Schedule 20 Annex 3: Empirical Approach to Impact Identification This section describes the overall framework and approach for impact evaluation providing context for the role that the baseline evaluation plays in the overall project evaluation strategy. To conduct causal inferences, the quantitative component of the evaluation from baseline to endline will employ a quasi￾experimental approach. As it is not feasible to randomize schools and students into treatment and non￾treatment groups in this type of project, it will not be possible to draw causal conclusions by observing simple before-and-after changes in outcomes (given that factors other than the treatment may influence the outcome over time). Further, evaluators cannot simply compare treatment and control groups due to selection bias and differences in unobservable characteristics between the groups coming from different districts. DiD combines these two methods to compare the before-and-after changes in outcomes for treatment and control groups and estimate the overall impact of the program. DiD takes the difference between the treatment group’s before and after outcomes. This is the first difference. In comparing the same group to itself, the first difference controls for factors that are constant over time in that group. Then, to capture time-varying factors, DiD takes the before-after difference in the control group, which was exposed to the same set of environmental conditions as the treatment group. This is the second difference. DiD analysis “cleans” all time-varying factors from the first difference by subtracting the second difference from it. This leaves us with the impact estimation – or the DiD. Put simply, the DiD compares the relative change of the treatment group with the change of the control group. Figure 1: DiD Impact Identification Source: (Population Health Methods, 2019) The validity of the DiD approach relies on the equal trend assumptions. Figure 2 represents an assumption across two groups with parallel lines. Treatment and control groups record different starting values of a given variable but should move in parallel over time until a project intervention is introduced at baseline. As portrayed in this example, members of both groups exhibit gradually higher values for the variable in question over time, even in the absence of an intervention. As shown in Figure 2, an accelerating result is observed after introducing an intervention to the treatment group. From that point forward, the treatment group (shown in green) follows a different trajectory for the outcome of interest. In the DiD approach, it is only the deviation from the parallel trend, rather than the full magnitude of change from baseline to midline Legend Treatment Group Control Group Time of intervention 21 or endline, that is attributed to project influence. That is, project impact is measured as the difference between the change, or difference, we would have expected given parallel trends and the change we observe. 22 Annex 4: Evaluation Framework Matrix Table 1: Evaluation matrix Aspect Evaluation question Methodologies / tools / respondents Relevance and coherence 1) Is the program relevant to the desired outcomes and objectives of the McGovern-Dole strategy, policy, and plan? • What are the key policies and priorities of the Government of Madagascar around [sustainable school feeding, strengthening food production capacity, elementary school education and literacy, other?] • What are the key gaps in knowledge and capacity of Government of Madagascar in these areas? • To what extent are the planned interventions in line with the policies and priorities of the Government of Madagascar? KII with USDA, Project staff and Ministry of Education Relevance and coherence 2) Isthe project’stheory of change realistic, appropriate, and practical to implement? Are there any modifications needed to the project’s design, structure, logic or management to facilitate the achievement of the desired outcomes and impact? KII with USDA, Project staff and Partners (CRS, FHI 360) FG with project beneficiaries Relevance and coherence 3) Are the project’s proposed strategies and interventions to increase knowledge and strengthen capacity of key stakeholders (i.e., GoM, schools, savings groups) appropriate and realistic to achieve the expected results and outcomes? KII with USDA, Project staff, Partners, Ministries FG with school staff FG with parents (including 23 saving group members) Relevance and coherence 4) Is the program relevant to address the real, and perceived needs of project beneficiaries (e.g., students, teachers, school administrators, parents/guardians, CWD, government officials)? How well does the program complement, leverage, and align with other ongoing education, nutrition, and school feeding programs within the country and other GoM plans and priorities? • How do key stakeholders (students, teachers, school administrators, parents/guardians, CWD, government officials) describe priority needs in key areas (e.g., health & nutrition, education/literacy, school feeding, poverty, agriculture and women’s economic and social empowerment etc poverty levels, other development indicators)? How do these vary for women versus men or other sub-groups? • What are the major challenges in achieving progress in these areas? • How do these need and challenges directly and indirectly influence the literacy outcomes in the project area? • To what extent the program objectives and outcomes address the community’s needs? KII with Project staff and partners, DREN (regional staff of the ministry of education), Chef CISCO (district education officer) FG with teachers, parents/guardi ans and school staff (teachers and school administrators ) Parent survey Student survey Relevance and coherence 5) Is the project design suited to improving the level of early grade literacy performance and overcoming the main challenges to improving literacy outcomes? KII with Project staff and partners, DREN, Chef CISCO FG with school staff EGRA Assessment 24 Classroom observation Relevance and coherence 6) Is the program designed to be flexible, for example, can the overall strategy, components, or specific activities be adjusted over time due to changing environment or in response to target population needs? KII with USDA, Project staff and partners Effectiveness 1) To what extent is the COVID-19 pandemic likely to influence program effectiveness/results and how best can the program address these influences? • What are the influences that COVID (and other relevant external factors) may have on program implementation? How can these impacts be mitigated? KII with USDA, Project staff, Partners and Ministry of Education Effectiveness 2) What are the main challenges or barriers to achieving desired outcomes across other related project components (i.e., health, nutrition, and dietary practices, strengthened capacity to manage school feeding activities, improved student attendance and attentiveness)? KII with USDA, Project staff, Partners and Ministries FG with school staff, FG with parents Student survey, parent survey Classroom observatio n Effectiveness 3) Is there a clear understanding of roles and responsibilities by all parties involved for implementation and monitoring? • What are perceived roles and responsibilities by all parties involved for implementation and monitoring? To what extent are these in line with program plans and how can they be improved/made clearer? KII with USDA, Project staff, Partners and Ministries 25 Effectiveness 4) Are the project’s monitoring and evaluation strategies realistic, appropriate, and practical to implement? • To what extent are the results framework and program approach valid for the local context? Is the theory of change realistic, appropriate and practical to implement? • What strategies or planned activities seemed effective for meeting the expected results and targets? Which were not? Why? • Do the design, structure, logic, or management need modifications to facilitate the achievement of desired outcomes and impact? • Are the program monitoring and evaluation strategies, including indicators, and targets suitable (i.e., SMART) to monitor and support the implementation of the targeted results? If not, what are the suggested changes? KII with USDA, Project staff, Partners and Ministries Efficiency 1) How efficient and realistic is the planned allocation of resources(human resources, time, expertise, funds etc.) to provide the necessary support and to achieve the broader program objectives? • How efficient is the project planning to engage key actors to finance program activities (e.g., contributions from governments & private entities to support school feeding, and education supplies? KII with USDA, Project staff and Partners Sustainability 1) Is the project’s sustainability plan realistic and appropriate? • What are the main barriers to achieving the sustainability plan of the program? KII with USDA, Project staff, Partners and Ministries FG with school staff and parents 26 Sustainability 2) What undeveloped opportunities might be at hand that can contribute to the sustainability of project benefits and outcomes? • Who needs to be engaged (private sector, community entities/associations, local government agencies, etc.) to ensure sustainability? KII with Project staff and partners, ministries, DREN, Chef CISCO FG with school staff and parents Sustainability 3) Doesthe project have the meansto addressthreats or gaps (i.e., climate-related shocks, COVID-19) that could impact progress toward sustainability and sustained outcomes/benefits of the project? • What do local organizations and public sector entities need to strengthen to be able to continue implementing activities beyond the end of the program? KII with Project staff and partners, ministries, DREN, Chef CISCO FG with school staff Sustainability 4) To what extent have stakeholders and local actors (e.g., GoM at all levels, private sector, community groups, other organizations implementing school feeding programs) been engaged and committed to ensuring progress toward a sustainable NSFP? • What specific institutional capacity needs (e.g., for key GoM, district, community and school level stakeholders) should the project address to foster greater host country engagement and accountability? KII with Project staff and partners, ministries, DREN, Chef CISCO FG with school staff and parents Sustainability 5) To what extent do stakeholders (i.e., GoM, communities, private sector) to provide monetary or in-kind support for school feeding and complementary activities, and are these sustainable inputs? KII with USDA, Project staff, Partners and Ministries FG with school staff and parents Parent survey 27 Sustainability 6) What commitment has been made in support of a gradual transition toward a locally owned and managed school feeding program by the GoM, schools, and communities? KII with USDA, Project staff, Partners and Ministries FG with school staff and parents Sustainability 7) Are all the elements of sustainability (capacity, motivation, resources, linkages) addressed in the project’s sustainability plan? KII with USDA, Project staff, Partners and Ministries Sustainability 8) What is specific capacity needs for project stakeholders (e.g., GoM, parents, school administrators, school management committees) that the project should address to foster greater ownership and accountability to sustain project benefits? Interviews with USDA, Project staff, Partners and Ministries FG with school staff and Parents 28 Annex 5: Data collection tools This section presents the data collection tools used throughout the Global Communities Mianatra project baseline evaluation. They are: • EGRA • Student questionnaire • Classroom observation tool • Parent questionnaire • Director survey and checklist • Qualitative data collection tools 29 EGRA INSTRUMENT Mada EGRA Baseline Il est important d’établir une relation détendue et enjouée avec les élèves qui vont être évalués, grâce à de simples conversations initiales (voir exemple ci-après). L’élève doit presque percevoir l’évaluation qui suit comme étant un jeu. Identifiez la langue que l’élève préfère utiliser pour communiquer. Lisez à haute voix clairement et lentement les sections encadrées UNIQUEMENT. Je vais d’abord t’expliquer pourquoi je suis ici en ce moment. En fait je travaille avec ton instituteur/institutrice, et je souhaiterais connaître comment vous apprenez à lire. Ton école a été choisi pour faire cette enquête et tu es parmi les élèves sélectionnés pour participer à la recherche si tu le veux bien. Je voudrais d’abord t’expliquer ce que nous allons faire, après tu me dira si tu veux participer ou non. En fait, nous allons faire de la lecture mais ce sera comme si nous jouons. Après cela, je vais te demander quelques questions. C’est cette tablette que nous utiliserons pour cela. Tu pourras refaire cette lecture à la fin de cette année scolaire si tu le souhaite. Mais bien sûr, tu peux toujours changer d’avis. Est-ce que tu as des questions ? Je voudrais te préciser que ceci n’est pas dutout un examen et tes réponses ne seront en aucun cas partagées ni à ton instituteur/institutrice ni à tes parents. Je vais prendre ton nom mais c’est seulement pour faciliter ta participation à la fin de l’année scolaire, personne ne saura que c’est toi qui a répondu ceci ou cela. Ta participation est vraiment importante mais tu peux toujours désister si tu ne souhaites pas participer. Tu peux ne pas répondre à certaines questions, tu peux aussi arrêter quand tu ne voudrais plus continuer As-tu des questions ? Est-ce que tu acceptes de participer ? Est-ce que nous pouvons commencer ? ☐ Si le consentement verbal n’est pas obtenu, remercier l’élève et sélectionnez “non ”sur l’écran. Sélectionnez ensuite “enregistrer ”et “démarrer un nouveau test.” DREN CISCO ZAP ECOLE ECOLE ID 30 1. Lettres - Famantarana litera Montrez à l’élève la feuille de lettres et groupes de lettres (graphèmes) dans le Cahier de Stimulus pendant que vous lui lisez les instructions suivantes : Faites démarrer le chronomètre quand l’élève commence à lire. Suivez la lecture de l’élève sur votre écran et relevez les réponses incorrectes en touchant du doigt les graphèmes incorrects sur l’écran. Les graphèmes incorrects apparaitront alors en bleu. Si l’élève s’autocorrige (donne une réponse incorrecte puis la corrige), touchez à nouveau l’item, ce qui fera apparaitre le graphème en gris. Ne dites rien sauf si l’élève ne répond pas et reste bloqué sur un mot pendant plus de 3 secondes. Dans ce cas, dites￾lui, “Continue”, en lui montrant le prochain mot. Notez le mot sur lequel l’élève était bloqué comme incorrect. Lorsque que la minute s’est écoulée et que le chronomètre s’arrête, l’écran s’affichera en rouge par flashs successifs. Cela signalera la fin du test. Demandez à l’élève de s’arrêter, indiquez le dernier mot lu en touchant l’écran afin qu’un crochet rouge apparaisse. Appuyez ensuite sur la touche “suivant”. Exemples: a s tr 1 2 3 4 5 6 7 8 9 10 p v E dr o d R f ts ntr (10) i A F nts t mp K a tr M (20) 60 seconds Voici un papier qui contient des lettres et des digrammes. Dis moi comment on prononce cette lettre. Par exemple, pour celle-ci [montre lui/la la lettre “a”] on la lis ainsi: “a” comme dans le mot “dada”. Allons faire un essai. Lis cette lettre.[Montre lui/la la lettre “s”] [Si sa réponse est vrai, dites ceci]: C’est Très bien! on lis cette lettre ainsi: “s” comme dans le mot ‘’sa’’. [Si sa réponse est fausse,alors dites ceci]: en fait on lis cette lettre ainsi: “s” comme dans le mot ‘’sa’’. Nous allons voir un autre exemple. Lis ce digramme. [Montre lui/la le ‘’tr’’] [Si sa réponse est vrai, dites ceci]: Très bien!, on lis ce digramme ainsi “tr” comme dans le mot “trano” [Si sa réponse est fausse,alors dites ceci]: en fait on lis ce digramme ainsi “tr” comme dans le mot “trano”... Quand je dis “Commence”, alors tu peux commencer à lire à partir d’ici [montrer la première lettre à l’aide de votre index], regarde bien les lignes et lis de gauche à droite. Lis bien, vite et fort s’il te plaît. Pointes avec ton index chaque lettre que tu lis. Lis bien et vite. Tu peux sauter s’il y a des lettres que tu ne sais pas lire et tu peux de suite continuer ta lecture. Pointe la première lettre avec ton index. Est-ce que tu es prêt(e)? Commence alors 31 m y ndr ng j e mb g nj I (30) nk L nd H z b Y R k T (40) TR nt z S h M n E O N (50) h O A e m O t r i v (60) nd R V S a i K j n mb (70) i O s I tr b y o nt L (80) p f A Y k N g a H n (90) ts ng N a mp o nj I d N (100) Merci bien! On peut passer à la prochaine activité! 2. Syllabes - Famantarana vaninteny Montrez à l’élève la feuille de syllabes dans le Cahier de Stimulus pendant que vous lui lisez les instructions suivantes : Faites démarrer le chronomètre quand l’élève commence à lire. Voici un papier qui contient des syllabes. Dis moi comment on prononce ces syllabes. Par exemple, cette syllabe [montre lui/la le “hi”] on la lis ainsi “hi” Allez, on va faire des essais. Lis cette syllabe [montre lui/la “la”]. [Si sa réponse est vrai, dites: Très bien!]: On la lis ainsi “la”. [Si sa réponse est fausse, dites:]: En fait on la lis comme ceci “la”. Nous allons voir un autre exemple. Lis cette syllabe [montre lui/la le “se”]. [Si sa réponse est vrai, dites: Très bien!]: Très bien! on la lis ainsi “la”. [Si sa réponse est fausse, dites]: En fait on la lis comme ceci “la”. Quand je dis “Commence”, alors tu peux commencer à lire à partir d’ici [montrer la première lettre à l’aide de votre index], regarde bien les lignes et lis de gauche à droite. Lis bien, vite et fort s’il te plaît. Pointes avec ton index chaque lettre que tu lis. Lis bien et vite. Tu peux sauter s’il y a des lettres que tu ne sais pas lire et tu peux de suite continuer ta lecture. Pointe la première lettre avec ton index. Est-ce que tu es prêt(e)? Commence alors 32 Suivez la lecture de l’élève sur votre écran et relevez les réponses incorrectes en touchant du doigt les graphèmes incorrects sur l’écran. Les graphèmes incorrects apparaitront alors en bleu. Si l’élève s’autocorrige (donne une réponse incorrecte puis la corrige), touchez à nouveau l’item, ce qui fera apparaitre le graphème en gris Ne dites rien sauf si l’élève ne répond pas et reste bloqué sur un mot pendant plus de 3 secondes. Dans ce cas, dites-lui, “Continue”, en lui montrant le prochain mot. Notez le mot sur lequel l’élève était bloqué comme incorrect. Lorsque que la minute s’est écoulée et que le chronomètre s’arrête, l’écran s’affichera en rouge par flashs successifs. Cela signalera la fin du test. Demandez à l’élève de s’arrêter, indiquez le dernier mot lu en touchant l’écran afin qu’un crochet rouge apparaisse. Appuyez ensuite sur la touche “suivant”. Si l’élève atteint la fin du test avant que l’écran ne s’affiche en rouge, arrêtez le chronomètre vous même au moment où l’élève lit le dernier mot du test. Puis touchez le dernier mot sur l’écran afin qu’un crochet rouge apparaisse. Appuyez ensuite sur la touche “suivant”. Règle d’auto-stop: Si l’élève ne donne aucune réponse correcte parmi les 5 premieres syllabes (la première ligne), l’écran s’affichera en rouge par flashs successifs et le chronomètre s’arrêtera automatiquement. Demandez gentiment à l’éleve de s’arrêter, interrompez l’épreuve et passez à la suivante. Exemples: hi la se 1 2 3 4 5 do mi dy tro mpo (5) ki da zi li vo (10) ndra ky n'ny hy tsa (15) mbo ri tsa po za (20) si nta so ndro vi (25) mpo tra mpi ni nga (30) bo ti zy tso ndri (35) ne ngo nka ke di (40) le me nja mpa nda (45) tsi ha ntsi mo ma (50) Merci bien! On peut passer à la prochaine activité! 33 3. Mots familiers - Famakiana teny mahazatra Montrez à l’élève la feuille de mots dans le Cahier de Stimulus pendant que vous lui lisez les instructions. Faites démarrer le chronomètre quand l’élève commence à lire le premier mot. Suivez la lecture de l’élève sur votre écran et relevez les réponses incorrectes en touchant du doigt les mots incorrects sur l’écran. Les mots incorrects apparaitront alors en bleu. Si l’élève s’autocorrige (donne une réponse incorrecte puis la corrige), touchez à nouveau l’item, ce qui fera apparaitre le mot en gris. Ne dites rien sauf si l’élève ne répond pas et reste bloqué sur un mot pendant plus de 3 secondes. Dans ce cas, dites-lui, “Continue”, en lui montrant le prochain mot. Notez le mot sur lequel l’élève était bloqué comme incorrect. Lorsque que la minute s’est écoulée et que le chronomètre s’arrête, l’écran s’affichera en rouge par flashs successifs. Cela signalera la fin du test. Demandez à l’élève de s’arrêter, indiquez le dernier mot lu en touchant l’écran afin qu’un crochet rouge apparaisse. Appuyez ensuite sur la touche “suivant”. Si l’élève atteint la fin du test avant que l’écran ne s’affiche en rouge, arrêtez le chronomètre vous même au moment où l’élève lit le dernier mot du test. Puis touchez le dernier mot sur l’écran afin qu’un crochet rouge apparaisse. Appuyez ensuite sur la touche “suivant”. Règle d’auto-stop: Si l’élève ne donne aucune réponse correcte parmi les 5 premiers mots (la première ligne), l’écran s’affichera en rouge par flashs successifs et le chronomètre s’arrêtera automatiquement. Demandez gentiment à l’éleve de s’arrêter, interrompez l’épreuve et passez à la suivante. Exemple: nono faly ireo 1 2 3 4 5 bota zaza mena mody loha (5) indray sary mazoto mbola tonga (10) Voici un papier qui contient plusieurs mots. Essais de lire ces mots comme tu peux. Ne lis plus les lettres, cette fois ci ce sont vraiment les mots qu’il faut lire. Par exemple, on lis “nono” et non ‘’n’’, ‘’o’’, ‘’n’’, ‘’o’’ Allons faire un essai. Lis ce mot [montre lui/la le “faly”] [Si la réponse est vrai, dites]: Très bien, on lis ce mot ainsi “faly” [Si la réponse est fausse, dites]: En fait on lis ce mot comme ceci “faly”. Nous allons voir un autre exemple. Lis ce mot [montre lui/la le “ireo”] [Si la réponse est vrai, dites]: Très bien, on lis ce mot ainsi “ireo”. [Si la réponse est fausse, dites]: En fait on lis ce mot comme ceci “ireo”. Est-ce que c’est bon, tu as bien compris? On peut continuer? Dès que je dis “commence”, alors pointe avec ton index les mots que tu vas lire. regarde bien les lignes et lis de gauche à droite. Cette fois-ci je ne vais plus dire quoique ce soit. C’est toi seul qui va lire le mot l’un après l’autre. Tu as bien compris? Mets ton index sur le premier mot. Es-tu prêt(e)? Lis bien et fort. Commence alors. 34 tsara izany may andro jiro (15) tamin’ny omby ireo harona tanana (20) ihany mihisty lakana fomba reraka (25) lamba zavatra saka rehetra lela (30) intsony kisoa trano ohatra mofo (35) marary mamy maso valo biby (40) sofina mafy mavo hoho kapa (45) tady izy orana seza doda (50) Merci bien! On peut passer à la prochaine activité! 4. Phrases absurdes Lisez les instructions à l’enfant, et passez en revue les exemples. Si l’enfant ne répond pas après 5 secondes, demandez-lui s’il/elle veut que vous répétiez la phrase. Si l’enfant ne répond toujours pas après 5 secondes, marquer “pas de réponse” et continuez avec la phrase suivante. Règle d’auto-stop : Si l’élève ne donne aucune réponse correcte parmi les 10 premières phrases, demandez gentiment à l’éleve de s’arrêter, interrompez l’épreuve et passez à la suivante. Correcte Incorrecte Pas de répo nse 1 Manidina eny amin’ny habakabaka ny vorona. Les oiseaux volent dans le ciel. (22) Vrai 2 Manana tongotra roa i Soa. Soa a deux pieds. (21) Vrai 3 Manoratra eny amin’ny solaitrabe ilay mpampianatra. Vrai C’est un exercice d’écoute. Je vais te lire les phrases suivantes. Ecoutes bien et pense bien à leurs senses. Les phrases peuvent être vraies ou fausses. Dis moi si la phrase est “vrai” ou “fausse”. On va prendre un exemple… “« Le soleil sait parler français. » [Si l’enfant répond correctement, dites] : Très bien [Si l’enfant ne répond pas correctement, dites] : Ecoutes une deuxième fois : « Le soleil sait parler français. » ce ne n’est pas vrai On va voir un autre exemple. « Les petits garçons jouent au ballon. » [Si l’enfant répond correctement, dites] : Très bien [Si l’enfant ne répond pas correctement, dites] : Ecoutes une deuxième fois : « Les petits garçons jouent au ballon. ». C’est vrai. On va maintenant essayer un autre exemple. « Tu peux voler comme un oiseau. » Tu es prêt ? On va donc commencer. 35 La maitresse écrit au tableau. (20) 4 Miteny malagasy ilay mpampianatra. Le maître parle en malagasy. (28) Vrai 5 Monina ao an-trano ny fosa. Le fosa vit à la maison. (24) Faux 6 Mandeha eny amin’ny arabe ny fiarakodia. La voiture roule sur la route. (11) Vrai 7 Mihinana saka ny totozy. La souris mange le chat. (10) Faux 8 Mafana ny afo. Le feu est chaud. (25) Vrai 9 Manatody ny akohovavy. La poule pond des œufs. (17) Vrai 10 Mihinana totozy ny saka. Le chat mange la souris. (7) Vrai 11 Mandeha eny ambony ranomasina ny fiarakodia. La voiture roule sur la mer. (6) Faux 12 Mijery amin'ny vavany i Jao. Jao voit avec sa bouche. (30) Faux 13 Mamily fiarakodia ny olona. Les gens conduisent les voitures. (23) Vrai 14 Mihinam-bary i Njara. Njara mange du riz. (18) Vrai 15 Maitso ny lanitra. Le ciel est vert. (27) Faux 16 Miposaka amin'ny maraina ny masoandro. Le soleil se lève le matin. (29) Vrai 17 Manatody vato ny akohovavy. La poule pond des cailloux. (9) Faux 18 Mihinana dite i Njara. Njara mange du thé. (14) Faux 19 Biby ny gidro. Le singe est un animal. (26) Vrai 20 Manoratra eny amin'ny solaitrabe ny ondry. Le mouton écrit au tableau. (3) Faux 21 Manana tongotra efatra i Soa. Soa a quatre pieds. (2) faux 22 Manidina eny amin'ny habakabaka ny sambo. Faux 36 Les bateaux volent dans le ciel. (1) 23 Mamily fiarakodia ny omby. Les zébus conduisent les voitures. (13) Faux 24 Monina ao an-trano ny saka. Le chat vit à la maison. (5) Vrai 25 Mangatsiaka ny afo. Le feu est froid. (8) faux 26 Biby ny seza. La chaise est un animal. (19) faux 27 Manga ny lanitra. Le ciel est bleu. (15) Vrai 28 Miteny malagasy ny akohovavy. Les poules parlent en malagasy. (4) Faux 29 Miposaka amin'ny hariva ny masoandro. Le soleil se lève le soir. (16) Faux 30 Mijery amin'ny maso i Jao. Jao voit avec les yeux. (12) Vrai Misaotra anao! hanao fampiasana hafa indray isika! 5. Histoire 1 min Montrez à l’élève la petite histoire dans le Cahier de Stimulus. Lisez à l’élève les instructions. Faites démarrer le chronomètre quand l’élève lit le premier mot. Suivez la lecture de l’élève sur votre écran et relevez les réponses incorrectes en touchant du doigt les mots incorrects sur l’écran. Les mots incorrects apparaitront alors en bleu. Si l’élève s’autocorrige (donne une réponse incorrecte puis la corrige), touchez à nouveau l’item, ce qui fera apparaitre le mot en gris. Ne dites rien sauf si l’élève ne répond pas et reste bloqué sur un mot pendant plus de 3 secondes. Dans ce cas, dites-lui, “Continue”, en lui montrant le prochain mot. Notez le mot sur lequel l’élève était bloqué comme incorrect. Lorsque que la minute s’est écoulée et que le chronomètre s’arrête, l’écran s’affichera en rouge par flashs successifs. Cela signalera la fin du test. Demandez à l’élève de s’arrêter, indiquez le dernier mot lu en touchant l’écran afin qu’un crochet rouge apparaisse. Appuyez ensuite sur la touche “suivant”. Si l’élève atteint la fin du test avant que l’écran ne s’affiche en rouge, arrêtez le chronomètre vous même au moment où l’élève lit le dernier mot du test. Puis touchez le dernier mot sur l’écran afin qu’un crochet rouge apparaisse. Appuyez ensuite sur la touche “suivant”. Quand l’élève a fini de lire, RETIREZ le passage de sa vue Règle d’auto-stop: Si l’élève ne réussit pas à lire correctement les premiers 7 mots, l’écran s’affichera en rouge par flashs successifs et le chronomètre s’arrêtera automatiquement. Demandez gentiment à l’éleve de s’arrêter, interrompez l’épreuve et passez à la suivante. Ecoute, voici une petite histoire. Lis la bien, fort et vite comme tu peux. Quand je dirais “Commence la lecture”, alors tu commences. S’il y a des mots que tu ne comprends pas, alors saute et continue ta lecture. Mets ton index sur le premier mot. Est-ce que tu es prêt(e)? Commence alors 37 Passage Tia fahadiovana i Mendrika. Mazoto mianatra izy. 7 Folo taona izy izao. Mirana ery izy mifoha maraina. Manomana ny fitaovana ho entiny mianatra izy. 23 Nandeha misasa sy miborosy nify izy. Lany anefa ny savony. 33 Nangataka tamin’ny reniny izy. Nanatitra savony vaovao ho azy ny reniny. 45 Misasa madio tsara izy. Faly be izy mianaka. Lasa nianatra i Mendrika nony avy eo. 60 Misaotra anao! hanao fampiasana hafa indray isika! 6. Histoire 2 min et comprehension de l’ histoire Montrez à l’élève la petite histoire dans le Cahier de Stimulus. Lisez à l’élève les instructions. Faites démarrer le chronomètre quand l’élève lit le premier mot. Suivez la lecture de l’élève sur votre écran et relevez les réponses incorrectes en touchant du doigt les mots incorrects sur l’écran. Les mots incorrects apparaitront alors en bleu. Si l’élève s’autocorrige (donne une réponse incorrecte puis la corrige), touchez à nouveau l’item, ce qui fera apparaitre le mot en gris. Ne dites rien sauf si l’élève ne répond pas et reste bloqué sur un mot pendant plus de 3 secondes. Dans ce cas, dites-lui, “Continue”, en lui montrant le prochain mot. Notez le mot sur lequel l’élève était bloqué comme incorrect. Lorsque que la minute s’est écoulée et que le chronomètre s’arrête, l’écran s’affichera en rouge par flashs successifs. Cela signalera la fin du test. Demandez à l’élève de s’arrêter, indiquez le dernier mot lu en touchant l’écran afin qu’un crochet rouge apparaisse. Appuyez ensuite sur la touche “suivant”. Si l’élève atteint la fin du test avant que l’écran ne s’affiche en rouge, arrêtez le chronomètre vous même au moment où l’élève lit le dernier mot du test. Puis touchez le dernier mot sur l’écran afin qu’un crochet rouge apparaisse. Appuyez ensuite sur la touche “suivant”. Quand l’élève a fini de lire, RETIREZ le passage de sa vue Règle d’auto-stop: Si l’élève ne réussit pas à lire correctement les mots nécessaires à la passation de la première question de compréhension, l’écran s’affichera en rouge par flashs successifs et le chronomètre s’arrêtera automatiquement. Demandez gentiment à l’éleve de s’arrêter, interrompez l’épreuve et passez à la suivante. Ecoute, voici une autre petite histoire. Lis la bien, fort et vite comme tu le peux. Je vais te poser quelques questions après. Quand je dirais “Commence la lecture”, alors tu commences. S’il y a des mots que tu ne comprends pas, alors saute et continue ta lecture. Mets ton index sur le premier mot. Est-ce que tu es prêt(e)? Commence alors 38 Posez toutes les questions qui sont présentées à l’écran. Elles correspondent automatiquement à la portion du texte que l’élève a lu. Sélectionnez les réponses de l’élève comme “Correcte” ou “Incorrecte”. La réponse sélectionnée apparaitra en jaune. Si l’élève ne répond pas dans les 10 secondes, sélectionnez “Pas de réponse” et passez à la question suivante. Ne répétez pas les questions. Seules les réponses identiques ou similaires aux réponses fournies dans ce protocole (listées à côté de chaque question) peuvent être considérées comme correctes. Si l’élève dit “je ne sais pas”, cochez la réponse “incorrecte”. Passage Ecoute, je vais te poser quelques questions, réponds comme tu l’entends. Tu peux consulter l’histoire pour y voir les réponses. Questions de compréhension Correcte Incorrecte Pas de réponse Zaza mazoto i Soa. Marary azy mafy ny lohany. 9 Inona no marary an’i Soa? [Loha] Mangovitra ny vatany. Miakatra be ny hafanany. 16 Manao ahoana ny hafanany? [miakatra] Tsy afaka nianatra izy izao. Nanatona mpitsabo izy sy ny reniny. 27 Nanatona an’iza i Soa sy ny reniny? [mpitsabo/dokotera] Nahazo tsindrona sy pilina izy. Efa nihatsara izao ny fahasalamany. Naterin’ ny reniny nianatra izy. 42 Iza no nanatitra an’I Soa nianatra? [reniny] Niezaka mafy i Soa. Nanampy azy ny mpampianatra. Nahazo naoty tsara Izy. Faly noho izany ny reniny. 59 Inona no nahafaly ny reniny? [naoty tsara] Misaotra anao! hanao fampiasana hafa indray isika! 7. Comprehension Orale L’épreuve n’est pas chronométrée et aucun mot écrit n’est montré à l’élève. Retirez le Cahier de Stimulus de la vue de l’élève. Lisez les instructions à l’élève. Vous allez lire le passage à haute voix UNE SEULE FOIS, lentement (à peu près un mot par seconde). Posez toutes les questions à l’élève. Ne laissez pas l’élève voir l’histoire ou les questions. Cette fois-ci, c’est moi qui va te lire une histoire. Je ne vais lire l’histoire qu’une seule fois alors sois très attentif/ve. Après, réponds aux questions selon ta compréhension. 39 Sélectionnez les réponses de l’élève comme “Correcte” ou “Incorrecte”. La réponse sélectionnée apparaitra en jaune. Si l’élève ne répond pas dans les 10 secondes, sélectionnez “Pas de réponse” et passez à la question suivante. Ne posez chaque question qu’une seule fois. Seules les réponses identiques ou similaires aux réponses fournies dans le protocole (listées à côté de chaque question) peuvent être considérées comme correctes. Si l’élève dit “je ne sais pas”, notez la réponse comme incorrecte. Correcte Incorrecte Pas de réponse Mianatra ao amin’ny taona faha firy i Jao? [Taona faharoa] Iza no miaraka amin’i Jao rehefa hianatra? [Namany] Inona no nozarain’izy ireo tamin’ilay mpangataka? [mofo] Iza no nisaotra? [mpangataka] Naninona no nankasitraka an’I Jao sy ny namany ilay mpampianatra? [asa soa/nizara] Merci ! On a terminé. Tu peux rentrer dans la salle de classe ! Mianatra ao amin’ny taona faharoa i Jao. Zaza mazoto sy tia namana izy. Lavitra ny trano fonenany ny sekoly. Miaraka amin’ny namany izy rehefa hianatra. Nifanena tamin’ny mpangataka izy ireo. Nitaraina noana ilay mpangataka. Nalahelo azy izy ireo. Nozarain’izy ireo taminy ny mofony. Nohaniny ilay mofo. Nisaotra ilay mpangataka. Samy lasa ry zareo. Nitantara izany tamin’ny mpampianany i Jao. Nankasitraka azy ireo ilay mpampianatra. 40 Classroom Observation Tool Cette section doit être remplie par l’enquêteur Date : (JJ-MM-AAAA) Code enquêteur: …………….. Nom de l’enquêteur : …………….. District : Beloha Tsihombe Bekily 1 2 3 Commune : Nom de l’école : Code répondant : Nº Question Réponse a- Matières b- Attention 01 Observation 1 1 2 3 4 5 6 7 8 9 10 11 12 13 □ Attentif □ Non Attentif □ Non observé 02 Observation 2 □ Attentif □ Non Attentif □ Non observé 1 2 3 4 5 6 7 8 9 10 11 12 13 03 Observation 3 1 2 3 4 5 6 7 8 9 10 11 12 13 □ Attentif □ Non Attentif □ Non observé 04 Observation 4 1 2 3 4 5 6 7 8 9 10 11 12 13 □ Attentif □ Non Attentif □ Non observé 05 Observation 5 □ Attentif □ Non Attentif □ Non observé 1 2 3 4 5 6 7 8 9 10 11 12 13 06 Observation 6 1 2 3 4 5 6 7 8 9 10 11 12 13 □ Attentif □ Non Attentif □ Non observé 41 07 Observation 7 1 2 3 4 5 6 7 8 9 10 11 12 13 □ Attentif □ Non Attentif □ Non observé 08 Observation 8 1 2 3 4 5 6 7 8 9 10 11 12 13 □ Attentif □ Non Attentif □ Non observé 09 Observation 9 1 2 3 4 5 6 7 8 9 10 11 12 13 □ Attentif □ Non Attentif □ Non observé 10 Observation 10 1 2 3 4 5 6 7 8 9 10 11 12 13 □ Attentif □ Non Attentif □ Non observé 42 Student Questionnaire Cette section doit être remplie par l’enquêteur Date : (JJ-MM-AAAA) Code enquêteur: …………….. Nom de l’enquêteur : …………….. District : Beloha Tsihombe Bekily 1 2 3 Commune : Nom de l’école : Code répondant : Bonjour, je m'appelle………………, un enquêteur d’ATW, un cabinet d’études indépendant. Nous faisons actuellement une enquête pour connaitre la situation de l’éducation et de la nutrition des enfants dans votre district. Je vais donc te poser quelques questionssur tes habitudes. Tes réponses serontstrictement confidentielles et ne seront pas partagées avec quiconque. Encore une fois, tu n’es pas obligé de participer si tu ne le veux pas. Si tu arrives à une question à laquelle tu préfères ne pas répondre, ce n’est pas grave, on peut passer. As-tu des questions ? [Laisser le temps pour poser des questions et répondre au besoin] Est-ce que tu es d’accord pour participer ? Peut-on commencer ? Nº Question Réponse 1. Information sur l’élève 101 Quel âge as-tu ? 102 Sexe (regardezl’enfant et cochez) o Masculin o Féminin 103 En quelle classe es-tu actuellement ? o T2 o T4 o T5 104 Est-ce que tu es en session du matin, de l’après-midi ou les deux ? o Matin o Après-midi o Les deux 105 As-tu mangé aujourd'hui avant de venir à l'école ? o Oui o Non 105 [Q104 = Non] Est-ce que cela t’arrive souvent de ne pas manger avant de venir à l’école o Souvent o Quelques fois o Jamais 43 106 As-tu mangé hier pendant la recréation ? [NB ou le dernier jour que tu es venu à l’école] o Oui o Non o Ne sait pas / Pas de réponse 2. Pratiques d’hygiène et de santé 201 A- Qu’est-ce que tu utilises pour te laver les mains à l’école ? [REPONSES MULTIPLES] (NB : Ne pas lire les réponses) • Savon • Liquide vaisselle • Cendre • Ne se lave pas les mains • Autre, préciser • Ne sait pas / Pas de réponse B- Qu’est-ce que tu utilises pour te laver les mains à la maison ? [REPONSES MULTIPLES] (NB : Ne pas lire les réponses) • Savon • Liquide vaisselle • Cendre • Ne se lave pas les mains • Autre, préciser • Ne sait pas / Pas de réponse 202 A quels moments de la journée te laves-tu les mains ? [REPONSES MULTIPLES] (NB : Ne pas lire les réponses) • Avant de préparer à manger • En préparant à manger • Après avoir préparé à manger • Avant de manger • Après avoir mangé • Après défécation • Après s'être mouché, avoir toussé ou éternué • Après avoir touché des déchets • Après le nettoyage des latrines • Après avoir travaillé dans les champs • Après le balayage • Jamais • Autre, préciser • Ne sait pas / Pas de réponse 44 203 Peux-tu me dire 3 choses qu'il faut faire pour améliorer la santé ou l’hygiène ? [REPONSES MULTIPLES] (NB : Ne pas lire les réponses) • Lavage des mains avec du savon • Utilisation des latrines pour se déféquer • Ne pas se déféquer à l’air libre • Alimentation équilibré et varié • Pratique d’alimentation du nourisson • Manger des aliments énergétiques • Other, please specify • 99 = Don’t know 3. Pratiques nutritionnelle et alimentaire Maintenant, j’aimerais te poser des questions sur tes habitudes alimentaires. 300 Est-ce que tu prends le petit déjeuner le matin avant d’aller à l’école ? CITER LES REPONSES o Toujours o Rarement o Jamais o Ne sait pas / Pas de réponse 301 [Q300 = Toujours OU rarement] Généralement, que manges-tu comme petit déjeuner ? [REPONSES MULTIPLES] • Riz sans mets • Riz avec des mets • Thé • Café • Lait • Pain • Manioc • Patate douce • Maïs • Autre, préciser • Ne sait pas / Pas de réponse 302 Est-ce que tu prends du gouter le matin à l’école ? CITER LES REPONSES o Toujours o Rarement o Jamais o Ne sais pas / Pas de réponse 303 [Q302 = toujours OU rarement] Généralement, que prends-tu comme gouter ? [REPONSES MULTIPLES] [NE PAS CITER LES REPONSES] • Thé • Café • Lait • Pain • Manioc • Patate douce 45 • Maïs • Biscuit • Autre, préciser • Ne sait pas / Pas de réponse 304 Est-ce que tu prends du déjeuner ? CITER LES REPONSES o Toujours o Rarement o Jamais o Ne sait pas / Pas de réponse 305 [Q304 = toujours OU rarement] Généralement, que manges-tu au déjeuner ? [REPONSES MULTIPLES] [NE PAS CITER LES REPONSES] • Riz sans mets • Riz avec des mets • Thé • Café • Lait • Pain • Manioc • Patate douce • Maïs • Autre, préciser • Ne sait pas / Pas de réponse 306 Est-ce que tu prends du gouter pendant l’après-midi à l’école ? CITER LES REPONSES o Toujours o Rarement o Jamais o Ne sait pas / Pas de réponse 307 [Q306 = toujours OU rarement] Généralement, que prends-du comme gouter l’après-midi ? [REPONSES MULTIPLES] [NE PAS CITER LES REPONSES] • Thé • Café • Lait • Pain • Manioc • Patate douce • Maïs • Autre à préciser • Ne sait pas / Pas de réponse 308 Est-ce que tu prends du diner le soir ? CITER LES REPONSES o Toujours o Rarement o Jamais o Ne sait pas / Pas de réponse 46 309 [Q308 = toujours OU rarement] Généralement, que prends-du comme diner le soir ? [REPONSES MULTIPLES] [NE PAS CITER LES REPONSES] • Riz sans mets • Riz avec des mets • Thé • Café • Lait • Pain • Manioc • Patate douce • Maïs • Autre, préciser • Ne sait pas / Pas de réponse 310 Peux-tu me dire des choses à faire pour avoir une bonne alimentation ? [REPONSES MULTIPLES] • Manger des aliments propres • Manger des aliments équilibrés et variés • Se laver les mains avant de manger • Manger des aliments riches en vitamine A • Manger des aliments riches en fer • Manger des fruits et légumes • Prendre ses repas à des horaires réguliers • Autre, préciser • Ne sait pas / Pas de réponse 311 Actuellement, est-ce que tu prends du repas scolaire à l’école ? o OUI o NON 312 [SI Q311 = OUI] Peux-tu me décrire ta satisfaction sur le goût du repas scolaire ? CITER LES REPONSES o Très délicieux o Délicieux o Pas vraiment délicieux o Pas du tout délicieux 4. Gestion de la menstruation (pour les filles) [102= Féminin] et [103= T4 OU T5] Je vais maintenant te poser quelques questions sur l’hygiène menstruelle 401 [SI Q102 = Féminin] As-tu déjà entendu parler des menstruations ? o Oui o Non Je vais te lire une série d'affirmations. Veuillez me dire si les énoncés sont vrais ou faux. 402 [SI Q401 = OUI] Les menstruations sont un processus biologique normal. o Vrai o Faux o Ne sait pas / Pas de réponse 47 403 [SI Q401 = OUI] Les menstruations sont un signe de puberté. o Vrai o Faux o Ne sait pas / Pas de réponse 404 [SI Q401 = OUI] Une fille est sale lorsqu’elle a ses règles. o Vrai o Faux o Ne sait pas / Pas de réponse 405 [SI Q401 = OUI] Quels moyens de protection pour les menstruations connais-tu ? NB : Ne pas lire les réponses • Serviettes hygiéniques • Tissus/linges • Coton/compresse • Tampon • Plusieurs culottes • Papier hygiénique • Ne sait pas / Pas de réponse 406 [SI Q102 = Féminin] As-tu déjà eu tes règles ? o Oui o Non o Ne sait pas / Pas de réponse 407 [SI Q406 = OUI] As-tu déjà manqué l’école parce que tu avais tes règles ? o Oui o Non o Ne sait pas / Pas de réponse 408 [SI Q407 = OUI] Combien de jours a tu été absente la dernière fois que tu as eu tes règles Jours 409 [SI Q407 = OUI] Pour quelles raisons n'es-tu pas allée à l’école ? NB : Ne pas lire les réponses • Peur des odeurs/fuites/me salir à l’école • Douleur • Pas d’endroit pour se laver à l'école • Les toilettes sont trop sales • Il n’y a pas de toilettes • Je n’ai pas de matériels pour me protéger à l’école • Autre, spécifier • Ne sait pas / Pas de réponse 410 [SI Q406 = OUI] Quel matériel de protection utilises-tu lorsque tu as tes règles ? NB : Ne pas lire les réponses • Serviettes hygiéniques • Tissus/linges • Coton/compresse • Tampon • Plusieurs culottes • Papier hygiénique • Autre, spécifier • Ne sait pas / Pas de réponse 48 411 [SI Q406 # Serviettes hygiéniques] Pourquoi n’utilises-tu pas de serviettes hygiéniques ? NB : Si nécessaire expliquer ce qu’est une serviette hygiénique • Pas d’argent • Pas de magasins/magasins trop loin • Honte d’acheter • Pas dans les habitudes culturelles • Autre, spécifier • Ne sait pas / Pas de réponse 412 [SI Q406 = Serviettes hygiéniques] Habituellement, combien de fois changes-tu de protections dans la journée durant tes règles ? o 1 fois o 2 fois o 3 ou plus o Ne sait pas / Pas de réponse 413 [SI Q406 = OUI] Habituellement, peux-tu changer facilement de protections à l’école lorsque tu en as besoin ? o Oui o Non o Ne sait pas / Pas de réponse 414 [Q413 = Non] Pourquoi ne peux-tu pas les changer facilement ? NB : Ne pas lire les réponses • Pas de toilettes • Toilettes mixtes • Portes des toilettes ne se ferment pas • Les toilettes sont loin/en dehors de l’école • Pas d’eau dans les toilettes • Pas de poubelles dans l’école • Autre, spécifier • Ne sait pas / Pas de réponse 5. Socio-économique 501 Est-ce que vous disposez des éléments que je vais citer suivant dans votre maison ? [NB : Lister les options] o Radio o Télévision o Téléphone o Bicyclette o Moto o Voiture o Eau de pompe o Electricité o Latrines o Aucun Merci beaucoup d’avoir répondu à mes questions. Fin du questionnaire 49 PARENT QUESTIONNAIRE Cette section doit être remplie par l’enquêteur Date : (JJ-MM-AAAA) Code enquêteur: …………….. Nom de l’enquêteur : …………….. District : Beloha Tsihombe Bekily 1 2 3 Commune : Nom de l’école : Code répondant : Bonjour, je m'appelle………………, un enquêteur d’ATW, un cabinet d’études indépendant. Nous faisons actuellement une enquête pour connaitre la situation de l’éducation et de la nutrition des enfants dans votre district. Votre enfant est parmi les élèves sélectionnés pour l’enquête. J’aimerai donc vous poser quelques questionssur vos habitudes. Vos réponses serontstrictement confidentielles et ne seront pas partagées avec quiconque. Maintenant, avez-vous des questions à poser sur le sondage ? [Laisser le temps pour poser des questions et répondre au besoin] Me donnez-vousla permission de poursuivre l’interview ? Oui / Non Nº Question Réponse 1. Caractéristiques du parent 100 Genre (regardezle parent et cochez) o Masculin o Féminin 101 Etes-vous le chef de foyer ? o OUI o NON 102 Quel âge avez-vous ? [NB : mettre 99 si la personne ne veut pas répondre] ans 103 Combien de personnes vivent dans votre foyer, y compris vous-même ? (NB : mettre 99 si la personne ne veut pas répondre) personnes 50 104 Quel est le niveau d’éducation le plus élevé que vous avez fait ? o N’a pas fréquenté l’école o Primaire o Secondaire o Lycée o Bacc + o Ne sait pas / Pas de réponse 105 Quelle est votre occupation principale ? o Pas de travail rémunéré o Ménagère o Travail agricole et/ou élevage o Propriétaire foncier o Marchand(e) / Commerçant o Travailleur(euse) de bureau o Retraité(e) o Autre (à préciser) o Ne sait pas / Pas de réponse 2. Education des enfants 201 Combien avez-vous de filles ? (NB : mettre 99 si la personne ne veut pas répondre) filles 202 [Q201 > 0] Combien avez-vous de filles inscrites à l’école ? filles 203 Combien avez-vous de garçons ? (NB : mettre 99 si la personne ne veut pas répondre) garçons 204 [Q205 > 0] Combien avez-vous de garçons inscrits à l’école ? garçons 205 Est-ce que vous avez des enfants qui ont déjà arrêté l’école ? o OUI o NON 206 [Q212 = OUI] En quelle classe ont-ils arrêté ? REPONSES MULTIPLES • Préscolaire • T1 • T2 • T3 • T4 • T5 51 207 [Q212 = OUI] Pouvez-vous me dire les raisons ? [REPONSES MULTIPLES] Open ended question • Manque de moyens • L’enfant n’avait plus envi • L’enfant a trouvé du travail • Il n’y a plus de place pour lui à l’école • Il a terminé la primaire et il n'y a pas d’école pour le second cycle • Il est déjà assez âgé et a honte d’aller à l’école • Autres à préciser • Ne sais pas / pas de réponse 208 En général, comment trouvez-vous le résultat scolaire des enfants ici dans votre village ? o Très bons o Bons o Moyens o Mauvais o Très mauvais 209 Qu’est-ce qui manque le plus pour avoir une éducation de qualité ? [REPONSES MULTIPLES] • Des infrastructures • Des enseignants • Des cantines scolaires • Des aides à domicile • Des cours particuliers • Autres à préciser 3. Santé des enfants 301 Quelles sont les maladies les plus fréquentes qui touchent vos enfants en âge scolaire ? [REPONSES MULTIPLES] • Diarrhée • Fièvre • Vomissement • Toux • Paludisme • Maladie respiratoire • Autre (à préciser) • Aucune 302 Durant ces deux dernières semaines, un de vos enfants a-t-il été touché par les maladies suivantes ? A- Diarrhée o OUI o NON o Ne sait pas/ Pas de reponse B- Fièvre o OUI o NON o Ne sait pas/ Pas de reponse C. Vomissement o OUI o NON o Ne sait pas/ Pas de reponse D. Toux o OUI o NON o Ne sait pas/ Pas de réponse 52 304 Selon vous, qu’est-ce qui manque le plus dans votre village pour pouvoir améliorer la santé des enfants ? [REPONSES MULTIPLES] • Centre de santé • Médecins qualifiés • Médicaments • Alimentation • Sensibilisation • Autre (à préciser) 4. Importance de l’éducation 401 Quelle importance accordez-vous à l’éducation scolaire de vos garçons ? o Très important o Important o Peu important o Pas du tout important o Ne sait pas / pas de réponse 402 Quelle importance accordez-vous à l’éducation scolaire de vos filles ? o Très important o Important o Peu important o Pas du tout important o Ne sait pas / Pas de réponse 403 Vous-même (ou autres membres de la famille) aidez-vous vos enfants pour leurs devoirs ? o Oui o Non o Ne sait pas / Pas de réponse 404 [403 = Oui] Si oui, pour quels types d’activité ? REPONSES MULTIPLES • Lire des lettres • Lire des mots • Lire un texte • Mathématiques • Faire réciter les leçons • Aide aux devoirs • Autre, préciser : 405 [Q403 = Non] Pourquoi ? [NB : Ne pas donner les réponses] REPONSES MULTIPLES • Les parents ne s’intéressent pas à l’école • Les parents pensent que ce n’est pas leur rôle • Les parents n’ont pas le temps • Les parents pensent que cela ne sert à rien • Les parents ne savent pas comment les aider ? • Les parents ne parlent pas assez bien la langue d’enseignement • Les parents ne savent pas lire • Autre, préciser : 53 406 Vous personnellement, pensez-vous qu’il est important pour les enfants de terminer l’école primaire ? o OUI o NON 407 Pouvez-vous me citer les avantages pour un enfant qui arrive à terminer l’école primaire ? [REPONSES MULTIPLES] • Réduire la pauvreté • Peut continuer au second cycle (CEG) • Dispose d’un diplôme (CEPE) • Peut avoir des bons travails • Acquérir des connaissances, des compétences et une éthique • Avoir une capacité à s’exprimer • Peut subvenir à ses besoins • Peut très bien parler en français • Peut réussir dans la vie • Est bien éduqué • Renforcement des capacités pour un développement ultérieur • S'occuper des parents lorsqu'ils sont âgés • Autres à préciser 408 Et vous personnellement, souhaiteriez-vous que vos enfants arrivent à terminer l’école primaire ? o OUI o NON 5. Association d’épargne et de crédit 501 Etes-vous membre d’une association d’épargne ou de crédit ? o OUI o NON = PASSER AU THEME 6 502 Pouvez-vous me dire le nom de cette association ? 503 Depuis combien de temps êtes-vous membre de cette association ? 504 Quel est votre rôle dans cette association ? o Président o Vice-président o Trésorier o Conseiller o Animateur o Simple membre o Autre (à préciser) 505 Quelles sont les activités de votre association ? [REPONSES MULTIPLES – ne pas lire les options] • Payer des cotisations • Se faire des épargnes • Réaliser des animations, sensibilisations • Chercher des formations • Autres spécifier 54 • Ne sait pas / Pas de réponse 506 Depuis que vous êtes membre de cette association, parvenez-vous à financer la scolarité de vos enfants sans problème ? o Oui o Non 507 [Q506= OUI] Quel type de support sont vous en mesure de fournir à vos enfants ? [REPONSES MULTIPLES – ne pas lire les options] • Matériels (livre, cahier, stylo, crayon) • Frais de scolarité • Uniforme scolaire, tablier • Cartables • autres à préciser 508 Quels sont les avantages que vous trouvez en adhérant une association d’épargne ? [REPONSES MULTIPLES – ne pas lire les options] • Pouvoir réaliser des économies • Ne pas dépenser l’argent n’importe comment • Pouvoir prêter de l’argent en cas d’urgence • Avoir une meilleure situation financière • Autres spécifier • Ne sait pas / Pas de réponse 6. Socio-économique 601 Est-ce que vous disposez des éléments que je vais citer suivant dans votre maison ? [NB : Lister les options] o Radio o Télévision o Téléphone o Bicyclette o Moto o Voiture o Eau de pompe o Electricité o Latrines 7. Implication dans le projet MIANATRA (écoles mianatra seulement) o Aucun [DISTRICT= 1 OU 2] 701 Etes-vous sélectionné ou impliqué dans la mise en œuvre du projet Mianatra ? o Oui o Non 702 [701= OUI] Connaissez-vous les activités prévues dans l’école par le projet Mianatra ? Oui Non Ne sais/Pas de réponse 703 Si oui, lesquelles ? Cuisine Magasinier Culture des champs/Jardin Construction Évènement autour de la lecture Club hygiène Autre, spécifier 704 [701= OUI] Oui Non 55 Souhaiteriez-vous participer aux activités du projet dans l’école de vos enfants ? Ne sais/Pas de réponse 705 Si non, que faudrait-il faire pour vous encourager à participer ? [NB : ne pas donner les réponses] Donner des repas Donner des aliments Partager des résultats Apporter plus de formation Améliorer la collaboration avec les enseignants Autre, spécifier PARENT QUESTIONNAIRE – HEALTH INDICATORS Nº Question Réponse 1. Caractéristiques du parent 101 Merci de me recevoir à nouveau aujourd’hui, j’ai juste 3 petites questions complémentaires. Est-ce que votre enfant a été absent de l’école durant ces deux dernières semaines ? o OUI o NON 102 [101=OUI] Pourriez-vous me dire la raison s’il vous plait ? o Malade o Paresseux o Faim et n’a pas voulu y aller o Fatigué o L’enfant a dû travailler o N’as pas fait ses devoirs o Ne se sent pas bien à l’école o Autres à préciser o Ne sais pas / Pas de réponse 103 Pouvez-vous nous dire de quelle maladie votre enfant a été atteint ? REPONSES MULTIPLES Fièvre Paludisme Diarrhée Vomissement Toux Maladie respiratoire Autre, spécifier Pas de réponse 104 Et qu’est-ce que vous avez fait ? REPONSES MULTIPLES o Aller à l’hopital o Consulter un médecin o Acheter des médicaments directs o Acheter des médicaments traditionnels o Autre à préciser 105 Et combien de jours votre enfant a￾t-il/elle absent ? o ………………….. jours Merci beaucoup d’avoir pris le temps de répondre ce sondage. Fin du questionnaire 56 DIRECTOR INTERVIEW AND SCHOOL INVENTORY Cette section doit être remplie par l’enquêteur Date : (JJ-MM-AAAA) Code enquêteur: …………….. Nom de l’enquêteur : …………….. District : Beloha Tsihombe Bekily 1 2 3 Commune : Nom de l’école : Bonjour ! Je m’appelle _____, un enquêteur d’ATW, un cabinet d’études indépendant. Nous faisons actuellement une enquête pour connaitre la situation de l’éducation et de la nutrition des enfants dans votre district. Votre école est parmi les établissements sélectionnés pour l’enquête. J’aimerai donc vous poser quelques questionssur l’environnement de cette école. Vos réponses serontstrictement confidentielles et ne seront pas partagées avec quiconque. Maintenant, avez-vous des questions à poser sur le sondage ? [Laisser le temps pour poser des questions et répondre au besoin] Me donnez-vous la permission de poursuivre l’interview ? Oui / Non Nº Question Réponse 1. Caractéristiques du directeur et capacités Je vais commencer par vous poser quelques questions vous même 101 Êtes-vous le directeur en titre de l'école ? o Oui o Non 102 Genre (regardezl’interlocuteur et cochez) o Masculin o Féminin 103 Quel âge avez-vous ? [NB : mettre 99 si la personne ne veut pas répondre] _______ ans 104 Quelle(s) classe(s) enseignez-vous ? T1 T2 T3 T4 T5 Aucune (Seulement directeur) 105 Quel type de contrat avez-vous ? o Fonctionnaire (Publique) o Vacataire o Maitre des parents (FRAM) o Maitre bénévole o Autre (Stagiaire…) 106 Quel est votre niveau d’étude ? o Master ( bacc +5) o Maitrise (bacc +4) 57 Nº Question Réponse o License ( bacc +3) o BEP (bacc +2) o Baccalauréat o BEPC o CEP 107 Quel est votre niveau de formation d’enseignant ? 108 Depuis combien d’années enseignez-vous ? _______ années 109 [Q100 = Directeur] Depuis combien d’années êtes-vous directeur d’établissement ? [NB : Dans cette école ou une autre école] _______ années 110 Avez-vous reçu une formation professionnelle (Formation continue) au cours des 3 dernières années [NB : Ne pas prendre en considération les réunions pédagogiques, inclure les formations de Nascent] o Oui o Non o Ne sait pas / Pas de réponse 2. Information sur l’école Je vais maintenant vous poser quelques questions sur votre école et les enseignants 200 Total élèves inscrits pour cette année scolaire 2022-2023 200 A Nombre élèves masculins 200 B Nombre élèves féminins 201 Quelles classes sont enseignées au sein de votre établissement ? [REPONSES MULTIPLES] Préscolaire T1 T2 T3 T4 T5 202 Combien y a-t -il de salles de classe utilisées dans votre école ? NB Compter même en matériaux provisoires _______ 204 Combien de sessions y-a-t-il dans votre établissement ? o Une session seulement o Deux sessions (matin et après-midi) 205 Combien d’enseignants de sexe masculin y-a-t ’il dans votre école ? [NB : Inclure le directeur] ______ 206 Combien d’enseignants de sexe féminin y-a-t ‘il dans votre école ? [Inclure la directrice] ______ 207 Dans quelles classes y a-t ‘il un maître des parents (FRAM) dans votre école ? Aucune Préscolaire T1 58 Nº Question Réponse T2 T3 T4 T5 208 T Les maîtres titulaires ont-ils été assidus à l’école depuis le début de l’année ? (NB : Oui vraiment = moins de 2 jours d’absence, pas du tout =plus 20 jours d’absence) o Oui vraiment o Oui plus ou moins o Non, pas vraiment o Non Pas de tout o Ne sait pas / pas de réponse 208 P [207 # aucune] Les maîtres FRAM ont-ils été assidus à l’école depuis le début de l’année ? (NB : Oui vraiment = moins de 2 jours d’absence, pas du tout =plus 20 jours d’absence) o Oui vraiment o Oui plus ou moins o Non, pas vraiment o Non Pas de tout o Ne sait pas / pas de réponse 209 T Les maîtres titulaires sont-ils ponctuels depuis le début de l’année ? (NB : Oui vraiment = pas de retard, pas du tout = plus de 2 heures de retard) o Oui vraiment o Oui plus ou moins o Non, pas vraiment o Non Pas de tout o Ne sait pas / pas de réponse 209 P [207 # aucune] Les maîtres FRAM ont-ils été ponctuels depuis le début de l’année ? (NB : Oui vraiment = pas de retard, pas du tout = plus de 2 heures de retard) o Oui vraiment o Oui plus ou moins o Non, pas vraiment o Non Pas de tout o Ne sait pas / pas de réponse 210 Votre école a-t-elle une association de parents d’élèves ? o Oui o Non 211 [Si oui 210] Comment qualifiez-vous le niveau d’activité de l’association des parents d’élèves ? [NB : Très active : Organise des réunions mensuellement et tient des procès-verbaux, Modérément active : se réunit une fois par trimestre] o Très active o Modérément active o Pas du tout active o Ne sait pas / Pas de réponse 212 Votre école a-t-elle une cantine fonctionnelle ? NB : En dehors des périodes de restriction Covid o Oui o Non 213 Votre école a-t-elle des vendeurs de snack et/ou repas ? NB : En dehors des périodes de restriction Covid o Oui o Non 214 Votre école dispose-t ’elle d’un club d’hygiène ? o Oui o Non 215 Votre école dispose-t ’elle d’une bibliothèque ? o Oui o Non 59 Nº Question Réponse 216 Votre école dispose-t-elle de coins de lecture dans certaines classes ? o Oui o Non 217 [Si oui 216] Lesquelles ? [REPONSES MULTIPLES] Préscolaire T1 T2 T3 T4 T5 218 Votre école a-t ’elle bénéficié de programme d’aide d’une ONG internationale (Autre que GC) dans les 3 dernières années ? o Oui o Non o Ne sait pas / Pas de réponse 219 [218=OUI] De quels types d’aide votre école a-t ‘elle bénéficié ? [REPONSES MULTIPLES] Forage ou puit Latrines Salle de classe Formation des enseignants Établissement des actes de naissance Distribution de matériel scolaire Dispositif de lavage de main Autre, spécifier 3. Infrastructure sanitaire de l’école Je vais maintenant vous poser quelques questions sur les infrastructures sanitaires de l’école 301 Qu'utilisent les élèves pour faire leurs besoins dans votre école ? [NB : Défécation à l’air libre : Pas de toilettes, brousse, champ, seaux toilettes/latrines suspendues, toilettes à compostage, ou autre Latrines non améliorée : à fosses sans dalle/trou ouvert Latrines améliorée : Latrines à fosses avec dalle latrines améliorées ventilées, ou chasse d’eau avec ou sans réservoir] Défécation à l’air (Pas de latrines) Trou ouvert, fosse sans dalle Latrines à dalle Latrines améliorée Ne sait pas / Pas de réponse 302 De combien de latrines fonctionnelles dispose votre école ? [NB : Exclure les latrines des enseignants, indiquer « 0 » si pas de latrines, si latrine partage avec une autre école, diviser par le nombre d’écoles] ____ 303 [Q302 > 0] Y a-t ‘il des latrines pour garçons et des latrines pour filles ? o Oui o Non 304 [Oui 303] La séparation fille/garçon est-elle respectée ? o Oui vraiment o Oui plus ou moins o Non, pas vraiment o Non, pas de tout o Ne sait pas / pas de réponse 305 [Q302 > 0] o Oui o Non 60 Nº Question Réponse Les latrines sont-elles à usage exclusif pour l'école ? [Utilisées uniquement par les élèves ou les enseignants] ? o Ne sait Pas / Pas de réponse 306 [Q302 > 0] Les latrines sont-elles sûres ? [Les latrines ferment, on ne peut pas voir à l’intérieur, la dalle est en bon état] o Oui vraiment o Oui plus ou moins o Non, pas vraiment o Non Pas de tout o Ne sait pas / pas de réponse 307 [302 > 0] Votre école dispose-t-elle d'une procédure de routine pour le nettoyage et d'entretien des latrines, qui garantit que des latrines propres et fonctionnelles sont disponibles à tout moment ? o Oui o Non o Ne sait Pas / Pas de réponse 308 [302 > 0] Comment qualifieriez-vous l’état des latrines ? [NB : Ne pas lire les réponses] o En bon état o Facile d’utilisation o Propres o Il n’y en a pas assez o Cassées ou en mauvais état o Besoin d’amélioration o Sentent mauvais o Sales o Pleines (besoin d’être vidangée] o Autre, précisez o Ne sait pas / Pas de réponse 309 [302 > 0] Pensez-vous que vos élèves filles utilisent régulièrement les latrines de l’école ? o Oui, vraiment o Oui, plus ou moins o Non, pas vraiment o Non, pas du tout o Ne sait pas / Pas de réponse 310 [302 > 0] Pensez-vous que vos élèves garçons utilisent régulièrement les latrines de l’école ? o Oui, vraiment o Oui, plus ou moins o Non, pas vraiment o Non, pas du tout o Ne sait pas / Pas de réponse 311 Pensez-vous qu’à l’école, les filles sont capables d’avoir une hygiène menstruelle appropriée ? o Oui, vraiment o Oui, plus ou moins o Non, pas vraiment o Non, pas du tout o Ne sait pas / Pas de réponse 312 [Q311= Non] Pourquoi les filles ne peuvent-elles pas avoir une hygiène menstruelle appropriée à l’école ? NB : Ne pas lire les réponses Pas de toilettes Pas de toilettes pour filles uniquement Les portes des toilettes ne se ferment pas Les toilettes sont loin/en dehors de l’école Pas d’eau dans les toilettes Pas de poubelles dans les toilettes Les toilettes sont salles Autres, spécifier Ne sait pas / Pas de réponse 61 Nº Question Réponse 313 De combien de dispositifs de lavage de main dispose votre école ? [Un seau dans la classe peut être considéré comme une station de lavage] _____ 314 Les stations de lavages disposent-elles généralement de savon ou de cendre ? o Oui, toujours o Oui, parfois o Non, jamais 315 Comment qualifiez-vous l’état des dispositifs de lavages ? o Parfait o Assez bon o Besoins d’amélioration o Mauvais o Ne sait pas / Pas de réponse 316 Existent-ils des systèmes de lavage de mains à côté des latrines ? o Oui o Non 317 Votre école dispose-t-elle d’un accès à l’eau ? o Oui, dans l’école o Oui, à proximité de l’école o Oui, mais loin de l’école o Non 318 [Oui Q317] De quel est le type d’accès à l’eau l’école dispose-t￾elle ? o Puit fermé o Puit ouvert o Forage à pompe solaire o Forage à pompe manuelle ou à pied o Forage à pompe à moteur o Eau de pluie o Eau en bouteille o Eau courante o Eau livrée par camion-citerne o Autre 319 [Oui Q317] L’approvisionnement en eau fonctionne-il ? o Oui, toujours o Oui, parfois o Non, jamais 320 Y-a-il assez d’eau à l’école pour mener à bien les activités suivantes : NB : lister les activités et cocher Boire Faire la cuisine Se laver les mains Faire du jardinage Approvisionner la communauté 321 [Q318= Eau courante] L’eau est-elle traité avant d’être bue ? o Oui, toujours o Oui, parfois o Non, jamais 322 [Q321= Non] Quelle méthode est utilisée pour le traitement de l’eau ? Bouillir l’eau Filtrer l’eau Acheter l’eau Ajouter de l’eau de Javel, solution chlorée ou des comprimés de chlore Ajouter de l’alun (Aluminium Sulfate) Ne sait pas / Pas de réponse Autre, spécifier 323 Comment l’eau pour boire est-elle stockée ? Seau ou bassine en plastique 62 Nº Question Réponse Seau ou bassine en tôle galvanisée ou céramique Bidon avec bouchon Dans le réservoir de collecte des eaux de pluie Il n’y a pas de stockage Ne sait pas / pas de réponse Autre, spécifier 4. Repas scolaire 401 Votre école fournit-elle actuellement des repas scolaires ? o Oui o Non 402 [401= OUI] Quels sont les repas que vous fournissez ? REPONSES MULTIPLES Riz avec du mets Pain Thé/café Lait Koba Autre, spécifier 403 Avec quelle fréquence ? o Tous les jours o 3-4 jours par semaine o 1-2 jours par semaine 404 Qui fournit la nourriture ? o Le gouvernement o ONG o Parents o Autre __________ 405 Depuis quand ? CITER LES REPONSES o Cette année scolaire o Depuis la dernière année scolaire o Avant de la dernière année scolaire 406 L’école a-t ’elle une cuisine ? o Oui o Non 407 L’école a-t ‘elle un magasin ? o Oui o Non 5. Commentaires 501 Faites-nous part de vos commentaires, si vous en avez éventuellement ? 502 En tant que l’intervieweur, veuillez fournir des commentaires supplémentaires (par exemple, problèmes, clarifications, commentaires). 63 Qualitative Data Collection Tools INTERVIEW GUIDE – USDA, USAID Bonjour, nous sommes l'équipe en charge de l'évaluation de base du projet GC Mianatra. Notre objectif est de recueillir des informations afin de mieux comprendre le programme et de faire des recommandations pour le projet et éventuellement pour de futurs programmes. Nous aimerions avoir votre aide dans ce domaine. Si quelqu'un n'est pas à l'aise ou a des problèmes spécifiques, nous pouvons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l'USDA et le gouvernement de Madagascar dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion durera environ une heure. Nous écrirons vos réponses afin de mieux saisir vos contributions et nous enregistrerons également la discussion. Si vous ne voulez pas participer, il n'y aura aucun problème. Vous pouvez retirer votre participation à tout moment. Avez-vous des questions ? [Donnez-leur le temps de réagir et de répondre aux questions si nécessaire]. Est-ce que vous nous donnez votre consentement ? " [Avez-vous obtenu un consentement verbal ? ☐ ] Interviewer : Merci de m’avoir accueilli aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle au sein de USDA ? 2. Depuis combien de temps occupez-vous votre poste ? 3. Pouvez-vous parler un peu de votre organisation et comment elle travaille à Madagascar dans le secteur de la sécurité alimentaire ? 4. Comment vous êtes impliqué dans le programme MIANATRA dans le sud de Madagascar ? Pertinence 5. Quels problèmes pensez-vous que le projet Mianatra tente de résoudre ? À votre connaissance, quelles sont les solutions mises en place pour résoudre ces problèmes ? Pensez-vous que cette approche est la bonne ? Si oui, pourquoi ? Si non, pourquoi ? Comment le contexte de la région Androy a-t-il été pris en compte dans la conception du programme ? 6. Pourquoi avoir choisi la région Androy, en particulier les deux districts Tsihombe et Beloha pour mettre en œuvre le projet MIANATRA ? Cohérence 7. Connaissez-vous la théorie du changement du projet ? Comment la trouvez-vous ? a. Est-elle réaliste, appropriée et pratique à mettre en œuvre ? b. Des modifications sont-elles nécessaires dans la conception, la structure, la logique ou la gestion du projet pour faciliter la réalisation des résultats et de l'impact souhaités ? 8. Qu’en est-elle de la conception du projet, comment le projet peut-il améliorer le niveau de performance en littératie dans les premières années et surmonter les principaux défis à l'amélioration des résultats en littératie ? 64 9. Le programme est-il conçu pour être flexible, par exemple, la stratégie globale, les composantes ou les activités spécifiques peuvent-elles être ajustées au fil du temps en raison de l'évolution de l'environnement ou en réponse aux besoins de la population cible ? a. Quels sont les facteurs sur lesquels la conception du programme devrait être flexible ? b. Dans quelle mesure pensez-vous que la structure du projet est conçue pour s'adapter à cela ? Comment ça pourrait être amélioré ? Efficacité 10. Quels sont les principaux défis ou obstacles à la réalisation des activités et atteindre les résultats souhaités dans les principales composantes du projet (c'est-à-dire la santé, la nutrition et les pratiques alimentaires, le renforcement de la capacité à gérer les activités d'alimentation scolaire, l'amélioration de l'assiduité et de l'attention des élèves) ? 11. Y a-t-il une compréhension claire des rôles et des responsabilités par toutes les parties impliquées pour la mise en œuvre et le suivi ? 12. Pourriez-vous m’expliquer un peu les rôles et les responsabilités de votre organisation dans la mise en œuvre et le suivi du programme ? 13. Selon vous, les stratégies de suivi et d'évaluation du projet sont-elles réalistes, appropriées et pratiques à mettre en œuvre ? 14. Dans quelle mesure le cadre de résultats et l'approche programme sont-ils valables pour le contexte local ? a. La théorie du changement est-elle réaliste, appropriée et pratique à mettre en œuvre ? b. Quelles stratégies ou activités planifiées semblent efficaces pour atteindre les résultats et cibles attendus ? Lesquelles ne l'étaient pas ? Pourquoi ? c. La conception, la structure, la logique ou la gestion nécessitent-elles des modifications pour faciliter la réalisation des résultats et de l'impact souhaités ? d. Les stratégies de suivi et d'évaluation du programme, y compris les indicateurs et les cibles, sont-elles adaptées (SMART) pour suivre et soutenir la mise en œuvre des résultats ciblés ? Si non, quels sont les changements suggérés ? Efficience 15. Dans quelle mesure l'allocation prévue des ressources (ressources humaines, temps, expertise, fonds, etc.) est-elle efficace et réaliste pour fournir le soutien nécessaire et atteindre les objectifs plus larges du programme ? a. Comment le projet envisage-t-il d'engager les acteurs clés pour financer les activités du programme (par exemple, les contributions des gouvernements et des entités privées pour soutenir l'alimentation scolaire et les fournitures scolaires) ? Durabilité 16. Quel est le plan pour assurer la durabilité du projet ? Pour vous, est-il réaliste et approprié ? a. Quels pourraient être les obstacles à la réalisation du plan de pérennisation du programme ? 17. Pour vous, quels sont les principaux défis à lesquels le projet doit faire face ? COVID-19 ? chocs liés au climat ? a. Le projet a-t-il les moyens de faire face aux menaces ? 65 18. Tous les éléments de durabilité (capacité, motivation, ressources, liens) sont-ils pris en compte dans le plan de durabilité du projet ? Leçons apprises et conclusion 19. Qu’est-ce que vous pouvez tirer comme leçon dans l’élaboration du projet MIANATRA ? 20. Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? 21. Y a-t-il des points que l'USDA/USA-Embassy souhaite spécifiquement qu'AtW prenne en compte lors de cette évaluation ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis 66 INTERVIEW GUIDE – Global Communities: Country Director - DCOP Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci de m’avoir accueilli aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pourriez-vous m’expliquer un peu les rôles et les responsabilités de votre organisation dans la mise en œuvre et le suivi du programme ? Pertinence 2. Parlons un peu des priorités et des politiques en ce qui concerne l’éducation à Madagascar, a. Selon votre connaissance, quelles sont les principales politiques et priorités du gouvernement de Madagascar concernant l'alimentation scolaire durable ? Et l'enseignement primaire, l’alphabétisation ? Selon vous, quelles sont les principaux manques du gouvernement de Madagascar dans ces domaines ? Connaissances ? Capacités ? b. Dans quelle mesure le programme MIANATRA s'aligne-t-il avec la stratégie nationale de l’éducation ? Dans quelle mesure le programme s'aligne-t-il avec d'autres priorités et politiques de l’éducation et des cantines scolaires ? S'il vous plaît, expliquez. Cohérence 3. Connaissez-vous le cadre de résultats du programme ? 4. Pouvez-vous m’expliquer quels sont les principaux objectifs que le programme tente d’atteindre ? 5. Dans quelle mesure trouvez-vous que les activités prévues puissent emmener à atteindre ces objectifs ? Expliquez s’il vous plait a. Dans quelle mesure les cantines scolaires permettent-elles de lutter contre la faim ? 6. Dans quelle mesure le programme complète-t-il, s'appuie-t-il et s'aligne-t-il sur d'autres programmes d'éducation, de nutrition et d'alimentation scolaire en cours dans le pays et sur d'autres plans et priorités du GoM ? 7. La conception du projet est-elle adaptée pour améliorer le niveau de performance en littératie dans les premières années et surmonter les principaux défis à l'amélioration des résultats en littératie ? 67 8. Le programme est-il conçu pour être flexible, par exemple, la stratégie globale, les composantes ou les activités spécifiques peuvent-elles être ajustées au fil du temps en raison de l'évolution de l'environnement ou en réponse aux besoins de la population cible ? Efficacité 9. Quelles sont les influences que le COVID (et d'autres facteurs externes pertinents) peut avoir sur la mise en œuvre du programme ? a. Comment ces impacts peuvent-ils être atténués ? est-ce que le programme a développé une stratégie à utiliser dans le cas d’une nouvelle fermeture des écoles ? 10. Quels sont les principaux défis ou obstacles à la réalisation des activités et atteindre les résultats souhaités dans les principales composantes du projet ? 11. Y a-t-il une compréhension claire des rôles et des responsabilités par toutes les parties impliquées pour la mise en œuvre et le suivi ? a. Selon vous, les stratégies de suivi et d'évaluation du projet sont-elles réalistes, appropriées et pratiques à mettre en œuvre ? Efficience 12. Dans quelle mesure l'allocation prévue des ressources (ressources humaines, temps, expertise, fonds, etc.) est-elle efficace et réaliste pour fournir le soutien nécessaire et atteindre les objectifs plus larges du programme ? a. Comment le projet envisage-t-il d'engager les acteurs clés pour financer les activités du programme (par exemple, les contributions des gouvernements et des entités privées pour soutenir l'alimentation scolaire et les fournitures scolaires) ? Durabilité 13. Quel est le plan pour assurer la durabilité du projet ? Pour vous, est-il réaliste et approprié ? a. Quels pourraient être les obstacles à la réalisation du plan de pérennisation du programme ? 14. Quelles opportunités non développées pourraient être à portée de main qui peuvent contribuer à la durabilité des avantages et des résultats du projet ? a. Qui doit être impliqué (secteur privé, entités/associations communautaires, agences gouvernementales locales, etc.) pour assurer la durabilité ? 15. Le projet a-t-il les moyens de faire face aux menaces ou aux lacunes (c'est-à-dire les chocs liés au climat, COVID-19) qui pourraient avoir un impact sur les progrès vers la durabilité et les résultats/avantages durables du projet ? a. Que doivent renforcer les organisations locales et les entités du secteur public pour pouvoir continuer à mettre en œuvre les activités au-delà de la fin du programme ? 16. Dans quelle mesure les parties prenantes (GoM, les communautés, le secteur privé) ont-elles fourni un soutien (monétaire ou en nature) pour l'alimentation scolaire et les activités complémentaires ? Ces apports sont-ils durables ? 68 17. Quel engagement a été pris pour soutenir une transition progressive vers un programme d'alimentation scolaire détenu et géré localement par le GoM, les écoles et les communautés ? 18. Tous les éléments de durabilité (capacité, motivation, ressources, liens) sont-ils pris en compte dans le plan de durabilité du projet ? 19. Quels sont les besoins spécifiques en capacités des parties prenantes du projet (le GoM, les parents, les administrateurs scolaires, les comités de gestion des écoles) auxquels le projet devrait répondre pour favoriser une plus grande appropriation et responsabilisation afin de maintenir les avantages du projet ? 20. Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 69 INTERVIEW GUIDE – Global Communities: MEAL Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci de m’avoir accueilli aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle au sein de cette organisation ? Pertinence 2. Parlons un peu des priorités et des politiques en ce qui concerne l’éducation à Madagascar ? a. Selon votre connaissance, quelles sont les principales politiques et priorités du gouvernement de Madagascar concernant [l'alimentation scolaire durable, le renforcement de la capacité de production alimentaire, l'enseignement primaire et l’alphabétisation ?] b. Selon vous, quelles sont les principaux manques du gouvernement de Madagascar dans ces domaines ? Connaissances ? Capacités ? c. Dans quelle mesure le programme MIANATRA s'aligne-t-il avec la stratégie nationale de l’éducation ? Dans quelle mesure le programme s'aligne-t-il avec d'autres priorités et politiques de l’éducation ? S'il vous plaît, expliquez. d. Les politiques et les actions que seront mises en œuvre pour l’éducation dans le cadre du projet Mianatra sont-elles conformes aux objectifs du gouvernement au niveau national ? Cohérence 3. La théorie du changement du projet est-elle réaliste, appropriée et pratique à mettre en œuvre ? a. Des modifications sont-elles nécessaires dans la conception, la structure, la logique ou la gestion du projet pour faciliter la réalisation des résultats et de l'impact souhaités ? b. Dans quelle mesure les cantines scolaires permettent-elles de lutter contre la faim 70 c. Pensez-vous que ce repas améliore la capacité des élèves à être attentifs à l'école et à mieux apprendre à lire ? Si oui, comment ? d. Pouvez-vous donner des exemples concrets ? Si non, pourquoi ? 4. La conception du projet est-elle adaptée pour améliorer le niveau de performance en littératie dans les premières années et surmonter les principaux défis à l'amélioration des résultats en littératie ? Efficacité 5. Quelles sont les influences que le COVID (et d'autres facteurs externes pertinents) peut avoir sur la mise en œuvre du programme ? a. Comment ces impacts peuvent-ils être atténués ? est-ce que le programme a développé une stratégie à utiliser dans le cas d’une nouvelle fermeture des écoles ? 6. Y a-t-il une compréhension claire des rôles et des responsabilités par toutes les parties impliquées pour la mise en œuvre et le suivi ? a. Selon vous, les stratégies de suivi et d'évaluation du projet sont-elles réalistes, appropriées et pratiques à mettre en œuvre ? b. Quels mécanismes avez-vous mis en place pour suivre les présences des élèves et établir des synthèses pour le suivi et évaluation ? 7. Dans quelle mesure le cadre de résultats et l'approche programme sont-ils valables pour le contexte local ? a. Quelles stratégies ou activités planifiées semblent efficaces pour atteindre les résultats et cibles attendus ? Lesquelles ne l'étaient pas ? Pourquoi ? b. La conception, la structure, la logique ou la gestion nécessitent-elles des modifications pour faciliter la réalisation des résultats et de l'impact souhaités ? c. Les stratégies de suivi et d'évaluation du programme, y compris les indicateurs et les cibles, sont-elles adaptées (SMART) pour suivre et soutenir la mise en œuvre des résultats ciblés ? Si non, quels sont les changements suggérés ? Efficience 8. Dans quelle mesure l'allocation prévue des ressources (ressources humaines, temps, expertise, fonds, etc.) est-elle efficace et réaliste pour fournir le soutien nécessaire et atteindre les objectifs plus larges du programme ? a. Comment le projet envisage-t-il d'engager les acteurs clés pour financer les activités du programme (par exemple, les contributions des gouvernements et des entités privées pour soutenir l'alimentation scolaire et les fournitures scolaires) ? Durabilité 9. Quel est le plan pour assurer la durabilité du projet ? Pour vous, est-il réaliste et approprié ? a. Quels pourraient être les obstacles à la réalisation du plan de pérennisation du programme ? 10. Quelles opportunités non développées pourraient être à portée de main qui peuvent contribuer à la durabilité des avantages et des résultats du projet ? 71 a. Qui doit être impliqué (secteur privé, entités/associations communautaires, agences gouvernementales locales, etc.) pour assurer la durabilité ? 11. Le projet a-t-il les moyens de faire face aux menaces ou aux lacunes (c'est-à-dire les chocs liés au climat, COVID-19) qui pourraient avoir un impact sur les progrès vers la durabilité et les résultats/avantages durables du projet ? a. Que doivent renforcer les organisations locales et les entités du secteur public pour pouvoir continuer à mettre en œuvre les activités au-delà de la fin du programme ? 12. Dans quelle mesure les parties prenantes et les acteurs locaux (le GoM à tous les niveaux, le secteur privé, les groupes communautaires, d'autres organisations mettant en œuvre des programmes d'alimentation scolaire) ont-ils été engagés à assurer des progrès vers un programme durable ? a. À quels besoins spécifiques en matière de capacité institutionnelle (par exemple, pour les principales parties prenantes au niveau du GoM, du district, de la communauté et de l'école) le projet devrait-il répondre pour favoriser un engagement et une responsabilisation accrus des districts, région et pays d’implantation du projet ? 13. Quel engagement a été pris pour soutenir une transition progressive vers un programme d'alimentation scolaire détenu et géré localement par le GoM, les écoles et les communautés ? 14. Tous les éléments de durabilité (capacité, motivation, ressources, liens) sont-ils pris en compte dans le plan de durabilité du projet ? 15. Quels sont les besoins spécifiques en capacités des parties prenantes du projet (le GoM, les parents, les administrateurs scolaires, les comités de gestion des écoles) auxquels le projet devrait répondre pour favoriser une plus grande appropriation et responsabilisation afin de maintenir les avantages du projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 72 INTERVIEW GUIDE – Global Communities: School Feeding Department – Commodity Director Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci de m’avoir accueilli aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle au sein de cette organisation ? 2. Pouvez-vous parler un peu de votre organisation et comment elle travaille à Madagascar ? Dans quel secteur ? 3. Comment vous êtes impliqué dans le programme MIANATRA dans le sud de Madagascar ? Pertinence 4. Selon votre connaissance, quelles sont les principales politiques et priorités du gouvernement de Madagascar concernant l'alimentation scolaire durable ? a. Selon vous, quelles sont les principaux manques du gouvernement de Madagascar dans ces domaines ? Connaissances ? Capacités ? b. Dans quelle mesure le programme MIANATRA s'aligne-t-il avec la stratégie nationale de l’éducation ? Dans quelle mesure le programme s'aligne-t-il avec d'autres priorités et politiques de l’éducation ? S'il vous plaît, expliquez. c. Les politiques et les actions que seront mises en œuvre pour l’éducation dans le cadre du projet Mianatra sont-elles conformes aux objectifs du gouvernement au niveau national ? Cohérence 5. La théorie du changement du projet est-elle réaliste, appropriée et pratique à mettre en œuvre ? a. Dans quelle mesure les cantines scolaires permettent-elles de lutter contre la faim b. Pensez-vous que ce repas améliore la capacité des élèves à être attentifs à l'école et à mieux apprendre à lire ? Si oui, comment ? 73 c. Pouvez-vous donner des exemples concrets ? Si non, pourquoi ? 6. Quels sont, selon vous, les besoins réels et perçus de bénéficiaires du projet ? Comment varient-ils pour les femmes par rapport aux hommes ou à d'autres sous-groupes ? Selon vous, quels sont les principaux défis pour progresser dans ces domaines ? a. Comment ces besoins et défis influencent-ils directement ou indirectement les résultats d'alphabétisation dans la zone du projet ? b. Dans quelle mesure les objectifs et résultats du programme répondent aux besoins de la communauté ? 7. Le programme est-il pertinent pour répondre aux besoins réels et perçus des bénéficiaires du projet : les élèves, les enseignants, les administrateurs scolaires, les parents/tuteurs, les représentants du gouvernement ? a. Dans quelle mesure le programme complète-t-il, s'appuie-t-il et s'aligne-t-il sur d'autres programmes d'alimentation scolaire en cours dans le pays et sur d'autres plans et priorités du GoM ? 8. La conception du projet est-elle adaptée pour améliorer le niveau de performance en littératie dans les premières années et surmonter les principaux défis à l'amélioration des résultats en littératie ? Efficacité 9. Quels sont les principaux défis ou obstacles à la réalisation des activités et atteindre les résultats souhaités dans les activités d'alimentation scolaire ? Durabilité 10. Quel est le plan pour assurer la durabilité du projet ? Pour vous, est-il réaliste et approprié ? a. Quels pourraient être les obstacles à la réalisation du plan de pérennisation du programme ? 11. Quelle stratégie allez-vous mettre en place pour assurer la continuité des cantines scolaires au-delà du programme ? 12. Quelles opportunités non développées pourraient être à portée de main qui peuvent contribuer à la durabilité des avantages et des résultats du projet ? a. Qui doit être impliqué (secteur privé, entités/associations communautaires, agences gouvernementales locales, etc.) pour assurer la durabilité ? 13. D'autres organisations mettant en œuvre des programmes d'alimentation scolaire ont-ils été engagés à assurer des progrès vers un programme durable ? 14. Dans quelle mesure les parties prenantes (GoM, les communautés, le secteur privé) ont-elles fourni un soutien (monétaire ou en nature) pour l'alimentation scolaire et les activités complémentaires ? Ces apports sont-ils durables ? 15. Quel engagement a été pris pour soutenir une transition progressive vers un programme d'alimentation scolaire détenu et géré localement par le GoM, les écoles et les communautés ? 16. Quels sont les besoins spécifiques en capacités des parties prenantes du projet (le GoM, les parents, les administrateurs scolaires, les comités de gestion des écoles) auxquels le projet devrait répondre pour favoriser une plus grande appropriation et responsabilisation afin de maintenir les avantages du projet ? 74 17. Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 75 INTERVIEW GUIDE – Global Communities: Village Savings and Loan Department Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci de m’avoir accueilli aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle au sein de cette organisation ? 2. Comment vous êtes impliqué dans le programme MIANATRA dans le sud de Madagascar ? Pertinence 3. Pouvez-vous m’expliquer les stratégies et interventions du projet pour renforcer les groupements d’épargne et de crédit au niveau des bénéficiaires du projet ? 4. Ces stratégies et interventions sont-elles appropriées et réalistes pour atteindre les résultats attendus ? Cohérence 5. Dans quelle mesure la mise en place de ces groupements peut-elle permettre d'améliorer les performances des élèves ? 6. Quels sont, selon vous, les besoins réels et perçus de bénéficiaires du projet ? Comment varient-ils pour les femmes par rapport aux hommes ? 7. Selon vous, quels sont les principaux défis pour progresser dans ces domaines ? 8. Dans quelle mesure les objectifs et résultats du programme répondent aux besoins de la communauté ? Efficacité 9. Quels sont les principaux défis ou obstacles à la réalisation des activités et atteindre les résultats souhaités ? 76 Durabilité 10. Quel est le plan pour assurer la durabilité du projet ? Pour vous, est-il réaliste et approprié ? a. Quels pourraient être les obstacles à la réalisation du plan de pérennisation du programme ? 11. Quels sont les besoins spécifiques en capacités des parents auxquels le projet devrait répondre pour favoriser une plus grande appropriation et responsabilisation afin de maintenir les avantages du projet ? 12. Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 77 INTERVIEW GUIDE – Global Communities: District Coordinator Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci de m’avoir accueilli aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle au sein de Global Communities ? 2. Comment vous êtes impliqué dans le programme MIANATRA dans le sud de Madagascar ? Pertinence 3. Selon vous, quels sont les besoins réels des enfants et des parents d’élèves dans votre district pour qu’ils puissent avoir un meilleur résultat scolaire ? 4. Les activités du programme sont-elles appropriées et réalistes pour répondre à ces besoins ? Cohérence 5. Concernant la théorie du changement du projet est-elle réaliste, dans quelle mesure les cantines scolaires permettent-elles de lutter contre la faim ? a. Pensez-vous que ce repas améliore la capacité des élèves à être attentifs à l'école et à mieux apprendre à lire ? Si oui, comment ? b. Pouvez-vous donner des exemples concrets ? Si non, pourquoi ? c. Dans quelle mesure les objectifs et résultats du programme répondent aux besoins de la communauté ? Efficacité 6. Pouvez-vous m’expliquer votre rôle dans la mise en œuvre et le suivi du projet ? 78 a. Selon vous, les stratégies de suivi et d'évaluation du projet sont-elles réalistes, appropriées et pratiques à mettre en œuvre ? Durabilité 7. Quelles opportunités non développées pourraient être à portée de main qui peuvent contribuer à la durabilité des avantages et des résultats du projet ? a. Qui doit être impliqué pour assurer la durabilité ? 8. Quels sont les besoins spécifiques en capacités des parties prenantes des parents, des administrateurs scolaires ou les comités de gestion des écoles auxquels le projet devrait répondre pour favoriser une plus grande appropriation et responsabilisation afin de maintenir les avantages du projet ? 9. Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 79 INTERVIEW GUIDE – CRS: Head of Program - DHoP Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci de m’avoir accueilli aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle au sein de cette organisation ? 2. Pouvez-vous parler un peu de votre organisation et comment elle travaille à Madagascar ? Dans quel secteur ? 3. Et comment CRS votre organisation est-elle impliquée dans le projet MIANATRA ? 4. Pourriez-vous m’expliquer un peu les rôles et les responsabilités de votre organisation dans la mise en œuvre et le suivi du programme ? Pertinence 5. Parlons un peu des priorités et des politiques en ce qui concerne l’éducation à Madagascar, a. Selon votre connaissance, quelles sont les principales politiques et priorités du gouvernement de Madagascar concernant l’assainissement et hygiène dans les écoles primaires ? b. Selon vous, quelles sont les principaux manques du gouvernement de Madagascar dans ces domaines ? Connaissances ? Capacités ? c. Les politiques et les actions que seront mises en œuvre dans le cadre du projet Mianatra sont-elles conformes à ces objectifs du gouvernement ? Notamment dans le domaine de l’eau, assainissement et hygiène. d. Dans quelle mesure les activités WASH peuvent-elles potentiellement permettre l'amélioration des apprentissages en lecture ? Cohérence 6. Dans quelle mesure le programme complète-t-il, s'appuie-t-il et s'aligne-t-il sur d'autres programmes d'éducation, de nutrition et d'alimentation scolaire en cours dans le pays et sur d'autres plans et priorités du GoM ? 80 7. Concernant le volet EAH du programme, est-il conçu pour être flexible, par exemple, la stratégie globale ou les activités spécifiques peuvent-elles être ajustées au fil du temps en raison de l'évolution de l'environnement ou en réponse aux besoins de la population cible ? Efficacité 8. Quels sont les principaux défis ou obstacles à la réalisation des activités et atteindre les résultats souhaités dans les principales composantes du projet ? 9. Y a-t-il une compréhension claire des rôles et des responsabilités par toutes les parties impliquées pour la mise en œuvre et le suivi ? b. Selon vous, les stratégies de suivi et d'évaluation du projet sont-elles réalistes, appropriées et pratiques à mettre en œuvre ? Efficience 10. Dans quelle mesure l'allocation prévue des ressources (ressources humaines, temps, expertise, fonds, etc.) est-elle efficace et réaliste pour fournir le soutien nécessaire et atteindre les objectifs plus larges du programme ? Durabilité 11. Quel est le plan pour assurer la durabilité du projet ? Pour vous, est-il réaliste et approprié ? a. Quels pourraient être les obstacles à la réalisation du plan de pérennisation du programme ? 12. Quelles opportunités non développées pourraient être à portée de main qui peuvent contribuer à la durabilité des avantages et des résultats du projet ? a. Qui doit être impliqué (secteur privé, entités/associations communautaires, agences gouvernementales locales, etc.) pour assurer la durabilité ? 13. Quels sont les besoins spécifiques en capacités des parties prenantes du projet (le GoM, les parents, les administrateurs scolaires, les comités de gestion des écoles) auxquels le projet devrait répondre pour favoriser une plus grande appropriation et responsabilisation afin de maintenir les avantages du projet ? Leçons apprises et conclusion 14. Qu’est-ce que vous pouvez tirer comme leçon dans l’élaboration du projet MIANATRA ? 15. Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 81 INTERVIEW GUIDE – CRS: WASH specialist Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci de m’avoir accueilli aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle au sein de CRS ? 2. Pouvez-vous m’expliquer les rôles et responsabilités de CRS dans le projet MIANATRA ? 3. Et vous personnellement, comment vous êtes impliqué dans le programme MIANATRA dans le sud de Madagascar ? Pertinence 4. Selon vous, quels sont les besoins réels de la communauté dans les deux districts du projet en ce qui concerne l’eau, l’assainissement et l’hygiène ? 5. Les activités du programme sont-elles appropriées et réalistes pour répondre à ces besoins ? 6. Dans quelle mesure les activités WASH peuvent-elles potentiellement permettre l'amélioration des apprentissages des élèves ? Cohérence 7. Pouvez-vous expliquer les activités du projet pour améliorer la situation en EAH des bénéficiaires ? Ecoles, élèves, ménages. Efficacité 8. Quels sont les principaux défis ou obstacles à la réalisation des résultats souhaités dans les composantes EAH du projet ? 9. Pouvez-vous m’expliquer votre rôle dans la mise en œuvre et le suivi du projet ? Selon vous, les stratégies de suivi et d'évaluation du projet sont-elles réalistes, appropriées et pratiques à mettre en œuvre ? Durabilité 10. Quelles opportunités non développées pourraient être à portée de main qui peuvent contribuer à la durabilité des avantages et des résultats du projet ? 82 b. Qui doit être impliqué pour assurer la durabilité ? 11. Quels sont les besoins spécifiques en capacités des parties prenantes des parents, des administrateurs scolaires ou les comités de gestion des écoles auxquels le projet devrait répondre pour favoriser une plus grande appropriation et responsabilisation afin de maintenir les avantages du projet ? 12. Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 83 INTERVIEW GUIDE – FHI 360 : Team lead education Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci de m’avoir accueilli aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle au sein de cette organisation ? 2. Pourriez-vous m’expliquer un peu les rôles et les responsabilités de votre organisation dans la mise en œuvre et le suivi du programme ? 3. Comment vous êtes impliqué dans le programme MIANATRA dans le sud de Madagascar ? Pertinence 4. Pouvez-vous me parler un peu des priorités et des politiques en ce qui concerne l’éducation à Madagascar ? a. Selon votre connaissance, quelles sont les principales politiques et priorités du gouvernement de Madagascar concernant i. L’alimentation scolaire durable, ii. Le renforcement de la capacité de production alimentaire, iii. L’enseignement primaire et l’alphabétisation ? b. Selon vous, quelles sont les principaux manques du gouvernement de Madagascar dans ces domaines ? Connaissances ? Capacités ? c. Dans quelle mesure le programme MIANATRA s'aligne-t-il avec la stratégie nationale de l’éducation ? Dans quelle mesure le programme s'aligne-t-il avec d'autres priorités et politiques de l’éducation ? S'il vous plaît, expliquez. d. Les politiques et les actions que seront mises en œuvre pour l’éducation dans le cadre du projet Mianatra sont-elles conformes aux objectifs du gouvernement au niveau national ? Cohérence 5. Connaissez-vous la théorie du changement du projet ? Si oui, est-elle réaliste, appropriée et pratique à mettre en œuvre ? 84 a. Des modifications sont-elles nécessaires dans la conception, la structure, la logique ou la gestion du projet pour faciliter la réalisation des résultats et de l'impact souhaités ? b. Dans quelle mesure les cantines scolaires permettent-elles de lutter contre la faim c. Pensez-vous que ce repas améliore la capacité des élèves à être attentifs à l'école et à mieux apprendre à lire ? Si oui, comment ? d. Pouvez-vous donner des exemples concrets ? Si non, pourquoi ? 6. Dans quelle mesure la mise en place de techniques ou méthodes spécifiques d'apprentissage de la lecture peut-elle permettre d'améliorer les performances des élèves en lecture ? Si oui, comment ? 7. Les stratégies et les interventions proposées par le projet pour accroître les connaissances et renforcer les capacités du Gouvernement sont-elles appropriées et réalistes ? 8. Selon vous, comment les principales les élèves décrivent-elles les besoins prioritaires dans les domaines l'éducation/l'alphabétisation, l'alimentation scolaire ? a. Y-a-t-il une différence entre les garçons et les filles ? b. Selon vous, quels sont les principaux défis pour progresser dans ces domaines ? 9. Et comment les parents décrivent-ils les besoins prioritaires dans le domaine de l’éducation/alphabétisation de leurs enfants ? 10. Et quels sont les besoins des enseignants, administrateurs scolaires pour améliorer la qualité de leurs travails et obtenir des meilleurs résultats sur l’éducation/alphabétisation ? a. Comment ces besoins et défis influencent-ils directement ou indirectement les résultats d'alphabétisation dans la zone du projet ? 11. Dans quelle mesure les objectifs et résultats du programme répondent aux besoins de la communauté ? 12. Le programme est-il pertinent pour répondre aux besoins réels et perçus des bénéficiaires du projet (les élèves, les enseignants, les administrateurs scolaires, les parents/tuteurs) ? a. Dans quelle mesure le programme complète-t-il, s'appuie-t-il et s'aligne-t-il sur d'autres programmes d'éducation, de nutrition et d'alimentation scolaire en cours dans le pays et sur d'autres plans et priorités du GoM ? 13. La conception du projet est-elle adaptée pour améliorer le niveau de performance en littératie dans les premières années et surmonter les principaux défis à l'amélioration des résultats en littératie ? Efficacité 14. Dans quelle mesure la pandémie COVID-19 est-elle susceptible d'influencer l'efficacité/les résultats du programme et comment le programme peut-il répondre au mieux à ces influences ? a. Quelles sont les influences que le COVID (et d'autres facteurs externes pertinents) peut avoir sur la mise en œuvre du programme ? b. Comment ces impacts peuvent-ils être atténués ? est-ce que le programme a développé une stratégie à utiliser dans le cas d’une nouvelle fermeture des écoles ? 15. Quels sont les principaux défis ou obstacles à la réalisation des activités et atteindre les résultats souhaités dans les principales composantes du projet : la nutrition et les pratiques alimentaires, le renforcement de la capacité à gérer les activités d'alimentation scolaire, l'amélioration de l'assiduité et de l'attention des élèves ? 16. Y a-t-il une compréhension claire des rôles et des responsabilités par toutes les parties impliquées pour la mise en œuvre et le suivi ? a. Selon vous, les stratégies de suivi et d'évaluation du projet sont-elles réalistes, appropriées et pratiques à mettre en œuvre ? Quels mécanismes avez-vous mis en place pour suivre les présences des élèves et établir des synthèses pour le suivi et évaluation ? 85 17. Dans quelle mesure le cadre de résultats et l'approche programme sont-ils valables pour le contexte local ? a. Quelles stratégies ou activités planifiées semblent efficaces pour atteindre les résultats et cibles attendus ? Lesquelles ne l'étaient pas ? Pourquoi ? b. La conception, la structure, la logique ou la gestion nécessitent-elles des modifications pour faciliter la réalisation des résultats et de l'impact souhaités ? c. Les stratégies de suivi et d'évaluation du programme, y compris les indicateurs et les cibles, sont-elles adaptées (SMART) pour suivre et soutenir la mise en œuvre des résultats ciblés ? Si non, quels sont les changements suggérés ? Durabilité 18. Quel est le plan pour assurer la durabilité du projet ? Pour vous, est-il réaliste et approprié ? a. Quels pourraient être les obstacles à la réalisation du plan de pérennisation du programme ? 19. Quelles opportunités non développées pourraient être à portée de main qui peuvent contribuer à la durabilité des avantages et des résultats du projet ? a. Qui doit être impliqué (secteur privé, entités/associations communautaires, agences gouvernementales locales, etc.) pour assurer la durabilité ? 20. Le projet a-t-il les moyens de faire face aux menaces ou aux lacunes (c'est-à-dire les chocs liés au climat, COVID-19) qui pourraient avoir un impact sur les progrès vers la durabilité et les résultats/avantages durables du projet ? a. Que doivent renforcer les organisations locales et les entités du secteur public pour pouvoir continuer à mettre en œuvre les activités au-delà de la fin du programme ? 21. À quels besoins spécifiques en matière de capacité institutionnelle (par exemple, pour les principales parties prenantes au niveau du GoM, du district, de la communauté et de l'école) le projet devrait-il répondre pour favoriser un engagement et une responsabilisation accrus des districts, région et pays d’implantation du projet ? 22. Quel engagement a été pris pour soutenir une transition progressive vers un programme d'alimentation scolaire détenu et géré localement par le GoM, les écoles et les communautés ? 23. Tous les éléments de durabilité (capacité, motivation, ressources, liens) sont-ils pris en compte dans le plan de durabilité du projet ? 24. Quels sont les besoins spécifiques en capacités des parties prenantes du projet (le GoM, les parents, les administrateurs scolaires, les comités de gestion des écoles) auxquels le projet devrait répondre pour favoriser une plus grande appropriation et responsabilisation afin de maintenir les avantages du projet ? 25. Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 86 INTERVIEW GUIDE – MEN : Training Department Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci d'avoir parlé avec moi aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle au sein de ce ministère ? 2. Que savez-vous du projet MIANATRA dans le sud de Madagascar ? 3. Comment vous êtes impliqué dans le programme MIANATRA dans le sud de Madagascar ? 4. Pourriez-vous m’expliquer un peu les rôles et les responsabilités de votre organisation dans la mise en œuvre et le suivi du programme ? Pertinence 5. Pouvez-vous me parler un peu des priorités et des politiques en ce qui concerne l’éducation à Madagascar ? a. Quelles sont les principales politiques et priorités du gouvernement de Madagascar concernant l'enseignement primaire et l'alphabétisation ? b. Selon vous, quelles sont les principaux manques du gouvernement de Madagascar dans ces domaines ? Connaissances ? Capacités ? c. Dans quelle mesure le programme MIANATRA s'aligne-t-il avec la stratégie nationale de l’éducation ? Dans quelle mesure le programme s'aligne-t-il avec d'autres priorités et politiques de l’éducation ? S'il vous plaît, expliquez. d. Les politiques et les actions que seront mises en œuvre pour l’éducation dans le cadre du projet Mianatra sont-elles conformes aux objectifs du gouvernement au niveau national ? Cohérence 6. Dans quelle mesure la mise en place de techniques ou méthodes spécifiques d'apprentissage de la lecture peut-elle permettre d'améliorer les performances des élèves en lecture ? Si oui, comment ? 7. Les stratégies et les interventions proposées par le projet pour accroître les connaissances et renforcer les capacités du Gouvernement sont-elles appropriées et réalistes ? 8. Selon vous, quels sont les besoins des enseignants, administrateurs scolaires pour améliorer la qualité de leurs travails et obtenir des meilleurs résultats sur l’éducation/alphabétisation ? 87 a. Comment ces besoins et défis influencent-ils directement ou indirectement les résultats d'alphabétisation dans la zone du projet ? 9. Dans quelle mesure les objectifs et résultats du programme répondent aux besoins de la communauté ? 10. Le programme est-il pertinent pour répondre aux besoins réels et perçus des enseignants, des administrateurs scolaires ? a. Dans quelle mesure le programme complète-t-il, s'appuie-t-il et s'aligne-t-il sur d'autres programmes d'éducation, de nutrition et d'alimentation scolaire en cours dans le pays et sur d'autres plans et priorités du GoM ? b. La conception du projet est-elle adaptée pour améliorer le niveau de performance en littératie dans les premières années et surmonter les principaux défis à l'amélioration des résultats en littératie ? Efficacité 11. Pour vous, quels sont les principaux défis ou obstacles pour améliorer l’alphabétisation des garçons et des filles ainsi que les performances scolaires ? a. Quels sont les principaux facteurs clés pour atteindre ces objectifs ? 12. Et quels pourraient être les principaux défis ou obstacles à l’amélioration de la formation pédagogique des enseignants ? a. Quels sont les principaux facteurs clés pour atteindre cet objectif ? 13. Avez-vous des suggestions ou recommandations pour un meilleur suivi du programme ? a. Quel système de suivi devrait être instaurer au niveau des chefs ZAP/CISCO ? Efficience 14. Dans quelle mesure l'allocation prévue des ressources (ressources humaines, temps, expertise, fonds, etc.) est-elle efficace et réaliste pour fournir le soutien nécessaire et atteindre les objectifs plus larges du programme ? 15. Dans quelle mesure votre ministère est-il en mesure de soutenir le programme en fournissant un soutien technique dans les écoles du programme ? Qu'en est-il de l'allocation des ressources pour la mise en œuvre et le suivi des activités clés de l'alimentation scolaire ? Durabilité 16. Quelles activités/stratégies clés le programme devrait-il s'efforcer/travailler d'intégrer dans le programme pour s'assurer que les activités et les résultats clés sont maintenus au-delà de la durée du projet ? 17. Quelles opportunités non développées pourraient être à portée de main qui peuvent contribuer à la durabilité des avantages et des résultats du projet ? a. Qui doit être impliqué (secteur privé, entités/associations communautaires, agences gouvernementales locales, etc.) pour assurer la durabilité ? 18. Quel type de stratégies le gouvernement a-t-il mis en place pour faire face aux chocs externes tel que le COVID-19 ? a. Quel est le plan du ministère de l’éducation en cas de covid et fermeture des écoles ? b. Capacité technologique pour prendre en charge la lecture/l'enseignement à distance 19. Pour vous, qui sont les parties prenantes/acteurs clés à prendre en compte pour assurer la pérennité des activités d'alimentation scolaire et d'amélioration de l'alphabétisation ? a. Gouvernement, secteur privé, groupes communautaires, autres organisations mettant en œuvre des programmes d'alimentation scolaire 20. Quelles structures doivent être établis/renforcés ? 88 21. Quel engagement a été pris pour soutenir une transition progressive vers un programme d'alimentation scolaire détenu et géré localement par le GoM, les écoles et les communautés ? Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 89 INTERVIEW GUIDE – MEN : School Feeding Department Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci d'avoir parlé avec moi aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle au sein de ce ministère ? 2. Que savez-vous du projet MIANATRA dans le sud de Madagascar ? 3. Comment vous êtes impliqué dans le programme MIANATRA dans le sud de Madagascar ? 4. Pourriez-vous m’expliquer un peu les rôles et les responsabilités de votre organisation dans la mise en œuvre et le suivi du programme ? Pertinence 5. Pouvez-vous me parler un peu des priorités et des politiques en ce qui concerne l’éducation à Madagascar ? a. Selon votre connaissance, quelles sont les principales politiques et priorités du gouvernement de Madagascar concernant : i. L'alimentation scolaire durable ii. Le renforcement de la capacité de production alimentaire iii. L'enseignement primaire et l’alphabétisation b. Selon vous, quelles sont les principaux manques du gouvernement de Madagascar dans ces domaines ? Connaissances ? Capacités ? c. Dans quelle mesure le programme MIANATRA s'aligne-t-il avec la stratégie nationale de l’éducation ? Dans quelle mesure le programme s'aligne-t-il avec d'autres priorités et politiques de l’éducation ? S'il vous plaît, expliquez. d. Les politiques et les actions que seront mises en œuvre pour l’éducation dans le cadre du projet Mianatra sont-elles conformes aux objectifs du gouvernement au niveau national ? Cohérence 6. Dans quelle mesure les cantines scolaires permettent-elles de lutter contre la faim a. Pensez-vous que ce repas améliore la capacité des élèves à être attentifs à l'école et à mieux apprendre à lire ? Si oui, comment ? b. Pouvez-vous donner des exemples concrets ? Si non, pourquoi ? 90 7. Les stratégies et les interventions proposées par le projet pour accroître les connaissances et renforcer les capacités du Gouvernement sont-elles appropriées et réalistes ? 8. Selon vous, comment les principales les élèves décrivent-elles les besoins prioritaires dans les domaines l'éducation/l'alphabétisation, l'alimentation scolaire ? a. Y-a-t-il une différence entre les garçons et les filles ? b. Selon vous, quels sont les principaux défis pour progresser dans ces domaines ? 9. Et comment les parents décrivent-ils les besoins prioritaires dans le domaine de l’éducation/alphabétisation de leurs enfants ? 10. Dans quelle mesure les objectifs et résultats du programme répondent aux besoins de la communauté ? 11. Le programme est-il pertinent pour répondre aux besoins réels et perçus des bénéficiaires du projet des élèves et leurs les parents/tuteurs ? a. Dans quelle mesure le programme complète-t-il, s'appuie-t-il et s'aligne-t-il sur d'autres programmes d'éducation, de nutrition et d'alimentation scolaire en cours dans le pays et sur d'autres plans et priorités du GoM ? 12. La conception du projet est-elle adaptée pour améliorer le niveau de performance en littératie dans les premières années et surmonter les principaux défis à l'amélioration des résultats en littératie ? Efficacité 13. Quels sont les principaux défis ou obstacles à la réalisation des activités et atteindre les résultats souhaités dans les principales composantes du projet : la nutrition et les pratiques alimentaires, le renforcement de la capacité à gérer les activités d'alimentation scolaire, l'amélioration de l'assiduité et de l'attention des élèves ? Durabilité 14. Quel est le plan pour assurer la durabilité du projet ? Pour vous, est-il réaliste et approprié ? a. Quels pourraient être les obstacles à la réalisation du plan de pérennisation du programme ? 15. Quelles opportunités non développées pourraient être à portée de main qui peuvent contribuer à la durabilité des avantages et des résultats du projet ? a. Qui doit être impliqué (secteur privé, entités/associations communautaires, agences gouvernementales locales, etc.) pour assurer la durabilité ? 16. Le projet a-t-il les moyens de faire face aux menaces ou aux lacunes (c'est-à-dire les chocs liés au climat, COVID-19) qui pourraient avoir un impact sur les progrès vers la durabilité et les résultats/avantages durables du projet ? a. Que doivent renforcer les organisations locales et les entités du secteur public pour pouvoir continuer à mettre en œuvre les activités au-delà de la fin du programme ? 17. À quels besoins spécifiques en matière de capacité institutionnelle (par exemple, pour les principales parties prenantes au niveau du GoM, du district, de la communauté et de l'école) le projet devrait-il répondre pour favoriser un engagement et une responsabilisation accrus des districts, région et pays d’implantation du projet ? 18. Quel engagement a été pris pour soutenir une transition progressive vers un programme d'alimentation scolaire détenu et géré localement par le GoM, les écoles et les communautés ? 19. Tous les éléments de durabilité (capacité, motivation, ressources, liens) sont-ils pris en compte dans le plan de durabilité du projet ? 20. Quels sont les besoins spécifiques en capacités des parties prenantes du projet (le GoM, les parents, les administrateurs scolaires, les comités de gestion des écoles) auxquels le projet devrait répondre pour favoriser une plus grande appropriation et responsabilisation afin de maintenir les avantages du projet ? 91 21. Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 92 INTERVIEW GUIDE – MEN : Monitoring and Evaluation Department Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci d'avoir parlé avec moi aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle au sein de ce ministère ? 2. Que savez-vous du projet MIANATRA dans le sud de Madagascar ? 3. Comment vous êtes impliqué dans le programme MIANATRA dans le sud de Madagascar ? 4. Pourriez-vous m’expliquer un peu les rôles et les responsabilités de votre organisation dans la mise en œuvre et le suivi du programme ? Pertinence 5. Pouvez-vous me parler un peu des priorités et des politiques en ce qui concerne l’éducation à Madagascar ? a. Selon votre connaissance, quelles sont les principales politiques et priorités du gouvernement de Madagascar concernant : i. L'alimentation scolaire durable ii. Le renforcement de la capacité de production alimentaire iii. L'enseignement primaire et l’alphabétisation ? b. Selon vous, quelles sont les principaux manques du gouvernement de Madagascar dans ces domaines ? Connaissances ? Capacités ? c. Dans quelle mesure le programme MIANATRA s'aligne-t-il avec la stratégie nationale de l’éducation ? Dans quelle mesure le programme s'aligne-t-il avec d'autres priorités et politiques de l’éducation ? S'il vous plaît, expliquez. d. Les politiques et les actions que seront mises en œuvre pour l’éducation dans le cadre du projet Mianatra sont-elles conformes aux objectifs du gouvernement au niveau national ? Cohérence 6. Dans quelle mesure la mise en place de techniques ou méthodes spécifiques d'apprentissage de la lecture peut-elle permettre d'améliorer les performances des élèves en lecture ? Si oui, comment ? 93 7. Les stratégies et les interventions proposées par le projet pour accroître les connaissances et renforcer les capacités du Gouvernement sont-elles appropriées et réalistes ? 8. Selon vous, comment les principales les élèves décrivent-elles les besoins prioritaires dans les domaines l'éducation/l'alphabétisation, l'alimentation scolaire ? a. Y-a-t-il une différence entre les garçons et les filles ? b. Selon vous, quels sont les principaux défis pour progresser dans ces domaines ? 9. Et comment les parents décrivent-ils les besoins prioritaires dans le domaine de l’éducation/alphabétisation de leurs enfants ? 10. Et quels sont les besoins des enseignants, administrateurs scolaires pour améliorer la qualité de leurs travails et obtenir des meilleurs résultats sur l’éducation/alphabétisation ? a. Comment ces besoins et défis influencent-ils directement ou indirectement les résultats d'alphabétisation dans la zone du projet ? 11. Dans quelle mesure les objectifs et résultats du programme répondent aux besoins de la communauté ? 12. Le programme est-il pertinent pour répondre aux besoins réels et perçus des bénéficiaires du projet (les élèves, les enseignants, les administrateurs scolaires, les parents/tuteurs) ? a. Dans quelle mesure le programme complète-t-il, s'appuie-t-il et s'aligne-t-il sur d'autres programmes d'éducation, de nutrition et d'alimentation scolaire en cours dans le pays et sur d'autres plans et priorités du GoM ? 13. La conception du projet est-elle adaptée pour améliorer le niveau de performance en littératie dans les premières années et surmonter les principaux défis à l'amélioration des résultats en littératie ? Efficacité 14. Dans quelle mesure la pandémie COVID-19 est-elle susceptible d'influencer l'efficacité/les résultats du programme et comment le programme peut-il répondre au mieux à ces influences ? a. Quelles sont les influences que le COVID (et d'autres facteurs externes pertinents) peut avoir sur la mise en œuvre du programme ? b. Comment ces impacts peuvent-ils être atténués ? est-ce que le programme a développé une stratégie à utiliser dans le cas d’une nouvelle fermeture des écoles ? 15. Quels sont les principaux défis ou obstacles à la réalisation des activités et atteindre les résultats souhaités dans les principales composantes du projet : la nutrition et les pratiques alimentaires, le renforcement de la capacité à gérer les activités d'alimentation scolaire, l'amélioration de l'assiduité et de l'attention des élèves ? 16. Y a-t-il une compréhension claire des rôles et des responsabilités par toutes les parties impliquées pour la mise en œuvre et le suivi ? a. Selon vous, les stratégies de suivi et d'évaluation du projet sont-elles réalistes, appropriées et pratiques à mettre en œuvre ? b. Quels mécanismes avez-vous mis en place pour suivre les présences des élèves et établir des synthèses pour le suivi et évaluation ? 17. Avez-vous des suggestions ou recommandations pour un meilleur suivi du programme ? a. Quel système de suivi devrait être instaurer au niveau des chefs ZAP/CISCO ? 18. Dans quelle mesure le cadre de résultats et l'approche programme sont-ils valables pour le contexte local ? a. Quelles stratégies ou activités planifiées semblent efficaces pour atteindre les résultats et cibles attendus ? Lesquelles ne l'étaient pas ? Pourquoi ? b. La conception, la structure, la logique ou la gestion nécessitent-elles des modifications pour faciliter la réalisation des résultats et de l'impact souhaités ? 94 c. Les stratégies de suivi et d'évaluation du programme, y compris les indicateurs et les cibles, sont-elles adaptées (SMART) pour suivre et soutenir la mise en œuvre des résultats ciblés ? Si non, quels sont les changements suggérés ? Durabilité 19. Quel est le plan pour assurer la durabilité du projet ? Pour vous, est-il réaliste et approprié ? a. Quels pourraient être les obstacles à la réalisation du plan de pérennisation du programme ? 20. Quelles opportunités non développées pourraient être à portée de main qui peuvent contribuer à la durabilité des avantages et des résultats du projet ? a. Qui doit être impliqué (secteur privé, entités/associations communautaires, agences gouvernementales locales, etc.) pour assurer la durabilité ? 21. Le projet a-t-il les moyens de faire face aux menaces ou aux lacunes (c'est-à-dire les chocs liés au climat, COVID-19) qui pourraient avoir un impact sur les progrès vers la durabilité et les résultats/avantages durables du projet ? a. Que doivent renforcer les organisations locales et les entités du secteur public pour pouvoir continuer à mettre en œuvre les activités au-delà de la fin du programme ? 22. À quels besoins spécifiques en matière de capacité institutionnelle (par exemple, pour les principales parties prenantes au niveau du GoM, du district, de la communauté et de l'école) le projet devrait-il répondre pour favoriser un engagement et une responsabilisation accrus des districts, région et pays d’implantation du projet ? 23. Tous les éléments de durabilité (capacité, motivation, ressources, liens) sont-ils pris en compte dans le plan de durabilité du projet ? 24. Quels sont les besoins spécifiques en capacités des parties prenantes du projet (le GoM, les parents, les administrateurs scolaires, les comités de gestion des écoles) auxquels le projet devrait répondre pour favoriser une plus grande appropriation et responsabilisation afin de maintenir les avantages du projet ? 25. Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 95 INTERVIEW GUIDE – Regional Agriculture and Livestock Department Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci d'avoir parlé avec moi aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle au sein du ministère de l’agriculture ? 2. Que savez-vous du projet MIANATRA dans les districts de Tsihombe et Beloha ? 3. Quel est votre rôle dans le projet ? Pertinence 4. Pouvez-vous me parler un peu des priorités et des politiques en ce qui concerne l’alimentation scolaire à Madagascar ? a. Le ministère de l’agriculture joue-t-il un rôle dans cela ? Si oui, lequel ? b. Quelles sont les principales politiques et priorités du gouvernement de Madagascar concernant le renforcement de la capacité de production alimentaire ? c. Selon vous, quelles sont les principaux manques du gouvernement de Madagascar dans ces domaines ? Connaissances ? Capacités ? Cohérence 5. Dans quelle mesure le programme complète-t-il, s'appuie-t-il et s'aligne-t-il sur d'autres programmes d'éducation, de nutrition et d'alimentation scolaire en cours dans le pays et sur d'autres plans et priorités du GoM ? 6. Selon vous, comment les enseignants ou les administrateurs scolaires décrivent-elles les besoins prioritaires dans les domaines l'alimentation scolaire ? 7. Et quels sont les besoins des agriculteurs pour pouvoir améliorer leur capacité de production et éventuellement approvisionner les écoles ? a. Selon vous, quels sont les principaux défis pour progresser dans ces domaines ? Efficacité 8. Et quels pourraient être les principaux défis ou obstacles à l’implémentation des cantines scolaires pour les enfants ? 96 a. Quels sont les principaux facteurs clés pour atteindre cet objectif ? 9. Avez-vous des suggestions ou recommandations pour l’approvisionnement des cantines scolaires dans ces zones d’intervention du projet ? Efficience 10. Dans quelle mesure l'allocation prévue des ressources (ressources humaines, temps, expertise, fonds, etc.) est-elle efficace et réaliste pour fournir le soutien nécessaire et atteindre les objectifs plus larges du programme ? 11. Dans quelle mesure votre ministère est-il en mesure de soutenir le programme en fournissant un soutien technique dans l’implémentation des cantines scolaires ? Creuser : appui sur la production des fermes scolaires et jardins scolaires Durabilité 12. Quelles activités/stratégies clés le programme devrait-il s'efforcer/travailler d'intégrer dans le programme pour s'assurer que les activités et les résultats clés sont maintenus au-delà de la durée du projet ? Creuser : appui sur la production des fermes scolaires et jardins scolaires, fermes des ménages 13. Quelles opportunités non développées pourraient être à portée de main qui peuvent contribuer à la durabilité des avantages et des résultats du projet ? a. Qui doit être impliqué (secteur privé, entités/associations communautaires, agences gouvernementales locales, etc.) pour assurer la durabilité ? 14. Pour vous, qui sont les parties prenantes/acteurs clés à prendre en compte pour assurer la pérennité des activités d'alimentation scolaire et d'amélioration de l'alphabétisation ? a. Gouvernement, secteur privé, groupes communautaires, autres organisations mettant en œuvre des programmes d'alimentation scolaire 15. Quelles structures doivent être établis/renforcés ? Creuser : appui sur la production des fermes scolaires et jardins scolaires, fermes des ménages 16. Quel engagement a été pris pour soutenir une transition progressive vers un programme d'alimentation scolaire détenu et géré localement par le GoM, les écoles et les communautés ? Creuser : appui sur la production des fermes scolaires et jardins scolaires, fermes des ménages 17. Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 97 INTERVIEW GUIDE – Regional WASH Department Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci de m’avoir accueilli aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle au sein du ministère de l’eau, assainissement et hygiène ? 2. Que savez-vous du projet MIANATRA dans les districts de Tsihombe et Beloha ? 3. Quel est votre rôle dans le projet ? Pertinence 4. Selon vous, quels sont les besoins réels de la communauté dans les deux districts du projet en ce qui concerne l’eau, l’assainissement et l’hygiène ? a. Comment pouvez-vous décrire la situation en eau, assainissement et hygiène des écoles primaires dans ces districts ? b. Quels sont les grands besoins de ces écoles en eau, assainissement et hygiène ? 5. Les activités du programme sont-elles appropriées et réalistes pour répondre à ces besoins ? 6. Dans quelle mesure les activités WASH peuvent-elles potentiellement permettre l'amélioration des apprentissages des élèves ? Cohérence 7. Dans quelle mesure le programme complète-t-il, s'appuie-t-il et s'aligne-t-il sur d'autres programmes d’eau, assainissement et hygiène qui sont en cours dans le pays ou sur d'autres plans et priorités du GoM ? 8. Selon vous, comment la population de ces deux districts décrivent-elle les besoins prioritaires dans les domaines l’eau, assainissement et hygiène ? 9. Et quels sont les besoins écoles dans les domaines de l’eau, assainissement et hygiène ? Efficacité 10. Quels sont les principaux défis ou obstacles à la réalisation des résultats souhaités dans les composantes EAH du projet ? 11. Pouvez-vous m’expliquer votre rôle dans la mise en œuvre et le suivi du projet ? 98 Selon vous, les stratégies de suivi et d'évaluation du projet sont-elles réalistes, appropriées et pratiques à mettre en œuvre ? Durabilité 12. Quelles opportunités non développées pourraient être à portée de main qui peuvent contribuer à la durabilité des avantages et des résultats du projet ? a. Qui doit être impliqué pour assurer la durabilité ? 13. Quels sont les besoins spécifiques en capacités des parties prenantes des parents, des administrateurs scolaires ou les comités de gestion des écoles auxquels le projet devrait répondre pour favoriser une plus grande appropriation et responsabilisation afin de maintenir les avantages du projet ? 14. Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 99 INTERVIEW GUIDE – National Education Department Bonjour, nous sommes l'équipe d'évaluation du programme Mianatra pour le compte de Global Communities. La finalité est de collecter des informations afin de mieux comprendre le programme et de formuler des recommandations pour le projet et éventuellement pour des futurs programmes. Nous aimerions avoir votre aide dans ce domaine. S'il y a quelqu'un qui n'est pas à l'aise ou qui a des questions particulières nous pourrons en discuter. Les résultats de l'enquête seront publiés sous la forme d'un rapport, qui sera partagé avec l’USDA et le Ministère de l’Éducation National dans le but de renforcer la mise en œuvre des programmes. Vos noms ne figureront pas dans ce rapport. Il ne serait pas possible de vous identifier. Notre discussion prendra environs une heure de temps. Nous allons écrire vos réponses et afin de mieux capter vos contributions nous allons aussi enregistrer la discussion. Si vous ne voulez pas participer, il n’y aura pas de problème. Vous pouvez retirer votre participation à tout moment. Avez-vous de questions ? [Donnez-leur le temps de réagir et répondre aux questions au besoin.] Est-ce que vous nous donnez votre consentement ?" [] Interviewer : Merci de m’avoir accueilli aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et me dire quelle est votre position actuelle ? 2. Que savez-vous du projet MIANATRA dans les districts de Tsihombe et Beloha ? 3. Quel est votre rôle dans le projet ? Pertinence 4. Selon votre connaissance, quelles sont les principales politiques et priorités du gouvernement de Madagascar concernant l'enseignement primaire et l’alphabétisation ? a. Selon vous, quelles sont les principaux manques de la DREN dans ces domaines ? Connaissances ? Capacités ? b. Dans quelle mesure le programme MIANATRA s'aligne-t-il avec la stratégie nationale de l’éducation ? Dans quelle mesure le programme s'aligne-t-il avec d'autres priorités et politiques de l’éducation ? S'il vous plaît, expliquez. c. Les politiques et les actions que seront mises en œuvre pour l’éducation dans le cadre du projet Mianatra sont-elles conformes aux objectifs du gouvernement au niveau national ? Cohérence 5. Dans quelle mesure les cantines scolaires permettent-elles de lutter contre la faim a. Pensez-vous que ce repas améliore la capacité des élèves à être attentifs à l'école et à mieux apprendre à lire ? Si oui, comment ? b. Pouvez-vous donner des exemples concrets ? Si non, pourquoi ? 6. Dans quelle mesure la mise en place de techniques ou méthodes spécifiques d'apprentissage de la lecture peut-elle permettre d'améliorer les performances des élèves en lecture ? Si oui, comment ? 7. Selon vous, comment les principales les élèves décrivent-elles les besoins prioritaires dans les domaines l'éducation/l'alphabétisation, l'alimentation scolaire ? a. Y-a-t-il une différence entre les garçons et les filles ? b. Selon vous, quels sont les principaux défis pour progresser dans ces domaines ? 100 8. Et comment les parents décrivent-ils les besoins prioritaires dans le domaine de l’éducation/alphabétisation de leurs enfants ? 9. Et quels sont les besoins des enseignants, administrateurs scolaires pour améliorer la qualité de leurs travails et obtenir des meilleurs résultats sur l’éducation/alphabétisation ? a. Comment ces besoins et défis influencent-ils directement ou indirectement les résultats d'alphabétisation dans la zone du projet ? 10. Dans quelle mesure les objectifs et résultats du programme répondent aux besoins de la communauté ? 11. Le programme est-il pertinent pour répondre aux besoins réels et perçus des bénéficiaires du projet (les élèves, les enseignants, les administrateurs scolaires, les parents/tuteurs) ? a. Dans quelle mesure le programme complète-t-il, s'appuie-t-il et s'aligne-t-il sur d'autres programmes d'éducation, de nutrition et d'alimentation scolaire en cours dans le pays et sur d'autres plans et priorités du GoM ? 12. La conception du projet est-elle adaptée pour améliorer le niveau de performance en littératie dans les premières années et surmonter les principaux défis à l'amélioration des résultats en littératie ? Efficacité 13. Quels sont les principaux défis ou obstacles à la réalisation des activités et atteindre les résultats souhaités dans les principales composantes du projet : la nutrition et les pratiques alimentaires, le renforcement de la capacité à gérer les activités d'alimentation scolaire, l'amélioration de l'assiduité et de l'attention des élèves ? 14. Y a-t-il une compréhension claire des rôles et des responsabilités par toutes les parties impliquées pour la mise en œuvre et le suivi ? a. Selon vous, les stratégies de suivi et d'évaluation du projet sont-elles réalistes, appropriées et pratiques à mettre en œuvre ? b. Quels mécanismes avez-vous mis en place pour suivre les présences des élèves et établir des synthèses pour le suivi et évaluation ? 15. Avez-vous des suggestions ou recommandations pour un meilleur suivi du programme ? a. Quel système de suivi devrait être instaurer au niveau des chefs ZAP/CISCO ? 16. Dans quelle mesure le cadre de résultats et l'approche programme sont-ils valables pour le contexte local ? a. Quelles stratégies ou activités planifiées semblent efficaces pour atteindre les résultats et cibles attendus ? Lesquelles ne l'étaient pas ? Pourquoi ? b. La conception, la structure, la logique ou la gestion nécessitent-elles des modifications pour faciliter la réalisation des résultats et de l'impact souhaités ? Durabilité 17. Quel est le plan pour assurer la durabilité du projet ? Pour vous, est-il réaliste et approprié ? a. Quels pourraient être les obstacles à la réalisation du plan de pérennisation du programme ? 18. Quelles opportunités non développées pourraient être à portée de main qui peuvent contribuer à la durabilité des avantages et des résultats du projet ? a. Qui doit être impliqué (secteur privé, entités/associations communautaires, agences gouvernementales locales, etc.) pour assurer la durabilité ? 19. Le projet a-t-il les moyens de faire face aux menaces ou aux lacunes (c'est-à-dire les chocs liés au climat, COVID-19) qui pourraient avoir un impact sur les progrès vers la durabilité et les résultats/avantages durables du projet ? a. Que doivent renforcer les organisations locales et les entités du secteur public pour pouvoir continuer à mettre en œuvre les activités au-delà de la fin du programme ? 101 20. Quel engagement a été pris pour soutenir une transition progressive vers un programme d'alimentation scolaire détenu et géré localement par le GoM, les écoles et les communautés ? 21. Quels sont les besoins spécifiques en capacités des administrateurs scolaires auxquels le projet devrait répondre pour favoriser une plus grande appropriation et responsabilisation afin de maintenir les avantages du projet ? 22. Y a-t-il autre chose que vous aimeriez me dire sur le projet MIANATRA ou votre expérience dans la conception de ce projet ? Interviewer : Voilà toutes mes questions. Y a-t-il autre chose que vous aimeriez ajouter à notre conversation ? Avez-vous des questions pour moi ? Sinon, merci beaucoup pour votre temps et pour avoir partagé vos avis. 102 INTERVIEW GUIDE – CHEF CISCO Interviewer : Merci d'avoir parlé avec moi aujourd'hui. Je voudrais commencer notre conversation par quelques questions générales. 1. Pouvez-vous vous présenter et m’expliquer un peu votre position ? 2. Depuis quand avez-vous travaillé dans cette zone ? Et depuis quand occupez-vous votre position actuelle ? 3. Comment trouvez-vous l’environnement de l’éducation dans cette zone ? 4. Selon vous, quels sont les plus grands problèmes de l’éducation dans cette zone ? a. Accès à l’éducation ? b. Problème d’infrastructures ? c. Problème d’enseignants qualifiés ? d. Motivation des élèves ? e. Motivation des élèves ? 5. Concernant les enseignants, ont-ils eu des formations préalables pour pouvoir enseigner ? Expliquez Et quels sont les problèmes que vous rencontrez en réalisant votre travail ? 6. De quelle manière la présence est-elle enregistrée au niveau de l’école ? 7. Qui est responsable de la tenue fiche de présence ? (Ex : Enseignants, directeur) ? 8. Comment pouvez-vous décrire les registres de fréquentation scolaire ? 9. S'il y a des lacunes dans la tenue des registres, que peut-on faire pour combler ces lacunes ? 10. Est-ce que vous jouez un rôle dans le suivi des données sur la fréquentation scolaire ? Explique-s’il vous plaît. 11. Y a-t-il déjà eu des lacunes (ou des trous) dans les données ? Si oui, comment ont-elles été résolues ? 12. Quelles sont les barrières qui empêchent l’inscription à l’école des enfants non scolarisés dans votre district ? 13. Connaissez-vous un programme ou projet sur le secteur de l’éducation qui s’est déjà déroulé dans cette zone ? 14. Si oui, lequel ? Quand ? Expliquer un peu le projet ? Quels seraient vos souhaits pour améliorer l’éducation des enfants dans votre établissement ? Avez-vous autres choses à me dire ? Avez-vous des questions à me poser ? Remerciement 103 FOCUS GROUP GUIDE— Parents of Students Introduction ! Présentation succincte d’ATW et des animateurs. ! Présentation du thème de la discussion et des objectifs ! Explication des règles de la discussion - Discussion libre, le plus spontanément et le plus sincèrement possible - Pas de censure (pas de faux ni vrai) - Enregistrement pour pouvoir faire les analyses - Se mettre à l’aise - Confidentialité de réponses données ! Tour de table présentation THEME I : EDUCATION 1. Education des enfants Comment trouvez-vous l’éducation des enfants dans votre village ? A quel âge vos enfants ont-ils commencé l’école ? Pourquoi ? Vos enfants sont-ils motivés à aller à l’école et étudier ? Quels sont leurs ambitions, leurs rêves ? Et comment trouvez-vous les résultats ? Etes-vous satisfaits ? A quelle fréquence vos enfants s’absentent-ils en classe ? Quelles sont les principales raisons ? Quels sont les obstacles à l'éducation de vos enfants ? Votre implication peut-elle aider à surmonter ces obstacles ? Si oui, quelles sont les différentes manières dont vous pouvez contribuer à surmonter ces obstacles ? Comment le programme peut-il vous aider à impliquer dans l'éducation de votre enfant ? 2. Problèmes liés à l’éducation Quels sont les grands problèmes liés à l’éducation que vous constatez dans votre village ? Quels sont les problèmes liés à l’accessibilité des enfants aux écoles ? Qu’en est-il des infrastructures publiques ? Est-ce que vous avez des enfants qui ne fréquentent pas l’école ? Quelles en sont les raisons ? Et est-ce qu’un de vos enfants a déjà abandonné l’école ? Quelles en sont les raisons ? Pour vous, existe-t-il une liaison entre la nutrition et l’éducation ? Pour vous, dans quelle mesure le manque de repas scolaires pour les élèves affecte-t-il les performances de votre enfant à l'école ? Voyez-vous un plus grand intérêt exprimé par votre enfant à aller à l'école en raison du programme d'alimentation scolaire ? Est-ce la même chose pour les garçons et les filles ? 3. Programme d’aide et souhaits Pouvez-vous me décrire quels sont les principaux besoins que vous ressentez pour améliorer l’éducation de vos enfants ? Quel genre de programme pensez-vous plus adaptés pour aider les enfants à réussir et à aller loin dans leurs études ? Quels seraient vos souhaits pour améliorer l’éducation de vos enfants ? Quelles sont les principales raisons pour lesquelles les enfants peuvent être absents de l'école ? Si à cause d'une maladie, quel type de maladies ? 104 La faim à la maison ou à l'école est-elle une de ces raisons ? S'il vous plaît, expliquez Les raisons des absences sont-elles les mêmes pour les filles et les garçons, ou y a-t-il des différences ? Qu'en est-il pour les filles et pendant les règles ? Y a-t-il des moments dans l'année où les parents gardent leurs enfants à la maison ? [Responsabilités ménagères, le travail, la pandémie mondiale] THEME II : EAU, ASSAINISSEMENT ET HYGIENE 1. Lavage des mains Selon vous, est-ce important de se laver les mains ? Pourquoi ? Comment vous lavez-vous les mains ? Qu’utilisez-vous ? Pourquoi ? Si vous n’utilisez pas de savon, pourquoi ? A quels moments vous l’avez-vous les mains ? Vous arrive-t-il d’en oublier de vous l’avez les mains sur un de ces moments ? Quelles seraient les raisons ? Avez-vous déjà entendu une sensibilisation sur le lavage des mains ? Comment avez-vous perçu cela ? Qu’avez-vous retenu ? Quels messages faites-vous passer à vos enfants concernant le lavage des mains ? Est-ce que vous faites un suivi de vos enfants ? 2. Latrines Est-ce que vous disposez d’une latrine à la maison ? Chez un voisin ? Si non, où est-ce que vous vous déféquez ? THEME III : SANTE 1. Accessibilité aux soins Est-ce que vous avez déjà fréquenté les CSB ? Si oui à quelles occasions ? Comment trouvez-vous les services offerts dans les CSB ? Quels seraient vos souhaits ? Si non, pourquoi n’avez-vous jamais fréquenté les CSB ? Quelles seraient les principales barrières qui vous empêchent d’aller au centre de santé ? Accessibilité ? Prix ? Services ? Quels sont les maladies les plus courantes qui touchent les enfants dans votre village ? Que faites-vous comme précaution ? Avez-vous autres choses à me dire ? Avez-vous des questions à me poser ? Remerciement 105 FOCUS GROUP GUIDE – Parents of Children Not Enrolelled in School Introduction ! Présentation succincte d’ATW et des animateurs. ! Présentation du thème de la discussion et des objectifs ! Explication des règles de la discussion - Discussion libre, le plus spontanément et le plus sincèrement possible - Pas de censure (pas de faux ni vrai) - Enregistrement pour pouvoir faire les analyses - Se mettre à l’aise - Confidentialité de réponses données ! Tour de table présentation THEME I : EDUCATION 1. Education des enfants Comment trouvez-vous l’éducation des enfants dans votre village ? A quel âge vos enfants ont-ils commencé l’école ? Pourquoi ? Nous savons que vos enfants ne vont pas à l’école, mais pour vous, trouvez-vous qu’ils motivés à aller à l’école et étudier ? Quels sont leurs ambitions, leurs rêves ? Et vous, avez-vous le souhait ou l’ambition de les inscrire à l’école ? 2. Problèmes liés à l’éducation Quels sont les grands problèmes liés à l’éducation que vous constatez dans votre village ? Quels sont les problèmes liés à l’accessibilité des enfants aux écoles ? Qu’en est-il des infrastructures publiques ? Quelles sont les principales raisons qui ont fait que vos enfants ne sont pas inscrits à l’école ? 3. Avis sur le programme de cantine scolaire Pouvez-vous me décrire quels sont les principaux besoins que vous ressentez pour améliorer l’éducation de vos enfants ? Quel genre de programme pensez-vous plus adaptés pour aider les enfants à réussir et à aller loin dans leurs études ? Si un programme d’alimentation scolaire s’implante sur votre commune, pensez-vous que ça serait une bonne ou une mauvaise chose pour l’éducation des enfants ? Est-ce que l’existence d’un programme pourrait vous inciter à faire inscrire vos enfants à l’école ? Pour vous, existe-t-il une liaison entre la nutrition et l’éducation ? Voyez-vous un plus grand intérêt exprimé par votre enfant à aller à l'école en raison du programme d'alimentation scolaire ? Est-ce la même chose pour les garçons et les filles ? THEME II : EAU, ASSAINISSEMENT ET HYGIENE 1. Lavage des mains Selon vous, est-ce important de se laver les mains ? Pourquoi ? Comment vous lavez-vous les mains ? Qu’utilisez-vous ? Pourquoi ? Si vous n’utilisez pas de savon, pourquoi ? A quels moments vous l’avez-vous les mains ? Vous arrive-t-il d’en oublier de vous l’avez les mains sur un de ces moments ? Quelles seraient les raisons ? 106 Avez-vous déjà entendu une sensibilisation sur le lavage des mains ? Comment avez-vous perçu cela ? Qu’avez-vous retenu ? Quels messages faites-vous passer à vos enfants concernant le lavage des mains ? Est-ce que vous faites un suivi de vos enfants ? 2. Latrines Est-ce que vous disposez d’une latrine à la maison ? Chez un voisin ? Si non, où est-ce que vous vous déféquez ? THEME III : SANTE 1. Accessibilité aux soins Est-ce que vous avez déjà fréquenté les CSB ? Si oui à quelles occasions ? Comment trouvez-vous les services offerts dans les CSB ? Quels seraient vos souhaits ? Si non, pourquoi n’avez-vous jamais fréquenté les CSB ? Quelles seraient les principales barrières qui vous empêchent d’aller au centre de santé ? Accessibilité ? Prix ? Services ? Quels sont les maladies les plus courantes qui touchent les enfants dans votre village ? Que faites-vous comme précaution ? Avez-vous autres choses à me dire ? Avez-vous des questions à me poser ? Remerciement 107 FOCUS GROUP GUIDE— Teachers Introduction ! Présentation succincte d’ATW et des animateurs. ! Présentation du thème de la discussion et des objectifs ! Explication des règles de la discussion - Discussion libre, le plus spontanément et le plus sincèrement possible - Pas de censure (pas de faux ni vrai) - Enregistrement pour pouvoir faire les analyses - Se mettre à l’aise - Confidentialité de réponses données ! Tour de table présentation 1. Environnement des enseignants Depuis combien d’année enseignez-vous ? Depuis quand enseignez-vous dans cette école ? Quelle classe enseignez-vous actuellement ? Comment êtes-vous parvenu à enseigner dans cette école ? Et comment vous sentez vous ? Etes-vous satisfait de votre situation actuelle ? Pourquoi ? Comment enregistrez-vous la présence dans votre classe ? A quelle fréquence ? Rencontrez-vous des difficultés dans le suivi des présences ? Expliquez s’il vous plaît Avez-vous déjà reçu des formations depuis que vous avez enseignez ? Si oui, quand et quelle formation ? De la part de qui ? Quel genre de programme ou de formation souhaiteriez-vous recevoir pour améliorer l’exercice de votre métier ? Qu’en est-il de votre relation avec vos supérieurs, sentez-vous supporter ou aider par votre chef ZAP ? Est-ce qu’il vient vous rendre visite ? A quelle fréquence ? Quels sont les plus grands problèmes que vous rencontrez en tant qu’enseignant ? 2. Education des enfants Comment trouvez-vous l’assiduité des enfants dans votre école ? Existe-t-il une différence entre les garçons et les filles ? Qui sont plus assidu ? Existe-t-il une différence entre les enfants les plus et les moins âgés ? Selon vous, quels sont les principaux facteurs qui font que les élèves ne sont pas assidus ? Comment trouvez-vous le résultat scolaire dans votre école ? Existe-t-il une différence entre les garçons et les filles ? Qui sont plus assidu ? Existe-t-il une différence entre les enfants les plus et les moins âgés ? Selon vous, quels sont les principaux facteurs qui peuvent améliorer les résultats des élèves en classe ? Comment trouvez-vous l’implication des parents dans l’éducation de leurs enfants ? Quels sont les principaux problèmes qui peuvent compromettre l’éducation des enfants dans votre école ? Si vous considérez les enfants qui fréquentent cette école, quels sont, selon vous, leurs besoins les plus urgents liés à leurs poursuites et à la réalisation de leur cursus scolaire ? Quelles sont les barrières qui empêchent l’inscription à l’école des enfants non scolarisés ? 3. Education et nutrition Comment trouvez-vous la situation nutritionnelle de vos élèves ? Vos élèves sont-ils inattentifs en classe en raison de manque de nourriture ? 108 Est-ce qu’il y une cantine scolaire dans votre école ? Si non, est-ce que cela avait déjà existé avant ? Qu’est-ce qui a fait que cette cantine ne fonctionne plus ? Avez-vous déjà pensé à une solution qui pourrait être apporter face à ce problème de nutrition des élèves ? Quels types d’aliments ces élèves ont-ils le plus besoin selon vous ? Pensez-vous que si ces enfants mangent bien, cela améliorerait leurs résultats en classe ? 4. Santé et hygiène Est-ce que vos élèves ratent souvent l’école en raison de maladie ? Quelles sont les maladies les plus fréquentes ? Selon vous, qu’est-ce qui fait que ces enfants attrapent souvent ces maladies ? Est-ce que vous faites des séances d’éducation ou de sensibilisation de vos élèves en ce qui concerne les bonnes pratiques d’hygiène ? Si oui, que faites-vous ? Les élèves retiennent-ils ? Si non, pourquoi ne faites-vous pas ? Quels seraient vos souhaits pour améliorer l’éducation des enfants dans votre établissement ? Avez-vous autres choses à me dire ? Avez-vous des questions à me poser ? Remerciement 109 Annex 6: Field report Training and pre-test: The training took place in Ambovombe from September 05 to 09. ATW was supported by the Technical Advisor, Alice Michelazzi, to conduct the training. It was attended by the M&E staff from Global Communities, 24 enumerators, and 12 supervisors who had been preselected to participate in the data collection. The training covered an introduction to the project and to data-collection tools and methods, practical application among trainees, field practice, debriefing and quiz sessions, constructive feedback, and logistics. The training also allowed data collectors to become familiar with two data-collection applications: Tangerine (for EGRA) and CSPro (for questionnaires). During the first two days, we trained all the field teams on the EGRA to be able to select the best afterwards. An assessment was carried out on the 3rd day to select those who will do the EGRA. From the afternoon of the 3rd day, the team was divided into two, one which will do the EGRA and the other which will do the surveys. The 5th day was devoted to a pre-test in 2 schools in Ambovombe followed by a debriefing session. The table below shows the details on the training agenda. Table 2: Training agenda Day 1 Monday 05 September Session Hourly ACTIVITIES Facilitators 1 8:00 - 8:30 Arrival and set up of participants 2 8:30 - 9:00 Official opening & Presentation 3 9:00 - 9:15 Organization of the training & Agenda Miranto 4 9:15 - 10:15 Presentation of the MIANATRA project Global Communities 10:15 - 10:30 Break 5 10:30 - 11:30 Evaluation Methodology & Sample Miranto 6 11:30 a.m. – 12:00 p.m. EGRA presentation Brief presentation of the EGRA instrument Alice 12:00 – 13:00 Lunch break 7 1:00 p.m. – 2:00 p.m. Principles of EGRA Administration Alice 8 2:00 p.m. – 3:00 p.m. Use of tablets Alice 9 3:00 p.m. – 4:00 p.m. Ownership of Scripts and Items - Introduction and Consent Alice 10 4:00 p.m. – 5:30 p.m. EGRA Subtask 1 – Letters Alice Day 2 Tuesday 06 September 1 8:00 - 8:30 Review day 1 and Program Day 2 Miranto & Alice 2 8:30 – 10:00 EGRA Subtask 2 – Syllables Alice 10:00 - 10:15 Break 110 3 10:15 – 11:00 EGRA Sub-Task 3 – Reading Familiar Words Alice 11:00 a.m. – 12:00 p.m. EGRA Sub-Task 4 – Silly Sentences 12:00 – 13:00 Lunch break 4 1:00 p.m. – 2:00 p.m. EGRA subtasks 5a and 5b – Reading and Comprehension of Text Alice 5 2:00 p.m. – 4:00 p.m. EGRA Practice – Sub-Tasks 1- 5 Alice 6 4:00 p.m. – 5:00 p.m. Closing and review of the day Miranto & Alice Day 3 Wednesday 07 September 1 8:00 - 8:30 Review day 2 and Program Day 3 Miranto & Alice 2 8:30 – 10:00 EGRA subtask 6 – Listening comprehension Alice 10:00 - 10:15 Break 3 10:15 – 11:30 AAM (Assessor Accuracy Measure) Miranto & Alice 4 11:30 a.m. – 12:00 p.m. Dispatch of teams, selection of those who will do the EGRA Miranto & Alice 12:00 – 13:00 Lunch break EGRA & OBSERVA TION TEAM 5 1:00 p.m. – 2:00 p.m. Questionnaire after EGRA Alice 6 2:00 p.m. – 3:00 p.m. Method for sampling students Alice 7 3:00 p.m. – 4:00 p.m. EGRA practice Alice 4:00 p.m. – 5:00 p.m. AAM (Assessor Accuracy Measure) Enumerators quiz Alice SURVEY TEAM 5 1:00 p.m. – 2:00 p.m. Student level surveys Miranto 6 2:00 p.m. – 3:00 p.m. Student questionnaire Miranto 7 3:00 p.m. – 5:00 p.m. Practice on student questionnaire Miranto Day 4 Thursday 08 September EGRA & OBSERVA TION TEAM 1 08:00 – 08:30 Review day 3 and Program Day 4 Alice 2 08:30 – 10:00 Class observation Alice 10:00 - 10:15 Break 3 10:15 – 12:00 Sampling method of students to observe Alice 12:00 – 13:00 Lunch break 4 1:00 p.m. – 4:00 p.m. Practice on classroom observation Alice 111 5 4:00 p.m. – 5:00 p.m. Organization of the teams and preparation of the pre-test Alice SURVEY TEAM 1 08:00 – 09:00 Parent-level surveys Miranto 2 09:00 – 10:00 Parent questionnaire Miranto 10:00 - 10:15 Break 3 10:15 – 12:00 Practice on parent surveys Miranto 12:00 – 13:00 Lunch break 4 1:00 p.m. – 2:00 p.m. School director Questionnaire and checklist Miranto 5 2:00 p.m. – 4:00 p.m. Qualitative surveys Miranto 6 4:00 p.m. – 5:00 p.m. Organization of the teams and preparation of the pre-test Miranto Day 5 Friday 09 September Pre-test at school level 2:00 p.m. – 4:00 p.m. Pre-test debriefing Miranto & Alice 4:00 p.m. – 5:00 p.m. Logistics and preparation for departure from the field Miranto & Alice 5:00 p.m. – 5:30 p.m. Closing of the training Miranto & Alice Data collection: The data collection in schools started on September 13 and continued through October 07, lasting four weeks. Organization of the team: one team made up of 4 supervisors and 8 enumerators were dedicated per district. Table 3: Team organization Function Supervisor 1 School director interview Focus group discussion Supervisor 2 Supervisor 3 Class observation Supervision of enumerators Supervisor 4 Enumerator 1 EGRA Enumerator 2 Enumerator 3 Enumerator 4 Student surveys Enumerator 5 112 Enumerator 6 Enumerator 7 Parent surveys Enumerator 8 Procedure to follow during the field: At the level of each school, below are the steps to follow to carry out collections: 1- Courtesy visit and request for consent from the school director: supervisors first reported to the school director with the mission order and explain to him the reason of the visit. Then, the supervisors requested consent from the school director so that the team can collect information from his school. 2- Realization of collections: After consent of the Director, the team proceeded to the data collection. Challenges encountered during the fieldwork: Table 4: Challenges encountered and solutions provided during the fieldwork Challenges Solutions Low number of students: The number of students we met in the schools was very low and we could not find the desired sample. In discussion with the Global Communities team and the technician, it was decided to add additional schools. Accessibility: The schools are located in remote places; access is very difficult and some schools have also been difficult to find. The team was transported by 4x4/all-terrain vehicles and for some schools, we recruited a local guide to direct us to the schools Contact with the school director the day before: The team must be in contact with the school director the day before the visit and some were difficult to reach because of the network. When the school director was unreachable by call, a supervisor had to visit the school the day before. Dialects: With the participants in very remote places, it was sometimes difficult to understand each other with the different dialects. Team members who are natives helped others to understand each other. 113 Annex 7: Data cleaning notes Data cleaning is divided into 7 steps: 1st step: Importing the csdb data from cspro to Excel and then STATA 2nd step: Control of the data and the other answers retrieved during the survey if there are errors or duplicates (to be deleted and corrected), then deletion of the data not consented to by the respondent 3rd step: Recoding of the other responses retrieved during the survey 4th step: Data consistency check by the statistician 5th step: Label the variables and the modalities of the variables in STATA 6th step: Transformation of text variables into numeric variables for data processing 7th step: Deletion of variables not useful for processing and establishment of a processing database to be communicated to the study manager 114 Annex 8: Data analysis plan Quantitative Data Analysis Plan Step 1: Data cleaning and survey weighting • Modification of data entry error based on information collected during data collection • Identification of invalid data • Computation of survey weights based on school enrollment and attendance in classroom • Implementation of survey weights in different datasets Step 2: Computation of EGRA scores and indicators • Computation of total correct, percent correct, zero-scores and fluency scores for each subtask • Computation of reading benchmarks • Verification of extremes fluency scores • Computation of indicators scores using weights to produce table of indicators values at baseline Step 3: Descriptive analysis of all datasets (using survey weights for all analysis) • Average total scores, percent correct, fluency scores. Proportion of zero-scores for each EGRA subtask. Proportion of students who read at benchmark. • Disaggregation of EGRA descriptive statistics by group (Treatment/Control) and gender. • Percentage for each indicator and disaggregation by group (Treatment/Control) and gender when possible. • Descriptive analysis of the classroom observation, disaggregation by group and gender. • Descriptive analysis (Average or percentage) of all items in the student questionnaire, disaggregation by group and gender. • Descriptive analysis (Average or percentage) of all items in the parent questionnaire, disaggregation by group. Step 4: Comparison analysis for equivalence between treatment and control group (using survey weights for all analysis) • Inferential analysis to compare treatment and control groups on EGRA. T-test or linear regression for total/percent correct and fluency scores and chi-square or logistic regression for percentage of zero-scores. • Inferential analysis to compare treatment and control groups on indicators values. Chi-square or logistic regression since all indicators represent percentage. • Inferential analysis to compare treatment and control on contextual factors 115 Qualitative Data Analysis Plan Step 1: Data cleaning • Cleaning based on any information received from quality control during data collection Step 2: Reading and developing coding • Transcription of audio recorded • First reading of partial materials to develop coding scheme Step 3: Coding of all the qualitative material • Applying coding to all qualitative material Step 4: Producing results from coding 116 Annex 9: Statistical weighting & adjustment factors The study is weighted in two levels: • The first level is based on the number of schools per district and the number of schools sampled in this district, i.e. the probability that a school is selected per district. 𝑃!"# = 𝑛!"# 𝑁!"#/%&'()&"( Of which " 𝑃!"# : The probability that a school will be selected in a district " 𝑛!"# : Number of schools selected in the district " 𝑁!"#/%&'()&"( : Number of schools in the district So the first level weight is the inverse of the probability of a school being selected in a district, which is: 𝑊!"# = * +!"# = ,!"#/%&'()&"( -!"# • The second level is based on the number of girls and boys in a class for each school and the number of girls and boys sampled in a class for each school, i.e., the probability that a girl or boy is selected in a class for each school is: 𝑃./"# = 𝑛0* !"#+ 𝑁.' !"#+ 𝑃1/"# = 𝑛2* !"#+ 𝑁1' !"#+ " 𝑛0* !"#+ : is the number of sampled girls of class "s" in the school ECLi " 𝑁.' !"#+ : is the number of girls in class "s" in the school ECLi " 𝑛2* !"#+ : is the number of boys sampled from class "s" in the school ECLi " 𝑁1' !"#+ : is the number of boys in class "s" in the school ECLi Thus, the weighting of the second level is based on the inverse of the probability of the number of sampled girls or boys of class "s" in the ECL schooli , in this case the weighting is o 𝑊' 3 , -./"# 3 𝑁.' !"#+ 𝑛0* !"#+ ' If girl o 𝑊' 3 , -0/"# 3 𝑁1' !"#+ 𝑛2* !"#+ ' If boy The final weight is equal to the weight of the first level multiplied by the weight of the second level 𝑊.3 𝑊!"# ∗ 𝑊' 117 Annex 10: Additional tables and figures Parents’ perception on child education: Table 10.1: Average number of children per parent vs average number of children enrolled per parent by control/project PROJECT CONTROL Average number of girls 3.1 2.6 Average number of girls enrolled in school 2.4 1.7 Average number of boys 2.7 2.7 Average number of boys enrolled in school 1.9 1.6 Figure 10.1: Percent of parents/guardians reporting each reason for their kids to stop school by control/project 0.3% 0.3% 0.3% 0.8% 0.2% 0.6% 0.3% 1.3% 6.1% 37.5% 0.1% 0.3% 0.7% 1.2% 1.3% 2.0% 2.6% 4.1% 13.2% 22.3% Orphan Pregnancy Health reason There is no more place for him at school Get married He is already old enough and ashamed to go to school He has finished primary and there is no school for the second cycle The child got a job The child no longer wanted Lack of funds Project Control 118 Figure 10.2: Percent of parents who help their children with their studies at home Figure 10.3: Percent of parents reporting each type of help 42% 24% Project Control 26% 28% 20% 7% 0% 13% 13% 7% 6% 0% Reading Maths Learning the lessons Doing homework Writing Project Control 119 Menstruation for girls Table 10.2: Summary statistics on girls in T4 PROJECT (n=281) CONTROL (n=88) Average age of girls in T4 15 13 Percent of girls who have already heard about menstruation 36% 44% Percent of girls who have already menstruated 21% 13% Table 10.3: Knowledge about menstruation by girls who have heard about it PROJECT (n=101) CONTROL (n=39) Menstruation is a normal biological process: - YES - NO - Do not know 93% 3% 4% 95% 0% 5% Menstruation is a sign of puberty: - YES - NO - Do not know 76% 14% 10% 80% 10% 10% A girl is dirty when she is on her period: - YES - NO - Do not know 71% 28% 1% 79% 15% 6% Table 10.4: Girls’ practices around menstruation, by girls who have already menstruated1 PROJECT (n=58) CONTROL (n=11) Percent of girls who have already missed school because of menstruation 26% 9% Average number of school days missed last time 3 days 3 days 1 Findings should be interpreted with caution due to the low number of respondents