McGovern-Dole International Food for Education and Child Nutrition III Project in Mali Midline Evaluation Report July 2018 Authors: Elnaz Safarha, M.S. Kajal Gulati, Ph.D. Marc Masson, B.S. Maria DiFuccia, M.A. Sakari Deichsel, M.A. Edoxi Kindané, M.S. Submitted to: Chief of Party: Eliane Kouton Catholic Relief Services – Mali Submitted by: Project Director: Elnaz Safarha IMPAQ International, LLC &zϮϬϭϱDĐ'ŽǀĞƌŶͲŽůĞ/ŶƚĞƌŶĂƚŝŽŶĂů&ŽŽĚĨŽƌĚƵĐĂƚŝŽŶ ĂŶĚŚŝůĚEƵƚƌŝƚŝŽŶ///WƌŽũĞĐƚŝŶDĂůŝ DŝĚůŝŶĞǀĂůƵĂƚŝŽŶReport Program: McGovern-Dole International Food for Education and Child Nutrition Agreement Number:FFE-688-2015/014-00 Funding Year: Fiscal Year 2015 Project Duration: 2015-2021 Implemented by: CRS DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov IMPAQ International, LLC Page i Mali MGD Midline Report July 2018 ACKNOWLEDGMENTS The authors would like to thank Catholic Relief Services (CRS) and the US Department of Agriculture (USDA) for their financial support. The authors also would like to thank the Ministry of Education of Mali and CRS staff, especially Roseline Kany Sidibe, for supporting and facilitating the rollout of the midline data collection. We have very much enjoyed our collaboration with CRS and would like to thank Eliane Kouton, Sylvain Guindo, Sekou Kayentao, and Amadou Isaac Yolo from CRS. Special thanks go to Almougaraita Maiga from EDC for his technical insights and support on training and pilot testing. We also thank Bado Cissoko and Amadou Cisse for their oversight and support in the field work. The authors thank all the enumerators and supervisors for their excellent work in the field. The findings, interpretations, and conclusions expressed herein are entirely those of the authors. They do not necessarily represent the views of IMPAQ, CRS, or USDA. IMPAQ International, LLC Page ii Mali MGD Midline Report July 2018 TABLE OF CONTENTS ACKNOWLEDGMENTS ........................................................................................................................ I TABLE OF CONTENTS......................................................................................................................... II TABLE OF EXHIBITS............................................................................................................................ V ACRONYM LIST ............................................................................................................................... VII EXECUTIVE SUMMARY ................................................................................................................... VIII Overview....................................................................................................................................viii Key Evaluation Outcomes..............................................................................................................ix Recommendations for the Project................................................................................................xii SECTION 1. INTRODUCTION................................................................................................................1 1.1 Program Background................................................................................................................1 1.2 Evaluation Background.............................................................................................................2 SECTION 2. EVALUATION METHODOLOGY ..........................................................................................4 2.1 Evaluation Questions................................................................................................................4 2.2 Evaluation................................................................................................................................6 2.2.1 Quantitative Evaluation Design................................................................................................... 6 2.2.2 Qualitative Evaluation Design ...................................................................................................10 2.3 Sampling Strategy Design .......................................................................................................11 2.3.1 Quantitative Sampling...............................................................................................................11 2.3.2 Qualitative Sampling .................................................................................................................13 2.4 Data Sources..........................................................................................................................15 2.4.1 Surveys ......................................................................................................................................15 2.4.2 Reading Assessment..................................................................................................................15 2.4.3 School Attendance Data............................................................................................................16 2.4.4 Key Informant Interview and Focus Group Discussion Data.....................................................16 SECTION 3. FIELDWORK....................................................................................................................17 SECTION 4. EVALUATION SAMPLES...................................................................................................19 4.1 Schools ..................................................................................................................................19 4.2 Students ................................................................................................................................21 4.3 Parents ..................................................................................................................................22 4.4 Household Environment.........................................................................................................23 4.5 Teachers ................................................................................................................................28 4.6 School Principals ....................................................................................................................30 IMPAQ International, LLC Page iii Mali MGD Midline Report July 2018 4.7 School Management Committees (SMCs)................................................................................31 SECTION 5. QUANTITATIVE OUTCOMES ............................................................................................33 5.1 Performance Evaluation .........................................................................................................33 5.1.1 School Outcomes.......................................................................................................................34 5.1.2 Student Outcomes.....................................................................................................................38 5.1.3 Parent Outcomes.......................................................................................................................46 5.1.4 Teacher Outcomes ....................................................................................................................56 5.1.5 School Principal Outcomes........................................................................................................61 5.1.6 SMC Outcomes..........................................................................................................................62 5.2 Impact Evaluation ..................................................................................................................66 SECTION 6. QUALITATIVE FINDINGS..................................................................................................70 6.1 Relevance ..............................................................................................................................71 6.1.1 Alignment with Community and Government Priorities ..........................................................71 6.1.2 Adjustments to Project Design..................................................................................................72 6.2 Effectiveness..........................................................................................................................72 6.2.1 Strategic Objective 1: Improved Overall Literacy of School-Aged Children..............................72 6.2.2 Strategic Objective 2: Increased Use of Health and Dietary Practices in Program Schools......74 6.3 Efficiency ...............................................................................................................................75 6.3.1 Use of Project Resources...........................................................................................................75 6.3.2 Achievement of Project Timeline..............................................................................................75 6.4 Perceived Impact....................................................................................................................76 6.4.1 Addressing Barriers to Education..............................................................................................76 6.4.2 Activities with the Greatest and Least Impact ..........................................................................76 6.4.3 Suggestions for Increasing Impact ............................................................................................77 6.5 Sustainability .........................................................................................................................77 6.5.1 Greatest Likelihood of Sustainability.........................................................................................77 6.5.2 Challenges to Sustainability.......................................................................................................78 SECTION 7. IMPLICATIONS FOR THEORY OF CHANGE AND PROJECT OPERATIONS..............................79 SECTION 8. CONCLUSION ..................................................................................................................81 8.1 Key Findings...........................................................................................................................81 8.1.1 Students’ Outcomes..................................................................................................................81 8.1.2 Parents Outcomes.....................................................................................................................82 8.1.3 Teachers and Principals’ Outcomes ..........................................................................................82 IMPAQ International, LLC Page iv Mali MGD Midline Report July 2018 8.1.4 SMCs’ Outcomes .......................................................................................................................83 8.1.5 Impact Outcomes......................................................................................................................83 8.1.6 Qualitative Findings...................................................................................................................83 8.2 Limitations.............................................................................................................................85 8.3 Recommendations .................................................................................................................86 REFERENCES ....................................................................................................................................88 APPENDICES ....................................................................................................................................89 APPENDIX A. MCGOVERN-DOLE ACTIVITIES AND RESULTS FRAMEWORKS ....................................... A-1 APPENDIX B: MCGOVERN-DOLE PROJECT INDICATORS.................................................................... B-1 APPENDIX C. ADDITIONAL TABLES AND COMPLEMENTARY OUTCOMES .......................................... C-1 APPENDIX D. EVALUATION INSTRUMENTS...................................................................................... D-1 IMPAQ International, LLC Page v Mali MGD Midline Report July 2018 TABLE OF EXHIBITS Exhibit 1: Map of Targeted Region in Mali .................................................................................................................... 1 Exhibit 2: Key Evaluation Questions.............................................................................................................................. 4 Exhibit 3: Cohort Comparison Approach to Project Evaluation Strategy ...................................................................... 7 Exhibit 4: Example of Calculations: Two-Year Program Effects..................................................................................... 9 Exhibit 5: Sampling Strategy for Midline Evaluation ...................................................................................................12 Exhibit 6: Qualitative Sampling Strategy .....................................................................................................................13 Exhibit 7: Participants in the Qualitative Focus Groups ..............................................................................................14 Exhibit 8: Participants in the Key Informant Interviews, Koulikoro.............................................................................14 Exhibit 9: Participants in the Key Informant Interviews, Mopti ..................................................................................14 Exhibit 10. ASER Reading Test Structure .....................................................................................................................15 Exhibit 11: Sample Distribution by Region and Type of Respondent ..........................................................................19 Exhibit 12: Handwashing Observation.........................................................................................................................20 Exhibit 13: Food Storage Observation .........................................................................................................................20 Exhibit 14: Canteen Observation................................................................................................................................. 21 Exhibit 15: Student Sample Composition ....................................................................................................................21 Exhibit 16: Parents Composition by Gender and Students’ Grade..............................................................................22 Exhibit 17: Caregiver Sample Composition..................................................................................................................23 Exhibit 18: Caregivers’ Educational Attainment by Region .........................................................................................24 Exhibit 19: Household Characteristics.........................................................................................................................24 Exhibit 20: Reasons for Child Not Attending School....................................................................................................25 Exhibit 21: Reasons for Child Dropping Out of School................................................................................................. 26 Exhibit 22: Reasons for Child Missing School Last Week.............................................................................................26 Exhibit 23: Households’ Access to Basic Services........................................................................................................27 Exhibit 24: Availability of Books and Reading Habits in Households...........................................................................28 Exhibit 25: Teachers’ Characteristics...........................................................................................................................29 Exhibit 26: Teachers’ Educational Attainment.............................................................................................................29 Exhibit 27: Teachers’ Employment Status...................................................................................................................30 Exhibit 28: Principals’ Characteristics..........................................................................................................................31 Exhibit 29: Principals’ Educational Attainment............................................................................................................31 Exhibit 30: SMC Characteristics...................................................................................................................................32 Exhibit 31: SMCs’ Educational Attainment by Region ................................................................................................. 32 Exhibit 32: McGovern-Dole Key Indicators..................................................................................................................33 Exhibit 33: Frequency of Schools with Key Infrastructures and Resources................................................................. 35 Exhibit 34: Frequency of Schools with Key Infrastructures and Resources................................................................. 36 Exhibit 35: Number of Non-Operational Days at the Canteen in March 2018 ............................................................37 Exhibit 36. Notice Board and Colored Report Cards...................................................................................................38 Exhibit 37: Sickness and Related Health Absences......................................................................................................39 Exhibit 38: Average number of days missed per student because of student health issues.......................................39 Exhibit 39: Knowledge and Self-reported Handwashing Practices at Critical Moments.............................................40 Exhibit 40. Knowledge on Intestinal Worms................................................................................................................41 Exhibit 41: Students’ Dietary Diversity ........................................................................................................................42 Exhibit 42. Minimum Acceptable Diet.........................................................................................................................43 Exhibit 43: Average Number of Enrolled Students who Regularly Attending in 2017–2018.......................................43 Exhibit 44: Percent of Students Regularly Attending (2016–2018) by Region ............................................................44 Exhibit 45: Distribution of Reading Skills by Grade Level (Percentage of Students) ...................................................45 Exhibit 46: Reading Proficiency at Grade Level by Grade and Gender........................................................................46 Exhibit 47: Food Security Status..................................................................................................................................47 Exhibit 48: Household Members Experiencing Reduced/Cut Meals during Food Insecurity ......................................48 Exhibit 49: Caregivers’ Health and Hygiene Knowledge..............................................................................................49 Exhibit 50: Caregivers’ Involvement in Preventive Activities by Region......................................................................49 IMPAQ International, LLC Page vi Mali MGD Midline Report July 2018 Exhibit 51: Caregivers’ Involvement in School Activities.............................................................................................51 Exhibit 52: General Assemblies Attended by Parents since the Start of the Previous School Year.............................51 Exhibit 53: Caregivers’ Aspirations for their Children by Children’s Gender...............................................................52 Exhibit 54: Caregivers’ Aspirations for their Children by Children’s gender................................................................52 Exhibit 55: Caregivers’ Reasons for Supporting Girls’ Education.................................................................................53 Exhibit 56: School Decision-Making Matrix in Koulikoro.............................................................................................54 Exhibit 57: School Decision-Making Matrix in Mopti...................................................................................................54 Exhibit 58: Changes in Households’ Livelihood due to Participation in SILC ...............................................................55 Exhibit 59: Different Types of Pre-Service Trainings....................................................................................................57 Exhibit 60: Teachers Who Received In-Service Trainings in Literacy and Pedagogy ...................................................57 Exhibit 61: Proportion of Teachers Trained in BLA by Grade and Region ...................................................................58 Exhibit 62: Timing of Training in BLA ...........................................................................................................................58 Exhibit 63: Proportion of Teachers Using Any of the BLA Techniques ........................................................................59 Exhibit 64: Teacher Use of BLA Equipment during Previous Month at Midline ..........................................................60 Exhibit 65: Challenges Faced by Principals in Supporting their Teachers....................................................................62 Exhibit 66: Principals’ Health and Hygiene Knowledge ...............................................................................................62 Exhibit 67: Different Topics in which SMCs Received Training....................................................................................63 Exhibit 68: Best Practices for Food Storage Reported by SMC Members ...................................................................64 Exhibit 69: SMC Members Passing Best Practices of Safe Food Storage.....................................................................64 Exhibit 70: SMC Members Passing on Best Practices of Safe Food Preparation .........................................................65 Exhibit 71: General Assemblies Organized by SMC Members for Parents..................................................................66 Exhibit 72: Program Effects Using Cohort Comparison Method .................................................................................68 Exhibit 73: Two-year Program Effect on Litarcy Outcomes.........................................................................................69 Exhibit 74: What Students Liked About Their Classroom and School......................................................................... C1 Exhibit 75: What Students Did Not Like About Their Classroom and School ............................................................. C1 Exhibit 76: Number of School Days Missed due to Students’ Illness in the Last Two Weeks..................................... C2 Exhibit 77: Type of Illness, If Student Missed School in the Last Two Weeks Due to Illness...................................... C2 Exhibit 78: Students Enrollment (2016-2018) ............................................................................................................ C3 Exhibit 79: Average Number of Students Enrolled (2016-2018) ................................................................................ C3 Exhibit 80: Percent of Students Regularly Attending (2016-2018)............................................................................. C4 Exhibit 81: Students Food Intake................................................................................................................................ C5 Exhibit 82: Reading Proficiency at Grade Level by Region.......................................................................................... C6 Exhibit 83: School Support Activities That Caregivers Participated In........................................................................ C6 Exhibit 84: Types of Academic Support Caregivers Provided to Students.................................................................. C7 Exhibit 85: Percentage of Teachers Trained in All Eight BLA Techniques at Midline.................................................. C7 Exhibit 86: Percentage of Grade 1 Teachers Trained in BLA Techniques ................................................................... C8 Exhibit 87: Percentage of Grade 2 Teachers Trained in BLA Techniques ................................................................... C8 Exhibit 88: Percentage of Grade 3 Teachers Trained in BLA Techniques ................................................................... C9 Exhibit 89: Percentage of Grade 4 Teachers Trained in BLA Techniques ................................................................... C9 Exhibit 90: Average Number of Techniques Used Among Those Trained In BLA ..................................................... C10 Exhibit 91: Percentage of Grade 1 Teachers Using BLA Techniques ........................................................................ C10 Exhibit 92: Percentage of Grade 2 Teachers Using BLA Techniques ........................................................................ C11 Exhibit 93: Percentage of Grade 3 Teachers Using BLA Techniques ........................................................................ C11 Exhibit 94: Percentage of Grade 4 Teachers Using BLA Techniques ........................................................................ C12 Exhibit 95: Percentage of Teachers Reporting Student Appreciation Of BLA Technique......................................... C12 Exhibit 96: Parents’ Wood Contributions to the Canteen, Reported by SMC Members.......................................... C13 Exhibit 97: Parents’ Food/Condiments Contribution to the Canteen, Reported by SMC Members........................ C14 IMPAQ International, LLC Page vii Mali MGD Midline Report July 2018 ACRONYM LIST ANPECTP Government Agency in Charge of Early Childhood Development APE ATE l’Association des Parents d’Elèves (Parent Association) Average Treatment Effect EDC Education Development Center EGRA Early Grade Reading Assessment ASER Annual Status of Education Report BLA Balanced Literacy Approach CGE Comité de Gestion d’Ecole (School Management Commitee) CRS Catholic Relief Services DAP ECOM EDC EGRA Development activity proposal Ecole Communautaire Education Development Center Early Grade Reading Assessment FAS Foreign Agricultural Service FGD Focus group discussion IDEN IFM IR ITE KII Departmental Inspector of National Education Institut de Formation des Maîtres Interactive Radio Technique Indirect treatment effect Key informant interview M&E Monitoring and evaluation MDE Minimum Detectable Effect MDMS Midday Meal Scheme MGD McGovern-Dole MONE Ministry of National Education PA PMP Parent associations Performance evaluation plan PTA SARPE SILC SMC SO TATE THR USDA Parent–teacher association Strategy Alternative de Recruitment du Personnel Enseignant Savings and internal lending community School management committee Strategic objective Total average treatment effect Take-home rations United States Department of Agriculture WASH Water, Sanitation, and Hygiene IMPAQ International, LLC Page viii Mali MGD Midline Report July 2018 EXECUTIVE SUMMARY Overview The United States Department of Agriculture (USDA) has funded Catholic Relief Services (CRS) for three project phasesthrough the through the McGovern-Dole (MGD) International Food for Education and Child Nutrition Program. The third phase, which is a five-year intervention (FY2016–FY2020), with a budget of $29,899,945, extends and expands on the previously implemented McGovern-Dole projects (MGD projects (I and II). The project activities in this third phase, which aims to improve literacy of school-aged children for 77,104 children in 264 primary schools in the Mopti and Koulikoro regions in Mali, include school meals, take-home rations (THR) and vitamin A and deworming medications distribution, school management committees (SMC) members capacity building, formation of savings and internal lending community (SILC) groups, expansion of illustrated report cards, and teachers as well as school administrators training on the balanced literacy approach (BLA). CRS selected IMPAQ International, LLC (IMPAQ) in 2015–2016 to design impact and performance evaluations of MGD III at baseline. The performance evaluation was designed to measure changes in outcomes over the life of the project with regards to USDA’s two strategic objectives (SO): (1) SO1 with a focus on students’ reading proficiency, attendance, participation in classroom activities, and teachers’ motivation, as well as parent/community engagement; and (2) SO2 with a focus on three key areas, including safe food preparation knowledge and storage practices, accessibility to preventive health interventions, and reducing health-related absences. Moreover, IMPAQ designed the impact evaluation to identify the causal effect of the BLA teacher training intervention on the literacy of primary-school￾aged children. At baseline, IMPAQ used quantitative and qualitative methods to establish baseline values for outcome indicators required by USDA, generated data to be used for comparative analysis, and validated project strategies and assumptions at baseline. In March 2018, CRS re-selected IMPAQ for the midline performance and impact evaluation of MGD III to assess progress in implementation, relevance of interventions, and appropriateness of strategies, management structures, and/or activities for improved project effectiveness, efficiency, impact, and sustainability. To evaluate the key outcome indicators and research questions, which outlined in detail in Section 2.1, as well as measure progress toward achieving them, IMPAQ collected survey data (students, the Annual Status of Education Report (ASER)1 , parents/caregivers, teachers/principals, and SMCs) using the same instruments used at baseline, as well as qualitative key informant interviews (KII), and focus group discussions (FGD) data. The purpose of this report is to assess the progress of MGD III in achieving the desired outcomes up to the midpoint of the project. In this report, we also discuss the impact of the two and three years of BLA intervention on literacy levels of both girls and boys in the targeted schools. The statistically significant gains in literacy associated with the program intervention is the main critical finding of this report. In addition, this report describes potential additional questions to be asked at endline evaluation and provides recommendations about the implementation of MGD III. 1 The Annual Status of Education Report (ASER) Center pioneered in 2005 a nationwide survey, composed of a reading and math test, to measure the achievements of children in primary school in rural India in reading and math. Since their inception, adapted versions of the tests have been implemented in Pakistan, Kenya, Mali, Senegal, Tanzania, Uganda, Ghana, and Burkina Faso (Pratham 2015; USAID 2012). IMPAQ International, LLC Page ix Mali MGD Midline Report July 2018 Key Evaluation Outcomes Below is a snapshot of the key outcomes for midline performance evaluation. More details are provided in Section 5 and Section 6. Key Performance Outcomes Students’ Outcomes We find an overall improvement in the reading ability of children across all grades, with no significant regional or gender differences. A significantly greater proportion of students in first grade (13 and 9 percentage points for girls and boys, respectively) could read simple sounds; second grade (12 and 14 percentage points for girls and boys, respectively) could decode simple words; third grade (20 and 23 percentage points for girls and boys, respectively) could read simple sentences, all statistically significant at the 1 percent level, with no big changes for girls and boys in fourth grade students. Moreover, the program interventions also seem to be related to children’s reported reasons for liking to attend school. Children in Koulikoro reported liking school and their classroom because of the useful skills they acquired and the ease of understanding lessons. In Mopti, children mentioned having access to food and the ease of understanding as their key reasons for liking to attend school. While the findings reveal an improvement in literacy and school attendance outcomes at midline, the progress on hygiene practices, which was the focus of MGD I and II activities, appears to be mixed, with wide variation across the two program sites. Progress on hygiene practices, such as washing hands, seems to be even slower. The average proportion of students who reported washing hands at critical moments is lower as compared to baseline. Moreover, the analysis also reveals that knowledge about handwashing practices, on average, did not change at midline as compared to baseline. While the actual reported washing of hands is lower possibly due to lack of proper infrastructure and facilities in the school, the lack of change in knowledge about handwashing practices is worth noting for strengthening program efforts. In the endline evaluation, we will also focus our efforts in understanding the reasons behind these changes in hygiene practices among students. Parents Outcomes At midline, we find an improved engagement of parents in their children’s education. Improved participation of parents for their children’s education is evidenced in several ways. First, parents’ awareness about the school notice boards increased and a higher proportion of parents received colored report cards. Moreover, the average number of caretakers who attended a school general assembly increased in Koulikoro but not in Mopti. Presumably, other concerns such as safety prevented caregivers from going to the school for general assemblies in Mopti. In contrast to the progress on improved engagement by parents for their children’s education, the analysis suggests a lack of change in the involvement of parents in their children’s preventative healthcare activities, such as providing iron and vitamin A supplementation. It is noteworthy that the data collection was conducted before the distribution of the mineral supplements, which is scheduled in November and May of each project year. Many parents cited that healthcare activities were too expensive or were not available or accessible. Lack of progress on these healthcare outcomes suggests that deeper, structural financial and geographical constraints might be contributing to a lack of progress on these outcomes. However, the project was not designed to focus on health care structures. MGD III aims to facilitate the distribution of the mineral supplements (e.g., vitamin A, iron, and deworming medications). The project IMPAQ International, LLC Page x Mali MGD Midline Report July 2018 also supports raising parents’ awareness about preventive health services by inviting local community health center staff in school to provide the communities with the appropriate health information At midline we also investigated decision-making within households and found differences in perceptions on decision-making for children’s education within the household. A majority of men reported themselves as being the primary decision-maker for children’s education. However, women suggested a relatively lower role of men in their children’s education. Women felt that they play a relatively higher role vis-à-vis men’s role in their children’s education – this is indicative of women exercising some say in their children’s education. However, both reports confirm that men tend to dominate the decision when it pertains to their children’s education. We also find no changes in parents’ aspirations for their children’s future between midline and baseline, suggesting the long period of time it takes to alter these deep-seated preferences of individuals. Teachers and Principals’ Outcomes Our survey results suggest that one of the key program interventions – providing BLA training to teachers – was administered as planned. Almost 100 percent of 1st to 3rd grade teachers in Koulikoro and 93.3 percent of teachers in Mopti received BLA teacher training. Moreover, over half the number of these teachers had received approximately 2-3 BLA trainings. These trainings also appear to have translated into actual application in the classroom, based on self-reported data by the teachers. Most teachers mentioned that they used at least one BLA technique in the classroom, and almost all teachers also reported using BLA equipment in the previous month in their classroom. Our results also suggest there were no key differences in handwashing practices by teachers between baseline and midline. However, this difference is being driven by the already high knowledge and reported practice of handwashing among teachers during both baseline and midline. SMCs’ Outcomes Improvements in the performance of SMCs were notable across several dimensions. First, almost all SMC members managed school canteens at midline. Many of these SMC members had also received training and many different kinds of trainings. Second, a higher proportion of SMCs monitored the practices of teachers, especially through classroom observations. Third, these SMCs also followed student progress through monitoring the colored bulletins. However, one of the key outcomes where our SMC outcomes showed mixed results were the deployment of school action plans. While in Koulikoro, there was a 20 percentage point increase in the SMCs that prepared a school action plan, there was a 12 percentage point reduction in Mopti. It is quite possible that political instability and strife in Mopti diminished the continuous operation of SMCs. As revealed in the interviews in Mopti with SMC members, illiteracy and challenges in filling forms for the plans was mentioned as one of the key constraining factors. Key Impact Outcomes Our cohort comparison quasi-experimental design shows a clear proof of program impact: children’s literacy outcomes improved by one level due to 2 years of exposure to a BLA-trained teacher and these outcomes increased by more than 1 level and a half due to 3 years of exposure to BLA-trained teachers. These findings are robust to adding other covariates. However, we do not find any evidence of spillover effects indicating that the program is successful in improving reading abilities of children due to intensive, prolonged exposure to more effective and well-trained teachers. IMPAQ International, LLC Page xiii Mali MGD Midline Report July 2018  Work on the school environment. Students cited the bad attitude of their teachers (e.g., hitting, yelling, harassing, and/or under-estimating children) as one of the most frequent reasons for what they do not like about their school and classroom. As recommended at baseline, the project should not only focus strengthening teachers’ pedagogical practices, but also on improving teachers’ attitudes and the school environment (explicitly addressing bullying) to create an atmosphere conducive to learning for students.  Target both male and female caregivers for improving outcomes such as children’s school attendance. The study also highlighted that men play a key role in decision-making on children’s educational attainment. Any long-run, sustained program impact would also work towards changing the attitudes of men towards the educational attainment of their children. Strategies specific to men can include sensitizing them as partners or allies, with common goals for the welfare of the family. Even in patriarchal or religious/Islamic communities, activities can focus on reinforcing the positive emotions of men as caregivers of their daughters. Since the program’s goal is to improve children’s health and educational outcomes, including both female and male caregivers will be critical for meeting the program’s objectives, especially improving the children’s school attendance and reducing their dropout rates.  Scale-up activities related to savings and internal lending. The SILC groups appear to offer the greatest path to sustainability after the project finishes, evidenced through CRS monitoring data and through the qualitative interviews with national and local stakeholders. The financial knowledge gained will last after the resources end, and several communities without SILC groups requested financial literacy training, knowing that this could help them maintain the canteens. Survey data also show that participation in the SILC groups helped parents in saving money for their children’s school fees as well as improving their livelihood, including their income, assets, and food security. Future iterations of the project many want to include livelihood activities to expand the benefits of SILC group participation beyond the project objectives.  Increase infrastructure-building to keep pace with awareness building, or temper program expectations. Awareness building appears to be happening faster than infrastructure building. For example, parents are aware of the importance of preventative health activities such as pre￾natal care and vaccinating their children, but the expense involved, or the distance required to access care is too great for many families. Another example is many canteen activities require water, but without access to a regular water source, these activities cannot be completed.  Include lessons on collaboration for teachers to increase the sustainability of BLA. Our impact evaluation shows that learning from BLA-trained teachers improves students’ literacy outcomes. The lack of spillover effects within the same school (that is, no improvement in literacy for students in the same school taught by non-trained teachers) is additional evidence that sustained exposure to a trained teacher is the mechanism for students’ improvement. However, the lack of increase for students in other classes also shows that teachers are not communicating or collaborating within their schools. Teachers are not sharing the techniques learned through BLA with other teachers. Because teacher turnover is so high, this threatens the sustainability of students’ gains in literacy. The program could add activities on school-wide collaboration, where teachers and principals share and practice what they’ve learned through BLA with the whole school.  Consider incentives for teachers burdened with additional work. Project staff reported that teachers are overwhelmed with extra managerial work related to the program (for example, inventory report, food delivery forms, etc.) In some schools, teachers are asked to help illiterate IMPAQ International, LLC Page xiv Mali MGD Midline Report July 2018 SMCs who are unable to complete school action forms and other required paperwork on their own. Project staff and/or partners should consider talking with teachers to get a better understanding of the extra burden involved and what could encourage adding extra duties. It may be that small grants or teacher recognition could help motivate those teachers who are reluctant to take on this extra work.  Put in place the revised strategy for sustainability, and include indicators to measure progress. Several activities will require external support to continue after the program ends. For example, in many communities, the amount of food families can contribute varies heavily from year to year based on climate. Teacher and SMC turnover also threaten sustainability, as without continuous training, new teachers and SMC members will not have the knowledge and skills to use BLA and run the canteens. CRS Mali’s revised strategy for sustainability recognizes that communities need additional government support, and therefore should advocate mayors’ offices (mandated to support school development under Malian decentralization law) to include financial support for school feeding and teacher training in their development action plans. Additional capacity building at the local level (such as mayor training) can support this. Monitoring progress towards the sustainability plan will help determine which strategies show promise and which need further revision. IMPAQ International, LLC Page 1 Mali MGD Midline Report July 2018 SECTION 1. INTRODUCTION The United States Department of Agriculture (USDA) has funded Catholic Relief Services (CRS) for three project phases through the McGovern-Dole (MGD) International Food for Education and Child Nutrition Program. The third phase, which is a five-year intervention (FY2016–FY2020) with a budget of $29,899,945, extends and expands on the previously implemented MGD projects (MGD I and II). This third phase aims to improve literacy of school-aged children for 77,104 children in 264 primary schools in the Mopti and Koulikoro regions in Mali. The purpose of this report is to assess the progress of MGD III in achieving the desired project outcomes. The report also describes potential additional questions to be asked at endline evaluation and provides recommendations about the implementation of MGD III. This report consists of seven sections. This introduction (Section 1) provides a brief overview of the program context for the MGD III evaluation. Section 2 outlines the mixed-methods evaluation approach, including research questions, sampling design and its modification, data tools, and data analysis. In Section 3, we describe field work for data collection. Section 4 describes the sample respondents and their key characteristics. Sections 5 and 6 present the quantitative and qualitative outcomes. Finally, Section 8 concludes the report with lessons learned, study limitations, and recommendations. 1.1 Program Background In response to low school attendance and the recurrent food crisis in Mali, CRS has implemented MGD I and II school feeding projects funded by USDA, in collaboration with local partners and the Ministry of National Education (MONE) in Mali in 2007-2010 (phase I) and 2011-2015 (phase II). The previous Mali MGD project (2011–2015) reached 76,411 primary and secondary students in 311 schools and has achieved positive results in enrollment and attendance rates, especially for girls, through provision of school meals and take-home rations. In addition, MGD I and II also focused on implementing activities such as establishing water points and school gardens, constructing latrines, and improving communities’ WASH knowledge and practices. These activities were specific to phase I and II. Although the enrollment and attendance improved during the first two phases of the project, there was concern about the quality of education, particularly literacy in primary grades. Therefore, in September 2015, USDA awarded CRS $29.9 million to implement the third phase of the MGD program, a five-year project (FY2016–FY2020) to build on previous successes with a main strategic objective of improving the literacy of 77,104 school-aged children in 264 primary schools in the regions of Mopti and Koulikoro (Exhibit 1). MGD III included elements of previous phases, including school meals, take-home rations (THR) and vitamin A and deworming medications distribution, and the formation of savings and internal lending community (SILC) groups. In addition, new project activities Exhibit 1: Map of Targeted Region in Mali IMPAQ International, LLC Page 2 Mali MGD Midline Report July 2018 for the third phase include school management committees (SMC) members capacity building, expansion of illustrated report cards, provision of literacy materials to schools, and training to teachers and administrators on the balanced literacy approach (BLA). (See the full list of MGD activities in all the three phases in Appendix A.) Based on the MGD III results framework (see Appendix A), higher literacy rates for school-aged children will be achieved by improving student attendance at high-quality literacy instruction in an environment in which students are enabled to participate actively in class. To improve attendance and attentiveness in 264 primary schools identified in the previous phases, this MGD project will support community-specific enrollment campaigns based on community-led barrier analysis results and continue school feedings to reduce short-term hunger and improve student attentiveness. Vitamin A and deworming medications will be distributed to enrolled students in Y1, 2, and 3, with a transition to local responsibility in Y3. Take home rations (THR) will be distributed to all fifth and sixth grade students in USDA-supported primary schools with a minimum 90% attendance to promote attendance and retention. CRS will also build sustained capacity in school feeding and early grade literacy instruction at the local, regional, and national levels. For school feeding, this includes tailored training to “graduate” SMCs in school management and matching grants to fund school action plans, incentivize graduation, and engender ownership by the decentralized structures that must provide a match. CRS will also provide technical support to the National School Canteen Center (CNCS) in response to weaknesses identified in their biennial evaluation of school feeding capacity. For early grade literacy instruction, this includes creating a network of government officials, school administrators, and teachers trained in the BLA, providing classroom kits, and monitoring student progress with the Early Grade Reading Assessment (EGRA). CRS is leading the implementation of MGD III in partnership with Amprode, Caritas Bamako, Caritas Mopti, Education Development Center, Inc. (EDC), and Guamina. In collaboration with the MONE, regional education offices (Academie), school district offices, and SMCs, CRS is building local capacity and promoting sustainability for school feeding and literacy activities where local, regional, and national education officials directly support monitoring and evaluation activities. 1.2 Evaluation Background CRS selected IMPAQ International, LLC (IMPAQ) in 2015–2016 to design the impact and performance evaluations of MGD III at baseline. IMPAQ used quantitative and qualitative methods to establish baseline values for outcome indicators required by USDA, refined targets for performance indicators, generated data to be used for comparative analysis, and validated project strategies and assumptions at baseline. In March 2018, CRS re-selected IMPAQ for the midline performance and impact evaluation of MGD III. Following the evaluation design at baseline, IMPAQ used quantitative and qualitative methods to assess progress in implementation, relevance of interventions, and appropriateness of strategies, management structures, and/or activities for improved project effectiveness, efficiency, impact, and sustainability. The midline evaluation consists of the following two components: 1. Performance evaluation. This evaluation is structured to measure changes in outcomes up to the midpoint of the project and to inform the overall evaluation results on MGD III core objectives. To accurately capture program performance longitudinally, IMPAQ measures the same program IMPAQ International, LLC Page 3 Mali MGD Midline Report July 2018 indicators at all three data collection points (baseline, midline, and endline). This report focuses on the midline evaluation findings and compares the performance of MGD III indicator outcomes between baseline and midline with regard to two of USDA’s strategic objectives (SO): SO1 with a focus on students’ reading proficiency, attendance, participation in classroom activities, and teachers’ motivation, as well as parent/community engagement; SO2 with a focus on three key areas, including safe food preparation knowledge and storage practices, accessibility to preventive health interventions, and reducing health-related absences. 2. Impact evaluation. This component of the evaluation seeks to identify the causal effect of the BLA teacher training intervention on the literacy of primary-school-aged children. In this report, we compare students’ outcomes at midline in second and third grades with their peers’ outcomes at baseline when their teachers did not receive any training. This report discusses the impact of the two and three years of BLA intervention on literacy levels of both girls and boys in the targeted schools. To evaluate the key outcome indicators and measure progress toward achieving them, as well as to address evaluation questions, which are outlined in detail in Section 2.1, IMPAQ collected survey data (students, parents/caregivers, teachers/principals, and SMCs) using the same instruments used at baseline, as well as qualitative key informant interviews, and focus group data. At midline, the qualitative research questions focused on learning what has occurred to date, including a formative assessment and suggestions for program process improvement. At endline, the qualitative research questions will focus on learning more about the potential for program sustainability and about promising practices and lessons learned. IMPAQ International, LLC Page 9 Mali MGD Midline Report July 2018 documented as a serious concern in developing countries. For example, grade 4 students in Comparison 3 at midline will not have been taught by a BLA-trained teacher but may have benefited from the BLA intervention through spillover effects. By comparing this group with the grade 4 students in Comparison 1 from baseline, we determine the three-year spillover effect on literacy progress of being in a BLA school. Source: IMPAQ. Limitations of the Cohort Comparison Method This is a quasi-experimental design that relies on the assumption that we are able to capture causal changes in literacy rates by measuring changes across cohorts. Our identification strategy rests on the assumption that there are no unobserved variables that affect both the probability of being part of the intervention group and the literacy rates of children. For example, particular educational policies enacted by the government in the same year of the intervention would potentially confound the cohort comparison approach. We also assume that the time effects experienced by students in different grades are the same—that is, grade 1 students experience the same time effects as grade 2 and grade 3 students. If there were any changes in the overall environment or any government policy that influenced only particular cohorts between baseline and midline, then this assumption may not necessarily hold. To safeguard from these threats and ensure the validity of our methodology, we have taken three different actions exploiting the structure of the program implementation and the data available:  We include time effects controls for all year-specific, individual-shared increases in literacy outcomes for all individuals. This addresses the identification threat regarding other educational policies being enacted. Exhibit 4: Example of Calculations: Two-Year Program Effects The average treatment effect on the treated after 2 years of exposure to the program (ATE2) is the difference in illiteracy prevalence for children in second grade at midline and baseline after controlling for any time effects between baseline and midline, as shown in Equation 1. 𝐴𝐴𝐴𝐴𝐴𝐴2 = (𝑃𝑃𝑡𝑡+2 2 − 𝑃𝑃𝑡𝑡 2 �������) two-year change in prevalence − (𝑃𝑃𝑡𝑡+2 1 − 𝑃𝑃𝑡𝑡 1 �������) time effect (1) The total average treatment effect on literacy levels after two years of exposure to the program (TATE2) is the weighted average of the ATE2 after two years of exposure to the program and the indirect treatment effect on the untreated (ITE) after being exposed to the project between baseline and midline. 𝐼𝐼 = (𝑃𝑃𝑡𝑡+2 4 − 𝑃𝑃𝑡𝑡 4 �������) spillover effect (2) 2 = 0.5 𝐴𝐴𝐴𝐴𝐴𝐴2 + 0.5 𝐼𝐼 (3) where  𝑃𝑃𝑡𝑡+2 2 is illiteracy prevalence of children in second grade in year 3 (midline)  𝑃𝑃𝑡𝑡 2 is illiteracy prevalence of children in second grade in year 1 (baseline)  𝑃𝑃𝑡𝑡+2 1 is illiteracy prevalence of children in first grade in year 3 (midline)  𝑃𝑃𝑡𝑡 1 is illiteracy prevalence of children in first grade in year 1 (baseline)  𝑃𝑃𝑡𝑡+2 4 is illiteracy prevalence of children in fourth grade in year 3 (midline)  𝑃𝑃𝑡𝑡 4 is illiteracy prevalence of children in fourth grade in year 1 (baseline) IMPAQ International, LLC Page 10 Mali MGD Midline Report July 2018  Threats arising from spillover effects are investigated through comparison across cohorts in the same school.  Additionally, our evaluation involves a substantial data collection on different variables. These variables are included in our specifications to control for other factors arising from students and teachers. Therefore, by taking advantage of the longitudinal data, the cohort implementation of the program, and a wide set of variables, our proposed quasi-experimental design is rigorous and allows us to mitigate many of the potential issues. Descriptive Performance Analysis In addition to these quasi-experimental methods, we also conducted descriptive performance analysis using data collected from parents who are SILC members and from fathers and mothers of students in grade 1 and grade 2, in addition to the data analysis from teacher, school principal, SMC, student, and parent surveys (see Data Sources for more detail about the survey instruments). In addition, we analyzed the school attendance data provided to IMPAQ by CRS through its M&E data collection system.  Descriptive analysis of SILC members’ data: Based on the data collected on SILC members in the parent survey, we provide descriptive characteristics on the SILC group participation. For example, we provide information on how SILC membership has influenced the children’s participation in schools and on the livelihoods of the household.  Comparative analysis of mothers’ and fathers’ response data: In the midline evaluation, we surveyed both fathers and mothers from grades 1 and 2 and elicited their responses on the same key performance indicators, such as their perceptions of the importance of education. We also asked a series of questions on their degree of decision making in the household, especially in the realm of children’s education. In many households in which a child’s father and mother are present, fathers often act as the decision makers in family matters, including children’s schooling. By comparing the responses of mothers and fathers, we analyzed the differences on educational awareness within a household and the differences in decision making. We performed t-tests to see whether the differences between fathers’ and mothers’ responses are also statistically significantly different.  Performance analysis using school attendance data: We used the attendance data provided by CRS to assess the trends in school attendance after different periods of program maturity and across the two key study areas. 2.2.2 Qualitative Evaluation Design Like the baseline, our qualitative design is a rapid-assessment approach using key informant interviews (KIIs) and focus group discussions (FGDs) with selected key project stakeholders and beneficiaries at both the national and community levels. For the midline evaluation, we included local administrative and education officials(including mayors), as mobilizing their support is a key aspect of program sustainability. We also added national stakeholders, including project staff, partners, and Ministry officials. We explain this in more detail in the section on sampling strategy (Section 2.3). IMPAQ International, LLC Page 11 Mali MGD Midline Report July 2018 We created role-specific interview and focus group protocols to question the identified stakeholders about their perceptions of the project implementation process, the project management, and successes and challenges. For the baseline, protocols focused on collecting information on current project realities and perceptions. For the midline, we included questions on impact, effectiveness, and sustainability (see the protocols in Appendix D, Evaluation Instruments). Qualitative Analysis Immediately after data collection, the qualitative data collectors emailed their notes in French to the IMPAQ team. After receiving the first set of notes, the team and data collectors held a debriefing over Skype to review the following:  Clarify any questions where respondents were confused.  Discuss any challenges in the field.  Identify what topics and issues needed further probing.  Determine how to adapt the guides in real time, if needed, to obtain more meaningful data.  Ensure that the research team shared a common understanding and interpretation of the main points and themes. Following this initial meeting, the data collectors and research team communicated through email to continue discussing these points. After the data collection was complete, the team reviewed and analyzed the debriefing notes, supplemented by interview recordings, to identify recurring patterns pertaining to the five research domains. 2.3 Sampling Strategy Design 2.3.1 Quantitative Sampling To implement a cohort comparison method, at baseline we sampled among schools where grades 1 through 4 were taught, there were no multi-grade classrooms for grades 1–4, and teachers taught only one grade (grades 1–4) per school. Following these criteria, we surveyed 2,464 students at baseline. In addition to students, we surveyed their mothers/caregivers, as well as their teachers, school principals, and SMCs in our sampled schools. For the midline evaluation, because four of our originally sampled 50 schools were closed as a result of political instability, we visited the 46 remaining sampled schools for the midline evaluation, surveying approximately 627 students from each grade (grades 1-4; 14 students, on average, per grade in each school), for a total sample of 2,506 students (1,216 boys and 1,290 girls). Although we surveyed fewer students from fewer schools, our power calculations show that our minimum detectable effect increases IMPAQ International, LLC Page 17 Mali MGD Midline Report July 2018 SECTION 3. FIELDWORK According to the US Department of Health and Human Services, this study was exempt from institutional review board (IRB) review because: (1) there is no biomedical testing involved in this research; and (2) this research is being conducted in an established or commonly accepted educational setting, “involving normal education practices.”7 Prior to administering the survey, enumerators were trained on procedures for contacting respondents, protecting respondent privacy and confidentiality, and securing the data. During the data collection, the survey team asked all respondents for their consent to proceed with the survey. With regards to students, we interviewed students after receiving their parents or school principals’ consent, as well as their assent. The survey team also assured children that their participation was voluntary, and that they could terminate the survey at any time they want. After data collection, the evaluation team also protected the privacy and confidentiality of respondents by storing the data in secure servers and separating personally identifiable information from the survey data. This section outlines the quantitative and qualitative fieldwork and subsequent data analysis for this evaluation. Data Collection Preparation In collaboration with CRS, IMPAQ trained 50 enumerators to collect the midline data in April and May 2018. To enhance the efficiency of midline data collection, CRS was able to invite 28 enumerators who collected baseline data to the training sessions. Enumerator training consisted of three days of theoretical indoor training and two days of pilot testing, followed by a debrief session and instrument refinement in Bamako. Enumerators received training on the rationale behind each survey item and learned how to collect data on paper and tablets. At the end of the training, based on a written test and the quality of pilot data collected from enumerators’ practice, in collaboration with CRS, we hired the 44 most experienced and skillful enumerators out 50 for the data collection. In addition to quantitative data collection training, we also provided a one-day training to the two local consultants hired by IMPAQ. Data Collection With CRS, we organized the 44 enumerators into 11 teams of four individuals and assigned each team to a region to survey. The final selected enumerators used tablets to conduct the in-person surveys and electronically submitted the surveys periodically during the fieldwork. Each team had a team leader. Most of the team leaders were already familiar with the project, the schools, and implementation of the ASER reading assessment. In addition to leadership tasks, they conducted school-level observations, including handwashing practices and canteen and food storage safety activities. In collaboration with CRS facilitators and IMPAQ fieldwork managers, team leaders also contacted school principals to coordinate their arrival before visiting the school; ensured that randomly selected students were still enrolled and available to be surveyed; and, finally, asked principals to call on selected students’ parents to be present on a day of data collection to give their consent and be surveyed. Two IMPAQ local consultants, one in each region, closely followed the teams of enumerators daily to oversee the quality of the data that enumerators collected and provide them with technical support. They also conducted qualitative interviews with local stakeholders and beneficiaries. 7 Exemption 45 CFR 46.101(b)(1). IMPAQ International, LLC Page 18 Mali MGD Midline Report July 2018 All enumerators regrouped with/contacted IMPAQ’s fieldwork managers several times during the day to debrief, submit daily data collection logs, submit electronic surveys, and review and plan for the next days of data collection. IMPAQ also met with the fieldwork managers on a daily basis to the extent possible during data collection to ensure a smooth collection and provide guidance on the potential challenges faced in the field. In addition, during the data collection, the data management team conducted quality checks on the uploaded data since the first day of data collection to ensure high-quality datasets. The team completed the fieldwork in 15 days. IMPAQ International, LLC Page 56 Mali MGD Midline Report July 2018 Source: Caregiver Survey; authors’ calculations, Midline: N=787 in Koulikoro, N=544 in Mopti. Total number of responses is larger than the number of respondents because respondents could select multiple choices. With regard to students’ education, respondents also frequently said that SILC membership contributed to their children’s education, with 40 percent in Koulikoro and 57 percent in Mopti saying it allowed them to save money for school fees; 33 percent in Koulikoro and 49 percent in Mopti saying it encouraged them to send their children to school; and 43 percent in Koulikoro and 32 percent in Mopti saying it helped them contribute to the school canteen. Only 9 percent in Koulikoro and 10 percent in Mopti of SILC members or associated caregivers said SILC membership did not contribute to the education of their children. 5.1.4 Teacher Outcomes From the teacher survey, we compared baseline and midline outcomes for teachers, including principals who taught in grades 1–4, by grade in the following four areas:  Pre-service and in-service trainings  Balanced literacy approach (BLA)  Pedagogical support and oversight  Health and hygiene knowledge Pre-Service and In-Service Trainings Received We examined the percentage of teachers who were formally trained to teach and the type of trainings they received beyond the BLA training. At baseline, about 83 percent of teachers in Koulikoro and 87 percent in Mopti reported having been formally trained to teach. At midline, these percentages increased in both regions, to 86 percent in Koulikoro and 91 percent in Mopti. As Exhibit 59 shows, there were no significant changes between teachers at baseline and midline in the formal training that they received. However, at midline, the majority had been recruited and trained through IFM32 (Institut de Formation des Maîtres) and SARPE (Strategy Alternative de Recruitment du Personnel Enseignant).33 The remainder were recruited and trained through ECOM (Ecole Communautaire)34 and other options. Overall, a higher percentage of teachers in the Mopti sample were trained through IFM and ECOM than in Koulikoro. 32 IFM is a teacher training school. All schools have a 4-year program for Grade 9 graduates and 2-year training program for Grade 12 graduates. The training program includes psychology, pedagogy, and subject matters such as science, mathematics, and languages. 33 SARPE is “a fast-track training route which involves taking slightly older students—again, with a minimum qualification of the DEF (although many will have received some further education)—and training them over what was 15 days and is now six months. SARPE is organized and taught by the local education authorities, with school advisors taking a prominent role in the training.” (“Mali: Teacher Preparation and Continuing Professional Development in Africa (TPA)”). Center for International Education (CIE). (2016). Mali: Teacher Preparation and Continuing Professional Development in Africa (TPA). Brighton, England: University of Sussex. http://www.sussex.ac.uk/cie/projects/completed/tpa/mali. 34 ECOM is a 45-day training program for community school teachers. Those teachers are hired and paid by communities but go through this government-supported training program. The program also includes psychology, pedagogy, and other subject matters. IMPAQ International, LLC Page 61 Mali MGD Midline Report July 2018 advisor had observed them during the lesson, with 54 percent saying that the regional supervisor had done so. The vast majority of surveyed teachers also received other types of support from their principal at midline. Most frequently, this was encouragement (63–64 percent overall) or teaching advice (77–85 percent overall). These outcomes were more or less the same compared with baseline, with no significant changes. These results suggest that the project activities on training school administrators for supporting teachers in their classrooms are being implemented as planned. Health and Hygiene Knowledge To measure teachers’ knowledge and practice of hygiene, we asked them about handwashing practices and intestinal worm prevention. Just as we did for caregivers and students, we first calculated the rate at which teachers identified at least the two critical moments (washing hands before eating and after using the latrines) and compared it to the rate at which teachers reported washing their hands for those two specific moments. In general, there were no regional differences among teachers when it came to handwashing and from baseline to midline, with 91–95 percent saying that people should wash their hands for the two moments considered critical and roughly 85 percent saying they actually washed their hands in those moments. On average, almost every person reported using soap when they washed their hands. The majority of teachers were able to cite at least two ways of preventing intestinal worms: 81 percent of teachers in Koulikoro were able to do so (significantly higher than at baseline), whereas only 68 percent were able to do the same in Mopti. 5.1.5 School Principal Outcomes This section presents baseline outcomes for all principals (N=44) in the following areas:  Pedagogical advisors and oversight of teachers  Health and hygiene knowledge As mentioned in Section 4.6, due to the small sample of principals, the results in this section presented with no statistical analysis (t-test of difference in means). Pedagogical Advisors and Oversight of Teachers To measure the extent to which principals supported and supervised teachers, we investigated how often principals observed their teachers during the reading–writing class, whether principals had difficulties with supporting their teachers, and the extent to which pedagogical advisors were helpful to principals. There were regional differences in the self-reported number of times principals observed their teachers during reading–writing lessons over an ordinary month. Unlike teachers’ responses, no principal said they never observed their teachers in a month, whereas principals in Koulikoro said they observed their teachers far more often (57 percent said more than 2 times a month) than did principals in Mopti (44 percent said they did so once a month). At baseline, most principals reported not having any difficulties with observing their teachers, with only 20 percent in Koulikoro and 32 percent in Mopti reporting such challenges. However, at midline, 69 IMPAQ International, LLC Page 67 Mali MGD Midline Report July 2018 baseline. The grade 1 students from midline and baseline are used to capture the time effects of the program intervention between baseline and midline. To obtain the two-year program effect of the program, we used the reading assessment data of both grade 2 and grade 1 students at midline and baseline. To estimate this effect, we ran the following regression model: Li = β0 + β1. Pi + β2. Ti + β3. Pi. Ti + β4. Xi + εi where • Li is the reading assessment score of student i • Pi is a dummy variable with value equal to 1 for student i at midline, and 0 otherwise • Ti is a dummy variable with value equal to 1 for grade 2 students at midline and baseline, and 0 otherwise • Xi represents other covariates such as the gender of student i, which could be potentially related to the literacy outcome of the student • β0 is the baseline average reading assessment score of grade 1 students • β1is the average change in reading assessment score for grade 1 students between midline and baseline and represents the time trend • β2 is the average difference between grade 2 and grade 1 reading assessment scores at baseline • β3 is the average two-year effect of being exposed to BLA-trained teachers on grade 2 students, and is the main estimate of interest • β4 is the change in student’s literacy scores if the student is a girl • εi is the random residual term To obtain more precise estimates, we included other covariates, such as the gender of the student in the preceding model. Because students are nested in a particular grade from a given school, literacy outcomes are related to each other (e.g., the residual term of a student in a given cohort in a particular school may be related to the residual term of his/her classmate). To account for this correlation of error terms across classmates in the same school, we estimated the standard errors after clustering for each separate cohort in each school (92 clusters for two grades in 46 schools). As a robustness check, we estimated our model by restricting the sample of grade 2 students at midline to only students who had been in the school since grade 1 (i.e., we excluded students who joined the school after baseline and may have been only partially exposed to the intervention). Similarly, we ensured that teachers who were associated with these students had received at least 2 years of teacher training (i.e., we excluded students of new teachers without BLA training). We estimated the three-year program effect in a similar manner as described previously, except that instead of using grade 2 student literacy outcomes, we used grade 3 student data at midline and baseline. We estimated the spillover effect by using the sample of grade 4 student outcomes at midline and baseline and tested whether the average outcomes varied significantly. Similar to the program effect model, we tested the robustness of our spillover estimates by adding other covariates to our spillover effect model. Exhibit 72 summarizes the impact results for two- and three-year program effects’ regression models and the spillover effects regression models. The row highlighted in gray represents the key program effects on improvements in children’s literacy outcomes, which is the key focus of the quasi-experimental evaluation. IMPAQ International, LLC Page 69 Mali MGD Midline Report July 2018 leads to changes in literacy outcomes, rather than sporadic exposures to BLA training methods used by teachers. The exhibit below further presents visually our results of the two-year program effect to facilitate easier comprehension of the intuition of the approach. As shown, (D–B) or β1 is the time trend as represented by the grade 1 students’ change in reading assessment scores. (B-A) or β2 is the average change in scores between grade 2 and grade 1 students at baseline. Our main outcome of interest is β3, which is measured as the difference between grade 2 scores at midline and baseline, after adjusting for the time effects. Exhibit 73: Two-year Program Effect on Litarcy Outcomes Source: IMPAQ IMPAQ International, LLC Page 72 Mali MGD Midline Report July 2018 6.1.2 Adjustments to Project Design Project staff and partners agreed that the objectives, targets, and timeframe were appropriate and realistic during the design phase. However, several external factors have forced adjustments from the original plan—most notably, schools closing temporarily or permanently because of insecurity in the program’s targeted area. Other external factors included the Ministry of Education’s decision to promote syllabic teaching approach across the country, and the frequent teachers’ strikes. Both of these caused some challenges to BLA trainings, as the project had to make some efforts to convince the government to allow for BLA training to continue in the project area and teacher strikes caused training events to be rescheduled or canceled. Finally, since the program started, Koranic schools have been increasing in the targeted area, particularly in Mopti. Local education officials confirmed that many parents started sending their children to Koranic schools for cultural reasons, the apparent security offered, and the simple fact that they are open, leading to a decrease in public school enrollment. These external factors required the original targets, objectives, and timeline of the MGD program to change. There were internal factors that affected the original program design as well. For example, capacity￾building activities at the local level took longer than planned. This was attributed to an overly ambitious SMC training, which covered too much information for one session. Local implementing partners decided to break the materials up over several sessions to make it easier for the SMC members to learn and process the topics. In addition, the local implementing partners expressed that the planned work in their regions was not appropriate given both their capacity and the capacity of the local stakeholders/beneficiaries, and therefore the rollout of activities was delayed accordingly. 6.2 Effectiveness Interview topics focused on the extent to which the project is achieving its two strategic objectives, successes and challenges of program implementation, and the effectiveness of management arrangements and stakeholder collaboration. 6.2.1 Strategic Objective 1: Improved Overall Literacy of School-Aged Children Successes According to stakeholders in the local education sector, the project is already improving the literacy of children in their communities. Several education officials spoke of the “new approach” to literacy that was helping children learn faster and increase their comfort in speaking and reading. Moreover, almost everyone said that the canteens played a crucial role in pupils’ attendance at school, facilitating learning. They said that before the project, pupils were obliged to return home for lunch, and as many live far from school, they were tired and unable to return in the afternoon. Having lunch at school means more students stay for the entire day. “Before the project, for students to master the alphabet, it took them two years. But now, with the new approach, we see pupils who are able to cite the alphabet at the first quarter of the first grade.” –County mayor IMPAQ International, LLC Page 73 Mali MGD Midline Report July 2018 From the perspective of the parents, the canteens have been a good strategy to increase school attendance. Like the local administrators, parents reported that having lunch at school encourages students to stay a full day at school. Parents also mentioned the take-home rations, saying that they not only incentivize children to go to school, but parents also strongly encourage their children to attend because it helps them to secure and save food for the rest of the family. For example, because of the donation of oil, mothers encourage their daughters to go to school rather than asking them to stay home to do chores. Program staff and partners said the BLA pedagogical techniques directly influenced children’s literacy. Project M&E data confirm this, as do the results of the impact evaluation (Section 5.2). Factors facilitating this success are the repeated trainings for teachers, which reinforce the new teaching methods, and the capacity building of the larger local education network, specifically school administrators, pedagogical advisors, and regional education offices. While most parents did not directly speak to the literacy instruction (although many SMCs mentioned this), parents did mention the colored reports designed to assess their children’s performance at school. Parents appreciated that these simple tools helped them to support their kids at home. Literate parents reported supporting their children by tutoring or mentoring them at home. Illiterate parents reported still being able to indirectly motivate their children’s performance by providing rewards for good results and ensuring that their children have breakfast and money before they go to school. Challenges Although the literacy of school-aged children is improving for now, stakeholders at both the national and local levels said that several factors challenge this objective. One is that teachers’ mobility could threaten the sustainability of the project’s results. Teachers who were trained in the new literacy approach will be relocated to other villages after three years of service, and the new teachers might not have received the training. Program staff also explained that some teachers and even parents resisted the BLA methods, as they did not believe that these methods would be effective. Furthermore, there is still a problem with teacher attendance, as teachers often don’t show up at school, or arrive very late. Like their students, these delays often occur because many of them must travel a long distance because they live far from their schools. It was also reported in some areas that female teachers have left because of security issues and fear of attacks. Parents (particularly those who are illiterate) noted that if teachers lived in the same village, they could provide additional support to their students in the evening after the school day. “We should acknowledge that there has been a great improvement in our children’s literacy skills. Before the project, pupils from first, second, and third grade were unable to read well. Now, pupils from first to second grade can read, write their names and those of their parents.” – SMC member “Today many kids are at school because of the canteen food. For example, the kids from Kimetogoro have abandoned school because they were obliged to walk long distance to return at their homes for the lunch at noon. Now they all come to school and stay there the whole day because they receive meal in the noon.” – Parent IMPAQ International, LLC Page 74 Mali MGD Midline Report July 2018 Program staff and partners also reported that teachers are overwhelmed with the tasks required through this program to 1) fill out food delivery sheets; and 2) work on the inventory report. These are in addition to their teaching schedules and responsibilities such as completing daily attendance records and creating colored report cards so illiterate parents can track their children’s progress. In some schools, SMC members are illiterate, and require the teachers to help them fill out forms such as the school action plan. Partners reported that many teachers resent this and are reluctant to do additional work without more support. Another challenge to the achievement of this objective is the local contribution of food for the canteen. Parents and local administrators stated that their contributions could be threatened by differing climatic conditions from one year to another, which could result in bad harvests. The contribution is also currently quite difficult for parents who are poor and/or have several children. In some localities, immigration abroad is a reason many students drop out of school. Money sent from their relatives in foreign countries or in larger cities are a main source of their livelihoods. As a result, some families encourage their children to immigrate. Girls are particularly encouraged to migrate to town to save money for their marriage trousseau, while boys are encouraged to immigrate to find work and “have adventures.” Besides immigration, children’s attendance is threatened by domestic activities like working in the fields for boys and doing housework for girls. However, due to the provision of food and local sensitization efforts, these are becoming less common reasons, and more children are attending school. Finally, some local focus group participants mentioned that the absence of electricity at home makes it difficult for children to study at night; therefore, the objective of improving literacy may not reach its full potential. 6.2.2 Strategic Objective 2: Increased Use of Health and Dietary Practices in Program Schools Successes Program staff and partners said that good health and dietary practices have increased, as deworming medication, vitamin A, and iron supplements have been distributed to children at all program schools. In addition, the quality of food at the canteens has led to healthier meals at lunch. From the local perspective, while participants spoke very favorably of the food ration provided by CRS for the canteens and the new literacy approach, the activities related to health, hygiene, and sanitation were mostly reviewed negatively. The next section describes the challenges. “We organized meetings many times with parents to sensitize them to contribute to the payment of foodstuff for the canteen, but some parents still don’t give their contribution. As a result, the canteen is sometimes closed for more than one week.” –Rural SMC “SMC members are all illiterate, we need functional literacy training to be more effective” – Male SMC “The most important source of incomes here are remittances sent by our family members abroad. Therefore, boys are dreaming of going abroad to help their families. The willingness to go to school is less important for them. As for girls, very early, they have to migrate to big towns to look for money for their marriage; which means once back, they are given in marriage.” –Mother’s focus group IMPAQ International, LLC Page 75 Mali MGD Midline Report July 2018 Challenges Issues related to water availability, the low flow of water pumps, and the remoteness or absence of health centers were most frequently stated by parents and SMC members as factors that negatively affect their children’s health and dietary practices. The most common example given by parents was that handwashing practices are impossible to sustain because of the unavailability of water at the schools. Other parents said that the scarcity of water also affected their gardening activities at the schools. SMC groups said that the program requires them to grow vegetables for the canteen, but without water, they have not been able to do so. Parents also reported being encouraged to visit health facilities for vaccinations or when their children are sick, but for the majority of community members, the distance is too great to walk. They therefore suggested having some sort of health center at the school level. 6.3 Efficiency Interview topics focused on efficiency in the use of project resources, the timeliness of project activities, and the project’s responses to internal and external factors that may hinder implementation. 6.3.1 Use of Project Resources Program staff described school closures as the main obstacle affecting the budget and project resources. School closures caused the budget not to be fully spent as planned—fewer schools means training fewer teachers, providing fewer supplies, and so on. In addition, the project staff also mentioned they had to revisit their budget allocations in response to changes in the beneficiaries’ need (e.g., increasing the number of training days for SMCs), which was underestimated. In such cases, the project used its saving from favorable exchange rates between the West African CFA franc and the US dollar. However, from the perspective of some local administrators, political differences within the education sector sometimes leads to inefficient allocation of resources. Mayors or leaders in the local education sector made decisions which affect project activities without involving other key actors. For example, it was reported that some mayors move teachers who are involved with the oppositional political party from MGD schools to schools not participating in the program. 6.3.2 Achievement of Project Timeline As mentioned in Section 6.1, Relevance, the timeline has been delayed because of several factors, including school closures, teacher strikes, syllabic teaching program promoted by the MONE, and local capacity. These factors combined with the expiration of food has made it difficult for the program to “The lack of water seriously undermines the cooking of food at school. We need water to ensure the sustainability of the canteen.” –Female SMC group “[Moving teachers] is done without any concern to allocate resources to real needs…For the project to be effective, strategies should be designed to actively involve all actors in the education sector. If all the actors are really involved in the life of the project, the objectives can be achieved without many issues.” –Local administrator IMPAQ International, LLC Page 76 Mali MGD Midline Report July 2018 distribute all the commodities. However, in such cases, the project has distributed the food through different channels (THR rather than school meals) before their expiration dates. 6.4 Perceived Impact Interview topics focused on stakeholders’ perception of the effect of the project on children’s education and health outcomes and the activities with the greatest and least medium- and long-term effects. Stakeholders were also asked to provide recommendations to improve the success of the project. 6.4.1 Addressing Barriers to Education At baseline, the main barriers to education reported by parents and SMC members were distance to school, cultural beliefs, and local insecurity. At midline, parents and SMC members reported that some of these barriers have been addressed by the project. Respondents stated that enrollment and attendance of school-aged children have greatly improved as a result of food provision at school. Even though many students still live far away from school, they are motivated to attend by knowing that food is available, and are more likely to stay a full day rather than going home for lunch and not returning. Parents and local education officials also stated that parents are now more concerned about their children’s education. Factors facilitating this change in belief have been greater community sensitization through door-to-door campaigning, and also the colored reports on students’ performance. For example, many parents said that having the report encouraged them to meet with teachers or the head of the school about how to encourage or work with their children to improve their academic performance. A few parents mentioned political insecurity as a deterrent to school attendance, although this topic came up less frequently at midline compared to baseline. However, stakeholders at both the national and local levels said that student safety and security is still a very serious barrier to education. 6.4.2 Activities with the Greatest and Least Impact From the local perspective (parents, SMC members, and education and administrative officials) the activities with the greatest impact are the food rations distributed to children (at lunch and THR) and the new literacy training approach. Local stakeholders overwhelmingly agreed that the activities with the least impact are those related to school gardens, community contributions, and sanitation and health. Again, the lack of water at the schools was the reason given for limited impact of the school gardens and the sanitation and health-related activities. For community contributions, the low and variable financial resources of the local families mean that many are unable to consistently help stock the canteens. In corroboration with local project beneficiaries, national project staff/partners and Ministry officials confirmed that the BLA literacy approach has the greatest impact. The improvement in learning and instruction has been documented through project M&E data (EGRA and classroom observations), and also through the external impact evaluation (Section 5.2). National stakeholders also believed that the financial knowledge building and livelihood training through the SILC groups are having a substantial impact as well. Project staff see evidence that SILC groups are growing and successfully managing funds “We were expected to provide the canteen with vegetables through our gardening activities, but the activities have stopped because of the lack of water.” –SMC member IMPAQ International, LLC Page 77 Mali MGD Midline Report July 2018 even without access to local banks.41 In addition, project monitoring data show that 239 SILC groups have contributed an average of $13 each to school canteens. However, staff mentioned that more activities needed to be done to ensure successful transfer of canteen activities to local stakeholders, including advocating for school feeding support from the mayors’ offices. 6.4.3 Suggestions for Increasing Impact Several parents appreciated the vegetable gardens initiated by the project, and believed that these gardens have an important potential to provide vegetables for the canteens. However, communities are unable to take the full advantage of these gardens because of the absence of a water source or because of the low flow rate of the existing pumps. Participants suggested that the next phase of the project should consider the provision of water sources to increase the impact of a fully functioning vegetable garden supplementing the canteens. Many discussants expressed regret that the project did not take into account provision of water sources and health facilities, which are serious issues for communities. Although the project enabled many parents to take interest in and support their children’s education, many stated that they are limited in their efforts by their lack of financial resources. For example, several parents said that even though they are sending their children to school, they are unable to buy supplemental learning materials such as notebooks and textbooks for their children. Many parents suggested that the project consider this aspect for the next round or in future interventions. Several parents and SMC members suggested adding a livelihood component to support parents with income￾generating activities. Parents stated that if they were supported in launching income-generating activities such as soap making, animal husbandry, or even savings and loans committees, this could help increase the impact of the project. Given that many parents are illiterate, the SMC members noted that their performance is therefore limited (e.g., record keeping is extremely difficult). Therefore, many parents and SMC membersrequested adult literacy training from the project to strengthen their capacities. 6.5 Sustainability To assess the sustainability of project interventions, the research team asked all stakeholders to identify major factors that are likely to influence the achievement or non-achievement of project sustainability and to describe current sustainability activities implemented by CRS and its partners. Respondents were also asked to recommend strategies to sustain activities after funding ends. 6.5.1 Greatest Likelihood of Sustainability Parents and local administrative officials are well aware that they are responsible for ensuring the program’s sustainability. Discussions with parents and SMC members showed that even with challenges, parents are enthusiastic about continuing the canteen activities. They plan to continue school feedings through local contributions and school gardens, although SMC members cautioned that they would not be able to provide the same quantity of food that CRS currently provides. 41 This was not corroborated through our local focus group discussions, in which parents and SMC members specifically asked for assistance in forming SILCs. However, SILC groups are not in every area of the project, and were not targeted for qualitative data collection. See Section 5.1.3 for findings from the SILC member survey. IMPAQ International, LLC Page 78 Mali MGD Midline Report July 2018 Project staff and partners said that the knowledge gained through the program would last beyond the program activities. For example, they are confident that the SILC groups will continue their activities, as the more established groups already function with minimal supervision. The literacy teaching approach will be sustainable to the extent that trained teachers stay in beneficiary communities (or if the incoming teachers are trained). Staff also hoped that when teachers move to other communities, they will bring the BLA techniques with them to their new schools. 6.5.2 Challenges to Sustainability The lack of income-generating activities for parents was stressed as a critical issue for sustainability. Poverty was cited as the main driver of parents’ inability to support their children’s education. Parents and SMC members stated that sometimes schools lack the means to cook because they have not received the contribution of the community to buy food. Especially during years with a poor harvest, parents are often unable to contribute to the canteens. From the perspective of some parents, they and other parents in their community lack means to buy school manuals, presenting a threat to their children’s academic performance. Some said that their children are sometimes forced to stay at home when teachers refuse class admittance when students don’t have required materials such as pens, copybooks, or slates. Most of the parents recognized that sending their children to school is worthwhile, because they are aware that education is key for their children to get better jobs and social recognition in their communities. Most would like to see their children reach secondary and even university education. However, secondary schools are generally located far from the communities where they live, and require costly school fees and food and board arrangements. As a result, most parents are aware that their children will stop school after the primary level. Finally, as mentioned earlier, the issue of water scarcity threatens the washing and sanitation practices and gardening activities. From the perspective of project staff, most did not think that the local communities would be able to continue the project as is without additional support. They believed that there is not yet sufficient ownership at the local level to manage the canteens. They are also concerned about the transfer of knowledge as trained teachers leave schools. Project staff also mentioned political threats to project sustainability, most likely the potential for conflicts closing schools, the opening of more Koranic schools, and newly elected local leadership not invested in the canteens and other activities. Recognizing that additional support was needed to ensure sustainability, project staff held three workshops at the national and regional levels in 2016 and 2017 to review the sustainability plan as originally proposed. They have since revised the plan, and are focusing on advocating mayors’ offices (mandated to support school development under Malian decentralization law) to include financial support for school feeding and teacher training in their development action plans. “The project meets some needs such as food at school, the quality of teaching, the rise of school enrollment and attendance; but a lot of things remain – access to water in school, food security during the drought, security of children on their way to school.” –SMC member IMPAQ International, LLC Page 79 Mali MGD Midline Report July 2018 SECTION 7. IMPLICATIONS FOR THEORY OF CHANGE AND PROJECT OPERATIONS Section 5 explained in detail the observed changes in outcomes since baseline (2016) by analyzing the performance evaluation data, collected by IMPAQ. Section 6 also provided a detailed analysis of qualitative interviews with national stakeholders, project implementers, local administrative and education officials, and parents and SMC members with regards to the relevance, effectiveness, efficiency, impact, and sustainability of the MGD III. In this section, we discuss the implication of those qualitative and quantitate findings for theory of change and project operations. By looking at the measured outputs and outcomes with regards to the theory of change (Appendix A), the observed changes in the performance evaluation data suggest that the project is moving towards the intended objectives, especially SO1 (i.e., improving literacy of school-aged children). In summary: • Providing BLA training to teachers, which is one of the key program interventions, was administered as planned to achieve MGD 1.1.342 and MGD 1.1.443 outcomes. These trainings also appear to have translated into actual application in the classroom, based on self-reported data by the teachers. Most teachers mentioned that they used at least one BLA technique in the classroom, and almost all teachers also reported using BLA equipment in the previous month in their classroom. Qualitative data confirms this, as local education administrators said that the BLA approach was helping children read at an earlier age than previously in their community. • There is evidence of the community understanding the benefits of education (MGD 1.1.5). Qualitative interviews show that both parents and education officials are more invested in children’s education, and parents in particular mentioned taking an active involvement in encouraging their children at school, through participating in SMCs, telling their children to do homework, and meeting with teachers. • At midline, a significantly lower proportion of students reported eating breakfast or a snack at home before going to school, especially in Mopti. However, almost all students ate lunch, similar to baseline. Although the quantitative data could not show where these children ate lunch, almost everyone in qualitative interviews said that the canteens played a crucial role in pupils’ attendance at school, facilitating learning (MGD 1.2.1.1 and 1.3.1.144). Parents and local administrators said that before the project, pupils were obliged to return home for lunch, and as many live far from school, they were tired and unable to return in the afternoon. Having lunch at school means more students stay for the entire day. • CRS’s attendance data were not available45 for baseline (2015-2016 academic year), thus assessing the changes on students’ attendance and enrollment over time was not be possible. However, the data show high enrollment, and consistent attendance for both boys and girls (MGD 1.3.4) in 2017-18 academic year, compared to the previous year (2016-2017). These results may also suggest improvement in communities’ understanding of educational benefits (MGD 1.3.5), especially for girls. However, the attendance data collected from teachers’ attendance logs should be interpreted with caution. It is because teachers may not routinely record attendance data. Even when they do and with the best intent, it is possible that teachers neglected to record missing children. 42 Improved literacy instructional materials 43 Increased skills and knowledge of teachers 44 Increased access to food (school feeding) 45 The implementation has started in October 2016 after the completion of baseline evaluation (May 2016). IMPAQ International, LLC Page 80 Mali MGD Midline Report July 2018 • The proportion of SMC members who received training since 2016 grew remarkably in both regions. They most frequently received training in food management, health, hygiene, and nutrition, and roles and responsibilities of the SMC. Moreover, the data show that SMC’s knowledge of good practices for food preparation and storage also increased greatly in both regions. These results suggest that the related activities were implemented as planned toward achieving MGD 2.1 outcome.46 However, these changes should be interpreted with caution for the performance evaluation. A new sample of individuals47 were selected at each data collection point (baseline and midline). Therefore, comparing mean outcomes at baseline and midline by using t-tests can only suggest a correlation between the observed changes and MGD III interventions. We cannot conclusively determine whether the interventions caused the changes. On the other hand, with regards to improved quality of literacy instruction (MGD 1.1) through BLA teacher training, our cohort comparison quasi-experimental design shows a clear proof of program impact on children’s literacy outcomes. The data show that students’ literacy outcomes improved by one level due to 2 years of exposure to a BLA-trained teacher and these outcomes increased by more than 1 level and a half due to 3 years of exposure to BLA-trained teachers. These findings are robust to adding other covariates. Next section will provide our recommendation based on these observed changes and our lessons learned. 46 Improved knowledge of safe food preparation and storage practices. 47 Exceptions to the new sample selection are described in detail in Section 4. IMPAQ International, LLC Page 81 Mali MGD Midline Report July 2018 SECTION 8. CONCLUSION This report presents the midline performance and impact evaluation results of the McGovern-Dole International Food for Education and Child Nutrition III project in Koulikoro and Mopti. The evaluation highlights changes in outcomes related to the core program objectives and allows us to set the stage for a fuller-examination of the program effects at endline. To accurately reflect program performance over time, we followed the same performance evaluation methodology and sampling strategy used during the baseline to generate an appropriate comparison with the midline using USDA guidelines. Our findings are based on data collected from: 1) surveys of students, parents, teachers, SMC members, and school teachers and principals, 2) reading assessment scores, 3) focus group discussions, and 4) key informant interviews. For our quantitative analysis, we employed pre-post comparison and cohort-comparison quasi-experimental methods in addition to descriptive data analysis. We collected data on more than 500 variables from 2, 506 primary school students, 2, 465 caregivers, 189 teachers, 44 school principals, and 45 SMC members. This section summarizes key findings in response to the main research questions, highlights study limitations, and provides recommendations for the overall project and the evaluation. 8.1 Key Findings We provide below the key findings from the performance evaluation related to students’, caregivers’, teachers’, and principals and SMC members’ knowledge of health and hygiene, results related to food security and dietary diversity, and findings related to the impact evaluation on children’s literacy outcomes. 8.1.1 Students’ Outcomes We find an overall improvement in the reading ability of children across all grades, with no significant regional or gender differences. A significantly greater proportion of students in first grade (13 and 9 percentage points for girls and boys, respectively) could read simple sounds; second grade (12 and 14 percentage points for girls and boys, respectively) could decode simple words; third grade (20 and 23 percentage points for girls and boys, respectively) could read simple sentences, all statistically significant at the 1 percent level, with no big changes for girls and boys in fourth grade. Moreover, the program interventions also seem to be related to children’s reported reasons for liking to attend school. Children in Koulikoro reported liking school and their classrooms because of the useful skills they acquired and the ease of understanding lessons. In Mopti, children mentioned having access to food and the ease of understanding as their key reasons for liking to attend school. While the findings reveal an improvement in literacy and school attendance outcomes at midline, the progress on hygiene practices, which was the focus of MGD I and II activities, appears to be mixed, with wide variation across the two program sites. Progress on hygiene practices, such as washing hands, seems to be even slower. The average proportion of students who reported washing hands at critical moments is lower as compared to baseline. Moreover, the analysis also reveals that knowledge about handwashing practices, on average, did not change at midline as compared to baseline. While the actual reported washing of hands is lower possibly due to lack of proper infrastructure and facilities in the school, the lack of change in knowledge about handwashing practices is worth noting for strengthening program efforts. In the endline evaluation, we will also focus our efforts in understanding the reasons behind these changes in hygiene practices among students. IMPAQ International, LLC Page 82 Mali MGD Midline Report July 2018 8.1.2 Parents Outcomes At midline, we find an improved engagement of parents in their children’s education. Improved participation of parents for their children’s education is evidenced in several ways. First, parents’ awareness about the school notice boards increased and a higher proportion of parents received colored report cards. Moreover, the average number of caretakers who attended a school general assembly increased in Koulikoro but not in Mopti. Presumably, other concerns such as safety prevented caregivers from going to the school for general assemblies in Mopti. In contrast to the progress on improved engagement by parents for their children’s education, the analysis suggests a lack of change in the involvement of parents in their children’s preventative healthcare activities, such as providing iron and vitamin A supplements. It is noteworthy that the data collection was conducted before the distribution of the mineral supplements, which is scheduled in November and May of each project year. Many parents cited that healthcare activities were too expensive or were not available or accessible. Lack of progress on these healthcare outcomes suggests that deeper, structural financial and geographical constraints might be contributing to a lack of progress on these outcomes. However, the project was not designed to focus on health care structures. MGD III aims to facilitate the distribution of the mineral supplements (e.g., vitamin A, iron, and deworming medications). The project also supports raising parents’ awareness about preventive health services by inviting local community health center staff in school to provide the communities with the appropriate health information. At midline we also investigated decision-making within households and found differences in perceptions on decision-making for children’s education within the household. A majority of men reported themselves as being the primary decision-maker for children’s education. However, women suggested a relatively lower role of men in their children’s education. Women felt that they play a relatively higher role vis-à-vis men’s in their children’s education – this is indicative of women exercising some say in their children’s education. However, both reports confirm that men tend to dominate the decision when it pertains to their children’s education. We also find no changes in parents’ aspirations for their children’s future between midline and baseline, suggesting the long period of time it takes to alter these deep-seated preferences of individuals. 8.1.3 Teachers and Principals’ Outcomes Our survey results suggest that one of the key program interventions – providing BLA training to teachers – was administered as planned. Almost 100 percent of 1st to 3rd grade teachers in Koulikoro and 93.3 percent of teachers in Mopti received BLA teacher training. Moreover, over half the number of these teachers had received approximately 2-3 BLA trainings. These trainings also appear to have translated into actual application in the classroom, based on self-reported data by the teachers. Most teachers mentioned that they used at least one BLA technique in the classroom, and almost all teachers also reported using BLA equipment in the previous month in their classroom. Our results also suggest there were no key differences in handwashing practices by teachers between baseline and midline. However, this difference is being driven by the already high knowledge and reported practice of handwashing among teachers during both baseline and midline. IMPAQ International, LLC Page 83 Mali MGD Midline Report July 2018 8.1.4 SMCs’ Outcomes Improvements in the performance of SMCs were notable across several dimensions. First, almost all SMC members managed school canteens at midline. Many of these SMC members had also received training and many different kinds of trainings. Second, a higher proportion of SMCs monitored the practices of teachers, especially through classroom observations. Third, these SMCs also followed student progress through monitoring the colored bulletins. However, one of the key outcomes where our SMC outcomes showed mixed results were the deployment of school action plans. While in Koulikoro there was a 20 percentage point increase in the SMCs that prepared a school action plan, there was a 12 percentage point reduction in Mopti. It is quite possible that political instability and strife in Mopti diminished the continuous operation of SMCs. As revealed in the interviews in Mopti with SMC members, illiteracy and challenges in filling forms for the plans was mentioned as one of the key constraining factors. 8.1.5 Impact Outcomes Our cohort comparison quasi-experimental design shows a clear proof of program impact: children’s literacy outcomes improved by one level due to 2 years of exposure to a BLA-trained teacher and these outcomes increased by more than 1 level and a half due to 3 years of exposure to BLA-trained teachers. These findings are robust to adding other covariates. However, we do not find any evidence of spillover effects indicating that the program is successful in improving reading abilities of children due to intensive, prolonged exposure to more effective and well-trained teachers. The statistically significant gains in literacy associated with the program intervention is the main critical finding of this report. 8.1.6 Qualitative Findings Summarized below are the main qualitative findings by relevance, effectiveness, efficiency, impact, and sustainability. Relevance:  Local stakeholders (parents, SMC members, and local administration officials) agreed that the project aligned with the priorities of their communities. They noted that food and take-home rations addressed food insecurity, as parents do not have to provide food (or money for food) and student attendance, as students are motivated to attend school because of the lunch and/or THR.  Elements of the original MGD project seemed to be overly ambitious, including the responsibilities of local implementing partners and the scope of the SMC training. As a result, some activities were delayed and/or adjusted to better align with local capacity.  External factors affected the original design of the MGD project in terms of the planned objectives, outcomes, targets, and timeframe. The most prevalent factor out of the project team’s control was the closing of schools due to political insecurity. Other factors include MONE’s decision to promote syllabic teaching approach across the country, including the project area with BLA training, frequent teacher strikes, and students moving from public schools to Koranic schools. IMPAQ International, LLC Page 84 Mali MGD Midline Report July 2018 Effectiveness:  Parents and local education and administrative officials confirmed that the literacy of school-aged children is improving. Parents appreciate having the color-coded reports to monitor their children’s progress.  Program staff and partners attributed improved literacy to the BLA pedagogical techniques, including the training and involvement of teachers, principals, pedagogical advisors, and local education offices.  Teacher turnover, low attendance, and lack of motivation have negatively affected the implementation of the BLA pedagogical techniques. Also, teachers seemed overwhelmed by the additional work due to the project, especially when they are tasked with assisting illiterate SMCs.  Immigration affects student enrollment, as in some communities, boys leave their villages to pursue opportunities abroad where they can send money home, while girls find work in big cities to support their wedding trousseaus.  Lack of water on school grounds prevents full effectiveness of the health and dietary objectives. No or limited water sources do not allow for handwashing, and also prevent SMC members from growing vegetables to supplement the canteen. Efficiency:  While overall program staff and partners reported that the project resources are being used efficiently, school closures and some adjustments to the project activities in response to changes in beneficiaries’ need have caused minor impediments in allocating budget resources.  Local administrative officials reported that in some communities there were political differences within the local education sector, specifically between the mayors and community leaders led to conflicts around the management of the schools. In other communities, the mayors often relocated teachers who were active in program schools without consideration for the needs of those schools.  External factors such as school closure and teachers’ strike, combined with the expiration of some of the commodities such as vegetable oil, made the food distribution difficult for the program. Impact:  Local and national stakeholders agreed that the BLA has a positive impact on improving children’s literacy.  Parents and SMC members found that school lunches and THR have a positive impact in their communities, as they address both hunger and school attendance.  Project staff said that creating SILC groups is having a positive impact for communities, as participants are able to financially manage canteens.  Parents and SMC members said that activities requiring water are not impactful, as they are not able to grow school gardens with the limited water supply. IMPAQ International, LLC Page 85 Mali MGD Midline Report July 2018  Community contributions to the canteens are less impactful than hoped for, as many families are too financially insecure to provide food or money. Sustainability:  The knowledge gained through BLA trainings and SILC/SMC groups will last longer than the program. However, transient teachers and community members mean that this knowledge may not stay in targeted communities.  Although local stakeholders said that they will continue the canteens after the program leaves, they acknowledged that they will not be able to provide the same quantity and quality of food.  Program staff expressed concern that local stakeholders lack capacity to take over the activities when CRS leaves. However, they are taking steps to revise the sustainability plan with a particular focus on the active support and involvement of the mayors’ offices, who are mandated to support school development under Malian decentralization law. 8.2 Limitations There are some limitations of the study that are worth noting:  An important limitation of the study is that it relied on self-reported data for a number of socially and culturally sensitive subjects, such as food consumption, hygiene practices, and student attendance. This self-reported data should be interpreted with caution and is particularly susceptible to social desirability bias. To help counter biases related to hygiene practices, we integrated observational data of children’s handwashing practices (at critical moments) at the school. However, this method has its limitations as many children did not eat or use the latrine during our school visit.  As described in Section 5, at each data collection point (baseline and midline), we selected new samples of individuals to survey. A limitation of this approach is that there might be systematic differences in the two sets of samples that could confound the observed changes in the outcomes of interest over time. Inspection of the data indicate that the baseline and endline samples were similar along many demographic covariates, except for household size, and accessibility to basic services, parents’ and teachers’ educational attainment, and literacy training received by teachers.  Another limitation is related to the small sample sizes for some indicators, in particular those related to the SMC and survey administrators’ outcomes. It is important to note that the results presented in those sections rely on very small sample sizes which reduce the accuracy of the indicators measured and the magnitude and direction of observed changes over time needs to be interpreted with caution.  Our impact evaluation design is a quasi-experimental methodology, which also rests on the assumptions that the time effects experienced by grade 1, 2, and 3 students are the same. Moreover, if any external policy changed the probability of participation in the program differentially for the grades, then the quasi-experimental methodology may not depict an accurate picture of the program impact on literacy. IMPAQ International, LLC Page 86 Mali MGD Midline Report July 2018  Our sample of schools from baseline to midline was also reduced to political instability in the country. As noted throughout the program report, we find differences in outcome achievement between Koulikoro and Mopti. It is quite possible that implementing the program as effectively in other Mopti schools was also challenging, and therefore, our analysis may be suggesting lack of improvement not because the program is not working but because of political strife in the region. 8.3 Recommendations We present the following recommendations to CRS based on both lessons learned from our experience in the field and our findings after analyzing the collected data. The recommendations are not intended to address all challenges identified through the study; instead they focus on the main drivers of project success.  Improve awareness among parents and children about basic hygiene practices. As the performance evaluation revealed, there is little progress on the adoption of hygiene practices by all the respondents, including children, parents, teachers, and SMC members. Although WASH related activities were only part of MGD II and the changes in these outcomes can be attributable to external factors such as lack of water sources, the respondents’ knowledge and their self￾reported hygiene practices are low with a gap between them. The project’s staff and partners should investigate this issue in further detail and adjust the focus of the project’s activities’ efforts, perhaps by SMC and teachers, to inculcate a culture of using basic hygiene practices by children. If children learn these in the school, it is quite possible that these practices may spillover in their homes as well.  Continue to provide and improve BLA trainings. The impact evaluation showed strong and significant improvement in students’ literacy outcomes due to exposure to BLA-trained teachers. The project activities should continue building on their level of efforts in training teachers and ensuring that teachers intensify the use of BLA techniques in the classroom.  Work on the school environment. Students cited the bad attitude of their teachers (e.g., hitting, yelling, harassing, and/or under-estimating children) as one of the most frequent reasons for what they do not like about their school and classroom. As recommended at baseline, the project should not only focus strengthening teachers’ pedagogical practices, but also on improving teachers’ attitudes and the school environment (explicitly addressing bullying) to create an atmosphere conducive to learning for students.  Target both male and female caregivers for improving outcomes such as children’s school attendance. The study also highlighted that men play a key role in decision-making on children’s educational attainment. Any long-run, sustained program impact would also work towards changing the attitudes of men towards the educational attainment of their children. Strategies specific to men can include sensitizing them as partners or allies, with common goals for the welfare of the family. Even in patriarchal or religious/Islamic communities, activities can focus on reinforcing the positive emotions of men as caregivers of their daughters.48 Since the program’s goal is to improve children’s health and educational outcomes, including both female and male caregivers will be critical for meeting the program’s objectives, especially improving the children’s school attendance and reducing their dropout rates. 48 http://www.un.org/womenwatch/daw/egm/men-boys2003/documents.html IMPAQ International, LLC Page 87 Mali MGD Midline Report July 2018  Scale-up activities related to savings and internal lending. The SILC groups appear to offer the greatest path to sustainability after the project finishes, evidenced through CRS monitoring data and through the qualitative interviews with national and local stakeholders. The financial knowledge gained will last after the resources end, and several communities without SILC groups requested financial literacy training, knowing that this could help them maintain the canteens. Survey data also show that participation in the SILC groups helped parents in saving money for their children’s school fees as well as improving their livelihood, including their income, assets, and food security. Future iterations of the project many want to include livelihood activities to expand the benefits of SILC group participation beyond the project objectives.  Increase infrastructure-building to keep pace with awareness building, or temper program expectations. Awareness building appears to be happening faster than infrastructure building. For example, parents are aware of the importance of preventative health activities such as pre￾natal care and vaccinating their children, but the expense involved, or the distance required to access care is too great for many families. Another example is many canteen activities require water, but without access to a regular water source, these activities cannot be completed.  Include lessons on collaboration for teachers to increase the sustainability of BLA. Our impact evaluation shows that learning from BLA-trained teachers improves students’ literacy outcomes. The lack of spillover effects within the same school (that is, no improvement in literacy for students in the same school taught by non-trained teachers) is additional evidence that sustained exposure to a trained teacher is the mechanism for students’ improvement. However, the lack of increase for students in other classes also shows that teachers are not communicating or collaborating within their schools. Teachers are not sharing the techniques learned through BLA with other teachers. Because teacher turnover is so high, this threatens the sustainability of students’ gains in literacy. The program could add activities on school-wide collaboration, where teachers and principals share and practice what they’ve learned through BLA with the whole school.  Consider incentives for teachers burdened with additional work. Project staff reported that teachers are overwhelmed with extra managerial work related to the program (for example, inventory report, food delivery forms, etc.) In some schools, teachers are asked to help illiterate SMCs who are unable to complete school action forms and other required paperwork on their own. Project staff and/or partners should consider talking with teachers to get a better understanding of the extra burden involved and what could encourage adding extra duties. It may be that small grants or teacher recognition could help motivate those teachers who are reluctant to take on this extra work.  Put in place the revised strategy for sustainability, and include indicators to measure progress. Several activities will require external support to continue after the program ends. For example, in many communities, the amount of food families can contribute varies heavily from year to year based on climate. Teacher and SMC turnover also threaten sustainability, as without continuous training, new teachers and SMC members will not have the knowledge and skills to use BLA and run the canteens. CRS Mali’s revised strategy for sustainability recognizes that communities need additional government support, and therefore should advocate mayors’ offices (mandated to support school development under Malian decentralization law) to include financial support for school feeding and teacher training in their development action plans. Monitoring progress towards the sustainability plan will help determine which strategies show promise and which need further revision. IMPAQ International, LLC Page 88 Mali MGD Midline Report July 2018 REFERENCES Food and Agriculture Organization (2010). Guidelines for Measuring Household and Individual Dietary Diversity. Rome, Italy: United Nations. Pratham. (2015). Pratham Around the World. Retrieved from http://www.prathamusa.org/programs/pratham-around-world. United States Department of Agriculture (USDA). (2012). Food for progress and McGovern-Dole indicators and definitions. Washington DC: Food Assistance Division, Office of Capacity Building, Foreign Agricultural Service. United States Department of Health and Human Services (USDHHS). Human Subject Regulations Decision Charts. Retrieved from http://www.hhs.gov/ohrp/policy/checklists/decisioncharts.html (2016, February 16). United States Agency for International Development (USAID). (2012). 2011 USAID education strategy reference materials. Washington, DC. Vagh, S.B. (2009). Evaluating the reliability and validity of the ASER testing tools. New Delhi: ASER Centre. Vanmeenen, G. (2010). Savings and Internal Lending Communities (SILC) - Voices from Africa. Nairobi: Catholic Relief Services. IMPAQ International, LLC Page 89 Mali FFE Midline Report July 2018 APPENDICES A. McGovern-Dole Results Activities and Frameworks B. McGovern-Dole Project Indicators C. Additional Tables and Complementary Outcomes D. Evaluation Instruments IMPAQ International, LLC Page A-2 Mali FFE Midline Report July 2018 MGD 1.2: Improved Attentiveness MGD 2.1: Improved Knowl edge of Sa fe Food Prepa ra ti on a nd Stora ge Pra cti ces MGD 2.5: Increased Acces s to Preventa ti ve Hea l th Interventi ons MGD 1.1.1: More Consistent Tea cher Attendance MGD 1.1.3: Improved Literacy Ins tructi ona l Ma teri a l s MGD 1.1.4: Increa s ed Ski l l s & Knowl edge of Tea chers MGD 1.1.5: Increa s ed Ski l l s & Knowl edge of School Admi ni s tra tors MGD 1.2.1: Reduced Short￾Term Hunger MGD 1.3.1: Increased Economi c & Cul tura l Incenti ves (Or Decrea s ed Di s i ncenti ves ) MGD 1.3.2: Reduced Health￾Related Abs ences MGD 1.3.3: Improved School Infrastructure MGD 1.3.4: Increa s ed Student Enrol l ment MGD 1.3.5: Increased Community Unders ta ndi ng of benefits of education Output: Tea cher recogni ti on Output: Di s tri buti on: Improved Literacy Ma teri a l s Output: Tra i ni ng: Tea chers Output: Tra i ni ng: School Admi ni s tra tors MGD S02: Increa s ed Us e of Hea l th & Di eta ry Practices CRS a s s i s t SMC to acces s funds through ma yors ’ office to meet s chool needs Output: Enrol l ment Ca mpa i gns Output: Tra i ni ng: School Ma na gement Committees Tra i ni ng: Commodi ty management, Food Prepa ra ti on a nd Stora ge Pra cti ces Di s tri buti on: De￾wormi ng medi ca ti on, vi ta mi ns & mi nera l s Output: Tea cher recogni ti on Output: Student Recognition Foundational Results MGD SO2: Increa s ed Us e of Hea l th a nd Di eta ry Practices Output: Ca pa ci ty Bui l di ng: Loca l , regi ona l , na ti ona l l evel MGD 1.4.1: Increa s ed Ca pa ci ty of Government Ins ti tuti ons MGD 1.4.3: Increa s ed Government Support Output: Ca pa ci ty Bui l di ng: Loca l , regi ona l , na ti ona l l evel Output: Ta ke Home Ra ti ons MGD 1.4.4: Increa s ed Enga gement of Loca l Orga ni za ti ons a nd Communi ty Groups Output: Form Sa vi ngs a nd Lendi ng Groups Output: Tra i ni ng: Government Offi ci a l s Output: Tra i ni ng: School Ma na gement Commi ttees MGD 1.2.1.1, 1.3.1.1: Increased Access to Food (School Feeding) Output: Provi de School Mea l s MGD 1.1: Improved Qua l i ty of Li tera cy Ins tructi on MGD 1.3: Improved Student Attenda nce MGD SO1: Improved Li tera cy of School -Age Chi l dren IMPAQ International, LLC Page D-1 Mali FFE Midline Report July 2018 APPENDIX D. EVALUATION INSTRUMENTS ASER Reading Assessment ASER Test Administration Instructions Student Survey Caregiver Survey Teacher/ Principal Survey SMC Survey Key Informant Interview Protocol Focus Group Discussion Protocol IMPAQ International, LLC Page D-2 Mali FFE Midline Report July 2018 ASER Reading Assessment ASER Test Administration Instructions IMPAQ International, LLC Page D-3 Mali FFE Midline Report July 2018 IMPAQ International, LLC Page D-4 Mali FFE Midline Report July 2018 IMPAQ International, LLC Page D-5 Mali FFE Midline Report July 2018 IMPAQ International, LLC Page D-6 Mali FFE Midline Report July 2018 BASELINE EVALUATION OF THE CRS FOOD FOR EDUCATION PROJECT IN MALI Student Survey BASE INFORMATION Enum_id Surveyor ID Date Date (JJ/MM/AAAA) Region Write the name of the region CAP Write the name of the Educational Animation Center schoolname Write the name of the school Select studentname_id Indicate the name and Unique Identity (ID) of the student CODE I__I__I__I__I__I__I__I__I grade Indicate the class of the respondent : (Indicate the class of the student that is written on your school sheet) 1. 1st year 2. 2nd year 3. 3rd year 4. 4th year I__I Dear Student: You have been selected to participate in a survey about health, nutrition, and education for the Cantine Scolaire (School Canteen) project. Your participation in this interview is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. If you accept, please respond to all questions as candidly as possible. If you do not know the answer to a question, you may simply say so. All responses will be kept strictly confidential. IMPAQ International, LLC Page D-7 Mali FFE Midline Report July 2018 consent Do you accept participation in this survey? 1. No  thanks 2. Yes  fname 88. Not found  thanks I__I *Select only one option If the response to “consent” is No or Not Found, thank the respondent and terminate the survey N.B: If the respondent refuses to answer any particular question, mark an « R » as the answer and move on the next question. Personal Information Great! Now, I will ask you a few questions about yourself… fname What is your first name? lname What is your family name? primecaren ame What is the first name of your mother OR responsible party/principal guardian? …………………………………………………………………………. primecarela st What is the last name of your mother OR responsible party/principal guardian? …………………………………………………………………………. Age1 Do you know your age? 0. No  “age2” 1. Yes  “gender” 88. Don’t know I__I *Choose only one option Age2 How old are you? …… *AGE INTERVAL from 4 to 19 *Write down -99 if the respondent refuse to answer Gender Are you a boy or a girl? 1. Male 2. Female I__I *Ask only if necessary *Select only one option newsch When you started at this school, in which grade were you? 1. 1st year *Read the list, but don’t read “Don’t know” IMPAQ International, LLC Page D-8 Mali FFE Midline Report July 2018 2. 2nd year 3. 3rd year 4. 4th year 88. Don’t know Class What is your current school year? 1. 1st year 2. 2nd year 3. 3rd year 4. 4th year I__I *If the student class is different from that noted on your student sheet, please check with the class master of the student *Select only one option Repeat1 Did you repeat a year? 0. No 1. Yes *Selection only one option Repeat2 Which class(es) did you repeat? 1. 1st year 2. 2nd year 3. 3rd year 4. 4th year *Choose all answers that apply schoolday When was the last day that you went to school? 1. Yesterday 2. Last Monday 3. Last Tuesday 4. Last Wednesday 5. Last Thursday 6. Last Friday 7. Last Saturday 8. It has been more than a week I__I *Select only one option IMPAQ International, LLC Page D-9 Mali FFE Midline Report July 2018 Schoolday2 How many days did you miss school last week? 0. None 1. 1 day 2. 2 days 3. 3 days 4. 4 days 5. All week 88. Don’t know I__I *Select only one option Environment and Participation at School Very good! Now, I would like to ask you a few questions about your school… Enviro1 a_new What do you like about your classroom and school? 0. Nothing 1. The lessons are easy to understand 2. The teacher teaches well 3. The teacher is kind and helpful 4. Learn useful skills and knowledge 5. Participate in activities/class games 6. Food is provided 7. Access to water 8. Access to good latrines that are in good condition and clean 9. Doing sport at school 10. Other (Specify :________________________) 88. Don’t know I__I I__I I__I I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply Enviro2 a_new What do you not like about your classroom and school? 0. Nothing 1. Teachers hits, yells, harasses, and/or under-estimates 2. Lessons are difficult to understand 3. Don’t learn interesting things/it is boring 4. Teacher is often not present at school 5. Lack of learning materials: that is books, board, etc. 6. School too far I__I I__I I__I I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply IMPAQ International, LLC Page D-10 Mali FFE Midline Report July 2018 7. Bad sanitary hygiene in the toilets, lack of toilets 8. Food provided is bad, no food provided 9. No access to water 10. Other students tease/intimidate me 11. Lack of clothes/uniforms 12. Other (Specify :________________________) 88. Don’t know I__I I__I Enviro3 Usually, does the master ask you questions during the class lesson? 1. Often 2. Sometimes 3. Rarely 4. Never 88. Don’t know I__I *Read the list to the respondent but don’t read « don’t know » *Select only one option Enviro4 Usually, do you try to answer the questions of your class master? 1. Often 2. Sometimes 3. Rarely 4. Never 88. Don’t know I__I *Read the list to the respondent but don’t read « don’t know » *Select only one option Enviro5 Usually, do you do lessons with the radio? 1. Often 2. Sometimes 3. Rarely 4. Never 88. Don’t know I__I *Read the list to the respondent but don’t read « don’t know » *Select only one option Enviro6 Usually, do you do the news of the class (that is, the master asks you what you did the previous day and you tell them how to write it on the board)? 1. Often 2. Sometimes 3. Rarely 4. Never 88. Don’t know I__I *Read the list to the respondent but don’t read « don’t know » *Select only one option IMPAQ International, LLC Page D-11 Mali FFE Midline Report July 2018 Enviro7 Usually, do you do class games? 1. Often 2. Sometimes 3. Rarely 4. Never 88. Don’t know I__I *Read the list to the respondent but don’t read « don’t know » *Select only one option Enviro8 Usually, does the master asks you to write about a subject of your choice? 1. Often 2. Sometimes 3. Rarely 4. Never 88. Don’t know I__I *Read the list to the respondent but don’t read « don’t know » *Select only one option Enviro9 Usually, does the master allow you to read a text of your choice? 1. Often 2. Sometimes 3. Rarely 4. Never 88. Don’t know I__I *Read the list to the respondent but don’t read « don’t know » *Select only one option Enviro1 0 Usually, is there someone at home who reads books to you? 1. Often 2. Sometimes 3. Rarely 4. Never 88. Don’t know I__I *Read the list to the respondent but don’t read « don’t know » *Select only one option Enviro1 1 Usually, do you read books for fun (that is, not required as homework)? 0. No 1. Yes I__I *Select only one option Project 1 What do you like about this project? 0. Nothing 1. Food 2. Rations to take home 3. the medicines (deworming, vitamin A, etc.) 4. Rewards 5. Radio/reading writing 6. The games 7. Other (Specify : ___) I__I I__I I__I I__I I__I I__I *Refer to the program that CRS is implementing in their school *Select all the responses that apply IMPAQ International, LLC Page D-12 Mali FFE Midline Report July 2018 Project 2 What do you not like about this project? 0. Nothing 1. Food (quality and/or quantity) 2. Rations to take home 3. the medicines (deworming, vitamin A, etc.) 4. Rewards 5. Radio/reading writing 6. The games 7. Other (Specify :____) I__I I__I I__I I__I I__I I__I *Refer to the program that CRS is implementing in their school *Select all the responses that apply Hygiene Thanks! I would like to ask you a few questions regarding hygiene… handwash In your opinion, when should someone wash their hands? 1. Before eating 2. Before touching or preparing food 3. Before giving food to someone 4. When hands are dirty 5. After touching a dirty object 6. After touching a pet 7. After using the latrines 8. After changing a baby’s diapers 9. Before prayers 10. Other (Specify :_________________________) 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply Hand1 How many times did you wash your hands yesterday? ……. *Interval between 0 and 20 *If 0, move to « Hand8 » Hand2 What were the reasons? 1. Before eating 2. Before touching or preparing food 3. Before giving food to someone 4. When hands are dirty 5. After touching a dirty object 6. After touching a pet I__I I__I I__I I__I I__I I__I *Do not give examples or read the list to the respondent IMPAQ International, LLC Page D-13 Mali FFE Midline Report July 2018 7. After using the latrines 8. After changing a baby’s diapers 9. Before prayers 10. Other (Specify :_________________________) 88. Don’t know I__I I__I I__I I__I *Choose all answers that apply Handa8 What do you usually use to wash your hands? 1. Just water 2. Water plus soap 3. Other (Specify :_____________________) I__I * Do not give examples or read the list to the respondent *Select only one option Worms In your opinion, how can one avoid to catch intestinal worms (in the belly)? 1. Avoid walking bare foot (wear shoes) 2. Do not bathe or swim in stagnant water 3. Eat meat that is well cooked 4. Avoid contact with contaminated water, and if necessary to wear boots and gloves 5. Wash hands with drinking water and soap before preparing food, before serving food or before eating 6. Wash hands with drinking water and soap after using the latrines 7. Protect food from flies, cockroaches, and dust 8. Store food in a pantry or a place that is clean and well-ventilated 9. Other (specify :______________________) 88. Don’t know I__I I__I * Do not give examples or read the list to the respondent *After the respondent has given one way, incite the respondent to give a second way: What other way can help avoid intestinal worms? Incite for 2 ways in total Health Thanks! Now, I would like to ask you a few questions about health… IMPAQ International, LLC Page D-14 Mali FFE Midline Report July 2018 Health1 During the last two weeks, have you been sick? 0. No  fs1 1. Yes  health1a I__I *Select only one option Health1a What did you have? 1. Diarrhea 2. Vomiting 3. Fever 4. Other (Specify :_________________) I__I I__I I__I I__I *Choose all answers that apply Health2 During the last two weeks, have you missed school because you were sick? 0. No  fs1 1. Yes  health3 I__I *Select only one option *If 1, move to ‘’health3’’ *If 2, move to ‘’fs1’’ Health3 During the last two weeks, how many days have you missed school because you were sick? 1. None 2. 1-3 days 3. 3-5 days 4. More than 5 days I__I *Select only one option Food Security Now, I would like you to think about all the meals that you’ve eaten yesterday… Fs1 According to you, was yesterday ‘ordinary/usual’ or was it a special occasion? 1. Ordinary/Usual 2. Special occasion (specify :____________________) I__I *Give examples of special occasions such as funeral and parties *Choose only one option Fs2 Now thinking about what you did yesterday, did you eat something before the morning meal? 0. No 1. Yes I__I *Select only one option Fs3 Yesterday, have you eaten something for the morning meal? 0. No  fs3 I__I *Select only one option IMPAQ International, LLC Page D-15 Mali FFE Midline Report July 2018 1. Yes  fs5 Fs4a Yesterday, were you satisfied after eating the morning meal? 1. I was satisfied  fs5 2. I could have eaten more  fs4b I__I *Select only one option Fs4b Yesterday, why didn’t you eat more food? 1. There was no food left 2. There was not anything that I like 3. Other (Specify :________________________) I__I *Select only one option Fs5 Yesterday, did you eat something between the morning meal and the mid-day meal? 0. No 1. Yes I__I *Select only one option Fs6 Yesterday, did you eat something for the mid-day meal? 0. No  fs8 1. Yes  fs7 I__I *Select only 1 option Fs7a Yesterday, were you satisfied after having eaten the mid-day meal? 1. I was satisfied  fs8 2. I was not satisfied  fs7b I__I *Select only one option Fs7b Yesterday, why didn’t you eat more food? 1. There was no food left 2. There was not anything that I like 3. Other (Specify :________________________) I__I *Select only one option Fs8 Yesterday, did you eat something between the mid￾day meal and the evening meal? 0. No 1. Yes I__I *Select only one option Fs9 Yesterday, did you eat something for the evening meal? 1. No 2. Yes I__I *Select only one option Fs10a Yesterday, were you satisfied after eating the evening meal? 1. I was satisfied fs11 I__I *Select only one option IMPAQ International, LLC Page D-16 Mali FFE Midline Report July 2018 2. I could have eaten more  fs10b Fs10b Yesterday, why did you not eat more food in the evening? 1. There was no food left 2. There was not anything that I like 3. Other (Specify :________________________) I__I *Select only one option We just talked about all the meals that you have eaten yesterday. Now, I would like you to take a few minutes to reflect on all the foods and beverages that you have eaten during the day and the night, either at home, at school or outside of home and school. Are you ready? Ok. Fs11 Can you cite all the food and beverages that you gave to [student’s name] yesterday? 1. Millet, rice, corn, sorghum, manioc 2. Nuts or beans (such as cowpeas, green peas, lentils) 3. Yoghurt, milk or cheese 4. Meat or fish 5. Eggs 6. Red palm oil or fruits or vegetables (such as carrots, squash, sweet potatoes, dark green leafy vegetables leaves, ripe mango, melon, apricot, ripe papaya, peach, red peppers, moringa leaves) 7. Other fruits and vegetables such as onion, eggplant, watermelon, oranges, green peppers, cabbage, tomato, dates, etc. 8. Don’t know I__I I__I I__I I__I I__I I__I I__I *Note all the food and beverages mentioned by the respondent on a separated sheet of paper. If dishes are mentioned, ask for the list of ingredients of each dish mentioned. *When the respondent is done, ask the respondent to assure you that he/she has mentioned all the food and beverages given to the child yesterday, including all snacks. Use time references if necessary (morning, afternoon, evening, night). *Select all answers that apply. *For each food group not mentioned, ask the respondent if a food from this group has IMPAQ International, LLC Page D-17 Mali FFE Midline Report July 2018 been consumed yesterday. READING TEST Here is the last series of questions! I would like to play a small game with you… readassess To what level has the student read? 0. 0 1. A 2. B 3. C 4. D 5. E 6. F 7. G 8. H 9. I 10. J 11. K I__I *Indicate the level of reading according to the test *Select only one option thanks Thanks a lot for answering my questions Note to enumerator: Fill out the next few questions based on the conducted survey Note1_enu merator What language was the survey conducted? 1. French 2. Local language 3. A combination of both Note2_enu merator What is the main language of instruction in this school? 1. French 2. Local language88. 88. Don’t knowNot French but not sure Note3_enu merator Enumerator Comment IMPAQ International, LLC Page D-18 Mali FFE Midline Report July 2018 MIDLINE EVALUATION OF THE CRS FOOD FOR EDUCATION PROJET IN MALI Caregivers Survey BASE INFORMATION Enum_ID Surveyor ID Date Date (JJ/MM/AAAA) Region Write the name of the region CAP Write the name of the Educational Animation Center schoolname Write the name of the school Select studentid Indicate the Unique Identity (ID) of the student CODE I__I__I__I__I__I__I__I__I Village Write the name of the village where the mother/responsible party of the student lives Dear Mother: You have been selected to participate in a survey about health, nutrition, and education for the Cantine Scolaire (School Canteen) project. Your participation in this interview is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. If you accept, please respond to all questions as candidly as possible. If you do not know the answer to a question, you may simply say so. All responses will be kept strictly confidential. IMPAQ International, LLC Page D-19 Mali FFE Midline Report July 2018 Consen t Do you accept participation in this survey? 1. Yes  match 2. No  thanks 9. Not found  thanks I__I *Select only one option If the response to “consent” is No or Not Found, thank the respondent and terminate the survey N.B: If the respondent refuses to answer any particular question, mark an « R » as the answer and move on the next question. PERSONAL INFORMATION Fname What is your first name? ____________________________ Lname What is your family name? ____________________________ Biomo What is your relationship with: ______________ [name of the student]? 1. Biological parent  age1 2. Principal responsible party  age1 3. Secondary principal party  age1 4. Other  thanks I__ I *Definition of the principal responsible party: person who takes care of the child and look after his/her health, physical well￾being and social development, in the short or long term, either at her own residence or that of the child. *Definition of the secondary responsible party : person in the household who takes care of the child IMPAQ International, LLC Page D-20 Mali FFE Midline Report July 2018 If answer to “biomo” is Other, thank the respondent and terminate the survey Age1 Can you give me your age? 0. No 1. Yes I__ I *If Yes, move to “age2” *If No, move to “gender” *Choose only one option Age2 How old are you? …… *AGE INTERVAL between 12 and 99 *If do not know -99 Gender What sex are you? 3. Male 4. Female I__ I *Ask only if necessary *Select only one option edu What is the highest level of education you have completed? 1. None 2. Some primary school but did not complete primary school 3. Completed primary school 4. Some secondary school but did not complete secondary school 5. Complete secondary school 6. Some university but did not complete university 7. Earned bachelor’s degree 8. More than bachelor’s degree 9. Professional school 10. Koranic school or medersa I__ I *Select only one option Household Environment Good! Now, I would like to ask you a few questions on [name of the student] and on your household… Distance1 How long does it take for [name of the student] to get to school? …. *In minutes -99 = if don’t know IMPAQ International, LLC Page D-21 Mali FFE Midline Report July 2018 Distance2 How does [name of the student] usually get to school? 1. On foot 2. Bicycle 3. Motorcycle 4. On the back of an animal 5. Public transport (bus, cab, cart) 6. Other (specify:______________________) 88. Don’t know I__I *Do not give examples or mention the list to the respondent *Select only one option Latrine1 Do you have a latrine where you live? 0. No  water 1. Yes  latrine2 I__I *Select only one option Latrine2 What type of latrine is it? 1. Installation with a mechanical or manual flush linked to a sewer or septic system or pit 2. Ventilated pit latrine 3. Latrine with a pit that has a slab 4. Latrine with a pit that has no slab 5. Bucket latrine 6. Other (specify:________________________ ) I__I *Specify to the respondent that it is the main latrine used by most members of the household *Do not give examples or mention the list to the respondent *Select only one option water What is the main source of drinking water of your household? 1. Running water in the yard (tap) 2. Running water in the community (fountain) 3. Private well 4. Public well 5. Distribution of water by a tank 6. Natural source of water (lake/oxbow lake, river, stream, etc.) 7. Other (Specify:_____________________) 88. Don’t know I__I *Define « household » or give examples : It’s a group of people generally related by blood or marriage, usually living together, producing together, and whose socio-economic authority is theoretically under just one person called the head of the household *Do not give examples or mention the list to the respondent IMPAQ International, LLC Page D-22 Mali FFE Midline Report July 2018 *Select only one option Elec1 Do you have electricity where you live? 1. Yes 2. No I__I *Select only one option *If No, move to “cookstove1” Elec2 For how many hours per day do you have electricity? 1. 0-1 hour 2. 1-3 hours 3. 3-5 hours 4. More than 5 hours I__I *Do not give examples or mention the list to the respondent *Select only one option hsize Including yourself, how many people are there in your household? …… *NB : the household could consist of only 1 member *INTERVAL from 1 to 60 Kid How many kids younger than 5 years old live in your household? …… *Interval from 0 to 30 Kid1 How many kids living in your household are old enough to go to school (6-15 years old)? …… * Interval from 0 to 30 *If 0, move to “Book1” Kid2 Among these kids, are there some who don’t go to school? 0. No  dropout1 1. Yes  kid2a I__I *Select only one option Kid2a Among those school-aged children, how many girls don’t go to school (never enrolled in school)? …… *Not to exceed the number in kid1 *If no girls are out of schools put 0 Kid2b Among those school-aged children, how many boys don’t go to school (never enrolled in school)? …… *Not to exceed the number in kid1- kid2a *If no boys are out of schools put 0 Kid3_girls What are all the reasons why these school￾aged girls don’t go to school? 1. I can’t afford school fees I__I I__I I__I *Skip if the answer to kid2a is zero IMPAQ International, LLC Page D-23 Mali FFE Midline Report July 2018 2. My child(ren) is/are not intelligent/capable enough 3. I need my child(ren) to help me at home/in the field 4. I need my child(dren) to work in order to support the family 5. The school quality is poor (that is, the students don’t learn anything and/or the teachers are aggressive and/or are not present, etc.) 6. The school is not safe 7. Girls are not supposed to go to school 8. Other (Specify:_______________________) I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply Kid3_boys What are all the reasons why these school￾aged boys don’t go to school? 1. I can’t afford school fees 2. My child(ren) is/are not intelligent/capable enough 3. I need my child(ren) to help me at home/in the field 4. I need my child(dren) to work in order to support the family 5. The school quality is poor (that is, the students don’t learn anything and/or the teachers are aggressive and/or are not present, etc.) 6. The school is not safe 7. Other (Specify:_______________________) I__I I__I I__I I__I I__I I__I I__I *Skip if the answer to kid2a is zero *Do not give examples or read the list to the respondent *Choose all answers that apply Dropout1 Has anyone in your household dropped out of school since the beginning of this year, that is since October 2017? 0. No  book1 1. Yes  drop_out2 I__I *Select only one option Dropout2 If yes, how many boys and girls in your household have dropped out since the beginning of the school year that is since last October? *Enter the number, but it cannot to exceed kid1 – kid2a – kid2b IMPAQ International, LLC Page D-24 Mali FFE Midline Report July 2018 1. Boys : _____ 2. Girls : _____ Dropout3_ girls What were the reasons for girls in your household to drop-out of school? 1. School is too far 2. School fees 3. Domestic work 4. Agricultural/other work 5. Girl’s sickness 6. Pregnancy 7. The girl(s) are not interested 8. Too much punishment at school 9. Humiliation of classmates 10. They got married 11. Other (specify):__________________ 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I I__I I__I *Select all that apply Dropout3_ boys What were the reasons for boys in your household to drop-out of school? 1. School is too far 2. School fee 3. Domestic work 4. Agricultural/other work 5. Child’s sickness 6. Child was not interested 7. Too much punishment at school 8. Humiliation of classmates 9. Other (specify):__________________ 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I I__I I__I *Select all that apply Book1 Approximately, how many books do you have at home, excluding school books? 0. No book 1. 1-5 2. 6-10 3. 11-20 4. More than 20 I__I *Select only one option *Confirm that the respondent understands that this does not refer to the Coran but to story books Book3 In general, do you or another adult in your household read books for [student’s name]? 1. Often I__I *Select only one option *Read the list to the respondent but don’t IMPAQ International, LLC Page D-25 Mali FFE Midline Report July 2018 2. Sometimes 3. Rarely 4. Never 88. Don’t know read « don’t know » or « refused » Attendanc e1 How many days did [student’s name] go to class in the past week? …… *A normal school week between 0-5 Attendanc e2 What were the reasons that [student’s name] that they missed schools last week? 1. School is too far 2. School fee 3. Domestic work 4. Agricultural/other work 5. Child’s sickness 6. Child was not interested 7. Too much punishment at school 8. Humiliation of classmates 9. Other (specify):__________________ 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I *Only ask if Attedance1 is not equal to 5 Health Thanks! Now, I would like to ask you a few questions regarding the health of [student’s name]… Health1 Have you ever engaged in the following preventive health care activities? 1. Vaccination 2. Deworming 3. Diet supplement in iron 4. Vitamin A supplement 5. Growth check-up 6. Prenatal care 7. Other (specify:____________________) I__I I__I I__I I__I I__I I__I *Read the list to the respondent *Ask Health1a if the respondent does not check ALL the answers (1 to 5) for the Health1 question. *Choose all answers that apply * Define « growth check￾up » : Visits to a health-care professional to monitor child growth in the first few years of life in order to check for delays in development and any problem IMPAQ International, LLC Page D-26 Mali FFE Midline Report July 2018 Health1a For the preventive health-care activities of [student’s name] that you didn’t do, what held you back? 1. The healthcare activities are too expensive 2. The healthcare activities are not available/too far away/not provided in my community 3. The healthcare activities are not important 4. I don’t have time 5. Religious reason 6. Other (Specify :______________________ ____) I__I I__I I__I I__I *Don’t cite the list of reasons *Choose all answers that apply Health3 During the last two weeks, has [student’s name] gotten sick? 0. No 1. Yes I__I *Select only one option *If Yes, move to “health3a” *If No, move to “handwash” Health3a What were the symptoms of that/those disease(s)? 1. Diarrhea 2. Vomiting 3. Fever 4. Other (Specify :_________________) I__I I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all answers that apply Health3b During the last two weeks, has [student’s name] missed school because he/she was sick? 2. No  handwash 3. Yes  health3c I__I *Select only one option *If Yes, move to “health3c” *If No, move to “handwash” Health3c During the last two weeks, how many days has [student’s name] missed school because he/she was sick? 5. 1-3 days 6. 3-5 days 7. More than 5 days I__I *Do not read list to the respondent *Select only one option IMPAQ International, LLC Page D-27 Mali FFE Midline Report July 2018 Hygiene Thanks! I would like to ask you a few questions regarding hygiene… handwas h In your opinion, when should someone wash their hands? 1. Before eating 2. Before touching or preparing food 3. Before giving food to someone 4. When hands are dirty 5. After touching a dirty object 6. After touching a pet 7. After using the latrines 8. After changing a baby’s diapers 9. Before prayers 10. Other (Specify :_________________________) 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply Hand1 How many times did you wash your hands yesterday? ……. *Interval between 0 and 20 *If 0, move to « Hand3 » Hand2 What were the reasons? 1. Before eating 2. Before touching or preparing food 3. Before giving food to someone 4. When hands are dirty 5. After touching a dirty object 6. After touching a pet 7. After using the latrines 8. After changing a baby’s diapers 9. Before prayers 10. Other (Specify :_________________________) 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply Hand3 What do you usually use to wash your hands? 1. Just water 2. Water plus soap 3. Other (Specify :_____________________) I__I * Do not give examples or read the list to the respondent *Select only one option IMPAQ International, LLC Page D-28 Mali FFE Midline Report July 2018 Worms In your opinion, how can one avoid to catch intestinal worms (in the belly)? 1. Avoid walking bare foot (wear shoes) 2. Not to bathe or swim in stagnant water 3. Eat meat that is well cooked 4. Avoid contact with contaminated water, and if necessary to wear boots and gloves 5. Wash hands with drinking water and soap before preparing food, before serving food or before eating 6. Wash hands with drinking water soap after using the latrines 7. Protect food from flies, cockroaches, and dust 8. Store food in a pantry or a place that is clean and well-ventilated 9. Other (specify :______________________) 88. Don’t know I__I I__I * Do not give examples or read the list to the respondent *After the respondent has given one way, incite the respondent to give a second way: What other way can help avoid intestinal worms? Incite for 2 ways in total Food Security Great! Now, I would like that you take a minute and reflect on all the food and beverages that you have given to [student’s name] yesterday during the day and evening either at home or outside of the house… Fs1 According to you, was yesterday ‘ordinary/usual’ or was it a special occasion? 1. Ordinary/Usual 2. Special occasion (specify :____________________) *If Gender = 1 (father/male caregiver)  Fs2_father *If Gender = 2 (mother/female caregiver), Fs2 I__I *Give examples of special occasions such as funeral and parties *Choose only one option Fs2 Can you cite all the food and beverages that you gave to [student’s name] yesterday? 1. Millet, rice, corn, sorghum, manioc 2. Nuts or beans (such as cowpeas) 3. Yoghurt, milk or cheese 4. Meat or fish I__I I__I I__I I__I I__I *Note all the food and beverages mentioned by the respondent on a separated sheet of paper. If dishes are mentioned, ask for the list of ingredients of each dish mentioned. IMPAQ International, LLC Page D-29 Mali FFE Midline Report July 2018 5. Eggs 6. Red palm oil or fruits or vegetables (such as carrots, squash, sweet potatoes, dark green leafy vegetables leaves, ripe mango, melon, apricot, ripe papaya, peach, red peppers, moringa leaves), etc. 7. Other fruits and vegetables such as onion, eggplant, watermelon, oranges, green peppers, cabbage, tomato, dates, lentils, peas, etc. Then, Move to fs3 I__I I__I *When the respondent is done, ask the respondent to assure you that he/she has mentioned all the food and beverages given to the child yesterday, including all snacks. Use time references if necessary (morning, afternoon, evening, night). *Select all answers that apply. *For each food group not mentioned, ask the respondent if a food from this group has been given to their kid yesterday. Fs2_f ather Can you cite all the food and beverages that [student’s name] eat yesterday? 1. Millet, rice, corn, sorghum, manioc 2. Nuts or beans (such as cowpeas) 3. Yoghurt, milk or cheese 4. Meat or fish 5. Eggs 6. Red palm oil or fruits or vegetables (such as carrots, squash, sweet potatoes, dark green leafy vegetables leaves, ripe mango, melon, apricot, ripe papaya, peach, red peppers, moringa leaves), etc. 7. Other fruits and vegetables such as onion, eggplant, watermelon, oranges, green peppers, cabbage, tomato, dates, lentils, peas, etc. 88. Don’t know I__I I__I I__I I__I I__I I__I I__I *Note all the food and beverages mentioned by the respondent on a separated sheet of paper. If dishes are mentioned, ask for the list of ingredients of each dish mentioned. *When the respondent is done, ask the respondent to assure you that he/she has mentioned all the food and beverages given to the child yesterday, including all snacks. Use time references if necessary (morning, afternoon, evening, night). *Select all answers that apply. *For each food group not mentioned, ask the respondent if a food from this group has been given to their kid yesterday. Fs3 Yesterday, has [student’s name] eaten something before the morning meal? I__I *Select only one option IMPAQ International, LLC Page D-30 Mali FFE Midline Report July 2018 0. No 1. Yes 88. Don’t know Fs4 Yesterday, has [student’s name] eaten something for the morning meal? 0. No 1. Yes 88. Do not know I__I *Select only one option Fs5 Yesterday, has [student’s name] eaten something between the morning meal and the mid-day meal? 0. No 1. Yes 88. Do not know I__I *Select only one option Fs6 Yesterday, has [student’s name] eaten something for the mid-day meal? 0. No 1. Yes 88. Do not know I__I *Select only one option Fs7 Yesterday, has [student’s name] eaten something between the mid-day meal and the evening meal? 0. No 1. Yes 88. Do not know I__I *Select only one option Fs8 Yesterday, has [student’s name] eaten something for the evening meal? 0. No 1. Yes 88. Do not know I__I *Select only one option Fs9 Yesterday, has [student’s name] eaten something after the evening meal? 0. No 1. Yes 88. Do not know I__I *Select only one option Now, I will read to you several statements that people have made regarding their food situation. For some of these statements, please tell me if the statement is often valid, sometimes valid, or never valid for your household during the last 12 months – that is, since last May. Fs16 The food supplies that we bought did not last at all, and we have no money to buy more. I__I *Select only one option IMPAQ International, LLC Page D-31 Mali FFE Midline Report July 2018 Is this often, sometimes, or never the case for your household during the last 12 months, that is, since last April? 1. Yes, often 2. Yes, sometimes 3. No, never 88. Don’t know Fs17 We couldn’t afford the luxury of eating balanced meals. Is this often, sometimes, or never the case for your household during the last 12 months? 1. Yes, often 2. Yes, sometimes 3. No, never 88. Don’t know I__I *Explain « balanced meals » *Select only one option Fs18 During the last 12 months, that is, since last April, have you one time eaten less than what you should because there was not enough food or money for food? 0. No 1. Yes 88. Don’t know I__I *Select only one option Fs19 During the last 12 months that is, since last April, were you hungry one time but did not eat because there was not enough food or money to buy food? 0. No 1. Yes 88. Don’t know I__I *Select only one option Fs20 During the last 12 months that is, since last April, have you or other adults in your household decreased the size of your meal OR skipped meals OR substitute some foods for other foods that are less nutritious because there was not enough food or money for food? 0. No 1. Yes I__I *If Yes, move to ’’fs20a’’ *If No or Don’t know, move to ‘’fs22’’ *Select only one option IMPAQ International, LLC Page D-32 Mali FFE Midline Report July 2018 88. Don’t know Fs20 a How many times did this happen? 1. Nearly every month 2. Some months but not every month 3. Only 1 or 2 months 88. Don’t know I__I *Select only one option Fs21 For whom in the household do you usually reduce the size of the meal? 1. Everyone 2. The women 3. The girls 4. The men 5. The boys 6. Other (Specify:___________________________) I__I I__I I__I I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all answers that apply Fs22 During the last 12 months, that is, since last April, how have you faced the fact that you did not have enough food for everyone in your household? 1. Reduce the number of meals of the household members 2. Reduce school fees of children 3. Borrow money to buy food 4. Receive food from members of the family, parents and neighbors 5. Cook everything that is available in the house for meals 6. Sell our cattle or other actives 7. Other (specify : ____________________) I__I I__I I__I I__I I__I I__I I__I *Ask Fs22 only if the respondent has responded YES OR YES OFTEN OR YES, SOMETIMES to at least one of the questions: ‘’fs16’’ OR ‘’fs17’’ OR ‘’fs18’’ OR ‘’fs19’’ OR ‘’fs20’’ *Don’t give examples or read the list to the respondent *Choose all answers that apply Parent Participation IMPAQ International, LLC Page D-33 Mali FFE Midline Report July 2018 Thanks! Now, I’d like to ask you a few questions regarding your engagement at the school of [student’s name]… Act2 Since the start of the school year, that is since last October, how many general assemblies have you participated in? 0. None 1. 1 to 3 assemblies 2. More than 3 assemblies 88. Don’t know I__I *Don’t read the list to the respondent *Choose only one option Act3 In your opinion, to what extent is the Comité de Gestion Scolaire (CGS) of the school of [student’s name] active/engaged? 1. Often 2. Sometimes 3. Rarely 4. Never I__I *Don’t read the list to the respondent *Choose only one option Act4 Have you participated in a support activity of the school of [student’s name], such as clean the latrines, the kitchen, the school premises, help the school as cook or shopkeeper, or other activities? 0. No 1. Yes I__I *If Yes, move to ’’Act5’’ *If No, move to ‘’Act6’’ *Select only one option Act5 To which activity/ies have you participated? 1. Clean the latrines, kitchen, school compound 2. Help the school as cook or shopkeeper 3. Contribute money and/or food to the school canteen 4. Support the garden/field of the school 5. Participation in a training 6. Participation in sensitization activities for the registration of children in schools 7. Other (Specify :____________________) I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all answers that apply IMPAQ International, LLC Page D-34 Mali FFE Midline Report July 2018 Act6 Are you engaged in the academic support of [student’s name]? 0. No 1. Yes I__I *Give examples if necessary : Help [student’s name] to read/do his homework ; follow his progress ; make sure he goes to school; make sure he has adequate time to do his homework; attend the Comité de Gestion Scolaire (CGS) meeting; etc. *Select only one option Act7 Can you state all the ways in which you are engaged in the education of [student’s name]? 1. Help to read/do his/her homework 2. Follow his/her progress 3. Make sure he/she goes to school 4. Make sure he/she has adequate time to do his homework 5. Attend the Comité de Gestion Scolaire (CGS) meeting 6. Other (Specify:_________________________) I__I I__I I__I I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all the answers that apply Score1 Is there a notice board at the school of [student’s name]? 0. No 1. Yes 88. Don’t know I__I *Select only one option *If Yes, move to ’’score2’’ *If No, move to ’’Report1’’ Score2 According to you, is this notice board at the school of [student’s name] useful? 0. No, not useful 1. Yes, useful 2. Both useful and not useful *If Yes, move to «Score 3» and don’t ask « Score4 » *If Not, move to « score4» and don’t ask « score3» *If Both useful and not useful move to « Score3 » AND ask « Score4 » *Select only one option IMPAQ International, LLC Page D-35 Mali FFE Midline Report July 2018 Score3 Can you give examples on the ways this board is useful? 1. Gives information about the attendance of students at school (school attendance) 2. Gives information about the school registration 3. Gives information about the performance of students at school 4. Gives information about the presence of teachers at school 5. Gives information about the performance of teachers at school 6. Gives information about the community contributions to the school (canteen) meals 7. Give information about the action plans of the school 8. Other (specify:_______________________) I__I I__I I__I I__I I__I I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all the answers that apply Score4 Can you give examples why this board is not useful? 1. The information on the board is not clear/legible 2. The information on the board does not inform me 3. The information on the board is not updated 4. The board is not displayed in an accessible place 5. Other (specify:____________________________) I__I I__I I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all the answers that apply Report1 Have you received a colored bulletin for [student’s name]? 0. No  teacheratt 1. Yes  report2 88. Don’t know  teacheratt I__I *Select only one option Report2 According to you, is this colored bulletin for [student’s name] useful? 0. No 1. Yes 2. Both useful and not useful *If Yes, move to «Report3» and don’t ask « Report4 » *If No, move to «Report4» and don’t ask « Report3» IMPAQ International, LLC Page D-36 Mali FFE Midline Report July 2018 *If Both useful and not useful, move to « Report3 » AND ask « Report4 » *Select only one option IMPAQ International, LLC Page D-37 Mali FFE Midline Report July 2018 Report3 Can you give me examples on how this colored bulletin is useful? 1. Gives examples about the performance of students at school 2. Other (specify:____________________________) I__I I__I *Don’t give examples or read the list to the respondent *Choose all the answers that apply Report4 Can you give me examples about why the colored bulletin is not useful? 1. The information on the bulletin is not clear/confusing/illegible 2. The information on the bulletin does not teach me anything new 3. The information on the bulletin is not updated 4. Other (specify:____________________________) I__I I__I I__I I__I *Only ask this question if the respondent responded No to «Report2 » *Don’t give examples or read the list to the respondent *Choose all the answers that apply Teacheratt In general, is the headmaster absent at school? 1. Often 2. Sometimes 3. Rarely 4. Never 88. Don’t know I__I *Read the list to the respondent but don’t read « don’t know » *Select only one option SILC group activities Great, thanks! Now I would like to ask you some questions about Saving and Internal Lending Community activities… silc1 Are you or any member of your household a member of a Saving and Internal Lending Community (SILC) ? 0. No one in my household is a member of CECI group  asp1 1. Yes, I am a member of CECI  silc2 2. Someone else in my household is a member of CECI group  silc2 88. Don’t know I__I *Select only one option IMPAQ International, LLC Page D-38 Mali FFE Midline Report July 2018 Silc2 What significant changes has participation in a SILC had on the life of your household or your own life? 1. Improved household’s income 2. Access to healthcare 3. Support on children’s education’s cost 4. Improved food security 5. Participation in village affairs 6. Connection with other community members and parents 7. Increased household’s asset (e.g., husbandry, households, equipment, personal items-cloths, jewels, etc.) 8. Others 9. No changes 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I *Select all that apply *Do not read the options to the respondent Silc3 To what degree, have your financial revenue improved with your participation in SILC group? 0. No improvement 1. Improved a little 2. Improved by a fair amount 3. Improved substantially 88. Don’t know I__I *Select only one option Silc4 How has participation in SILC group contributed to the education of your children? 1. Encouraged enrollment of children in school 2. Allowed you to save more for school 3. Monitor your child’s school performance 4. Contribute to the school canteen 5. Other (specify) *Do not read the options *Select all that apply IMPAQ International, LLC Page D-39 Mali FFE Midline Report July 2018 6. No change 88. Don’t know Silc5 If given a chance, will you or your household continue your participation in SILC group activities after CRS Food for Education (FFE) project ends? 0. No 1. Yes 88. Don’t know I__I *Select only one option Educational Aspiration of Mothers Great! Now, I would like to ask you a few questions regarding the education of [student’s name] Asp1 When [student’s name] will be close to 20 years old, what job do you think he/she will do? 1. Blue collar (Jobs that do not require a high level of education) 2. White collar (Jobs that require a high level of education) 3. Other (specify:___________________________ _) 88. Don’t know I__I *If the respondent chooses a blue collar job, write down 1 *If the respondent chooses a white collar job, write down 2 *The examples given have the goal to help the enumerators pick the right answer. But don’t give IMPAQ International, LLC Page D-40 Mali FFE Midline Report July 2018 Blue Collar job examples a. Full time housewife b. Agricultural worker c. Construction worker (building) d. Office secretary e. Religious leader/priest/sheik f. Merchants/sellers g. Military h. Mason i. Sportsman/Sportswoma n j. Domestic employee k. Driver l. Mechanic m. Cab driver n. Farmer o. Public servant p. Firefighter q. Traditional profession r. Fisherman s. Policeman/policewoma n t. Singer White Collar Job examples a) Veterin ary b) Preside nt of the country c) Professi onal cook d) Jurist (judge, notary, lawyer, etc.) e) Scientis t f) Dentist g) Univers ity profess or h) Manag er i) Tax collecto r j) Physicia n k) Accoun tant l) Actor/a ctress m) Artist n) Enginee r o) Nurse p) High school teacher examples or read the list to the respondent *Select only one option IMPAQ International, LLC Page D-41 Mali FFE Midline Report July 2018 q) Painter /decora tor r) Busines sman/b usiness woman s) Airplan e pilot t) Comput er scientis t u) Head of school/ universi ty Asp2 Ideally, what level of education would you want [student’s name] to reach? 1. None 2. Some primary school 3. Complete primary school 4. Some secondary school 5. Complete secondary school 6. Some university 7. Obtain a bachelor’s degree 8. More than a bachelor’s degree 9. Professional school 88. Don’t know I__I **Don’t give examples or read the list to the respondent *Select only one option Asp3 Do you expect [student’s name] to reach that level of education? 0. No 1. Yes I__I *If yes, move to ’’Girlsch1 ’’ *If No, move to ‘’Asp4’’ *Select only one option IMPAQ International, LLC Page D-42 Mali FFE Midline Report July 2018 Asp4 Why do you think that [student’s name] will not reach this level of education? 1. I can’t afford school fees for my kid 2. My kid is not intelligent/capable enough 3. I need my kid to help me at home/in the field 4. I need my kid to work to support the family 5. Other (Specify:_______________________) I__I I__I I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all the answers that apply Girlsch1 In the past, it was mostly boys who went to school. Nowadays, both boys and girls go to school. According to you, is this a good or bad thing? 1. Good 2. Bad 3. Both good and bad 88. Don’t know I__I *Select only one option *If Good, move to « girlsch2» and don’t ask « Girlsch3 » *If Bad, move to « girlsch3» and don’t ask « Girlsch2 » *If Both good and bad, move to « girlsch2» *If Don’t know, thank the respondent and move to observations If the response to “girlsch1” is Don’t know, thank the respondent and move to observations. Girlsch2 Why do you think that the schooling of girls is a good thing? 1. Improve the standard of living of the whole family 2. Improve their health (the girls) 3. Improve the health of the children that they will have 4. The girls will also be able to blossom 5. Allow girls to find a better job 6. Other (specify:_______________________) I__I I__I I__I I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all the answers that apply Girlsch3 Why do you think that the schooling of girls is a bad thing? 1. Girls are supposed to stay at home 2. Girls don’t need to go to school 3. Girls don’t work outside of the house, so, what is the point? 4. Girls should not be outside in public 5. There is not girl-only school and they should not go to school with boys 6. School is dangerous for girls 7. Other (specify :_________________) I__I I__I I__I I__I I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all the answers that apply IMPAQ International, LLC Page D-43 Mali FFE Midline Report July 2018 Decision 1 Who makes a decision for your boy (male children) in your household to attend school? 1. Mother 2. Father 3. Both parents together 4. The child himself 5. Someone else (specify): ___________________ 88. Don’t know / No response I__I *Do not read the exapmles to the respondent *Select only one option *If the respondent indicates that the decision is made by the child with one or more parent (s), categorize the response as the decision of the parent (s) (i.e. choose one of the answers 1, 2, or 3) Decision 2 Who makes a decision for your girl (female children) in your household to attend school? 1. Mother 2. Father 3. Both parents together 4. The child himself 5. Someone else (specify): ___________________ 88. Don’t know / No response I__I *Do not read the examples to the respondent *Select only one option *If the respondent indicates that the decision is made by the child with one or more parent (s), categorize the response as the decision of the parent (s) (i.e. choose one of the answers 1, 2, or 3) Decision 3_boys Who in your household usually have the final say in the payment of the boys’ school fees?: 1. Mother 2. Father 3. Both parents 4. The child himself 5. Someone else (specify): ___________________ 88. Don’t know / No response I__I *Do not read the examples to the respondent *Select only one option Decision 3_girls Who in your household usually have the final say in the girl’s school fees?: 1. Mother 2. Father I__I *Do not read the examples to the respondent *Select only one option IMPAQ International, LLC Page D-44 Mali FFE Midline Report July 2018 3. Both parents 4. The child herself 5. Someone else (specify): ___________________ 88. Don’t know / No response thanks Thanks a lot for answering my questions IMPAQ International, LLC Page D-45 Mali FFE Midline Report July 2018 Midline EVALUATION OF THE CRS FOOD FOR EDUCATION PROJET IN MALI Teacher/Principal Survey Base Information Enum Surveyor (last name and first name) Date Date (DD/MM/YYYY) Region Write the name of the region CAP Write the name of the Educational Animation Center schoolname Write the name of the school Dear Director/Teacher : You have been selected to participate in a survey about health, nutrition, and education for the Cantine Scolaire (School Canteen) project. Your participation in this interview is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. If you accept, please respond to all questions as candidly as possible. If you do not know the answer to a question, you may simply say so. All responses will be kept strictly confidential. consent Do you accept participation in this survey? 0. No  thanks the respondent and terminate the survey 1. Yes  fname I__I If the response to “consent” is No or Not Found, thank the respondent and terminate the survey. Personal Information IMPAQ International, LLC Page D-46 Mali FFE Midline Report July 2018 Excellent! Now I would like to ask a few questions about you… fname What is your name? ___________________________ lname What is your last name? ___________________________ Age1 Do you know your age? 0. No 1. Yes 2. Don’t know I__I *If Yes, go to “age2” *If No or Don’t know, go to “gender” *Only select one option Age2 How old are you? …… *AGE INTERVAL between 15 and 99 gender Enter the teacher’s sex? 1. Male 2. Female I__I *Only ask if neccesary language Which language do you speak best? 1. French 2. Local Language 3. Other (specify:__________________) I__I *Select only one option Language2 In which language do you use to teach? 1. French 2. Local Language 3. Both I__I *Select only one option Language3 In which language are you are more comfortable teaching? 1. French 2. Local Language 3. Other (specify:__________________) I__I *Select only one option Edu What is the highest level of education that you achieved? 1. DEF 2. BAC 3. BT1 4. BT2 5. CAP 6. Bac+2 (DEUG, DUT) 7. Bac+3 8. Bac+4 I__I *Select only one option IMPAQ International, LLC Page D-47 Mali FFE Midline Report July 2018 9. Bac+5 10. Other (Specify : ____________________) Teach Do you teach 1st, 2nd, 3rd, or 4th grade in this school? 0. No 1. Yes I__I *If yes, go to “ Teach1” *If no, go to “principal” Teach1 What class do you teach? 1. First grade 2. Second Grade 3. Third Grade 4. Fourth Grade I__I I__I I__I I__I *Choose all the answers that apply Teach2 Have you had this class since the beginning of school? 0. No 1. Yes I__I *Choose only one option *HINT to the enumerator: if s/he was a replacement teacher in the middle of the school-year Teachlen How long have you been teaching? 1. Less than one year 2. 1 to 2 years 3. 3 to 5 years 4. 6 years or more I__I *Choose only one option same_classe Are your children from different class years sitting together in the same calss at the same time? 0. No 1. Yes *Choose only one option *Relevant only if selected more than 1 option for teach1 *If No, skip to Kid1 Kid How many students are in your class? …. *Indicate a number, 0 to 150 Kid1 How many students are in your 1st grade class? …. *Indicate a number, 0 to 150 IMPAQ International, LLC Page D-48 Mali FFE Midline Report July 2018 Kid2 How many students are in your 2nd grade class? …. *Indicate a number, 0 to 150 Kid3 How many students are in your 3rd grade class? …. *Indicate a number, 0 to 150 Kid4 How many students are in your 4th grade class? …. *Indicate a number, 0 to 150 emp What is your employment status? 1. Government official 2. Community official 3. Government contractor 4. IFM intern 5. Volunteer 6. Other I__I *Read the list to the respondent *Select only one option TRAINING AND KNOWLEDGE Thank you! Now, I would like to ask a few questions on the type of training and degrees you have received in the past… Train1 Were you trained to teach? 0. No 1. Yes I__I * Select only one option *If No, go to “Train4” Train2 What is your training in teaching? 1. SARPE 2. ECOM 3. IFM I__I * Select only one option IMPAQ International, LLC Page D-49 Mali FFE Midline Report July 2018 4. IPEG 5. Hégire 6. None 7. Other (Specify : ____________________) Train3 Was this training validated by a diploma, certificate, and attestation of success? 1. Diploma 2. Certificate 3. Attestation of success 4. Other I__I * Select only one option Train4 Were you trained on the balanced approach? 0. No  train7 1. Yes  train4a I__I * Select only one option Train4a How many times did you receive the balanced approach training? 1. Only once 2. Twice 3. Three times 4. Four times 5. More than four times 88. Don’t know I__I * Select only one option Train5 When were you trained on the balanced approach? 1. Before December 2015 2. December 2015 3. March 2016 4. September 2016 5. December 2016 6. September 2017 7. December 2017 8. Other (Specify:___________________________) I__I I__I I__I I__I * Select only one option Train6 On which techniques of the balanced approach were you trained? 1. Interactive radio educatoin (EIR) 2. News of the class 3. Text decoding 4. Guided reading 5. Language mechanism games 6. Guided writing 7. Invented writing 8. Spontaneous writing I__I I__I I__I I__I I__I I__I I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all answers that apply IMPAQ International, LLC Page D-50 Mali FFE Midline Report July 2018 9. Other (Specify: ______________________) Train6a Do you use the activities of the balanced approach in your class? 0. No 1. Yes I__I If Yes, go to “Train6b” If No, go to “Train7” * Select only one option Train6b What activities of the balanced approach do you use during the Language and Communication lessons? 1. Interactive radio educatoin (EIR) (in theory, every day) 2. News of the class (in theory, every day) 3. Text decoding 4. Guided reading 5. Language mechanism games (in theory, every day) 6. Guided writing 7. Invented writing 8. Spontaneous writing 9. Other (Specify: ______________________) I__I I__I I__I I__I I__I I__I I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all answers that apply Train6c What are the activities of the balanced approach that your students appreciate most in your class? 1. Interactive radio educatoin (EIR) 2. News of the class 3. Text decoding 4. Guided reading 5. Language mechanism games 6. Guided writing 7. Invented writing 8. Spontaneous writing 9. Other (Specify: ______________________) I__I I__I I__I I__I I__I I__I I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all answers that apply Train6d_ma terial Do you use the balanced appraoch teaching materials in your class? 0. No  «Train9» 1. Yes  «Train6e_material» I__I * Select only one option IMPAQ International, LLC Page D-51 Mali FFE Midline Report July 2018 Train6e_ma terial How often did you use the Balanced Approach equipment during the last month? 0. Never 1. Rarely 2. Sometimes (sometimes) 3. Often (frequently) I__I * Select only one option Train6f According to you, does the balanced approach help improve your students' level of reading and writing? 1. Not at all 2. A little 3. Moderately 4. A lot 88. Do not know I__I * Select only one option Train6g Which aspects of the balanced approach techniques are you satisfied with? 0. Nothing 1. the formations 2. Supervision of the project 3. Supervision of the principal 4. Supervision of CAP officers 5. The material distributes 6. Student participation 7. Other (specify) 88. Don’t know I__I I__I I__I I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all answers that apply Train7 Have you received another training in the didactics/teaching of reading-writing since the beginning of the school year, that is since last October? 0. No  train7a 1. Yes  88. Don’t know  train7a I__I *Specify to respondent that this does not include the balanced approach * Select only one option Train7a Who supported the training? 1. CAP 2. CRS/EDC/other partners 3. Other (Specify:_________________________) I__I I__I I__I *Choose all the answers that apply Train7a Did you participate in another training in teaching since last October? 0. No  train9 1. Yes  train8 1. Don’t know  train9 I__I *Specify to respondent that this does not include the balanced approach IMPAQ International, LLC Page D-52 Mali FFE Midline Report July 2018 * Select only one option Train8 Who supported the training? 4. CAP 5. CRS/EDC/other partners 6. Other (Specify:_________________________) I__I I__I I__I *Choose all the answers that apply Attend3 Over the course of an ordinary week, how frequently does the director observe you during a lesson of reading-writing? 0. Never 1. 1-2 days 2. 3-4 days 3. Every day 4. Not applicable (respondant is a principal) I__I * Don’t read this list to the respondent * Select only one option Attend3_m onth Over the course of an ordinary month, how frequently does the director observe you during a lesson of reading-writing? 0. Never 1. Once month 2. 2 times a month 3. More than 2 times a month 4. Not applicable (respondent is director) I__I * Don’t read this list to the respondent *If 1, 2, or 3, go to “Attend3a” *If Never, go to “Attend3b” * Select only one option IMPAQ International, LLC Page D-53 Mali FFE Midline Report July 2018 Attend3a In your opinion, are the observations of the director useful? 1. Often (frequently) 2. Sometimes (a few times) 3. Rarely 4. Never I__I *Don’t ask this question if the answer to attend3 and attend3_month are “never” *Read the list to the respondent * Select only one option Attend3b Other than observations, does the director offer other types of support? 0. None 1. Encouragements/Congratulations 2. Teaching advice 3. Other (Specify : __________________) I__I I__I I__I I__I * Do not read the list to the respondent * Choose all the answers that apply Attend3c Apart from the school director, who else has observed you during a lesson of reading-writing? 0. No one 1. Pedegogical advisor 2. The regional supervisor 3. Other (Specify : __________________) I__I I__I I__I * Choose all the answers that apply Attend3d How many times have you been observed by people other than the Director during the year? 0. Never 1. Once 2. Twice 3. More than twice 88. Don’t know I__I * Select only one option Attend4 Usually, do your students participate during the lesson? 1. Often (frequently) 2. Sometimes (a few times) 3. Rarely 4. Never  handwash I__I *Read the list to the respondent * Select only one option Attend4a Usually, with regards to student’s participation during the lesson, is there a difference between the girls and the boys? 0. Yes  attend4b 1. No  handwash I__I * Select only one option IMPAQ International, LLC Page D-54 Mali FFE Midline Report July 2018 Attend4b Who participates most during the lesson— the girls or the boys? 1. The girls 2. The boys I__I * Select only one option Hygiene We’re almost done! Now, I will ask you a few questions on hygiene… handwas h In your opinion, when should someone wash their hands? 1. Before eating 2. Before touching or preparing food 3. Before giving the food to another person 4. When the hands are dirty 5. After touching a dirty object 6. After touching a domestic animal 7. After using the latrines 8. After changing a baby’s diaper 9. Before the prayer 10. Other (Specify :_________________________) 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I I__I I__I * Do not read the list to the respondent *Choose all the answers that apply Hand1 How many times have you washed your hands yesterday? ……. *0 = < & <20 *If 0, go to Hand3 Hand2 What were the reasons? 1. Before eating 2. Before touching or preparing food 3. Before giving the food to another person 4. When the hands are dirty 5. After touching a dirty object 6. After touching a domestic animal 7. After using the latrines 8. After changing a baby’s diaper 9. Before the prayer 10. Other (Specify :_________________________) 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I I__I I__I * Do not read the list to the respondent *Choose all the answers that apply IMPAQ International, LLC Page D-55 Mali FFE Midline Report July 2018 Hand3 Usually, what do you use to wash your hands? 1. Simple water 2. Water plus soap 3. Other (Specify :_____________________) I__I * Do not read the list to the respondent *Select only one option IMPAQ International, LLC Page D-56 Mali FFE Midline Report July 2018 Worms1 By what way can we prevent intestinal worms? 1. Avoid walking barefoot (wear shoes) 2. Don’t bathe yourself or swim in stagnant water 3. Eat meat that is cooked appropriately 4. Avoid contact with contaminated water, but if necessary wear boots and gloves 5. Wash hands with clean water and soap before preparing food, before serving food, or before eating 6. Wash hands with water that is clean and soap after using the latrines 7. Protect food against flies , cockroaches, and dust 8. Keep food in a pantry, or a place that is clean and well aerated 9. Other (specify:__________________) 88. Don’t know I__I I__I *Do not give examples or read the list to the respondent *After the respondent gives a way, encourage the respondent to give a second way: By what other way can we prevent intestinal worms? *Encourage to obtain 2 ways total Stuprop In your opinion, during a normal day, how many students among your students wash their hands before eating at school? 1. None 2. Less than half 3. Around half 4. More than half 5. Almost all 6. All 88. Don’t know I__I *Select only one option Director Now, I would like to know if you serve as director of the school? principal Are you the director in this school? 0. No  thank the respondent and end the survey 1. Yes  principal1 I__I *Select only one option If the response to “principal” is No, thank the respondent and end the survey IMPAQ International, LLC Page D-57 Mali FFE Midline Report July 2018 Principal1 How long have you been director of this school? 1. Less than a year 2. 1 to 2 years 3. 3 to 5 years 4. 6 years or more I__I *Select only one option Principal2 Have you been trained to track and support your teachers in your teaching of the reading￾writing? 0. No  principal3 1. Yes  principal2a I__I *Select only one option Principal2a When were you trained the last time in the use of tools for supervising teachers in the teaching of reading-writing? 0. Never 1. December 2017 2. September 2017 3. January 2016 4. Other (Specify) 88. Dont know I__I *Select only one option Principal2b Have you received another training on the supervision of teaching of reading-writing since the beginning of the school year, which is since last October? 0. No  principal3 1. Yes  principal2c 88. Don’t know  principal3 I__I *Select only one option Principal2c Who supported the training? 2. CAP 3. CRS/EDC/other partners 4. Other (Specify ---------------) I__I I__I I__I *Select more than one option Principal3 During a normal week, at which frequency do you observe your teachers during a lesson of reading-writing? 1. Never 2. 1-2 days 3. 3-4 days I__I * Don’t read the list to the respondent *Select only one option IMPAQ International, LLC Page D-58 Mali FFE Midline Report July 2018 4. Every day Principal3_month During a normal month, at which frequency do you observe your teachers during a lesson of reading-writing? 0. Never 1. 1 time 2. 2 times 3. More than 2 times I__I * Don’t read the list to the respondent *Select only one option Principal4 Do you have any challenges observing/supporting your teachers? 0. No  principal6 1. Yes  principal5 I__I *Select only one option Principal5 What are these challenges? 1. Lack of time 2. Lack of material resources (books, chalk, etc.) 3. Don’t know how to support them 4. Other (Specify :_______________) I__I I__I I__I I__I * Don’t read the list to the respondent * Choose all the answers that apply Principal6 How often do the pedegogical advisors help you in your work? 1. Often (frequently) 2. Sometimes (a few times) 3. Rarely 4. Never I__I *Read the list to the respondent *Select only one option PHYSICAL ASSETS OF THE SCHOOL Since you are the director, that is, the first responsible for the school, I would like to ask you a few questions about the physical assets of the school… Asset1 Does the school have an aerated place to store food? 0. No 1. Yes I__I *Select only one option IMPAQ International, LLC Page D-59 Mali FFE Midline Report July 2018 Asset2 Does the school have pallets or an elevated surface for storing food? 0. No 1. Yes I__I *Select only one option Asset3 Is there a kitchen in the school? 0. No 1. Yes I__I *Select only one option Asset4 Is there any source of water available for the school? 0. No 1. Yes I__I *Select only one option Asset4a What is the main source of water available for the school? 1. None 2. Faucet, running water SOMAGEP 3. Borehole/village pump 4. Improved well (protected) 5. Traditional well (non protected) 6. Surface water (backwater, river, stream) 7. Rainwater 8. Other (specify:__________________) I__I *Do not give examples or read the list to the respondent *Select only one option Asset4b How far from the school is the main water source? 1. In the school compound 2. Less than 15 minutes on foot 3. More than 15 minutes on foot 88. Don’t know I__I *Read the list to the respondent, but don’t read “don’t know” *Select only one option Asset4c Do you currently have any problems in accessing drinking water access problems? 0. No  asset5 1. Yes  asset4d I__I *Select only one option Asset4d What kind of problems do you have for accessing drinking water? 1. Broken pump 2. Water point occupied by animals 3. Drying up of water point 4. Water point is used for agriculture 5. Other (specify : _________________) I__I I__I I__I I__I I__I *Don’t give examples or read the list to the respondent *Choose all the answers that apply IMPAQ International, LLC Page D-60 Mali FFE Midline Report July 2018 Asset5 Are there sanitary installations inside the school? (ex. latrines, toilets, etc.) ? 0. No 1. Yes I__I *Select only one answer Asset6 Are there sanitary installations inside the school in seperate blocs for girls and boys? 0. No 1. Yes I__I * Select only one answer Asset7 Is there a sufficient availability of reading materials? 0. No 1. Yes I__I * Select only one answer SCHOOL CANTEEN Very good! Now, I would like to ask you a few questions on the school canteen… Canteen1_ne w In March 2018, how many days did the canteen not work? 0. 0 days of no operation – worked full month 1. Between 1-5 days of non-functioning 2. More than 5 days of non-operation 88. Don’t know I__I * Select only one answer Canteen2_ne w What were the reasons that the canteen did not work? 1. Absence of stoves 2. Absence of teachers 3. Food breakdown of the project 4. Lack of condiments 5. Other (to be specified) 88. Don’t know I__I I__I I__I I__I I__I *Don’t read the list to the respondent *Select all the responses that apply. Canteen3_ne w To what extent, are you satisfied with the participation of management committees in the operation of the canteen? 0. Not satisfied 1. A little satisfied 2. Satisfied 3. Very satisfied 88. Don’t know I__I * Select only one answer IMPAQ International, LLC Page D-61 Mali FFE Midline Report July 2018 Canteen4 Do you think the canteen activity should continue in your school after the withdrawal of the project? 0. No 1. Yes 88. Don’t know I__I * Select only one answer Canteen5 Who do you think should take charge of the canteen after the withdrawal of the project? 1. Parents 2. Mayor’s Office 3. State 4. Other (to be specified) 88. Don’t know I__I * Select only one answer school_image [Take a photo of the school when principal=1] thanks Thank you very much for having answered my questions. IMPAQ International, LLC Page D-62 Mali FFE Midline Report July 2018 Midline EVALUATION OF THE CRS FOOD FOR EDUCATION PROJECT IN MALI SMC Survey BASE INFORMATION Enum_id Surveyor ID Date Date (DD/MM/YYYY) Region Write the name of the region CAP Write the name of the Educational Animation Center schoolname Write the name of the school Dear Sir/Madam: You have been selected to participate in a survey about health, nutrition, and education for the Cantine Scolaire (School Canteen) project. Your participation in this interview is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. If you accept, please respond to all questions as candidly as possible. If you do not know the answer to a question, you may simply say so. All responses will be kept strictly confidential. consent Do you accept participation in this survey? 3. No  thank the respondent and terminate the survey 4. Yes  I__I *select only one option If the response to “consent” is No, thank the respondent and terminate the survey IMPAQ International, LLC Page D-63 Mali FFE Midline Report July 2018 N.B: If the respondent refuses to answer any particular question, mark an « R » as the answer and move on the next question. PERSONAL INFORMATION Fname What is your first name? _________________________ Lname What is your last name? ______________________ Match 2 Are you : 1. School Principal 2. Teacher 3. Parent of a student 4. Other (Specify : _____________________) I__I *Choose only one option *Respondent should select “Other” only if all other options don’t apply Match 3 What is your role in the CGS? 1. Director of the CGS 2. President of the supply/food store 3. Administrative secretary 4. Responsible for canteen 5. Responsible for Scofi 6. Other (Specify:________) I__I Age1 Can you state your age? 0. No  “gender” 5. Yes  “age2” I__I *If Yes, move to “age2” *If No, move to “gender” *Choose only one option Age2 How old are you? …… *AGE INTERVAL between 12 and 99 Gender What is your gender? 3. Male 4. Female I__I *Ask only if necessary *Select only one option edu What is the highest level of education you have completed? 11. None 12. Some primary school but did not complete primary school 13. Completed primary school 14. Some secondary school but did not complete secondary school 15. Complete secondary school I__I *Select only one option IMPAQ International, LLC Page D-64 Mali FFE Midline Report July 2018 16. Some university but did not complete university 17. Earned bachelor’s degree 18. More than bachelor’s degree 19. Professional school 20. Koranic school or medersa 21. Other (specify: ________) TRAINING AND KNOWLEDGE OF CGS Good! Now, I would like to ask you a few questions on the training that you received and your knowledge… Train1 Does the CGS manage the school canteen? 0. No 1. Yes I__I *Select only one option Train2 Did the CGS members receive training since 2016? 0. Non  Train4 1. Oui  Train3 I__I *Select only one option If Yes, move to Train3 If No, move to Train4 Train3 In what fields have the CGS members received training? 1. Food management 2. Health, hygiene, and nutrition 3. Democratic set-up of CGS 4. Roles and responsibilities of CGS 5. Annual action plan development 6. Resource mobilization 7. Follow-up and evaluation strategy 8. Other (Specify :__________________) I__I I__I I__I I__I I__I I__I I__I I__I *Do not give examples or mention the list to the respondent *Choose all answers that apply Train4 Can you name the main responsibilities of the CGS? 1. Children follow-up (child retention in school, schooling progress of children, etc.) 2. Teacher follow-up (being present, etc.) 3. Canteen management 4. Resource mobilization for the school (financial and/or material) 5. Hygiene and cleanliness of children 6. Ensure communication between school and community (communication) I__I I__I I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply IMPAQ International, LLC Page D-65 Mali FFE Midline Report July 2018 7. Ensure school development and maintenance (maintenance of buildings, latrines, water check-points) 8. Annual report of activities to the population 9. Advocate for support from the town hall/CAP 10. Other (Specify :__________________) 88. Don’t know I__I I__I I__I I__I Train5 In your opinion, do you think that the responsibilities assigned to the CGS are too heavy? 0. No  train8 1. Yes  train6 I__I *Choose only one option *If Refused, move to “Train6” Train6 In your opinion, which responsibilities should be kept? 1. Children follow-up (child retention in school, schooling progress of children, etc.) 2. Teacher follow-up (presence, etc.) 3. Canteen management 4. Resource mobilization for the school (financial and/or material) 5. Hygiene and cleanliness of children 6. Ensure communication between school and community (communication) 7. Ensure school development and maintenance (maintenance of buildings, latrines, water check-points) 8. Annual report of activities to the population 9. Advocate for support from the town hall/CAP 10. Other (Specify :__________________) 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply Train7 In your opinion, who should be in charge of the responsibilities that should not be kept? 1. School principal 2. Teacher 3. Parents 4. Government/Ministry of Education 5. Other (specify :_____________) 88. Don’t know I__I I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply *Probe based on what they mentioned in the previous question – responsibilities that they IMPAQ International, LLC Page D-66 Mali FFE Midline Report July 2018 you think should not be kept for CGS members Train8 Does the training(s) received allow you to fulfill your responsibilities? 1. All 2. Mostly 3. Some 4. None I__I *Read the list to the respondent *Choose only one option Train9 In your opinion, what are the best practices for food storage? 1. The bags should be at one meter from the wall and roof 2. The bags should be placed on elevated palettes/platforms 3. The shop should be swept 4. The shop should be well-ventilated 5. The shop should be well secured 6. The food should be classified by type 7. The food should be well stacked to facilitate the inventory (not mixed) 8. Other (Specify :__________________) 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply Train10 In your opinion, what are good practices for food preparation, from a hygiene perspective? 1. Keep the surfaces that are used to prepare food clean 2. Wash vegetables, fruits and ingredients with drinking water 3. Cook meat, fish thoroughly 4. Respect the steps in food preparation 5. Never mix raw food and food already cooked 6. Never save meals to be warmed up and served the next day 7. Serve meals of the day when they are hot 8. Not let meals exposed to the open air 9. Place meals in clean plates/cups 10. Other (Specify :__________________) 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply IMPAQ International, LLC Page D-67 Mali FFE Midline Report July 2018 handwas h In your opinion, when should someone wash their hands? 11. Before eating 12. Before touching or preparing food 13. Before giving food to someone 14. When hands are dirty 15. After touching a dirty object 16. After touching a pet 17. After using the latrines 18. After changing a baby’s diapers 19. Before prayers 20. Other (Specify :_________________________) 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply Hand1 How many times did you wash your hands yesterday? ……. *Interval between 0 and 20 *If 0, move to « Hand3 » IMPAQ International, LLC Page D-68 Mali FFE Midline Report July 2018 Hand2 What were the reasons? 1. Before eating 2. Before touching or preparing food 3. Before giving food to someone 4. When hands are dirty 5. After touching a dirty object 6. After touching a pet 7. After using the latrines 8. After changing a baby’s diapers 9. Before prayers 10. Other (Specify :_________________________) 88. Don’t know I__I I__I I__I I__I I__I I__I I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply Hand3 What do you usually use to wash your hands? 1. Just water 2. Water plus soap 3. Other (Specify :_____________________) I__I * Do not give examples or read the list to the respondent *Select only one option Worms In your opinion, how can one avoid to catch intestinal worms (in the belly)? 1. Avoid walking bare foot (wear shoes) 2. Not to bathe or swim in stagnant water 3. Eat meat that is well cooked 4. Avoid contact with contaminated water, and if necessary to wear boots and gloves 5. Wash hands with drinking water and soap before preparing food, before serving food or before eating 6. Wash hands with drinking water soap after using the latrines 7. Protect food from flies, cockroaches, and dust 8. Store food in a pantry or a place that is clean and well-ventilated 9. Other (specify :______________________) 88. Don’t know I__I I__I * Do not give examples or read the list to the respondent *After the respondent has given one way, incite the respondent to give a second way: What other way can help avoid intestinal worms? Incite for 2 ways in total SCHOOL CANTINE MANAGEMENT IMPAQ International, LLC Page D-69 Mali FFE Midline Report July 2018 Thanks! Now, I would like to ask you a few questions about the general assemblies and management of the school canteen… SMC1 Since the beginning of the school year, which is since last October, how many general assemblies have been organized by the CGS between parents and students to discuss school life? 0. None 1. 1 to 3 general assemblies 2. More than 3 general assemblies 88. Don’t know I__I *Don’t read the list to the respondent *Choose only one option SMC13 During the last three months, that is since February, has the community ensured the maintenance of the shop? 0. Non 1. Oui I__I *Choose only one option SMC14 During a week of canteen, how many days did parents/students contribute for the wood? 0. Aucun None 1. 1 to 2 days 2. 3 to 6 days 3. Every day 88. Don’t know I__I *Don’t give examples or read the list to the respondent *Choose only one option SMC15 During one week of canteen, how many days have parents contributed to condiments (vegetables, salt, potassium, etc)? 0. None 1. 1 to 2 days 2. 3 to 4 days 3. All of the 5 days 88. Don’t know I__I *Don’t give examples or read the list to the respondent *Choose only one option SMC15 _a Since the beginning of the school year, did the parents contribute to condiments (vegetables, salt, potassium, etc) of the cantine? 0. No 1. Yes I__I *Choose only one option IMPAQ International, LLC Page D-70 Mali FFE Midline Report July 2018 SMC17 Since the beginning of the school year, that is since last October, have you prepared your annual school action plan? 0. No 1. Yes I__I *If Yes, move to “SMC18” *If No, move to “Edu1” *Select only one option SMC18 Since the beginning of the school year, that is since last October, what is the level of implementation of the annual school action plan (the overall activities to do at school during the year) : 1. 0% 2. 1-25% 3. 26-50% 4. 51-75% 5. 76-100% I__I *Do not read the list to the respondent, give examples if necessary: school garden, choral singing, fencing work, latrine, kitchen, classrooms, etc. *Select only one option TEACHING FOLLOW-UP Edu1 Has CGS been informed of the pedagogical approach of the program used by teachers? 0. No 1. Yes I__I *Select only one option Edu1_ precise Has the CGS been informed about the balanced approach used by teachers? 0. No 1. Yes 88. don’t know I__I *Select only one option Edu2 Does the CGS follow the practices of the teacher? 0. No  edu4 1. Yes  edu3 I__I *Select only one option Edu3 How does the CGS follow the practices of the teacher? 1. Class observation 2. Notebook preparation 3. Asking questions to children 4. Other (Specify :___________________________) I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply IMPAQ International, LLC Page D-71 Mali FFE Midline Report July 2018 Edu4 Does the CGS follow the correct conservation and use of pedagogical materials? 0. No  edu6 1. Yes  edu5 I__I *If Yes, move to “Edu5” *If No, move to ‘’Edu6‘’ *Choose only one option Edu5 How does the CGS follow the correct conservation and use of pedagogical materials? 1. Visiting the local area where materials are stored 2. Inventory 3. Class observation 4. Other (Specify :______________________) I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply Edu6 Does the CGS follow the children’s progress? 0. No  edu7 1. Yes  edu8 I__I *Select only one option Edu7 How does the CGS follow the children’s progress? 1. Results of compositions 2. Community tools of learning evaluations (Beekungo, EGRA lite, other) 3. Colored bulletins 4. Other (Specify :______________________) I__I I__I I__I I__I *Do not give examples or read the list to the respondent *Choose all answers that apply Edu8 Have students’ parents questioned the CGS on the pedagogical approach of teachers? 0. No 1. Yes I__I *Select only one option Edu9 Does the CGS monitor the presence of teachers? 0. No  thanks 1. Yes  edu10 I__I *Select only one option Edu10 How does the CGS monitor the presence of children's teachers? 1. Daily check 2. Keeping an attenance register 3. Follow up with the principal 4. Other (specify: ______________________) I__I *Select only one option thanks Thank you very much for having answered my questions. IMPAQ International, LLC Page D-72 Mali FFE Midline Report July 2018 Administrative and Education Officials Background 1. What is your title? How long have you been with [agency/organization]? Can you tell me about what your role has been related to the FFE project? How long have you been involved with this project? Relevance 2. What do you think is the main goal of the Food for Education project? What, specifically, is it trying to achieve? (Probe on short-term versus long-term goals) Do you think these are reasonable goals? Why/why not? 3. Were you involved or consulted in the design phase? If so, in what ways? Was this adequate? 4. What are the barriers to education in your region (Koulikoro/Mopti)? Are there different barriers for boys and girls? In what ways do you think the FFE project took these socio￾economic, cultural, and political situations into consideration during the design phase? Organization: Title: Gender: IMPAQ International, LLC Page D-73 Mali FFE Midline Report July 2018 5. How well do you think that the FFE project is aligned with your region’s (Koulikoro/Mopti) priorities, policies, programs and laws? 6. Do you think the planning and organization of the FFE project were well-planned and realistic in terms of its objectives, desired outcomes, targets, and timeframe? Why or why not? 7. Since the project began, have beneficiaries’ needs changed over time in a way that has affected the project? If so, how has the project responded to changing needs? Effectiveness 8. To what extent do you think the FFE project is meeting its goals? In what ways, if any, does it fall short? How can it be improved? 9. What outputs or outcomes has the project achieved so far? Are the implementation of some activities more successful than others? If so, which ones? Why? 10. Overall, what are the successes and challenges experienced in the implementation process? How can they be addressed for better achievements in the future? IMPAQ International, LLC Page D-74 Mali FFE Midline Report July 2018 Efficiency 11. To what extent does the FFE project collaborate with your government department/organization? Is this an adequate level of collaboration? Why or why not? How can it been improved? 12. Do you think the project is implemented in the most efficient way? In what ways can it be more efficient? Impact 13. What are the impacts, if any so far, of the activities on communities where the FFE project is being implemented? Which activities do you think have the greatest impacts? Why? 14. Is there a difference in the way that the activities are affecting boys and girls? 15. In your opinion, is there anything about the project that could be strengthened or done differently? What were the overall challenges you see? What are the overall successes of the FFE project? How can they be taken into account for future projects? 16. In what ways, if any, have political conflicts had impacts on community response and commitment to this project? IMPAQ International, LLC Page D-75 Mali FFE Midline Report July 2018 Sustainability 17. From your perspective, which activities and processes will be sustainable beyond project funding and which will not be sustainable? Please explain. What are the biggest challenges to sustainability? Is the project prepared to overcome these challenges? 18. What role, if any, will your agency/organization have in ensuring sustainability of the outcomes? What measures, and which specific efforts, have been undertaken already to prepare for the phase out of the project’s funding? 19. Do you have any recommendations to help ensure sustainability of the project? Conclusion 20. Is there anything that I did not ask about that you would like to share with me, or do you have any additional thoughts about what we have discussed today? Thank you for your time and comments. IMPAQ International, LLC Page D-76 Mali FFE Midline Report July 2018 National Government Stakeholders Background 1. What is your title? How long have you been with [government agency/organization]? Can you tell me about what your role has been related to the FFE project? How long have you been involved with this project? Relevance 2. What do you think is the main goal of the Food for Education project? What, specifically, is it trying to achieve? (Probe on short-term versus long-term goals) Do you think these are reasonable goals? Why/why not? 3. Were you involved or consulted in the design phase? If so, in what ways? Was this adequate? 4. What are the barriers to education in Mali (or region, if talking to regional official)? Are there different barriers for boys and girls? In what ways do you think the FFE project took these socio-economic, cultural, and political situations into consideration during the design phase? Organization: Title: Gender: IMPAQ International, LLC Page D-77 Mali FFE Midline Report July 2018 5. How well do you think that the FFE project is aligned with Mali’s (or the region’s) priorities, policies, programs and laws? Effectiveness 6. To what extent do you think the FFE project is meeting its goals? In what ways, if any, does it fall short? How can it be improved? Efficiency 7. In what ways do you think the FFE project used existing capacity or structures to address the educational and health needs of students? 8. To what extent does the FFE project collaborate with your government department/organization? Is this an adequate level of collaboration? Why or why not? How can it been improved? Impact 9. What impact do you think the FFE project has had so far, if any, in the communities where it is implemented? Which activities have had the greatest impacts so far? 10. In your opinion, is there anything about the project that could be strengthened or done differently? What were the overall challenges you see? What are the overall successes of the FFE project? How can they be taken into account for future projects? IMPAQ International, LLC Page D-78 Mali FFE Midline Report July 2018 11. In what ways, if any, have political conflicts had impacts on community response and commitment to this project? Sustainability 12. From your perspective, which activities and processes will be sustainable beyond grant funding and which will not be sustainable? Please explain. What are the biggest challenges to sustainability? Is the project prepared to overcome these challenges? 13. What role, if any, will your agency/department have in ensuring sustainability of the outcomes? Conclusion 14. Is there anything that I did not ask about that you would like to share with me, or do you have any additional thoughts about what we have discussed today? Thank you for your time and comments. Project Staff/Partners Background Organization: Title: Gender: IMPAQ International, LLC Page D-79 Mali FFE Midline Report July 2018 1. What is your title? How long have you been with [organization]? What are your main responsibilities in your position, particularly related to the FFE project? How long have you been involved with this project? Relevance 2. What do you think is the main goal of the Food for Education project? What, specifically, is it trying to achieve? (Probe on short-term versus long-term goals) Do you think these are reasonable goals? Why/why not? 3. Were you involved or consulted in the design phase? If so, in what ways? What are the strengths of the project’s design? What are its weaknesses? 4. Do you think the planning and organization of the FFE project were well-planned and realistic in terms of its objectives, desired outcomes, targets, and timeframe? Why or why not? 5. In what ways/how well do you think the FFE’s goals fit with the government’s educational and health priorities goals? (national/regional/local) 6. Since the project began, have beneficiaries’ needs changed over time in a way that has affected the project? If so, how has the project responded to changing needs? IMPAQ International, LLC Page D-80 Mali FFE Midline Report July 2018  Effectiveness 7. Are the project’s planned activities proceeding on schedule? What has helped stay on schedule and what has made it difficult? 8. What outputs or outcomes has the project achieved so far? Are expected results occurring as planned? To what extent will the objectives be achieved? 9. Are the implementation of some activities more successful than others? If so, which ones? Why? 10. How effective is the monitoring strategy for collecting regular and reliable data on the work? What are the strengths and gaps in the monitoring system? 11. Overall, what are the successes and challenges experienced in the implementation process? How can they be addressed for better achievements in the future? Efficiency 12. To your knowledge, to what extent are the planned activities being implemented according to the budget? IMPAQ International, LLC Page D-81 Mali FFE Midline Report July 2018 13. What obstacles in allocating budget resources have arose? How were they overcome and at what cost? 14. Do you think the project is implemented in the most efficient way? In what ways can it be more efficient? Are objectives being achieved on time? Why or why not? Impact 15. What are the impacts, if any so far, of the activities on communities where the FFE project is being implemented? Which activities do you think have the greatest impacts? Why? 16. Is there a difference in the way that the activities are affecting boys and girls? 17. What has been impact of the project’s activities on the communities in terms of a. Building organizational capacities (SMC, SILC) b. Raising awareness in parents /teachers/students (for example, on the importance of education, promoting attendance for students and teachers) 18. How are community-based structures (e.g. schools, SMC, SILC) supporting project implementation? Are they on track to assume ownership of key activities beyond the life of the project? Are they satisfied with their participation? How might they be encouraged and/or supported to participate more? IMPAQ International, LLC Page D-82 Mali FFE Midline Report July 2018 19. How have capacity building activities for SMC improved their capacities? What obstacles persist? What more should be done to ensure they will have the capacity to manage the school canteens beyond the life of the project? 20. What innovations, lessons learned and good practices can be documented so far? Sustainability 21. From your perspective, which activities and processes will be sustainable beyond project funding and which will not be sustainable? Please explain. What are the biggest challenges to sustainability? Is the project prepared to overcome these challenges? 22. How has the FFE Project engaged other stakeholders (e.g., government and local organizations) to sustain the project activities and outcomes after the project funding ends? What will be their role be in sustaining the outcomes of the project at the district and community levels? What systems do you think they have in place to sustain the outcomes? 23. Do any socio-cultural or political aspects endanger the sustainability of the project and what actions are being taken to sensitize local institutions and target groups to these issues? 24. What measures, and which specific efforts, have been undertaken already to prepare for the phase out of the project’s funding? IMPAQ International, LLC Page D-83 Mali FFE Midline Report July 2018 25. Do you have any recommendations to help ensure sustainability of the project?    Conclusion 26. Is there anything that I did not ask about that you would like to share with me, or do you have any additional thoughts about what we have discussed today?  Thank you for your time and comments. IMPAQ International, LLC Page D-84 Mali FFE Midline Report July 2018 Parents: Focus Group Discussion • Here as researchers for the FFE project. We are not funding any programs or school, we are providing feedback to program implementers • After we finish, we will ask parents who participate in the SMC to stay a few minutes longer to answer additional questions about their activities in the group • Everything is confidential, we will not record anyone’s name or share anything they say with teachers, principals, or any other community members. Respect each other and do not repeat this conversation outside of here • No right or wrong answers – it’s ok to disagree, because we want everyone’s opinion. Everyone should speak freely, and respect each other • We will be here for one hour • You do not have to answer a question if you do not want • Can we record the discussion for notes? • Do you have any questions for us before we begin? 1. In some communities, not all children are able to attend school on a regular basis. Does this happen in your community? Are there some children who attend school more than others? What prevents some children in this community from going to school? (Probe to understand if there are different things that prevent boys and girls from going to school). 2. Have any barriers to sending children to school been addressed through the FFE program activities (such as take home rations or daily hot meals)? What about parents’ attitudes towards the value of education? (Spend time on this question and probe – for example, if they parents say “school feeding” ask specifically what the previous barrier was, and how the project is helping. Try to get a lot of responses here – keep following up, “is there anything else?”) School: Gender: Number of Participants: IMPAQ International, LLC Page D-85 Mali FFE Midline Report July 2018 3. What do you think would encourage children in your community to go to school more often? What are the specific encouragements needed for girls? For boys? (Spend time on this question and probe – for example, if they parents say “money” ask specifically what the costs are, and why they are prohibitive. Try to get a lot of responses here – keep following up, “is there anything else?”) 4. What do you like about the school your child attends? What, if anything, could be better? Do you know your child’s teacher or teachers? What do you like (or not like) about your child’s teacher or teachers? 5. How far do most children in this area get in school? Elementary school? Beyond? Do most children in the community go as far in school as they would like? If not, what stops them? 6. How far would you like to see your own child/children go in school? Why? What difference will it make it their lives to have this level of education? Do you think your attitude in this is different or similar to other parents in your community? 7. Do you encourage your children to study at home? Why? Why not? How else are you involved in your child’s education? Is this typical of families in your community? 8. What were you hoping [activity/program] would do for your family? For your community? Has [activity or program] met your expectations? Why or why not? IMPAQ International, LLC Page D-86 Mali FFE Midline Report July 2018 9. Do you think [school feeding/take home rations] support children to go to school? Why or why not? (Probe for any differences between boys and girls) 10. From your perspective, does the overall program adequately address the needs of children and their families? Why or why not? What would you do differently to better support children and their families? 11. Overall, have you had a positive or negative experience in [activity or program]? Please explain. 12. Is there anything that I did not ask about that you would like to share with me, or do you have any additional thoughts about what we have discussed today? Thank you for your time! If you are a member of the SMC, I will ask you to please stay a few more minutes for some additional questions related to your activities with this group. If you are not a member, you may now leave. IMPAQ International, LLC Page D-87 Mali FFE Midline Report July 2018 School Management Committee: Focus Group Discussion Thank you for staying. I am going to ask you 1. What SMC responsibility do you like the most? Why? 2. What SMC responsibility do you like the least? Why? What is the most challenging aspect? 3. Have you been a part of or seen any successful activities through your work as a member of the SMC? How did it make you feel? What aspects of the activity(ies) do you feel were the most successful? (Probe: were there any successful outcomes that were surprising to you?) 4. Were there any activities the SMC tried to do this year or last year which were not successful? Which were the least successful? What needs do you think are still unmet? (Probe: were there any negative outcomes that were surprising to you?) 5. From your perspective, does the overall program adequately address the needs of children and their families? Do you think that children in your community have access to quality education? Why or why not? What would you do differently to better support children and their families? School: Gender: Number of Participants: IMPAQ International, LLC Page D-88 Mali FFE Midline Report July 2018 6. In what ways, if any, has the FFE project improved your community? What about the abilities and capacities of your SMC? 7. In your opinion, is there anything about the project that could be strengthened or done differently? 8. From your perspective, which activities and processes will be sustainable beyond project funding and which will not be sustainable? Please explain. What are the biggest challenges to sustainability? Is the SMC prepared to overcome these challenges? 9. What role, if any, will your SMC have in ensuring sustainability of the outcomes? What measures, and which specific efforts, have been undertaken already to prepare for the phase out of the project’s funding? What support does your SMC need to manage activities in the future? 10. Do you have any recommendations to help ensure sustainability of the project? 11. Have you learned anything from participating in the SMC? Overall, has it been a positive or negative experience? Why? Thank you for your time!