McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 1 of 46 December 2017 Title: Project (FFE-116-2017/024-00) Name: Mercy Corps Year: 2017-2021 Baseline Report: Phase I March 2018 McGovern-Dole International Food for Education and Nutrition Program Food Assistance Division, Office of Capacity Building and Development McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 2 of 46 December 2017 Table of Contents List of Graphs and Tables......................................................................................................... 3 Methodology
......................................................................................................................... 5 Primary Data Collection ................................................................................................................... 5 Limitations....................................................................................................................................... 9 Baseline Results..................................................................................................................... 11 Implications for Program Implementation............................................................................. 35 Attachment I: FGD Participant Profile.................................................................................... 38 Attachment II: Additional Literacy Assessment Results ......................................................... 40 Attachment III: Map of Baseline Assessment Locations......................................................... 46 McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 3 of 46 December 2017 List of Graphs and Tables Table 1: Baseline Sample Size ....................................................................................................................... 5 Table 2: Literacy Assessment Sample Georgaphic Distribution.................................................................... 6 Table 3: Phase I Schools Geographic Distribution ........................................................................................ 8 Table 4: Phase I School Administration Members...................................................................................... 15 Table 5: Reasons for Missing School........................................................................................................... 19 Table 6: Home Learning Environment ........................................................................................................ 33 Figure 1: Reading Assessment Sample by Age and Gender.......................................................................... 7 Figure 2: Phase I Primary Grade Students Gender Disaggregation ............................................................ 11 Figure 3: Phase I Primary School Total Enrollment..................................................................................... 11 Figure 4: Phase I Prmary School Enrollment by Province ........................................................................... 12 Figure 5: Phase I Feeding Programs in 2017-2018 School Year.................................................................. 14 Figure 6: Phase I Recent School Administration Training ........................................................................... 16 Figure 7: Phase I Existing Kitchen Equipment............................................................................................. 17 Figure 8: Self-Reported Attendacne for Second Grade Students for Past Week........................................19 Figure 9: Number of Full-Time Cooks ......................................................................................................... 23 Figure 10: Full-Time Cooks' Education........................................................................................................ 23 Figure 11: Type of Water Source in Phase I Schools................................................................................... 24 Figure 12: Type of Sanitation Faciliaties in Phase I Schools........................................................................ 25 Figure 13: Number of Toilets per School .................................................................................................... 25 Figure 14: Types of Activities Funded by School Fiduciary Boards............................................................. 27 Figure 15: Types of Readers in Literacy Assessment .................................................................................. 28 Figure 16: Type of Readers by Gender........................................................................................................ 28 Figure 17: Comprehension Results by Language ........................................................................................ 29 Figure 18: Classroom Resources................................................................................................................. 29 McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 4 of 46 December 2017 Figure 19: Teacher's Activities During Class................................................................................................ 30 Figure 20: Classroom Observation Results ................................................................................................. 31 Figure 21. Home learning activities with household members.................................................................. 32 Figure 22. Number of household members who engage in learning activities with the child...................33 Figure 23. Participation in community literacy activities............................................................................ 34 Figure 24. Subtask average percent correct ............................................................................................... 40 Figure 25. Average Letter ID percent correct by gender and class............................................................. 41 Figure 26. Most-Used Words percent correct by gender (n=768).............................................................. 41 Figure 27. Decodable words percent correct by gender ............................................................................ 42 Figure 28. Percentage of students able to read at least five words in 30 seconds on Russian and Kyrgyz reading passages, by gender and class language................................................................................ 43 Figure 29. Oral reading fluency (words correct per minute) on Kyrgyz and Russian reading passages by gender and class language.................................................................................................................. 44 Figure 30. Distribution of Oral Reading Fluency scores in class language.................................................. 45 McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 5 of 46 December 2017 Methodology The primary purpose of this baseline is to establish initial benchmarks against which McGovern-Dole Food for Education 2017 performance will be measured, whilst further informing the program and indicator design. These data will be used throughout the program implementation cycle in following ways: • Adapt program design and adjust initial indicator targets to be more reflective of the existing conditions in the target communities; • to identify challenges to program implementation and to make recommendations to address those challenges; and • To establish benchmark values against which change in outcomes will be measured at the midterm and final evaluation stages and ensure that program implementation is on track. This baseline assessment covered 151 new schools that will be introduced to the Phase I McGovern-Dole Food for Education in 2018. As described in the M&E Plan, the baseline assessment employed a mixed￾methods approach and consisted of three main primary data collection components: 1. A structured questionnaire survey in all 151 schools; 2. A structured literacy assessment and classroom observation in a sample of 50 schools; and 3. A series of FGDs with parent and caretakers in a sample of 32 schools. In addition, Mercy Corps conducted a review of available secondary data on national statistics relevant for several indicators (mentioned throughout the analysis). For some indicators, the baseline value was established based on previous Mercy Corps program results in Kyrgyzstan that used identical indicators. Another round of baseline data collection will be conducted in 2019 for approximately 190 Phase II schools. Primary Data Collection Primary data for this baseline assessment was collected by three teams: • Mercy Corps Program Officers, who were responsible for conducting FGDs with parents and other caretakers; • A local data collection company hired by MC and responsible for surveying 151 selected schools; • Save the Children program staff and 20 enumerators who focused on conducting ALOT and classroom observations. Program M&E Plan provides more details on the data collection tools used. Total sample size and types of respondents are presented in the table below: Table 1: Baseline Sample Size Data collection method Respondent Total sample Structured questionnaire School directors 151 McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 6 of 46 December 2017 Literacy assessment For the literacy assessment, Save the Children randomly selected 50 schools from a total of 223 schools to be included in Phase I of the program: 84 schools currently in FFE12 Ext and 139 schools selected for Phase I. To account for the larger average size of schools in the Southern region, which tend to be larger than in the North due to higher population density, 30 schools out of 50 were chosen from Southern provinces (Osh, Jalal-Abad, and Batken). The geographic composition of the literacy component sample is presented in the table below: Table 2: Literacy Assessment Sample Geographic Distribution Oblast Schools Children enrolled in primary grades Batken 5 80 Chuy 14 195 Issyk-Kul 1 16 Jalal-Abad 15 240 Naryn 1 16 Osh 10 160 Talas 4 61 TOTAL 50 768 Within each school, enumerators attempted to randomly select and interview 16 students. If fewer than 16 students were available, we interviewed all of the available students. We strove to derive a representative sample of Grade 2 students, so when more than one second grade class existed at a school, we sampled as evenly from each class as possible. The sample was not purposefully stratified by age or gender. The figure below demonstrates the composition of the sample by age and gender. Nearly all children in the sample were seven to nine years old, with a few younger and older students. FGD Parents of primary grade students 329 ALOT Second grade students 768 Structured classroom observation Primary grade teachers 43 McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 7 of 46 December 2017 Figure 1: Reading Assessment Sample by Age and Gender The test was conducted using ALOT methodology, which provides tools to measure critical components of children’s early grade reading, including components skills such as oral reading fluency and non-word reading, and a robust measure of reading comprehension. Save the Children has previously used ALOT in their work with American Institutes for Research, and it is very similar to the Early Grade Reading Assessment (EGRA), which was previously used in Kyrgyzstan, but includes more comprehension questions and more precise translations. However, the reading passages in Kyrgyz and Russian in this test were taken from the previously used EGRA since they had gone through a thorough validation process. In addition to the sample for analysis, we interviewed 34 additional students with two enumerators at the same time to assess the inter-rater reliability of the instruments. Overall, we found that the instruments performed excellently, with a very high degree of agreement between enumerators. 1 School Survey A local data collection company specializing in social research for international NGOs and donors in the Kyrgyz Republic conducted a baseline assessment in Phase I schools. The company was selected through a competitive process, which included a screening of 11 total applications. After the selection, MC M&E and program staff developed tailored training materials for the team of 10 highly experienced enumerators. Since most of them are based in their respective oblasts, they were invited for a 2-day workshop in Bishkek for the training and questionnaire pilot. The workshop was conducted by MC on Jan 31 – Feb 2, 2018. The pilot took place in two schools on the outskirts of Bishkek, where the conditions resemble those in the target rural areas. The team and MC then came back to the office to review the results and answer any remaining questions. The questionnaire was updated based on the feedback from the team. A total of 151 schools were assessed, of which 12 are currently in reserve and will be included in Phase II in 2019. During preparatory phone calls to the schools to explain the purpose of the survey and schedule an interview, Mercy Corps learned that one of the selected schools decided to postpone their participation 1 An inter-reliability analysis was performed after the data collection. 2 197 503 63 2 1 0 100 200 300 400 500 600 6 7 8 9 10 11 Number of children Age 52% 48% Boys Girls McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 8 of 46 December 2017 in the program until Phase II. At this point, 1 school from the reserve group was moved to Phase II, which reduced the total number of schools surveyed to 151 from the initially planned 152. The table below shows distribution of schools by province. Overall, the majority of assessed schools are located in Chuy province (67 out of 151). Table 3: Phase I Schools Geographic Distribution Province Phase I schools Reserve North Naryn 11 0 Chuy 60 6 Talas 3 0 Issyk-Kul 9 1 South Batken 8 0 Jalal-Abad 34 2 Osh 15 2 Total 140 11 Exact locations of Phase I schools, main and reserve, can be seen in Attachment III. Field data collection in the target schools took place over the period of February 5-22, 2018. On average, each visit took about 1.5 hours, including observations. The interviews were conducted in either Russian or Kyrgyz, depending on the respondents' preference. Overall, field data collection went smoothly and without any major challenges or delays despite unusually cold weather, which led to a week-long nationwide school closure in late January and early February 2018. The few challenges that were encountered were resolved on the spot: • Several schools selected for Phase I are located in extremely hard-to-reach areas, particularly in the winter, which led to unexpected changes in the schedule to account for the road conditions. However, it did not impact the overall schedule and the work was completed on time. • Data entry mistakes were identified on an ongoing basis by both the contractor's PM and MC and corrected right away. The following week, Mercy Corps and the enumerators conducted two rounds of data verification and cleaning. The final dataset was submitted for analysis on March 2, 2018. McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 9 of 46 December 2017 Focus Group Discussions (FGDs) During the same period, Feb 5-22, Mercy Corps Program Officers conducted 32 FGDs with parents and other caretakers of primary school students enrolled in target schools. Each team consisted of two Program Officers with experience in field data collection and delivering workshops and trainings in a school setting. Sampling for FGDs consisted of several steps. First, the total sample size was calculated assuming the total target population of parents and caretakers for Phase I to be 1,750 people (estimated target for this activity). At 95/5 confidence level and interval, the sample size was 315 people. Since a typical FGD consists of no more than 10 participants, this yielded a total of 32 groups, which were proportionally allocated to the two regions based on the number of Phase I schools: 12 in the South (40% of the total number of schools) and 20 in the North (60%). Lastly, target schools were randomly selected from the list of schools for each region. 10 parents or caretakers of primary school students were invited to participate in the discussion, and in some cases groups consisted of up to 12 participants, which yielded a total of 329 people. Efforts were made to ensure representation on the school level, however such external factors as availability and willingness to participate were also taken into account. While this sample is not meant to be representative of the entire target population, it is possible to draw reliable conclusions about the situation in the community and several neighboring communities due to their homogeneity evident in the previous school feeding programs implemented by Mercy Corps. MC M&E and key program staff developed a semi-structured guide for the discussions, which was designed for a 1.5-hour conversation2 and covered three main topics: • General knowledge and awareness of good child nutrition practices and correlated health issues; • Nutrition practices at home; • Understanding of and attitudes towards a school feeding program, including interest in and ability to support one at the school. The data were then analyzed by topic and by region to be used in updating and tailoring training modules for parents and caretakers. Limitations • The primary limitation of this baseline assessment is in its generalizability. While the school survey covered every school selected for Phase I, including 12 reserve schools, the findings from the qualitative data are not meant to be representative of the entire target population, but are limited to the specific communities the data were collected in. The sampling strategy for the literacy component ensured that the sample is representative of children at schools in Phase I targeted 2 This time limit was decided upon in consultation with the Mercy Corps team and target schools’ director to be compatible with the invited participants’ busy schedules during the day since it is not customary to conduct FGDs or workshops in the evening. McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 10 of 46 December 2017 by Save the Children. We cannot definitively state that our findings would hold for children studying in Phase II schools. o Despite this limitation, given that selection criteria and geographical reach will be the same for both phases of the project, it is unlikely that there would be any meaningful differences between those schools and the schools included in this study. o Alternatively, if there are systematic differences between the schools in two phases, another baseline literacy assessment could be considered. • The classroom observation comes with a separate set of limitations. An observation influences the behaviors of the teacher and children. The lessons that we observed and report in this report may be the examples of “best case” of the teacher’s lessons rather than typical of the day-to-day reading lesson children experience and findings should be interpreted with some caution. • Finally, all associations in this report are inherently correlational and we do not assert any causal inference to these findings. McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 11 of 46 December 2017 Baseline Results This section outlines key baseline results for the standard USDA Performance Indicators outlined in the Performance Monitoring Plan. Where necessary and possible, further analysis was conducted to look into relevant disaggregation and provide more details to support programmatic adjustments discussed in the last section of this report. The Performance Monitoring Plan with updated indicator targets and baseline values is submitted as a separate attachment. Number of students enrolled in schools receiving USDA assistance The total number of students in four primary grades in all 151 surveyed schools is 39,159. However, 6,695 students are enrolled in 12 schools that are currently on the reserve list for various reasons and will most likely be included in Phase II in 2019. Therefore, 32,464 students are enrolled in the 139 target schools selected for Phase I, which is slightly higher than the initially estimated 32,0003 . Overall, there are slightly more boys than girls enrolled in primary grades in the target schools. Figure 2: Phase I Primary Grade Students Gender Disaggregation Overall, school sizes range from small schools with only 30-50 primary grade students to significantly larger ones. The total distribution of schools by size and by region is presented in the chart below. Figure 3: Phase I Primary School Total Enrollment (Grades 1-4) 3 However, if we add the number of primary grade students in the 84 schools that are already in the program and will be rolled over to Phase I, the total number of students will be slightly over the total target of 48,000 for FY 2018 (currently at 50,730 students at the end of 2017-2018 school year, but it might change slightly for the nextschool year with new enrollment). 51.6% 48.4% Male Female 0 5 10 15 20 25 30 30-49 50-99 100-199 200-399 400-599 600-800 Number of schools Number of primary grade students North South McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 12 of 46 December 2017 Regional analysis showed that, although there are fewer schools selected in the Southern provinces (58 out of 139), the total number of students is higher than in the North because the schools are larger on average. 55.3% of all primary grade students in the target schools are located in the three Southern provinces: Jalal-Abad, Osh, and Batken. The chart above also shows that the majority of smaller schools are located in the Northern region, while the largest schools are in the South. This distribution is consistent with the current population density in the Kyrgyz Republic, which is significantly higher in the South. Further analysis of school size by province shows that most smaller schools are located in Chuy province, while larger schools tend to be located in Jalal-Abad, and Osh provinces. As mentioned previously, Chuy province also has the highest number of target schools for Phase I – 59 out of 139. Figure 4: Phase I Primary School Enrollment by Province There is no significant difference in gender distribution between Northern and Southern schools, with male students being slightly more prevalent than female across all schools, as shown in the chart above. In practice, school size is often directly correlated with available infrastructure and resources relevant for implementing school feeding program, such as the average size of school kitchen and dining area, 0 5 10 15 20 30-49 50-99 100-199 200-399 400-599 Number of schools Number of students School Size by Province: North Chuy Issyk-Kul Naryn Talas 0 2 4 6 8 10 12 14 50-99 100-199 200-399 400-599 600-800 Number of schools Number of students School Size by Province: South Batken Djalal-Abad Osh McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 13 of 46 December 2017 sanitation facilities, key administration staff, etc. Baseline data and more details on these variables is provided under relevant indicators in this section. Number of school-age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance This indicator is directly related to the indicator above since all the children enrolled in primary grades (1- 4) will be eligible for receiving daily school meals, therefore the target for this indicator is established as identical to total enrollment – 32,464 students. Furthermore, some schools might choose to include students in prep classes4 in their school feeding programs, as some have done during the course of FFE12. However, the government budget is not allocated for these students since they are not officially enrolled yet, and the USDA-provided supplementary commodities are not planned for them for the same reason. In this case, school feeding for prep students is only possible with additional help from their parents and the community. While not all schools will be able to arrange sufficient support in this case, there is precedent, and the baseline shows that all but two of Phase I schools have prep classes for the entire school year (480 hours) and plan to continue enrolling prep classes every year for the foreseeable future. On average, including prep class students into the feeding program would mean a 20% increase in the total number of students receiving school meals. Number of daily school meals (breakfast, snack, lunch) provided as a result of USDA assistance Under the current Kyrgyz law5 , all public schools in the country are obligated to have a feeding program for primary grades (1-4), which is funded by the government. However, the current law only covers a serving of one glass of tea and one bun or muffin per student. Indeed, the baseline assessment confirmed that 100% of schools selected for Phase I have this feeding program in place and receive government funding: either 7 or 10 KGS per child per day (more for high-altitude locations). However, some schools have already started providing full hot meals to primary grade students on their own initiative, using funds from the parents and local government. Of the 151 surveyed schools, 27% supplemented the standard tea and bun menu with hot meals at least 1-2 days per week, while further 7% of schools used the government budget allocated to them under the law mentioned above to create a full 5-day hot meals menu. 4 Prep classes are part of the new MoES-supported preschool education program where children aged 5-7, who are getting ready to enroll in first grade the following school year, have the option of attending preparatory classes at that school a year in advance. The current program consists of 480 academic hours and includes reading, writing, and math classes. While the teacher salaries are funded from the national budget, all other related expenses, including school meals, are covered by the parents. 5 “Establishing School Feeding Program in Public Schools in the Kyrgyz Republic”, June 27, 2002. McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 14 of 46 December 2017 Figure 5: Phase I Feeding Programs in 2017-2018 School Year The majority of schools (90%) only use government funds to provide school meals, while the remaining schools collect money from parents in order to diversify meals provided to primary grade students. The baseline value for this indicator will still be established at zero since the hot meals provided in these schools are not supported by USDA at the time of data collection. Number of school administrators and officials trained or certified as a result of USDA assistance Over the course of the program, Mercy Corps plans to organize a series of trainings and workshops to the key members of school administrations, namely director, deputy director, accountant, nurse, and supply and operations manager. These trainings will cover the basics of establishing and managing a school feeding program, as well as topics relevant for their immediate job functions, such as accounting, project management, etc. However, it is important to highlight that the number and type of key administration members depends on the school size. Specifically, in smaller schools it is possible to only have a director and a deputy, while larger schools are likely to employ a full-time accountant and several operations and maintenance managers. The baseline results for our target 139 schools reflect this, with 22 smaller schools only employing a director and a deputy director, who often also functions as a teacher. This factor was taken into account in revision of the target for this indicator – two administration members from these 22 schools will be invited to trainings instead of the initially estimated five. The table below provides total numbers of key school administration staff currently employed full-time in all 139 target schools, excluding kitchen staff which is described in more detail under the relevant indicator. 7% 66% 27% Feeding Program 2017-2018 School Year Tea & bun Hot meals Both McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 15 of 46 December 2017 Table 4: Phase I School Administration Members Administration Member Total Number Director 139 Deputy Director 139 Operations and Supply Manager 112 Deputy Director for Primary Grades 36 Nurse 18 Accountant 7 Total 451 Assuming no major changes in school staff for the next academic year, the target for this indicator for Phase I will be established at 451 school administration members6 . Furthermore, Mercy Corps gathered information on the previous exposure and awareness of the key topics relevant to the program by asking whether school administration had training on any of the topics listed below in the last five years. The results show that 100% of the target schools have had at least one training or a professional seminar on relevant topics in the last five years, with the majority of school administrators having participated in some form of literacy or teaching skills training. It should be noted that is it not uncommon for members of school administration to also work as teachers, particularly in smaller schools that do not get allocated dedicated staff members for certain roles, such as accounting and methodological work7 . 6 It should be noted here that in some cases certain positions exist (i.e. funded), but were vacant at the time of the baseline data collection. The question asked specifically about school admin members currently employed by the school. 7 56% of the target schools surveyed in this baseline reported having at least one member of school administration teach on a regular basis. McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 16 of 46 December 2017 Figure 6: Phase I Recent School Administration Training The baseline results show that while some schools have already received training and/or orientation seminars on the topics pertinent to establishing and managing a hot meals program, such as safe food storage and handling or child nutrition, most schools have not. In addition, 76.3% of all surveyed schools have had some experience working with international NGOs and 68.4% have had to apply for various infrastructure grants. This finding shows that the majority of school administrators should be aware of the requirements and procedures they will have to follow when working with Mercy Corps on establishing a hot meals program. Number of education facilities rehabilitated/constructed as a result of USDA assistance (kitchen infrastructure and equipment) This indicator assumes that every school that participates in the program will have their kitchen unit rehabilitated in some way with USDA support; therefore, the target also assumes 100% coverage. However, the scope of rehabilitation will vary for each school, from building a new kitchen where the existing ones are too small to install new kitchen equipment. Mercy Corps grants committee will work with each school to develop a tailored kitchen rehabilitation plan taking into account baseline results and available resources. If it becomes evident that, due to lack of resources, a school is not able to conduct sufficient rehabilitation activities in order to meet the national standards for a school feeding program, it will be moved to Phase II and replaced with a suitable school from the reserve list. Mercy Corps’ experience with implementing previous programs shows that approximately 5% of schools need more time to gather sufficient resources for kitchen rehabilitation. It should be noted that 85.5% of all school buildings assessed in this baseline (151) are considered to be “standard”, i.e. designed and constructed to be a school from the beginning as opposed to a communal building being adapted for educational purposes out of necessity. In practice, this means that these schools have sufficiently large kitchens and dining hall areas, as well as necessary infrastructure (water supply, sewage, heating systems, etc.). However, it is also possible that even though the infrastructure 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Teaching skills, literacy instruction Budget planning, financial management Grant application and management School feeding progrm management WASH Good nutrition, child nutrition Other Food safe storage and handling Recent School Administration Training McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 17 of 46 December 2017 exists, it may not be fully functional due to a variety of reasons. In this case, Mercy Corps will focus on repairing and complementing existing facilities. 12% of schools have kitchen units smaller than 15 sq. m, which is the minimum required size to be able to install the necessary equipment for preparing hot meals, according to the MoES regulations8 . In terms of infrastructure and equipment, more than half of all the schools (64%) have water heating units in the kitchen, with only three schools relying solely on central heating, which can be unreliable and is often shut off during the summer. About a third of schools (33%) do not currently have access to hot water in the kitchen. In terms of kitchen equipment required for preparing daily hot meals in accordance with all the rules and regulations of the national center for disease control and prevention, the situation varies from almost nothing to a full set, as can be seen in the chart below. Figure 7: Phase I Existing Kitchen Equipment As can be seen, not even half of all the schools have a dedicated sink large enough for washing dishes and produce that meet national requirements for a school feeding program. In fact, even simple dishware is not always sufficient, which means that students sometimes have to share glasses and plates9 . At the same time, the majority of schools are equipped with at least a small fridge and a stove, often left over from the school feeding system that existed under the Soviet Union. Further analysis shows that only 11% of the target schools currently have a set of kitchen equipment, while 70% have at least the bare minimum (a fridge and a stove). 8“Establishing School Feeding Program in Public Schools in the Kyrgyz Republic”, June 27, 2002. 9 While rare, this was also mentioned in one of the FGDs with parents without prompting form the moderators, who raised their concerns about the risk of spreading diseases in such conditions. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Dishes Stove Fridge Cookware Cutting table Shelves Oven Kitchen sink Kitchen Equipment McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 18 of 46 December 2017 In addition, current norms for hygiene and safe food preparation require school kitchens to have a dedicated sink for the kitchen staff, separate from the dish washing sink, to be able to wash their hands regularly. At the moment, only 62% of schools have at least one dedicated sink for the kitchen staff. Number of students regularly (80%) attending USDA supported classrooms/schools Attendance in public schools, especially in primary grades, is consistently high in the Kyrgyz Republic. However, there is no standardized measure of attendance on the national level, therefore no reliable national statistics. While the attendance data are collected by the MoES regularly, in practice the methodology varies between different districts and the results are generally reported without any verification. At the same time, calculating the baseline value using MGD methodlogy would exceed the scope of the baseline study and significantly increase the time of each interview above the agreed upon one hour10. However, attendance rates were already being collected by Mercy Corps using the MGD methodlogy in the FFE12 program schools, most of which are located in the same districts. Based on the considerations outlined above, it is reasonable to establish the baseline value for Phase I schools as 90%11 and the target at 95% of the total number of enrolled primary grade students(the values in Attachment D are in absolute numbers). A more detailed analysis of the attendance data collected during the literacy portion of this baseline shows that there are, in fact, frequent absences from class. However, they tend to be rather short (less than 20% of the total number of school days) and therefore are not counted under the standard MGD indicator but are included in this report to provide more context on the classroom environment assessed during the literacy baseline. In the sample of second grade students interviewed by SC for the literacy baseline, only 66% reported attending school all five days of the previous week. Nearly one in five children missed at least one day of school in the previous week. 10 Since the baseline was conducted while school was in session, beneficiaries’ work responsibilities and time had to be taken into considertation while scheduling the assessment. 11Average primary school attendance taken during the 2017-2018 academic year (for primary schools only, not including kindergartens). McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 19 of 46 December 2017 Figure 8: Self-Reported Attendance for Second Grade Students for Past Week When asked further about the reasons, the majority of children stated that illness was a factor for missing school. A sizeable number of children also mentioned weather, and a quarter more missed school for other reasons, perhaps related to the nationwide closing of schools due to extremely cold temperatures during the data collection period. Table 5: Reasons for Missing School Work at home 2% Sick 63% School far away 1% Weather 11% Other reason 27% Number of individuals trained in child health and nutrition as a result of USDA assistance This indicator measures the number of health professionals, primary health care workers and community health workers who successfully complete 16-hour training sessions on child health and child nutrition delivered by Mercy Corps over the course of the program. The baseline value for this indicator is set at zero and the target is estimated based on the number of full-time nurses working in schools in Phase I (18) and community-based nurses, who normally serve about 3 schools each on average. Therefore, the new target for Phase I is 58 individuals. Previously, Mercy Corps considered parents of the primary grade students as the main target audience for this training based on the assumption that there were no health care workers available at schools who would be able to attend the training. However, the baseline results show that there are, in fact, full-time nurses working at the target schools or at least in the target locations. Moreover, since they are 66% 15% 8% 11% Self-Reported Attendance Did not miss school Once Twice Three or more times McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 20 of 46 December 2017 government staff, it will be more realistic for them to attend a full 16-hour training than for parents, who are typically busy with work and other activities during the day. Perhaps more importantly, this approach will allow Mercy Corps to follow the rationale behind this indicator – to build human capital of the health professionals and support institutional capacity building in Kyrgyzstan. Nevertheless, Mercy Corps recognizes that the parents of primary grade students play a crucial role in ensuring proper nutrition and health and hygiene practices not only at schools, but also at home. Therefore, Mercy Corps proposed to target parents at the HH level through an adapted behavior change strategy focused on nutrition and child health. The strategy will be implemented through trained community-based change agents selected from the target population, who will be responsible for delivering a series of workshops to their communities on the key topics with close support from the Mercy Corps staff. The strategy is currently being finalized with the technical support from Mercy Corps HQ nutrtion and a behavior change expert and will be shared with USDA before the activities begin. To better understand the current knowledge and practices at home and to establish baseline values, MC proposes to conduct a KAP-based HH survey measuring key issues in child nutrtion and health. After the activities are over, Mercy Corps wil conduct a similar study to measure behavior change. This two-part study is proposed to be conducted as one of the special studies under this program (described in detail in the USDA-approved M&E Plan). In addition, Mercy Corps already collected parents’ perceptions of and thoughts on good nutrition through a series of FGDs conducted as part of this baseline. The purpose of the FGDs was to better understand the overall perceptions and knowledge about child nutrition and school feeding in order to inform training modules design. During the trainings, Mercy Corps will address the following key points identified during the discussions: • Overall, almost everyone had at least a general idea about the common health problems in the country and their connection to nutrition. In particular, anemia was mentioned as the main health problem among young children and women, which is consistent with the 2012 Demographic and Health Survey findings for the Kyrgyz Republic12. Predictably, participants with higher education level tended to be more aware of various nutrition-related health issues. • At the same time, participants did not have clarity on what qualities fresh produce and other commodities should have to be considered nutritious and safe for consumption and how to select packaged items in stores. The most commonly named issue with conventional food items available on local markets and in stores was lack of information on their provenance13, as well as certain concerns about the perceived quality of products imported from China, whether or not supported by evidence. 12According to the survey, 46% of children under five had some form of anemia. Kyrgyzstan Demographic and Health Survey. MoH, 2012. 13National regulations on labels on imported products exist in the Kyrgyz Republic, but are not strictly enforced. It is not unusual to find items with no Kyrgyz or Russian labels or stickers on the package. McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 21 of 46 December 2017 • There seems to be a consistent perception among the participants across all locations that young children are not interested in having breakfast before going to school. Participants gave several possible explanations, ranging from children simply not being hungry in the morning to parents not having enough time to cook before taking children to school. 20% of participants stated in the pre-discussion survey that their children did not eat breakfast that day. o During follow-up discussions, some participants admitted that children may, in fact, experience short-term hunger in the first half of the day since even tea and muffins are not served to all students at the same time due to space and staff limitations in schools: some receive it earlier in the morning, while others have to wait until almost lunch. • Another perception shared by the majority of the participants who were concerned about the idea of healthy food and good nutrition. Most participants were concerned that eating healthy would be very expensive and time-consuming since it requires rare ingredients not commonly available on local markets, and the recipes are perceived to be rather complicated. When prompted further, it appeared that most of the respondents who expressed such concerns did not have first-hand experience with preparing healthy food. • Participants expressed strong interest in learning ways to adapt and substitute for hard-to-find or expensive ingredients in healthy recipes and to provide alternatives to junk food for their children that are easy to prepare taking into account the local conditions and preferences. There is a particular interest in learning more about traditional meals14 in terms of nutritional value and appropriateness for young children, as well as possible alternatives. o This seems to be especially relevant to high-altitude and remote communities that often experience difficulties with accessibility supplies, including fresh produce. • Almost everyone agreed that a hot meals program at school would be beneficial for their children and expressed general willingness to support it. However, when prompted further, respondents were not able to name any specific ways, aside from financial support, they could participate in the process, particularly in terms of monitoring how their contributions are used. o Several participants also had concerns about the quality of food served at schools, which were related to lack of standard training for school cooks and quality monitoring from the relevant government agencies. It should be noted that for the key points outlined above, no major differences were identified between different locations. However, in more detailed conversations about traditional food and culinary preferences there were some ethnic differences15, which will also be taken into account during trainings. 14Namely, rich meat and high-carb dishes that are commonly eaten by even the youngest children, as well as fermented beverages with low alcohol content. Tea is the most popular drink in the country and is consumed by almost everyone regardless of age. 15Primarily Uzbek and Dungan communities. McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 22 of 46 December 2017 Summary of the demographic profile of the parents and caretakers who participated in FGDs can be found in Attachment I. Number of students receiving de-worming medication In the Kyrgyz Republic, the Ministry of Health conducts periodic nationwide deworming among primary school students with nearly 100% coverage. The last round of deworming took place in 2017 and subsequent lab tests showed that the current prevalence of parasitic infections in the population is 18.3%, which is significantly lower than 78.4% recorded during the first national deworming study in 200716. Therefore, the target for this indicator could be established at 100% coverage of all primary grade students, but will be specified when the next round of national deworming is announced. It is not likely to happen in 2018, but could be possible to include as a target in the PhaseII baseline (FY19). At the moment, all values are at zero in Attachment D. Number of individuals trained in safe food preparation and storage as a result of USDA assistance This indicator tracks the activity aimed at providing school cooks with professional training with a focus on child nutrition. Since every participating school must have at least one trained full-time cook on staff, the target for this indicator is established at 139. However, the baseline results show that not all of the surveyed schools currently employ at least one full￾time cook. In fact, 19.2% only have kitchen assistants, while a few other schools have two full-time cooks. This disparity can be explained by the stipulations of the law governing provision of meals (tea and bun) in public schools, mentioned earlier. Specifically, only one kitchen assistant is allocated for schools with fewer than 100 primary grade students, while schools with 200 and more students should have a full-time cook. The difference between the two positions under the current law is that a full-time cook is allowed to prepare hot meals, while a kitchen assistant lacks the necessary training, but is allowed to handle and serve tea and buns or muffins. Taking into account that the Kyrgyz government currently only provides tea and buns to public schools, 10 schools with large number of students chose to spend their allocated budget on several kitchen assistants rather than a cook since higher skill level had not been needed. In such cases, school directors will nominate one of the kitchen assistants to attend the USDA-supported professional training over the summer of 2018 and to be subsequently promoted to a full-time cook. 16Report on Mass Deworming Among Primary School Students in Batken, Jalal-Abad, Osh, Issyk-Kul, and Chuy Provinces of the Kyrgyz Republic. MoH, 2017. McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 23 of 46 December 2017 Figure 9: Number of Full-Time Cooks Further analysis of the baseline data shows that among the currently employed cooks, not everyone has training pertinent to school feeding and/or child nutrition. In fact, only 16% of the cooks do, while the majority (47%) have been trained in other related fields, such as line cook, and more than a third have no professional or technical degree at all. Figure 10: Full-Time Cooks' Education Number of schools using an improved water source At the time of baseline data collection, only 5% of the target schools did not have access to an improved water source. It should be noted that these schools do, in fact, have access to water, but it is either not completely in line with national and international WASH standards or is not sufficiently reliable. For example, one school has a functioning indoor plumbing system, but water is only available for three hours per day. In other cases, plumbing is currently being constructed and schools have to rely on deliveries from outside the village. none one two 19.2% 74.8% 6.0% 0% 10% 20% 30% 40% 50% 60% 70% 80% % of all schools Schools with Full-Time Cooks 37% 16% 47% Cook Education None Child nutrition Other McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 24 of 46 December 2017 However, as can be seen in the chart below, the majority of schools have either a plumbing system or a protected water pump outside, while a quarter of schools have access to both. Figure 11: Type of Water Source in Phase I Schools The baseline value for this indicator is relatively high because access to an improved water source is one of the main criteria for a school to participate in the Food for Education program. Moreover, this requirement is also outlined in the national disease control and prevention regulations regarding school feeding. Therefore, only a few schools require assistance with improving their water source. The baseline value is established at 133, and the target is 139 (100%). Furthermore, 52% of schools currently have hot water, either through central heating or school-based water heating systems. Number of schools with improved sanitary facilities The most common type of improved latrines encountered in the Kyrgyz Republic is a simple pit with reinforced walls and a concrete slab, which is the minimum requirement for a sanitary facility in public schools under current laws. The baseline results confirm this, with 61% of all surveyed schools having this type of latrine. Further 23% have an improved pit latrine with ventilation (VIP latrine). Sewage and septic tank systems are rare, which is not surprising considering the rural location of the majority of our target schools, where sewage systems are not common and septic tanks tend to be rather difficult and costly to construct given the rocky soil in high-altitude regions. Therefore, the baseline value for this indicator is established at 121 (the total number of schools with any kind of improved sanitation facilities at the time of the baseline). 57% 12% 26% 5% Type of Water Source Plumbing Pump outisde Both Other McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 25 of 46 December 2017 Figure 12: Type of Sanitation Facilities in Phase I Schools It should also be noted that slightly under half of the target schools have more than one toilet, sometimes of different structural types. For example, 19% of surveyed schools have toilets both inside the building and outside. During the interview, respondents admitted that indoor toilets are not always functioning for a variety of reasons, so schools choose to have an additional latrine outside, often constructed with limited resources and not in line with international or national WASH standards. Figure 13: Number of Toilets per School Regardless of the type of latrine, sanitary conditions in schools remain poor overall. For example, only 20% of schools have flush water in at least one latrine and only 38% have handwashing facilities either inside or next to the latrines, predominantly without soap. At the time of data collection, 70% of latrines had not been cleaned, meaning that pits or septic tanks were almost or already at capacity. 61% 23% 12% 9% 6% Pit with slab VIP Pit Septic tank Sewage Type of Latrines 56% 28% 6% 10% Number of Toilets per School 1 2 3 4 and more McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 26 of 46 December 2017 Number of Parent Teacher Associations or similar “school” governance structures contributing to their schools a s a result of USDA assistance In the Kyrgyz Republic, the absolute majority of public schools have at least one active Parents’ Committee (PC), which is responsible for monitoring and supporting a number of school functions, including organizing and fundraising for various events and school infrastructure needs. All but one of the Phase I schools currently have at least one PC, of which 87.4% meet regularly during the school year – at least once per quarter and sometimes more frequently. However, since USDA-supported activities have not yet begun, the baseline value for this indicator is established at zero, but the target reflects possible 100% coverage of all target schools given pre-existing school governance structures. Number of public-private partnerships formed as a result of USDA assistance and Value of new public and private sector investments leveraged as a result of USDA assistance Under these indicators, we will consider any cash or in-kind donations and regular contributions to the school feeding program from private citizens and private sector, such as parents of the primary grade students and local businesses. 30% of all surveyed schools have unofficial (voluntary) monthly donations from the parents. However, since the USDA-supported activities have not started yet, the baseline value for the number of public￾private partnerships formed is established at 0, but the target is 139 (100%). The rationale behind the 100% target is that each school will need to have additional contributions from parents in order to be able to provide diverse school meals using supplemental commodities provided by USDA. The baseline also revealed that on average, parents provide approximately 53 KGS per month per child (80 cents) for various school needs. Taking into account the average household income in the target communities17, it is possible to assume that this is a reasonable amount to ask from parents forsupporting a school feeding program. Therefore, this value was used in estimating the target value of new private investments leveraged as a result of the program. The value of public funds will depend on the scope and type of infrastructure work required in each school and is estimated at 1,000 USD per school based on the results of the previous program18. It should also be noted that some schools also have so-called fiduciary boards, responsible primarily for collecting and managing private donations from the local community. In 2014, the Government of the Kyrgyz Republic issued its first law aimed at regulating and governing these groups, including outlining requirements for formal registration and opening a dedicated bank account. Currently, the law is under revision; nevertheless, this is the preferred form of community contributions to school needs, including meals, and Mercy Corps will work to support and provide technical assistance to the remaining target 17As reported by FGD participants. 1830% of the total cost of the project, which is typically around 3,500 USD per school. McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 27 of 46 December 2017 schools to either establish a fiduciary board and/or register an official bank account for already existing ones. At baseline, 65% of the target schools had officially registered fiduciary boards in addition to PCs, of which almost half (46%) already had a bank account. However, according to the interviewed directors, not all of the existing fiduciary boards are currently operational due to lack of funds. At the time of the baseline, the already functioning fiduciary boards allocated funds for the following activities and needs: Figure 14: Types of Activities Funded by School Fiduciary Boards Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text The outcome measure for this indicator is the percent of readers with comprehension due to the structure of the test. Being a “reader” means being able to read at least 5 words of the test passage within 30 seconds, which is considered a pre-requisite to move on to the comprehension part of the test. After finishing the passage, children who were “readers” were asked a series of ten questions to gauge their comprehension. The first seven questions were literal comprehension questions about details from the story. These were followed by two inferential questions that require the child to “connect-the-dots” of the story, and one evaluative question that asks children to apply the information of the story to an external situation. A student is considered a “reader with comprehension” if he or she is able to answer at least 80% of relevant questions correctly. Under these conditions, only 39% of the sample of second grade students could be considered “readers with comprehension” in their class language (Russian or Kyrgyz). 51.5% 49.5% 45.5% 41.6% 6.9% School repairs Supplies and equipment National holidays and other celebrations Teacher salaries and bonuses School feeding program Type of Funded Activities McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 28 of 46 December 2017 Figure 15: Types of Readers in Literacy Assessment Overall, fewer than four in ten students could read with comprehension in the language of the instruction. Female students had higher results at 44% than male students (34%): Figure 16: Type of Readers by Gender The baseline results show that boys are also three times more likely to be a non-reader than girls (7% for boys could not read the passage in their class language, compared to just 2% for girls). A significant difference was also observed in students’ ability to comprehend the text by class language (Russian or Kyrgyz). As the chart below demonstrates, children in Kyrgyz classes outperformed on the Kyrgyz reading passage and children in Russian classes far outperformed on the Russian reading passage. Overall, the Russian passage appeared to be more challenging. At the same time, while considerable variation by gender were observed in other areas of reading assessment19, there were no significant differences between boys and girls and the average performance on comprehension questions. 19Further analysis on reading and comprehension skills of the sampled students can be found in Attachment II. 5% 56% 39% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Non-reader Reader without comprehension Reader with comprehension 7%** 2% 59% 53% 34% 44%** 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Male Female Non-reader Reader without comprehension Reader with comprehension McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 29 of 46 December 2017 Figure 17: Comprehension Results by Language Number of textbooks and other teaching and learning materials provided as a result of USDA assistance As part of the classroom observation portion of the literacy baseline, enumerators recorded the presence of various resources in the classroom, which revealed some notable deficiencies. Figure 18: Classroom Resources Kyrgyz-language classrooms Russian-language classrooms Print materials are displayed in Russian 9% 88% Print materials are displayed in Kyrgyz 97% 38% Print materials are displayed in another language 11% 0% Storybooks are available in Russian 14% 63% Storybooks are available in Kyrgyz 49% 50% Children have enough writing materials 89% 100% Children's work is displayed 26% 25% As outlined in the table above, while print materials are usually displayed in the language of instruction, classrooms rarely display multilingual print. At the same time, while students have plenty of textbooks, there is a lack of age-appropriate storybooks: just half of all schools had any storybooks available to children. A 2012 EGRA conducted in Kyrgyzstan identified a lack of suitable early grades reading materials as a major constraint on children’s literacy learning, at least for the Russian and Kyrgyz languages. 65%*** 17% 37% 52%*** 0% 10% 20% 30% 40% 50% 60% 70% Kyrgyz passage (n=730) Russian passage (n=671) Average percent correct Kyrgyz Classes Russian Classes McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 30 of 46 December 2017 In order to address this deficiency, Save the Children will purchase age-appropriate reading materials such as those developed by the Aga Khan Foundation for its Reading for Children project, as well as commercially published materials where these are available at an affordable cost. These materials will be made available at Reading Corners established at selected schools20. Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance In order to better understand how reading is being taught to students and identify areas where improvements can be made, highly-trained enumerators conducted structured “snapshot” observations in second grade classrooms in the sampled schools. Enumerators coded the activities that teachers were engaged in every five minutes to give us an understanding of how class time was being used over the course of the lesson. The chart below presents the results of the observed classes at each of the five “snapshots”. Figure 19: Teacher's Activities During Class Unsurprisingly, the beginning of the class is dominated by teacher-led instruction and preparation/transition activities, meaning the teacher is leading the class in the instructions for an activity or preparing students. We see the rest of the teacher’s time dominated by reading (or listening to students read), assisting students with creative literacy activities such as games or crafts, and assisting with group work. Encouragingly, according to the observation, we find teachers did not spend any time “off-task” and the amount of time spent on instruction and transitional activities is low. The next chart summarizes the teaching skills that teachers focused on in their lessons. Enumerators observed teachers specifically addressing reading comprehension in nearly all of the lessons, but writing was observed in fewer than half of all lessons. 20 See attached PMP for additional (custom) indicators related to this activity. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0:05 0:10 0:15 0:20 0:25 Teacher-led instruction Creative literacy activity Reading Writing Group work, pair work, or individual work Transitions / preparing activites McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 31 of 46 December 2017 Figure: Percentage of Lessons That Included Instruction on Specific Literacy Skill In general, enumerators had a high opinion of the classrooms they observed. In a majority of lessons observed, enumerators gave the highest rating for teacher’s positive interactions with students and attentiveness to gender in the classroom. Ratings were more mixed which it came to the lesson plan: just half of teachers had a detailed lesson plan. Figure 20: Classroom Observation Results 12% 70% 21% 28% 86% 95% 40% 86% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Letter identification Vocabulary Phonemic /Phonological awareness Decoding reading using phonics knowledge Reading fluency Reading comprehension Writing Listening and speaking 2% 7% 2% 51% 12% 21% 42% 42% 23% 86% 79% 51% 56% 26% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Teacher-student interactions Teacher's gender attitudes Teacher's lesson plan Teaching aids in classroom Reading environment Needs Improvement Adequate Good McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 32 of 46 December 2017 Additional Baseline Data for Custom Indicators on Literacy In light of recent negotiations with the currently ongoing Time to Read project implemented by USAID/Chemonics, Save the Children made several adjustments to initially proposed activities and indicators in order to avoid potential duplication of efforts. As a result, targets for standard indicators have been lowered due to reduction in associated activities, while several new, custom indicators have been developed, which is reflected in the section below and in the Performance Monitoring Plan. However, it should be noted that the negotiation process is still ongoing, so further changes to indicators and targets are possible. Finalized programmatic changes will be discussed in detail with USDA at a later stage and outlined in the first Biannual Report at the end of April 2018. Home learning activities and environment In the past, Kyrgyzstan had a strong culture of reading in homes and communities and a well-developed network of public libraries. This culture has eroded in the face of contemporary economic realities, but its roots remain. The experience of the Quality Reading Project has been that schools and communities respond enthusiastically to activities promoting children’s literacy beyond the classroom. The figure below presents information about the learning activities that children reported engaging in at home. Figure 21. Home learning activities with household members No significant differences were observed in the number of learning activities by child’s gender or class language. However, there is a noticeable gender dynamic regarding with whom children are engaging in learning activities. Children reported engaging in learning activities with their mothers and sisters significantly more than with their fathers and brothers. 77% of children reported that their mother helped them with homework and 61% reported reading with their mother. Conversely, 50% of children reported that their father helped them with homework in the past week and only one in three children reported any other learning activity with their father. 21% 29% 32% 49% 9% 18% 24% 34% 36% 61% 10% 18% 35% 41% 50% 77% 10% 21% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Brother Sister Father Mother Grandfather Grandmother Tells a story Reads Help with homework McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 33 of 46 December 2017 Overall, children get the most engagement in the form of help with homework. Figure 22 demonstrates the proportion of children engaging in reading activities by the number of household members they report engaging in with them. On average, children had 2.6 household members help them with homework in the past week, compared to just two that they reported reading together with, and 1.7 that they reported told them a story. Figure 22. Number of household members who engage in learning activities with the child As can be seen, nearly all children reported receiving homework help from at least one household member. However, one in seven children reported not reading with anyone in the household, and one in four children reported not hearing a story from any household member. The table below presents a summary of the types of reading materials and toys children reported having at home. As we can see, nearly all children had textbooks at home. However, storybooks were considerably rarer. One in four children do not have any storybooks at home. Table 6: Home Learning Environment Does your family own the following reading materials? Textbooks 99% Religious books 72% Newspapers 71% Storybooks 73% Total number of types of reading materials 3.12 Community-level literacy engagement In addition to children’s home literacy environment, we also asked about their engagement with literacy in the community as a whole. Significant differences by gender were observed in terms of participation in literacy-related community activities. The next figure shows that girls are 17% (11 percentage points) 24% 62% 14% Tell you a story None 1 to 3 4 or more 14% 69% 17% Reading together None 1 to 3 4 or more 4% 72% 25% Help with homework None 1 to 3 4 or more McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 34 of 46 December 2017 more likely to report reading outside of school and over 40% (8 percentage points) more likely to report helping someone else to read than boys. Figure 23. Participation in community literacy activities 65% 55% 19% 21% 76%** 63%* 27%** 20% 0% 10% 20% 30% 40% 50% 60% 70% 80% you read outside of school? you help someone to read? you read or share books with someone? anyone in your community read to you? In the past week, did.... Boys Girls McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 35 of 46 December 2017 Implications for Program Implementation Based on the findings of this assessments, Mercy Corps and Save the Children consider making the following adjustments to program activities in Kyrgyzstan: • Given that there are fewer schools in Phase I (139) than estimated at the time of proposal submission (160), school-based indicator targets should be revised to reflect this. All relevant changes to indicator targets can be seen in Attachment D. • Based on the FGD findings and extensive previous implementation experience, Mercy Corps proposes to revise the approach to providing training on child health and nutrition under the standard MGD indicator #18. Instead of targeting the parents under this indicator, Mercy Corps proposes to focus on primary and community health care workers, as suggested in the indicator definition. The baseline showed that there are currently 18 full-time nurses in 18 schools, in addition to community nurses assigned to every village (on average, one nurse for every three schools). • At the same time, Mercy Corps will target parents of primary grade students at the HH-level through a tailored behavior change strategy in order to improve their nutrition and child health knowledge and practices at home. A custom indicator is suggested in the indicator plan (Attachment D), and the full strategy will be shared with USDA at a later stage. o Mercy Corps will also conduct a focused pre/post HH-level study using the KAP methodology to measure current knowledge and behaviors and subsequent change. This is proposed as one of the special studies under this program (one study split in two stages, pre and post). • To avoid duplication of efforts with the ongoing literacy project implemented by USAID/Chemonics, Mercy Corps and Save the Children have revised targets for the standard literacy indicators given the reduction in classroom-based literacy activities (# 2, 5, 6, 26). o Specifically, the LOP target for indicator #26 was established at 49% based on the estimation that it is reasonable to expect an improvement of 10 percentage points from the baseline (39%) over the course of the program. Initially, the target was set higher based on the previously conducted studies using different methodlogy. At the same time, several custom indicators have been developed to track and measure complementary literacy activities agreed upon with USAID/Chemonics. Please refer to the updated Attachment D for details. • Baseline results, as well as implementation experience for similar programs do not indicate a problem with children missing school due to perceived high opportunity cost of missed labor. Attendance is consistently high regardless of vulnerability profile of the household. Therefore, in McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 36 of 46 December 2017 the Kyrgyz context the standard indicator #17 - Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance – will measure the total number of primary grade students receiving USDA-provided meals in school. • Baseline results indicate several schools with critical sanitation conditions (unimproved latrines). In order to provide adequate conditions for improving WASH practices in schools, Mercy Corps plans to reallocate several infrastructure projects, initially planned for Year 3 of the program, to Year 1 or 2 for these schools and consider waiving community contribution, if required. • Similarly, infrastructure projects will be accelerated in Phase I schools with unimproved or unreliable water supply. Similar approach could be taken to reserve schools that are likely to be included in Phase II, since reliable water supply is one of the main conditions for a school to be able to establish a hot meals program. • Overall, given the baseline results it is reasonable to assume that, similarly to Phase I, there will be a portion of smaller schools in non-standard buildings suggested by the MoES Phase II with infrastructure deficiencies. Knowing this, Mercy Corps plans to start the selection process with the MoES to identify Phase II schools as soon as possible and start infrastructure projects in advance. This approach will allow us to ensure that small and remote schools could still be included in the program even if their current infrastructure does not meet program selection criteria. • Focus more actively on working with existing fiduciary boards and encouraging schools to create them in order to formalize parental contributions to school feeding program, which is in line with the current law. • Cooperate with WFP to actively advocate timely kitchen staff recruitment and reimbursement with the MoES to ensure that all target schools have trained full-time cooks by Fall 2018 in order to start providing hot meals at the beginning of school year, as planned. • Encourage family members to engage more with their children in learning activities, and beyond just assisting them with homework, could help build children’s overall language and comprehension skills. o The home environment portion of the child assessment tool revealed that there is substantial room for improvement when it comes to children’s exposure to literacy activities outside of school. • There is also a considerable gender gap when it comes to reading. While most teachers appeared to have positive attitudes towards gender, there may be room for improvement and other areas that can help reduce this gap. McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 37 of 46 December 2017 o For example, campaigns to promote reading more generally may be a way to incentivize reading for all students, particularly for young boys who, as the literacy baseline findings show, tend to fall somewhat behind female students and do not tend to see reading as an activity of interest to them. o Other gender-specific strategies might be useful to help reduce this gap, such as involving older students and siblings in informal reading activities with primary grade students. • Results from the classroom observations indicate deficiencies in the reading environment, especially in regard to the availability of reading materials to children. Save the Children will support establishment of Reading Corners in target schools, which will be equipped with age￾appropriate reading materials. • More activities could be considered to improve students’ understanding of the reading material. While second grade student seem to be able to decode words, they struggle with comprehension as indicated by a relatively low baseline value for “readers with comprehension”. McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 38 of 46 December 2017 Attachment I: FGD Participant Profile While the sample for FGDs conducted during this baseline assessment is not representative of the entire beneficiary population, it does provide certain insights on general household characteristics of the target communities. On average, each household had three children (younger than 18), with only 16% of families having one child. 10% had five children or more. In terms of household income, half of respondents indicated that they had up to 10,000 KGS per month in total income for the family. Further 27% make between 10,000 and 15,000 KGS per month. It should be noted that the average salary in the Kyrgyz Republic in 2017 was estimated to be 15,000 KGS21. However, it should also be noted that this finding does not take into account cattle or any other productive assets at home; the question only asked about monthly cash income specifically since the program encourages contributions from parents to the school hot meals program. Furthermore, 56% of the households with three children and 60% of the households with four children had less then 10,000 KGS in monthly income. The main source of formal employment among the respondents was public sector, i.e. local government, teaching, etc. Further 12% indicated commercial farming as the main source of income, while only 6% worked in the private sector. The majority of respondents – 36% – said they were unemployed at the time, however, upon further questioning, it became evident the majority of them were actually stay-at-home mothers and were not actively looking for formal employment. As expected, the majority of caretakers are stay-at-home mothers, which is consistent with the prevailing social structure in rural communities across Kyrgyzstan. Of the 329 participants, only 11% were male. In 21Data from the National Committee for Statistics of the Kyrgyz Republic. 16.0% 21.2% 29.5% 23.3% 10% Total Number of Children in HH 1 2 3 4 5 or more McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 39 of 46 December 2017 terms of the relationship with children enrolled in selected primary schools, the majority of respondents were parents, followed by grandparents. The level of education among the FGD participants varied, but the absolute majority had finished at least secondary school. 20% of the respondents had a TVET degree, while another 20% had finished college (4 or 5-year program). 1.5% 11.3% 1.2% 85.3% 0.6% Type of Relationship Aunt/uncle Grnadparent Older sibling Parent Other McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 40 of 46 December 2017 Attachment II: Additional Literacy Assessment Results As mentioned previously, the literacy assessment consisted of several tasks measuring different subskills. In this section, each subskill is analyzed in more detail, identifying the most difficult aspect of each task and identifying, when present, differences by gender and language of instruction. While children demonstrated reasonably strong results in the subskills of reading, when it came to measuring reading comprehension, the results are mixed. As mentioned earlier, children read reading passages in both Kyrgyz and Russian. Children that were able to read at least five words of the passage within 30 seconds are considered “readers” (if they could not, they were deemed “non-readers”). After reading the passage, if children could successfully answer at least eight out of ten comprehension questions correctly were considered “readers with comprehension.” In general, girls demonstrate significantly better literacy skills on nearly every subskill. Figure 24. Subtask average percent correct Letter Identification The first literacy-related task that children completed was a letter identification task. Enumerators showed children a grid of all of the letters in the alphabet and asked them to identify them in turn. For children in both Kyrgyz classes and Russian classes, the two most difficult letters were “Ь” and “Ъ” (“soft” and “hard” marks). This is unsurprising as they have no sound of their own. As Figure 22 shows, we find a small but significant difference by both language of instruction and gender in letter identification performance. 96% 96% 88% 51% 90% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Letter ID Most-used Words Decodable Words Listening Comprehension Vocabulary McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 41 of 46 December 2017 Figure 25. Average Letter ID percent correct by gender and class Note: ~ = p < 0.10; * = p < 0.05; ** = p < 0.01; *** = p < 0.001 Word Recognition: Most-Used Words and Decodable Words The Most-used Words (MUW) sub-test consists of a chart of 20 common words from grade-level textbooks the student attempts to read. We used the list of most-used words from the EGRA implemented in Kyrgyzstan in previous years22. Again, we find strong results on average. We observe no significant differences by class language. We find that girls scored significantly higher than boys as shown in Figure 23. All of the Kyrgyz and Russian words had correct response rates above 90 percent, but the most difficult word in Russian was “какие” (which kind) and in Kyrgyz was “калды” (remained). Figure 26. Most-Used Words percent correct by gender (n=768) Note: ~ = p < 0.10; * = p < 0.05; ** = p < 0.01; *** = p < 0.001 The Decodable Words subtest was also adapted from the EGRA and consists of a chart of 20 nonsense (invented) words that children sound out. The words were developed to mimic the phonological structure 22 The EGRA test was adapted/created by American Institutes of Research for the evaluation of the USAID Quality Reading Program implemented by Save the Children. 96% 97%*** 96% 98%*** 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Male Female Kyrgyz Russian Student gender Class language Average percent correct 94% 97%** 0% 20% 40% 60% 80% 100% Male Female Student gender Average percent correct McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 42 of 46 December 2017 of actual words in Russian and Kyrgyz. Figure 24 demonstrates children’s average scores on this task by both gender and class language. As the figure shows, girls scored 9 percentage points better on Decodable Words than boys, and children in Russian classes scored 4 percentage points better than children in Kyrgyz classes. Figure 27. Decodable words percent correct by gender Note: ~ = p < 0.10; * = p < 0.05; ** = p < 0.01; *** = p < 0.001 Language skills: Listening Comprehension and Vocabulary The Letter ID, Most Used Words, and Decodable Words subtasks test various components of children’s ability to decode text. In order for students to read with comprehension, they need not only the ability to decode, but also the language skills to process the words they read. The Listening Comprehension and Vocabulary subtasks test exactly this. Both tasks were taken from the EGRA used in Kyrgyzstan. The Vocabulary task was based on the format of the Peabody Picture Vocabulary Test and comprised a series of words that children were asked to identify a picture of (out of four choices). The Listening Comprehension subtask consisted of a story read to children, with four comprehension questions (three literal and one evaluative) asked orally afterwards. Performance on the Vocabulary test was strong, with the average child getting 9 out of 10 (90%) of questions correct. The Listening Comprehension subtask was considerably more difficult, with children only able to answer half (51%) of the questions correctly on average. We observe no differences between Kyrgyz and Russian classes or boys and girls on these two tasks. The most difficult Vocabulary item was “painter” for both Kyrgyz and Russian students. On the Listening Comprehension test, both Kyrgyz and Russian students found the final literal comprehension question very difficult: just 30% of children could successfully answer it. 86% 97%** 87% 91%* 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Male Female Kyrgyz Russian Student gender Class language Average percent correct McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 43 of 46 December 2017 Reading fluency, accuracy, and comprehension The ultimate goal of reading text is to be able to read with comprehension. The final two literacy tasks were designed to measure this goal directly. Children read aloud passages in Russian and Kyrgyz and, if they were able to read them, answered 10 comprehension questions about each. We adapted the two passages from the reading comprehension passages from EGRA and increased their length and detail with grade-appropriate modifications. If a child could read at least five words correctly within 30 seconds, we considered the child a “reader” and gave them as much time as required in order to complete reading the passage. Figure 28. Percentage of students able to read at least five words in 30 seconds on Russian and Kyrgyz reading passages, by gender and class language Note: ~ = p < 0.10; * = p < 0.05; ** = p < 0.01; *** = p < 0.001 Figure 25 displays the percentage of “readers” on the Russian and Kyrgyz reading passages by gender and class language. A significantly higher percentage of female students were readers than male students. This was especially true on the Kyrgyz reading passage. Girls were nine percent points more likely to be a reader than boys. We also examined the difference by class language. Students in Russian language of instruction classes were more likely to be able to read the Russian passage than students in Kyrgyz language of instruction classes. Interestingly, the reverse was not true. There was no difference in being a “reader” for students from Kyrgyz and Russian language of instruction classes on the Kyrgyz passage. Within the population of “readers” we also asses the fluency and accuracy of their reading. Fluency measure assess the number of words per minute that children correctly read. Accuracy measures the percentage of words read correctly. 82% 93%*** 93% 97%** 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Russian passage Kyrgyz passage Percentage of readers Male Female 85% 98%*** 95% 97% Russian passage Kyrgyz passage Kyrgyz-language class Russian-language class McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 44 of 46 December 2017 Figure 26 presents the differences in children’s average fluency scores (as correct words per minute). In this statistic, we only include “readers”. Again, girls are outperforming boys and are reading significantly faster on both reading passages. When we break down results by class language, we find that Kyrgyz language classes perform better on the Kyrgyz reading passage than Russian classes, and vice versa. Figure 29. Oral reading fluency (words correct per minute) on Kyrgyz and Russian reading passages by gender and class language Note: ~ = p < 0.10; * = p < 0.05; ** = p < 0.01; *** = p < 0.001 Figure 27 presents the overall distribution of fluency scores among readers in the language of instruction. Two out of three readers scored between 10-40 words correct per minute. Just 28% of students (29% of readers) were able to read above 40 words correct per minute, often used as a benchmark for minimum requirements for reading with comprehension. 28.5 34.5*** 32.2** 27.4 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 Male Female Kyrgyz Russian Student gender Class language Average words correct per minute Kyrgyz reading passage (n=730) 25.4 29.2*** 25.8 34.8** Male Female Kyrgyz Russian Student gender Class language Russian reading passage (n=671) McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 45 of 46 December 2017 Figure 30. Distribution of Oral Reading Fluency scores in class language The findings regarding Accuracy are more mixed. Girls read slightly more accurately than boys on the Kyrgyz reading passage (95% vs. 93%), but there were no significant differences on the Russian reading passage. Children in Russian classes read the Russian reading passage more accurately than children in Kyrgyz classes (96% vs. 90%), but there were no differences in accuracy by class language on the Kyrgyz reading passage. 4% 67% 29% 0 5 10 15 20 25 Frequency 0 20 40 60 80 100 Oral Reading Fluency (correct words per minute) in class language Reader: 10-40 cwpm Reader: 40+ cwpm Non-reader McGovern-Dole Food for Education 2017-2021 Baseline Report: Phase I Page 46 of 46 December 2017 Attachment III: Map of Baseline Assessment Locations Red = 141 schools selected for Phase I Green = 11 reserve schools