USAID AENN NFLC Cohort 2 Baseline Assessment 1 Addressing Education in Northeast Nigeria Non-Formal Learning Center Cohort 2: Baseline Assessment Approved Date: Agreement Number: 72062018CA00001 Activity Start Date and End Date: Total Award Amount: $14,876,124 AOR Name: Submitted by: Ayo Oladini, Chief of Party FHI 360 No. 4, Hamman Maiduguri Street, Ali Kotoko Close, off Lagos Street, behind MTN office, old GRA, Maiduguri, Borno, Nigeria Tel: (234) 9-4615555 Email: SOladini@FHI 360.org AENN NFLC Cohort 2 Baseline Assessment March 2021 Contracted under 72062018CA00001 AENN DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. USAID AENN NFLC Cohort 2 Baseline Assessment 3 ACRONYMS ACES Assessment of Children’s Emotional Skills AENN Addressing Education in Northeast Nigeria AOR Agreement Officer Representative CBO Community-Based Organization CWPM Correct words per minute EGMA Early Grade Mathematics Assessment EGRA Early Grade Reading Assessment HLE Home Literacy Environment IDP Internally Displaced Person LGAs Local Government Areas LSPM Letter sounds per minute MOE Ministry of Education NFLC Non-Formal Learning Center ORF Oral reading fluency PCA Principal Components Analysis SEL Socio-emotional Learning SES Socioeconomic Status UNICEF United Nations Children’s Fund USAID United States Agency for International Development USAID AENN NFLC Cohort 2 Baseline Assessment 4 TABLE OF CONTENTS EXECUTIVE SUMMARY............................................................................................................5 DATA COLLECTION STRATEGY .................................................................................5 LEARNER DEMOGRAPHICS .........................................................................................5 READING AND NUMERACY OUTCOMES....................................................................5 SOCIAL-EMOTIONAL LEARNING OUTCOMES ............................................................6 SCHOOL SAFETY OUTCOMES.....................................................................................7 1. INTRODUCTION.............................................................................................................8 PURPOSE .....................................................................................................................8 ACTIVITY DESCRIPTION .............................................................................................8 2. DATA COLLECTION STRATEGY......................................................................................9 3. RESULTS: DESCRIPTIVE ANALYSIS ..................................................................................10 LEARNER SAMPLE......................................................................................................10 LEARNER DEMOGRAPHICS .......................................................................................10 OUTCOMES: EGRA/MA..............................................................................................12 OVERALL EGRA OUTCOMES ......................................................................................................13 OVERALL EGMA OUTCOMES .....................................................................................................15 SOCIAL-EMOTIONAL LEARNING OUTCOMES ..........................................................17 SCHOOL SAFETY OUTCOMES...................................................................................19 4. CONCLUSION...............................................................................................................21 USAID AENN NFLC Cohort 2 Baseline Assessment 5 EXECUTIVE SUMMARY This report summarizes academic and social emotional learning outcomes among a representative sample of learners who participated in the USAID Addressing Education in Northeast Nigeria (AENN) Activity’s second cohort of basic and post-basic level non-formal learning centers (NFLCs) in Borno and Yobe States. In addition, basic level NFLCs are divided between Hausa and Kanuri as the language of instruction as determined by community preference, and post-basic level NFLCs include English instruction. Within the NFLCs, AENN uses a condensed curriculum that is aligned to the formal curriculum and is divided into two levels (Basic Literacy, which is equivalent to primary 1-3, and Post-Literacy, which is equivalent to primary 4-6), according to the national non-formal education policy. Students attending the NFLC Basic Literacy program received classes four times per week for two hours and 15 minutes per day for approximately seven months. Each day, learners participated in 45-minute lessons for Hausa or Kanuri literacy, mathematics, and social emotional learning (SEL). Learners attending the NFLC Post Literacy received classes four times per week for three hours per day. For weeks one through eight of the program, Post Literacy lessons are an English Bridging program. Beginning in week nine, each day learners participated in 45-minute lessons each for English language and literacy, mathematics, science, and Nigerian History and Values with English as language of instruction. Data Collection Strategy Master Trainers working for the AENN program administered the baseline assessment with a sample of learners across AENN supported NFLCs. Assessment tools included the Early Grade Reading Assessment (EGRA), Early Grade Mathematics Assessment (EGMA), two assessments of social￾emotional learning (Children’s Stories and Mental Health), and a student-level Safer Learning Environments survey. Respondents included four randomly selected learners from each AENN NFLC across Yobe and Borno states. The content of the EGRA administered to each learner aligned with the language of instruction in that NFLC (Hausa, Kanuri or English). Baseline data for Cohort 2 NFLCs was collected between January 12 and February 4, 2021. Learner Demographics In the second year of implementation of AENN, programming shifted to providing non-formal learning centers in Hausa, Kanuri, and post-basic (English) to better serve the out-of-school child population in Borno and Yobe. Over two-thirds of the children are enrolled in basic Hausa NFLCs, 12.5% enrolled in basic Kanuri, and approximately 20% of learners enrolled in post-basic literacy. Depending on the type of NFLC, girls comprise between 56 and 59.6% of all learners and between 56% and 65% are internally displaced persons (IDPs). Among Hausa NFLCs, about 43% speak Hausa as their home language and 36% speak Kanuri, and 7.1% speak Fulatanci. About 77.5% of learners enrolled in basic Kanuri NFLCs speak Kanuri as their home language, while only 8.9% and 2.4% speak Hausa or Fulatanci, respectively. The data also show that learners in Hausa NFLCs are slightly more likely to report some disability (physical and/or cognitive) relative to learners in Kanuri basic literacy and in post-basic literacy learning centers. In terms of socioeconomic standing and home learning environment, learners are distributed somewhat uniformly across NFLC types. Lastly, about 25%-29% of learners in basic Hausa and post-basic NFLCs report working outside of their own homes, while 14% of Kanuri basic learners report working. Reading and Numeracy Outcomes Overall, reading and numeracy outcomes do not appear to be dissimilar to cohort 1’s baseline averages. Moreover, nearly half of learners enrolled in basic level NFLCs (54% for Hausa and 46% for Kanuri) receive USAID AENN NFLC Cohort 2 Baseline Assessment 6 scores of zero on the letter sound identification and syllable reading subtasks of the Early Grade Reading Assessment (EGRA). Meaning that these learners, at baseline, are unable to identify a single letter sound or read a single syllable. While on average, learners in Hausa and Kanuri basic literacy NFLCs can identify between 10.8 and 12.6 letters per minute and read between 13 and 13.5 syllables per minute. We also find that 57% of Hausa basic learners, 64.5% of Kanuri basic learners, and 48.4% of post-basic learners are not able to read a single word in the NFLC’s respective language. We also observe substantive variation in learners’ average fluency scores with Hausa learners averaging 12 correct words read per minute while Kanuri learners average 4.2 words per minute, and post-basic learners average 16.8 words per minute. However, we emphasize that these assessments are not comparable as the language of the assessment is different and the age groups are different between basic and post-basic level NFLCs. Although these results will serve as a useful point of reference for the learners’ learning growth trajectories by endline. For numeracy, the Early Grade Math Assessment (EGMA) data show that learners in basic level NFLCs receive average scores of 55% for Hausa learners and 50.4% for Kanuri learners on the number identification subtask. In terms of the addition subtask, learners in basic NFLCs average approximately 35- 36% for both Hausa and Kanuri NFLCs, and approximately 30% on the subtraction subtask for the same learners. In post-basic level NFLCs, learners on average score 69% on number identification, 43.4% on addition, and 35% on subtraction. Learners who received a positive score on the standard addition or subtraction subtask were required to complete a higher difficulty addition/subtraction subtask comprised of 5 arithmetic exercises of additional difficulty. Here, we find that the eligible learners average 40% on the second level addition subtask and 35% on the second level subtraction subtask. Social-emotional Learning Outcomes We report learners’ Social and emotional learning outcomes by using measures of hostile attribution bias where we calculate learners’ tendency to attribute hostile intent from a third party, and measures of conflict resolution strategies based on hypothetical (purposefully vague) scenarios. In addition, we measure children’s mental well-being using a standard mental health module measuring the frequency of depressive symptoms or thoughts among the learners. We calculate that, on average, learners in Hausa basic NFLCs would attribute hostile intent to another actor in about 42% of all presented scenarios (8 stories in total). Among Kanuri learners, we find that they attribute hostile intent in 30.9% of the scenarios, while learners in post-basic literacy would do so in about 46.6% of the scenarios. Despite the variation in hostile attribution bias among learners from different types of NFLCs, the intervention would aim to decrease these rates of hostile attribution bias by endline. When asked how to resolve the conflict present in these scenarios, we find that learners would resort to aggression 11% of the time among Hausa learners, 7.9% of the time among Kanuri learners, and 12.3% of the time among post-basic learners. Across all types of NFLCs learners would choose to disengage in between 36% and 41.8% of the presented scenarios, and would resort to problem solving in between 44% and 50.5% of the time. When measuring learners’ mental wellness, we find that learners in Kanuri centers have the relatively lowest mental health index and those in post-basic having the highest. Relative to learners in Hausa basic NFLCs, Kanuri learners have a lower mental health index score by 0.68 standard deviations, while learners in post-basic have a higher mental health index score by 0.22 standard deviations. We note that the mental health index measure presented in this report is a norm-referenced measure rather than a criterion￾referenced one. As such, the mental health index, as constructed, does not enable us to make determinations of absolute mental wellness. USAID AENN NFLC Cohort 2 Baseline Assessment 7 School Safety Outcomes Finally, in this report, we present findings from our school climate and safety perceptions modules. At baseline of cohort 2, learners in Hausa basic NFLCs average 82% on the overall school safety score and 80.6% on the overall school climate score. Learners in Kanuri basic NFLCs average 741.% and 70.9% on the overall school safety and climate scores, respectively. Learners in post-basic level NFLCs score more similarly to learners in Hausa basic NFLCs with overall scores of 80.3% and 84.5% in terms of school safety and climate, respectively. The most notable finding from the individual items comprising the school safety composite score is that between 67% and 71.4% of learners across all types of NFLCs report having a safe place to play in the community, while that rate varies between 83.3% and 89.8% at school. In terms of school climate, we find that between 30% and 38% of learners, depending on the NFLC type, report that boys and girls are not nice to each other, and between 18% and 26% report that students often treat children with disabilities unkindly. USAID AENN NFLC Cohort 2 Baseline Assessment 8 1. INTRODUCTION Purpose This report summarizes changes in academic and social emotional learning among a representative sample of learners who participated in AENN’s second cohort of Basic and Post-Basic Literacy level non-formal learning centers (NFLCs), from January 6th, 2021 through June 2021. Activity Description The AENN Activity targets the immediate educational needs of children and youth ages 6-15 who are internally displaced, and their host communities, in eight local government areas (LGAs) across Borno and Yobe, while building long-term resilience across the system. In Borno state, in Year 1 and 2, AENN was implemented in Maiduguri Metropolitan Council, Jere, Monguno, Hawul, and Dikwa. In Yobe, the LGAs include Damaturu, Potiskum, and Bade. The program was rolled out in 150 communities in Year 1. In Year 2, the Activity expanded to additional communities in those LGAs with 900 total NFLCs, distributed between Hausa and Kanuri Basic Literacy and Post Literacy based on community needs. Within one month of the start of lessons in 2020, however, all in-person learning was suspended due to the Covid-19 pandemic and the Activity pivoted to distance learning. The Activity succeeded in engaging 2020 learners in home learning through phone, radio, and home learning kits despite to the challenges of restricted travel and gatherings, limited technology access, and increased displacement. Upon reopening of NFLCs in January 2021, all 2020 enrolled learners were offered the opportunity to re-enroll in NFLCs, with any remaining slots open to community enrollment. To meet the educational needs of out-of-school children and youth, AENN provides quality basic education for 6-15 year olds via non-formal learning centers (NFLCs). These classes are primarily delivered in formal school buildings in the afternoon (after formal school lesson is over) or in temporary learning spaces that have been established by United Nations Children’s Fund (UNICEF) or other education actors. Community-Based Organizations (CBOs) that oversee the implementation of non-formal activities provide seating mats, WASH materials, and other support the process of making the classrooms safer and more comfortable for learners. Additionally, CBOs provide learners with snacks and female learners with hygiene kits to promote regular attendance. The size of classes ranges between 50-60 learners per class. Classes are delivered by trained local learning facilitators. In Year 1, all NFLCs were delivered in Hausa language. In Year 2, Kanuri language learning was added as well as the introduction of post-basic literacy level NFLCs. The Kanuri language NFLC follows the same typical model of the Hausa NFLCs, while the post-basic literacy NFLCs where children who have completed basic level literacy transition to learning in English. All non-formal education activities are accompanied by safety interventions and community-level activities that support enrollment and awareness around the importance of education. Within the NFLCs, AENN uses a condensed curriculum that is aligned to the formal curriculum and is divided into two levels (Basic Literacy, which is equivalent to primary 1-3, and Post-Literacy, which his equivalent to primary 4-6) according to the national non-formal education policy. This assessment only examines students participating in Basic Literacy (NFLC Level 1). Students attending the NFLC Basic Literacy program receive classes 4 times per week for 2 hours and 15 minutes per day for approximately nine months (although due to time constraints the first cohort was completed in seven months). Each day, learners participate in 45-minute lessons for literacy, math, and social emotional learning (SEL). After completing the Basic Literacy program, learners can either mainstream into formal schools at grade 4 or continue into the two-year Post-Literacy program. After completing the Post-Literacy program, learners can mainstream into formal schools at grade 7 (junior secondary school). Learners attending the NFLC Post Literacy received classes four times per week for three hours per day. For weeks one through eight USAID AENN NFLC Cohort 2 Baseline Assessment 9 of the program, Post Literacy lessons are an English Bridging program. Beginning in week nine, each day learners participated in 45-minute lessons each for English language and literacy, mathematics, science, and Nigerian History and Values with English as language of instruction. 2. DATA COLLECTION STRATEGY This study measures literacy outcomes using the Early Grade Reading Assessment (EGRA) in Hausa for the Hausa NFLCs, in Kanuri for the Kanuri NFLCs, and in English for the post-basic literacy NFLCs. The reading assessment subtasks for both Hausa and Kanuri include letter sounds, syllables, oral reading fluency (ORF), and reading comprehension. For the English version of the assessment, we administer oral vocabulary and spoken vocabulary instead of letter sounds and syllables. We also use three Early Grade Mathematics (EGMA) subtasks to measure numeracy outcomes, including number identification, addition, and subtraction for all three types of NFLCs. However, for the post-basic NFLCs we also include level-2 addition and subtraction subtasks that are higher in difficulty relative to the regular subtasks. We deploy several social-emotional learning (SEL) and safe learning environment modules, including the Children's Stories module measures learners' hostile attribution bias, which is their intent to attribute hostile intent to a third party, as well as conflict resolution strategies, which identifies whether a learner uses aggressive or problem-solving strategies to resolve a conflict, or disengages from the conflict. Drawing from USAID's Safer Learning Environment Toolkit, we use a series of school climate questions to measure learners' perception of the teacher and classroom environment. We also ask a series of questions to measure learners' perceptions of safety in the classroom and community, and to estimate participation in home learning during the school closure. AENN staff trained Master Trainers (MTs) in Borno and Yobe states to administer this baseline learning assessment. Innocent Chukwu and Akale Konduge trained 37 MTs in Borno State from January 7-11, then travelled to Yobe State to train an additional 27 MTs from January 15-19. During each 5-day training the MTs learned to use the Tangerine software and Android tablets to administer the EGRA and EGMA as a group. Then, MTs were divided into three groups according to the language taught in the NFLCs each is assigned to support. The 41 MTs proficient in Hausa, and the 9 MTs proficient in Kanuri were trained to administer an EGRA including Letter Sound and Syllable Reading subtests. The remaining 17 MTs were trained to administer an English EGRA with Oral Vocabulary and Spoken Vocabulary subtests, in place of Letter Sounds and Syllables. MTs practiced administering the assessments in pairs, as well as with pupils from local schools. At the end of the training each MT was assigned between 5-20 NFLCs to assess, within the communities they support. Each MT was trained in just one version of the EGRA, and only assigned NFLCs matching that language. Data Collection tool place from January 12-February 4. Upon arriving at an NFLC each MT randomly sampled 4 learners and administered the assessment to each. Any learner refusing consent returned to class, and a random replacement was selected by the MT. USAID AENN NFLC Cohort 2 Baseline Assessment 10 3. RESULTS: DESCRIPTIVE ANALYSIS Learner Sample This analysis is drawn from a sample of 3,032 learners across 711 NFLCs, 2,048 of whom enrolled in Hausa language NFLCs (n=486), 382 in Kanuri NFLCs (n=82), and 602 in post-basic literacy NFLCs (n=144). Table 1, below, illustrates the overall distribution of learners by state, LGA, and type of NFLC. Table 1. Learner sample by state and LGA at baseline and endline Borno Yobe LGA Hausa Kanuri Post-basic LGA Hausa Kanuri Post-basic Jere 0.37 0.13 0.31 Bade 0.43 --- 0.27 Kondunga 0.05 0.03 --- Damaturu 0.38 --- 0.49 MMC 0.58 0.47 0.65 Potiskum 0.19 --- 0.23 Mongudo --- 0.38 0.05 --- Observations 1193 382 444 Observations 855 --- 158 Learner Demographics Table 2 displays learner demographic characteristics by state and NFLC type. The sample consists of over 56% females and at least 56% being internally displaced persons (IDP) across all NFLC types. However, we see that children in Kanuri and Post-Basic NFLCs are slightly more likely to be IDPs while learners enrolled in Hausa NFLCs are more likely to be from the host community, about 42% in Hausa relative to 33% in Kanuri and Post-Basic. In terms of home language, we find that about 43% of learners enrolled in the Hausa NFLCs speak Hausa as their home language and 36% speak Kanuri. At the same time, we see that about 78% of learners in the Kanuri NFLCs speak Kanuri while only 9% speak Hausa. Lastly, in Post￾Basic Literacy, we find that 34% of learners’ home language is Hausa and 45% speak Kanuri. At least in Kanuri and Hausa instruction, there appears to be some level of mismatch between the learners’ first language and the NFLC language of instruction that may be driven by geographical and local population composition constraints. Table 2 also shows that learners are approximately evenly distributed between the three types of NFLCs in terms of socioeconomic status with all NFLCs reporting at least 69% of learners considered of low socioeconomic standing. We collected data on children’s home learning environment (HLE) which consists of asking the learners if they have access to reading material other than schoolbooks at home, if someone reads to them at home, and if they receive help with their homework. We use these variables to proxy for the home learning environment via indexing using a Principal Components Analysis (PCA). The index is then used to determine whether a child has an above average (high) HLE or a below average (low) one. The data shows that learners according to their HLE status are somewhat uniformly distributed across NFLC types with high and low HLE learners each comprising approximately half the sample. Across all NFLCs we find that 8.8% of children report either a physical or cognitive disability with Hausa NFLCs having a relatively higher percentage of children with disabilities. Lastly, we collect data on children’s living situation where we find that less than 3% of learners do not live with a relative (mother, father, or other family member). USAID AENN NFLC Cohort 2 Baseline Assessment 11 Table 2. Learner demographic characteristics by state at baseline and endline Hausa Kanuri Post-Basic N Pct N Pct N Pct Demographics: Female 2048 57.6 382 56.0 602 59.6 Ate before school 2048 80.9 382 79.1 602 82.7 IDP 2048 55.8 382 64.9 602 64.1 Host community 2048 41.6 382 33.2 602 32.6 Returnee 2048 2.7 382 1.8 602 3.3 Home Language: Hausa 2048 42.6 382 8.9 602 34.1 Fulatanci 2048 7.1 382 2.4 602 5.1 English 2048 0.2 382 0.0 602 0.0 Kanuri 2048 35.9 382 77.5 602 45.2 Other 2048 14.3 382 11.3 602 15.6 Socioeconomic Status: Low SES 2048 68.8 382 75.1 602 72.9 High SES 2048 31.2 382 24.9 602 27.1 Home Learning Environment: Low HLE 2048 56.4 382 56.0 602 48.5 High HLE 2048 43.6 382 44.0 602 51.5 Disability Status: Has a physical disability 2048 6.7 381 3.9 601 1.7 Has a cognitive disability 2048 5.6 381 2.1 601 6.3 Child lives with: Lives with mother 2048 89.6 382 86.9 602 78.1 Lives with father 2048 73.7 382 71.2 602 58.8 Lives with other family member 2048 74.0 382 74.9 602 78.9 Lives with adult not family 2048 2.6 382 1.8 602 3.2 Table 3 describes the distribution of learners’ time outside of the NFLC environment including whether they engage in some extra-household work, the number of days they were absent from the NFLC, the number of days they spent with their friends outside of school, and lastly we include a measure of whether the learner was subject to any type of victimization (being hit, pinched, hit with a stick, or criticized) from adults, other children, and in general. Table 3 shows that a quarter of all learners in Hausa NFLCs engage in some form of work outside of their home, half of whom work in a household for another family. The proportion of children who report working is 14% in Kanuri NFLCs and 29% in Post-Basic NFLCs with the modal type of work is also working in a household for another family. In addition, children reported being, on average, absent from the NFLC by between 0.5 and 1 day in the past week. Children also reported spending time with friends outside of school by about 3.5 days a week, a pattern that is consistent across NFLC types. USAID AENN NFLC Cohort 2 Baseline Assessment 12 Table 3. Learner demographic characteristics by state at baseline and endline (cont.) Hausa Kanuri Post-Basic N Pct N Pct N Pct Child work: Works outside home or in fields 2048 24.6 382 14.1 602 29.4 Work in HH for another family 529 46.7 56 39.3 182 23.6 Work in market 529 14.6 56 5.4 182 12.6 Work in a store 529 2.5 56 0.0 182 6.6 Work in the fields 529 12.1 56 5.4 182 17.0 Other work 529 24.2 56 50.0 182 40.1 Time-Use (days per week): Absent last week 1956 0.5 355 1.0 600 0.7 With friends outside school 2048 3.4 382 3.5 602 3.6 Victimization: Victimized in last two weeks 2022 29.9 354 50.0 593 38.6 Victimized by other children 2039 25.0 369 38.5 597 32.2 Victimized by adults 2041 16.4 375 34.1 600 23.0 Outcomes: EGRA/MA In this section, we calculate mean EGRA/MA subtask scores by type of NFLC type. For the Hausa and Kanuri basic literacy NFLCs, the specific reading subtask measures include letter sounds identified correctly per minute, syllables read correctly per minute, words (from a passage) read correctly per minute, and reading comprehension questions answered correctly, in the NFLCs’ respective language of instruction. For the post-basic literacy NFLCs, learners were assessed in English with timed reading subtasks include oral vocabulary (OVPM), spoken vocabulary (SVPM), and oral reading fluency measured in words read correctly per minute (CWPM). We also report the proportion of children who receive scores of zero on each reading subtask, e.g. a score of zero on the ORF subtask signifies that the learner was not able to read a single word from the provided passage. It is important to note, however, that although some of the subtasks share the same scoring scale, we cannot make inferential comparisons of reading performance between learners from different NFLC types as they comprise different populations and assessments that are not identical. In addition, we compute the proportion of learners who fall into different reading proficiency levels based on their oral reading fluency performance. The Nigerian Educational Research Development Council (NERDC), alongside other agencies and donors, is currently developing a National Reading Framework to identify reading performance standards for 10 languages (including English) across grades 1-6. So far, the draft framework proposes only one performance standard: learners should read 18-36 words per minute in Hausa by the end of Grade 2. NERDC and its partners will continue to finalize performance standards for additional languages and grades in the coming months. While Nigeria’s performance standards are being finalized, AENN will adapt the Grade 2 Performance Standard for Basic Hausa and Kanuri and Post￾Basic English. These standards may be revised when the the National Reading Framework is finalized. AENN’s proposed performance standards are listed below. USAID AENN NFLC Cohort 2 Baseline Assessment 13 Table 4. Learner demographic characteristics by state at baseline and endline (cont.) Does not meet minimum performance standards Approaches minimum performance standards Meets minimum performance standards Exceeds minimum performance standards Basic Hausa 0 correct words per minute (cwpm) 1-17 cwpm 18-36 cwpm 37+ cwpm Rationale: While Basic Hausa is intended to cover grades 1-3, the accelerated nature of the program (3 years into 9 months) means that children have far less reading practice time. Hence a Grade 2 equivalent is appropriate for Hausa Basic. Basic Kanuri 0 cwpm 1-13 cwpm 14-29 cwpm 30+ cwpm Rationale: Kanuri words are typically longer than Hausa words, even at the Basic level. Kanuri words in the AENN books are roughly 20% longer than Hausa words. Therefore, we expect a lower total number of words read per minute in Kanuri. Post Basic English Year 1 0 cwpm 1-17 cwpm 18-36 cwpm 37+ cwpm Rationale: In Post Basic Year 1, children must transition to a new language with a less transparent orthography (English). As children learn the complex rules of English reading, we expect that their Hausa skills will support a roughly equivalent reading level in English at the end of the year to their Hausa or Kanuri of the previous year. Post Basic English Year 2 0 cwpm 18-36 cwpm 37-59 cwpm 60+ cwpm Rationale: In Post Basic Year 2, children will continue learning the complexities of English reading (which typically takes three years to acquire even for native English speakers). We expect children to read at least 37 cwpm as a minimum standard by the end of Post Basic Year 2. Lastly, for math, we assess learners on their ability to identify numbers correctly, complete addition exercises, and complete subtraction exercises. Each of the three math subtasks are scored by adding the number of questions that learners answered correctly and dividing by 20, which is the total number of questions for each math subtask. The math assessment follows the same structure and content for Hausa, Kanuri, and Post-Basic NFLCs, with Post-Basic completing a second more difficult level of addition and subtraction exercise. This is done in aims of capturing the higher level of learners taking the assessment. Overall EGRA Outcomes Table 5 displays, first, the proportion of learners who received a score of zero on each EGRA subtask, the average score on the subtasks, and the average score of learners on the reading comprehension component. Here, we find that between 53% and 57% of learners in the Hausa NFLCs are not able to identify a single letter sound, a single syllable, or read a single word. In the Kanuri NFLCs, the data show that 47.5% of learners cannot identify a single letter sound, 45.7% cannot read a single syllable, and 64.5 could not read a single word in Kanuri. For post-basic literacy English assessment, we find that just under half the learners are not able to read a single word in English. In terms of the average EGRA scores, learners in the Hausa centers can identify 12.6 letter sounds per minute, read 13.5 syllables per minute, and 12 words read correctly per minute. Learners in the Kanuri centers average 10.8 LSPM and 13 SPM in terms of letter sounds and syllables. However, we find that the USAID AENN NFLC Cohort 2 Baseline Assessment 14 average reading fluency is 4.2 words correctly per minute, well below the minimum standard for reading proficiency in Kanuri. Lastly, learners in post-basic literacy averaged 16.8 words per minute on the reading fluency subtask (ORF), 7.5 spoken vocabulary words per minute, and 4.9 oral vocabulary words per minute. When examining reading comprehension performance, we observe that, on average, learners can answer 28.8%, 14.5%, and 20.7% among learners in the Hausa, Kanuri, and Post-Basic NFLCs, respectively. Note that reading comprehension is scored as the percentage of 5 comprehension questions following the reading passage answered correctly. Further, learners are only prompted to the reading comprehension portion if they were able to read at least able to attempt the reading passage. An average score of 20%, for instance, would signal that among learners who could read at least 1 word, they, on average, could answer 1 question out of 5 correctly. Table 5. Mean EGRA subtask scores at baseline, by NFLC type Hausa Kanuri Post-Basic N Mean N Mean N Mean Percentage Receiving Zero: Letter sounds (%) 1911 52.9 362 47.5 --- --- Syllables (%) 1860 54.2 352 45.7 --- --- Oral Reading Fluency (%) 1869 57.0 369 64.5 548 48.4 Timed Subtasks: Letter sounds (LSPM) 1911 12.6 362 10.8 --- --- Syllables (SPM) 1860 13.5 352 13.0 --- --- ORF (CWPM) 1869 12.0 369 4.2 548 16.8 Spoken vocabulary (SVPM) --- --- --- --- 548 7.5 Oral vocabulary (OVPM) --- --- --- --- 516 4.9 Comprehension: Reading comprehension (% correct) 826 28.8 51 14.5 261 20.7 Figure 1, plots the distribution of learners’ ORF scores along the performance standards for each of the NFLC types. Again, seeing as the assessments are for different languages, the cutoff scores for different proficiency levels will not be the same. Figure 1. Distribution of EGRA subtask scores at baseline, by NFLC type The graph, again, shows the proportion of learners who cannot read a single word as not meeting the minimum standard for proficiency. The graph also shows the proportion of students who are approaching the minimum standard and the proportion who meet the minimum standard across the three types of assessments. For Hausa, we find that 14.2% and 12.5% meet and exceed the minimum standard, respectively. Meaning that just under 27% of all learners are at least proficient in terms of reading fluency in Hausa. In Kanuri, we find that only about 11.6% are identified as at least meeting the minimum performance standard for reading in Kanuri. Learners in the post-basic literacy centers show a slightly USAID AENN NFLC Cohort 2 Baseline Assessment 15 more even spread across the performance categories with 30.3% of learners meeting the minimum proficiency level of 18 words per minute or exceeding it. Overall EGMA Outcomes The Early Grade Math Assessment is structured such that each subtask consists of 20 exercises for number identification, addition, and subtraction. Note that the same version of EGMA was administered to all children across all types of NFLCs. However, for the post-basic literacy NFLC, learners were administered an additional 5 addition and subtraction exercises of a higher level of difficulty. All EGMA subtask scores reported here are measured as the percentage answered correctly, out of 20 questions for the standard EGMA and out of 5 for the higher level addition and subtraction. Table 6. Mean standard EGMA subtask scores at baseline, by NFLC type Hausa Kanuri Post-Basic N Pct correct N Pct correct N Pct correct Number identification 2048 54.8 382 50.4 602 69.1 Addition 2048 35.9 382 35.1 602 43.4 Subtraction 2048 29.8 382 29.5 602 34.9 Addition – level 2 -- -- -- -- 490 40.0 Subtraction – level 2 -- -- -- -- 392 34.7 Table 6 shows that learners in Hausa and Kanuri basic literacy NFLCs perform similarly across all the EGMA subtask with average number identification scores of 54.8% and 50.4%, respectively. Addition and subtraction scores were, on average, similar as well with means of approximately 35% and 29%, respectively. Learners in post-basic literacy averaged 69% on the number identification subtask, 43% on the standard addition subtask, and 35% on the standard subtraction subtask. Lastly, among post-basic literacy learners who did not receive a score of zero on the addition and/or subtraction subtasks, mean level-2 addition and subtraction scores are 40% and 34.7%, respectively. Figure 2, on the following page, plots the distribution of learner scores along each standard subtask, stratified by NFLC type. Across all subtasks and consistently across most NFLC types, the modal score range is 0-20% signaling that there is a substantial proportion of the learner population who do not possess adequate math skills. The data show a pattern where number identification scores are higher than addition and subtraction, while also showing a distribution that is less skewed toward the lowest score range. We also observe that learners are more likely to receive high scores on number identification than on addition or subtraction. Another pattern that emerges from the graph is that learner scores on subtraction are the lowest, on average, and are skewed more toward the low end than in addition. This potentially signals that learners find subtraction as somewhat more difficult than addition, a finding that appears consistent across all NFLC types. USAID AENN NFLC Cohort 2 Baseline Assessment 16 Figure 2. Distribution of EGMA subtask scores at baseline, by NFLC type USAID AENN NFLC Cohort 2 Baseline Assessment 17 Social-emotional Learning Outcomes In addition to assessing children’s reading and math skills using EGRA/MA, we assessed learner Socio￾emotional Learning (SEL) outcomes using two modules adapted for the northeast Nigeria context. We deployed the Children’s Stories module, which includes an ambiguous scenario in which one child does something to another child in a story. The learner is asked to interpret whether the child in the story intended to hurt the other child or if it was an accident. This module helps to measure hostile attribution bias, which assesses the tendency of children to attribute hostile intent to a third party in response to a provocation. Additionally, the Children’s Stories module measures conflict resolution strategies employed by respondents by asking how the respondent would react in the scenario presented. Conflict resolution strategies include responses that are categorized as aggression, disengagement, or problem solving. We also employ a mental health module that probes children for symptoms of depression. We measure hostile attribution bias as the percentage of stories where the learner attributed hostile intent on the third party in each story. Here, we find that learners in Hausa NFLCs, on average, attributed hostile intent in the stories they listened about 42% of the time. In other words, children would point toward malicious intent on the part of the culprit, rather than attributing the act to an accident or some other non-malicious reason. Learners in post-basic literacy NFLCs also exhibited a similar level of hostile attribution in their stories at about 46.6%. On the other hand, learners in the Kanuri NFLCs displayed a lower hostile attribution bias of approximately 31%, which is at least 11 percentage points lower than their Hausa and post-basic literacy counterparts. Table 7. Mean Social and Emotional Learning scores at baseline, by NFLC type Hausa Kanuri Post-Basic N Pct N Pct N Pct Hostile attribution bias 2048 42.0 382 30.9 602 46.6 Conflict Resolution Strategy: Aggression 2048 11.3 382 7.9 602 12.3 Disengagement 2048 40.0 382 41.8 602 36.4 Problem Solving 2048 44.0 382 46.7 602 50.5 Table 7 also shows that children in Hausa, Kanuri, and post-basic literacy would resort to aggression as a means to resolve the conflict in the stories by between 7.9% and 12.3% of the time. Here, we find that learners in the Kanuri centers to have the lowest likelihood to resort to aggression at 7.9%, while learners in the Hausa and post-basic literacy centers exhibit a higher likelihood of aggression as a strategy by about 3.4-4.4 percentage points. Learners are then divided somewhat evenly between resolving the conflicts by disengaging from the conflict or by attempting to problem solve. In Hausa and Kanuri centers, we observe that learners are likely to disengage or problem solve between 40% and 46.7% of the time. Among post￾basic literacy learners, we find that they are relatively more likely to engage in problem solving than to disengage by about 14 percentage points. These results may be indicative of the age difference between the basic and post-basic learners as the children in the post-basic centers are older. Figure 3, plots the same information present in Table 7 in bar graph form. USAID AENN NFLC Cohort 2 Baseline Assessment 18 Figure 3. Distribution of Social and Emotional Learning scores at baseline, by NFLC type In terms of mental health, we calculate a mental health index as a composite score computed from a Principal Components Analysis (PCA) of 6 questions on the frequency of depressive symptoms including feeling nervous, hopeless, restless, sadness, and worthless, as well as feeling that any activity requires a lot of effort. As such, the mental health index is scored as a z-score where each score is the number of standard deviations above or below the average of zero. This means that higher mental health index scores indicate relatively better mental health status. Figure 4 plots the overall distribution of the mental health index for each type of NFLC and we find that learners in Kanuri centers have a lower average score as well as a distribution that is skewed farther toward the bottom end of the constructed mental health scale, relative to the other learners in the sample. Specifically, we calculate that the average mental health index for Hausa learners is +0.04, -0.64 for Kanuri learners, and +0.26 for post-basic learners. This signals that learners in Kanuri centers, on average, have worse mental health outcomes relative to their counterparts by about 0.68 standard deviations on this index. Figure 4. Distribution of learner mental health index (higher is better), by NFLC type USAID AENN NFLC Cohort 2 Baseline Assessment 19 School Safety Outcomes To measure pupils’ perceptions of the NLFC learning environment, we use the school climate questionnaire from USAID’s Safer Learning Environment Toolkit, which assesses if children feel that their teachers create a safe learning environment and whether children are treated fairly and with respect. We also measure safety of learners by asking four questions related to community and school safety. Table 8. Overall and individual school safety and climate scores at baseline, by NFLC type Hausa Kanuri Post-Basic N Pct N Pct N Pct Individual School Climate Items: Students treat each other with respect 1988 96.8 357 98.3 595 86.9 Teachers listen to students 1934 98.1 292 87.3 592 96.6 Boys and girls are not nice to each other 1873 38.1 307 30.3 590 36.1 Students often treat disabled students unkindly 1873 22.9 300 26.0 585 18.3 Teachers treat girls and boys equally 1858 90.6 304 91.4 583 91.9 Teachers treat students of all races the same 1840 88.7 305 91.8 583 86.3 Students asked to help decide what is best 1734 84.5 276 92.4 569 85.4 Students rewarded when they do well 1887 95.0 283 92.2 585 92.3 Students know who to report to when they experience or witness violence 1817 96.8 280 93.9 582 96.2 Individual School Safety Items: Feel safe at school 1991 90.8 367 78.2 599 86.5 Have a safe place to play at school 2048 83.3 382 89.8 602 86.2 Feel safe in the community 2017 86.8 362 64.4 601 81.7 Have a safe place to play in the community 2048 71.4 382 70.7 602 67.2 Overall Score: Safety score (%) 2048 82.1 382 74.1 602 80.3 School climate score (%) 2036 80.6 373 70.9 601 84.5 Table 8 shows the mean school climate and safety scores from each type of NFLC, by item and overall. The overall school climate score is computed as the simple mean across all 9 individual items included in the school climate module, while the overall school safety score is the simple mean across the 4 individual item included in the school safety module. All scores are thus expressed as percentages. Overall, learners report overall school climate scores of 80.6%, 70.9%, and 84.5% among those enrolled in Hausa, Kanuri, and post-basic literacy centers, respectively. We observe a similar pattern in terms of school safety scores, with learners in the Kanuri centers reporting the lowest scores relative to their Hausa and post-basic literacy counterparts. Figure 5 plots overall score distributions of school safety and school climate for each type of NFLC. Interestingly, we find that in all types of NFLCs, over 53% of learners report scores of 100% on the school safety perception measure with 59% of learners in the Hausa centers scoring 100%. Similarly, at least 29% of learners across all NFLC types score 100% on the school climate measure with a variation of at most 3 percentage points between the centers. However, we also see that learners in Kanuri centers are most likely to score lower than 44% on the school climate measure. This again may be a signal that learners in the Kanuri centers have different experiences in the community at large as well as in their respective learning centers than their counterparts. USAID AENN NFLC Cohort 2 Baseline Assessment 20 Figure 5. Distribution of learners’ perceptions of school safety and school climate, by NFLC type USAID AENN NFLC Cohort 2 Baseline Assessment 21 4. CONCLUSION Overall, the findings in this report appear to be in line with expectations based on cohort 1’s baseline results. Although it is important to note that the results from the two cohorts are not directly comparable since all learners in the first cohort attended Hausa basic literacy NFLCs, while this year the learners were split between Hausa and Kanuri basic literacy as well as a proportion attending post-basic literacy with English instruction. At baseline for cohort 1, we observe that 53% of learners were not able to read a single word on the fluency passage, while for cohort 2 about 57% of Hausa learners, 64.5% of Kanuri learners, and 48.4% of post-basic learners were not able to read a single word. This, again, does not necessarily signify an improvement or regression in learner performance as the assessments are not identical between administrations and center types and therefore not directly comparable. This comparison is meant only as a point of reference. In terms of EGMA performance, the cohort 1 average scores were 43%, 30%, and 25% for number identification, addition, and subtraction, respectively. Whereas for cohort 2, learners in Hausa and Kanuri basic literacy centers averaged between 51% and 55% on number identification, between 35% and 36% on addition, and between 29.5% and 29.8% on subtraction. Learners in the higher level post-basic literacy centers averaged higher scores with 69% on number identification, 43% on addition, and 35% on subtraction. SEL measures were somewhat similar between last year’s baseline and this year’s as mean hostile attribution scores were 40% for cohort 1 and between 31% and 47%, depending on the type of learning center, for cohort 2. In terms of conflict resolution, we found that cohort 1 learners resorted to aggression to resolve conflicts about 15% of the time, while that rate was between 8% and 12% among cohort 2 learners. Cohort 1 learners relied on disengagement to resolve conflict 34% of the time, this year’s cohort would employ disengagement between 36% and 42% of the time. Lastly, we find that this year’s learners would attempt to problem solve when facing a conflict between 44% and 51% of the time, while learners would employ problem solving 51.5% of the time based on last year’s baseline. Although, these contrasts in SEL measures are only baseline-to-baseline, it will be interesting to observe comparisons in SEL baseline-to-endline trajectories between the two cohorts. In terms of the mental health index, this is the first year of implementing this module, as such, we will not be able to compare between the two cohorts. However, we were able to discern some patterns in the data showing that learners in Kanuri centers had lower mental health scores than their counterparts in Hausa and post-basic literacy. At the same time, we observe that learners in post-basic literacy had the highest mean mental health index scores signifying a lower likelihood and frequency of reporting symptoms of depression. Finally, in terms of perceptions of school climate, we calculate average scores between 71% and 84.5% at baseline for cohort 2, with an overall average (across the full sample) of 80.1%. The baseline average of the school climate score from this year relative to last year’s baseline average are nearly identical whereby the mean score last year was 80.2%. Further, we calculate mean school safety scores between 74.1% and 82.1% at baseline for cohort 2 and an overall average of 80.8%. Unlike the findings from the school climate scores, the mean cohort 1 school safety score was 90.6% which is almost 10 percentage points higher than this year’s average score. As with typical baseline analyses, it is difficult to make any inferential conclusions or discern final patterns in the data. Even when contrasting between this year and last year’s baseline results as year-to-year many factors may change especially after a pandemic, disruption of services, and normal time trends outside of the program’s control. However, it will be interesting to observe the baseline-to-endline growth trajectories in terms of participants’ learning, SEL, and mental well-being.