DISCLAIMER: This evaluation inception report was produced at the request of the United States Agency for International Development. It was prepared independently by Ivy Meja, PhD., (Evaluation Team Leader), Romylyn Metila, PhD., Arlyne Marasigan, PhD., and the USAID/Philippines Collaborating, Learning, and Adapting for Improved Development (CLAimDev) activity. The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development of the United States Government. ADVANCING BASIC EDUCATION IN THE PHILIPPINES (ABC+) MID-TERM PERFORMANCE EVALUATION REPORT MARCH 31, 2023 SOURCE: WEBSITE OF RTI INTERNATIONAL 1 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV ABSTRACT This mid-term performance evaluation examined the relevance, effectiveness, and sustainability of the Advancing Basic Education in the Philippines (ABC+) program. The evaluation used a theory-based, mixed￾methods design to understand ABC+’s performance. To describe other outcomes brought by COVID-19, which emerged during the first year of the program, outcome harvesting (OH) and the most significant change technique were used. The evaluation focused on ABC+ activities in Regions V (Bicol), VI (Western Visayas), and a small portion of Bangsamoro Autonomous Region in Muslim Mindanao (BARMM) (Maguindanao Division in BARMM). The evaluation conducted four activities in collecting data: 1) electronic surveys for individuals such as teachers and school heads (intermediate result (IR)1 and IR2), 2) key informant interviews (KIIs) for system capacity-related topics with implementing partners and the Philippines Department of Education’s (DepEd) central, regional, and division offices (IR3), 3) focus group discussions and OH workshops on early-grade instruction and delivery (IR1) and access to materials (IR2), and 4) case studies for in-depth analysis of home learning partners and private sector engagements. The evaluation team found that ABC+ activities are relevant and aligned with DepEd policies and programs, particularly those related to learning recovery and continuity for literacy development. During the pandemic, ABC+ worked strategically by streamlining activities on teacher training, materials development, and system capacitation. Beneficiaries appreciated the close, consistent monitoring and intensive, flexible teacher support. As a result, ABC+ is a reliable and robust partner of schools. The evaluation also found that the teacher training and supplemental reading materials developed by ABC+ are effective in helping learners to read in their mother tongue and the Filipino language. The learning recovery plan, one of the policies enhanced with ABC+ support, was crucial in improving teachers’ instruction to help K-3 learners catch up with their reading delays. The DepEd system’s capacity to analyze and make decisions based on the data from comprehensive literacy assessment is very helpful in monitoring children’s reading outcomes. There is evidence that although the logic model of ABC+ was affected by the pandemic, the results are still promising because the program model is agile, i.e., with the return to face-to-face classes, ABC+ can pivot to achieve the intended outputs and outcomes. An unintended outcome brought by COVID-19 is the learning recovery plan co-developed by Region V and ABC+. This has contributed to IR1 and IR2. The evaluation also found that ABC+ contributed to enhancing policies related to curriculum development and assessment at the K-3 levels. Finally, the review found promising elements towards sustainability, as the project strengthened local ownership and leadership through diverse strategies and activities. Further collaboration with the private sector is imperative. Recommendations and implications for stakeholders are discussed in this report. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 2 ADVANCING BASIC EDUCATION IN THE PHILIPPINES (ABC+) Mid-Term Performance Evaluation Report March 2023 Contract/Award Number: 72049220D00004/72049222F00004 DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development of the United States Government. 3 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV CONTENTS ABSTRACT ................................................................................................................................. 1 ACRONYMS................................................................................................................................ 9 EXECUTIVE SUMMARY.......................................................................................................... 10 RECOMMENDATIONS ON RELEVANCE ...........................................................................................................12 RECOMMENDATIONS ON EFFECTIVENESS ....................................................................................................13 RECOMMENDATIONS ON SUSTAINABILITY..................................................................................................13 EVALUATION PURPOSE AND EVALUATION QUESTIONS .......................................... 15 RELEVANCE..................................................................................................................................................................15 EFFECTIVENESS ...........................................................................................................................................................15 SUSTAINABILITY.........................................................................................................................................................16 LEARNING QUESTIONS ..........................................................................................................................................16 BACKGROUND........................................................................................................................ 17 THEORY OF CHANGE (TOC) ...............................................................................................................................17 EVALUATION METHODS AND LIMITATIONS................................................................. 19 LOCATION AND GEOGRAPHIC SCOPE...........................................................................................................19 EVALUATION DESIGN .............................................................................................................................................19 DATA COLLECTION.................................................................................................................................................19 QUANTITATIVE METHOD.............................................................................................................................19 QUALITATIVE METHOD.................................................................................................................................20 TRIANGULATION. ............................................................................................................................................20 EVALUATION DESIGN MATRIX...................................................................................................................20 VALIDATION................................................................................................................................................................20 KNOWN LIMITATIONS TO THE EVALUATION DESIGN ...........................................................................21 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ............................................. 24 RELEVANCE: EVALUATION QUESTION 1 ........................................................................................................24 RELEVANCE: EVALUATION QUESTION 2 ........................................................................................................25 RELEVANCE: EVALUATION SUB-QUESTIONS ................................................................................................28 SUB-QUESTION 1.1.....................................................................................................................................28 SUB-QUESTION 1.2.....................................................................................................................................28 KEY MESSAGE ON RELEVANCE...................................................................................................................29 CONCLUSIONS ON RELEVANCE ...............................................................................................................29 RECOMMENDATIONS ON RELEVANCE ..................................................................................................30 EFFECTIVENESS: EVALUATION QUESTION 1..................................................................................................31 IR 1: IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS ................................31 IR 2: IMPROVED ACCESS TO QUALITY MATERIALS ............................................................................32 IR 3: IMPROVED EDUCATION SYSTEM CAPACITY AND COMMITMENT TO MANAGE AND IMPLEMENT INTERVENTIONS SUPPORTING EFFECTIVE EARLY-GRADE INSTRUCTION.....33 EFFECTIVENESS: EVALUATION SUB-QUESTIONS..........................................................................................33 KEY MESSAGE ON EFFECTIVENESS ............................................................................................................40 CONCLUSIONS ON EFFECTIVENESS ........................................................................................................41 RECOMMENDATIONS ON EFFECTIVENESS ...........................................................................................41 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 4 SUSTAINABILITY: EVALUATION QUESTION......................................................................................................43 KEY MESSAGE ON SUSTAINABILITY.............................................................................................................49 CONCLUSIONS ON SUSTAINABILITY.........................................................................................................49 RECOMMENDATIONS ON SUSTAINABILITY............................................................................................50 LIST OF ANNEXES (REDO THIS)........................................................................................1 ANNEX 1. AVAILABLE DATA..............................................................................................1 ANNEX 2. SITE AND RESPONDENT SELECTION CRITERIA........................................2 ANNEX 3. SAMPLING FRAME (INDIVIDUALS, INSTITUTIONS, ORGANIZATIONS)4 ANNEX 4. TARGET SAMPLE (INDIVIDUALS, INSTITUTIONS, ORGANIZATIONS) 5 ANNEX 5. DATA COLLECTION TOOLS ...........................................................................7 ANNEX 6. DOCUMENTS FOR DESK REVIEW ................................................................30 ANNEX 7. VERIFIED OUTCOMES.....................................................................................33 ANNEX 8. LEVEL 1 MOST SIGNIFICANT STORY .........................................................63 ANNEX 9. LEVEL 2 MOST SIGNIFICANT STORY ........................................................124 ANNEX 10. LEVEL 3 MOST SIGNIFICANT STORY ......................................................131 ANNEX 11. LOCALIZED SUSPENSION OF OBSERVATION IN REGION V ............136 ANNEX 12. LIST OF KII AND FGD PARTICIPANTS ....................................................137 ANNEX 13. SURVEY RESULTS.........................................................................................139 ANNEX 14. LIST OF CODES FOR THE THREE EVALUATION PARAMETERS AND RELIABILITY........................................................................................................................150 ANNEX 15. INFORMANTS’ IDENTIFIED POLICIES THAT ALIGN WITH ABC+....169 ANNEX 16. ACTIVITY SUMMARY INFORMATION OF ABC+ ...................................172 ANNEX 17. STATEMENT OF WORK ..............................................................................173 ANNEX 18. EVALUATION DESIGN AND METHODOLOGY......................................219 ANNEX 19. LEARNING QUESTIONS..............................................................................224 ANNEX 20. VALIDATION WORKSHOP AND SURVEY ..............................................226 ANNEX 20-A. VALIDATION SURVEY FREQUENCY TABLES....................................251 TABLE 1...................................................................................................................................251 TABLE 2...................................................................................................................................251 TABLE 3...................................................................................................................................252 TABLE 4...................................................................................................................................252 TABLE 5...................................................................................................................................252 TABLE 6...................................................................................................................................253 TABLE 7...................................................................................................................................253 5 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV TABLE 8...................................................................................................................................253 TABLE 9...................................................................................................................................254 TABLE 10 .................................................................................................................................254 TABLE 11 .................................................................................................................................254 TABLE 12 .................................................................................................................................255 TABLE 13 .................................................................................................................................255 TABLE 14 .................................................................................................................................255 TABLE 15 .................................................................................................................................256 TABLE 16 .................................................................................................................................256 TABLE 17 .................................................................................................................................256 TABLE 18 .................................................................................................................................257 TABLE 19 .................................................................................................................................257 TABLE 20 .................................................................................................................................257 TABLE 21 .................................................................................................................................258 TABLE 22 .................................................................................................................................258 TABLE 23 .................................................................................................................................259 TABLE 24 .................................................................................................................................259 TABLE 25 .................................................................................................................................259 TABLE 26 .................................................................................................................................259 TABLE 27 .................................................................................................................................260 TABLE 28 .................................................................................................................................260 TABLE 29 .................................................................................................................................260 TABLE 30 .................................................................................................................................261 TABLE 31 .................................................................................................................................261 TABLE 32 .................................................................................................................................261 TABLE 33 .................................................................................................................................262 TABLE 34 .................................................................................................................................262 ANNEX 20-B. VALIDATION SURVEY FREQUENCY TABLES ................................263 BY REGION ..........................................................................................................................263 TABLE 1...................................................................................................................................263 TABLE 2...................................................................................................................................263 TABLE 3...................................................................................................................................263 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 6 TABLE 4...................................................................................................................................264 TABLE 5...................................................................................................................................264 TABLE 6...................................................................................................................................264 TABLE 7...................................................................................................................................265 TABLE 8...................................................................................................................................265 TABLE 9...................................................................................................................................265 TABLE 10.................................................................................................................................265 TABLE 11.................................................................................................................................266 TABLE 12.................................................................................................................................266 TABLE 13.................................................................................................................................266 TABLE 14.................................................................................................................................267 TABLE 15.................................................................................................................................267 TABLE 16.................................................................................................................................267 TABLE 17.................................................................................................................................268 TABLE 18.................................................................................................................................268 TABLE 19.................................................................................................................................269 TABLE 20.................................................................................................................................269 TABLE 21.................................................................................................................................269 TABLE 22.................................................................................................................................270 TABLE 23.................................................................................................................................270 TABLE 24.................................................................................................................................270 TABLE 25.................................................................................................................................270 TABLE 26.................................................................................................................................271 TABLE 27.................................................................................................................................271 TABLE 28.................................................................................................................................272 TABLE 29.................................................................................................................................272 TABLE 30.................................................................................................................................272 TABLE 31.................................................................................................................................273 TABLE 32.................................................................................................................................273 TABLE 33.................................................................................................................................273 TABLE 34.................................................................................................................................274 ANNEX 21. TEACHERS’ EVALUATION OF ABC+ CONTRIBUTION TO DEVELOPING LEARNERS’ NUMERACY SKILLS...........................................................274 7 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV ANNEX 22. SELECTED MOST SIGNIFICANT CHANGE STORY...............................275 ANNEX 23. INCEPTION REPORT ...................................................................................276 ACRONYMS.........................................................................................................................280 EXECUTIVE SUMMARY.....................................................................................................281 PURPOSE OF THE EVALUATION...................................................................................284 BACKGROUND...................................................................................................................286 THEORY OF CHANGE (TOC)..........................................................................................287 RESULTS FRAMEWORK....................................................................................................290 EVALUATION QUESTIONS .............................................................................................298 RELEVANCE.................................................................................................................................................................. 298 EFFECTIVENESS ........................................................................................................................................................... 298 SUSTAINABILITY......................................................................................................................................................... 299 EVALUATION DESIGN AND METHODOLOGY ...........................................................300 LOCATION AND GEOGRAPHIC SCOPE........................................................................................................... 300 EVALUATION DESIGN ............................................................................................................................................. 300 METHODS ............................................................................................................................................................ 301 DESK REVIEW ..................................................................................................................................................... 301 SURVEY.................................................................................................................................................................. 301 KEY INFORMANT INTERVIEWS................................................................................................................... 301 FOCUS GROUP DISCUSSION....................................................................................................................... 301 OUTCOME HARVESTING WORKSHOP AND MOST SIGNIFICANT CHANGE WORKSHOP302 EVALUATION OF SUPPLEMENTARY READING MATERIALS (SRM)................................................ 302 CLASS OBSERVATION..................................................................................................................................... 303 SITE SELECTION CRITERIA..................................................................................................................................... 303 KII PARTICIPANTS AND COMPOSITION CRITERIA ............................................................................ 304 FGD PARTICIPANTS AND COMPOSITION CRITERIA......................................................................... 304 SURVEY.................................................................................................................................................................. 304 OH PARTICIPANT COMPOSITION AND SELECTION CRITERIA.................................................... 304 MSC PARTICIPANT COMPOSITION AND SELECTION CRITERIA.................................................. 305 DATA ANALYSES........................................................................................................................................................ 305 QUANTITATIVE ................................................................................................................................................. 305 QUALITATIVE ..................................................................................................................................................... 306 TRIANGULATION............................................................................................................................................. 306 EVALUATION DESIGN MATRIX............................................................................................................................ 306 KNOWN LIMITATIONS TO THE EVALUATION DESIGN ........................................................................... 306 THREATS TO INTERNAL VALIDITY .................................................................................................................... 306 DATA MANAGEMENT AND ETHICAL CONSIDERATIONS....................................................................... 317 WORKPLAN................................................................................................................................................................. 317 ANNEX 1. AVAILABLE DATA..........................................................................................318 ANNEX 2: SITE AND RESPONDENT SELECTION CRITERIA....................................319 ANNEX 3: SAMPLING FRAME (INDIVIDUALS, INSTITUTIONS, ORGANIZATIONS)322 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 8 ANNEX 4: TARGET SAMPLE (INDIVIDUALS, INSTITUTIONS, ORGANIZATIONS)325 ANNEX 5: DATA COLLECTION TIMELINE...................................................................328 ANNEX 6: SURVEY FOR TEACHERS ..............................................................................329 ANNEX 7: SURVEY FOR SCHOOL HEADS....................................................................339 ANNEX 8: KEY INFORMANT INTERVIEW QUESTIONNAIRE ..................................346 ANNEX 9: FOCUS GROUP DISCUSSION PROTOCOL ...............................................350 III. FGD QUESTIONS.................................................................................................................................................. 350 ANNEX 10: SRM EVALUATION TOOL FOR TEACHERS............................................355 ANNEX 11: ABC + GANTT CHART................................................................................358 ANNEX 12: OH WORKSHOP QUESTIONNAIRE .........................................................362 ANNEX 13: MOST SIGNIFICANT CHANGE STORY COLLECTION FORM.............363 ANNEX 14: MSC GUIDELINES FOR STORY VERIFICATION AND SELECTION ...367 ANNEX 15: VIEWING TOOL FOR RECORDED CLASSES...........................................368 ANNEX 16: DOCUMENTS FOR DESK REVIEW ............................................................370 ANNEX 17: ANTICIPATED SCHEDULES AND LOGISTICAL ARRANGEMENTS...375 ANNEX 24. EVALUATION TEAM....................................................................................377 ANNEX 25. REFERENCES .................................................................................................380 ANNEX 26. NONDISCLOSURE AND CONFLICT OF INTEREST AGREEMENTS...381 9 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV ACRONYMS ABC+ Advancing Basic Education in the Philippines BARMM Bangsamoro Autonomous Region in Muslim Mindanao CRLA Comprehensive Rapid Learning Assessment DepEd Department of Education EGR early-grade reading F2F face-to-face GESI gender equality and social inclusion HLP home learning partner IP implementing partner IR Intermediate Result KII key informant interview KPI key performance indicator LGU local government unit MSCT Most Significant Change Technique MTB-MLE Mother Tongue-Based-Multilingual Education NEAP National Educators’ Academy of the Philippines NGO non-government organization OH Outcome Harvesting PY Program Year RTI Research Triangle Institute International SEL socio-emotional learning SRM supplemental reading materials TEI teacher education institution UP-NISMED University of the Philippines National Institute for Science and Mathematics Education Development USAID U.S. Agency for International Development USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 10 EXECUTIVE SUMMARY This report presents the performance evaluation results of Advancing Basic Education in the Philippines (ABC+) using the three evaluation criteria of relevance, effectiveness, and sustainability. Specifically, the topline evaluation questions were: To what extent does the innovation introduced by ABC+ on improved early-grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade teaching correspond with the objectives of the Department of Education (DepEd) of the Philippines (priorities, policies, the needs, and the practical requirements)? To what extent has ABC+ contributed to improving early-grade literacy, the better transition of reading ability to Filipino and English, more solid acquisition of foundational math concepts and skills, and improved social and emotional learning for children in the early grades (kindergarten to Grade 3) in the Philippines? To what extent did ABC+ achieve its outputs and outcomes in the three intermediate results (IRs) on improved early-grade instruction and delivery, access to quality materials, and education capacity and commitment to manage and implement interventions supporting effective early-grade teaching? Will the achievements in improved early-grade reading, math, and social and emotional skills for children in the early-grades persist after the project’s conclusion? The evaluation also assessed ABC+’s performance in achieving its three intermediate results (IRs): IR 1: Improved early-grade instruction and delivery. IR 2: Improved access to quality materials IR 3: Improved education capacity and commitment to manage and implement intervention support in early-grade instruction. The evaluation team used a theory-based, mixed-methods design to assess ABC+’s performance. Specifically, a concurrent mixed methods evaluation approach using quantitative and qualitative methods was selected. In response to the challenges of early literacy development in mother tongue-based multilingual education (MTB-MLE), USAID/Philippines (USAID/PH), the Philippine DepEd, and Research Triangle Institute International (RTI) and its subcontractors co-developed the Advancing Basic Education in the Philippines project or ABC+. This five-year USAID-funded project aims to improve early-grade reading and emphasize using mother tongue languages to transition learning in Filipino and English for children in kindergarten to grade 3. It also supports a more solid acquisition of foundational mathematical concepts and skills and improves social and emotional learning. ABC+ integrates gender equality and social inclusion (GESI) in its design and programming. 11 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV Although ABC+’s objectives and design remain the same, the context within which it was implemented was significantly changed by the COVID-19 pandemic. Community restrictions necessitated by the pandemic, particularly school closures, affected ABC+’s intended inputs and activities. As a result, unexpected outcomes other than those reflected in the results framework are possible, mainly as the project was implemented within the unprecedented contextual changes in education due to COVID-19 and school closures. In addition to using theory-based mixed methods to answer the evaluation questions, the evaluation also used Outcome Harvesting (OH) to uncover unexpected outcomes not envisioned in the theory of change. The evaluation design has three levels of analysis: • Individuals supported by ABC+, such as teachers, school heads, faculty, and students. • Institutions focusing on system capacity, including the DepEd Central Office, Regional and Division levels, schools; and • Community organizations and private sector affiliates are involved in ABC+ implementation. The evaluation focused on school divisions supported by ABC+ in Regions V (Bicol), VI (Western Visayas), and a small portion of Bangsamoro Autonomous Region in Muslim Mindanao (BARMM) (Maguindanao Division in BARMM). The selection criteria used two filters, geographic location with the associated linguistic focus and participant profile. Participants were teachers and officials from DepEd Regional and Division levels. For the first filter, Bicol, Western Visayas, and Maguindanao Schools Division in BARMM were automatically selected as part of ABC+. The following filter was the school divisions that received ABC+ assistance. For the selection criteria of respondents, there were three filters. The first filter was the geographic location (Regions V, VI, and a small portion of BARMM). The second filter was a proportional sampling from selected divisions. Selected divisions were based on the scale used by ABC+, where the most used languages were the basis for site selections. Additional selection criteria included literacy rate, type of schools, proximity to the city, mode of transportation to reach the site, and availability of electricity and internet connectivity for remote data collection. Finally, using these criteria, ABC+ schools were randomly selected. The evaluation used four activities to collect data: 1) electronic surveys for individuals such as teachers and school heads (IR1 and IR2), 2) key informant interviews (KIIs) with implementing partners and the DepEd’s central, regional, and division offices to probe system capacity (IR3), 3) focus group discussions (FGDs) and OH workshops on early-grade instruction and delivery (IR1) and access to materials (IR2), and 4) a case study for in-depth analysis of home learning partners (HLPs) and private sector engagement. The evaluation team found that ABC+ activities are relevant and highly aligned with DepEd policies and programs. They were instrumental in implementing learning continuity and learning recovery programs, especially literacy instruction, which received the most significant level of ABC+ support. For example, ABC+ has already conducted several activities demonstrating the availability of resources to strengthen DepEd’s literacy and numeracy programs and improve English proficiency while recognizing linguistic USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 12 diversity long before the recent Basic Education Report 20,23,1 delivered by Vice President Sara Duterte.2 Since the ABC+ activities for developing numeracy have just started, improvement in learners’ numeracy skills is expected to manifest in the coming years. Survey data show that ABC+’s contribution to socio￾emotional learning (SEL) integration was achieved to a large extent. Due to the absence of face-to-face instruction, ABC+ support for SEL integration was primarily undertaken through supplementary reading materials (SRMs). Findings revealed that ABC+’s locally developed age and developmentally-appropriate reading materials for the K-3 levels help learners read in their mother tongue and Filipino language. The unintended outcome brought about by COVID-19 is the learning recovery plan, which contributed to IR1 and IR2. ABC+’s contribution to enhancing policies was most observed in curriculum development and assessment at the K-3 levels. ABC+’s potential for sustainability was observed in Regions V and VI based on identified mechanisms such as training given by ABC+ and DepEd in partnership with the National Educators’ Academy of the Philippines (NEAP), the commitment of teacher trainers during the training of trainers (TOT) to continuously adapt ABC+ instructional strategies, license attribution of the SRMs and creative commons. The creative commons or distribution of copyrighted SRMs were given for free and was influential in that everyone may have a copy or reproduce without worrying about copyright issues. School heads and teachers recognized the effectiveness and efficiency of Comprehensive Rapid Learning Assessment (CRLA) when communicating and monitoring learners’ progress. Updating CRLA and possible innovations could make this CRLA more relevant to teachers and learners. Through the Mandanas-Garcia Ruling implementation, the LGU has the prerogative to allocate additional funds to deliver basic education services. The LGU also participates in the local school board meeting to assist DepEd with other financial resources allocation. These mechanisms mitigate the limited budget allocation of DepEd, which is vital to sustaining the ABC+ program after its conclusion. Based on the ABC+ sustainability milestones, the private sector contributes to the program’s sustainability by providing financial and other support systems to schools and communities. However, they have limited involvement, so further collaboration is imperative. Private sector organizations, such as the Rotary Club International and Coca-Cola Foundations, donated school supplies, photocopying machines, and cash. Their presence in the school and community projects and programs sustains DepEd's direction to achieve early-grade literacy. To a large extent, these sustainability mechanisms contribute to improving the ABC+ program leading to its potential to last long-term. RECOMMENDATIONS ON RELEVANCE For ABC+ and DepEd, the team recommends that processes for initiating local policy enhancements be shared through fora or meetings so that other regions can also identify local policies that can be 1 Duterte, S.Z. (2023). Basic Education Report 2023 [PowerPoint slides]. Department of Education. Office of the Secretary, Philippines. 2 Note, however, that this is only true for Regions V and VI as the host institutions of the ABC+ intervention. 13 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV enhanced. ABC+ can assist schools in disseminating best practices, especially in implementing recovery programs that concentrate on literacy development. ABC+ can likewise support schools to create platforms for giving or sharing good practices, such as school visits for benchmarking or an online/face￾to-face (F2F) forum or presentation. Since numeracy instruction is designed to be a pilot model for developing a National Framework on Numeracy, ABC+ could explicitly create and establish a supplementary framework that targets numeracy instruction in the key stages. This framework may emerge from the set and implemented lesson plans from its ongoing project with the University of the Philippines National Institute for Science and Mathematics Education Development (UP-NISMED). ABC+ and the DepEd regional offices should bolster efforts for SEL integration, especially in Regions V and VI. F2F class observations can help determine the training support that can be provided to help teachers increase their level of SEL integration. RECOMMENDATIONS ON EFFECTIVENESS For ABC+, the evaluation team recommends gathering real-time data, via classroom observations in the three regions, on teachers’ abilities to implement the science of teaching beginning reading, which was a focus of ABC+’s training. A research write-shop for K-3 teachers may increase teaching reading efficacy or belief in teachers' capacity to develop reading skills among learners. Classroom-based research outputs of teachers may serve as additional evidence of the effectiveness of ABC+ interventions. Efforts should be made to institutionalize improved literacy training for teachers and simultaneously encourage host institutions to commit to the continuity of production of quality SRMs. The DepEd regional and division offices may assist teachers in applying what they learned from the teacher training through mentoring and monitoring activities. This will serve as a refresher approach since most of the training was done during the height of the pandemic. Locally trained ABC+ facilitators may include this in their monitoring and evaluation activities in their regions and divisions. The NEAP may continue to explore a training design and schedule that will support and cater to teachers’ needs, situations, and well-being. For instance, teachers expressed the need to moderate the number of training tasks/assignments, especially considering that some teachers hold other jobs. SRMs were found to have the most significant contribution to improving learners’ reading capacity in their mother tongue, Filipino, and English. Therefore, local government units (LGUs) should ensure that SRM hard copies are protected by providing space to store them safely and maintain the production and replenishment of SRMs to different divisions by conducting an annual inventory of equipment supporting the year-round material availability. RECOMMENDATIONS ON SUSTAINABILITY The recommendations on sustainability are entrusted to three agencies/organizations: ABC+, DepEd’s Central, Regional, and Division Offices, and LGUs. The evaluation team recommends that ABC+: USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 14 • assists DepEd in scaling up collaboration with the private sector to outsource funds for Program Years (PY) 4 and 5. • strengthens and formalizes collaboration with teacher education institutions in the region through Memorandums/Agreements of Understanding to prepare and train future in-service teachers to promote and support early-grade literacy for PY 4-5. • encourages the active participation of volunteer parents, youth organizations, and other willing individuals in the community during the school learning action cell session for PY 4-5. • collaborates with schoolteachers to conduct action research focusing on classroom and community-based reading literacy and numeracy for future reference and an additional action plan toward PY 4-5 sustainability. • lays the groundwork for transitioning ownership of the program from implementing partner (IP) to host institutions and organizations, in particular continuing to improve literacy training for teachers and encouraging host institutions to commit to the production of quality SRMs; • explicitly develops and designs a supplementary framework that targets numeracy instruction in the critical stages since numeracy instruction is intended to be a pilot model that will support the plan for creating a National Framework on Numeracy. This framework may emerge from lesson development and implementation with UP-NISMED as their partner institution; and • assists regions in disseminating ideas to improve or contextualize national or regional policies that ABC+ already supports. Data revealed that due to partner schools’ confidence in ABC+ support, DepEd regional offices expanded regional policies that involved only literacy to include numeracy. Some regional policies were also enhanced in terms of budget allocation. Processes for initiating these regional policy enhancements can be shared with other regions through fora or meetings so that other regions can also identify local policies that can be enhanced or further contextualized. The DepEd Central Office may adopt enhanced policies on assessment and educational sector management at the national level. ABC+ provided technical support on early-grade-related policies to Regions V and VI. Policies were effective, and this may work in other regions. The regional and division office may support teachers, students, community volunteers, and HLPs through well-being mechanisms such as counseling, open dialogue, psychosocial training, and distributing well-being flyers. LGUs may also assist DepEd in sustaining the financial needs of schools and the ABC+ program by allocating additional funds to the following: teacher training, technology transfer capacity-building program of school personnel, implementation and monitoring of the program, and maintaining barangay reading centers and learning hubs through HLPs and SRMs production and distribution. LGUs and local stakeholders should collaborate to mobilize, support, and sustain the established program. Given the limited human and financial resources of numerous schools in Regions V and VI and selected schools in BARMM, especially in remote areas, educational public-private partnership is one possible means to ensure the program’s sustainability. 15 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV EVALUATION PURPOSE AND EVALUATION QUESTIONS The United States Agency for International Development/Philippines (USAID/PH) commissioned an independent third-party performance evaluation of Advancing Basic Education in the Philippines (ABC+), which is implemented by Research Triangle Institute (RTI) International and its partners - The Asia Foundation, SIL LEAD, and Florida State University. This evaluation provides USAID/PH evidence to gauge the extent to which ABC+ is achieving its outputs and outcomes in the three intermediate results (IRs) on improved early-grade instruction and delivery (IR1), improved access to quality materials (IR2), and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction (IR3). The evaluation focused on the performance of ABC+ from 2019 to 2022, assessing its relevance, effectiveness, and sustainability. The following are the evaluation questions: RELEVANCE • To what extent does the innovation introduced by ABC+ on improved early-grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction correspond with the objectives of the Department of Education of the Philippines (DepEd) (priorities, policies, the needs, and the practical requirements)? • Did ABC+’s design and implementation consider the available capability and capacities of the recipients (human resources and organizations) and responses to any change in context (e.g., public health emergency such as COVID-19)? • How did ABC+ incorporate gender equality and social inclusion (GESI) analysis? To what extent has ABC+ implementation supported GESI? • To what extent has ABC+ contributed to improving early-grade literacy, the better transition of reading ability to Filipino and English, more solid acquisition of foundational math concepts and skills, and improved social and emotional learning for children in the early grades (kindergarten to Grade 3) in the Philippines? EFFECTIVENESS 1. To what extent did ABC+ achieve its outputs and outcomes in the three IRs on improved early￾grade instruction and delivery, improved access to quality materials, and enhanced education capacity and commitment to manage and implement interventions supporting early-grade instruction? 2. Which of the three IRs contributed the most to improving early-grade reading, math, and social and emotional skills for children in the early-grades (kindergarten to Grade 3)? What were the explanatory factors, if any, for achievement, overachievement, and underachievement or non￾achievement? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 16 3. Have there been any outcomes brought by COVID-19 in implementing ABC+ that would tell us whether the program improved early-grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction? 4. How has ABC+ contributed to enhancing policies on early-grade reading instruction and learning outcomes in kindergarten to Grade 3 (K–3)? SUSTAINABILITY • Will the achievements in improved early-grade reading, math, and social and emotional skills for children in the early grades persist after the project’s conclusion? • How did ABC+ strengthen local ownership and leadership on the enhanced instruction and learning outcomes in kindergarten to Grade 3? The evaluation results will inform the design of the Mission’s next-generation basic education activities. The intended audiences of this evaluation are USAID-Philippines education and program officers and USAID staff worldwide interested in basic education programs, specifically at K-3 levels; Philippine and U.S. stakeholders, including those in other U.S. government agencies and organizations; the DepEd in the Philippines; and other researchers and organizations interested in the use of mother tongue-based multilingual education (MTB-MLE) to strengthen literacy and numeracy in primary education, especially within the context of the recent pandemic. Annex 16 shows the activity summary information of ABC+. LEARNING QUESTIONS USAID requested a review of four learning questions during the evaluation inception meeting. While the evaluation team collected and analyzed data regarding these questions, the team conducted the analysis separately from the topline questions on relevance, effectiveness, and sustainability. Refer to Annex 19 for details. 17 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV BACKGROUND DESCRIPTION OF THE PROBLEM AND CONTEXT ABC+ is a five-year USAID-funded program aimed at improving the early-grade reading, math, and social and emotional skills of children in the early grades (kindergarten to Grade 3) in the Philippines. The prominent feature of early-grade instructions in the K-12 curriculum in the Philippines is the MTB-MLE policy. From kindergarten through Grade 3, learners progressively acquire literacy skills in their native language (MT), Filipino, and English, transitioning to instruction in Filipino and English starting in Grade 4. In a country with 180-plus native languages, the delivery of MTB-MLE throughout the country can be daunting. The baseline study by ABC+ reflects that many children are left behind in school. Grade 2 learners’ decoding skills are underdeveloped, and their reading and comprehension abilities are weaker than their oral language skills. Although supplemental materials are available at the DepEd, teachers do not use these as often as intended. In response to the challenges of early literacy development in the context of MTB-MLE, USAID/Philippines, DepEd, and RTI and its subcontractors co-developed a project to improve early￾grade basic language skills, with an emphasis on enhancing early-grade reading (EGR) in the mother tongue and preparing to transition to Filipino and English. ABC+ supports improved early-grade literacy, a better transition of reading ability to Filipino and English, more solid acquisition of foundational math concepts and skills, and improved social and emotional learning. ABC+ works with the Philippine DepEd in Manila, as well as with its regional- and division-level offices in Regions V (Bicol) and VI (Western Visayas) and selected districts of Maguindanao in the Bangsamoro Autonomous Region in Muslim Mindanao (BARMM). The estimated beneficiaries are two million students, 36,000 teachers, 6,600 school heads, and 644 supervisors. In March 2020, schools closed due to COVID-19, disrupting ABC+’s implementation, which started four weeks earlier. Most activities became remote, including teacher training. In December 2021, selected DepEd schools piloted face-to-face (F2F) classes. Beginning in January 2022, DepEd progressively expanded F2F classes in 6,000 schools as part of its learning recovery plan. ABC+ partnered with DepEd Region V to contribute to this effort in preparing the Recovering for Academic Achievement by Improving Instruction through Sustainable Evidence-Based Learning Programs (RAISE). This three-year learning recovery plan aims to ensure that the COVID-19 generation of learners does not suffer a disadvantage compared with past and future generations. THEORY OF CHANGE (TOC) ABC+’s design assumes that: “IF quality of instruction is improved (IR1), quality and availability of materials are enhanced (IR2), and education system capacity to support early-grade learning (IR3) is improved, THEN it will lead to positive and sustained gains in students’ basic skills.” ABC+ seeks to achieve its overall objective through the following IRs: IR 1: Improved early-grade instruction and delivery systems USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 18 IR 2: Improved access to quality materials IR 3: Improved education system capacity and commitment to manage and implement interventions supporting effective early-grade instruction. Detailed background descriptions are in the Statement of Work (Annex 17). 19 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV EVALUATION METHODS AND LIMITATIONS LOCATION AND GEOGRAPHIC SCOPE The evaluation covered Regions V (Bicol), VI (Western Visayas), and a small portion of BARMM (Cotabato, Maguindanao Division I in BARMM), in which ABC+ is being implemented. The evaluation used the division, district, and school levels in its sampling frame to purposively select informants using ABC+ participant lists. EVALUATION DESIGN Anchored on the theory of change, the evaluation used a theory-based, mixed-methods design to assess ABC+’s performance. The evaluation methodology used concurrent mixed methods3 to better understand the implementation of ABC+ through qualitative and quantitative assessment inquiries. The selection of key informant interviews (KIIs) and focus group discussion (FGD) participants was purposive, while the survey respondents were randomly sampled. The minimum acceptable response rate for the adapted survey was 50 percent (Babbie, 1990).4 Although the ABC+ activity’s objectives and design did not change with the onset of COVID-19, the pandemic significantly altered the context within which it was implemented. To capture unintended outcomes, the evaluation used Outcome Harvesting (OH) to uncover unexpected results and contributions not envisioned in the theory of change.5 DATA COLLECTION In this evaluation, mixed methods were used to analyze the implementation of the ABC+ interventions. The quantitative method used an online survey, and qualitative methods included literature reviews, KIIs, FGDs, OH workshops, and the Most Significant Change Technique (MSCT). Using multiple data sources through mixed methods enabled the evaluation to cover a broad spectrum, conduct a more in-depth analysis, and achieve a more holistic understanding of data through triangulation (Yin, 2003)6. Methods were conducted both virtually and in person. The sequence of data collection was as follows: desk review, survey, KIIs, FGDs, OH workshops, MSCT, and evaluation of supplementary materials. The evaluation team also conducted a validation workshop on the accuracy of the evaluation information. QUANTITATIVE METHOD The evaluation used descriptive statistics and analytic data visualizations for the quantitative data and disaggregated data by sex and location to the extent possible. This approach is consistent with USAID’s embedded monitoring and evaluation in the Program Cycle.7 Data from the survey were analyzed 3 Mertens, D. M. (2017). Mixed methods design in evaluation. Thousand Oaks: CA: Sage. 4 Babbie E. Survey Research Methods. Belmont, Calif: Wadsworth; 1990. 5 The World Bank. (2014). Outcome-based Field Learning Guide. Washington, D.C. (p.13) 6 Yin, R.K. (2003). Case Study Research: Design and Methods. Sage. Thousand Oaks, California. 7 USAID Learning Lab: Monitoring in the Program Cycle USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 20 through descriptive statistics. Analysis of progress of ABC+ toward indicators derived from ABC+ quarterly/annual reports. However, the absence of needed data during the scheduled time for analyses precluded: 1) comparison of division and school-level data at the start and midline; and 2) determination of the effect size of interventions by cohort, divisions, sex, and location. Classroom observation was also not done due to the COVID-19 pandemic restrictions imposed by DepEd. Other survey items are found in Annex 13. QUALITATIVE METHOD The qualitative data were recorded and transcribed in worksheets. They were subjected to content analysis using computer-assisted qualitative data analysis software (CAQDAS). The evaluation team coded the responses and computed the estimated reliability index. A constant comparison method complemented the index where the crew checked the coherence of responses, codes, and themes to ensure that the results had no external threats to the audience of this performance evaluation. The team also used content analysis on the outcomes harvested. Qualitative data analysis was used for FGD, KII, MSCT, and OH data to find emerging themes. Computer software, such as SPSS and NVivo, was used for data processing and analysis in quantitative and qualitative studies. The codes used and reliability results are found in Annex 14. TRIANGULATION. The evaluation team combined and compared the quantitative and qualitative analysis results for a holistic understanding of ABC+ concerning relevance, effectiveness, and sustainability. EVALUATION DESIGN MATRIX. The design matrix in Table 1 summarizes the evaluation design and methods. The details of the evaluation methodology, such as the data collection and its corresponding participants, data management, and ethical consideration, can be found in Annex 18. VALIDATION The validation workshop was conducted via Zoom. Representatives from Regions V, VI, and selected school districts in BARMM participated in the workshop held on February 24, 2023. The ABC+ evaluation findings, conclusions, and recommendations were presented, and participant feedback was collected. The validation workshop used a tool that consists of 34 items that cover statements capturing the results and recommendations under relevance (with 11 words), effectiveness (with 11 statements), and sustainability (with 12 comments). It was sent to the participants two weeks before the evaluation workshop together with the presentation of the findings, to give time for the participants to review and evaluate the results. Interviews and narratives from the participants supplemented this tool. There were unexpected technical glitches (i.e., difficulty connecting to the internet) and overlapping activities by the attendees during the virtual validation (i.e., ABC+ workshop and last-minute DepEd 21 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV seminar simultaneously happening with the validation workshop). As a result, some attendees were either unable to attend or were distracted during the process. A validation survey was also conducted, which yielded 65 respondents, and this regional breakdown: Region V – 17, Region VI – 40, and BARMM – 8. Overall, albeit using a small number of informants, the validation survey results generally confirm the midline evaluation findings and recommendations for ABC+. Across regions, respondents’ answers confirm the veracity of the presented findings and support the given recommendations. While some responses were received for statements that disconfirm findings (i.e., not true in our locale), this is a very small number. It may be an indication though that program implementation has not covered the entirety of a target location. The validation survey results, combined with the data gathered from the conducted online validation workshop, confirm the midline evaluation findings for ABC+. The documentation of the workshop and the survey can be found in Annex 20. KNOWN LIMITATIONS TO THE EVALUATION DESIGN This performance evaluation has four potential limitations: reduced sample size, inaccessible sites, ability to recall information for self-reported data and limited access to documents. First, some respondents selected for the KII and FGD could not participate due to prior commitments, health reasons/COVID-19, weak internet connectivity, and power outages. This resulted in a reduced number of actual versus expected participants. Second, some study sites were inaccessible due to the distance from the regional or district offices (i.e., limited areas were visited in BARMM compared to those in the city proper of Cotabato). Third, the self-reported data, the survey, KIIs, and FGDs depended on what people could remember. Lastly, accessing documents where ABC+ contact persons were no longer connected with their schools or transferred to other schools or offices took much work. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 22 TABLE 1. EVALUATION DESIGN MATRIX EVALUATION QUESTIONS SPECIFIC QUESTIONS DESIGN SUGGESTED DATA SOURCES SUGGESTED DATA COLLECTION METHODS 1. Relevance: To what extent does the innovation introduced by ABC+ on improved early-grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early￾grade instruction correspond with the most recent objectives of the Department of Education of the Philippines (priorities, policies, the needs, and the practical requirements)? 1.1. Did ABC+’s design and implementation consider the available capability and capacities of the recipients (individual and organizational) and responses to any change in context (e.g., public health emergency such as COVID-19)? Qualitative Project documents and reports IPs: RTI (the prime partner), The Asia Foundation, SIL LEAD, and Florida State University Key stakeholders: DepEd, school boards, community, parent-teacher associations Document review KIIs (Central Office, regional offices, school divisions) 1.2. How did ABC+ incorporate gender equality and social inclusion (GESI) analysis? To what extent has ABC+ implementation supported GESI? Qualitative Project documents and reports Document review FGD: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators To what extent has ABC+ contributed to improved early-grade literacy, the transition of reading ability to Filipino and English, the acquisition of foundational math concepts and skills, and social and emotional learning for children in the early grades (kindergarten to Grade 3) in the Philippines? Quantitative Qualitative Survey Project documents and reports IPs: The Asia Foundation and SIL LEAD Online survey from Document review 2. Effectiveness: To what extent did ABC+ achieve its outputs and outcomes in the three intermediate results (IRs) on improved early-grade instructions and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early￾grade instruction? 2.1. Which of the three IRs contributed the most to the achievement of improving early-grade reading, math, and social and emotional skills for children in the early-grades (kindergarten to grade 3) Quantitative Qualitative Survey Project documents and quarterly reports Assessment tools Key stakeholders: DepEd, school boards, community, parent-teacher associations Online survey form Document review KIIs (Central Office, Regional Offices, School Divisions) FGD: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators 2.2 Have there been any outcomes brought by COVID-19 in implementing ABC+ that would tell us whether the program improved early-grade instructions and delivery, improved access to quality materials, and improved education capacity and Qualitative Project documents and reports Outcome Harvesting workshop and Most Significant Change Technique Document review 23 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV TABLE 1. EVALUATION DESIGN MATRIX EVALUATION QUESTIONS SPECIFIC QUESTIONS DESIGN SUGGESTED DATA SOURCES SUGGESTED DATA COLLECTION METHODS commitment to manage and implement interventions supporting early-grade instruction? 2.3. How has ABC+ contributed to enhancing policies on EGR instruction and learning outcomes in kindergarten to Grade 3? Qualitative Key stakeholders: DepEd, school boards, community, parent-teacher associations KIIs (Central Office, regional offices, school divisions) FGD: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators 3. Sustainability: Will the achievements in improved early-grade reading, math, and social and emotional skills for children in the early-grades persist after the project’s conclusion? How did ABC+ strengthen local ownership and leadership on the enhanced instruction and learning outcomes in kindergarten to Grade 3? Quantitative Qualitative Organization records Policies, personnel records, and student records Key stakeholders: DepEd, school boards Online survey form Document review KIIs (Central Office, regional offices, school divisions) FGDs: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators Site visit to check the institutional capacity USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 24 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS Two questions guided the evaluation of ABC+’s relevance: the first asked about how the program innovations aligned with DepEd objectives and policies, and in contrast, the second asked about how ABC+ improved four learning aspects of children in the early-grades. RELEVANCE: EVALUATION QUESTION 1 To what extent does the innovation introduced by ABC+ on improved early-grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction correspond with the objectives of the Philippines DepEd (priorities, policies, needs, and the practical requirements)? FGD and survey data findings show that ABC+ innovations aligned highly with DepEd policies. Its activities were instrumental in partner schools’ implementation of learning continuity, especially for literacy instruction. Data show that at the institutional level, ABC+ activities have aligned much or somewhat with DepEd programs and policies. Annex 15 presents a list of national and regional DepEd policies that ABC+ activities align with, according to informants. Figure 1 below unpacks policies and programs that align with ABC+ innovations. Informants explained that ABC+ is so aligned with their activities that it does not appear like a “separate program because all the initiatives support what we are already doing.” Another explained, “What ABC+ has been doing is that they made sure that all their undertakings are aligned with the BARMM’s Ministry of Basic, Higher, and Technical Education (MBHTE) and DepEd.” Results revealed that partners enhanced or contextualized existing policies due to their ABC+ experience or confidence in ABC+ support. Figure 1. ABC+ Initiatives For Improved Literacy Instructions Among DepEd policies and programs, ABC+ was particularly relevant in supporting DepEd’s learning continuity and learning recovery programs. As schools grappled with remote learning and the new demands of education, ABC+ supported schools’ learning continuity programs by providing 25 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV supplemental reading materials (SRMs) to support reading practice. ABC+ teacher training also provided teachers with strategies they applied in preparing the modules for learners. ABC+ ensured that instructional materials were distributed on time with the help of stakeholders. Teacher training and materials development continued during the pandemic through online platforms and virtual communication. The pandemic heightened the need for teacher training and materials development, and ABC+’s ability to supply these heightened its relevance, mainly because the regular training for teachers stopped and schools did not have adequate resources to support learners remotely to develop their reading skills. According to one interviewed teacher, “ABC+ joined us at the height of the pandemic, so the experience was challenging, but at least, with their help, everything became easier.” ABC+’s relevance extends to the present, as schools try to recover learning lost during the pandemic. The materials that ABC+ provided and the training knowledge and skills teachers gained are valuable in learning recovery programs. As explained by a teacher, “ABC+ is very relevant, and it is what we need right now [learning recovery].” RELEVANCE: EVALUATION QUESTION 2 To what extent has ABC+ contributed to improving early-grade literacy, the better transition of reading ability to Filipino and English, more solid acquisition of foundational math concepts and skills, and improved social and emotional learning for children in the early grades (kindergarten to Grade 3) in the Philippines? Improving early-grade literacy and the better transition of reading ability to Filipino and English. The data presented could be more nuanced for transitioning reading skills to Filipino and English. Due to the learning gap incurred during the pandemic, the data imply that much of the reading skills described by informants are only for the mother tongue and Filipino, and transitioning the reading skills from the mother tongue to the second language will be taken care of after establishing foundational reading skills. Survey results show that 80 percent of teachers in all the target regions feel that ABC+ initiatives have been instrumental in improving early-grade literacy instruction. These findings are supported by qualitative data where informants described ABC+ as their “partner” in literacy teaching. Figure 1 above unpacks the extent of ABC+’s contribution to literacy improvement across individual, institutional, and organizational levels. ABC+ initiated collaboration with parents by capacitating home learning partners (HLPs) for remote learning and using SRMs that targeted literacy development. For teacher training, informants identified the content on pedagogy and technology use as crucial topics that improved their reading instruction. Teachers reported that the provision of literacy instructional materials improved their literacy instruction because materials were accessible via the Learning Resource Portal and were ready to print and easy to use. One informant shared that “learning became easier for children because of the materials provided by ABC+.” At the institutional level, ABC+ contributed to school programs supporting literacy, learning continuity and recovery, materials development and distribution, and assessment-informed instruction. ABC+ contributed to school programs by providing teacher training and materials. The most significant ABC+ contribution was its support of learning continuity and recovery. A teacher shared that ABC+ helped them design and implement an especially relevant intervention USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 26 during learning recovery because there were so many non-readers. This effort to focus on fundamental skills such as literacy skills for learning recovery aligns with the World Bank’s Reach, Assess, Prioritize, Increase, Develop (RAPID) Framework for learning recovery8, and the National Learning Recovery Plan was presented in DepEd’s Basic Education Report 2023.9 On materials development and distribution, ABC+ assistance supported teachers’ skills development in materials writing and evaluation or quality assurance. Moreover, SRMs, which are leveled readers in the local language, help children practice their reading skills and develop an interest in reading. From a target of 576 titles, ABC+ produced 1,402 titles of EGR materials. These materials were made available to 750,479 learners (from a target of 523,201). Schools implemented assessment-informed instruction through ABC+’s provision of the comprehensive rapid literacy assessment tool, which diagnosed learners’ reading skills and provided guidance for differentiated instruction tailored to learners’ diagnosed reading levels. Starting with an assessment to inform recovery learning intervention is consistent with the World Bank’s RAPID framework to regain learning lost during the pandemic. ABC+’s collaboration with teacher education institutions (TEIs) resulted in the establishment of a pre-service course in literacy instruction. Through this partnership, ABC+ prepared pre-service teachers (including current and incoming teachers) early on for reading instruction. Collaboration with the community involving the barangay, the police, private radio stations, and others provided the wraparound support needed to ensure learning continuity. The barangay and police helped ABC+ and partner schools deliver instructional materials to learners, while the partnership with radio stations enabled the broadcasting of lessons. ABC+ also initiated meetings to establish memorandums of agreement between schools and some stakeholder groups to ensure learning continuity. Meetings initiated by discussions paved the way for some stakeholder groups to express their commitment to supporting school programs. 1. Some of the following unexpected outcomes emerged from ABC+ activities: 2. Teacher training and materials development produced a ripple effect. FGD informants from Regions V and VI said that trained teachers went on to train intermediate-grade teachers in their school and early-grade teachers in other schools. One teacher informant shared that the training has “enriched [me]…in so many ways,” She planned to cascade these skills with other teachers in the Division. 3. FGD informants reported that learners at home used the materials that ABC+ produced, unlike the pre-pandemic practice where children were not allowed to bring home instructional resources. An informant from Region V shared that “learning became easier for children through the materials provided by ABC+.” Informants explained that materials benefited not only the learners but also their siblings and parents, some of whom had been illiterate and learned to read along with their children through the materials. An informant from Region VI said that “parents learned together with their kids” and that “illiterate 8 World Bank, the Bill and Melinda Gates Foundation, FCDO, UNESCO, UNICEF, and USAID. 2022. Guide for Learning Recovery and Acceleration: Using the RAPID Framework to Address COVID-19 Learning Losses and Build Forward Better. Washington, DC: World Bank. 9 Duterte, S.Z. (2023). Basic Education Report 2023 [PowerPoint slides]. Department of Education. Office of the Secretary, Philippines. 27 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV parents were able to learn through reading materials put up [made available through hubs] in sitios and haciendas.” 4. In Region V, informants shared that schools reproduced ABC+ materials. Any excess they received was shared with other schools, so schools beyond partner schools also benefited. Developing a more solid acquisition of foundational math concepts and skills. Although literacy has traditionally been ABC+’s primary focus, numeracy development is also becoming an additional target since it has been identified as a skill for which children likewise need help. However, the project’s numeracy focus has been initiated only recently. In 2021, ABC+'s efforts to improve numeracy began with research in collaboration with the University of the Philippines National Institute for Science and Mathematics Education Development (UP-NISMED). The study looked into the state of early-grade math instruction, and results were shared with DepEd. Based on the research results, math instruction pivots based in the research results were designed, piloted, and evaluated. In 2022, ABC+ conducted teacher training on numeracy in Regions V and VI (Teacher Training I). Consultations with the Early Numeracy Technical Working Group began, and Teacher Training 2, which trained participants to connect formal and informal mathematics for children, was conducted in Regions V and VI. Math instructional strategies in the two regions were piloted by distributing ABC+ home learning resource kits that included materials for numeracy and learning action cell modules on early-grade mathematics. Survey findings show that five out of ten teachers in the regions reported moderate ABC+ contributions to developing math skills. These contributions were reported on training, materials, and program support. These findings are validated by a comment shared by a Region VI informant at the evaluation team's online Results Validation Workshop with DepEd. The informant explained that more activities on developing numeracy could be organized in the following stages of implementation. Activities for developing mathematics skills were only recently implemented, and their impact will only be noticed after some time. Informants shared that numeracy materials developed are slated for distribution to partner schools soon. Training support was scheduled for late 2022 at the time of this evaluation. This training involves integrating vital instructional strategies and collaborative lesson planning, which will be demonstrated to children or peers and revised based on trainer feedback. Class observations for developing numeracy skills are also scheduled for early 2023. Improving social and emotional learning. ABC+ integrated efforts to strengthen socio￾emotional learning (SEL) through instructional leadership training, believing that school leaders are critical in fostering learning environments that encourage SEL. The ABC+ Quarter 3 report states that the training taught DepEd leaders to foster conditions resulting in a positive learning environment. The training provided resources and toolkits that featured SEL in school management and showed supervisors and school heads how to integrate SEL and GESI into their school programs and interventions. The report also noted that ABC+ embedded SEL in SRM development by featuring an SEL orientation and workshop in materials and training. Moreover, SEL messaging was strongly encouraged in the themes of produced SRMs. Teacher Training 2 also taught teachers how to integrate SEL/GESI themes in book selection and other class exercises. Based on the survey of teachers, ABC+ helped in improving learners’ SEL to a large extent. Almost eight out of ten teachers in BARMM stated this was the case, while nearly six out of ten Regions V USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 28 and VI teachers expressed the same. Qualitative data show that ABC+ helped improve SEL among learners through teacher training, which taught strategies for this. However, teachers’ explanations of SEL integration in surveys and FGDs were mainly about the values inculcated in learners through SRMs. They explained that SRMs helped in learners’ value formation. According to teachers, “Children learned moral lessons and applied these to their life.” Given the cessation of classroom face-to-face instruction during the pandemic, it appears that values in SRMs became the primary source for SEL integration. Future training on SEL can be used to check the depth and accuracy of teachers’ understanding of SEL integration, and actual SEL integration can be monitored and confirmed in face-to-face classes. RELEVANCE: EVALUATION SUB-QUESTIONS Program evaluation on relevance has two sub-questions: one is about the inclusion and implementation of GESI, and the other is on whether ABC+ considered the capabilities and capacities of recipients in response to the changing context of the COVID-19 pandemic. Sub-Question 1.1 How did ABC+ incorporate GESI analysis? To what extent has ABC+ implementation supported GESI? Surveyed teachers claimed that ABC+ training helped them integrate GESI into their instruction. Annex 13 shows that almost eight out of ten teachers in BARMM stated that ABC+ contributions in their GESI integration were helpful to a large extent. In comparison, nearly six out of ten teachers in Region V and VI evaluated ABC+ initiatives as useful to a great extent. While these data are favorable, teachers from Region V and VI could be further supported in the following years of program implementation so that more teachers in these areas could integrate GESI into their instruction. At the individual level, ABC+ incorporated and promoted GESI through training that developed teachers’ and HLPs’ awareness of the concept. At the institutional level, the contribution of ABC+ GESI training was manifested in GESI integration in the SRMs that teachers and writers developed and in teachers incorporating GESI in their instruction. One informant explained that featuring different tribes, different colors, and skin “was a way of supporting GESI.” Another informant said ABC+ training “made the teachers aware of gender stereotypes or potentially discriminating ideas in pictures and words in books given to learners.” Informants also explained that the provision of materials and activities that benefited struggling and special learners provided a means of living up to the principles of social inclusion. Another informant expressed that their improved awareness of gender stereotypes has also influenced how they assign tasks to learners. One informant said, “girls are not the only ones asked to sweep the floor. Even boys are asked to sweep the floor.” Sub-Question 1.2 Did ABC+’s design and implementation consider the available capability and capacities of the recipients (individuals and organizations) and responses to any change in context (e.g., public health emergency such as COVID-19)? The most significant change in context that the ABC+ program had to adapt to was the COVID-19 pandemic. Indeed, ABC+ made several adjustments in consideration of the capabilities and capacities 29 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV of recipients in the face of COVID-19. All adjustments were due to the prohibited or restricted face-to-face interactions, which were most strict in early 2020. Though program activities had to be conducted remotely, ABC+ continued its activities by adapting to restrictions on physical interactions. The instructional and technological support provided by ABC+ aligns with World Bank’s RAPID framework for learning recovery and DepEd’s Basic Education Report 2023, particularly the advice to increase the efficiency of instruction by supporting teachers through upskilling in technology used for remote learning. Key project pivots included the following: • Because there was no face-to-face training and physical contact at the onset of the pandemic, ABC+ provided pre-training orientation on the technology used to prepare partners for online training platforms. • ABC+ also provided communication and Wi-Fi (internet) support or allowance to beneficiaries to ensure they could sustain online training participation. • ABC+ reached out to recipients, monitored their progress, and sought to identify concerns through online means (virtual chat groups, emails). • ABC+ utilized an adaptive training schedule that accounted for teachers’ busier schedules and other pandemic-related concerns. Training activities were made more flexible because they could be completed online or offline. • ABC+ provided modules and materials for the training to give teachers more time to prepare. • ABC+ also provided a longer time to finish training outputs. • Eventually, some areas accommodated limited face-to-face training, allowing ABC+ to conduct onsite training only with smaller groups of trainees and with adjustments to fit into the “new normal” setup. Health protocols were followed strictly during training, and onsite COVID-19 testing was conducted to protect the health of participants and staff. • Because a limited number of people were available to help with materials distribution, and due to limited transportation resources, ABC+ collaborated with the community, like the barangay and the police, for systematic distribution of materials from schools to the houses of learners to ensure the continuity of learning activities. KEY MESSAGE ON RELEVANCE The emerging key message on relevance is that ABC+ teacher training, materials development, and community collaboration efforts played a crucial role in capacitating schools to ensure learning continuity during the pandemic. With ABC+ support, partner schools' learning continuity and recovery efforts would have been unrestricted and less than optimal in terms of timeliness, efficiency, and quality. CONCLUSIONS ON RELEVANCE 1. ABC+ provided a complete and streamlined package of activities spanning teacher training, materials development, and system capacitation that was highly relevant to DepEd policies and programs, particularly on learning continuity and learning recovery. While other programs typically target only one goal, ABC+ recognized the value of the relationship among training, materials, and the overall system and worked on all three seamlessly and USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 30 simultaneously. The efficient streamlining is seen in the complementary design and activities for teacher training, materials development, and system capacitation. 2. ABC+’s highest level of support went toward literacy instruction. This affirms the achievement of the program’s primary goal, which is to improve children’s literacy skills. Initiatives for the three IRs benefited the improvement of children’s literacy skills. 3. ABC+ initiatives for developing numeracy skills have just started and have yet to mature. Learners’ mathematics skills are expected to improve once initiatives are implemented more fully. 4. SEL integration was more evident in developed materials than in instruction, possibly due to the absence of face-to-face instruction. Now that learners and teachers are back in their physical classrooms, the effects of initiatives on SEL integration in teaching can be monitored and enhanced. RECOMMENDATIONS ON RELEVANCE 1. ABC+ and DepEd could help regions give ideas to enhance or contextualize national or regional policies that ABC+ already supports. Data revealed that due to partner schools’ confidence in ABC+ support, some regional policies that involved only literacy had been expanded by partners to include numeracy. Some policies were also enhanced in terms of budget allocation. Processes for initiating these local policy enhancements can be shared with other regions through fora or meetings so that other regions can also identify local policies that can be enhanced or further contextualized. 2. ABC+ may assist schools in disseminating best practices, especially in implementing recovery programs concentrating on literacy development. Findings show that some partner schools in Region V have developed implementation efficiency and achieved targeted literacy outcomes for their learning recovery programs. Schools can be supported in creating platforms for disseminating or sharing good practices, such as school visits for benchmarking or an online or F2F forum or presentation. 3. Because numeracy instruction is designed to be a pilot model that will support the plan for developing a National Framework on Numeracy, ABC+ may strategically develop and design a supplementary framework that targets numeracy instruction in the key stages. This framework may emerge from the lesson development and implementation with UP-NISMED as their partner institution. 4. ABC+ and DepEd regional office could bolster efforts for SEL integration, especially in the instruction of Region V and VI teachers. Generally, findings showed teachers’ explanation of SEL integration was limited to values development via SRMs. This may have been due to the absence of opportunities to implement SEL integration in F2F classes. With the resumption of F2F classes, teachers’ accuracy of understanding SEL integration can be checked in their class instruction, and appropriate training can be offered based on class observation data. While Regions V and VI data show that the extent of the help of ABC+ training for SEL integration was achieved to a great extent, these results (57.8 percent for Region V and 59.5 percent for Region VI) are much lower compared to BARMM’s (72.9 percent). F2F class observations can help determine the training support needed to help teachers in these regions increase their level of SEL integration. 31 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV EFFECTIVENESS: EVALUATION QUESTION 1 To what extent did ABC+ achieve its outputs and outcomes in the three IRs on improved early-grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction? Answering this question required assessing whether ABC+ produced the intended outputs and outcomes. The main unit of analysis measures achievement, underachievement, or overachievement in the three IRs. IR 1: IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS Table 2, which depicts ABC+’s sub-IRs for IR1, shows that only six out of thirteen key performance indicators (KPIs) were completed under IR 1.1. These were mostly on capacity building of teachers, school heads, and faculty from teacher education institutions. There is an overachievement for one KPI for IR1.2. The overachievements in some of the KPIs in IR.11 and one KPI in IR1.2 were connected to pivots to virtual training, which reached more than the expected number of teachers and school heads. Many teachers were reached and the training approach during the pandemic was effective, as reflected in Annex 13-Table 21, where 94.2 percent of the participant ratings indicated that ABC+ contributed to their instructional practices. However, the effectiveness of ABC+’s professional development programs cannot be measured because classroom observations to assess teachers’ application of skills have not started yet. Thus, there is no data to confirm whether the intended outcome of teachers delivering quality instruction is consistent with the training and materials received. If classroom observations are not happening soon, teachers might forget what was taught to them. The time component of teaching, where learners and students interact, is essential to achieve the learning outcomes of improved early-grade instruction. The outcomes for sub-IRs 1.3 and 1.4 (related to math instruction and differentiated instruction, respectively) will be needed until year three of the program to mature. However, preparatory activities such as pilot and evaluation of innovations in mathematics are already achieved but still need to be adapted for broader application. The numeracy support program was only recently implemented, and results to date are moderate. According to the course leader on numeracy, several teacher training courses have been conducted and, in Program Year 3 (PY3), there will be a collaborative development of numeracy lessons and implementation for students or teachers. Although it is a pilot model for numeracy instruction, it is hoped that their effort will gain traction as the program matures. TABLE 2. EXTENT OF ACHIEVEMENT IN IR1 SUB-IRS EXTENT OF ACHIEVEMENT Sub-IR 1.1: Early-grade instructional practices for mother tongue, math, and SEL improved Overachieved (six KPIs) Sub-IR 1.2: School heads and district and division supervisors’ capacity to support and monitor instruction improved Overachieved Sub-IR 1.3: Innovations in math instruction piloted, evaluated, and, as appropriate, adapted for broader application Progressing USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 32 IR 2: IMPROVED ACCESS TO QUALITY MATERIALS There is overachievement in three out of the four sub-IRs of IR2. IR2 has 75 percent progress and the most significant number of outcomes achieved (Table 3). Sub-IR 2.1. refers to improved capacity to develop leveled materials (i.e., reading materials specially designed for different reading skills or grade levels), and sub-IR 2.3 refers to the ability to ensure that the provision of learning materials is continued. The data for sub-IR 2.1 from the program’s Qualitative Data Report on Achievements shows that the distribution of produced leveled readers for primary and pre-primary grade learners has exceeded targets. For primary learners, the report states that as of Quarter 4 of Year 3 of program implementation, 496.5 percent of the learners targeted have already received decodable, leveled SRMs against a target of 95 percent. The percentage of pre-primary grade learners that received leveled readers has also surpassed the mark of 95 percent as early as Year 2 of program implementation. The percentage of primary grade learners that received leveled readers continues to rise until Year 3 of implementation. Based on these data and the quick rate of materials production, sub-IR 2.3 has also been overachieved, as implementers have ensured the ongoing provision of materials and have exceeded targets. Sub-IR 2.2 pertains to developing materials that support SEL integration. While reports indicate that leveled readers have been developed and distributed to children, no available data specifically refer to SEL integration in these materials. However, the implementing partner confirmed that materials development always considered SEL integration, which was part of the process. This means that each material they produce is designed to promote SEL integration. The increased availability of future data may enable a better evaluation of the project's performance in terms of SEL integration and effectiveness. Finally, sub-IR 2.4 pertains to the improved functionality and usability of the learning resource portal. The program set a target of 20 percent of teachers meeting expectations for this result. By Quarter 10 Ibid. Sub-IR 1.4: Innovations to support differentiated instruction piloted, evaluated, and, as appropriate, adapted for broader application Progressing TABLE 3. EXTENT OF ACHIEVEMENT IN IR2 SUB-IRS EXTENT OF ACHIEVEMENT Sub-IR 2.1: Capacity to develop leveled, mother tongue supplementary10 reading materials improved Overachieved (six KPIs) Sub-IR 2.2: Capacity to develop and provide materials supportive of students’ SEL improved Overachieved Sub-IR 2.3: Capacity to ensure the ongoing provision of learning resources for grades K– 3 developed Progressing Sub-IR 2.4: Functionality and usability of the learning resources portal improved Progressing 33 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV 4 of Year 3 implementation, this target has been over-achieved, with 27.5 percent of teachers being registered and meeting expectations. IR 3: IMPROVED EDUCATION SYSTEM CAPACITY AND COMMITMENT TO MANAGE AND IMPLEMENT INTERVENTIONS SUPPORTING EFFECTIVE EARLY-GRADE INSTRUCTION Table 4 shows that against a performance target of 36, ABC+ supported 38 parents-teachers associations or community-based governance structures exceeding the level of expected accomplishment in IR 3.5. Specifically, it shows significant progress (82 percent) in the number (294 out of 360) of parents or community members trained to support children’s education. While sub-IR 3.1 is progressing and 3.2 is overachieved, sub-IRs 3.3 and 3.4 are still underachieved, at 40 percent and 22 percent, respectively. These sub-IRs refer to the number of LGUs with targeted allocations to support learning in grades K-3 and the number of private sector partners supporting evidence￾based EGR interventions. One possible explanation for the underperformance could be the diversion of funding and other resources to respond to community health crises brought about by COVID-19. EFFECTIVENESS: EVALUATION SUB-QUESTIONS Program evaluation on effectiveness has three sub-questions. The results for two questions are presented here: one investigates which IRs contributed the most to the learning outcomes and the other looks at unintended outcomes brought by COVID-19. Sub-Question 2.1 TABLE 4. EXTENT OF ACHIEVEMENT IN IR3 SUB-IRS EXTENT OF ACHIEVEMENT Sub-IR 3.1: Policies to support MT instruction and provision of learning resources are effectively implemented Progressing Sub-IR 3.2: DepEd capacity and commitment to using data to inform decision-making for budgeting, targeting resources, communicating and monitoring expectations, and implementing improvements in early-grade instruction reinforced Overachieved Sub-IR 3.3: LGUs and local stakeholders mobilized to support and sustain ABC+- introduced improvements in early-grade instruction Underachieved Sub-IR 3.4: Private sector engagement increased in support of ABC+-introduced improvements in early-grade instruction Underachieved Sub-IR 3.5: Parent and community involvement in supporting early-grade learning increased Achieved USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 34 Which of the three IRs contributed the most to improving EGR, math, and social and emotional skills for children in the early grades (kindergarten to Grade 3)? What were the explanatory factors, if any, for achievement, overachievement, and underachievement or non-achievement? IR2 (provision of learning materials and resources) contributed the most to the access of quality materials to the EGR and SEL skills of children in grades K–3. IR2 has an estimated 75 percent progress achievement from the KPIs as it has the most significant outputs and outcomes achieved during the ABC+ implementation. Testimonials from interviews with key informants and FGDs with participants from varied backgrounds indicate 43 percent progress against its target. Accessible and quality materials as the output were effective in supporting the beginning reading skills of K–3 learners during remote learning. This ensured more opportunities to practice reading various reading titles. SRMs were produced in two languages: the children's mother tongue and the Filipino language. Home learning partners and teachers whose mother tongue differs from the one used in the reading materials can still teach the learners because of its Filipino translations. Qualitative data from KIIs also showed that learners benefited the most in this program (n = 55, 100 percent), which is connected to SRMs under IR2. An analysis of harvested outcomes also validates this. Results show that most outcomes about learners’ improved reading skills correspond to ABC+ interventions (Annex 7). For example, ABC+ activities such as the production of learning guides and SRMs, delivery of materials to schools and HLPs, books received by schools and HLPS, school heads training teachers on the development of SRMs, teachers trained how to conduct a proper read￾aloud lesson, and most importantly the learning recovery plan contributed to the improved learning outcomes of K-3 learners. Specifically, the delay in reading abilities of K-3 learners was accelerated in the LRP of ABC+ and host regions. This is very evident in OH data of one teacher where learners were already observed reading out loud in the fifth week of the program instead of on its eighth week: “Nakikita na ang progress sa mga bata kahit five weeks pa lang. Kasi ito ay total ng eight weeks. Iyong nakakarecover, nakakapagbasa na. Nakakabasa na iyong umulit ng Grade 2 modular instruction.” (Even if we are only in the fifth week of the program, we can now observe the progress of learners. This [learning recovery plan] runs for eight weeks. Learners recover and can read texts. Those who were part of the modular instruction in Grade 2 and were not promoted to Grade 3 can now read in the classroom). However, some challenges were observed. For instance, not all schools have sufficient library space to store some SRMs for classroom use, and some schools lack libraries. Many teachers constructed dedicated reading and numeracy corners in their classrooms. Classrooms, including libraries, are often prone to typhoons, floods, landslides, and fires. Given the limited capacity of teachers, and school personnel, and most especially, limited Maintenance and Other Operating Expenses from DepEd, every school faces challenges in protecting and providing space to store SRMs properly. Despite the challenges posed by the pandemic, the ABC + program remained effective in improving early-grade literacy and achieved its target outcomes. Ability to adapt quickly. The program was highly responsive to the needs during implementation and adjusted its initial intervention plan to provide appropriate and relevant activities during the pandemic. These strategies were effective and useful to the target stakeholders, making the teachers’ “presence” felt in the learners' homes. The Regional Director of DepEd Region V said, “Strategies formulated by ABC+ helped the region bridge the learning gap during the pandemic. The strategies such as blended learning, use of technology, and home learning guide for home learning platform were successfully applied amid the challenging situation”. 35 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV ABC+ materials enhanced the reading of learners, including their siblings. Initially, the goal was to produce and distribute 8,608,310 materials, but to date, 11,634,849 EGR materials have been made and distributed to K–3 learners. The principal of Montes II Elementary School of Iloilo City Division stated in her interview: “ABC+ materials are very timely during the pandemic, especially that we have distributed it to our children that helped and enhanced them in their readings not only to our learners but also to their brothers and sisters at home.” In the FGDs, informants explained that ABC+’s SRMs contributed to literacy because they are in the local language, which parents could understand and explain to their children. The materials were also developed according to grade levels and were aligned with DepEd’s Most Essential Learning Competencies. Teachers reported that materials “strengthened children’s reading comprehension, widened their vocabulary, and developed their fluency.” ABC+ also strives to complement the government’s policy direction and, to this end, complements program implementation with efforts to advocate for the program to LGUs/DepEd to generate ownership among stakeholders. At the Results Validation Workshop with DepEd, one informant said that the produced SRMs in the local language were very helpful in developing children’s reading skills that their school has decided that they will continue to use the SRMs even if DepEd had ordered the discontinuation of the teaching of the mother tongue as a language subject. Facilitative and collaborative. ABC+ strengthened the collaboration between home and school by engaging HLPs and community members to partake in the program. ABC+ also gained the DepEd’s trust in the process of program implementation. According to the DepEd Region VI Education Program Specialist, MTB- MLE and Filipino Coordinator-Curriculum and Learning Management Division Regional Office, “ABC+ is facilitative and collaborative. They provided budget support and training and also selected the right people to partner with us. Hence, despite the pandemic, ABC+ still achieved its outcomes. ABC+ has gained the full support of the leaders in the regions.” Continued/sustained monitoring during the pandemic. Consistent program monitoring at all levels ensured that activities were on track and the program delivered its outputs. While in the middle of the pandemic, the program shifted to utilizing online platforms to achieve its objectives. “Even during the pandemic, ABC+ is still highly effective because of the dedication and hard work of the project staff of ABC+. They were persistent in following up/monitoring. They won’t stop until work is done,” according to a Roxas City Focal Person. With the return of in-person activities, program effectiveness should continue relatively quickly. The Schools Division Superintendent of Maguindanao II Division thought they should “only rate four out of five on the effectiveness since the implementation is still halfway.” Similarly, according to the Assistant Schools Division Superintendent, Special Geographic Area Division, MBHTE of BARMM, “Strategies must be implemented in a F2F setting. If you want to see the effectiveness of the strategies, it should be done face-to-face. Some strategies require a teacher knowledgeable on how to conduct the activity.” Sub-Question 2.1 Have there been any outcomes brought by COVID-19 in implementing ABC+ that would tell us whether the program improved early-grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 36 There are two parts to the section of this sub-question. First are the results of outcomes harvested from evaluation participants, and the second is the selected most significant change story. The OH method defines an outcome as a change in an individual, group, community, organization, or institution’s behavior, relationships, actions, activities, policies, or practices. It could also be an unexpected or unintended demonstrated change in awareness, knowledge, or skills, collaborative effort, or the use of knowledge or innovative solutions. Outcomes might also describe unexpected or unintentional, more profound institutional changes relating to policy, citizen engagement, government accountability, and organizational arrangements. The evaluation team asked key informants and FGD participants to describe what they perceived as the outcomes for each IR and ABC+’s contributions. Since ABC+ is only approaching its fourth year, the OH focused only on the project's processes and how they affected its inputs and outputs. Figure 2 presents the harvested outcomes associated with the three IRs. These outcomes include intended and unintended outcomes expressed by the participants during the OH and MSCT workshops. Figure 2. Verified Harvested Outcomes Per IR Figure 2 shows that the harvested intended outcomes in IR1 are increased teaching efficacy, increased capacity to teach, increased appreciation of teachers’ role in beginning reading, and developed precision in identifying specific learner support in reading. This can be attributed to the various teacher training that ABC+ delivered to the recipient schools across covered regions, cultivating confidence, competencies, and strategies among teachers in teaching literacy to K–3 learners. One of the FGD participants praised the quality of the program’s materials and training: “The materials distributed to the school were beneficial in teaching reading, especially the picture book 37 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV for the kinder. The teachers benefited from the training. They were refreshed with the strategies and techniques in teaching reading.”11 In IR2, the evaluation team confirmed four intended outcomes: learners’ increased love for reading, an adaptation of ABC+ instructional and home learning strategies, teachers’ increased capacity in curriculum development, and cognitively aligned reading materials. This supports ABC+’s provision of contextualized and developmentally appropriate SRMs to K–3 learners (access and use of materials), capacitation of HLPs (training of parents and guardians on how to teach their children at home), and capacitation of teachers (training, workshops, and monitoring). In IR3, the harvested intended outcomes are improved capacity at the school level, parental support in reading, and a whole community approach to beginning reading. The harvested outcomes can be attributed to the expressed support and involvement of various stakeholders, awareness, and capacity building of parents and other community members in reading practice, the commitment of multiple stakeholders to monitoring, and the support of the ABC+ program to DepEd policy. Two significant unintended outcomes emerged towards the end of PY3. These are improved criteria for teachers’ competence (IR1) and the learning recovery curriculum (IR3). Improved teacher competence was not brought about by the pandemic but was a positive outcome. The development of MTB-MLE-based instructional materials was added as a criterion for K–3 teachers’ competency for professional promotion. ABC+’s materials workshops helped regional DepEd leadership realize the importance of materials development and the corresponding value of capacitating teachers. Teachers can earn a professional promotion by being involved in developing or producing MTB-MLE instructional materials. A materials development focal person reported this as a welcome recognition of their efforts and an additional motivation to keep producing leveled local SRMs. The strategies at the K–3 level of the learning recovery plan developed and initiated by DepEd Region V were influenced by instructional practices that ABC+ introduced during teacher training 1 and 2. As a partner of DepEd Region V, ABC+ provided critical technical inputs to the region’s learning recovery plan and program, also known as the Recovering for Academic Achievement by Improving Instruction through Sustainable Evidence-based learning (RAISE) program. Part of the RAISE plan was to start the school year with an eight-week learning recovery curriculum, focused solely on directly supporting learning remediation and intervention on literacy and numeracy for Grades 1–3. ABC+ developed the assessment tools and capacity building for teachers using the said tools. These two unintended and significant outcomes ensure that the outcomes outlined in the theory of change will be fully achieved in two years because they are integrated into the host institutions’ policies and are connected to the ABC+ program. BOX 1. MOST SIGNIFICANT CHANGE STORY One of the intended outcomes of ABC+ is the whole community approach (upper portion of IR3). This outcome connects to the emerging most meaningful change story that underwent three selection levels, titled “Nothing is Impossible for the Learning of Children” by a Master Teacher from Region V. The story was chosen because it illustrates the classic adage, “It takes a village to raise a child.” The story cuts across the 11 The program used the 5Ts framework (Teach, Time, Text, Tongue, and Test) as a scientific approach to teaching reading in the early-grades. Details of this approach are further discussed in the value-added discussion at the end of this section. This unique and novel approach still needs to be employed by the host regions before ABC+. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 38 intermediate results (IRs) of ABC+ and captures all its goals. The story was about reading materials and, just as equally, about training and systems ability. It tells the story of a teacher empowered by ABC+ training. The teacher extended her initiative to develop children’s literacy by becoming an advocate and engaging the community to launch a literacy project during the pandemic. According to one story selector, “It is a complete story since it captures several activities…it captures what has been happening on the ground, it captures the series of pivots and innovations, and it is the story of the two years of learning during the pandemic” and, ultimately, presents the fruit of the activity (see Annex 22 for the actual story). Analysis showed that ABC+ influences the story because of the training received by the teacher. Two levels of program recipients were seen, the individual (teacher) and the community. Also, there are two types of changes (outcomes). These are changes in behavior (an initiative of the teacher) and changes in the community’s activity (provision for learning home library). In summary, although ABC+ activities underwent several pivots, most harvested outcomes were aligned with the intended outcomes outlined in the theory of change. And as planned, these contributed to early-grade instruction. Sub-question 2.3 How has ABC+ contributed to enhancing policies on EGR instruction and learning outcomes in kindergarten to Grade 3? ABC+ enhanced three types of policies: 1) strengthening data-driven planning and investment programming through strengthening the assessment systems for Grade 1–3 learners, 2) multi￾stakeholder sectoral planning to improve education financing for early-grade learning, and 3) contextualization of policy. Strengthening data-driven planning and investment programming through strengthening the assessment systems for Grade 1–3 learners. ABC+’s introduction/co￾development of the Comprehensive Rapid Learning Assessment (CRLA) tools with DepEd Region V, VI and selected Maguindanawan-speaking districts in BARMM paved the way to creating a systematic data dashboard that enabled the teachers to quickly identify learners who are eligible to undergo designed remedial intervention strategies based on their learning needs. Region V has begun incorporating CRLA results as one of the key performance indicators in their quarterly Monitoring and Evaluation sessions. In Region VI, CRLA was instrumental in profiling learners and data-driven interventions. According to Region VI Focal Person, non-readers in Filipino and English languages decreased - “Profiling of learners to make materials and training for teachers suited to students were created from the CRLA. Non-Filipino readers dropped from 34.3 percent to 11 percent, and non-English readers dropped from 14.8 percent to 6.6 percent.” Schools then continued bridging the learning gaps by maximizing CRLA results in formulating plans and activities even after concluding the 8-week DepEd Regions V and VI curriculum. status of K BOX 2. BENEFITS OF CRLA CRLA has proven effective in reporting class, school, and regional data on the status of K-3 reading skills. It allows educators to identify children’s reading levels and inform instructional decisions and interventions quickly and easily. One of the key benefits of CRLA is its effectiveness in reporting data at various levels. For example, a teacher can determine her students’ reading levels at the beginning of the school year. This data can then be aggregated to provide a picture of the reading skills of the entire class. Similarly, data from multiple classes can be combined to provide an overview of the reading skills of an entire school. This data can then inform decisions about curriculum, instructional strategies, and interventions. Data are also collected mid and the end of the school year. This provides information on the progress of reading skills of K-3 learners based on the visualized data. Another benefit of CRLA is the ease with which data can be visualized. It collects and analyzes data, displaying the results in an easy-to-understand format. This makes it easy for educators, administrators, and policymakers to quickly identify trends and patterns in the data at division and regional levels. The most significant benefit of CRLA is its scalability. It is technology-based and uses Looker Studio, which can be availed for free via a web browser (Google Chrome). Hands-on training related to its use is easy to learn by teachers, supervisors, and regional coordinators. 39 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV Multi-stakeholder sectoral planning to improve education financing for early-grade learning. ABC+ introduced a model for sub-sector planning called the multi-stakeholder local education sectoral mechanism (ESM). ESM was designed to strengthen the existing policies and practices relating to decentralized planning and financing for early-grade education by LGUs and DepEd regions and division offices. ABC+ introduced a structured, standardized process and tools/rubrics for determining the gaps along the three areas of access, quality, and governance and facilitated the prioritization of programs, projects, and activities (PPAs). Some of these priority PPAs were considered in the LGUs’ Executive-Legislative Agenda and School Education Fund, thus contributing to improving learning outcomes and strengthening the link between DepEd and LGUs. These DepEd and LGU partners include those in Tabaco City, Legazpi City, Negros Occidental, Antique, Capiz, and Sorsogon. Contextualization of policy. Regions contextualized the National Policy on Bawat Bata Bumabasa (3 Bs) by localizing the 3Bs in their policy/program. To date: • Region V spearheaded the Bawat Batang Bikolano Bihasang Bumasa (5Bs) • Region VI has its Pagbabasa ng may Unawa Nakatutulong Labanan ang Agam-agam (PUNLA) • BARMM has a Bawat Batang Bangsamoro Bumabasa at Bumibilang (5Bs) program. The National Memorandum prompted the contextualization of policies that established a standard school-based reading program that responds to improving learners’ reading challenges. Region 5 ABC+ Focal Person explained: “In Region V, we created 5Bs or the Bawat Batang Bikolano Bihasang Bumasa (Every Bikolano Child Reads Very Well), before the pandemic, to respond to 3Bs with a focus to establish standards for schools for implementation of school-based programs in implementing the reading program.” ABC+ uses a whole community approach to the early-grade instruction ecosystem. This approach added three micro-ecosystems: teacher education institutions, local government units, and home learning partners. In this ecosystem, the roles of this micro-ecosystem became explicit in ABC+. For example, in teacher education institutions, the courses offered did not have existing modules for their pre-service teachers. ABC+ helped the faculty in charge of courses write and produce the Beginning Reading and Literacy Development modules. The home learning partners were guided on how to use the home learning modules for K–3 learners. A case study in Albay Central School describes this whole community approach. Additionally, ABC+ used core elements—the 5Ts of Teach, Time, Text, Tongue, and Test—found to improve early-grade literacy instruction and learner outcomes. These provide structure to this reading intervention and ensure the approach is inclusive. During the complex pandemic context, ABC+ effectively used 3Ts: Text (SRMs), Tongue (mother tongue-based materials), and Test, which supported incremental progress for K–3 learners. For teachers and school heads, the training component, which is the teaching part of the model, deepened pedagogical knowledge and increased teaching efficacy about MT as a separate subject. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 40 Classroom observations on how teachers apply skills from the training to help learners read have yet to be possible due to the pandemic and remain an urgent concern for the full achievement of the program goal. The last T – Test - has a significant contribution as well but must be fully understood by the entire system using the suggested approach of the evaluation team - the use of 3Ds (Data, Discourse, and Design). Data should be continuously generated via tests, discourses on the implication of the data should happen at all levels, from regional to classroom levels, and design how resources (e.g., human, financial) may be used to improve EGR. BOX 2. ABC+ PROGRAM IN ALBAY CENTRAL SCHOOL (REGION V) In 2019, ABC+ was introduced to Region V and Region VI and selected the Maguindanawan-speaking district in BARMM. The program, primarily in partnership with the DepEd, aims to improve basic literacy skills in the early grades (K–3), focusing on three IRs: IR 1: Improve early-grade instruction and delivery systems, IR 2: Increase access to quality, leveled, locally developed supplementary reading materials to support early literacy, and IR 3: Support the enabling environment by improving education system capacity and commitment to manage and implement interventions supporting effective early￾grade instruction. The alignment of the ABC+ program to the DepEd policies proved mutually reinforcing, providing credibility to the various program activities, which optimized the implementation of the existing DepEd policies on literacy, specifically in the native language. The national memorandum on 3Bs, Bawat Bata Bumabasa, was contextualized by Region V into 5Bs (Bawat Batang Bikolano Bihasang Bumasa) along with the implementation of the eight-week learning recovery program to address learning losses. As soon as Region V adopted the ABC+ program, many schools immediately implemented and supported it, including Albay Central School (ACS), one of the pioneer schools under the Legazpi City Division. ACS is currently notable for its exemplary performance across the three IRs. The Training of Trainers (TOT) and teacher training designed by ABC+, DepEd, and the National Educators’ Academy of the Philippines strengthen teaching skills and strategies on how to teach the mother tongue to early grades. The need for teacher training emanated from research-based data and assessments. Teachers diligently adopted the teaching strategies and teaching processes, including integrating gender equality and social inclusion, and social-emotional learning. During the COVID-19 pandemic, they learned how to utilize and navigate different platforms and made the teaching-learning experience more fun and effective amidst the pandemic. The skilled trainers or the TOT inspired and motivated teachers to perform and show their commitment to improving their teaching. In addition, the schoolteachers were also trained to utilize their dashboard to conduct systematic assessments using the Comprehensive Rapid Literacy Assessment, which then served as their basis for data-driven interventions. ABC+ provided supplementary reading materials (SRMs) to support early-grade literacy (i.e., wordless, decodable, and leveled readers in English, Filipino, and MT small and big books) for K–3 learners. With the support of the barangay and volunteer parents, ACS was able to collaborate and orient the six catchment barangays to the program and distribute SRMs. While one barangay among the six catchment areas could put up a community learning hub, others established reading centers, which benefited not only the learners but also illiterate community members. “The reading materials provided by ABC+ enhanced their interest in reading, which made it easier for the kids to understand the lessons,” a parent explained. Some teachers and parents encountered difficulties translating mother tongue words due to multiple variations of the Bikol language. Some K–3 teachers lacked confidence due to their infrequent use of Filipino and English. A supervisor-teacher￾parent support system was developed through the virtual platform to support them. School coordinators became more responsive to the needs of the teachers and students in implementing Mother Tongue-Based Multilingual Education. The intensified monitoring was also vital in ensuring that activities were on track and facilitating proper rollout to the target partners/participants. ABC+ and the ACS principal and teachers engaged the community, including home learning partners, municipal/barangay LGUs, and non-government and government organizations, to improve early-grade literacy despite the school closure. Education partners were able to augment educational resources during the implementation of the program by reproducing and distributing SRMs. In addition, the LGU of Legazpi City leveraged support through the School Education Fund. With the help of ABC+ and the collaboration with DepEd and other stakeholders, the whole community approach successfully promoted early-grade literacy. ABC+’s support system played a significant role in improvements in early-grade literacy. The region regarded the ABC+ program as highly responsive to the educational challenges brought by the pandemic, and DepEd Region V was, therefore, very supportive and committed to sustaining what ABC+ started. KEY MESSAGE ON EFFECTIVENESS Following the evaluation criteria on effectiveness, the emerging key message in this evaluation parameter is that even in the context of the pandemic, ABC+ used an agile program model, which allowed for positive results. It quickly adjusted to ensure that intended outputs and outcomes will be 41 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV achieved. Although some activities were delivered in an alternative format, there is still fidelity to the program design. Outcomes of ABC+ are starting to surface incrementally, especially since face-to￾face classes are now in full swing. CONCLUSIONS ON EFFECTIVENESS ABC+ activities on professional development for teachers, school heads, and supervisors, production, and access of quality SRMs that can be publicly shared, and technical assistance on the learning recovery plan were instrumental in catching up with the reading delays of K-3 learners brought about by the pandemic. The ABC+ efforts on the DepEd system’s capacity to strengthen the use of assessment data are very helpful in monitoring children’s reading outcomes. • From the qualitative findings and secondary quantitative data from participant feedback, it is clear that the ABC+’s provision of contextualized, localized, age and developmentally appropriate SRMs; capacitation of home learning partners on how to teach their children at home; and teachers’ capacitation on designing and implementing early-grade instruction using the science of reading, workshops, and monitoring together are helping the learners to read in their mother tongue and even in the Filipino language. • From the harvested outcomes, it can be concluded that the unintended outcome brought by COVID-19 is the learning recovery plan. This plan contributed to improved instruction and access to quality materials. Regional focal persons, supervisors, school heads, teachers, and MTB-MLE coordinators collaborated on an eight-week recovery plan to ensure that K-3 students could catch up and recover their learning losses in reading skills. This catch-up strategy was supported technically by ABC+. • The evaluation team concludes that ABC+’s contribution to enhancing policies was observed most in curriculum development and assessment at the K–3 levels. For the curriculum development, this was evident in the budgetary support of ABC+ in the production of SRMs and the learning recovery plan for students to catch up on their reading losses brought on by the pandemic. For the assessment, the CRLA was very helpful in monitoring children’s reading outcomes and how they can be addressed. Data dashboards showing children who can read were effective in the host regions ensuring that technical support was given to schools in need. Further, the institutionalization of these policies was observed to be a top￾down approach, where the regional office provides the implementing rules applied to the classroom level. Specifically, Region V has begun incorporating CRLA results as one of their key performance indicators in their quarterly Regional and Division Monitoring Evaluation and Adjustment sessions. RECOMMENDATIONS ON EFFECTIVENESS On the effectiveness of capacity development for teachers, school heads, and supervisors and the effect of SRMs on improving reading of learners in their mother tongue, Filipino, and even in the English language, the following recommendations are addressed to ABC+, DepEd-Regional and Division Office, National Educators Academy of the Philippines (NEAP), and the LGU: 1. ABC+ could: • gather real-time data on teachers’ abilities to implement the 5Ts of teaching to understand better actual teaching and learning practices in K–3 instruction. This is better done via classroom observations in the three regions; and USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 42 • add training and technical assistance on research in instructional differentiation and focus on the transition of ownership of the program to host institutions and organizations. For the first recommendation, a research write-shop where K–3 teachers are taught how to design and implement research may increase teaching reading efficacy and, at the same time, provide classroom research evidence on the effectiveness of ABC+. For the second recommendation, efforts should be made to institutionalize improved literacy training for teachers and, at the same time, encourage host institutions to commit to the continuous production of quality SRMs. 2. DepEd-Regional and Division Offices could provide financial and technical assistance for teachers to continue applying what they learned from the teacher training. This, in turn, will serve as a refresher approach, as most of the training was conducted during the height of the pandemic. Locally trained ABC+ facilitators may include this in their monitoring and evaluation activities in their regions and divisions. 3. The NEAP could continue exploring a training design and schedule to support and cater to teachers’ needs, situations, and well-being. While several pivots have been made from the original training design to cater to the evolving circumstances of teachers during the pandemic, participants relayed some additional suggestions in training design and schedule. They expressed the need to moderate the number of training tasks/assignments so that they can rest at night. Training also must consider that training participants regularly do other jobs as teachers. Finally, while online data allowance was provided, some teachers had to travel to a more urbanized part of their area to access stable Wi-Fi connections, incurring added cost of transport. 4. Since SRMs from IR2 contributed the most to learning outcomes, space should be allocated to store them in safer places. Collaborating with provincial, municipal, and even barangay offices and officials is imperative to address challenges in keeping quality SRMs. School heads and teachers could seek help and request a library room or building to store all the SRMs safely. Some schools do not have printers or photocopying machines to reproduce SRMs in case schools need extra or additional copies. Schools that lack printers or copy machines to produce SRMs should be given a more significant budget by their LGU to ensure that copies of SRMs are readily available; this would be particularly important for schools with no access to the internet. Finally, the private sector and LGUs could help transport SRMs for remote storage. On the contribution of the learning recovery plan as the unintended outcome: 1. ABC+ assists schools in disseminating best practices, especially in implementing recovery programs concentrating on literacy development. Findings show that some partner schools in Region V have developed implementation efficiency and achieved targeted literacy outcomes for their learning recovery programs. Schools can be supported to create platforms for sharing good practices, such as school visits for benchmarking or an online/F2F forum or presentation; and 2. Because numeracy instruction is designed to be a pilot model that will support the plan for developing a National Framework on Numeracy, ABC+ can strategically use this as an opportunity to develop and design a supplementary framework that targets numeracy 43 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV instruction in the key stages. This framework may emerge from the lesson development and implementation with UP-NISMED as their partner institution. On enhanced policies on curriculum development and assessment at the K–3 levels, the evaluation team recommends that ABC+ help regions disseminate ideas to strengthen or contextualize national or regional policies that ABC+ already supports. Data revealed that due to partner schools’ confidence in ABC+ support, some regional policies that involved only literacy had been expanded by partners to include numeracy. Some policies were also enhanced in terms of budget allocation. Processes for initiating these local policy enhancements can be shared with other regions through fora or meetings so that other regions can also identify local policies that can be enhanced or further contextualized. SUSTAINABILITY: EVALUATION QUESTION Will the achievements in improved early-grade reading, math, and social and emotional skills for children in early grades persist after the project’s conclusion? To gauge if the achievements in improved early-grade reading, math, and social and emotional skills for children will persist beyond the project, the sustainability milestone of ABC+ was reviewed and analyzed based on the findings and core principles of sustainability, namely: human, economic, social, and environment. Midline Sustainability Analysis. The Years 1–5 plans were revisited, including the IRs and KPIs,12 to decide if the set goals were achieved and, if not, what alternative plans of action were made to achieve the goals (unexpected outcomes). A plan of action (i.e., local strategies and activities spearheaded by DepEd and ABC+ in collaboration with stakeholders) was undertaken in the context of the COVID-19 pandemic. Indeed, the pandemic was a great challenge to ABC+ and DepEd to ascertain the program's sustainability from the program’s start-up to midline evaluation because ABC+ has a limited timeline of five years. 12 Research Triable Institute, ABC+ Monitoring, Evaluation, and Learning Plan, 2019 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 44 Figure 3. Early-grade Ecosystem Toward Resilience and Sustainability Figure 3 shows the early-grade ecosystem toward resilience and sustainability during the implementation of the ABC+ program amidst the pandemic. ABC+ and DepEd collaboration and partnership with agencies and community to promote and achieve target early-grade literacy is about core principles of sustainability (i.e., human, social, economic, and environmental). In the human aspect of sustainability, the project supplied technical assistance for teachers through focal persons (the proper communication and monitoring of expectations from division to school) and training of trainers (technical assistance from the regional level to division and district levels). The continuous implementation of physical and online learning to learn about and develop adaptive teaching strategies and classroom assessments also contributed to sustainability. Furthermore, the pre-service training preparation given by the identified TEIs in their respective region continuously contextualized the development of teaching and learning materials based on context. Training pre￾service teachers is a succession model for DepEd in-service teachers to guarantee many well-trained in-service teachers. The social aspect of sustainability is manifested through partnerships with HLPs and community stakeholders (i.e., B/LGU, NGOs, the private sector, etc.). ABC+ trained parents and community members/volunteers from barangays who will serve as a trainer of trainers to promote and use Parents and Community Session Guides to build the capacity of the parents and community members (i.e., barangay-based organizations of youth, the elderly, women, parents volunteers, and barangay officials and volunteers). In addition, ABC+ partnered with the Department of the Interior and Local Government Academy (DILG-LGA) to develop a model for Early-grade Literacy education governance to capacitate local communities. The economic aspect of sustainability is seen in the continuous production, printing, and usability of quality-assured SRMs and the accessibility of platforms such as TV, radio, and social media. Inventory of equipment for all the schools managed by the LGU will provide helpful information to decide which schools require an additional budget for the production and supply of SRMs to ensure that the reading practices of K-3 learners are well supported. ABC+, DepEd, and NEAP utilized cost-effective strategies: 1) the use of CRLA to develop Learning Recovery programs and Literacy Developments programs in three regions (Regions V and VI and BARMM); 2) the migration of modules from the Teacher Professional Development programs to DepEd Professional Development and Learning Management System, making them more accessible to many; 3) continuous development of SRMs that have been stored to the Learning Portal for future utilization; and 4) the use of Learn Kernel13 as another intervention for ABC + Beginning Reading Program. The environmental aspect of sustainability refers to creating safe learning spaces like community reading centers, learning hubs, and reading and numeracy corners inside the classroom. ABC+ supported DepEd to establish and gain support from the local community and mobilized them to develop learning hubs and community reading centers/learning centers to create a good learning environment. Inclusivity and building resilience also contribute to sustainability, mainly by protecting the learning resources developed. It is also important to acknowledge the role of all stakeholders, 13 A Learn Kernel is a rechargeable, tablet-like device that functions even without internet access, which many educators believe is ideal for use in remote and rural communities. 45 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV from top agencies down to grassroots levels, given the numerous challenges such as environmental disaster risks (climate change, typhoons, flooding, landslides, earthquakes, and volcanic eruptions), as well as geographic and political-economic factors (far and remote school, limited livelihood, malnutrition among learners). Based on the evaluation data, the team found sustainability mechanisms that will likely support improved EGR and literacy beyond the program’s life. See Table 6. Under IR1, collaboration with NEAP is an excellent mechanism to adapt the ABC+ program continuously. Many teachers appreciated ABC+ training, which helped them improve their literacy instruction. Informants explained, “The Professional Development Information Technical Assistance provided for NEAP is a good start, and DepEd is adopting the system.” Another informant stated, “The training is NEAP-certified and Philippine Regulatory Commission-recognized. One of the directions is to build up the capacities of the division trainers of Negros Occidental and other Schools Division Offices so that they will be given NEAP-Regional recognition as Regional Learning Facilitators on the ABC+ training program and other programs intended for K-3 teachers.” Based on these statements, ABC+, in collaboration with NEAP, showed significant improvement in teacher professional development. At the same time, ABC+ focal persons, teachers, and informants saw the long-term importance of capacitating the teachers through professional development. The instructional approaches and strategies they acquired through training were of immense help to them as a teacher and for improving early-grade literacy. Under IR2, the creative commons as the license attribution, where program recipients may distribute copyrighted SRMs freely, were helpful to many teachers and parents. Teachers could easily access and reproduce the SRMs. Likewise, parents recognized the value of these learning materials. During the KII, one of the informants stated, “In the future, we can sustain the ABC+ development materials,” while another mentioned that “I printed out reading materials and let the students read them; if they were able to read the materials, I gave them stamps depending on the number of words they’ve read.” Under IR3, teachers and school heads proved the remarkable functionality of CRLA in many participating schools from Regions V and VI and selected school districts in BARMM. If this initiative introduced by ABC+ in collaboration with DepEd continues, this would be a positive indication of the program’s sustainability. TABLE 6. ABC+ AREAS OF ACHIEVEMENT AND IDENTIFIED SUSTAINABILITY MECHANISMS INTERMEDIATE RESULTS ACHIEVEMENT OF ABC+ SUSTAINABILITY MECHANISM IR 1 Capacity development for teachers Training by NEAP and commitment of teacher trainers to continuously adapt ABC+ instructional strategies IR 2 Quality-assured SRMs License attribution of the SRMs is under creative commons IR 3 Precision in assessment, formation of sectoral educational management, and robust community support system Use and functionality of CRLA in communicating and monitoring. The Mandanas-Garcia Ruling Parent-Teacher Community Association Involvement USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 46 Given the relevance of the policies and effectiveness of training, teaching, and learning materials, how to continuously finance the program will be one of the biggest challenges after the conclusion of the ABC+ program. DepEd adaptation and mainstreaming of ABC+ interventions would ensure their inclusion in the DepEd and local government budgets and their sustainability. Based on evaluation team findings from FGDs and KIIs, ABC+ and DepEd strengthened the ongoing collaboration with provincial, district, and municipal education committees to gain financial support for training and SRM production and distribution. The education committee from the governor’s and members of Congress’ offices supported ABC+ and DepEd’s efforts to distribute and raise funds for the program. In addition, the local school board offers assistance to schools and learners through the Mandanas-Garcia ruling by giving more funds to improve school facilities and infrastructure. NGOs, alumni, and other stakeholders provided financial support and in-kind donations such as printers, bond papers, photocopying machines, small and big books, and other school supplies and kits. During the school closure brought about by the COVID-19 pandemic, education leaders experienced limited manpower. In response, ABC+ and DepEd worked with the Parent-Teacher Community Association (PTCA) to mobilize volunteer parents, youth organizations, NGOs, and B/LGU to assist schools in reproducing and distributing learning materials. The continuous aid of PTCA, especially during the pandemic, is another robust sustainability mechanism to ensure achievements in improved early-grade reading and SEL continue. Parents, teachers, and community efforts were crucial because they directly interacted with learners. Teachers could guide parents on how to teach their children at home. In regions V and VI, teachers, through FGDs, and school heads, through surveys, expressed their readiness to continue ABC+ activities after the project ends. Teachers articulated that they learned more strategies during the training and recognized the support of the ABC+ program. Furthermore, the survey shows that more than 50 percent of the participants answered, “strongly ready” and “ready.” See Annex 13 for more detail. Identified Successful Elements towards Sustainability. Teachers' and school heads’ responses to the survey aligned with the identified successful elements towards sustainability, such as 1) continuous support and institutionalization on contextualized policies and memos and collaboration with stakeholders (internal and external) through MOAs and MOUs, 2) established functional data￾driven planning and investment programming through strengthening the assessment systems, 3) consistent and continuous development and production of quality assured SRMs, 4) continuous innovation, implementation and monitoring of innovative programs on literacy with GESI and SEL, 5) continuous innovation in the teacher training/professional development and making it meaningful to teachers, and 6) continuous support for teachers to improve learners’ performance, a promising indication that program interventions and achievements might continue after ABC+ closes out. 47 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV Figure 4. Successful Elements Toward Sustainability The stairs in Figure 4 represent the elements of sustainability wherein the first step is considered the first element. This creates chain reactions to achieve sustainability of the ABC+ program. Once the agencies or organizations establish collaborative relationships, the intervention only needs continuous support and institutionalization of relevant contextualized policies and memos. Partnerships with external stakeholders and the private sector through Memorandums of Agreement/Understanding (MOA/MOU) were also great support systems. One of the key informants in Region VI stated, “Our mayor also pledged his support. LSBD (Local School Board Development) funds were used as well. Signed proposal for training of teachers and parents on the proposed remedial instruction, as ideas came from the ABC+ project.” Once policies and memos are in place, other elements will follow, such as: established functional data-driven planning and investment programming through strengthening the assessment systems through license attribution (assessment tools were ready and in place and matched with teaching and learning materials); continuous development and production of quality-assured SRMs; continuous innovation, implementation, and monitoring of literacy programs that integrated GESI and SEL at the onset; continuous innovation in teacher professional development and making this meaningful to teachers; and continuous support for teachers to improve learners’ performance. How did ABC+ strengthen local ownership and leadership on the enhanced instruction and learning outcomes in kindergarten to Grade 3? The peak of the COVID-19 pandemic resulted in school closures across the country. The pandemic hurt the entire education system, more specifically, its delivery. However, despite experiencing various calamities and disasters, many educators were able to innovate through technology and local communities were capacitated to provide additional support. ABC+ and DepEd central office collaborated with various local agencies on how to continue education and work around the barriers brought on by the pandemic. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 48 As shown in Figure 5 below, DepEd regional offices could contextualize policies through memos disseminated to school divisions to innovate programs and adapt them to their context and pandemic restrictions. One of the education leaders stated, “At our level, this provided us guidance that aside from the regular work, we need to include certain activities in the budget to ensure continuity of the interventions.” This informant also noted that they “consider that newly hired teachers and teachers assigned to kindergarten to Grade 3 classes should receive the Teachers Training 1 and 2 as part of their induction program.” Each school was able to implement innovative school activities based on the needs of the learners and parents through the support of the local community. Figure 5. Strategies and Activities that Strengthen Local Leadership and Ownership Toward Sustainability ABC+ assisted and capacitated communities through diverse local activities, as shown in Figure 5. The following activities strengthened local ownership toward sustainability: 1. Establishment of an enhanced support system of stakeholders where the barangay LGU, through the local school board ensures an ample School Education Fund. 2. Partnership with the DILG-LGA to develop modules to capacitate communities for early￾grade learning education governance. 3. Linkages and partnerships with the private sector and NGOs in the form of cash and in-kind donations help improve facilities and infrastructure to support the reading literacy program; 4. Engagement with HLPs, which led to the construction of barangay reading centers and parent volunteers, and guiding volunteer parents in the delivery and distribution of modules during school closures; 5. Conduct regular home visitation and organizing of physical and virtual group study and buddy systems to continue and reinforce the ongoing literacy program; and 6. Increasing teachers’ awareness led to guiding the parents on how to help their children read. Some parents and guardians were encouraged and supported to become teacher/parent 49 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV volunteers to help develop home-based and community-based supplemental learning materials. The reflected and observed strategies and activities concurred with the survey result on the extent to which ABC+ activities involved community stakeholders (i.e., LGU, NGO, etc.) to improve early￾grade instruction. Almost 50 percent of the participants (school heads) responded to this statement with “to a great extent” or “moderate extent.” KEY MESSAGE ON SUSTAINABILITY Based on the collected data and reviewed documents on ABC’s sustainability milestones, the critical indication of sustainability is the resilience built during the peak of the COVID-19 pandemic through the commitment of all education stakeholders, from DepEd to HLPs in the community. This was made possible because of ABC+ initiatives and collaboration that focused on the following: 1) consistent and continuous capacitation of teachers, including monitoring and evaluation using accessible technology; 2) continuous innovations in teaching and learning modalities; and 3) continuous robust support system provided by various stakeholders to sustain ABC+ program interventions even after its project conclusion. Given the limited funds and human resources of several schools, DepEd and ABC+ collaboration with other government agencies, private sector, and community stakeholders is an excellent means to achieve continuity of the designed training program (i.e., pre-service and in-service), reproduction and distribution of SRMs, and improvement of physical infrastructure and facilities. These are future strategies that can be explored. Enhancing literacy achievement is not limited to curriculum and the four corners of the classroom; instead, it goes beyond “schooling.” It is what we call “community,” a living laboratory for and of learners where each member of the whole community plays a significant role. CONCLUSIONS ON SUSTAINABILITY The achievements in improved early-grade reading and social and emotional skills for children in early grades will likely continue after the ABC+ project conclusion because of the identified mechanisms and elements that have been employed toward the achievement of sustainability. The sustainability mechanisms include 1) training by NEAP and commitment of teacher trainers to adapt ABC+ instructional strategies continuously, 2) license attribution of the SRMs is under creative commons, 3) use of CRLA in communicating and monitoring, 4) the Mandanas-Garcia Ruling, and 5) PTCA involvement. The successful elements of sustainability are: 1. continuous support and institutionalization of contextualized policies and memos and collaboration with stakeholders (internal and external) through MOAs and MOUs. 2. established functional data-driven planning and investment programming by strengthening the assessment systems; 3. consistent and continuous development and production of quality-assured SRMs; USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION | 50 4. continuous innovation, implementation, and monitoring of innovative programs on literacy with GESI and SEL. 5. continuous innovation in the teacher training/professional development and making it meaningful to teachers; and 6. continuous support for teachers to improve learners’ performance. The community’s resilience, the adaptation of DepEd, ABC+, and other agencies to deliver education amidst the pandemic, and constructive collaboration through collaboration with TEIs, LGU, NGOs, and other stakeholders in the community also contribute to sustainability. However, there was limited collaboration with the private sector, as indicated in the five-year sustainability milestone of ABC+. At the regional level, ABC+ and DepEd collaborate with TEIs to train pre-service teachers in preparation for their in-service tasks to support and continuously improve early-grade literacies (i.e., reading and numeracy). ABC+ established local ownership and leadership, resulting in various innovative strategies and activities based on the needs and conditions of the communities. Adapting the whole community approach helped education leaders, teachers, parents, and learners build resilience to sustain the community’s practices on literacy. The concept of the whole community approach is firmly anchored on the core principles of sustainability (i.e., human, social, environmental, and economic). RECOMMENDATIONS ON SUSTAINABILITY To sustain the ABC+ program, 1. ABC+ could: • assist DepEd in scaling up collaboration with the private sector to outsource funds for the continuity of the ABC+ program while transitioning for PY 4-5 and before its conclusion. • strengthen collaboration with TEIs in the region through MOA/MOU to prepare and train future in-service teachers to promote and support early-grade literacies for PY 4-5. • encourage active participation of volunteer parents, youth organizations, and other willing individuals in the community during the school learning action cell session for PY 4-5. • collaborate with schoolteachers to do action research focusing on classroom and community-based reading literacy and numeracy for future reference and create an additional action plan toward sustainability PY 4-5. 51 | ABC+ MID-TERM PERFORMANCE EVALUATION USAID.GOV 2. The DepEd-Regional and Division Office could support teachers, students, community volunteers, and HLPs’ resilience through contextualized well-being mechanisms such as counseling, open dialogue, psychosocial training, and distributing well-being flyers, PY 4-5. 3. The DepEd-Central Office could: • adopt enhanced institutionalized policies on using CRLA as an assessment and educational sector management tool at the national level. ABC+ provided technical support on early-grade-related policies to Regions V and VI. Education policies related to MTBMLE were effective, and this may work in other regional contexts; and • collaborate with LGUs on the process needed to enable annual education financial review and to plan to sustain the needs in achieving continuous improvement on early￾grade literacy. 4. The LGU could: • assist DepEd in sustaining the financial needs of the ABC+ program by allocating additional funds to the following: teacher training, technology transfer capacity building program for school personnel, implementation and monitoring of the program, maintaining barangay reading centers and learning hubs through HLPs, and SRM production and distribution in PY 4-5; and • collaborate with LGUs and local stakeholders to mobilize, support, and sustain the established program. Given the limited human and financial resources of numerous schools in regions V and VI and selected schools in BARMM, especially in remote areas, educational public-private partnership is crucial to learn the program’s sustainability before and even after its conclusion. U.S. Agency for International Development Annex 2 Building, U.S. Embassy, 1201 Roxas Blvd., Ermita, Manila 1000 LIST OF ANNEXES ANNEX 1. AVAILABLE DATA ANNEX 2. SITE AND RESPONDENT SELECTION CRITERIA ANNEX 3. SAMPLING FRAME (INDIVIDUALS, INSTITUTIONS, ORGANIZATIONS) ANNEX 4. TARGET SAMPLE (INDIVIDUALS, INSTITUTIONS, ORGANIZATIONS) ANNEX 5. DATA COLLECTION TOOLS ANNEX 6. DOCUMENTS FOR DESK REVIEW ANNEX 7. VERIFIED OUTCOMES ANNEX 8. LEVEL 1 MOST SIGNIFICANT STORY ANNEX 9. LEVEL 2 MOST SIGNIFICANT STORY ANNEX 10. LEVEL 3 MOST SIGNIFICANT STORY ANNEX 11. LOCALIZED SUSPENSION OF OBSERVATION IN REGION V ANNEX 12. LIST OF KII AND FGD PARTICIPANTS ANNEX 13. SURVEY RESULTS ANNEX 14. LIST OF CODES FOR THE THREE EVALUATION PARAMETERS AND RELIABILITY ANNEX 15. INFORMANTS’ IDENTIFIED POLICIES THAT ALIGN WITH ABC+ ANNEX 16. ACTIVITY SUMMARY INFORMATION OF ABC+ ANNEX 17. STATEMENT OF WORK ANNEX 18. EVALUATION DESIGN AND METHODOLOGY ANNEX 19. LEARNING QUESTIONS ANNEX 20. VALIDATION WORKSHOP AND SURVEY ANNEX 21. TEACHERS’ EVALUATION OF ABC+ CONTRIBUTION TO DEVELOPING LEARNERS’ NUMERACY SKILLS ANNEX 22. SELECTED MOST SIGNIFICANT CHANGE STORY ANNEX 23. INCEPTION REPORT ANNEX 24. EVALUATION TEAM ANNEX 25. REFERENCES ANNEX 26. NONDISCLOSURE AND CONFLICT OF INTEREST AGREEMENTS 1 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 1. AVAILABLE DATA TYPE TIMEFRAME QUALITY RELIABILITY REFERENCE Baseline survey February 22-24, 2020 Includes baseline values for ABC+ that were used to establish key performance indicators (KPIs) and relevant context indicators Collected by the MEL team Baseline report June 2020 Data sources in each IRs July 2019- September 2020 Includes accomplishments for each IRs for Year 1 of the program (PY1) Collected by the MEL team Year 1-Annual Progress Report: July 2019- September 2020 Accomplishments In each IRs October 2020- September 2021 Includes accomplishments for each IRs for Year 2 of the program (PY2) Collected by the MEL team Year 2-Annual Progress Report: October 2020- September 2021 Accomplishments In each IRs May 2022 Includes accomplishments for each IRs for Year 3 of the program (PY3)-Q2 Collected by the MEL team Quarterly Progress Report-Quarter 2 FY_2022 as of May 6 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 2 ANNEX 2. SITE AND RESPONDENT SELECTION CRITERIA SAMPLING PROCEDURES IN SELECTING TEACHERS AND SCHOOL HEADS FOR THE SURVEY USAID ABC+ Evaluation Goal: Sampling of Teachers and School Heads for the Survey Type of Sampling: Stratified Random Sampling 1. Determine the population from where the samples will be taken from ● Population (Universe): ___________________________ ● Accessible population: ___________________________ ● Note: Are there areas that are not accessible for any reason? If any, they will be removed from the sampling frame. 2. Determine the total number of divisions, schools, and participating teachers from the identified population. ● Total number of divisions: _________ ● Total number of schools: _________ ● Total number of participating teachers: _________ 3. Calculate the sample size using Cochran’s formula with a 95% confidence interval and a 3% margin of error. 4. Identify the geographic location of each division/province and determine the corresponding proportion of participating teachers. Distribute the desired sample size based on the proportion of each geographic category. ● Table 1 shows the distribution of all participating teachers and desired sample size by geographic category. 5. Randomly select the schools until the desired number of teachers-respondents per division/area is achieved. The school heads from the selected schools will automatically be the respondents to represent the school. An online randomizer (i.e., random.org) will be used. Table 1. Site and Selection Sheet GEOGRAPHIC CATEGORY NO. OF SCHOOLS / SCHOOL HEADS NO. OF MALE TEACHERS NO. OF FEMALES TEACHERS ALL TEACHERS PROP. MALE PROP. FEMALE SAMPLE SIZE BY AREA SAMPLE SIZE (MALE) SAMPLE SIZE (FEMALE) COHORT 1: Region V Albay Camarines Norte Legazpi City Sorsogon Region VI Escalante City Iloilo City Negros Occidental Roxas City 3 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV GEOGRAPHIC CATEGORY NO. OF SCHOOLS / SCHOOL HEADS NO. OF MALE TEACHERS NO. OF FEMALES TEACHERS ALL TEACHERS PROP. MALE PROP. FEMALE SAMPLE SIZE BY AREA SAMPLE SIZE (MALE) SAMPLE SIZE (FEMALE) COHORT 2: Region V Iriga City Camarines Sur (Rinconada) Masbate Region VI Antique Capiz COHORT 3 Region V Tabaco City Camarines Sur (Central Bikol) Naga City Catanduanes Region VI Bago City Cadiz City Kabankalan Iloilo Prov. La Carlota City Silay City Passi City Bacolod City Sagay City San Carlos City TOTAL USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 4 ANNEX 3. SAMPLING FRAME (INDIVIDUALS, INSTITUTIONS, ORGANIZATIONS) NUMBER OF SCHOOLS FEMALE MALE TOTAL TRAINED TEACHERS* Cohort 1 Region V Albay 176 1117 101 1218 Camarines Norte 253 1917 141 2058 Legazpi City 44 475 17 492 Sorsogon 109 609 60 669 Region VI Escalante City 27 247 13 260 Iloilo City 54 764 17 781 Negros Occidental 500 3505 237 3742 Roxas City 46 360 20 380 Cohort 2 BARMM Cotabato City 11 91 6 97 Maguindanao I 10 8 2 10 Maguindanao II 95 312 36 348 SGA 50 245 37 282 Region V Camarines Sur 153 973 65 1038 Iriga City 40 303 14 317 Masbate 127 720 43 763 Tabaco City 40 365 19 384 Region VI Antique 485 2211 145 2356 Cohort 3 Region VI Bacolod City 46 120 4 124 La Carlota City 24 156 7 163 Sagay City 49 119 3 122 San Carlos City 28 36 36 Total 2364 14652 986 15638 *NEAP-accredited training courses 5 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 4. TARGET SAMPLE (INDIVIDUALS, INSTITUTIONS, ORGANIZATIONS) UNIT OF ANALYSIS INSTITUTIONS ORGANIZATIONS INDIVIDUALS TOTAL METHOD CENTRAL ROS (COHORT 1, 2, BARMM) COHORT 3 (INTERVENTION) DOS THIS LGU (WITH CPES) PRIVATE SECTOR SCHOOL HEADS TEACHERS CURRICULUM DEVELOPERS TEI FACULTY Surveys 2 3 10 17 2 Midline participants Midline participants Bloom and Let’s Read Platform trainees BSU WVS U TBD KII 2 3 X 17 2 X X X X X 24 FGD X X 1 group (reps of 10 cities) 8 FGDs with school heads and teachers (7 languages + Maguindanawn) 1 FGD with BSU and WVSU 10 OH workshops Level 3 Leve l 2 Level 1 (2 groups) Level 1 (6 groups: 5 participants for each group) 3 Case study X 3 (1 school/IR) 1 X X X X X X X X X 4 Classroom observations X X X n= 68 (17 x 4 class es: K-3) X X X X X X X X 68 Document review Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes TBD Note: X = not applicable USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 6 Projected participants and their backgrounds KII ● DepEd leaders who are in key RO and DO positions and MTB-MLE Coordinators in the Regions, Divisions, and schools who are knowledgeable and actively involved in teacher training, materials development, and activities for their sustainability. ● ABC+ leaders and staff who played key roles in the interventions. ● Teachers from small, big, and GIDCA schools who participated in the interventions and were part of the baseline and midline assessments. FGD ● 1-2 DepEd Teachers who were part of the baseline and midline assessment, representatives of school size or type (big schools, small schools, and GIDCA), and have participated in several ABC+ interventions. ● 1-2 DepEd Leaders who may represent either the Division, District, or school and/or an MTB￾MLE focal person at any level. ● 1-2 ABC+ leaders or focal persons who are knowledgeable and directly involved in interventions. ● 1-2 Parents or HLPs, one of whom assisted a male child, and the other female in their studies, have received ABC+ SRMs, and have participated in ABC+ activities for remote learning (e.g., text blasts, webinars, etc.) ● 1-2 institutional partners who occupy a key position or have been involved or knowledgeable in ABC+ interventions. ● 1-2 community members representing LGU, business groups, advocacy groups, civil society, etc. who have participated in ABC+ activities. If relevant or applicable, they should come from cities that have piloted an MSE program. If circumstances permit, an FGD involving Grade 2 learners can be conducted. The projected participants and their backgrounds are as follows: ● 5-7 Grade 2 male and female learners from ABC+ beneficiary schools, whose teachers have participated in various training, have received SRMs and other kits/packages, and whose respective HLPs have been supported by ABC+ activities. Parents of these children should give consent for their children’s FGD participation. Children should also give their assent in verbal and written means. 7 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 5. DATA COLLECTION TOOLS Annex 5.1 SURVEY FOR TEACHERS Good day! We would like to let you know that as a recipient of the Advancing the Basic Education in the Philippines Program (ABC+) of USAID, you were selected to take part in the survey. This activity is part of the evaluation study of the ABC+ Program. Before you decide to participate in this study you must understand why this is being done and what it will involve. Please read the following information carefully and ask the evaluator if there is anything that is not clear or if you need more information. PROCEDURE In this survey, we will be asking you some questions about the nature of your participation and outputs attributed to the ABC+ Program. The survey can be completed in 30 minutes. Your participation in this evaluation study can provide insights into how the ABC+ program can move forward to improve instruction, quality, access to reading materials, and system capacity to sustain efforts at the early-grade level. CONFIDENTIALITY For this study, your identity will be anonymous. Every effort will be made by the evaluator to preserve your confidentiality. CONTACT INFORMATION If you have questions at any time about this study, you may contact the evaluator whose contact information is provided below: Randolf S. Sasota randolf.sasota@gmail.com VOLUNTARY PARTICIPATION Your participation in this study is voluntary. It is up to you to decide whether or not to take part in this study. If you decide to take part in this study, you will be asked to sign a consent form by entering your name in the consent section of this google form. CONSENT I have read and understood the provided information. I understand that my participation is voluntary and that I am free to withdraw at any time, without giving a reason and without cost. I understand that I will be given a copy of this consent form. We request you to answer all the questions. For the questions with a response scale, please tick the answer based on your degree of agreement or disagreement. For open-ended questions, please type your answer in the provided box. Once you finish, please click the “DONE” button, and your responses will be automatically sent to us. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 8 A. General information Name (Optional): ________________________ Date: _____________ Age: Sex at birth: Male/Female (For discussion) Highest Educational Attainment: Institutional Affiliation: Town/City: Province: Region: Designation: From 2019 to the present, did you participate in ABC+ activities? o Yes o No If yes, in what year? o 2019 o 2020 o 2021 o 2022 What was the form of training? o Virtual o In-person What was your role in the training you engaged in? o A trainee o A trainer o Others: _____ (Please indicate.) B. Relevance: 1. Which of the following statements best describes how ABC+ compares with other professional development in which you have participated during the past six months? ● ABC+ was more useful than other professional development on MTB-MLE that I have participated in. ● ABC+ was about the same as other professional development that I have participated in ● ABC was less useful other than the professional development that I have participated in ● I don’t have an opinion. ● I don’t have an opinion because I have not participated in any other professional development in the last six months ● Others (Please indicate.) 9 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV 2. List at least three adjustments that ABC+ implemented in their activities/programs to account for the changes brought by the COVID-19 pandemic. 3. How have your students responded to ABC+ supplementary reading materials (SRMs)? 4. How about parents and your co-teachers? Provide any feedback on children's responses to ABC+ SRMs from them. 5. On a scale of 1-3, where 1 is the highest and 3 is the lowest, rate the extent to which ABC+ considered trainees' capacity and individual situations in each of the adjustments you listed above. 6. Do you know of any DepEd policy (local, regional, or national) on early-grade instruction that has been modified or enhanced because of ABC+ activities/programs? If yes, explain briefly how ABC+ activities influenced the policy modification or enhancement _____________________________________________________________ 7. Recall the time before participating in the ABC+ program and compare it now in terms of your early-grade instructions and delivery. To what extent has the program been useful in improving early-grade instructions and delivery since its implementation? ● ABC+ program has been very useful in improving early-grade instructions and delivery. ● ABC+ program has been somewhat useful in improving early-grade instructions and delivery. ● ABC+ program has been bit useful in improving early-grade instructions and delivery. ● ABC+ program has not been useful at all in improving early-grade instructions and delivery. C. Effectiveness 1. Indicate the extent to which the ABC+ met your professional learning and teaching needs in early-grade instruction ● It addressed my professional learning and teaching needs completely. ● It addressed some of my professional learning and teaching needs. ● It did not address my professional learning and teaching needs. ● This professional development did not help much because I was already familiar with this topic. 2. To what extent has ABC+ training helped you improve your instruction in the following: To a great extent To a moderate extent To a limited extent Literacy Mathematics Integration of SEL Integration of GESI? 3. As you plan your early-grade instruction, are there any activities especially intended for boys? ● Yes ● No 4. If your answer above is yes, can you describe it in brief? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 10 5. As you plan your early-grade instruction, are there any activities especially intended for girls? ● Yes ● No 6. If your answer above is yes, can you describe it in brief? 7. How did you implement instruction during the pandemic? Check all that apply: ● Use of modules, implemented with the help of home learning partners ● Online learning using Moodle and conferencing platforms (Zoom, Google Meet) ● House-to-house visits to teach students face-to-face ● Others: (Please specify.) 8. To what extent were you still able to implement skills and knowledge acquired from ABC+ training using the instructional delivery approach used during the pandemic? ● To a great extent ● To a moderate extent ● To a limited extent 9. If your answer above is to a great extent/a limited extent, provide the factors which limited the application of acquired skills in the used delivery mode. 10. Around how many school days were lost in your school due to COVID-19 restrictions in the following school years? ● 2020-2021 ● 2021-2022 11. Give 1-3 suggestions for ABC+ in terms of their conduct of teacher training (for literacy instruction, math instruction, GESI, etc.) 12. Give 1-3 suggestions for ABC+ in terms of their conduct of materials development (training and workshops, distribution of materials, etc.) 13. In your community or school, do boys perform better in school than girls or is it the other way around? Give possible reasons as to why this could be the case in your context. 14. On a scale of 1-3 where 1 is the highest and 3 is the lowest, rank the following ABC+ interventions based on the extent of their contribution to improving your instruction for learners' basic skills in the early grades, especially in reading in the MT and transition to Filipino and English. ● Strengthening capacity development for teachers and school heads ● Developing and providing access to quality reading materials ● Strengthening the system capacity of the Department of Education in your regions/divisions to sustain efforts of ABC+ (i.e., collaboration with NGOs and LGU to improve the system and policies) Please explain ranking: ___________________________________________________________ 11 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV D. Sustainability 1. How likely will you apply continuously what you have learned in ABC+ even after the program implementation? ___ Very Unlikely ___ Unlikely ___ Neutral ___Likely ___Very Likely 2. Which of the following statements best describes the likelihood that you will apply what you learned in ABC+? ● I have already practiced it in my classroom. ● I have already practiced in my classroom, and it seemed to work well. ● I have already practiced in my classroom, and it seemed to work well. I will keep on practicing it. ● I have already practiced in my classroom, but it was not appropriate for my learners. ● I look forward to practicing in my classroom in the next few weeks ● I look forward to practicing in my classroom later this academic year ● I would like to practice, but I don’t have sufficient materials that I need. ● I would like to practice it, but I don't have the chance. ● I don’t think that these things will work with my learners. ● "I would like to practice it, but ________ (Give reason here.)." 3. Which of the following support systems do you need to sustain the good practices that you acquired from ABC+ training programs? ● Continues quality professional teacher training in the next 2-3 years ● Accessibility of materials through radio, TV-based instructions, and other social media platforms. ● Availability of the most appropriate instructional technology for their developmental stage, grade, and subject matter needs. ● Availability of quality supplementary reading materials (big and small books) ● Availability of quality learning materials for early-grade learning ● Availability of infrastructures (e.g., buildings, roads, power supplies) and facilities (e.g., libraries, computer rooms, learning hubs/spaces) for K-3 to strengthen their acquired literacy skills in their mother tongue (MT), especially to learners with special needs. ● School/Division-level policy to collaborate with the local government unit (LGU) for improving early-grade reading, math, and social and emotional skills ● School/Division-level policy to collaborate with non-government organizations (NGOs), for improving early-grade reading, math, and social and emotional skills 4. Generally, what are parents and community members' attitudes toward Mother Tongue-Based Multilingual Education (MTB-MLE)? ● Very positive and highly supportive ● Moderately positive and supportive ● Unaware of MTB-MLE and passive ● Moderately negative and skeptical ● Very negative and highly opposed 5. To what extent have ABC+ activities involved community stakeholders (LGUs, NGOs, elders, etc.) for the improvement of early-grade instruction? 6. How has ABC+ activities supported your school in reaching out to the last mile and GIDCA schools? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 12 7. What are your observations about the coverage that ABC+ activities have received in the news media, for example in newspapers and online news, social media platforms, and during television and radio broadcasts? ● inspiring ● informative ● encouraging ● no idea ● others (Please specify: ________) 8. Provide examples of instances when you saw news reporters and journalists covering ABC+ activities in your school or district. 9. Based on your training experience, what component of ABC+ programs need policy recommendations to ascertain sustainability? ● Collaboration with stakeholders ● Budget allocation ● Time allocation for the training ● The ratio of teaching and learning materials ● Quality of teaching and learning materials ● Integrating socio-emotional learning skills ● Mainstreaming or highlighting gender sensitivity and girls’ empowerment ● Promoting social inclusion when it comes to religious belief and indigenous people’s practices ● Providing access to the internet through wireless and wired means and other options 10. List three to five existing and/or potential “barrier/s” that could hinder the sustainability of ABC+ programs once the USAID engagement ends. 11. List three to five best practices that emerged during the implementation of ABC+ programs that could be sustained by your school/division after the USAID engagement ends. 13 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Annex 5.2 SURVEY FOR SCHOOL HEADS Good day! We would like to let you know that as a recipient of the Advancing the Basic Education in the Philippines Program (ABC+) of USAID, you were selected to take part in the survey. This activity is part of the evaluation study of the ABC+ Program. Before you decide to participate in this study you must understand why this is being done and what it will involve. Please read the following information carefully and ask the evaluator if there is anything that is not clear or if you need more information. PROCEDURE In this survey, we will be asking you some questions about the nature of your participation and outputs attributed to the ABC+ Program. The survey can be completed in 10-15 minutes. Your participation in this evaluation study can provide insights into how the ABC+ program can move forward to improve instruction, quality, access to reading materials, and system capacity to sustain efforts at the early-grade level. CONFIDENTIALITY For this study, your identity will be anonymous. Every effort will be made by the evaluator to preserve the confidentiality of your answers. CONTACT INFORMATION If you have questions at any time about this study, you may contact the evaluator whose contact information is provided below: Randolf S. Sasota randolf.sasota@gmail.com VOLUNTARY PARTICIPATION Your participation in this study is voluntary. It is up to you to decide whether or not to take part in this study. If you decide to take part in this study, you will be asked to sign a consent form by entering your name in the consent section of this Google form. CONSENT I have read and understood the provided information. I understand that my participation is voluntary and that I am free to withdraw at any time, without giving a reason and without cost. I understand that I will be given a copy of this consent form. We request you to answer all the questions. For the questions with a response scale, please tick the answer based on your degree of agreement or disagreement. For open-ended questions, please type your answer in the provided box. Once you finish, please click the “DONE” button, and your responses will be automatically sent to us. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 14 A. General information Name (Optional): Age: Sex at birth: Male/Female (For discussion) Highest Educational Attainment: Institutional Affiliation: Town/City: Province: Region: Designation: From 2019 to the present, did you participate in ABC+ activities? o Yes o No If yes, in what year? o 2019 o 2020 o 2021 o 2022 What was the form of training? o Virtual o In-person What was your role in the training you engaged in? o A trainee o A trainer o Others: _______ (Please indicate.) B. Relevance 1. Which of the following statements best describes how ABC+ compares with other professional development in which you have participated during the past six months? ● ABC+ was more useful than other professional development that I have participated in. ● ABC+ was about the same as other professional development that I have participated in ● ABC was less useful other than the professional development that I have participated in ● I don’t have an opinion. ● I don’t have an opinion because I have not participated in any other professional development in the last six months 15 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV 2. To what extent do ABC+ activities align with existing DepEd policies and/or programs on: ● early-grade instruction ● access to quality materials ● improved education capacity 3. As you observed teachers at the K-3 level, are there any activities especially intended for boys? ● Yes ● No 4. If your number 2 answer is yes, can you describe it in brief? 5. As you observe teachers at the K-3 level, are there any activities especially intended for girls? ● Yes ● No 6. If your number 3 answer is yes, can you describe it in brief? 7. List at least three adjustments that ABC+ implemented in their activities/programs to account for the changes brought by the COVID-19 pandemic. 8. On a scale of 1-3, where 1 is the highest and 3 is the lowest, rate the extent to which ABC+ considered beneficiaries' capacity and individual situations in each of the adjustments you listed above. 9. Do you know of any DepEd policy (local, regional, or national) on early-grade instruction that has been modified or enhanced because of ABC+ activities/programs? ● Yes ● No 10. If yes, explain briefly how ABC+ activities influenced the policy modification or enhancement. C. Effectiveness 1. Indicate the extent to which the ABC+ met your professional learning needs in the supervision of early-grade instruction ● It addressed my professional learning needs completely ● It addressed some of my professional learning needs ● It did not address my professional learning needs ● This professional development did not help much because I was already familiar with this topic 2. To what extent has ABC+ training helped you in mentoring K-3 teachers to improve their instruction in the following: To a great extent To a moderate extent To a limited extent Literacy Mathematics Integration of SEL Integration of GESI? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 16 3. In your institution, in which of the following aspects are ABC+ activities most successful? ● Strengthening capacity development for teachers and school heads ● Developing and providing access to quality reading materials ● Strengthening the system capacity of the Department of Education in your regions/divisions to sustain efforts of ABC+ 4. Please provide specific examples in your #3 answers and why these are the most effective/successful. 5. On a scale of 1-3 where 1 is the highest and 3 is the lowest, rank the following ABC+ interventions based on the extent of their contribution to improving learners' basic skills in the early grades, especially in reading in the MT and transition to Filipino and English ● Strengthening capacity development for teachers and school heads ● Developing and providing access to quality reading materials ● Strengthening the system capacity of the Department of Education in your regions/divisions to sustain efforts of ABC+ 6. How did you implement instruction during the pandemic? Check all that apply: ● Use of modules, implemented with the help of home learning partners ● Online learning using Moodle and conferencing platforms (Zoom, Google Meet) ● House-to-house visits by teachers who teach students Face to face ● Others: (please specify) 7. To what extent were teachers still able to implement skills and knowledge acquired from ABC+ training using the instructional delivery approach used during the pandemic? ● To a great extent ● To a moderate extent ● To a limited extent 8. If your answer above is to a great extent/a limited extent, provide the factors which limited the application of acquired skills in the used delivery mode. 9. Around how many school days were lost in your school due to COVID-19 restrictions in the following school years? ● 2020-2021 ● 2021-2022 10. Give 1-3 suggestions for ABC+ in terms of their conduct of teacher training (for literacy instruction, math instruction, GESI, etc.) 11. Give 1-3 suggestions for ABC+ in terms of their conduct of materials development (training and workshops, distribution of materials, etc.) 12. In your community or school, do boys perform better in school than girls or is it the other way around? Give possible reasons as to why this could be the case in your context. D. Sustainability 1. Which of the following statements best describes the likelihood that you will apply what you learned in ABC+? ● I have already practiced in my school. ● I have already practiced in my school, and it seemed to work well. ● I have already practiced in my school, but it was not appropriate for our K-3 learners. 17 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ● I look forward to practicing in my school in the next few weeks. ● I look forward to practicing in my school later this academic year. ● I would like to practice, but we don’t have the materials that we need in our school. ● I don’t think that these things will work with our K-3 learners. 2. Please rate your readiness in continuing ABC+ activities after the project ends ● Strongly ready ● Ready ● Not ready ● Strongly not ready ● Do not know 3. What plans do you have to sustain/replicate/continue the program after the project ends? 4. To what extent have ABC+ activities involved community stakeholders (LGUs, NGOs, elders, etc.) for the improvement of early-grade instruction? ● To a great extent ● To a moderate extent ● To a limited extent 5. How has ABC+ activities supported your school in reaching out to last-mile schools and GIDCA? 6. What could be the major challenges or obstacles in implementing your plans after the project ends? How do you intend to overcome them? 7. What are some of your predictions about how the educational ideas, methods, and materials developed under ABC+ will continue to be used after the project ends? 8. Generally, what are parents and community members' attitudes toward Mother Tongue-Based Multilingual Education (MTB-MLE)? ● Very positive and highly supportive ● Moderately positive and supportive ● Unaware of MTB-MLE, passive ● moderately negative and skeptical ● very negative and highly opposed 9. What are your observations about the coverage that ABC+ activities have received in the news media, for example in newspapers and online news, social media platforms, and during television and radio broadcasts? Provide examples of when you saw news reporters and journalists covering ABC+ activities in your school or district. ● inspiring ● informative ● encouraging ● no idea ● others (please specify: ________) Our survey about ABC+ is now concluded. Your comments today have been very valuable and useful for our evaluation of the project. Thank you for providing your time to assist us. The USAID evaluation report of ABC+ will later be publicly released. We will contact you in the future if we need to make some clarifications on your responses. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 18 Annex 5.3 KEY INFORMANT INTERVIEW QUESTIONNAIRE Date: Preliminaries 1. Introduction of CLAimDev a. Recording b. Screenshot of KII participant 2. Introduction of participant a. Giving of name b. Giving of agency or affiliation c. Giving of location: school, division, region 3. Reading of house rules (RFAs: in charge) a. To allow our conversation to flow more freely, I’d like to go over some ground rules. b. There are no “wrong answers,” just your observations and opinions. c. As much as possible please answer each question. d. Whatever is shared during the discussion/interview will remain confidential. e. Let me know about your questions now. Are there any questions? KII with DepEd Leaders (From Region, Division, District, or School) ● What Regional, Division, or District policy (e.g., memorandum or order, regulations, programs, etc.) are ABC+ activities on SRM development and distribution in your locale aligned with? Please name these policies and explain how ABC+ activities are aligned with them. ● How has ABC+ activities enriched your processes for materials development (i.e., contextualization and localization) and distribution? In what ways has ABC+ enhanced your operations for this dimension? ● What are the considerations to contextualize the designed/developed SRMs ● How many SRMs has your school/division/region received from ABC+ interventions? Who has these SRMs or who is accountable for them? ● How frequently do the teachers and students use the designed/developed SRMs ● What feedback have you received from teachers, parents/HLPs, and their children about the quality and effectiveness of these SRMs? Describe how the children responded to the SRMs and their use. Did the SRMs affect children’s interest in reading in any way? ● Given all the ABC+ activities that you engaged with, which group do you think benefited the most (i.e., teachers, learners, school heads)? Why this group? You may rank the groups and explain why. ● In what ways are the School, Division, and Region ensuring the strides made in the following through ABC+ activities will be sustained long after the program has concluded o Development of quality SRMs through a workshop, QA, inclusion of gender-fair/SEL themes, etc. o Implementation and monitoring SRMs to ensure the proper inclusion of gender-fair/SEL themes o Partial evaluation of intervention and revision of SRMs for continuous improvement and 19 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV quality assurance o Accessibility of quality SRMs through funding and ensuring that GIDCA schools are provided with them. o Continuous Professional development and training of teachers and curriculum developers ● What issues and concerns have you experienced from teachers, curriculum developers/content writers about the quality, effectiveness, and sustainability of the intervention? KII with Teachers ● What Regional, Division, or District policy (e.g., memorandum or order, regulations, programs, etc.) are ABC+ activities on SRM development and distribution in your locale aligned with? Please name these policies and explain how ABC+ activities are aligned with them. ● How have ABC+ activities enriched your processes for materials development and distribution? In what ways has ABC+ enhanced your operations for this dimension? ● How has ABC+ activities enriched your instruction and delivery? In what ways has ABC+ enhanced your teaching skills? Do you get to apply what you have learned from ABC+ training in your classes? If yes, what are the effects of your application of skills and knowledge? ● How many SRMs has your school/division/region received from ABC+ interventions? Who has these SRMs or who is accountable for them? ● What feedback have you received from co-teachers, parents/HLPs, and their children about the quality of these SRMs? ● How different are these SRMs from the ones you had? Why? ● Have you integrated these SRMs into your instruction? If yes, how? What effect, if any, did the SRMs have on your instruction? Did you notice any change in the students when you started using the SRMs? Tell us about it. ● How did children respond to the use of SRMs? ● In what ways are the School, Division, and Region ensuring the strides made in the following through ABC+ activities will be sustained long after the program has concluded o Development of quality SRMs through a workshop, QA, inclusion of gender-fair/SEL themes, etc. o Implementation and monitoring SRMs to ensure the proper inclusion of gender-fair/SEL themes o Partial evaluation of intervention and revision of SRMs for continuous improvement and quality assurance o Accessibility of quality SRMs through funding and ensuring that GIDCA schools are provided with them. o Continuous Professional development and training of teachers and curriculum developers o Review of existing policy on the inclusion of gender-fair/SEL themes for future reference ● What issues and concerns have you experienced from parents and their children during the implementation of the ABC+ intervention (Sustainability, IR 1,2,3) USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 20 ABC+ Partners (LGU-LSB, Private Sector, and NGOs) ● Have you collaborated with DepEd for materials development previously? Based on your experience, how has ABC+ activities enriched DepEd processes for materials development and distribution? In what ways has ABC+ enhanced operations for this dimension? ● How have you collaborated with DepEd for teacher training and professional development previously? Based on your experience, how has ABC+ activities enriched DepEd processes for teacher training and professional development, policy review of education system capacity, and commitment to manage and implement interventions? In what ways has ABC+ enhanced operations for this dimension? ● How was your collaboration experience with ABC+ concerning teacher training and professional development to improve the instruction and delivery of intervention activities and to possible policy review of education system capacity and commitment to manage and implement interventions? ● Would you be interested to continue with these activities and policy review of the education system capacity and commitment to manage and implement interventions with DepEd even after the conclusion of ABC+ engagement? Why or why not? ● Has DepEd expressed any interest in engaging you again in future activities similar to the ones you participated in for ABC+? ● What issues and concerns have you experienced with DepEd as LGU and NGO partners during the implementation of the ABC+ interventions? 21 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Annex 5.4 FOCUS GROUP DISCUSSION PROTOCOL Date: Preliminaries 1. Introduction of CLAimDev a. Recording b. Screenshot of KII participant 2. Introduction of participant a. Giving of name b. Giving of agency or affiliation c. Giving of location: school, division, region 3. Reading of house rules (RFAs: in charge) a. To allow our conversation to flow more freely, I’d like to go over some ground rules. b. In this FGD, only one person speaks at a time. c. Everyone doesn’t have to answer every single question. d. Whatever is shared during the discussion will remain confidential. e. There are no “wrong answers,” just different opinions. f. Since our time is limited there may be a need to ask you to stop and change topics from time to time. g. Let me know about your questions now. Are there any questions? FGD Questions 1. Relevance ● How have ABC+ activities contributed to the enhancement of policies on instruction and learning outcomes in kindergarten to grade 3? ● How have ABC+ activities for professional training and materials development enriched your instruction? ● What materials development and distribution policies and processes that were already in place before were enhanced or made more efficient or more effective by ABC+ interventions? ● Do you know of any DepEd policy (local, regional, or national) on early-grade instruction that has been modified or enhanced because of ABC+ activities/programs? If yes, explain briefly how ABC+ activities influenced the policy modification or enhancement. a. On the effect of the pandemic: ● How has the COVID-19 pandemic affected ABC+ activities in terms of: o improving early-grade instruction and delivery, o improving access to quality materials, and o improving education capacity and o managing and implementing interventions supporting early-grade instruction? ● List at least three adjustments that ABC+ implemented in their activities/programs to account for the changes brought by the COVID-19 pandemic. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 22 ● On a scale of 1-3, where 1 is the highest and 3 is the lowest, rate the extent to which ABC+ considered trainees' capacity and individual situations in each of the adjustments you listed above. ● Around how many school days were lost due to COVID-19 restrictions in the past school years? ● To what extent were you still able to implement skills and knowledge acquired from ABC+ training using the instructional delivery approach used during the pandemic? ● How did home learning partners (HLPs) implement the ABC+ program? ● How was the learning of children from Geographically Isolated, Disadvantaged, and Conflict￾Affected Areas (GIDCA) monitored, especially during the pandemic? What is the status of their learning compared to their peers? b. On Gender Equality and Social Inclusion (GESI): ● Are the needs of boys and girls addressed in classroom instruction? Cite examples or instances where this was observed. ● How have ABC+ activities promoted gender equality and social inclusion (GESI) in your instruction delivery? ● What was your experience in teacher training and delivery of instruction that highlighted GESI? ● Have you integrated GESI into your lessons? If yes, how did you do that? How would you describe the experience of integrating it with instruction or introducing it to learners? o How did the girls respond to it? How about the boys? Do you have any teaching activities that you implement for boys only or for girls only? Why? Where did you get that idea? What other strategies or activities of yours differentiate boys from girls? What made you decide to do it that way? How did the boys and girls respond to these activities? 2. Effectiveness a. On Program Contribution ● Compared with your previous programs in early-grade literacy, what is/are the unique features, if any, of ABC+ in the improvement of early-grade instruction and delivery, access to materials, and capacity of your division/school? ● To what extent have skills and knowledge acquired from ABC+ activities been applied in instruction? ● How useful are knowledge and skills gained from ABC+ activities in supporting your schools’ instruction and quality and accessibility of materials in the early grades? ● Recall the time before participating in the ABC+ program and compare it now in terms of your early-grade instruction and delivery. To what extent has the program been useful in improving early-grade instruction and delivery since implementation? 23 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ● How have ABC+ activities improved teachers’ and school heads’ capacity to design, deliver, or supervise instruction at early-grade levels? ● Give 1-3 suggestions for ABC+ in terms of their conduct of teacher training (for literacy instruction, math instruction, GESI, etc.). ● What learning support do children need to make learning more effective? How about parents/guardians, what support system do they need to make children’s learning more effective and meaningful? b. On Supplementary Reading Materials (SRMs) ● Did all enrolled children here receive an SRM package? What is in the package? How and when were these delivered? Was delivery timely? Did schools with learners from GIDCA in the area also receive packages? How did the school ensure that learners from GIDCA had access to SRMs? ● Do you think SRMs are properly contextualized and appropriate to the learning needs of children? Why? ● How did you and/or the children use the SRMs and how often? ● Do the children use the SRMs on their own? If yes, how? ● What effect did SRM use have in the teaching process? In student response (engagement and learning)? ● Do the SRMs have any effect or change on children’s reading or learning how to read? Why do you associate these effects or changes with SRM use? ● Are SRMs easy to use and follow by the parents/ home learning partners (HLPs)? ● What are the three things that you and/or the children like best about the SRMs? Why? ● What are three opportunities for SRM improvement? What else would you like to see in the future for SRMs? ● What are the three challenges for SRM improvement? c. GESI in SRMs ● How have ABC+ activities promoted GESI in materials development? ● Do boys and girls find the ABC+ SRMs stimulating and relevant to their age group and gender? ● What was your experience in using SRMs that highlight GESI? Did the materials appeal to boys only, girls, or both? What makes you say that? d. Socio-emotional learning (SEL) in SRM ● Did the SRMs feature social or cultural values? If yes, which stories are these, and what values were presented? How did the children respond to these stories? Do you think they understood the values presented? What makes you say that? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 24 3. Sustainability ● What ABC+ activities would you like to continue even after the engagement with ABC+ has ended? Why? ● List three to five best practices that emerged during the implementation of ABC+ programs that could be sustained by your school/division after the engagement ends. What plans does your school have for sustaining or replicating ABC+ activities after the project ends? ● What could be the major challenges or obstacles in implementing the plan of your school after the project ends? How do you intend to overcome them? ● Generally, what are parents and community members' attitudes toward Mother Tongue￾Based Multilingual Education (MTB-MLE)? ● Has the region, division, or school made policy, budget, staffing, etc. provisions for continuing ABC+ activities for teacher training, SRM development and distribution, and policy review on the implementation and continuous improvement of MTB-MLE even after the program has ended? Discuss each provision. ● To what extent have ABC+ activities involved community stakeholders (LGUs, NGOs, elders, etc.) for the improvement of early-grade instruction? ● What emerging partnerships that were initiated by your region/division/school may support the sustainability of gains of learners in early-grade literacy? ● What component/s of the ABC+ program need policy recommendations to ascertain sustainability? ● What are three opportunities for teacher professional development? What else would you like to see in the future for teacher professional development? ● What are the three challenges in policy implementation and improvement? What else would you like to see in the future in terms of education system capacity and commitment to manage ABC+ intervention programs? ● What are three opportunities for policy implementation and improvement? Provide examples of instances when you saw news reporters and journalists covering ABC+ activities in your school or district. What impressions did these give you? Why? 25 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Annex 5.5 OUTCOME HARVESTING WORKSHOP QUESTIONNAIRE RELEVANCE EFFECTIVENESS SUSTAINABILITY IR 1 Teacher Training What have been the outcomes of ABC+ interventions on DepEd policies (memos, plans, priorities, etc.) on teacher training, SRMs, and student learning (literacy, math, and SEL)? What specific ABC+ activities led to these outcomes? What have been the outcomes of ABC+ teacher training? What specific ABC+ activities led to these outcomes? What have been the outcomes of ABC+ activities to ensure local ownership and continued support for teacher training, SRM development, and systems capacity? What specific ABC+ activities led to these outcomes? IR 2 Materials (SRMs) What have been the outcomes of ABC+ activities for materials development? ABC+ produced SRMs? What specific ABC+ activities led to these outcomes? IR 3: System Capacity What have been the outcomes of ABC+ interventions on DepEd capacity and processes and community participation for the improvement of teacher training, SRM development, and student learning? What specific ABC+ activities led to these outcomes? What have been the outcomes of ABC+ activities for developing DepEd systems and capacity and community participation to support early-grade instruction? What specific ABC+ activities led to these outcomes? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 26 Annex 5.6 MOST SIGNIFICANT CHANGE IN STORY COLLECTION FORM Most Significant Change Story Collection Form Background USAID and DepEd would like to capture stories of significant change that may have resulted from the activities and outputs of ABC+ in [location]. This will help ABC+ in improving their work and enable them to celebrate successes with you. The stories and information collected from these interviews have the following objectives: ● Identify areas that need improvement in ABC+ activities ● Identify what has already been achieved ● Collect the experiences of people in [location] concerning ABC+ activities Confidentiality Clause and Contact Details ● If the storyteller wants to remain anonymous, don’t record their name or contact details – just write ‘storyteller’ or similar descriptions. Name of storyteller: __________________________________________________ Group Membership (tick one): • Parent/Guardian • Teacher • School Head • DepEd Leader (State position:_______) • Trainer (State affiliation: ________) • Community Member (State affiliation: _________) • Others: (State affiliation: __________) Name of the person recording the story: _______ Represented School: _______ Confidentiality May we use your stories for reporting to USAID, or sharing with other people in the region? Do you [storyteller]: (tick one) ● Agree to write down your story and share it with others. ● Afree to have your name in the story (tick one)? ● Agree to use your story in a publication? ● Agree to use your photo in the story? 27 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV District: ______ Division: ______ Region: _____ Date of Recording: _______ Questions 1. Tell me about how you learned about [innovation] and how you got involved with [innovation]. __________________________________________________________________________ ______________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ 2. Please take a few minutes to think about all the changes that have happened in the past [X] years of implementing [innovation]. For teachers and parents: Think of changes related to experiencing and using the [innovation] For trainers and focal persons: Think of changes related to including the [innovation] in your operations Pause here to allow the storyteller to think about all the changes. Allow him or her to write down notes if s/he wants to. Give the storyteller some thinking and note-taking time 3. From your point of view describe a story that best illustrates the most significant change that you have experienced or observed as a result of [innovation] being offered in the school or the community. ▪ There may have been many changes, great and small, positive and negative. Choose the change that you feel is the most significant. ▪ Describe who was involved, what happened, where, and when. ▪ Include enough detail to make it understandable to someone not familiar with your experience or what you have witnessed. ▪ If choosing one change is difficult, feel free to describe more than one. Another form will be used for that story. ▪ The significant change you choose can be in: o The lives of the beneficiaries (teachers, learners, parents) o The lives of the individuals in the community o The lives of colleagues with whom you worked o The wider community or the DepEd system o Policy (memos, regulations, etc.) and operations (processes, practices) ____________________________________________________________________ ____________________________________________________________________ USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 28 ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ 4. Give your story a title. ____________________________________________________________________ 5. Why is this story significant to you? ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ 6. How has the [innovation] contributed to the significance of the story? ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ -end of the form- 29 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Annex 5.7 MOST SIGNIFICANT CHANGE GUIDELINES FOR STORY VERIFICATION AND SELECTION Story Verification Guide for Level 2 Participants (ABC+ and DepEd trainers/focal persons) ● Is the storyteller a confirmed beneficiary of ABC+ activities and outputs? ● Has the storyteller been a beneficiary long enough to express the details given in the story? ● Are the details (when and where) in the story consistent with ABC+ activities? ● Have you heard of similar stories reported by other beneficiaries before? ● Is the story credible? Do the reported experiences portray what is likely or realistic based on the context or nature of the intervention and other factors? Significance Criteria for Level 2 and Level 3 Participants ● How close is the story’s illustration of events to ABC+ goals? ● How impactful is the change described in the story? ● Does the story reflect the intended and unintended goals of ABC+? ● How well does the story capture stakeholder perspective vis-à-vis implemented ABC+ activities? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 30 ANNEX 6. DOCUMENTS FOR DESK REVIEW Table 1: Existing Policies (Orders and Memoranda) and Program of the Department of Education on MTB-MLE at the National Level ISSUANCE DATE TITLE RELEVANT IRS IN DOCUMENT HB No. 2188 July 20, 2022 An Act Suspending the Implementation of the Use of Mother Tongue as the Medium of Instructions for Kindergarten to Grade 3 IR1, IR2 DO 21, s. 2019 August 22, 2019 Policy Guidelines on the K to 12 Basic Education Program IR1, IR2 DO 55, s. 2015 December 8, 2015 Utilization of Language Mapping Data for Mother Tongue￾Based Multilingual Education (MTB-MLE) Program Implementation IR2 DO 28, s. 2013 July 5, 2013 Additional Guidelines to DepEd Order No. 16, s. 2012 [Guidelines on the Implementation of the Mother Tongue￾Based Multilingual Education (MTB- MLE)] IR1, IR2 DO 16, s. 2012 February 17, 2012 Guidelines on the Implementation of the Mother Tongue￾Based Multilingual Education (MTB-MLE) IR1, IR2 DO 74, s. 2009 July 14, 2009 Institutionalizing Mother Tongue-Based Multilingual Education (MTB-MLE) IR1, IR2 Table 2: Department of Education Orders and Memoranda on Materials ISSUANCE DATE TITLE RELEVANT IRS IN DOCUMENT DO 44, s. 2013 September 26, 2013 Moratorium on the Procurement of Supplementary Reading, Reference, and Other Instructional Materials (IMs) IR2 DO 13, s. 2012 February 3, 2012 Guidelines on the Allocation, Delivery, and Distribution of Instructional Materials (IMs) to Support the K to 12 Curriculum IR2 DO 90, s. 2011 November 10, 2011 Guidelines on the Utilization of Downloaded Funds for Materials Development and Production of Mother Tongue￾Based Multilingual Education (MTB-MLE) Program IR2 31 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Table 3: Department of Education Orders and Memoranda on Teacher Training/Professional Development ISSUANCE DATE TITLE RELEVANT IRS IN DOCUMENT DO 24, s. 2020 September 7, 2020 National Adoption and Implementation of the Philippine Professional Standards for School Heads IR3 DO 42, s. 2017 August 11, 2017 National Adoption and Implementation of the Philippine Standards for Teachers IR3 DO 12, s. 2015 April 10, 2015 Guidelines on the Early Language, Literacy, and Numeracy Program: Professional Development Component IR1, IR3 DM 35, s. 2012 February 17, 2012 Training of Trainers for Kindergarten Teachers on Mother Tongue-Based Multilingual Education (MTB-MLE) IR1, IR3 DO 18, s. 2011 February 23, 2011 Guidelines on the Conduct of Mother Tongue-Based Multilingual Education Training IR1 DM 495, s. 2010 December 8, 2010 National Training of Trainers on the Mother Tongue-Based Multilingual Education (MTB-MLE) Program for Phases III and IV IR1 DO 18, s. 2011 February 23, 2011 Guidelines on the Conduct of Mother Tongue-Based Multilingual Education Training IR1 Table 4: Department of Education Reports and Documents TITLE RELEVANT IRS IN DOCUMENT School Improvement Plans (SIP) of selected schools and case study schools IR1, IR2, IR3 Regional Basic Education Plan of Regions V, VI, and BARMM IR1, IR2, IR3 Division Education Development Plans IR1, IR2, IR3 Learning Recovery Plans of Regions, Divisions, and Schools IR1, IR2, IR3 Table 5: Existing Policies (Orders and Memoranda) and Program of the Department of Education on MTB-MLE at the Regional Level ISSUANCE DATE TITLE RELEVANT IRS IN DOCUMENT General Policies DO 21, s. 2019 August 22, 2019 Policy Guidelines on the K to 12 Basic Education Program IR1, IR2 DO 55, s. 2015 December 8, 2015 Utilization of Language Mapping Data for Mother Tongue-Based Multilingual Education (MTB-MLE) Program Implementation IR2 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 32 ISSUANCE DATE TITLE RELEVANT IRS IN DOCUMENT DO 28, s. 2013 July 5, 2013 Additional Guidelines to DepEd Order No. 16, s. 2012 [Guidelines on the Implementation of the Mother Tongue-Based Multilingual Education (MTB- MLE)] IR1, IR2 DO 16, s. 2012 February 17, 2012 Guidelines on the Implementation of the Mother Tongue-Based Multilingual Education (MTB-MLE) IR1, IR2 DO 74, s. 2009 July 14, 2009 Institutionalizing Mother Tongue-Based Multilingual Education (MTB-MLE) IR1, IR2 Materials DM 441, s. 2019 (Region V, Division of Naga) October 1, 2019 Guidelines and Process for LRMDS Assessment and Evaluation of Locally Developed and Procured Materials IR2, IR3 Table 6: ABC+ Documents TITLE RELEVANT IRS IN DOCUMENT Fiscal Year 2022 Quarter 2 Report January 1-March 31, 2022 ABC+: Advancing Basic Education in the Philippines IR1, IR2, IR3 Fiscal Year 2021 Quarter 3 Report April 1-June 30, 2021 IR1, IR2, IR3 Project Year 1 Annual Progress Report July 2019-September 2020 IR1, IR2, IR3 Project Year 2 Annual Progress Report October 2020-September 2021 IR1, IR2, IR3 ABC+: Advancing Basic Education in the Philippines Baseline Report June 2020 IR1, IR2, IR3 Country Development Cooperation Strategy (CDCS) November 25, 2019 - November 24, 2024 IR1, IR2, IR3 Monitoring, Evaluation Learning Plan Advancing Basic Education in the Philippines (ABC+) IR1, IR2, IR3 GESI Plan Update IR1, IR2, IR3 Draft ABC+ MEL Plan September 27, 2019 IR1, IR2, IR3 Draft ABC+ MEL Plan September 27, 2019, unannotated IR1, IR2, IR3 USAID/Philippines Basa Pilipinas Program Early-grade Reading Assessment Final Evaluation Report: 2018 IR1, IR2, IR3 ABC+ Gender Equity and Social Inclusion Accomplishments 2020-2021 IR1, IR2, IR3 ABC+ Gender Indicators Highlighted IR1, IR2, IR3 ABC+ Gender and Inclusion Plan 2019 IR1, IR2, IR3 Program Cycle Guidance How to Note Evaluation Statements of Work IR1, IR2, IR3 33 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 7. VERIFIED OUTCOMES IR 1 IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Fully Agree Partially Agree Disagree Explanation/ Remarks Evaluators' Remark R5 1 T1 and T2 Strategies and techniques learned in TT1 and TT2 (Teachers Training 1, Teachers Training 2) become part of the training culture. Adaptation of ABC+ instructional strategies X Our training includes project creation and direct application that the teachers can use in the classroom [Not enough significant activities are listed to validate the credibility of the outcome.] R5 3 Development of the tool Conduct of Comprehensive Rapid Literacy Assessment Provision of appropriate intervention Provision of appropriate intervention X Teachers used the CRLA at the Beginning of the School Year to assess learners' reading profiles. No corresponding interventions are available in CRLA. Just the learner's profile. R5 5 Development of Bicol Language Identification Tool The language of the learners was identified. Provision of appropriate materials considering the language of the learner. Provision of appropriate intervention X BLIT was facilitated to learners and schools to identify their Mother Tongue and in turn, the EGRMs that will be distributed to their school. R5 6 The teachers and parents developed a Home Learning Guide and contextualized the Marungko booklet. Parents and HLP attended capacity building facilitated by the trained teachers, specifically on Marungko and the Learning Experience Approach and the use of Teachers attended the hybrid training of ABC+ -TT1 and TT2. Parents and Home Learning Partners applied the knowledge gained during the capacity building in beginning reading to children including the struggling readers. Adaptation of ABC+ home learning strategies X The outputs during the training are expected to be applied in the classroom and in the homes which are being monitored and expected to be translated into results during the EoSY CRLA. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 34 IR 1 IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Fully Agree Partially Agree Disagree Explanation/ Remarks Evaluators' Remark ELKONIN Boxes. R5 7 Selection of writers in the region, Grades 1 to 3 teachers. 2 weeks intensive crafting of 8th week curriculum (selected writers from the region) Developed the quality assured 8th week learning recovery curriculum for Grades 1 to 3, lesson maps, and worksheets. Implementation and monitoring of the 8th week LRC or Learning Recovery Curriculum to address teaching gaps. Learning Recovery Curriculum X In response to the learning gaps of K-3 learners in the region brought about by the pandemic, the 8 week curriculum is a brave move by the region to dedicate the first 8 weeks of reading instruction to focus on building foundational schools (ie. Literacy and numeracy). R5 8 Teachers prepared the needed preparations for teaching beginning literacy. Teachers applied the KSAs learned in the training. Teachers attended the hybrid TT1 on beginning reading. Beginning reading teachers gained confidence in teaching the learners about beginning literacy. Increase teaching efficacy X Although TT1 does not heavily focus on teaching beginning reading on language transition, strategies for beginning reading are still taught. R5 9 Applying best practices. Sharing of best practices among teachers. Needs assessment for teachers. Classroom management is easily handled by the adviser. Increase teaching efficacy X Teach back/demo of teachers is part of the teacher training R5 10 Teachers applied the strategies learned from training. Trained teachers train teachers in other schools. Teachers from selected schools attended the ABC+ training. Teaching strategies and abilities were enhanced. Increase teaching efficacy X Inputs from trainers were passed down to the training participants through the roll-out where the trainees trained their teachers in the other divisions or schools. R5 11 Trained teachers serve as resource Trained teachers initiated online Teachers attended the ABC+ training. Teacher empowerment Increase teaching efficacy X As part of the roll-out, the trained teachers become part of the 35 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 1 IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Fully Agree Partially Agree Disagree Explanation/ Remarks Evaluators' Remark persons to other districts that invited them and F2F seminars with other K-3 teachers. "pool of trainers" for that region/division and are expected to pass down their knowledge to their districts or other schools. R5 13 Teachers gained mastery in providing appropriate activities and teaching strategies. Teachers prepared a lesson plan with differentiated activities as one output during the seminar. The teacher attended training on different teaching strategies. The teacher was able to apply appropriate teaching strategies in different learning situations. Adaptation of ABC+ instructional strategies X Teacher participants submit outputs during the training that includes applications of differentiated instructions and strategies. Research in differentiated instruction based on CRLA assessment R5 14 Reading materials developed by teachers were given as take￾home activities. The school conducted parent training on teaching techniques. ABC materials were utilized in the reading session during home visitation Parents' burden on the reading habit of their children is minimized. Parental support in reading X The ABC+ -developed materials were distributed to the homes. Parents were trained and were given guides in facilitating reading at home. R5 16 The school implemented the Project Initiative dubbed Project Syllable Mastery in Katon Way (SMK) from 2020-2021 to present the help of the teachers, home learning partners, BLGU, and other stakeholders. School heads facilitated in the TT1, TT2, and School Head’s RILT. The reading crisis is addressed continuously Increased readers X As part of the output of School Heads during the ILT, they are expected to roll out their plan which was supported and reinforced by the region-wide initiative for Learning Recovery and Continuity Plan. R5 17 Teachers Teachers The school head More pupils were able Increased X Need to know which USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 36 IR 1 IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Fully Agree Partially Agree Disagree Explanation/ Remarks Evaluators' Remark attended training in ABC+ learned different teaching strategies/meth ods for teaching children. monitored the implementation of the ABC+ Reading Strategies through Alternative Classroom observations. to read and learn with the new teaching strategy learned from ABC+ number of independent readers set of students and from which schools. The midline results will verify this. R5 22 Sa ating division at municipal level training, naeencourage natin yung mga teachers na gawin yung mga tinuturong strategies ng ABC+. Na kahit medyo kailangan ng effort ay worthwhile at meaningful naman ang magiging results para sa mga bata. Naranasan ko mismo yung effort na ginagawa upang magturo ng isang lesson sa pagbabasa dahil sa ginawa naming demo teaching. Natuto ako kung panu mag turo ng pagbabasa mula sa mga resource person mula sa ABC+ Nadagdagan ang effort at tiyaga ng mga guro sa pagtuturo ng pagbabasa Increased appreciation on teachers role in beginning reading X The strategies taught in the training added to the teachers' knowledge and increased their appreciation for beginning reading. R5 23 Kasama kong nag attend ng training of the trainers ang mga administrators tulad ng PSDS at EPS Sila mismo ay naranasan nila ang maghanda ng mga kagamitan sa pagtuturo sila rin ay naging demo teachers Nakita at nalaman ng mga School Heads kung gaano kahirap at kagastos ang kailangang gawin upang magturo ng tamang pamamaraan ng pagbabasa sa Nabigyang halaga ng mga School Heads at administrators ang mga ginagawa ng mga primar grade teachers sa pagtuturo ng pagbabasa. Increased appreciation of teachers' role in beginning reading X This is reflected in the evaluation after the training 37 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 1 IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Fully Agree Partially Agree Disagree Explanation/ Remarks Evaluators' Remark municipal at district training. R5 24 The assessment was conducted to the teachers' needs. FGD was conducted with a focus on teachers' difficulties. Sharing of best practices. Teachers applied their learnings from their co￾teachers. Classroom management is easily handled by the adviser. Increase teaching efficacy X The teach-back or demo allows teachers to practice classroom management strategies they’ve learned during the training, in the roll￾out. R5 26 Teachers attended ABC+ training Selected teachers and school heads were trained in RTT1 and RTT2 Produced quality reading materials. Development of appropriate reading materials Innovation in teaching X The reading materials that the learners and schools received are based on their mother tongue. The BLIT and CRLA ensure that the learners are reading materials according to their reading profile R5 29 LAC Session Capacitating volunteer teachers and intermediate teachers on Marungko, Fuller, and CPA Increased capacity to teach X LAC sessions serve as follow-through and reinforcement to ensure that the strategies are applied in the classroom. R6 2 Teachers received the ABC+ materials Conducted the orientation on the utilization of ABC+ materials Distribution of ABC+ materials according to reading profile Teachers become aware of the importance of giving appropriate reading materials to the learner so they can relate and have meaningful reading experiences, provision of appropriate intervention Developed precision in identifying specific learner support in reading X Awareness as a foundational objective is key in providing the appropriate intervention for the learners. R6 3 Issued Regional Memo on Conducted workshop on Developed wordless picture Confidence and readiness of teachers in Developed precision in X With the teaching demo conducted as USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 38 IR 1 IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Fully Agree Partially Agree Disagree Explanation/ Remarks Evaluators' Remark indexed/ mapped-out EGRMs and MELCs by grade level per quarter the mapping/ indexing of EGRMs and MELCs by grade level per quarter books, decodable, and leveled readers in the early-grades. the delivery of instructions guided by the Regional Memo on the indexed/mapped￾out EGRMs and MELCs by grade level per quarter identifying specific learner support in reading part of the training, the confidence and readiness of the teachers when cascading the strategies are already in place. R6 4 Learners received the ABC+ materials Conducted orientation to the parents on the use of the HGL Made use of the reading materials. Learners can relate to the reading materials. This arouses their interest and love for reading because the reading materials were appropriate to their reading profile. Cognitively aligned reading materials X The ABC+ -developed reading materials that are aligned with the learner's reading profile make reading interesting for them. What are these types of materials based on the learning profile? R6 5 Conducted Training￾Workshop of Potential Learning Resource Evaluators (LREs) - QUALITY Assurance of Grade Reading Materials (EGRMs) Video Lessons Developed Editing Guides for the Four Mother Tongues a.Akeanon b. Hiligaynon c. Kinaray-a d. Sinugbuanong Binisaya Revisited the Four-Minima per Mother Tongue a. Teacher's Resource Book - Orthography b. Learner's Materials c.Teacher's Guide d. Literary Pieces COMPETENT Pool of LREs in four major languages of the Region. (Kinaray=a, Hiligaynon, Akeanon and Sinugbuanong Bisaya) Increased capacity in curriculum development X The training aims to create a cadre of master trainers in the region that can train other divisions. R6 8 ABC+ Early￾grade Learning Materials (EGL) Delivered to the School Orientation to parents regarding utilization of the EGL during the HPTA Meeting Instructions made by parents on how to use EGL Learners are very interested to read the EGL Cognitively aligned reading materials (the required difficulty level of the materials match the X The outcome indicated was not necessarily supported by the activities indicated. Recommendation: convert materials to courseware 39 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 1 IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Fully Agree Partially Agree Disagree Explanation/ Remarks Evaluators' Remark ability level of learners. Thus, learners interact with the materials) R6 9 ABC+ SRMs were given to learners Parents were oriented on how to use the ABC+ SRMs in school Parents/learning facilitators/guardia ns scheduled the reading session for their children Learners and parents together read the ABC+SRMs Educational pause for the family X This is part of the program design -- the HLP guides are meant to facilitate reading at home. R6 11 Materials were delivered/Levele d Reader books Teachers were oriented on how to use the materials Materials are given to children who are identified to be struggling Materials were utilized for struggling readers Developed precision in identifying specific learner support in reading X The current reading materials that have been distributed to the homes and classrooms are aligned with the learner's reading profile based on CRLA results. R6 13 Materials were delivered to the school and given to the teachers Teachers were being oriented on how to use te materials The materials were given to the pupils during read aloud session Learners enjoy reading the book Cognitively aligned reading materials X The activities listed are not quite substantial enough to contribute to the outcome. R6 14 The learners developed their love for reading by using the materials The font style was readable, and the pictures were attractive for the kids. The context and pictures of the materials were appropriate for the learners. The learners enjoy reading the material. Cognitively aligned reading materials X With the appropriate materials, learners' interest in reading will be piqued. R6 15 ABC+/ SRMs were delivered to children Parents were oriented on how to use ABC+ learning materials ParentsParents/ho me learning facilitators/guardia ns scheduled the reading time with their children Learners enjoy listening to stories read by their parents/ guardians and home learning facilitators Cognitively aligned reading materials X The teaching strategies to parents and HLP guides were aimed to encourage a love for reading at school and home among learners. If consistently done right, this contributes to the outcome USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 40 IR 1 IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Fully Agree Partially Agree Disagree Explanation/ Remarks Evaluators' Remark indicated. R6 16 Reading habits were done at home with the help of parents. SRMs were delivered and used by K to Grades 1-3 learners. Parents were oriented on the use of SRMs. The reading habit was practiced at home. Adaptation of ABC+ home learning strategies (Reading habit was practiced at home). X Parents were trained on the use of HLP guides to allow their kids to read at home. R6 17 Parents were sent consent that their children will be one of the participants in the conduct of Field Testing. -Parents were reminded that the conduct of this testing is to test only the effectiveness and appropriateness of the materials for children's level. The Grade 2 Learners were enjoying reading the materials since they used colored pictures. Field Testing of RM'S in Grade 2. Cognitively aligned reading materials X Field testing was done as part of the materials development process. R6 18 Decodable, leveled readers and picture books were introduced to parents and learners Parents were given help in utilizing the books Utilization of decodable, leveled reader and picture books Learners are enjoying reading and listening to story books Cognitively aligned reading materials X The teaching strategies to parents and HLP guides were aimed to encourage a love for reading at school and home among learners. If consistently done right, this contributes to the outcome indicated. R6 19 Story Books are given to the Learners Parents oriented by teachers on the use of the resources Parents interacted with the learners with the use of the Story Book Learners can relate what they read from the Story Book Cognitively aligned reading materials X Comprehension skills are considered in the book development. R6 20 ABC+SRMs will be reproduced Teachers conducted remedial reading Developed precision in X The home package that was distributed to the 41 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 1 IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Fully Agree Partially Agree Disagree Explanation/ Remarks Evaluators' Remark to have a 1:1 identifying specific learner support in reading learner's home was used for individualized learning, esp. as a reading remediation. R6 22 Materials were delivered Teachers were oriented on how to use the materials decodable books are an easy step￾by-step process of introducing letters for beginning readers/learners Decodable books are utilized for beginning reading learners Developed precision in identifying specific learner support in reading X Appropriate materials are given according to the reading profile of learners R6 23 Decodable leveled Readers and Picture books were introduced to learners and parents Parents were oriented on how to use and utilized the books Utilization of Levelled Reader and Picture Books Learners Enjoying Reading Books Cognitively aligned reading materials X With the appropriate materials, learners' interest in reading will be piqued. R6 24 Materials were delivered to the school and given to the teachers Teachers were being oriented on how to use te materials The materials were given to the pupils during read aloud session Learners especially those who are struggling with reading were able to love reading decodable reading books Cognitively aligned reading materials X With the appropriate materials, learners' interest in reading will be piqued. R6 30 Big books downloaded to the school Contextualizati on of the materials by the teachers during LAC Orientation to parents by the teachers A better understanding of the learners on the context of the stories as parents shared stories with them Adaptation of ABC+ home learning strategies (Reading habit was practiced at home). X Comprehension skills are considered in the book development. R6 33 Orientation on their Roles as HLPs Conduct Capacity Building for Parents Parents serve as HLPS Collaboration with Parents and Community Engagement The whole community approach to beginning reading X This is part of the program design -- the HLP guides are meant to facilitate reading at home. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 42 IR 1 IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Fully Agree Partially Agree Disagree Explanation/ Remarks Evaluators' Remark R6 34 Teachers identify the reading level of the learners teachers scheduled a remedial reading session for struggling learners Teachers use the ABC+/SRMs as the remedial reading material for his/her learner. Pupils love to read SRMs because the text and illustrations are simple and clear. Cognitively aligned reading materials X Learners enjoy reading because they were given reading materials appropriate to their reading level. BARMM 1 No response After attending the 5 days of comprehensive training, the 12 teachers gathered different teaching strategies and used them in teaching. Orientation of parents on how to use the SRMs The teacher can now easily identify the pupils' level of reading and strategies to be used for the facilitation of reading. Developed precision in identifying specific learner support in reading X They used the CRLA to determine the learner's reading profile. BARMM 2 No response The teachers shared their ideas and experiences during the seminar. Using teaching strategies such as SEL, GESI, LEA, GRR, etc. Teachers are fully capacitated in implementing ABC+ Activities. Increase teaching efficacy X The demo teaching as part of the training ensured that the teachers are capable of cascading the training to other teachers and the learners themselves. BARMM 3 Teacher to Teacher mentoring activities Provided various activities and technical assistance Enhanced reading program of the school Improved policy at the school level X The strategies and output during the training are a value-add to whatever program/s the school has. BARMM 4 Teachers' enhancement training Utilized/adapted learned strategies (BDA, GRR, etc.) Conducted Action Research based on the result of the Literacy Test Application of learned Knowledge and Skills Adaptation of ABC+ instructional strategies X Check-in/ monitoring sessions/surveys are done to ensure that the teachers apply the strategies they learned from the training. 43 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 1 IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Fully Agree Partially Agree Disagree Explanation/ Remarks Evaluators' Remark BARMM 5 No response Teachers applied different strategies using SEL and GESI, for example, EL Konin Boxes. Teachers applied Marungko Approach in teaching reading in connection with the SEL and GESI. Teachers are now capacitated to make materials for teaching and reading, for example, El Konin Boxes Marungko Approach. Increase teaching efficacy X This is part of their outputs during training. BARMM 6 No response Teachers assessed the learners by conducting reading, using the Assessment Tool from CRLA. Teachers identified the learners reading ability using the assessment tool from CRLA. More learners can read words, phrases, and sentences suited to their reading ability. Cognitively aligned reading materials X Through the CRLA tool, teachers know the reading profile of their students so that learners read according to their reading ability. BARMM 7 No response Training of Trainers on Early Literacy Strategies. Training of Teachers (K-3) on Early Literacy Strategies. (1) Effective reading strategies were applied by teachers in school. (2) Integration of SEL and GESI was done. Increase teaching efficacy X Part of the program design -- supported by concrete outputs from the training. BARMM 10 Teachers were sent to ABC+ training Teachers applied the reading strategies to learners according to their level Teachers monitored the reading progress of learners. Teachers acquired different reading strategies from ABC+ training Adaptation of ABC+ instructional strategies X Part of the program design and supported by check-in and monitoring. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 44 IR 2 IMPROVED ACCESS TO QUALITY MATERIALS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factors Fully Agree Partially Agree Disagree Explanation/ Remarks R5 1 Parents were engaged in encouraging the child to read more. Teachers use the decodable book for oral reading in class. My daughter learned to love reading Bicol because of the decodable reader book from ABC+. It has child-friendly words and short stories and illustrations. So now, no more crying while reading. ' Learners' increased love for reading child-friendly words, illustrations, parents' reading X Need to verify which learner is from what school. The learner received appropriate material based on her reading profile, thus the expected interest in reading. R5 2 Materials were properly distributed to our volunteers in the learning hub and parents. The Barangay council invited the parents how to use and to be understood properly by learners. The learners from our barangay easily understood every word they read and were happy to say that parents and barangay council volunteers had good results for our learners. Increased number of readers Access to materials, capacitated parents on materials used X Need to know which learners are from which school or community. The midline results will verify this. R5 3 TT1 and TT2 SEL and GESI inclusion in the SLACS and INSET Social Emotional Learning and Gender Equity and Social Inclusion were considered by the teachers in preparing the HLG and Learning Activity Sheet. GESI integration in materials development GESI inclusion in training X This is part of the teachers' output during the training, which is expected to be applied in their lessons. R5 4 Attendance to TT1 and TT2. Production of Home Learning Guides. HLP was able to guide the learners. Adaptation of ABC+ instructional strategies Capacitation through training X HLPs were given step-by￾step guides, including scripts and videos to facilitate this. R5 5 Evaluation of teacher-made materials for early-grades. Teachers became confident and used the gained knowledge in conducting training￾workshop. Increased confidence in teaching and Increased capacity in Capacitation through training X This is evidenced by their input to their SRM development process. 45 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 2 IMPROVED ACCESS TO QUALITY MATERIALS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factors Fully Agree Partially Agree Disagree Explanation/ Remarks -Enhanced/improved the process of SRMs development. materials development R5 6 Pupils were able to use LAS made by teachers who attended TT1 and TT2. Facilitated learning of pupils at home. Adaptation of ABC+ instructional strategies Capacitation through training X Supplemental activity sheets were given to learners to reinforce learning both in school and at home. R5 7 Pilot study reading assessment for Grades 3 and 4. Reading levels revealed supplementary reading materials were given to parents for pupil use. X Although assessment in school was intended for K-3, some anecdotes state that it's also being used on higher grade levels to determine the reading profile of students, thus providing appropriate reading materials for home use. R5 8 FB live workshop in using learning materials, and storybooks for grade 3. Masa madali maintindihan ng mga bata yung story; Nadevelop yung bonding namin ng mga bata na educational. Improved reading skills and Improved parent-child relationship Capacitation through orientation X Need to know which learners are from which school or community. The midline results will verify this. R5 9 ILT as a participant ILT as a Trainer Instructional Leaders Training-TOT participants became well-versed in using online platforms; Enhanced instructional leadership skills among school heads. Increased teaching efficiency and increased leadership capacity Capacitation through training X Schools Heads were exposed to and trained to adapt to technology, esp. during the pandemic. They were expected to cascade their learnings to their school and other teachers. R5 10 Orientation was conducted to the parents on the use of books, the The books were shared by parents with their children and other Books were received by the learners through their parents. Higher level of interest of parents and children in reading books/stories in pairs and by groups. Adaptation of ABC+ instructional strategies Access to materials, capacitated parents on materials used X This is part of the strategies taught to parents in reinforcing reading at home. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 46 IR 2 IMPROVED ACCESS TO QUALITY MATERIALS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factors Fully Agree Partially Agree Disagree Explanation/ Remarks responsibilities of parents, and the content of SRMs. children. A better way for parents to teach reading is through the SRMs. R5 11 Primary teachers attended the RTOT to enhance teaching reading strategies. Several teachers serve as facilitators in the RTOT in conducting the training for teachers. Learn the proper way to conduct read-aloud lessons. Teachers are now equipped with the necessary skills in teaching reading, able to deliver read-aloud sessions to the learners. Increased teaching capacity Capacitation through orientation X The teach-back or demo teaching was done to test their application of knowledge learned during the workshop. R5 12 Teacher writers undergo intensive training in crafting learning materials. Produce quality￾assured learning materials to support the teaching￾learning process. ABC+ SRMS was delivered to the school. Lighten the daily activities of teachers in creating meaningful lessons. Increased teaching efficacy Capacitation through orientation X Teachers ensure that appropriate learning materials are being given and used by their learners. R5 13 Conduct of regional workshop on learning resource evaluation for central Bicol and Minasbate. July 18-22, 2022. Masbate teacher￾writers include evaluation in the material development. Improved materials development process Capacitation through orientation X Teachers evaluated their locally-developed learning materials R5 14 Discussion with the (PSB) Personnel School Board about the locally developed materials and give a ceiling point. Inclusion of locally developed materials in criterion A (CIM Curriculum Instruction Materials) for teacher growth and development promotion. Innovation in Teacher Professional Development Awareness through training X Need to know which school. Although they are expected to include this in their SIP, a verification must still be conducted that this has indeed transpired. R5 15 Attended capacity-building Develop QADEGRMs Join the search and promotion Promotion of teacher￾writers to master Innovation in teacher Awareness through X Although this training is a good training ground for 47 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 2 IMPROVED ACCESS TO QUALITY MATERIALS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factors Fully Agree Partially Agree Disagree Explanation/ Remarks activities on EGRMs development conducted by the ABC+ project. for Masbateño teachers and learners. for teachers and MTs. teachers and outstanding teachers/Master teachers. professional development training promotion, this needs verification in terms of which school or division does this. R5 16 Constant follow-up on the use of materials (online and modular) and kumustahan session (2020) Held an online orientation (FB live) on the use of the materials. Receive ABC+ materials and give them to the kids. Students and parents enjoyed reading and were motivated to read more. Increased interest in reading by learners and Increased interest in reading by parents Access to materials, capacitated parents on materials used X Need to verify which students and parents are from which school since seems to be just an anecdotal R5 18 Teachers were trained in materials development. The school training workshop was conducted. SH gives TA on development of classroom Ims IMs preparation made more meaningful Teachers' appreciation of the materials development process Awareness through training X Strategies shared during the training are supposed to reflect+I60 direct application through the development of IMs for classrooms, which in this case seems true. R5 19 Provision of SRMs from ABC Parents' orientation on SRMs utilization. Take-home readings and home-based learning packages are provided to the learners. Improved reading habits of learners Improved reading habits of learners Access to materials, capacitated parents on materials used X Need to know which learners are from which school or community. The midline results will verify this. R5 20 ABC+ materials were distributed to primary Teachers. Primary teachers utilized the ABC+ Materials in teaching Reading Comprehension to both the Instructional and Pupils had more practice reading the ABC+ materials through the guidance of parents at home. Both instructional and independent readers are excited to gain more learning insights through reading. Increased interest in reading by learners Access to materials, capacitated parents on materials used X Need to know which learners are from which school or community. The midline results will verify this. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 48 IR 2 IMPROVED ACCESS TO QUALITY MATERIALS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factors Fully Agree Partially Agree Disagree Explanation/ Remarks Independent Readers in English and Filipino. R5 21 ABC+ SRMs were read by the pupils during the vacant time. The teacher used the books in their oral reading. ABC+ SRMs were borrowed by the parents to read with their children at home. The materials engaged students to be more inclined to read and boosted their interest in making connections with print materials. Increased interest in reading by learners Access to materials, parents' support in reading practice X Exposing the learners to printed reading materials in both school and at home will increase their connection with reading. R5 22 Teachers were also trained in the school in terms of the development of learning materials. Teachers were also able to train other teachers in the school to write storybooks, especially in the mother tongue. Instructional material development has become more significant. Teachers' Appreciation of materials Development Process Cascaded training by participants and Awareness through training X Ownership of the development process is one objective of the project, thus involving the teacher-writer from the community itself. R5 23 Parents are very active in giving follow-up to their children at home when it comes to teaching reading. More children enjoy the materials developed by the team. Minasbate stories developed were more interesting for the children to read since they can relate to the language used. Learners enjoy listening to stories read by their parents. Increased interest in reading by learners Access to materials, parents' support in reading practice X Need to know which learners are from which school or community. Seems an anecdotal R5 24 School heads are giving technical assistance not only to the teachers but as well as the students and parents of the community. Capacity building for teachers and parents of the community. Capacitation through training and close monitoring of leaders X The two activities indicated are part of the training design. R5 25 Grades 1 to 3 Kami po ay Ako po ay nag Narefresh at nadagdagan Increased Cascaded X The training was designed to 49 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 2 IMPROVED ACCESS TO QUALITY MATERIALS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factors Fully Agree Partially Agree Disagree Explanation/ Remarks teachers were trained in teaching beginning reading and mother tongue, Filipino and English, and Basics in Numeracy naging trainers na sa division level attend ng training of Trainers sa Regional level ang kaalaman ng mga guro sa pagtuturo ng beginning reading at basics ng arithmetic at numeracy. teaching capacity training and capacitation through training share added knowledge and strategies in teaching literacy and maths. R5 27 School heads are giving technical assistance not only to the teachers but as well as the students and parents of the community. Capacity building for teachers and parents of the community. Capacitation through training X REPEAT OF # 24. The 2 activities indicated are part of the training design. R5 28 Grades 1 to 3 teachers were trained in teaching beginning reading and mother tongue, Filipino and English, and Basics in Numeracy Kami po ay naging trainers na sa division level Ako po ay nag attend ng training of Trainers sa Regional level Narefresh at nadagdagan ang kaalaman ng mga guro sa pagtuturo ng beginning reading at basics ng arithmetic at numeracy. Increased teaching capacity Capacitation through training X REPEAT OF #25. The training was designed to share added knowledge and strategies in teaching literacy and maths R5 30 First week of implementation medyo struggle mga teachers kaya inaassist pa Pag nagtatanong po ako sa mga teachers, nakakabasa na Nakakatuwa na naririnig na malakas na nagbabasa ang mga grade 2 at 3 Nakikita na ang progress sa mga bata kahit five weeks pa lang. Kasi ito ay total ng 8 weeks. Iyong nakakarecover, Increased reading skills of learners Capacitation through training, support from mentors X Although this is anecdotal, the pre-test and formative assessment in place can attest to this. This needs further verification of which USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 50 IR 2 IMPROVED ACCESS TO QUALITY MATERIALS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factors Fully Agree Partially Agree Disagree Explanation/ Remarks natin sila, provide technical assistance learners. nakakapagbasa na. Nakakabasa na iyong umulit ng grade 2 modular instruction. learners are from which school though. R5 31 The pupils were encouraged to read ABC+ SRMs materials. Teachers used ABC+ SRM materials in teaching. The parents also cooperated with the teachers in helping their children to learn. Motivated pupils in reading especially story books. Increased interest in reading by learners Access to materials, teachers' use of materials, and parents' support in reading practice X Strategies taught during training had the objective of increasing motivation once all learners are exposed to reading. R5 32 Based on the assessment conducted by the advisers, there was a lot of struggling readers in our school. Then here comes the book from the ABC+ (Decodable reader). Advisers checked which among those delivered books will suit the learner's needs. The reading coordinator of the school together with our school head facilitated a program for our struggling leaders. Students were engaged in the learning process. Parents and teachers are engaged in the teaching process. Increased interest in reading by learners, involvement of parents in teaching and learning Learner assessment, access to materials, intervention design X This is how the training is designed, and it will be more effective once both teachers and parents are involved in teaching the learners how to read. R5 33 Introduced reading sessions on Fridays Oriented and Capacitated Parents through PLAC on the conduct of oral reading as learning partners. Received technical assistance from MTs, HS, EPS Learners are now fond of listening to storybooks read by their parents. Increased interest in reading by learners Capacitated parents, technical assistance received from leaders X The activities are initiatives and direct applications of what they learned from the training. This however further verification which learners and which school. R5 34 Each classroom had put up reading corners displaying teacher-made Scheduled reading hour Culminating reading activity in every class. Learners have developed a genuine love of reading. Increased interest in reading by learners Access to materials, reading instruction X The activities are initiatives and direct applications of what they learned from the training. This however further verification which 51 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 2 IMPROVED ACCESS TO QUALITY MATERIALS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factors Fully Agree Partially Agree Disagree Explanation/ Remarks reading materials and ABC+ books. learners and which school. R5 35 Used different reading strategies in reading Utilization of the arts of questioning with an emphasis on HOTS and gradual psychological unfolding (GPU) Use of worksheets, LAS, and OSL. Learners have improved reading comprehension levels. Increased reading skills of learners Reading instruction, instructional materials X Need to know which learners are from which school or community. The midline results will verify this. R5 36 ABC+ SRMs, Decodable, and labeled reader books were distributed per class from grades 1-3. Teachers distributed the books to the learners. Teachers used the books as instructional and supplementary materials for reading and listening activities. (8-week curriculum) Learners are fond of reading and listening to stories found in the SRMs. Increased interest in reading by learners Access to materials, reading instruction X All 3 activities are part of the design, and if all 3 are present, the expected outcome is increased love for reading of students R5 37 Teachers have been identified as core writers of worksheets and SRMs. They attended the training workshop conducted by the ABC+ Materials were prepared, evaluated, and distributed throughout the region. Learners enjoyed answering the worksheets Increased study skills of learners Capacitation through training and workshop, access to materials, use of materials X All 3 activities are part of the program design. The outcome needs further verification though. R5 38 Teachers allotted time for project DEAR. Teachers encouraged learners to read and retell the story. Utilization of different reading materials. Learners have improved the GLR-CT. Increased interest in reading by learners Reading program, reading instruction, use of materials X Need to know which learners are from which school or community. The midline results will verify this. R6 1 Conducted survey of training of Conducted training on strategies for Coordinate with the HRDD on the MandE results on Improved competence and readiness of teachers in the delivery Increased teaching capacity Capacitation through training, close X The strategies taught during the workshop and the training design itself are USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 52 IR 2 IMPROVED ACCESS TO QUALITY MATERIALS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factors Fully Agree Partially Agree Disagree Explanation/ Remarks Early-grades Teacher language learning and transition: Improving early￾grade literacy in school and at home the knowledge gained in the training conducted. of instructions coordination, monitoring supposed to make the teachers ready for the delivery of instructions. R6 2 Application of the teaching strategies learned from the training. Equip teachers on teaching early-grades literacy which leads to improved self￾confidence in the delivery of the lesson. Increased teaching capacity Capacitation through training X The demo teaching during the workshop is meant to see improvement in the teacher's confidence in delivering the lesson. R6 4 Enhanced/impro ved teaching skills especially in teaching early-grade literacy Teachers attended training The assessment shows improvement in the reading profile of the learners Increased reading skills of learners Capacitation through training X Need further verification-- which learners from which school and division R6 6 Conducted Teachers Training for teachers (TTI) The practices of the teachers are now more aligned in teaching literacy in the classroom. Increased teaching capacity Capacitation through training X Not many significant activities are listed to validate the credibility of the outcome. R6 7 Teachers attended a series of ABC+ training sessions. Teachers were refreshed on several techniques to be used in teaching reading using EGMs. Teachers applied the techniques learned in the teaching-learning process, especially in literacy development. More learners were helped in the aspect of reading and comprehension and their love of reading is being developed. Increases reading skills and Increased interest in reading by learners Capacitation through training, improved reading instruction X The program design is meant to directly impact the learners of the teachers who attended the training. R6 10 Attended the training of trainers at the Regional level. Joined the workshop of ABC+ within our Division of Escalante City. The teachers used the strategies learned in the training or workshop. The teachers gained additional knowledge, especially in teaching literacy and numeracy. Increased teaching capacity capacitation through training, improved reading X This is a default outcome of any training. Part of the program design. 53 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 2 IMPROVED ACCESS TO QUALITY MATERIALS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factors Fully Agree Partially Agree Disagree Explanation/ Remarks instruction R6 12 Teachers closely monitored learners' progress through home visitation, especially during the pandemic. Conducted orientation to teachers who were not participants of TT1. Teachers' better understanding of how to help learners read. Increased teaching capacity Teachers' close monitoring of learners, capacitation through training X Part of the program design -- supported by concrete outputs from the training. R6 13 Teachers were taught how to make/develop stories according to different stages suited to the learners' reading level. Teachers were able to develop stories using the Blooms App. Teachers were motivated to develop more stories of different stages for learners' use. Teacher trainees realized that developing storybook for children has stages to follow according to the reading level of the learners. Teachers' appreciation of the materials development process Capacitation and awareness through training X Part of the program design -- supported by concrete outputs from the training. R6 16 Teachers applied strategies they learned in the training like SEL. Teachers made flashcards, and charts to supplement the learners' reading materials to enhance the phonics and word recognition of learners. Teachers use flashcards, and charts in his/her reading session, and teachers consider his/her learners as unique individuals. Teachers used SEL to know the behavior of every learner to gather learners' formative feedback. Teachers' behavior towards teaching reading was evident after attending the training. Increased teaching capacity Capacitation and awareness through training X Supported by concrete outputs during the training USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 54 IR 2 IMPROVED ACCESS TO QUALITY MATERIALS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factors Fully Agree Partially Agree Disagree Explanation/ Remarks R6 18 During virtual training, besides gaining knowledge and skills, we also widened our world by meeting new acquaintances which developed into friendships. We, teachers, enjoy the comfortable venues during face-to-face training set-ups. Virtual friendship went strong as trainees see each other. Cooperation, collaboration, and sharing of good practices were manifested. Innovation in teaching Formed bonds or relationships, comfortable training venues X Part of the program design -- collaboration and sharing of best practices R6 19 Teachers' attendance at training on literacy strategies teachers harvested additional SRMs from the LR Portal Utilization of SRMs from the LR Portal Improved learners' engagement in reading activities Increased interest in reading by learners capacitation through training, access to materials, and use of materials X Need further verification. The outcome indicated was not necessarily supported by the activities indicated. R6 21 Demonstration of reading strategies through SLAC. Recalling salient points of TT1 via SLAC> Strengthen reading strategies. Increased teaching capacity capacitation through training X Part of the program design-- supported by concrete outputs during the training BARMM 1 No response Conduct pre and post￾assessment using the Comprehensive Rapid Literacy Assessment (CRLA) Tool Assessment skills of parents in using SRMs More learners can easily identify sounds; read the CVC pattern and phrases. Increased reading skills of learners Diagnosing learners' needs through assessment, capacitation of parents X Need further verification -- which learner from school and division. Midline results will verify this. BARMM 2 No response No response Implementation/U se of SRMs inside the house/home. The learners are now fond of reading books, specifically, the books that are distributed both for the full Increased interest in reading by learners access to materials X Appropriate materials are given according to the reading profile of learners, thus making it more interesting for them 55 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 2 IMPROVED ACCESS TO QUALITY MATERIALS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factors Fully Agree Partially Agree Disagree Explanation/ Remarks refresher, moderate refresher, and grade￾ready pupils because of the quality of the materials. BARMM 3 Exercise on CVC Pattern (Literacy domain conducted by ABC+) More activities on phonemic awareness Construction of simple sentences and reduced the number of slow readers Learners' reading skills are improved. Improved reading skills of learners Reading instruction X Need further verification -- which learner from school and division. Midline results will verify this. BARMM 4 No response The teacher gave short story/reading materials to the parents The parent reads the story to the child The learners are more interested in storytelling by their parents Increased interest in reading by learners Access to materials, parents' support in reading practice X Need further verification -- which learner from school and division. BARMM 5 No response The CRLA reading materials made by the teachers were distributed to the learners. Learners are confident to read the materials given to them. Learners are now fond of reading using the books given by the ABC+ Increased interest in reading by learners Teachers created materials, access to materials X Appropriate materials are given according to the reading profile of learners, thus making it more interesting for them. BARMM 6 No response The parents were oriented on how to conduct the CRLA and why they become a partner of teachers in assessing the learners. The parents were always teaching their children at home. It is now easy for parents to teach their children using the given reading materials by the teachers. Parental support in reading Capacitation of parents, parents' support in reading, access to materials, use of materials X Parents are provided with HLP guides that they can use at home. BARMM 7 Translated English stories Received SRMs for Grade 2 and Teachers developed Teachers are more effective in conducting Increased teaching Teachers created X With the teaching demo conducted as part of the USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 56 IR 2 IMPROVED ACCESS TO QUALITY MATERIALS CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factors Fully Agree Partially Agree Disagree Explanation/ Remarks to Magindanawn (online). 3 students (Filipino and English). decodable materials and leveled reading materials in Magindanawn using Blooms App. reading sessions. Reading sessions become more enjoyable to learners. capacity materials, access to materials training, the confidence and readiness of the teachers when cascading the strategies are already in place. BARMM 8 Video lessons in Magindanawn in support of BRP (Beginning Reading Program) of ABC+. Oriented parents on the use of the SRMs. Conducting reading sessions with kids becomes easy because they are oriented with the materials and availability of materials. Increased teaching efficacy Capacitation through training, parent involvement in materials development X Supplemental guides, such as HLP guides and soon BRP videos, are given to parents to help facilitate reading at home BARMM 10 Distributed ABC+ reading materials to parents Oriented parents on the use of ABC+ reading materials and how to teach reading. Used ABC+ materials during sessions More learners can produce letter sounds and read syllables and simple words Improved reading skills of learners Access to materials, capacitated parents on materials use, use of materials X Need further verification -- which learner from school and division. Midline results will verify this. BARMM 11 No response Used the CRLA Assessment Tool to determine learners' reading ability Used as pre-test and post-test assessments CRLA Assessment Results helped the teachers to choose the reading materials to be used per learners' reading level Increased teaching efficacy Diagnosing learners' needs through assessment X CRLA was used at the beginning of the school year to assess the learners of their reading profile 57 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 3 IMPROVED EDUCATION SYSTEM CAPACITY AND COMMITMENT TO MANAGE AND IMPLEMENT INTERVENTIONS SUPPORTING EFFECTIVE EARLY-GRADE INSTRUCTION CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factor Fully Agree Partially Agree Disagree Explanation/ Remarks R5 1 Orientation on how to conduct the Comprehensive Rapid Literacy Assessment (CRLA) Radiated through division training for K-3 Utilized by all Grades 1-3 Identified the expected level of the child to give the appropriate intervention. Improved grade level Capacitation through training and technical support on the use of CRLA X CRLA was done at the Beginning of the School year in all regions of Bicol R5 2 Intensified partnerships with different sectors like Alumni Implementation of School￾initiated project SAKKUM (Susihon kag Asikasuhon an Kabataan sa Komunidad agud Urupod na Magbasa) Establishment of School/Classroom LR Nook and Purok Reading Centers for reading intervention sessions. Increased number of books and material donors. Increased number of reading materials Additional learning materials from ABC+ 1:1 ratio X Initiatives from the community and private sectors facilitated the faster distribution of ABC+ - developed materials R5 3 Provisions of donations like reading materials, snacks, and school supplies by various stakeholders. Conduct school-based SMK Awarding Day to recognize the support and effort of the teachers, pupils, parents, and other stakeholders. Involvement of the stakeholders particularly BLGU and the planning, implementation, monitoring, and evaluation phases of reading initiatives entitled Project SMK (Syllable Mastery in Katon Way). Love for reading was intensified among the pupils, parents, teachers, head, and the community. Improved home￾community partnership Strengthen HLP due to school closure because of pandemic X Need to know which community, learners, parents, and teachers. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 58 IR 3 IMPROVED EDUCATION SYSTEM CAPACITY AND COMMITMENT TO MANAGE AND IMPLEMENT INTERVENTIONS SUPPORTING EFFECTIVE EARLY-GRADE INSTRUCTION CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factor Fully Agree Partially Agree Disagree Explanation/ Remarks R5 4 Brgy donated books and school reading materials in school. ABC+ became a partner with the community ABC+ became a partner with the community Engaging parents/community through orientation and home partner sessions strengthened home￾school relationships which resulted in collaboration in developing reading habits among learners. Improved home￾community collaboration Access to materials, parents' support in reading practice, and teacher monitoring practices X This is part of the output of the PCE work under the IR3 strand. R5 6 Grades 1 to 3 teachers attended the training for the eight-week LRP Teachers and school heads oriented the parents about the regional policy regarding the 8-week recovery program through a meeting. It was explained to the parents the need to recover their children when it comes to literacy and numeracy and they were also informed about how this will be done. Parents and other members of the community become more aware of the need to help their children learn to read by following up at home. Improved home￾community partnership; Awareness and capacitation of parents, parents and community support in reading, use of materials X Creating awareness as a foundational goal is an important aspect of the program and is intensified by collaboration among schools, parents, and the community. 59 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 3 IMPROVED EDUCATION SYSTEM CAPACITY AND COMMITMENT TO MANAGE AND IMPLEMENT INTERVENTIONS SUPPORTING EFFECTIVE EARLY-GRADE INSTRUCTION CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factor Fully Agree Partially Agree Disagree Explanation/ Remarks R5 8 Teachers of the advanced class (Special Science ES) attended the CRLA training Trained teachers initiated to donate books on beginning reading and educational supplies to identified indigent slow and non-readers through HRPTA The support system of stakeholders; Brigada Eskwela/Pagbasa The enhanced support system of stakeholders (i.e catchment barangays of the school, parents, alumni, private institutions, NGOs, LGUs) Increased stakeholders' support system Support and involvement of stakeholders X The activities indicated concrete collaboration efforts by schools, teachers, parents, and other stakeholders in the community. Thus, creating a solid support system to support student learning. R5 10 Programs and projects of the school were actively participated in and attended by the officials Recognition was given to the active stakeholders during the PRAISE Awards Night for the teachers, non￾teaching personnel, and all the stakeholders Strengthen partnership with the BLGU Improved and strengthen the partnership Support and commitment of stakeholders X Recognition of active stakeholders can be a clear indicator of strengthened partnership--not just mere participation in activities. R5 12 The school is a recipient of learning materials and digital tools from USAID. The school became the benchmark of best practices of other schools. Increased literacy benchmarking Capacitation through training and commitment to supporting early-grade instruction X The ABC+ interventions are meant to be benchmarks for other schools in terms of improving literacy among their learners. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 60 IR 3 IMPROVED EDUCATION SYSTEM CAPACITY AND COMMITMENT TO MANAGE AND IMPLEMENT INTERVENTIONS SUPPORTING EFFECTIVE EARLY-GRADE INSTRUCTION CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factor Fully Agree Partially Agree Disagree Explanation/ Remarks R5 13 Intensified conduct of School LAC sessions regarding Research and Innovations The district conducted Best Practices and Innovations Participated in the Division Summit on Best Practices and Innovations. Influenced schools to innovate practices. Increased and sustain literacy innovation practices Capacitation through training and commitment to supporting early-grade instruction X Thinking of innovations on the part of the school is part of the program design. R5 15 Functionalized Brigada Pagbasa Program Invited storytellers from stakeholders during culminating Strengthen parents' participation as home-learning partners. Developed partnerships with stakeholders Increased stakeholders partnership Support and commitment of stakeholders X Mobilizing community partners is a good indicator of a strong partnership with stakeholders. R5 19 Sponsored ABC+ training/ capability for teachers, and webinars with free load allowances. Extended resources shared riso machine and bond papers to aid the printing of modules in the pandemic times. Development of video production of ABC+ learning pocket using Kotobe in creating stories. Development of the projects/programs of the Division on Literacy and Numeracy Improved literacy program X The activities listed show the sustainability of the program in the schools R5 20 Organize district inspectorate teams/ delivery teams Connected with Rotary Club, PNP, and LGU partners in the delivery of EGRMs to the school. Delivered EGRMs to schools through Rotary Club and PNP Establishment of linkages and partnership Increased partnership establishment Support and commitment of stakeholders X Involving stakeholders and making them co-own the program is a good indicator of sustainability. 61 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV IR 3 IMPROVED EDUCATION SYSTEM CAPACITY AND COMMITMENT TO MANAGE AND IMPLEMENT INTERVENTIONS SUPPORTING EFFECTIVE EARLY-GRADE INSTRUCTION CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factor Fully Agree Partially Agree Disagree Explanation/ Remarks R5 23 Conducted program launching of sustainability of project parent, and teacher as a response to ABC+ PIP (Project Initiative Plan) for 5 years with the teachers, home learning partners, and LGU. Received voluntary donations (materials from the member of the municipal council) Organize the different groups according to the catchment areas and target skills; Scheduling external stakeholders specifically from LGU as volunteer tutors. Distribution of materials to each group. Implementation of the program as scheduled. Sustainability/Developm ent of the program with enhanced participation of the parents, HLP, learners, and LGU. Improved and sustain HLP Support and commitment of stakeholders, parents' support in reading practice, and teacher monitoring practices X Involving stakeholders and making them co-own the program is a good indicator of sustainability. BARMM 1 No response Use of ABC+ and SRMs during remedial class. Implementation of MBHTE Guidelines in the project 5Bs. ABC+ supported the MBHTE's 5Bs (Bawat Bata sa Bangsamoro Bumabasa at Bumibilang) project improved the MBHTE support system Support and commitment and monitoring of stakeholders (MBHTE, DepEd, and ABC+) X Through training and EGRMs USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 62 IR 3 IMPROVED EDUCATION SYSTEM CAPACITY AND COMMITMENT TO MANAGE AND IMPLEMENT INTERVENTIONS SUPPORTING EFFECTIVE EARLY-GRADE INSTRUCTION CODE ACTIVITIES OUTCOME EXPERT'S REMARKS Region and Outcome Number Activity 1 Activity 2 Activity 3 Code Explanatory Factor Fully Agree Partially Agree Disagree Explanation/ Remarks BARMM 3 Provided technical assistance to improve skills Monitored and reported practices to enhance skills Instructional supervision to follow-up ABC+ activities. Sustainability/developm ent of teaching strategies learned from ABC+ Improved teaching strategies Capacitation through training X Check-in and monitoring are done to ensure sustainability BARMM 4 No response The school conducted a workshop for teachers to strengthen the 5Bs program. The school conducted an orientation for the implementation of 5Bs. The MBHTE formulated a program named 5Bs and the ABC+ program contributed to it Improved ABC+ and MBHTE literacy program Support DepEd policy and ABC+ program X Through technical assistance and training BARMM 6 No response No response Integrated SEL and GESI in Teaching Reading ABC+ supported the MBHTE's 5Bs (Bawat Bata sa Bangsamoro Bumabasa at Bumibilang) project Improved ABC+ GESI in MBHTE ; Support DepEd policy and ABC+ program X Not enough significant activities are listed to validate the credibility of the outcome. 63 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 8. LEVEL 1 MOST SIGNIFICANT STORY14 RESPONDENT NUMBER SCHOOL DISTRICT DIVISION REGION DATE OF RECORDING 1 Padang Elementary School 10 Legazpi City V 10/03/2022 2 Sto. Domingo Central School Sto. Domingo Albay V 10/03/2022 3 No Response No Response Legazpi City V 10/03/2022 4 Albay Central School 4 Legazpi City V 10/03/2022 5 Bagumbayan Central School 3 Legazpi City V 10/03/2022 6 No Response No Response Masbate V 10/03/2022 7 Calayucay Elementary School Sto. Domingo Albay V 10/03/2022 8 No Response 2 Legazpi City V 10/03/2022 9 Albay Central School 4 Legazpi City V 10/03/2022 10 Albay Central School 4 Legazpi City V 10/03/2022 11 Bagumbayan Central School 3 Legazpi City V 10/03/2022 12 Calayucay Elementary School Sto. Domingo Albay V 10/03/2022 13 Bagumbayan Central School 3 Legazpi City V 10/03/2022 14 San Isidro Elementary School Central Iriga City V 09/30/2022 15 San Isidro Elementary School Central Iriga City V 09/30/2022 16 Calongay Elementary School Pilar 4 Sorsogon Province V 09/30/2022 17 No Response No Response Masbate V 09/30/2022 18 San Isidro Elementary School Central Iriga City V 09/30/2022 19 San Isidro Elementary School Central Iriga City V 09/30/2022 20 San Isidro Elementary School Central Iriga City V 09/30/2022 21 San Isidro Elementary School Central Iriga City V 09/30/2022 22 Calongay Elementary School Pilar 4 Sorsogon Province V 09/30/2022 23 Escalante Central Elementary School No Response No Response VI 10/07/2022 24 Escalante Central Elementary School No Response No Response VI 10/07/2022 25 Escalante Central Elementary School No Response No Response VI 10/07/2022 26 No Response No Response No Response VI 10/07/2022 27 No Response No Response No Response VI 10/07/2022 28 No Response IV Iloilo City VI 10/06/2022 29 Jaro 1 Elementary School No Response Iloilo City VI 10/06/2022 30 Iloilo Central Elementary School I - City Proper Iloilo City VI 10/06/2022 31 Arevalo Elementary School IX-AREVALO Iloilo City VI 10/06/2022 32 No Response II Iloilo City VI 10/06/2022 14 No Response data are portions left by the respondent as they submit their most significant story USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 64 33 No Response No Response SDO- Iloilo City VI 10/06/2022 34 No Response I-ILOILO CITY Iloilo City VI 10/06/2022 35 Arevalo Elementary School IX Iloilo City VI 10/06/2022 36 A. Montes 1 Elementary School II Iloilo City VI 10/06/2022 37 ICES I Iloilo City VI 10/06/2022 38 No Response IX Iloilo City VI 10/06/2022 39 No Response IX Iloilo City VI 10/06/2022 40 No Response IX Iloilo City VI 10/06/2022 41 No Response VI Iloilo City VI 10/06/2022 42 No Response No Response No Response VI 10/06/2022 43 No Response No Response CLMD 10/06/2022 44 No Response No Response No Response VI 10/06/2022 45 No Response No Response No Response VI 10/06/2022 46 No Response No Response Education Support Services VI 10/06/2022 47 Iloilo Central Elementary School I SDO-Iloilo City VI 10/06/2022 48 Bubong Elementary School VI Cotabato City BARMM 10/02/2022 49 Bubong Elementary School VI Cotabato City BARMM 10/02/2022 50 Bubong Elementary School VI Cotabato City BARMM 10/02/2022 51 Bubong Elementary School VI Cotabato City BARMM 10/02/2022 52 Mohammad Integrated School VII Cotabato City BARMM 10/02/2022 53 Mohammad Integrated School VII Cotabato City BARMM 10/02/2022 54 Mohammad Integrated School VII Cotabato City BARMM 10/02/2022 55 Pagalamatan Elementary School VI Cotabato City BARMM 10/02/2022 56 Pagalamatan Elementary School VI Cotabato City BARMM 10/02/2022 57 Pagalamatan Elementary School VI Cotabato City BARMM 10/02/2022 58 Pagalamatan Elementary School VI Cotabato City BARMM 10/02/2022 59 Tamontaka Central School VI Cotabato City BARMM 10/02/2022 60 Tamontaka Central School VI Cotabato City BARMM 10/02/2022 61 Tamontaka Central School VI Cotabato City BARMM 10/02/2022 62 Tamontaka Central School VI Cotabato City BARMM 10/02/2022 63 Usman Baunga Elementary School VI SDOCC BARMM 10/02/2022 64 Usman Baunga Elementary School VI Cotabato City BARMM 10/02/2022 65 Usman Baunga Elementary School VI Cotabato City BARMM 10/02/2022 66 Usman Baunga Elementary School VI Cotabato City BARMM 10/02/2022 40 No Response IX Iloilo City VI 10/06/2022 41 No Response VI Iloilo City VI 10/06/2022 42 No Response No Response No Response VI 10/06/2022 43 No Response No Response CLMD 10/06/2022 65 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S 1 No Response I am a school head of one of the schools in the Legazpi City Division for seven years. As a new school head, the training provided by ABC+ gives me a wide perspective on how to handle, manage, and delivers learning to our pupils during the pandemic. I trained for teachers training on teaching strategies in reading 2020 and teachers training 2 on the use of supplementary reading materials in the early grade for schools and community and instructional leaders’ training for school heads and supervisors, which is why it gives a deeper understanding and knowledge on ABC+ program and activities. Before the ABC+ training, Social and Emotional Learning (SEL) and Gender Equity and Social Inclusiveness (GESI) were not considered in choosing SRMs in providing technical assistance to teachers, especially on how they are integrated into the lesson. But after the ABC+ SEL and GESI were included not only in learners but also in dealing with teachers, parents, and the community. I became more acquainted that social and emotional learning is not only applicable to learners but also in dealing with adult learners. Teachers also take into consideration the theme of the story that SEL and GESI must be included in the story read pupils also became aware of how to respect children of different races, colors, religions, and etcetera. Inclusion of this I can say that we produce learners who have respect for others. This is the lifelong learning I gained from ABC+. No Response No Response No Response 15 Tell me how you learned about [innovation] and got involved with [innovation]? 16 From your point of view, describe a story that best illustrates the most significant change that you had experienced or observed as a result of [innovation] being offered in the school or in the community. 17 Give your story a title. 18 Why is this story significant for you? 19 How has the [innovation] contributed to the story’s significance? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 66 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S 2 I learned ABC+ activities just last July 2022 when [name], Albay Div. EPS for English and MTB informed me that ABC+ will deliver books to school so I interviewed teachers regarding activities done by ABC+. I got interested because of the things I learned from the said interviews. I started asking teachers how they used the materials from ABC+ and requested dialogs with teachers on how they can maximize the use of the said learning packages. I am a school head for 23 years, but I experienced ABC+ activities just for 2 months a significant change I experienced happened last August 22, 2022, when the 3 directors from ABC+ visited Sto. Domingo CS to distribute books to our learners. The said activity made our learners very happy because they received books and it made our learners more interested in learning. Teachers appreciated the leveled reading materials because they are using them to help our struggling readers in school. It is very appropriate for our learners' needs. The grades 1 to 3 learners who are also having the eight-week learning recovery curriculum are helped by the said materials. Teachers informed me that our learners have improved reading habits because they are so eager to read starting with sounds, especially with Grade 1 pupils. The learners started to borrow different books from the teachers. ABC+ in Sto. Domingo CS The story is significant to me because even if I had experienced ABC+ activities for just a short time I noted improvement in our learners, teachers, and parents. The innovation contributed much to the pupils' and teachers' recovery plans. 3 No Response I am the Education Program Supervisor in the Learning Resource Management System (LRMS) of SDO Legazpi City. Being the LR-EPS, I am expected to deliver my job excellently in line with the development of learning resources which I found difficult during my first four Thank You For the Help! No Response No Response 67 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S years in the service. The year 2019 if I remember it right when I first met ABC+. I attended a workshop on the development of early-grades materials initiated by Region 5 in partnership with ABC+ and USAID. I served as a writer then, and months and months passed several training-workshop conducted by ABC+ where I was involved not only as a writer but as an evaluator of teacher’s made SRMs as well. I experienced how ABC+ managed a workshop and writing shop that helped me most in my profession now. I can consider what I learned today and how I served Deped Legazpi as a good evaluator of learning material, ABC+ is part of it. I learned the process to develop quality reading materials that my co-worker in Deped is also learning from me. I am very grateful that in every activity ABC+ is conducting in the region. I am always part of it if not all. I will be forever grateful. Kudos to all the personnel of Team RV-ABC+. Thank you for the help. 4 __Through a memo. I am a school head of the biggest elementary school in the School Division of Legazpi City. It was in the year of the pandemic, that was June 2020 when I first learned about ABC+. I was informed by the Chief of the Curriculum Implementation Division that our school will pilot the same program. From there, I attended training and orientation and I became the resource speaker of Teachers’ Training 1 and 2. The teachers were also trained and the parents as Home partners. The training and experiences that I gained in this program helped me a lot. First I am refreshed of the 14 domains of literacy and strategies in reading. I was able to regain my confidence as a speaker trainer. I could deliver my topic well because of the various training on the Training of Trainers Conducted. Aside from this, my ICT competence was enhanced. Imagine I have to present slides in Zoom (meeting platform). Google Docs, google forms, messenger, and the interaction board were so My Experience in ABC+ This story became a part of who I am today as a leader. It contributed much to my personal development and we got recognition and awards out of this program. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 68 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S much fun but the most exciting part is facilitating the breakout room and using Jamboard. It was really fun and exciting. The learnings and experiences gained from the ABC+ made me a better individual, and adaptive leader, and discovered my potential as a school leader. These experiences and learnings greatly helped me to transfer what I learned, motivate teachers and parents as agents of learning, and their result to improved achievements of our learners. 5 In ABC+, I am a participant in ILT and a trainer from TT1 and TT2 in our division. ABC+ was brought to the field by the Deped Regional Office. ABC+ changed the lives of many public elementary schools in Region 5 and Region 6. Amidst the pandemic, this partner organization paves the way for catapulting changes in the educational system as they introduce the virtual training of K-3 teachers on early-grade literacy. I remember the first time that we were selected to attend the regional learning for teacher training 1. I barely slept the whole night thinking about what will happen in the virtual room, what would my face be like on the screen, and worried that I might lose connection. But these worries turned into an excellent experience training like this is more thrilling and exciting because there are breakout sessions wherein you can talk to your team members, to present a document is not that easy before, but with patience, we were able to conquer our fears in using IT. This remarkable experience happened with ABC+ and now I am more confident in using it in the field. ABC+ training like TT2 and ILT pave the way for more understanding of literacy concepts to guide teachers in the field on how they will conquer the learning of their learners through the help of the Home Learning Partners-the parents. The use of HLG, LAS, and HLPs is a realization of how ABC+ helps schools in alleviating learning in those trying times and made an impact on the lives of young learners. An Experience with ABC+ It is significant because of new experiences that ABC+ has provided among the schools￾trainings, materials, tools and even load allowance-these is proof that they wanted to help YOUNG MINDS. Parents, teachers, and school heads are aware of the basic literacy skills that will hone the learners to be productive learners through the pandemic. 69 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S 6 No Response Isa ako sa pinakamapalad na naging bahagi ng ABC+ project activities. Isang supervisor ng Deped Masbate Province na minsang nangarap na maging bahagi ng curriculum development ng ABC+. Nakapagsulat na din ako ng mga kontekstwalizadong mga kwentong pambata ngunit sa aking pananaw at palagay ay may kulang pa at dapat punuan. Salamat at ako ay nabigyan ng pagkakataon na maipakilala ni [name] isang regional supervisor sa CLMD-Region 5 kay Ginang [names]. Sila ang mga instrument upang ang pangarap ko at pangarap ng bawat Masbateñong bata na magkaroon ng libro na gamit ang sariling lengwahe ay natupad. Noong panahon na nagsisimula pa lang ang implementasyon ng K to 12 ay para bagang isang alon ng Ticao at Burias na hindi malaman kung saan patungo. Maraming balakid kagaya ng matataas at matatarik na bundok ng madaanan at iba pang kabundukan sa probinsya ng Masbate. Mga balakid na sa tingin ko ay posibleng alisin o tanggalin kung sama sama at buong puso at husay na gagawin. Nangyari ang lahat sa mahaba at patuloy na pag hahawan ng balakid, isang balakid na kung saan hindi makakabuo ng mga magagandang kwentong pambata na hango sa sariling kultura, imahinasyon at kontekstong Masbateño kung hindi ay Minasbate Writing Guide na gagamitin bilang gabay sa pagbuo ng kwentong Minasbate. Nangyari ito buhat sa masusi at mahusay na prosesong pinagtulungan ng mga language experts sa tulong ng ABC+. Dahil sa hindi pa tapos ang working orthography ng Orawa Kag Orgulyo Ta, Igwa’n ABC+ Kag Deped Para sa Bata Na Masbateño No Response No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 70 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S Masbate, ito ang ginamit sa paggamit ng wika, gamit at iba pa na kung saan wala sa dating development activities. Malayo sa dati, nagbago ang lahat! Naipinta ang isang napakagandang senaryo ng Masbate sa pagbuo ng mga kwentong pambata gamit ay sariling lenggwahe at konteksto, iyan ay dahil sa ABC+. Dumaan man sa maraming pagsubok sa book development activities nandyan na ang poor internet connectivity, frequent brownouts, transportasyon, mga Gawain ng mga punong manunulat, at higit sa lahat ay hindi natin nakikitang kalaban “ang covid-19” ang lahat ng ito ay mga inspirasyon sa pagbuo ng pangarap ko at ng bawat batang Masbateño! Ang lahat ng kabahagi ng proyektong ito ay nagdiwang at patuloy na mag diriwang sa parangal na iginawad ng ABC+ project noong ang bawat punong manunulat ay itinanghal sa isang seremonya na dinaluhan ng DEPED officials at ABC+. Isang karangalan hindi hindi naming makakalimutan na minsan pa ang bawat isa ay nabigyang halaga, ang bawat isa ay mayroon ng maiiwang legacy sa bawat batang Masbateño dahil orawa at orgulyo ko usad ako na Masbateño! Higit sa lahat ang pagpupunyagi sa naging instrumento ang aming mga pangarap ay ABC+. Ako ay patuloy na sumulat ng mga kwento ng probinsya ko dahil karangalan ko na mayroong ABC+ at Deped para sa bawat batang Masbateño! 7 Have attended several trainings with ABC+; Have I have been a trainee, trainer, and facilitator in TT1, TT2, and ILT SPD of ABC+ and all these have made changes in me as a teacher, leader, and school head. ABC+ Changing Lives It has brought great significance to the community. The innovation served as the spice of the 71 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S been a trainer also. Having been trained in Literacy Instruction has equipped me to hand down what I have gained to my colleagues or classroom teachers. As a trainer for ILT SPD, this gives me the confidence to manage my fellow school heads. More so to serve as an inspiration that amidst work overload a school head trained by ABC+ can handle every situation with a happy disposition and positivity as one of the gained values for ABC+. Every hard situation I face with a light heart when as a school leader you were able to immerse yourself in the pros and cons of organizational management. Among my involvement in ABC+, the most significant is to be the proponent of the project Initiation Plan (PIP), the Parent Partner, Teacher PA. It was very challenging at the very start not before the preparation but most of all has encouraged the parents to attend the capacity building because I thought that no parent will accept that his child is branded as “slow”. With some strategies appearing and the school's sincerity to offer assistance to their children, parents attended the capacity building with many attendees. Parents are Home Learning Partners attended with active engagement. In its second year of implementation, the project was sustained and gave birth to a new project, PAREALNA (Parent Reading and Learning Numeracy Advocates). Parents/Home Learning Partners are more active and enthusiastic to know their schedules and how they are going to conduct the program. Every schedule is a new challenge for them. The learners seem active in class participation as observed by the teacher. The teacher has put pride in reading. As the school head of Calayucay ES, I take the lead and impose literacy on the learners through ABC+ in the face-lift of the school. The time may come that ABC+ disconnects from DepED but its legacy shall remain in the heart of Calayucay Elementary School. story that made it remarkable. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 72 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S 8 "I am a school head of one of the remote schools in Legazpi City Division. I handle 10 teachers, teaching 310 learners. No Response No Response No Response No Response 9 No Response Ako po ay isang Parent na displaced sa trabaho during pandemic. Sa pag-aaral ng mga bata medyo hands-on kami. Inaalam naming kong kumusta pag-aaral nila, yong numeracy and Reading skills pero parang nakukulangan kami as parents lalo na sa reading. Ng nagkaroon ng FB Live workshop ang ABC+ Reading a story sa mga learning material ng ABC+ Nakita ko yong interest nila sa mga story, mas madali kasi nila maintindihan at alam nila kaagad kong ano ang ibig sabihin ng mga salita. Malaking tulong sa amin as a parent ang program ng ABC+. Dahil dito mas dumalas ang aming pagbabasa ng mga anak ko. Sil ana mismo ang nagbabasa ng mga kwento. Natutunan nila na mahalin ang pagbabasa kaya nagging storyteller sila sa mga pinsan nila na mga bata pa. Thank you sa ABC+ kasi nagkaroon kami ng Bonding na Educational. Bonding na Educational Why is this story significant for me? Kasi sa panahon ng pandemic, sobrang hirap kahit na yong pag-aaral ng mga bata nagging mahirap pero dahil sa ABC+ ang pag￾aaral nila ay naging bonding moment naming pamilya. No Response 10 No Response I am a parent of two, one in primary and the other one in high school. ABC+ was introduced to us by our teachers at Albay Central School. They provide free books that contain short stories with illustrations. My daughter’s mother tongues are Tagalog and English. I was struggling to teach her to read Bicol because she doesn’t like it, whenever I gave her Bicol books she would cry. After the ABC+ books were given to us by the school there was a big change. She learns to love the book little by little. She first colors the illustration, and she reads. After some time, she finishes the Bicol No More Crying Drama No Response No Response 73 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S book crying. And from that moment, no more crying drama when reading Bicol. That is why I am grateful to ABC+ for using a very child-friendly word in making the materials. This change is very important to me because it is so hard to teach my daughter if she hates doing what I tell her to do. Now she enjoys reading bicol books thanks to ABC+. 11 No Response Ako ay isang parent ng 4 na estudyante na kasalukuyang nag-aaral sa Bagumbayan Central School. Bilang isang solo parent na nag-aalaga sa 4 na anak sobrang hirap lalo na po ang time ng pandemic kasi lahat sila sakin umaasa. In fast 2 years ABC+ ALS module malaki rin ang naitulong sa mga bata as a partner of school kaming mga parents ang nagtuturo sa kanila. Maraming positive and negative effect sa mga learners, in my own experience sa mga anak ko nandyan yung minsan napagalitan ko kasi di tinatapos yung ALS naglalaro muna sympre as bilang isang ina gusto na matapos ang lesson kasi di lang yon ang kailangan kong bigyan ng pansin kasi apat po sila. No Response No Response No Response 12 Nakilala ko sa paaralan ang ABC+ tapos parent teacher pa po.- Happy na makatulong sa ibang mga bata na hirap magbasa na ma eshare ang aking talino at kakayahan makapagturo at maipagbahagi ang kakayahan ko para makapagpabasa sila at maturuan ng magandang aral upang sa paglaki nila ay dadalhin nila ito sa pagtanda nila. Maging maayos ang kanilang buhay at makatulong pa sila sa kanilang pamilya. Para naman sa kanila ang pag-aaral nila kasi ito na magkaroon sila ng magandang buhay. Para hindi sila maging palaboy sa kalsada. Makatulong sa ibang bata Para sa akin mahalaga kasi gusto kong maibahagi ang aking kakayahan sa ibang mga bata. So maibahagi ko ang aking kakayahan sa ibang bata 13 No Response Ako po ay grade 3 pupil parents. Nalaman ko sa ABC program and Activities ang mahalagang detalye para maguide ko sa tamang kaalaman ang aking anak. PROGRAM No Response No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 74 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S Before noong wala pa ang ABC+ program wala akong kaideya kung saan magsisimula sa pagtuturo sa aking anak. After kung malaman ang ABC+ program nabigyan ako ng guide kung ano ang mga dapat gawin. Nabago nito ang daily routine naming mag-ina. Mas nabibigyan naming ng oras ang study time kaysa sa “playtime”. Nagkaroon kami ng bonding time as a mother and daughter through this program. Malaki ang naging epekto nitong ABC+ program dahil natutulungan naming ang aming anak sa kanilang STUDIES 14 Several years ago, I heard that ABC+ was a donor of books to some schools of the Iriga City Division. And as years went by, it offered seminars and training to all teachers in the country. And all of a sudden, I was one of the participants in the two phases of virtual seminars of ABC+ in our division. The training was very challenging considering the signal of the internet and the inlined activities in the training manual. But it was very educational and fruitful. It considered all the necessary tools a teacher should possess. The teaching strategies were all new and I feel so effective. The discussion of the topics was so comprehensive and so helpful to us teachers. ABC Plus: You Opened Us The most significant experience I had in applying my learning in ABC+, was when I made a non-reader pupil literate. I was confident that my traditional strategies were effective but I failed. After applying the appropriate activities in teaching phonics phonological awareness, my pupil with a reading disability can now read phrases, sentences, and short stories. The teaching innovations offered by ABC+ to teachers capacitated us to become more effective and efficient in molding the early-grader to become literate and ready to tackle higher lessons in the coming grades. 15 No Response Isa,dalawa….Dalawang taon? Mga panahong lumipas nang magkaroon ng pandemya na sobrang nakaapekto sa lahat. Bawal lumabas, bawal makihalubilo sa ibang tao. Paano ang mga bata? Paano sila matututo? Sigurado ko marami ang hindi marunong magbasa pagdating ng face to face classes. "Face to Face No Response No Response No Response 75 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S classes?" Gusto ko ng makita ang mag-aaral ko. Sabik na sabik na akong maturuan sila. Sa ngayon. marami pang magulang ang nagdadalawang isip na papasukin ang kanilang mga anak. Dahil sa takot na dulot ng Covid-19. Bilang isang guro, ako'y tumulong sa pagpapalaganap ng wastong edukasyon , para sa bata at sa bayan. Hindi ko hahayaang maraming bata ang mawawalan ng pag￾asa na makapag-aral at makapagtapos ng baitang. Maraming bata ang tuwang-tuwa na bumalik sa eskwela. Gustong-gusto na nilang matutong magbasa. Napaisip ako, Aha! Kailangan kong magsipag , gumawa ng mga bagay na makakatulong sa mga bata upang lalo pa silang magsikap at matuto. "Gagamitin ko ang iba't-ibang estratehiya, kagamitan at kaalaman ko sa paglahok sa ABC+ at iba pang palihan. Araw -araw ay nagkakaroon kami ng "pagkukwento" sa silid-aralan. Upang lalo silang ganahan na pumasok araw-araw. Kitang-kita ko ang ngiti sa kanilang mga labi. At ako rin ang lubos na nagagalak dahil buong kong naramdaman ang pagmamahal ko sa aking mag￾aaral. 16 My journey in ABC+ started through a division memorandum when my name was included as one of the regional trainers in TT1, TT2, and tilt. The reading crisis during the pandemic time was a tough challenge for me as a school leader. It was so timely that ABC+ had partnered with DepEd where it became a part of my ABC+ journey. With all the training that I attended as a regional trainer and even facilitated with the teachers and school heads in the division, somehow and somewhat the learnings on reading strategies and creating a positive climate in the school and at home helped the learners improve their literacy domains. Through my project initiative as a final output which was dubbed Project SMK (Syllable Mastery in Katon way), the number of frustrated readers was My ABC+ reading journey The story is significant to me because it benefits most of our learners in the school. Reading innovation became an instrument in addressing the reading crisis in the school. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 76 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S lessened. The said project which I implemented in the school reaped a National Award as one of the Noteworthy Literacy Practices and school supplies worth of 10thousand pesos conducted and sponsored by ABC+ during the Learning Conference via Zoom. Indeed, it was realized with flying colors because of the distributed leadership I employed with the external and internal stakeholders in the school. The love for reading among the pupils, teachers, parents, and the community was intensified because of the training conducted by ABC+ and more so the collaborative effort by various stakeholders in combating struggling readers in the school. 17 I learned about ABC+ activities from our Education Program Supervisor in LRMS, Sir Juancho P. Azares. A year ago, he invited me to join and participate in the teachers' training for teachers in the development of Minasbate Storybooks in Early-grade. It was indeed magical and a roller coaster of an event for me because ever since then, I have been an active and privileged participant in ABC+ training and an active user of their materials. Indeed, my life has changed because of this partnership. Many changes have happened in the past year of implementing ABC+ activities. One of the things that I have witnessed is that children are showing signs of change in behavior in terms of their fondness and love for reading and appreciating books. Teachers were provided great opportunities because they were able to make Minasbate stories for their students, which helped the kids and helped them grow professionally. The training provided by ABC+ molded and honed the skills of the teachers on how they could teach their learners scientifically, which is beneficial in the field. Among these changes, the most significant I encountered and experienced as a result of ABC+ activities was when I was allowed to finally fulfill my 77 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S long-time dream—my childhood dream of becoming a writer of children’s books. Ever since I was a kid, I have wanted to write and publish stories that my readers would enjoy and relate to. My dream has finally turned into reality. ABC+ became the bridge to my childhood dream. I never, in my wildest dream, thought that I could become a quality assurance of the books that we developed during those months of training. I feel so privileged and honored because of these opportunities. That is why I am forever grateful and in debt to Sir Juancho Azares and ABC+. It was truly significant for me because not only were my dreams fulfilled, but it also changed the lives and views of the learners when it comes to reading and appreciating books. 18 No Response Teaching my students how to read is one of the challenging tasks as a Grade Two teacher. In my four years of teaching in the public school and handling the last section, I'm thankful that I am one of the participants during the two years of ABC+ Training. ABC+ taught me a lot about what strategies in reading I will use for my students. Different reading strategies and approaches were discussed and I learned a lot from it. I unlearned and learned a lot of things that are very beneficial on my part as a teacher. I can now choose the right strategy that is suited to my learners, especially in their reading level. I want to instill in them the habit and love for reading so that they will enjoy and understand what they are reading. I want to teach them that reading activities are not a burden but an enjoyable activity that they will learn and explore a lot of things. ABC+ training is an avenue that taught me a lot and will forever be grateful for my learners and their future. Reading is fun with ABC+ No Response No Response 19 No Response I am Joanalyn L. Sarmiento. I was a teacher for 10 years. I enjoyed being a teacher and learning day by day. "ABC+, Continuing No Response No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 78 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S ABC+ was introduced to me when I was assigned as ICT support during the ABC+ Regional Training of Teachers (Strategies for Language Learning and Transition: Improving Early-grade Literacy in School and at Home) and luckily I was assigned as a trainer during the Training in the use of SRMs. It was a privilege for me to have heard the well-versed trainers/speakers from the ABC+ team. I do love listening to them and discussing things that best fit the educational needs nowadays. As an intermediate teacher for seven years, everything was new to me. When our school head assigned me to the primary level, I thought it was easy thinking that my lessons would be more basic than before. But I was wrong, it is more challenging that will require me to be more innovative and creative so that my learner would learn and even enjoy learning. ABC+ has helped me to rediscover my purpose as a teacher. I have learned different strategies that would improve and develop the learners' literacy levels. different pedagogies and effective learning approaches that will cater to the needs of our learners. This capacitated us with more precious techniques on how to bridge the gap of learning among our pupils. Embracing the change along with our dream for our learners. I have been a learner and I do appreciate my teachers even now because of what I am today. It was even more significant knowing that what I have learned in ABC+ will transpire on our vision as educators. Taking the challenging role of teaching children will not stop with my dream but pursue what I can for them. Journey to Literacy" 20 ABC+ was introduced through a series of training given to teachers. A native teacher of SDO Iriga City, seasoned by time from the private school teaching experience and public school teaching experience, sprouted a very idealistic and strict teacher in the primary grades. Believing that every day is a chance to make a difference in the lives of every child, this teacher gives her best armed the ABC+ A Life￾Changing Tool It is significant because of the change it made to my learner (Onin-not the real name) and me as his teacher. The innovation is significant because it helped me as a teacher to better 79 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S best pedagogies in teaching but does not after ABC+ came about. ABC+ is a very short word yet makes a significant change in the life of a teacher and the lives of her school children. ONIN (not the real name of a learner) is a very shy boy unfriendly and so withdrawn. Among the learners in the class, Onin got the teacher's attention but to no avail, failed, and felt so frustrated. After a while was tapped to attend the different trainings given by the ABC+ RT1 -RT2 that significantly the teachers to understand all the types of learners she had especially her learner Onin. Onin is dyslexic and has language deficiency syndrome. Diagnosing his problem is not the end of it all but was able to address the needs of Onin Onin now is a changed boy -an active and participative boy. understand my learners and address their/his needs 21 Bilang isang magulang ng batang lalaki nakilahok ako sa programa ng pamahalaan sa pagbabasa ng aking anak sa bahay upang lalo siya mahasa sa pagbabasa at maintindihan niya rin ang kaniyang binabasa. At dahil nga sa aking pakikiisa sa mga programa ng paaralan masasabi na nakapagpabago sa mga bata ang mga programa ngayon sa mga bata dahil sa panahon noong ako ay nag-aaral pa hindi ganito kaganda pa ang itinuturo ng mga guro sa amin pero ngayon dahil sa mga programa sa komunidad ay napaganda na ang ugnayan ng paaralan at komunidad. Dati rati mas gusto ng anak ko at ng mga bata na lumabas lang pero ngayon mas gusto niya na magbasa lamang sa bahay. Ugnayan ng Komunidad at ng Paaralan Dahil sa ito ay nakapagpabago sa mga bata. 22 Ako po ay isang Brgy. Kagawad sa aming barangay at ako po ay naimbitahan sa Launching at Nagkaroon ng malaking pagbabago sa mga mag-aaral lalo na sa kanilang reading abilities sa tulong ng mga projects ng ABC+ na dinaluhan na trainings ng mga teachers na kung saan ito ay naging kapaki-pakinabang sa mga mag-aaral dahil natututo silang magbasa dahil sa mga Projects ng ABC+ at dahil na rin sa mga guro ng My wonderful journey with ABC+ Ako ay nagpapasalamat at proud na naging bahagi ako sa ABC+ projects at masaya ako sa resulta ng Ang nasaksihan kong ABC+ Project and Initiatives ng Calongay Elementary USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 80 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S Orientation and nalaman ko po na ito ay proyekto ng ABC+ sa paaralan na pinangalanan ng proponent na si madam Cherry na Project SMK (Syllable Mastery in Katon Way) bilang project initiative at simula noon nakita ko na maganda ang objectives ng projects lalo na ang magbebenipisyo nito ay ang mga mag-aaral na may suliranin sa pagbabasa or mahina pa sa pagbasa. Kasama ako sa pagmonitor nito at nasaksihan ko mismo na epektibo ang project at halos lahat na recipient ay halos natutong magbasa ng dahil sa project na ito. Calongay Elementary School na kung saan sila ang nagsakripisyo sa preparations lalo na sa pagcraft ng mga initiatives para matulungan ang mga mag-aaral na may kahinaan sa pagbabasa..Salamat sa mga tumulong, ang ABC+, mga Guro sa Calongay Elementary School and sa suporta ng BLGU. kanilang mga projects at sa maliit kong ambag, naging parte ako ng project. School na Project SMK (Syllable Mastery in Katon Way) ito ay isang makabuluhan at Malaki ang naitulong sa mga mag-aaral. 23 No Response I know this program ABC+ by joining a series of Me Level-Up No Response No Response 81 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S workshops and training. Before ABC+ I was a shy type of person and after ABC+ I gained self-confidence and know to open up my insights. The change that happened with ABC+ is always good since by the time I joined the training or seminars there is a cash allowance or load allowance which helps the financial burden of the teachers and the venue is comfortable if it is face-to-face. It is significant in my teaching field because it harnesses my skills in the different teaching strategies to be used by attending a series of workshops. Thank you ABC+ for being part of this program. with ABC+ 24 No Response I am Mary Grace A, Bacharo from Escalante Central Elementary School, Schools Division of Escalante City. I got to know the ABC+ program by attending a virtual seminar last 2020 and it was my first time. At first, I was not excited about this training because I know that after every seminar, there will be an output. Nasabi ko sa sarili ko na, "Naku, dagdag trabaho naman." But while listening to the facilitators in every session, I've learned and realized that this program suits us, and it is a big help for us teachers since it offered teaching strategies for early-grade language learning and for improving literacy in school and at home, I'm also enjoying this program because there is a load allowance for teachers and aside from that, ABC+ also provided materials for teachers and learners which is very helpful for the children. It is significant in my teaching career because it helps in my professional development. Sometimes, I can say to myself that teaching is hard, and at times it can seem like my energy is in short supply, But I realized that we all need a little inspiration to remind us why we do what we do. My Journey with ABC+ No Response No Response 25 No Response I am Teacher Sheila Mae H. Vasquez, a Grade 1 teacher at Escalante Elementary School. I heard about ABC+ Journey with ABC+ No Response No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 82 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S programs when the division memo was handed to us. And I am one of the participants attending the ABc+ training. I thought ABC+ training was just another bunch of tasks to do. But it changed my perception when I started attending a series of training and workshops. The ABC+ programs changed me a lot in terms of my teaching styles in teaching my learners that are struggling readers. It broadens my knowledge and understanding of how to deal with them. When I was chosen to be one of the decodable writers, I was about to give up the task but when I continued my journey I was able to love my task as a writer and was able to write two decodable books for stages 3 and 5 levels. It is an unforgettable experience to be part of the ABC+ family and able to apply what I have learned to my school station. 26 No Response I’m Ma. Rowena T. Elisterio, Principal II of Anini-y Central School, Anini-y, Antique. When I was informed that I am one of the participants in the ABC+ webinar for Instructional Leadership Training(Strengthening Learning Conditions For Easy Literacy) and on Strategies For Language Learning and Transition: Improving Early￾grade Literacy in School and at Home, respectively, I was very excited. I put in my mind that with so much work that I am doing in school and the district, I will give time to attend this webinar because what kind of Leader I am if I could not give technical assistance to my teachers in terms of reading readiness of their learners. This is a perennial problem not only for the school but the whole Deped Community. It is a must to find/give solutions to lessen this problem. Before the ABC+ webinar, I learned the different teaching strategies and approaches in some of the seminars way back before the pandemic and during my college days. But when I attended the ABC+ webinar, the strategies, and approaches that I’ve learned deepened. The learnings I My ABC+ journey para sa ang bagong kabataan No Response No Response 83 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S gained are beneficial because I taught the teachers how to use and integrate them into their lessons. It changed the atmosphere of the classroom because the teachers served only as learning facilitators inside the classroom. Learners became very active and participative during class discussions as observed during classroom visitation. The changes were very good on the part of the teachers because their struggling learners or full refreshers became confident to read letter sounds, phrases, and syllables. After a six-week curriculum conducted by the school, mid-CRLA was conducted, and results showed that there is an increase in the previous data. Happiness could be in the eyes of the teachers upon knowing the result. The teachers also enjoyed making flashcards, and print reading materials because the school supported their needs as well as the people in the community. The training was very significant because I was given a chance to become one of the trainers for my Grade 1 to 3 teachers in the district. I will never forget that once I was a part of the program I helped the teachers improve their teaching style in reading. To the teachers who are the front liners and who imparted what they learned during the echo seminar to their learners. They enjoyed and were very patient in teaching their learners and were very positive that all of them(learners) will become readers. They did home visits to check whether the learners improved their reading skills. To the parents, guardians, and learning facilitators who find it hard to teach their children/apos how to read after two years of using modules. Now the parents/guardians were taught during homeroom meetings how to read letter sound so that they can teach their children/apos at home. To the learners who are the clientele who craved to learn how to read, now they have progressed in their reading. Thank you, ABC, + for the wonderful experiences. Saludo ako. Sana marami pang trainings USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 84 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S to come. 27 No Response I am Rea M. Venegas, a teacher from CRAMS, SDO Antique. I have been teaching for 16 years and 15 of those are spent with my first graders. Now, how did I get acquainted with ABC+? I was chosen to be part of the Language Editor team when ABC+ selected teachers to craft Kinaray-a books for k-3 learners. And I must say, I have learned a lot. I can’t say it was an easy job, it was the toughest one I’ve had with ABC+ to date. But I can surely say that ABC+ has the toughest training, and toughest outputs to submit but has all the smartest members they can get! They can be very intimidating but having to train under their wing will surely equip you with new ideas, and more strategies that are very effective in the teaching-learning process. ABC+ has capacitated me with a lot of ideas that I can use in my class. Now that we are slowly getting back our freedom from the shackles of COVID-19. I am excited to try them for my learners. The ABC+ No Response No Response 28 No Response ABC+ in the life of an Education Leader! Assigned working in the Curriculum Implementation Division which focuses on the teaching-learning process of Instruction which is the heart of the Department of Education. In this division, we are facing many challenges not only brought about by the Corona Virus Pandemic but by the low academic performance of our learners especially in the area of literacy and numeracy. ABC+ was introduced to us as one of the pilot divisions in Region VI. Orientation, Seminar Workshops were conducted for Teachers and School Leaders on Learning Materials Deut Teaching Strategies, and others. Before ABC+ was implemented, we experienced many problems or challenges in the delivery of Early-grade Instruction especially in Reading due to the presence of several learners under the frustration level. But with ABC+: A Heaven-Sent Gift No Response No Response Chosen Story 85 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S the partnership with ABC+ and its implementation of the Program, Projects, and Activities, there’s a decrease in the number of non-readers. Good changes happened in our learners and teachers. As well in school and in the community, learners’ achievement improved, and teachers’ confidence levels up. Parents were happy with the materials received by their children and the improved socio-emotional aspect of their kids. ABC+ played an important role not only in the lives of the learners but also in the teaching competence of our learning facilitators. The community is very grateful for the program because it played an important role in the total development of the child. They observed their children were focused on their studies and minimized their time on computer games. Indeed ABC+ is God sent to us! It’s a gift from heaven. A big salute to the men and women of ABC+. Our Big Congratulations! 29 No Response One Friday afternoon, the District Supervisor told the School Head that the school where she was assigned was chosen as a pilot school for ABC+. ABC+ is a new thing for the School Head. She had no idea or background of this program/project. With a series of communications circulated, the school head had an initial grasp of this program. One day the focal person, Sir Loloy went to the school and discussed details of the implementation. Despite the surge of the pandemic, virtual orientation, training, and workshops were conducted with supervisors, school heads, teachers, and parents. Focal persons were designated, schedules were made, agreements were set and rules and regulations were implemented. With their reach, risks - taking moves, and challenging decisions A. Montes 1 Elementary School was TRANSFIGURED to another level. The School Head was moved to pursue and implement the program to cater to the needs of the learners and have the opportunity of coming to school "ABC+ A. Montes 1 amidst Pandemic Education" "Conquering Pandemic through ABC+ Program" No Response No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 86 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S even for a rare chance of sitting inside the classroom taking the CRLA. Teachers' hunger to see learners and teach in person was given a chance to be real. New training makes them more adept at the new tools, approaches, and strategies to make children read. Their competencies were sharpened and polished parent￾partners relationship with the teachers and principal were closely developed. Their taking on more serious responsibilities was evident and their helpfulness with one another was felt because of the risk during the pandemic. The School heads' competencies were challenged and strengthened. ABC+ was indeed a blessing and a challenge to the whole school community. It makes the school community stronger and more resilient in the delivery of DepEd services and continuous learning=teaching experiences in a risk￾taking manner. The School Head with the teachers develops a more cohesive relationship professionally and personally. Parents and learners were in one way or another developing a sufficient way of pursuing learning. As a whole, A. Montes 1 Elementary School becomes an institution amidst the pandemic. 30 No Response Being a teacher for more than a decade, teaching has been a way of life. ABC+ came into the picture when ICES became the pilot school; for the implementation of the ABC+ Project. Orientations, training, and check-in meetings were some of the activities that helped everyone to grasp the essence of the program. Teachers, school heads, and supervisors were introduced to RLA, CRLA, strategies, approaches for early-grade literacy, supplementary reading materials development, BASA materials, and a lot more. ABC+ paved the way for numerous and diverse learning experiences not only for the learners but as well as for the teachers and instructional leaders. The program equipped the teachers with different skills and ABC+: A Worthwhile Journey No Response No Response 87 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S highlighted the important role of assessment to provide appropriate intervention to the learners. With ABC+, the teachers, the school, and the community are more involved in helping the learners to foster a love for reading as well as learning. Teachers were able to make innovations and interventions with the help received from the program. They were able to get to know their learners personally and provide appropriate intervention and assistance in their learning. Parents were also involved and became part of the learning process. The program helped them to see the important role they have in the learning process, and that reading and learning is a family affair. The journey with ABC+ is a meaningful one. It touches everyone's life and the impact is lasting. May they continue their vision and mission of promoting literacy to the learners and training and instructional leaders. 31 ABC+ gave me lots of opportunities to build myself and to develop professionally. I started to become a participant in the training, turned into a trainer and later then became a writer. I was able to learn more reading strategies because of the TT1 and then I learned how to An ordinary classroom teacher who was once inspired by ABC+ to become a part of bringing change to the lives of Filipino children, I became one of the writers of the ABC+ leveled reader book. Throughout the reading process, I am amazed by the project of ABC+ to promote change in the lives of Filipino children by providing them with books for them to love reading. The materials that were made would promote learning and would help the children learn how to read and appreciate the essence of reading. With the efforts we gave to create the stories and for the illustrators to make their masterpieces. I know then children would love the materials and could help them learn reading interestingly. It is the best feeling to share things that could change the lives of the children. Good thing that also ABC+ let us experience how to become a writer with the help they gave us throughout our writing process. Sharing my effort through writing a story for our learners is a significant event for me as a teacher. "Not Just an Ordinary Teacher" This story is significant for me because I was allowed to be a part of the changing lives of Filipino children, The ABC+ provided everything. I need to make my pupil read. I was able to create a clear goal using the strategies and materials that were introduced. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 88 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S facilitate learning when I became a trainer. The best part, I was able to make stories because of the opportunity to become an ABC+ writer. At the end of the process of our writing journey, we were able to conduct field testing of our stories. Seeing the children appreciate and love the story is the best fulfilling journey for us teacher-writer. Contributing change to their lives is a reward the teacher could never have. With the stories I wrote, I am sure pupils may enjoy them and they could learn something from them. I made those stories to give a change to their lives and for them to love reading. 32 No Response My ABC+ journey started when our school was chosen as one of the pilot schools in Region VI, Division of Iloilo City. At first, I feel disturbed since for me this will give me additional work as a teacher. I experienced lots of challenges and I considered it to be a great help for me as a teacher. The training, check-in meetings with ABC+ personnel, and other related activities help me grow professionally. Since I am a newly assigned Grade 1 teacher, I appreciate the knowledge and skills I learned from ABC+. Things went smoothly in doing my tasks every day applying the things I learned. The supplementary materials I received also help improve the literacy skills of my learners. The distribution of the materials was very timely amid the pandemic. Learners enjoy readable reading books because of the attractive illustrations which encourage them to read more. It arouses their interest and develops their love for reading. The CRLA also helped me assess my learners' reading profiles. This enables me indirectly my learner's needs. Innovations were conducted after the assessment. This is a significant change in the lives of these learners. Parents were able to provide study corners at home. This will allow reading with kids. This develops a strong bond between parent and child relationships. With ABC+, more teachers, home learning partners, and learners will be helped. "My ABC+ Journey" No Response No Response 89 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S 33 No Response Extending for a considerable distance marked by inequalities and surface coarseness, narrow and steep….such are the steps leading to the post of Chief of the Curriculum Implementation Division. It opened doors to opportunities. It gave me ABC+. The ABC+ that came to the lives of the educators in SDO-Iloilo City gave meaning and extended even more, one's love for teaching and care for learners. The training for teachers in K to 3, the materials that were made or grafted for leveled readers, and the decodable materials were very useful. The teachers who came in contact with learners and folks at homemade teaching were even more meaningful. As teachers' attestations reveal to the CID Chief, from the start learners were identified for readiness, They loved how the ABC+ has made things a lot easier. The task of making learners read and develop their love for reading changed. Teaching leveled reading came out to be very interesting. At the end of the school year, when the readership of learners is improved, the teachers can only thank and keep on thinking of "The ABC+ that was." The change in teachers' behavior toward teaching reading has significantly changed. Training and materials were a real tandem. "ABC+: An Insight to a Tandem" The story is significant for me because it is my journey as a Grade 3 Teacher who will teach learners and attain 5Bs. No Response 34 No Response Sa isang mababang paaralan sa rehiyon anim ako nagtuturo. Baguhan poa lang ako sa serbisyo bilang isang public school teacher. Nagtuturo ako sa bansang Cambodia bago pa man ito. Napakahirap sa mga unang araw ko dahil maliban sa pandemya na mayroon tayo ay sobrang iba ang mga gawain ihambing sa mga nakagawian kong gawain. Tandang-tanda ko pa sa unang araw ko sa trabaho. Ang mga guro doon ay abalang-abala sa kanilang virtual training sa ABC+. Iyon din ang araw na una kong narinig ang salitang ABC+. Ako'y napaisip na may konting sabik kung ano ang ibig sabihin nito. Hanggang dumating na ang araw na "Salamat, ABC+" No Response No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 90 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S hinihintay ko, Napasama na ako sa group chat ng ABC+ sa aming paaralan. Paunti-unti nagkaroon ako ng ideya kung ano ito. Mas lalong lumalim ang aking pag-unawa sa programa dahil sa lingguhang check-in meeting namin kasama ang mga ABC+ facilitators at mga guro sa aming paaralan. Minsan nga naging moderator pa ako. Halong kaba at saya ang aking nararamdaman sa araw na iyon, dahil nga baguhan pa lang ako, Lumipas pa ang ilang buwan mas lalo kong naappreciate ang ginagawa ng ABC+. Napakalaki ang tulong na naibibigay sa akin bilang guro, sa mga bata at sa mga magulang. Isang araw may isang nanay na nagkwento sa akin na hindi lamang ang aking estudyante ang nagbabasa kundi gusto na rin ito ng mga kapatid. Napakasarap pakinggan na nagustuhan ng mga bata ang pagbasa ng aklat. Dumaan pa ang araw may nakita na naman akong isang magandang dulot ang programa. Pandemya sa mga araw na 'yon kaya nahihirapan akong i-identify and level ng reading ng aking mga mag-aaral. Salamat sa CRLA ng ABC+. Kahit ngayon na face to face na, ginagamit pa rin namin ito. Salamay ABC+! 35 No Response Ako isa ka maestra nga buta sa pagpaningwa sa mga hilikuton nga makabulig sa akon kaugalingon nga mahibaluan ang mga butang kag stratihiya para itudlo sa akon mga kabataan kag makabulig sa akon kaupod nga mga manunudlo. Daku nga bulig nga nangin parte ako sang ABC+ tungod ako isa ka ABC+ Facilitator kag writers, ini akon gid nahibaluan paagi sa DepEd, sa suporta sang akon principal Dr. Noren L. Eseo kag District Supervisor Ma'am Roberta C. Jison. Isa ka bag-o nga pamilya ang akon nabtyagan sa ABC+. Sila nakabulig sa akon nga makaobra sang isa ka istorya nga may inspirasyon sa pagsulat kag pagpaminsar. Tudo suporta nga ako makahibalo sa paggamit sang application sa laptop para maaply koa ang akon istorya. ABC+ ang Inspirasyon Ko No Response No Response Chosen Story 91 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S Sila ang alagyan kon sa diin tudo-tudo ang ila suporta. Isa na diri ang pagtiner sa lain-lain nga mga hotel, makahampanay sa kaupod na mga manunudlo sa bilog nga Panay. Isa ka maayo nga hitabu nga nagin parte ako sang ABC+ ang pagbulig nga nakaobra sang istorya nga mahatag sa mga kabataan nga makabasa kag magin materials ka mga manunudlo sa pagtudlo. Paagi sa supplementary materials kami makabulig man sa mga ginikanan kag stakeholders nga magamit nila sa pagtudlo sa barangay. Daku ang mabulig sang ABC+ sa amon nga mga manunudlo tungod sa istorya nga ginhatag nila padayon nga nakahatag sang mga balasahon sa amon kabataan kag mas nadevelop nila ang pag-intiendi sa ila ginbasa. Nakaapekto ini sa mga manunudlo, ginikanan, estudyante, kag stakeholders tundog magamit gid ini indi lang sa eskwelahan kundi sa balay kag sa baranggay. 36 No Response I was once an Intermediate teacher for more than 20 years, specifically Grade 5 and 6. And it's now my 4th year of teaching Grade 1. At first, it takes time to adjust -the tasks, patience, etc. I realized later as time goes on that I love teaching Grade 1 learners. Making them read and count was my happiness. So challenging! ABC+ was just a timely and great blessing to me as a newly assigned teacher in Grade 1. Our school was chosen as a Pilot school. We were monitored closely and had check-in meetings once a month or as needed, as to the activities, problems encountered interventions, and so forth. ABC+ and our school became close partners in molding the barren K-3 teachers, and gained a lot of wisdom and knowledge through sharing of experiences, problems, and interventions. I gained a lot of knowledge in my teaching. The training provided by ABC+ was so helpful to me and a big bonus because I was chosen as one of the Regional Trainers of ABC+ for K-3. Before, I was like an empty tablet, with not enough My Journey as a Grade 1 Teacher No Response No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 92 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S skills in teaching Grade 1 Learners. But with ABC+ as our partner, I gained confidence in teaching these little ones. I enjoyed teaching Grade 1 learners. And recently, I was awarded as Principal's Ace in Videolar Presentation during my Classroom Observation. 37 No Response Hi! I am a teacher in Region VI. I am a passionate and dedicated Grade 2 teacher of one of the well-known schools here in Region Vi. I am one of the teachers and our school has chosen to be a pilot school in implementing the ABC program and activities. My experience in the ABC program has been meaningful and fruitful as a teacher because it helps me to gain strategies and techniques, to widen and strengthen the program with the supervision of our school head, focal persons, co-teacher, as well as stakeholders. The ABC+ program experience was a roller coaster journey because as a teacher, this was new to me and my work experience in the DepEd could not be sufficient to overcome the challenges in the said program. But with the help of the orientation, training, check-in meetings, collaborative ideas, and by the grace of God we built strong relationships among us as an individual. Today, ABC+ programs have a big impact on the educational system because they focus on literacy, reading skills, and comprehension of every learner, the changes subject to intervention to the teacher, parents, and learners. These changes for the parents are positive and there are negatives. Some parents embrace that his/her child needs help and others can take the results of the reading assessment which they lead not to engage in the intervention because some of the parents have low literacy. So, little by little we always give assurance to the parents of the benefits of the said program. The change is significant because of this program, we teachers discover the needs of every learner. The effects are; improved interest of the child in reading, Reading is the best weapon for every Child No Response No Response 93 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S and it strengthens the relationship among teachers, parents, and stakeholders. 38 No Response As soon I regained consciousness of the complexity of what it is to become one of the instrumental people in education in the new normal of education. I fondly remember the time when I met Sir Loloy of ABC+, I have known him since the school was chosen as the venue for Launching National Reading Month way back when I was assigned as a school head in my previous school. He coordinated in the school to prepare everything for launching, they invited stakeholders from the different sectors to do the story reading, this is also the time that I Know the ABC+ that helps the DepEd and education stakeholders in addressing the interconnected factors that contribute to low education outcomes in the poorest preparing areas in the Philippines. My experiences in ABC+ are very timely because I was able to attend orientation and Crafted Literacy Program, which addressed the issues in literacy. This program is Project 3BsC (Basa sa Balay kag sa Baranggay). This will focus on developing learners. Macro skills, reading, writing, speaking, writing, and literacy. Through ABC+ and this program is a strategic endeavor designed to achieve our goals of producing a community of independent readers through well-integrated literacy instruction responsible to create learning with a more lasting impact. Through ABC+ and the project, 3BS was developed to reduce the number of full refresher learners to zero. My Never￾Ending ABC+ Journey No Response No Response 39 No Response I was very fortunate to be part of the advocacy of ABC+ to improve the quality of education of our learners. I would like to capture my experiences in the 3Gs￾Growth, Greatness, and Grace. I have witnessed how the lives of our educators and esp. our learners, have improved. There is growth in terms of the level of My 3Gs with ABC+ The story is significant because it is my journey. A journey rich with the best people esp. from the ABC+ No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 94 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S knowledge and skills among teachers, learners, parents, external stakeholders, and instructional leaders. With this personal growth comes Greatness-greatness in the quality of work we saw produced and the amount of time, energy, and dedication we have invested to facilitate the transfer of learning from ABC+ activities to its rightful beneficiaries. Lastly, I also see my work in ABC+ as Grace. It is with the Lord's grace that we have moved despite the insurmountable obstacle brought about by the pandemic. There were so many personal and professional struggles I have undergone. But I believe it is also grace that will sustain me as I continue my mission of being one of the advocates of learning and empowerment for our struggling learners and all people in our educational system helping each child, one pupil at a time. I want to echo the works of Chito Salavar. "How we do God's work is where our talents can best be served" So all the insights I gained from my various ABC+ experiences became part of my journey for growth, greatness, and grace. I believe that I was led to this beautiful world of ABC+ because that is where my talents as a supervisor can best be served, doing God's work. program, as well as my fellow educators. 40 No Response Almost five years ago when we were crafting the DepEd LAC toolkit, when I first met Ms. Ina Aquino and Mr. Loloy Gelmo. We got a chance to talk, collaborate, and have coffee memories in one of the best-known coffee houses in Zamboanga, I thought it ends there, however, in one of the projects introduced to DepEd, the ABC+ birth to Region VI was started with a visit of Mr. Gelmo to Iloilo Central Els. As principal of the school, I've cited his concerns and introduced myself that once upon a time we met in a workshop, and our stories did not end there having more and more expectations and activities. He laid down his intent to borrow a few teachers and learners to host the Launching and ABC+ The ABC+ Journey for Literacy No Response No Response 95 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S Project in WV which will be allowed by several Central, Regional, and Division officials, I really welcomed the request and prepared the school, teachers, and learners for the said request, the launching was done in a hotel with a lot of significant people, there the program was introduced, what surprised me most, that many of my co-writers in the LAC Toolkit were there present too, and of course the presence of Ms. Ina Aquino, we were divide into smaller groups and to craft our dashboard in reading performance and other key indications in MTB, Filipino, and English, we have to present to the group that we need the program, we need to satisfy and convince that we are ready to be the pilot of the program, I was tasked to present the Division output from a smaller workshop. I was so surprised that the group from the ABC+ Co with Ro were convinced that we will be the initial division of the program, knowing of the presence and little background of the program I was identified as pilot school and school heads as the lead of the program, my joining with ABC+ started in their series of activities and workshops were attended by me. Countless conducts of FGDs, school visits, and data analysis were done by them, we feel proud of their visits from the local guests and foreign guests looking for why we are to be considered pilot school for the program, I dwelt so many activities to prefill from conferences, assessment, materials development, contextualization, field visits among others, the relationship with ABC+ has been more intimate the pandemic strikes, I thought we will cease with the program, however it was even more intense , the program has reshaped its delivery of remote learning process, more and more trainings were introduced, the challenges of the pandemic didn't hinder but even more our bond stronger when we collaborated in how things will be done with resilience and strong connection of making change happen, I have embraced the program USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 96 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S for the teachers and most especially the Filipino learners we serve, the BC+ is a greatest gift for the Ilonggo Learners to read and write with confidence and have sense of pride. 41 No Response I have been a reading advocate since I became interested in being an educator, which means I started to love reading more during my college years. Reading can take you around the world. ABC+ programs focus on the importance of reading and I am happy and fortunate to take part in some of its activities I am a Pablo Sohok District Supervisor assigned to supervise schools implementing the ABC+ programs and projects. My involvement with the program/project not only starts and ends with supervision but extended to being an EGRA Assessor. I was an EGRA Assessor for Baseline Collection and Midline Collection of Data. Having gone through many trainings on how to conduct assessments was an eye-opener. The tedious training we had on how to assess pupils' reading skills, including their word recognition, listening, and oral-reading comprehension plus this speed or fluency gave us the confidence to do our task efficiently. I played two different roles during these two assessment periods; first as an Assessor and second as an Interviewer. Both assignments were challenging. My experience made me realize how important the roles of teachers and parents as HIP (especially during the time of pandemic) are in helping their learners/children to read. The use of decodable readers, if had been maximized, assisted both teachers and parents to develop reading comprehension among our learners. Children who were exposed to books and were read frequently by their HIPs tend to be a reader with comprehension over those who were deprived of the books and follow-up. These facts were revealed during the interview of HIPs (parents)during the Midline Assessment/ A reading advocate and EGRA assessment No Response No Response 97 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S 42 No Response "Hay naku! Diyos ko Lord, bakit ba sa akin pa ibinigay itong coordinatorship ng Mother tongue Based Multilingual Education," reklamo ko sa kasama ko sa office na si Sir Donald. "Hoy, di ba nga 3 years kang coordinator ng MTB-MLE doon sa Antique sa pinakamamahal moy Kinaray-a language. Kayang-kaya mo yan, " sagot si Sir Donald. OO, sa Antique pa lang una kong naranasan bilang Education Program Supervisor sa FIlipino and pagiging coordinator ng MTMLE ng Kinaray-a language. Diyos ko po, super hirap at super dami ng travel at trabaho. Oo, tinanggap ko ang trabahong iyon dahil wala namang choice diba? Kahit nakakasama ng sabihing "Para sa Bata Para sa Bayan", yuck ewan ko ba. Aba'y sa totoo lang sukang￾suka ako sa linggwahe parang Ex ko pinabayaan ko. "Hoy, wala ka na naman sa sarili mo. Anong iniisip mo? Si MTB-MLE o si Ex mo? tanong ni Sir Donald habang tinatapik ang balikat ko. " Weeh... Si MTB. Alam kong malaking pasanin ito. Hindi ito Ex ko na pwedeng balik￾balikan." Isang araw noon...."Sir TIno, memo oh, punta ka raw ng Manila dadalo sa Launching ng ABC+," sabi ni RF ang Secretary sa office. "Salamat Sir RF," sabi ko habang nakamot noo na tinatanggap ang memo. Binasa ko ang memo at walang magawa , nangutang kay Maam Nona at lumipad ng Manila. Simula nong araw na iyon, nagsimula na ang kalbaryo ko sa MTB￾MLE. Diyos ko dinagdagan pa ng ABC+ na ito. May panahon pa kaya ako sa lovelife ko or even sa Ex ko? Training dito, training doon, CRLA here and there, speaker dito, speaker doon. Ok lang din sana kasi libre, pinapakain at pinapatulog ka sa magagarang hotel. Diba class? Heto pa ha, pag Quality assure ng mga Early-grade reading Materials sa Kinaray-a, Akeanon, Hiligaynon at iba pa na lengguahe ng rehiyon. Aba'y 483 titles din iyon. Pagsasapuna ng mga Editing Guide etc. etc. My God, mary and Joseph, hindi na ako makahinga. Talagang parang tuluyan nang "Masarap na Kalbaryo sa Puso ng Ex Ko" No Response No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 98 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S magkakalayo kami ng Ex ko nito. Parang hindi na bolera sa akin ang "Para sa Bata, Para sa bayan!" "Hello, Sir Loloy...ah ok po...a sige po prepare lang po letter kay RD para malaman nila ni Maam Ellen.. TY po," tinawagan ako ni Sir Loloy and head ng ABC+ sa rehiyon. Punta daw sa Makati para sa launching ng pinaghirapan naming dalawang mudules na gagamitin ng mga kumukuha ng edukasyon sa mga Teacher Education Institution. Lipad na naman, 10% na lang ang tsansa ng balikan namin si Ex walang time na sa kanya. Dumating ako ng Makati, sa sobrang pagod ay napaidlip ako. "Puntahan ko kaya si Ex, may time pa ako bago bumalik ng Iloilo,"sabi ko sa sarili. Puntahan ko si Ex, miss na miss ko na sya. Pagbabakasakali na makapagbalikan kami.Sa pintuan pa lang ng bahay nila tanaw ko na siya, naghihintay sa akin. Nakadopa ang dalawang kamay. Ibig sabihin buong puso niya akong tatanggapin, lumuhod ako sa harao niya, at humingi ng tawad sa lahat ng oras, pinabayaan at hindi siya napuntahan para man lang mabisita sya. Tinulungan ko siyang mabuti, naramdaman ko ang sakripisyo nita sa mga sagot dulot ng korona sa kanyang ulo. Sa mga dugo na dumaloy sa dalawa niyang mga paa at palad, nakita ko ang pagdaloy ng dugo na huhugas ng aking sakripisyo sa MTB=MLE. Napagtanto ko nang mga oras na yon na tama siya kailangang magsakripisyo para sa kapakanan ng lahat. Ring....Ring... "Shit, hala ka! nagsisimula na ang opening program, panaginip lang pala," bulong ko sa sarili.. Kinuha ko ang CP ko. "yes Sir Loloy, coming, bababa na po," sagot ko. Ex, salamat ngayon yayakapin ko na ang MTB-MLE, ABC+ at siyempre ikaw; Isasapuso ko na rin and "Para sa Bata Para sa Bayan" dahil heto ang mandato ko. 43 The office where the writer belongs is mandated to Before the ABC+ as a partner, we organized our team as Regional Learning Resource Development and Evaluators but with our ABC+ partner, the tasks were Together We Stand: RO 6 and ABC+ No Response No Response 99 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S ensure the effective and efficient delivery and management of the articulated basic education curriculum. The pressing challenge is how to raise the bar of excellence in reading and numeracy. We have crafted and implemented innovations and initiatives to address this. One of the communications routed to my office was the letter of the ABC+ that seeks approval for a short meeting with the Head of office and party to discuss the project. We had an initial talk with the representative about the project and it motivated us to give a thumbs up to the lighter because of shared responsibilities because the team focused on a particular project in giving Technical assistance to slow-performing schools division in Reading. A great or challenging change in the landscape of curriculum delivery during the pandemic challenges each one how education will continue and the learning outcomes be assessed. But with the strong commitment of the ABC+, they were able to develop a tool for Comprehensive and Rapid Learning Assessment (CRLA) and conducted it at the Beginning and End of the school year, making the DepEd Region 6 confident to say that the teaching=learning facilitated by teachers, parents, and responsible adults are meaningful. The combined efforts of DepEd Regional Office VI and ABC+paints history on the educational journey of Region VI in terms of LR provision, teacher training, and learning assessment. May their journey continue... USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 100 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S meeting schedule. During the orientation meeting, we found out that there are greater possibilities that can help us in terms of expertise, logistics monitoring, and evaluation to improve access and quality. The result of the initial meeting was shared during the meeting of education leaders with testimonial support from the Head of Office and other recipient regions. 44 No Response Indirectly with ABC+. That's how I describe Ting. Ting is more into Arts, more into Aesthetics, and much more into skill performances. Not so on literacy and numeracy. One day, Ting bumped into Loloy the ABC+ guy. He usually has business with Ting's colleagues Tino And Does. They always talked as to the witty-gritty of their upcoming write shop. Hearing the conversion from a distance, the first word that struck Ting is BOREDOM! Days went by...Ting saw herself die from curiosity what's keeping Tino and Do busy. Both of them are affixing their signature page by page on the stocked piles of papers. Right there and then, she asked "From a Distance" No Response No Response 101 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S her colleagues and TIno shared the story behind those bundles of papers. Tino's sharing is not just merely a story but it produces a STORYBOOK! Yes, a storybook! Ting immediately scanned the books and was engulfed by the amazing story and hooked into the pictures on those books. Even it changed her perspective and she did her research about ABC+. Ok...Ahem....wow! All that she utters is what she found out about ABC+'s partnership with DepEd. Since then, Ting admired, respected, and supported the noble endeavor of ABC+ from a distance! 45 March 8, 2019- a DepEd Memorandum was routed to the Curriculum and Learning Management Division (CLMD) from the Office of the Regional Director (ORD) inviting participants to attend the ABEP Co-Design Workshop on March 11-14, 2019 at PanPacific Hotel, manila. Two (2) CLMD Representatives were chosen and I was one of them. It was during this workshop that I came to know "Excuse me, Maam Daisy," Don said as he met Ms. Daisy Santos of the Bureau of Learning Resources at the hotel lobby. "Can I ask you a question?" he added. "Yes, Sir. What is it about?" she smilingly replied. "If I may ask, do we have standards or guidelines in developing decodable and leveled readers for our key Stage 1 learners?" "Ah, that's still a work in progress," she immediately replied. "Oh, thank you, Maam Daisy," Don said as he moved his way back to the workshop session room. At the back of his mind was "How can we develop those learning processes if we do not have the standards or guidelines?" -UNFINISHED- "The Long Wait is Over" No Response No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 102 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S about AB+ Project. From then on, I was involved in almost all of them the activities of the learning resources component as the Learning Resources Management Section Education Program Supervisor of DepEd Region VI. 46 As Chief of the Education Support Services Division of the Regional Oddice VI, I was invited to attend the orientation with the identified beneficiaries such as Iloilo City, Negros Occidental, Roxas City, Antique, and Escalante City. It was because our office takes charge of forging partnerships with stakeholders After a series of conferences, consultations, engagements, and agreements, DepEd has already established a stable partnership with ABC+. We have availed of gadgets such as risograph which was used in the reproduction of self-learning modules during the pandemic and provided supplies during the expansion of the limited face-to-face. Recently, we were invited to an Industry Leaders Conference where linkages were expanded to help DepEd provide learning resources. Private sector engagements to its fullest for mental health education, Child-Protection and Save the Children initiatives. ABC+ is also an affirmed partner of the Adopt-a-School Program. It got its award during the National Brigada Eskwela Awards. Knowing ABC+ proves that education is everybody's business. "What's your Business?" No Response No Response 103 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S through a Memorandum of Understanding or Agreement in the provision of additional education resources needed by the school's division, let it learning resources, teacher training, provision of gadgets or infrastructure, and wellness. Then a series of conferences with the top management and division partnership coordinators were conducted. 47 No Response I am a grade 2 class teacher at the Iloilo Central Elementary School. No Response No Response No Response 48 No Response Ako po si Noraisah T. Guiahal, may asawa. Graduate po ako ng BEED. Two years po ako nagturo ng Kindergarten sa isang private school, dito sa Cotabato Shariff Kabunsuan College Inc. Hindi po ako pinalad na maging Licensed Professional Teacher. Proud parent stakeholder ng BES. Nagkaroon po ako ng idea tungkol sa ABC+ through the initiative of BES teachers, dati, wala akong alam sa ABC+ dahil pero dahil doon Parents with a Purpose No Response No Response Chosen Story "Ang tulong ng ABC+ sa amin" - Ito po ang story na napili namin USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 104 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S nagkaroon ako ng idea about Advancing Basic Education in the Philippines MBHTE-BARMM. As a parent maraming nabago sa buhay ko, sana magtuloy￾tuloy na ang programang ito para kahit papaano meron kaming/ako ambag na maitulong sa mga teacher sa pagtuturo sa pamamagitan ng pag-gabay sa module ng mga anak namin. kung gagawan ko ng titulo ang storya ng buhay ko ay "Parent with a Purpose". Dahil sa ABC+ Program, feeling ko hindi lang ako mother ng four boys ko, kundi role model nila at least of the other mothers. Happy to serve. Bubong Elementary School. 49 No Response Ako si Catherine May N. Velarde, Teacher-I na nagtuturo sa Paaralang Elementarya ng Bubong. Walong taon na rin akong nagtuturo sa unang baitang. Marami na rin akong mga seminars and training na nadaluhan. Nakilala ko ang ABC+ noong nagkaroon kami ng virtual orientation noong January 27, 2020 Pagkatapos ng Virtual orientation ay binigyan kami ng Assessment Tool na kung saan ay susukatin ang kaalaman ng aming mag-aaral sa Literacy. Ito ay CRLA o Comprehensive Rapid Literacy Assessment. Noong March 25 to 27, 30- 31 ay nagkaroon kami ng Teacher’s Training tungkol sa ABC+, dito ko nakilala si ABC+. Sa limang araw na training/workshop, marami akong natutunan sa ABC+. Natutunan ko ang iba’t ibang estratehiya at Teknik sa pagturo ng pagbasa, pagbasa ng salita, parirala, pangungusap, at kwento. Nariyan ang SEL at GESI at marami pang iba. Natutunan o nakilala sa ABC+ ang iba’t ibang approach na ginagamit sa pagtuturo na alam kong ginagamit ko na sa mga nakaraang taon ng aking pagtuturo. Ngunit may tamang ngalan o tawag pala rito. Malaki ang naitulong ng ABC+ sa aming mga guro, magaaral at mga magulang dahil ito ay isa sa nagbibigay daan sa amin kung paano naming Ang Paglalakbay na kasama si ABC+ No Response This story is significant for me because I learned a lot from ABC+. 105 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S mauunawaan ang aming mga mag-aarl na may iba’t ibang kakayahan sa pagbabasa. Sa tulong ng ABC+ nabibigyan namin ng kaalaman ang mga magulang ng mga bata at nagiging partner namin sila sa paghubog ng aming mga mag-aaral at sa bagong henerasyon na darating. 50 No Response Ako po si Devorah Kasim Datumanguda, bagong punong guro ng Mababang Paaralan ng Bubong. Nakilala ko si ABC+ sa maiksing panahon. ABC+ na handang tumulong, magbigay ng mga stratehiya sa mga guro, mag-aaral at magulang upang higit na umunlad ang pagkatuto ng pagbasa, pang-unawa ng mga mag-aaral sa pamamagitan ng pagbibigay ng mga babasahin (SRMs). Bilang isang punong-guro, marami pa akong nalahukan na training, ngunit ang ABC+ ay hindi ko makakalimutan. Natutunan ko ang SEL and GESI, dahil sa ABC+. Ang ABC+ program ay malaki ang naiambag niyang pagbabago sa larangan ng pagtuturo ng mga guro at pagkatuto ng mga bata dahil sa mga activities nito na madali maunawaan ng guro at bata kasama na ang mga magulang. Ang ABC+ ay napakahalaga sa pagbibigay ng pagbabago. Mula sa akin bilang punong guro at sa mga guro sa larangan ng stratehiya sa pagtuturo ng mga bata. Natuto ang bata sa pagkilala ng mga letra, hanggang sa pagbasa ng mga salita, mga magulang na handang gumabay sa kanilang mga anak upang matuto. ABC+ ang Tunay na Gabay ABC+ gives more strategies to the Teacher and learners learn more reading. ABC+ Program gives importance to the part of parents to the learning process of their children. ABC+ is also a program composed of School and Community involvement to give quality learning to the learners. 51 No Response Ako po si Norhana D. Aliman, apat na taon ng nagtuturo ng Grade 3 sa pampublikong Paaralan ng BARMM, particular sa Bubong Elementary School, Cotabato City. Noong January 27, 2022, nagkaroon ng Ang Tulong ng ABC+ sa amin It is a very significant story because it is a form of gratefulness to the ABC+ program The ABC+ provided everything. I need to make USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 106 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S online orientation ang ABC+ para sa lahat ng pampublikong guro ng Cotabato City Division na nagtuturo ng Kindergarten, Grade 1, Grade 2 at Grade 3. Noong araw na rin una kong nalaman ang patungkol sa ABC+ at sa iba pang aktibidad nito. Noong March 25- 27, 30-31 2022, nagkaroon kami ng five days Comprehensive Training tungkol sa ABC+ programs and activities. Before the training, curious ako sa objective and purposes ng training na ito. Nang Nakita ko ang programs, nagtatala pa ako sa mga term tulad ng SEL and GESI, CPR and LEA at sa iba pang term. At nang magsimula na ang training, napagtanto kong para ito sa mga bata ng Bangsamoro. Noong hindi pa kami naka-training, nagtuturo lang kami ngunit ang available reading materials sa school, ang process naman na ginagamit naming ay hindi ganun ka organized. Ngunit nang matapos ko ang training ng ABC+, nagkaroon na ako ng idea sa pagpapabasa sa mga bata. Napakalaki ng naitulong ng ABC programs at materials sa akin. Bukod sa napapanahon ito sa bagong programa ng MBHTE, ang 5Bs ay mas napadali ang pag-assess sa mga bata. Hindi na rin magkakasakit ang ulo ko sa paggawa ng SRMs, dahil provided na ito ni ABC+. Mas malakas na rin ang involvement and participation ng parents at stakeholders dahil aware sila sa mga programs and activities ng ABC+. Dahil sa pagtutulungan, kasama ang mga magulang at tagapag-alaga sa pamumuno ng aming punong-guro, naging maayos ang Sistema sa pagtuturo sa mga bata at unti-unti ay napapabasa na namin sila. to make our teaching life easier as a teacher. my pupil read. I was able to create a clear goal using the strategies and materials that were introduced. 52 No Response I am Faudzia Esmael Butocan. A parent of three pupils from Mohammad Integrated School. As a parent and a vendor, it is not easy for me to teach my three children. I have to work hard to feed them. But as a parent, it is my responsibility as well to do other jobs for them, I have to teach them to help them gain the knowledge Education is the value of Every Human Being. The significance of my story helps motivate my co￾parents to guide and not teach their children or push We know the innovations are good and bad but I do believe that if we can guide our Chosen Story A busy parent but still manages to teach/follow up with children and it 107 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S that they deserve most. I am very grateful that the ABC+ program helped my children's needs concerning the learning materials that they'd been sharing with us. Even the teachers always encouraged the parents to guide them in this crisis time of the pandemic. If I compare my experience before and after ABC+, it is far from my expectation. My children learned a lot from those inside the program of ABC+, though there are times when I wasn't able to teach my children well. The advantage of ABC+ is that during this pandemic, I have time for my children to teach them, though changes may happen. Modules helped a lot to boost my children’s knowledge. Their time is not wasted playing and doing nonsense. The effects are really good because I do believe that modules help strengthen the learner's knowledge and produce more thinking activities. Because I believe that education is the value of every human being. their children to stop learning, instead push them to learn and dream high. children on how to use them properly, I mean "TIME MANAGEMENT WILL DO", they will learn to discipline themselves from inside to outside. also encourages and motivates children to pursue education despite the pandemic. 53 No Response I am Fairudz W. Bangen currently teaching at Mohammad Integrated School. I have been teaching the Grade 3 level for almost five years. I just know the ABC+ since they conducted training, particularly for Kinder to Grade 3 Teachers. During the training, I have learned a lot of strategies that can be utilized to improve the literacy of our Bangsamoro learners. I have also learned that it is important to integrate SEL and GESI in my teaching. Before, I had difficulty with how I could improve the reading skills of my learners. Alhamdulillah, I learned different strategies from ABC+. Elkonin boxes help with letter sounding, it is easy to read CVC words, then from words to phrases and phrases to words. GESI and SEL "The Change Brought by the ABC+" The story is significant for me because it is my journey as a Grade 3 Teacher who will teach learners and attain 5Bs. No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 108 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S also remind me to integrate this in teaching because before, I just picked pictures for my teaching materials without considering SEL and GESI. I consider that the change is good because I can see the improvement in my learners. They are improving when it comes to reading. As a teacher, the change is good because it enhances my teaching strategies and shows the impact of it on my learner’s development. 54 No Response I am Saria D. Ali, a Principal II of Mohammad Integrated School located at Biniruan, Poblacion 9, Cotabato City. I first learned about ABC+ back when we had the training last March 2022, and the activities that were facilitated and conducted there were quite amusing, strategies were refreshed, and teachers were so excited about the training. Before the ABC+ Training, we usually had practice of some of the activities similar to those of the ABC+, few teachers were already practicing some of those strategies though some were not. Changes included the interests and motivation of teachers to make reading more interesting and enjoyable. The change is good. Teachers are now guided with the materials and more clearly defined in terms of tools that are utilized. The change is significant for me since the school is using the tool already and the effect on teachers is to easily identify the reading level of pupils what is good in there is that there is an assessment of reading levels with enough time, provided with materials. The most important thing is the innovation where teachers conducted Action Research, utilized this research, and even adopted it by other teachers. Further, this also enhanced our SMILE Project where the focus is Synergy in Mentoring for Intensified Learning Enhancement. No Response No Response No Response 109 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S 55 I learned the inclusion of GESI and SEL in teaching, working with other teachers from other schools. Ako po si Consmarie T. Guillen, naging guro sa loob ng dalawampung (20) taon. Mula sa 11 taon sa pribadong paaralan hanggang sa mailipat ako sa pampublikong paaralan ng kung saan nasabi kong ito ang tunay na buhay ng pagtuturo. Napadpad sa malayong isala ng Timako, mga mag-aaral at magulang na nasa laylayan ng kahirapan. Taong 2013 ng ako ay mapadpad roon at sa loob ng siyam na taon, kahit papaano ay masasabi kong hirap sa pagbabasa ang mga kabataan doon. March 27, 2022, nagkaroon ng limang araw na training na kung saan sa training ito ay higit pang naragdagan ang kaalaman ng mga kaguruan sa kanilang mga istratehiya sa pagbabasa at pagbibilang. Ang mga ibinahaging aklat ng ABC+ ay nagagamit at kinalulugdan ng mga kabataan. Panahon ng Summer Reading, mas pinaigting ang program ng pagbasa sa mga batang nasa frustration level. At Alhamdulillah, mas naging positibo sila sa kanilang kagustuhang matuto sa ngayon, mas kailangan ng mga batang ito ang commitment, passion ng mga kaguruan upang higit pang matuto ang kabataang ito. Sa ngayon higit pang isinusulong ang programa ng MBHTE ng 5Bs sa lahat ng paaralan gamit ang mga natutunan sa training ng ABC+. “ABC+ Mabuhay, Sigay ka Cotabato City Division”. "May Pag-asa sa Pagbasa" Because of this, I was completely aware of the need for pupils to learn to read, count, and write. (Dahil dito ko lubusang naramdaman ang pangangailangan ng mga mag-aaral na matutong bumasa, magbilang, at magsulat.) ABC+ innovations can be of great help on children’s learning preferences. (Malaki ang maitutulong ng mga innovation ng ABC+, kagustuhan sa pagkatuto ng bata.) 56 I learned the ABC+ activities during the conducting seminar last March 2022 integrating SEL and GESI. Ako po si Gingging E. Acob, nagtuturo sa Pagalamatan Elementary School bilang guro ng ika-dalawang baiting (grade 2). Bilang isang guro ng nasa ikalawang baiting kailangan ko po ng maraming strategy sa pagtuturo ng pagbasa (o reading) para mai-apply ko sa mga bata. Laking pasasalamat ko at isa ako sa mga nakalahok sa seminar/workshop ng ABC+ at dahil dito marami akong natutunan na istratehiya sa pagtuturo ng pagbasa. Pagkatapos ng aming seminar/workshop sa ABC+ "Ang Aking Mag-aaral" No Response No Response N/A N/A USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 110 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S masaya at nagagalak akong gamitin ang mga istratehiya na aking natutunan sa aking mag-aaral at nakitra kong Malaki ang naitulong ng ABC+ sa aking mga mag-aaral. 57 I learned the ABC+ activities during the conduct of a seminar or training of PATHWAYS last February 28 to March 4, 2022, entitled Training of Literacy and Use of SRMs: Integrating (SEL) Socio-Emotional Learning and Gender Equality and Social Inclusion (GESI) in Learning Language and Literacy in school and at home. I am AMIRA T. HASIM, a teacher 1 of Pagalamatan Elementary School. I know this ABC+ Program/Activities during the conduct of Training for Trainers on Literacy Strategies and use of SRMs: Integrating SEL and GESI in Learning Language and Literacy in School and at home last February 28 to March 4, 2022. Before ABC+, I can say that my knowledge about some strategies that I learned from my previous seminar is not vividly clear to me, because every time I have seminars I am comparing what I have learned from it, and every time I am comparing it, there are some points that they contradict with each other. After I have learned the ABC+, all strategies, techniques, and some points that may help the teachers to apply it in the classroom were strengthened. It helps me as an educator to have a clear knowledge about some strategies that can be used by us in teaching our learners, especially in Literacy and Numeracy integrating the SEL and GESI. There are some changes in terms of teaching my pupils, and these changes help me to promote equal treatment to my learners, especially now that I have learned the SEL and GESI. This change is very important because it positively affects the learners. I can say that my learners develop a lot, it may not be perfect but gives a positive impact on the young Bangsamoro learners in our school and this BARMM region. Our goal is to promote quality and competitive Bangsamoro Learners in this region. "A Love for a Children" No Response No Response Chosen Story (1) The story is clear and it talks about children and (2) The story shows love for the children. 58 No Response As a parent, I learned how to use the reading materials that CRLA provided in my child’s teaching reading through school orientation. (Bilang magulang, natutunan ko sa pamamagitan ng orientation ng school No Response No Response No Response N/A N/A 111 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S sa kung paano gamitin ang mga reading materials na ibinigay ng CRLA sa pagtuturo ng aking anak sa pagbabasa.) 59 Through the Division Memorandum from Cotabato City and MBHTE￾BARMM cascading ABC+ programs and projects. Si Omar isang batang hindi gaano na develop sa kaniya ang pagbabsa. Sa tuwing pinapabasa ni Ma’am para ma-enhance ang reading skills niya ay wala siyang concentration, and always he said “no” at “tinatamad po ako, Ma’am”. Makabasa naman po siya ngunit hindi gaano ka-fluent magbasa. Pagdating sa bahay si Ina Namraida ay ganoon din, nahihirapan magbasa sa parehong dahilan parating sinasabi na “tinatamad ako magbasa”. Isang araw po sa dahilang project: Nanay Ko! Guro Ko! Mas nabigyan pansin at diin si Omar na madagdagan ang interest na magbasa. Ang proyektong ito ay para sa mas malakas na kolaborasyon ng teacher at nanay para sa continuity of reading skills kabilang ang numeracy development. Simula noong pumasok si ABC+ kapansin-pansin ang mga pagbabagong ugali kay Omar towards reading skills development. "Project: Nanay Ko! Guro Ko!" The change in Omar’s attitude towards reading with heart through the project. A new “strategy” that can easily grasp the heart of a certain child (Omar) through a strong collaboration of teachers and parents. 60 I have been an ABC+ translator online and have been a trainee to be a trainer to my fellow teachers to share better knowledge from ABC+ programs and to make it easier for us to teach or to guide our daily teaching, especially to the strategies we Ako ay naging isang translator na manunulat ng ABC+ sa pamamagitan ng online at naging trainee ako upang maging trainer sa kapwa ko guro upang mabahagi ang magandang kaalaman mula sa ABC+ na programa upang mas madali ito sa aming pagtuturo o maging gabay ito sa aming araw-araw na pagtuturo lalo na sa mga strategies na aming natutunan. Kami ay nagkaroon ng CRLA Assessment upang alamin o malaman ang kakayahan ng aming mag-aaral upang maging gabay natin ito sa kung saan ang mas nangangailangan ng gabay gamit ang 5B’s na programa ng MBHTE sa aming mga mag-aaral at ito ay matututukan upang si equity ay maging equality ang kaalaman para sa mga bata. Ibinahagi rin naming ito sa aming mga magulang kung paano gumamit ng SRM’s at mga strategies na aming natutunan mula sa training "Pagtutulung an sa Kaalaman" Teachers and parents have had a better interconnection to learn more about how to teach Bangsamoro children or our students through the use of the ABC+ program, SRMs, video lessons, and short stories. (Nagkaroon ng mas magandang interconnection ang This is significant because we know that the wisdom of knowledge is in the teacher’s and parent’s connection or association to achieve better children’s learning with the help of ABC+. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 112 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S have learned. (Ako ay naging isang translator na manunulat ng ABC+ sa pamamagitan ng online at naging trainee ako upang maging trainer sa kapwa ko guro upang mabahagi ang magandang kaalaman mula sa ABC+ na programa upang mas madali ito sa aming pagtuturo o maging gabay ito sa aming araw-araw na pagtuturo lalo na sa mga strategies na aming natutunan.) naming na hatid ng ABC+. Ang kaalaman na ito ng mga magulang ay kanilang ginagamit sa kanilang tahanan upang turuan ang kanilang mga anak. Sa mga guro at magulang lagging naming isinasa alang-alang ang SEL at GESI na programa ng ABC+. guro at magulang upang mas matutukan kung paano maturuan ang mga batang Bangsamoro o aming mga mag-aaral sa pamamagitan ng paggamit ng ABC+ program, SRMs, video lessons, and short stories.) (Naging makabuluhan ito dahil malalaman natin na ang karunungan ay nasa samahan ng guro at magulang upang makamit ang mas magandang kaalaman o learning ng mga bata sa tulong din ng ABC+.) 61 I was invited by ABC+ as a writer during the development of Decodable and Leveled Reading materials using Blooms App. I’ve been a writer in several workshops on developing reading materials in Magindanawn before the coming of ABC+ and before the inclusion of Cotabato City in BARMM. I am so grateful that my skill in contextualized material development is being recognized by our division and thus always included in every program and activity that ABC+ is conducting in support of the MBHTE’s 5B’s Program. I am invited as a writer of Magindanawn stories, a member of the Quality Assurance Team of Video Lesson Script in Matya Tanu, Magindanaon (BRP), and a Language Coach in Video "The Contextualize r" This story is significant since all the activities that I am involved with are aiming to support the 5 Bs program of the MBHTE and as an intervention to the CRLA result. The contextualized stories and video lessons for beginning reading are so significant since the teachers’ skills were developed during the Chosen Story (1) Involved in the development of materials developed by ABC+; (2) Developed materials are useful to teachers and learners, and 113 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S Lesson Recording by ABC+. My skills were more developed and I am happy too that I was able to recommend some of my co-teachers who have talents in broadcasting so that they can also share in the achievement of ABC+ to support ABC+. I learned a lot in script development and in the skills of broadcasting, even in writing stories (listening to small books, and big books) using Blooms App because I engaged with ABC+. Looking forward to the time that these video lessons will be out in the field because I know it would be very beneficial to the learners, especially to the beginning reading learners. development of the materials and hopefully, the product will help in filling the academic gap, especially in Literacy of the Learners. (3) Currently involved in developing the Beginning Reading materials in support with 5B’s. 62 It was introduced by the teachers to the parents and performed some activities for the learners to easily understand the context of the materials. I am Baiheria, a devoted parent for my seven-year-old son. He is a grade 2 learner from TCS. As a parent, the most significant change I had was the bonding my son and I build all throughout. When the ABC+ introduced to the parents, we were able to understand the mission-vision of the program. The parent got involved which is very important because my son and I spent more time to each other. Ang lagi niyang sinasabi sakin ay “ina let’s read a story” mas nagkaroon kami ng conversation para pag-usapan ang mga bagay na dati ay nasa loob lang niya. Ngayon na mas involved na ako as “nanay” mas napapa-intindi ko na sa kaniya ng maayos at mas naging Madali sa kaniya ang matuto. Mas naging interesado ito sa kaniyang pag-aaral, natutunan ko din ang tamang approach para mas matulungan ko siya since parent also become their teacher at home. Mas napaintindi ko sa kaniya na gender equality is very important. Mahalaga bilang magulang na ma-engaged sa learning process ng anak dahil kami, mga nanay, ang kasama nila sa bahay. Dahil sa program na ito mas naintindihan ko ang effort and sacrifice of teachers to teach our children. I am also thankful because I got the chance to be my son’s teacher at home and I can say that mas naintindihan ko "Teacher at Home" It is significant because my son and I build a firm bonding. My son learns to open himself to me. Innovation is very important especially in the learning process because I believe that learning is a continuous process. It never ends and every day is a new thing to learn. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 114 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S ang tamang pamamaraan upang matuto ng maayos at maging epektibo sa mga bata. Mahirap din dahil minsan kailangan hanapin ang kiliti at interest ng bata. I am thankful to BAC+ because I was able to understand the process for my son’s learning journey. 63 Teaching reading with the integration of SEL and GESI. Ako si Leonila A. Palma, punong guro ng Usman Baunga Elementary School, Tamontaka 2, Cotabato City. Isa sa mga paaralan ng Cotabato City ng Dibisyon ng Lungsod ng Cotabato. Nakilala ko and ABC+ dahil sa inimbitahan ako at ang aking mga guro na dumalo sa isang makulay na training sa pagtuturo ng pagpapabasa ng mga kabataang may kahirapan sa larangan ng pagbabasa. Ang CRLA assessment tool ang naging gabay upang lalong makilala namin ang reading level ng mga bata at mabigyan ang bawat isa ng kaukulang lunas sa pagpapabasa. Bago ko makilala sa ABC+, nag-aassess lang kami sa pagpapabasa sa pamamagitan ng sari￾sariling reading tool. Dahil kay ABC+ binigyan kami ng gabay upang lalong makilala ang reading level nila at maibigay sa kanila ang wastong reading material na naaayon sa kanilang ability. Dahil dito nabago ang pananaw ng bawat guro sa pagtuturo ng pagbasa at naging masigasig ang mga guro sa pagtuturo dahil sa kanilang bagong kaalaman sa ABC+. Sadyang mahalaga na mapaunlad ang kaalaman ng ating mga guro upang lalong mahasa ang kani-kanilang pagtuturo. Dahil dito, lahat ng batang Bangsamoro ay matuturuan at lalong lalo na, “Walang Batang Bangsamoro ang Mapag￾iiwanan” at dahil sa 5B’s “Bawat Batang Bangsamoro ay Bumabasa at Bumibilang.” "ABC+ at its Best" We learned a lot! NR I love this story. (1) It is a message for Bangsamoro Children. (2) It shows how their reading teaching differs before and after they've known ABC+. (3) It was also shared how she gave importance to the sustainability of the ABC+ program. 64 I am Razul Omar, and as a teacher, I have learned that ABC+ activities are significant in I am Razul Omar, a teacher-1 from Usman Baunga Elementary School. I noticed a lot of changes in children’s reading skills. Before, the children were unable to read and through ABC+ training and activities, I used and shared what I learned to develop the "Halina't Tayo'y Magbasa" The story is important to me because it has something to do with the children’s This is beneficial because it shows a significant change in the N/A N/A 115 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S teaching reading. Through this, I found out what appropriate interventions or strategies can be used to teach children to read and develop their reading skills. (Ako si Razul Omar, bilang isang guro, natutunan ko na ang ABC+ activities ay napakahalaga sa pagtuturo ng pagbabasa. Sa pamamagitan nito, nalaman ko kung ano ang angkop na interventions ang pwedeng i-apply sa pagkatuto ng mga bata.) children’s reading skills. (Ako si Razul Omar, isang teacher 1 sa Usman Baunga Elementary School. Marami akong napansin na pagbabago sa mga bata. Dati ang mga bata ay hindi nakakabasa at sa pamamagitan ng ABC+ training and activities, nagamit ko ang mga natutunan ko upang matutong magbasa ang mga mag-aaral.) preferences towards teaching reading. (Mahalaga ang kwento sa akin dahil ito ay may kinalaman sa pagkatao ng mga bata sa pagbasa.) children’s reading capabilities. (Nakakatulong ito dahil ito ay nagpapakita ng pagbabago sa pagbasa ng mga bata.) 65 I learned about ABC+ activities when there was a writeshop about making stories (Decoding/Levele d Reading Materials) using Ako po si Parida B. Radia. Nagtuturo sa Usman Baunga Elementary School. Nagkaroon ng pagbabago sa aking pagtuturo lalo na sa pagbabasa ng mga mag-aaral. Nalaman ko po kung ano ang dapat basahin ng mga mag-aaral o kung saan dapat magsimula sa pagtuturo ng pagbabasa sa mga mag-aaral. Naging madali sa mag-aaral ang pagkatuto sa pagbasa dahil sa mga ginawang reading materials na umaayon sa kanilang "Pagbasa, naging Madali" This story is important to me because it made my teaching reading better and more efficient. (Mahalaga sa akin This has become important because of the beneficial changes in our (teachers') reading. Chosen Story N/A USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 116 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S the Blooms App. I am one of those invited to participate as a writer on such a writeshop. We made different stories according to the level of the child. This was conducted in Emanor, Cotabato City. (Nalaman ko po ang ABC+ activities noong nagkaroon ng writeshop tungkol sa paggawa ng stories (Decoding/Levele d Reading Materials) gamit and Blomms App. Isa po ako sa inanyayahang sumali bilang writer sa naturang writeshop. Gumawa po kami ng iba't-ibang kwento ayon sa level ng bata. Naganap po ito sa EM Manor Cotabato City.) level. Ang pagbuo ng babasahin na naayon sa level nila ang naging susi upang mas aktibo silang sumali sa pagbabasa. Naging madali din sa akin bilang guro ang pagtuturo ng pagbasa dahil sa mga nabuong babasahin na ito. ang kwentong ito dahil naging matagumpay ang pagtuturo ko sa pagbabasa.) teaching. (Naging mahalaga po ito dahil sa pagbabago ng ating pagtuturo sa pagbasa.) 66 No Response I am Evangeline U. Cornelio, a grade 2 parent, sharing No Response No Response No Response 117 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S my story about ABC+. It was February when the school principal of UBES called for a meeting for parents’ orientation regarding ABC+. It is a guide for children regarding reading materials. As a parent, I see my child’s development, just like how he recognizes letters by using the reading materials. (Ako po si Evangeline U. Cornelio, isang Grade 2 parent, sharing my story about ABC+. Month of february ng nagpatawag ng meeting ang school principal ng UBES para sa orientation ng parents patungkol sa ABC+. Ito ay guide para sa mga bata regarding sa reading materials. Bilang magulang po nakikita ko ang development ng anak ko, tulad sa kung paano na ito nakaka-recognize ng mga letters by using the reading materials. ) 40 No Response Almost five years ago when we were crafting the DepEd LAC toolkit, when I first met Ms. Ina Aquino and Mr. Loloy Gelmo. We got a chance to talk, collaborate, and have coffee memories in one of the best-known coffee houses in Zamboanga, I thought it ends there, however, in one of the projects introduced to DepEd, the ABC+ birth to Region VI was started with a visit of Mr. Gelmo to Iloilo Central Els. As principal of the school, I've cited his concerns and introduced myself once upon a time we met in a workshop, our stories did not end there having more and more expectations and activities. He laid down his intent to borrow a few teachers and learners to host the Launching and ABC+ Project in WV which will be allowed by several Central, Regional, and Division officials, I really welcomed the request and prepared the school, teachers, and learners for the said request, the launching was done in a hotel with a lot of significant people, there the program was introduced, what surprised me most, that many of my co-writers in The ABC+ Journey for Literacy No Response No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 118 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S the LAC Toolkit were there present too, and of course the presence of Ms. Ina Aquino, we were divide into smaller groups and to craft our dashboard in reading performance and other key indications in MTB, Filipino, and English, we have to present to the group that we need the program, we need to satisfy and convince that we are ready to be the pilot of the program, I was tasked to present the Division output from a smaller workshop. I was so surprised that the group from the ABC+ Co with Ro were convinced that we will be the initial division of the program, knowing of the presence and little background of the program I was identified as pilot school and school heads as the lead of the program, my joining with ABC+ started in their series of activities and workshops were attended by me. Countless conducts of FGDs, school visits, and data analysis were done by them, we feel proud of their visits from the local guests and foreign guests looking for why we are to be considered pilot school for the program, I dwelt so many activities to prefill from conferences, assessment, materials development, contextualization, field visits among others, the relationship with ABC+ has been more intimate the pandemic strikes, I thought we will cease with the program, however it was even more intense , the program has reshaped its delivery of remote learning process, more and more trainings were introduced, the challenges of the pandemic didn't hinder but even more our bond stronger when we collaborated in how things will be done with resilience and strong connection of making change happen, I have embraced the program for the teachers and most especially the Filipino learners we serve, the BC+ is a greatest gift for the Ilonggo Learners to read and write with confidence and have sense of pride. 41 No Response I am a reading advocate since I became interested to be A reading No Response No Response 119 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S an educator, which means I started to love reading more during my college years. Reading can take you around the world. ABC+ programs focus on the importance of reading and I am happy and fortunate to take part in some of its activities I am a Pablo Sohok District Supervisor assigned to supervise schools implementing the ABC+ programs and projects. My involvement with the program/project not only starts and ends with supervision but extended to being an EGRA Assessor. I was an EGRA Assessor for Baseline Collection and Midline Collection of Data. Having gone through several trainings on how to conduct assessments was an eye-opener. The tedious training we had on how to assess pupils' reading skills, including their word recognition, listening, and oral-reading comprehension plus this speed or fluency gave us the confidence to do our task efficiently. I played two different roles during these two assessment periods; first as an Assessor and second as an Interviewer. Both assignments were challenging. My experience made me realize how important the roles of teachers and parents as HIP (especially during the time of pandemic) are in helping their learners/children to read. The use of decodable readers, if had been maximized, assisted both teachers and parents to develop reading comprehension among our learners. Children who were exposed to books and were read frequently by their HIPs tend to be a reader with comprehension over those who were deprived of the books and follow-up. These facts were revealed during the interview of HIPs (parents)during the Midline Assessment/ advocate and EGRA assessment 42 No Response "Hay naku! Diyos ko Lord, bakit ba sa akin pa ibinigay itong coordinatorship ng Mother tongue Based Multilingual Education," reklamo ko sa kasama ko sa office na si Sir Donald. "Hoy, di ba nga three years kang coordinator ng MTB-MLE doon sa Antique sa "Masarap na Kalbaryo sa Puso ng Ex Ko" No Response No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 120 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S pinakamamahal moy Kinaray-a language. Kayang-kaya mo yan, " sagot si Sir Donald. OO, sa Antique pa lang una kong naranasan bilang Education Program Supervisor sa FIlipino and pagiging coordinator ng MTMLE ng Kinaray-a language. Diyos ko po, super hirap at super dami ng travel at trabaho. Oo, tinanggap ko ang trabahong iyon dahil wala namang choice diba? Kahit nakakasama ng sabihing "Para sa Bata Para sa Bayan", yuck ewan ko ba. Aba'y sa totoo lang sukang￾suka ako sa linggwahe parang Ex ko pinabayaan ko. "Hoy, wala ka na naman sa sarili mo. Anong iniisip mo? Si MTB-MLE o si Ex mo? tanong ni Sir Donald habang tinatapik ang balikat ko. " Weeh... Si MTB. Alam kong malaking pasanin ito. Hindi ito Ex ko na pwedeng balik￾balikan." Isang araw noon...."Sir TIno, memo oh, punta ka raw ng Manila dadalo sa Launching ng ABC+," sabi ni RF ang Secretary sa office. "Salamat Sir RF," sabi ko habang nakamot noo na tinatanggap ang memo. Binasa ko ang memo at walang magawa , nangutang kay Maam Nona at lumipad ng Manila. Simula nung araw na iyon, nagsimula na ang kalbaryo ko sa MTB￾MLE. Diyos ko dinagdagan pa ng ABC+ na ito. May panahon pa kaya ako sa lovelife ko or even sa Ex ko? Training dito, training doon, CRLA here and there, speaker dito, speaker doon. Ok lang din sana kasi libre, pinapakain at pinapatulog ka sa magagarang hotel. Diba class? Heto pa ha, pag Quality assure ng mga Early-grade reading Materials sa Kinaray-a, Akeanon, Hiligaynon at iba pa na lengguahe ng rehiyon. Aba'y 483 titles din iyon. Pagsasapuna ng mga Editing Guide etc. etc. My God, mary and Joseph, hindi na ako makahinga. Talagang parang tuluyan nang magkakalayo kami ng Ex ko nito. Parang hindi na bolera sa akin ang "Para sa Bata, Para sa bayan!" "Hello, Sir Loloy...ah ok po...a sige po prepare lang po letter kay RD para malaman nila ni Maam Ellen.. TY po," tinawagan ako ni Sir Loloy and head ng ABC+ sa 121 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S rehiyon. Punta daw sa Makati para sa launching ng pinaghirapan naming dalawang mudules na gagamitin ng mga kumukuha ng edukasyon sa mga Teacher Education Institution. Lipad na naman, 10% na lang ang tsansa ng balikan namin si Ex walang time na sa kanya. Dumating ako ng Makati, sa sobrang pagod ay napaidlip ako. "Puntahan ko kaya si Ex, may time pa ako bago bumalik ng Iloilo,"sabi ko sa sarili. Puntahan ko si Ex, miss na miss ko na sya. Pagbabakasakali na makapagbalikan kami.Sa pintuan pa lang ng bahay nila tanaw ko na siya, naghihintay sa akin. Nakadopa ang dalawang kamay. Ibig sabihin buong puso niya akong tatanggapin, lumuhod ako sa harao niya, at humingi ng tawad sa lahat ng oras, pinabayaan at hindi siya napuntahan para man lang mabisita sya. Tinulungan ko siyang mabuti, naramdaman ko ang sakripisyo nita sa mga sagot dulot ng korona sa kanyang ulo. Sa mga dugo na dumaloy sa dalawa niyang mga paa at palad, nakita ko ang pagdaloy ng dugo naghuhugas ng aking sakripisyo sa MTB=MLE. Napagtanto ko nang mga oras na yon na tama siya kailangang magsakripisyo para sa kapakanan ng lahat. Ring....Ring... "Shit, hala ka! nagsisimula na ang opening program, panaginip lang pala," bulong ko sa sarili.. Kinuha ko ang CP ko. "yes Sir Loloy, coming, bababa na po," sagot ko. Ex, salamat ngayon yayakapin ko na ang MTB-MLE, ABC+ at siyempre ikaw; Isasapuso ko na rin and "Para sa Bata Para sa Bayan" dahil heto ang mandato ko. 43 The office where the writer belongs is mandated to ensure the effective and efficient delivery and management of the articulated Before the ABC+ as a partner, we organized our team as Regional Learning Resource Development and Evaluators but with our ABC+ partner, the tasks were lighter because of shared responsibilities because of the team which focused on a particular project in giving Technical assistance to low-performing schools division in Reading. A great or challenging change in the landscape of curriculum delivery during the pandemic Together We Stand: RO 6 and ABC+ No Response No Response USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 122 RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S basic education curriculum. The pressing challenge is how to raise the bar of excellence in reading and numeracy. We have crafted and implemented innovations and initiatives to address this. One of the communications routed to my office was the letter of the ABC+ that seeks approval for a short meeting with the Head of office and party to discuss the project. We had an initial talk with the representative about the project and it motivated us to give a thumbs up to the meeting schedule. During the orientation meeting, we found out that challenges each one how education will continue and the learning outcomes be assessed. But with the strong commitment of the ABC+, they were able to develop a tool for Comprehensive and Rapid Learning Assessment (CRLA) and conducted it at the Beginning and End of the school year, making the DepEd Region 6 confident to say that the teaching=learning facilitated by teachers, parents, and responsible adults are meaningful. The combined efforts of DepEd Regional Office VI and ABC+paints history on the educational journey of Region VI in terms of LR provision, teacher training, and learning assessment. May their journey continue... 123 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONDENT NUMBER QUESTION 115 QUESTION 3 (STORY)16 QUESTION 4 (TITLE)17 QUESTION 518 QUESTION 619 REMARKS REASON/S there are greater possibilities that can help us in terms of expertise, logistics monitoring, and evaluation to improve access and quality. The result of the initial meeting was shared during the meeting of education leaders with testimonial support from the Head of Office and other recipient regions. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 124 ANNEX 9. LEVEL 2 MOST SIGNIFICANT STORY20 STORY CODE TITLE STORY REASON L201 Parents with a Purpose Ako po si [NAME] may asawa. Graduate po ako ng BEED. 2 years po ako nagturo ng Kindergarten sa isang private school, dito sa Cotabato Shariff Kabunsuan College Inc. Hindi po ako pinalad na maging Licensed Professional Teacher. Proud parent stakeholder ng BES. Nagkaroon po ako ng idea tungkol sa ABC+ through the initiative of BES teachers, dati, wala akong alam sa ABC+ dahil pero dahil doon nagkaroon ako ng idea about Advancing Basic Education in the Philippines MBHTE-BARMM. As a parent maraming nabago sa buhay ko, sana magtuloy-tuloy na ang programang ito para kahit papaano meron kaming/ako ambag na maitulong sa mga teacher sa pagtuturo sa pamamagitan ng pag-gabay sa module ng mga anak namin. kung gagawan ko ng titulo ang storya ng buhay ko ay "Parent with a Purpose". Dahil sa ABC+ Program, feeling ko hindi lang ako mother ng four boys ko, kundi role model nila at least of the other mothers. Happy to serve. Bubong Elementary School. "Ang tulong ng ABC+ sa amin" - Ito po ang story na napili namin L202 Education is the value of Every Human Being. "I am [NAME]. A parent of 3 pupils from Mohammad Integrated School. As a parent and a vendor, it is not easy for me to teach my three (3) children. I have to work hard to feed them. But as a parent, it is my responsibility as well to do other jobs for them, I have to teach them to help them gain the knowledge that they deserve most. I am very grateful that the ABC+ program helped my children's needs concerning the learning materials that they'd been sharing with us. Even the teachers always encouraged the parents to guide them in this crisis time of the pandemic. If I compare my experience before and after ABC+, it is far from my expectation. My children learned a lot from those inside the program of ABC+, though there are times when I wasn't able to teach my children well. The advantage of ABC+ is that during this pandemic, I have time for my children to teach them, though changes may happen. Modules helped a lot to boost my children’s knowledge. Their time is not wasted playing and doing nonsense. The effects are really good because I do believe that modules help strengthen the learner's knowledge and produce more thinking activities. Because I believe that education is the value of every human being." We know the innovations are good and bad but I do believe that if we can guide our children on how to use them properly, I mean "TIME MANAGEMENT WILL DO", they will learn to discipline themselves from inside to outside. L203 "A Love for a Children" "I am [NAME], a teacher 1 at Pagalamatan Elementary School. I learned about this ABC+ Program/Activity during the conduct of Training for Trainers on Literacy Strategies and Use of SRMs: Integrating SEL and GESI in Learning Language and Literacy in School and at home last February 28 to March 4, 2022. Before ABC+, I can say that my knowledge about some strategies that I learned from my previous seminar is not vividly clear to me, because every time I have seminars I am comparing what I have learned from it, and every time I am comparing it, there are some points that they contradict with each other. (1) The story is clear and it talks about children and (2) The story shows love for the children. 20 Removed actual names for privacy and confidentiality. 125 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV STORY CODE TITLE STORY REASON After I have learned the ABC+, all strategies, techniques, and some points that may help the teachers to apply it in the classroom were strengthened. It helps me as an educator to have a clear knowledge about some strategies that can be used by us in teaching our learners, especially in Literacy and Numeracy integrating the SEL and GESI. There are some changes in terms of teaching my pupils, and these changes help me to promote equal treatment to my learners, especially now that I have learned the SEL and GESI. This change is very important because it positively affects the learners. I can say that my learners develop a lot, it may not be perfect but gives a positive impact on the young Bangsamoro learners in our school and this BARMM region. Our goal is to promote quality and competitive Bangsamoro Learners in this region. " L204 The Contextualizer "I’ve been a writer in several workshops on developing reading materials in Magindanawn before the coming of ABC+ and before the inclusion of Cotabato City in BARMM. I am so grateful that my skill in contextualized material development is being recognized by our division and thus is always included in every program and activity that ABC+ is conducting in support of the MBHTE’s 5B’s Program. I am invited as the writer of Magindanawn stories, a member of the Quality Assurance Team of Video Lesson Script in Matya Tanu, Magindanaon (BRP), and a Language Coach in Video Lesson Recording by ABC+. My skills were more developed and I am happy too that I was able to recommend some of my co￾teachers who have talents in broadcasting so that they can also share in the achievement of ABC+ to support ABC+. I learned a lot in script development and in the skills of broadcasting even in writing stories (listening to small books, big books) using Blooms App because I engaged with ABC+. Looking forward to the time that these video lessons will be out in the field because I know it would be very beneficial to the learners, especially to the beginning reading learners. " The contextualized stories and video lessons for beginning reading are so significant since the teachers’ skills were developed during the development of the materials and hopefully, the product will help in filling the academic gap, especially in Literacy of the Learners. L205 "Pagbasa, naging Madali" Ako po si [NAME]. Nagtuturo sa Usman Baunga Elementary School. Nagkaroon ng pagbabago sa aking pagtuturo lalo na sa pagbabasa ng mga mag-aaral. Nalaman ko po kung ano ang dapat basahin ng mga mag-aaral o kung saan dapat magsimula sa pagtuturo ng pagbabasa sa mga mag-aaral. Naging madali sa mag-aaral ang pagkatuto sa pagbasa dahil sa mga ginawang reading materials na umaayon sa kanilang level. Ang pagbuo ng babasahin na naaayon sa level nila ang naging susi upang mas aktibo silang sumali sa pagbabasa. Naging madali din sa akin bilang guro ang pagtuturo ng pagbasa dahil sa mga nabuong babasahin na ito. This has become important because of the beneficial changes in our (teachers') reading. teaching. L206 ABC+, Salamat sa Pagkatuto Ako ay isang magulang mula sa Tiwi Central School na buo ang suporta at pagpapahalaga sa pag-aaral ng aking mga anak. Wala naman akong gaanong naging problema sa aking mga anak dahil sinanay ko sila sa pagbabasa mula sa kanilang murang edad at ito ay kanilang nakahiligan. Madalas akong sa eskwelahan dahil isa akong GPTA official at teacher. Kung minsan nasaksihan ko na maraming mga bata ang talagang hirap bumasa at walang hilig sa pagbabasa. Isa sa dahilan nila ang kakulangan ng mga babasahin kaya’y hindi atraktibo sa kanila ang mga aklat sa silid-aralan. Nasaksihan ko ang araw na namigay ng mga babasahin ang ABC+ at ako’y nakibasa rin. Sobra akong humanga at bumilib dahil madaling unawain at may makulay na larawan na nakakatawag pansin sa mga bata. Yung tipong kapag pinanood USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 126 STORY CODE TITLE STORY REASON mo ang larawan ay ma-enganyo ka ring basahin ang kwento. Simula noon ay marami na akong nakitang mga bata na nagbabasa nito sa ilalim ng puno at sa covered court at nagkukwentuhan sila tungkol dito. Tunay na napakaganda naidulot nito sa mga bata. Nadevelop ang love for reading nila. Kasi tunay namang napaka-ayos ng pagkakasulat maging ang ilustrasyon o larawan. Madaling intindihin at malinis ang pagkakagawa. Bilang isang magulang, ako’y umaasa na sana’y mapagpatuloy ang mga ganitong proyekto bilang suplemento sa mga batang nahihirapan magbasa at magkaroon ng hilig dito. Nawa’y lalo pang pagpalain ng maykapal ang mga taong nasa likod ng proyektong ito. L207 An Eye Sore, An Eye Opener Learning and development may be overrated but it is integral and indispensable in any office or organization. I have been in the service since I was 20 years old – so 20 years now and half of it handling an administrative position. I was lucky enough to be tapped either as a participant or a trainer. I met ABC+ when all efforts to improve the quality of education are at a crossroads. This was due to the limitations set by the pandemic. This was the time when I learned about the different modalities of conducting training. Especially virtual sessions. It was ABC+ who introduced me to Zoom, GMeet, and Google Classroom! Imagine, handling sessions in person to explaining my topic in front of a computer. I first doubted its efficiency and effectiveness knowing that the internet connection in Legazpi is unreliable. Well, I was proven wrong especially when I changed roles from a participant to a trainer. I can confidently handle a virtual learning session starting from scheduling a Zoom meeting, annotating a screen, handling breakout rooms, and other nitty-gritty. My left eye may have been permanently marked due to eye stress during the more than two weeks of computer radiation. I know that many have enlightened beliefs about online learning. I also believe that our gains ripple infinitely. (Naging mahalaga po ito dahil sa pagbabago ng ating pagtuturo sa pagbasa.) L208 Walang imposible sa pagkatuto ng mga bata Sa panahon ng pandemya at lahat ng paaralan at nagsara ako ay isang guro na naghanap ng paraan kung paano maipagpapatuloy ang pagkatuto ng mga batang nasa unang grado. Ako ay si [NAME], nagtuturo sa Mababang Paaralan ng San Jose, bayan ng Malilipot, Dibisyon ng Albay. Isang guro na handang maglingkod para sa mga bata. Naging trainer ako ng ABC+ TT1 at TT2.Bago magkaroon ng pandemya, ako’y isang guro na ginagampanan lamang ang aking responsibilidad para sa pagkatuto ng mga bata sa loob ng silid-aralan. Subalit nagkaroon ako ng malaking inspirasyon mula sa mga pagsasanay na aking nadaluhan sa ABC+. Naglunsad ako ng isang inisyatibo upang makapagpatuloy ang mga bata na mahikayat matuto, magbasa at mag-aral sa kaunting oras kahit nakasara ang paaralan ng dalawang taon.Humingi ako ng tulong sa komunidad namin lalo na sa mga may kaalaman magturo, sila ang naging kaagapay ko para maitayo ang Learning Home-Libraries. Isang lugar na puwedeng magsilbing paaralan, may mga aklat na puwedeng basahin, well-structured din tulad ng isang klasrum at may guro na magtuturo at gagabay sa mga bata, ang mga volunteer tutors. Sa umpisa mahirap mabuo ang proyekto, dahil may iilan na negatibo ang pananaw, subalit mas nakakarami pa rin ang handa at bukas palad na tumulong sa pamamagitan ng serbisyo at material.Sa kasalukuyan na ang gaming paaralan ay nasa full face-to-face classes na at ang mga namamahala sa home libraries ay pumapasok sa amin bilang Learning Support Aide, pansamantala muna naming tinigil ang pagpunta dito ng mga bata. Magkakaroon ng bagong paghahanda at iskedyul ang mga batang gagabayan sa lugar na ito.Hindi maikakaila na makabuluhan ang papel ng komunidad sa pagkatuto ng mga bata. Anumang hamon sa pagpapatuloy ng pagkatuto ay malalampasan kung tulong￾tulong ang bawat – isa. 127 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV STORY CODE TITLE STORY REASON L209 MY ABC+ READING JOURNEY The reading crisis during the pandemic time was a tough challenge for me as a school leader. It was so timely that ABC+ had partnered with DepEd where it became a part of my ABC+ journey. With all the training that I attended as a regional trainer and even facilitated with the teachers and school heads in the division, somehow and somewhat the learnings on reading strategies and creating a positive climate in the school and at home helped the learners improve their literacy domains. Through my project initiative as a final output which was dubbed Project SMK (Syllable Mastery in Katon way), the number of frustrated readers was lessened. The said project which I implemented in the school reaped a National Award as one of the Noteworthy Literacy Practices and school supplies worth of 10thousand pesos conducted and sponsored by ABC+ during the Learning Conference via Zoom. Indeed, it was realized with flying colors because of the distributed leadership I employed with the external and internal stakeholders in the school. The love for reading among the pupils, teachers, parents, and the community was intensified because of the training conducted by ABC+ and more so the collaborative effort by various stakeholders in combating struggling readers in the school. L210 From Tiny Dot to Gem I am a simple classroom teacher from the far-flung school of Sto. Domingo District. Even though I am the District MTB Coordinator, I have limited participation in workshops and training outside the district. Our Division and Education Supervisor in MTB-MLE introduced to me the ABC+ program and activities. They included me in the Regional Training of Trainers (RTOT) somewhere in August 2019. From that training, I was assigned to train many teachers from our district and other districts as well. Since then, a big change has come into my life. Since then, I am always identified as a facilitator every time our district conducts training about literacy. More workshops and training from ABC+ were given to me as part of my professional growth. Last July 2022, another opportunity was given to me and that was to be part of the Division of Albay Core Team for the 8- Week Curriculum. All the confidence and abilities I am well equipped for today are all because of ABC+ training. They transformed me from an ugly duckling to a beautiful swan. From a tiny dot to a gem. Likewise, I became an effective teacher more than before. Words of gratitude were owed to Deped RO5, Deped Albay Division, ABC+, and Sto. Domingo District for giving me a chance to showcase my talent and skills as a teacher and trainer as well. My enthusiasm to continue to grow is still burning. I still wait for another chance to pursue it. L211 ABC+, A Bridge to One's Childhood Dream I learned about ABC+activities from our Education Program Supervisor in LRMS, [NAME]. A year ago, he invited me to join and participate in the teachers' training for teachers in the development of Minasbate Story Books in Early-grade. It was indeed magical and a roller coaster of an event for me because ever since then, I have been an active and privileged participant in ABC+ training and an active user of their materials. Indeed, my life has changed because of this partnership. Many changes have happened in the past year of implementing ABC+ activities. One of the things that I have witnessed is that children are showing signs of change in behavior in terms of their fondness and love for reading and appreciating USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 128 STORY CODE TITLE STORY REASON books. Teachers were provided with great opportunities because they were able to make Minasbate stories for their students, which helped the kids and helped them grow professionally. The training provided by ABC+ molded and honed the skills of the teachers on how they could teach their learners scientifically, which is beneficial in the field. Among these changes, the most significant I encountered and experienced as a result of ABC+ activities was when I was allowed to finally fulfill my long-time dream – my childhood dream of becoming a writer of children’s books. Ever since I was a kid, I have wanted to write and publish stories that my readers would enjoy and relate to. My dream has finally turned into reality. ABC+ became the bridge to my childhood dream. I never, in my wildest dream, thought that I could become a quality assurance of the books that we developed during those months of training. I feel so privileged and honored because of these opportunities. This is why I am forever grateful and in debt to Sir Juancho Azares and ABC+. It was truly significant for me because not only were my dreams fulfilled, but it also changed the lives and views of the learners when it comes to reading and appreciating books. L212 ABC+ and me It was during the early period of the pandemic when I was called to attend the very first ritual training that I have experienced in my whole life, the capacity building for Learning Resource Evaluators sponsored by ABC+. Even if it was my first time, I never hesitated to attend because I’m a person who is a risk-taker by nature and very willing to learn things that would enhance my knowledge, skills, and attitudes – the 3 main areas of focus that I would like to improve as an individual – because I am dealing with different kinds of people. At this point, I am thankful to our LRMS Supervisor, [NAME] for believing in my capacity. Had it not been for Sir Juancho, I would not be able to discover my potential as a learning resource evaluator and quality assurer of Minasbate Learning Resource Materials. The story I have written was significant to me because the experience made a meaningful change in my life as a person and as a learning resource evaluator. The real-life experience I had with the activities that ABC+ exposed me to was a real behavior-changing experience. I never realized that I could be equipped with enough quality knowledge to assure learning resource materials in Minasbate. But with the support of the ABC+, the Minasbate Editing Guide, the Load Allowance, and the very accommodating facilitators, I was able to help develop quality learning resource materials that grade 1-3 learners of Minasbate can use. Without the innovation from ABC+, my story would have been insignificant. Imagine the big number of Grades 1-3 learners in the province of Masbate who benefit from the locally developed learning resource materials. Indeed, ABC+ was able to make an educational breakthrough that will remain forever igniting in the hearts of the teachers, learners, and parents of Calasuche Elementary School and most schools of the Masbate province. Hat’s off to ABC+ Team. L213 ABC+: A Heaven￾Sent Gift ABC+ in the life of an Education Leader! Assigned working in the Curriculum Implementation Division which focuses on the teaching-learning process of Instruction which is the heart of the Department of Education. In this division, we are facing many challenges not only brought about by the Corona Virus Pandemic but by the low academic performance of our learners especially in the area of literacy and numeracy. ABC+ was introduced to us as one of the pilot divisions in Region VI. Orientation, Seminar Workshops were conducted for Teachers and School Leaders on Learning Materials Deut Teaching Strategies, and others. Before ABC+ was implemented, we experienced many problems or challenges in the 129 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV STORY CODE TITLE STORY REASON delivery of Early-grade Instruction especially in Reading due to the presence of many learners under the frustration level. But with the partnership with ABC+ and its implementation of the Program, Projects, and Activities, there’s a decrease in the number of non-readers. Good changes happened in our learners and teachers. As well in school and in the community, learners’ achievement improved, and teachers’ confidence levels up. Parents were happy with the materials received by their children and the improved socio-emotional aspect of their kids. ABC+ played an important role not only in the lives of the learners but also in the teaching competence of our learning facilitators. The community is very grateful for the program because it played an important role in the total development of the child. They observed their children were focused on their studies and minimized their time on computer games. Indeed ABC+ is God sent to us! It’s a gift from heaven. A big salute to the men and women of ABC+. Our Big Congratulations! L214 My 3Gs with ABC+ I was very fortunate to be part of the advocacy of ABC+ to improve the quality of education of our learners. I would like to capture my experiences in the 3Gs- Growth, Greatness, and Grace. I have witnessed how the lives of our educators and esp. our learners, have improved. There is growth in terms of the level of knowledge and skills among teachers, learners, parents, external stakeholders, and instructional leaders. With this personal growth comes Greatness-greatness in the quality of work we saw produced and the amount of time, energy, and dedication we have invested to facilitate the transfer of learning from ABC+ activities to its rightful beneficiaries. Lastly, I also see my work in ABC+ as Grace. It is with the Lord's grace that we have moved despite the insurmountable obstacle brought about by the pandemic. There were so many personal and professional struggles I have undergone. But I believe it is also grace that will sustain me as I continue my mission of being one of the advocates of learning and empowerment for our struggling learners and all people in our educational system helping each child, one pupil at a time. I want to echo the works of Chito Salavar. "How we do God's work is where our talents can best be served" So all the insights I gained from my various ABC+ experiences became part of my journey for growth, greatness, and grace. I believe that I was led to this beautiful world of ABC+ because that is where my talents as a supervisor can best be served, doing God's work. L215 Masarap na Kalbaryo sa Puso ng Ex Ko "Hay naku! Diyos ko Lord, bakit ba sa akin pa ibinigay itong coordinatorship ng Mother tongue Based Multilingual Education," reklamo ko sa kasama ko sa office na si Sir Donald. "Hoy, di ba nga 3 years kang coordinator ng MTB-MLE doon sa Antique sa pinakamamahal moy Kinaray-a language. Kayang-kaya mo yan, " sagot si Sir Donald. OO, sa Antique pa lang una kong naranasan bilang Education Program Supervisor sa FIlipino and pagiging coordinator ng MTMLE ng Kinaray-a language. Diyos ko po, super hirap at super dami ng travel at trabaho. Oo, tinanggap ko ang trabahong iyon dahil wala namang choice diba? Kahit nakakasama ng sabihing "Para sa Bata Para sa Bayan", yuck ewan ko ba. Aba'y sa totoo lang sukang-suka ako sa linggwahe parang Ex ko pinabayaan ko. "Hoy, wala ka na naman sa sarili mo. Anong iniisip mo? Si MTB-MLE o si Ex mo? tanong ni Sir Donald habang tinatapik ang balikat ko. " Weeh... Si MTB. Alam kong malaking pasanin ito. Hindi ito Ex ko na pwedeng balik-balikan." Isang araw noon...."Sir TIno, memo oh, punta ka raw ng Manila dadalo sa Launching ng ABC+," sabi ni RF ang Secretary sa office. "Salamat Sir RF," sabi ko habang nakamot noo na tinatanggap ang memo. Binasa ko ang memo at walang magawa , nangutang kay Maam Nona at lumipad ng Manila. Simula nong araw na iyon, nagsimula na ang kalbaryo ko sa MTB-MLE. Diyos ko dinagdagan pa ng ABC+ na ito. May panahon pa kaya ako sa lovelife ko or even sa Ex ko? Training dito, training doon, CRLA here and there, USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 130 STORY CODE TITLE STORY REASON speaker dito, speaker doon. Ok lang din sana kasi libre, pinapakain at pinapatulog ka sa magagarang hotel. Diba class? Heto pa ha, pag Quality assure ng mga Early-grade reading Materials sa Kinaray-a, Akeanon, Hiligaynon at iba pa na lengguahe ng rehiyon. Aba'y 483 titles din iyon. Pagsasapuna ng mga Editing Guide etc. etc. My God, mary and Joseph, hindi na ako makahinga. Talagang parang tuluyan nang magkakalayo kami ng Ex ko nito. Parang hindi na bolera sa akin ang "Para sa Bata, Para sa bayan!" "Hello, Sir Loloy...ah ok po...a sige po prepare lang po letter kay RD para malaman nila ni Maam Ellen.. TY po," tinawagan ako ni Sir Loloy and head ng ABC+ sa rehiyon. Punta daw sa Makati para sa launching ng pinaghirapan naming dalawang mudules na gagamitin ng mga kumukuha ng edukasyon sa mga Teacher Education Institution. Lipad na naman, 10% na lang ang tsansa ng balikan namin si Ex walang time na sa kanya. Dumating ako ng Makati, sa sobrang pagod ay napaidlip ako. "Puntahan ko kaya si Ex, may time pa ako bago bumalik ng Iloilo,"sabi ko sa sarili. Puntahan ko si Ex, miss na miss ko na sya. Pagbabakasakali na makapagbalikan kami.Sa pintuan pa lang ng bahay nila tanaw ko na siya, naghihintay sa akin. Nakadopa ang dalawang kamay. Ibig sabihin buong puso niya akong tatanggapin, lumuhod ako sa harao niya, at humingi ng tawad sa lahat ng oras, pinabayaan at hindi siya napuntahan para man lang mabisita sya. Tinulungan ko siyang mabuti, naramdaman ko ang sakripisyo nita sa mga sagot dulot ng korona sa kanyang ulo. Sa mga dugo na dumaloy sa dalawa niyang mga paa at palad, nakita ko ang pagdaloy ng dugo na huhugas ng aking sakripisyo sa MTB=MLE. Napagtanto ko nang mga oras na yon na tama siya kailangang magsakripisyo para sa kapakanan ng lahat. Ring....Ring... "Shit, hala ka! nagsisimula na ang opening program, panaginip lang pala," bulong ko sa sarili.. Kinuha ko ang CP ko. "yes Sir Loloy, coming, bababa na po," sagot ko. Ex, salamat ngayon yayakapin ko na ang MTB-MLE, ABC+ at syempre ikaw; Isasapuso ko na rin and "Para sa Bata Para sa Bayan" dahil heto ang mandato ko. L216 Inspiration in a Change I am [NAME] School Head of Barbaza Central School, District of Barbaza, Division of Antique. I am 60 years old and 28 years in the service. ABC+ was introduced to us a couple of years ago and we were the venue in the training of ABC+ for the teachers in kindergarten to grade 3. I was not a member of the training but I stayed in the venue for the whole duration of the seminar as a spectator/observer. There’s a saying that nothing is permanent in this world except change, but I would like to say that change in ABC+ is spectacular. I can see it from the daily activities and reactions of the teachers. It was indeed a significant change especially on the part of the teachers for they are all motivated, excited, and passionate enough to make a difference in the lives of the learners where the foundation of learnings in the grade levels they are holding are very important and fundamental. 131 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 10. LEVEL 3 MOST SIGNIFICANT STORY21 CODE ORIGINAL TITLE /ENGLISH TRANSLATION STORY ORIGINAL VERSION ENGLISH TRANSLATION REASON FOR SELECTION L301 Walang Imposible sa Pagkatuto ng mga Bata/ “Nothing is impossible for the learning of children” At the time of the pandemic when all schools had closed, I was a teacher who found ways to continue the learning of children in the first grade. I am Teacher Lita, currently teaching at San Jose Elementary School, in the town of Malilipot, Albay Division. A teacher ready to serve the children. I became a trainer of ABC+ TT1 and TT2. Before the pandemic, I just did my responsibility as a teacher for the learning of the children in the classroom. However, I was greatly inspired by the various ABC+ training that I have attended. I launched an initiative to help encourage children to continue to learn, read, and study within a short period despite the closure of schools for two years. I asked for help from the community, especially from those who have teaching knowledge. We worked together to build the Learning Home-Libraries, a place that can be considered a school with books that can be read by the children. It is well-structured just like a real classroom with teachers, the volunteer tutors, who teach and guide the children. At first, it was difficult to put up and develop the project because others had negative views about it. However, more people were ready and generous in helping in terms of providing service and materials. As our school resumes full face-to-face classes, the people supervising the home libraries became our Learning Support Aides in school, while we temporarily closed the home libraries for the children. There will be new preparations and schedules for children who will be part of this project. It is undeniable that the community plays an important role in At the time of the pandemic when all schools had closed, I was a teacher who found ways to continue the learning of children in the first grade. I am Teacher Lita, currently teaching at San Jose Elementary School, in the town of Malilipot, Albay Division. A teacher ready to serve the children. I became a trainer of ABC+ TT1 and TT2. Before the pandemic, I just did my responsibility as a teacher for the learning of the children in the classroom. However, I was greatly inspired by the various ABC+ training that I have attended. I launched an initiative to help encourage children to continue to learn, read, and study within a short period despite the closure of schools for two years. I asked for help from the community, especially from those who have teaching knowledge. We worked together to build the Learning Home￾Libraries, a place that can be considered a school with books that can be read by the children. It is well-structured just like a real classroom with teachers, the volunteer tutors, who teach and guide the children. At first, it was difficult to put up and develop the project because others had negative views about it. However, more people were ready and generous in helping in terms of providing service and materials. As our school resumes full face-to-face classes, the -Community partnership comes in -Ability of the teacher to mobilize the community, - Sustainability through the community, going beyond the comfort zone -Convinced learners to learn using ABC+ materials -Created and initiated programs in the community to help the children learn how to read; initiated a project for children to learn to read. -ABC+ serves as inspiration and it radiates impact to the community 21 Removed actual names for privacy and confidentiality. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 132 CODE ORIGINAL TITLE /ENGLISH TRANSLATION STORY ORIGINAL VERSION ENGLISH TRANSLATION REASON FOR SELECTION the learning of children. Thus, whatever challenges in the continuation of learning will be overcome if everyone will help. people supervising the home libraries became our Learning Support Aides in school, while we temporarily closed the home libraries for the children. There will be new preparations and schedules for children who will be part of this project. It is undeniable that the community plays an important role in the learning of children. Thus, whatever challenges in the continuation of learning will be overcome if everyone will help. L302 A Love for a Children I am [NAME], a teacher 1 at Pagalamatan Elementary School. I learned about this ABC+ Program/Activity during the conduct of Training for Trainers on Literacy Strategies and Use of SRMs: Integrating SEL and GESI in Learning Language and Literacy in School and at home last February 28 to March 4, 2022. Before ABC+, I can say that my knowledge about some strategies that I learned from my previous seminar is not vividly clear to me, because every time I have seminars I am comparing what I have learned from it, and every time I am comparing it, there are some points that they contradict with each other. After I have learned the ABC+, all strategies, techniques, and some points that may help the teachers to apply it in the classroom were strengthened. It helps me as an educator to have a clear knowledge about some strategies that can be used by us in teaching our learners, especially in Literacy and Numeracy integrating the SEL and GESI. There are some changes in terms of teaching my pupils, and these changes help me to promote equal treatment to my learners, especially now that I have learned the SEL and GESI. This change is very important because it positively affects the learners. I can say that my learners develop a lot, it may not be perfect but gives a positive impact on the young Bangsamoro learners in our school and this BARMM region. Our goal is to promote quality and competitive Bangsamoro Learners in this region. (1) The story is clear and it talks about children and (2) The story shows love for the children. -The teacher portrays the authentic experience of being a teacher; applying all the strategies and techniques learned from ABC+ Teacher who has a passion to improve herself for the learners -GESI and SEL -Love for children and direct application; promote equal treatment for my students; realization with direct application - Knowledge of GESI; shows the impact of GESI on children 133 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV CODE ORIGINAL TITLE /ENGLISH TRANSLATION STORY ORIGINAL VERSION ENGLISH TRANSLATION REASON FOR SELECTION L303 ABC+ and Me It was during the early period of the pandemic when I was called to attend the very first ritual training that I have experienced in my whole life, the capacity building for Learning Resource Evaluators sponsored by ABC+. Even if it was my first time, I never hesitated to attend because I’m a person who is a risk￾taker by nature and very willing to learn things that would enhance my knowledge, skills, and attitudes – the 3 main areas of focus that I would like to improve as an individual – because I am dealing with different kinds of people. At this point, I am thankful to our LRMS Supervisor, Sir [NAME] for believing in my capacity. Had it not been for Sir Juancho, I would not be able to discover my potential as a learning resource evaluator and quality assurer of Minasbate Learning Resource Materials. The story I have written was significant to me because the experience made a meaningful change in my life as a person and as a learning resource evaluator. The real-life experience I had with the activities that ABC+ exposed me to was a real behavior-changing experience. I never realized that I could be equipped with enough quality knowledge to assure learning resource materials in Minasbate. But with the support of the ABC+, the Minasbate Editing Guide, the Load Allowance, and the very accommodating facilitators, I was able to help develop quality learning resource materials that Grade 1-3 learners of Minasbate can use. Without the innovation from ABC+, my story would have been insignificant. Imagine the big number of Grades 1- 3 learners in the province of Masbate who benefit from the locally developed learning resource materials. Indeed, ABC+ was able to make an educational breakthrough that will remain forever igniting in the hearts of the teachers, learners, and parents of Calasuche Elementary School and most schools of the Masbate province. Hat’s off to ABC+ Team. The story of how ABC+ helps the Minasbate community Developed teacher’s KSA and considered ABC+ as an educational breakthrough Made me feel like a part of the teacher's journey of the teachers; based on this, they discovered that they have the potential to learn the story. The writer’s experience was extended and was inspired by reading materials from ABC+ L304 ABC+, Salamat sa Pagkatuto/ ABC+, Thank you for the Learning Ako ay isang magulang mula sa Tiwi Central School na buo ang suporta at pagpapahalaga sa pag-aaral ng aking mga anak. Wala naman akong gaanong naging problema sa aking mga anak dahil sinanay ko sila sa pagbabasa mula sa kanilang murang edad at ito ay kanilang nakahiligan. I am a parent from Tiwi Central School who fully supports and values my children's education. I didn't have much problem with my children because I trained them to read from a young age and they enjoy reading. I often go to school because I am a The story of a parent is so legit, explaining the LMs, easy to understand, colorful, well-written Perspective of parents and USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 134 CODE ORIGINAL TITLE /ENGLISH TRANSLATION STORY ORIGINAL VERSION ENGLISH TRANSLATION REASON FOR SELECTION Madalas akong sa eskwelahan dahil isa akong GPTA (General Parent￾Teachers Association) official at teacher. Kung minsan nasaksihan ko na maraming mga bata ang talagang hirap bumasa at walang hilig sa pagbabasa. Isa sa dahilan nila ang kakulangan ng mga babasahin kaya’y hindi atraktibo sa kanila ang mga aklat sa silid-aralan. Nasaksihan ko ang araw na namigay ng mga babasahin ang ABC+ at ako’y nakibasa rin. Sobra akong humanga at bumilib dahil madaling unawain at may makulay na larawan na nakakatawag pansin sa mga bata. Yung tipong kapag pinanood mo ang larawan ay ma-enganyo ka ring basahin ang kwento. Simula noon ay marami na akong nakitang mga bata na nagbabasa nito sa ilalim ng puno at sa covered court at nagkukwentuhan sila tungkol dito. Tunay na napakaganda naidulot nito sa mga bata. Nadevelop ang love for reading nila. Kasi tunay namang napaka- ayos ng pagkakasulat maging ang ilustrasyon o larawan. Madaling intindihin at malinis ang pagkakagawa. Bilang isang magulang, ako’y umaasa na sana’y mapagpatuloy ang mga ganitong proyekto bilang suplemento sa mga batang nahihirapan magbasa at magkaroon ng hilig dito. Nawa’y lalo pang pagpalain ng maykapal ang mga taong nasa likod ng proyektong ito. GPTA (General Parent-Teachers Association) official and teacher. Sometimes, I witness that many children struggle to read and have no interest in reading. One of their reasons is the lack of reading materials or that classroom books are unappealing to them. I was present on the day when ABC+ distributed reading materials, and I read them, too. I was very impressed because the books were easy to understand and had colorful pictures that attracted children's attention -- they are the kind with pictures that will entice you to read the stories, too. Since then, I have seen many children reading these books under the tree and at the covered court, and they talk about the stories. It, indeed, did an outstanding job with the children. Their love for reading was developed because the writing and even the illustration or picture are very good. They are easy to understand and were made very well. As a parent, I hope that projects like these can be continued as a supplement for children who struggle to read so that they will develop an interest in books. May God continue to bless the people behind this project. descriptions of the features of books... Many children learned how to read when ABC+ books arrived; learners were encouraged to read, addressed the literacy problems An inspirational story that even parents are “nakibasa” (also read) and learn. Several children read the books. L305 ABC+, A Bridge to One's Childhood Dream I learned about ABC+activities from our Education Program Supervisor in LRMS, [NAME]. A year ago, he invited me to join and participate in the teachers' training for teachers in the development of Minasbate Story Books in Early-grade. It was indeed magical and a roller coaster of an event for me because ever since then, I have been an active and privileged participant in ABC+ training and an active user of their materials. Indeed, my life has changed because of this partnership. Many changes have happened in the past year of implementing ABC+ activities. One of the things that I have witnessed is that children are showing signs of change in behavior in terms of their fondness and love for reading and appreciating books. Teachers were provided with great opportunities because they were able to make Minasbate stories for their students, which helped the kids and helped them grow professionally. The training provided That someone achieved her dream through ABC+ is a pleasant thought Dreams do come true to be a writer. No great happiness when your dream comes true This story impacted the learner and teachers and most is the writer through ABC+ 135 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV CODE ORIGINAL TITLE /ENGLISH TRANSLATION STORY ORIGINAL VERSION ENGLISH TRANSLATION REASON FOR SELECTION by ABC+ molded and honed the skills of the teachers on how they could teach their learners scientifically, which is beneficial in the field. Among these changes, the most significant I encountered and experienced as a result of ABC+ activities was when I was allowed to finally fulfill my long-time dream – my childhood dream of becoming a writer of children’s books. Ever since I was a kid, I have wanted to write and publish stories that my readers would enjoy and relate to. My dream has finally turned into reality. ABC+ became the bridge to my childhood dream. I never, in my wildest dream, thought that I could become a quality assurance of the books that we developed during those months of training. I feel so privileged and honored because of these opportunities. This is why I am forever grateful and in debt to Sir Juancho Azares and ABC+. It was truly significant for me because not only were my dreams fulfilled, but it also changed the lives and views of the learners when it comes to reading and appreciating books. L306 My ABC+ Reading Journey The reading crisis during the pandemic time was a tough challenge for me as a school leader. It was so timely that ABC+ had partnered with DepEd where it became a part of my ABC+ journey. With all the training that I attended as a regional trainer and even facilitated with the teachers and school heads in the division, somehow and somewhat the learnings on reading strategies and creating a positive climate in the school and at home helped the learners improve their literacy domains. Through my project initiative as a final output which was dubbed Project SMK (Syllable Mastery in Katon way), the number of frustrated readers was lessened. The said project which I implemented in the school reaped a National Award as one of the Noteworthy Literacy Practices and school supplies worth of 10thousand pesos conducted and sponsored by ABC+ during the Learning Conference via Zoom. Indeed, it was realized with flying colors because of the distributed leadership I employed with the external and internal stakeholders in the school. The love for reading among the pupils, teachers, parents, and the community was intensified because of the training conducted by ABC+ and more so the collaborative effort by various stakeholders in combating struggling readers in the school. The story of how ABC+ turns a successful story that reduced the frustration of readers What ABC+ did was good, for example, the Grade 1 learner then is a Grade 3 learner now, and ABC+ helped with learning recovery USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 136 ANNEX 11. LOCALIZED SUSPENSION OF OBSERVATION IN REGION V Source: Department of Education Region V. (2022). Temporary suspension of classroom observations at all levels. 137 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 12. LIST OF KII AND FGD PARTICIPANTS22 NAME CODES REFERENCES MODIFIED ON MODIFIED BY CLASSIFICATION FGD_Teachers_Iriga 1 1 26/10/2022 3:RSS FGD_Parents – Albay 1 1 26/10/2022 3:RSS FGD_Cohort 3 – Virac 1 1 26/10/2022 3:RSS KII_Cohort 3 – Catanduanes 1 1 26/10/2022 3:RSS FGD_Teachers - Naga 1 1 26/10/2022 3:RSS FGD_Teachers - Legazpi 1 1 26/10/2022 3:RSS FGD_Teachers – Roxas 1 1 26/10/2022 3:RSS FGD_Teachers – Legazpi – Sorsogon 1 1 26/10/2022 3:RSS FGD_Teachers – Iriga – Camarines Sur 1 1 26/10/2022 3:RSS FGD_Teachers – Cotabato 1 1 26/10/2022 3:RSS FGD_School Heads – SGA 1 1 26/10/2022 3:RSS FGD_Teachers – SGA 1 1 26/10/2022 3:RSS KII_Parents 1 1 26/10/2022 3:RSS FGD_TEIs – Bicol University – West Visayas University 1 1 26/10/2022 3:RSS FGI 1 1 26/10/2022 3:RSS KII_Focal Person – Escalante 1 1 26/10/2022 3:RSS KII_Focal Person – Negros Occidental 1 1 26/10/2022 3:RSS KII_ Focal Person – Roxas 1 1 26/10/2022 3:RSS KII_Focal Person – Lacuesta 1 1 26/10/2022 3:RSS KII_School Head - Iloilo 1 1 26/10/2022 3:RSS KII_School Head - Cotabato 1 1 26/10/2022 3:RSS KII_School Head - Maguindanao 1 1 26/10/2022 3:RSS KII_Focal Person – SGA 1 1 26/10/2022 3:RSS KII_ Focal Person - Bansil 1 1 26/10/2022 3:RSS KII_DepEd CO - 1 1 1 26/10/2022 3:RSS KII_NGO Rotary- Legazpi 1 1 26/10/2022 3:RSS KII_DepEd CO – 2 1 1 26/10/2022 3:RSS KII_DpeEd CO – 3 1 1 26/10/2022 3:RSS KII_DepEd CO and RO - 1 1 1 26/10/2022 3:RSS KII_ DepEd CO – 4 1 1 26/10/2022 3:RSS KII_ DepEd CO – 5 1 1 26/10/2022 3:RSS KII_ DepEd CO – 6 1 1 26/10/2022 3:RSS 22 Actual file names were renamed for privacy and confidentiality. Renamed with generic modifier. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 138 NAME CODES REFERENCES MODIFIED ON MODIFIED BY CLASSIFICATION KII_DepEd CO and RO - 2 1 1 26/10/2022 3:RSS KII_DepEd CO and RO - 3 1 1 26/10/2022 3:RSS KII_DepEd - Albay 1 1 26/10/2022 3:RSS KII_DepEd – Camarines Norte 1 1 26/10/2022 3:RSS KII_DepEd – Camarines Sur 1 1 26/10/2022 3:RSS KII_DepEd – Iriga 1 1 26/10/2022 3:RSS KII_DepEd – Legazpi 1 1 26/10/2022 3:RSS KII_DepEd – Masbate 1 1 26/10/2022 3:RSS KII_DepEd – Naga 1 1 26/10/2022 3:RSS KII_DepEd – Sorsogon 1 1 26/10/2022 3:RSS KII_DepEd – Tabaco 1 1 26/10/2022 3:RSS KII_DepEd CO - 7 1 1 26/10/2022 3:RSS KII_DepEd CO - 8 1 1 26/10/2022 3:RSS KII_DepEd CO - 9 1 1 26/10/2022 3:RSS KII_DepEd CO and RO - 4 1 1 26/10/2022 3:RSS KII Focal Person - Capiz 1 1 26/10/2022 3:RSS KII Focal Person - Antique 1 1 26/10/2022 3:RSS KII TEI – Bicol University 1 1 26/10/2022 3:RSS KII TEI – West Visayas University 1 1 26/10/2022 3:RSS KII_Focal Person - Escalante 1 1 26/10/2022 3:RSS KII_Focal Person - Iloilo 1 1 26/10/2022 3:RSS KII_Focal Person - Roxas 1 1 26/10/2022 3:RSS KII_Focal Person - Capiz 1 1 26/10/2022 3:RSS KII_Focal Person - Antique 1 1 26/10/2022 3:RSS KII_Focal Person -Negros Occidental 1 1 26/10/2022 3:RSS KII_School Head_ Iloilo 1 1 26/10/2022 3:RSS FGD_Teachers - Iloilo 1 1 26/10/2022 3:RSS KII_HLP- Iloilo 1 1 26/10/2022 3:RSS FGD_Teachers - Escalante 1 1 26/10/2022 3:RSS KII_Cohort 3 - Roxas 1 1 26/10/2022 3:RSS KII Cohort 3 - Bacolod 1 1 26/10/2022 3:RSS FGD_HLP - Iloilo 1 1 26/10/2022 3:RSS FGD_Teachers - Iloilo 1 1 26/10/2022 3:RSS 139 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 13. SURVEY RESULTS ANNEX 13.1 SURVEY RESULTS (SCHOOL HEADS) RESPONSES REGION BARMM (n=16) V (n=23) VI (n=187) TOTAL (n=226) SEX Female 81.3 60.9 79.7 77.9 Male 18.8 39.1 20.3 22.1 TOTAL 100.0 100.0 100.0 100.0 HIGHEST EDUCATIONAL ATTAINMENT College graduate 18.8 8.7 20.3 19 Doctorate degree holder 18.8 21.7 7.5 9.7 Doctorate degree level with units earned 12.5 12.0 9.6 10.2 Master’s degree holder 31.3 12.0 18.7 19.0 Master’s degree level with units earned 18.8 43.5 43.9 43.0 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q11 (Does your school offer multi-grade classes?) No 75 100 84.5 85.4 Yes 25 0 15.5 14.6 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q12 (Does your school have SPED or children with special needs classes?) No 75 87 86.1 85.4 Yes 25 13 13.9 14.6 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q13 (Does your school have learners from IP community?) No 68.8 82.6 79.7 79.2 Yes 31.3 17.4 20.3 20.8 TOTAL 100.0 100.0 100.0 100.0 PARTICIPATION IN ABC+ ACTIVITIES SINCE 2019 No 0 0 0 0 Yes 100 100 100 100 TOTAL 100.0 100.0 100.0 100.0 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 140 RESPONSES REGION BARMM (n=16) V (n=23) VI (n=187) TOTAL (n=226) YEAR OF PARTICIPATION IN ABC+ ACTIVITIES23 2019 0 10 13.8 12.8 2020 0 26.7 20.7 20.2 2021 26.7 36.7 41.7 40.5 2022 73.3 26.7 23.9 26.5 FORM OF TRAINING Combination of Virtual and Face-To-Face 12.5 4.3 20.3 18.1 Face-To-Face 75 8.7 15 18.6 Virtual 12.5 87 64.7 63.3 TOTAL 100.0 100.0 100.0 100.0 ROLE IN THE TRAINING Trainee 87.5 87 85 85.4 Trainer 12.5 13 15 14.6 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q18 (Which of the following statements best describes how ABC+ compares with other professional development in which you have participated during the past six months?) ABC+ was MORE USEFUL than other professional development on MTB-MLE that I have participated in. 68.8 69.6 77.7 75.7 ABC+ was ABOUT THE SAME as other professional development that I have participated in 0 0 0 0 I don’t have an opinion 0 0 1 0.9 I don’t have an opinion because I have not yet participated in any other professional development in the last six months 6.3 17.4 2.1 4.0 Others 0 0 1.6 1.3 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q19.1 (To what extent do ABC+ activities align with existing DepEd policies and/or programs on Early-grade instruction?) Much aligned 81.3 82.6 82.4 82.3 Somewhat aligned 18.8 13 13.4 13.7 Fairly aligned 0 4.3 4.3 4 Not at all aligned 0 0 0 0 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q19.2 (To what extent do ABC+ activities align with existing DepEd policies and/or programs on Access to quality material?) Much aligned 75 82.6 77 77.4 Somewhat aligned 25 13 4.8 4.4 23 Multiple responses 141 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONSES REGION BARMM (n=16) V (n=23) VI (n=187) TOTAL (n=226) Fairly aligned 0 4.3 4.8 4.4 Not at all aligned 0 0 0 0 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q19.2 (To what extent do ABC+ activities align with existing DepEd policies and/or programs on Improved education quality?) Much aligned 75 78.3 80.7 80.1 Somewhat aligned 25 17.4 14.4 15.5 Fairly aligned 0 4.3 4.8 4.4 Not at all aligned 0 0 0 0 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q20 (As you observed teachers at the K-3 levels, are there any activities especially intended for boys?) No 81.3 87 78.6 79.6 Yes 18.8 13 21.4 20.4 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q22 (As you observed teachers at the K-3 levels, are there any activities especially intended for girls?) No 75 73.9 73.8 73.9 Yes 25 26.1 26.2 26.1 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q26 (Do you know of any DepEd policy (local, regional, or national) on early-grade instruction that has been modified or enhanced, or enhanced because of ABC+ activities/programs?) No 43.8 26.1 42.8 41.2 Yes 56.3 73.9 57.2 58.8 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q28 (Recall the time before participating in the ABC+ program and compare it now in terms of your early￾grade instruction and delivery. To what extent has the program been useful in improving early-grade instructions and delivery since its implementation?) ABC+ program has been VERY USEFUL in improving early-grade instructions and delivery 68.8 82.6 79.1 78.8 ABC+ program has been SOMEWHAT USEFUL in improving early-grade instructions and delivery 31.3 8.7 17.1 17.3 ABC+ program has been A BIT USEFUL in improving early-grade instructions and delivery 0 8.7 2.1 2.7 ABC+ program has NOT BEEN USEFUL at all in improving early-grade instructions and delivery 0 0 1.6 1.3 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q29 (Indicate the extent to which the ABC+ met your professional learning and teaching needs in early￾grade instructions.) It addressed my professional learning and teaching needs COMPLETELY 43.8 60.9 61.0 59.7 It addressed SOME of my professional learning and teaching needs 56.3 39.1 38.0 39.4 It did NOT address my professional learning and teaching needs 0 0 0.5 0.4 This professional development DID NOT HELP MUCH because I was already familiar with this topic 0 0 0.5 0.4 TOTAL 100.0 100.0 100.0 100.0 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 142 RESPONSES REGION BARMM (n=16) V (n=23) VI (n=187) TOTAL (n=226) RESPONSES TO Q26 (To what extent has ABC+ training helped you in mentoring K-3 teachers to improve their literacy instruction?) To great extent 75 82.6 81.3 81 To a moderate extent 25 17.4 16.6 17.3 To a limited extent 0 0 2.1 1.8 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q26 (To what extent has ABC+ training helped you in mentoring K-3 teachers to improve their mathematics instruction?) To great extent 50 65.2 64.2 63.3 To a moderate extent 43.8 30.4 30.5 31.4 To a limited extent 6.3 4.3 5.3 5.3 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q26 (To what extent has ABC+ training helped you in mentoring K-3 teachers to improve their instruction in the integration of socio-emotional learning (SEL)?) To great extent 68.8 82.6 73.8 74.3 To a moderate extent 31.3 17.4 23.5 23.5 To a limited extent 0 0 2.7 2.2 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q26 (To what extent has ABC+ training helped you in mentoring K-3 teachers to improve their instruction in integration of gender equality and social inclusion (GESI)?) To great extent 75 82.6 77 77.4 To a moderate extent 25 17.4 21.4 21.2 To a limited extent 0 0 1.6 1.3 TOTAL 100.0 100.0 100.0 100.0 ABC+ ACTIVITIES MOST SUCCESSFUL ASPECT Strengthening capacity development for teachers and school heads 36.7 45.2 40.7 40.9 Developing and providing access to quality reading materials 33.3 33.3 37.7 36.9 Strengthening the system capacity of the Department of Education in your regions/division to sustain efforts of ABC+ 30 21.4 21.5 22.2 TOTAL 100.0 100.0 100.0 100.0 MODES OF IMPLEMENTING INSTRUCTION DURING THE PANDEMIC Use of modules, implemented with the help of home learning partners 60 62.2 50.7 52.3 Online learning using Moodle and conferencing platforms (Zoom, GoogleMeet) 12 13.5 18.2 17.4 House-to-house visits by teachers who teach students face-to-face 4 0 1.7 1.7 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q35 (To what extent were teachers still able to implement skills and knowledge acquired from ABC+ training using the instructional delivery approach used during the pandemic?) To great extent 56.3 69.6 49.2 51.8 To a moderate extent 37.5 17.4 46.0 42.5 143 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONSES REGION BARMM (n=16) V (n=23) VI (n=187) TOTAL (n=226) To a limited extent 6.3 13.0 4.8 5.8 TOTAL 100.0 100.0 100.0 100.0 LIKELIHOOD OF CONTINUOUS APPLICATION OF LEARNING IN ABC+ EVEN AFTER THE PROGRAM IMPLEMENTATION Least unlikely 0 0 0 0 Unlikely 0 0 0.5 0.4 Maybe/Neutral 6.3 4.3 6.4 6.2 Likely 25 26.1 30.5 29.6 Most likely 68.8 69.6 62.6 63.7 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q40 (Which of the following statements best describes the likelihood that you will apply what you learned in ABC+?) I have already practiced it in my school 18.8 21.7 28.3 27 I have already practiced in my school and it seemed to work well 62.5 60.9 58.8 59.3 I have already practiced in my school, but it was not appropriate for our K-3 learners 0 0 1.6 1.3 I look forward to practicing in my school in the next few weeks 0 0 4.8 4 I look forward to practicing in my school later this academic year 6.3 13 2.7 4 I would like to practice, but I don’t have sufficient materials that we need in our school 12.5 4.3 3.7 4.4 I don’t think that these things will work with our K-3 learners 0 0 0 0 TOTAL 100.0 100.0 100.0 100.0 READINESS IN CONTINUING ABC+ ACTIVITIES AFTER THE PROJECT ENDS Strongly ready 25 34.8 23 24.3 Ready 68.8 56.5 72.9 72.6 Not ready 6.3 8.7 2.1 3.1 Strongly not ready 0 0 0 0 Do not know 0 0 0 0 TOTAL 100.0 100.0 100.0 100.0 ABC+ ACTIVITIES’ EXTENT OF INVOLVEMENT OF COMMUNITY STAKEHOLDERS (LGUS, NGOS, ELDERS, ETC.) FOR THE IMPROVEMENT OF EARLY-GRADE INSTRUCTION To great extent 31.3 52.2 40.1 40.7 To a moderate extent 62.5 39.1 51.9 51.3 To a limited extent 6.3 8.7 8 8 TOTAL 100.0 100.0 100.0 100.0 GENERAL ATTITUDE OF PARENTS AND COMMUNITY MEMBERS TOWARDS MOTHER TONGUE-BASED MULTILINGUAL EDUCATION (MTB-MLE) Very positive and highly supportive 50 30.4 28.9 30.5 Moderately positive and supportive 50 56.5 61.5 60.2 Unaware of MTB-MLE and passive 0 4.3 3.7 3.5 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 144 RESPONSES REGION BARMM (n=16) V (n=23) VI (n=187) TOTAL (n=226) Moderately negative and skeptical 0 4.3 3.2 3.1 Very negatively and highly opposed 0 0 2.7 2.2 No response 0 4.3 0 0.4 TOTAL 100.0 100.0 100.0 100.0 OBSERVATIONS ABOUT THE COVERAGE THAT ABC+ ACTIVITIES HAVE RECEIVED IN THE NEWS MEDIA Inspiring 25 47.8 26.2 28.3 Informative 37.5 21.7 48.7 45.1 Encouraging 18.8 13 4.3 6.2 No idea 18.8 13 4.3 6.2 Others 0 4.3 0 0.4 TOTAL 100.0 100.0 100.0 100.0 145 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 13.2 SURVEY RESULTS (TEACHERS) RESPONSES Region BARMM (n=59) V (n=161) VI (n=1372) TOTAL (n=1592) SEX Female 89.8 95 95 94.8 Male 10.2 5 5 5.2 TOTAL 100.0 100.0 100.0 100.0 HIGHEST EDUCATIONAL ATTAINMENT College graduate 27.1 40.4 43.9 42.9 Doctorate degree holder 1.7 1.2 0.2 0.4 Doctorate degree level with units earned 1.7 1.9 1.5 1.6 Master’s degree holder 40.7 12.4 10.1 11.4 Master’s degree level with units earned 28.8 44.1 44.3 43.7 TOTAL 100.0 100.0 100.0 100.0 PARTICIPATION IN ABC+ ACTIVITIES SINCE 2019 No 11.9 5.6 5.5 5.7 Yes 88.1 94.4 94.5 94.3 TOTAL 100.0 100.0 100.0 100.0 YEAR OF PARTICIPATION TO ABC+ ACTIVITIES24 2019 8.6 21.1 21.3 20.8 2020 0 40.4 41.0 39.4 2021 24.1 49.7 55.5 53.7 2022 67.2 32.9 40.2 40.5 FORM OF TRAINING Combination of Virtual and Face-To-Face 3.4 15.5 24.6 22.9 Face-To-Face 94.9 6.8 18.4 20 Virtual 1.7 77.6 57.1 57.1 TOTAL 100.0 100.0 100.0 100.0 ROLE IN THE TRAINING Trainee 79.7 92.5 90.3 90.1 Trainer 20.3 7.5 9.7 9.9 24 Multiple responses USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 146 RESPONSES Region BARMM (n=59) V (n=161) VI (n=1372) TOTAL (n=1592) TOTAL 100.0 100.0 100.0 100.0 STATUS IN TERMS OF IMPLEMENTATION OF BUDGET OF WORK Ahead 3.4 5 1.5 1.9 Delayed 3.4 2.5 3.9 3.7 On-time 93.2 92.5 94.6 94.3 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q17 (Which of the following statements best describes how ABC+ compares with other professional development in which you have participated during the past six months?) ABC+ was MORE USEFUL than other professional development on MTB-MLE that I have participated in. 66.1 76.4 70 70.5 ABC+ was ABOUT THE SAME as other professional development that I have participated in 27.1 18 25.4 24.7 I don’t have an opinion 1.7 2.5 1.9 1.9 I don’t have an opinion because I have not yet participated in any other professional development in the last six months 0 1.9 1.9 1.9 Others 0 0 0 0.9 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q22 (Recall the time before participating in the ABC+ program and compare it now in terms of your early￾grade instruction and delivery. To what extent has the program been useful in improving early-grade instructions and delivery since its implementation?) ABC+ program has been VERY USEFUL in improving early-grade instructions and delivery 83.1 81.4 80.3 80.5 ABC+ program has been SOMEWHAT USEFUL in improving early-grade instructions and delivery 10.2 16.8 17.6 17.3 ABC+ program has been A BIT USEFUL in improving early-grade instructions and delivery 1.7 1.9 1.7 1.7 ABC+ program has NOT BEEN USEFUL at all in improving early-grade instructions and delivery 0 0 0.4 0.3 No Response 5.1 0 0 0.2 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q23 (On a scale of 1 to 10, rate your overall knowledge, attitudes, and practices (KAP) in delivering early￾grade reading (EGR) instruction BEFORE participating in the ABC+ training.) 1 0 0 0.2 0.2 2 0 0 0.3 0.3 3 1.8 1.2 0.9 1.0 4 3.5 2.5 2.7 2.7 5 15.8 11.2 17.5 16.8 6 8.8 11.2 17.5 16.8 7 35.1 21.1 21.1 21.6 8 24.6 36 24.3 25.5 9 1.8 9.9 13.2 12.5 10 8.8 6.8 6.3 6.5 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q24 (On a scale of 1 to 10, rate your overall knowledge, attitudes, and practices (KAP) in delivering early￾grade reading (EGR) instruction AFTER participating in the ABC+ training.) 147 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONSES Region BARMM (n=59) V (n=161) VI (n=1372) TOTAL (n=1592) 3 0 0.6 0.1 0.2 4 0 0 0.1 0.1 5 1.8 0 1.5 1.3 6 0 2.5 2.2 2.1 7 17.5 8.1 7.8 8.2 8 28.1 28.6 33.5 32.8 9 36.8 40.4 38.6 38.7 10 15.8 19.9 16.3 16.7 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q26 (Indicate the extent to which the ABC+ met your professional learning and teaching needs in early￾grade instructions.) It addressed my professional learning and teaching needs COMPLETELY 66.1 64 59.4 60.1 It addressed SOME of my professional learning and teaching needs 25.4 36 40.3 39.3 It did NOT address my professional learning and teaching needs 0 0 0.1 0.1 This professional development DID NOT HELP MUCH because I was already familiar with this topic 0 0 0.1 0.1 No response 8.5 0 0 0.3 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q27.1 (To what extent has ABC+ training helped you in mentoring K-3 teachers to improve their literacy instruction?) To great extent 66.1 65.8 64.1 64.3 To a moderate extent 23.7 31.1 33.6 33 To a limited extent 1.7 3.1 2.3 2.4 No response 8.5 0 0 0.3 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q27.2 (To what extent has ABC+ training helped you in mentoring K-3 teachers to improve their mathematics instruction?) To great extent 40.7 47.2 44 44.2 To a moderate extent 47.5 50.3 52.6 52.1 To a limited extent 1.7 2.5 3.5 3.3 No response 10.2 0 0 0.4 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q27.3 (To what extent has ABC+ training helped you in mentoring K-3 teachers to improve their instruction in the integration of socio-emotional learning (SEL)?) To great extent 72.9 57.8 59.5 59.9 To a moderate extent 16.9 37.3 38 37.2 To a limited extent 3.4 5 2.4 2.7 No response 6.8 0 0 0.3 TOTAL 100.0 100.0 100.0 100.0 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 148 RESPONSES Region BARMM (n=59) V (n=161) VI (n=1372) TOTAL (n=1592) RESPONSES TO Q27.3 (To what extent has ABC+ training helped you in mentoring K-3 teachers to improve their instruction in the integration of gender equality and social inclusion (GESI)?) To great extent 72.9 59 58.9 59.4 To a moderate extent 16.9 38.5 38.8 38 To a limited extent 3.4 2.5 2.3 2.3 No response 6/8 0 0 0.3 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q20 (As you observed teachers at the K-3 levels, are there any activities especially intended for boys?) No 81.4 67.7 64.3 65.3 Yes 10.2 32.2 35.7 34.3 No response 8.5 0 0 0.3 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q30 (As you plan your early-grade instruction, are there any activities especially intended for girls?) No 76.3 95.8 61.2 62.2 Yes 15.3 34.2 38.8 37.4 No response 8.5 0 0 0.3 TOTAL 100.0 100.0 100.0 100.0 MODES OF IMPLEMENTING INSTRUCTION DURING THE PANDEMIC Use of modules, implemented with the help of home learning partners 98.2 95 97.2 97 Online learning using Moodle and conferencing platforms (Zoom, GoogleMeet) 10.9 24.2 20.6 20.9 House-to-house visits by teachers who teach students face-to-face 25.5 42.2 42.8 42.2 TOTAL 100.0 100.0 100.0 100.0 COMPONENTS OF ABC+ PROGRAMS THAT NEED POLICY RECOMMENDATIONS TO ASCERTAIN SUSTAINABILITY Collaboration with stakeholders 20.3 19.3 17.9 18.2 Budget allocation 8.5 21.7 13.6 14.3 Time allocation for the training 6.8 11.2 9.7 9.7 The ratio of teaching and learning materials 16.9 8.7 32.4 30.7 Quality of teaching and learning materials 20.3 19.9 32.4 30.7 Integrating socio-emotional learning skills 1.7 1.9 1.5 1.5 Mainstreaming or highlighting gender sensitivity and girl’s empowerment 1.7 0 0.8 0.8 Promoting social inclusion when it comes to religious belief and indigenous people’s practices 5.1 2.5 0.9 1.2 Providing access to the internet through wireless and wired means and other options 6.8 14.9 14 13.8 No Response 11.9 0 0 0.4 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q37 (On a scale of 1-3 where 1 is the highest and 3 is the lowest, rank strengthening capacity development for teachers and school heads based on the extent of their contribution to improving your instruction for learners’ basic skills in the early grades, especially in reading in the MT and transition to Filipino and English.) 1 58.2 57.1 48.9 50.1 149 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONSES Region BARMM (n=59) V (n=161) VI (n=1372) TOTAL (n=1592) 2 40 34.8 43 42.1 3 1.8 8.1 8 7.8 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q37 (On a scale of 1-3 where 1 is the highest and 3 is the lowest, rank developing and providing access to quality reading materials based on the extent of their contribution to improving your instruction for learners’ basic skills in the early grades, especially in reading in the MT and transition to Filipino and English.) 1 60 52.8 47.7 48.6 2 36.4 36 41.7 41 3 3.6 11.2 10.6 10.4 TOTAL 100.0 100.0 100.0 100.0 RESPONSES TO Q37 (On a scale of 1-3 where 1 is the highest and 3 is the lowest, rank strengthening system capacity of the Department of Education in your regions/divisions to sustain efforts of ABC+ (i.e., collaboration with NGOs and LGU to improve the system and policies) based on the extent of their contribution to improving your instruction for learners’ basic skills in the early grades, especially in reading in the MT and transition to Filipino and English.) 1 43.6 50.3 43.2 43.9 2 54.5 38.5 45.4 45 3 1.8 11.2 11.5 11.1 TOTAL 100.0 100.0 100.0 100.0 LIKELIHOOD OF CONTINUOUS APPLICATION OF LEARNINGS IN ABC+ EVEN AFTER THE PROGRAM IMPLEMENTATION Least likely 0 0 0.8 0.7 Unlikely 0 0 0.4 0.4 Neutral 10.2 11.2 10.3 10.4 Likely 30.5 37.9 47.4 45.8 Most likely 54.2 50.9 41.1 42.6 No response 5.1 0 0 0.2 TOTAL 100.0 100.0 100.0 100.0 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 150 ANNEX 14. LIST OF CODES FOR THE THREE EVALUATION PARAMETERS AND RELIABILITY NAME DESCRIPTION FILES REFERENCES Contribution to children’s literacy Any effort (training or materials) provided by ABC+ to improve learners’ literacy Examples: Mention of Teacher training on literacy instruction; Mention of SRMs 80 732 Contribution to children’s numeracy Any effort (training or materials) provided by ABC+ to improve learners’ numeracy Examples: Mention of Teacher training on numeracy instruction; Mention of SRMs related to numeracy 22 73 Contribution to socio￾emotional learning Any effort of ABC+ to promote socio-emotional learning; could possibly overlap with some GESI efforts; keywords could be values, “moral lesson,” etc. Examples: Could be ABC+ training that made teachers aware of socio-emotional learning, and values-oriented teaching; Could be ABC+ materials that reflect values 23 44 Enhanced pedagogical content knowledge (PCK) Any activities referring to improved instructions and strategy in early-grade GIDCA and Last Mile schools 72 328 GESI (Gender equality and social inclusion) promotion Any effort of ABC+ (training or materials) to promote GESI Examples: Could be ABC+ training that made teachers aware of gender differences of children and how it could affect teaching and learning; Could be mention of ABC+ materials development process and how it made sure that GESI was upheld; Could be ABC+ materials that reflect GESI, that is, manifesting indicators like the equal number of male and female characters, featuring a differently-abled character, etc. 20 151 Goal Alignment Correspondence of ABC+ with the targets or objectives of the school, division, or region in developing literacy, numeracy, GESI, etc. This may include ABC+ training and materials that have the same goals as the school, division, or region. Examples: Mention of activities (like teacher training) or outputs (like SRMs) that are described to have the same goals as the school 58 180 Notable Remarks Any quotable lines (English, Filipino, or in the local language) about the codes for Relevance 49 124 Pandemic pivots Any adjustment made by ABC+ in teacher training or materials development to accommodate the changes brought about by the pandemic. Any description of the nature of ABC+ training and/or process of developing and distributing materials. Opinion of stakeholders on the pivots made by ABC+ Examples: Description of shifting from face-to-face 42 145 151 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV NAME DESCRIPTION FILES REFERENCES to online activities Mention of the use of online tools, giving of internet load, implementation of hybrid modes of training, follow-ups done, following health pr Parental and community engagement and or involvement 27 65 Policy Alignment Correspondence of ABC+ with DepEd policies and priorities which are formalized through official documents (e.g., orders or memos in school, division, region, or national level) Examples: Mention of memos or orders that align with ABC+ activities 76 375 Reaching out to marginalized groups Any effort of ABC+ to include any marginalized group in policy and interventions. Marginalized groups could be: those who live in GIDCA areas or those enrolled in last mile schools, special (SPED) pupils, learners from low-income families, etc.) Examples: Mention of ABC+ reaching out to marginalized groups (making sure that SRMs are delivered to them, or that their teachers are included in training) 12 14 Support to GIDCA and Last Mile Schools Contextualization of training and activities in GIDCA and Last Mile 4 5 Support to GIDCA and Last Miles Schools 2 2 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 152 NAME DESCRIPTION FILES REFERENCES 4.1. (TEIs) How would DepEd sustain /replicate/continue the ABC+ programs? What are your strategic plans of action when the project is over and USAID turnover the responsibility to the institution to observe and practice autonomy? 0 0 Address the learning gaps 0 0 Conduct research 2 3 Continuous monitoring and evaluation 0 0 Digital file of the locally developed 0 0 Funding support 1 1 Institutionalization of the CRLA 0 0 Instructional delivery and materials 2 3 Provision of technical assistance 0 0 Replicate course program 3 7 Share best practices 1 1 Trainor of Trainer 0 0 4.2. (TEIs) Are there plans to design and offer a parallel course in a relevant master's program at your institution? Does your institution have annexes (branches)? Are there plans to offer the course in relevant programs in those annexes? (TEIs) 0 0 Early childhood education program 1 2 MA Preschool education program 2 4 ABC+ strengthens the IRs How was ABC+ strengthen the following in your school (improved early-grade instruction and delivery, access to quality materials and education capacity and commitment) 0 0 Enhance their (learners) imagination 0 0 More comfortable in using the technology 0 0 Feedback received What feedback have you received from parents or home learning partners in their children about the quality and effectiveness of the supplementary materials? 0 0 Children are very happy 2 2 Easily explain it to their children 1 1 153 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV NAME DESCRIPTION FILES REFERENCES Parents there are happy 1 1 Suited for the learners 0 0 Understand the story through the picture 1 2 FGD 4.1 What are your views (pananaw) about the MTB-MLE (Mother tongue- based multilingual education) policy and its role (papel) in early-grade education? (4.3 eval question and IR3 learning question) 1 1 Big help when it comes to teaching the learners 3 8 Create confusion to those learners who are good in English 2 2 Develop of self confidence 3 5 Difficult to understand 2 4 Easy to understand 3 6 Effective to facilitate learning 2 4 Implementation review 1 2 Medium of instruction 2 2 No background in English it is a great help 0 0 Not to make a subject the MTB 1 3 Opportunity to express themselves 2 3 Removes barrier to learning 1 3 Scaffold learning among learners 1 4 FGD 4.2 How would you sustain /replicate/continue (maipagpapatuloy) the ABC+ programs after its conclusion? 1 1 Adopt and utilize materials 2 2 Allotted funds for trainings 4 9 Another innovation to continue zero non-reader 1 2 Capacitate parents and teachers 2 5 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 154 NAME DESCRIPTION FILES REFERENCES Capacity building of teachers 4 7 Conducting assessment using CRLA 2 3 Continuous conduct of assessment and planning 2 2 Created an extension program 1 1 Financing the reading materials 1 1 Highlights and the edit the input 0 0 Improve and enhance the ABC+ 1 1 Include in Annual Improvement Plan 1 3 Institutionalize contextualization process introduced by ABC+ 2 5 Integrate ABC+ processes 2 5 Printing of materials 3 5 Production of the stories 1 1 Recommend using CRLA as standard tool 1 2 Regular monitoring of strategies 1 3 Tap the stakeholders the community 2 2 FGD 4.3 What were the lessons learned that may support the sustainability of the project in the context of the Department of Education, home learning partners, and LGU? 0 0 Adopt all the learnings gained 0 0 Augment the logistical needs 1 1 Capacity building 1 1 Logistical requirements for trainings 1 1 Printing of learning materials 1 1 Prioritize the learning needs of all 1 1 155 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV NAME DESCRIPTION FILES REFERENCES the learners Support through the SEF 1 1 FGI 13.2 Are there plans to re-design and offer a parallel professional development for teachers? 0 0 Expand the support in learning research materials 1 2 Scale up 1 2 Train the parents 1 4 Training of trainers 1 2 FGI 14. What potential challenges and risk management that you may encounter in the next two years and after the conclusion of the ABC+ project? 0 0 Long as we have frustrated reader 1 2 Homo - All Teachers 1. What ABC+ activities would you like to continue even after the engagement with ABC+ has ended? Why? 0 0 Assessments and supplemental materials 4 7 Integrated with monitoring 1 2 Training for teachers 5 14 Homo - All Teachers 10. What are three challenges in the policy implementation and improvement? What else would you like to see in the future in terms of education system capacity and commitment to manage ABC+ intervention programs? 0 0 Allocate funds for monitoring from ABC+ 3 6 Commitment of teachers 1 3 Consistency of program 1 3 Funding teaching materials for reading 2 3 Innovate instructional materials 1 2 MandE for sustainability 1 3 Maximize support 2 4 More teachers trainings 3 8 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 156 NAME DESCRIPTION FILES REFERENCES Quality materials 2 4 Reward system on ABC+ implementation 1 3 Support from administration 2 4 Support from community 1 3 Support from LGU 2 6 Support from parents 2 5 Support from stakeholders 2 5 Homo - All Teachers 11. What are three opportunities for policy implementation and improvement? 0 0 Enhance teachers skills 1 5 MBHTE and ABC+ partnership 1 3 Programs under MBHTE 1 3 Quality education as a whole 3 7 Homo - All Teachers 2. List three to five best practices that emerged during the implementation of ABC+ programs that could possibly be sustained by your school/division after the engagement ends. What plans does your school have for sustaining or replicating ABC+ activities after the project ends? 0 0 Action plan 1 3 Blended mode of training 1 3 Contextualized materials, strategies 3 5 Continuous improvement 1 3 Gap in the implementation 1 2 GESI and SEL inclusions 1 3 Home learning partners (HLP) 1 1 Institutionalize the program 1 3 Integrate LAC sessions 2 6 Online platform 1 3 Replicate and reproduce SRMs 2 8 Strengthened teaching strategies 2 3 Teacher Training 3 6 157 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV NAME DESCRIPTION FILES REFERENCES Homo - All Teachers 3. What could be the major challenges or obstacles in implementing the plan of your school after the project ends? How do you intend to overcome them? 0 0 Ample time on monitoring and supervision 1 3 Commitment of teachers 1 3 Lack of support from home learning partners 1 2 Learning hubs 2 4 Limited materials and budget 3 4 More partnership 1 1 Reproduction of SRMs 1 5 Time constraints 3 7 Transportation 1 3 Homo - All Teachers 4. Generally, what are parents and community members' attitudes towards Mother Tongue-Based Multilingual Education (MTB-MLE)? 0 0 Acceptance of community 1 3 Hard time translating 1 2 Linguistically diverse context 3 5 Many are very much happy 1 2 Not so okay 1 1 Positive attitudes 1 3 Prefer Filipino language 4 4 School’s MOI is Filipino 1 2 School’s MTB-MLE subject is Filipino 1 2 Teachers are not speakers of the learners’ mother tongue 2 3 Homo - All Teachers 5. Has the region, division, or school made policy, budget, staffing, etc. provisions for continuing ABC+ activities for teacher training, SRM development and distribution, and policy review on the implementation and continuous improvement of MTB-MLE even after the program has ended? 0 0 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 158 NAME DESCRIPTION FILES REFERENCES Discuss each provision. Implementation (ObEx) Object of Expenditures 2 4 MOOE allocation in support to the ABC+ 1 1 None 3 8 Homo - All Teachers 6. To what extent have ABC+ activities involved community stakeholders (LGUs, NGOs, elders, etc.) for the improvement of early-grade instruction? 1 1 Always supportive 3 5 BLGU is very supportive 2 4 Gave materials and sometimes finances 1 1 Homo - All Teachers 7. What emerging partnerships that were initiated by your region/division/school may support the sustainability of gains of learners in early-grade literacy? 0 0 Partnership Barangay council 2 2 Partnership with LGU 2 3 Private and Public Partnership 1 3 SERG; LandD trainings 1 3 Homo - All Teachers 8. What component/s of the ABC+ program need policy recommendations to ascertain sustainability? 0 0 Advocacy or Program needs to be strengthened 1 3 Benchmarking or partnership in Barangays 2 3 Care for non-readers (CNR) and ELLN 1 3 Community engagement 1 3 Development of SRMs 2 4 SGA sa usapin ng SEF 1 3 Supplementary materials 2 6 Teacher trainings 3 5 159 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV NAME DESCRIPTION FILES REFERENCES Training on basic reading instruction 2 3 Homo - All Teachers 9. What are three opportunities for teacher professional development? What else would you like to see in the future for teacher professional development? 0 0 Committed and creative teachers 1 4 Developing the SRMs 3 5 Make original content 1 1 Masters Education Scholarships 1 3 Materials development 1 5 More trainings on MandE 3 6 Provision of the relevant tools 3 6 Skilled reading teachers 3 5 Teachers to innovate materials 2 3 Technical assistance 2 5 KII 4.1 Are policies in place to continue with these ABC+ initiatives after 2024? If yes, what are these policies? 0 0 Capacity building for teachers 6 15 Community-based literacy program 4 4 Contextualization of policy 7 10 Continue on developing learning materials 16 29 CRLA will be used as assessment tool 10 26 DepEd policy memos and orders 23 44 Institutionalization of assessment 5 7 K-12 Curriculum 2 2 Learning Recovery Plan 9 15 Monitoring and Evaluation 2 3 MT utilization 2 2 MTB-MLE policy 1 1 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 160 NAME DESCRIPTION FILES REFERENCES None 3 3 Not yet 4 6 Parent capacity training 2 2 Reading program 4 5 KII 4.2 Are there plans to design and offer a parallel professional development for teachers? 0 0 GIDCA and Last Mile Schools Training 1 3 Learning action cell (LAC) 0 0 Regional Learning Facilitators 1 4 School head and superintendent training 3 6 Sustain LAC sessions 2 2 Teacher training 22 43 Technical assistance 1 2 Training on Language Transition 1 3 Upskilling of story book writers and illustrators 2 5 Use of Supplemental Reading Materials 5 7 KII 4.3 What potential challenges and risk management that you may encounter in the next two years and after the conclusion of the ABC+ project? 0 0 Change in management 7 12 Early childhood teacher 1 1 External Funding 3 3 External partners 2 2 Financial capacity 6 9 IT technical support 2 3 Lack of books 2 2 Lack of budget or funding 14 32 Lack of teachers in preschool 1 1 161 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV NAME DESCRIPTION FILES REFERENCES Learning needs 3 3 Lifespan of materials 5 7 Losing strong partner 4 4 MTB both as subject as a medium of instruction 3 4 Priorities of new administration 3 8 Reproduction and distribution of the materials 7 13 Revise the curriculum 2 5 Sustain the program and activities 9 17 KII 4.4 What were the lessons learned that may support the sustainability of the project in the context of the Department of Education and local actors? 0 0 Capacity building for all 17 22 Committed and dedicated advocates of the program 14 22 Efficacy and efficiency of the program 8 11 MandE is to be strengthened 2 3 More learners who are reading 2 6 Numeracy Assessment 2 3 Reading materials contextualized 2 7 Real-time monitoring 0 0 KII 4.5 What plans do you have in order to sustain/replicate/continue the program after the project ends? (Sustainability, IR 3) 1 1 Assess the Arabic learning level 2 3 Compilation of materials 6 7 Contextualization of learning materials 11 14 Continual teacher training and upgrading 12 23 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 162 NAME DESCRIPTION FILES REFERENCES Coordinate with the parents 1 2 Language identification tool or orthography 2 3 Linkages on our LGUs and private sector 6 6 Monitoring and assessment tool from ABC+ 4 7 Plan to institutionalize assessment of ABC+ 5 6 LGU NGO DepEd Capacity and Commitment What were LGU, NGO, DepEd capacity and commitment programs to inform decision making for budgeting, targeting resources, communication, and monitoring expectations and implementing improvements in early-grade instruction? 4 4 Donations of school supplies 0 0 Established partnership 1 1 Utilize reading center 0 0 Workshops with LGU 0 0 Partnership with the barangay or other community groups - Additional programs or assistance Do you have a partnership with the barangay or other community groups? What additional programs or assistance introduced by ABC+ was provided to strengthen your partnership with the community? 1 1 Barangay to assist the reading difficulties 0 0 Barangay was the one to prepare the venue 0 0 Support of the barangay official 0 0 Recommendations 7 14 All regions or scale up 2 4 Community-based literacy 2 2 Coordination with community 0 0 Data-driven decision 1 1 Funding source 4 6 163 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV NAME DESCRIPTION FILES REFERENCES Importance of stakeholders 2 3 Inter-collaboration with BLGU 1 1 Monitoring and evaluation 3 3 Policy alignment 0 0 Program adaptation 5 5 Program contextualization 2 3 Support system 3 3 Teacher training 4 5 Technical support 2 3 Tertiary level 2 5 Themes A: FGD B: FGD C:FGD D: FGD E: FGI F: KII G: KII H: KII I: KII 1. Active participation 2 0 0 0 0 0 1 3 1 2. Appropriateness of materials 4 1 1 2 0 3 7 6 5 3. Collaboration 1 0 1 0 0 1 3 0 0 4. Conducive PD environment 4 0 0 0 0 0 2 0 2 5. Development of socio-emotional learning (SEL) based materials 0 1 0 0 0 1 0 1 1 6. Development of MTB-MLE materials 0 0 0 0 0 17 3 1 2 7. Differentiated instruction 0 0 0 0 0 2 3 2 0 8. GESI promotion 2 1 1 0 0 8 3 1 4 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 164 Themes A: FGD B: FGD C:FGD D: FGD E: FGI F: KII G: KII H: KII I: KII 9. Instructional innovation 0 1 0 1 0 1 0 3 0 10. IR1 0 0 0 0 0 5 1 0 0 11. IR2 2 0 5 0 0 1 1 0 0 12. IR3 5 0 1 1 1 3 2 0 3 13. Learners 1 0 4 0 0 0 0 2 0 14. LGU 1 5 1 0 0 2 1 0 3 15. Love of language 0 0 0 1 0 2 0 3 0 16. Mobilization of local stakeholders 1 3 0 0 0 4 0 6 2 17. MTB-MLE coordinators 4 0 0 0 0 7 2 0 10 18. Overlapping of activities 0 0 0 0 0 0 1 3 0 19. Pandemic pivots 0 0 0 1 0 0 31 4 0 20. Parents 10 0 0 0 0 5 6 1 10 21. Persistent 4 1 0 0 0 3 7 2 15 22. Policy alignment 1 0 0 0 1 0 1 1 0 23. Presence of community hub 0 0 0 0 0 0 0 0 0 24. Professional development of school heads 2 0 0 0 0 0 0 0 0 165 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Themes A: FGD B: FGD C:FGD D: FGD E: FGI F: KII G: KII H: KII I: KII 25. Professional development of teachers 1 0 1 0 0 2 10 0 0 26. Quality 3 0 0 0 0 17 4 0 3 27. Reading progress 0 0 0 0 0 0 4 1 2 28. Reliable collaborator 2 1 0 1 0 8 5 0 3 29. Satisfied 0 0 2 0 0 0 5 2 2 30. School heads 1 0 0 0 0 5 2 0 19 31. School-initiated plan for K-3 resources 0 0 0 0 0 8 0 2 0 32. Stimulating 2 1 0 0 0 2 0 0 2 33. Teachers 0 0 0 0 0 12 5 4 5 34. Teaching innovation 6 5 0 0 0 2 3 1 13 35. Use of various learning materials 0 0 0 0 0 1 10 0 0 36. User friendly 1 0 0 0 0 0 0 0 1 37. Value for money 0 0 0 0 0 1 1 0 2 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 166 Themes 1 : F G D 2 : F G D 3 : F G D 4 : F G D 5 : F G D 6 : F G D 7 : F G D 8 : F G D 9 : F G D 1 0 : F G D 1 1 : F G D 1 2 : F G D 1 3 : F G D 1 4 : F G D 1 5 : F G D 1 6 : F G D 1 7 : K I I 1 8 : K I I 1 9 : K I I 2 0 : K I I 2 1 : K I I 2 2 : K I I 2 3 : K I I 2 4 : K I I 2 5 : K I I 2 6 : K I I 2 7 : K I I 2 8 : K I I 2 9 : K I I 3 0 : K I I A. Contribution in children’s literacy 34 18 12 19 17 0 0 0 0 0 0 0 1 1 6 2 22 21 24 23 43 13 0 0 0 0 0 0 0 0 B. Contribution in children’s numeracy 1 6 0 2 0 0 0 0 0 0 0 0 1 1 0 0 2 2 0 1 1 0 0 0 0 0 0 0 0 0 C. Contribution in socio￾emotional learning 1 2 1 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 3 5 3 1 0 0 0 0 0 0 0 0 D. Enhanced pedagogical content knowledge (PCK) 26 14 5 14 4 0 0 0 0 0 0 0 1 0 6 1 9 2 21 41 36 6 0 0 0 0 0 0 0 0 E. GESI (Gender 0 6 1 4 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 167 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV equality and social inclusion promotion F: Goal Alignment 12 6 3 5 2 0 0 0 0 0 0 0 2 0 0 0 7 5 12 9 20 13 0 0 0 0 0 0 0 0 G. Notable remarks 10 9 1 3 9 0 0 0 0 0 0 0 0 0 1 0 4 2 4 5 10 7 0 0 0 0 0 0 0 0 H: Pandemic pivots 4 5 8 3 1 0 0 0 0 0 0 0 0 1 1 0 5 2 5 3 5 0 0 0 0 0 0 0 0 0 I: Parental and community engagement and or involvement 5 5 14 5 5 0 0 0 0 0 0 0 0 1 0 0 2 2 7 3 2 0 0 0 0 0 0 0 0 0 J: Policy Alignment 18 9 1 2 0 0 0 0 0 0 0 0 2 0 1 0 53 20 11 5 7 2 0 0 0 0 0 0 0 0 K: Reaching out to marginalized groups 0 1 1 2 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 2 0 0 0 0 0 0 0 0 0 L: Support to GIDCA and Last Mile 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 168 Schools M: Support to GIDCA and Last Miles Schools 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 169 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 15. INFORMANTS’ IDENTIFIED POLICIES THAT ALIGN WITH ABC+ Issuance Date Title National Policies RA 9155 “Governance of Basic Education Act of 2001” August 11, 2001 AN ACT INSTITUTING A FRAMEWORK OF GOVERNANCE FOR BASIC EDUCATION, ESTABLISHING AUTHORITY AND ACCOUNTABILITY, RENAMING THE DEPARTMENT OF EDUCATION, CULTURE, AND SPORTS AS THE DEPARTMENT OF EDUCATION, AND FOR OTHER PURPOSE DepEd Order No. 012, s. 202 March 15, 2022 POLICY GUIDELINES FOR THE PROVISION OF LEARNING RESOURCES AND NEEDED DEVICES AND EQUIPMENT AND FUNDING RELEVANT ACTIVITIES FOR THE IMPLEMENTATION OF THE BASIC EDUCATION￾LEARNING CONTINUITY PLAN DepEd Order No 032, s. 2020 October 15, 2020 GUIDELINES ON THE ENGAGEMENT OF SERVICES OF LEARNING SUPPORT AIDES TO REINFORCE THE IMPLEMENTATION OF THE BASIC EDUCATION LEARNING CONTINUITY PLAN IN TIME OF THE COVID-19 PANDEMIC DepEd Order No. 074, s. 2009 July 14, 2009 INSTITUTIONALIZING MOTHER TONGUE-BASED MULTILINGUAL EDUCATION DepEd Order No. 016, s. 2012 February 17, 2012 GUIDELINES ON THE IMPLEMENTATION OF THE MOTHER TONGUE￾BASED MULTILINGUAL EDUCATION (MTB-MLE) DepEd Order No. 070, s. 2011 September 08, 2011 GUIDELINES ON THE UTILIZATION OF FUNDS FOR EVERY CHILD A READER PROGRAM (ECARP) DepEd Order No. 018, s. 2017 April 19, 2017 GUIDELINES ON THE UTILIZATION OF THE 2017 EVERY CHILD A READER PROGRAM FUNDS FOR THE EARLY LANGUAGE, LITERACY, AND NUMERACY PROGRAM: PROFESSIONAL DEVELOPMENT COMPONENT DepEd Order No. 031, s. October 02, THE INTERIM GUIDELINES FOR ASSESSMENT AND GRADING IN USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 170 Issuance Date Title 2020 2020 LIGHT OF THE BASIC EDUCATION LEARNING CONTINUITY PLAN DepEd Order No. 032, s. 2015 July 29, 2015 ADOPTING THE INDIGENOUS PEOPLES EDUCATION CURRICULUM FRAMEWORK DepEd Order No. 022, s. 2016 April 19, 2016 IMPLEMENTING GUIDELINES ON THE ALLOCATION AND UTILIZATION OF THE INDIGENOUS PEOPLES EDUCATION (IPed) PROGRAM SUPPORT FUND FOR FISCAL YEAR (FY) 2016 DepEd Memorandum No. 173, s. 2019 November 22, 2019 HAMON: BAWAT BATA BUMABASA (3Bs Initiative) DepEd Order No. 021, s. 2019 August 22, 2019 POLICY GUIDELINES ON THE K TO 12 BASIC EDUCATION PROGRAM DepEd Order No. 014, s. 2018 March 26, 2018 POLICY GUIDELINES ON THE ADMINISTRATION OF THE REVISED PHILIPPINE INFORMAL READING INVENTORY DepEd Order No. 045, s. 2002 September 11, 2002 READING LITERACY PROGRAMS IN THE ELEMENTARY SCHOOLS DepEd Order No. 012, s. 2020 June 19, 2020 ADOPTION OF THE BASIC EDUCATION LEARNING CONTINUITY PLAN FOR THE SCHOOL YEAR 2020-2021 IN LIGHT OF THE COVID-19 PUBLIC HEALTH EMERGENCY DepEd Order No. 014, s. 2018 March 26, 2018 POLICY GUIDELINES ON THE ADMINISTRATION OF THE REVISED PHILIPPINE INFORMAL READING INVENTORY Regional Policies RM No. 769, s. 2022 October 06, 2022 ASSESSMENT ACROSS KEY STAGES AND LEARNING AREAS 171 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Issuance Date Title RM. No. 412, s. 2022 June 08, 2022 CONDUCT OF THE REGION-WIDE END-OF-SCHOOL-YEAR RAPID LITERACY ASSESSMENT RM No. 127 s. 2022 October 19, 2022 GUIDELINES ON THE 8-WEEK RECOVERY CURRICULUM OF THE POST-IMPLEMENTATION ASSESSMENT ACTIVITIES RM No. 841, s. 2022 August 03, 2022 GENERAL GUIDELINES ON THE LEARNING RECOVERY PROGRAM IMPLEMENTATION THROUGH THE “BAWAT BATA SA BANGSAMORO AY BUMABASA AT BUMIBILANG or 5Bs PROGRAM” RM No. 047, s. 2020 June 05, 2020 POLICY GUIDELINES ON THE CONDUCT OF ONLINE QUALITY ASSURANCE OF LEARNING RESOURCES RM No. 74, s. 2022 June 06, 2022 IMPLEMENTATION OF THE 8-WEEK LEARNING RECOVERY PROGRAM IN GRADES 1 TO 3 FOR THE SCHOOL YEAR 2022-2023 RM No. 764, s. 2022 October 05, 2022 PERFORMANCE EVALUATION OF ADVANCING THE BASIC EDUCATION (ABC+) IN THE PHILIPPINES USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 172 ANNEX 16. ACTIVITY SUMMARY INFORMATION OF ABC+ TABLE 1. ACTIVITY SUMMARY INFORMATION25 TITLE ABC+: ADVANCING BASIC EDUCATION IN THE PHILIPPINES Cooperative Agreement Number 72049219CA00005 Start and end dates July 1, 2019, to June 28, 2024 Funding levels $38.5 million Implementing partners RTI International Technical lead on all intermediate results (IRs). Overall project management, quality assurance, and timely reporting. Sub-awardees The Asia Foundation Local government capacity building for sustainability; strengthening partnerships with the private sector (IR 3). Mother tongue materials development (IR 2). SIL LEAD Language policy and capacity building for developing reading materials (IRs 2 and 3). Florida State University Partnering with universities in the regions to develop pre-service modules for mother-tongue literacy and math instruction (IR 1). UP-NISMED 25 Fiscal Year 2022, Quarter 2 Report January 1–March 31, 2022. ABC+: Advancing Basic Education in the Philippines. 173 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 17. STATEMENT OF WORK PERFORMANCE EVALUATION STATEMENT OF WORK ADVANCING BASIC EDUCATION IN THE PHILIPPINES (ABC+) JULY/2022 RTI INTERNATIONAL USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 174 TABLE OF CONTENTS ACRONYMS III PURPOSE OF THE EVALUATION 1 SUMMARY INFORMATION 1 BACKGROUND 2 DESCRIPTION OF THE PROBLEM AND CONTEXT 2 THEORY OF CHANGE (TOC) 4 RESULTS FRAMEWORK 11 EVALUATION QUESTIONS 13 RELEVANCE 13 EFFECTIVENESS 14 SUSTAINABILITY 15 EVALUATION DESIGN AND METHODOLOGY 16 LOCATION AND GEOGRAPHIC SCOPE 16 EVALUATION DESIGN 16 DATA COLLECTION 16 DATA ANALYSES 18 QUANTITATIVE 18 QUALITATIVE 18 TRIANGULATION 18 EVALUATION DESIGN MATRIX 18 DELIVERABLES AND REPORTING REQUIREMENTS 20 PRE-FIELD PHASE 20 1. IN-BRIEF MEETING WITH USAID 20 175 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV INCEPTION PHASE 20 2. INCEPTION REPORT WITH EVALUATION DESIGN, WORK PLAN, SCHEDULE, AND TOOLS 20 FIELDWORK PHASE 20 3. MID-TERM BRIEFING AND INTERIM MEETINGS 20 ANALYSIS PHASE 21 4. DRAFT EVALUATION REPORT AND USAID OUT-BRIEFING 21 LEARNING AND DISSEMINATION PHASE 21 5. LEARNING EVENT(S) AND PRODUCTS FOR EVALUATION UTILIZATION 21 6. LEARNING EVENT SYNTHESIS 21 EVALUATION TEAM COMPOSITION 21 STAFFING 21 EVALUATION ADVISOR (TEAM LEADER) 22 TECHNICAL SPECIALIST: PROFESSIONAL DEVELOPMENT SPECIALIST 22 TECHNICAL SPECIALIST: EDUCATIONAL ADMINISTRATION SPECIALIST 23 FIELD REGIONAL ASSISTANTS (UP TO 3 MEMBERS) 23 DATA ANALYST 24 PROJECT ASSISTANT 24 LEVEL OF EFFORT (LOE) ESTIMATE FOR THE EXTERNAL EVALUATION TEAM 25 PERIOD OF PERFORMANCE AND ESTIMATED EVALUATION SCHEDULE 25 FINAL REPORT FORMAT 27 EVALUATION REPORT QUALITY CRITERIA 28 DATA REQUIREMENTS 29 ANNEX 1. AVAILABLE DATA 30 ANNEX 2: EVALUATION DESIGN MATRIX 31 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 176 177 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ACRONYMS BARMM Bangsamoro Autonomous Region in Muslim Mindanao CO Central Office (DepEd) DepEd Department of Education EGR Early-grade reading assessment GESI Gender equity and social inclusion KPI Key performance indicator LGU Local Government Unit LOI Language of instruction MTB-MLE Mother Tongue-Based Multilingual Education OH Outcome Harvesting PTC Parent Teacher Community Qual Qualitative Quan Quantitative RO Regional Office RTI Research Triangle Institute SD School Division SEL Socio Emotional learning TAF The Asia Foundation ToC Theory of Change TOCOR Task Order Contracting Officer’s Representative TPACK Technological, pedagogical, and content knowledge USAID U.S. Agency for International Development USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 178 PURPOSE OF THE EVALUATION The United States Agency for International Development/Philippines (USAID/PH) is commissioning a third-party performance evaluation of ABC+: Advancing Basic Education in the Philippines implemented by RTI International, the prime partner, The Asia Foundation, SIL Lead, and Florida State University. Through this evaluation, USAID/PH will have evidence to gauge the extent to which ABC+ achieves its outputs and outcomes in the three intermediate results (IRs) on improved early￾grade instructions and delivery (IR1), improved access to quality materials (IR2), and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction (IR3). The evaluation will focus on the performance of ABC+ from 2019 to 2022. Specifically, it will assess its relevance, effectiveness, and sustainability through a series of evaluation questions. ABC+ works to support the Philippines Department of Education’s goal to increase and sustain basic skills among K-3 learners. Results of the evaluation will inform the design of the Mission’s next-generation basic education activities. The intended audiences of this evaluation are USAID/PH program officers and USAID staff worldwide interested in basic education programs, specifically at K-3 levels. Philippine and U.S. stakeholders, including those in other U.S. government agencies and organizations, the Department of Education (DepEd) in the Philippines, United States, and worldwide, and other researchers and organizations interested in basic education. SUMMARY INFORMATION Figure 1 summarizes the ABC+ activity information.26 26 Fiscal Year 2022 , Quarter 2 Report January 1–March 31, 2022. ABC+: Advancing Basic Education in the Philippines. 179 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Figure 1 - Activity Summary Information Title ABC+: Advancing Basic Education in the Philippines Cooperative Agreement Number 72049219CA00005 Start and end dates July 1, 2019, to June 28, 2024 Funding levels $38.5 million Implementing partners RTI International Technical lead on all intermediate results (IRs). Overall project management, quality assurance, and timely reporting. Sub-awardees The Asia Foundation Local government capacity building for sustainability; strengthening partnerships with the private sector (IR 3). Mother tongue materials development (IR 2). SIL LEAD Language policy and capacity building for developing reading materials (IRs 2 and 3). Florida State University Partnering with universities in the regions to develop pre-service modules for mother-tongue literacy and math instruction (IR 1). UP-NISMED Early Numeracy Initiative pilot and research (IR 1) Sectors or topics Early reading, math, and social and emotional skills for children in the early￾grades (Kindergarten through grade 3 [K–3]) in the Philippines. BACKGROUND DESCRIPTION OF THE PROBLEM AND CONTEXT The ABC+: Advancing Basic Education in the Philippines is a five-year USAID-funded program aimed at improving the early-grade reading, math, and social and emotional skills of children in the early￾grades (kindergarten to grade 3) in the Philippines27. The prominent feature of early-grade 27 Monitoring, evaluation, and learning plan. ABC+: Advancing Basic Education in the Philippines USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 180 instructions in the K to 12 Curriculum in the Philippines is the Mother Tongue-Based Multilingual Education (MTB-MLE) Policy. From kindergarten to grade 3 (K– 3), learners progressively acquire literacy skills in their mother tongue (MT), Filipino, and English, transitioning to instruction in Filipino and English starting in grade 4. In a country with 180 plus native languages, the delivery of MTB-MLE throughout the country can be daunting. The baseline study conducted by ABC+ reflects this, showing that many children are being left behind in school, particularly boys and rural communities. Grade 2 learners’ decoding skills are underdeveloped, and reading and comprehension abilities are weaker than their oral language skills. Although supplemental materials are available at the Department of Education, teachers did not use them in their instructions. In response to the challenges of early literacy development in the context of MTB-MLE, the United States Agency for International Development (USAID), the Philippines Department of Education (DepEd), and RTI International (RTI) and its subcontractors co-developed a project targeted at improving basic skills in the early-grades, with an emphasis on improving early-grade reading in the MT and transition to Filipino and English. The project also supports improved early-grade literacy, better transition of reading ability to Filipino and English, more solid acquisition of foundational math concepts and skills, and improved social and emotional learning. ABC+ will work with the Philippine Department of Education (DepEd) in Manila, as well as with its regional- and division-level offices in Regions V (Bicol) and VI (Western Visayas) and selected districts of Maguindanao in the Bangsamoro Autonomous Region in Muslim Mindanao (BARMM).28 Figure 2 shows the estimated numbers of beneficiaries such as students, teachers, school heads, and supervisors. Figure 2 Estimated Number of Beneficiaries (Life of Project) In March 2020, schools closed due to COVID-19 disrupting ABC+’s implementation. Most activities became remote, including teacher training. In December 2021, selected DepEd schools piloted face￾to-face classes. Beginning in January 2022, DepEd progressively expanded face-to-face classes as part of its Learning Recovery Plan. ABC+ partnered with DepEd Region V to contribute to this effort in preparing The Recovering for Academic Achievement by Improving Instruction through 28 Monitoring and Evaluation Learning Plan: Advancing Basic Education in the Philippines (ABC+) 181 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Sustainable Evidence-Based Learning Programs (RAISE). This 3-year learning recovery plan aims to ensure that the COVID-19 generation of learners does not suffer a disadvantage compared with past and future generations. It hopes to enable this cohort of learners to fully recover the learning lost during the pandemic and prioritizes remedial or accelerated education approaches to address challenges and get all learners to grade-level proficiency. RAISE also invests in appropriate resources to offset learning losses and minimize drop-out rates.29 For the school reopening set in SY 2022-2023, ABC+ continued to facilitate the collaboration between the Office of Health and the Office of Education and selected implementing partners to support the reopening of schools for face-to-face classes. ABC+ gained additional funding to provide support in BARMM and Region VI schools. As part of this collaboration, the Breakthrough Action, part of the USAID/Philippines ReachHealth project, developed the BIDA Kid Campaign, which focuses on motivating children and their families to follow health protocols for a safe return to school. ABC+ also has resumed field visits to meet with partners and observe activity implementation. THEORY OF CHANGE (TOC) ABC+’s design assumes that “IF quality of instruction is improved (IR 1), quality and availability of materials are enhanced (IR 2), and education system capacity to support early-grade learning (IR 3) is improved, THEN it will lead to positive and sustained gains in students’ basic skills. ABC+ seeks to achieve its overall objective through the following Intermediate Results (IR): IR 1: Improved early-grade instruction and delivery systems IR 2: Improved access to quality materials IR 3: Improved education system capacity and commitment to manage and implement interventions supporting effective early-grade instruction Specifically, the theory of change for each IR is as follows: IR 1. IF teachers master and apply effective reading instructional practices for the early-grades, instructional supervisors have a better capacity to support developmentally appropriate instructional practices that foster children’s cognitive, social, and emotional development; higher education institutions are better equipped to prepare pre-service teachers for early-grades instruction and to pilot instructional innovations; and there is increasing evidence on what works to strengthen instructional practices in the early-grades, THEN teachers will be more effective in providing instruction that develops students’ basic skills in the early-grades. IR 2. IF local capacity to develop quality early reading materials is strengthened, critical gaps (language and level) in the supply of materials are filled, inefficiencies in the book chain are resolved, and digital versions of materials are made more easily accessible, THEN children will have more equitable access to greater quantities of quality reading materials, in the right language and at the right level. 29 ABC+ Fiscal Year 2022 Quarter 2 Report, January 1-March 31, 2022 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 182 IR 3. IF policies to support mother tongue (MT) instruction and provision of learning resources are effectively implemented, local government units (LGUs) and local stakeholders are mobilized to support and sustain ABC+, introduce improvements in early-grade instruction, and private sector engagement in support of ABC+, introduce improvements in early-grade instruction is increased, and parent and community involvement in supporting early-grade learning is increased, THEN the education system will be better positioned to deliver and sustain quality learning opportunities in early-grades. Figure 3 illustrates the theory of change. Figure 3 - ABC+ Theory of Change Diagram The ABC+ Logical Framework Model (LogFrame) derives from the theory of change. Figure 4 details the full LogFrame model mapping key activities (inputs/processes) to outputs to IR-level behavioral change outcomes and associated KPIs. There are 22 KPIs drawn from the LogFrame, including two specific gender equality and social inclusion (GESI) embedded directly under IRs 1 and 2 (indicators 1.1 d and 2.2) 183 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Figure 4 - Detailed Log Frame Activity (Processes/Inputs) Outputs IR-Level Behavior Change Outcomes Corresponding KPI(s) ABC+ Goal-Level Theory of Change and Logical Framework Teachers, school heads, and supervisors trained and supported to deliver effective early-grade reading (EGR) instruction K–3 classrooms receive an increased number of quality, grade-appropriate reading materials The enabling environment (e.g., policies, partnerships, and resources) is fostered to focus support on EGR best practices and inputs K–3 students receive improved instruction K–3 students have greater access to and use of quality reading materials Schools, classrooms, and teachers benefit from increased support K–3 students increase the rate of literacy foundational skills in their MT K–3 students transition to acquiring literacy skills in Filipino Schools, school divisions, regional offices (ROs), LGUs, and the private sector increase focus and support for EGR resources and best practices Goal-level (GL)1. Percent of learners targeted for US Government (USG) assistance who attain a minimum grade-level proficiency in reading at the end of grade 2 in MT (ES.1-1- outcome) GL2. Percent of learners targeted for USG assistance who attain a minimum grade-level proficiency in reading at the end of grade 2 in Filipino (ES.1-1-outcome) GL3. Percent of learners targeted for USG assistance with an increase of at least one proficiency level in reading at the end of grade 2 in MT (ES.1-48-outcome) GL4. Percent of learners targeted for USG assistance with an increase of at least one proficiency level in reading at the end of grade 2 in Filipino (ES.1-48-outcome) GL5. Number of learners in primary schools reached with USG assistance (ES.1-3-output) GL6. Number of learners reached in reading USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 184 Activity (Processes/Inputs) Outputs IR-Level Behavior Change Outcomes Corresponding KPI(s) programs at the primary level (ES.1-5-output) IR 1: Improved early-grade instruction and delivery systems Cadre of master trainers trained in instructional best practices K–3 teachers trained in instructional best practices GESI awareness assessment and content integrated into training modules Teachers equipped with the knowledge, skills, and self-efficacy to deliver effective EGR instruction Teachers’ sensitivity and awareness of GESI-related issues increased Demonstrably positive changes in teachers’ knowledge, attitudes, and practices in delivering EGR instruction Teachers apply GESI-sensitive instructional practices to differentiate between boys’ and girls’ unique learning needs 1.1a. Percent of grade 2 teachers delivering quality instruction with fidelity to the training and materials, according to a clearly defined set of observable practices (outcome) 1.1b. Percent of grade 2 teachers indicating positive beliefs and knowledge of EGR best practices (output) 1.1c. Number of primary school educators who complete professional development activities on implementing evidence-based reading instruction with USG assistance (ES.1- 7) 1.1d. Percent of school heads demonstrating practical knowledge and understanding of GESI best practices (GESI-output) School heads trained to foster a greater SEL environment in the classrooms GESI awareness assessment and content integrated into SEL modules School heads have the knowledge and skills to promote SEL in their schools School heads and teachers actively promote a positive SEL environment 1.1e. Percent of school heads demonstrating practical knowledge and understanding of SEL best practices (custom-output) 185 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Activity (Processes/Inputs) Outputs IR-Level Behavior Change Outcomes Corresponding KPI(s) Pre-service course modules and syllabi adapted to deliver best practices in student￾teacher preparation for EGR instruction Pre-service instructors trained on EGR best practices for preparing student-teachers Instructors equipped with the knowledge, skills, and materials to improve their preparation for EGR courses Instructors apply knowledge, skills, and best practices in delivering EGR courses 1.1f. Number of teacher education institutions (TEIs) that adopt ABC+-introduced modules on early-grade literacy and numeracy instruction (output) School heads and supervisors trained to monitor, support, and provide effective feedback to K–3 teachers Teacher observation and feedback tools, as well as Learning Action Cells (LAC) materials adapted and provided to school heads and supervisors School heads and supervisors equipped with tools and skills to support and mentor teachers to improve their EGR instructional practices School heads and supervisors actively monitor, support, and provide effective feedback to teachers to improve their EGR instructional practices LACs become more effective venues for peer-to-peer learning 1.2. Percent of school heads and supervisors actively providing effective instructional support and feedback to K–3 teachers according to clearly defined expectations (outcome) ABC+ trains selected teachers in innovative math instructional practices Selected math teachers equipped with the knowledge and skills to deliver innovative instruction in mathematics Math teachers apply knowledge and skills of innovative math instruction in the classroom 1.3. Percent of mathematics teachers delivering innovative math instruction according to a clearly defined set of observable practices in fidelity to the training (outcome) ABC+ trains selected teachers on Tangerine:ClassTM application to improve their continuous assessment and differentiated instructional practices in EGR Selected K–3 teachers equipped with the skills, knowledge, tools, and technology to utilize the Tangerine:ClassTM application Teachers effectively use Tangerine:ClassTM to implement their routine assessment and apply differentiated instructional practices 1.4. Percent of teachers using Tangerine:ClassTM appropriately according to a clearly defined set of observable practices in fidelity to the training (outcome) USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 186 Activity (Processes/Inputs) Outputs IR-Level Behavior Change Outcomes Corresponding KPI(s) IR 2: Improved access to quality materials Story writers and illustrators trained to utilize low-cost technologies to develop quality leveled supplemental MT reading materials Leveled supplemental reading materials in target MTs produced and distributed to K–3 classrooms GESI awareness issues and content integrated into supplemental materials K–3 classrooms receive leveled supplemental reading materials in target MTs Story writers and illustrators equipped with the knowledge and tools to develop quality supplemental materials in MT Teachers and students use high￾quality MT reading materials in the classrooms 2.1. Percent of K–3 classrooms in which teachers and learners are regularly using supplemental learning materials according to a clearly defined set of observable criteria (outcome) DepEd central office (CO), ROs, and Schools Divisions supported to develop high-quality SEL materials DepEd CO, ROs, and Schools Divisions equipped with the knowledge, skills, and resources to produce high-quality SEL materials DepEd CO, ROs, and Schools Divisions produce and disseminate SEL materials 2.2. Percent of titles produced by ABC+ that meet GESI-appropriate standards based on a checklist of clearly observable criteria (GESI￾output) Supplemental reading materials in Filipino and English procured and distributed to K–3 classrooms K–3 classrooms receive supplemental learning materials in Filipino and English Teachers and students use high￾quality English and Filipino supplemental materials in the classroom 2.3. Percent of primary-grade learners targeted for USG assistance who have the appropriate variety of decodable, leveled, and supplementary readers in the LOI (ES.1- 45) The LRP system is improved in terms of its functionality, user experience, and accessibility An improved LRP interface is tested and deployed to participating regions Teachers and administrators increasingly access and download materials from the LRP 2.4. Content, access, and usability of LRP enhanced according to system metrics and key informant teachers and administrators (outcome) 187 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Activity (Processes/Inputs) Outputs IR-Level Behavior Change Outcomes Corresponding KPI(s) IR 3: Improved education system capacity and commitment to manage and implement interventions supporting effective early-grade instruction Teacher observation and feedback tools adapted from the Philippine Professional Standards for Teachers (PPST) and provided to school heads and supervisors, along with reporting and feedback templates School heads and supervisors are equipped with the knowledge, tools, and skills to monitor teachers and report on instructional practices School heads and supervisors report summarized results, aggregated at the division level, and include them in the school monitoring, evaluation and adjustment process, quarterly division monitoring, evaluation, and adjustment process (DMEA) meetings 3.2a. Number of Schools Divisions incorporating timely indicators on teachers’ quality of EGR instruction in their DMEA process (outcome) DepEd CO, ROs, and Schools Divisions engaged to review existing plans and budgets and to include in their 2021 budgets resource allocations to cover additional EGR-related costs Documented reviews of existing allocations to identify opportunities to better align resources with priority areas of investment for improving implementation of policies in support of early-grades literacy and learning outcomes Budgets and plans incorporate additional allocations or programs that focus specifically on literacy and early-grade learning outcomes 3.2b. Percent change in the allocation of resources for learning materials and teacher professional development at central, regional, and division levels LGUs engaged in identifying current gaps in Special Education Fund (SEF) guidelines, analyzing the prevailing patterns of resource allocation and expenditure, and strengthening local school board (LSB) planning and prioritization Documented analyses of LSB/SEF funding and resource allocation patterns LGU/LSB plans and SEF allocation priorities incorporate additional allocations or programs that focus specifically on literacy and early-grade learning outcomes 3.3 Percent change in the allocation of SEF funds from LGUs in target regions for printing and distribution of learning resources and teacher participation in professional development USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 188 Activity (Processes/Inputs) Outputs IR-Level Behavior Change Outcomes Corresponding KPI(s) Build coalitions of stakeholders DepEd assisted in identifying and arranging private sector engagement, e.g., resources, technical expertise, advocacy, and advice, in support of ABC+ and DepEd objectives related to improving early-grade learning Documented analyses of private sector landscape, identification and inventory of potential partnerships, and areas of engagement Private sector partners are increasingly mobilized to focus resources and programs to support MTB-MLE and early￾grade learning programs 3.4. Number of private-sector partners engaged to support evidence-based EGR interventions (outcome) Outreach strategies that can enlist greater parental and community participation in parent-teacher associations (PTAs), civil society organizations, and LSB activities Provide materials to help parents engage in home literacy activities Materials and assistance delivered to selected schools and Schools Divisions to strengthen PTA and community-engagement programs Parents and communities are increasingly engaged and support MTB-MLE and early-grade learning outcomes 3.5. Number of PTAs or community-based school governance structures supported with USG assistance (ES.1-13) 189 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Results Framework ABC+ operates according to the Results Framework shown in Figure 5. This framework summarizes how ABC+’s strategies contribute to results. Figure 5 - Results Framework ABC+ uses the results framework in Figure 5 as a tool to monitor and manage its progress. It sets out (1) development outcomes, (2) results, and (3) metrics to measure effectiveness and efficiency. The results framework shows the hierarchy of results expected within ABC+. To situate the early-grade learning ecosystem for improved learning outcomes in reading, mathematics, and SEL, Figure 6 illustrates a complementary framework for this. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 190 Figure 6 - Complementary Results Framework IR 1: Improved early-grade instruction and delivery systems IR 2: Improved access to quality materials IR 3: Improved education system capacity and commitment to manage and implement interventions supporting effective early￾grade instruction Theory of change (IR!) IF teachers master and apply effective reading instructional practices for the early-grades, instructional supervisors have a better capacity to support developmentally appropriate instructional practices that foster children’s cognitive, social, and emotional development; higher education institutions are better equipped to prepare pre-service teachers for early￾grades instruction and to pilot instructional innovations; and there is increased evidence on what works to strengthen instructional practices in the early-grades, THEN teachers will be more effective in providing instruction that develops students’ basic skills in the early-grades. Theory of change (IR2) IF local capacity to develop quality early reading materials is strengthened, critical gaps (language and level) in the supply of materials are filled, inefficiencies in the book chain are resolved, and digital versions of materials are made more easily accessible, THEN children will have more equitable access to greater quantities of quality reading materials, in the right language and at the right level. Theory of change (IR3) IF policies to support mother tongue (MT) instruction and provision of learning resources are effectively implemented, local government units (LGUs) and local stakeholders are mobilized to support and sustain ABC+-introduced improvements in early-grade instruction, private sector engagement in support of ABC+-introduced improvements in early-grade instruction is increased, and parent and community involvement in supporting early-grade learning is increased, THEN the education system will be better positioned to deliver and sustain quality learning opportunities in early-grades. Figure 3. Complementary Results Framework for ABC+ Theory of Change Program Goal Goal: Improved Learner Outcomes in Reading, Mathematics, and SEL Goal Level Indicators (GLIs) GL1. Percent of learners targeted for USG assistance who attain a minimum grade-level proficiency in reading at the end of grade 2 in MT (ES.1-1-outcome) GL2. Percent of learners targeted for USG assistance who attain a minimum grade-level proficiency in reading at the end of grade 2 in Filipino (ES.1-1-outcome) GL3. Percent of learners targeted for USG assistance with an increase of at least one proficiency level in reading at the end of grade 2 in MT (ES.1-48-outcome) GL4. Percent of learners targeted for USG assistance with an increase of at least one proficiency level in reading at the end of grade 2 in Filipino (ES.1-48-outcome) GL5. Number of learners in primary schools reached with USG assistance (ES.1-3-output) GL6. Number of learners reached in reading programs at the primary level (ES.1-5-output) 191 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Sub-IR 1.1: Early-grade instructional practices for MT, math, and SEL improved Sub-IR 2.1: Capacity to develop leveled, MT supplementary reading materials improved Sub-IR 3.1: Policies to support MT instruction and provision of learning resources are effectively implemented Sub-IR 1.2: School heads’ and district and division supervisors’ capacity to support and monitor instruction improved Sub-IR 2.2: Capacity to develop and provide materials supportive of students’ SEL improved Sub-IR 3.2: DepEd capacity and commitment to use data to inform decision-making for budgeting, targeting resources, communicating and monitoring expectations, and implementing improvements in early-grade instruction reinforced Sub-IR 1.3: Innovations in math instruction piloted, evaluated and, as appropriate, adapted for wider application Sub-IR 2.3: Capacity to ensure ongoing provision of learning resources for grades K–3 developed Sub-IR 3.3: LGUs and local stakeholders mobilized to support and sustain ABC+-introduced improvements in early-grade instruction Sub-IR 1.4: Innovations to support differentiated instruction piloted, evaluated, and, as appropriate, adapted for wider application Sub-IR 2.4: Functionality and usability of the learning resources portal improved Sub-IR 3.4: Parent and community involvement in supporting early-grade learning increased Intermediate Results (IRs) Sub￾IRs Outcome indicators Percent of grade 2 teachers, primary school teachers school heads, TEIS demonstrating knowledge and adopting materials on EGR, GESI, and SEL - 1.1 a, 1.1.b, , 1.1.c, , 1.1.d, , 1.1.e, , 1.1.f 1.2. Percent of school heads and supervisors actively providing effective instructional support and feedback to K–3 teachers according to clearly defined expectations (custom-outcome) 1.3. Percent of mathematics teachers delivering innovative math instruction according to a clearly defined set of observable practices in fidelity to the training (custom-outcome) 1.4. Percent of teachers using Tangerine:ClassTM appropriately according to a clearly defined set of observable practices in fidelity to the training (custom-outcome) 2.1. Percent of K–3 classrooms in which teachers and learners are regularly using supplemental learning materials according to a clearly defined set of observable characteristics (custom-outcome) 2.2. Percent of titles produced by ABC+ that meet GESI-appropriate standards based on a checklist of clearly observable criteria (GESI￾output) 2.3. Percent of primary-grade learners targeted for USG assistance who have the appropriate variety of decodable, leveled, and supplementary readers in the LOI (ES.1-45-output) 2.4 Content, access, and usability of LRP enhanced according to system metrics and key informant teachers and administrators (custom-outcome) 3.2a. Number of Schools Divisions incorporating timely indicators on teachers’ quality of EGR instruction in their DMEA process (custom￾outcome) 3.2b. Percent change in allocation of resources for learning materials and for teacher professional development at central, regional, and division levels 3.3. Percent change in allocation of SEF and other resources from LGUs in target regions for early-grade literacy and numeracy activities (custom-outcome) 3.4. Number of private sector partners engaged to support evidence￾based EGR interventions (custom-outcome) 3.5. Number of PTAs or community-based school governance structures supported with USG assistance (ES1-13-output) Perfor mance Indicat ors USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 192 Specific strategies to achieve the objectives include the following: ● Capacity development for K-3 teachers for effective reading instructional practices ● Strengthen capacity to develop quality reading materials for a specific language of instruction (LOI) ● Capacity development for instructional supervisors to support developmentally appropriate instructional practices that foster children’s cognitive, social, and emotional development ● Enhance management capacity to support mother tongue (MT) instruction and provision of learning resources EVALUATION QUESTIONS In this evaluation, the evaluand is ABC+. The evaluation will focus on the performance of the ABC+ project in achieving its stated objectives in relation to its three (3) intermediate results (IRs). The questions focus on the relevance, effectiveness, and sustainability of interventions of ABC +. For relevance, the set of questions will be on the alignment of the ABC+ project to the needs, policies, and priorities of the Department of Education on increasing and sustaining basic skills among the K-3 learners. Additionally, it will assess whether the evaluand is sensitive and responsive to context, quality of design, and responsiveness over time.30 For effectiveness, it will provide judgment on the success and fidelity of the project to the logic model. Lastly, for sustainability, it will assess the continuity of the project and will also look at the significance and contribution of ABC+. The evaluation will seek to identify new practices or steps toward a long-term change if any, and the effects of ABC+’s interventions on social actors such as the individual, group, community, organization, or institution. After the in-brief with USAID and initial consultations with ABC+ and its major stakeholders, the evaluation team will refine the questions in this SOW. The main question RELEVANCE 1. To what extent does the innovation introduced by ABC+ on improved early-grade instructions and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction correspond with the most recent objectives of the Department of Education of the Philippines (priorities, policies, the needs, and the practical requirements)? 2. To what extent has ABC+ contributed to improving early-grade literacy, the better transition of reading ability to Filipino and English, more solid acquisition of foundational math concepts and skills, and improved social and emotional learning for children in the early￾grades (kindergarten to grade 3) in the Philippines? 30 OECD (2021). Applying Evaluation Criteria Thoughtfully, OECD Publishing, Paris, https://doi.org/10.1787/543e84ed-en. 193 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Some possible sub-questions under the relevance questions that the evaluation team will discuss during the Initial Briefing include: ● How were ABC+’s activities relevant in reaching out to marginalized groups for incorporation into policy and intervention in the three IRs? ● How did ABC+ incorporate gender equality and social inclusion (GE and SI) analysis? To what extent has ABC+ implementation supported GE and SI? ● Did ABC+’s design and implementation consider the available capability and capacities of the recipients (human resources and organizations) and responses to any change in context (e.g., public health emergency such as COVID-19)? EFFECTIVENESS 3. To what extent did ABC+ achieve its outputs and outcomes in the three (3) intermediate results (IRs) on improved early-grade instructions and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction? Some possible sub-questions under the relevance questions that the evaluation team will discuss during the Initial Briefing include: ● Which of the three (3) IRs contributed the most to the achievement of improving early-grade reading, math, and social and emotional skills for children in the early-grades (kindergarten to grade 3)? What were the explanatory factors, if any, for achievement, overachievement, and underachievement or non-achievement? ● To what extent is the target group reached? Were results delivered for men and women, boys and girls, and persons with disability? ● Which among the following elements of instruction and learning outcomes (professional and curriculum development and stakeholders’ participation) in K–3 has the ABC+ made the greatest impact? In what ways did COVID-19 affect the performance of each of the IRs? ● Are teachers, curriculum developers, school administrators, and pre-service faculty improving their technological, pedagogical, and content knowledge (TPACK) on EGR and SEL (e.g., instructional practices and materials)? ● How have policies in EGR instruction contributed to change in enhancing instructions and learning outcomes in kindergarten to grade 3 (K–3)? ● Have there been any outcomes brought by COVID-19 in implementing ABC+ that would tell us whether the program improved early-grade instructions and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction? ● Have there been any negative or unexpected outcomes resulting from project activities, and if so, what are they, and what can we learn from these outcomes? SUSTAINABILITY USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 194 4. Will the achievements in improved early-grade reading, math, and social and emotional skills for children in the early-grades persist after the project’s conclusion? Some possible sub-questions under the relevance questions that the evaluation team will discuss during the Initial Briefing include: ● To what extent are the Department of Education and local actors (financially, personnel￾wise, and in terms of organization) capable and prepared to maintain the positive effects without support in the long term? ● To what extent was sustainability addressed at the design stage and during the project? What are the major factors (including risks) influencing sustainability? In what contexts were the efforts on sustainability applicable (CO, ROs, SD, school, and community level)? ● To what extent did the ABC+ strengthen local ownership and leadership on the enhanced instruction and learning outcomes in kindergarten to grade 3 (K–3)? ● What are the visible challenges and opportunities regarding the project’s sustainability, and how likely is their occurrence? ● To what extent are the target social actors capable of sustaining the positive effects of ABC+ without support in the long term? LEARNING QUESTION IR 1 Do ABC+ trained teachers differentiate between boys’ and girls’ unique learning needs? IR 2 Do boys and girls find the ABC+ supplemental reading materials stimulating and relevant to their age-group and gender? IR 3 How can regions/divisions/LGUs better support and reach GIDCA, remote and Last Mile schools? How do attitudes and perceptions of parents and other community stakeholders vary in relation to early-grade education and MTB-MLE in particular? EVALUATION DESIGN AND METHODOLOGY LOCATION AND GEOGRAPHIC SCOPE The evaluation will cover Regions V (Bicol), VI (Western Visayas), and a small portion of BARMM (Maguindanao Division in BARMM) in which ABC+ is being implemented. The evaluation will use the division, district, and school levels in its sampling frame to select purposively informants using ABC+ participant lists. EVALUATION DESIGN Anchored by the theory of change, the evaluation will use a theory-based, mixed-methods design to understand ABC+’s performance. The evaluation methodology will use a concurrent mixed methods evaluation, this means that QUAN and QUAL methods occur concurrently. The selection of key informant interviews will be purposive. The evaluation team will process data for any surveys when the received responses reach the minimum acceptable response rate of 50% (Babbie, 1990). 195 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ABC+’s operating context is entirely different from its initial design and first year of implementation. Community restrictions brought about by COVID-19 affected the intended inputs and activities of ABC+. The program was implemented only 5 months before COVID. As a result, outcomes other than those reflected in the results framework are possible, especially when implemented in a complex environment, such as the unprecedented contextual changes in education due to COVID and school closures. The evaluation will use Outcome Harvesting (OH) as part of the approach to uncover unexpected outcomes not envisioned in the theory of change, in addition to the QUAN and QUAL methods used to evaluate ABC+’s performance related to theorized outcomes. The OH method defines an outcome as a change in an individual, group, community, organization, or institution’s behavior, relationships, actions, activities, policies, or practices.31 It could also be an unexpected or unintended demonstrated change in awareness, knowledge, or skills, collaborative action, or the use of knowledge or innovative solutions. Outcomes might also describe unexpected or unintended deeper institutional changes relating to policy, citizen engagement, government accountability, and organizational arrangements.32 The evaluation will ask key informants and focus group discussion participants to describe what they perceived as the outcomes for each IRs and the ABC+’s contributions to the outcomes. DATA COLLECTION The evaluation may use the following data collection methods: (a) document review, particularly on the delivery of outputs and use of resources; (b) Key Informant Interviews (KII); (c) focus group discussions (FGD); (d) purposive surveys; and (e) case studies. Combining these methods will allow for greater data consistency and triangulation of information for validity. Key informant interviews and focus group discussions should be with USAID personnel, officials of the Department of Education Central Office (Cos), Regional Offices (ROs), School Divisions (SDs), and selected beneficiaries (teachers, students, parents, school boards) and partners from the private sector. The main stakeholders are composed of the national and regional offices of the following: (a) Department of Education (DepEd); (b) Local Government Unit; (c) Parent-Teacher Community (PTC); and (d) private sectors. The evaluation design will include a selection of sites for key informant interviews and a list of participants (teachers, school administrators, parents, and private partners) for the case studies for USAID/PH’s approval. The evaluation team will also collect additional information from performance monitoring reports, school records, milestone reports, quarterly progress reports, and annual reports. If the community restrictions in the three targeted regions (Bicol, Western Visayas, Maguindanao Schools Division in BARMM) make in-person data collection unfeasible, the evaluation will use remote methods. Electronic activity records, such as scanned documents, may be substituted for in￾person visual reviews of activity records. Data gathering with remote data collection may be pursued via online or smartphone surveys using Google forms, Microsoft forms, and similar computer and mobile-aided measures and using video conferences through Zoom, and Google Meet, among others. 31 Wilson-Grau, R. and Britt, H. (2012). Outcome Harvesting. Cairo: Ford Foundation. (p.1) 32 The World Bank. (2014). Outcome-based Field Learning Guide. Washington, D.C. (p.13) USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 196 The data collection methods discussed here are indicative. The evaluation team may propose other data collection methods during the inception phase. The data collection phase will begin with six weeks of fieldwork, including two weeks of reviewing documents collected onsite. Figure 7 is a visual representation of data collection, evaluators-in￾charge, and expected participants. Figure 7 - Data Collection Timeline DATA ANALYSES QUANTITATIVE The evaluation will use descriptive statistics and analytic data visualizations for the quantitative data and disaggregated data for gender, sex, and location to the extent possible. The said approach to quantitative analysis is consistent with USAID’s embedded monitoring and evaluation in the Program Cycle.33 QUALITATIVE The capta (qualitative data) will be recorded and transcribed in worksheets. It will be subjected to content analysis using computer-assisted qualitative data analysis software (CAQDAS). The evaluation team will code the responses and compute the estimated reliability index. A constant comparison method will complement the index where the team will check the coherence of responses, codes, and themes to ensure that the results will have no external threats to the 33 https://usaidlearninglab.org/monitoring-toolkit?tab=2 197 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV audience of this performance evaluation. The team also will use content analysis on the outcomes harvested. TRIANGULATION The evaluation team will combine and compare the quantitative and qualitative analysis results. Comparing the two results will provide a more complete understanding of the ABC+ project in terms of relevance, effectiveness, and sustainability. The evaluation also will use joint displays for a mixed￾methods approach. EVALUATION DESIGN MATRIX The design matrix in Figure 8 summarizes the evaluation design and methods. The evaluation team will develop a more complete version along with the evaluation design and methodology in the Inception Report. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 198 Figure 8 - Evaluation Design Matrix Evaluation Design Matrix Questions Suggested Data Sources Suggested Data Collection Methods Suggested Data Analysis Methods Relevance: To what extent does the innovation introduce by ABC+ on improved early-grade instructions and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction correspond with the most recent objectives of the Department of Education of the Philippines (priorities, policies, the needs, and the practical requirements)? Project Documents and Reports IPs: RTI International, the prime partner, The Asia Foundation, SIL Lead, and Florida State University. Key stakeholders: DepEd, School Boards, Community, Parent Teachers’ Association) Document Review Baseline data from (1) GAD Checklist for Education Projects, (2) situational analysis of the educational provision for children with disabilities, and (3) Basa Pilipinas study of beliefs and practices KIIs(CO, ROs, SDs) FGD: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators, parents, private partners Simple surveys Site visits and remote data collection approaches Qualitative and quantitative analyses Effectiveness: To what extent did ABC+ achieve its outputs and outcomes in the three (3) intermediate results (IRs) on improved early-grade instructions and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions Project Documents and Quarterly Reports Key stakeholders: DepEd, School Boards, Community, Parent Teachers’ Association KIIs(CO, ROs, SDs) FGD: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators, parents, private partners Qualitative and quantitative analyses 199 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Evaluation Design Matrix Questions Suggested Data Sources Suggested Data Collection Methods Suggested Data Analysis Methods supporting early-grade instruction? IPs: RTI International, the prime partner, The Asia Foundation, SIL Lead, and Florida State University. Simple surveys Site visits and remote data collection approaches Sustainability: Will the achievements in improved early-grade reading, math, and social and emotional skills for children in the early-grades persist after the project’s conclusion? Project Documents and Quarterly Reports Key stakeholders: DepEd, School Boards, Community, Parent Teachers’ Association IPs: RTI International, the prime partner, The Asia Foundation, SIL Lead, and Florida State University. KIIs(CO, ROs, SDs) FGD: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators, parents, private partners Workshop on outcome mapping for each IRs Simple surveys Site visits and remote data collection approaches Qualitative and quantitative analyses USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 200 DELIVERABLES AND REPORTING REQUIREMENTS PRE-FIELD PHASE IN-BRIEF MEETING WITH USAID The evaluation team will have an in-briefing with USAID/PH. The meeting will consist of introductions and a discussion of the evaluators’ understanding of the assignment, initial assumptions, evaluation questions, preliminary thoughts on the evaluation design and methodology, and work plan. INCEPTION PHASE INCEPTION REPORT WITH EVALUATION DESIGN, WORK PLAN, SCHEDULE, AND TOOLS The team will draft an evaluation design and work plan for USAID/PH. The evaluation design will include (1) a detailed evaluation design matrix (including the key questions, methods, and data sources to be used to address each question and the data analysis plan for each question); (2) draft data collection instruments both for quantitative and qualitative methods with their main features; (3) sample frame (list of potential informants/respondents and sites to be visited); (4) known limitations to the evaluation design. In addition to the design, the work plan will finalize the anticipated schedule and logistical arrangements and include a list of key stakeholders at the national and regional levels and the geographic areas the evaluation will cover. Upon receipt of USAID/PH comments and feedback, the evaluation team will revise the Inception Report and submit it to USAID/PH for final approval before beginning the evaluation research. The Inception report should be submitted within 30 business days of the mobilization of an evaluation team. FIELDWORK PHASE MID-TERM BRIEFING AND INTERIM MEETINGS After completing the fieldwork phase, the evaluators must hold a mid-term briefing with USAID/PH. The briefing should provide a progress report on the status of the evaluation and fieldwork, including potential challenges and emerging opportunities. The briefing shall be held no later than fifteen (15) days after completing primary data gathering and fieldwork. The evaluation team will also provide USAID/PH with periodic feedback on the progress of the evaluation, as agreed upon during the first briefing. If desired or necessary, the evaluation team will brief USAID weekly. The Out Briefing should be conducted within 15 business days of fieldwork and data analysis completion. The Out Briefing will include the evaluation’s emerging findings and conclusions based on the field research and the preliminary data analysis. ANALYSIS PHASE DRAFT EVALUATION REPORT AND USAID OUT-BRIEFING The draft evaluation report will be consistent with the guidance provided in the section on the Final Report Format. The report will address each of the questions identified in the SOW and any other issues that have a bearing on the objectives of the evaluation. Once the initial draft evaluation 201 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV report is submitted, the Offices of Program Resources Management and Education will have ten business days to review and consolidate comments on the initial draft and provide the comments to the principal investigator. After receiving comments, the evaluation team will have ten business days to submit a revised, final (second) draft report. USAID/PH will have ten business days to review and reply with comments on the final (second) draft report. The Draft final report should be submitted within 30 business days of completing the Out Briefing. LEARNING AND DISSEMINATION PHASE LEARNING EVENT(S) AND PRODUCTS FOR EVALUATION UTILIZATION CLAimDev will organize at least two learning events for key stakeholders within three months of USAID’s acceptance of the final report. The learning events will disseminate evaluation findings, explore good practices and lessons learned related to key ABC+, and highlight ways to enhance the sustainability of results. At least one learning event’s target audience will be USAID/Philippines, and one learning event’s target audience will be USAID’s external stakeholders. Based on written technical directions Task Order Contracting Officer’s Representative (TOCOR) and the availability of funds, the Contractor will organize and conduct additional learning events. CLAimDev will design and produce learning products to increase evaluation utilization within three months of USAID acceptance of the final report. CLAimDev will finalize the number and type of learning products based on the evaluation’s results and the availability of funds in consultation with the TOCOR. LEARNING EVENT SYNTHESIS This document synthesizes the discussions from the ABC+ Performance Evaluation Learning Event with various stakeholders. It will capture the salient points from the plenary and breakout discussions. This document is intended to be a reference for stakeholders working in the field of basic education and for future USAID programming activities. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 202 EVALUATION TEAM COMPOSITION STAFFING Three CLAimDev staff members will oversee and manage the evaluation. 1. Chief of Party (CLAimDev) 2. Principal Investigator (CLAimDev) 3. Evaluation Specialist (CLAimDev) The external evaluation team will conduct the evaluation. 1. Team Leader Evaluation Advisor (STTA) 2. Professional Development Specialist (STTA) 3. Educational Administration Specialist (STTA) 4. Region V Field Regional Evaluation Assistant (STTA) 5. Region VI Field Evaluation Assistant (STTA) 6. BARMM Field Evaluation Assistant (STTA) 7. Data Analyst (STTA) 8. Evaluation Assistant (STTA) EVALUATION ADVISOR (TEAM LEADER) ● Evaluation professional with at least ten (10) years of experience conducting evaluation studies, preferably in the education sector and as a Team Leader or Team Member on a USAID activity. ● A Master’s degree in education or related discipline is required. ● Ability to write message-driven evaluation reports. ● Demonstrated ability to lead and manage evaluation or research teams. ● Willingness and ability to work as part of a team. ● Excellent English communication skills, both written and oral. Evaluation reports drafted by candidates may be requested. TECHNICAL SPECIALIST: PROFESSIONAL DEVELOPMENT SPECIALIST ● Professionals with expertise in professional development (e.g.) teacher training, curriculum development, and research on instructional delivery in early-grades (K-3) with expertise in: 203 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV o mathematics o socioemotional learning o mother tongue reading materials, and/or o gender equality and social inclusion (GESI) ● Experience conducting evaluations of development assistance and programs/projects is strongly preferred. ● At least seven years of experience in a relevant discipline is required. ● A Master’s degree in education or related disciplines is required. ● Willingness and ability to work together as part of a team. ● Demonstrated knowledge of monitoring and evaluation systems. ● Excellent English communication skills, both written and oral. Sample studies or published work may be requested from candidates ● Experience in qualitative data management is an advantage TECHNICAL SPECIALIST: EDUCATIONAL ADMINISTRATION SPECIALIST ● Professionals with expertise in educational administration, preferably in policy development with expertise/experience in: o local government involvement, o private sector engagement, or o parent and community involvement ● Experience conducting evaluations of development assistance and programs/projects is strongly preferred. ● At least seven years of experience in a relevant discipline is required. ● A Master’s degree in education or related disciplines is required. ● Willingness and ability to work together as part of a team. ● Demonstrated knowledge of monitoring and evaluation systems. ● Excellent English communication skills, both written and oral. Sample studies or published work may be requested from candidates ● Experience in qualitative data management is an advantage FIELD REGIONAL ASSISTANTS (UP TO 3 MEMBERS) USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 204 ● Master’s degree in the social sciences or related disciplines is required. ● Familiarity with USAID and international development programming is required. ● Evaluation and K-3 professionals with at least five (5) years of experience are preferred. ● Willingness and ability to work together as part of a team ● Excellent English communication skills, both written and oral ● Sample studies or published work may be requested from candidates ● Basic knowledge in qualitative data analysis is an advantage DATA ANALYST ● Professional with expertise in qualitative and quantitative data collection, processing, analysis, interpretation, and presentation/visualization. ● At least 3-5 years of experience in a relevant discipline is preferred. ● A Master’s degree in social sciences or related disciplines is preferred. ● Functional knowledge of relevant data processing, analysis, and presentation software is a must (e.g., SPSS, NVivo, Tableau). ● Willingness and ability to work together as part of a team. ● Demonstrated knowledge of data management and analysis. ● Excellent English communication and data presentation skills, both written and oral. PROJECT ASSISTANT ● Experience providing support services, preferably in evaluation engagements, including formatting interview guides and survey questionnaires using remote data collection tools. ● Experience in data processing (quantitative and qualitative data) as well as logistic support in scheduling and arranging consultation meetings, travel arrangements, and venues for learning events. 205 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV LEVEL OF EFFORT (LOE) ESTIMATE FOR THE EXTERNAL EVALUATION TEAM The LOE estimate for the evaluation is 672 person-days, as detailed in Figure 9. Figure 9 - External Evaluation Team Level of Effort External Evaluation Team Level of Effort by Key Task Key Tasks Team Leader Professional Developmen t Specialist Education Administratio n Specialist Field Research Assistant s (3) Data Analyst Project Assistan t Total Pre-field Phase 9 9 9 0 9 9 45 Inception Phase 22 22 22 0 22 22 110 Field Data Gathering and Analysis 45 45 45 135 45 45 360 Draft Report Preparation 10 10 10 30 10 10 80 Final Report Writing 7 7 7 0 7 7 35 Learning and Dissemination 14 14 14 0 0 0 42 Sub-total 107 107 107 165 93 93 672 PERIOD OF PERFORMANCE AND ESTIMATED EVALUATION SCHEDULE The performance period for this evaluation will cover an approximate nine-month period from the STTA team’s deployment through the learning and dissemination events. The conduct of the evaluation through the submission of the final report will cover an approximate six to seven-month period. Based on USAID availability and schedules, the learning and dissemination events will be within three months of the final report submission. Figure 10 details the evaluation, learning, and dissemination tasks and schedule. Figure 10 - Estimated Evaluation Schedule Task Number Task Name Estimated Duration Estimated Start Estimated Finish USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 206 Task Number Task Name Estimated Duration Estimated Start Estimated Finish Pre-field Phase 1 USAID issues STRIDE on-demand task order 0 days 1-Jul-22 2-Jul-22 2 Issue STTA contract to Team Leader (TL) 0 days 13-Jul-22 15-Jul-22 2a Team Leader Updates SOW and Schedule 3 days 18-Ju1 20-Jul 2b Issue STTA contracts to 2 Specialists 0 days 18-Jul 22 July 3 Document review 5 days 25-Jul-22 29-Jul-22 4 Evaluation team planning session 3 days 1-Aug-22 3-Aug-22 5 In-brief meeting with USAID 0 days 5-Aug-22 5-Aug-22 Inception Phase 6 Initial consultations with IP and major stakeholders 2 days 8-Aug-22 9-Aug-22 7 Document review and analysis of reports and studies from implementing partners and clients 10 days 10-Aug-22 20-Aug-22 Prepare inception report with evaluation design, methodology, tools, and schedule 8 days 22-Aug-22 30-Aug-22 8 Submit inception report to COP for review 2 days 31-Aug-22 1-Sep-22 9 Submit inception report to USAID for approval 0 days 5-Sep-22 5-Sep-22 10 Revise and submit inception report 2 days 6-Sep-22 7-Sep-22 Data Gathering and Analysis 11 Pilot tools and finalize 7 days 8-Sept-22 15-Sep-22 12 Schedule data collection interviews and arrange logistics 2 days 16-Sep-22 17-Sep-22 13 Data collection 19 days 19-Sep-22 10-Oct-22 13a Presentation on Preliminary Impression 1 day 11-Oct-22 11-Oct-22 14 Data processing and analysis 12 days 12-Oct-22 25-Oct-22 15 USAID Out-brief 1 day 26-Oct-22 26-Oct-22 16 Consultation meeting with IPs 1 day 28-Oct-22 28-Oct-22 Draft Report Writing Phase 17 Draft report preparation 10 days 29-Oct-22 9-Nov-22 18 Submit draft report for PI and COP review 0 days 10-Nov-22 11-Nov-22 19 Comments on draft report from PI and COP 5 days 14-Nov-22 18-Nov-22 Final Report Writing Phase 207 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Task Number Task Name Estimated Duration Estimated Start Estimated Finish 20 Final report revisions 7 days 21-Nov-22 28-Nov-22 21 Submit final report for PI and COP review 0 days 29-Nov-22 29-Nov-22 22 Submit final report for Home Office review 0 days 1-Dec-22 1-Dec-22 23 Submit final report draft to USAID 0 days 8-Dec-22 8-Dec-22 24 USAID Comments on final report 5 days 9-Dec-22 14-Dec-22 25 Final report revisions based on USAID comments 7 days 2-Jan-23 10-Jan-23 26 COP and Home Office report finalization 5 days 11-Jan-23 17-Jan-23 27 Final report submission 0 days 17-Jan-23 17-Jan-23 28 Final report approval 5 days 18-Jan-23 24-Jan-23 29 Final report submitted to the DEC; data uploaded to DDL 0 days 24-Jan-23 24-Jan-23 Learning and Dissemination Phase February – April 2023 30 Learning Event 1: Findings, conclusions, and recommendations with one (1) learning event’s target audience will be USAID/Philippines (#1) 5 days TBD TBD 31 Learning Event 2: Dissemination of Findings and Lessons Learned with another learning event’s target audience will be USAID’s external stakeholders (#2) 3 days TBD TBD 32 Submit supplementary report 6 days TBD TBD FINAL REPORT FORMAT The evaluation findings, conclusions, and recommendations will be consolidated into a message￾oriented final report. The final evaluation report should include an executive summary; a background of the local context and the activity; the evaluation questions; the methodology or methodologies; limitations to the evaluation; findings, conclusions, and recommendations; and good practices and lessons learned, as applicable. The report should be formatted according to USAID’s evaluation report template, with estimated page counts, as listed below. 1. Abstract (1/2 page) 2. Executive summary (3 - 5 pages) 3. Evaluation Purpose (1/2 page) 4. Background on the context and the strategies/projects/activities being evaluated (1 page) USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 208 5. Evaluation Questions (1/2 page) 6. Methodology (1 page) 7. Limitations to the evaluation (1/2 page) 8. Findings, conclusions, and (if applicable) recommendations (30 - 32 pages) 9. Annexes The report should not exceed 40 pages, including the abstract and executive summary. The executive summary should be no more than 3–5 pages long and summarize the purpose, the background of the activity being evaluated, main evaluation questions, methods, findings, conclusions and recommendations, and lessons learned (if applicable). The report will explain the evaluation methodology, with details included in an Annex. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (e.g., selection bias, recall bias, among others) The annexes to the report shall include the following: (a) the evaluation SOW; (b) the evaluation inception report; (c) statements of difference, if any, regarding significant unresolved differences of opinion by funders, implementers, or members of the evaluation team; (d) all tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides; (e) sources of information, properly identified and listed; and (f) disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of conflicts of interest or describing existing conflicts of interest. The Principal Investigator will ensure that the final evaluation report is publicly available through the USAID Development Experience Clearinghouse within 90 calendar days of the official completion date of the evaluation contract. EVALUATION REPORT QUALITY CRITERIA Per USAID ADS 201.3.5.17, draft evaluation reports must undergo a peer review organized by the office managing the evaluation. The following criteria will serve as the basis against which the report is reviewed: ● Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity. ● Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. ● The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report. ● Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently be revised and documented in consultation and agreement with USAID. ● Evaluation methodology should be explained in detail and sources of information properly 209 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV identified. ● Limitations to the evaluation should be adequately disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). ● Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. ● Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. ● If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. ● If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific. DATA REQUIREMENTS All quantitative data collected by the evaluation team will be provided in an electronic file in an easily readable format. The data will be organized and fully documented for use by those not fully familiar with the activity or the evaluation. USAID will retain ownership of all datasets developed. All datasets created or collected will be submitted to the DDL in accordance with the terms and conditions of the Task Order for this evaluation. This is in keeping with Executive Order 13642 and the OMB Open Data Policy (M-13- 13), which states that an agency’s “public data listing may also include, to the extent permitted by law and existing terms and conditions, datasets that were produced through agency-funded grants, contracts, and cooperative agreements.” USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 210 ANNEX 1. AVAILABLE DATA Figure 11 - Available Data Type Timeframe Quality Reliability Reference Baseline survey February 22-24, 202o Includes baseline values for ABC+ that were used to establish key performance indicators (KPIs) and relevant context indicators Collected by the MEL team Baseline report June 2020 Accomplishments In each IRs July 2019- September 2020 Includes accomplishments for each IRs for Year 1 of the program (PY1) Collected by the MEL team Year 1-Annual Progress Report: July 2019- September 2020 Accomplishments In each IRs October 2020- September 2021 Includes accomplishments for each IRs for Year 2 of the program (PY2) Collected by the MEL team Year 2-Annual Progress Report: October 2020- September 2021 Accomplishments In each IRs May 2022 Includes accomplishments for each IRs for Year 3 of the program (PY3)-Q2 Collected by the MEL team Quarterly Progress Report￾Quarter 2 FY_2022 as of May 6 211 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 2: EVALUATION DESIGN MATRIX Figure 12 - Evaluation Design Matrix Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) 1. Relevance: To what extent does the innovation introduce by ABC+ on improved early￾grade instructions and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction correspond with the most recent objectives of the Department of Education of the Philippines (priorities, policies, the needs, and the practical requirements)? To what extent has ABC+ contributed to improving early-grade literacy, the better transition of reading ability to Filipino and English, more solid acquisition of foundational math concepts and skills, and improved social and emotional learning for children in the early￾grades (kindergarten to grade 3) in the Philippines? Qual Project Documents and Reports IPs: RTI International, the prime partner, The Asia Foundation, SIL Lead, and Florida State University. Key stakeholders: DepEd, School Boards, Community, Parent Teachers’ Association) Document Review KIIs(CO, ROs, SDs) Note: Develop survey instruments on educational innovation. Use the identified innovation pillars by OECD 1.2. How relevant were the activities conducted by ABC+ in reaching out to the marginalized groups to be incorporated into policy and intervention in the Qual Project Documents and Reports Document Review FGD: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 212 Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) three IRs? 1.3. In what ways was the intervention based on sound gender equality and social inclusion (GE and SI) analysis? To what extent has ABC+ been implemented to promote GE and SI? Project Documents and Reports IPs: The Asia Foundation and SIL LEAD Document Review Did the design and implementation of the ABC+ consider the available capability and capacities of the recipients (human resources, organizations) as well as the responses to any change in context (e.g., public health emergency such as COVID-19) in the course Qual Project Documents and Reports IPs: RTI International, the prime partner, The Asia Foundation, SIL Lead, and Florida State University. Key stakeholders: DepEd, School Boards Document Review Baseline data34 from (1) GAD Checklist for Education Projects, (2) situational analysis of the educational provision for children with disabilities, and (3) Basa Pilipinas study of beliefs and practices 34 ABC+ Gender and Inclusion Plan 2019 213 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) of its lifespan? KIIs (CO, ROs, SDs) Effectiveness: To what extent did ABC+ achieve its outputs and outcomes in the three (3) intermediate results (IRs) on improved early-grade instructions and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction? 2.1. Which of the three (3) IRs contributed the most to the achievement of improving early-grade reading, math, and social and emotional skills for children in the early￾grades (kindergarten to grade 3) Qual Project Documents and Quarterly Reports Assessment tools Key stakeholders: DepEd, School Boards, Community, Parent Teachers’ Association) Document Review KIIs(CO, ROs, SDs) FGD: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators 2.2. To what extent is the target group reached? Were results delivered for men and women, boys and girls, and persons with disability? Quan-Qual Project Documents and Reports Document Review WEBSITE ANALYTICS (ABC+ Tangerine) Frequency and duration of webpage visits, click￾through patterns, and sequence of pages viewed. Survey items or questions: How many schools adopted policies in EGR, GESI, and SEL? 2.3. Which among the following elements of instruction and learning outcomes (professional Qual Key stakeholders: DepEd, School Boards, Community, Parent Teachers’ Association) KIIs (CO, ROs, SDs) FGD: trainers, trainees, curriculum developers (storytellers, illustrators), USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 214 Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) and curriculum development and stakeholders’ participation) in K–3 has the ABC+ made the greatest impact? In what ways did COVID-19 affect the performance at each IRs? school administrators 2.4. Are teachers, school administrators, and pre￾service faculty improving their technological, pedagogical, and content knowledge (TPACK) on EGR and SEL (e.g., instructional practices and materials)? Quan-Qual Project Documents and Reports IPs: RTI International, the prime partner, The Asia Foundation, SIL Lead, and Florida State University. Key stakeholders: DepEd, School Boards Document Review KIIs (CO, ROs, SDs) FGD questions What skills and knowledge did participants acquire (trainees, parents, learners)? To what extent have skills and knowledge been applied in instructions (as observed by school heads and supervisors) and curriculum development storytellers and illustrators)? 215 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) 2.5 How have policies in EGR instruction contributed to change in enhancing instructions and learning outcomes in kindergarten to grade 3 (K–3)? Quan-Qual Project Documents and Reports IPs: RTI International, the prime partner, The Asia Foundation, SIL Lead, and Florida State University. Key stakeholders: DepEd, School Boards 2.6. Have there been any outcomes brought by COVID-19 in the implementation of ABC+ that would tell us whether the program improved early-grade instructions and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting Qual Document review Workshop on Outcome harvesting with social actors USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 216 Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) early-grade instruction? 2.7. Have there been any negative or unexpected outcomes as a result of project activities, and if so, what are they, and how can we learn from these outcomes? Qual Document review Workshop on Outcome harvesting with social actors Sustainability: Will the achievements in improved early-grade reading, math, and social and emotional skills for children in the early￾grades persist after the conclusion of the project? To what extent are the Department of Education and local actors (financially, personnel￾wise, and in terms of organization) capable and prepared to maintain the positive effects without support in the long term? Organization records Policies, personnel records, and student records. Key stakeholders: DepEd, School Boards Document Review KIIs(CO, ROs, SDs) FGDs: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators Site visit to check institutional capacity 3.2.To what extent was sustainability addressed at the design stage and during the project? What Policy documents Sustainability Plan, PY! Document Review KIIs (CO, ROs, SDs, School KII/FGD questions Were there new policies crafted, approved, and 217 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) are the major factors (including risks) influencing sustainability? In what contexts were the efforts on sustainability applicable (CO, ROs, SD, school, and community level)? report, Table 19, p.74 Key stakeholders: DepEd, School Boards heads) Site visit to check institutional capacity (infrastructure at community level, if any) implemented by COs, ROs, SDs to support the training of teachers and the development of materials and SEL? How long is the access to the LRP after the project ceases? 3.3 To what extent did the ABC+ strengthen local ownership and leadership on the enhance instruction and learning outcomes in kindergarten to grade 3 (K–3)? Qual Document Review FGD (School boards, School heads, parents, private sector) FGD question Are there any initiatives that pertain to scaling up and out beyond the project life? 3.4 What are the visible challenges and opportunities regarding the sustainability of the project, and how likely is their occurrence? What were the strengths of the professional development, curriculum development, and policies? What were the weaknesses, and how can this be improved? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 218 Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) 3.5 To what extent are the target groups capable of sustaining the positive effects of ABC+ without support in the long term? KIIs(CO, ROs, SDs) FGDs: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators FGD questions: -What were the lessons learned that may support the sustainability of the project in the context of the Department of Education and local actors? How is accuracy in the content of MT, Math, GESI, and SEL assessed? Were the classroom observation tools used aligned with the training design on early-grade learning? Are home literacy materials easy to follow? 219 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 18. EVALUATION DESIGN AND METHODOLOGY DATA COLLECTION METHODS In this evaluation, mixed methods were used to analyze the implementation of the ABC+ interventions. The quantitative method used an online survey, and qualitative methods included literature reviews, key informant interviews (KII), focus group discussions (FGD), Outcomes Harvesting (OH) workshops, and Most Significant Change Technique (MSCT). Using multiple data sources through mixed methods enabled the evaluation to cover a wide spectrum, conduct a more in-depth analysis, and achieve a more holistic understanding of data through triangulation (Yin, 2003). Some of these methods were conducted virtually, while others were conducted in person. The sequence of data collection was as follows: desk review, survey, KIIs, FGDs, OH workshops, MSCT, and evaluation of supplementary materials. DESK REVIEW The desk review of documents started early in data collection, and this involved reading reports, policies, plans, etc., to gather data to answer the evaluation questions. A template matrix to organize reviewed data was used to gather evidence from reading documents. Desk review data helped answer the evaluation questions across all IRs and informed the crafting of questions for KIIs, FGDs, and workshops for OH and MSC. See Annex 14 for the types of documents reviewed. SURVEY Simple surveys were administered online. Participants were teacher trainees, supervisors, school heads, LGU, and private sector focal persons. The survey tool gathered data related to the evaluation questions on the project's relevance, effectiveness, and sustainability. It included questions to measure project relevance, satisfaction, and effectiveness to recipients. There were also questions to measure the relevance of ABC+’s intervention to existing policies at the central office, regional, and division levels of DepEd. KEY INFORMANT INTERVIEWS KIIs were conducted via phone and virtual conferencing platforms. Featured questions were informed by desk review data and the project context presented by IP during the initial consultation of the evaluation team. Questions were emailed to informants should they be unavailable for calls or online conferencing. FOCUS GROUP DISCUSSION A total of ten FGDs were conducted online and on-site by gathering seven to eleven informants representing different stakeholder groups. Participants were grouped according to the topics they would discuss. One group comprised representatives of 10 cities (DepEd heads, trainers for instruction and materials development, community members (LGU representatives, etc.), eight (8) with school heads and teachers (i.e., 7 languages + Maguinanawn), and one FGD with BSU and WVSU. The FGDs talked mainly about the program’s interventions and their perceived outcomes (see Annex 21). OUTCOME HARVESTING WORKSHOP AND MOST SIGNIFICANT CHANGE WORKSHOPS USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 220 Outcome Harvesting Workshop (OHW) and Most Significant Change Technique (MSC) were conducted to complement each other. OH prompts were informed first by the desk review data, which was used for crafting workshop prompts, which key program organizers validated. Finalized versions were used in a workshop. Using meta cards, participants (social actors) representing different groups of intervention recipients were gathered in a virtual and in-person workshop to respond to the prompts and discuss their answers. Collected outcomes and outcome descriptions were verified and categorized (domains of change) by program implementers (change agents). Refer to Annex 16 for the verified outcomes. The outcomes harvested and the activities that lead to them are appropriate springboards for the MSC stories of the participants. OH is good at identifying intended and unintended outcomes and gives a group perspective to outcomes or changes, while MSC presents a change through the perspective of an individual. While OH outcomes tend to be general, MSC data can be rich and vivid. Picking up from the identified outcomes and domains of change in virtual and face-to-face OH workshops, participants in the MSC Level 1 workshop (i.e., teachers, school heads, community/Barangay leaders, and parents, etc.) were asked to write down their most significant change stories in relation to their experience with ABC+ activities or outputs. A template was used to help participants recall changes they experienced or witnessed in relation to ABC+ activities (see Annex 12). The template guided participants in providing details and structure to their stories. Participants were given a chance to write more than one story. Once stories were written, the group read their stories and chose the most significant story based on the prompt given to them. Annex 17 presents the 66 Level 1 stories from the regions and the selected 16 stories from the set. The set of selected stories from Level 1 was passed on to Level 2 workshop participants (i.e., DepEd and ABC+ trainers, DepEd school heads and supervisors, and focal persons) who met online to verify the content of selected stories. After deliberating, they chose the six most significant stories from the set. Please see Annex 18 for Level 2 MSC Stories and the reasons for selecting them. These were passed on to Level 3 story selectors (i.e., Key leaders of DepEd and ABC+) who selected one most significant change story. In selecting MSC stories, Level 2 and 3 selectors were given a set of criteria they may use (see Annex 19). Emerging themes from the stories and their differences or similarities were generated to complement OH results and better understand the effects of ABC+ implementation. EVALUATION OF SUPPLEMENTARY READING MATERIALS (SRM) Initially, developed SRMs were supposed to undergo evaluation; however, upon learning from data collected at the outset that the SRMs had already undergone a long and stringent quality assurance as part of their development, it was decided to forego this data collection activity. CLASS OBSERVATION Originally, the plan was to conduct class observations to gather data on instruction and materials used. However, at the time of data collection, DepEd’s moratorium for class observations was enforced, and schools were strongly advised to focus on learning recovery instruction and forgo other activities (refer to Annex 20, a sample of localized suspension of observations in Region V). At the same time, the pandemic was still ongoing, and class observations would have to be communicated in advance, with health protocols followed. These requisites could not be accommodated at the time of data collection. 221 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV SITE SELECTION CRITERIA The main stakeholders were composed of the following: (a) DepEd central, regional, and local offices; (b) Local Government Units; (c) Parent Community Engagement (PCE); and (d) private sectors. The analysis was at three levels: individual, institution, and organization. The evaluation design included a selection of sites for KIIs and FGDs, and a list of participants (education leaders, teachers, school heads, parents, LGU, NGOs, and private partners) for the case studies for USAID/PH’s approval. Specifically, the selection criteria were on two levels, geographic location with the associated linguistic focus and participant profile. For the first level of selection criteria for the regions, Bicol, Western Visayas, and Maguindanao Schools Division in BARMM, who were part of the project, were automatically selected. The next filter looked into the school divisions in the three regions that received the assistance of ABC+, specifically schools’ proximity to RO (ARPY2, p.9.). A sampling frame of schools from the implementing partner guided the evaluation team for its specific selection criteria at the level of respondents, making sure the geographic locations and the seven languages used in the project were well represented. However, for KII, a census of high-level positions involved in early-grade literacy was interviewed (i.e., DepEd Central, Directors of Regions V, VI, and BARMM, chief educational supervisors, and educational supervisors in charge of early-grade literacy). Table 3 shows the summary of data collection methods and corresponding participants. KII PARTICIPANTS AND COMPOSITION CRITERIA KII requires in-depth information from informants who were selected based on their key positions (if from DepEd and ABC+) and intensive involvement in ABC+ interventions. See Annex 21 for the list of KII participants and their backgrounds. FGD PARTICIPANTS AND COMPOSITION CRITERIA FGD's success depends largely on the varied backgrounds of informants and the perspectives they contribute to the discussion. Their response to one another’s answers also matters. To ensure that quality FGD is observed, the number of participants was limited from 6-12. A maximum of two representatives per stakeholder group were invited to participate. The FGD participants and their backgrounds are found in Annex 21. Table 3. Summary of Data Collection Methods and Corresponding Participants Method Description Document Review Annual Reports, DepEd documents Key Informant Interviews (KII) Central Office, ROs, and DOs Focus Group Discussions (FGD) Seven (7) language groups composed of teachers, school heads LGU, and private sector focal persons Simple surveys Teacher trainees, supervisors, school heads, LGU, and private sector focal persons USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 222 Case studies LGUs/Schools (proximity to RO) Outcome Harvesting Workshop Teachers, school heads, supervisors, parents, LGU officials SURVEY Stratified random sampling was employed in selecting the respondents for the online surveys. Cochran’s formula with a 95% confidence interval and a 3% margin of error was used in determining the sample size. Annex 3 presents the sampling procedures and template table showing the distribution of all participating schools, teachers, and desired sample size by geographical area. Sample selection was guided by proportional sampling to ensure that schools from Region V, Region VI, and selected schools in BARMM are fairly represented. OH PARTICIPANT COMPOSITION AND SELECTION CRITERIA OH participants were those closest to program implementation, such as field implementers (e.g., focal persons, teachers, trainers, writers and illustrators, QA evaluators, etc.). A few members who were knowledgeable about the program but detached from the intervention were also invited (e.g., program designers and key leaders). MSC PARTICIPANT COMPOSITION AND SELECTION CRITERIA These were informants that represented different groups in program implementation. The following are the project-level groups and the backgrounds of their members. MSC workshops, virtual and face-to-face, were held for each ABC+ beneficiary region. Level 1: Intervention recipients and trainers: trained teachers and their school heads, parents of children who received SRMs, and community members representing a Division where OH was conducted. Level 2: Intervention implementers from DepEd, ABC+, and partners: focal persons, regional or division trainers (different from the ones in Level 1), writers and illustrators, materials QA evaluators, etc., representing a Division where OH was conducted. Level 3: Program Leadership from DepEd and ABC+: program designers, policy, and decision￾makers, Central, Regional, or Division DepEd leaders, etc. The evaluation team also collected additional information from school records, school improvement plans, learning recovery program plans, etc. Documents were sourced by taking photographs of documents with the permission of the school head. Electronic files were requested and sent to the team by email when possible. The team also used Google forms and similar computer and mobile-aided measures, such as using a mobile phone and video conferences through Zoom and Google Meet. The data collection phase began with four weeks of fieldwork, including two weeks of reviewing documents collected onsite. A visual representation of data collection and expected participants can be found in Annex 4. DATA ANALYSES 223 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV QUANTITATIVE The evaluation used descriptive statistics and analytic data visualizations for the quantitative data and disaggregated data for sex and location to the extent possible. The said approach to quantitative analysis was consistent with USAID’s embedded monitoring and evaluation in the Program Cycle. Data from the survey were analyzed through descriptive statistics. Analysis of progress of ABC+ toward indicators from the quarterly/annual reports. However, the absence of needed data during the scheduled time for analyses precluded the comparison of division and school-level data at the start and midline and the determination of the effect size of interventions by cohort, divisions, sex, and location. Classroom observation was not also done due to the COVID-19 pandemic restrictions imposed by DepEd. Thus, employing descriptive statistics on classroom observation data collected during fieldwork was not included in the data analyses of this evaluation report. Other survey items are found in Annex 22. QUALITATIVE The qualitative data were recorded and transcribed in worksheets. It was subjected to content analysis using computer-assisted qualitative data analysis software (CAQDAS). The evaluation team coded the responses and computed the estimated reliability index. A constant comparison method complemented the index where the team checked the coherence of responses, codes, and themes to ensure that the results had no external threats to the audience of this performance evaluation. The team also used content analysis on the outcomes harvested. Qualitative data analysis was used for FGD, KII, MSC, and OH. Expected information was about themes gathered from the discussion to analyze the evaluation criteria indicators. In both quantitative and qualitative analyses, computer software such as the SPSS and NVivo was used for data processing and analysis. The codes used and reliability results are found in Annex 23. TRIANGULATION The evaluation team combined and compared the quantitative and qualitative analysis results. Comparing the two results provided a complete understanding of the ABC+ project regarding relevance, effectiveness, and sustainability. EVALUATION DESIGN MATRIX The design matrix in Table 4 summarizes the evaluation design and methods. KNOWN LIMITATIONS TO THE EVALUATION DESIGN This performance evaluation has five potential limitations: reduced sample size, inaccessible sites, participant bias, ability to recall information for self-reported data, and limited access to documents. First, on sample size - some respondents of the KII, and FGD were not able to participate due to prior commitments, health reasons/COVID-19, weak internet connectivity, and power outages that resulted in a reduced number of actual participants compared with the number of invited ones. Second, some study sites were not accessible due to the distance from the ROs/DOs. For instance, areas visited in BARMM were limited to those in the city proper, Cotabato. Third, in the self￾reported data, the survey, KII, and FGD relied on what respondents said. Responses depended on what people could remember because these activities or events occurred in the past. Lastly, the access to documents where some project contact persons were probably no longer connected with USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 224 their schools or transferred to other schools or offices (principal and other office staff assigned to another school) and needed more time to look for documents, which resulted in delays or limited access to documents. DATA MANAGEMENT AND ETHICAL CONSIDERATIONS The evaluation team ensured that ethical conventions in program evaluations were adhered to. The team secured informed consent from all participants in FGD and KII and all survey respondents as part of the data-gathering process. The evaluation team prepared the Participant Informed Consent Forms customized for various types of respondents. Moreover, the team observed cultural sensitivity and DNH principles in the entire spectrum of the evaluation process. The participants received information during the informed consent process about the: (1) purpose of the evaluation, (2) the extent of their involvement, (3) their right to refuse or discontinue participation, (4) the potential benefits of the evaluation, and (5) confidentiality of their responses. All interview data is stored on secure computers in Panagora’s Manila office. The Panagora Home Office ensured that personally identifiable data were not included in any public reports, including the final reports submitted to the USAID DEC. As per the nature of this performance evaluation, interview data sets will not be submitted to USAID or any third party. The following sections present the gathered data, analysis, and conclusions for ABC+Relevance, Effectiveness, and Sustainability. Results are organized according to the topline evaluation questions which are followed by data for the learning questions. ANNEX 19. LEARNING QUESTIONS Do ABC+-trained teachers differentiate between boys’ and girls’ unique learning needs? (IR 1- Learning question)? Teachers were asked if they differentiate instruction for boys and girls as they plan for their activities. As shown in Table 6, overall, teachers reported that they do not plan specific activities intended exclusively for either sex, as reported by the majority of the respondents highlighted in red (65.3% for boys, 62.2% for girls). Table 6. Percentage distribution of teachers planning for sex-specific activities For Girls For Boys Teachers planning sex-specific activities 37.4 % 34.4 % Teachers not planning sex-specific activities 62.2 % 65.3 % No response 0.3% 0.3% Total 100% 100% Note: Numbers are derived from the survey questionnaire administered online. Highlighted in blue are the percentage of teachers who reported that they plan specific activities for boys or girls. Fewer (respondents) (37.4%) reported that they consider assigning activities for girls when they develop their plan for early-grade instruction. Teachers try to incorporate activities in the lessons that are intended for girls such as newspaper butterfly crafts, art activities, drawing, 225 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV coloring, cutting and pasting, painting, and making 3D dolls, so girls can express their thoughts and imagination and at the same time develop their fine motor skills. A respondent also reported that girls should be trained in doing household chores. It was also reported that they try to integrate GESI and SEL in their instruction for girls. Other notable activities that the respondents expressed are role-playing, dancing, and singing. In addition, it was also expressed that girls are more performers in literacy and that they are more interested and excited about reading activities. For boys, respondents (34.4%) reported using play-based and game-based learning like a scavenger hunt for more physical activity to encourage more participation among them. Group and collaborative activities that promote and integrate GESI and SEL were also reported by the respondents to address the needs of boys and girls and at the same time improve their socialization skills. In addition, the respondents also expressed that boys are mostly in need of learning intervention than girls thus they must be given separate learning practices as well. It can be gleaned from the narratives expressed by the respondents (37.4% for Girls; 34.4% for Boys), there seem to be stereotyped notions of how boys and girls learn differently. Stereotypical beliefs that boys are naturally more active than girls and that girls are for “girls’ activity” only and other gendered expectations of what boys and girls can only do and or are allowed to do. There is a need to have sound teacher training on gender awareness focusing on how gender dynamics in the classroom impact learning especially since face-to-face classes are now being implemented. ABC+ program may build on their existing GESI training and dig deeper into gender awareness training focusing on the teachers’ targeted knowledge and skills in promoting and teaching literacy across gender in the classroom setting. The result further suggests to also include differentiated instruction in teacher training to meet the individual needs of all learners across gender. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 226 ANNEX 20. VALIDATION WORKSHOP AND SURVEY I. Validation Workshop of Evaluation Findings, Conclusions, and Recommendations on USAID’s Advancing Basic Education in the Philippines (ABC+) February 24 (Friday), 2023 | 9:00 a.m. - 11:00 a.m. | Online: Zoom A. Background Rationale and objectives USAID/PH commissioned CLAimDev to conduct a third-party mid-term performance evaluation of Advancing Basic Education in the Philippines (ABC+). Valuing the inputs of the activity’s primary beneficiaries, the ABC+ Evaluation Team will share their findings with the Philippines’ Department of Education (DepEd). Validation of evaluation results is an important component of providing useful information to various stakeholders in ABC+. This will ensure that the entire ecosystem and all actors in the early-grade reading instruction are informed of the results of the evaluation. Furthermore, it provides an avenue for verifying the accuracy of the findings of the evaluation by facilitating discussion on the possible reasons for the said findings. This activity is intended to be conducted with evaluation participants of the ABC+ program. Specifically, the validation workshop aims to: ● Present the major findings of the Performance Evaluation of ABC+ ● Collect feedback from the participants on the accuracy of the information gathered by external evaluators Methodology The validation workshop was conducted on February 24 (Friday), 2023 at 9:00 a.m. to 11:00 a.m. through online (via Zoom) modality and was participated by representatives from the following organizations and/or offices: (1) ABC+ Evaluation Team; (2) Evaluation respondents from DepEd/MBHTE Regional Office, DepEd/MBHTE Division Office, and DepEd/MBHTE Schools; (3) USAID; and (4) CLAimDev (see Annex A for the complete list). Prior to the workshop, an advanced copy of the slide deck that reflected the results of the evaluation was provided to participants. During the validation activity, a prerecorded video of evaluation results was played. Responding to the validation tool and workshop immediately followed after the presentation. The participants were grouped into three breakout rooms. In each room, remarks on their validation responses were shared with the group. B. Workshop Content Evaluation findings presented by the team 227 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV The ABC+ evaluation findings were presented through a pre-recorded video presentation by the STTA Team: Dr. Ivy Mejia (Evaluation Advisor/Team Leader), Dr. Arlyne Marasigan (Technical Specialist for Policy in Education), and Dr. Romylyn Metila (Technical Specialist for MTB-MLE). Please refer to the attached link to the video presentation (see Annex B). The outline of the presentation is shown below: ● Purpose of ABC+ ● Purpose of the performance evaluation ● Evaluation questions ● Evaluation design and methodology ● Results: Relevance, Effectiveness, Sustainability ● Recommendations Validation tool used The validation tool, for the ABC+ evaluation results, consists of 34 items which cover statements capturing the results and recommendations under relevance (with 11 statements), effectiveness (with 11 statements), sustainability (with 12 statements). The validation tool was sent to the participants two weeks before the evaluation workshop together with the presentation of the findings so as to give time for the participants to review and evaluate the findings (see Annex C for the validation tool). This tool was supplemented by interviews and narratives from the validation workshop participants. The participants provided their feedback on statements related to the findings and recommendations presented under relevance, effectiveness, and suitability. C. Workshop Proper After the pre-recorded video presentation and answering of the validation tool, Dr. Mejia then proceeded to the workshop proper where the plenary was distributed to three rooms for Region V (co-facilitated by Dr. Marasigan and Ms. Lisbog), Region VI (co-facilitated by Prof. Metila and Ms. Oliverio) and BARMM (co-facilitated by Dr.. Sasota and Dr. Bazer). The workshop proper lasted for about 45 minutes. The following table presents the consolidated inputs of each regional representative and other invitees for the three evaluation parameters and their feedback on the recommendations. D. Post Validation Meeting Action Points USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 228 1. Note the strengths and limitations of the validation workshop. The Evaluation Team will note the pros and cons, and the limitations of the virtual validation platform, especially with respect to the articulation of the explanations to the survey responses. 2. Add a frequency table on the evaluation questions which will provide granularity to the YES answer of the participants in the validation exercise. 3. Generate a summary of statistics for the validation tool and submit on March 5, 2023 4. Revise the draft report and submit on February 28, 2023 229 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Region Relevance Effectiveness Sustainability Region V Item no. 1: “This is very true” because ABC+ training/s helped us a lot; teachers were taught to navigate different platforms; it helped a lot in the implementation of the learning continuity plan Albay Central School had online classes, and SDO management and parents were very supportive; training/s were conducted for parents, teachers, and learners; teachers were motivated by the innovations initiated by ABC+. Item no. 3: “This is very true” in our local (Masbate City), especially in the Early-grade Level. ABC+’s workshops and innovation were a great help, especially since the city lacks orthography; there were parameters shared by ABC+ which were applied at the early￾grade level. Item no. 4: Item no. 13: Bicol Region has different variations of languages thus, the MTB-MLE-based instructional materials helped a lot in educating the learners, especially that the teachers were trained. Item no. 14: Learners’ love for reading was developed due to accessibility and timely delivery of materials. Some of the teachers were trained to prepare/write their own IMs contextualized in their own areas. Item no. 19: Came up with Policies/Activities to improve learners under full refresher; results of CRLA included in the reports and monitoring to make doable guidelines/activities to improve districts with poor performance in the USY Remediation and intervention activities Item no. 34: Suggestion: Programs should have tied up with other agencies. Touch participation of DSWD especially 4Ps – most of the 4Ps recipients are behind in literacy due to lack of nutrition; Encourage volunteers from mothers of 4Ps beneficiaries. Region V highlighted the need for collaboration with GPH programs such as the 4Ps which address related issues affecting marginalized students, such as malnutrition. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 230 Region Relevance Effectiveness Sustainability “At a moderate degree” Question: Why do we have this description, “at a moderate degree”? Would it affect our answer? In Camarines Sur, there were several initiatives, tapping the assistance of UP NISMED, distributed materials for the early-grade learners, especially on mathematics; Although there were only selected schools that benefitted, the initiatives are very important especially since the focus now on region 5 is on literacy and numeracy. Answer: The ABC+ program began in 2019 and first started with literacy; the UP NISMED training was in the latter part and hence provided conservative indicators only. Suggestion: Could we separate the contributions for the development of our learners’ math skills as per quality and quantity of the training/s? Maybe it is only in the implementation that we are on a moderate degree since it was conducted in the latter part. Item no. 22: Recommendation: Provide training/s on Socio-Emotional aspect/ Wellness of teachers 231 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Region Relevance Effectiveness Sustainability Item no. 5: “This is very true” because ABC+ was able to consider SEL in all the materials Item no. 6: Suggestion: Gender equality and social inclusion are different and hence need to be separated. There is a separate criterion on how to grade the material as to gender equality and social inclusion. These two are also separated in the SDG - #5 Gender Equality and #10 Reduced Inequality ABC+ training helped us integrate gender equality in our instruction to a great extent. ABC+ training helped us integrate social inclusion into our instruction to a great extent. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 232 Region Relevance Effectiveness Sustainability Region VI The participants did not go through each item one by one for comments, explanations, and discussions. Most of our participants were in a seminar for a walkthrough activity of ABC+. However, they indicated in a text message that they were able to respond to the Validation Tool forwarded to them weeks before. Below are the comments made by the participants who made it to the workshop. ABC+ Focal Person from Capiz: I mostly answered yes and had not much explanation since they have little experience and are just starting with the program. ABC+ Focal Person from Escalante: Was attending a seminar for a walkthrough with ABC+ and was not able to concentrate well during the workshop especially since she was to present a report on the topic given to her. ABC+ Focal Person from Iloilo City: She made mention that she was already done with answering the Validation Tool and does not need much explanation anymore. She said that the ABC+ looked more at the literacy program where materials are already complete which need no explanation. As the ABC+ focal person in Iloilo City, she plans on cascading everything she learned to 10 schools in Iloilo City. She mentioned further that the numeracy program should be given lots of focus by ABC+, where concepts included values and alignment to policies. ABC+ Focal Person from Negros Occidental: He answered the Validation Tool and tried to get in the Zoom meeting several times but the connection was slow. He was able to get into the meeting after the breakout session and was able to have a short talk with Ms. Metila while the rest of the participants already transferred to other breakout rooms. He also said that the developed supplementary reading 233 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Region Relevance Effectiveness Sustainability materials in their school were so good in helping children learn how to read that they decided in their school that they will continue using the materials even if DepEd had ordered that the Mother Tongue will not be taught as a subject anymore. ABC+ Focal Person from Antique: Our ABC+ Focal Person from Antique was also attending a DepEd Seminar though he indicated to attend the workshop. He tried to connect to the Zoom meeting but his wifi connection was so slow and was not able to make it for the scheduled workshop. ABC+ Focal Person from Roxas City: Though she indicated long before attending the workshop, she was not able to make it due to some emergency. She was asked if she could have someone to take her place or represent her but to no avail since it was a non-working holiday and no work for DepEd people. Overall, there was not much recommendation given by the participants except one participant, that is, to have a lot of focus next time on the numeracy program of ABC+. The rest did not make any recommendations anymore. BARMM Item no. 1: ABC+ was able to help us a lot when it comes to literacy for our K-3 teachers and learners; they provided us with trainings and approaches that teachers can deliver to their learners which is aligned to MBHTE Item no. 12: Very true because some of those learnings are influenced by ABC+ because this is their foundation to read from L1 to L2 to L3; it should be research-based and backed up by empirical studies; Thanks to ABC+ “BARMM has no IR3” Noted was the qualification of the BARRM responses relative to sustainability, and that BARRM is not ready for a handover from ABC+. At the same time, IR3 activities were not implemented for the BARRM so USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 234 Region Relevance Effectiveness Sustainability Item no. 2: On the contextualization, it is really aligned with the MBHTE’s 5Bs program especially for K-3 learners; the strategies and materials are relevant to the learning areas Item no. 3: Very useful for our children especially on the materials (books and content) Item no. 4: It is moderate because ABC+ here in BARMM just focus mainly on literacy but we are hoping to have numeracy soon Item no. 13: We actually have pool of writers that we recognize and the trainings that ABC+ provided are of great help Item no. 14: Children now love reading especially the materials given; we hope that there are more learning materials that we can get from ABC+ Item no. 15: Parents were oriented on how to use those materials with the children; trainings on home learning strategies maybe the validation questions on sustainability are not applicable35 . 35 Note on BARMM: ABC+ implements only a limited set of interventions in the Bangsamoro Autonomous Region of Muslim Mindanao (BARMM). The goal- and IR-level outcome indicators are the product of many different activities that work together across sub-IRs to produce the desired outcome. Only a few indicators under IR1 were implemented in BARMM. IR 2 and IR 3 were not part of intervention. The theory of change is not applicable when only a few of the activities are selectively implemented as they are in BARMM (Source: Baseline Data Report, p.6) 235 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Region Relevance Effectiveness Sustainability Item no. 5: Very true because teachers were trained on SEL and GEDSI (GESI) Item no. 7: ABC+ has many considerations during the pandemic- they looked into where they could come in so that they can align with what we have here in MBHTE; ABC+ considered everything including the pandemic to help K-3 learners and teachers; they conducted training/s online and provided learners materials to be used by learners at home during the pandemic ABC+ is very helpful, especially in formulating the Adaptive Learning Modules (ALMs); MBHTE has this ALMs- and we use these during the pandemic at home; ABC+ is helpful in giving training/s on SEL and GESI where teachers are advised in integrating these to learners’ learning; We also have Item no. 16: Yes, and teachers were “refreshed” because ABC+ helps the continuity of the learning; although limited and we believe that teachers cascaded the training/s to their respective districts Item no. 17: Helpful and true; actually, even before the pandemic, we have created big books with the parents and teachers Item no. 18: The technical assistance from ABC+ is very helpful knowing for a fact that it is a big agency and we’re having difficulty when it comes to sustainability; good to know that ABC+ is here to help Item no. 20: USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 236 Region Relevance Effectiveness Sustainability this training for parents to help. Item no. 8: Most of the policies in MBHTE are all anchored on national policies and that contextualization is really important here Item no. 9: It is very helpful and doable and maybe we can add numeracy later on like what we have on ELLN Item no. 11: We hope that we have this as well in BARMM 100% agree on the recommendation Item no. 21: It can be but we are still new here as a division As of now, it would be better if we go hand￾in-hand with ABC+ as we cannot do it yet alone for now. Item no. 23: Not 100% because we have different attitudes but hopefully with the help of school leaders through their mentoring, for sustainability, yes, there is, hopefully. 237 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Annex A: Attendance List USAID/Philippines: 1. Monica Ang-Tan – Agreement Officer Representative 2. Albert Aquino – Contracting Officer Representative 3. Thomas LeBlanc – Director, Office of Education 4. Yvette Malciohn – Acting Director of Education Panagora Group: USAID-CLAimDev 1. David Callihan – Chief of Party, CLAimDev 2. Gil Dy-Liacco – Senior Evaluation and Research Specialist 3. Noriel Sicad – Senior Monitoring and Evaluation Specialist/Principal Investigator 4. Noymee Anne Lopez – MER Assistant 5. Romark Jan Sta. Clara – MER Assistant 6. Jerry Jose – ELA Specialist 7. Nancy Samonte – Communications Assistant 8. Marian Gamboa – Learning Coordinator DepEd/MBHTE Representatives: Region V 1. Dinnah Baňares - ABC+ Focal, Masbate City 2. Melissa Carretero – former principal of Albay Central School (ACS) 3. Emma Soriano – ABC+ Focal, Camarines Norte 4. Mai Anne Rondola – ABC+ Focal, Albay 5. Preciosa Dela Vega – ABC+ Focal, Camarines Sur 6. Maricel Dineros – ABC+ Focal, Sorsogon Province Region VI 1. Perly Mapa – Division Filipino and MTB Coordinator, ABC+ Focal Person of Escalante City 2. Ruth Gervero – Education Program Supervisor-SPEC/ALS, ABC+ Focal Person of Capiz BARMM 1. Dr Rasna Latiph – EPS/ABC+ Focal, Maguindanao II Division 2. Dr Noria Mandigan – ASDS, SGA Division 3. Dr Sarah Bansil – EPS/ABC+ Focal, SGA Division STTA-ABC+ Evaluation 1. Ivy Mejia, PhD – Evaluation Advisor/Team Leader 2. Arlyne Marasigan, PhD – Technical Specialist for Policy in Education 3. Romylyn Metila, PhD – Technical Specialist for MTB-MLE USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 238 4. Randolf Sasota, PhD – Data Analyst 5. Ferlie Famaloan – Project Assistant 6. Sheila Oliverio – Regional Field Assistant 7. Jovy Lisbog – Regional Field Assistant 8. Saddam Bazer, PhD – Regional Field Assistant Annex B: ABC+ Evaluation Findings Pre-recorded Video Presentation [Click the link for a copy of the Video Presentation.] Annex C: Validation Tool for the ABC+ Evaluation Results (as formatted on Google Survey Form) Good day! The following survey questions consist of the findings and recommendations on the evaluation of the ABC+ project. This survey aims to validate the evaluation results. Kindly answer the following questions by ticking the appropriate answer based on your own perception or experience. We request you also to provide a short explanation for your answer to each question. Thank you. - Evaluation Team for ABC+ Project Division o Albay o Camarines Norte o Legazpi City o Sorsogon o Escalante City o Iloilo City o Negros Occidental o Roxas City o Camarines Sur o Iriga City o Masbate o Tabaco City o Antique o Bacolod City o La Carlota City o Sagay City o San Carlos City 239 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV o Cotabato City o Maguindanao I o Maguindanao II o SGA Region o V o VI o BARMM Date (MM/DD/YYYY) Relevance 1. ABC+ innovations were very much aligned with DepEd policies, and its activities were especially instrumental in our implementation of learning continuity, especially for literacy instruction. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don't have enough information on this. Explanation for the answer to Question 1 2. We enhanced or contextualized some existing policies due to our ABC+ experience or because we have gained confidence in ABC+ support. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don't have enough information on this. Explanation for the answer to Question 2 3. ABC+ initiatives have been very useful in improving our early-grade literacy instruction. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don't have enough information on this. Explanation for the answer to Question 3 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 240 4. ABC+ contributions for the development of our learners’ math skills are at a moderate degree. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don't have enough information on this. Explanation for the answer to Question 4 5. ABC+ helped us in improving learners’ socio-emotional learning (SEL) to a great extent. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don't have enough information on this. Explanation for the answer to Question 5 6. ABC+ training helped us integrate gender equality and social inclusion in our instruction to a great extent. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don't have enough information on this. Explanation for the answer to Question 6 7. ABC+ made several adjustments in consideration of our capabilities and capacities in the face of the COVID-19 pandemic. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don't have enough information on this. Explanation for the answer to Question 7 Recommendations on Relevance 8. Help Regions disseminate ideas to enhance or contextualize DepEd National or Regional policies that ABC+ is already supporting 241 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV o This is very helpful and/or doable. o This is somewhat helpful and/or doable. o This is not helpful and/or doable. I’d suggest something else. Explanation for the answer to Question 8 / Other suggestion 9. Assist schools in disseminating best practices, especially in implementing recovery programs, most of which concentrate on literacy development. o This is very helpful and/or doable. o This is somewhat helpful and/or doable. o This is not helpful and/or doable. I’d suggest something else. Explanation for the answer to Question 9 / Other suggestion 10. Since numeracy instruction is designed to be a pilot model that will support the plan for developing a National Framework on Numeracy, ABC+ may explicitly develop and design a supplementary framework that targets numeracy instruction in the key stages. o This is very helpful and/or doable. o This is somewhat helpful and/or doable. o This is not helpful and/or doable. I’d suggest something else. Explanation for the answer to Question 10 / Other suggestion 11. Bolster efforts for SEL integration, especially in the instruction of Region V and VI teachers. o This is very helpful and/or doable. o This is somewhat helpful and/or doable. o This is not helpful and/or doable. I’d suggest something else. Explanation for the answer to Question 11 / Other suggestion Effectiveness 12. There are improved reading skills in Filipino and English language in our K-3 learners which is influenced by ABC+. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 12 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 242 13. The development of MTB-MLE-based instructional materials was added as a criterion for K-3 teachers’ competency (e.g., ranking and or promotion). o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 13 14. We observed increased love for reading in our learners. This is connected to ABC+’s provision of contextualized and developmentally appropriate supplementary reading materials (SRMs) to K-3 learners. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 14 15. Home learning partners (HLPS) such as parents, adapted ABC+’s instructional and home learning strategies because of trainings received on how to teach their children at home. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 15 16. K-3 teachers’ increased capacity curriculum development were obtained from various ABC+’s teacher training, workshop, and monitoring programs. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 16 243 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV 17. Locally developed reading materials that are age and developmentally appropriate at the K-3 levels are helping our learners to read in their mother tongue and even in the Filipino language o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 17 18. The learning recovery plan with technical assistance from ABC+ contributed to improved early￾grade instruction and delivery and improved access to quality K-3 materials. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 18 19. ABC+’s contribution to the enhancement of policies was observed more in curriculum development and assessment at the K-3 levels. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 19 Recommendations on Effectiveness 20. Add training and technical assistance on research in instructional differentiation in K-3 levels. o This is very helpful and/or doable. o This is somewhat helpful and/or doable. o This is not helpful and/or doable. I’d suggest something else. Explanation for the answer to Question 20 / Other suggestion USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 244 21. Focus on the transition of ownership of the program from the implementing partner (RTI) to host institutions (DepEd) and organizations (LGU, HLPs). o This is very helpful and/or doable. o This is somewhat helpful and/or doable. o This is not helpful and/or doable. I’d suggest something else. Explanation for the answer to Question 21 / Other suggestion 22. Continue to explore a training design and schedule that will support and cater to teachers’ needs, situations, and well-being. o This is very helpful and/or doable. o This is somewhat helpful and/or doable. o This is not helpful and/or doable. I’d suggest something else. Explanation for the answer to Question 22 / Other suggestion Sustainability 23. ABC+ achievement in improving early-grade reading will persist/ be sustained after the project’s conclusion. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 23 24. ABC+ achievement in improving early-grade SEL will persist/ be sustained after the project’s conclusion. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 24 25. The monitoring and evaluation processes using accessible technology and innovations in teaching and learning modalities and strong support system to be provided by various stakeholders, could lead to sustainability of the ABC+ program even after its project conclusion. o This is very true in our locale. 245 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 25 26. ABC+ initiatives and collaboration with NGO and community stakeholders strengthen local ownership. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 26 27. ABC+ initiatives and collaboration with DepEd RO, DO, and LGU strengthen local leadership. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 27 28. ABC+ collaboration with education and community leaders and volunteers, teachers, and parents helped their respective community(ies) to build resilience especially during the COVID-19 pandemic. o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 28 29. The concept of the whole community approach warrants the sustainability of the ABC+ program o This is very true in our locale. o This is somewhat true in our locale. o This is not true in our locale. o I don’t have enough information on this. Explanation for the answer to Question 29 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 246 Recommendations on Sustainability 30. Guide community LGU and volunteer parents to effectively implement HLP approaches o This is very helpful and/or doable. o This is somewhat helpful and/or doable. o This is not helpful and/or doable. I’d suggest something else. Explanation for the answer to Question 30 / Other suggestion 31. Assist community to continuously monitor community learning hubs o This is very helpful and/or doable. o This is somewhat helpful and/or doable. o This is not helpful and/or doable. I’d suggest something else. Explanation for the answer to Question 31 / Other suggestion 32. Ensure ample supply of quality of supplementary reading materials in different barangay reading centers o This is very helpful and/or doable. o This is somewhat helpful and/or doable. o This is not helpful and/or doable. I’d suggest something else. Explanation for the answer to Question 32 / Other suggestion 33. Encourage and collaborate with the private sector to sustain the needs (i.e., facilities and infrastructure) of schools and community to support continuous improvement literacies (i.e., reading, numeracy, GESI and SEL) o This is very helpful and/or doable. o This is somewhat helpful and/or doable. o This is not helpful and/or doable. I’d suggest something else. Explanation for the answer to Question 33 / Other suggestion 34. Encourage more volunteers to participate in literacy programs not just in the school but in the community as well. o This is very helpful and/or doable. o This is somewhat helpful and/or doable. o This is not helpful and/or doable. I’d suggest something else. Explanation for the answer to Question 34 / Other suggestion 247 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Annex D: Validation Workshop Photos USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 248 II. VALIDATION SURVEY The Evaluation Team conducted a validation survey reaching back to the 177 KII and FGD initial evaluation respondents. The survey yielded 65 respondents, including the 11 who participated in the validation workshop. The regional breakdown of survey respondents is as follows: Region V – 17, Region VI – 40, and BARMM – 8. The validation tool has 11 questions each for relevance and effectiveness, and 12 questions for sustainability. Frequency tables for responses to the 34 questions are shown in Annex 20-A. Tables showing responses by region are in Annex 20-B. Relevance. The Relevance component of the evaluation is composed of two topline questions: the first one is about how ABC+ activities correspond with DepEd objectives, and the second is about the extent of contribution of ABC+ to: a) the improvement of early￾grade literacy and transition to Filipino and English reading, b) acquisition of foundational mathematics skills, and c) improved social and emotional learning. For the first topline question, validation survey results confirmed the evaluation findings that ABC+ activities are very much aligned with DepEd objectives. All survey respondents confirmed this to be true, with 62 of the total 65 respondents indicating that this was very true in their locale, while three indicated that this was somewhat true in their context (Annex 20-A, Table 1). The evaluation result stating that existing policies were contextualized or enhanced because schools gained confidence from ABC+ support was confirmed by 51 of 65 respondents to be very true in their locale, while 12 stated that it was somewhat true in their context (Annex 20-A, Table 2). The survey answers of the big 249 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV majority of respondents support the findings of the evaluation study for the first topline question, confirming that ABC+ activities are indeed relevant to DepEd policies and activities in their respective contexts. The second topline question for Relevance is about the extent of ABC+ contribution in literacy instruction, developing mathematics, and improving SEL. For contribution on literacy instruction, all respondents indicated that ABC+ initiatives had been very useful, with 57 of 65 saying that this was very true in their locale, and 8 stating that this was somewhat true in their place (Annex 20-A,Table 3). The evaluation study’s finding that ABC+ contribution for developing mathematics skill was at a moderate level was indicated by half of the respondents (33 of 65) to be very true in their context, 23 said that it was somewhat true in their locale, while 5 said that this was not true in their place, and four did not have information about it (Annex 20-A, Table 4) This result indicates that ABC+ initiatives for developing mathematics skills is possibly more differentiated across regions. Nevertheless, it should be noted that the majority of respondents did agree with the finding. A regional breakdown of results for this finding show that the majority of respondents selected answers that indicate the finding was true in their locale, while a small number of respondents answered that the finding was not true in their context or that they did not have information about it (Annex 20-B, Table 4). The evaluation finding that ABC+ initiatives helped in improving SEL to a great extent was confirmed to be true by 61 of 65 respondents. Of these 61, 53 regarded this finding to be very true in their context, while 8 indicated that it was somewhat true in their locale (Annex 20-B, Table 5) Other results evaluating ABC+ Relevance were about the adjustment of ABC+ design and implementation to recipients’ capabilities and capacities in the light of the COVID-19 pandemic. Evaluation findings show that ABC+ made several adjustments for their recipients and this was confirmed by all 65 respondents to be true in their locale: 58 regarded it as very true, while 7 indicated that it was somewhat true (Annex 20-B, Table 7). Evaluation also found that ABC+ initiatives helped teachers in integrating Gender Equality and Social Inclusion in instruction to a great extent. Sixty-two of 65 respondents confirmed this to be true, with 53 of the 65 regarding it as very true, while 9 indicated that it was somewhat true. Only one thought that it was not true in his locale, while one did not have information on this (Annex 20-A, Table 6). Effectiveness. The purposes of evaluating the effectiveness of ABC+ were to describe the extent of achievement in the three IRs, identify outcomes brought about by the pandemic, and how it contributed to the enhancement of policies on EGR instruction. Validation survey results confirmed the evaluation findings that the intended outcomes in each IR are contributing to the overall outcome of improved reading skills. For IR 1, K-3 teachers’ increased capacity curriculum development were obtained from various ABC+ teacher training, workshop, and monitoring programs. Table 16 (Annex 20-A) confirms that this was very true for 85% (f = 55) of the respondents. Table 13 (Annex 20-A) indicates that the development of MTB-MLE-based instructional materials was added as a criterion for K-3 teachers’ competency (e.g., ranking and or promotion) and was very true for more than half of the participants (58%, f = 38). USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 250 For IR2, improved reading skills in Filipino and English language and increased love for reading among the learners are connected to ABC+’s provision of contextualized and developmentally appropriate supplementary reading materials (SRMs) to K-3 learners. This evaluation finding was very true in the context of 90% (f = 54) of the validation survey respondents (Annex 20-A, Tables 12 and 14). For IR3, 72% (f = 47) of home learning partners (HLPS), such as parents, adapted ABC+’s instructional and home learning strategies because of training received on how to teach their children at home (Annex 20-A, Table 15). According to 86% of the respondents (f = 56), the learning recovery plan is an unintended outcome brought about by the Pandemic and was also helping in improving the instruction and access to quality materials for the K-3 learners (Annex 20-A, Table 18). The respondents also confirmed that ABC+’s contribution to the enhancement of policies was observed more in curriculum development and assessment at the K-3 levels (86%, f = 56) as reflected in Table 19, Annex 20-A. Participants also agreed that the recommendation of the evaluation team for additional training and technical assistance on research in instructional differentiation in K-3 levels are doable and helpful to their locale (Annex 20-A,Table 20). Sustainability As the ABC+ program commenced a few months before the onset of the COVID-19 pandemic, it was indeed a great challenge to ascertain the program's sustainability since ABC+ has a limited timeline of five years. Under sustainability, the topline question is whether improved achievements in reading, math, and social and emotional skills for children in the early-grades persist after the project’s conclusion. Likewise, this component seeks to observe IR3 which pertains to improved education system capacity and commitment to manage and implement interventions supporting effective early-grade instruction. The validation survey results confirmed the evaluation findings that ABC+ activities most likely will be sustained or will persist even after the project’s conclusion, Tables 23 and 24, Annex 20-B, show the distribution of responses per region. Almost all survey respondents (56 out of 65) confirmed that improving early-grade reading is very true for their locale while 55 out of 65 respondents confirmed improvements in early-grade SEL. Moreover, 55 out of 65 respondents confirmed (Table 25, Annex 20-A) that the monitoring and evaluation processes using accessible technology and innovations in teaching and learning modalities and strong support systems provided by various stakeholders, could lead to the sustainability of the ABC+ program. Respondents confirmed that ABC+ initiatives and collaboration with DepEd, Regional Offices, Divisions Offices, NGOs, and community stakeholders strengthen local ownership across the regions. The same is true with collaboration with education and community leaders and volunteers, teachers, and parents who helped their respective communities to build resilience, especially during the COVID-19 pandemic (Tables 26- 28, Annex 20-A). Many of the respondents affirmed that the concept of the whole community approach warrants the sustainability of the ABC+ program (Table 29, Annex 20-A), where LGU and volunteer parents effectively implement the HLP approaches, assist the community to 251 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV continuously monitor learning hubs, and encourage and collaborate with the private sector to sustain the needs (i.e., facilities and infrastructure) of schools and the community to support continuous improvement literacies (i.e., reading, numeracy, GESI and SEL) (Tables 30-34, Annex 20-A). In addition, the respondents suggested that ABC+ should tie up with other agencies, such as with the Department of Social Welfare and Development, especially with its 4Ps (Pantawid Pamilyang Pilipino Program), a program that provides conditional cash grants to the poorest of the poor, to improve the health, nutrition, and the education of children aged 0-18. Most of the 4Ps recipients are behind in literacy due to lack of nutrition. The respondents also suggested collaboration with mothers of 4Ps beneficiaries, as volunteers. Conclusion Overall, albeit using a small number of informants, the validation survey results generally confirm the midline evaluation findings and recommendations for ABC+. Across regions, respondents’ answers confirm the veracity of the presented findings and support the given recommendations. While some responses were received for statements that disconfirm findings (i.e., not true in our locale), this is a very small number. It may be an indication though that program implementation has not covered the entirety of a target location. The validation survey results, combined with the data gathered from the conducted online validation workshop, confirm the midline evaluation findings for ABC+. ANNEX 20-A. VALIDATION SURVEY FREQUENCY TABLES RELEVANCE TABLE 1 Distribution of Responses to How True is Statement 1. ABC+ innovations were very much aligned with DepEd policies, and its activities were especially instrumental in our implementation of learning continuity, especially for literacy instruction RESPONSE FREQUENCY (n=65) This is somewhat true in our locale. 3 This is very true in our locale. 62 TOTAL 65 TABLE 2 Distribution of Responses to How True is Statement 2. We enhanced or contextualized some existing policies due to our ABC+ experience or because we have gained confidence in ABC+ support USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 252 RESPONSE FREQUENCY (n=65) I don't have enough information on this. 1 This is not true in our locale. 1 This is somewhat true in our locale. 12 This is very true in our locale. 51 TOTAL 65 TABLE 3 Distribution of Responses to How True is Statement 3. ABC+ initiatives have been very useful in improving our early grade literacy instruction RESPONSE FREQUENCY (n=65) This is somewhat true in our locale. 8 This is very true in our locale. 57 TOTAL 65 TABLE 4 Distribution of Responses to How True is Statement 4. ABC+ contributions for the development of our learners’ math skills are at a moderate degree RESPONSE FREQUENCY (n=65) I don't have enough information on this. 4 This is not true in our locale. 5 This is somewhat true in our locale. 23 This is very true in our locale. 33 TOTAL 65 TABLE 5 Distribution of Responses to How True is Statement 5. ABC+ helped us in improving learners’ socio￾emotional learning (SEL) to a great extent RESPONSE FREQUENCY (n=65) I don't have enough information on this. 2 253 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV This is not true in our locale. 2 This is somewhat true in our locale. 8 This is very true in our locale. 53 TOTAL 65 TABLE 6 Distribution of Responses to How True is Statement 6. ABC+ training helped us integrate gender equality and social inclusion in our instruction to a great extent RESPONSE FREQUENCY (n=65) I don't have enough information on this. 2 This is not true in our locale. 1 This is somewhat true in our locale. 9 This is very true in our locale. 53 TOTAL 65 TABLE 7 Distribution of Responses to How True is Statement 7. ABC+ made several adjustments in consideration of our capabilities and capacities in the face of the COVID-19 pandemic RESPONSE FREQUENCY (n=65) This is somewhat true in our locale. 7 This is very true in our locale. 58 TOTAL 65 TABLE 8 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 8. Help Regions disseminate ideas to enhance or contextualize DepEd National or Regional policies that ABC+ is already supporting RESPONSE FREQUENCY (n=65) USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 254 This is somewhat helpful and/or doable. 12 This is very helpful and/or doable. 53 TOTAL 65 TABLE 9 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 9. Assist schools in disseminating best practices, especially in implementing recovery programs, most of which concentrate on literacy development RESPONSE FREQUENCY (n=65) This is somewhat helpful and/or doable. 7 This is very helpful and/or doable. 58 TOTAL 65 TABLE 10 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 10. Since numeracy instruction is designed to be a pilot model that will support the plan for developing a National Framework on Numeracy, ABC+ may explicitly develop and design a supplementary framework that targets numeracy instruction in the key stages RESPONSE FREQUENCY (n=65) This is not helpful and/or doable. I’d suggest something else. 1 This is somewhat helpful and/or doable. 15 This is very helpful and/or doable. 49 TOTAL 65 TABLE 11 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 11. Bolster efforts for SEL integration, especially in the instruction of Region V and VI teachers 255 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONSE FREQUENCY (n=65) This is not helpful and/or doable. I’d suggest something else. 3 This is somewhat helpful and/or doable. 14 This is very helpful and/or doable. 48 TOTAL 65 EFFECTIVENESS TABLE 12 Distribution of Responses to How True is Statement 12. There are improved reading skills in Filipino and English language in our K-3 learners which is influenced by ABC+ RESPONSE FREQUENCY (n=65) I don’t have enough information on this. 2 This is somewhat true in our locale. 9 This is very true in our locale. 54 TOTAL 65 TABLE 13 Distribution of Responses to How True is Statement 13. The development of MTB-MLE-based instructional materials was added as a criterion for K-3 teachers’ competency (e.g., ranking and or promotion) RESPONSE FREQUENCY (n=65) I don’t have enough information on this. 10 This is not true in our locale. 9 This is somewhat true in our locale. 8 This is very true in our locale. 38 TOTAL 65 TABLE 14 Distribution of Responses to How True is Statement 14. We observed increased love for reading in our learners. This is connected to ABC+’s provision of contextualized and developmentally appropriate supplementary reading materials (SRMs) to K-3 learners USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 256 RESPONSE FREQUENCY (n=65) I don’t have enough information on this. 1 This is somewhat true in our locale. 10 This is very true in our locale. 54 TOTAL 65 TABLE 15 Distribution of Responses to How True is Statement 15. Home learning partners (HLPS) such as parents, adapted ABC+’s instructional and home learning strategies because of trainings received on how to teach their children at home RESPONSE FREQUENCY (n=65) I don’t have enough information on this. 3 This is not true in our locale. 5 This is somewhat true in our locale. 10 This is very true in our locale. 47 TOTAL 65 TABLE 16 Distribution of Responses to How True is Statement 16. K-3 teachers’ increased capacity curriculum development were obtained from various ABC+’s teacher training, workshop, and monitoring programs RESPONSE FREQUENCY (n=65) I don’t have enough information on this. 1 This is not true in our locale. 1 This is somewhat true in our locale. 8 This is very true in our locale. 55 TOTAL 65 TABLE 17 Distribution of Responses to How True is Statement 17. Locally developed reading materials that are age and developmentally appropriate at the K-3 levels are helping our learners to read in their mother tongue and even in the Filipino language 257 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV RESPONSE FREQUENCY (n=65) This is not true in our locale. 1 This is somewhat true in our locale. 5 This is very true in our locale. 59 TOTAL 65 TABLE 18 Distribution of Responses to How True is Statement 18. The learning recovery plan with technical assistance from ABC+ contributed to improved early grade instruction and delivery and improved access to quality K-3 materials RESPONSE FREQUENCY (n=65) I don’t have enough information on this. 1 This is not true in our locale. 1 This is somewhat true in our locale. 7 This is very true in our locale. 56 FREQUENCY 65 TABLE 19 Distribution of Responses to How True is Statement 19. ABC+’s contribution to the enhancement of policies was observed more in curriculum development and assessment at the K-3 levels RESPONSE FREQUENCY (n=65) I don’t have enough information on this. 2 This is somewhat true in our locale. 7 This is very true in our locale. 56 TOTAL 65 TABLE 20 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 20. Add training and technical assistance on research in instructional differentiation in K-3 levels RESPONSE FREQUENCY USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 258 (n=65) This is somewhat helpful and/or doable. 10 This is very helpful and/or doable. 55 TOTAL 65 TABLE 21 Distribution of Responses to How True is Statement 21. Focus on the transition of ownership of the program from the implementing partner (RTI) to host institutions (DepEd) and organizations (LGU, HLPs) RESPONSE FREQUENCY (n=65) This is somewhat helpful and/or doable. 20 This is very helpful and/or doable. 45 TOTAL 65 TABLE 22 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 22. Continue to explore a training design and schedule that will support and cater to teachers’ needs, situations, and well￾being RESPONSE FREQUENCY (n=65) This is not helpful and/or doable. I’d suggest something else. 2 This is somewhat helpful and/or doable. 5 This is very helpful and/or doable. 58 TOTAL 65 SUSTAINABILITY 259 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV TABLE 23 Distribution of Responses to How True is Statement 23. ABC+ achievement in improving early grade reading will persist/ be sustained after the project’s conclusion RESPONSE FREQUENCY (n=65) I don’t have enough information on this. 1 This is somewhat true in our locale. 8 This is very true in our locale. 56 TOTAL 65 TABLE 24 Distribution of Responses to How True is Statement 24. ABC+ achievement in improving early grade SEL will persist/ be sustained after the project’s conclusion RESPONSE FREQUENCY (n=65) I don’t have enough information on this. 1 This is somewhat true in our locale. 9 This is very true in our locale. 55 TOTAL 65 TABLE 25 Distribution of Responses to How True is Statement 25. The monitoring and evaluation processes using accessible technology and innovations in teaching and learning modalities and strong support system to be provided by various stakeholders, could lead to sustainability of the ABC+ program even after its project conclusion RESPONSE FREQUENCY (n=65) I don’t have enough information on this. 1 This is somewhat true in our locale. 9 This is very true in our locale. 55 TOTAL 65 TABLE 26 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 260 Distribution of Responses to How True is Statement 26. ABC+ initiatives and collaboration with NGO and community stakeholders strengthen local ownership RESPONSE FREQUENCY (n=65) I don’t have enough information on this. 2 This is not true in our locale. 2 This is somewhat true in our locale. 12 This is very true in our locale. 49 TOTAL 65 TABLE 27 Distribution of Responses to How True is Statement 27. ABC+ initiatives and collaboration with DepEd RO, DO, and LGU strengthen local leadership RESPONSE FREQUENCY (n=65) This is not true in our locale. 1 This is somewhat true in our locale. 6 This is very true in our locale. 58 TOTAL 65 TABLE 28 Distribution of Responses to How True is Statement 28. ABC+ collaboration with education and community leaders and volunteers, teachers, and parents helped their respective community(ies) to build resilience especially during the COVID-19 pandemic RESPONSE FREQUENCY (n=65) I don’t have enough information on this. 1 This is not true in our locale. 1 This is somewhat true in our locale. 9 This is very true in our locale. 54 TOTAL 65 TABLE 29 261 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Distribution of Responses to How True is Statement 29. The concept of the whole community approach warrants the sustainability of the ABC+ program RESPONSE FREQUENCY (n=65) I don’t have enough information on this. 1 This is not true in our locale. 1 This is somewhat true in our locale. 15 This is very true in our locale. 48 TOTAL 65 TABLE 30 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 30. Guide community LGU and volunteer parents to effectively implement HLP approaches RESPONSE FREQUENCY (n=65) This is not helpful and/or doable. I’d suggest something else. 1 This is somewhat helpful and/or doable. 16 This is very helpful and/or doable. 48 TOTAL 65 TABLE 31 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 31. Assist community to continuously monitor community learning hubs RESPONSE FREQUENCY (n=65) This is not helpful and/or doable. I’d suggest something else. 2 This is somewhat helpful and/or doable. 17 This is very helpful and/or doable. 46 TOTAL 65 TABLE 32 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 262 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 32. Ensure ample supply of quality of supplementary reading materials in different barangay reading centers RESPONSE FREQUENCY (n=65) This is not helpful and/or doable. I’d suggest something else. 4 This is somewhat helpful and/or doable. 11 This is very helpful and/or doable. 50 TOTAL 65 TABLE 33 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 33. Encourage and collaborate with the private sector to sustain the needs (i.e., facilities and infrastructure) of schools and community to support continuous improvement literacies (i.e., reading, numeracy, GESI and SEL) RESPONSE FREQUENCY (n=65) This is not helpful and/or doable. I’d suggest something else. 1 This is somewhat helpful and/or doable. 10 This is very helpful and/or doable. 54 TOTAL 65 TABLE 34 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 34. Encourage more volunteers to participate in literacy programs not just in the school but in the community as well RESPONSE FREQUENCY (n=65) This is not helpful and/or doable. I’d suggest something else. 1 This is somewhat helpful and/or doable. 6 This is very helpful and/or doable. 58 TOTAL 65 263 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 20-B. VALIDATION SURVEY FREQUENCY TABLES BY REGION TABLE 1 Distribution of Responses to How True is Statement 1. ABC+ innovations were very much aligned with DepEd policies, and its activities were especially instrumental in our implementation of learning continuity, especially for literacy instruction across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is somewhat true in our locale. 0 3 0 3 This is very true in our locale. 17 37 8 62 TOTAL 17 40 8 65 TABLE 2 Distribution of Responses to How True is Statement 2. We enhanced or contextualized some existing policies due to our ABC+ experience or because we have gained confidence in ABC+ support across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don't have enough information on this. 0 1 0 1 This is not true in our locale. 0 1 0 1 This is somewhat true in our locale. 2 9 1 12 This is very true in our locale. 15 29 7 51 TOTAL 17 40 8 65 TABLE 3 Distribution of Responses to How True is Statement 3. ABC+ initiatives have been very useful in improving our early grade literacy instruction across Region REGION TOTAL RESPONSE V (n=65) (n=17) VI (n=40) BARMM (n=8) This is somewhat true in our locale. 2 5 1 8 This is very true in our locale. 15 35 7 57 TOTAL 17 40 8 65 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 264 TABLE 4 Distribution of Responses to How True is Statement 4. ABC+ contributions for the development of our learners’ math skills are at a moderate degree across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don't have enough information on this. 2 0 2 4 This is not true in our locale. 0 4 1 5 This is somewhat true in our locale. 2 18 3 23 This is very true in our locale. 13 18 2 33 TOTAL 17 40 8 6 TABLE 5 Distribution of Responses to How True is Statement 5. ABC+ helped us in improving learners’ socio-emotional learning (SEL) to a great extent across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don't have enough information on this. 1 0 1 2 This is not true in our locale. 1 1 0 2 This is somewhat true in our locale. 2 5 1 8 This is very true in our locale. 13 34 6 53 TOTAL 17 40 8 65 TABLE 6 Distribution of Responses to How True is Statement 6. ABC+ training helped us integrate gender equality and social inclusion in our instruction to a great extent across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don't have enough information on this. 2 0 0 2 This is not true in our locale. 0 1 0 1 This is somewhat true in our locale. 3 6 0 9 This is very true in our locale. 12 33 8 53 TOTAL 17 40 8 65 265 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV TABLE 7 Distribution of Responses to How True is Statement 7. ABC+ made several adjustments in consideration of our capabilities and capacities in the face of the COVID-19 pandemic across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is somewhat true in our locale. 0 6 1 7 This is very true in our locale. 17 34 7 58 TOTAL 17 40 8 65 TABLE 8 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 8. Help Regions disseminate ideas to enhance or contextualize DepEd National or Regional policies that ABC+ is already supporting across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is somewhat helpful and/or doable. 0 11 1 12 This is very helpful and/or doable. 17 29 7 53 TOTAL 17 40 8 65 TABLE 9 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 9. Assist schools in disseminating best practices, especially in implementing recovery programs, most of which concentrate on literacy development across Region REGION FREQUENCY (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is somewhat helpful and/or doable. 0 6 1 7 This is very helpful and/or doable. 17 34 7 58 TOTAL 17 40 8 65 TABLE 10 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 10. Since numeracy instruction is designed to be a pilot model that will support the plan for developing a National Framework on Numeracy, ABC+ may explicitly develop and design a supplementary framework that targets numeracy instruction in the key stages across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is not helpful and/or doable. I’d suggest something else. 0 1 0 1 This is somewhat helpful and/or doable. 1 10 4 15 This is very helpful and/or doable. 16 29 4 49 TOTAL 17 40 8 65 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 266 TABLE 11 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 11. Bolster efforts for SEL integration, especially in the instruction of Region V and VI teachers across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is not helpful and/or doable. I’d suggest something else. 0 1 2 3 This is somewhat helpful and/or doable. 2 9 3 14 This is very helpful and/or doable. 15 30 3 48 TOTAL 17 40 8 65 TABLE 12 Distribution of Responses to How True is Statement 12. There are improved reading skills in Filipino and English language in our K-3 learners which is influenced by ABC+ across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 1 0 1 2 This is somewhat true in our locale. 4 4 1 9 This is very true in our locale. 12 36 6 54 TOTAL 17 40 8 65 TABLE 13 Distribution of Responses to How True is Statement 13. The development of MTB-MLE-based instructional materials was added as a criterion for K-3 teachers’ competency (e.g., ranking and or promotion) across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 3 5 2 10 This is not true in our locale. 1 7 1 9 This is somewhat true in our locale. 1 6 1 8 This is very true in our locale. 12 22 4 38 TOTAL 17 40 8 65 267 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV TABLE 14 Distribution of Responses to How True is Statement 14. We observed increased love for reading in our learners. This is connected to ABC+’s provision of contextualized and developmentally appropriate supplementary reading materials (SRMs) to K-3 learners across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 0 0 1 1 This is somewhat true in our locale. 3 5 2 10 This is very true in our locale. 14 35 5 54 TOTAL 17 40 8 65 TABLE 15 Distribution of Responses to How True is Statement 15. Home learning partners (HLPS) such as parents, adapted ABC+’s instructional and home learning strategies because of trainings received on how to teach their children at home across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 2 1 0 3 This is not true in our locale. 1 3 1 5 This is somewhat true in our locale. 3 4 3 10 This is very true in our locale. 11 32 4 47 TOTAL 17 40 8 65 TABLE 16 Distribution of Responses to How True is Statement 16. K-3 teachers’ increased capacity curriculum development were obtained from various ABC+’s teacher training, workshop, and monitoring programs across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 1 0 0 1 This is not true in our locale. 0 0 1 1 This is somewhat true in our locale. 1 6 1 8 This is very true in our locale. 15 34 6 55 TOTAL 17 40 8 65 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 268 TABLE 17 Distribution of Responses to How True is Statement 16. K-3 teachers’ increased capacity curriculum development were obtained from various ABC+’s teacher training, workshop, and monitoring programs across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 1 0 0 1 This is not true in our locale. 0 0 1 1 This is somewhat true in our locale. 1 6 1 8 This is very true in our locale. 15 34 6 55 TOTAL 17 40 8 65 Distribution of Responses to How True is Statement 17. Locally developed reading materials that are age and developmentally appropriate at the K-3 levels are helping our learners to read in their mother tongue and even in the Filipino language across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is not true in our locale. 1 0 0 1 This is somewhat true in our locale. 1 2 2 5 This is very true in our locale. 15 38 6 59 TOTAL 17 40 8 65 TABLE 18 Distribution of Responses to How True is Statement 18. The learning recovery plan with technical assistance from ABC+ contributed to improved early grade instruction and delivery and improved access to quality K-3 materials across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 0 1 0 1 This is not true in our locale. 0 0 1 1 This is somewhat true in our locale. 3 3 1 7 This is very true in our locale. 14 36 6 56 TOTAL 17 40 8 65 269 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV TABLE 19 Distribution of Responses to How True is Statement 19. ABC+’s contribution to the enhancement of policies was observed more in curriculum development and assessment at the K-3 levels across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 0 0 2 2 This is somewhat true in our locale. 2 4 1 7 This is very true in our locale. 15 36 5 56 TOTAL 17 40 8 65 TABLE 20 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 20. Add training and technical assistance on research in instructional differentiation in K-3 levels across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is somewhat helpful and/or doable. 1 9 0 10 This is very helpful and/or doable. 16 31 8 55 TOTAL 17 40 8 65 TABLE 21 Distribution of Responses to How True is Statement 16. K-3 teachers’ increased capacity curriculum development were obtained from various ABC+’s teacher training, workshop, and monitoring programs across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 1 0 0 1 This is not true in our locale. 0 0 1 1 This is somewhat true in our locale. 1 6 1 8 This is very true in our locale. 15 34 6 55 TOTAL 17 40 8 65 Distribution of Responses to How True is Statement 21. Focus on the transition of ownership of the program from the implementing partner (RTI) to host institutions (DepEd) and organizations (LGU, HLPs) across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is somewhat helpful and/or doable. 5 14 1 20 This is very helpful and/or doable. 12 26 7 45 TOTAL 17 40 8 65 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 270 TABLE 22 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 22. Continue to explore a training design and schedule that will support and cater to teachers’ needs, situations, and well-being across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is not helpful and/or doable. I’d suggest something else. 0 1 1 2 This is somewhat helpful and/or doable. 1 3 1 5 This is very helpful and/or doable. 16 36 6 58 TOTAL 17 40 8 65 TABLE 23 Distribution of Responses to How True is Statement 23. ABC+ achievement in improving early grade reading will persist/ be sustained after the project’s conclusion across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 0 1 0 1 This is somewhat true in our locale. 3 5 0 8 This is very true in our locale. 14 34 8 56 TOTAL 17 40 8 65 TABLE 24 Distribution of Responses to How True is Statement 24. ABC+ achievement in improving early grade SEL will persist/ be sustained after the project’s conclusion across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 0 1 0 1 This is somewhat true in our locale. 4 5 0 9 This is very true in our locale. 13 34 8 55 TOTAL 17 40 8 65 TABLE 25 Distribution of Responses to How True is Statement 16. K-3 teachers’ increased capacity curriculum development were obtained from various ABC+’s teacher training, workshop, and monitoring programs across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 1 0 0 1 This is not true in our locale. 0 0 1 1 This is somewhat true in our locale. 1 6 1 8 This is very true in our locale. 15 34 6 55 TOTAL 17 40 8 65 271 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Distribution of Responses to How True is Statement 25. The monitoring and evaluation processes using accessible technology and innovations in teaching and learning modalities and strong support system to be provided by various stakeholders, could lead to sustainability of the ABC+ program even after its project conclusion across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 0 1 0 1 This is somewhat true in our locale. 3 6 0 9 This is very true in our locale. 14 33 8 55 TOTAL 17 40 8 65 TABLE 26 Distribution of Responses to How True is Statement 26. ABC+ initiatives and collaboration with NGO and community stakeholders strengthen local ownership across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 0 2 0 2 This is not true in our locale. 1 1 0 2 This is somewhat true in our locale. 3 8 1 12 This is very true in our locale. 13 29 7 49 TOTAL 17 40 8 65 TABLE 27 Distribution of Responses to How True is Statement 27. ABC+ initiatives and collaboration with DepEd RO, DO, and LGU strengthen local leadership across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is not true in our locale. 1 0 0 1 This is somewhat true in our locale. 2 4 0 6 This is very true in our locale. 14 36 8 58 TOTAL 17 40 8 65 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 272 TABLE 28 Distribution of Responses to How True is Statement 16. K-3 teachers’ increased capacity curriculum development were obtained from various ABC+’s teacher training, workshop, and monitoring programs across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 1 0 0 1 This is not true in our locale. 0 0 1 1 This is somewhat true in our locale. 1 6 1 8 This is very true in our locale. 15 34 6 55 TOTAL 17 40 8 65 Distribution of Responses to How True is Statement 28. ABC+ collaboration with education and community leaders and volunteers, teachers, and parents helped their respective community(ies) to build resilience especially during the COVID-19 pandemic across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 1 0 0 1 This is not true in our locale. 0 1 0 1 This is somewhat true in our locale. 4 5 0 9 This is very true in our locale. 12 34 8 54 TOTAL 17 40 8 65 TABLE 29 Distribution of Responses to How True is Statement 29. The concept of the whole community approach warrants the sustainability of the ABC+ program across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 0 0 1 1 This is not true in our locale. 1 0 0 1 This is somewhat true in our locale. 2 12 1 15 This is very true in our locale. 14 28 6 48 TOTAL 17 40 8 65 TABLE 30 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 30. Guide community LGU and volunteer parents to effectively implement HLP approaches across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is not helpful and/or doable. I’d suggest something else. 0 0 1 1 This is somewhat helpful and/or doable. 3 13 0 16 This is very helpful and/or doable. 14 27 7 48 TOTAL 17 40 8 65 273 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV TABLE 31 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 31. Assist community to continuously monitor community learning hubs across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is not helpful and/or doable. I’d suggest something else. 1 1 0 2 This is somewhat helpful and/or doable. 3 13 1 17 This is very helpful and/or doable. 13 26 7 46 TOTAL 17 40 8 65 TABLE 32 Distribution of Responses to How True is Statement 16. K-3 teachers’ increased capacity curriculum development were obtained from various ABC+’s teacher training, workshop, and monitoring programs across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) I don’t have enough information on this. 1 0 0 1 This is not true in our locale. 0 0 1 1 This is somewhat true in our locale. 1 6 1 8 This is very true in our locale. 15 34 6 55 TOTAL 17 40 8 65 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 32. Ensure ample supply of quality of supplementary reading materials in different barangay reading centers across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is not helpful and/or doable. I’d suggest something else. 1 0 3 4 This is somewhat helpful and/or doable. 2 9 0 11 This is very helpful and/or doable. 14 31 5 50 TOTAL 17 40 8 65 TABLE 33 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 33. Encourage and collaborate with the private sector to sustain the needs (i.e., facilities and infrastructure) of schools and community to support continuous improvement literacies (i.e., reading, numeracy, GESI and SEL) across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is not helpful and/or doable. I’d suggest something else. 1 0 0 1 This is somewhat helpful and/or doable. 3 7 0 10 This is very helpful and/or doable. 13 33 8 54 TOTAL 17 40 8 65 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 274 TABLE 34 Distribution of Responses to How Helpful and Doable ABC+ Program is in Statement 34. Encourage more volunteers to participate in literacy programs not just in the school but in the community as well across Region REGION TOTAL (n=65) RESPONSE V (n=17) VI (n=40) BARMM (n=8) This is not helpful and/or doable. I’d suggest something else. 1 0 0 1 This is somewhat helpful and/or doable. 2 3 1 6 This is very helpful and/or doable. 14 37 7 58 TOTAL 17 40 8 65 ANNEX 21. TEACHERS’ EVALUATION OF ABC+ CONTRIBUTION TO DEVELOPING LEARNERS’ NUMERACY SKILLS USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 275 ANNEX 22. SELECTED MOST SIGNIFICANT CHANGE STORY USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 276 ANNEX 23. INCEPTION REPORT 277 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV PERFORMANCE EVALUATION ADVANCING BASIC EDUCATION IN THE PHILIPPINES (ABC+) Inception Report September 2022 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 278 CONTENTS ACRONYMS 3 EXECUTIVE SUMMARY 4 PURPOSE OF THE EVALUATION 6 BACKGROUND 7 DESCRIPTION OF THE PROBLEM AND CONTEXT 7 EVALUATION QUESTIONS 8 RELEVANCE 8 EFFECTIVENESS 8 SUSTAINABILITY 9 EVALUATION DESIGN AND METHODOLOGY 10 LOCATION AND GEOGRAPHIC SCOPE 10 EVALUATION DESIGN 10 DATA COLLECTION 11 SITE SELECTION CRITERIA 13 DATA ANALYSES 15 TRIANGULATION 16 EVALUATION DESIGN MATRIX 16 KNOWN LIMITATIONS TO THE EVALUATION DESIGN 16 THREATS TO INTERNAL VALIDITY 17 DATA MANAGEMENT AND ETHICAL CONSIDERATIONS 28 WORKPLAN 28 ANNEX 1. AVAILABLE DATA 29 ANNEX 2: SITE AND RESPONDENT SELECTION CRITERIA 30 ANNEX 3: SAMPLING FRAME (INDIVIDUALS, INSTITUTIONS, ORGANIZATIONS) 33 279 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 4: TARGET SAMPLE (INDIVIDUALS, INSTITUTIONS, ORGANIZATIONS) 36 ANNEX 5: DATA COLLECTION TIMELINE 39 ANNEX 6: SURVEY FOR TEACHERS 40 ANNEX 7: SURVEY FOR SCHOOL HEADS 47 ANNEX 8: KEY INFORMANT INTERVIEW QUESTIONNAIRE 54 ANNEX 9: FOCUS GROUP DISCUSSION PROTOCOL 57 ANNEX 10: SRM EVALUATION TOOL FOR TEACHERS 62 ANNEX 11: ABC + GANTT CHART 65 ANNEX 12: OH WORKSHOP QUESTIONNAIRE 69 ANNEX 13: MOST SIGNIFICANT CHANGE STORY COLLECTION FORM 70 ANNEX 14: MSC GUIDELINES FOR STORY VERIFICATION AND SELECTION 74 ANNEX 15: VIEWING TOOL FOR RECORDED CLASSES 75 ANNEX 16: DOCUMENTS FOR DESK REVIEW 78 ANNEX 17: ANTICIPATED SCHEDULES AND LOGISTICAL ARRANGEMENTS 83 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 280 ACRONYMS BARMM Bangsamoro Autonomous Region in Muslim Mindanao CO Central Office (DepEd) DepEd Department of Education EGR Early-grade reading assessment GESI Gender equity and social inclusion KPI Key performance indicator LGU Local Government Unit LOI Language of instruction MTB-MLE Mother Tongue-Based Multilingual Education OH Outcome Harvesting PTC Parent Teacher Community Qual Qualitative Quan Quantitative RO Regional Office RTI Research Triangle Institute SD School Division SEL Socio Emotional Learning TAF The Asia Foundation ToC Theory of Change TOCOR Task Order Contracting Officer’s Representative TPACK Technological, pedagogical, and content knowledge USAID U.S. Agency for International Development 281 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV EXECUTIVE SUMMARY This inception report presents the methodology the ABC+ external performance evaluation team will use. The methodology will emphasize three evaluation criteria (relevance, effectiveness, and sustainability) to assess ABC+’s performance in achieving three intermediate results (IR): 1. IR 1: improved early-grade instruction and delivery, 2. IR 2: improved access to quality materials, and 3. IR 3: improved education capacity and commitment to manage and implement interventions supporting early-grade instruction. The evaluation will focus on the performance of the ABC+ project in achieving its stated objectives in relation to its three (3) intermediate results (IRs). The questions focus on the relevance, effectiveness, and sustainability of interventions of ABC+. Specifically, the topline questions are the following: 1. To what extent does the innovation introduced by ABC+ on improved early-grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction correspond with the objectives of the Department of Education of the Philippines (priorities, policies, the needs, and the practical requirements)? 2. To what extent has ABC+ contributed to improving early-grade literacy, the better transition of reading ability to Filipino and English, more solid acquisition of foundational math concepts and skills, and improved social and emotional learning for children in the early-grades (kindergarten to grade 3) in the Philippines? 3. To what extent did ABC+ achieve its outputs and outcomes in the three (3) intermediate results (IRs) on improved early-grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction? 4. Will the achievements in improved early-grade reading, math, and social and emotional skills for children in the early-grades persist after the project’s conclusion? The evaluation team will use a theory-based, mixed-methods design to understand ABC+’s performance. It will use a concurrent mixed methods evaluation approach using quantitative and qualitative methods. In response to the challenges of early literacy development in the context of Mother Tongue-Based Multilingual Education (MTB-MLE), USAID Philippines, the Philippine Department of Education, and Research Triangle Institute International (RTI) and its subcontractors co-developed a project referred to as Advancing Basic Education in the Philippines or ABC+. This is a five-year USAID-funded program aimed at improving early-grade reading, with an emphasis on the mother tongue and transition to USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 282 Filipino and English for children in kindergarten to grade 3. It also supports a more solid acquisition of foundational mathematical concepts and skills and improved social and emotional learning. ABC+ also integrates gender equality and social inclusion (GESI) in its design and programming. Although the ABC+ activity’s objectives and design remain the same, the context within which it was implemented was significantly changed by the COVID-19 pandemic. Community restrictions brought about by the pandemic affected ABC+’s intended inputs and activities. As a result, unexpected outcomes other than those reflected in the results framework are possible, especially when implemented in a complex environment, such as the unprecedented contextual changes in education due to COVID and school closures. In addition to theory-based mixed methods to answer the evaluation questions, the evaluation will use Outcome Harvesting (OH) as part of the approach to uncover unexpected outcomes not envisioned in the theory of change. There are three levels of analysis in the evaluation design: 1) individuals focusing on ABC+ assistance recipients such as teachers, school heads, faculty, and students. 2) institutions focusing on system capacity, including the Department of Education, Regional and Division levels, and schools; and 3) community organizations and private sector affiliates involved in ABC+ implementation. The evaluation will focus on ABC+ activities in Regions V (Bicol), VI (Western Visayas), and a small portion of BARMM (Maguindanao Division in BARMM). The selection criteria will use two filters, geographic location with the associated linguistic focus and participant profile. For the first filter, Bicol, Western Visayas, and Maguindanao Schools Division in BARMM, which are automatically selected as part of the project. The next filter would be the school divisions that received the assistance of ABC+. For the selection criteria of respondents, there will be three filters. The first filter is the geographic location (Regions V, VI, and a small portion of BARMM). The second filter is a proportional sampling from selected divisions. Selected divisions are based on the scale used by ABC+, where the most used languages were the basis for site selections. Additional selection criteria for divisions are the literacy rate, type of schools, proximity to city, mode of transportation to reach the site, and availability of electricity and internet connectivity for remote data collection. Lastly, the third filter will randomly choose participants who received support from ABC+. The evaluation will use four activities to collect data: 1) Electronic surveys for individuals such as teachers and school heads (IR1 and IR2), 2) Key Informant Interviews (KIIs) for system capacity-related topics with Implementing Partners, DepEd Central, Regional and Division Offices as respondents (IR3), 3) Focus Group Discussions (FGDs) and OH workshops for early-grade instructions and delivery (IR1) and access to materials as the topics (IR2), and 4) Case studies for in-depth analysis of home learning partners and private sector engagements. The evaluation team will also collect secondary data from the implementing partner and its subcontractors, and other sources. Data analysis will include descriptive statistics, content analysis, and policy analysis. This inception report also outlines the major tasks and timelines that will guide the evaluation team. 283 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 284 PURPOSE OF THE EVALUATION The United States Agency for International Development/Philippines (USAID/PH) is commissioning a third-party performance evaluation of ABC+: Advancing Basic Education in the Philippines implemented by RTI International, the prime partner, The Asia Foundation, SIL Lead, and Florida State University. Through this evaluation, USAID/PH will have evidence to gauge the extent to which ABC+ is achieving its outputs and outcomes in the three intermediate results (IRs) on improved early-grade instruction and delivery (IR1), improved access to quality materials (IR2), and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction (IR3). The evaluation will focus on the performance of ABC+ from 2019 to 2022. Specifically, it will assess its relevance, effectiveness, and sustainability through a series of evaluation questions. ABC+ works to support the Philippines Department of Education’s goal to increase and sustain basic skills among K-3 learners. Results of the evaluation will inform the design of the Mission’s next-generation basic education activities. The intended audiences of this evaluation are USAID/PH education and program officers and USAID staff worldwide interested in basic education programs, specifically at K-3 levels, Philippine and U.S. stakeholders, including those in other U.S. government agencies and organizations, the Department of Education (DepEd) in the Philippines, and other researchers and organizations interested in the use MTB-MLE to strengthen literacy and numeracy in basic education, especially this time of the pandemic. TABLE 1. ACTIVITY SUMMARY INFORMATION36 TITLE ABC+: ADVANCING BASIC EDUCATION IN THE PHILIPPINES Cooperative Agreement Number 72049219CA00005 Start and end dates July 1, 2019, to June 28, 2024 Funding levels $38.5 million Implementing partners RTI International Technical lead on all intermediate results (IRs). Overall project management, quality assurance, and timely reporting. Sub-awardees The Asia Foundation Local government capacity building for sustainability; strengthening partnerships with the private sector (IR 3). Mother tongue materials development (IR 2). SIL LEAD Language policy and capacity building for developing reading materials (IRs 2 and 3). Florida State University Partnering with universities in the regions to develop pre-service modules for mother-tongue literacy and math instruction (IR 1). UP-NISMED 36 Fiscal Year 2022, Quarter 2 Report January 1–March 31, 2022. ABC+: Advancing Basic Education in the Philippines. 285 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 286 BACKGROUND DESCRIPTION OF THE PROBLEM AND CONTEXT The ABC+: Advancing Basic Education in the Philippines is a five-year USAID-funded program aimed at improving the early-grade reading, math, and social and emotional skills of children in the early￾grades (kindergarten to grade 3) in the Philippines.37 The prominent feature of early-grade instructions in the K - 12 curriculum in the Philippines is the Mother Tongue-Based Multilingual Education (MTB-MLE) Policy. From kindergarten through grade 3 (K–3), learners progressively acquire literacy skills in their mother tongue (MT), Filipino, and English, transitioning to instruction in Filipino and English starting in grade 4. In a country with 180 plus native languages, the delivery of MTB-MLE throughout the country can be daunting. The baseline study conducted by ABC+ reflects this, showing that many children are being left behind in school, particularly boys and rural communities. Grade 2 learners’ decoding skills are underdeveloped, and their reading and comprehension abilities are weaker than their oral language skills. Although supplemental materials are available at the Department of Education, it appeared that these were not utilized by teachers as often as intended. In response to the challenges of early literacy development in the context of MTB-MLE, the United States Agency for International Development (USAID), the Philippines Department of Education (DepEd), and RTI International (RTI) and its subcontractors co-developed a project targeted at improving basic language skills in the early-grades, with an emphasis on improving early-grade reading in the MT and for the transition to Filipino and English. ABC+ supports improved early-grade literacy, better transition of reading ability to Filipino and English, more solid acquisition of foundational math concepts and skills, and improved social and emotional learning. ABC+ works with the Philippine DepEd in Manila, as well as with its regional- and division-level offices in Regions V (Bicol) and VI (Western Visayas) and selected districts of Maguindanao in the Bangsamoro Autonomous Region in Muslim Mindanao (BARMM).38 Figure 1 shows the estimated numbers of beneficiaries such as students, teachers, school heads, and supervisors. 37 Monitoring, evaluation, and learning plan. ABC+: Advancing Basic Education in the Philippines 38 Monitoring and Evaluation Learning Plan: Advancing Basic Education in the Philippines (ABC+) 287 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Figure 1. Estimated Number of Beneficiaries (Life of Project) In March 2020, schools closed due to COVID-19 disrupting ABC+’s implementation, which started four weeks earlier. Most activities became remote, including teacher training. In December 2021, selected DepEd schools piloted face-to-face classes. Beginning in January 2022, DepEd progressively expanded face-to-face classes in 6,000 schools as part of its Learning Recovery Plan. ABC+ partnered with DepEd Region V to contribute to this effort in preparing The Recovering for Academic Achievement by Improving Instruction through Sustainable Evidence-Based Learning Programs (RAISE). This 3-year learning recovery plan aims to ensure that the COVID-19 generation of learners does not suffer a disadvantage compared with past and future generations. It hopes to enable this cohort of learners to fully recover the learning lost during the pandemic. It prioritizes remedial or accelerated education approaches to address challenges and get all learners to catch up grade-level proficiency. RAISE also invests in appropriate resources, such as program and curriculum guides, assessment tools, and learning materials39, to offset learning losses and minimize drop-out rates.40 41 For the school reopening set for the 2022-2023 school year, ABC+ continued to facilitate collaboration between the Department of Health and the Department of Education and selected implementing partners to support the reopening of schools for face-to-face classes. As part of this collaboration, the Breakthrough Action, part of the USAID/Philippines Reach Health project, developed the BIDA Kid Campaign, which focuses on motivating children and their families to follow health protocols for a safe return to school. ABC+ gained additional funding to provide support in BARMM and Region VI schools. ABC+ also has resumed field visits to meet with partners and observe continuous implementation and monitoring of the program. In addition, to carry out the necessary support system needed. THEORY OF CHANGE (TOC) ABC+’s design assumes that “IF quality of instruction is improved (IR 1), quality and availability of materials are enhanced (IR 2), and education system capacity to support early-grade learning (IR 3) is improved, THEN it will lead to positive and sustained gains in students’ basic skills. ABC+ seeks to achieve its overall objective through the following Intermediate Results (IR): IR 1: Improved early-grade instruction and delivery systems IR 2: Improved access to quality materials IR 3: Improved education system capacity and commitment to manage and implement interventions supporting effective early-grade instruction Specifically, the theory of change for each IR is as follows: IR 1. IF teachers master and apply effective reading instructional practices for the early-grades, instructional supervisors have a better capacity to support developmentally appropriate 39 https://www.pna.gov.ph/articles/1179348 40 ABC+ Fiscal Year 2022 Quarter 2 Report, January 1-March 31, 2022 41 Bicol educators join DepEd's learning recovery program - https://www.pna.gov.ph/articles/1179348 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 288 instructional practices that foster children’s cognitive, social, and emotional development; higher education institutions are better equipped to prepare pre-service teachers for early-grades instruction and to pilot instructional innovations; and there is increasing evidence on what works to strengthen instructional practices in the early-grades, THEN teachers will be more effective in providing instruction that develops students’ basic skills in the early-grades. IR 2. IF local capacity to develop quality early reading materials is strengthened, critical gaps (language and level) in the supply of materials are filled, inefficiencies in the book chain are resolved, and digital versions of materials are made more easily accessible, THEN children will have more equitable access to greater quantities of quality reading materials, in the right language and at the right level. IR 3. IF policies to support mother tongue (MT) instruction and provision of learning resources are effectively implemented, local government units (LGUs) and local stakeholders are mobilized to support and sustain ABC+, introduce improvements in early-grade instruction, and private sector engagement in support of ABC+, introduce improvements in early-grade instruction is increased, and parent and community involvement in supporting early-grade learning is increased, THEN the education system will be better positioned to deliver and sustain quality learning opportunities in early-grades. Figure 2 illustrates the theory of change. 289 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Figure 2. ABC+ Theory of Change Diagram USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 290 The ABC+ Logical Framework Model (LogFrame) derives from the theory of change. Table 2 details the complete LogFrame model mapping key activities (inputs/processes) to outputs to IR-level behavioral change outcomes and associate key performance indicators (KPIs). There are 22 KPIs drawn from the LogFrame, including two specific gender equality and social inclusion (GESI) embedded directly under IRs 1 and 2 (indicators 1.1 d and 2.2). RESULTS FRAMEWORK ABC+ operates according to the Results Framework shown in Figure 3. This framework summarizes how ABC+’s strategies contribute to results. 291 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 292 Figure 3. Results Framework 293 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV TABLE 2. DETAILED LOG FRAME ACTIVITY (PROCESSES/INPUTS) OUTPUTS IR-LEVEL BEHAVIOR CHANGE OUTCOMES CORRESPONDING KPI(S) ABC+ Goal-Level Theory of Change and Logical Framework Teachers, school heads, and supervisors trained and supported to deliver effective early-grade reading (EGR) instruction K–3 classrooms receive an increased number of quality, grade-appropriate reading materials The enabling environment (e.g., policies, partnerships, and resources) is fostered to focus support on EGR best practices and inputs. K–3 students receive improved instruction K–3 students have greater access to and use of quality reading materials Schools, classrooms, and teachers benefit from increased support K–3 students increase the rate of literacy foundational skills in their MT K–3 students transition to acquiring literacy skills in Filipino Schools, school divisions, regional offices (ROs), LGUs, and the private sector increase focus and support for EGR resources and best practices Goal-level (GL)1. Percent of learners targeted for US Government (USG) assistance who attain a minimum grade-level proficiency in reading at the end of grade 2 in MT (ES.1-1-outcome) GL2. Percent of learners targeted for USG assistance who attain a minimum grade-level proficiency in reading at the end of grade 2 in Filipino (ES.1-1- outcome) GL3. Percent of learners targeted for USG assistance with an increase of at least one proficiency level in reading at the end of grade 2 in MT (ES.1-48- outcome) GL4. Percent of learners targeted for USG assistance with an increase of at least one proficiency level in reading at the end of grade 2 in Filipino (ES.1-48- outcome) GL5. Number of learners in primary schools reached with USG assistance (ES.1-3-output) GL6. Number of learners reached in reading programs at the primary level (ES.1-5-output) IR 1: IMPROVED EARLY-GRADE INSTRUCTION AND DELIVERY SYSTEMS Cadre of master trainers trained in instructional best practices K–3 teachers trained in instructional best practices GESI awareness assessment and content integrated into training modules Teachers equipped with the knowledge, skills, and self-efficacy to deliver effective EGR instruction Teachers’ sensitivity and awareness of GESI-related issues increased Teachers differentiate between boys’ and girls’ learning needs.” 1.1b. Percent of grade 2 teachers indicating positive beliefs and knowledge of EGR best practices (output) 1.1c. Number of primary school educators who complete professional development activities on implementing evidence-based reading instruction with USG assistance (ES.1-7) 1.1d. Percent of school heads demonstrating practical knowledge and understanding of GESI best practices (GESI-output) School heads trained to foster a greater School heads have the knowledge and skills to promote SEL in their School heads and teachers actively promote 1.1e. Percent of school heads demonstrating practical knowledge and understanding of SEL best USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 294 TABLE 2. DETAILED LOG FRAME ACTIVITY (PROCESSES/INPUTS) OUTPUTS IR-LEVEL BEHAVIOR CHANGE OUTCOMES CORRESPONDING KPI(S) SEL environment in the classrooms GESI awareness assessment and content integrated into SEL modules schools a positive SEL environment practices (custom-output) Pre-service course modules and syllabi adapted to deliver best practices in student-teacher preparation for EGR instruction Pre-service instructors trained on EGR best practices for preparing student￾teachers Instructors equipped with the knowledge, skills, and materials to improve their preparation for EGR courses Instructors apply knowledge, skills, and best practices in delivering EGR courses 1.1f. Number of teacher education institutions (TEIs) that adopt ABC+-introduced modules on early-grade literacy and numeracy instruction (output) School heads and supervisors trained to monitor, support, and provide effective feedback to K–3 teachers Teacher observation and feedback tools, as well as Learning Action Cells (LAC) materials adapted and provided to school heads and supervisors School heads and supervisors equipped with tools and skills to support and mentor teachers to improve their EGR instructional practices School heads and supervisors actively monitor, support, and provide effective feedback to teachers to improve their EGR instructional practices LACs become more effective venues for peer-to-peer learning 1.2. Percent of school heads and supervisors actively providing effective instructional support and feedback to K–3 teachers according to clearly defined expectations (outcome) ABC+ trains selected teachers in innovative math instructional practices Selected math teachers equipped with the knowledge and skills to deliver innovative instruction in mathematics Math teachers apply knowledge and skills of innovative math instruction in the classroom 1.3. Percent of mathematics teachers delivering innovative math instruction according to a clearly defined set of observable practices in fidelity to the training (outcome) ABC+ trains selected teachers on the Tangerine: ClassTM application to improve their continuous assessment and differentiated instructional practices in EGR Selected K–3 teachers equipped with the skills, knowledge, tools, and technology to utilize the Tangerine: ClassTM application Teachers effectively use Tangerine: ClassTM to implement their routine assessment and apply differentiated instructional practices 1.4. Percent of teachers using Tangerine: ClassTM appropriately according to a clearly defined set of observable practices in fidelity to the training (outcome) IR 2: IMPROVED ACCESS TO QUALITY MATERIALS Story writers and illustrators trained to utilize low-cost technologies to develop quality leveled supplemental MT reading materials. Leveled supplemental reading materials in target MTs produced and distributed to K–3 classrooms. GESI awareness issues and content K–3 classrooms receive leveled supplemental reading materials in target MTs Story writers and illustrators equipped with the knowledge and tools to develop quality supplemental materials in MT Teachers and students use high-quality MT reading materials in the classrooms 2.1. Percent of K–3 classrooms in which teachers and learners are regularly using supplemental learning materials according to a clearly defined set of observable criteria (outcome) 295 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV TABLE 2. DETAILED LOG FRAME ACTIVITY (PROCESSES/INPUTS) OUTPUTS IR-LEVEL BEHAVIOR CHANGE OUTCOMES CORRESPONDING KPI(S) integrated into supplemental materials DepEd central office (CO), ROs, and Schools Divisions supported to develop high-quality SEL materials DepEd CO, ROs, and Schools Divisions equipped with the knowledge, skills, and resources to produce high￾quality SEL materials DepEd CO, ROs, and Schools Divisions produce and disseminate SEL materials 2.2. Percent of titles produced by ABC+ that meet GESI-appropriate standards based on a checklist of clearly observable criteria (GESI-output) Supplemental reading materials in Filipino and English procured and distributed to K–3 classrooms K–3 classrooms receive supplemental learning materials in Filipino and English Teachers and students use high-quality English and Filipino supplemental materials in the classroom 2.3. Percent of primary-grade learners targeted for USG assistance who have the appropriate variety of decodable, leveled, and supplementary readers in the LOI (ES.1-45) The LRP system is improved in terms of its functionality, user experience, and accessibility An improved LRP interface is tested and deployed to participating regions Teachers and administrators increasingly access and download materials from the LRP 2.4. Content, access, and usability of LRP enhanced according to system metrics and key informant teachers and administrators (outcome) IR 3: IMPROVED EDUCATION SYSTEM CAPACITY AND COMMITMENT TO MANAGE AND IMPLEMENT INTERVENTIONS SUPPORTING EFFECTIVE EARLY-GRADE INSTRUCTION Teacher observation and feedback tools adapted from the Philippine Professional Standards for Teachers (PPST) and provided to school heads and supervisors, along with reporting and feedback templates School heads and supervisors are equipped with the knowledge, tools, and skills to monitor teachers and report on instructional practices School heads and supervisors report summarized results, aggregated at the division level, and include them in the school monitoring, evaluation and adjustment process, quarterly division monitoring, evaluation, and adjustment process (DMEA) meetings 3.2a. Number of Schools Divisions incorporating timely indicators on teachers’ quality of EGR instruction in their DMEA process (outcome) DepEd CO, ROs, and Schools Divisions engaged to review existing plans and budgets and to include in their 2021 budgets resource allocations to cover additional EGR-related costs Documented reviews of existing allocations to identify opportunities to better align resources with priority areas of investment for improving implementation of policies in support of early-grades literacy and learning outcomes Budgets and plans incorporate additional allocations or programs that focus specifically on literacy and early-grade learning outcomes 3.2b. Percent change in the allocation of resources for learning materials and teacher professional development at central, regional, and division levels LGUs engaged in identifying current gaps in Special Education Fund (SEF) guidelines, analyzing the prevailing patterns of resource allocation and expenditure, and strengthening local school board (LSB) planning and prioritization. Build coalitions of stakeholders Documented analyses of LSB/SEF funding and resource allocation patterns LGU/LSB plans, and SEF allocation priorities incorporate additional allocations or programs that focus specifically on literacy and early-grade learning outcomes 3.3 Percent change in the allocation of SEF funds from LGUs in target regions for printing and distribution of learning resources and teacher participation in professional development USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 296 TABLE 2. DETAILED LOG FRAME ACTIVITY (PROCESSES/INPUTS) OUTPUTS IR-LEVEL BEHAVIOR CHANGE OUTCOMES CORRESPONDING KPI(S) DepEd assisted in identifying and arranging private sector engagement, e.g., resources, technical expertise, advocacy, and advice, in support of ABC+ and DepEd objectives related to improving early-grade learning Documented analyses of private sector landscape, identification and inventory of potential partnerships, and areas of engagement Private sector partners are increasingly mobilized to focus resources and programs to support MTB-MLE and early-grade learning programs 3.4. Number of private-sector partners engaged to support evidence-based EGR interventions (outcome) Outreach strategies that can enlist greater parental and community participation in parent-teacher associations (PTAs), civil society organizations, and LSB activities Provide materials to help parents engage in home literacy activities Materials and assistance delivered to selected schools and Schools Divisions to strengthen PTA and community-engagement programs Parents and communities are increasingly engaged and support MTB-MLE and early￾grade learning outcomes 3.5. Number of PTAs or community-based school governance structures supported with USG assistance (ES.1-13) 297 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ABC+ uses the results framework in Figure 3 as a tool to monitor and manage its progress. It sets out (1) development outcomes, (2) results, and (3) metrics to measure effectiveness and efficiency. The results framework shows the hierarchy of results expected within ABC+. The following are activities which target learning outcomes in reading, mathematics, and SEL 4. Capacity development for K-3 teachers for effective reading instructional practices 5. Capacity development for the production of quality reading materials for a specific language of instruction (LOI) 6. Capacity development for instructional supervisors to support developmentally appropriate instructional practices that foster children’s cognitive, social, and emotional development 7. Management capacity for supporting mother tongue (MT) instruction and provision of learning resources At the program goal level, the project, in its third year of implementation, has achieved the following: 8. From a target of 41.5% of proficient readers in MT at the end of Grade 2, only 12.5% were assessed to be demonstrating this indicator (GL1) 9. From a target of 42.2 % of proficient readers in Filipino at the end of Grade 2, only 12.8 % were assessed to be demonstrating this indicator (GL2) 10. From a target of 50% of schools’ heads demonstrating practical knowledge in understanding of SEL best practices, 45.2% were reported to be practicing a such skill (GL3) 11. From a target of 60% of school heads and supervisors actively promoting effective instructional support and feedback to K-3 teachers, 54.9% were reported to be integrating this into their supervisory roles (GL4) 12. From a target of 923 titles, a total of 814 primary textbooks and other TLMS were already provided 13. From a target of 360 parents and community members to train, a total of 176 individuals already received training on supporting children’s education 14. From a target of 8,608, 310 reading materials, a total of 11,634, 849 early-grade reading materials were produced and distributed to schools and K-3 learners 15. From a target of 576 titles, a total of 1,402 titles of early-grade reading materials were developed 16. From a target of 523, 201 learners, a total of 750, 479 learners benefitted from the project USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 298 EVALUATION QUESTIONS The evaluation will focus on the performance of the ABC+ project in achieving its stated objectives in relation to its three (3) intermediate results (IRs). The questions focus on the relevance, effectiveness, and sustainability of interventions of ABC+. For relevance, the set of questions will be on the alignment of the ABC+ project to the needs, policies, and priorities of the Department of Education on increasing and sustaining basic skills among K-3 learners. Additionally, it will assess whether the evaluation is sensitive and responsive to context and quality of design over time.42 For effectiveness, it will provide judgment on the success and fidelity of the project to the logic model. Lastly, for sustainability, it will assess the continuity of the project and will also look at the significance and contribution of ABC+. The evaluation will seek to identify new practices or steps toward a long-term change, if any, and the effects of ABC+’s interventions on social actors such as the individual, group, community, organization, or institution. The main questions are the following: RELEVANCE 17. To what extent does the innovation introduced by ABC+ on improved early-grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction correspond with the objectives of the Department of Education of the Philippines (priorities, policies, the needs, and the practical requirements)? 1. Did ABC+’s design and implementation consider the available capability and capacities of the recipients (human resources and organizations) and responses to any change in context (e.g., public health emergency such as COVID-19)? 2. How did ABC+ incorporate gender equality and social inclusion (GESI) analysis? To what extent has ABC+ implementation supported GESI? 18. To what extent has ABC+ contributed to improving early-grade literacy, the better transition of reading ability to Filipino and English, more solid acquisition of foundational math concepts and skills, and improved social and emotional learning for children in the early-grades (kindergarten to grade 3) in the Philippines? EFFECTIVENESS 19. To what extent did ABC+ achieve its outputs and outcomes in the three (3) intermediate results (IRs) on improved early-grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction? 42 OECD (2021). Applying Evaluation Criteria Thoughtfully, OECD Publishing, Paris, https://doi.org/10.1787/543e84ed-en. 299 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV 3. Which of the three (3) IRs contributed the most to improving early-grade reading, math, and social and emotional skills for children in the early-grades (kindergarten to grade 3)? What were the explanatory factors, if any, for achievement, overachievement, and underachievement or non-achievement? 4. Do ABC+ trained teachers to differentiate between boys’ and girls’ unique learning needs? (IR1- learning question) 5. Do boys and girls find the ABC+ supplemental reading materials stimulating and relevant to their age group and gender? (IR2- learning question) 6. Have there been any outcomes brought by COVID-19 in implementing ABC+ that would tell us whether the program improved early-grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction? 7. How have ABC+ contributed to the enhancement of policies on EGR instruction and learning outcomes in kindergarten to grade 3 (K–3) SUSTAINABILITY 20. Will the achievements in improved early-grade reading, math, and social and emotional skills for children in the early-grades persist after the project’s conclusion? 8. How did ABC+ strengthen local ownership and leadership on the enhanced instruction and learning outcomes in kindergarten to grade 3 (K–3)? 9. How can regions/divisions/LGUs better support and reach GIDCA, remote, and Last Mile schools? (IR3- learning question) 10. How do attitudes and perceptions of parents and other community stakeholders vary in relation to early-grade education and MTB-MLE in particular? (IR3- learning question) USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 300 EVALUATION DESIGN AND METHODOLOGY LOCATION AND GEOGRAPHIC SCOPE The evaluation will cover Regions V (Bicol), VI (Western Visayas), and a small portion of BARMM (Cotabato, Maguindanao Division I in BARMM) in which ABC+ is being implemented. The evaluation will use the division, district, and school levels in its sampling frame to select purposively informants using ABC+ participant lists. EVALUATION DESIGN Anchored by the theory of change, the evaluation will use a theory-based, mixed-methods design to understand ABC+’s performance. The evaluation methodology will use a concurrent mixed methods evaluation43, this means that quantitative and qualitative methods are used concurrently. The reason why concurrent mixed-methods was used in this study was to understand the implementation of ABC+ through qualitative and quantitative evaluation inquiries. The selection of key informant interviews and FGD participants will be purposive while the selection of respondents for the survey will be randomly sampled. The evaluation team will process data for any surveys when the received responses reach the minimum acceptable response rate of 50% (Babbie, 1990). Although the ABC+ activity’s objectives and design remain the same, the context within which it was implemented was significantly changed by the COVID-19 pandemic. Program implementation had to be adapted to changing health protocols, school, community, and family situations. Due to these unforeseen changes brought by pandemic, evaluation analysis will be set against what was done and how relevant and effective this was given the scenario it was in. Community restrictions brought about by COVID-19 affected the intended inputs and activities of ABC+ and so unintended outputs and outcomes are expected. The program was implemented only five months before COVID-19 pandemic. As a result, outcomes other than those reflected in the results framework are possible, especially when implemented in a complex environment, such as the unprecedented contextual changes in education due to COVID-19 and school closures. The evaluation will use Outcome Harvesting (OH) as part of the approach to uncover unexpected outcomes and contributions not envisioned in the theory of change, in addition to the quantitative and qualitative methods used to evaluate ABC+’s performance related to theorized outcomes. The OH method defines an outcome as a change in an individual, group, community, organization, or institution’s behavior, relationships, actions, activities, policies, or practices.44 It could also be an unexpected or unintended demonstrated change in awareness, knowledge or skills, collaborative action, or the use of knowledge or innovative solutions. Outcomes might also describe unexpected or unintended deeper institutional changes relating to policy, citizen engagement, government accountability, and organizational arrangements.45 43 Mertens, D. M. (2017). Mixed methods design in evaluation. Thousand Oaks: CA: Sage. 44 Wilson-Grau, R. and Britt, H. (2012). Outcome Harvesting. Cairo: Ford Foundation. (p.1) 45 The World Bank. (2014). Outcome-based Field Learning Guide. Washington, D.C. (p.13) 301 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV The evaluation will ask key informants and focus group discussion participants to describe what they perceived as the outcomes for each IRs and the ABC+’s contributions to the outcomes. However, since it is entering only its 4th year, the harvesting of outcomes will only focus on the project's processes and how they affected the project's inputs and outputs. DATA COLLECTION METHODS In this evaluation, mixed methods will be used to fully realize the ground-level implementation of the ABC+ intervention program. The quantitative method will use a survey and qualitative methods will include desk review, key informant interview (KII), focus group discussion (FGD), Outcomes Harvesting (OH) workshops, and Most Significant Change Technique (MSCT). The use of multiple data sources through mixed methods will enable the evaluation to cover a wide spectrum, conduct more in-depth analysis, and achieve a more holistic understanding of data through the process of triangulation (Yin, 2003). Some of these methods will be conducted virtually, and while others will be conducted in-person. The sequence of data collection is as follows: desk review, survey, KIIs, FGDs, OH workshops, MSCT, and evaluation of supplementary materials DESK REVIEW The desk review of documents starts early in data collection, and this involves reading reports, policies, plans, etc. to gather data to answer the evaluation questions. A template matrix to organize reviewed data will be used to gather evidence from reading documents. Desk review data will help answer the evaluation questions across all IRs and will also inform the crafting of questions for KIIs, FGDs, and workshops for OH and MSC. See Annex 19 for the types of documents that will be reviewed. If an Early Language, Literacy, and Numeracy Assessment (ELLNA) was conducted and data are available for all schools under ABC+ and can be shared by the Bureau of Education Assessment, then this information will be considered. However, this type of data can often be difficult to access. SURVEY Simple surveys will be administered online. Participants will be teacher trainees, supervisors, school heads, LGU, and private sector focal persons. The survey tool will gather data related to the evaluation questions on the relevance, effectiveness, and sustainability of the project. It will include questions to measure satisfaction and effectiveness of the program to recipients and an inventory of innovations introduced by the host institution as part of sustainability. There are also questions to measure the relevance of ABC+’s intervention to existing policies at the central office, regional, and division levels of DepEd. KEY INFORMANT INTERVIEWS KIIs will be conducted via phone or virtual conferencing platforms. Featured questions are informed by desk review data and the project context presented by IP during the initial consultation of the evaluation team. Questions will be emailed to informants should they be unavailable for calls or online conferencing. FOCUS GROUP DISCUSSION USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 302 Two FGDs will be conducted on-site by gathering seven (7) to eleven (11) informants representing different stakeholder groups. Participants will be grouped according to the topics they will discuss. One group will comprise DepEd heads, trainers for instruction and materials development, community members (LGU representatives, etc.), and teachers who will talk mainly about the conducted interventions and their perceived outcomes. The second FGD will involve teachers and parents discussing the outcomes and elaborating on their experiences from the provided interventions. Key informant interviews and focus group discussions should be with USAID personnel, officials of the Department of Education Central Office (COs), Regional Offices (ROs), School Divisions (SDs), and selected beneficiaries (teachers, students, parents, school boards) and partners from the private sector. OUTCOME HARVESTING WORKSHOP AND MOST SIGNIFICANT CHANGE WORKSHOP Outcome Harvesting Workshop (OHW) and Most Significant Change Technique (MSC) will be conducted to complement each other. OH prompts will be informed first by the desk review data, which will be used for crafting workshop prompts, which key program organizers will validate. Finalized versions will be used in a workshop. Using meta cards, participants (social actors) representing different groups of intervention recipients will be gathered in a workshop to respond to the prompts and discuss their answers. Collected outcomes and outcome descriptions will be verified and categorized (domains of change) by program implementers (change agents). Outcomes harvested and the activities that led to them are appropriate springboards for the MSC stories of the participants. OH is good at harvesting intended and unintended outcomes and gives a group perspective to outcomes or changes, while MSC presents a change through the perspective of an individual. While OH outcomes tend to be general, MSC data can be rich and vivid. Picking up from the identified outcomes and domains of change in the OH workshop, participants in the MSC Level 1 workshop (i.e., teachers and parents, etc.) will be asked to write down or audio record their most significant change stories in relation to their experience with ABC+ activities or outputs. A created template will be used to help participants recall changes they have experienced or witnessed in relation to ABC+ activities. The template will also guide participants in providing details and structure to their stories. Participants may write more than one story. Once stories are written down or recorded, the group reads their stories and chooses the most significant story based on the prompt given to them. The selected stories and the reasons for choosing them are noted. The set of selected stories from Level 1 is passed on to Level 2 workshop participants (i.e., DepEd and ABC+ trainers and focal persons) who will meet to verify the content of collected stories and choose the most significant stories from the set. This set will then be passed on to Level 3 informants (i.e., Key leaders of DepEd and ABC+) who will select the final set of most significant stories. In selecting MSC stories, Level 2 and 3 participants will be provided with a given set of criteria that they may use, or alternatively, they can create their own set of criteria for choosing stories. Emerging themes from the stories and the differences or similarities among them may be used to complement OH results to gain a better understanding of the effects of ABC+ implementation. Data from MSC stories may be used for the selection of case studies in each IRs. EVALUATION OF SUPPLEMENTARY READING MATERIALS (SRM) The SRM evaluation will be conducted by having teachers choose three SRMs to evaluate using the provided tool. These SRMs should be the most frequently used ones by teachers and parents or 303 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV those that are children’s reported or observed favorite. Teachers fill out the evaluation form electronically and email these along with photos or video recordings of the pages of the evaluated SRMs. CLASS OBSERVATION Class observation will use schools’ submitted video recordings of K-3 classes. Videos will be viewed and evaluated using an adapted tool that will look primarily at the application of strategies taught in ABC+ training, the teachers’ use of the local language, and the integration of SRM use in lessons. Combining these methods will allow for greater data consistency and triangulation of information for validity. SITE SELECTION CRITERIA The main stakeholders are composed of the following: (a) Department of Education Central, Regional, and Local Offices (DepEd); (b) Local Government Unit; (c) Parent Community Engagement (PCE); and (d) private sectors. The analysis is at three levels: individual, institution, and organization. The evaluation design will include a selection of sites for KIIs and FGDs, and a list of participants (education leaders, teachers, school heads, parents, LGU, NGOs and private partners) for the case studies for USAID/PH’s approval. Specifically, the selection criteria are in two levels, geographic location with the associated linguistic focus46, and participant profile. For the first level of selection criteria for the regions, Bicol, Western Visayas, and Maguindanao Schools Division in BARMM47 who are part of the project are automatically selected. The next filter would look into the school divisions in the three regions that received the assistance of ABC+, specifically schools’ proximity to RO (ARPY2, p.9.). A sampling frame of schools from the implementing partner would guide the evaluation team for its specific selection criteria at the level of respondents, making sure the geographic locations and the seven languages used in the project are well represented. However, for KII, a census of high-level positions involved in early-grade literacy will be interviewed (i.e., DepEd Central, Directors of Regions V, VI, and BARMM, chief educational supervisors, and educational supervisor in charge of early-grade literacy). Table 3 shows the summary of data collection method and corresponding participants. Table 3. Summary of data collection method and corresponding participants. Method Target participants/Documents 46 Project Year 2 Annual Progress Report: October 2020–September 2021, p. 12) 47 Note on BARMM: Bangsamoro Autonomous Region of Muslim Mindanao (BARMM) was excluded from the survey because ABC+ implements only a limited set of interventions in the region. The goal- and IR-level outcome indicators are the product of many different activities that work together across sub-IRs to produce the desired outcome. The theory of change is not applicable when only a few of the activities are selectively implemented as they are in BARMM (Source: Baseline Data Report, p.6) USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 304 Document review Annual Reports, DepEd documents Key Informant Interviews (KII) Central Office, ROs, and DOs Focus Group Discussions (FGD) Seven (7) language groups composed of teachers, school heads LGU, and private sector focal persons Simple surveys Teacher trainees, supervisors, school heads, LGU and private sector focal persons f Case studies LGUs/Schools (proximity to RO) Outcome Harvesting Workshop Teachers, school heads, parents, LGU officials Review basic education statistics Division-level data (if possible) KII PARTICIPANTS AND COMPOSITION CRITERIA KIIs will require in-depth information from informants so that they will be selected based on their key positions (if from DepEd and ABC+) and intensive involvement in ABC+ interventions. The projected KII participants and their backgrounds are found in Annex 6. FGD PARTICIPANTS AND COMPOSITION CRITERIA FGD’s success depends largely on the varied backgrounds of informants and perspectives they contribute to the discussion. Their response to one another’s answers will also matter. To ensure that quality FGD will take place, the total number of participants will be limited from 6-12. A maximum of 2 representatives per stakeholder group will be invited to participate. The projected FGD participants and their backgrounds are found in Annex 6. SURVEY In selecting the respondents for the online surveys, stratified random sampling will be employed. Cochran’s formula with a 95% confidence interval and a 3% margin of error will be used in determining the sample size. Annex 4 presents the sampling procedures and template table showing the distribution of all participating schools, teachers, and desired sample size by geographical area. Sample selection will be guided by proportional sampling to ensure that schools from Region V, Region VI, and selected schools in BARMM are fairly represented. OH PARTICIPANT COMPOSITION AND SELECTION CRITERIA OH participants will be those closest to program implementation, such as field implementers (e.g., focal persons, teachers, trainers, writers and illustrators, QA evaluators, etc.). A few members who are knowledgeable about the program but detached from the intervention will also be invited (e.g., program designers and key leaders). 305 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV MSC PARTICIPANT COMPOSITION AND SELECTION CRITERIA This will be composed of informants that represent different groups in program implementation. The following are the project-level groups and the backgrounds of their members. An MSC workshop will be held for each ABC+ beneficiary region. Level 1: Intervention recipients and trainers: trained teachers and their school heads, parents of children who received SRMs, and community members representing a Division where OH was conducted. Level 2: Intervention implementers from DepEd, ABC+, and partners: focal persons, regional or division trainers (different from the ones in Level 1), writers and illustrators, materials QA evaluators, etc., representing a Division where OH was conducted. Level 3: Program Leadership from DepEd and ABC+: program designers, policy, and decision￾makers, Central, Regional, or Division DepEd leaders, etc. The evaluation team will collect additional information from performance monitoring reports, school records, milestone reports, quarterly progress reports, and annual reports. If the community restrictions in the three targeted regions (Bicol, Western Visayas, Maguindanao Schools Division in BARMM) make in-person data collection unfeasible, the evaluation will use remote methods. Electronic activity records, such as scanned documents, may be substituted for in￾person visual reviews of activity records. Data gathering with remote data collection may be pursued via online or smartphone surveys using Google forms, Microsoft forms, and similar computer and mobile-aided measures and using video conferences through Zoom, and Google Meet, among others. The data collection phase will begin with six weeks of fieldwork, including two weeks of reviewing documents collected onsite. A visual representation of data collection and expected participants can be found in Annex 5. DATA ANALYSES QUANTITATIVE The evaluation will use descriptive statistics and analytic data visualizations for the quantitative data and disaggregated data for sex and location to the extent possible. The said approach to quantitative analysis is consistent with USAID’s embedded monitoring and evaluation in the Program Cycle.48 21. Data from the survey will be analyzed through descriptive statistics 22. Comparison of division and school-level data at the start and midline, if possible 23. Descriptive statistics on classroom observation data collected during fieldwork, if possible 48 https://usaidlearninglab.org/monitoring-toolkit?tab=2 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 306 24. Analysis of progress of ABC+ toward indicators from the quarterly/annual reports 25. Effect size of interventions by cohort, divisions, sex, and location, if possible QUALITATIVE The qualitative data will be recorded and transcribed in worksheets. It will be subjected to content analysis using computer-assisted qualitative data analysis software (CAQDAS). The evaluation team will code the responses and compute the estimated reliability index. A constant comparison method will complement the index where the team will check the coherence of responses, codes, and themes to ensure that the results will have no external threats to the audience of this performance evaluation. The team also will use content analysis on the outcomes harvested. 26. Qualitative data analysis will be used for FGD, KII, MSC, and OH. Expected information will be about themes gathered from the discussion to analyze the evaluation criteria indicators. In both quantitative and qualitative analyses, computer software such as the SPSS and NVivo will be used for data processing and analysis. TRIANGULATION The evaluation team will combine and compare the quantitative and qualitative analysis results. Comparing the two results will provide a complete understanding of the ABC+ project in terms of relevance, effectiveness, and sustainability. The evaluation also will use joint displays for a mixed￾methods approach. EVALUATION DESIGN MATRIX The design matrix in Table 4 summarizes the evaluation design and methods. KNOWN LIMITATIONS TO THE EVALUATION DESIGN This performance evaluation has five potential limitations, namely: reduced sample size, inaccessible sites, Hawthorne effect, ability to recall information for self-reported data, and limited access to documents. First, the sample size, some respondents of the online Survey, KII, and FGD may not be able to participate due to prior commitments, health reasons/COVID-19, weak internet connectivity, and power outages that will result in a reduced number of samples. Second, some study sites might not be accessible due to COVID-19 quarantine restrictions, distance from the ROs/DOs, and unexpected weather disturbances. Third is the Hawthorne effect, where teachers perform mainly for observation either through an in-person visit or recorded video lessons. Fourth, in the self-reported data where the survey, KII, and FGD rely on what respondents will say, and responses will depend on what people can remember because these activities or events took place in the past. Lastly, the access to documents where some project contact persons are probably no longer connected with their schools or transferred to other school or office (principal and other office staff assigned to another school) and will need more time to look for documents, which will result in delay or limited access to documents. THREATS TO INTERNAL VALIDITY 307 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV There may be a tendency for some informants in KIIs, FGDs, and workshops for OH and MSC to manifest participant bias, social desirability bias, sponsor bias, and acquiescence bias. To prevent these biases from manifesting, informants will be assured that their responses are confidential and that the evaluators are not connected to ABC+. To prevent social desirability bias, indirect questioning will be applied, and questions will be phrased to assure informants that it is fine to answer in a way that is not socially desirable. Informants will be assured that there are no right, or wrong answers and data will be verified to avoid inflated self-reports. To prevent sponsor bias, the interviewer/facilitator for KIIs, FGDs, and OH and MSC workshops will maintain a neutral stance and will be careful not to reinforce positive responses from participants. To prevent acquiescence bias, questions will elicit elaborated and accurate responses from informants. In FGDs and workshops, moderators will ensure that the process is inclusive and participatory, all informants contribute to the discussion, and no one monopolizes discussions or activities. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 308 TABLE 4. EVALUATION DESIGN MATRIX Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) 1. Relevance: To what extent does the innovation introduce by ABC+ on improved early￾grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction correspond with the most recent objectives of the Department of Education of the Philippines (priorities, policies, the needs, and the practical requirements)? To what extent has ABC+ contributed to improving early-grade literacy, the better transition of reading ability to Filipino and English, more solid acquisition of foundational math concepts and skills, and improved social and emotional learning for children in the early￾grades (kindergarten to grade 3) in the Philippines? Qual Project Documents and Reports IPs: RTI International, the prime partner, The Asia Foundation, SIL Lead, and Florida State University. Key stakeholders: DepEd, School Boards, Community, Parent Teachers’ Association) Document Review KIIs (CO, ROs, SDs) Note: Develop survey instruments on educational innovation (see Annex 8). Use the identified innovation pillars by OECD 1.2. How relevant were the activities conducted by ABC+ in reaching out to the marginalized groups to be incorporated into policy and intervention in the Qual Project Documents and Reports Document Review FGD: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators 309 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV TABLE 4. EVALUATION DESIGN MATRIX Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) three IRs? 1.3. In what ways was the intervention based on sound gender equality and social inclusion (GE and SI) analysis? To what extent has ABC+ been implemented to promote GE and SI? Project Documents and Reports IPs: The Asia Foundation and SIL LEAD Document Review Did the design and implementation of the ABC+ consider the available capability and capacities of the recipients (human resources, organizations) as well as the responses to any change in context (e.g., public health Qual Project Documents and Reports IPs: RTI International, the prime partner, The Asia Foundation, SIL Lead, and Florida State University. Key stakeholders: DepEd, School Boards Document Review Baseline data49 from (1) GAD Checklist for Education Projects, (2) situational analysis of the educational provision for children with disabilities, and (3) Basa Pilipinas study of beliefs and 49 ABC+ Gender and Inclusion Plan 2019 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 310 TABLE 4. EVALUATION DESIGN MATRIX Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) emergency such as COVID-19) in the course of its lifespan? practices KIIs (CO, ROs, SDs) Effectiveness: To what extent did ABC+ achieve its outputs and outcomes in the three (3) intermediate results (IRs) on improved early-grade instructions and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction? 2.1. Which of the three (3) IRs contributed the most to the achievement of improving early-grade reading, math, and social and emotional skills for children in the early￾grades (kindergarten to grade 3) Qual Project Documents and Quarterly Reports Assessment tools Key stakeholders: DepEd, School Boards, Community, Parent Teachers’ Association) Document Review KIIs (CO, ROs, SDs) FGD: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators 2.2. To what extent is the target group reached? Were results delivered for men and women, boys and girls, and persons with disability? Quan-Qual Project Documents and Reports Document Review WEBSITE ANALYTICS (ABC+ Tangerine) Frequency and duration of webpage visits, click￾through patterns, sequence of pages viewed. Survey items or questions: How many schools adopted policies in EGR, GESI, and SEL? 311 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV TABLE 4. EVALUATION DESIGN MATRIX Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) 2.3. Which among the following elements of instruction and learning outcomes (professional and curriculum development and stakeholders’ participation) in K–3 has the ABC+ made the greatest impact? In what ways did COVID-19 affect the performance at each IRs? Qual Key stakeholders: DepEd, School Boards, Community, Parent Teachers’ Association) KIIs (CO, ROs, SDs) FGD: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators 2.4. Are teachers, school administrators, and pre￾service faculty improving their technological, pedagogical, and content knowledge (TPACK) on EGR and SEL (e.g., instructional practices and materials)? Quan-Qual Project Documents and Reports IPs: RTI International, the prime partner, The Asia Foundation, SIL Lead, and Florida State University. Key stakeholders: DepEd, School Boards Document Review KIIs (CO, ROs, SDs) FGD questions What skills and knowledge did participants acquire (trainees, parents, learners)? To what extent have skills and knowledge been applied in USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 312 TABLE 4. EVALUATION DESIGN MATRIX Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) instructions (as observed by school heads and supervisors) and curriculum development storytellers and illustrators)? 2.5 How have policies in EGR instruction contributed to change in enhancing instructions and learning outcomes in kindergarten to grade 3 (K–3)? Quan-Qual Project Documents and Reports IPs: RTI International, the prime partner, The Asia Foundation, SIL Lead, and Florida State University. Key stakeholders: DepEd, School Boards 2.6. Have there been any outcomes brought by COVID 19 in the implementation of ABC+ that would tell us whether the program improved early-grade Qual Document review Workshop on Outcome harvesting with social actors 313 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV TABLE 4. EVALUATION DESIGN MATRIX Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) instructions and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction? 2.7. Have there been any negative or unexpected outcomes as a result of project activities, and if so, what are they, and how can we learn from these outcomes? Qual Document review Workshop on Outcome harvesting with social actors Sustainability: Will the achievements in improved early-grade reading, math, and social and emotional skills for To what extent are the Department of Education and local actors (financially, personnel￾wise, and in terms of Organization records Policies, personnel records, and student records. Document Review KIIs (CO, ROs, SDs) FGDs: trainers, trainees, curriculum developers USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 314 TABLE 4. EVALUATION DESIGN MATRIX Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) children in the early￾grades persist after the conclusion of the project? organization) capable and prepared to maintain the positive effects without support in the long term? Key stakeholders: DepEd, School Boards (storytellers, illustrators), school administrators Site visit to check institutional capacity 3.2. To what extent was sustainability addressed at the design stage and during the project? What are the major factors (including risks) influencing sustainability? In what contexts were the efforts on sustainability applicable (CO, ROs, SD, school, and community level)? Policy documents Sustainability Plan, PY1 report, Table 19, p.74 Key stakeholders: DepEd, School Boards Document Review KIIs (CO, ROs, SDs, School heads) Site visit to check institutional capacity (infrastructure at community level, if any) KII/FGD questions Were there new policies crafted, approved, and implemented by COs, ROs, SDs to support the training of teachers and development of materials and SEL? How long is the access to the LRP after the project ceases? 3.3 To what extent did the ABC+ strengthen local ownership and leadership on the enhance instruction and Qual Document Review FGD (School boards, School heads, parents, FGD question Are there any initiatives that pertain to scaling up and out beyond the 315 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV TABLE 4. EVALUATION DESIGN MATRIX Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) learning outcomes in kindergarten to grade 3 (K–3)? private sector) project life? 3.4 What are the visible challenges and opportunities regarding the sustainability of the project, and how likely is their occurrence? What were the strengths of the professional development, curriculum development, and policies? What were the weaknesses, and how can this be improved? 3.5 To what extent are the target groups capable of sustaining the positive effects of ABC+ without support in the long term? KIIs (CO, ROs, SDs) FGDs: trainers, trainees, curriculum developers (storytellers, illustrators), school administrators FGD questions: -What were the lessons learned that may support the sustainability of the project in the context of the Department of Education and local actors? How is accuracy in the content of MT, Math, GESI, and SEL assessed? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 316 TABLE 4. EVALUATION DESIGN MATRIX Evaluation Questions Specific questions Design Suggested Data sources Suggested Data Collection Methods Sample survey items/FGD/KII questions (For the Eval Team-Pre￾field Phase) Were the classroom observation tools used aligned to the training design on early-grade learning? Are home literacy materials easy to follow 317 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV DATA MANAGEMENT AND ETHICAL CONSIDERATIONS The evaluation team will ensure that ethical conventions in program evaluations are adhered to. The team will secure informed consent from all participants in FGD and KII and all survey respondents as part of the data gathering process. The evaluation team will prepare the Participant Informed Consent Forms customized for various types of respondents. Moreover, the team will observe cultural sensitivity and DNH principles in the entire spectrum of the evaluation process. The participants will receive information during the informed consent process about the: (1) purpose of the evaluation, (2) the extent of their involvement, (3) their right to refuse or discontinue participation, (4) potential benefits of the evaluation, and (5) confidentiality of their responses. Data gathering will ensue only when the participant expresses voluntary participation. All interview data will be stored on secure computers in Panagora’s Manila office. The Panagora Home Office will ensure that personally identifiable data will not be included in any public reports, including the final reports submitted to the USAID DEC. As per the nature of this performance evaluation, interview data sets will not be submitted to USAID or any third party. WORKPLAN The team anticipates the following major tasks and milestones. Detailed activities are contained in the Gantt chart in Annex 11. WORKPLAN MAJOR TASK MILESTONE Inception report to COP for review week of Sep 12-16, 2022 End Data collection (19 days) October 10, 2022 Data processing and analysis (12 days) October 25, 2022 USAID Out-brief October 26, 2022 Out-briefing with IP Week of Nov 1-4 Final Report drafting November 21-28, 2022 Submit Final report for Home Office review December 1, 2022 Submit Final report revisions based on USAID comments Week of January 2-10, 2022 Learning Event 1 TBD Learning Event 2 TBD USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 318 ANNEX 1. AVAILABLE DATA TABLE 5. AVAILABLE DATA Type Timeframe Quality Reliability Reference Baseline survey February 22-24, 2020 Includes baseline values for ABC+ that were used to establish key performance indicators (KPIs) and relevant context indicators Collected by the MEL team Baseline report June 2020 Data sources in each IRs July 2019- September 2020 Includes accomplishments for each IRs for Year 1 of the program (PY1) Collected by the MEL team Year 1-Annual Progress Report: July 2019- September 2020 Accomplishments In each IRs October 2020- September 2021 Includes accomplishments for each IRs for Year 2 of the program (PY2) Collected by the MEL team Year 2-Annual Progress Report: October 2020- September 2021 Accomplishments In each IRs May 2022 Includes accomplishments for each IRs for Year 3 of the program (PY3)-Q2 Collected by the MEL team Quarterly Progress Report￾Quarter 2 FY_2022 as of May 6 319 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 2: SITE AND RESPONDENT SELECTION CRITERIA SAMPLING PROCEDURES TO BE USED IN SELECTING TEACHERS AND SCHOOL HEADS FOR THE SURVEY USAID ABC+ Evaluation Goal: Sampling of Teachers and School Heads for the Survey Type of Sampling: Stratified Random Sampling 1. Determine the population where the samples will be taken from Population (Universe): ___________________________ Accessible population: ___________________________ Note: Are there areas that are not accessible for any reasons? If any, they will be removed from the sampling frame. 2. Determine the total number of divisions, schools, and participating teachers from the identified population. Total number of divisions: _________ Total number of schools: _________ Total number of participating teachers: _________ 3. Calculate the sample size using Cochran’s formula with a 95% confidence interval and 3% margin of error. 4. Identify the geographic location of each division/province and determine the corresponding proportion of participating teachers. Distribute the desired sample size based on the proportion of each geographic category. Table 1 shows the distribution of all participating teachers and desired sample size by geographic category. 5. Randomly select the schools until the desired number of teachers-respondents per division/area is achieved. The school heads from the selected schools will automatically be the respondents to represent the school. An online randomizer (i.e., random.org) will be used. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 320 Geographic Category No. of Schools / School Heads No. of Male Teachers No. of Females Teachers All Teachers Prop. Male Prop. Female Sample Size by Area Sample Size (Male) Sample Size (Female) COHORT 1: Region V Albay Camarines Norte Legazpi City Sorsogon Region VI Escalante City Iloilo City Negros Occidental Roxas City COHORT 2: Region V Iriga City Camarines Sur (Rinconada) Masbate Region VI Antique Capiz COHORT 3 Region V Tabaco City 321 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Geographic Category No. of Schools / School Heads No. of Male Teachers No. of Females Teachers All Teachers Prop. Male Prop. Female Sample Size by Area Sample Size (Male) Sample Size (Female) Camarines Sur (Central Bikol) Naga City Catanduanes Region VI Bago City Cadiz City Kabankalan Iloilo Prov. La Carlota City Silay City Passi City Bacolod City Sagay City San Carlos City TOTAL USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 322 ANNEX 3: SAMPLING FRAME (INDIVIDUALS, INSTITUTIONS, ORGANIZATIONS) Number of Schools Female Male Total Trained Teachers Cohort 1 Region V Albay 176 1117 101 1218 Camarines Norte 253 1917 141 2058 Legazpi City 44 475 17 492 Sorsogon 109 609 60 669 Region VI Escalante City 27 247 13 260 Iloilo City 54 764 17 781 Negros Occidental 500 3505 237 3742 Roxas City 46 360 20 380 Cohort 2 323 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV BARMM Cotabato City 11 91 6 97 Maguindanao I 10 8 2 10 Maguindanao II 95 312 36 348 SGA 50 245 37 282 Region V Camarines Sur 153 973 65 1038 Iriga City 40 303 14 317 Masbate 127 720 43 763 Tabaco City 40 365 19 384 Region VI Antique 485 2211 145 2356 Cohort 3 Region VI USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 324 Bacolod City 46 120 4 124 La Carlota City 24 156 7 163 Sagay City 49 119 3 122 San Carlos City 28 36 36 Total 2364 14652 986 15638 NEAP-accredited training courses 325 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 4: TARGET SAMPLE (INDIVIDUALS, INSTITUTIONS, ORGANIZATIONS) Unit of analysis Institutions Organizations Individuals Total Method Central ROs (Cohort 1, 2 BARM M) Cohort 3 (intervention) DOs TEIs LGU (with CPEs) Private Sector School heads Teachers Curriculum developers TEI faculty Surveys 2 3 10 17 2 Midline participants Midline participants Bloom and Let’s Read Platform trainees BSU WVSU TBD KII 2 3 X 17 2 X X X X X 24 FGD X X 1 group (reps of 10 cities) 8 FGDs with school heads and teachers (7 languages + Maguindanawn) 1 FGD with BSU and WVSU 10 OH workshops Level 3 Level 2 Level 1 (2 groups) Level 1 (6 groups: 5 participants for each group) 3 Case study X 3 (1 school/I R) 1 X X X X X X X X X 4 Classroom observations X X X N= 68 (17 x 4 X X X X X X X X 68 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 326 Unit of analysis Institutions Organizations Individuals Total Method Central ROs (Cohort 1, 2 BARM M) Cohort 3 (intervention) DOs TEIs LGU (with CPEs) Private Sector School heads Teachers Curriculum developers TEI faculty classe s: K￾3) Document review Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes TBD 327 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Projected participants and their backgrounds KII ● DepEd leaders who are in key RO and DO positions and MTB-MLE Coordinators in the Regions, Divisions, and schools who are knowledgeable and actively involved in teacher training, materials development, and activities for their sustainability. ● ABC+ leaders and staff who played key roles in the interventions. ● Teachers from small, big, and GIDCA schools who participated in the interventions and were part of the baseline and midline assessments. FGD ● -2 DepEd Teachers who were part of the baseline and midline assessment, representatives of school size or type (big schools, small schools, and GIDCA), and have participated in several ABC+ interventions. ● 1-2 DepEd Leaders who may represent either the Division, District, or school and/or an MTB-MLE focal person in any level. ● 1-2 ABC+ leader or focal person who are knowledgeable and directly involved in interventions. ● 1-2 Parents or HLPs, one of whom assisted a male child, and the other female in their studies, have received ABC+ SRMs, and have participated in ABC+ activities for remote learning (e.g., text blast, webinars, etc.) ● 1-2 institutional partners who occupy a key position or have been involved or knowledgeable in ABC+ interventions. ● 1-2 community members representing LGU, business groups, advocacy groups, civil society, etc. who have participated in ABC+ activities. If relevant or applicable, they should come from cities who have piloted an MSE program. If circumstances permit, an FGD involving Grade 2 learners can be conducted. The projected participants and their backgrounds are as follows: ● 5-7 Grade 2 male and female learners from ABC+ beneficiary schools, whose teachers have participated in various training, have received SRMs and other kits/packages, and whose respective HLPs have been supported by ABC+ activities. Parents of these children should give consent for their children’s FGD participation. Children should also give their assent in verbal and written means. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 328 ANNEX 5: DATA COLLECTION TIMELINE 329 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 6: SURVEY FOR TEACHERS Good day! We would like to let you know that as a recipient of the Advancing the Basic Education in the Philippines Program (ABC+) of USAID, you were selected to take part in the survey. This activity is part of the evaluation study of the ABC+ Program. Before you decide to participate in this study it is important that you understand why this is being done and what it will involve. Please read the following information carefully and ask the evaluator if there is anything that is not clear or if you need more information. PROCEDURE In this survey, we will be asking you some questions about the nature of your participation and outputs attributed to the ABC+ Program. The survey can be completed in 30 minutes. Your participation in this evaluation study can provide insights on how the ABC+ program can move forward to improve instruction, quality, and access to reading materials, and system capacity to sustain efforts at the early-grade level. CONFIDENTIALITY For the purposes of this study, your identity will be anonymous. Every effort will be made by the evaluator to preserve your confidentiality. CONTACT INFORMATION If you have questions at any time about this study, you may contact the evaluator whose contact information is provided below: Randolf S. Sasota randolf.sasota@gmail.com VOLUNTARY PARTICIPATION Your participation in this study is voluntary. It is up to you to decide whether or not to take part in this study. If you decide to take part in this study, you will be asked to sign a consent form by entering your name in the consent section of this google form. CONSENT I have read and understood the provided information. I understand that my participation is voluntary and that I am free to withdraw at any time, without giving a reason and without cost. I understand that I will be given a copy of this consent form. We request you to answer all the questions. For the questions with a response scale, please tick the answer based on your degree of agreement or disagreement. For open-ended questions, please type your answer in the provided box. Once you finish, please click the “DONE” button and your responses will be automatically sent to us. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 330 331 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV A. General information Name: _________________ Date Name (Optional): Age: Sex at birth: Male/Female (For discussion) Highest Educational Attainment: Institutional Affiliation: Town/City: Province: Region: Designation: From 2019 to the present, did you participate in ABC+ activities? o Yes o No If yes, in what year? o 2019 o 2020 o 2021 o 2022 What was the form of training? o Virtual o In-person What was your role in the training you engaged in? - A trainee - A trainer - Others: _____ (please indicate) USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 332 333 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Relevance: 1. Which of the following statements best describes how ABC+ compares with other professional development in which you have participated during the past six months? ● ABC+ was more useful than other professional development on MTB-MLE that I have participated in. ● ABC+ was about the same as other professional development that I have participated in ● ABC was less useful other than the professional development that I have participated in ● I don’t have an opinion. ● I don’t have an opinion because I have not participated in any other professional development in the last six months ● Others (Please indicate) 2. List at least three adjustments that ABC+ implemented in their activities/programs to account for the changes brought by the COVID-19 pandemic. 3. How have your students responded to ABC+ supplementary reading materials (SRMs)? 4. How about parents and your co-teachers? Provide any feedback on children's response to ABC+ SRMs from them. 5. In a scale of 1-3, where 1 is the highest and 3 is the lowest, rate the extent to which ABC+ considered trainees' capacity and individual situations in each of the adjustments you listed above. 6. Do you know of any DepEd policy (local, regional, or national) on early-grade instruction that have been modified or enhanced because of ABC+ activities/programs? If yes, explain briefly how ABC+ activities influenced the policy modification or enhancement. 7. Recall the time before participating in the ABC+ program and compare it now in terms of your early-grade instructions and delivery. To what extent has the program been useful in improving early-grade instructions and delivery since the implementation? ● ABC+ program has been very useful in improving early-grade instructions and delivery. ● ABC+ program has been somewhat useful in improving early-grade instructions and delivery. ● ABC+ program has been a bit useful in improving early-grade instructions and delivery. ● ABC+ program has not been useful at all in improving early-grade instructions and delivery. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 334 Effectiveness 1. Indicate the extent to which the ABC+ met your professional learning and teaching needs in early-grade instruction ● It addressed my professional learning and teaching needs completely. ● It addressed some of my professional learning and teaching needs. ● It did not address my professional learning and teaching needs. ● This professional development did not help much because I was already familiar with this topic. 2. To what extent has ABC+ training helped you improve your instruction in the following: To a great extent To a moderate extent To a limited extent Literacy mathematics Integration of SEL Integration of GESI? 3. As you plan your early-grade instruction, are there any activities especially intended for boys? ● Yes ● No 4. If your answer above is yes, can you describe it in brief? 5. As you plan your early-grade instruction, are there any activities especially intended for girls? ● Yes ● No 6. If your answer above is yes, can you describe it in brief? 7. How did you implement instruction during the pandemic? Check all that apply: ● Use of modules, implemented with the help of home learning partners ● Online learning using moodle and conferencing platforms (Zoom, Google Meet) ● House to house visit to teach students face-to-face ● Others: (please specify) 8. To what extent were you still able to implement skills and knowledge acquired from ABC+ training using the instructional delivery approach used during the pandemic? 335 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ● To a great extent ● To a moderate extent ● To a limited extent 9. If your answer above is to a great extent/a limited extent, provide the factors which limited the application of acquired skills in the used delivery mode. 10. Around how many school days were lost in your school due to COVID-19 restrictions in the following school years? ● 2020-2021 ● 2021-2022 11. Give 1-3 suggestions for ABC+ in terms of their conduct of teacher training (for literacy instruction, math instruction, GESI, etc.) 12. Give 1-3 suggestions for ABC+ in terms of their conduct of materials development (training and workshops, distribution of materials, etc.) 13. In your community or school, do boys perform better in school than girls or is it the other way around? Give possible reasons as to why this could be the case in your context. 14. In a scale of 1-3 where 1 is the highest and 3 is the lowest, rank the following ABC+ interventions based on the extent of their contribution to improving your instruction for learners' basic skills in the early-grades, especially in reading in the MT and transition to Filipino and English. ● Strengthening capacity development for teachers and school heads ● Developing and providing access to quality reading materials ● Strengthening system capacity of the Department of Education in your regions/divisions to sustain efforts of ABC+ (i.e., collaboration with NGOs and LGU to improve the system and policies) Please explain ranking. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 336 Sustainability 1. How likely will you apply continuously what you have learned in ABC+ even after the program implementation? __ Very Unlikely ___ Unlikely _____ Neutral ____Likely ______Very Likely 2. Which of the following statements best describes the likelihood that you will apply what you learned in ABC+? ● I have already practiced it in my classroom. ● I have already practiced in my classroom, and it seemed to work well. ● I have already practiced in my classroom, and it seemed to work well. I will keep on practicing it. ● I have already practiced in my classroom, but it was not appropriate for my learners. ● I look forward to practicing in my classroom in the next few weeks ● I look forward to practicing in my classroom later this academic year ● I would like to practice, but I don’t have sufficient materials that I need. ● I would like to practice it, but I don't have the chance. ● I don’t think that these things will work with my learners. ● "I would like to practice it, but ________ (give reason here)." 3. Which of the following support systems do you need in order to sustain the good practices that you acquired from ABC+ training programs? ● Continues quality professional teacher training in the next 2-3 years ● Accessibility of materials through radio, TV based instructions and other social media platforms. ● Availability of most appropriate instructional technology for their developmental stage, grade and subject matters needs. ● Availability of quality supplementary reading materials (big and small books) ● Availability of quality learning materials for early-grade learning ● Availability of infrastructures (e.g., buildings, roads, power supplies) and facilities (e.g., libraries, computer rooms, learning hubs/spaces) for K-3 to strengthen their acquired literacy skills in their mother tongue (MT) especially to learners with special needs. ● School/Division-level policy to collaborate with the local government unit (LGU) for improving early-grade reading, math, and social and emotional skills 337 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ● School/Division-level policy to collaborate with non-government organizations (NGOs), for improving early-grade reading, math, and social and emotional skills 4. Generally, what are parents and community members' attitudes towards Mother Tongue-Based Multilingual Education (MTB-MLE)? ● Very positive and highly supportive ● Moderately positive and supportive ● Unaware of MTB-MLE and passive ● Moderately negative and skeptical ● Very negative and highly opposed 5. To what extent have ABC+ activities involved community stakeholders (LGUs, NGOs, elders, etc.) for the improvement of early-grade instruction? 6. How have ABC+ activities supported your school in reaching out to last mile and GIDCA schools? 7. What are your observations about the coverage that ABC+ activities have received in the news media, for example in newspapers and online news, social media platforms, and during television and radio broadcasts? ● inspiring ● informative ● encouraging ● no idea ● others (please specify: ________) 8. Provide examples of instances when you saw news reporters and journalists covering ABC+ activities in your school or district. 9. Based on your training experience, what component of ABC+ programs need policy recommendations to ascertain sustainability? ● Collaboration with stakeholders ● Budget allocation ● Time allocation for the training ● Ratio of teaching and learning materials ● Quality of teaching and learning materials ● Integrating socio-emotional learning skills USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 338 ● Mainstreaming or highlighting gender sensitivity and girls’ empowerment ● Promoting social inclusion when it comes to religious belief and indigenous people’s practices ● Providing access to internet through wireless and wired means and other options 10. List 3-5 existing and/or potential “barrier/s” that could hinder the sustainability of ABC+ programs once the USAID engagement ends. 11. List three to five best practices that emerged during the implementation of ABC+ programs that could possibly be sustained by your school/division after the USAID engagement ends. 339 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 7: SURVEY FOR SCHOOL HEADS SURVEY School heads Good day! We would like to let you know that as a recipient of the Advancing the Basic Education in the Philippines Program (ABC+) of USAID, you were selected to take part in the survey. This activity is part of the evaluation study of the ABC+ Program. Before you decide to participate in this study it is important that you understand why this is being done and what it will involve. Please read the following information carefully and ask the evaluator if there is anything that is not clear or if you need more information. PROCEDURE In this survey, we will be asking you some questions about the nature of your participation and outputs attributed to the ABC+ Program. The survey can be completed in 10-15 minutes. Your participation in this evaluation study can provide insights on how the ABC+ program can move forward to improve instruction, quality, and access to reading materials, and system capacity to sustain efforts at the early-grade level. CONFIDENTIALITY For the purposes of this study, your identity will be anonymous. Every effort will be made by the evaluator to preserve the confidentiality of your answers. CONTACT INFORMATION If you have questions at any time about this study, you may contact the evaluator whose contact information is provided below: Randolf S. Sasota randolf.sasota@gmail.com VOLUNTARY PARTICIPATION Your participation in this study is voluntary. It is up to you to decide whether or not to take part in this study. If you decide to take part in this study, you will be asked to sign a consent form by entering your name in the consent section of this google form. CONSENT I have read and understood the provided information. I understand that my participation is voluntary and that I am free to withdraw at any time, without giving a reason and without cost. I understand that I will be given a copy of this consent form. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 340 We request you to answer all the questions. For the questions with a response scale, please tick the answer based on your degree of agreement or disagreement. For open-ended questions, please type your answer in the provided box. Once you finish, please click the “DONE” button and your responses will be automatically sent to us. A. General information Name (Optional): Age: Sex at birth: Male/Female (For discussion) Highest Educational Attainment: Institutional Affiliation: Town/City: Province: Region: Designation: From 2019 to the present, did you participate in ABC+ activities? o Yes o No If yes, in what year? o 2019 o 2020 o 2021 o 2022 What was the form of training? o Virtual o In-person What was your role in the training you engaged in? A trainee A trainer Others: _______ (please indicate) Relevance 341 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV 1. Which of the following statements best describes how ABC+ compares with other professional development in which you have participated during the past six months? ● ABC+ was more useful than other professional development that I have participated in. ● ABC+ was about the same as other professional development that I have participated in ● ABC was less useful other than the professional development that I have participated in ● I don’t have an opinion. ● I don’t have an opinion because I have not participated in any other professional development in the last six months 2. To what extent does ABC+ activities align with existing DepEd policies and/or programs on: ● early-grade instruction ● access to quality materials ● improved education capacity 3. As you observed teachers at the K-3 level, are there any activities especially intended for boys? ● Yes ● No 4. If your number 2 answer is yes, can you describe it in brief? 5. As you observe teachers at the K-3 level, are there any activities especially intended for girls? ● Yes ● No 6. If your number 3 answer is yes, can you describe it in brief? 7. List at least three adjustments that ABC+ implemented in their activities/programs to account for the changes brought by the COVID-19 pandemic. 8. In a scale of 1-3, where 1 is the highest and 3 is the lowest, rate the extent to which ABC+ considered beneficiaries' capacity and individual situations in each of the adjustments you listed above. 9. Do you know of any DepEd policy (local, regional, or national) on early-grade instruction that have been modified or enhanced because of ABC+ activities/programs? ● Yes ● No 10. If yes, explain briefly how ABC+ activities influenced the policy modification or enhancement. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 342 Effectiveness 1. Indicate the extent to which the ABC+ met your professional learning needs in supervision of early-grade instruction ● It addressed my professional learning needs completely ● It addressed some of my professional learning needs ● It did not address my professional learning needs ● This professional development did not help much because I was already familiar with this topic 2. To what extent has ABC+ training helped you in mentoring K-3 teachers to improve their instruction in the following: To a great extent To a moderate extent To a limited extent Literacy mathematics Integration of SEL Integration of GESI? 3. In your institution, in which of the following aspects are ABC+ activities most successful? ● Strengthening capacity development for teachers and school heads ● Developing and providing access to quality reading materials ● Strengthening the system capacity of the Department of Education in your regions/divisions to sustain efforts of ABC+ 4. Please provide specific examples in your #3 answer and why these are the most effective/successful. 5. In a scale of 1-3 where 1 is the highest and 3 is the lowest, rank the following ABC+ interventions based on the extent of their contribution to improving learners' basic skills in the early-grades, especially in reading in the MT and transition to Filipino and English ● Strengthening capacity development for teachers and school heads ● Developing and providing access to quality reading materials ● Strengthening system capacity of the Department of Education in your regions/divisions to sustain efforts of ABC+ 343 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV 6. How did you implement instruction during the pandemic? Check all that apply: ● Use of modules, implemented with the help of home learning partners ● Online learning using moodle and conferencing platforms (Zoom, Google Meet) ● House to house visit by teachers who teach students Face to face ● Others: (please specify) 7.To what extent were teachers still able to implement skills and knowledge acquired from ABC+ training using the instructional delivery approach used during the pandemic? ● To a great extent ● To a moderate extent ● To a limited extent 8. If your answer above is to a great extent/a limited extent, provide the factors which limited the application of acquired skills in the used delivery mode. 9. Around how many school days were lost due in your school due to COVID-19 restrictions in the following school years? ● 2020-2021 ● 2021-2022 10. Give 1-3 suggestions for ABC+ in terms of their conduct of teacher training (for literacy instruction, math instruction, GESI, etc.) 11. Give 1-3 suggestions for ABC+ in terms of their conduct of materials development (training and workshops, distribution of materials, etc.) 12. In your community or school, do boys perform better in school than girls or is it the other way around? Give possible reasons as to why this could be the case in your context. Sustainability USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 344 1. Which of the following statements best describes the likelihood that you will apply what you learned in ABC+? ● I have already practiced in my school. ● I have already practiced in my school, and it seemed to work well. ● I have already practiced in my school, but it was not appropriate for our K-3 learners. ● I look forward to practicing in my school in the next few weeks. ● I look forward to practicing in my school later this academic year. ● I would like to practice, but we don’t have the materials that we need in our school. ● I don’t think that these things will work with our K-3 learners. 2. Please rate your readiness in continuing ABC+ activities after the project ends ● Strongly ready ● Ready ● Not ready ● Strongly not ready ● Do not know 3. What plans do you have in order to sustain/replicate/continue the program after the project ends? 4. To what extent has ABC+ activities involved community stakeholders (LGUs, NGOs, elders, etc.) for the improvement of early-grade instruction? ● To a great extent ● To a moderate extent ● To a limited extent 5. How have ABC+ activities supported your school in reaching out to last mile schools and GIDCA? 6. What could be the major challenges or obstacles in implementing your plans after the project ends? How do you intend to overcome them? 7. What are some of your predictions about how the educational ideas, methods, and materials developed under ABC+ will continue to be used after the project ends? 8. Generally, what are parents and community members' attitudes towards Mother Tongue-Based Multilingual Education (MTB-MLE)? ● Very positive and highly supportive ● Moderately positive and supportive ● Unaware of MTB-MLE, passive ● moderately negative and skeptical ● very negative and highly opposed 9. What are your observations about the coverage that ABC+ activities have received in the news media, for example in newspapers and online news, social media platforms, and during television and radio broadcasts? Provide examples of when you saw news reporters and journalists covering ABC+ activities in your school or district. ● inspiring 345 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ● informative ● encouraging ● no idea ● others (please specify: ________) Our survey about ABC+ is now concluded. Your comments today have been very valuable and useful for our evaluation of the project. Thank you for providing your time to assist us. The USAID evaluation report of ABC+ will later be publicly released. We will contact you in the future if we need to make some clarifications on your responses. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 346 ANNEX 8: KEY INFORMANT INTERVIEW QUESTIONNAIRE Date: I. Preliminaries 1. Introduction of CLAimDev a. Recording b. Screenshot of KII participant 2. Introduction of participant a. Giving of name b. Giving of agency or affiliation c. Giving of location: school, division, region 3. Reading of house rules (RFAs: in-charge) To allow our conversation to flow more freely, I’d like to go over some ground rules. . There are no “wrong answers,” just your observations and opinions. a. As much as possible please answer each question. b. Whatever is shared during the discussion/interview will remain confidential. f. Let me know about your questions now. Are there any questions? KII with DepEd Leaders (From Region, Division, District, or School) ● What Regional, Division, or District policy (e.g., memorandum or order, regulations, programs, etc.) are ABC+ activities on SRM development and distribution in your locale aligned with? Please name these policies and explain how ABC+ activities are aligned with them? ● How have ABC+ activities enriched your processes for materials development (i.e., contextualization and localization) and distribution? In what ways has ABC+ enhanced your operations for this dimension? ● What are the considerations to contextualize the designed/developed SRMs 347 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ● How many SRMs has your school/division/region received from ABC+ interventions? Who has these SRMs or who is accountable for them? ● How frequent do the teachers and students use the designed/developed SRMs ● What feedback have you received from teachers, parents/HLPs and their children about the quality and effectiveness of these SRMs? Describe how the children responded to the SRMs and their use? Did the SRMs affect children’s interest in reading in any way? ● Given all the ABC+ activities that you engaged with, which group do you think benefited the most (i.e., teachers, learners, school heads)? Why this group? You may rank the groups and explain why. ● In what ways are the School, Division, and Region ensuring the strides made in the following through ABC+ activities will be sustained long after the program has concluded o Development of quality SRMs through workshop, QA, inclusion of gender-fair/SEL themes, etc. o Implementation and monitoring SRMs to ensure the proper inclusion of gender￾fair/SEL themes o Partial evaluation of intervention and revision of SRMs for continuous improvement and quality assurance o Accessibility of quality SRMs through funding and ensuring that GIDCA schools are provided with them. o Continuous Professional development and training of teachers and curriculum developers What issues and concerns have you experienced from teachers, curriculum developers/content writers about the quality, effectiveness, and sustainability of the intervention? KII with Teachers ● What Regional, Division, or District policy (e.g., memorandum or order, regulations, programs, etc.) are ABC+ activities on SRM development and distribution in your locale aligned with? Please name these policies and explain how ABC+ activities are aligned with them? ● How have ABC+ activities enriched your processes for materials development and distribution? In what ways has ABC+ enhanced your operations for this dimension? ● How have ABC+ activities enriched your instruction and delivery? In what ways has ABC+ enhanced your teaching skills? Do you get to apply what you have learned from ABC+ training in your classes? If yes, what are the effects of your application of skills and knowledge? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 348 ● How many SRMs has your school/division/region received from ABC+ interventions? Who has these SRMs or who is accountable for them? ● What feedback have you received from co-teachers, parents/HLPs and their children about the quality of these SRMs? ● How different are these SRMs from the ones you had? Why? ● Have you integrated these SRMs in your instruction? If yes, how? What effect, if any, did the SRMs have on your instruction? Did you notice any change in the students when you started using the SRMs? Tell us about it. ● How did children respond to the use of SRMs? ● In what ways are the School, Division, and Region ensuring the strides made in the following through ABC+ activities will be sustained long after the program has concluded o Development of quality SRMs through workshop, QA, inclusion of gender-fair/SEL themes, etc. o Implementation and monitoring SRMs to ensure the proper inclusion of gender￾fair/SEL themes o Partial evaluation of intervention and revision of SRMs for continuous improvement and quality assurance o Accessibility of quality SRMs through funding and ensuring that GIDCA schools are provided with them. o Continuous Professional development and training of teachers and curriculum developers o Review of existing policy on the inclusion of gender-fair/SEL themes for future reference What issues and concerns have you experienced from parents and their children during the implementation of the ABC+ intervention (Sustainability, IR 1,2,3) ABC+ Partners (LGU-LSB, Private Sector and NGOs) ● Have you collaborated with DepEd for materials development previously? Based on your experience, How have ABC+ activities enriched DepEd processes for materials development and distribution? In what ways has ABC+ enhanced operations for this dimension? 349 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ● How have you collaborated with DepEd for teacher training and professional development previously? Based on your experience, How have ABC+ activities enriched DepEd processes for teacher training and professional development, policy review of education system capacity and commitment to manage and implement interventions? In what ways has ABC+ enhanced operations for this dimension? ● How was your collaboration experience with ABC+ with regard to teacher training and professional development to improve the instruction and delivery of intervention activities and to possible policy review of education system capacity and commitment to manage and implement interventions? ● Would you be interested to continue with these activities and policy review of education system capacity and commitment to manage and implement interventions with DepEd even after the conclusion of ABC+ engagement? Why or why not? ● Has DepEd expressed any interest in engaging you again in future activities similar to the ones you participated in for ABC+? ● What issues and concerns have you experienced with DepEd as LGU and NGO partners during the implementation of the ABC+ interventions USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 350 ANNEX 9: FOCUS GROUP DISCUSSION PROTOCOL Date: I. Preliminaries 1. Introduction of CLAimDev a. Recording b. Screen shot of FGD participants 2. Introduction of participants a. Giving of name b. Giving of agency or affiliation c. Giving of location: school, division, region 3. Reading of house rules (RFAs: in-charge) To allow our conversation to flow more freely, I’d like to go over some ground rules. a. In this FGD, only one person speaks at a time. b. Everyone doesn’t have to answer every single question. c. Whatever is shared during the discussion will remain confidential. d. There are no “wrong answers,” just different opinions. e. Since our time is limited there may be a need to ask you to stop and change topics from time to time. f. Let me know about your questions now. Are there any questions? III. FGD QUESTIONS RELEVANCE § How have ABC+ activities contributed to the enhancement of policies on instruction and learning outcomes in kindergarten to grade 3? § How have ABC+ activities for professional training and materials development enriched your instruction? 351 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV § What materials development and distribution policies and processes that were already in place before were enhanced or made more efficient or more effective by ABC+ interventions? § Do you know of any DepEd policy (local, regional, or national) on early-grade instruction that has been modified or enhanced because of ABC+ activities/programs? If yes, explain briefly how ABC+ activities influenced the policy modification or enhancement. On the effect of the pandemic § How has COVID-19 pandemic affected ABC+ activities in terms of: improving early-grade instruction and delivery, improving access to quality materials, and improving education capacity and managing and implementing interventions supporting early-grade instruction? § List at least three adjustments that ABC+ implemented in their activities/programs to account for the changes brought by the COVID-19 pandemic. § In a scale of 1-3, where 1 is the highest and 3 is the lowest, rate the extent to which ABC+ considered trainees' capacity and individual situations in each of the adjustments you listed above. § Around how many school days were lost due to COVID-19 restrictions in the past school years? § To what extent were you still able to implement skills and knowledge acquired from ABC+ training using the instructional delivery approach used during the pandemic? § How did home learning partners (HLPs) implement the ABC+ program? § How was the learning of children from Geographically Isolated, Disadvantaged and Conflict￾Affected Areas (GIDCA) monitored, especially during the pandemic? What is the status of their learning compared to their peers? On Gender Equality and Social Inclusion (GESI) § Are the needs of boys and girls addressed in classroom instruction? Cite examples or instances where this was observed. § How have ABC+ activities promoted gender equality and social inclusion (GESI) in your instruction delivery? § What was your experience in teacher training and delivery of instruction that highlighted GESI? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 352 § Have you integrated GESI in your lessons? If yes, how did you do that? How would you describe the experience of integrating it with instruction or introducing it to learners? How did the girls respond to it? How about the boys? § Do you have any teaching activity that you implement to boys only or to girls only? Why? Where did you get that idea? What other strategies or activities of yours differentiate boys from girls? What made you decide to do it that way? How did the boys and girls respond in these activities? EFFECTIVENESS On Program Contribution § Compared with your previous programs in early-grade literacy, what is/are unique features, if any, of ABC+ in the improvement of early-grade instruction and delivery, access to materials, and capacity of your division/school? § To what extent have skills and knowledge acquired from ABC+ activities been applied in instruction? § How useful are knowledge and skills gained from ABC+ activities in supporting your schools’ instruction and quality and accessibility of materials in the early-grades? § Recall the time before participating in the ABC+ program and compare it now in terms of your early-grade instruction and delivery. To what extent has the program been useful in improving early-grade instruction and delivery since implementation? § How have ABC+ activities improved teachers’ and school heads’ capacity to design, deliver, or supervise instruction in early-grade levels? § Give 1-3 suggestions for ABC+ in terms of their conduct of teacher training (for literacy instruction, math instruction, GESI, etc.). § What learning support do children need to make learning more effective? How about parents/guardians, what support system do they need to make children’s learning more effective and meaningful? On Supplementary Reading Materials (SRMs) § Did all enrolled children here receive an SRM package? What is in the package? How and when were these delivered? Was delivery timely? Did schools with learners from GIDCA in the area also receive packages? How did the school ensure that learners from GIDCA had access to SRMs? § Do you think SRMs are properly contextualized and appropriate to the learning needs of children? Why? 353 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV § How did you and/or the children use the SRMs and how often? § Do the children use the SRMs on their own? If yes, how? § What effect did SRM use have in the teaching process? In student response (engagement and learning)? § Do the SRMs have any effect or change on children’s reading or learning how to read? Why do you associate these effects or changes to SRM use? § Are SRMs easy to use and follow by the parents/ home learning partners (HLPs)? § What are the three things that you and/or the children like best about the SRMs? Why? § What are three opportunities for SRM improvement? What else would you like to see in the future for SRMs? § What are three challenges for SRM improvement? GESI in SRMs § How have ABC+ activities promoted GESI in materials development? § Do boys and girls find the ABC+ SRMs stimulating and relevant to their age-group and gender? § What was your experience in using SRMs that highlight GESI? Did the materials appeal to boys only, girls only, or both? What makes you say that? Socio-emotional learning (SEL) in SRM § Did the SRMs feature social or cultural values? If yes, which stories are these and what values were presented? How did the children respond to these stories? Do you think they understood the values presented? What makes you say that? SUSTAINABILITY § What ABC+ activities would you like to continue even after the engagement with ABC+ has ended? Why? § List three to five best practices that emerged during the implementation of ABC+ programs that could possibly be sustained by your school/division after the engagement ends. What plans does your school have for sustaining or replicating ABC+ activities after the project ends? § What could be the major challenges or obstacles in implementing the plan of your school after the project ends? How do you intend to overcome them? § Generally, what are parents and community members' attitudes towards Mother Tongue￾Based Multilingual Education (MTB-MLE)? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 354 § Has the region, division, or school made policy, budget, staffing, etc. provisions for continuing ABC+ activities for teacher training, SRM development and distribution, and policy review on the implementation and continuous improvement of MTB-MLE even after the program has ended? Discuss each provision. § To what extent has ABC+ activities involved community stakeholders (LGUs, NGOs, elders, etc.) for the improvement of early-grade instruction? § What emerging partnerships that were initiated by your region/division/school may support the sustainability of gains of learners in early-grade literacy? § What component/s of the ABC+ program needs policy recommendations to ascertain sustainability? § What are three opportunities for teacher professional development? What else would you like to see in the future for teacher professional development? § What are three challenges in the policy implementation and improvement? What else would you like to see in the future in terms of education system capacity and commitment to manage ABC+ intervention programs? § What are three opportunities for policy implementation and improvement? § Provide examples of instances when you saw news reporters and journalists covering ABC+ activities in your school or district. What impressions did these give you? Why? 355 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 10: SRM EVALUATION TOOL FOR TEACHERS USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 356 357 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 358 ANNEX 11: ABC + GANTT CHART Task Name Estimated Duration Estimated Start Estimated Finish Adjusted dates with this Inception Report July August September October November Pre-field Phase USAID issues STRIDE on-demand task order 0 days 1-Jul-22 2-Jul-22 Issue STTA contract to Team Leader (TL) 0 days 13-Jul-22 15-Jul-22 Team Leader Updates SOW and Schedule 3 days 18-Ju1 20-Jul Issue STTA contracts to 2 Specialists 0 days 18-Jul 22 July Document review 5 days 25-Jul-22 29-Jul-22 4-10-Aug-22 Evaluation team planning session 2 days 1-Aug-22 3-Aug-22 11-12-Aug￾22 In-brief meeting with USAID 0 days 5-Aug-22 5-Aug-22 31-Aug-22 Inception Phase Initial consultations with IP and major stakeholders (Scheduled on September 6 and 9, 2022) 2 days 24 Aug-22 25-Aug-22 6-9-22 359 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Task Name Estimated Duration Estimated Start Estimated Finish Adjusted dates with this Inception Report July August September October November Document review and analysis of reports and studies from implementing partners and clients 10 days 13-Aug-22 22-Aug-22 Prepare inception report with evaluation design, methodology, tools, and schedule (September 15, 2022) 8 days 25-Aug-22 30-Aug-22 Submit inception report to COP for review 2 days 31-Aug-22 1-Sep-22 15-16-Sep￾22 Submit inception report to USAID for approval 0 days 5-Sep-22 5-Sep-22 17-Sep-22 Revise and submit inception report (September 19, 2022) 2 days 6-Sep-22 7-Sep-22 18-19-Sep￾22 Data Gathering and Analysis Pilot tools and finalize 7 days 8-Sept-22 15-Sep-22 Schedule data collection interviews and arrange logistics 2 days 16-Sep-22 17-Sep-22 Data collection 19 days 19-Sep-22 10-Oct-22 Presentation on Preliminary Impression 1 day 11-Oct-22 11-Oct-22 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 360 Task Name Estimated Duration Estimated Start Estimated Finish Adjusted dates with this Inception Report July August September October November Data processing and analysis 12 days 12-Oct-22 25-Oct-22 USAID Out-brief 1 day 26-Oct-22 26-Oct-22 Consultation meeting with IPs 1 day 28-Oct-22 28-Oct-22 Draft Report Writing Phase Draft report preparation 10 days 29-Oct-22 9-Nov-22 Submit draft report for PI and COP review 0 days 10-Nov-22 11-Nov-22 Comments on draft report from PI and COP 5 days 14-Nov-22 18-Nov-22 Final Report Writing Phase Final report revisions 7 days 21-Nov-22 28-Nov-22 Submit final report for PI and COP review 0 days 29-Nov-22 29-Nov-22 361 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV Task Name Estimated Duration Estimated Start Estimated Finish Adjusted dates with this Inception Report December January Submit final report for Home Office review 0 days 1-Dec-22 1-Dec-22 Submit final report draft to USAID 0 days 8-Dec-22 8-Dec-22 USAID Comments on final report 5 days 9-Dec-22 14-Dec-22 Final report revisions based on USAID comments 7 days 2-Jan-23 10-Jan-23 COP and Home Office report finalization 5 days 11-Jan-23 17-Jan-23 Final report submission 0 days 17-Jan-23 17-Jan-23 Final report approval 5 days 18-Jan-23 24-Jan-23 Final report submitted to the DEC; data uploaded to DDL 0 days 24-Jan-23 24-Jan-23 Learning and Dissemination Phase Learning Event 1: Findings, conclusions, and recommendations with one (1) learning event’s target audience will be USAID/Philippines (#1) 5 days TBD TBD Learning Event 2: Dissemination of Findings and Lessons Learned with another learning event’s target audience will be USAID’s external stakeholders (#2) 3 days TBD TBD Submit supplementary report 6 days TBD TBD USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 362 ANNEX 12: OH WORKSHOP QUESTIONNAIRE Relevance Effectiveness Sustainability IR 1 Teacher Training What have been the outcomes of ABC+ interventions on DepEd policies (memos, plans, priorities, etc.) on teacher training, SRMs, student learning (literacy, math, and SEL)? What specific ABC+ activities led to these outcomes? What have been the outcomes of ABC+ teacher training? What specific ABC+ activities led to these outcomes? What have been the outcomes of ABC+ activities to ensure local ownership and continued support for teacher training, SRM development and systems capacity? What specific ABC+ activities led to these outcomes? IR 2 Materials (SRMs) What have been the outcomes of ABC+ activities for materials development? ABC+ produced SRMs? What specific ABC+ activities led to these outcomes? IR 3: System Capacity What have been the outcomes of ABC+ interventions on DepEd capacity and processes and community participation for the improvement of teacher training, SRM development, and student learning? What specific ABC+ activities led to these outcomes? What have been the outcomes of ABC+ activities for developing DepEd systems and capacity and community participation to support early-grade instruction? What specific ABC+ activities led to these outcomes? 363 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 13: MOST SIGNIFICANT CHANGE STORY COLLECTION FORM USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 364 365 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 366 367 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 14: MSC GUIDELINES FOR STORY VERIFICATION AND SELECTION Story Verification Guide for Level 2 Participants (ABC+ and DepEd trainers/focal persons) § Is the storyteller a confirmed beneficiary of ABC+ activities and outputs? § Has the storyteller been a beneficiary long enough to express the details given in the story? § Are the details (when and where) in the story consistent with ABC+ activities? § Have you heard of similar stories reported by other beneficiaries before? § Is the story credible? Do the reported experiences portray what is likely or realistic based on the context or nature of the intervention and other factors? Significance Criteria for Level 2 and Level 3 Participants § How close is the story’s illustration of events to ABC+ goals? § How impactful is the change described in the story? § Does the story reflect the intended and unintended goals of ABC+? § How well does the story capture stakeholder perspective vis-à-vis implemented ABC+ activities? USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 368 ANNEX 15: VIEWING TOOL FOR RECORDED CLASSES 369 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 370 ANNEX 16: DOCUMENTS FOR DESK REVIEW Existing Policies (Orders and Memoranda) and Program of the Department of Education on MTB￾MLE at the National Level Issuance Date Title Relevant IRs in Document HB No. 2188 July 20, 2022 An Act Suspending the Implementation of the Use of Mother Tongue as the Medium of Instructions for Kindergarten to Grade 3 DO 21, s. 2019 August 22, 2019 Policy Guidelines on the K to 12 Basic Education Program IR1, IR2 DO 55, s. 2015 December 8, 2015 Utilization of Language Mapping Data for Mother Tongue-Based Multilingual Education (MTB-MLE) Program Implementation IR2 DO 28, s. 2013 July 5, 2013 Additional Guidelines to DepEd Order No. 16, s. 2012 [Guidelines on the Implementation of the Mother Tongue-Based Multilingual education (MTB- MLE)] IR1, IR2 DO 16, s. 2012 February 17, 2012 Guidelines on the Implementation of the Mother Tongue-Based Multilingual Education (MTB-MLE) IR1, IR2 DO 74, s. 2009 July 14, 2009 Institutionalizing Mother Tongue-Based Multilingual Education (MTB-MLE) IR1, IR2 Department of Education Orders and Memoranda on Materials Issuance Date Title Relevant IRs in Document 371 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV DO 44, s. 2013 September 26, 2013 Moratorium on the Procurement of Supplementary Reading, Reference, and other Instructional Materials (IMs) IR2 DO 13, s. 2012 February 3, 2012 Guidelines on the Allocation, Delivery, and Distribution of Instructional Materials (IMs) to Support the K to 12 Curriculum IR2 DO 90, s. 2011 November 10, 2011 Guidelines on the Utilization of Downloaded Funds for Materials Development and Production of Mother Tongue-Based Multilingual Education (MTB-MLE) Program IR2 Department of Education Orders and Memoranda on Teacher Training/Professional Development Issuance Date Title Relevant IRs in Document DO 24, s. 2020 September 7, 2020 National Adoption and Implementation of the Philippine Professional Standards for School Heads IR3 DO 42, s. 2017 August 11, 2017 National Adoption and Implementation of the Philippine Standards for Teachers IR3 DO 12, s. 2015 April 10, 2015 Guidelines on the Early Language, Literacy, and Numeracy Program: Professional Development Component IR1, IR3 DM 35, s. 2012 February 17, 2012 Training of Trainers for Kindergarten Teachers on Mother Tongue-Based Multilingual education (MTB-MLE) IR1, IR3 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 372 DO 18, s. 2011 February 23, 2011 Guidelines on the Conduct of Mother Tongue￾Based Multilingual Education Training IR1 DM 495, s. 2010 December 8, 2010 National Training of Trainers on the Mother Tongue-Based Multilingual Education (MTB￾MLE) Program for Phases III and IV IR1 DO 18, s. 2011 February 23, 2011 Guidelines on the Conduct of Mother Tongue￾Based Multilingual Education Training IR1 Department of Education Reports and Documents Title Relevant IRs in Document School Improvement Plans (SIP) of selected schools and case study schools IR1, IR2, IR3 Regional Basic Education Plan of Regions V, VI, and BARMM IR1, IR2, IR3 Division Education Development Plans IR1, IR2, IR3 Learning Recovery Plans of Regions, Divisions, and Schools IR1, IR2, IR3 Existing Policies (Orders and Memoranda) and Program of the Department of Education on MTB￾MLE at Regional Level Issuance Date Title Relevant IRs in Document DO 21, s. 2019 August 22, 2019 Policy Guidelines on the K to 12 Basic Education Program IR1, IR2 373 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV General Policies DO 55, s. 2015 December 8, 2015 Utilization of Language Mapping Data for Mother Tongue-Based Multilingual Education (MTB-MLE) Program Implementation IR2 DO 28, s. 2013 July 5, 2013 Additional Guidelines to DepEd Order No. 16, s. 2012 [Guidelines on the Implementation of the Mother Tongue-Based Multilingual education (MTB- MLE)] IR1, IR2 DO 16, s. 2012 February 17, 2012 Guidelines on the Implementation of the Mother Tongue-Based Multilingual Education (MTB-MLE) IR1, IR2 DO 74, s. 2009 July 14, 2009 Institutionalizing Mother Tongue￾Based Multilingual Education (MTB￾MLE) IR1, IR2 Materials DM 441, s. 2019 (Region V, Division of Naga) October 1, 2019 Guidelines and Process for LRMDS Assessment and Evaluation of Locally Developed and Procured Materials IR2, IR3 ABC+ Documents Title Relevant IRs in Document Fiscal Year 2022 Quarter 2 Report January 1-March 31, 2022 ABC+: Advancing Basic Education in the Philippines IR1, IR2, IR3 Fiscal Year 2021 Quarter 3 Report April 1-June 30, 2021 IR1, IR2, IR3 Project Year 1 Annual Progress Report July 2019-September 2020 IR1, IR2, IR3 Project Year 2 Annual Progress Report October 2020-September 2021 IR1, IR2, IR3 USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 374 ABC+: Advancing Basic Education in the Philippines Baseline Report June 2020 IR1, IR2, IR3 Country Development Cooperation Strategy (CDCS) November 25, 2019 - November 24, 2024 IR1, IR2, IR3 Monitoring, Evaluation Learning Plan Advancing Basic Education in the Philippines (ABC+) IR1, IR2, IR3 GESI Plan Update IR1, IR2, IR3 Draft ABC+ MEL Plan September 27, 2019 IR1, IR2, IR3 Draft ABC+ MEL Plan September 27, 2019, unannotated IR1, IR2, IR3 USAID/Philippines Basa Pilipinas Program Early-grade Reading Assessment Final Evaluation Report: 2018 IR1, IR2, IR3 ABC+ Gender Equity and Social Inclusion Accomplishments 2020-2021 IR1, IR2, IR3 ABC+ Gender Indicators Highlighted IR1, IR2, IR3 ABC+ Gender and Inclusion Plan 2019 IR1, IR2, IR3 Program Cycle Guidance How to Note Evaluation Statements of Work IR1, IR2, IR3 375 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 17: ANTICIPATED SCHEDULES AND LOGISTICAL ARRANGEMENTS USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 376 377 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 24. EVALUATION TEAM Ivy P. Mejia is a researcher and evaluation methodologist. Her work revolves around pre-service and in-service training, lesson study, program evaluation, data science in education using R-language, and qualitative data analysis using NVivo collaboration cloud and ATLAS.ti. She is a member of the National Council for Measurement in Education, a group of measurement scientists in education. As an evaluation methodologist, she served as an evaluation consultant to United Nations Educational, Scientific and Cultural Organization (UNESCO) and USAID activities. She was a technical specialist in the Science Technology Research, and Innovation for Development (STRIDE) performance evaluation. At present, she is a science education specialist and the co-chair of the Knowledge and Research Management Group at the UP-NISMED. Dr. Mejia is also a Professorial Lecturer at the University of the Philippines (UP) Diliman College of Education teaching educational assessment courses. In the ABC+ evaluation, she was the team leader, advising and spearheading the team on the study implementation, data analysis, and presentations to USAID and other stakeholders. She also steered the team in interpreting results related to IR1, the effectiveness-related evaluation questions, and related data on professional developments of program recipients. Romylyn A. Metila is a researcher focusing on multilingual education, particularly, the Philippines’ MTB-MLE. She was part of several commissioned studies, including the multiphase study, Understanding Best Practices in MTB-MLE in the Philippines, by the Assessment, Curriculum, and Technology Research Centre (ACTRC), and the Review of Component 1 of the Learning, Equity and Accountability Program Support (LEAPS) Project, by World Bank. Her dissertation was about the design of a context-based MTB-MLE evaluation model. In the ABC+ evaluation, she served as the expert on MTB-MLE-related activities and results. She also guided the team in interpreting results related to IR2, relevance-related evaluation questions, and related data on literacy and materials development. Arlyne C. Marasigan is a researcher with published works on MTB-MLE and education for sustainability. She worked with UP-Center for Integrative and Development Studies (UP-CIDS) under the Islamic Studies Program and published a discussion paper entitled Sustainability Concerns of Madrasah Education Program: Basis for Philippine Islamic and Madrasah Education Policy Review. Her dissertation, A Critical Analysis of Environmental Pedagogies in the Philippine Teacher Education: A Call for a Paradigm Shift towards Sustainability, was about the environmental sustainability framework for teacher education. From 2011 to 2019, she was also a member of the Education Co￾Research for Sustainability, a Grant-in-Aid for Scientific Research in Japan’s National Institute for Educational Policy Research. For ABC+ evaluation, she served as the expert on policy who advised the evaluation team on MTB￾MLE related activities and results. She was also the one assigned to steer the team in interpreting results related to IR3, Sustainability-related Evaluation questions, and data related to system capacity and commitment to manage and implement intervention supporting effective early-grade instruction. Randolf S. Sasota is a senior lecturer of the UP College of Education since 2012, handling courses in statistics applied to education, including the methods, data analysis, tests and measurements, test USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 378 and scale development, and psychometric theory. He is also a Supervising Science Research Specialist and Officer-in-Charge in the Science and Technology Manpower Education Research and Promotions Division of the Department of Science and Technology - Science Education Institute (DOST-SEI). This division is mandated to promote science to the youth, conduct research and evaluation studies on science education and science and technology human resource development, and provide management information systems in the Institute. He is currently taking up his PhD in Education Major in Research and Evaluation. In the ABC+ Performance Evaluation, he served as the data analyst. Saddam "Sam" Bazer is a licensed professional teacher and a senior technical adviser of Save the Children Philippines for their education projects in the Bangsamoro Autonomous Region in Muslim Mindanao (BARMM). Sam completed his PhD in Education at the School of Education, Shaanxi Normal University, Xi'an City, China, as a scholar. He finished his Bachelor of Elementary Education major in English at Mindanao State University (MSU) – Iligan Institute of Technology as cum laude. He earned his Master of Arts in Educational Technology at UP Diliman with the highest distinction and as a Presidential scholar. He received an Academic Excellence Award for his graduate studies and is a member of the Pi Gamma Mu International Honor Society. Sam has developed expertise in education research and education technology for government and intergovernmental organizations. His extensive experience includes supporting education innovations such as the MTB-MLE, literacy and reading interventions, and the open educational resources development. He engaged with different institutions and organizations including MSU, UP, ACTRC, Education Development Center, DepEd Central Office, and the Ministry of Basic, Higher, and Technical Education (MBHTE) in BARMM. In the ABC+ Evaluation, Dr. Bazer was a regional field assistant who led the data gathering and coordination with the MBHTE and supported the team in the data analysis and report writing. Maria Sheila ``Shy” Luna-Oliverio is a Science Research Specialist of the DOST-SEI Capacity Building Program in Science and Mathematics Education stationed at West Visayas State University La Paz, Iloilo City. She is in-charge of the graduate scholarship program assisting PhD and Master in Biology, Mathematics and Physics scholars. Shy has a lot of monitoring and evaluation experience in her field of work. Being based in Western Visayas, she served as the regional field assistant of the ABC+ Performance Evaluation. She supported the evaluation team in the data collection, liaise with regional stakeholders, and performed tasks for the completion and success of the evaluation. Jovy A. Lisbog is a development worker who is engaged with a local NGO based in Region V, which is championing community-based inclusive development programs including inclusive education. She has relevant monitoring and evaluation experience in the development sector. Jovy served as a regional field assistant of the ABC+ Performance Evaluation in the Bicol Region. She supported the evaluation team in data collection, liaised with stakeholders, and performed tasks in the completion of the evaluation. Ferlie Rose Ann Famaloan has a Masters in Cultural Anthropology and Development Studies from Katholieke Universiteit Leuven in Belgium with a distinction of cum laude. Ferlie was a magna cum laude graduate with a Bachelor of Arts in Anthropology from the University of the Philippines Diliman and took up Master of Science in Medical Anthropology at the University of the Philippines Manila. She has extensive experience in research and development. Her current research projects and interests include research and development, project management, health systems research, mental health, public health, and medical anthropology. As part of the ABC+ evaluation team, she 379 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV was a research assistant handling stakeholder coordination, scheduling data collection, and compiling and laying out technical reports and presentations. USAID.GOV ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES | 380 ANNEX 25. REFERENCES 1. Duterte, S.Z. (2023). Basic Education Report 2023 [PowerPoint slides]. Department of Education. Office of the Secretary, Philippines. 2. Mertens, D. M. (2017). Mixed methods design in evaluation. Thousand Oaks: CA: Sage. 3. Babbie E. Survey Research Methods. Belmont, Calif: Wadsworth; 1990. 4. The World Bank. (2014). Outcome-based Field Learning Guide. Washington, D.C. (p.13) 5. Yin, R.K. (2003). Case Study Research: Design and Methods. Sage. Thousand Oaks, California. 6. USAID Learning Lab: Monitoring in the Program Cycle 7. Research Triangle Institute. (2019). Monitoring, Evaluation, and Learning Plan. 8. Department of Education Region V. (2022). Temporary suspension of classroom observations at all levels. 381 | ABC+ MID-TERM PERFORMANCE EVALUATION - ANNEXES USAID.GOV ANNEX 26. NONDISCLOSURE AND CONFLICT OF INTEREST AGREEMENTS 1. Ivy Mejia- Evaluation Advisor/Team Leader Mejia_Non Disclosure Agreement.pdf 2. Romylyn Metila - Technical Specialist for MTB￾MLE Metila_Non Disclosure Agreement.pdf 3. Arlyne Marasigan - Technical Specialist for Policy in Education Marasigan_Non Disclosure Agreement.pdf 4. Randolf Sasota - Data Analyst Sasota_Non Disclosure Agreement.pdf 5. Saddam Bazer - Field Regional Assistant Bazer_Non Disclosure Agreement.pdf 6. Maria Sheila Oliverio - Field Regional Assistant Oliverio_Non Disclosure Agreement.pdf 7. Jovy Lisbog - Field Regional Assistant Lisbog_Non Disclosure Agreement.pdf 8. Ferlie Famaloan - Project Assistant Famaloan_Non Disclosure Agreement.pdf