This publication was produced for review by the United States Agency for International Development (USAID). It was prepared by Emily Morris, Yvette Tan, EnCompass LLC and its partner MSI, a Tetratech company, for the Data and Evidence for Education Programs (DEEP), Contract No. GS-10F-0245M. The views expressed herein do not necessarily reflect the views of USAID. TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY August 2021 PHOTO | USAID/CAMBODIA Suggested Citation Morris, Emily and Yvette Tan. Toolkit for Designing a Comprehensive Distance Learning Strategy. Washington DC: USAID, 2021. Acknowledgements Lead Authors: Emily Morris, Yvette Tan. Rebecca Rhodes, Saima Malik, Rebeca Martinez, and Elena Walls provided notable contributions and guidance. The authors would like thank the COVID-19 Education Response Task Team, several members of USAID’s Center for Education (including Brooke Estes, Bethany Johnson, Joshua Josa, Leah Maxson, Olga Merchan, Nina Papadopoulous, Heather Risley, and Nina Weisenhorn), USAID Regional Bureau colleagues, and a number of staff in USAID Missions for their critical insights at various stages of the development of this product. The authors would also like to thank Mary Burns and Anna Farrell for their review, comments, and contributions. Thank you to Denisse Leatxe (Txiki Txoko) for graphic development. All icon graphics by Txiki Txoko, with some modified icons from The Noun Project. i | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV CONTENTS ACRONYMS AND ABBREVIATIONS.......................................................................................................III BACKGROUND TO THE TOOLKIT ........................................................................................................1 INTRODUCTION TO DISTANCE LEARNING ......................................................................................3 What is distance learning?...................................................................................................................................... 3 How does distance learning work? ...................................................................................................................... 3 Who is the intended audience for distance learning?...................................................................................... 4 What are the overarching instructional goals for distance learning?........................................................... 6 What are the key requirements for successful distance learning?................................................................ 6 What are the different ways technology may be used in distance learning?.............................................. 7 OVERVIEW OF A COMPREHENSIVE DISTANCE LEARNING STRATEGY.................................10 What is a comprehensive distance learning strategy?....................................................................................10 How does a comprehensive distance learning strategy support resilience to crises?...........................13 What are the phases for operationalizing a comprehensive distance learning strategy?......................14 OUTLINE OF A COMPREHENSIVE DISTANCE LEARNING STRATEGY DOCUMENT..........16 COMPONENT 1: ANALYZE THE EXISTING DISTANCE LEARNING LANDSCAPE AND DEVELOP A STRATEGIC VISION.............................................................................................................18 COMPONENT 2: DESIGN A DISTANCE LEARNING APPROACH...............................................36 COMPONENT 3: DEVELOP A MONITORING, EVALUATION, AND LEARNING PLAN......43 COMPONENT 4: CREATE A BUDGET ..................................................................................................53 CONCLUSION...............................................................................................................................................60 REFERENCES ...................................................................................................................................................61 ANNEXES ........................................................................................................................................................67 Annex A. Roadmap for Measuring Distance Learning...................................................................................67 Annex B: Key terms and definitions................................................................................................................... 68 LIST OF EXHIBITS Exhibit 1: The Four Components in Designing a Comprehensive Distance Learning Strategy...................... 2 Exhibit 2: Distance Learning Modalities........................................................................................................................ 3 Exhibit 3: Distance Learning Basics............................................................................................................................... 4 Exhibit 4: Distance Learning Modalities by Education Levels.................................................................................. 5 Exhibit 5: Overarching Instructional Goals of Distance Learning .......................................................................... 6 Exhibit 6: Key Requirements for Successful Distance Learning.............................................................................. 7 ii | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Exhibit 7: SAMR Technology Integration Definitions and Examples ..................................................................... 8 Exhibit 8: Planning the Comprehensive Distance Learning Strategy...................................................................12 Exhibit 9: The Phases of Operationalizing a Comprehensive Distance Learning Strategy.............................14 Exhibit 10: Illustrative Outline for a Distance Learning Strategy .........................................................................16 Exhibit 11: Guiding Questions for an Inclusive Vision............................................................................................18 Exhibit 12: Links between Distance Learning Vision and Goals...........................................................................20 Exhibit 13: Distance Learning Goals and Instructional Goals: Illustrative Examples.......................................22 Exhibit 14: Distance Learning as a Component of Broader ICT Strategies......................................................27 Exhibit 15: Steps for Integrating Institutional Capacity-Building into the Distance Learning Strategy (Steps 1 and 2) .............................................................................................................................................................................. 31 Exhibit 16: Contextual Factors, Descriptions, and Distance Learning Examples.............................................33 Exhibit 17: Guiding Questions for Selecting a Modality .........................................................................................36 Exhibit 18: Key Questions in Considering Which Modalities to Implement.....................................................37 Exhibit 19: Decision Tree for Choosing How Content Will Be Delivered (Remotely, In Person, or Hybrid)............................................................................................................................................................................... 40 Exhibit 20: Steps for Developing an Institutional Capacity-Building Plan (Step 3)...........................................42 Exhibit 21: Guiding Questions for Developing a MEL Plan....................................................................................43 Exhibit 22: Aims of Monitoring and Evaluating Distance Learning.......................................................................45 Exhibit 23: Illustrative Conceptual Framework Showing Alignment Between Vision, Goals, and Subgoals .............................................................................................................................................................................................. 47 Exhibit 24: Three Domains of Measuring Distance Learning ................................................................................48 Exhibit 25: Metrics for the Technological Landscape and Access to Programming and Content, by Domain .............................................................................................................................................................................................. 48 Exhibit 26: Metrics for Measuring Quality of Programming and Materials, by Domain...................................49 Exhibit 27: Decision Tree for Determining Data Collection Technology .........................................................51 Exhibit 28: Guiding Questions for Creating a Budget.............................................................................................53 Exhibit 29: Roadmap for Measuring Distance Learning..........................................................................................67 LIST OF BOXES Box 1. Guiding Principles for Designing a Comprehensive Distance Learning Strategy .................................. 2 Box 2. Three Main Principles of UDL........................................................................................................................... 9 Box 3. Guiding Principles for Designing a Comprehensive Distance Learning Strategy ................................10 Box 4. USAID Definitions of Resilience in Education.............................................................................................13 Box 5. Examples of Marginalized Groups.................................................................................................................. 25 iii | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV ACRONYMS AND ABBREVIATIONS ECD Early childhood development EIE Education in emergencies COVID-19 Coronavirus disease 2019 HIV/AIDS Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome IAI/IRI Interactive Audio Instruction/Interactive Radio Instruction ICA Institutional Capacity Assessment ICT Information communications technology ICT4E Information communications technology for education INEE Inter-agency Network for Education in Emergencies IVR Interactive voice response LGBTQIA+ Lesbian, gay, bisexual, transgender, queer, intersex, asexual, + stands for all of the other sexualities, sexes, and genders not represented LMS Learning management system M&E Monitoring and evaluation MEL Monitoring, evaluation, and learning NGO Non-governmental organization SAMR Substitution, augmentation, modification, and redefinition SIM Subscriber Identification Module SMS Short Message Service TTI Teacher Training Institute UDL Universal Design for Learning UNDP United Nations Development Programme UNESCO United Nations Educational, Scientific and Cultural Organization USAID United States Agency for International Development 1 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV BACKGROUND TO THE TOOLKIT Distance learning is broadly defined as teaching and learning where educators and learners are in different physical spaces. The instructional goals of distance learning can range from serving as the main form of instruction to providing supplementary, complementary, or additional teaching and learning support. Whatever the instructional goals, distance learning can increase educational access and improve teaching and learning if designed intentionally, equitably, and inclusively. The purpose of this toolkit is to provide practical guidance (tools, examples, and resources) for designing a comprehensive distance learning strategy that covers an entire education sector or system. A comprehensive distance learning strategy responds to the needs of a context1 in the short-term and simultaneously lays out a medium- to long-term approach that reflects principles of universal design for learning (UDL) (see Box 2 for UDL principles). In addition, a comprehensive distance learning strategy can ensure that when crises or shocks occur, quality teaching and learning can continue remotely and that support mechanisms are in place when learners return to learning after emergencies. Finally, the comprehensive distance learning strategy can add to decision-makers’ resources for increasing their education systems’ resilience. The primary audience for this toolkit includes decision-makers and representatives from government education institutions responsible for developing and overseeing distance learning efforts. Donor institutions, USAID Missions, non-governmental organizations, civil society organizations, and other education stakeholders working on distance learning efforts may also find this toolkit useful. After using this toolkit, users will be able to: 1. Understand what distance learning is, why it is important, and how it works; and 2. Create a comprehensive distance learning strategy that meets the needs of learners and educators in a given context and builds the resilience of education systems. The toolkit is organized into the four components illustrated in Exhibit 1, and provides tools and action points that guide users in gathering, capturing, and analyzing relevant data and making decisions for developing their comprehensive distance learning strategy. Action points may be executed sequentially, simultaneously, or in any order, depending on the context and the extent to which distance learning programming already exists. 1 For this toolkit, “context” also refers to “country” depending on the geographical scope. 2 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Exhibit 1: The Four Components in Designing a Comprehensive Distance Learning Strategy Box 1. Guiding Principles for Designing a Comprehensive Distance Learning Strategy 1. Increase equitable access to distance learning opportunities, especially for marginalized learners. 2. Ensure high-quality and inclusive design, content, and programming. 3. Build capacity to plan, develop, and implement quality distance learning opportunities. These principles are based on ethical and evidence-based practice in developing educational strategies and are adapted from USAID’s Delivering Distance Learning in Emergencies publication and echo the cross-cutting considerations included in USAID’s Returning to Learning During Crises Toolkit. 3 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV INTRODUCTION TO DISTANCE LEARNING WHAT IS DISTANCE LEARNING? Distance learning is broadly defined as teaching and learning where educators and learners are in different physical spaces.2 Often used synonymously with distance education,3 distance learning takes place through one of four modalities: audio/radio, video/television, mobile phone, and/or online learning platforms. Printed texts frequently accompany these modalities and could also be a fifth modality in cases where technology is not used for teaching and learning, such as correspondence (or print￾based) education. This toolkit includes print-based learning as a fifth modality. No one distance learning modality is ideal for teaching all skills to all learners in all contexts. Therefore, a multi-modal approach is emphasized throughout this toolkit to ensure that content is accessible to learners and educators from different contexts, ages, and demographics. While educators and learners are typically located in different physical spaces, in some cases, distance learning can take place through in-person groups with a trained educator or facilitator who supports learners in completing their lessons, or a caregiver who supervises a pod or group of learners. Examples include in-person listener groups for Interactive Radio Instruction (IRI) and Interactive Audio Instruction (IAI) or pods (small groups meeting together) of primary school learners engaged in online learning and overseen by a caregiver.4 Distance learning can also be facilitated through a combination of remote and in-person activities (e.g., individual, remote reading instruction at home plus in-person instruction), referred to as hybrid distance learning. HOW DOES DISTANCE LEARNING WORK? Distance learning can be implemented in different ways. Learners may participate asynchronously, completing the learning activities in their own time using one of the five modalities (e.g., learners watch a video of a science experiment in their own time). Learners may also participate synchronously, where 2 Content in this “Introduction” section is based on USAID’s Delivering Distance Learning in Emergencies publication. 3 Some assert that “distance education” emphasizes the processes of teaching as well as learning, but others argue that teaching is also implied in “distance learning.” This toolkit uses the term distance learning to be consistent. 4 USAID's Interactive Audio and Radio Instructional Resources page provides guidance on using IRI and IAI as well as multiple links to country-specific IRI and IAI programming. Exhibit 2: Distance Learning Modalities 4 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV teaching and learning happen simultaneously in real time, either in an online space (e.g., virtual classroom connected to a learning management system or LMS5) or through a concurrent broadcast (e.g., educational television program). Finally, learners may participate in a mixed approach that employs both synchronous and asynchronous learning. Learning occurs using one or more modalities and educators can communicate with learners through a variety of technologies and means. For example, educators, learners, and families may communicate through phone calls, web-based social media, text messages, emails, and/or a messaging platform in an LMS. Distance learning can be assessed through individual or group work, and for formative or summative purposes, as will be discussed in greater detail under Component 3. Exhibit 3: Distance Learning Basics This image was adapted from www.pearsoned.com/tips-moving-class-online-quickly/. WHO IS THE INTENDED AUDIENCE FOR DISTANCE LEARNING? Distance learning can be designed for learners of all ages and education levels, from preschool learners to adults who are enrolled in higher education or nonformal learning programs. Distance learning can 5 Learning management systems are the software programs used for facilitating all aspects of online learning, such as Google Classroom. 5 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV also be designed for educators, caregivers, and family members. However, as detailed below, some modalities and programming are better suited for certain age groups or education levels than others.6 • Radio/audio programming transmitted through a broadcast requires less technological literacy than other modalities. In general, radio/audio programming has been tried and studied for younger audiences as well as out-of-school youth and adult learners such as in-service educators. Audio programming, such as podcasts, can also be used for nonformal learners as well as secondary and tertiary learners. • Television/video programming has been designed for all ages, including educational television broadcasts for preschoolers and interactive video lectures for secondary and tertiary learners. • Mobile phone programming varies greatly in design and technological literacy required but can be designed for all age levels. • Online learning is more commonly used for older youth and adults who have higher levels of technological literacy and reading ability, but it has been increasingly used for primary school-aged children as well. • Correspondence (print-based) or self-directed learning is used with all ages of learners with the content tailored to literacy levels. Exhibit 4 shows the suitability of the different modalities to the different education levels. Note that each modality can be accessed on different devices. For example, online learning can be accessed on a mobile phone, computer, or tablet, and radio programming can be listened to on a radio, mobile phone, computer, or tablet. Exhibit 4: Distance Learning Modalities by Education Levels All modalities need to be accompanied with appropriate levels of support when used with young children, low-literacy users, and learners with disabilities. Users may need support in operating the devices (e.g., mobile phone, tablet, computer), as well as in accessing programming and content (e.g., software programs or interfaces). 6 See USAID Delivering Distance Learning in Emergencies: Literature Review for evaluative studies by different age groups and levels. 6 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV WHAT ARE THE OVERARCHING INSTRUCTIONAL GOALS FOR DISTANCE LEARNING? Distance learning can enhance teaching and learning in any context in order to ensure flexibility, continuity, and differentiated instruction for learners and educators. Distance learning can be designed for any one of the following: the main form of instruction, complementary instruction, supplementary instruction, or additional educational support (see Exhibit 5 below and sub-component 1A to identify and detail instructional goals and subgoals). Exhibit 5: Overarching Instructional Goals of Distance Learning WHAT ARE THE KEY REQUIREMENTS FOR SUCCESSFUL DISTANCE LEARNING? Distance learning (and teaching) require access to technology and high-quality, inclusive instructional design. Technology includes devices, infrastructure, and technology interfaces or software, and, for this toolkit, can also include correspondence (print-based) education content. Instructional design refers to the technological knowledge (how to use devices), content knowledge (what is being taught), and pedagogical knowledge (how content is being taught) conveyed through distance learning. Instructional design takes into consideration diverse groups of learners and encompasses remote, in-person, or hybrid distance learning formats.7 7 Adapted from Judith Harris, Punya Mishra, and Matthew J. Koehler, “Teachers’ Technological Pedagogical Content Knowledge and Learning Activity Types: Curriculum-Based Technology Integration Reframed,” Journal of Research on Technology in Education 41, no. 4 (2009): 393–416, and Mary Burns, “For Want of a Good Theory: Considerations for Technology Integration in Well-resourced Schools.” In A Closer Look at Educational Technology, edited by M.A. Clausen. New York, NY: Nova 7 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Exhibit 6: Key Requirements for Successful Distance Learning For distance learning to be aligned with the guiding principles (Box 1), the technology devices used need to be accessible to all learners, including the most marginalized, and especially those with disabilities. Instructional design, content, and programming need to be high-quality, inclusive, and engaging for all learners. Furthermore, sufficient infrastructure should be in place alongside adequate technological, content, and pedagogical knowledge and expertise. Where infrastructure and expertise do not exist, they must be built. WHAT ARE THE DIFFERENT WAYS TECHNOLOGY MAY BE USED IN DISTANCE LEARNING? Distance learning falls under the larger fields of educational technology (EdTech) and information communications technology for education (ICT4E). As a field, EdTech examines the “process of analyzing, designing, developing, implementing, and evaluating the instructional environment and learning materials in order to improve teaching and learning.”8 ICT4E is often used synonymously with educational technology and emphasizes strengthening education systems through technology in addition to improving teaching and learning.9 EdTech encompasses all teaching and learning using technology in remote, in-person, and hybrid formats.10 EdTech can be integrated into distance learning in different ways and for different instructional goals. Technology may be used as a way of replicating in-person instruction (e.g., holding a virtual lecture when in-person classes are not feasible) or it may also be used as a critical tool to redefine learning and broaden the possibilities of what and how content is taught. 8 Serhat Kurt, “Educational Technology: An Overview,” Educational Technology, Educational Technology Consulting Services, November 18, 2015, https://educationaltechnology.net/educational-technology-an-overview/ 9 USAID, “Three Ps of ICT4E: Principles, Partnerships, and Programs,” EducationLinks, USAID, January 17, 2020, https://www.edu-links.org/learning/three-ps-ict4e-principles-partnerships-and-programs 10 Definitions for in-person, remote, and hybrid can be found in the first section of this introduction and in the discussion of sub-component 2B. 8 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV The Substitution, Augmentation, Modification and Redefinition (SAMR) framework11 (Exhibit 7) is one way to conceptualize the different ways technology is used to design meaningful distance learning. This toolkit draws on the SAMR framework because it provides a common language for thinking through how and why a given technology is being used in teaching and learning. Exhibit 7 lays out the four different ways technology can be integrated into teaching and learning. Exhibit 7: SAMR Technology Integration Definitions and Examples SAMR CATEGORY DEFINITION ADAPTED FOR DISTANCE LEARNING EXAMPLE OF DISTANCE LEARNING (VIDEO PROGRAMMING) ENHANCEMENT OF TEACHING AND LEARNING Substitution Technology acts as a direct substitute to in-person learning with no functional change. Educators deliver lessons as they would have in-person through virtual means (e.g., video conferencing or phone call). Augmentation Technology acts as a direct substitute to in-person learning with functional improvements. Educators record lessons they would deliver in an in￾person class but add images or animations to show examples, as well as simultaneous sign language interpretation and closed captioning for deaf and hard￾of-hearing learners. Note that closed captioning also helps learners who do not speak the language of instruction as their first language or struggle to comprehend diverse accents. TRANSFORMATION OF TEACHING AND LEARNING Modification Technology allows for significant task redesign. A video lesson is designed and scripted filming actual places, people, and phenomena that could not be captured through text or photo descriptions. Redefinition Technology allows for creation of previously inconceivable new tasks. A video lesson is designed and scripted filming actual places, people, and phenomena that could not be captured through text or photo descriptions. Real time features are integrated into programs where viewers can text or call a designated number and, through interactive voice recognition, get specialized information. Source: Puentedura 2010 and Burns 2019. Strategy development teams can use the SAMR framework when deciding on modalities and instructional goals. High-quality and inclusive distance learning programming and content require intentional planning and design to ensure that what is being taught is responsive to learners and educators in the intended learning environment. 11 Ruben Puentedura, “On the Impact of the SAMR Model,” Common Sense Education, Common Sense, January 2, 2020, https://www.commonsense.org/education/videos/ruben-puentedura-on-the-impact-of-the-samr-model 9 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Sometimes substitution or augmentation of in-person learning content is the necessary solution, especially as part of short-term strategies and quick pivots to distance learning.12 Modifying or redefining content takes time, and is often only feasible in medium- to long-term distance learning strategies. For example, when educational radio or video programs designed for in-person use in an early grade classroom with a trained educator are suddenly used in home environments during school closures, the programs may be used to substitute for in-person learning in the short-term. However, for the medium￾to long-term, they will need to be modified or even redefined, so caregivers can use them and they are accessible to all learners. Therefore, the end goal of a comprehensive distance learning strategy is to ensure that all activities integrate technology in a way that best supports teaching and learning for all learners, whether through substitution, augmentation, modification, or redefinition. All integration of technology for distance learning should take into account the principles of UDL, to the greatest extent possible (see Box 2): “UDL is an approach to instruction that prioritizes meeting the needs of learners with disabilities.”13 Learners with and without disabilities often experience greater academic success in classrooms with UDL. Box 2. Three Main Principles of UDL 1. Use multiple means of engagement, fostering learners’ motivations in a variety of ways. 2. Use multiple means of representation, presenting information to learners in a variety of ways. 3. Use multiple means of actions and expressions, enabling learners to express what they learn in a variety of ways. These principles are from USAID’s Universal Design for Learning to Help all Children Read: Promoting Literacy for Learners with Disabilities. The SAMR framework can be used as a resource for ensuring that planned distance learning interventions are not only accessible, but also engaging and inclusive for all.14 12 Mary Burns, “For Want of a Good Theory: Considerations for Technology Integration in Well-Resourced Schools,” in A Closer Look at Educational Technology, ed. Maria A. Clausen (New York, NY: Nova Science Publishers, 2019) and H.L., “SAMR Model: A Practical Guide for EdTech Integration,” Schoology, October 30, 2017, https://www.schoology.com/blog/samr-model-practical-guide-edtech-integration. 13 Hayes, Anne, Ann Turnbull, and Norma Moran, “Universal Design for Learning to Help All Children Read: Promoting Literacy for Learners with Disabilities (First Edition),” Global Reading Network, USAID, May 2019: vii. https://www.globalreadingnetwork.net/sites/default/files/media/file/Literacy%20for%20All%20Toolkit_0.pdf. 14 Further information on how ICT can be used to support the implementation of UDL principles can be found in USAID’s working paper: Using Information Communications Technologies to Implement Universal Design for Learning. 10 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV OVERVIEW OF A COMPREHENSIVE DISTANCE LEARNING STRATEGY The guiding principles of this toolkit ensure that distance learning strategies are designed to be comprehensive, equitable, inclusive, and feasible. A comprehensive distance learning strategy contributes to building the resilience capacities of education systems, thereby ensuring that when crises or shocks occur and in-person learning is disrupted, quality teaching and learning can continue remotely. Box 3. Guiding Principles for Designing a Comprehensive Distance Learning Strategy 1. Increase equitable access to distance learning opportunities, especially for marginalized learners. 2. Ensure high-quality and inclusive design, content, and programming. 3. Build capacity to plan, develop, and implement quality distance learning opportunities. These principles are based on ethical and evidence-based practice in developing educational strategies and are adapted from USAID’s Delivering Distance Learning in Emergencies publication. WHAT IS A COMPREHENSIVE DISTANCE LEARNING STRATEGY? A comprehensive distance learning strategy takes into account both short-term distance learning needs and more medium- to long-term plans and initiatives.15 A comprehensive distance learning strategy defines: • Instructional goals (overarching and detailed) for the use of distance learning; • Technologies that will be used (single or multiple modalities); • How technology will be integrated; • Intended groups that will be engaged in distance learning, especially the most marginalized; • Approaches and methods for monitoring and evaluating teaching and learning; and • Costs for implementing a comprehensive distance learning strategy. 15 The terms and short-, medium-, and long-term strategies were adapted from the USAID Return to Learning During Crises Toolkit. 11 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV SHORT TERM DISTANCE LEARNING STRATEGY MEDIUM TO LONG TERM DISTANCE LEARNING STRATEGY A short-term distance learning strategy is a temporary plan that is employed only until in￾person teaching and learning can resume or a challenge in an education system can be addressed. Under a short-term distance learning strategy, educational opportunities are provided as a temporary measure to a specific group(s) of learners, whose learning has been disrupted by a particular event or circumstance. A short-term strategy may be suitable in a crisis or emergency when in-person learning has been disrupted, or in non-emergency circumstances like while a community is building or repairing a new school and students are temporarily without a physical space. A medium- to long-term distance learning strategy is an intentional and integral part of an education system or teaching and learning activities. Under a medium-to-long term distance learning strategy, high-quality and inclusive distance learning is provided to learners on an ongoing basis. A medium- to long-term strategy considers how distance learning can be used to support all different educational levels of learners and their progress toward different instructional goals. While it is difficult to quantify the exact duration of a short-, medium-, or long-term strategy, as this will vary greatly depending on context, it is important to distinguish between whether a given distance learning intervention is a temporary and reactive solution to an event or circumstance, or a more intentional and ongoing approach. Comprehensive distance learning strategies allow decision-makers to plan for using distance learning modalities to increase educational access and quality for populations on a continuous basis. As demonstrated in Exhibit 8, some users of this toolkit may start planning for the medium- to long term first, while others may first start with planning for distance learning in the short-term, as necessitated by a shock or acute crisis. Ideally, medium- to long-term planning for distance learning should be put into place before a short-term strategy ends or before learners return to in-person learning.16 16 See the USAID Return to Learning During Crises Toolkit for guidance on this process. 12 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Exhibit 8: Planning the Comprehensive Distance Learning Strategy Adapted from The Brookings Institution and U.S. Department of Education, Office of Elementary and Secondary Education. While planning for (and implementing) distance learning, it is important to set up enabling conditions that can support instructional delivery in learning institutions and/or homes. Among the enabling conditions for distance learning shown in Exhibit 8, technological infrastructure must first and foremost be in place, and educators must have the technological, content, and pedagogical knowledge to implement inclusive and high-quality distance learning. This requires working with education stakeholders and institutions (educators and administrators) to ensure that there is the capacity to implement distance teaching and learning. Additionally, it is critical to work with community and health institutions to ensure the wellbeing of learners, educators, and families who will be engaged in distance learning. For example, in the case of early childhood education, teaching and learning may be implemented collaboratively by education, community, and health institutions to ensure that children are safe, nourished, and able to meet developmental expectations in their classes. 13 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV HOW DOES A COMPREHENSIVE DISTANCE LEARNING STRATEGY SUPPORT RESILIENCE TO CRISES? The USAID Return to Learning During Crises Toolkit states that “short-term responses can contribute to building transformative resilience capacities both during and after a crisis.”17 Furthermore, it explains that “distance learning, alternative education opportunities, and other education innovations should continue to be developed, tested, adapted, and scaled in order for the education system to better respond during future crises.”18 Box 4 outlines USAID’s definition of resilience in education. Box 4. USAID Definitions of Resilience in Education19 Resilience capacities are the “assets, skills, knowledge, resources, and networks that are used to anticipate and deal with the consequences of shocks or stressors in a way that reduces their overall impacts.”20 There are three forms of resilience capacities. • Absorptive: The ability of individuals, households, communities, or institutions to minimize exposure and sensitivity to shocks and stressors through preventive measures and appropriate coping strategies to avoid permanent, negative impacts. • Adaptive: The ability of individuals, households, communities, or institutions to make informed choices and changes in livelihood and other strategies in response to longer-term social, economic, and environmental change. • Transformative: The ability of communities and institutions to establish an enabling environment for systemic change through their governance mechanisms, policies and regulations, cultural and gender norms, community networks, and formal and informal social protection mechanisms. Planning for medium- and long-term distance learning helps prepare education systems to sustain learning in the face of shocks and acute crises. In addition, distance learning capacities can enhance the education system’s ability to serve as a resource to disseminate life-saving information, strengthen social and human capital, reduce chronic vulnerability, and promote inclusive growth.21 17 Boisvert, Kayla, Nina Weisenhorn, and Jamie Bowen. “Returning to Learning during Crises: Decision-making and Planning Tools for Education Leaders.” USAID, 2020, https://www.edu-links.org/resources/returning-learning￾during-crises-toolkit. 18 Boisvert, Weisenhorn, and Bowen 2020. 19Dr. Ritesh Shah, “Transforming Systems in Times of Adversity: Education and Resilience,” Education in Crisis and Conflict Network, USAID, 2019, https://www.eccnetwork.net/resources/transforming-systems-times-adversity￾education-and-resilience-white-paper. 20Shah 2019, p.6. 21Shah 2019. 14 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV ADDITIONAL RESOURCES • Transforming Systems in Times of Adversity: Education and Resilience White Paper • USAID and INEE’s Checklist for Information Communications Technology Interventions to Support Education in Crises and Conflict Settings WHAT ARE THE PHASES FOR OPERATIONALIZING A COMPREHENSIVE DISTANCE LEARNING STRATEGY? Whether in an emergency or non-emergency context, there are five phases for operationalizing distance learning: 1) planning and conducting a needs assessment; 2) planning and designing the distance learning approach; 3) building educators’ and learners’ capacity; 4) setting up monitoring and evaluation (M&E) systems; and 5) implementing distance learning. The components (and sub-components) outlined in this toolkit are the guidance for completing the planning phases one and two. Exhibit 9 illustrates how the four components of designing a comprehensive distance learning strategy guide the first two phases of planning (see USAID Delivering Distance Learning in Emergencies: An Overview). Exhibit 9: The Phases of Operationalizing a Comprehensive Distance Learning Strategy 15 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV In Component 1 of this toolkit, users analyze the distance learning landscape for the first phase of planning. In Component 2, users design a distance learning approach. In Component 3, users develop a monitoring, evaluation, and learning (MEL) framework. In Component 4, users create a budget that guides planners through the second phase of planning. The components outlined in this toolkit are designed to help those developing comprehensive distance learning strategies to operationalize distance learning initiatives that will ultimately increase their education system’s efficacy and resilience. ADDITIONAL RESOURCES • USAID’s Delivering Distance Learning in Emergencies: A Review of Evidence and Best Practices • USAID’s Delivering Distance Learning in Emergencies: An Overview • EDC’s Distance Education for Teacher Training: Modes, Models and Methods 16 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV OUTLINE OF A COMPREHENSIVE DISTANCE LEARNING STRATEGY DOCUMENT The structure of a distance learning strategy will vary depending on the context. An illustrative outline is provided below for users to adapt to their needs. Exhibit 10: Illustrative Outline for a Distance Learning Strategy SECTION DETAILED INFORMATION TOOLS FOR COMPLETING ACTION POINTS A. Introduction (Component 1: sub￾component 1A) • Purpose of strategy • Vision and instructional goals Tool 1: Vision and Instructional Goals B. Background (Component 1: sub￾components 1B-1G) Needs assessment data on: • Marginalized groups • Existing education frameworks • Technology infrastructure • Existing programming • Institutional capacity • Contextual factors Tool 2: Data and Strategies for Marginalized Groups Tool 3: Education Frameworks Checklist Tool 4: Technology Infrastructure Landscape Tool 5: Distance Learning Inventory Tool 6: Institutional Capacity Checklist Tool 7: Distance Learning Contextual Considerations C. Strategic Plan (Component 2) • Tech infrastructure commitments • Tech knowledge commitments • Content commitments • Pedagogical knowledge commitments Tool 4: Technology Infrastructure Landscape Tool 5: Distance Learning Inventory Tool 6: Institutional Capacity Checklist Tool 7: Distance Learning Contextual Considerations D. MEL Plan (Component 3) • MEL goals, indicators, and plan Tool 8: MEL Plan E. Budget (Component 4) • Costs for developing and implementing the strategy 17 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV In writing a distance learning strategy, a team should consider the following recommendations:22 1. Meet short-term needs of educators and learners while planning for medium- to long￾term goals. Prioritize immediate needs of stakeholders while continuing to plan for and allocate resources for medium- and long-term strategies. 2. Actively communicate, consult, and collaborate with educators, families, communities, education technology entities, and other education stakeholders. 3. Allocate sufficient resources to ensure that there is relevant technological, content, and pedagogical knowledge for implementing distance learning. 4. Plan to monitor, evaluate, and learn. Incorporate metrics that capture reach, engagement, and outcomes in the final strategy. Use these metrics to adjust implementation of the strategy to promote more meaningful distance learning. 22 Recommendations derived from USAID’s Distance Learning in Emergencies and Return to Learning Toolkit. 18 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV COMPONENT 1: ANALYZE THE EXISTING DISTANCE LEARNING LANDSCAPE AND DEVELOP A STRATEGIC VISION In completing Component 1, users respond to the guiding questions from the table below to build an inclusive vision for their comprehensive distance learning strategy. The guiding questions are accompanied by action points that help users to gather and analyze data and make decisions for their relevant learning landscape. Users can execute action points sequentially, simultaneously, or in varying orders, depending on their local context, but they should work iteratively to continuously refine and update their determined vision as they go through the subsequent components of this toolkit. Exhibit 11: Guiding Questions for an Inclusive Vision GUIDING QUESTIONS ACTION POINTS AND TOOLS A. What is the vision for distance learning and what are the instructional goals to achieve this vision?  Determine the vision and the instructional goals to achieve the vision. Tool 1: Vision and Instructional Goals (with Tools 2, 3, & 4) B. Which marginalized groups should be prioritized in a comprehensive distance learning strategy?  Identify marginalized groups that should be prioritized in the strategy. Tool 2: Data and Strategies for Marginalized Groups C. How does the vision for distance learning and the use of educational technology align with existing government education policy, strategy, or planning frameworks?  Determine where distance learning fits into current policy, strategy, or planning frameworks.  Determine which marginalized groups are identified in these frameworks. Tool 3: Education Frameworks Checklist D. What technology infrastructure exists and what modalities for delivering distance learning are feasible in the context?  Determine if existing technological infrastructure is sufficient for supporting distance learning vision and instructional goals.  Identify which groups do not have equal and equitable access to technology. Tool 4: Technology Infrastructure Landscape E. What distance learning programming and content already exist in the context? Who are the intended users? Has the content been evaluated?  Identify current distance learning initiatives and content.  Identify who the intended users are.  Identify whether the content has been evaluated. Tool 5: Distance Learning Inventory F. What governmental institutional capacity exists for implementing distance  Identify existing institutional capacity for supporting distance learning, including teacher training. 19 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV GUIDING QUESTIONS ACTION POINTS AND TOOLS learning to achieve the vision? What kind of teacher education/training exists?  Identify capacity for implementing UDL in distance learning. Tool 6: Institutional Capacity Checklist G. What contextual factors need to be considered to achieve the vision?  Identify the contextual factors that may influence equitable implementation of distance learning and achieving the vision. Tool 7: Distance Learning Contextual Considerations Write it up:  Document the learning landscape and strategic vision in your comprehensive distance learning strategy document. A. What is the vision for distance learning and what are the instructional goals to achieve this vision? A distance learning vision encompasses the purpose of distance learning (what), through which modalities programming and content will be delivered (how), who the intended users are (for whom), and the main reasons distance learning is needed (why). The vision and goals encapsulate the principles (see Box 1), as well as the goals, subgoals, and specific instructional subgoals of distance learning. Exhibit 12 shows how the vision is linked to the goals, as demonstrated further in the MEL framework that will be introduced in Component 3. 20 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Exhibit 12: Links between Distance Learning Vision and Goals 21 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV The distance learning goals reflect the key requirements of distance learning (see Exhibit 6). The instructional subgoals are short-, medium-, or long-term and capture what learners, educators, and other users will gain from distance learning. It is important to determine these instructional subgoals early on in the design process, as they will help guide users in determining what modalities, technology infrastructure, programming and content, and other contextual factors are in place. Development of the vision and instructional goals should involve a wide range of stakeholders, including central and decentralized government institutions, non-governmental and community-based organizations, disabled persons’ organizations, youth-led and youth-serving organizations, teachers and teacher unions, and technology companies. Determining the instructional goals in the design phase also helps ensure the comprehensive distance learning strategy is inclusive of, and responsive to, the unique needs of different target groups. The distance learning goals and subgoals will be essential to Component 3 where users will develop a MEL framework that lays out how these subgoals will be measured. Exhibit 13 provides several overarching instructional goals for implementing distance learning alongside illustrative examples to help users determine their vision for distance learning and measurable subgoals. In Tool 1: Vision and Instructional Goals, users will complete their own table with instructional goals determining if the technology integration is for substituting, augmenting, modifying, or redefining teaching and learning (Exhibit 7: SAMR Framework). 22 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Exhibit 13: Distance Learning Goals and Instructional Goals: Illustrative Examples23 INSTRUCTIONAL GOALS ILLUSTRATIVE EXAMPLES MEASURABLE SUBGOALS (FROM EXHIBIT 12) SHORT-TERM To serve as the main form of instruction during disruptions of in-person instruction. • Continue to provide lessons and content during temporary closures of learning institutions (e.g., airing radio/television programs for home use during a health emergency; virtual teacher training). • Provide access to learners until permanent education institutions are in place (e.g., for geographically remote communities waiting for a formal school to be built in their area). • Help learners retain knowledge and skills (e.g., ensure learners are still practicing literacy, numeracy, or social and emotional skills development). Ensure or increase access to distance learning or information. AND Ensure or improve quality and inclusivity of distance learning. To provide information to communities and/or to maintain contact with educators, learners, and families (additional support). • Provide needed distance learning information to educators, learners, and families (e.g., how to facilitate school closures). • Provide information critical to health and wellbeing of learners and educators (e.g., public service announcement on health information or pressing environmental issues). • Maintain contact with learners, educators, and families (e.g., mobile phone call check-in with learners and caregivers). Ensure or increase access to distance learning or information. To provide teaching and learning support (complementary, supplementary, or additional). • Provide support to learners, educators, caregivers, and others taking up distance learning (e.g., guidance to caregivers on how to use mobile phone applications for child’s learning, coaching group for educators to maintain contact with learners). • Provide learners who need additional complementary or supplementary resources and practice (e.g., additional literacy practice and tutoring in preparation for an exam). Ensure or improve quality and inclusivity of distance learning instruction and support. MEDIUM- TO LONG-TERM To serve as the main form of instruction. • Provide or increase remote distance learning access for learners who cannot participate in in-person learning and/or seek flexible educational opportunities (e.g., working youth, children from agricultural families who need to work Ensure or increase access to distance learning or information. 23 Refer to Annex A for definitions of technical terms. 23 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV INSTRUCTIONAL GOALS ILLUSTRATIVE EXAMPLES MEASURABLE SUBGOALS (FROM EXHIBIT 12) during harvest times, nursing mothers, university students seeking online degrees). • Provide or increase hybrid distance learning access for learners who seek in￾person and remote distance learning options (e.g., youth who are working and need remote training in addition to practicing vocational skills in-person). AND Ensure or improve quality and inclusivity of distance learning. • Build critical technological knowledge and skills (e.g., word processing or coding skills; specialized training on braille machines with experts outside of the context). • Promote better teaching through virtual training (e.g., teacher training program that in-service educators can use in their classroom to improve pedagogical practices). • Increase knowledge of how to use devices. • Increase knowledge of how to teach through distance learning. To complement in-person learning (within the curriculum). • Provide distance learning content for enhancing curriculum (e.g., learners use educational app games or radio programming after school to practice applying math content from their lessons; learners practice collaborative skills). • Provide professional development, training, and support (e.g., training modules that supplement in-person learning, social media groups that offer virtual coaching). • Ensure or increase access to distance learning or information. • Increase knowledge of how to teach through distance learning. To supplement in-person learning (beyond the curriculum). • Further educational learning in home or community setting (e.g., educational TV broadcasts that teach or reinforce literacy and socioemotional learning). • Provide on-going pedagogical support for educators (e.g., community of practice and peer groups via social media group). • Provide on-going technology support for educators (e.g., community of practice and peer groups via social media group). • Ensure or increase access to distance learning or information. • Increase knowledge of how to teach through distance learning. • Ensure or improve quality of technology instruction. To provide teaching and learning support (additional support). • Provide planned and intentional support to learners, educators, caregivers, and others (e.g., guidance to caregivers on how to support learners’ literacy development at home using print texts; remote coaching group to help educators uptake new curricula). Ensure or improve quality and inclusivity of distance learning instruction and supports. 24 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV In the strategy design process, the creation of a vision and accompanying instructional goals should occur in an iterative fashion and prioritize marginalized groups (see sub-component 1B) and will be revisited by users in multiple steps throughout the toolkit. ACTION POINTS 1. In Tool 1: Vision and Instructional Goals, modify and adapt the instructional goals and illustrative examples (from Exhibit 13) to the relevant context. 2. In Tool 1: Vision and Instructional Goals, create a vision statement that encapsulates these goals. 3. Once data have been entered into Tool 2: Data and Strategies for Marginalized Groups (per sub-component 1B) return to Tool 1: Vision and Instructional Goals to ensure that the vision and instructional goals adequately reflect marginalized groups’ needs. B. Which marginalized groups should be prioritized in a comprehensive distance learning strategy? In order to ensure the strategy is inclusive, it is critical to identify which groups have been historically marginalized in terms of access to, and participation in, high-quality in-person and distance learning. Once demographic data and known barriers to inclusion for each marginalized group have been analyzed per the action points for this sub-component 1B, users should review and revise their draft vision and instructional goals from sub-component 1A to ensure inclusivity. Definitions of marginalized groups will vary by country and context. Some illustrative examples are presented in Box 5. Individuals and groups may be marginalized because of intersectional individual or group demographics and/or be systematically excluded from equal and equitable educational opportunities because of a myriad of legal, political, social, educational, and economic conditions. While systematically determining marginalized groups is a complex process, and oftentimes contentious, it is critical to ensuring that these populations are not only prioritized in the approach, but that measures are taken to prevent furthering digital divides and opportunity gaps. 25 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Box 5. Examples of Marginalized Groups According to UNESCO’s Global Education Monitoring Report, those who are marginalized are “of a group within a given culture, context or history at risk of being excluded and discriminated against because of the interplay of differing personal characteristics or grounds.”24 Specific groups who fall within this definition vary by context. Individuals and groups may be marginalized because of their: disability; sex; sexual orientation (LGBTQIA+) and/or gender identity; ethnicity, race, language, or religion; socioeconomic status or caste; residency in rural or high-density areas (e.g., poor access to basic needs and resources); residence in a crisis or conflict area (e.g., child affected by war, internally and externally displaced learners); health status (e.g., people living with HIV or terminal diseases); age (e.g., overaged learners); and/or lack of social, economic, and/or political protections (e.g., child laborers, victims of trafficking, youth unwillingly married, migrant workers, people of diverse political opinions).25 Marginalized populations can represent intersectional identities and be marginalized in multiple ways. “These groups often experience discrimination in the application of laws and policy and/or access to resources, services, and social protection, and may be subject to harassment, violence and/or persecution.”26 It is important to note that, although radio/audio, television/video, and correspondence (print-based) initiatives have a long history of reaching marginalized learners,27 distance learning can exacerbate opportunity gaps if not carefully designed to reach, engage, and support the learning outcomes of learners from marginalized populations. This is especially true of online learning and mobile phone modalities, where the digital divide and opportunity gaps are often higher than when using radio and television modalities. Additionally, distance learning is sometimes used as a long-term approach to reaching excluded marginalized groups in contexts where it should be used only as a short-term solution. When a short￾term solution is implemented in place of a long-term approach for marginalized groups, equity gaps will be amplified and marginalized groups will not receive the quality of education responsive to their learning needs. Therefore, completing this step is crucial to developing a distance learning strategy that is inclusive. 24 UNESCO, “Global Education Monitoring Report 2020: Inclusion and Education: All means all,” UNESCO Digital Library, UNESCO, 2020, https://unesdoc.unesco.org/ark:/48223/pf0000373718, p. 420. 25 Adapted from USAID, DCHA/DRG/HR, “Suggested Approaches for Integrating Inclusive Development Across the Program Cycle and in Mission Operations: Additional Help for ADS 201,” USAID, July 2018, https://usaidlearninglab.org/sites/default/files/resource/files/additional_help_for_ads_201_inclusive_development_1 80726_final_r.pdf and USAID, “USAID Education Policy,” USAID, November 2018, https://www.usaid.gov/sites/default/files/documents/1865/2018_Education_Policy_FINAL_WEB.pdf. 26 USAID, “USAID Education Policy,” USAID, November 2018, https://www.usaid.gov/sites/default/files/documents/1865/2018_Education_Policy_FINAL_WEB.pdf, pp. 2 and 18. 27 United Nations, “Convention on the Rights of Persons with Disabilities - Article 9,” United Nations Department of Economic and Social Affairs, United Nations, 2006, https://www.un.org/development/desa/disabilities/convention-on-the-rights-of-persons-with-disabilities/article-9- accessibility.html. 26 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV ACTION POINTS 1. In Tool 2: Data and Strategies for Marginalized Groups, identify all known marginalized groups in a given context and include data on demographics and barriers to inclusion. Ensure that target groups from Tool 1 are integrated here. 2. In Tool 2: Data and Strategies for Marginalized Groups, note any modifications that need to be made to instructional goals to prioritize marginalized groups. 3. Go back to Tool 1: Vision and Instructional Goals, review instructional goals, and modify to ensure that marginalized groups are prioritized. 4. Describe marginalized groups and how they should be prioritized in the comprehensive distance learning strategy document (Section B per the outline). ADDITIONAL RESOURCES • FHI 360’s Gender and Information Communication Technology (ICT) Survey Toolkit • USAID’s Gender and ICT Training Course • USAID’s Return to Learning During Crises Toolkit—Tool 0.1 Equity and Inclusion Self-Assessment • USAID’s How-To Note DRAFT Education and Inclusion Guidance Disability Inclusive Education • UNESCO’s Global Education Monitoring Report 2020: Inclusion and Education: All Means All • USAID and INEE’s Checklist for Information Communications Technology Interventions to Support Education in Crises and Conflict Settings C. How does the vision for distance learning and the use of educational technology align with existing government education policy, strategy, or planning frameworks? Identifying where distance learning fits into the existing frameworks, policies, strategies, and plans for the use of technology is important for understanding how a distance learning strategy can be designed, funded, and implemented. Distance learning may be part of: a) a country's new plans and policies per an educational reform, b) an overarching ICT strategy, and/or c) a specialized ICT4E strategy. Exhibit 14 shows how distance learning strategies fit within broader ICT strategies. An overarching ICT or EdTech country strategy, where it exists, usually includes plans on how to build or expand technological infrastructure, increase digitization and automation through ICT in public and private sector agencies, and improve ICT skills and utilization across sectors.28 Moreover, it may have been developed under ministries and agencies outside of the education sector, such as those at the head 28 Edmond Gaible et al., “First Principles: Designing Effective Education Programs Using Information and Communication Technology (ICT) – Compendium,” EducationLinks: Information and Communications Technology for Education (ICT4E) Toolkit (USAID, June 2011), https://www.usaid.gov/sites/default/files/documents/1865/E1- FP_ICT_Compendium.pdf. 27 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV of government (e.g., prime minister’s office), and/or those that work on issues related to labor, women, and children. Knowing these ICT-related plans will help toolkit users define the vision for distance learning more clearly and better position the comprehensive distance learning strategy in relation to other education and ICT-related strategies and policies. Exhibit 14: Distance Learning as a Component of Broader ICT Strategies In contexts with an ICT4E strategy, there may be language on the use of ICT in teaching and learning, (e.g., what technology devices may be distributed to educators and learners, and/or plans to expand internet connectivity to education institutions), or specific mention of distance learning that users of this toolkit may want to adopt or expound upon for the distance learning strategy. In cases where there is a stand-alone distance learning strategy, the approach often provides alternative learning for target populations in geographically remote areas, targets nonformal learners (e.g., adults or out-of-school youth) or addresses higher education strategies (e.g., open universities). This toolkit may be used to improve an existing distance learning strategy and make it more comprehensive. In all cases, teams creating their comprehensive distance learning strategies will benefit from using the information collected under this sub-component about existing ICT or EdTech country strategies to further refine the vision developed under sub-component 1A. These iterative revisions will create a well-grounded and actionable comprehensive distance learning strategy. 28 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV ACTION POINTS 1. Review the country/context’s ICT or EdTech-related policy(s), plan(s), and strategy(s). 2. In Tool 3: Education Frameworks Checklist, use the questions to analyze distance learning language in existing frameworks and identify what is covered or not in these strategies. 3. In Tool 3: Education Frameworks Checklist, document any language that describes the vision or instructional goals of distance learning. 4. Go back to Tool 1: Vision and Instructional Goals and integrate this language into the vision and instructional goals as needed. 5. Describe any mention of distance learning in the various education frameworks in the comprehensive distance learning strategy document (Section B per the outline). ADDITIONAL RESOURCES • USAID’s ICT4E Toolkit • USAID’s EQUIP 1: Designing Effective Education Programs Using ICT • Lesotho's Education Sector Plan 2016–2026 (an example of a country’s education plan that includes use of distance learning for out-of-school youth and adult learners) • Malaysia’s ICT Transformation Plan for the Ministry of Education 2019–2023 (an example of a country’s ICT plans for education) • Philippine’s Digital Strategy 2011–2015 (an example of a country’s ICT plan that includes education￾related objectives and goals) D. What technology infrastructure exists and what modalities for delivering distance learning are feasible in the context? Gathering data on existing technological infrastructure helps identify which modalities and technology devices are feasible for implementing distance learning in a given context. Infrastructure includes: • Connectivity and coverage: refers to access to electricity, reach and range of radio and TV broadcast coverage, mobile phone reach and coverage, and internet access and connectivity. • Technology devices: refers to ownership of hardware (i.e., radio, TV, mobile phone and computer). • Software access: refers to the availability and feasibility of files, applications, LMSs, and software programs necessary for the different hardware. • Assistive/adaptive technology: refers to the availability of technology for people with disabilities. • Government infrastructure: refers to the existence of government-owned or run infrastructure related to radio, TV, mobile phone, and the internet as well as the existence of any government agreements with providers. The technology infrastructure landscape can be assessed through reviewing industry reports (e.g., telecom, media), national statistics, research, etc. and conducting interviews with experts and key 29 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV players, including representatives from the government and various technology companies and stakeholders. Policies or programs that support or hinder access should also be documented (e.g., restriction of social media platforms, free internet access for learners) and analyzed further in sub￾component IF of the toolkit (contextual factors). ACTION POINTS 1. In Tool 4: Technology Infrastructure Landscape, include any data on technology device access, connectivity and coverage, software access, accessibility/assistive technology features, and policies/programs that support or hinder access. 2. Describe the existing technology infrastructure in the comprehensive distance learning strategy document (Section B per the outline). ADDITIONAL RESOURCES • UNESCO’s Distance Learning Solutions • World Bank’s Knowledge Pack: Education TV • World Bank’s Knowledge Pack: Educational Radio • World Bank’s Knowledge Pack: Mobile Distance & Hybrid Education Solutions E. What distance learning programming and content already exist in the context? Who are the intended users? Has the content been evaluated? Taking inventory of what distance learning initiatives exist—including curricula, content, and materials— is critical to understanding what distance learning modalities have been tried and the breadth of the expertise and capacity that exists in a given context. This information can be gathered through an inventory of the existing content and programming that has been produced by all stakeholders, (i.e., government office, NGOs, other organizations). ACTION POINTS 1. In Tool 5: Distance Learning Inventory, enter information on existing and past distance learning initiatives. This includes information on target groups (intended users) and how programming and content has been assessed or evaluated. 2. In Tool 5: Distance Learning Inventory, in the “analyzing content” section of Tool 5, map existing content from action point 1 by education level and modality. (Note that you will need to copy and paste information from the “inventory table” at the top of Tool 5 to complete mapping.) Analyze coverage and gaps. 3. Go back to Tool 1: Vision and Instructional Goals and adjust any instructional goals and target groups based on what programming and content exists and what needs to be created. 4. Describe the existing programming and content in the comprehensive distance learning strategy document (Section B per the outline). 30 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV ADDITIONAL RESOURCES • World Bank’s Lessons for Education During the Covid-19 Crisis: Continuity Stories • World Bank’s How countries are using edtech (including online learning, radio, television, texting) to support access to remote learning during the COVID-19 pandemic • Global Digital Library and Global Digital Library Radio • USAID’s Distance Learning Interactive Audio and Radio Instruction Online Library F. What institutional capacity exists for implementing distance learning to achieve the vision? What kind of teacher education/training exists? Key to developing a comprehensive distance learning strategy is understanding which governmental or parastatal entities and agencies have the institutional capacity to design and develop distance learning content, train educators, provide leadership, and coordinate the different actors and stakeholders involved in distance learning implementation. Institutional capacity refers to both operational and leadership capacity (e.g., an institution’s mandate, policies, and systems) and human capacity (e.g., technological knowledge, content knowledge, and pedagogical knowledge).29 Exhibit 15 lays out the steps in integrating Institutional Capacity Development into the distance learning strategy, starting with identifying the knowledge, skills, and experience of organizations involved in distance learning (Step 1) and then identifying areas where capacity needs to be built (Step 2). In contexts where stakeholders have little to no prior institutional knowledge, skills, and experience in distance learning, the first step is to identify areas where capacity needs to be built. 29 USAID, “Institutional Capacity Assessment Tool G2G Toolkit,” EducationLinks (USAID, November 2013), https://www.edu-links.org/resources/institutional-capacity-assessment-tool. Institutional capacity is further discussed in sub-component 2C. 31 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Exhibit 15: Steps for Integrating Institutional Capacity-Building into the Distance Learning Strategy (Steps 1 and 2) To complete the institutional capacity checklist (Action Points 1–3 below), users will review whether or not: • Distance learning content development teams exist; • Pre- and in-service educator trainings are in place; and • Leadership has the capacity to guide and coordinate distance learning program design and delivery. Data on institutional capacity can be collected by reviewing human resource expertise in distance learning in education agencies, coursework or plans in teacher training institutions, and government sector reports and documents. Conducting key informant interviews or surveys with government officials and technical experts to identify existing capacity and gaps will also be critical. In sub-component 2D, users will use this needs assessment data to develop an Institutional Capacity Development Plan for the strategy. ACTION POINTS 1. In Tool 6: Institutional Capacity Checklist, answer the guiding questions and, where relevant, copy and paste the names of institutions and organizations from Tool 5: Distance Learning Inventory. Rate (using best estimate) from 0–3 (with 0 = no capacity, 1 = limited capacity, 2 = some capacity, and 3 = strong capacity) the extent of their capacity and provide the required details for "yes" answers. 2. In Tool 6: Institutional Capacity Checklist, summarize and list the organizations with capacity, then identify the institutional factors that need to be considered in emergency contexts and areas where capacities for a particular modality need to be developed or strengthened (i.e., technological, content, and pedagogical knowledge, coordination, oversight, and M&E capabilities). 3. Describe the existing institutional capacity for implementing distance learning in the comprehensive distance learning strategy document (Section B per the outline). 32 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV ADDITIONAL RESOURCES • Omidyar Network’s Scaling Access and Impact: Realizing the Power of EdTech • USAID’s Institutional Capacity Assessment (ICA) Tool Government to Government Education Toolkit • USAID’s Human and Institutional Capacity Handbook 2011 G. What contextual factors need to be considered to achieve the vision? After the vision, infrastructure (current frameworks, technology, and programming), and institutional capacity have been identified, it is critical to identify any macro-level sociological, regulatory, or political economy factors that could impact the operationalization and implementation of the distance learning strategy in positive, negative, or neutral ways. These contextual factors may have emerged in different ways throughout the needs assessment process, but the action points below systematically ensure that they are assessed in the strategy. Based on existing literature, there are six key contextual factors that may impact distance learning content development and implementation. These should be taken into consideration early on when planning a distance learning strategy. These factors are outlined in the table below. 33 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Exhibit 16: Contextual Factors, Descriptions, and Distance Learning Examples CONTEXTUAL CONSIDERATIONS DESCRIPTION EXAMPLES (USING MOBILE PHONE MODALITY) Educational paradigms • Any historical or contemporary institutions, practices, or measures that influence which learning or teaching content and pedagogical approaches must be followed. • Historical legacies that influence who has access to or who is targeted through distance learning. • Low-tech distance learning modalities (radio, paper correspondence) have a long history of being used for remote, marginalized communities and high-tech distance learning modalities (computers, tablets) have a history of being used for privileged learning communities. Mobile phone programming has been overlooked. • Under colonial rule, rural communities were excluded from schooling and provided with correspondence (print-based) education. Rural residents are still less likely to have access to mobile and online learning. Sociolinguistic and cultural factors • Any official or unofficial sociolinguistic practices, and power structures, that influence which languages are used for/dominate the different distance learning modalities. • Social or cultural practices or norms that influence who has access to technology devices and content. • Social or cultural practices or norms that influence how marginalized groups are portrayed in programming and content. • Language of instruction policy in national (dominant) or lingua￾franca language requires distance learning content to be produced in national languages only, despite the fact that the majority of educators and learners are not proficient in that language. • Mobile phones and apps are in the dominant national language(s), making them inaccessible to the majority of teachers and learners. • Mobile phone app ownership is encouraged among elder males in the family but discouraged for youth, and especially young women. • Mobile phone companies and app developers are typically not representative of local or marginalized groups and programming and content does not reflect these groups or reproduces harmful stereotypes. Political will and prioritization • Political agendas that influence which distance learning modalities are integrated and financed. • Political measures that may enable or restrict (i.e., censor) distance learning technology software and interfaces or programming and coherence between government agencies and policies. • Potential for unethical or corrupt practices. • Political platforms built on expanding mobile phone and tablet access as a way of “modernizing” the education sector. • Blocking of social media apps; government censure of content that alludes to ethnic conflicts; cross-sectoral commitment to expanding mobile phone education. • Potential for mobile phone companies getting contracts for distance learning if they agree to support political candidates. Economic development • Economic development imperatives that link ICT, digital inclusion, or distance learning to • Mobile phones are part of the “vision” strategy for advancing economic development. 34 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV CONTEXTUAL CONSIDERATIONS DESCRIPTION EXAMPLES (USING MOBILE PHONE MODALITY) national education growth (and poverty reduction) strategies. • There is increasing use of phones for economic purposes (e.g., banking, money transfer, business transactions, etc.). Regulatory environment • Factors that enable or hinder security of technology (e.g., cyber security) in the country. • Conditions that privilege external and large￾scale tech companies over local or small-scale companies. • Registration of technology and domains and other regulatory factors. • Policy on SIM card registration that restricts mobile phone users and invalidates non-registered lines. • Agreements with international or local-monopoly mobile phone providers at the national level, and control over mobile phone satellites and infrastructure (and pricing). • Few in-country domains and educational apps are restricted. Edtech supply and climate • History and climate of ICT nationalization or privatization and the extent to which the public and private sectors partner on access to technology devices, software, connectivity, and infrastructure of mobile phone, radio, TV, Internet, printing, etc. • Competition among sectors that influences pricing. • “EdTech entrepreneurs have access to capital through appropriate business models, allowing them to survive”30 • Education institutions can access products that meet their needs. • There are eight telecom industries in the country, only one of which is partly government owned. While some of these telecom providers have offered zero rating for Internet access and phone data for learning, this is not accessible across communities. There are no public-private partnerships. • Universal Service Funds restricted to certain companies with limited service reach. • Health sector purchasing massive packages that have increased prices and increased SMS bundles and prices. • Public or private teams have access to capital and resources they need to develop high-quality content. • Costing, procuring, and delivering technology and content meets education institutions’ needs. Sources 31 30 Pouezevara, Sarah, Ignacio Jara Valdivia, Mike Michalec, Talitha Amalia, and Sybille Fleischmann, “Scaling Access and Impact: Realizing the Power of Edtech,” Imaginable Futures, Omidyar Network, March 2019: 10, https://www.imaginablefutures.com/learnings/scaling-access-impact-realizing-power-edtech/. 31Burns, Mary, “For Want of a Good Theory: Considerations for Technology Integration in Well-Resourced Schools,” Essay. In A Closer Look at Educational Technology, edited by Maria A. Clausen. NY, NY: Nova Science Publishers, 2019; Pouezevara, Sarah, Ignacio Jara Valdivia, Mike Michalec, Talitha Amalia, and Sybille Fleischmann, “Scaling Access and Impact: Realizing the Power of Edtech,” Imaginable Futures, Omidyar Network, March 2019: 10, https://www.imaginablefutures.com/learnings/scaling-access-impact-realizing-power-edtech/; World Bank EdTech Team, “EdTech Knowledge Pack: Innovation Ecosystems,” The World Bank Education and Technology: EdTech Publications, The World Bank Group, August 25, 2020, http://pubdocs.worldbank.org/en/466031598013786493/World-Bank-EdTech-Innovation-Ecosystems-Knowledge-Pack-July17 35 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV ACTION POINTS 1. In Tool 7: Distance Learning Contextual Considerations, document and discuss important contextual considerations that may influence the distance learning strategy in a given context. Provide illustrative and real examples, separating by modalities as relevant. 2. Describe the relevant contextual factors in the comprehensive distance learning strategy document (Section B per the outline). ADDITIONAL RESOURCES • USAID’s Discussion Note: Complexity-Aware Monitoring • USAID’s Tips on Context Monitoring • World Bank’s Education and Technology Publications • Omidyar Network’s Scaling Access and Impact: Realizing the Power of EdTech 36 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV COMPONENT 2: DESIGN A DISTANCE LEARNING APPROACH In this component, users will analyze information gathered in Component 1 to determine which modalities will be included in the strategy and for what purposes. In the same tools used in Component 1 (Tools 4–7), users will utilize the needs assessment data to determine the technological, content, and pedagogical knowledge the strategy will cover (including education levels and demographics) and detailed strategies for reaching marginalized users. Exhibit 17: Guiding Questions for Selecting a Modality GUIDING QUESTIONS ACTION POINTS AND TOOLS A. Which modalities are feasible given the existing technology infrastructure? How are marginalized groups reached? (technological infrastructure)  Determine which modalities will be used and how to reach those without technology devices and infrastructure. Tool 4: Technology Infrastructure Landscape Tool 7: Distance Learning Contextual Considerations B. What content will learners receive and how will the content be delivered (remotely, in-person, and/or hybrid)? What content will best serve marginalized learners? Do educators and learners have the requisite technological knowledge? (content and technological knowledge)  Determine which technological and content knowledge will be covered.  Determine the intended (target) users.  Determine if the distance learning approach should be fully remote, in-person, or hybrid. Tool 5: Distance Learning Inventory C. What institutional capacity-building approach will ensure effective and equitable implementation of distance learning? (pedagogical knowledge)  Determine which pedagogical knowledge is needed for implementing distance learning. Tool 6: Institutional Capacity Checklist Tool 7: Distance Learning Contextual Considerations Write it up:  Document the approach (modality, content, and institutional capacity) in your comprehensive distance learning strategy. A. Which modalities are feasible given the existing technology infrastructure? How are marginalized groups reached? Identifying the modalities that are appropriate for implementing distance learning in a given context is key to developing a comprehensive distance learning strategy that is user-centered, designed for scale, and takes into account the needs of the most marginalized learners and educators. In this analysis, users draw on data about the technology infrastructure from sub-component 1D (Tool 4: Technology Infrastructure Landscape). 37 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Drawing on sub-component 1D data, users identify modalities for implementing their strategy, based on coverage, connectivity, and device access. Users also determine how to reach marginalized groups who lack access to existing infrastructure and devices. To the greatest extent possible, the strategy should lay the groundwork for a multi-modal approach where the different modalities match the demographics, educational levels, and goals of the learners and educators. A multi-modal approach may be used within an education level (e.g., TV and radio for primary levels, video and online learning for secondary levels) or across education levels (e.g., radio for primary, TV for secondary, and online for tertiary), depending on what is sustainable and cost-effective in the context. Exhibit 18 poses key questions regarding technology and instructional design that must be considered in order to make choices about which modalities will be used and how marginalized groups will be included. (Note that these key questions relate to all of the key requirements for successful distance learning that were introduced in Exhibit 6). Exhibit 18: Key Questions in Considering Which Modalities to Implement32 Source33 32 Coverage, defined as the percentage of a target population reached via a given technology, is also an important consideration in choosing modalities. The Diffusion of Innovations Theory was used to determine the percentage of learners and educators (>80 percent) that should have the necessary device or connectivity coverage for distance learning to succeed. A target for coverage of >80 percent helps ensure equity and inclusion. 33 Adapted from World Bank EdTech Team, “EdTech Knowledge Pack on Remote Learning Response to COVID￾19,” The World Bank Education and Technology: EdTech Publications, The World Bank Group, April 8, 2020, http://pubdocs.worldbank.org/en/925611587160522864/KnoweldgePack-COVID19-RemoteLearning￾LowResource-EdTech.pdf. 38 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV As the exhibit above indicates, both the technology landscape and instructional designs used will influence which distance learning modalities will be used and which learner populations will be reached. In some cases, a technology device may be widely available (e.g., radio) but may not be the best modality for a given subject or content (e.g., computer programming). In other cases, a technology device may be well-suited to certain instructional designs but may not be accessible to all users (e.g., computers for populations without connectivity). ACTION POINTS 1. In Tool 4: Technology Infrastructure Landscape, answer the questions on coverage and connectivity and analyze which modalities are feasible and how to reach groups that are not covered. 2. In Tool 4: Technology Infrastructure Landscape, answer questions on device access and analyze which devices are feasible and how to reach marginalized groups. 3. In Tool 4: Technology Infrastructure Landscape, answer questions on software access and analyze which interfaces are feasible and how to reach marginalized groups. 4. Use the analyses in Tool 4: Technology Infrastructure Landscape to identify which modalities should be used in the strategy, and how marginalized groups will be reached. Include a description on the choice of modalities in the comprehensive distance learning strategy document (Section C per the outline). ADDITIONAL RESOURCES • World Bank’s Remote Learning Response to COVID-19 • World Bank’s Knowledge Pack: Education TV • World Bank’s Knowledge Pack: Educational Radio • World Bank’s Knowledge Pack: Mobile Distance & Hybrid Education Solutions B. What content will learners receive and how will the content be delivered (remotely, in-person, and/or hybrid)? What content will best serve marginalized learners? Do educators and learners have the requisite technological knowledge? After determining which of the five modalities (if not all) the strategy will cover, the next step is to identify what new content, if any, should be developed to address the gaps identified in sub-component 1E (Tool 5). In identifying which content to use, the team should also consider how marginalized groups will be prioritized. Instructional designs may include: • Fully remote distance learning instruction: Educators and learners are in different physical spaces and use one of the five modalities. For example, a university offering online classes using synchronous and asynchronous teaching and learning through a learning management system like 39 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Google Classroom. Another example is a secondary school educator using correspondence (print￾based) instruction. • Distance learning instruction in groups: Distance learning content is received through in￾person groups with a trained facilitator using the distance learning programming to deliver the content knowledge or lesson; for example, IAI listener groups. Another example is when learners are studying in a learning pod (small group of learners meeting together) using mobile phones and video instruction with the support of a rotating caregiver while the educator is in a different physical space. • Hybrid distance learning instruction: Hybrid instruction often happens in one of two ways. First, where some of the instruction is conducted online and some in-person (e.g., online degree program with in-person three-month residency); second, when instruction occurs synchronously with learners in different places (some learners are in-person and others join from a distance). Hybrid classrooms may use any of the four main distance learning modalities to allow learners who are not physically present to participate. For example, in-service educators complete video and print-based modules on their own (asynchronous) three times a month but come together once a month for an in-person class. Strategy design teams need to determine what content their learners need and what instructional design approach they will use. Users also need to determine whether learners’ needs can be entirely met with existing content, or whether new content is needed. The decision tree in Exhibit 19 can assist strategy design teams in decision-making. 40 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Exhibit 19: Decision Tree for Choosing How Content Will Be Delivered (Remotely, In Person, or Hybrid) 41 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Equity and inclusion considerations should be paramount during the decision-making process guided by the decision tree shown in Exhibit 19. Accessibility features for all technologies should be sufficiently budgeted to ensure inclusivity and accessibility of learning. Plans for ensuring technological knowledge should be integrated into the design. ACTION POINTS 1. In Tool 5: Distance Learning Inventory, (based on work from sub-component 1E), group content by grade level and modality and analyze what content can be used or adapted according to instructional goals and equity/inclusion data. 2. In Tool 5: Distance Learning Inventory, identify what new content will be needed by modality and education level, how content will be administered (i.e., remotely, in person, or hybrid), how needs of marginalized groups are prioritized (e.g., language minority, learners with limited to no access to device), and what technological knowledge will be covered. 3. Describe the new programming and content that will be created, and how marginalized groups will be prioritized in the comprehensive distance learning strategy document (Section C per the outline). ADDITIONAL RESOURCES. • World Bank’s Knowledge Pack: Education TV • World Bank’s Knowledge Pack: Educational Radio • World Bank’s Knowledge Pack: Mobile Distance & Hybrid Education Solutions • USAID’s Best Practices on Effective Remote Socioemotional and Soft Skills Interventions in Distance Learning C. What institutional capacity-building approach will ensure effective and equitable implementation of distance learning? In sub-component 1F, users identified institutions’ and organizations’ existing capacities and areas where capacity needs to be increased. As stated in that section, institutional capacity refers to both operational and leadership capacity (e.g., an institution’s mandate, policies, and systems) and human capacity34 (e.g., technological knowledge, content knowledge, and pedagogical knowledge). The next step in the institutional capacity-building process is to design an Institutional Capacity Development Plan for the strategy that can later be operationalized during the implementation phase. This plan will identify the human resources (i.e., content developers, trainers, leaders, and coordinators), technology, and processes that need to be established or strengthened, and the cost to do so. The guiding questions in Tool 6: Institutional Capacity Checklist walk users through the critical components of creating a capacity-building plan that can be integrated into the distance learning strategy. 34 USAID, “Institutional Capacity Assessment Tool G2G Toolkit,” Education Links (USAID, November 2013), https://www.edu-links.org/resources/institutional-capacity-assessment-tool. 42 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Exhibit 20: Steps for Developing an Institutional Capacity-Building Plan (Step 3) ACTION POINTS 1. In Tool 6: Institutional Capacity Checklist, identify the technical team, distance learning experts, and technology devices needed. Start thinking about costs for distance learning content development, educator training, and leadership and coordination for implementation. 2. In Tool 7: Distance Learning Contextual Considerations, review the relevant contextual considerations that could hinder or enable capacity-building of target institutions (i.e., government institutions and teacher training institutions). Take note in Tool 6: Institutional Capacity Checklist of any contextual considerations that need to be made. 3. Describe the institutional capacity-building plan (i.e., the human resources and technology available and needed, the processes that need to be established or strengthened, and the cost to do so) in the comprehensive distance learning strategy document (Section C per the outline). ADDITIONAL RESOURCES • USAID’s Human and Institutional Capacity Handbook • UNICEF’s Contingency planning, risk reduction, preparedness and response framework • World Bank’s EdTech Decision Tree for Ministries of Education (K-12 focus) • USAID & INEE’s Checklist for ICT Interventions to Support Education in Crises & Conflict Settings • USAID’s ICT4E How-To Note • USAID’s Using ICT to Implement Universal Design for Learning • UNDP’s A Capacity Development Plan for Civil Society Organizations in the Pacific 43 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV COMPONENT 3: DEVELOP A MONITORING, EVALUATION, AND LEARNING PLAN This section guides users through the process of creating an overarching MEL Plan for the comprehensive distance learning strategy.35 Users will adapt the vision and instructional goals created in Components 1 and determine performance indicators to measure whether the vision and goals are being achieved as designed. How the data will be collected and methods for capturing data will also be determined. The content in this component is adapted from USAID’s A Roadmap for Measuring Distance Learning: A Review of Evidence and Emerging Best Practices, which was written to guide the design of monitoring and evaluation of distance learning interventions (See the Roadmap Figure in Annex A). Designing a MEL Plan for a comprehensive distance learning strategy can take place concurrently with action points in Component 1 (e.g., as users gather information for Tool 1: Vision and Instructional Goals and Tool 2: Data & Strategies for Marginalized Groups). As with all action points, the MEL Plan should prioritize marginalized groups and follow the strategy’s guiding principles of equitable access, high-quality and inclusive programming, and capacity-building. Exhibit 21: Guiding Questions for Developing a MEL Plan GUIDING QUESTIONS36 ACTION POINTS A. What are the aims (internal and external) of monitoring and evaluating distance learning? (Step 1)  Determine the aims a of monitoring and evaluating strategy implementation. Tool 8: MEL Plan B. What will be measured and how? (Step 2)  Paste the strategy’s vision and any measurable goals from Tool 1: Vision and Instructional Goals into Tool 8: MEL Plan.  Create context-specific, adapted distance learning goals, subgoals (including instructional goals) in line with the strategy’s vision. Tool 8: MEL Plan  Create context-specific performance indicators to measure the distance learning goals and subgoals. Tool 8: MEL Plan C. How will data be collected (in person, remotely, or an integrated approach)? (Step 3)  Determine how data will be collected (in person, remotely, integrated). Tool 8: MEL Plan 35 More detailed MEL frameworks and plans should be created for each distance learning initiative under this strategy. 36 The “steps'' from that roadmap are presented here as “guiding questions'' and guide users through the process of developing a MEL Plan. 44 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV GUIDING QUESTIONS36 ACTION POINTS  Determine through which modality data will be collected (phone, tablet, computer). Tool 8: MEL Plan D. What methods should be used to capture data per the performance indicators? (Step 4)  Determine what method (instruments, protocol) should be used for collecting data and how to include marginalized groups. Tool 8: MEL Plan Write it up:  Document how this overarching MEL framework will be used to monitor and evaluate progress toward the vision in your comprehensive distance learning strategy. As strategy teams develop the MEL Plan, they should consider the following overarching recommendations for measuring distance learning: 1. Integrate in-person and remote data collection approaches, use multi-modal interfaces37 for collecting data, and employ mixed-methods to measure distance learning. Integrating in-person and remote data collection (e.g., in-person testing and remote interviews), using multi-modal interfaces (e.g., phone calls and SMS surveys), and mixed-methods approaches (e.g., interviews, surveys, and photographs) helps promote greater participation and leads to more accurate results. 2. Encourage innovative solutions to measure reach, engagement, and outcomes during a quick pivot to distance learning, while also developing high-quality MEL strategies for the longer term. The guidance in this section helps teams think about short￾term MEL needs while working toward longer-term strategies for assessing the effectiveness of distance learning. 3. Design equitable monitoring and evaluation approaches and conduct systematic equity analyses of distance learning initiatives. Evaluative approaches to distance learning must strive to measure and analyze whether marginalized individuals and groups are being systematically included or excluded through distance learning programming as well as during in￾person and remote data collection. A. What are the aims (internal and external) of monitoring and evaluating distance learning? The first step in designing a MEL Plan is to determine the overarching aims and objectives for monitoring and evaluation that guide the strategy. 37 Using different interfaces or approaches for data collection expands the reach of who can participate in data collection. For example, in the case of surveys, participants with low reading levels or low vision may be better served by Interactive Voice Response, (IVR), or Computer-Assisted Telephone Interviewing, (CATI), but SMS messaging is more accessible to participants who are hard of hearing. 45 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Exhibit 22 highlights learning and accountability as the two key aims for measuring distance learning. Within these aims, there are internal and external objectives that should be considered when creating a MEL Plan, to ensure appropriate data are collected. Internally, (within government units, learning institutions, etc.), data can be used to inform program (initiative) management and teaching and learning (learning), and/or to guide adaptation and sustainability of programming (accountability). Externally, MEL data can be used to generate knowledge and further the evidence base (learning), and inform costing, scaling, and replicability at different levels, such as district, regional, or national levels (accountability). Exhibit 22: Aims of Monitoring and Evaluating Distance Learning Source: Adapted from Hempel and Fiala (2012). In Tool 8: MEL Plan, users will determine the overarching aims, but as the MEL Plan is developed the aims for each of the goals and subgoals should also be considered. ACTION POINT • In Tool 8: MEL Plan, determine the overarching aims of monitoring and evaluation in the strategy. B. What will be measured and how? DEVELOPING THE VISION, GOALS, AND SUBGOALS The second step in designing the MEL Plan is to determine the strategy’s MEL framework.38 The MEL framework visualizes the vision, goals, and subgoals for distance learning. The MEL framework should 38 See USAID’s Results Framework Template provided in the Additional Resources of this section for a modifiable template. 46 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV include: a) a measurable distance learning vision, b) distance learning goals, and c) distance learning subgoals. A measurable vision defines: • What the purpose is (i.e., improving learning outcomes and wellbeing of learners); • How it will be delivered or through which modalities (e.g., radio, video, mobile phone, online, paper-based or multi-modal); • For whom (e.g., all ages and education levels, specific marginalized groups prioritized); • By whom (e.g., Ministry of Education and Ministry of Youth); and • Why distance learning is needed (e.g., increasing equitable access). A measurable goal and subgoal shows: • Direction of change (i.e., improved or increased); • What is being measured (i.e., connectivity, technological knowledge); and • For whom (e.g., grade one learners). Goals in the comprehensive distance learning strategy are aligned with the key requirements for successful distance learning. For example, Goal 2: Increased technological knowledge of primary school educators and students. Subgoals under Goals 2–4 correspond to instructional goals, which are short-, medium-, or long-term goals that capture what learners, educators, and other users will gain from distance learning. For example, Subgoal 2.1: Increased knowledge of how to use technology devices. Users will adapt and develop the vision, goals, subgoals, and specific performance indicators in Tool 8: MEL Plan. Creating a conceptual framework (such as the one shown in Exhibit 23) helps to show the alignment between the vision, goals, and subgoals and clearly outline the theory of change. The MEL Plan developed in Tool 8: MEL Plan will be included in the strategy document. 47 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Exhibit 23: Illustrative Conceptual Framework Showing Alignment Between Vision, Goals, and Subgoals 48 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV DEVELOPING PERFORMANCE INDICATORS The next step in developing the MEL Plan is to create performance indicators in Tool 8. Performance indicators measure the achievement of the strategy’s goals and subgoals and can be quantitative or qualitative, but should clearly outline what is being achieved (e.g., mastery in using devices), for whom (e.g., preschool educators), and how the data are being measured (e.g., proportion or percentage). Distance learning performance indicators will vary based on the context, modalities, intended (target) users, etc., but should at a minimum capture three domains: reach (access to technology devices, infrastructure, connectivity, etc.), engagement (participation in and use of programming), and outcomes (change in knowledge, skills, attitudes, and behaviors). Exhibit 24: Three Domains of Measuring Distance Learning Within these three domains, a variety of performance indicators should be developed to capture changes in technological landscape (infrastructure, device access, etc.), technological knowledge, content knowledge, and pedagogical knowledge. Exhibit 25 and Exhibit 26 provide access and quality metrics for guiding performance indicator development. Exhibit 25: Metrics for the Technological Landscape and Access to Programming and Content, by Domain REACH ENGAGEMENT OUTCOMES CONTENT KNOWLEDGE LEARNING SOCIAL AND EMOTIONAL LEARNING Infrastructure Infrastructure and connectivity (Internet and phone data) coverage Technology & Accessibility Access to technology devices (hardware) and software; access to assistive technology Utilization of technology devices and software as intended; access to assistive technology Change in technological literacy 49 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV REACH ENGAGEMENT OUTCOMES CONTENT KNOWLEDGE LEARNING SOCIAL AND EMOTIONAL LEARNING Programming Basic listenership, viewership, and usership by target audiences (frequency and duration of use) Participation in programming as intended and completion Change in subject matter, content knowledge, and skills acquisition and retention Change in social and emotional and soft skills, attitudes, and beliefs Change in behaviors Accompanying Materials Access to accompanying materials Utilization of accompanying materials as intended Cost Unit cost of reaching learners Unit cost of engaging learners Cost of improving outcomes, in dollars per unit of measure of an outcome Exhibit 26: Metrics for Measuring Quality of Programming and Materials, by Domain REACH ENGAGEMENT OUTCOMES CONTENT KNOWLEDGE LEARNING SOCIAL AND EMOTIONAL LEARNING INTENDED AUDIENCE Interaction Opportunities for interaction built into the programming Extent to which users interact in programming as intended Change in subject matter, content knowledge, and skills acquisition and retention Change in social and emotional and soft skills, attitudes, and beliefs Change in behavior Quality and Relevance Adherence to curriculum, grade level, scheduling, geographical reach, and other parameters facilitating use Level of quality; relevance to developmental stage and age, gender, language, etc. Response Number of users who share feedback by demographics Level of interest in programs; popularity of programming UNINTENDED AUDIENCE Shadow Audience39 Shadow audience access Shadow audience participation and response When developing performance indicators, the distance learning principles should be considered. These are: a) increasing equitable access to distance learning, b) ensuring high-quality and inclusive design, content, and programming, and c) building capacity for successful implementation (Box 1). 39 A shadow audience consists of users who utilize distance learning programming even though they were not among the originally targeted beneficiaries (e.g., a younger sibling who watches a program with their older sibling for whom it is designed). 50 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV ACTION POINTS In Tool 8: MEL Plan: 1. Paste the adapted vision, goals, and subgoals from Tool 1: Vision and Instructional Goals. 2. Adapt and create context-specific distance learning goals and subgoals (including instructional goals) in line with the strategy’s vision. 3. Create performance indicators to measure the adapted distance learning goals and subgoals. C. How will data be collected (in person, remotely, or an integrated approach)? For collecting performance indicator data, a combination of in-person and remote data collection approaches should be used. The decision tree in Exhibit 27 walks users through the key considerations for determining which technology to use for collecting data, whether in-person or remotely. 51 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Exhibit 27: Decision Tree for Determining Data Collection Technology 52 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV In Tool 8: MEL Plan, users will decide whether each performance indicator will be measured in person, remotely, or an integrated in-person and remote approach. If users are collecting any data remotely, strategy developers should also decide which modality (online, mobile phone, tablet, computer) they will use to collect the data. Column J of Tool 8: MEL Plan allows planners to list the modality they will use to collect data for each indicator. ACTION POINTS In Tool 8: Mel Plan: 1. Determine how data will be collected (in person, remote, integrated). 2. Determine through which modality data will be collected (e.g., phone, tablet, computer). D. What methods should be used to capture data per the performance indicators? A wide range of quantitative and qualitative methods can be used to collect data toward the performance indicators in a strategy. In A Roadmap for Measuring Distance Learning: A Review of Evidence and Emerging Practices there is detailed guidance on how to decide the most relevant methods— quantitative, qualitative, or mixed methods—for collecting data. In Tool 8: MEL Plan, users will decide if each performance indicator will be measured using quantitative, qualitative, or mixed methods. ACTION POINTS 1. In Tool 8: MEL Plan, determine what method (instruments, protocols) should be used for collecting data and how to include marginalized groups. 2. Include and describe the Tool 8: MEL Plan in the comprehensive distance learning strategy document (Section D per the outline). ADDITIONAL RESOURCES • USAID’s Results Framework Template • USAID’s A Roadmap for Measuring Distance Learning: A Review of Evidence and Emerging Practices • USAID’s Technical Note: Developing Results Frameworks • USAID’s Tips on Selecting Performance Indicators • Kellogg Foundation’s Logic Model Development Guide • UNDP’s Handbook on Planning, Monitoring and Evaluating for Development Results • World Bank’s Monitoring & Evaluation: Some Tools, Methods, & Approach • The Fletcher School’s Hitachi Center for Technology and International Affairs and Besa: Catalyzing Strategic Change’s Technology for Evaluation in Fragile and Conflict Affected States: An introduction for the digital immigrant evaluator, Working Paper 53 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV COMPONENT 4: CREATE A BUDGET This component guides users through the cost categories to consider in order to create a budget to operationalize a distance learning strategy. These illustrative categories were designed to be consistent with those commonly used in government and non-government implementers' budgets, but with nuances specific to distance learning. There is no tool provided as it is assumed users will draw on budget templates/formats used in their particular contexts. Per the principles of this toolkit, it is important that sufficient financial resources be dedicated to operationalizing a comprehensive distance learning strategy. The purpose of distance learning is not to promote savings, but rather to promote meaningful and high-quality teaching and learning that meets the vision and instructional goals outlined in the strategy. Exhibit 28: Guiding Questions for Creating a Budget GUIDING QUESTIONS ACTION POINTS A. What are the general cost categories in a comprehensive distance learning strategy budget?  Understand how budgets for distance learning are organized along standard costs. B. What are the labor costs to consider in a comprehensive distance learning strategy budget?  Identify labor costs to be budgeted by phase. C. What are the travel costs to consider in a comprehensive distance learning strategy budget?  Identify travel costs to be budgeted by phase. D. What are the equipment and supply costs to consider in a comprehensive distance learning strategy budget?  Identify equipment and supplies costs to be budgeted by phase. E. What are other implementation and direct costs to consider in a comprehensive distance learning strategy budget?  Identify other implementation and direct costs to be budgeted by phase. F. What are other administrative costs to consider in a comprehensive distance learning strategy budget?  Identify other administrative costs to be budgeted by phase. A. What are the general cost categories in a comprehensive distance learning strategy budget? Determining detailed costs is crucial to planning and designing a comprehensive distance learning strategy. This section illustrates types of costs likely to be incurred during the design and development, implementation, and monitoring and evaluation phases of a comprehensive distance learning strategy. For institutional capacity-building costs, see some of the preliminary costs designated in Tool 6: Institutional Capacity Checklist. Budgets are commonly organized along the following standard cost categories, also referred to as ingredients: 54 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV • Labor costs: international and local salaries/payments, benefits, fringe, and allowances; • Travel costs: international and local transportation and per diem for planning, implementing, and MEL; • Equipment and supplies: relevant hardware and software; • Other implementation and direct costs: training participant costs, accessibility, and reasonable accommodations; and • Other administrative costs: overhead if implemented through a grant or contract. The cost categories for development of any distance learning program will vary depending on the context, as will costs by category. In contexts where the required technology infrastructure is limited or non-existent, the cost for equipment and supplies will take up a bigger proportion of the budget. For example, for the production of a series of 100 pre-recorded lessons for radio instruction in Malawi, 65 percent of the total budget covered labor costs while 35 percent of the total budget went to other implementation and direct costs.40 It is important to avoid underestimating costs of distance learning. Building program designers’ capacity, piloting and formative evaluation of content, disseminating and maintaining equipment, and monitoring and evaluating teaching and learning at scale are all elements of distance learning implementation that are frequently under-budgeted. Allocating sufficient resources to these areas is essential to achieving the vision and instructional goals. ACTION POINT • In a budget (to be determined by the user), document the different costs per the illustrative categories outlined below (Component 4B–4F). Include this budget in the comprehensive distance learning strategy document (Section E of the outline). ADDITIONAL RESOURCE. • USAID’s Cost Measurement guidance has detailed information on how to budget for USAID￾funded initiatives. B. What are the labor costs to consider in a comprehensive distance learning strategy budget? Labor costs are grouped by different phases of designing and implementing distance learning and the different types of expertise and labor needed. Labor costs include international and local salaries, honorariums, or other forms of payment for work and services. Under this section, benefits, fringe, and allowances would be budgeted if they apply. 40 Heather Carroll et al. “COVID-19: Interactive Radio and Audio Instruction (IRI) - Implementation Guidance,” Save the Children’s Resource Center, Save the Children, 2020, https://resourcecentre.savethechildren.net/node/17890/pdf/learn_covid￾19_tool_interactive_radio_and_audio_instruction_implementation_guidance.pdf 55 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Planning and conducting needs assessment • Technical expert(s) to conduct and lead needs assessment • Data collectors who collect primary data • Other analysts and/or researchers with specific expertise Planning and designing distance learning approach • Technical expert(s) for developing program content (e.g., curriculum design or mapping, digital content and materials, training for writers or developers, instructional design) • Technical expert(s) for developing/programming technological software interfaces (e.g., developers for audio, video, mobile phone app, or online learning management systems) • Accessibility expert(s) to ensure content and technology is accessible to persons with disabilities • Writing/editorial staff for review of content (e.g., writers, editors) • Formative evaluation specialist(s) to oversee continuous piloting and assessment of content (for all modalities) • For audio and video content only: studio/production staff Building capacity • Technical expert(s) to train distance learning instructional designers • Technical expert(s) to train distance learning educators and administrators (e.g., head teachers, principals to supervise or mentor educators) • Participant(s) of technical working groups and advisory boards • Staff in charge of managing partnerships Setting up MEL systems • Monitoring specialist(s) time to develop a database and a monitoring plan • MEL expert(s) to train MEL staff on ongoing data collection and reporting • MEL team staff for ongoing data collection, data processing, and analysis (e.g., data entry) • Evaluation specialist(s) to design and manage evaluation activities to measure outcomes and impact Implementing distance learning • Educators who will implement distance learning • Administrators who will oversee and manage distance learning educators • Staff to manage administrative and other tasks (e.g., financial staff, drivers) • Staff to supervise implementation and report back to stakeholders, including donors • Staff to provide technical help or support (e.g., IT desk) • Community mobilization or outreach specialist(s) to help inform intended users about programming • Other relevant labor costs 56 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV C. What are the travel costs to consider in a comprehensive distance learning strategy budget? Travel costs refer to international and local travel, transportation, and per diem for staff and personnel. Below are travel cost considerations by phase: Planning and conducting needs assessment • Transportation, per diem, and all other travel-related expenses (e.g., lodging) for bringing together experts and staff to plan and conduct needs assessment Planning and designing distance learning approach • Transportation, per diem, and all other travel-related expenses (e.g., lodging) for bringing production team and experts together Building capacity • Transportation, per diem, and all other travel-related expenses (e.g., lodging) for bringing instructional design training team, educator training team, and experts together • Transportation, per diem, and all other travel-related expenses (e.g., lodging) for convening technical working groups, advisory boards, and other activities Setting up MEL systems • Transportation, per diem, and all other travel-related expenses (e.g., lodging) for MEL trainers • Transportation, per diem, and all other travel-related expenses (e.g., lodging) for MEL teams to collect data Piloting and refining distance learning • Transportation, per diem, and all other travel-related expenses (e.g., lodging) for working with educators (e.g., petrol, travel stipends) and administrators • Transportation, per diem, and all other travel related expenses (e.g., petrol to pick up materials, repair equipment) for educators and administrators implementing distance learning Implementing distance learning • Transportation, per diem, and all other travel-related expenses (e.g., lodging) for working with educators (e.g., petrol, travel stipends) and administrators • Transportation, per diem, and all other travel related expenses (e.g., petrol to pick up materials, repair equipment) for educators and administrators implementing distance learning 57 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV D. What are the equipment and supply costs to consider in a comprehensive distance learning strategy budget? Equipment and supply costs refer to both durable (i.e., any item that has an expected life of two years or more) and expendable (i.e., any item that has an expected life of two years or less) equipment and supplies. Below are equipment and supply cost considerations by phase: Planning and conducting needs assessment • Office furniture for team conducting needs assessment • Office supplies for team conducting needs assessment • Technical equipment and hardware (e.g., computers, hard drives, servers, routers, modems, Wi-fi boosters, satellite dish) for team conducting assessment • Equipment for collecting data (e.g., mobile phones, tablets) • Software licenses and fees (e.g., analysis software, databases, LMS platforms) for team conducting needs assessment Planning and designing distance learning approach • Office furniture for content development team • Office supplies for content development team • Technical equipment and hardware (e.g., computers, hard drives, servers, routers, modems, Wi-fi boosters, satellite dish, projectors) for content development team • For audio and video content only: studio equipment for audio and video production (e.g., sound boards, mixers, computers, cameras, microphones) • Software licenses and fees (e.g., mobile phone apps, online learning management system, video and audio editing software) Building capacity • Equipment and materials for training (e.g., projector, servers, modems) Setting up MEL systems • Office furniture for MEL team • Office supplies for MEL team • Technical equipment and hardware (e.g., computers, hard drives, servers, routers, modems, Wi-fi boosters, satellite dish) for MEL team • Equipment for collecting data (e.g., mobile phones, tablets) • Software licenses and fees (e.g., analysis software, databases, LMS platforms) for MEL team Implementing distance learning • Printing costs of materials (e.g., supplementary materials) • Office furniture for educators and administrators 58 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV • Office supplies for educators and administrators • Technical equipment and hardware (e.g., computers, hard drives, servers, routers, modems, Wi-fi boosters, satellite dish, satellite phone) for educators and administrators • Technology devices for end users/learners (e.g., TV, radio, phones, tablets, or computers) • Software licenses and fees (e.g., LMS platforms, phone apps) E. What are the other implementation and direct costs to consider in a comprehensive distance learning strategy budget? Other direct costs refer to training participant costs, costs related to accessibility and reasonable accommodations. Below are examples of other direct cost considerations by phase: Planning and conducting needs assessment • Participant costs (e.g., handouts, rental of training venue) for any relevant training on planning and needs assessment • Ongoing phone, internet, and connectivity for team conducting needs assessment Planning and designing distance learning approach • Ongoing phone, internet, and connectivity for content development team • For audio and video content only: − Licensing fees for copyrighted video or audio content − Renovation/construction or lease of recording studio/space Building capacity • Participant costs (e.g., handouts, rental of training venue) for any relevant training on instructional design and development Setting up MEL systems • Ongoing phone, internet, and connectivity for MEL teams • Training costs for data collectors (e.g., rental of training venue, training or workshop materials) • Remote data collection costs (e.g., SMS costs, phone data) Implementing distance learning • Participant costs (e.g., handouts, rental of training venue) for any relevant training for educators and administrators • Ongoing phone, internet, and connectivity for educators and administrators as well as end users/learners • Distribution and delivery costs of equipment and materials (e.g., to schools, learning centers) • For audio and video content only: ongoing costs for air time on radio or TV 59 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV F. What are the other administrative costs to consider in a comprehensive distance learning strategy budget? Other administrative costs refer to overhead costs if implemented through a grant or contract. Below are examples of other administrative cost considerations across all phases: • Office and operating expenses (e.g., rent, utilities) • Bank fees and other administrative costs • Ongoing communications costs (e.g., administrative mailings) 60 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV CONCLUSION This toolkit provides an outline of a comprehensive distance learning strategy with detailed tools and action points to guide decision-making in the design process. Distance learning can meet multiple educational needs when it is an intentional and integral part of an education sector’s plans and strategies; when programming and content are high-quality, inclusive, and equitable; and when short-term and long￾term approaches build on needs and capacity in the learning landscape. Building a comprehensive distance learning strategy will ensure that there are clear, measurable, inclusive, and equitable short-term and long-term plans for providing distance learning, and that these plans address diverse education and age levels, types of learners and institutions, and instructional goals. Strategies should clearly prioritize the most marginalized groups, and should be integrated into countries’ and systems’ long-term education frameworks and budgets. The existence of comprehensive distance learning strategies helps ensure that systems are resilient to shocks and stressors in the short￾term and that education meets the needs of all learners and educators in the long-term. 61 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV REFERENCES Auerswald, Colette L., Amber Akemi Piatt, and Ali Mirzazadeh. “Research with Disadvantaged, Vulnerable and/or Marginalized Adolescents.” Innocenti Research Briefs, No. 2017/06. New York: United Nations, June 2017. https://doi.org/10.18356/a054ca5b-en. Bloome, Anthony. “Information and Communications Technology for Education (ICT4E) Toolkit.” EducationLinks: Information and Communications Technology for Education (ICT4E) Toolkit. USAID, January 1, 2020. https://www.edu-links.org/index.php/resources/information-and-communication￾technology-education-ict4e-toolkit. Bloome, Anthony and Cynthia Chassy. Information and Communication Technology for Education (ICT4E) How-To Note. United States Agency for International Development, Bureau for Economic Growth, Education, and the Environment, Office of Education, 2019. https://www.edu￾links.org/sites/default/files/media/file/USAID%20ICT4E%20How-To%20Note%20Final_Feb20.pdf. Boisvert, Kayla, Nina Weisenhorn, and Jamie Bowen. “Returning to Learning during Crises Toolkit.” EducationLinks. USAID, October 2020. https://www.edu-links.org/resources/returning-learning-during￾crises-toolkit. Bub, Kristen, and Kelsey Dalrymple. Best Practices on Effective SEL/Soft Skills Interventions in Distance Learning. Rockville, MD and Arlington, VA: EnCompass LLC and MSI, a Tetra Tech company, 2020. Burns, Mary. “For Want of a Good Theory: Considerations for Technology Integration in Well￾Resourced Schools.” Essay. In A Closer Look at Educational Technology, edited by Maria A. Clausen. New York, NY: Nova Science Publishers, 2019. Burns, Mary. “Distance Education for Teacher Training: Modes, Models and Methods.” Education Development Center: International Development. EDC, Inc., 2011. http://idd.edc.org/resources/publications/modes-models-and-methods. Burns, Mary, Mohammad Issack Santally, Roshan Halkhoree, Kevin Roopesh Sungkur, Bhavish Juggurnath, and Yousra Banoor Rajabalee. “Information and Communications Technologies and Secondary Education in Sub-Saharan Africa: Policies, Practices, Trends and Recommendations.” Mastercard Foundation, November 2019. https://mastercardfdn.org/wp-content/uploads/2019/11/ICT-in-Secondary￾Education.pdf. Carroll, Heather, Luke Stannard, and Julia Finder. “COVID-19: Interactive Radio and Audio Instruction (IRI) - Implementation Guidance.” Save the Children’s Resource Center. Save the Children, 2020. https://resourcecentre.savethechildren.net/node/17890/pdf/learn_covid￾19_tool_interactive_radio_and_audio_instruction_implementation_guidance.pdf. EnCompass LLC, Management Systems International (MSI), and Data and Evidence for Education Programs (DEEP). “Best Practices in Generating Data on Learners with Disabilities.” EducationLinks. USAID, October 2020. https://www.edu-links.org/resources/best-practices-generating-data-learners￾disabilities. Gaible, Edmond, Tony Bloome, Analice Schwartz, Janel Hoppes Poché, and Wayan Vota. “First Principles: Designing Effective Education Programs Using Information and Communication Technology 62 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV (ICT) – Compendium.” EducationLinks: Information and Communications Technology for Education (ICT4E) Toolkit. USAID, June 2011. https://www.usaid.gov/sites/default/files/documents/1865/E1- FP_ICT_Compendium.pdf. GDL Radio. Accessed December 6, 2020. https://gdlradio.org/. Global Digital Library. Accessed December 6, 2020. https://digitallibrary.io/. Global Education Monitoring Report Team. “Global Education Monitoring Report, 2020: Inclusion and Education: All Means All.” UNESCO Digital Library. UNESCO, 2020. https://unesdoc.unesco.org/ark:/48223/pf0000373718?fbclid=IwAR2L8j0jjrRBoLGFDf65VEjdGnt_nTMtH OT1yb-ZFco_Z57Iobr8HO5-WXI. Government of Lesoto, Ministry of Education and Training. “Education Sector Plan 2016 - 2026.” Global Partnerships for Education. Government of Lesoto, 2016. https://www.globalpartnership.org/sites/default/files/education_sector_plan_2016-2026._lesotho_0.pdf. Harris, Judith, Punya Mishra, and Matthew J. Koehler. “Teachers’ Technological Pedagogical Content Knowledge and Learning Activity Types: Curriculum-Based Technology Integration Reframed.” Journal of Research on Technology in Education 41, no. 4 (2009): 393–416. Hayes, Anne, Ann Turnbull, and Norma Moran. “Universal Design for Learning to Help All Children Read: Promoting Literacy for Learners with Disabilities (First Edition).” Global Reading Network. USAID, May 2019. https://www.globalreadingnetwork.net/sites/default/files/media/file/Literacy%20for%20All%20Toolkit_0.p df. Hempel, Kevin and Nathan Fiala. Measuring Success of Youth Livelihoods Interventions: A Practical Guide Monitoring and Evaluation. The World Bank, 2012. Highet, Catherine, Hannah Skelly, and Alexandra Tyers. “Gender and Information Communication Technology (ICT) Survey Toolkit.” USAID, April 2019. https://www.usaid.gov/sites/default/files/documents/15396/Gender_and_ICT_Toolkit.pdf. H.L. “SAMR Model: A Practical Guide for EdTech Integration.” Schoology, October 30, 2017. https://www.schoology.com/blog/samr-model-practical-guide-edtech-integration. Josa, Josh and Cynthia Chassy. “How-to Note Disability Inclusive Education.” EducationLinks. USAID, November 2018. https://www.edu-links.org/sites/default/files/media/file/How￾ToNote_DisabilityInclusiveEducation_0.pdf. Kurt, Serhat. “Educational Technology: An Overview.” Educational Technology. Educational Technology Consulting Services, November 18, 2015. https://educationaltechnology.net/educational-technology-an￾overview/. Malaysia, Ministry of Education. “ICT Transformation Plan for the Ministry of Education 2019 - 2023.” Malaysian Government. Ministry of Education, 2019. https://www.moe.gov.my/images/KPM/UKK/2019/02_Feb/ITP_2.0_Brochure_4portal_A3_printing_SEC. pdf. 63 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV Morris, Emily, and Anna Farrell. “Delivering Distance Learning in Emergencies.” EducationLinks. USAID, April 2020. https://www.edu-links.org/resources/delivering-distance-learning-emergencies. Office of Elementary and Secondary Education. “Communities Come Together to Support STEM Education.” Office of Elementary and Secondary Education, November 19, 2015. https://oese.ed.gov/2015/11/communities-come-together-to-support-stem-education/. Panoply Digital. Gender Digital Divide Online Course. USAID. Accessed December 6, 2020. Online course, 60:00. https://www.panoplydigital.com/gender-and-ict-online-course/. Philippines, Department of Information and Communications Technology. “Philippine’s Digital Strategy 2011 - 2015.” Philippine Government. Department of Information and Communications Technology, 2011. https://dict.gov.ph/wp-content/uploads/2014/06/philippine-digital-strategy-2011-2015.pdf. Pouezevara, Sarah, Ignacio Jara Valdivia, Mike Michalec, Talitha Amalia, and Sybille Fleischmann. “Scaling Access and Impact: Realizing the Power of Edtech.” Imaginable Futures. Omidyar Network, March 2019. https://www.imaginablefutures.com/learnings/scaling-access-impact-realizing-power-edtech/. Puentedura, Ruben. “On the Impact of the SAMR Model.” Common Sense Education. Common Sense. January 2, 2020. https://www.commonsense.org/education/videos/ruben-puentedura-on-the-impact-of￾the-samr-model Puentedura, Ruben. “SAMR and TPCK: Intro to Advanced Practice.” Hippasus, December 8, 2010. http://hippasus.com/resources/sweden2010/SAMR_TPCK_IntroToAdvancedPractice.pdf. Rogers, Everett M. Diffusion of Innovations 3rd Edition. New York: The Free Press: New York, 1983, p. 247. Scharbatke-Church, Cheyanne, and Aditi Patel. “Technology for Evaluation in Fragile and Conflict Affected States: An Introduction for the Digital Immigrant Evaluator.” The Fletcher School’s Hitachi Center for Technology and International Affairs and Besa: Catalyzing Strategic Change, April 2016. https://sites.tufts.edu/ihs/files/2018/02/Technology-and-Evaluation-Hitachi-Paper.pdf. Shah, Ritesh. “Transforming Systems in Times of Adversity: Education and Resilience.” Education in Crisis and Conflict Network. USAID, 2019. https://www.eccnetwork.net/resources/transforming￾systems-times-adversity-education-and-resilience-white-paper. Tacquard, Stephanie. “Tips for Moving a Class Online Quickly: Pearson Blog.” Pearson, March 13, 2020. https://www.pearsoned.com/tips-moving-class-online-quickly/. UNDP. “A Capacity Development Plan For CSOs in the Pacific.” United Nations Development Programme. UNDP Pacific Centre, June 2015. http://www.undp.org/content/dam/rbap/docs/Research%20&%20Publications/democratic_governance/U NDP_PC_DG_A_Capacity_Assessment_of_CSOs_in_the_Pacific.pdf. UNDP. “World's Most Marginalized Still Left behind by Global Development Priorities: UNDP Report.” Human Development Reports, March 23, 2017. http://hdr.undp.org/en/content/world%E2%80%99s￾most-marginalized-still-left-behind-global-development-priorities-undp-report. 64 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV United Nations. “Convention on the Rights of Persons with Disabilities - Article 9.” United Nations Department of Economic and Social Affairs. United Nations, 2006. https://www.un.org/development/desa/disabilities/convention-on-the-rights-of-persons-with￾disabilities/article-9-accessibility.html. UNESCO. “Distance Learning Solutions.” UNESCO, January 15, 2020. https://en.unesco.org/covid19/educationresponse/solutions. UNESCO. “Global Education Monitoring Report 2020: Inclusion and education: All means all.” UNESCO Digital Library. UNESCO, 2020. https://unesdoc.unesco.org/ark:/48223/pf0000373718. UNICEF, Education Section. “COVID-19 Education: Contingency Planning, Risk Reduction, Preparedness and Response Framework.” April 2020. https://www.unicef.org/lac/en/media/12531/file. USAID. “Checklist for Promoting Equity and Inclusion.” 2020. (not final yet) (https://docs.google.com/document/d/1jkX05ApARfw-jvxtFAWqQGcGaaHlO9CH8LxdzxC-IQM/edit. USAID. “Distance Learning Interactive Audio and Radio Online Library.” EducationLinks. Accessed December 6, 2020. https://www.edu-links.org/distance-learning-interactive-audio-and-radio-online￾library. USAID. “Human and Institutional Capacity Development Handbook.” USAID Learning Lab. USAID, August, 2011. https://usaidlearninglab.org/sites/default/files/resource/files/HICD%20Handbook%202011%20-%2008.pdf. USAID. “Institutional Capacity Assessment Tool G2G Toolkit.” EducationLinks. USAID, November 2013. https://www.edu-links.org/sites/default/files/media/file/ICA_Tool_FINAL_Sept2014.pdf. USAID. “Three Ps of ICT4E: Principles, Partnerships, and Programs.” EducationLinks. USAID, January 17, 2020. https://www.edu-links.org/learning/three-ps-ict4e-principles-partnerships-and-programs. USAID. “Tips on Learning from Context Formal and Informal Approaches to Understanding the Local Political Economy.” Learning Lab. Accessed December 6, 2020. https://usaidlearninglab.org/sites/default/files/resource/files/tips_on_context_monitoring_- _formal_to_informal_.pdf. USAID. “USAID Education Policy.” USAID, November 2018. https://www.usaid.gov/sites/default/files/documents/1865/2018_Education_Policy_FINAL_WEB.pdf. USAID Bureau for Policy, Planning, and Learning. “Discussion Note: Complexity-Aware Monitoring.” USAID Learning Lab. USAID, July 2018a. https://usaidlearninglab.org/sites/default/files/resource/files/cleared_dn_complexity￾aware_monitoring.pdf. USAID Bureau for Policy, Planning, and Learning. “Technical Note: Developing Results Frameworks.” USAID, July 2013. https://www.usaid.gov/sites/default/files/documents/1865/_508_RF_Technical_Note_Final_2013_0722.p df. 65 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV USAID/DCHA/DRG/HR. “Suggested Approaches for Integrating Inclusive Development Across the Program Cycle and in Mission Operations: Additional Help for ADS 201.” USAID, July 2018b. https://usaidlearninglab.org/sites/default/files/resource/files/additional_help_for_ads_201_inclusive_devel opment_180726_final_r.pdf. USAID and INEE (Hartenberger-Toby, Lisa and Richmond, Simon). “Checklist for Information and Communications Technologies (ICT) Interventions to Support Education in Crisis and Conflict Settings.” USAID, August 2018. https://www.eccnetwork.net/resources/checklist-information-and￾communications-technologies-ict-interventions-support-education. USAID Office of Trade and Regulatory Reform and Bureau of Economic Growth, Education, and Environment. “RF Template V1 Without Sub-IRs: Program Cycle: Project Starter.” USAID, June 12, 2019. https://www.usaid.gov/project-starter/program-cycle/cdcs/strategy-and-results-framework/rf￾template-v1-without-sub-irs. Vegas, Emiliana, and Rebecca Winthrop. “Beyond Reopening Schools: How Education Can Emerge Stronger than before COVID-19.” Brookings. Brookings, September 8, 2020. https://www.brookings.edu/research/beyond-reopening-schools-how-education-can-emerge-stronger￾than-before-covid-19/. World Bank. Education and Technology: EdTech Publications. Accessed December 6, 2020. https://www.worldbank.org/en/topic/edutech/brief/education-and-technology-publications. World Bank. “How Countries Are Using EdTech (Including Online Learning, Radio, Television, Texting) to Support Access to Remote Learning during the COVID-19 Pandemic.” World Bank. Accessed December 6, 2020. https://www.worldbank.org/en/topic/edutech/brief/how-countries-are-using-edtech￾to-support-remote-learning-during-the-covid-19-pandemic. World Bank. "Lessons for Education During the Covid-19 Crisis: Continuity Stories." Accessed December 6, 2020. https://documents.worldbank.org/en/publication/documents￾reports/documentlist?colti=Education%20Continuity%20Stories. World Bank EdTech Team (Hallie Rocklin Applebaum et al.). “EdTech Knowledge Pack: Innovation Ecosystems.” The World Bank Education and Technology: EdTech Publications. The World Bank Group, August 25, 2020. http://pubdocs.worldbank.org/en/466031598013786493/World-Bank-EdTech￾Innovation-Ecosystems-Knowledge-Pack-July17. World Bank EdTech Team (Sharan Zacharia et al.). “EdTech Knowledge Pack: Education Radio.” The World Bank Education and Technology: EdTech Publications. The World Bank Group, August 6, 2020. http://pubdocs.worldbank.org/en/351561596545287034/EduRadio-KnowledgePack-WorldBank￾31July2020. World Bank EdTech Team (Sharon Zacharia et al.). “EdTech Knowledge Pack: Education Television Programming.” The World Bank Education and Technology: EdTech Publications. The World Bank Group, June 1, 2020. http://pubdocs.worldbank.org/en/267791593613610668/Education-TV-Knowledge￾Pack-WorldBank-Edtech-Team. World Bank EdTech Team. “EdTech Knowledge Pack: Mobile Distance & Hybrid Education Solutions.” The World Bank Education and Technology: EdTech Publications. The World Bank Group, July 2020. 66 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV http://pubdocs.worldbank.org/en/685691598013656403/WorldBank-EdTech-Team-Knowledge-Pack￾MobileDistance-HybridEducationSolutions-JUL2020. World Bank EdTech Team. “EdTech Knowledge Pack on Remote Learning Response to Covid-19.” The World Bank Education and Technology: EdTech Publications. The World Bank Group, April 8, 2020. http://pubdocs.worldbank.org/en/925611587160522864/KnoweldgePack-COVID19-RemoteLearning￾LowResource-EdTech.pdf. 67 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV ANNEXES ANNEX A. ROADMAP FOR MEASURING DISTANCE LEARNING Exhibit 29: Roadmap for Measuring Distance Learning 68 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV ANNEX B: KEY TERMS AND DEFINITIONS Terms around distance learning change over time and by context. The definitions below pertain to this toolkit but may be defined differently in other circumstances. TERM DEFINITION Asynchronous distance teaching and learning Occurs at different times AND in different places (e.g., recording lectures and having learners respond with questions and comments on a discussion board on their own time).a Basic phone “A wireless handheld device that allows users to make and receive calls...[also] capable of sending and receiving text messages. As these devices evolved, they became smaller and more features were added, such as multimedia messaging service (MMS), which allowed users to send and receive images.”b Blended learning “Any formal education program in which a student learns at least in part through online learning, with some element of student control over time, place, path, and/or pace.”c An example of a blended learning model is flipped classrooms, where video lectures or other online activities are assigned to learners and are completed before in-person discussion sessions with instructors take place. Blended learning is often used synonymously with hybrid learning. Computer-Assisted Telephone Interviewing “In the CATI research mode, interviewers make calls themselves, and they are the ones noting down the respondents’ answers and reading the next question from a script. CATI interviewers use specialized software to dial phone numbers, record the answers they are given, and read the next question based on any skip logic that is included.”d Complementary educational resources and materials Materials and resources that reinforce or enhance teaching and learning of formal or nonformal curriculum (e.g., short video episodes that depict a historical figure or demonstrate a science experiment).e, f Cookies tracking “A computer ‘cookie’ is ... a term for a packet of data that a computer receives and then sends back without changing or altering it...When you visit a website, the website sends the cookie to your computer. Your computer stores it in a file 69 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV TERM DEFINITION located inside your web browser.”g Cookies allow for automatic tracking and data collection. Correspondence education “A method of providing education for students, primarily adults, who receive lessons and exercises through the mail or some other device and, upon completion, return them for analysis, criticism, and grading. It is extensively used by business and industry in training programs, by men and women in the armed forces, and by the governments of many nations as part of their educational program. It supplements other forms of education and makes independent study programs readily available.”h Distance learning (or distance education) Teaching and learning where educators and learners are in separate physical spaces. Distance learning can occur through one of four modalities: audio/radio, video/television, mobile phone, or online learning platforms. Printed and digital texts (e.g., teachers’ guides and student materials) often accompany these modalities and could be a fifth modality in cases where technology is not (or cannot be) used for teaching and learning (e.g., correspondence learning). Distance learning can be synchronous or asynchronous. e-Learning e-Learning is often used synonymously with online or Web￾based learning.i More broadly, “e-learning content is developed according to a set of learning objectives and is delivered using different media elements, such as text, graphics, audio and video. It must provide as much learning support as possible (through explanations, examples, interactivity, feedback, glossaries, etc.), in order to make learners self-sufficient. However, some kind of support, such as e-mail-based technical support or e-tutoring, is normally offered to learners.”j Feature phone “A feature phone is a type of mobile phone that has more features than a standard cellphone but is not equivalent to a smartphone …Typically, a feature phone has the basic characteristics of a mobile phone and has capabilities such as a portable media player, digital camera, personal organizer and Internet access.”k Fidelity of implementation and design The “degree to which an intervention or program is delivered as designed/intended.”l In the case of distance learning, degree 70 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV TERM DEFINITION to which technology, programming, and materials are used as designed. Gender-responsive teaching and learning “Teaching methods and learning materials that take into account the specific learning needs of female and male students.”m Hybrid Distance Learning Instruction in which a combination of remote distance learning and in-person distance learning is used. Hybrid learning has two common uses. First, where some of the instruction is conducted online and other in-person (e.g., online degree program with in-person three-month residency); second, when instruction occurs synchronously with learners in different places (some learners are in-person and others join from a distance). Hybrid classrooms may use any of the four main distance learning modalities to allow learners who are not physically present to participate (e.g., some learners are in a classroom with the instructor while other learners join via a livestream of the class). Inclusive education “Having one system of education for all students, at all levels (early childhood, primary, secondary, and post-secondary), with the provision of supports to meet the individual needs of students. Inclusive education focuses on the full and effective participation, accessibility, attendance, and achievement of all students, especially those who, for different reasons, are excluded or at risk of being marginalized."n Interactive audio/radio instruction Interactive audio instruction (IAI) is a distance education approach that uses interactive pedagogies to engage listeners in active and quality teaching and learning through pre-recorded audio programs. Audio lessons guide educators and learners “through activities, games, and exercises that teach carefully organized knowledge and skills.”o Programs draw on songs, stories, and culturally based knowledge and content. Audio programs can either be digitized and listened to on an audio device (IAI) or broadcast through radio (IRI). IAI can be used with learners and educators in a range formal and non-formal settings to “improve educational quality and teaching practices.” Interactive Voice Response (IVR) “Interactive Voice Response (IVR) is an automated telephony system technology that interacts with the callers, gathers the required information, and routes the calls to the particular 71 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV TERM DEFINITION appropriate recipient ... Conversations are either pre-recorded or generated audio which assist, direct, or route calls automatically without a live operator. Within these interactions, clients can communicate by using either the touch￾tone keypad selection or voice telephone input. The responses take the form of voice, call-back or any other related media.”p (User) interface “User interface (UI) is a broad term for any system, either physical or software based, that allows a user to connect with a given technology.”q Learning management system “Learning management systems (LMS) are software platforms for instructors to manage and organize educational courses online and provide students a single location for all course material ... LMSs are composed of a document management component and communication capability, allowing teachers to upload course files such as rubrics, assignments, calendars, and gradebooks, as well as communicate with students via chat rooms or forums. Many LMSs may also offer online assessment functionality, such as quizzes and tests, or offer features that integrate multimedia components such as videos and photos. Students can use LMSs to submit assignments, and many platforms allow instructors to grade assignments within the platform.”r Learning outcomes “Learning outcomes are statements that describe the knowledge or skills students should acquire by the end of a particular assignment, class, course, or program, and help students understand why that knowledge and those skills will be useful to them. They focus on the context and potential applications of knowledge and skills, help students connect learning in various contexts, and help guide assessment and evaluation.”s Log file (customized for education app) “A log file is a file that keeps a registry of events, processes, messages, and communication between various communicating software applications and the operating system. Log files are present in executable software, operating systems, and programs whereby all the messages and process details are recorded. Every executable file produces a log file where all activities are noted.”t 72 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV TERM DEFINITION Messaging services: • SMS (Short Message Service) • MMS (Multimedia Messaging Service) Text and multimedia messaging technologies. “SMS stands for Short Message Service ... it is one of the oldest texting technologies. It is also the most widespread and frequently used. MMS stands for Multimedia Messaging Service. It was built using the same technology as SMS to allow SMS users to send multimedia content. It’s most popularly used to send pictures, but can also be used to send audio, phone contacts, and video files.”u Massive Open Online Courses (MOOCS) An online distance education mechanism (platform) where teaching and learning is global (e.g., a general class on monitoring and evaluation). Some MOOC providers (e.g., Coursera, Udemy) offer a certificate or credit for a cost. They can be taught asynchronously, with active teacher monitoring, or be completely automated. Nonformal education “Non-formal education takes place both within and outside educational institutions and caters to people of all ages. It does not always lead to certification. Non-formal education programmes are characterized by their variety, flexibility, and ability to respond quickly to new educational needs of children or adults. They are often designed for specific groups of learners such as those who are too old for their grade level, those who do not attend formal school, and adults. Curricula may be based on formal education or on new approaches. Examples include accelerated ‘catch-up’ learning, after-school programmes and literacy- and numeracy-focused programmes. Non-formal education may lead to late entry into formal education programmes, in which case it is sometimes called ‘second chance’ education.”v Online education A distance learning modality that refers to teaching and learning that occurs via the Internet. Online education (or online learning) can be used to supplement in-person education (e.g., learners follow along on tablets during a guided reading exercise) or be the primary mode of delivery in distance learning settings (also called online distance education). Online learning can be asynchronous (where learners control time and pace) or synchronous (where teaching and learning happens simultaneously in real time either in an online space or through a concurrent broadcast). 73 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV TERM DEFINITION Smart phone “A smartphone is a mobile phone with highly advanced features. A typical smartphone has a high-resolution touch screen display, WiFi connectivity, Web browsing capabilities, and the ability to accept sophisticated applications.”w Supplementary educational materials Materials that do not follow an official curriculum and are not designed specifically for classroom instruction purposes, but that contain educational content (e.g., educational TV series and educational app games).x, y Survey software “An application used to collect feedback from a targeted sample through a computer-assisted method, which comes in different ways. At its core survey software solutions help you design, send, and analyze surveys, usually via the Internet and using drag-and-drop tools and automated functionality.”z For example, Tangerine, KoBo Toolbox, Survey to Go, Survey Monkey, and Qualtrics. Synchronous distance teaching and learning Occurs simultaneously, but not in the same physical space. It often refers to online learning that happens in real time via digital, video, audio, or online forums (e.g., class discussion on Zoom).aa Universal Design for Learning (UDL) “Universal design for learning (UDL) is a set of principles for designing curriculum that provides all individuals with equal opportunities to learn. UDL is designed to serve all learners, regardless of ability, disability, age, gender, or cultural and linguistic background. UDL provides a blueprint for designing goals, methods, materials, and assessments to reach all students including those with diverse needs. UDL is an approach to instruction that prioritizes meeting the needs of learners with disabilities.”bb 74 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV a Great Schools Partnership. “Asynchronous Learning.” Great Schools Partnership, August 29, 2013a. https://www.edglossary.org/asynchronous-learning/ b Techopedia. “Mobile Phone.” Techopedia, August 20, 2020. https://www.techopedia.com/definition/2955/mobile￾phone c Maxwell, Clifford. “What Blended Learning Is and Isn’t.” Blended Learning Universe, March 4, 2016. https://www.blendedlearning.org/what-blended-learning-is-and-isnt/ d Elliott, Roxana. “Interactive Voice Response vs. Computer Assisted Telephone Interviewing for survey research” (blog). Geopoll, October 3, 2019. https://www.geopoll.com/blog/interactive-voice-response-vs-computer-assisted￾telephone-interviewing-research/ e Casado, Carlos Andrés Gilarranz and Rodriguez-Sinobas, Leonor. “Design and application of complementary educational resources for self-learning methodology.” Geophysical Research Abstracts Vol. 18, EGU2016-5584, 2016 EGU General Assembly. 2016. f Yuan, Liu. “Investigation and research on flipped classroom of "History of Chinese Arts and Crafts" based on WeChat Platform.” Advances in Social Science, Education and Humanities Research, volume 344. 2019. g Norton. “What are cookies?” Norton. n.d. https://us.norton.com/internetsecurity-privacy-what-are-cookies.html h Britannica. “Correspondence education” Britannica. September 7, 2007. https://www.britannica.com/topic/correspondence-education i Burns, Mary. “Distance Education for Teacher Training: Modes, Models, and Methods.” Washington, DC: EDC, 2011. http://idd.edc.org/resources/publications/modes-models-and-methods j Ghirardini, Beatrice. “E-learning methodologies - A guide for designing and developing e-learning courses.” Rome: FAO Trust, 2011. http://www.fao.org/3/i2516e/i2516e.pdf k Techopedia. “Feature Phone.” Techopedia, February 5, 2016a. https://www.techopedia.com/definition/26221/feature-phone l USAID. USAID Education Policy. Washington, D.C.: USAID, 2018a. https://www.usaid.gov/documents/1865/2018- usaid-education-policy m USAID. “Introduction to Gender-Responsive Teaching Methods.” Washington, D.C.: USAID 2018b. https://www.mcsprogram.org/wp-content/uploads/dlm_uploads/2018/11/PowerPoint-Introduction-to-Gender￾Responsive-Teaching-Methods.pdf n USAID. USAID Education Policy. Washington, D.C.: USAID, 2018a. https://www.usaid.gov/documents/1865/2018- usaid-education-policy o Ho, Jennifer and Hetal Thukral. Tuned in to student success: assessing the impact of Interactive Radio Instruction for the hardest to reach. Washington, D.C.: Education Development Center, 2009. p Mitranescu, Miruna. “IVR Definition and Benefits.” Aircall (blog), August 25, 2016. https://aircall.io/blog/call￾center/interactive-voice-response/ q Techopedia. “User Interface (UI).” Techopedia, November 11, 2016b. https://www.techopedia.com/definition/4685/user-interface-ui r G2. “Best Learning Management System.” G2. n.d. https://www.g2.com/categories/learning-management-system￾lms s University of Toronto. “What are learning outcomes?” University of Toronto, 2020. https://teaching.utoronto.ca/teaching-support/course-design/developing-learning-outcomes/what-are-learning￾outcomes/ t Techopedia. “Log File.” Techopedia, January 11, 2017. https://www.techopedia.com/definition/5445/log-file u Twilio. “What are SMS and MMS and How do They Differ?” Twilio. n.d. https://www.twilio.com/learn/messaging/what-are-sms-and-mms v Human Rights Council. Report of the Special Rapporteur on the right to education: realizing the right to education through non-formal education. Human Rights Council, June 23, 2017. https://www.ohchr.org/EN/HRBodies/HRC/RegularSessions/Session35/Documents/A_HRC_35_24_AUV.docx w Techopedia. “Smartphone.” Techopedia, February 25, 2019. https://www.techopedia.com/definition/2977/smartphone 75 | TOOLKIT FOR DESIGNING A COMPREHENSIVE DISTANCE LEARNING STRATEGY USAID.GOV x Thakur, Vijay Singh. “Using supplementary materials in the teaching of English: Pedagogic scope and applications.” English Language Teaching, 8, 1-6. 2015. Retrieved from: https://files.eric.ed.gov/fulltext/EJ1084305.pdf y Reddy, R. Sucharan. “Supplementary materials to enhance language skills of learners.” International Journal of Social Science and Interdisciplinary Research, 2, 143-150. 2013. Retrieved from: http://www.indianresearchjournals.com/pdf/IJSSIR/2013/September/16.pdf z Robinson, Christopher. “What is survey software? A comprehensive guide to benefits, features, types, pricing and more.” Finances Online, 2020. https://financesonline.com/survey-software-comprehensive-guide-benefits-features￾types-pricing/ aa Great Schools Partnership. “Synchronous Learning.” Great Schools Partnership, August 29, 2013b. https://www.edglossary.org/synchronous-learning/ bb Teaching Excellence in Adult Literacy (TEAL). Universal Design for Learning. American Institutes for Research, 2010. https://lincs.ed.gov/sites/default/files/2_TEAL_UDL.pdf