Evaluation of the Jordan Education Initiative Report: Task 3 REVIEW OF THE TECHNOLOGY EMPLOYED TO DELIVER E-LEARNING December 10, 2007 This publication was prepared by RTI International on behalf of Education Development Center, Inc., for the United States Agency for International Development. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 2 of 61 Report: Task 3 Review of the Technology Employed to Deliver E-Learning Submitted to: USAID/Jordan through Education Development Center, Inc. Submitted by: Gordon M. Cressman, RTI International John Daly, Consultant to RTI International Project: Evaluation of the Jordan Education Initiative Prime Contract: Assistance to Basic Education / Basic Education, EDH-I-02-05-00031-00 Task Order: 02 The Authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Cover Photo Credits (Clockwise from top left): G.M. Cressman, G.M. Cressman, G.M. Cressman, D. Light Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 3 of 61 Acknowledgements This report would not have been possible without the support, cooperation, and organizational skills of the Program Management Office (PMO) of the Jordan Education Initiative (JEI). Projects Coordinator, Luma Atallah, and In-classroom Technology Track Manager, Niveen Jum’a, provided exceptional support, managing to arrange critical meetings, school visits, and focus groups in an incredibly short amount of time. To a great degree this confirmed the exceptional abilities of the PMO team and their relationships with the Ministry of Education and Discovery Schools. The entire PMO team, including Executive Director Haif Bannayan, was transparent and direct in demanding a clear and objective report with specific recommendations. The Authors also thank the many principals, teachers, lab technicians, and JEI partner organizations who invested time participating in interviews and focus groups. Finally we thank the United States Agency for International Development for providing the support, objectives, and encouragement for this important assessment. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 4 of 61 Abbreviations 1–12............Grades 1–12 AD...............(Microsoft) Active Directory ADL.............Advanced Distributed Learning Initiative ADSL ..........Asymmetric Digital Subscriber Line CA...............Computer Associates Inc. CD-ROM.....Compact Disk–Read Only Memory CDRW ........Compact Disk Read Write CEB.............Computer Engineering Bureau CIDA ...........Canadian International Development Agency CJI...............Connecting Jordanians Initiative CLI ..............Cisco Learning Institute CMS............Content Management System CoSN ..........Consortium for School Networking CPU ............Central Processing Unit DHCP..........Dynamic Host Configuration Protocol DoD.............(U.S.) Department of Defense DSS.............Decision Support System DVD ............Digital Versatile Disk EDC ............Education Development Center, Inc. ECE.............Early Childhood Education e-content.....Electronic content e-curricula...Electronic curricula e-learning....Any technologically mediated learning EFL .............English as a Foreign Language EMIS...........Education Management Information System ERfKE.........Education Reform for a Knowledge Economy ERP.............Enterprise Resource Planning GB...............Gigabyte Gbps ...........Gigabits per second GE...............Gigabit Ethernet GIS..............Geographical Information System GPS ............Global Positioning System IAB ..............Interactive White Board ICDL............International Computer Drivers License ICT ..............Information and Communication Technology IMS..............Instructional Management System ISA ..............(Microsoft) Internet Security and Acceleration Server ITG..............Integrated Technology Group IWB.............Interactive White Board JEI...............Jordan Education Initiative JT................Jordan Telecom JTG.............Jordan Telecom Group KB ...............Kilobyte Kbps............Kilobits per second LAN.............Local Area Network LMS.............Learning Management System LTO.............Linear Tape-Open M&E............Monitoring and evaluation Mbps...........Megabits per second MEPI...........Middle East Partnership Initiative Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 5 of 61 MIT..............Massachusetts Institute of Technology MoE.............Ministry of Education MoICT.........Ministry of Information and Communication Technology NAS.............Network Attached Storage NBN ............National Broadband Network NGO............Nongovernmental Organization OPLP..........One Laptop per Child OSS ............Open Source Software PC...............Personal Computer PDA.............Personal Digital Assistant PMO............Project Management Office PO...............Purchase Order PoE.............Power over Ethernet QRC............Queen Rania Center RAID ...........Redundant Array of Inexpensive Disks ROI..............Return on investment SAN.............Storage Area Network SBN.............School Broadband Network SCORM......Sharable Content Object Reference Model SIS/SMS.....Student Information and School Management System SJE .............Supporting Jordan’s Education Project SLA.............Service-level agreement SME............Subject Matter Expert SMS............System Management Server SQL.............Structured Query Language TCO ............Total Cost of Ownership U.K. ............United Kingdom U.S. ............United States (of America) UBN ............University Broadband Network UNDP..........United Nations Development Programme UNESCO....United Nations Educational, Scientific and Cultural Organization USAID.........United States Agency for International Development USB.............Universal Serial Bus VOI..............Value of Investment VoIP............Voice over Internet Protocol WAN ...........Wide Area Network WB..............World Bank WEF............World Economic Forum WSIS...........World Summit on the Information Society WSUS.........Windows Software Update Services Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 6 of 61 Contents Acknowledgements ....................................................................................................................................3 Abbreviations ..............................................................................................................................................4 Contents......................................................................................................................................................6 1 Introduction ..........................................................................................................................................8 2 The Education System Context ........................................................................................................10 3 Developing E-content........................................................................................................................11 3.1 Technical standards and guidelines............................................................................................. 12 3.2 The role of public-private partnerships......................................................................................... 13 3.3 Monitoring and evaluation (M&E)................................................................................................. 14 3.4 Updating e-content........................................................................................................................ 15 3.5 Conclusions................................................................................................................................... 16 4 Serving E-content ..............................................................................................................................16 4.1 EduWave....................................................................................................................................... 16 4.2 Server infrastructure, hosting, support, and capacity planning................................................... 18 4.3 Conclusions................................................................................................................................... 19 5 Accessing E-content from the Classroom ........................................................................................19 5.1 Computer labs............................................................................................................................... 19 5.2 Laptop computers ......................................................................................................................... 21 5.3 Interactive white boards................................................................................................................ 23 5.4 Connecting to EduWave and the Internet.................................................................................... 24 5.4.1 ADSL................................................................................................................................... 24 5.4.2 National Broadband Network (NBN)................................................................................. 24 5.4.3 Internet access ................................................................................................................... 26 5.4.4 Security and bandwidth management............................................................................... 27 5.5 Comments from teachers............................................................................................................. 29 5.6 Conclusions................................................................................................................................... 30 6 Maintaining Access to E-content.......................................................................................................31 6.1 Organization, communication, and coordination......................................................................... 31 6.2 Technical support, maintenance, and replenishment ................................................................. 32 6.2.1 Problem reporting, escalation, and response................................................................... 32 6.2.2 The JEI intern program ...................................................................................................... 33 6.2.3 Monitoring and evaluation (M&E)...................................................................................... 34 6.3 Strategic planning, management, and financing of technology.................................................. 35 6.4 Conclusions................................................................................................................................... 38 7 Conclusions .......................................................................................................................................39 Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 7 of 61 7.1 The role of the JEI......................................................................................................................... 39 7.2 Selecting ICT models.................................................................................................................... 40 7.3 ICT models piloted in Discovery Schools .................................................................................... 41 7.3.1 Computer labs .................................................................................................................... 41 7.3.2 Laptop computers and portable data-show projectors..................................................... 42 7.3.3 Media rooms and interactive white boards ....................................................................... 43 7.4 Other possible ICT models........................................................................................................... 43 7.5 Developing and maintaining e-content ........................................................................................ 46 7.6 Scaling-up results.......................................................................................................................... 47 7.7 General conclusions ..................................................................................................................... 48 8 Recommendations.............................................................................................................................48 8.1 Develop simple, sustainable solutions to critical infrastructure problems.................................. 49 8.2 Expand the internship program to further strengthen technical support capacity ..................... 49 8.3 Prove these solutions work in Discovery Schools before scaling up ......................................... 49 8.4 Develop a model for scaling up that fosters, rewards, and disseminates success ................... 50 8.5 Expand public-private partnerships to include employers of knowledge workers..................... 50 Annex A: Schedule of Meetings and Names of Primary Contacts.........................................................51 Annex B: Focus Group Script and Questions..........................................................................................54 Annex C: List of Reference Documents and Internet Resources...........................................................57 Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 8 of 61 1 Introduction The Jordan Education Initiative (JEI) was created in 2003, with the assistance of the World Economic Forum (WEF), to apply public-private partnerships to improve the application of information and communication technology (ICT) in grades 1–12 in Jordanian schools. The four primary objectives of the JEI are as follows:1 1. Improve the development and delivery of education to Jordan’s citizens through public￾partnerships, and, in the process, help the government of Jordan achieve its vision for education as a catalyst for social and economic development 2. Encourage the development of an efficient public-private model for the acceleration of educational reforms in developing countries based on unleashing the innovation of teachers and students through the effective use of ICT 3. Build the capacity of the local information technology industry for the development of innovative learning solutions in partnership with world-class firms, creating economic value that will lead to mutually beneficial business opportunities 4. Leverage an environment of national government commitment and corporate citizenship to build a model of reform that can be exported to and replicated in other countries. To achieve these objectives, a coalition was formed involving international and domestic companies and Jordanian government agencies. The efforts of this coalition have been coordinated and managed by the JEI Program Management Office (PMO). As indicated by Objective 4 above, the JEI has been viewed by the WEF as a flagship effort to produce a model, lessons-learned, and best practices that can be applied to other countries with similar objectives. Under the project, content has been developed to enrich the curriculum in core subjects, including math, science, Arabic, English as a Foreign Language (EFL), and ICT. This content is provided electronically and used by students in computer labs and by teachers equipped with laptops and “data￾show” projectors2 in the classroom. The program has focused on almost 100 selected “Discovery Schools” where teachers and laboratory technicians have been trained. The WEF project was limited to 3 years. The JEI PMO intends to continue as a nongovernmental organization (NGO) and is in the process of making that transition, which involves new sponsorship, new facilities, some new staffing, and new functions. The Authors form part of a larger team conducting a four-part assessment of JEI program strategy and implementation. The four parts of the assessment are as follows: ƒ Task 1: Assess the impact of using e-learning modules on students/teachers, schools, and the overall educational system ƒ Task 2: Assess the role of public-private partnerships ƒ Task 3: Assess the hardware and delivery infrastructure employed to deliver e-learning 1 Jordan Education Initiative Brochure, October 2004. 2 The term “data-show” is commonly used in Jordan to refer to a stationary or portable projector designed to project video and computer displays. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 9 of 61 ƒ Task 4: Assess the total cost of ownership for employing e-learning as a core part of instruction within the schools. The focus of this report is Task 3, which assesses JEI strategy and implementation with respect to information technology models and infrastructure intended to support the use of electronic content for teaching and learning. Task 3 of the JEI assessment consists of three major tasks as follows: 1. Review, in coordination with the EDC Team for Tasks 1 and 2, the existing and planned applications of technology supported by JEI partners, as well as for future partnerships 2. Review the hardware and software requirements for JEI activities 3. Review the capacity for JEI hardware and infrastructure delivery and management. While it is a crucial issue for the JEI at it moves forward, this report does not assess the educational results, outcomes, or impacts of the e-curriculum or ICT models introduced by the JEI. Those issues are being addressed in other parts of this overall assessment. This report documents the findings of the Authors based on a review of background documents and in￾person site visits, interviews, and focus group discussions conducted by the Authors in Amman, Jordan, between August 31 and September 11, 2007. We have done our best to document our observations carefully and accurately, and to base our conclusions and recommendations on this data alone. However, we are aware of the limitations of this data and take full responsibility for any inaccuracies in this report. While this report focuses on information technology models and infrastructure deployed by the JEI, it does so within the context of overall JEI objectives. These objectives require transferring successful ICT models to the Ministry of Education (MoE) and eventually introducing these models to all or most Jordanian public schools. Thus this report evaluates ICT models deployed by the JEI within 100 selected Discovery Schools in Amman, as well as the challenges of transferring successful models to the MoE and scaling them nationally. This report also considers the role of the JEI and public-private partnerships in this process to date, and whether and how those roles may or should change. Chapter 2 of this report briefly summarizes the education system context in Jordan, focusing on the intended role of the JEI and ICT in education reform in Jordan. Chapter 3 discusses the process of developing, testing, monitoring, and updating electronic content (e-content) for use by teachers and students in the blended learning approach promoted by the JEI. Chapter 4 assesses key infrastructure for serving e-content to the schools, including the EduWave software platform and MoE server infrastructure, hosting facilities, and related technical support. Chapter 5 assesses ICT models for accessing e-content in the classroom, including stationary computer labs, mobile laptop computers and projectors, and interactive white boards, as well as the network infrastructure necessary to connect these access points to the centralized e-content server facility. Chapter 6 assesses critical elements of maintaining access to e-content, including organization, communication, coordination, technical support systems and capacity, and strategic planning and management. Chapter 7 summarizes major conclusions of this assessment. Finally, Chapter 8 provides specific recommendations for the JEI. Annex A contains a list of the meetings, site visits, and focus groups conducted by the Authors during this period. Annex B contains a script and list of questions used to guide focus group discussions conducted by the Authors with principals and teachers in two Discovery Schools. Annex C contains a list of background documents, some referenced specifically in this report and others providing general background for this assessment. This report is to be submitted to the team leading the overall assessment of the JEI, and is intended to be the basis for a portion of their overall report. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 10 of 61 “The United Nations Educational, Scientific and Cultural Organization (UNESCO) 2002 report Information and Communication Technology in Education describes a four stage model showing a continuum of approaches to ICT development in schools. The model describes school jurisdictions moving through the stages: Emerging ¨ Applying ¨ Infusing ¨ Transforming Using this model, Jordan is likely at the “applying” stage—a stage where: ƒ most of the computers in schools are in computer labs, ƒ teachers still dominate the learning environment, and ƒ ICT is beginning to be used in various subjects and not only for teaching about ICT. Jordan is poised to move into the “infusing” stage. At this stage, ICT is integrated or embedded across the curriculum. Schools begin to use a range of computer-based technologies in laboratories and classrooms. Teachers begin to explore how ICT can change their personal productivity and teaching approaches.” 2 The Education System Context Jordan is conducting a major reform of its educational system under a program called Educational Reform for the Knowledge Economy (ERfKE). The ERfKE program seeks to provide a human resource development structure needed for Jordan’s emergence as a knowledge economy. The project is comprised of four components that target education policy and strategy, curriculum and teacher upgrading, infrastructure and physical upgrading, and also early childhood education (ECE). The project includes training a total of 60,000 teachers and administrators in different levels of ICT skills and uses of ICT in classrooms, and produces electronic content consistent with approved curricula and a blended learning approach. It is supported by the World Bank (WB) in partnership with the governments of Canada, the United States, Britain, and Japan, and is scheduled to be completed by early 2008. The JEI operates in the context of ERfKE. Thus the MoE specified the curriculum to be used. So too, it defined the blended learning approach to be taken, combining textbooks designed for a nationwide curriculum with the developed courseware enriching and complementing classroom presentations. The national system of school computer labs also defined the approach, as did major factors such as student-teacher ratios and students per classroom. Specifically, Jordanian schools do not have computer-to-student ratios comparable to those in other, richer countries. Moreover, the MoE’s limited resources defined the rapidity with which the corps of teachers could be trained in basic ICT skills and in pedagogic approaches to use the technology effectively. The four-stage model referenced in the Supporting Jordan’s Education (SJE) Project/ Canadian International Development Agency (CIDA) report of 20063 is useful: Exhibit 1 shows the organizational structure of the JEI. Overall strategic direction is invested in an executive steering committee, which is a subset of a larger stakeholder committee. Implementation is the responsibility of the PMO, which is advised by an academic advisory board. The activities of the JEI are organized into three main tracks. One of these, Track 1 Discovery Schools, is divided into three subtracks consisting of in-classroom technology, electronic curricula (e-curricula), and training. Each 3 Classroom and Computer Lab Deployment Strategy in Government Schools Grades 1–12. The Directorate of Curriculum and Textbooks of the MoE and the Supporting Jordan’s Education Project, SJE/CIDA, September 2006. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 11 of 61 track and subtrack has an assigned manager within the PMO. This report focuses exclusively on Track 1. Exhibit 1 Organizational Structure of the Jordan Education Initiative track 1 discovery schools track 1b e-curricula track 1a In-classroom technology track 1c training track 2 lifelong learning track 3 ICT industry development program management office (PMO) Based in Jordan Full time program managers Overall program responsibility academic advisory board Stakeholder Committee Executive steering committee 3 Developing E-content E-curricula development is one of three JEI subtracks under Track 1 Discovery Schools. JEI has played a critical role in supporting the development of e-curricula for the following subjects: ƒ Math ƒ Science ƒ Arabic ƒ EFL ƒ Civics ƒ ICT The JEI played a key role in evaluating and implementing e-content in Discovery Schools. This has included helping to move content from lesson plans to story boards to electronic media, following up on e-content development, quality assurance, and coordination. The JEI conducts weekly meetings among e-content developers and MoE Subject Matter Experts (SMEs) to identify and resolve obstacles. The JEI also helps to follow up implementation and address problems with ICT infrastructure and teacher training. The JEI has also played a role in facilitating and accelerating procedures through the MoE bureaucracy. The Digitization Department of the MoE Curricula and School Textbooks Directorate believes JEI has played a critical role in The Digitization Department of the MoE Curricula and School Textbooks Directorate believes the JEI has played and continues to play a critical role in developing e-content. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 12 of 61 developing e-content. They believe strongly that they could not do their work without the support of the JEI. The JEI has also played an important role in making connections between the companies involved in content production and delivery to schools, as well as training and planning for roll out in Discovery Schools. Rubicon, developer of e-math and e-science content, believes that JEI could do more to facilitate communication among the various stakeholders involved in developing e-content. Rubicon also reported that they would like more and more timely feedback from teachers. They have not received any feedback from students or parents. The lack of effective mechanisms to gather, analyze, and distribute feedback from the schools to service providers is significant and surprising. We found that e-content developed for EduWave4 is not the only e-content used by teachers and students. Teachers reported that they frequently used content from the Internet to supplement textbook material, and that students may use the Internet for class projects. Teachers and students also developed their own e-content using Microsoft PowerPoint, Microsoft Movie Maker, and other software. We observed some of this during our visits to Discovery Schools. This Individually developed e-content could be a valuable resource if such content could be shared easily. Such a sharing facility should enable teachers to comment on content contributed by peers and nominate contributions for special recognition. Peer and official review of shared content could also make sure contributed content is appropriate within the official curriculum. While EduWave has some group collaboration features that might be used to do this, they are not specifically designed for this purpose and have yet to be introduced into Discovery Schools. 3.1 Technical standards and guidelines Guidelines and standards for developing e-curricula5 for Discovery Schools were developed through a series of workshops in April and May of 2004. Workshop participants included the JEI, the MoE, and the Integrated Technology Group (ITG). ITG is the developer of the EduWave e-learning platform software. Draft guidelines and specifications were reviewed again during a workshop in October 2004. The latest version, Version 9, was published on September 9, 2004. These guidelines and standards have not been updated since then. The guidelines and standards are generally clear and encompass a wide range of current and future technical delivery options. Section 5 of the Guidelines and Standards for e￾Curriculum in JEI Discovery Schools details technical guidelines and standards for developing e-content. The standards call for e-content to be accessible from a computer device over a broadband6 Wide Area Network (WAN) or broadband Local Area Network (LAN) (wired or wireless). The standards also mention that e-content may be accessed via low bandwidth or dial-up modem connections, or wireless Personal Digital Assistants 4 EduWave is a comprehensive e-learning software platform developed by the Integrated Technology Group (ITG). EduWave is discussed in more detail in Chapter 4 of this report. 5 In this report, the terms “e-curricula” and “e-content” are used interchangeably. 6 The term “broadband” is not defined, but is a relative term generally used in data communications to refer to speeds of 200 kilobits per second (Kbps) or greater. The guidelines call for protection against computer viruses and other malicious programs that could compromise the integrity and stability of the content. The JEI could help improve communications among stakeholders involved in developing e-content, and could improve feedback mechanisms from teachers to developers. Additional e-content created by teachers and students could be shared online using a peer-review system that would reward innovation. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 13 of 61 (PDAs). Off-line use of e-content is listed as a “future option.” The guidelines also call for protection against computer viruses and other malicious programs that could compromise the integrity and stability of the content. As described elsewhere in this report, this protection is not in place, and viruses and other malicious software are a significant source of computer instability and computer performance problems in the schools. The technical standards call for all e-content to be compatible with the Sharable Content Object Reference Model (SCORM), a standard of the Advanced Distributed Learning (ADL) Initiative, an initiative to standardize development and implementation of learning technologies across the United States Department of Defense (DoD). SCORM is a widely recognized standard for packing e-content so that it is portable among many different Learning Management Systems (LMS). This ensures that the e-content is compatible with EduWave. The only major area of controversy concerning the specifications appears to be the requirement that any rich media files (e.g. Flash animation or videos) be less than 250 kilobytes (KB) in size for acceptable performance over narrowband7 connections. Rubicon, one of the JEI e￾content development partners, believes that this size limitation is excessively restrictive, and does not permit them to package e-content in the most effective way for use in computer labs. The e-science content is particularly rich in animated content. Rubicon has packaged e-science for grades 1–12 using “supermedia,” large files that enable a teacher or student to navigate through the rich content without having to make repeated requests to EduWave for successive portions. Rubicon believes strongly that the current limitation on the size of rich media files is too restrictive for developing effective e-content in technical subjects in higher grades. The guidelines mention off-line use of e-content as a future option. Rubicon staff stated that it should be possible to cache e-content on a server in each school. They report that e-math content occupies 0.5 gigabytes (GB) of disk space and e-science content occupies 0.7GB of disk space. Because this content is typically updated once per calendar year, it is reasonable to consider caching this content locally in each school. This issue is discussed further in Section 5. 3.2 The role of public-private partnerships Private sector partners have played a key role in developing e-content. Rubicon, a Jordanian company specializing in multimedia e-education and 3D animation, and the Cisco Learning Institute (CLI), developed the e-math content with support from Cisco Systems Inc., and the e-science content with support from FastLink (now rebranded as Zain). e-Dimension, a subsidiary of the Jordan Telecom Group (JTG), has developed e-Arabic content with support from France Telecom, which holds a majority interest in JTG. Rubicon and CLI have collaborated to produce the e-EFL content with support from the United States Agency for International Development (USAID) Middle East Partnership Initiative (MEPI). Jordanian firm Menhaj Educational Technologies has developed the ICT e-content with support from Microsoft, and the e-Civics content with support from MEPI. There does not appear to be a close collaborative relationship between ITG and all e-content developers. The ITG does not believe that Rubicon, and perhaps other third-party e-content 7 The term “narrowband” generally refers to communication connections that provide less than 200Kbps bandwidth. The term is most commonly used to refer to dial-up modem connections. The JEI has played an important role in bringing the various stakeholders together to discuss and resolve problems. Rubicon believes strongly that the current limitation on the size of rich media files is too restrictive for developing effective e-content in technical subjects in higher grades. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 14 of 61 developers, have made the best use of EduWave features. The fact that the ITG owns both the LMS platform and a competing subsidiary (JAID Productions) places the ITG in a commanding position with respect to other e-content developers. However, we heard from the ITG and other stakeholders that the JEI has played an important role in bringing the various stakeholders together to discuss and resolve problems. 3.3 Monitoring and evaluation (M&E) The staff of the JEI PMO is quite small (eight people at the time of this assessment), and does not appear to be able to closely monitor the development of the e-content by the partner firms. The evaluation of that content, in terms of its suitability to complement the Jordanian public school curriculum, the degree to which it complements textbooks and its acceptance by teachers is done by a committee convened by the MoE. The JEI worked with the MoE to develop a “site survey” to monitor and evaluate the roll out of e-content in Discovery Schools. The site survey collects data from each Discovery School on the use of e-content by subject and grade, the use of computer labs and laptop computers, training of teachers in the use of ICT and e-content, and the status and technical support of computer and network infrastructure. The JEI normally conducts site surveys every 3 months during the school year. Each site survey requires 3 weeks for data collection, processing, analysis, and reporting. The data are collected and entered by NetCorps interns managed by JEI and are processed and analyzed by the JEI team. JEI conducted the most recent site survey, its 12th, in May 2007. Work is already underway on the 13th site survey. JEI track managers share and discuss results with relevant MoE central directorates before they are presented and discussed with the Secretary General of the MoE and with His Excellency the Minister of Education. Results are also shared with field officers and directors in Discovery School education directorates. The JEI site survey has been critical in monitoring progress in introducing e-content and information technology in Discovery Schools and has been used very effectively by the JEI team to identify problems, and to take and monitor corrective action. It is the only objective M&E system in place to monitor the process of introducing e-content into schools and the status and technical support of the necessary ICT infrastructure. Site survey reports show that the JEI team has used the results to clearly identify emerging problems and to recommend specific corrective action. JEI reports also show that they have used this tool very effectively for planning and resource allocation in Discovery Schools. The JEI collects information from teachers regarding difficulties in using e-content through direct observation and interviews. There is no regular monitoring system to measure the reliability and performance of e-content delivery by EduWave over the network to the classroom. Lab technicians have been instructed to test and report bandwidth performance periodically, but the MoE has no automated system to measure bandwidth performance or analyze network traffic at the school level. Neither the ITG, nor e-content developers, nor the MoE regularly observe the use and performance of the e-content delivery system in the classroom. Therefore EduWave e-content may be underutilized for a significant period of time before the MoE becomes aware there are technical problems with delivery. Once teachers lose confidence in the delivery of e-content to the classroom, it may be difficult to win them back. In addition, the MoE has The JEI site survey has been used very effectively by the JEI team to identify problems, and to take and monitor corrective action. There is no regular monitoring system to measure the reliability and performance of the e-content delivery system to the classroom. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 15 of 61 no way to determine easily whether problems are due to inadequate PC maintenance, malicious software in computer labs, unreliable or slow network connections to schools, inadequate performance of the MoE data center, excessive bandwidth demand, or poor EduWave performance. Hence when significant problems become apparent to the MoE, much time, effort, and money may be wasted identifying the problem and implementing an effective solution. Most teachers interviewed during this assessment reported that the use of e-content and ICT were powerful tools in engaging and holding the interest of students. In higher grades, however, many expressed concern over their ability to prepare students for the final Tawjihi test8 , which is based on textbooks, while also using e-content in a blended learning approach. No changes have been made to the Tawjihi test in coordinating it with the introduction of e-content. JEI site survey results clearly show a reduction in use of e-content at higher grade levels. National test scores in Jordan have decreased. This decrease may be caused by many factors, including the challenge of absorbing many Iraqi children into the system. Several JEI stakeholders, including the MoE, appear to be considering seriously the need for monitoring the outcomes of introducing e-content and blended learning approaches. Such monitoring is essential to make adjustments in strategy and tactics to improve results. Now in its third year, use of e-math may have reached a point where intermediate outcomes can be evaluated. There are some useful guides to designing M&E systems to evaluate the effectiveness of ICT investments in education. These include guidelines developed by the Central Regional Educational Laboratory (2004) in the United States, and a comprehensive handbook developed by Wagner and others (2005) for the World Bank InfoDev program. The JEI could play an important role in facilitating and designing a more effective system for monitoring and evaluating education outcomes. While the MoE should be responsible for implementing an M&E system for education outcomes, the system should be designed with the help of national and international experts in this field. Without such a system, Jordan has no way to measure whether the JEI is achieving its intended results. 3.4 Updating e-content E-content is reviewed and updated each year. Updated e￾content must be reviewed and approved by the MoE Curricula and School Textbooks Directorate before it is accessible via EduWave. Now in its third year of use, e￾math has been through this upgrade cycle twice. Guidelines and technical standards for e-content have not been reviewed and updated during this period, though information and communication technology continues to change. In addition to changes in technology, feedback from teachers and subject matter experts should also be considered in reviewing and updating e-content. Finally, it is important to coordinate and review the work of e-content developers in updating e-content and incorporating new technology. The MoE needs to develop effective formal policies and procedures for reviewing and updating e-content guidelines, standards, and content. The JEI could play an important role in facilitating this work, and could also help to coordinate and monitor the implementation and testing of updates by e-content developers. 8 Tawjihi is the general secondary examination in Jordan. Students in secondary academic and vocation programs in Jordan sit for the general secondary examination at the end of two years. Those who pass are awarded the Tawjihi (General Secondary Education Certificate). Without a more effective system for monitoring and evaluating education outcomes, Jordan has no way to measure whether the JEI is achieving its intended results. The MoE needs to develop effective formal policies and procedures for reviewing and updating e-content guidelines, standards, and content. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 16 of 61 3.5 Conclusions 1. The JEI has played and continues to play a critical role in developing e-content, coordinating and facilitating the work of the various stakeholders, and providing solutions to bureaucratic obstacles within the MoE. The JEI has also played an important role in bringing the various stakeholders together to discuss and resolve problems. 2. The lack of a continuous feedback from the classroom to e-content developers is a significant weakness in e-content development. The JEI could help improve communications among stakeholders involved in developing e-content, and could improve feedback mechanisms from teachers to developers. 3. Teachers and students are creating their own content using a variety of software tools. Additional e-content created by teachers and students could be shared online using a peer￾review system that would reward innovation. EduWave might be used to provide this sharing facility. 4. E-content guidelines call for protection against computer viruses and other malicious programs that could compromise the integrity and stability of the content, yet viruses and other malicious software are a major cause of computer instability and performance problems in schools. It is essential for the MoE to implement a simple solution that it can afford to sustain. 5. The current limitation on the size of rich media files may be too restrictive for developing effective e-content in technical subjects in higher grades. This restriction may no longer be necessary as connectivity improves or schools are able to cache and serve e-content locally. 6. The JEI site survey has been used very effectively by the JEI team to identify problems, and to take and monitor corrective action in Discovery Schools. However, there is no regular monitoring system to measure the reliability and performance of the e-content delivery system to the classroom. The JEI could play an important role in facilitating and designing effective monitoring and evaluation systems. 7. The MoE should be responsible for implementing an M&E system for education outcomes. The system should be designed with the help of national and international experts in this field. Without such a system, Jordan has no way to measure whether the JEI is achieving its intended results. 8. The MoE needs to develop effective formal policies and procedures for reviewing and updating e-content guidelines, standards, and content. The JEI could play an important role in facilitating this work. 4 Serving E-content 4.1 EduWave Schools have access to e-content through the EduWave™ e-learning platform developed by the ITG. The ITG began developing EduWave in 2000. EduWave is a Web-based application designed for centralized management of e-content and other data. The software includes LMS, Content Management System (CMS), Instructional Management System (IMS), Student Information and School Management System (SIS/SMS), Education Management Information System (EMIS), Decision Support System (DSS), and Enterprise Resource Planning (ERP) features. The system also Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 17 of 61 includes authoring and individual student assessment tools, as well as collaborative tools, such as discussion forums and audio/video conferencing. EduWave is a large, complex software product. In January 2003, the MoE signed an agreement with ITG to use the product to deliver e-content nationally to an estimated 1.2 million students in 3,200 schools nationally. Other EduWave actual or planned deployments to date include two school districts in New Jersey, U.S., and schools in Bahrain. ITG’s business model for EduWave focuses on large￾scale implementation at the school-system level. EduWave is not designed for implementation in individual schools, but is designed to serve all information to all schools within a school system from a single central location. EduWave’s current centralized deployment architecture depends on the reliability and performance of connectivity to each computer in each computer lab and to each laptop computer used to access e-content. ITG is aware of reports of performance problems accessing e-content from schools, but blames this primarily on connectivity problems and resistance to change on the part of teachers. ITG believes that some teachers need to be “pushed” to use the e￾content. ITG receives information concerning EduWave performance in school computer labs through the Queen Rania Center (QRC). ITG has not sent its software engineers into computer labs to observe how teachers use EduWave or to observe performance problems firsthand. EduWave does not support a distributed model that would enable off-line access to data and content at the school level while automatically synchronizing data and content with a centralized database. While ITG offers an off-line viewer for EduWave e-content, the MoE has not purchased this component. ITG is adamant that its centralized technical approach is the best solution and does not appear to be open to considering a model that caches content at the school level. ITG does not believe the MoE has the technical capacity or funding to maintain an EduWave server installation in each school. The MoE is currently using LMS, CMS, IMS, and SIS/SMS elements of EduWave to manage e-content and student information, including end-of-year grades. According to ITG, schools posted some 70 million to 80 million records via the EduWave SIS/SMS module at the end of the previous school year. The MoE is not using any of the collaboration and information-sharing tools offered by EduWave. These include e-mail, online multithreaded discussion forums, and built-in audio/video conferencing. Some of these features might be used in the future by teachers to share information about using EduWave features and e-content, and to share other e-content that they develop, such as PowerPoint presentations. Audio/video conferencing might be used to enable a teacher in one school to teach students in other schools. ITG believes that JEI has been excessively focused on e-content while ignoring the potential of other EduWave features. However, ITG also believes that JEI has played an important role in getting the various stakeholders to discuss and resolve problems concerning the deployment and use of e-content using EduWave. The architecture of EduWave depends on the reliability and performance of connectivity to each computer in each computer lab and to each laptop computer used to access e-content. ITG believes that JEI has been excessively focused on e-content while ignoring the potential of other EduWave features. ITG has not sent its software engineers into computer labs to observe how teachers use EduWave or to observe performance problems first-hand. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 18 of 61 Load-testing software can simulate the expected number of students at each level and subject during peak usage based on the expected roll out of e-content to schools, and as enrollment is expected to increase over time. 4.2 Server infrastructure, hosting, support, and capacity planning EduWave was hosted on MoE servers at the QRC during initial roll out to schools. During this period, according to Jordan Telecom (JT), there were many problems caused by lack of an Internet firewall. EduWave is now hosted in the MoE data center colocated at the JT Hashem Data Center. JT provides data center space, electricity, climate control, fire suppression, and security. JT’s role in MoE data center operations is to provide these basic facility services, technically manage and configure data center equipment, replace faulty hardware, and install new hardware. ITG manages all EduWave server operations remotely, including scheduled backups to Linear Tape-Open (LTO) tape media. JT is in the process of converting the Hashem Data Center into a secondary disaster recovery data center. While other colocated customer data centers may be moved to a new JT data center, the MoE data center will remain in its current location. Based on discussions with JT operations personnel at the Hashem Data Center and separately with ITG, there may be some confusion concerning the roles and responsibilities of the three parties (JTG, ITG, and the MoE) with respect to EduWave server operations. Neither JTG nor ITG seems clear regarding responsibilities for data security. JTG does not have access to server Administrator accounts and believes it should be responsible for operating system and routine tape backup operations. The MoE data center currently consists of five 92-unit (U) racks installed in a separate enclosed space in a larger raised-floor data center area. The racks include network devices, servers, disk storage arrays, and backup tape drives. The EduWave application runs on 15 blade servers running Microsoft Windows 2003 Server and Internet Information Server 6 operating behind Microsoft load balancing. Relational database management is handled by two clustered servers running Microsoft Structured Query Language (SQL) Server. In addition there are two servers dedicated to Microsoft System Management Server (SMS), which are discussed later in this report. Disk storage arrays include Redundant Array of Inexpensive Disks (RAID) 1 mirrored hard disk drives on rack-mounted servers and blade servers, a Storage Area Network (SAN) disk array, and Network Attached Storage (NAS). There is significant expansion capability in the SAN disk array and some expansion capability in the existing racks. There has been considerable discussion regarding the decision to serve EduWave e-content to all schools from this single location. Reliability and performance problems reported by schools accessing e-content via EduWave have resulted in some finger pointing among ITG, JT, the National Broadband Network (NBN)9 , and the MoE. Although the hardware configuration at the Hashem Data Center should have the capacity to meet the demand, performance problems are blamed variously on the 9 The National Broadband Network is also known as the National Broadband Learning and Research Network. Exhibit 2 The Ministry of Education data center co-located at the Jordan Telecom Hashem Data Center. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 19 of 61 Hashem Data Center, ADSL (Asymmetric Digital Subscriber Line) connections to schools, the reliability of the NBN, and virus problems in school computer labs. JT reports that Central Processing Unit (CPU) utilization of the servers at the Hashem Data Center is low, even during peak demand, but the question of the capacity of the Hashem Data Center servers can be determined reliably through load testing. Load-testing software can simulate the expected number of students at each level and subject during peak usage based on the expected roll out of e￾content to schools, and as enrollment is expected to increase over time. This testing can be used to verify the capacity of the Hashem Data Center to meet projected demand, and to plan in advance for any necessary increases in server or storage capacity. Such load testing should be done jointly by the ITG and the MoE. Results can be used to proactively plan data center upgrades and to help determine whether data center performance plays any role in EduWave performance problems reported by schools. 4.3 Conclusions 1. As ITG’s main reference site, the MoE should insist that ITG work with content developers, network connectivity providers, and server hosting engineers to observe problems firsthand and diagnose and resolve them collectively. ITG should send its software engineers into computer labs to observe how teachers use EduWave and to observe performance problems. 2. Reliability and performance problems with connectivity have inhibited use of e-content. There appears to be no technical reason why e-content must be served live to all schools from a single location. It should not be necessary to replicate all EduWave functions at the school level. The JEI and the MoE should work with ITG to enable schools to use e-content off-line. This may include developing a simple school-level proxy caching solution. 3. The JEI should examine other EduWave features to determine whether they may be useful and how and when they may be introduced in Discovery Schools. 4. Server and storage capacity at the Hashem Data Center appear to be more than sufficient and can be expanded if necessary. Load-testing software could be used to simulate usage based on the expected roll out of e-content to schools, and as enrollment is expected to increase over time. This would help eliminate any questions of server performance and could be used to plan capacity increases well in advance of need. 5 Accessing E-content from the Classroom 5.1 Computer labs The computer lab is the primary ICT model used by the JEI in Discovery Schools. Each Discovery School is equipped with one to six computer labs. Most Discovery Schools have at least two com puter labs. Each Discovery School computer lab is equipped with 10 to 20 industry-standard desktop PCs (personal computers) and a single server. MoE standards call for each computer lab to be equipped with at least 20 PCs. Most Discovery Exhibit 3 Students at Al-Shifa’ Bint Awf School in one of two computer labs equipped with Windows desktop PCs. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 20 of 61 School computer labs meet this standard. The PCs are equipped with CD-ROM drives used for some e-content and for student projects. The servers currently provide Dynamic Host Configuration Protocol (DHCP) services for dynamic IP address assignment, and Microsoft ISA (Internet Security and Acceleration Server), primarily to provide proxy caching services for Internet content accessed via an ADSL connection. It is not clear whether the stateful and application-layer firewall features of ISA were ever configured. The PCs and servers are connected to a switched school LAN. In schools not yet connected to the NBN, this LAN is connected via the ISA server to an ADSL modem for Internet services and access to EduWave. In NBN-connected schools, this switched network is connected directly to an NBN switch. Network equipment in each computer lab is installed in a wall-mounted rack with a locking cabinet. The multiple computer labs in a school are connected to a main computer lab that provides the ADSL or NBN connection and houses the DHCP/ISA server. Including PCs in Discovery School computer labs, there are approximately 65,000 PCs in computer labs distributed among the 3,200 schools in Jordan, an average of 20 per school. Nearly one-third of these are seven years old. PC procurement has largely been driven by donor funding, resulting in a patchwork of makes and models, complicating technical support. Equipment provided to Discovery Schools through the JEI is relatively new and a few Discovery Schools have as many as six computer labs. This is not the situation in all schools. At Salah Al-Din School (male), a Discovery School, one of three labs serving 1,300 boys could not be used because most of the old PCs were not functional. The student-teacher ratio in this school is 45:1, while the student-working computer ratio is as high as 6:1, forcing some students to stand and wait their turn at the computers. Al-Nuzha Elementary School (female), which is not a Discovery School, has an enrollment of 1,166 and one computer lab equipped with 14 old PCs loaned from other schools. Many parties interviewed during this assessment, including ITG, Rubicon, teachers, principals, and personnel in various central MoE directorates, know that the ratio of students to working computers in computer labs is the most serious constraint to the effective use of e-content. The Classroom and Computer Lab Deployment Strategy conducted by the Directorate of Curriculum and Textbooks with CIDA’s SJE Project in 2006 estimated that 2,180 new computer labs would be needed throughout the Kingdom in 2008–2009. This analysis was based on curriculum demands for time in computer labs. It did not consider the negative impact on the effectiveness of that time caused by inoperable equipment and problems accessing e-content over the network. Good technical support and periodic planned replacement of equipment are necessary if computer labs are to meet the demand for access to e-content. Some computer labs are arranged in lecture hall fashion, with desks arranged in rows facing the teacher. Other labs have been rearranged in the form of a “U,” with desks arranged around the perimeter, making it easier for the teacher to monitor the activity of the students and to provide hands-on help. Computer lab rooms, however, are not constructed with this in mind. The size and shape of the rooms often do not consider the space Many parties interviewed during this assessment know that the ratio of students to working computers in computer labs is the most serious constraint to the effective use of e-content. Exhibit 4 A new computer lab at Salah Al-Din School cannot be used because electrical outlets are in the wrong location. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 21 of 61 required for the equipment and electrical outlets are often installed in locations that make it difficult to connect the PCs to electrical power. At Salah Al-Din School, one lab equipped with 10 new PCs could not be used because electrical outlets were not installed in appropriate locations. The JEI could play an important role in improving communication and coordination between MoE directorates responsible for constructing and equipping computer labs. 5.2 Laptop computers The second ICT model implemented by the JEI in Discovery Schools includes providing portable laptop computers and “data-show” projectors to teachers. Each Discovery School is equipped with six to seven laptop computers and “data-show” projectors. Laptop computers and projectors were procured through a variety of different donors and tenders, resulting in five different makes and models of laptops and five different makes and models of projectors. The JEI installed Wi-Fi wireless access points in each Discovery School to provide wireless local area network connectivity to these laptop computers throughout each school. Wireless access points were also procured through several different tenders, resulting in at least two different makes and models being deployed. Wireless access points are normally mounted high on interior walls and are provided with power using Power over Ethernet (PoE) via a Cisco switch installed in the wall-mounted rack of the main computer lab. Students normally remain in their classrooms where specialized teachers come throughout the day to teach the various subjects. Laptop computers and projectors enable teachers to display EduWave e￾content, and other content they or their students have prepared, in the classroom. The wireless networks enable teachers to bring a laptop computer to any classroom, connect to the wireless network from any location in the classroom, and display EduWave e-content. The JEI maintains detailed data on Discovery Schools including student enrollment, class size, network infrastructure, computer lab equipment, and teacher training. The JEI uses this information to carefully allocate the available laptop computers and projectors. The JEI also uses periodic site surveys to monitor the functional status of this equipment. Policies regarding the use of and responsibility for laptop computers and projectors are determined by the MoE but vary widely in implementation by school principals. MoE policy permits teachers to take laptop computers home after school hours if they sign a “loan” form, taking responsibility for any damage to the equipment caused by misuse. However, some principals prohibit teachers from taking laptop computers home and require the computers to be placed in locked storage cabinets or locked computer labs after school hours. Projectors may also be left locked in a computer lab, because they are too heavy to carry on public transport and are not useful at home. Teachers may not take the risk of using the laptop computers at home, or may decide that the computers are safer in their care at home than if left locked in the school after hours. The JEI initially assigned laptop computers and projectors to individual teachers. This made sense based on the roll out of e-content and teacher training. Beginning with the introduction of e-EFL content, the JEI is planning to introduce a model in which teachers share a limited pool of laptop computers. Some school principals have introduced a laptop sharing model on their own. While each teacher would, of course, prefer to have an individually-assigned laptop, the shared model appears to The JEI could play an important role in improving communication and coordination between MoE directorates responsible for constructing and equipping computer labs. A shared laptop computer model is an efficient use of scarce resources. The MoE should encourage consistent implementation of a policy that permits teachers to take laptop computers home. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 22 of 61 be a practical compromise and an efficient allocation of scarce resources. A shared laptop computer model should include consistent policies across schools regarding the use of and responsibility for the equipment, and secure storage in the school when teachers do not take laptop computers home. While many teachers were initially afraid to use this equipment, most teachers interviewed during this assessment believe the laptop computers and projectors are valuable in the classroom to present e￾content. One Arabic teacher reported that, while it had initially taken him longer to prepare for lessons using the laptop, he now found it quicker, easier, and more effective to prepare lessons using the laptop than using traditional tools. Most teachers said that it was extremely useful to be permitted to take laptop computers home to prepare lessons, to improve their own ICT skills, and to spread the general culture of using ICT. While teachers in some areas may have desktop PCs at home, they find it better to prepare lessons at home on the same laptop computer they use in the classroom. Many Discovery School teachers also use laptops at home to manage grades for students and to do some statistical analysis of exam grades. The most common request we heard was for more laptop computers and projectors. Teachers and principals in several schools visited said that laptop computers and projectors were more useful than computer labs, because the schools could not possibly keep pace with increased enrollment by adding computer labs. The two models are, of course, not equivalent. Computer labs enable individual students to access e-content, and make it possible to track individual student progress and to adapt e-content to the needs of each student automatically. Computer labs also require physical space that may not exist and take time to construct. Teachers, most of whom have other responsibilities after school, cannot stay to prepare lessons in computer labs. A shared pool of laptop computers enables teachers to prepare lessons at home, and, with a projector and wireless network, make it possible to use that lesson material and EduWave content in any classroom. Laptop computers, projectors, and wireless networks can be deployed more quickly than computer labs. The two models also present very different Total Cost of Ownership (TCO) and Value of Investment (VOI) issues. Laptop computers cost two to three times as much as desktop PCs. Desktop PCs may continue to function and provide adequate performance for 5 to 7 years. Laptop computers may continue to function and provide adequate performance for 4 to 5 years. Batteries in laptop computers may need to be replaced every 1 to 3 years at a cost of US$120 to US$160. Projector lamps, which may need to be replaced in less than a year of use, can cost 40 to 50 percent of the original cost of the projector. Laptop computers are also more expensive and difficult to repair. Field replaceable components are generally limited to batteries, keyboards, CDRW/DVD drives, hard disk drives, memory, and wireless network adapters. Failure of other components, such as displays and motherboards, normally require replacement of the entire laptop chassis or repair by a manufacturer-authorized repair center. The JEI conducted a thorough analysis of laptop and projector hardware failures and repair issues in Discovery Schools as part of its 12th site survey, covering the period of May 18 through June 8, 2007. Results show that while one percent of the desktop PCs were out of service, eight Teachers and principals in several schools visited said that laptop computers and projectors were more useful than computer labs, because they could not possibly keep pace with increased enrollment by expanding computer labs. The JEI should lead the TCO and VOI analysis and dialogue with the MoE and other stakeholders prior to piloting ICT models. There should be clear understanding among the parties of direct and indirect cost implications and responsibilities throughout the technology life cycle before investments have been made. Computer labs and use of laptop computers by teachers present very different total cost of ownership (TCO) and value of investment (VOI) issues. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 23 of 61 We do not believe that additional investments in IWBs should be dismissed out-of-hand. We caution that the VOI of this technology be evaluated carefully. percent of the laptops and two percent of the projectors were out of service. The 3-year warranty on most of the laptop computers and projectors has now expired. None of the laptop computers and projectors is covered under existing MoE maintenance agreements. Nearly 40 percent of laptop repairs have taken 1 to 3 months, while 44 percent have taken longer. While the JEI and the MoE are aware of these issues, it appears dialogue around TCO and VOI is just beginning. The MoE ICT Directorate has resisted JEI efforts to find a sustainable solution to laptop computer maintenance. The JEI should lead the TCO and VOI analysis and dialogue with the MoE and other stakeholders prior to piloting ICT models. There should be clear understanding among the parties of direct and indirect cost implications and responsibilities throughout the technology life cycle before investments have been made. 5.3 Interactive white boards A limited number of schools have been equipped with interactive white boards (sometimes abbreviated IWB or IAW). These were purchased from two separate manufacturers. They require a ceiling mounted data-show projector and desktop PC or laptop computer with the appropriate software installed. The correspondence between the IWB touch-sensitive surface and the projector image requires periodic re-calibration. An IWB enables users to draw electronically on any image projected by the computer, to interact with the computer through a touch-sensitive surface, and to print, store, transmit, and reuse any image on the IWB. Some IWBs have been installed in computer labs or standard classrooms. Others have been installed in special shared “media” rooms equipped with other audiovisual equipment. We were initially skeptical that schools would use the IWBs effectively enough to justify the TCO. In other words, we doubted their educational value would be sufficient to justify investing in this technology, particularly when the focus has been in reducing student-computer ratios in computer labs to increase access to EduWave e-content. At Al-Shifa’ Bint Awf School (female), we saw an IWB being used very effectively to engage a class of students. While the exercise we observed could have been done with a conventional blackboard or whiteboard and paper handouts, teachers reported that, as with other ICT, the use of the IWB helped to engage students and hold their interest. These observations are consistent with studies of IWB use in the classroom in other countries, including Australia, Canada, the United Kingdom (U.K.), and the U.S. (Higgins et al., 2005; Becta, 2005; Somekh et al., 2007). These studies indicate positive changes in the patterns of classroom interaction that are sustained for at least a year after IWBs become embedded in teaching practices. Student achievement results suggest a measurable positive impact in the first year, but these results may not be sustained in the second year, suggesting that early improvement may be an artifact of change. Exhibit 5 A student at Al-Shifa’ Bint Awf School uses an interactive whiteboard to demonstrate the process of creating a topographic map. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 24 of 61 We do not believe additional investments in IWBs should be dismissed out-of-hand. Results of studies in other countries suggest there are positive learning benefits, at least in the short term. We caution that the VOI of this technology be evaluated carefully. Al-Shifa’ Bint Awf School is an extraordinary school because of exceptional leadership. IWBs may be used effectively only in schools that have such leadership and should therefore be provided to schools based on their level of commitment. The MoE should be fully aware of and commit to the cost of sustaining this technology before installing it in additional schools. 5.4 Connecting to EduWave and the Internet All e-content is stored centrally in EduWave databases at the Hashem Data Center and is served on￾demand to users. Teachers and students may access e-content on EduWave from school computer labs and networked laptop computers connected to the Internet via ADSL or the NBN. Teachers and students may also access EduWave from any Internet connection, including ADSL connections in schools not yet connected to the NBN, from homes with Internet connectivity, and from any Internet café. 5.4.1 ADSL Until recently most schools in the country, including Discovery Schools, were connected to the Internet via JT ADSL connections providing a nominal bandwidth of 512Kbps. According to the MoE, 1,000 schools in remote parts of the country do not have ADSL connections, although this may be reduced to 500 by the end of 2007. Speed tests from schools have indicated actual bandwidth to be between 46Kbps and just over 400Kbps.10 This bandwidth may be shared by between 13 and more than 60 computers in each school. ADSL connections were running close to 100 percent of capacity when computer labs were in use. Significant improvement was noticed when JT increased bandwidth to 1 megabit per second (Mbps) using ADSL2. Traffic monitoring by JT indicated that a large percentage of traffic from each school is generated by malicious software (viruses, worms, and spyware). JT believes this traffic needs to be prevented and controlled at the school level. This is consistent with comments made by Cisco Systems and NBN network engineers. 5.4.2 National Broadband Network (NBN) The NBN is a high-capacity fiber-optic networking project designed to interconnect all institutions of learning and some 44 government agencies nationwide. The NBN project was initiated by the Ministry of Information and Communication Technology (MoICT) in 2002 under the Connecting Jordanians Initiative (CJI). The NBN project team includes networking and telecommunication professionals experienced in designing, implementing, and managing networks on this scale. The NBN is being constructed in several phases. The first phase, the University Broadband Network (UBN), has been completed and now connects eight public universities at nine sites. This phase was completed in 2004, with operations successfully handed off to the universities. The NBN began the second phase, the School Broadband Network (SBN), in 2003. This phase is designed to connect 3,200 public schools, 100 Knowledge Stations, 17 public community colleges, and 12 learning resource centers throughout the Kingdom. The SBN and UBN are interconnected. Within the schools, the SBN is commonly referred to as the NBN. 10 Eng. Othman Al-Suqi, Cisco Systems Inc., Resident Consultant, Jordan Education Initiative, Track Manager: In-Classroom Technology. Traffic monitoring by Jordan Telecom indicates that a large percentage of traffic from each school is generated by malicious software (viruses, worms, and spyware). Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 25 of 61 The SBN is a hub-and-spoke, point-to-point Gigabit Ethernet (GE) network with fault tolerance in the form of additional fiber spokes that form interconnected logical rings. The Amman metro ring uses nine primary spokes to connect aggregation points, each of which contains a distribution network connecting up to 20 schools. Most aggregation points are located in schools. These aggregation points have been installed in secure rooms equipped with voltage regulation, climate control, fire suppression, and remote video monitoring equipment. Existing switch equipment at each aggregation point is capable of connecting up to 64 schools and includes firewall services and intrusion detection services. The QRC data center is also connected to the Amman metro ring. Additional rings will be constructed to interconnect aggregation points that will provide a distribution network for connecting schools in other parts of the country. The national network has been divided into eight such modules, of which the Amman metro ring is Module 1. The national overlay network is also an optical ring structure that will interconnect the eight regional modules and the data centers via 1 gigabit per second (Gbps) (1GE) or 10Gbps (10GE) fiber optic connections. A total of 231 schools, including 89 Discovery Schools, are now connected to the NBN via fiber optic connections that provide 100Mbps of bandwidth to each school. The bandwidth is sufficient to support video conferencing, Voice over Internet Protocol (VoIP), and other advanced services. While this has resulted in a major performance improvement in accessing EduWave e-content, NBN-connected schools still report a variety of access and performance problems. Teachers at several schools reported poor performance when an entire class attempted to load the same learning object at the same time. Teachers at Al-Shifa’ Bint Awf Girls School reported that in some cases it may take 15 minutes to load a learning object during a 45 minute class session, so they downloaded the entire e-science curriculum for local access. With lab seats and time a scarce commodity, any loss of class time is significant. Continued problems with the reliability and performance of access to EduWave e-content are not necessarily caused by NBN technical problems. Computer lab technicians, JEI interns, education directorate field officers, and network engineers report that computer viruses, worms, and spyware are rampant in computer labs. These malicious software programs can consume significant processor and bandwidth resources, resulting in poor performance. Schools switching back to old ADSL connections for Internet access, then back to the NBN for performance, can disrupt network addressing. Schools that turn off electrical power mains when they close can also disrupt network operations. Even if all these problems are corrected, it is safe to assume that connectivity to EduWave servers at the Prince Hashem Data Center will not be 100 percent available. Server capacity at the Prince Hashem Data Center should be capable of keeping pace with the roll out of e-content to 3,200 schools, but the reliability and performance of connectivity to the data center will continue to limit access to e￾content and effective use of limited computer lab time. The MoE should consider seriously whether mission-critical e-content that is updated on an annual basis can be cached at the school level, eliminating total dependence on connectivity between the school and the Hashem Data Center. As the flagship EduWave reference site, the MoE should have the leverage to insist on an effective solution. The ITG has built a large, integrated, technologically complex software product assuming that adequate connectivity will exist. According to the ITG, they rely entirely on second-hand information from the QRC regarding e-content access and performance problems in the classroom. The ITG does not send its own technical The reliability and performance of connectivity to the data center will continue to limit access to e￾content and effective use of limited computer lab time. The JEI could play a key role in helping to bring all parties to the table to clarify roles, discuss the problems, agree on corrective actions, and monitor progress towards the common objective of making this system work for the Kingdom. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 26 of 61 personnel to observe how teachers are using the product and what problems they are encountering. Considering that this is the company’s largest and highest profile installation, this is difficult to understand. They are clearly aware of e-content access and performance problems in classrooms, but blame this on connectivity. This response should not be acceptable to the MoE. Thus far, finger pointing between ITG, JT, the NBN, and the MoE has not been productive. The JEI could play a key role in helping to bring all parties to the table to clarify roles, discuss the problems, agree on corrective actions, and monitor progress towards the common objective of making this system work for the Kingdom. Considering that the core e-content is normally updated annually, it is difficult to make the argument that it must be served live, on-demand, to all 3,200 schools from a single central location. It should be possible to move to a model that distributes, caches, and manages e-content at the school level. According to Rubicon, content files for e-math occupy a total of approximately 0.5GB of disk space, while content files for e-science occupy a total of 0.7GB. Both subjects could be stored on a single USB flash drive. Caching e-content closer to the edge of the network would reduce dependence on connectivity, improve performance and reliability, and make it easier to scale the content delivery system. While not absolutely necessary, it is also technologically possible to store and manage student information at the school level while synchronizing these data automatically with a central database for system-wide monitoring and evaluation. Such an approach would make the system less dependent on connectivity, more fault tolerant, and more scalable. ITG has been very resistant to the idea of local caching of content on school servers. In part, this resistance may be based on the assumption that this caching would require a massive investment in changing EduWave architecture. It may also be based on the assumption that this would essentially require a local copy of EduWave in each school. ITG points out that the MoE is not likely to have the resources to support 3,200 Web application and database servers located throughout the Kingdom. This is a very important and valid point, but each Discovery School already has a Microsoft Windows server in the main computer lab to provide DHCP and ISA services. A less radical architectural change that uses existing servers may be a feasible compromise. The ITG may be able to combine its existing off-line viewer with a system that provides e-content updates to school servers automatically. An EduWave proxy agent on each school server could download updates when available and serve e￾content from local server storage. This could help resolve the most critical e-content access and performance problems without a massive investment to provide off-line access to all EduWave functions. 5.4.3 Internet access Teachers interviewed during this assessment appeared to have a strong understanding of the blended learning approach. In addition to e-content available via EduWave, teachers reported using a variety of content from the Internet to supplement e-content and textbook material. Schools may use the Internet to access e-content on EduWave as well as content on the Internet. JT maintains two international Internet connections through New York (USA), one through Frankfurt (Germany) and one through Paris (France). The current agreement between the MoE and JT provides a total of 4Mbps of shared bandwidth to All content files for e-math and e￾science occupy could be stored on a single 2GB Universal Serial Bus (USB) flash drive. An EduWave proxy agent on each school server could download updates when available and serve e-content from local server storage. Jordan Telecom tests indicate aggregate Internet bandwidth demand by schools ranges from 25Mbps to 30Mpbs. The current MoE agreement provides 4Mbps. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 27 of 61 users in 3,200 schools and education directorates to access the Internet. According to JT, this bandwidth is 100 percent utilized during the school year. JT conducted an internal study over an 8- month period during the school year that showed actual total aggregate Internet bandwidth demand by MoE users ranged from 25Mbs to 30Mbs. JT provided these results to the MoE and suggested that the MoE increase maximum shared Internet bandwidth from 4Mbps to 20Mbps. JT reports that they have very little information on the connection of schools to the NBN and have not projected the impact this may have on Internet bandwidth requirements. Considering their own experience with fiber cuts and reports of similar problems with the NBN, JT believes that ADSL connections in schools should be used as backup for NBN connections. As mentioned at other points in this report, malicious software and network traffic generated by malicious software continues to affect the stability and performance of PCs and laptop computers, to consume NBN and Internet bandwidth, and may be a significant factor in the performance of e-content served by EduWave from the Hashem Data Center. The MoE has installed a Symantec security appliance at the Hashem Data Center, but JT engineers believe this is undersized for current network traffic and is not in the correct location within the network topology. The NBN is a private intranet designed to interconnect educational and government institutions. The NBN is not an Internet Service Provider and providing Internet access has not been within the scope of the NBN program. The issue of Internet access was not raised until the NBN began switching schools from ADSL. The MoE assumed that the NBN would provide Internet access, but never raised this question with the MoICT NBN project team. This is symptomatic of serious communication problems between the NBN project team and the MoE. The MoE did not designate a single point of contact with NBN or form a team to ensure coordination and communication among the various relevant MoE departments. Lack of strong coordination and communication between the NBN team and the MoE, and within the MoE has resulted in several problems that have affected the reliability of the NBN network. NBN connections to schools were made during the vacation period, when schools were closed. School staff at some schools kept electrical power in their schools turned off during this period, preventing the NBN team from testing these connections. At the beginning of the school year, when power was restored, the NBN team discovered many breaks in the fiber and other connectivity problems that prevented access to EduWave e-content. Lack of understanding within the MoE of the NBN scope and network bandwidth and traffic management issues in general are also hampering resolution of the Internet connectivity issue. The JEI is playing an important role in facilitating communication and helping to reach a solution to the Internet access issue. The NBN team is aware that there are virtually no network security policies or systems within MoE-controlled portions of the network, including school LANs and computer labs that are heavily infected with malicious software. Without adequate security within MoE-controlled portions of the network, providing Internet access to schools through the NBN increases security threats to other NBN users and has the potential to disable NBN and EduWave services through denial-of-service attacks. 5.4.4 Security and bandwidth management JEI track managers, JT, the NBN team, Computer Engineering Bureau (CEB) support technicians, JEI interns, education directorate field officers, computer lab technicians, and teachers all cited computer viruses as the most critical problem affecting their ability to use ICT in the classroom. Computer viruses, worms, spyware, and other forms of malicious software can consume CPU capacity and network bandwidth, resulting in poor computer and network performance. JT and Cisco Systems networking The JEI is playing an important role in facilitating communication between stakeholders and helping to reach a solution to the Internet access issue raised by the roll out of the NBN. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 28 of 61 engineers report that as much as 80 percent of network traffic from the schools may be “unwanted” traffic generated by malicious software. There is no standard antivirus or anti-spyware software deployed on school computers. Computer Associates, Inc. (CA), a JEI contributor, offered to provide roughly US$2.5 million worth of antivirus software for PCs, laptop computers, and servers. The MoE reportedly hesitated to accept this offer without assurances that free antivirus updates would be provided beyond the initial 3-year CA commitment. In the absence of any MoE guidance or standards, computer lab technicians and JEI interns resort to installing whatever antivirus software they can find. Other basic security measures are also absent. Antivirus protection must be combined with a system for distributing critical security patches to servers, PCs, and laptops to provide basic protection against malicious software. The MoE currently has no standardized solution for distributing and installing critical security patches to the schools. There is no acceptable use policy, and no policy regarding the use of peer-to-peer software, or installation of unauthorized software. There is no traffic monitoring or bandwidth management at the school level to prevent peer-to-peer and other unwanted traffic from consuming network bandwidth. ISA servers in each main computer lab may have provided stateful and application-layer firewall services for ADSL connections. When schools are connected to the NBN, these servers are no longer positioned to provide these security services. NBN switch infrastructure at aggregation points can provide firewall and intrusion-detection services for the NBN, but it is not clear whether these services have been configured in coordination with the MoE and are sufficient to protect schools from denial-of￾service attacks that may originate on the Internet or NBN. There is currently no firewall appliance between the NBN and each school LAN. The MoE did install a Symantec Gateway Security appliance at the Hashem Data Center, but, as mentioned previously in this report, JT engineers believe this is undersized for current network traffic and is not in the correct location within the network topology. NBN engineers also reported that this device alone cannot provide adequate security for the schools or the MoE data center. There is also no Internet-filtering software to prevent students and teachers from accessing inappropriate Internet content. The Authors suspected access to inappropriate content posed a serious risk to support from teachers, parents, and principals for the use of ICT in the schools. While fear of inappropriate content may have inhibited use of the Internet a few years ago, this was not the case in interviews and focus groups conducted during our assessment. Teachers and principals are well aware of the ease of intentional or accidental access to inappropriate content, but this now appears to be outweighed by the value of appropriate content for enriching the curriculum. Teachers and principals would, however, like to have filtering software installed if possible to reduce this risk. One school installed large mirrors in the back of each computer lab to enable instructors to monitor what their students are viewing on computer displays. In another school, the Queen Rania (girls), the principal had reconfigured computer lab desks into a “U” shape along the walls. Several additional desks were grouped together to form a central island containing six additional work stations and a teacher controlled-computer. All monitors were easily visible to the teacher. Consultants to the MoE have recommended the use of Microsoft SMS to provide remote control, patch management, software distribution, and automated hardware and software inventory services. Microsoft Active Directory (AD) will be used to provide central authentication and authorization services, and will serve as the foundation for SMS implementation. Microsoft’s licensing terms for the MoE are so preferential that alternate technologies were not considered and no Microsoft’s licensing terms for the MoE are so preferential that alternate technologies were not considered and no TCO analysis was conducted. Numerous users cited computer viruses as the most critical problem affecting their ability to use ICT in the classroom. In the absence of any MoE guidance or standards, computer lab technicians and JEI interns resort to installing whatever antivirus software they can find. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 29 of 61 TCO analysis was conducted. Because local Microsoft engineers had never designed and deployed these products on a network of more than 3,200 sites and 65,000 clients, more experienced Microsoft engineers have been brought in for this project. Microsoft will train two engineers in the MoE ICT Directorate, but MoE consultants admitted these engineers were not likely to remain with the MoE for long. The MoE is clearly just beginning to realize the enormity of the challenge of providing basic security within school networks. While Microsoft SMS and Active Directory are capable systems in experienced hands, they are complex and require well-trained system engineers to maintain. There are also many complex enterprise antivirus and antispyware products on the market that can be managed from a single location on a large network. However, MoE financial, organizational, and human capacity constraints limit the organization’s ability to maintain and sustain complex systems. Considering these capacity constraints, the JEI can help the MoE to focus on resolving the most critical security problems using the simplest possible solutions. These should be standardized, documented, promoted, and monitored. For example, nearly all antivirus software has a feature to automatically update virus signatures and the virus scanning engine on a configurable schedule. Many of these products, including free versions, can be configured to retrieve updates from any designated directory on any accessible computer. A server at the Hashem Data Center could check for and retrieve antivirus updates daily. School servers could be configured to retrieve antivirus updates daily from a designated directory on the server at the Hashem Data Center. PCs in computer labs and laptop computers could be configured to retrieve antivirus updates from a standard directory on the school server. This simple cascading system does not require expensive software and conserves network bandwidth, because computers in schools get updates locally. It also ensures that updates are distributed to each computer quickly and automatically. This simple system requires standardization, good documentation, training field officers and computer lab technicians, and regular monitoring. Microsoft Windows Software Update Services (WSUS) can be used in the same way to enable the school server to download and distribute Microsoft security patches and updates automatically at the school level. WSUS is available free from Microsoft. WSUS does not require the use of Microsoft Active Directory for patch distribution. School servers could retrieve patches and updates from a central WSUS server at the Hashem Data Center, making it possible to control patch distribution centrally if necessary. Microsoft Windows Update on PCs and laptops in each school could be configured to retrieve security patches and updates from the school server. This saves network bandwidth, because computers in each school get their updates locally. As with the antivirus update system example, this system must be rigidly standardized, well documented, communicated to field officers and lab technicians, and monitored regularly. 5.5 Comments from teachers We visited several schools and had the opportunity to meet with teachers and principals. Some teachers had relatively little experience with the e-content for several reasons, including teaching in fields for which the e-content had not been long available. There appeared to be considerable differences among the Discovery Schools we visited in terms of the utilization of e-content. Generally, however, the teachers we spoke with reported favorably on the incorporation of computers in their programs, both in the classroom and in the computer lab; those who had the opportunity to take laptops home reported favorably on the incidental benefits of having the computers. They appreciated the MoE financial, organizational and human capacity constraints limit their ability to maintain and sustain complex systems. The JEI can help the MoE to focus on resolving the most critical security problems using the simplest possible solutions. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 30 of 61 training they had received and generally wanted more, especially if it could be scheduled at times and places convenient to them. Teachers and principals we interviewed reported that they sought to use the e-content in class about twice a week, once in the computer lab and once using the laptop and projector. They informed us that initially the preparation time for each of those classes was several hours per class and was quite burdensome, but that after 1 or 2 years experience with the technology, preparation time was reduced. Teachers reported some tension between the need to prepare students to do well on exams, which are focused on content within the textbooks, and use of the e-content, which is designed to supplement and complement the textbooks. The tension seems to be especially high in the highest grades (10–12). However, we also observed students in one computer lab who were working with e-content prepared by their teacher and another class in which a teacher was using an IWB with e-content she had developed to teach geography. We were told that teachers used selected e-content from the Internet, prepared their own PowerPoint presentations, and used the technology in other ways independent of the e-content available through EduWave. We were told that it was especially onerous to have to prepare backup lessons for use when the computer technology failed, which appears to be a common occurrence. Moreover, the computer lab problems were such that it frequently took two periods of lab time to complete a unit designed for one. While other sources suggested that there were differences in teacher experiences with the technology according to the subjects taught, we did not have an opportunity to investigate that issue with the teachers themselves. 5.6 Conclusions 1. Reducing the ratio of students to working computers in computer labs is critical to increase student access to e-content. Problems coordinating computer lab construction, aging equipment, lack of planned equipment replacement, inadequate computer and network security, and connectivity problems are limiting the effective use of existing computer labs. These problems affect JEI efforts in Discovery Schools and threaten the scale up of JEI innovations to all Jordanian schools. 2. Sharing laptops among teachers is an efficient use of scarce resources. Shared laptop computers and projectors can be used by teachers to increase access to e-content when construction of new computer labs cannot keep pace with increased enrollment. 3. Teachers who take laptop computers home benefit by continuing to learn and prepare lesson plans outside normal school hours. The MoE should encourage consistent implementation of a policy that permits teachers to take laptop computers home. 4. Computer labs and use of laptop computers by teachers present very different TCO and VOI issues. The JEI should lead the TCO and VOI analysis and dialogue with the MoE and other stakeholders prior to piloting ICT models. There should be clear understanding among the parties of direct and indirect cost implications and responsibilities throughout the technology life cycle before investments have been made. 5. We were initially skeptical of the value of additional investments in IWBs. We saw an IWB being used very creatively in one school to engage students in learning. Studies in other counties report similar results and suggest that there are at least short-term learning benefits. We suggest that the VOI of this technology be evaluated carefully. IWBs may be a cost￾effective way to increase student access to e-content. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 31 of 61 6. Computer and network security is inadequate. The prevalence of malicious software is the most serious technical problem affecting computer labs and network bandwidth. The MoE needs to implement basic security policies, procedures, and tools, including client antivirus and antispyware software, and firewall appliances at the school level. The MoE should favor simple documented standards, tools, procedures, and training over complex technical systems that may be more difficult to sustain. 7. Preferential commercial software volume licensing agreements can be very beneficial, but the MoE should make sure such arrangements do not eliminate consideration of alternative technologies. The MoE should examine the technical objectivity of consultants carefully and choose technical solutions only after carefully considering TCO, VOI, and sustainability. 8. Internet bandwidth for schools appears to be inadequate. Once malicious software is under control, the MoE should examine whether existing bandwidth provided through JT should be increased to meet the legitimate needs of the schools. 9. The MoE should form an NBN task force that represents the relevant directorates and has a leader that serves as a single point of contact for the NBN team. The task force should work with the NBN team to improve communication and coordination with schools, to resolve policy and technical issues, such as network security and bandwidth management, and to make sure the MoE is making the most effective use of NBN potential. 10. The NBN provides sufficient bandwidth to support new applications, including voice and video conferencing. The JEI should lead efforts to explore whether some of these applications could be used to further JEI objectives. 11. Delivery of e-content to schools via EduWave will continue to be vulnerable to problems with network reliability and performance. There appear to be no technical reasons why e-content could not be cached at the school level. Many schools already have suitable servers. The MoE should leverage its status as an EduWave reference site to encourage ITG to explore this option. 6 Maintaining Access to E-content 6.1 Organization, communication, and coordination There are some experienced personnel and excellent capacity within individual MoE directorates, but directorates often function in separate disconnected silos, and the vision and strategy of MoE executive leadership does not appear to be communicated well throughout the organization. For example, we visited a new computer lab in which the location of electrical outlets prevented installation of new computers, and therefore use of the lab, in a school already critically short of lab facilities. Teachers showed how room size, desks, and electrical outlets limited their ability to arrange computer labs in the most effective way for teaching their students. There also does not appear to be a single designated point of contact within the MoE to communicate with and coordinate the activities of MoE directorates in support of the NBN roll out. The JEI plays an important role in trying to facilitate communication among MoE directorates and between the MoE and private sector partners. No other organization appears to be positioned to play this important role moving forward. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 32 of 61 Capacity is also a problem. As might be expected in a public sector organization, there are challenges retaining personnel. We were told by the ICT Directorate and ICT consultants to the MoE that the MoE has difficulty retaining personnel with strong IT skills. Skilled and particularly certified IT personnel quickly get higher-paying positions in the private sector. A seemingly constant stream of new projects also prevents personnel in the ICT Directorate from adequately evaluating project results; making adjustments; resolving fundamental deficiencies in policies, procedures, and systems; and focusing on longer-term planning. The JEI plays an important role in trying to facilitate communication among MoE directorates and between the MoE and private-sector partners. The JEI has also played a role in increasing MoE technical capacity, particularly through the JEI internship program. No other organization appears to be positioned to play this important role moving forward. 6.2 Technical support, maintenance, and replenishment Roles, responsibilities, support contacts, and problem escalation procedures are not well communicated or well understood by all parties. Many computer lab technicians and education directorate field officers are not well trained in diagnosing, communicating, and resolving problems. The JEI intern program has been very effective in strengthening the capacity of school lab technicians. The existing after-warranty maintenance contract appears to be performing reasonably well, but does not cover laptop computers. The help desk at QRC is not adequately staffed or equipped. The continuing series of projects has made it difficult for the MoE to monitor and evaluate warranty support, after-warranty support, and help desk operations effectively, and manage the infrastructure proactively. The JEI site survey of Discovery Schools is the most useful and effective M&E system available. These basic issues, described in more detail below, need to be resolved before the use of e-content and JEI technology models are scaled beyond Discovery Schools. 6.2.1 Problem reporting, escalation, and response Each school has at least one computer lab technician; some schools have more than one. The number of lab technicians in a school is not necessarily consistent with the number of computer labs; it may also reflect the leadership and initiative of the school principal. Computer lab technicians are normally secondary school diploma holders and have not attained CompTIA A+11 computer support certification. Computer lab technicians are responsible for providing on-site technical assistance to teachers and students; scheduling use of computer labs; maintaining computer labs, school LANs, laptop computers, and projectors; and diagnosing and resolving simple software and hardware problems. Lab technicians are the first responders to any problems concerning information technology in the school. It is not clear whether lab technicians have written position descriptions. If a lab technician is unable to resolve a problem, he or she submits a written paper report to the Maintenance and Network Department of the relevant education directorate. Lab technicians do not have access to an online help desk ticketing system. Many education directorate offices have very few telephone lines, and normally no single person in an education directorate is responsible for answering calls. Paper mail is checked once a week, slowing response time. 11 CompTIA A+ is a vendor-neutral, internationally recognized certification. It confirms a technician’s ability to perform tasks such as installation, configuration, diagnosing, preventive maintenance, and basic networking Fundamental issues of technical support, maintenance, and equipment replenishment need to be resolved before the use of e￾content and JEI technology models are scaled beyond Discovery Schools. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 33 of 61 A liaison field officer from the education directorate will then visit the school to investigate the problem. For some equipment, as explained below, lab technicians in Discovery Schools also have been authorized to contact a contracted maintenance firm directly to resolve the problem. If the problem appears to be with the hardware, the field officer determines whether the equipment is under warranty or covered by an after-warranty maintenance contract. If the equipment is under warranty, the field officer is responsible for arranging warranty service with the appropriate local vendor via phone, fax, or e-mail. We were told by field officers and CEB support technicians that they sometimes must make multiple visits to a school to resolve a single problem due to lack of understanding of the technology in the school or to poor communication of the problem. Out-of-warranty desktop PCs, printers, data-show projectors, low-end servers, and LAN switches have been covered under a maintenance contract between the MoE and CEB since 2005. There are two years remaining on this contract. CEB has had some six contracts with the MoE since 2000, when they installed 20,000 APB desktop PCs and 900 servers in the schools. When warranties expire for desktop PCs, printers, data-show projectors, low-end servers, and LAN switches, this equipment automatically falls under the CEB maintenance contract. CEB affixes an asset management tag to each piece of equipment covered under the contract and seals the case to prevent unauthorized access. In addition to a central office in Amman, CEB maintains three regional offices to meet the response times specified in the maintenance agreement. “Response” means a CEB support technician visits the site to investigate the problem. Servers and other critical devices require 12-hour response. PCs and printers require a 24-hour response, while other equipment requires a 48-hour response. The responsible CEB regional office may aggregate maintenance tickets for rural schools before traveling to these sites. Initially there were complaints about CEB performance. This has reportedly improved, particularly since CEB opened regional offices. If the equipment is not under warranty and is not covered by an after-warranty maintenance agreement, the problem is reported to the maintenance department of the central MoE ICT Directorate. From here, there does not appear to be any clear procedure. The equipment may be out of service for months or years until the maintenance department issues a purchase order (PO) for replacement components. At this point it is a best effort system, with no service-level agreement (SLA). Most laptop computers in Discovery Schools now fall into this void. JEI data show that laptop computers may be out of service for years without any action taken by the maintenance department. The MoE operates a help desk at QRC. A help desk staff of four persons uses two telephone lines to support 3,200 schools. One staff member is seconded by JT to deal with ADSL connectivity problems. Two staff members take calls concerning problems with EduWave, and one staff member seconded by Microsoft answers questions about Microsoft Windows and Office. There is no clearly established central help desk, central help desk ticketing system, or documented problem escalation procedure. There are no signs on walls or signs on equipment in computer labs to explain who should be contacted for help and how to contact them. Many teachers, principals, computer lab technicians, and field officers are not clear who to contact for various hardware and software problems. Teachers occasionally contact Rubicon regarding problems with EduWave access or responsiveness. Lab technicians and field officers occasionally contact CEB regarding problems with Internet access and performance. 6.2.2 The JEI intern program The JEI intern program has played a major role in strengthening the capacity of computer lab technicians in Discovery Schools, overcoming resistance to new technology, and helping teachers and principals make more effective use of the technology. The intern program has been very effective in transferring knowledge and strengthening the capacity of lab technicians. The JEI plays an active role in recruiting and managing the interns and could play a key role in scaling the program up to strengthen Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 34 of 61 The JEI intern program has been very effective in overcoming resistance to change, transferring knowledge, and strengthening the capacity of lab technicians. technical capacity in education directorate maintenance and network departments as well as in the central ICT Directorate. The JEI worked with NetCorps Jordan to recruit and screen interns with bachelor’s degrees in computer science or a related discipline. Interns are motivated to gain experience in a rich technical environment, developing management, interpersonal, and communication skills that can help them advance to better positions. Each intern commits to a 1-year (school year) term of service, but many have returned for a second year before moving to private sector jobs. Interns receive stipends as compensation. The program currently recruits 25 interns each year. Interns work in teams of five. Each team is responsible for supporting 20 Discovery Schools. Male interns may be assigned to visit some female schools, but female interns are not assigned to male schools. When possible, each intern is assigned to Discovery Schools closest to their home. Interns are primarily responsible for making sure e-content is available to students and teachers and for collecting data for periodic site surveys. Interns provide technical support to teachers and lab technicians in using e-content and help trouble-shoot hardware, software, and connectivity problems. Interns also help to arrange and manage JEI events. The JEI intern program intends to increase focus on change management in the schools. The JEI developed special training materials and training for interns tailored to Discovery Schools. The JEI meets with interns weekly or biweekly to update them on activity and changes in Discovery Schools, so they can communicate this information to lab technicians, teachers, and principals. Interns play a key role in providing JEI with feedback from the schools. Some principals were initially reluctant to permit interns to work in the schools. It took some time for schools to see their value, but soon schools began asking for help from the interns. All 100 Discovery School principals took the time to attend the most recent JEI intern graduation in recognition of their contributions to the use of ICT in Discovery Schools. Indirect evidence indicates that interns have been very effective in strengthening the capacity of lab technicians. CEB reports that lab technicians in Discovery Schools can identify and describe problems much more accurately than lab technicians in other schools, enabling CEB to resolve problems faster and more efficiently. 6.2.3 Monitoring and evaluation (M&E) The MoE currently has no comprehensive asset management system for computer and networking equipment. JEI interns have created a Microsoft Access database in each Discovery School to manage computer and network equipment. These databases are maintained by computer lab technicians. CEB has a database of all equipment covered under their maintenance contract, and has provided this to the central maintenance department several times in the form of an SQL script, but the maintenance department has not yet been able to use this to help build a comprehensive ICT asset management database. CEB uses internal help desk ticketing software to track all reported problems, and uses this software to monitor the frequency of problems by type as well as their response times. Education directorate field officers have online access to this ticketing system, but report 90 percent of all problems to CEB by fax. The central maintenance department of the ICT Directorate accumulates approximately 30 binders of paper maintenance tickets The MoE currently has no comprehensive asset management system for computer and networking equipment and no central help desk ticketing system. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 35 of 61 each month. These are reviewed by the maintenance department and CEB each quarter. Payment is issued to CEB by the MoE based on the results of this quarterly review. This gives the maintenance department good general overall knowledge of major trends in hardware failures. While, the maintenance department has access to the CEB ticketing system, it does not appear to monitor this regularly. They do not appear to produce their own reports or receive regular reports from CEB to monitor trends in equipment failure and CEB response. CEB does maintain and use this information internally. The QRC help desk has no central help desk ticketing software; therefore it is not possible for the central maintenance department to monitor trends in QRC help desk requests or the response of the help desk team. The JT help desk ticketing system is not accessible to the ICT Directorate. While there are many good free and commercial alternatives for help desk ticketing systems, the QRC is reportedly developing one in-house. Requests for help may be entered into the CEB ticketing system, entered into the JT ticketing system, or not entered into any help desk ticketing system. This makes it impossible for maintenance departments at any level to get a complete picture of trends and plan interventions based on data. The JEI site survey of Discovery Schools currently provides the most useful M&E information on trends in hardware failures and the availability of ICT infrastructure. 6.3 Strategic planning, management, and financing of technology Becta (2006), the leading government agency in the U.K. concerned with ICT in education, concludes that education leaders must accurately and strategically plan their finances if they are to achieve a robust, reliable, and sustainable ICT infrastructure. There is little visible evidence of long-term IT strategic planning or management within the MoE. Previous assessments have pointed out the criticality of this issue, particularly with respect to the need to scale up the number of computer labs and lab seats to meet projected needs. The combination of uncoordinated donor-driven equipment inputs and government procurement regulations has resulted in a multitude of makes and models of equipment complicating support and maintenance, driving up support costs. The MoE does not have a long-term strategic plan for managing equipment replacement to refresh the technology and minimize support costs. In the absence of a long-term technology management plan, infrastructure and infrastructure support costs are driven by uncontrolled and uncoordinated donor inputs. The MoE cannot control support costs and sustain ICT infrastructure required to expand e￾content beyond Discovery Schools until these basic issues are addressed. Discovery Schools have benefited from injections of new equipment and the active support of the JEI PMO and the JEI intern program, but even this cannot be sustained without long￾term planning for support costs and equipment replenishment. The JEI has provided valuable input to the MoE toward TCO analysis and long-term planning, and the MoE has now reached a point where it must face these issues. The JEI is in a position to bring the MoE, independent consultants, and private sector partners to the table to develop a long-term plan for scaling up and The JEI site survey of Discovery Schools currently provides the most useful M&E information on trends in hardware failures and the availability of ICT infrastructure. In the absence of a long-term technology management plan, infrastructure and infrastructure support costs are driven by uncontrolled and uncoordinated donor inputs. The JEI is in a position to bring the MoE, independent consultants, and private sector partners to the table to develop a long-term plan for scaling and sustaining the ICT infrastructure. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 36 of 61 sustaining the ICT infrastructure. Becta (2006) reports that technical support, including maintenance, accounts for an average of 58 percent of ICT costs in primary schools and 62 percent in secondary schools in the U.K. Troni and Silver (2006), in studies done for Gartner Inc., found that implementing policies, best practices, and processes were the primary opportunities for organizations to reduce the TCO12 of their PCs. TCO case studies by the Consortium for School Networking (CoSN) of U.S. school districts show that lack of hardware and software standards makes it difficult to impossible to apply such best practices in managing technology to reduce support costs (CoSN 2004). In general, technology support costs are inversely proportional to the degree of hardware and software standardization and management. Careful standardization and management of PCs and laptops lowers support costs and increases reliability. In a study of ICT use in K–12 schools in California, U.S., Kusserow, Lavin, and Movassaghi (1999) cite standardization as a best practice in containing technical support costs. Gartner’s Desktop PC TCO model confirms that well-managed PCs can be significantly less expensive to operate than unmanaged systems (Troni and Silver, 2006). Gartner models also show that the TCO of unmanaged laptop computers can exceed those of unmanaged PCs by 24 percent (ibid.). There are five different makes and models of laptop computers, five different makes and models of data-show projectors, and at least two different makes and models of wireless network access points in Discovery Schools. There are at least seven makes and models of desktop PCs in computer labs. Different makes and models are the result of different tenders, some issued by donors, others issued by the MoE. This makes it difficult to stock spare replacement equipment and parts on site, such as data-show projector bulbs, PC power supplies, and laptop power adapters. It also makes it very difficult to manage standard tested software images, which are essential to controlling software reliability and performance when computers are shared.13 While Dell can be expected to donate Dell computers, government procurement regulations prohibit the ministries from specifying make and model. Technical specifications must be generic, and the lowest competitive bidder must receive the order, regardless of the TCO. There is no concept of managed “strategic sourcing” to negotiate discounted prices for a limited number of makes and models of equipment from a few suppliers each year. This lowers the cost of maintaining and supporting the technology and increases reliability. PCs in Discovery School computer labs are 3 years old or less. Nearly one-third of all PCs in computer labs are 7 years old. Some hardware components, such as power supplies, CD-ROM drives, and hard disk drives, become increasing unreliable as they are repeatedly subjected to stress, and compatible replacement parts become increasingly difficult to find. CEB reported to the Authors increasing difficulty in getting replacement parts for the older PCs. In general, downtime and maintenance costs increase as computers age. While it is true that direct maintenance costs are included in the CEB maintenance contract, the indirect costs of decreased computer availability are not. The issue of software compatibility is more complex. At some point in time, manufacturers, such as Microsoft, cease providing security updates for older software versions. For most of its products, Microsoft provides product security patches for 10 years, including a minimum 2-year support overlap between new and previous software versions. Once this period has ended, security updates are no longer available. In addition, advances in technologies used to deliver e-content may become incompatible with older computers, or may not perform adequately on older PCs. This is also true of the learning management software platform. For example, ITG may release upgrades to EduWave that require Web browser features that are incompatible with older operating systems running on older PCs. 12 Gartner defines TCO as the holistic view of costs across enterprise boundaries over time (Mieritz and Kirwin 2005). 13 Gartner models indicate that using standard software images and automated imaging software can reduce deployment costs by up to US$578 per PC in the U.S. (Troni, F. and Silver M., 2006). Many school districts in the U.S. use standard software images and re-image computers at least once a year to reduce software problems (CoSN 2004). Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 37 of 61 New operating system versions are also often incompatible with older PCs or require expensive hardware upgrades. At some point, it is less costly to replace older PCs than to continue to maintain them. This point depends on many factors, including how the computers are used, how critical they are to ongoing operations, and how difficult and costly they are to maintain. However, planned replacement on some regular cycle is essential to maintaining computer and network infrastructure. The Authors found no evidence that the MoE has routine planned replacement or is coordinating donor inputs strategically by using a long-term hardware standardization and replacement plan. It is reasonable to examine whether surplus PCs from organizations with shorter equipment replacement cycles might be refurbished to provide low-cost replacements for older PCs. Bakia (2000) notes that surplus donated computers can carry indirect costs that exceed any cost savings. Consultants to the MoE reported to the Authors that a TCO analysis convinced the MoE to abandon the idea of using refurbished PCs in computer labs. However, considering their age, more than a third of the PCs in computer labs throughout the Kingdom are little different than refurbished PCs. The same TCO analysis that showed that new PCs would cost less than refurbished PCs when all costs were considered may also apply to the oldest PCs in computer labs now. School systems with managed equipment replenishment typically replace PCs on a 3- to 5-year cycle to make sure PCs meet the technical requirements of current software, are reasonably reliable, and can be maintained at reasonable cost. Even in the most developed countries a 5-year replenishment cycle for desktop PCs is common to manage technical support costs and avoid technological obsolescence. Many school systems lease computing equipment to address the issue of planned replacement and to lower overall support costs. An analysis by the Robert Frances Group (2005), targeted at U.S. organizations, suggests that a 3-year PC replacement cycle lowers TCO, and recommends leasing as an effective method for enforcing this policy and managing costs. Osin (1998) estimates costs for computer labs in Israel using a 6-year PC replacement cycle. However, the same analysis assumes PC replacement on an 8-year cycle in developing countries based only on the difficulty of obtaining replacements. This analysis does not consider TCO, including the relationship between equipment age and maintenance cost, or issues of software compatibility. Bakia (2000) estimates the TCO for computers in classrooms in Barbados, Turkey, Chile, and Egypt using planning documents (Barbados, Chile, and Egypt) and data from site visits (Turkey). These indicated an assumed 5-year replacement cycle for computers. This study recommends replacing computers on a 5-year cycle and suggests that computer replacement be a line item in recurring cost budgets to avoid technological obsolescence. Case studies of U.S. school districts in the states of Missouri, Texas, Virginia, and Wisconsin in 2004 by CoSN showed that planned replacement cycles in these districts ranged from 4 to 5 years (CoSN, 2004). Several of these school districts lease computers to amortize costs and facilitate planned replacement. In general CoSN recommends replacing computers on a 5-year cycle.14 There are several well-recognized and tested TCO and VOI models for education. Scrimshaw (2002) reviews the literature on the use of TCO applied to ICT in education and identifies key models. Gartner (2005) provides a comprehensive definition of TCO as applied in Gartner’s Decision Engine for Costs Management software tool and in Gartner’s research. In the U.S., the CoSN has worked with Gartner and others to develop a TCO framework specifically for primary and secondary schools (Gartner 2003). Beginning in 2003, CoSN and Gartner have applied the CoSN/Gartner TCO Tool to case studies of school districts in eight U.S. states. The CoSN VOI model adds consideration of benefits to the TCO analysis and is designed as a practical approach to estimating the Return on Investment (ROI) of ICT in education.15 Becta (2006) has also developed a TCO model that includes direct and indirect costs, as 14 http://www.classroomtco.org/checklist/replacement_costs.html 15 http://www.edtechvoi.org/methodology/ Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 38 of 61 well as considering educational and management outcomes. Between 2002 and 2005, Becta worked with schools in England and Wales to test and refine this TCO model. The JEI has raised the TCO issue with the MoE. The JEI and external ICT consultants have presented the MoE with TCO analyses of existing technology models. The MoE is just beginning to understand the importance of this issue. MoE ICT consultants also appear to be focusing on this issue and report some progress. It is imperative to resolve fundamental issues regarding MoE long-term planning, strategic management, technical support, and funding of ICT before expanding beyond the 100 Discovery Schools. Until the MoE has a long-term technology management and support plan against which donor inputs are programmed by the MoE, infrastructure replenishment and technical support costs will be donor-driven and ultimately unsustainable. 6.4 Conclusions 1. The JEI PMO plays an important role in trying to facilitate communication among MoE directorates and between the MoE and private sector partners. No other organization appears to be positioned to play this important role moving forward. 2. Fundamental issues of technical support, maintenance, and equipment replenishment need to be resolved before the use of e-content and JEI technology models are scaled beyond Discovery Schools. 3. The JEI intern program has been very effective in overcoming resistance to change, transferring knowledge, and strengthening the capacity of lab technicians. It may be possible to expand this program to strengthen technical support capacity in education directorates and the central ICT Directorate. 4. The MoE needs to adequately staff and equip the QRC help desk as the single service point of contact for lab technicians, field officers, and users. The MoE also needs to clarify policies and procedures for problem reporting and escalation. 5. The JEI site survey of Discovery Schools currently provides the most useful M&E information on trends in hardware failures and the availability of ICT infrastructure. A unified help desk ticketing system and a hardware and software asset management system would provide the MoE with better information for technology management and long-term planning. The existing CEB database provides a good foundation. 6. The MoE needs to program donor inputs of hardware and software against a long-term technology management plan to reduce infrastructure support costs and improve the availability of equipment needed to access e-content. While it may take time, the MoE should also pursue changes in procurement regulations that would permit increased standardization and strategic sourcing to lower the TCO of information technology in the schools. Until the MoE has a long-term technology management and support plan against which donor inputs are programmed by the MoE, infrastructure replenishment and technical costs will be donor￾driven and ultimately unsustainable. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 39 of 61 7 Conclusions 7.1 The role of the JEI The JEI has been a highly visible initiative in the Jordanian educational system. We were told in interviews at the Curriculum and Textbook Directorate that the high level of support given to the JEI enabled it to overcome some bureaucratic hurdles that would have considerably hampered the MoE if it had tried to carry out this program alone. The JEI PMO in particular has provided important strategic, technical, and managerial capacity. Specifically, the JEI PMO has provided the following: ƒ PMO personnel have been regularly and visibly involved in coordinating meetings among stakeholders and service providers. This has included meetings of JEI governing and advisory bodies, e-content developers and the Curriculum and Textbook Directorate, the ICT Directorate, JT, and NBN, and others. ƒ The JEI PMO has devoted considerable effort to strengthening technical capacity within Discovery Schools and the most relevant directorates of the MoE. The JEI internship program seems to have been especially important in supporting computer lab technicians in their functions and in encouraging teachers to use e-content, thereby facilitating implementation of the e-curriculum pilots and roll out. ƒ JEI site surveys, designed and managed by the PMO, appear to provide a unique and important source of information on the progress of ICT innovation in the pilot schools. ƒ The PMO has maintained close contact with MoE personnel in the curriculum directorate and provided significant support for developing and testing e-content. ƒ The PMO has also played an important role in providing feedback to e-content developers from teachers and students testing the technology, and to the MoE concerning the roll out process and its lessons for nationwide scale up. We encountered very strong support for the JEI from MoE personnel in some areas such as e￾curriculum and maintenance. One private sector interviewee said that as a result of the JEI, Jordan now has a road map to the development of a more intensive and beneficial application of ICT to education. Others were complimentary about the persistence of JEI staff and the benefits that they had brought to the ERfKE. It is difficult to imagine these initiatives continuing to move forward without the continued active participation of a program with the stature and resources of the JEI. In addition, the PMO has played a vital role in coordinating and managing the initiative. The ICT models piloted tested in Discovery Schools are serving to adapt best practices from other countries and reveal significant scale-up challenges. We encourage the JEI to strengthen the M&E system for measuring results and to continue investigating potentially useful ICT models. However, the key challenge for the JEI is to facilitate simple and effective solutions to fundamental problems that threaten scale up, and a system that encourages scale up based on initiative and rewards at the school level. These issues are discussed in more detail in following sections. The high level of support given to the JEI enables it to overcome some bureaucratic hurdles to achieve results that would not be possible without the JEI. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 40 of 61 7.2 Selecting ICT models An evaluation of MoE computer lab deployment strategy (The Directorate of Curriculum and Textbooks and the Supporting Jordan’s Education Project, 2006) reported the following: A common finding in the literature is that the early uses of ICT to enhance learning could be described as “doing old things in new ways.” A good example of this is digitized textbooks. A well-designed electronic textbook may be more attractive than a traditional textbook. It may have animation and sound, and it may hold students’ attention better than a traditional textbook. But “doing old things in new ways” does not represent a new and effective use of ICT. ICT should be used to do “new things in new ways.” One fundamental concern identified during our short visit, a concern we could not adequately investigate, was whether JEI was in fact promoting doing “new things in new ways,” or would more accurately be described as helping the MoE “to do old things in new ways.” The factors that determine the best choice of ICT are many and varied. Different models are appropriate in different environments. Thus, in ICT in education, “best practice” tends to be a concept better applied at the management level rather than for the choice of model. Some of those best practices are as follows: ƒ Tailoring the technology to the instructional needs, affordability constraints, and other constraints on a case-by-case basis The National Education Strategy has established key curriculum principles and defined knowledge economy skills: ƒ The curriculum supports the acquisition of Knowledge Economy skills in communication, management of information, use of numbers, critical thinking, creativity and innovation, problem solving, personal management, and teamwork. ƒ The acquisition of higher order skills—analysis, synthesis, evaluation, and application—is part of the essential learning outcomes and serves as the basis for assessment strategies. Among the many progressive statements in the National Education Strategy and in the ERfKE vision documents; these two principles establish the criteria for selecting the most effective uses of ICT to support learning. During the review of the International literature on ICT these principles match the best in the world. Jordan’s vision is a world-class vision. School jurisdictions around the world are doing exactly what Jordan is doing, trying to increase access to ICT for students to enhance learning and increase achievement. The literature consistently highlights that greater access to ICT in and of itself does not improve student learning. Only when: ƒ The pedagogy is changed to a student-centered approach ƒ Teachers are engaged in lifelong professional development ƒ The curriculum is changed to encourage collaborative learning, self-directed study, authentic tasks … will increased access to ICT enhance learning. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 41 of 61 ƒ Measuring and monitoring total cost of ownership, including non-quantifiable factors, such as the adequacy of technology to prepare students for their roles in the information economy ƒ Using EMIS effectively for informed decision making ƒ Focusing on simple, sustainable solutions. Far more ICT failures occur from an overly ambitious effort than from too modest an approach. Experience in many countries over many years demonstrates that most very ambitious ICT projects fail to meet their objectives. In the case of Jordan, the ability to scale benefits depends on the ability of the MoE to scale and sustain ICT infrastructure, which requires focusing on developing and promoting simple, standardized systems and basic technology planning and management capacity. We were informed that early in its life, JEI considered a wide variety of ICT devices for the schools, but eventually focused mainly on funding from international private sector organizations for Jordanian educational content developers. They in turn created content first in mathematics, then in science, Arabic, EFL, and other fields. Other forms of private sector involvement included training by the Intel Teach program and seconding private sector technical specialists to the project. Discovery Schools are a central element to JEI’s approach. ICT models and e-content were first pre￾piloted and then pilot tested in a subset of these schools and, when ready, rolled out to remaining Discovery Schools. When that roll out is successful, the technology will be handed over to the MoE to scale up to 3,200 schools nationwide. The system of nearly 100 Discovery Schools in Amman was organized to include a wide range of school conditions. JEI has encouraged the creation of the lab technician position based on experience in the schools, and has created an internship program that involves young professionals to encourage change, provide assistance to discovery school teachers and lab technicians, solve problems, and more recently to conduct site surveys that provide data on system performance to the JEI and the MoE. In general Discovery Schools appear to have stronger ICT infrastructure and are better run than one can expect for in many schools in Jordan. Many schools in more remote areas of the country do not have adequate access to electricity and are not yet connected to the Internet. The JEI is aware of this. The ICT models piloted in Discovery Schools are somewhat conservative applications of models widely used in other countries. The JEI has been careful to introduce and adapt these to the Jordanian context. 7.3 ICT models piloted in discovery schools The JEI has piloted two ICT models as the primary vehicles for testing the delivery and use of e￾content: computer labs, and portable laptop computers and projectors issued to teachers. These and other possible ICT models are discussed in more detail in following sections. 7.3.1 Computer labs The computer lab model concentrates computers in a shared network facility, where close technical support is provided by a lab technician, security is provided by strong steel doors and bars on windows, and climate control is provided by a split-unit air conditioner. The computer lab model requires allocating scarce space to a shared facility, but reduces the cost of providing adequate technical support, security, and climate control. Concentrating many computers in one room also simplifies and reduces the cost of connecting them to e-content and the Internet. Finally, it enables a lab technician to provide close support for a teacher who may be leading a class of 45 students without any other teaching assistance. The computer lab model, Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 42 of 61 already present in some form in non-Discovery Schools, should be a cost-effective way of sharing scarce resources. Computer labs require space that must be reallocated from existing uses or must be newly constructed, and construction is a time-consuming process. Principals, teachers, and lab technicians must schedule shared use of the facility carefully to maximize use. It also takes time to move students in and out of the facility, taking precious class time. Finally, as enrollment increases, the number of computer labs must increase to maintain reasonable student-computer ratios. Demand projections by the Directorate of Curriculum and Textbooks and the Supporting Jordan’s Education Project (2006) suggest that, based on curriculum e-content requirements, 2,180 new computer labs will be needed by the 2008–2009 school year. While this number may be high, it is unlikely that anywhere near this many computer labs can be constructed and equipped to keep pace with increasing enrollment. In addition there are problems maintaining the availability of computers in existing labs. It is likely that the supply of computer labs will continue to lag behind demand, limiting access to e-content. 7.3.2 Laptop computers and portable data-show projectors The JEI provided selected teachers with laptops and portable data-show projectors to enable them to use e-content in the classroom. The JEI initially intended to provide a laptop to each teacher involved in classroom teaching of the e-content. More recently JEI has been considering a shared laptop model. The JEI provided Discovery Schools with wireless networks to enable Internet connectivity in the classroom. Teachers have received training through the International Computer Drivers License (ICDL) and Intel Teach programs. In interviews we conducted for this assessment, teachers were uniformly positive regarding the usefulness of the laptops and portable projectors. While implementation of MoE policies regarding this equipment varied among schools we visited, and the issue of laptop maintenance has become critical, this ICT model has several positive attributes. Laptop computers and portable data-show projectors can be distributed quickly and can be shared among teachers. Wireless network connectivity can also be deployed quickly and, if standardized, requires very little technical support and maintenance. This model also conforms to Jordan’s classroom model, in which students normally stay in an assigned classroom while specialized teachers visit them throughout the week according to the subjects of the curriculum. If permitted by their principal to take laptop computers home, teachers may also use them after school hours to continue learning and preparing lesson plans and exercises. Studies by Gartner and others conclude that the TCO for laptop computers is higher than that of desktop PCs. Laptop computers are more expensive to purchase and repair, and have lower performance than desktop PCs at the same price point. Lower performance means they generally become technologically obsolete sooner than desktop PCs as newer software demands greater processor performance. However, desktop PCs cannot provide the benefits of mobility. These benefits include being taken from classroom to classroom, shared among teachers, and used to prepare lesson plans and exercises at home. This model provides greater access to more e-content in more classrooms. While it does not provide students that same level of small-group and individual e-content interaction as computer labs, laptop computers make it possible to provide some level of classroom access to e-content as enrollment increases. It is likely that the supply of computer labs will continue to lag behind demand, limiting access to e-content. Laptop computers and portable data-show projectors make it possible for teachers to provide some level of classroom access to e-content as enrollment increases. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 43 of 61 7.3.3 Media rooms and interactive white boards Recently other technological approaches have drawn interest. These include media rooms and IWBs. Media rooms concentrate limited ICT resources, such as laptop computers, data-show projectors, IWBs, and other electronic media tools. Like computer labs, media rooms can be equipped with adequate security and climate control, and concentrate scarce resources so they can be supported more easily than if they are scattered among many classrooms. Media rooms are more flexible in their space requirements, and require less equipment than computer labs. They are more likely to be able to use existing space, and can be equipped more quickly and at a lower cost than computer labs. Teachers bring their students to a media room for e-content enabled lessons. As with computer labs, some class time is lost moving students to and from media rooms, and students have less individual contact with e-content than in computer labs. However, as discussed in section 5.3, Jordanian experience with IWBs appears consistent with results in other countries that suggest at least short-term gains in student achievement. Data regarding sustained gains are less conclusive. However, there is enough evidence to support a more thorough analysis of the VOI and sustainability of IWBs and media rooms in the Jordanian context. 7.4 Other possible ICT models The JEI is aware that the range of technological options deployed in Discovery Schools is limited. This has been driven by the need to support the deployment of EduWave e-content and to remain within the envelope of MoE sustainability while increasing MoE technical capacity. ICT includes a wide range of devices including graphing calculators, handheld PDAs, tablet computers, video cameras, mobile phones, science probe-ware devices, Global Positioning System (GPS) receivers, and others being developed and released daily. Some technologies that might be considered include the following: ƒ Translating and adapting ICT-supported testing and assessment tools, such as needed assessments for individual learning problems; monitoring individualized instruction strategies as students move ahead on their own pace; and for assessing basic skills (reading and math skills in particular). ƒ Using available software and voice recording technology to test reading ability in Arabic and English at the primary level. ƒ Introducing, in higher grades, ICT-supported skills assessment tools benchmarked for international certification of interest to industries such as information technology, financial management, and civil engineering. ƒ Introducing assistive technology applications for special learning needs, such as students with hearing, vision, or mobility impairments. ƒ Using arcade-quality, interactive simulation “games” on selected topics, such as water, conflict, building technologies, and transportation systems. ƒ Using Geographical Information System (GIS) mapping and monitoring systems for school facilities and e-content in geography and other subjects. In this section we discuss several ICT models that are being used or are being considered by other education systems. These include the following: There is enough evidence to support a more thorough analysis of the VOI and sustainability of IWBs and media rooms in the Jordanian context. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 44 of 61 ƒ Thin client computers ƒ Free and open source software ƒ Mobile laptop computer labs ƒ One laptop per child, or the one-to-one model ƒ Live and prerecorded video streaming of lectures by outstanding teachers to school media rooms over the NBN. Thin client computers come in several configurations ranging from bespoke terminals to reconfigured surplus desktop PCs. They are most commonly used to replace industry standard desktop PCs (sometimes called fat clients) in computer labs. The advantages of thin client computers may include lower initial per student costs and lower technical support costs. Some thin client computers have no internal hard disks. Some use open source software with no recurrent licensing costs. They may require less space and run quieter than desktop PCs. They may not require climate control. Because the hardware and software is simpler, more standardized, and less common, they may be less subject to malicious software, more reliable, and require less technical support. A study by Light and Gersick (2003) of the use of the thin client model in U.S. schools reported significantly fewer technical support problems and lower costs than fat client models. However, thin client computers may also require more server resources and may not support multimedia software and hardware required by e-content. During this assessment, we visited two computer labs equipped by the MoE with thin client computers on a pilot basis. We heard from several sources that this experiment was generally considered a failure due to the inability of the selected thin client computers to run software used in the curriculum and by the e-content. This software includes Microsoft Office, Microsoft Movie Maker, Microsoft Internet Explorer Version 7, Adobe Flash, and Adobe Shockwave. Becta (2007) documents similar issues in case studies of 12 U.K. public schools using thin client technology. Given the software requirements of the curriculum and e-content, we believe the JEI is better served by focusing on increased standardization and better management of fat client computer labs. Open source software (OSS) is software for which the underlying programming source code is available to users so that they may study, copy, modify, make new versions to meet specific needs, and provide copies of the software to other users. OSS licenses carry a Copyright statement, but normally retain rights of attribution only, offer the underlying source code on request, and require users to apply the same licensing terms if they release modified versions of the software. OSS often carries no licensing costs. Companies in the OSS market earn revenue by charging for specially packaged versions of the software and by providing expert technical support. The most recognized example of OSS is the Linux operating system. Popular versions include Red Hat Linux16 and Ubuntu Linux,17 which is also available in a version packaged specifically for use by educators. Other relevant examples include OpenOffice,18 which is compatible with Microsoft Office; the Mozilla Firefox Web browser,19 which competes with Microsoft Explorer; and the moodle20 Web-based CMS. In general, schools use OSS to reduce costs (Becta, 2005). At least one school district in California converted to OSS recently to reduce costs (Loftus, 2007). In addition to reducing or eliminating software licensing fees, OSS can often be used effectively on older, less powerful computers. OSS can also provide more technical flexibility and broader experience for students, and is less frequently targeted by malicious software. Studies in England (Becta, 2005) indicate that, on average, the TCO of computers is significantly lower in schools using OSS than in schools using proprietary software. 16 www.redhat.com 17 www.ubuntu.com 18 www.openoffice.org 19 www.mozilla.org 20 www.moodle.org Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 45 of 61 However, the literature suggests there are many factors to consider, including compatibility with the curriculum and e-curriculum technical requirements. Without more careful analysis of the specific case of Jordan we cannot conclude that OSS is a realistic, lower￾cost alternative. However, we observe that the influence of private sector partners may eliminate any consideration of OSS by the MoE. While the input of private sector partners has been a vital part of the JEI, we believe it is important for the MoE to remain objective and open in analyzing the VOI of technical alternatives. The MoE should be particularly aware of potential technical bias on the part of consultants provided by private sector partners. Education systems in several other countries use mobile laptop computer labs to bring computer labs to students. Laptop computers are typically stored in a mobile cart that provides wireless network connectivity, a data-show projector, a printer, and an electrical charging system for the laptops. The laptop computers and other equipment can be stored in the locked cart when not in use. These systems are commercially available from many different sources. On the surface, this appears to be a good match to the class model in Jordan, in which students remain in their assigned classroom while specialized teachers come to them throughout the day. However, initial and recurrent costs for these systems are likely to be higher than for fixed computer labs using desktop PCs or thin client computers. Users of this approach report other problems, including short laptop battery life. We learned during this assessment that the MoE has experimented with laptop computers and data-show projectors on carts, and shared among teachers. These had very limited mobility in multistory schools without elevators, which include most schools in the Kingdom. Considering these issues, it is unlikely that mobile laptop computer labs would be a cost-effective model for Jordan. In the most developed countries, some school districts are piloting a model that provides an industry standard laptop computer to each student. In the U.S., this model is called one-to-one computing. Case studies by the CoSN (2006, 2007) provide examples of the model’s benefits, costs, and challenges. The high initial and recurrent costs of this model are likely to confine it to the most affluent school systems for the near future. In November 2005, at the World Summit on the Information Society (WSIS) in Tunis, Tunisia, Nicolas Negroponte and Kofi Annan unveiled a working low-cost laptop computer designed to be issued to each individual child for learning inside and outside the classroom. This model is based on constructionist learning theories developed by Seymour Papert, Alan Kay, Negroponte, and others. The One Laptop per Child (OLPC) association founded by Negroponte and others at the Massachusetts Institute of Technology (MIT) Media Lab is a nonprofit organization dedicated to developing and promoting this model. At the 2006 World Economic Forum in Davos, Switzerland, the United Nations Development Programme (UNDP) announced it would back the laptop. While the unit price target remains US$100, current projections place the unit cost at about US$175. These announcements sparked great interest and controversy. While there is great skepticism that this model is sustainable or the most cost-effective use of resources, particularly in developing countries, some 15 countries have expressed some commitment to the OLPC model. Intel subsequently released their Classmate PC, which is a laptop hardware referenced model designed to meet OLPC learning objectives using Intel technology. Intel advanced its efforts rapidly, moving from concept to volume manufacturing of US$200 laptops in 18 months. In July 2007, Intel announced that it had joined the board of OLPC, ending direct and open competition between the two initiatives. While the input of private sector partners has been a vital part of the JEI, we believe it is important for the MoE to remain objective and open in analyzing the VOI of technical alternatives. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 46 of 61 We were informed that the JEI had considered the OLPC model and rejected it as unrealistic in Jordan. Certainly the cost of rolling out one computer per child to 3,200 schools and 1.2 million students would be prohibitive now. Also, as mentioned previously in this report, many case studies show that initial cost is a small percentage of the overall TCO for computer hardware. However, progress to date suggests the initial unit cost of entry-level OLPC laptop computers should come down from the US$200 range to the US$100 range within 18 to 24 months. Jordan’s economy has shown steady growth, and the Kingdom is committed to improving education as a means toward a knowledge economy. At some point in the future, the OLPC model will become affordable in Jordan, at least for some educational applications. The way in which individual personal computers are best used in education differs from the way computers are best used when shared, whether in a computer laboratory or by a teacher in the classroom. Many questions remain concerning the cost effectiveness of the OLPC model, but ongoing pilot projects in other countries are beginning to provide useful information. As the JEI experience has shown, a good program to develop such content in Arabic that is compatible and complementary to the ongoing curriculum, to test it in a set of Discovery Schools, and evaluate its costs, effectiveness, and rollout potential, will take several years. If such an effort does not begin soon, it is likely that the relevant information for decisions regarding expansion of school computer networks will not be available when it is needed by the MoE. We suggest that the JEI continue analyzing the potential VOI of the OLPC model and consider a program to design and pilot OLPC-based educational content for the future. Finally, the NBN opens exciting new possibilities to the JEI. Provided security policies and systems are put into place to control malicious software, the NBN provides far more bandwidth than required to deliver existing e-content to schools. The NBN provides sufficient bandwidth to support video conferencing between schools, and streaming of video content live or on-demand. This could enable an exceptional or specialized teacher in one school to teach students gathered in media rooms in other schools around the Kingdom. It could also enable on-demand video streaming of prerecorded lectures and demonstrations. Both techniques could enable university professors to provide expert instruction in specialized subjects to secondary schools. According to ITG, EduWave now offers a component that supports video conferencing. We encourage the JEI to explore these possibilities. 7.5 Developing and maintaining e-content Under the JEI, private sector partners have developed content complementary to official textbooks and standard curriculum, and have provided teachers with suggestions for selected content modules linked to specific lessons and objectives within the curriculum. This has been an important component of the current JEI technological model. While e-content provided through EduWave must pass extensive appropriateness and quality review by the Curricula and School Textbooks Directorate, teachers and students also use resources from the Internet and develop some of their own e-content. Two related issues suggest that the current centralized e-content strategy might be expanded to leverage the initiative of teachers: 1. The curriculum is changing and the texts may already be outdated, so that a more flexible system of providing enrichment content to teachers might be useful 2. There is a huge and constantly increasing amount of content available on the Internet, far exceeding that developed by the private partners that could be used by teachers. A carefully designed Web site could provide teachers access to the following: We suggest that the JEI continue analyzing the potential VOI of the OLPC model and consider a program to design and pilot OLPC-based educational content for the future. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 47 of 61 ƒ e-content developed and contributed by teachers and students; ƒ links to appropriate Internet content contributed and classified by teachers; ƒ comments on e-content resources by teachers who have used them; ƒ ratings for the resources from students and teachers; and ƒ review and oversight by the Curriculum and Textbook Directorate. Such a Web site could include strong ordering algorithms to present the most relevant resources to teachers searching for e-content, and algorithms to suggest resources to teachers that have been used by other teachers with similar patterns of resource use. This would leverage and reward the initiative of teachers, and would harness the peer review capacity of the teaching community to enrich and improve e-content resources. This strategy explicitly recognizes the following: ƒ Broadened authority of teachers to select appropriate materials to meet their specific class needs of the moment ƒ Broadened interest in enabling students and parents to select enrichment materials not assigned by teachers. According to information provided by ITG, the EduWave platform already contains at least some features that could be used to implement this strategy. 7.6 Scaling up results The Hawthorne effect21 suggests that almost any experimental program may have benefits that sustained when the change becomes institutionalized. Moreover, the resource-intensive efforts made by JEI to insure adequate development and testing of their innovations may not be replicable on a larger scale in the MoE without increased capacity and external technical assistance. We were told that students are getting better at using the technology, and that the full impact of the introduction of computers in the schools will not be realized until students have been exposed to the technology for a number of years as they progress through the grades. On the other hand, it is likely to require years for a senior teacher to fully master the art and craft of enriching his or her teaching using ICT; again, the short period of this work may fail to identify, much less measure, the long term effects of introducing ICT. Thus the effects observed during the past year or two by JEI may not be fully representative of the results of expanding the program in the future and indeed may not be sustained over the long run. The approach piloted by JEI and its partners leaves it to the teacher to determine how and how much to use the technology. Added to the great variability among teachers, classes, students, and communities, there are great differences in how different teachers would use the same technological options in similar circumstances. Again, the complexity of this situation would seem to make scientific analysis of the experience of JEI difficult, and introduce uncertainty in the projection of the observed results. We question some of the human factors in the system being piloted. We were told that no significant campaign had been made to inform parents about the use of ICT in education and to enlist their support and involvement in the JEI program. Similarly, the incentive systems for teachers and principals 21 The Hawthorne effect refers to a phenomenon which is thought to occur when people observed during a research study temporarily change their behavior or performance (this can also be referred to as demand characteristics). Others have broadened this definition to mean that people’s behavior and performance change following any new or increased attention. The Hawthorne studies have had a dramatic effect on management in organizations and how people react to different situations. The term gets its name from a factory called the Hawthorne Works, where a series of experiments on factory workers were carried out between 1924 and 1932. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 48 of 61 seemed weak. The JEI team also cited the motivation and capability of the principal as the single most important critical success factor in each school. Informing and engaging parents, encouraging and rewarding the initiative of principals and teachers, and publicizing achievements and disseminating best practices could be major elements in scaling up and sustaining effective ICT models. 7.7 General conclusions The JEI has made considerable progress introducing ICT and its use in pedagogy to improve education outcomes. Much more remains to be done in making the educational system adequate for Jordan’s success in a global knowledge economy. The process of ICT innovation is just beginning in Jordan. JEI represents an important instrument for increasing the speed and quality of the innovation process—it has played a critical role in selecting, adapting, and piloting ICT models in Jordanian education. Much more attention needs to be paid to the hand off of materials from Discovery Schools to the MoE for national scale up. Outsourcing at least some technical support appears to be effective, but requires both technical and management competence in the MoE that will be difficult to develop and maintain. The innovations planned for the MoE should stretch but need not greatly exceed its capabilities to manage the innovation process JEI-led initiatives in monitoring and evaluation are essential and should be strengthened: increased input from teachers, students, and parents, and more systematic and rigorous monitoring and evaluation are needed to accelerate the evaluation of technology models and improvement of e￾content. The JEI is likely to face serious challenges as it makes the transition from a high-profile WEF project to an operational NGO with continuing responsibilities for both technology innovation and for assisting public education officials with the scale up of the innovations. The JEI PMO is a small organization with enormous responsibilities to help meet the Kingdom’s objective of building a knowledge economy. At present no other Jordanian organization has the prominence, experience, and capacity to play this role. To achieve its mission the JEI will need continued high-level support, additional capacity, and the continued funding support of the Government of Jordan, private sector partners, and multilateral donors. 8 Recommendations In this section we present specific recommendations concerning scaling up from pilot models in Discovery Schools to full-scale deployment. These recommendations do not address the crucial issue of monitoring and evaluating learning results, outcomes, and impacts. Those issues are addressed separately as part of this overall JEI assessment effort. We believe JEI priorities moving forward should be as follows: 1. Solve the most critical information technology management problems using simple, sustainable systems. 2. Solve the human capacity issue for technical support by expanding the internship program to address weaknesses at the education directorate and central ICT Directorate levels. 3. Prove these solutions work in Discovery Schools before scaling up. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 49 of 61 It is essential for the JEI to work with the MoE to resolve key infrastructure support challenges for the 100 Discovery Schools before these models are introduced to other schools in Jordan. 4. Develop a model for scaling up successful models and approaches using a strategy that fosters, rewards, and disseminates leadership and innovation at the school level. 5. Institutionalize the process of improving e-content and innovating in the use of ICT in grades 1–12 in public school education, using the Discovery Schools as a test bed for future improvements and innovations. 6. Expand public-private partnerships to include employers of knowledge workers. Achieving these objectives will require the status, organizational, and technical capacity of the JEI program, and a successful strategy for handing off solutions to the MoE. The following sections discuss each of these key recommendations in more detail. 8.1 Develop simple, sustainable solutions to critical infrastructure problems Desktop PCs and data-show projectors are important aspects of ICT enrichment of education in grades 1–12 and should be affordable in Jordan. It should also be possible to sustain a limited number of laptop computers and projectors shared among teachers. The development of Arabic e-content specifically designed to enrich the curriculum provides essential tools for the blended learning approached introduced by the JEI. The hardware and software infrastructure for such services are mature and their use is justified to achieve ERfKE objectives. The JEI has selected ICT models that should be scalable and within the capacity of the MoE to sustain. The MoE does not yet, however, have the capacity to manage, support, or sustain the ICT infrastructure. Fundamental security, management, technical support, and long-range planning are not in place. There is some evidence that the JEI and MoE ICT consultants have made progress in helping the MoE in some of these areas. 8.2 Expand the internship program to further strengthen technical support capacity The JEI internship program appears to have been very successful in overcoming resistance to change and increasing technical support capacity at the school level. The program has been effective in transferring capacity from young university graduates to computer lab technicians. The interns have also helped to encourage teachers and overcome resistance to change. Many interns have completed their terms and been successful in entering the private sector. Schools and interns benefit. We believe it should be possible to expand this program to strengthen technical support capacity in health directorates as well as to help the ICT Directorate manage a seemingly continuous stream of initiatives and mandates. Interns could also help the ICT Directorate with the challenge of developing and implementing sustainable and standardized solutions for security patch management and virus protection. Finally, the intern program could be expanded to help scale up ICT innovation from Discovery Schools to non-Discovery Schools. 8.3 Prove these solutions work in Discovery Schools before scaling up ICT models introduced in Discovery Schools have been successful, but have also highlighted key challenges of technology management, maintenance, technical support, and security. Discovery Schools provide an excellent environment in which to develop, test, and Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 50 of 61 adjust solutions to these issues. We believe it is essential for the JEI to continue working closely with the MoE and other JEI partners to resolve these key challenges for the 100 Discovery Schools before these models are introduced to other schools in Jordan. 8.4 Develop a model for scaling up that fosters, rewards, and disseminates success The JEI has observed that effective use of ICT to improve teaching and learning is closely related to the initiative and motivation of school principals and teachers. Strategy for scaling up the use of e-content should be based on a system that motivates principals and teachers; rewards motivation, innovation, and achievement; and shares and promotes innovations and best practices. Exemplary schools and teachers can be used as “vanguards” through reward and exchange programs that demonstrate what can be achieved and that help to promote change. ICT upgrades would be introduced into schools that meet certain basic criteria and have demonstrated their intention to use the technology effectively. This “organic” scaling approach also enables support system capacity to be scaled at a sustainable rate. Such a strategy will help avoid the waste and negative affects that result when technological innovations are forced on users without demonstrating potential benefits, overcoming resistance to change, or providing adequate technical support. 8.5 Expand public-private partnerships to include employers of knowledge workers Current JEI private sector partners are primarily suppliers of technology or technology services. The JEI has benefited from significant partner contributions of equipment, software, services, and technical expertise. The primary ERfKE objective is to create knowledge workers for a knowledge economy. We believe it may be useful to broaden the set of private sector partners to include additional employers of knowledge workers, such as representatives of the financial services, tourism and transportation, and Internet-enabled service-export industries. Similarly, partnerships might be considered with institutions of higher education to strengthen the analytic capacity of the program, and/or with international organizations involved in e-learning, such as the relevant portions of UNESCO. The JEI should broaden the set of private sector partners to include consumers of knowledge workers, such as representatives of the financial and business decision support sectors. Strategy for scaling up the use of e-content should be based on a system that motivates principals and teachers; rewards motivation, innovation, and achievement; and shares and promotes innovations and best practices. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 51 of 61 Annex A: Schedule of Meetings and Names of Primary Contacts Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 52 of 61 Time Function Topic Attendees Title Organization Location Status Day 1: Sunday Sept. 2 8:30– 9:30 Meeting JEI PMO Meeting Luma Atallah Projects Coordinator JEI MoICT ok Niveen Jum’a Technology Track Coordinator JEI 2:30– 3:30 Meeting Technology track meeting Niveen Jum’a Technology Track Coordinator JEI MoICT ok 3:30– 4:30 Meeting Cisco meeting Othman Al￾Suqi MoICT ok Day 2: Monday Sept. 3 9:00– 11:00 Meeting JEI PMO Meeting Haif Bannayan Executive Manager JEI MoICT ok Luma Atallah Projects Coordinator JEI Niveen Jum’a Technology Track Coordinator JEI Shaden Hendawi Head of PMO JEI 1:00– 3:00 Meeting MoE/ ICT Staff meeting Mohammad Qubbaj Head of network division Ministry of Education MoE ok Fatima Kilani Head of projects division Ministry of Education Khalil Awarntani Head of maintenance division Ministry of Education 4:00– 5:00 Meeting Intel meeting Rula Habash Intel MoICT ok Day 3: Tuesday Sept. 4 9:00– 10:30 Meeting Digitization dept. meeting Qasem Al￾Khateeb Head of digitization dept. Ministry of Education Curricula Directorate ok 11:00– 1:00 Meeting MoE/ ICT directorate plans for DS Bassam Kahhaleh IT Advisor Ministry of Education MoE ok Niveen Jum’a Technology Track Coordinator JEI 3:00– 5:00 Meeting Maintenance company meeting Ayman Zghoul CEB CEB ok Musheer Isma’eel CEB Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 53 of 61 Time Function Topic Attendees Title Organization Location Status Day 4: Wednesday Sept. 5 8:30– 10:30 Visit DS visits Ahnaf Bin Qais ok 12:00– 2:00 Meeting Rubicon meeting Reem Al-Far Rubicon Rubicon ok 3:00– 5:00 Meeting USAID meeting David Bruns USAID US Embassy ok Day 5: Thursday Sept. 6 8:30– 12:30 Visits DS visits Al-Shifa’ Bint Awf Salah Al-Din ok 1:00– 3:00 Meeting NBN meeting Nisreen Araj MoICT MoICT ok Ali Baydoon MoICT Abdullah Abdel Qader MoICT 3:00– 5:00 Meeting NCJ meeting Rami Karmi NCJ MoICT ok Mustafa Qtaishat NCJ Asma Abu Hussein JSS Day 6: Sunday Sept. 9 11:30– 1:30 Visit DS focus group Queen Rania DS ok 2:00– 4:00 Meeting JT meeting Mohammad Al-Azzeh JT Hashim Station ok 4:30– 5:30 Meeting ITG meeting Batool Ajlouni VP/Business Development ITG ITG ok Day 7: Monday Sept. 10 9:00– 10:30 Visit ERfKE school visit Nuzha elementary female ok 11:30– 1:30 Visit DS focus group Jameel Shaker DS ok 2:30– 4:30 Meeting Wrap up meeting with JEI PMO Haif Bannayan Executive Manager JEI MoICT ok Luma Atallah Projects Coordinator JEI Shaden Hendawi Head of PMO JEI Osama Obaidat JEI Niveen Jum’a Technology Track Coordinator JEI Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 54 of 61 Annex B: Focus Group Script and Questions These consultants are here working on behalf of USAID. They have been asked to help the JEI to evaluate the activities of the JEI. In this focus group session we would like to learn more about how the technology introduced by the JEI has affected your ability to teach and the learning achievement of your students. We would like to know your opinions and suggestions. All of your responses will be anonymous. We will consider the views of the entire focus group, but we will not record your names or associate your responses with your names. Our objective is to learn information that may be helpful to the JEI to improve their program. 1) Has the technology and training you have received through the JEI program changed the way you teach your students? a) If so, what do you believe are the major changes? 2) Has the technology introduced through the JEI program affected the academic performance of your students? a) If so, how has it affected their performance? 3) What has been the role of the principal in supporting your use of this technology for teaching? 4) Has the JEI been effective in helping you to use the new technology? 5) What changes to you believe the JEI should make to improve this program? 6) Have you experienced any difficulties using the ICT introduced through the JEI? a) If so, could you describe these difficulties? b) How have these difficulties affected your ability to teach your students? 7) Did the parents of your students have any concerns about the changes that the JEI were introducing in your school? a) If so, can you describe these concerns? b) Has the attitude of the parents changed since the JEI began? c) If so, how has their attitude changed? 8) Approximately what percentage of your students has computers (PCs) at home? Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 55 of 61 9) Approximately what percentage of your students has access to the Internet from home? 10) Do the students transfer any work from home to school electronically? a) If so, how do they do this? By e-mail? USB flash drive? CD-ROM? 11) During a typical school week, approximately how many times do you use the computer lab for teaching your students? 12) About how many total hours would you estimate your students spend in the computer lab each week? 13) In your opinion, has this time been very productive? 14) Has the use of the e-content in the computer lab improved the academic performance of your students? a) If not, what do you believe needs to be changed or improved to increase the positive impact of this technology on the learning of the students? 15) Have you experienced any difficulties in using the computer labs? a) If so, could you describe these difficulties? b) How have these difficulties affected your ability to teach your students? 16) Are you using e-content through EduWave to teach your students? a) If so what e-content are you using to teach your students? b) How has the use of this e-content affected the learning achievement of your students? c) Have you encountered any problems in accessing or using the e-content through EduWave? d) If so, what problems have you encountered? e) How has this affected your ability to teach your students? f) In your opinion, what is the quality of the e-content available to you through EduWave? g) What changes do you believe should be made in the e-content available to you through EduWave? h) What changes do you believe should be made in EduWave? 17) What is your opinion of the quality of the e-content available to you through EduWave? 18) In addition to the e-content available through EduWave, what other e-content and software, if any, are you using to teach your students? 19) What additional e-content would you like to see made available to you for teaching your students? 20) Do you use the Internet at all in your teaching? Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 56 of 61 a) If so, can you explain how you use the Internet? b) Do the parents of your students have any concerns about the use of the Internet by your students? 21) If so, what concerns have they expressed to you? 22) When your students are in the computer lab, do they have access to the Internet? a) If so, can you describe how your students are using the Internet in your classes? b) Have you experienced any problems with the way your students are using the Internet or the type of material they have access to on the Internet? c) Have you experienced any problems in the speed of access to the Internet from the computer lab? 23) Do you use a portable laptop and projector for teaching? a) If so, can you describe how you are using this technology? b) How has the use of the laptop and projector affected your ability to teach your students? c) How has the use of this technology affected the learning achievement of your students? d) Have you experienced any problems using the portable laptop computers and projectors? e) If so, can you describe these problems? f) How have these problems affected your ability to teach your students? g) What changes to you believe should be made in the laptop program to make it better? 24) Does your school have a Smartboard? a) If so, have you used the Smartboard in your classes? b) Do you believe the Smartboard has been useful? 25) If so, can you describe how the Smartboad has been useful? 26) Are there other technologies that you believe would be useful for teaching your students? Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 57 of 61 Annex C: List of Reference Documents and Internet Resources Documents Al-Nahar, T., Chang, M., and Koda, Y. 2004. The Jordanian Use of ICT in Education Policy and Practice: An Arab and a World Perspective. Interim report of a joint study between Government of Jordan, National Center for Human Resource Development, and World Bank—Middle East and North Africa Region, Jordan Education Initiative. Alnoaimi, T. 2005. JEI Deployment and Scaling in the context of wider ERfKE. Microsoft PowerPoint presentation. Anavitarte, L. and Jump, A. 2007. Intel and OLPC Partnership is Good for Intel. ID Number: G00150448. Gartner Inc. Bakia, M. 2000. The Costs of Computers in Classrooms: Data from Developing Countries. Mimeograph. The World Bank, Washington, D.C. Becta. 2006. ICT Investment Planner—Guidance Notes. Becta. 2005. How can the use of an interactive whiteboard enhance the nature of teaching and learning in secondary mathematics and modern foreign languages? Becta Research. 2007. Thin Client technology in schools, a summary of research findings. Becta. 2006. Managing ICT Costs in Schools. Becta. 2005. Open Source Software in Schools, A case study report. Becta. 2005. Open Source Software in Schools, A study of the spectrum of use and related ICT infrastructure costs. Becta. 2005. Using Open Source Software in Schools, Information Sheet. Becta. 2003. What the Research Says about Interactive Whiteboards. Cawthera, A. 2002. Computers in Secondary Schools in Developing Countries: Costs and Other Issues. (Including Original Data from South Africa and Zimbabwe). World Bank, Washington, D.C. Central Regional Educational Laboratory (NCREL). 2003. enGauge 21st Century Skills: Literacy in the Digital Age. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 58 of 61 Chemonics International. 2006. Jordan ICT Sector Strategy. USAID Contact 278-C-00-02-00210-00, Achievement of Market-Friendly Initiatives and Results (AMIR II). Cisco Systems Inc. 2006. Cisco Corporate Citizenship Report. Consortium for School Networking. 2003. Understanding the Total Cost of Ownership of School Networks: A Presentation for School Leaders. Microsoft PowerPoint presentation. Consortium for School Networking. 2007. A Report and Estimating Tool for K–12 School Districts, Value of Investment, Arizona VOI Case Study, One-to-one Student Computing. Consortium for School Networking. 2006. A Report and Estimating Tool for K–12 School Districts, One￾to one Student Computing, Total Cost of Ownership, Value of Investment, District 2 TCO/VOI Case Study. Consortium for School Networking. 2006. A Report and Estimating Tool for K–12 School Districts, One￾to one Student Computing, Total Cost of Ownership, Value of Investment, District 3 TCO/VOI Case Study. Consortium for School Networking. 2007. A Report and Estimating Tool for K–12 School Districts, Value of Investment, Iowa VOI Case Study, Formative Assessment. Consortium for School Networking. 2004. A Report and Estimating Tool for K–12 School Districts, Missouri District Case Study. Consortium for School Networking. 2004. A Report and Estimating Tool for K–12 School Districts, Texas District Case Study. Consortium for School Networking. 2004. A Report and Estimating Tool for K–12 School Districts, Virginia District Case Study. Consortium for School Networking. 2004. A Report and Estimating Tool for K–12 School Districts, Wisconsin District Case Study. Consortium for School Networking. 2007. A Report and Estimating Tool for K–12 School Districts, Value of Investment, Wisconsin VOI Case Study, Online Learning. Curricula and Textbooks Administration. 2007. E-Curricula. Microsoft PowerPoint presentation. Denning, T. 2005. Evaluation Framework, Jordan Education Initiative, Draft. Keele University. The Directorate of Curriculum and Textbooks and the Supporting Jordan’s Education Project. 2006. Classroom and Computer Lab Deployment Strategy in Government Schools Grades 1-12. Working Paper. SJE/CIDA. El-Berr, S. 2004. Non-German Donor Activities in Education in Jordan. Center for Development Research (ZEF). University of Bonn, Germany. Eyango, V. M., and AlShaer, M. 2003. Education Assessment for Jordan 2004–2009 Country Strategy. USAID/Jordan. Garter Inc. 2006. Spend Less, Get More: 25 IT Cost Containment Techniques. Gartner Inc. Gartner Inc. 2003. A Report and Estimating Tool for K-12 School Districts, Why Total Cost of Ownership (TCO) Matters. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 59 of 61 Higgins, S. et al. 2005. Embedding ICT in The Literacy and Numeracy Strategies, Final Report. Centre for Learning and Teaching, School of Education, Communication and Language Sciences, University of Newcastle upon Tyne. Holzberg, Carol S. 2005. Total Cost of Ownership (TCO). techLEARNING. Integrated Technology Group (ITG). 2004. EduWave™, the e-Learning Platform. Jordan Education Initiative. Jordan Education Initiative. 2007. 10th Site Survey, Summary Results Presentation. Microsoft PowerPoint presentation. Jordan Education Initiative. 2007. 12th Site Survey, May 18th–Jun 8th, 2007, Summary Results Presentation. Microsoft PowerPoint presentation. Jordan Education Initiative. 2006. JEI E-Curricula Master Plan. Microsoft Excel spreadsheet. Jordan Education Initiative. 2005. Contributing Organizations, Jordan Education Initiative. Jordan Education Initiative. 2005. The Jordan Education Initiative, a public-private partnership model for effective and advanced learning deployment. Trifold brochure. Jordan Education Initiative. 2004. The Jordan Education Initiative, a public-private partnership model for effective and advanced learning deployment. Brochure. Jump, A. 2007. Report Highlight for Dataquest Insight: Forecast Scenarios for Education Ultralow-Cost Mobile PCs, Worldwide. ID Number: G00151988, Gartner Inc. Kusserow, T., Lavin, J., and Movassaghi, M. 1999. Managing Technology Efficiently in California K-12 Schools: Policies & Practices for Minimizing the Total Cost of Ownership (TCO). Richard & Rhoda Goldman School of Public Policy, University of California, Berkeley. Lars M. and Kirwin B. 2005. Defining Gartner Total Cost of Ownership. Gartner Inc. McKensey & Company. 2005. Building Effective Public-Private Partnerships: Lessons Learnt from the Jordan Education Initiative, An Initiative of the World Economic Forum and Government of Indonesia. Mieritz, L. and Kirwin, B. 2005. Defining Gartner Total Cost of Ownership. Gartner Inc. Ministry of Education, 2006. National Education Strategy. Directorate of Educational Research and Development. Ministry of Education. 2004. The Development of Education National Report of the Hashemite Kingdom of Jordan. Managing Directorate of Educational Research and Development. Presented to the 47th Session of the International Conference on Education in Geneva. Ministry of Information and Communication Technology (MoICT). 2006. Request for Proposal on Supply and Installation of Network Equipment for Module 8, Volume II: Technical Specifications, Drawings, and Bill of Quantities. National Broadband Network. Osin, L. 1998. “Computers in Education in Developing Countries: Why and How?” Education in Technology Series, Vol. 3, No. 1, The World Bank, Washington, D.C. Light, D. and Gersick, A. 2003. A Study of the Efficacy of a Thin Client Technology Infrastructure. Center for Children & Technology, Education Development Center Inc. Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 60 of 61 Loftus, J. 2007. Microsoft Windows ousted at California school district. Enterprise Linux News. Robert Frances Group. 2005. The Downside of Keeping PCs beyond Three Years and How Leasing Can Help. Scrimshaw, P. 2002. Total Cost of Ownership, A review of the literature, A report to Department for Education and Skills (DfES). ICT in Schools Research and Evaluation Series—No. 6, BECTA. Somekh, B. et al. 2007. Evaluation of the Primary Schools Whiteboard Expansion Project. Report to the Department for Children, Schools and Families, Centre for ICT, Pedagogy and Learning, Education & Social Research Institute, Manchester Metropolitan University, Becta. Troni, F. and Silver M. 2006. Use Processes and Tools to Reduce TCO for PCs, 2005–2006 Update. Gartner Inc., ID Number: G00136769. UNESCO. 2002. Information and Communication Technology in Education, A Curriculum for Schools and a Programme for Teacher Development. University of Keele. 2006. Jordon JEI Project, A baseline report prepared for the British Council in Amman, Baseline Data Collection—Teachers and Principals. USAID. 2007. EFfKE Support Project (ESP) Fact Sheet. USAID. 2005. EQ Review, Educational Quality in the Developing World. EQUIP 1 Project Newsletter, Vol. 3, No. 3. Wagner, D. et al. 2005. Monitoring and Evaluation of ICT in Education Projects, A Handbook for Developing Countries. Information for Development Program, InfoDev. Web Sites CEB (formerly Computer & Engineering Bureau) www.ceb.com.jo ChangeAgent for Arab Development and Education Reform www.caderco.com Information Technology Association of Jordan www.intaj.net Integrated Technology Group www.itgsolutions.com Jordan Education Initiative www.jei.org.jo The Jordan Training Technology Group—Rubicon www.rubicon.com.jo Ministry of Information and Communication Technology, Kingdom of Jordan www.moict.gov.jo NetCorps Jordan Report: Task3, Review of the Technology Employed to Deliver E-Learning December 10, 2007 Page 61 of 61 www.globalnetcorps.org/course/view.php?id=178 The Consortium for School Networking http://www.cosn.org/ Taking TCO to the Classroom http://www.classroomtco.org/ CoSN/Gartner TCO Tool & Case Studies http://www.classroomtco.org/gartner_intro.html The 1 to 1 Institute http://www.one-to-oneinstitute.org/ Central Regional Educational Laboratory (NCREL), enGuage®: A Framework for Effective Technology Use http://www.ncrel.org/engauge/ Becta http://www.becta.org.uk One-to-One Information Resources www.k12one2one.org