Evaluation of the Jordan Education Initiative Report: Task 4 COST ASSESSMENT OF TECHNOLOGY SUPPORT FOR E-LEARNING IN JORDAN April 17, 2008 This publication was prepared by RTI International on behalf of Education Development Center, Inc., for the United States Agency for International Development. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 2 of 29 Report: Task 4 Cost Assessment of Technology Support for E-Learning Note: The Task 4 report builds on the Task 3 report, which provides a detailed review of the technology employed to deliver e-learning in Jordan. The Task 3 report is not reviewed further in this report. Submitted to: USAID/Jordan through Education Development Center, Inc. Submitted by: Frank Method, RTI International Project: Evaluation of the Jordan Education Initiative Prime Contract: Assistance to Basic Education / Basic Education, EDH-I-02-05-00031-00 Task Order: 02 The Author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Cover Photo Credits (Clockwise from top left): G.M. Cressman, G.M. Cressman, G.M. Cressman, D. Light Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 3 of 29 Acknowledgements This report would not have been possible without the support, cooperation, and organizational skills of the Program Management Office (PMO) of the Jordan Education Initiative (JEI). Projects Coordinator, Luma Atallah, and In-classroom Technology Track Manager, Niveen Jum’a, provided excellent support, managing to arrange critical meetings and school visits on short notice and providing background data from the JEI site surveys and other documents both during the visit and following. Though the visit to Amman was immediately prior to the national election and many of the key people were thus unavailable, every effort was made to arrange essential meetings. Throughout this visit and the previous visits, the PMO team, including Executive Director Haif Bannayan, was candid and transparent in discussing the known problems with implementation to date and direct in demanding an objective report with specific recommendations. Finally we acknowledge and thank the United States Agency for International Development (USAID) for providing the support for this assessment. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 4 of 29 Abbreviations ADSL ..........Asynchronous Digital Subscriber Line AEL.............Appalachian Regional Education Lab CBJ .............Central Bank of Jordan CD...............Compact Disc CoSN ..........Consortium for School Networking COW...........Computers on Wheels DSL.............Digital Subscriber Line EDC ............Education Development Center, Inc. EDSS..........Electronic Data Storage System EduWave....Education Portal Developed by Integrated Technology Group EMIS...........Education Management Information System EMIS...........Education Management Information System ERfKE.........Education Reform for a Knowledge Economy GNP............Gross National Product IAETE .........Institute for the Advancement of Emerging Technologies ICO..............Initial Cost of Ownership ICT .............Information and Communication Technology IT.................Information Technology ITG..............Integrated Technology Group IWB.............Interactive White Board JEI...............Jordan Education Initiative JOD.............Jordanian Dinar MoE.............Ministry of Education MoICT.........Ministry of Information and Communication Technology NBN ............National Broadband Network NCSA..........National Council of School Administrators NER ............Net Enrollment Ratio NGO............Nongovernmental organization PC...............Personal Computer PDA.............Personal Digital Assistant PMO............Project Management Office R*TEC.........Technology in Education Resource Center RCO............Real Cost of Operations ROI..............Return on Investment TBO.............Total Benefit of Operations TCO Total Cost of Ownership TCO ............Total Cost of Ownership TEI ..............Total Economic Impact UIS..............UNESCO Institute for Statistics UNESCO....United Nations Educational, Scientific, and Cultural Organization USAID.........United States Agency for International Development VOI..............Value of Investment WEF............World Economic Forum Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 5 of 29 Contents Acknowledgements ......................................................................................................................... 3 Abbreviations................................................................................................................................... 4 1 Cost Analysis............................................................................................................................. 6 1.1 Purpose............................................................................................................................... 6 1.2 Summary............................................................................................................................. 6 2 Cost Factors .............................................................................................................................. 8 3 Cost Scenarios ......................................................................................................................... 10 4 Cost Structure Summary ......................................................................................................... 15 5 Limiting Factors for Cost Analysis ........................................................................................... 16 6 Conclusions and Recommendations....................................................................................... 19 Annex A: Comparative Costing Models and Approaches ............................................................. 22 Annex B: Meetings ........................................................................................................................ 25 Annex C: Approach and Scope for Task Four Consultancy.......................................................... 27 Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 6 of 29 1 Cost Analysis 1.1. Purpose To assess costs of Jordan Education Initiative (JEI) activities and inputs in support of e-learning in Discovery Schools and the cost implications of scaling up the JEI technology platforms and e￾learning approaches to additional schools as part of the national Education Reform for the Knowledge Economy (ERfKE) programs. The scope for the Task Four Assessment is: a) to assess the investments to date of the JEI initiatives; b) to project the Total Costs of Operations (TCO) of an expanded use of the JEI e￾content and required ICT support in ERfKE schools over a relevant period (such as 5 years), including investment costs and all recurrent costs to the extent they can be identified (training, maintenance, technical support, virus protection and other software additions, software upgrades, and ongoing refinement of e-content); and c) on the basis of these projections, make recommendations on feasible scenarios for scaling up the information and communication technology (ICT) provision and e-learning approach. See Annex C for details on scope and approach. 1.2. Summary Two costing scenarios were developed, Scenario One approximating the incremental costs to the Ministry of Education’s ERfKE schools and Scenario Two approximating the costs of scaling up the inputs provided under JEI for the Discovery Schools. Costs are estimated on the basis of Initial Cost of Ownership (ICO) including infrastructure, hardware, and software, and on the basis of TCO, including the ICO estimates plus recurrent costs for software, maintenance and parts, laboratory technicians, and interns or similar technical support personnel and operations. Costs are then estimated for average costs per school and for increments of 100 schools, for each of the next 5 years and for TCO over 5 years, including an inflation factor of 6 percent. A 5-year period to estimate TCO approximates the useful life of the computers and other hardware investments. Though some elements, such as computer lab rooms, will have a longer useful life and careful maintenance may extend the life of some of the hardware, the costs are annualized to facilitate longer term planning for replacement costs. Costs of software and hardware are judged unlikely to reduce significantly in 5 years. Excel worksheets are provided separately to facilitate re-estimation as decisions are made on the options for provision and as more precise unit cost data are obtained. For Scenario One, incremental costs to ERfKE, the annualized costs (recurrent costs plus hardware and infrastructure amortized on the basis of estimated useful life) are $27,743 per school, $3,468 per teacher, and $28 per student. For Scenario Two, the annualized costs of scaling up the inputs provided under JEI (excluding any additional computer laboratories and associated personal computers [PCs]) are $17,724 per school, $2,215 per teacher, and $19 per student, per year.1 The overall assessment is that the total costs will be large relative to Jordan’s economy and education budget. Scenario One requires an initial investment of $7,495,000 per 100 schools and has recurrent costs of $1,374,000 per year. Scenario Two (assuming no additional laboratories) 1 UNESCO estimates teacher-to-student ratios at 1:21, but class size typically is 25–40 students. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 7 of 29 requires an initial investment of $4,450,000 per 100 schools and has recurrent costs of $910,000 per year. Jordan currently provides a higher proportion of its public budget for education than most countries. Data from the United Nations Educational, Scientific, and Cultural Organization (UNESCO) show that public expenditure on education as a percentage of gross national product (GNP) is about 4.9 percent. This is about 20.6 percent of public expenditure, with about two-thirds (13 percent) of that amount allocated to elementary and secondary education.2 Jordan also allocates about 4.8 percent of primary expenditure for teaching materials—which gives Jordan the relatively high rank of 9th out of 30 countries in a recent UNESCO Institute for Statistics (UIS) survey.3 Given its current high level of education expenditure, it is unlikely at least in the short term that Jordan will be able to finance from its own resources the requirements for scaling up the e￾learning ICT requirements for all schools. Such financing will have to be accomplished in planned stages over a number of years and will depend critically on the availability of external funding and partnerships. A period of at least 5 years appears required for full-scale implementation. Assuming substantial continued external funding and partnerships providing in-kind inputs, it is likely that the initial investment costs will be met more easily than the recurrent budget requirements of ongoing technical support, maintenance, software licenses, training, laboratory technicians, and other personnel costs. These costs, including replacing equipment and updating software over time, must be anticipated and budgeted annually. Some amount of ongoing content enrichment and curriculum refinement also must be assumed, optimally up to 10 percent annually of the original e-learning development costs, including related assessment, research, and performance monitoring. Such costs will need to be included in regular Ministry budgets. Project funding by external funders and partners is likely to be more responsive to the initial investment requirements than to the recurrent cost requirements over time, as has been the case during the development phases of the JEI. Though expensive in terms of total cost, the ICT support for e-learning under the JEI is judged to be relatively cost efficient compared with the experience in other countries, and with respect to identified options for cost savings, discretionary choices on hardware and software, and the current level of technical support and maintenance under the approaches supported by JEI. See the discussion in Annex A: Comparative Models and Approaches. The main options for reducing investment costs are to manage with fewer additional computer laboratories, to use fewer Interactive White Boards (IWBs), and to target the initial phases of scale up on schools with at least two existing computer laboratories with well maintained PCs. Alternatively, it can be assumed that the costs of computer laboratories and associated PCs are already included in the plans of the ICT Directorate for all ERfKE schools, and/or that the initial stages of scale up of the JEI approach will concentrate on schools with at least two computer laboratories. It also is possible for some of the e-learning content, e.g., e-Arabic, to be implemented with little reliance on computer labs. Other subjects, particularly e-mathematics, would require adequate computer laboratory access. Excluding the laboratory costs more accurately reflects the incremental costs of the JEI e-learning approach but understates the full costs of the support required under ERfKE. Some additional investments are needed beyond those provided for the Discovery Schools, particularly for antivirus software, more adequate help desk support, development of multimedia resource rooms, computer access for students outside the classroom, and content support in 2 UNESCO Institute for Statistics (UIS) http://stats.uis.unesco.org/unesco/TableViewer/document.aspx?ReportId=121&IF_Language=eng&BR_Country=4000. 3 http://www.nationmaster.com/graph/edu_spe_on_tea_mat-education-spending-on-teaching-materials. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 8 of 29 addition to that of the existing e-learning curriculum. Because these costs depend on policy and strategy decisions that have not yet been made, the additional options are not fully reflected in the scenarios mentioned above, except for antivirus software. The full value of the ICT investments in support of the e-learning curriculum will be enhanced through making computers more accessible to students outside classroom hours and outside the schools, and through more emphasis on distributed learning and problem-solving projects involving students and teachers in clusters or other pairings of schools. Instructional effectiveness, and therefore value of the investments, also can be improved through lab/student management software enabling the teacher to individualize student activity and to monitor individual activity more effectively. Among the options for improving cost efficiency are to increase lab and computer usage and adopt strategies for reducing the amount of lost time due to inoperable computers (PCs as well as laptops), projectors, and other hardware. Computer usage could be improved through greater use of computers in labs and/or in multimedia rooms outside of classroom hours, through scheduling changes that allow the labs to be used for longer periods (thus reducing setup time and time required for students and teachers to move in and out of the labs), and through improving maintenance by increased monitoring of lab computer maintenance and perhaps by maintaining decentralized stocks of replacement parts. Such options present administrative difficulties, but do not increase costs significantly. Also, lost time due to connectivity problems could be reduced at little additional cost by increased use of CDs or other storage strategies to overcome bandwidth and connectivity problems. Whether the approach is cost effective in addition to being cost efficient requires a judgment on the instructional effectiveness and learning gains under the JEI approach, as well as on other judgments of program effectiveness by the Ministry, such as increases in ICT skills, development of the ICT sector in Jordan, and use of ICT outside the schools. The learning gains to date appear modest and are difficult to attribute solely to the ICT and e-learning approach. However, it is too early to make a determination because the transformational processes are still under way and the e-learning for two subjects was fully rolled out only in the last year. (See the discussion under Task One Report.) However, there is some evidence from teacher feedback that the e￾content is helpful, that students are engaged and, after initial adjustment and training in how to prepare lessons using the technology, that teachers generally welcome the approach and find it helpful in lesson preparation. 2 Cost Factors The major cost drivers are as follows. 1. Computer laboratories and associated PCs, software, and laboratory technicians are the most expensive component. It is not known at this time how many additional laboratories will be required. The number will depend on which schools are selected for the initial stages of scale up. For the 100 Discovery Schools, which were selected in part for adequate laboratories, 30 new labs were added.4 Available labs were used for about 44 percent of the available hours, with a realistic maximum being on the order of 60–70 percent 4 Site survey number 11, slide 23 Cost drivers: ƒ Computer laboratories ƒ Laboratory technicians ƒ Datashow projectors ƒ Productivity and Antivirus software ƒ Maintenance ƒ Interactive White Boards ƒ Laptops ƒ PCs for laboratories ƒ Support personnel Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 9 of 29 because of scheduling difficulties. The Scenario One estimate assumes, somewhat arbitrarily, that half of the additional schools selected for the upcoming stage will require an additional computer laboratory. Most additional laboratories are expected to be reconfigured classrooms; thus, it is assumed an additional classroom will be required. Each new computer laboratory is estimated at $42,000 for a 50 m2 laboratory, at $850 per m2 including the costs of a replacement classroom. This is the minimum size for a laboratory that can accommodate 21 workstations and 25–40 students at a time. A more adequate laboratory size, allowing for better configuration of work stations, of about 60 m2 would cost proportionately more. Assuming a 20-year useful life for the laboratories spreads the costs to about $2,000 per year. N.B. It is difficult to judge how many additional laboratories may be needed. The current e￾learning approach is designed to include substantial laboratory time. However, laboratory use varies greatly by subject (with e-mathematics making most use of the labs and other subjects, such as e-Arabic, using the labs least). Further, laboratory use falls off sharply in the upper grades, particularly as examinations approach. Some laboratory capacity is needed aside from the e-learning content objectives. For example, the laboratories are used to introduce basic computer skills, particularly in the primary grades. To date there has been little experimentation with an approach relying mainly on classroom use of the e-learning content or close monitoring and analysis of the differences in learning outcomes between schools and classes making greater or lesser use of the laboratories. This is a researchable factor with major implications on the costs of needed ICT investment and on the feasibility of rapid scale up of the e-learning approaches. 2. Laboratory technician salaries are JD 271 per month or $4,592 per year. One technician is needed for a maximum of three laboratories, preferably one for every two laboratories. As the e-learning approach scales up, it will be increasingly difficult to attract, train and, retain sufficient numbers of well-qualified and motivated technicians at this salary rate and this cost may be expected to rise. Laboratory technician training costs are included under the operations estimate. 3. Datashow projectors are estimated at $4,000 each, with a useful life of 6 years. Depending on specifications, these projectors range from $3,000 to $8,000, and more. A minimum of eight projectors will be required per school, plus one for a multimedia room where such rooms are provided. 4. Antivirus software will be about $1,800 per school and productivity software (MS Office 2007) will be $52 per computer. Both estimates are for site licenses that must be renewed annually. The Ministry may be able to negotiate slightly better rates on a volume basis or on a concessional basis, but these rates are fairly standard elsewhere, including in the United States. 5. Maintenance will be at least $16 per computer per year and parts replacement will be about $450 annually for each laptop and datashow projector combined. The parts that fail or are damaged most frequently are batteries, chargers, projector lamps, and laptop monitors. 6. Interactive White Boards (IWBs) are estimated at $3,200 each. Depending on vendor and specifications, IWBs can range from $1,800 to $5,000. On an annualized basis, with an estimated useful life of 4 years, they are about $800 per year. Whiteboards require a ceiling-mounted projector and a PC or laptop, i.e., they are an addition to, not a substitute for, laptops provided to teachers. It is not known at this time how many IWBs actually will Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 10 of 29 be provided; the scenario estimates are on based on 50 IWBs being provided, covering half the 100 schools. 7. Laptops will be needed for each e-learning teacher, with eight teachers per school. At midrange specifications, these laptops will cost about $1,200 each. Maintenance and parts will be additional. A useful life of 4 years is assumed; with good maintenance, this might be extended to 4.5 or 5 years. 8. PCs are estimated at $900 each for midrange specifications, not including software. Maintenance and spare parts are likely to be less than for laptops. A useful life of 6 years is assumed; with good maintenance, this might be extended at most an additional year. N.B. PCs are included in the Scenario One estimate, which includes additional computer laboratories, but are excluded from the Scenario Two estimate. 9. Other costs for operations and for interns or equivalent personnel providing support for data collection and site surveys are difficult to attribute on a per school basis, but are essential to the effective implantation of the overall approach. These costs are quite low, about 2.2 percent of the total costs for 100 schools. Because a sampling approach can be used for site surveys and monitoring, these costs should decline as a percentage as the e-learning approach scales up. However, expenditures to support these functions are essential to ensure the efficient use of the other investments. 10. The costs of laptop and PC computers under Scenario One are about 14 percent of the total investment on an annualized basis. PCs are about 5.7 percent and laptops are about 8.4 percent of the total investment on an annualized basis. Recurrent costs, including software and maintenance/parts, lab technicians, and operations support, are about 47 percent of the total costs on an annualized basis, datashow projectors are the most expensive hardware input, representing about 19 percent of the annualized costs. Under Scenario Two, excluding laboratory and PC costs, laptops are 13.5 percent of the annualized costs, projectors are 30.1 percent, and recurrent costs are 51.4 percent. Thus, although it may be possible to procure PCs and laptops at slightly lower unit costs, total annualized costs would reduce only marginally, at most 1–2 percent, and any savings may be offset by lower reliability and higher recurrent maintenance costs. The main cost driver will be the degree to which the e-learning approach depends on the availability of laboratory time. 3 Cost Scenarios ICO and TCO were estimated on two scenarios: Scenario One assumes additional computer laboratories will be required in the expansion schools, while Scenario Two assumes that no additional computer laboratory infrastructure will be needed and/or that the first priority for expansion will be to schools with at least two existing computer laboratories with functioning PCs. Cost estimation scenarios are based on increments of 100 schools. This is on a sufficient scale to allow bulk purchasing, system licenses for software, and scaled arrangements for maintenance and technical support. It is assumed that Jordan may decide to scale up to a larger number of schools, perhaps 500 in the next 2 or 3 years and eventually to all 3,000 plus schools. The full costing is a matter of policy judgments on the rate of expansion and scale. Depending on these judgments, the cost estimates can be made as multiples of 100 school increments using the accompanying Excel spreadsheets. N.B. It is not expected that there will be significant reductions in unit costs for faster scale up to larger numbers of schools. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 11 of 29 For the first scenario, the full cost of implementing the JEI e-learning approach in additional schools would require adding a computer lab, additional PCs, and IWBs in about half the schools. This scenario approximates the actual cost of implementation as part of the ERfKE schools initiative. On this basis, the initial investment costs (ICO) are about $7,495,000 per 100 schools, with a recurrent cost in year one of about $1,360,000. Annualizing the costs (amortizing the hardware and infrastructure on the basis of the expected useful life, plus the recurrent costs) results in first year estimates of $27,607 per school, $3,451 per teacher, and $28 per student. Subsequent years are estimated with a 6 percent inflation adjustment on recurrent costs.5 These unit costs, though substantial relative to Jordan’s financing capacity and other education requirements, compare favorably with the unit costs of ICT provision for education in other countries, including the United States. For example, estimates of costs of computers per student made in 2000 for Turkey, Chile, and Egypt reported per student costs of $32, $56, and $75 respectively.6 N.B. Full comparable comparisons on a per school basis are not possible given the many differences in level and type of hardware provided and the different levels of support. See the discussion in Annex One: Comparative Models and Approaches. For the second scenario, approximating the current costs of technology inputs and support under JEI, the assumption is that the costs of additional computer laboratories would be provided under Ministry programs for ERfKE and/or that the next stages of scaling up the e-learning approach would focus on those schools with two or more existing computer laboratories. It also is possible that some of the smaller schools could manage with only one laboratory, though with some scheduling difficulties. This scenario is not actually a cost savings, as the necessary infrastructure will eventually be required in all ERfKE schools, but a focus on the schools with two or more laboratories is the likely scenario for the next stage of scale up. Excluding the need for additional laboratories, PCs for the laboratories and the associated software and maintenance, the initial investment cost (ICO) is about $4,661,000 per 100 schools, with a recurrent cost in year one of about $1,132,000. Annualizing the costs (amortizing the hardware and infrastructure on the basis of the expected useful life, plus the recurrent costs) results in first year estimates of $17,840 per school, $2,239 per teacher, and $18 per student, per year. Optional scenarios include providing different numbers of IWBs, providing one additional laptop per school for use as a spare, the possibility that excellent maintenance could extend the useful life of computers and other infrastructure, and using computers on wheels (COW carts) as an alternative to providing additional computer laboratories. ƒ In both scenarios, IWBs add about $160,000 per 100 schools, on the assumption that half the schools would receive an IWB each for use in multimedia classrooms. There is anecdotal evidence that teachers find the IWBs useful and helpful in engaging students, but there is not yet any evidence (pending the Part One assessment) of their cost￾effectiveness in comparison with providing teacher laptops, datashow projectors, or additional laboratory time. Thus, providing IWBs remains a discretionary option. The 5 Central Bank of Jordan (CBJ) estimate for 2006 was 6.25% 6 Marianne Bakia. “The Costs of Computers in Classrooms: Data from Developing Countries,” 2000 Annualizing costs for the first scenario (with additional labs) results in first year estimates of $27,607 per school, $3,451 per teacher, and $28 per student. Annualizing costs for the second scenario (JEI inputs only) results in first year estimates of $17,840 per school, $2,239 per teacher and $18 per student. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 12 of 29 number of IWBs provided could be reduced or increased depending on funding availability and further experience with the use of IWBs for interactive teaching. ƒ The provision of one additional laptop per school would add about $121,000 for 100 schools, including software licenses. If used as spares, the additional laptops would not require additional maintenance. Assuming a teacher cost of $3,500 per year, reducing total lost time per school due to inoperable laptops by as little as 2 weeks a year would be cost effective. Actual downtime under JEI for the Discovery Schools was under 3 percent, which is remarkably low.7 This operability rate is unlikely to be maintained in a scaled up program with less intense monitoring and support. Moreover, of those machines that had operability problems, the downtime was more than 6 months in 60 percent of the cases.8 ƒ Excellent maintenance may extend the useful life of computers, IWBs, and other hardware and infrastructure. The costs for the scenarios above are on the basis of an average of 4 years of useful life for laptops and IWBs and 6 years for PCs and projectors. These are conservative estimates. If, for example, laptops could be maintained in useful condition for an average of 5 years each rather than the 4 years used in the scenarios, the annualized savings would be about $60 per laptop or $480 per school on an annualized basis ($1,920 per year rather than $2,400). The key to improved maintenance, whether provided by Ministry technicians or by service contracts and warranties, appears to be in providing close monitoring. It is likely the Ministry will have to consider decentralizing some of the maintenance functions, possibly using local ICT firms or strengthening capacities at the district level. There are many possible combinations of warranties and service contracts, school system personnel, and use of local firms for maintenance and technical support. Most large systems with multiple districts decentralize these functions to some extent and/or rely on vendors and contractors to provide support. As an example, in a California district of 140,000 students, “As machines are purchased, the district works with the vendors to provide a CD with images of standard applications, including legacy systems, specifically configured for them. Computers are purchased with a three-year warranty. District technicians provide warranty repair work and charge back to the participating vendors. [The District IT Department] supports the standard client devices.”9 ƒ A major limiting factor on the feasible rate of expansion will be the rate at which lab technicians can be trained and arrangements made for technical support and maintenance for the expansion schools. One alternative is for the Ministry to develop service contracts with local information technology (IT) firms and nongovernmental organizations (NGOs) for maintenance and technical support. There are unlikely to be significant cost savings in such arrangements. However, if such arrangements result in more responsive maintenance and technical support, there may be improvements in cost effectiveness. Such arrangements should be accompanied by systematic monitoring, assessment, and reporting, with explicit roles for the school teams in the assessments. Another option, possibly in combination with such service contracts, is to use students for some of the maintenance and technical support; this would be feasible particularly at the 7 Site survey number 11, slide 26 8 Site Survey number 11, slide 40 9 One of a set of case studies by the Gartner Group for the North Central Regional Education laboratory, included in http://www.classroomtco.org/2003_case_studies/california.pdf, included in Taking TCO to the Classroom, http://www.classroomtco.org/gartner_intro.html#case Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 13 of 29 The availability of trained lab technicians may be a constraint for expansion, limiting the rate of ICT expansion for lab-dependent schools to no more than 100-200 schools a year for the next 2 to 3 years. secondary level and possibly earlier and would contribute to the objective of students’ increasing ICT skills. The availability of trained and motivated lab technicians may be a more limiting constraint. Scaling up to 3,000 or more schools, and assuming at least 2 laboratories per school, will require 3,000 to 5,000 trained lab technicians. Assuming some attrition, this will require training upwards of 500 new lab technicians a year on a continuing basis. The variables affecting the feasible rate of recruitment, training, and retention of such technicians are not known in sufficient detail to suggest a maximum rate of expansion. The cost estimate for technician salaries is somewhat higher than current beginning salaries, allowing for some incentives to improve recruitment and retention. JEI has developed a training program of about 120 hours, run by the JEI technology track coordinator. It may be possible to develop an ICT-supported training program and to use well-selected and experienced lab technicians to assist with training. On this basis it may be possible to have three or four training programs a year initially, producing 80–100 new technicians and to increase the number of programs in subsequent years as more experienced trainers are available. This suggests that the rate of ICT expansion for lab-dependent schools will have to be set at not more than 100–200 schools a year for the next 2 or 3 years. Possibly a faster rate could be achieved by developing training programs in partnership with some of the community colleges or universities. In any case, it is strongly recommended that expansion plans not exceed the rate at which lab technicians can be trained and reliable arrangements made for maintenance and technical support. Any rate that exceeds these support capacities is likely to be very costly in terms of inadequate maintenance and difficulties in using the available technology effectively in the schools. ƒ COW carts could be used to offset the cost of an additional laboratory, at least in some schools where time or space does not permit construction of an additional classroom to replace the one to be repurposed for use as a computer laboratory. However, COW carts are only marginally more cost effective than laboratories, particularly on an amortized basis, and have some disadvantages in terms of use in Jordanian classrooms. Assuming similar specifications as other laptops provided to teachers, and the same numbers of laptops (20) provided for student use in the laboratories, the initial costs to provide 20 COW carts to each school would be about $24,000 per school.10 Wireless access points are already included in the estimates. The initial cost of $24,000 for 20 laptop COW carts compares with an estimated $42,000 for a new laboratory and $18,000 for laboratory PCs. However, annualizing the laboratory infrastructure on the basis of 20 years of useful life and the PCs on the basis of 6 years of useful life also results in an estimate over 5 years of about $24,000. Recurrent costs also appear similar. Maintenance and parts costs for the laptops are likely to be somewhat higher than for the PCs, but the laptops will use substantially less electricity. Among the factors affecting the use of COW carts with the e-learning approach developed under JEI are the following: many schools are multistory, without elevators, limiting the movement of the carts; setup of the laptops in the classroom takes some time away from the relatively short (40– 45 minute) classes; unless the classroom is wired as a lab, the laptops will be used mainly on battery power, necessitating frequent recharging, limiting use to not more than 2 hours (or two consecutive classes), and resulting in more frequent battery replacement; and a lab technician may not be available in the classroom to assist with setup and to load content prior to the lesson. Conclusion: although COW carts may be an option in selected schools, the option of configuring 10 Most COW cart vendors recommend a specialized laptop that is relatively rugged and that can withstand frequent recharging. COW carts also typically include a wireless printer Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 14 of 29 and equipping a computer laboratory appears cost efficient with approximately the same cost over 5 years. ƒ Whatever are the Ministry choices on the types and numbers of hardware to be provided, it is important to standardize the selection with a limited number of vendors and models. Laptops provided for the Discovery Schools included Dell, HP, and two ACER models, and projectors included 5 different models—2 models by Beng and 3 models by HP. Jordan may wish to work with more than one vendor, and there may be more than one partner providing hardware and infrastructure support, but it should be possible to have a single vendor and a single standard for hardware and software for all schools in each cluster and/or district. Standardization of software provision as well as hardware has significant advantages, both in terms of initial volume purchasing and in terms of technical support, maintenance, and training. In the California case study previously cited, where purchasing is done at the school level and technical support and training is provided at the district level, schools purchased or acquired through donations a variety of hardware and chose different software packages. The inefficiencies that resulted raised costs at the district or systems level even when donations or other ad hoc acquisition may have reduced initial costs at the school level. “This diversity creates inefficiency from a technical and an academic perspective. The district loses leverage in technical training, has less of an ability to take advantage of volume purchasing discounts, and leaves itself vulnerable to technical problems related to interactions among configurations (for example, one product causing the other to fail or work unreliably). The current situation limits the ability of the teaching staff to share best practices in technology integration or learn from each other’s mistakes. The operational inefficiency, combined with limited staffing, hinders the district from taking full advantage of its technology investments.”11 ƒ Actual unit costs of the ICT provision under the development phase of JEI were difficult to estimate, given the substantial in-kind support by partners. The estimates used for the projected scenarios are considered order-of-magnitude correct, but the actual costs will depend on negotiated procurements, choices of vendors, arrangements for maintenance and technical support, decisions on the number of IWBs, and the number of new computer labs. Unit cost estimates for laptops and desktops were obtained from JEI for midrange computers. Cost estimates for constructing new labs, reconfiguring existing classrooms, and installing wireless access points were obtained from comparisons with other projects, including support from the United States Agency for International Development (USAID) for school buildings under ERfKE. Software estimates were obtained through review of vendor quotes for large school systems in the United States. Costs of IWBs, datashow projectors, and COW carts were obtained from reviews of online vendor listings. Salaries for computer lab technicians and Netcorps interns were obtained from JEI. The estimate for maintenance was obtained from the ICT Department of the Ministry of Education (MoE). Bulk purchasing and system licenses are assumed throughout. Although it is expected that Jordan will continue to receive support from external donors and technology partners, it is impossible to predict at this stage what in-kind support may be provided; for this reason, all costs are estimated as actual costs, whether financed by Jordan or provided by partners. 11 http://www.classroomtco.org/2003_case_studies/california.pdf p.8 Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 15 of 29 4 Cost Structure Summary We used the following data to calculate the projected cost of both scenarios. Exchange rate: 1 Jordanian Dinar (JOD) = $1.412 (as of November 2007) Inflation rate: 6 percent (Central Bank of Jordan [CBJ] estimate was 6.25 percent for 2006) Projected Cost, Scenario One Cost Assumptions Cost in $ Cost in JOD Initial investment per 100 schools 1 lab for half the schools, 21 PCs, 8 laptops, 8 projectors, wireless access points for 8 classrooms, antivirus, and productivity software each computer $7,495,000 JOD 5,308,074 Recurrent cost per 100 schools Lab technicians, maintenance, software licenses, parts, and operations (plus 6% inflation) $1,730,000 JOD 972,800 Annual cost per school Amortization of investments (lab, PCs, and laptops) plus recurrent cost per school $27,743 JOD 19,648 Annual cost per teacher 8 e-learning teachers per school $3,468 JOD 2,456 Annual cost per student Assume 25 students per class, 5 classes per week $28 JOD 20 Projected Cost, Scenario Two Cost Assumptions Cost in $ Cost in JD Initial investment per 100 schools 1 lab for half the schools, 21 PCs, 8 laptops, 8 projectors, wireless access points for 8 classrooms, antivirus, and productivity software each computer $4,450,000 JOD 3,151,558 Recurrent cost per 100 schools Lab technicians; maintenance; software licenses; parts; operations (plus 6% inflation) $910,394 JOD 644,755 Annual cost per school Amortization of investments (lab, PCs, and laptops) plus recurrent cost per school $17,724 JOD 12,552 Annual cost per teacher 8 e-learning teachers per school $2,215 JOD 1,569 Annual cost per student Assume 25 students per class, 5 classes per week $18 JOD 18 Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 16 of 29 5 Limiting Factors for Cost Analysis The following are limiting factors for the Task Four cost analysis. E-learning requirements are additional to existing and planned ICT investments. Independent of the investments required for the support e-learning technology under the JEI, Jordan is making (and already has made) major investments in ICT infrastructure under its ERfKE programs and under related programs such as the National Broadband Network (NBN). These include DSL and ADSL connections for most schools and plans to roll out broadband connections in stages for most schools (including 84 of the JEI Discovery Schools to date); computer labs (1 or 2 in most schools, 3 or more in most of the Discovery Schools); computers and associated furniture, software, and ICT instructors for the labs; and the EduWave portal developed locally by ITG and to which the Ministry made a commitment prior to the start of the JEI. These investments serve a number of education objectives additional to support of e￾learning, including Education Management Information System (EMIS) and Education Decision Support Systems (EDSS) data management functions of the Ministry, ICT training for students and teachers, Internet access, and communications. The above investments are additional to the costs of e-learning development and ICT support under JEI. In any case, there is no obvious or methodologically sound way to attribute or apportion the costs of these underlying investments to the individual Discovery Schools or the specific activities supported under the JEI. They obviously are substantial costs, and much of the JEI e-learning activity would not be feasible without this underlying infrastructure, but the costs of the underlying infrastructure is not included in the analysis for this report. There is no single model for the Discovery Schools and ERfKE schools. Schools in Jordan serve a variety of grade levels: grades 1–4, 1–7, 1–8, 1–12, 3–7, 4–7, 4–8, 4–9, 4– 10, 4–12, 5–12, 6–9, 6–12, 7–10, 7–12, 8–12, 9–12, 10–12, and 11–12. The 100 Discovery Schools were selected based mainly on ICT criteria (proximity to the Queen Rania Data Center in Amman, existence of two or more labs, and the expectation that these schools would be among the first set of schools to be connected by broadband) and, to some extent, the interest of the school principals in participating. However, the Discovery Schools were not otherwise chosen on the basis of any education control criteria for optimizing the experimental validity of the e-learning and associated ICT inputs under JEI. This selection criterion has the merit of creating a large number of alternative contexts in which to develop and apply the e-learning approaches, but it makes it impossible to discuss the approaches as a single school model. For this reason, the approach in this report will be to focus on the costs per class and per teacher. Development costs and use of ICT differ by e-learning content area. The e-learning content was developed with different timelines for initial content development, piloting, and roll out, and was started at different grade levels and in different schools. It is beyond the scope of this report to attempt to distinguish the differences in cost across each of the content areas, except to note the differences in external partner support for content development. For the most part, the costs of ICT provision are the same, but there are differences in the use of the technology (e.g., e-mathematics making somewhat greater use of the labs and e-Arabic using mainly the laptops in the classroom). The 100 Discovery Schools were selected following criteria that had the merit of creating a large number of alternative contexts in which to develop and apply the e￾learning approaches, but this diversity also makes it impossible to discuss a single JEI school model. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 17 of 29 Projections of future costs depend on a number of factors for which decisions have not yet been made. In addition to decisions on the technical specifications of new hardware and software, the Ministry is understood to be considering an education cluster strategy that includes creating cluster resource centers. Such centers would provide pedagogic and professional development support. They would require the ability to cache and retrieve content at the cluster level, and perhaps at the school level. It also is possible that some of the technical support and maintenance for school ICT would be provided at the cluster level. Additional decisions likely to affect the use of ICT and e-learning may include curriculum revisions and integration of content (e.g., integrated science), thereby permitting more flexible scheduling of laboratory use and some use of ICT for e-training and professional support for teachers. Decisions on the above are not expected to significantly change the ICT requirements at the schools, or their costs, but will affect the use of the technology and thus the value and return on investment (ROI). Some cost factors will differ in the next phases. The JEI contributions to date have been mainly for experimental, developmental, and demonstration activities, with full scale up only occurring within the last year. The costs of the initial experimentation, e-content development, and piloting are one-time costs that provide only a partial basis for estimating future costs. Future costs may not include all of the technology and will require a lower level of ongoing investment in curriculum and materials development. There is no objective basis for estimating the ongoing costs of curriculum development and iterative improvement of the e-learning materials. However, on the basis of a complete review and revision of materials at least once in 10 years, an ongoing effort of at least 10 percent of the initial investment costs of the e-learning materials development is recommended. The Discovery Schools were selected in part because they had at least two computer labs. Many of the other schools had only one laboratory, and some had none. A minimum of two computer laboratories per school appears to be needed, meaning that future costs will include either constructing additional laboratories or repurposing classrooms (adding wireless access points and switches). In most cases, given crowded conditions in existing classrooms and a growing enrollment, repurposing of a classroom will require building an additional classroom. Selected and motivated teachers and some of the better principals of the Discovery Schools were involved in developing e-content and piloting the approaches. The Discovery Schools involved selected and motivated teachers and some of the better principals of schools. As the approach is rolled out to additional schools, additional training, supervision, and monitoring of the implementation process will be required, at least at the levels of the scale up activities in the Discovery Schools and possibly greater. There is at present no basis for estimating the additional level of effort; it is expected that, at least initially, teacher training and support will be on the same basis as for the scale up (post-pilot) phases of the JEI. Given that there now is a cadre of teachers experienced with the e-content, if it proves possible to use a peer-to-peer approach for the additional schools, the training costs should be comparable, and possibly slightly lower, for the additional schools. The activities in the Discovery Schools benefited from the work of Netcorps interns and a significant number of seconded personnel from the Ministry’s Directorates of ICT, Curriculum, and Training, as well as the coordinators of the JEI program management office, plus seconded personnel and external technical assistance from the JEI partners. Key activities included conducting a series of site surveys and related data collection and monitoring; training laboratory As JEI is scaled up, the existing cadre of trained teachers will make it possible to use a peer-to￾peer approach for the additional schools, possibly making training costs slightly lower. Maintenance and other support personnel will be crucial for the successful scalability of JEI. This will require additional trained personnel and some decentralization of functions. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 18 of 29 technicians; maintaining equipment; providing technical and troubleshooting support; and conducting related workshops, consultations, and training for the school teams. It is not clear how these functions will be accomplished at a larger scale, but it is clear that the same level of support cannot be maintained by central Ministry personnel, interns, and project partners. Providing such a level of support will require additional trained personnel and is likely to require some decentralization of functions to the district or cluster levels and possibly some contracting with Jordanian IT firms. The costs of operations support by JEI or the equivalent functions by Ministry units or other entities are likely to continue at about the same level. The costs of the JEI include operations support for training, site surveys, in-classroom support for the teachers and lab technicians, collection of daily data on the connectivity status, troubleshooting in schools, and other responsibilities. These costs will continue, either through a continued role for JEI or through other arrangements by the Ministry. Other costs of the JEI PMO, such as planning and liaising with the JEI partners, reporting to the World Economic Forum (WEF), and a variety of executive and coordinating functions, also will need to continue but will not increase proportionately to the larger scale of e-learning activities. As the approach is scaled up, these executive and coordinating functions will need to become more integrated with ongoing Ministry functions. For cost assessment purposes, the costs of these functions are assumed to be relatively the same [in absolute terms but significantly less on a per-school basis. N.B. No assumptions are made as to the continuing role of JEI or the form it may take in the next phases. The costs to date for the development phases of the JEI activities provide only a partial guide for the costs in the next phases. Total investment in the development phase was about $25 million, including in-kind contributions, of which about $19 million was provided by Jordanian and international partners and $6 million from Ministry resources.12 Other estimates are $18 million (Ministry of Information and Communication Technology [MoICT])13 and $22 million (McKinsey).14 The McKinsey Group’s review of the partnership commitments for the first 3 years (as of the end of 2004, 20 months into the project) estimated the contributions were 73 percent financial, 14 percent goods/equipment, and 13 percent human resources. According to the McKinsey report,15 $18 million of the total $22 million was spent on e-learning activities, including curriculum content development ($9.5 million), in-classroom technology ($8.2 million), and teacher training ($0.3 million). Although the JEI estimate of early 2007 is slightly higher, the proportions do not appear to have changed significantly from those assessed by McKinsey. Global private sector partner contributions were 83 percent of total support for JEI, 50 percent of the total financing ($11 million). Donor funding was 32 percent of the total ($7 million), Jordanian government investment was 11 percent of the total ($2.5 million), and Jordanian private sector investment was 7 percent of the total ($1.3 million, mainly FastLink for e-science). N.B. Jordanian government contributions appear underestimated, given the contribution of personnel and existing infrastructure in the schools and nationally, particularly for the EduWave portal and the NBN. To date, about 43 percent of the total has been spent on e-curriculum development and piloting. On the basis of an estimated $9.5 million for e-content development over 3 years, a 10 percent 12 2006 estimate, World Bank Mid-Term Report, http://www.jei.org.jo/LessonsLearned/JEI_world%20bank%20med_%20210106.ppt#261,5 , Slide 5 13 MOICT Web site lists $15 million from partners and $3 million from Jordan Government sources 14 See McKinsey, “Building Effective Public-Private Partnerships; Lessons from the Jordan Education Initiative,” pp. 24-26. Available at: http://www.jei.org.jo/KnowledgeCenterfiles/McKinsey%20Final%20Report_May%2005.pdf 15 Ibid., 15. See figure 13, p. 26. Contributions for JEI calculated as 73% financial, 14% goods and equipment, and 13% human resources. Global private sector partner contributions were 50% of the total financing. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 19 of 29 estimate for ongoing e-curriculum content development and refinement would be $300 thousand a year. For future deployment and scale up; teacher training; technology; technology support and maintenance; and assessment/monitoring/surveys are likely to require more support than e￾curriculum development/revision. Recognizing the continuing need for policy, strategy, curriculum, and technology choices such as those noted above, and the degree to which some of these will be specific to the country, Jordan may wish to develop its own costing model or toolkit to facilitate ICT planning and budgeting for use of ICT in education. See Annex A: Comparative Models and Approaches for partial models. Such a TCO model and planning toolkit would be particularly useful and needed if some of the strategic planning, budgeting, and purchasing decisions are decentralized to the district or school level. Such toolkits are less useful or needed where most decisions are made centrally and where the curriculum and pedagogic approaches are relatively standard with little opportunity for alternative approaches by schools. Key steps and decisions in the development of a Jordanian costing model or toolkit are: 1. Developing assessment processes to specify learning objectives—nationally and by school teams; 2. Developing assessment processes on existing school capacities—infrastructure, connectivity, staffing, management, technical support/maintenance, and pedagogic support systems; 3. Aligning step 2 with step 1 using scenario-based planning approaches; 4. Making decisions on the diversity of school models to be supported and whether to cover all schools, prioritized by level. Generally, the more local adaptation and innovation is encouraged, the more useful will be the costing models and planning toolkit; 5. Making policy decisions on centralized versus local purchasing, technical support, and maintenance that require initial policy decisions on decentralization, clustering, and school governance; 6. Budgeting and financing decisions—purchasing, licensing, and leasing, with particular attention to recurrent costs and amortization assumptions; 7. Developing standardized purchasing/pricing based on negotiated systems licenses and bulk purchasing. Costs can be estimated on the basis of bulk purchasing, leasing, and systems licensing agreements with vendors whether the purchasing is to be initiated by decisions at the school, district, regional, or national level; 8. Developing monitoring and performance assessment arrangements, with metrics, both for learning outcomes and for hardware and software functionality and use. 6 Conclusions and Recommendations 1. Scale up should be incremental over a period of several years, perhaps 5 years. In addition to the availability of financing, limiting factors will include the rate at which laboratory technicians can be recruited and trained, the rate at which teachers can be Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 20 of 29 trained on e-learning approaches, decisions on selecting schools with at least two functioning computer laboratories, and/or decisions to add laboratories. It is beyond the scope of this assessment to assess these factors in any detail, but the judgment is that the next stage will have to be on the order of 500 additional schools over the next 2 years, possibly fewer. 2. Critical decisions for how quickly the ICT provision and e-learning approach can be scaled up include decisions on which schools and/or districts and clusters to target for the next stage, which will affect the number of additional labs needed; decisions on decentralization and outsourcing of maintenance and technical support; a judgment on how rapidly laboratory technicians can be recruited and trained; and a judgment on the rate at which additional teachers can be trained in the new approaches. Other factors, such as the rates of procurement and infrastructure improvement, are largely budgetary and can likely be scaled up as needed. 3. The ICO investment costs for hardware and infrastructure may be more easily financed or supported by external funders and in-kind provisions by partners. It is critical that the recurrent costs of maintenance, technical support, training, monitoring/assessment, and operations support be anticipated and budgeted for. Donated or externally funded infrastructure and hardware will have the same recurrent costs as other Ministry procurements. 4. Standardization of hardware for each school or cluster of schools will help to reduce costs and improve the efficiency of maintenance. If, as is likely, it is necessary to use more than one vendor, for example if some of the hardware is provided by more than one funder or partner, the Ministry should endeavor to limit the diversity of hardware at any one school or cluster of schools. 5. The Ministry should be able to negotiate volume purchases of hardware as well as system licenses for software and favorable rates for maintenance and warranties. Such warranties and/or funding for maintenance will be needed for donated hardware as well. 6. A judgment will be needed on whether to purchase extended warranties from vendors (typically 3 years) or to contract for maintenance using Jordanian firms and/or NGOs. The costs are likely to be about the same, but the adequacy, particularly the response times, may be quite different. The experience to date under JEI is that contracted maintenance combined with close monitoring has resulted in maintenance response times (i.e. time lost due to inoperable hardware) somewhat less than for computers covered by vendor warranties. 7. It is unlikely that the Ministry will be able to provide efficient and responsive maintenance for a significantly larger number of schools through reliance on centrally reporting ICT Directorate staff. Consideration should be given to contracting with more than one Jordanian firm for maintenance, preferably serving districts, municipalities, or clusters of schools. This might be combined with a system of spare parts warehoused and resupplied on a decentralized basis. 8. Support for the site surveys and for related monitoring of ICT use and operability is critical to overall efficient use of the technology and to effective implementation of e￾learning. These costs are quite modest and could be increased with more support for research and analysis on classroom practices and the effective use of the technology. 9. Some provision will be needed for ongoing refinement of the e-learning materials and approaches. Curriculum and materials development should be an ongoing and iterative process; it would be a false economy to assume that the e-learning materials have been fully developed. Accepting that the e-learning approaches were supported with $9.5 Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 21 of 29 million for the 3-year development phase, a continuing commitment to refine and augment the materials might be on the order of 10 percent of the development costs, or $300,000 annually. On going refinement and additional content development would require a combination of research, testing/assessment and materials development through an organization like the proposed Queen Rania Center for Education Technology and Innovation as well as operational support from the Ministry’s Curriculum Directorate. 10. The site surveys by JEI have been useful in monitoring the use of technology by teachers and for guiding the development and pilot phases of JEI. However, there has been inadequate support to date for carefully structured research on changes in classroom pedagogy, the use of the technology by students, and the effectiveness of the ICT provision and e-learning materials in terms of specific learning outcomes. IWBs and COW carts have been provided in limited numbers, but there does not appear to have been much systematic research or monitoring on how they have been used or their effectiveness against learning outcome objectives. 11. Similarly, there are many other technologies and applications of technology that have been proposed during the development period of JEI, including use of PDAs and cell phones, a variety of data storage arrangements, additional functionality for EduWave, and increased availability of peripheral equipment (particularly printers and copiers) in the schools. There likely will be additional technology options proposed in future years. It is strongly suggested that, as part of any partner offers to provide technology on a demonstration or experimental basis, funds must be made available for the systematic assessment of and monitoring against specific learning outcomes. 12. None of the existing cost estimation and cost management toolkits reviewed for this assessment are fully applicable to Jordan’s administrative and policy context and to the ICT approaches employed in support of e-learning. Jordan may find it useful to develop its own toolkit, based on Jordanian conditions and the relatively uniform and centralized ERfKE systems management and school support models. Such a toolkit may be particularly useful if Jordan moves toward a cluster-based or school-based planning and assessment approach. 13. The operations support from the JEI PMO has been very important to the success of the JEI activities to date. Whatever form the JEI may take in the future, at least the equivalent level of operations support will be required as the e-learning approaches and technology support are scaled up. The site surveys, data collection, and monitoring are particularly important. In addition to the specific operations functions of the JEI PMO, it also has functioned as an opportunity for seconded personnel from the Ministry to gain valuable experience with analytic and technical support and management functions in a team environment that is difficult to duplicate in a larger and more bureaucratic Ministry. 14. Finally, the main recommendation for improving cost-effectiveness is to increase the use of the technology by students, both in the classroom and labs and for projects and other learning activities outside classroom hours. Reducing costs by limiting ICT provision is unlikely to increase the effectiveness of the investment in supporting more active learning by students and more interactive pedagogy by teachers. Scheduling changes to allow for longer time in the classrooms and laboratories; further training and pedagogic support for teachers; and greater use of multimedia rooms and arrangements for students to access computers outside of class time, either in the schools or in community centers are among the options to consider. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 22 of 29 Annex A:Comparative Costing Models and Approaches There are three main analytic approaches to the assessment of costs for ICT applications in education:16 1. Initial Cost of Ownership (ICO)—focus is on the investment cost, typically focusing on hardware, software, infrastructure, installation costs, connectivity, and related design and planning costs but not recurrent management/monitoring/assessment, maintenance and content development, teacher training, and related pedagogic support 2. Total Cost of Ownership (TCO)—including all costs (to the extent they can be identified and estimated) over a relevant period, typically 3 to 5 years, with emphasis on the recurrent budget and support requirements, amortizing the investment costs over relevant discount periods. TCO is most needed in cases where an investment decision has already been made and the goal is to select a solution out of a small choice of potential solutions. 3. Value of Investment (VOI)—assessment of ICO and/or TCO in terms of expected outcomes and impacts, considering alternative strategies for application and use of the technology and assigning quantitative values to the extent possible. Useful mainly for design teams. A number of toolkits and templates exist for use in TCO assessment.17 Most are designed for school and district teams in the initial stages of ICT planning. They typically require weeks, if not months, of analysis, participatory planning, and design choices by the school or district teams. None were judged feasible for use in the Task Four assessment or fully applicable to Jordan. Jordan may find it useful to develop its own toolkit, based on Jordanian conditions and the relatively uniform and centralized ERfKE systems management and school support models. Such a toolkit may be particularly useful if Jordan moves toward a cluster-based or school-based planning and assessment approach Similarly, a review of the available case studies in the international literature on costs of ICT infrastructure and support for education does not reveal any directly comparable models, either in 16 Related terms, used more by businesses, are Return on Investment (ROI), Total Benefit of Operations (TBO), Real Cost of Operations (RCO), and Total Economic Impact (TEI). Their focus includes impacts on productivity and comparisons with other business opportunities. For a succinct explanation of these terms, see http://www.business-value￾group.com/whitepaper-basics1.html?&L=1 17 Most toolkits are designed for use by district and school planning teams in the initial stages of assessment, design, and planning of school or district systems and support estimation of TCO on a multiyear basis. See in particular, K–12 Ownership Calculator, Integrated Technology Education Group for Institute for the Advancement of Emerging Technologies (IAETE) at the Appalachian Regional Education Lab (AEL). For other TCO toolkits, see: • Building the 21st Century School (http://archive.ncsa.uiuc.edu/idt/). North Central Technology in Education Resource Center (R*TEC) and National Council of School Administrators (NCSA). • Taking TCO to the Classroom (http://www.classroomtco.org/). Consortium for School Networking (CoSN) • Technology Integration 2000–2002 (http://www.edfacilities.org/rl/technologyII.cfm). National Clearinghouse for Educational Facilities Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 23 of 29 terms of the ICT infrastructure and e-learning approaches or in terms of cost analysis methodologies. This is in part due to the differences in approach, the differences in scale, the fact that most documented case studies are of school-based or district-based ICT planning and purchasing, and the fact that costs differ greatly across countries, including the adequacy of existing infrastructure and costs of time for teachers and technical support and maintenance.18 The approach supported under JEI, and expected to be supported as e-learning systems are scaled up for other ERfKE schools, differs from other countries on each of these dimensions. In particular, Jordan is developing a whole country emphasis on a unitary basis rather than targeting individual schools or sets of schools; the JEI approach is focused on ICT support for e-teaching rather than on increased ICT use by students; and Jordan already has substantial existing and planned ICT infrastructure, including full enrollment capacity K–12 in all parts of the country, connectivity for most schools with ADSL, and eventually broadband and portal support for e￾learning and other systems management functions through EduWave. These and other factors make the relatively unique approach developed under the JEI difficult to compare directly with the experience of other countries. 1. There are several significant differences between the e-learning approach in Jordan and the models and approaches in other countries. 2. The JEI e-learning is a “blended learning” approach, complementary and supplementary to the curriculum for all public schools in Jordan. It is possible to assess the incremental costs of the e-content, but the full cost of the curriculum implementation include the core costs of the overall education delivery and pedagogic support in Jordan’s schools. 3. Most documented experience is of initiatives either for elementary education, or for intermediate/middle schools and secondary schools. A factor affecting cost efficiency of ICT investments in many countries is that too many students do not go beyond elementary school and even fewer complete secondary school for a variety of reasons. In such cases, the value of ICT investments for the elementary grades is questionable, given the many other requirements for improving capacity and enrollment. This does not appear to be a factor in Jordan, where there is near-complete enrollment through secondary school, the highest in the region.19 Secondary completion rates are 69.5 percent for males and 76.3 percent for females.20 Essentially all children complete at least primary education, with 97 percent continuing to secondary education. The secondary school Net Enrollment Ratio (NER) is 79 percent (2005) versus the regional average of 60 percent NER, with female NER at 80 percent.21 Thus investments in ICT for the elementary grades lead to computer familiarity and ICT skills that support further learning in the secondary grades.22 4. Both the JEI development activities to date and the planned scaling up for the ERfKE schools include grades 1–12. It is not possible in most cases to separate the provision of computer laboratories for elementary or intermediate/secondary schools in Jordan; in most cases they are in the same building and treated as one school with multiple configurations of grade level. 5. ICT support for e-learning under the JEI is provided mainly to teachers, unlike in most other case studies in the literature that focus on the provision of ICT for use by students. 18 In addition, most documented case studies are of ICT initiatives from 2002 or earlier, with consequent difficulties of adjusting for current prices and for newer hardware and software options. 19 For comparison, the completion rates in the United States are only 71.8% male, 79.2% female. 20 UNESCO, Secondary completion ratios (2003/4) http://www.uis.unesco.org/template/publications/wei2006/Chap1_Tables.xls 21 http://stats.uis.unesco.org/unesco/TableViewer/document.aspx?ReportId=121&IF_Language=eng&BR_Country=4000 22 ICT skill development and computer familiarity developed in the elementary grades should not be considered: “wasted” if the student does not continue through secondary education; the skills will still be of value, and increasingly needed, for most occupations and for functioning in the information environment which is developing in Jordan. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 24 of 29 6. Most case studies are of assisted projects, or of school and district ICT initiatives which differ in technology provision or approach from those used in neighboring schools or districts, thus facilitating comparative assessments and research. Jordan is committed to systemwide reforms, where all schools are provided with the same resources and support and are expected to implement the same curriculum with comparably trained teachers. Though clearly it will take some time to achieve uniform support, and there will remain some uneven provision for a variety of reasons, the system-wide commitments do not encourage ad hoc approaches or alternative provision .. Exceptions may be made for piloting promising innovations, but the pilots will be viewed mainly in terms of their potential to scale to other schools, not just in terms of their feasibility, , utility, or even demonstrated success under ad hoc and nonreplicable conditions. See discussion of other specifics for JEI and for Jordan under limiting factors (Section 5 in the Task Four report). Partially relevant ICT costing approaches are found in the following papers. • Computers in Secondary Schools in Developing Countries: Costs and Other Issues, Andy Cawthera, World Bank, 2002 http://info.worldbank.org/etools/docs/library/36057/computer_costs_cawthera_2001.pdf This paper provides a useful literature review and comparative costs for 3 different levels of technology provision—basic, basic plus, and deluxe provision. The analysis focuses mainly on secondary education and ICT provision for student use. Though comparisons are made for the total investment for each country, the case examples are on costs for individual schools, both poorly resourced schools and well-endowed schools, including private schools. Most discussion focuses on options for reducing costs, using donated and recycled computers, and on strategies for increasing students’ computer utilization rates. • The Costs of Computers in Classrooms: Data from Developing Countries, Marianne Bakia, World Bank, 2000 http://www.classroomtco.org/cic.pdf This paper reviews costs from primary school projects in Barbados, Turkey, Chile, and a secondary school in Egypt. The comparisons illustrate major differences in costs across countries, both in cost components and in the percentages of total costs for components. • Computers in Education in Developing Countries: Why and How?, Luis Osin, Center for Educational Technology, Tel Aviv, technical note for World Bank, 1998 Although the cost estimates in this paper are somewhat dated, the paper provides a good example of costing and discounting methodologies, including training, courseware, and operations costs as well as hardware, software, and infrastructure. Examples provided focus on technology for student use at the secondary level. • Making the Case: Mobile Wireless Computer Labs as a Cost Effective Alternative to Fixed Desktop Computer Labs in Schools, Earthwalk, November 2005, http://www.earthwalk.com/_EW_html/_downloads/Mobile_Vs_Fixed.pdf This paper reviews the costs of Smartcarts for use in classrooms as an alternative to lab￾based ICT infrastructure, including comparative costs of the two approaches. Laptops and Smartcarts using wireless connections are justified as achieving savings in terms of lower infrastructure costs (labs, wiring, and furniture) and energy efficiency, which offset the higher ICO investment costs. The paper does not compare the replacement costs or useful life of the two ICT approaches, nor does it address higher costs of maintenance, including replacing batteries and chargers and other hardware failures. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 25 of 29 Annex B: Meetings JEI • Haif Banayan, Executive Director • Niveen Jum’a, Technology Track Coordinator • Luma Atallah, Project Coordinator • Ammar Abdullah Alamleh, Netcorps intern coordinator ICT Directorate • Eng. Mazzen Amarin, Director World Links • Lana Noureddin Abzakh (accompanied on school visits) Integrated Solutions, Jordan Telcom (formerly e-Dimensions) • Lana Katbeh (Content Development) • Kholoud Totah (Head of Corporate Social Responsibility Strategy) Rubicon • Isam Ajoubi, Managing Director Janeel Shaker (boys’ school) • Bilal Al-Dabbas (principal) • Khaldoun Rhahadh (English teacher) • Computer lab technician, presentation by students and math teacher on LEGO Energy Robotics model Queen Rania (girls’ school) • Khaldoun M. Rhahadh • Computer lab technician, CEB maintenance technician USAID • David Bruns • Maha Alshaer ERfKE Support Project • Jeff Coupe, Chief of Party (COP) USAID/ERfKE School Building Project • Sarah Woodhead, COP, Camp Dresser McKee (CDM) Queen Rania Center • Met with Jeff Coupe, Education Support Program COP • Met briefly with Go Ota, JICA team leader, Capacity Development of Learning Resources Centers for Science Education Utilizing ICT • Observed planning workshop for principals on School Development Units Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 26 of 29 ERfKE • Ian McLellan, Director ERfKE Coordinating Unit • Brenda Cooke, Senior Advisor: Governance, Management and Accountability Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 27 of 29 Annex C: Approach and Scope for Task Four Consultancy November 10–20, 2007 Purpose: Assess costs of JEI activities and inputs in support of e-learning in Discovery Schools and the cost implications of scaling up the JEI technology platforms and e-learning approaches as part of the ERfKE programs for all public schools in Jordan. Approach: The approach will need to address three analytic problems. 1. There is considerable variance among the Discovery Schools in the hardware provided (laptops, datashow projectors, IWBs, and COW carts), the number of computer labs, and the number of PCs in the labs. There also are differences in the utilization of the technology, by level and by size of school. 2. Major components of the technology support and e-learning support is provided under ongoing Ministry programs (ERfKE and other) that are not separately budgeted or attributed for the Discovery Schools as a group or individually. 3. Estimation of the likely costs of scaling up the e-learning approaches, and the technology required for such approaches, depend on policy and strategy decisions that for the most part have not yet been made, by the Ministry or by JEI. The assessment (and the subsequent report) will be in three parts. Part One: Assessment of the costs to date of JEI and JEI partners; support for hardware provision; additional connectivity (mainly wireless), e-learning module development; and related training for teachers, lab technicians, technical support, and other activities directly related to the technology provision and e-learning program development (such as the costs of site surveys, workshops, and monitoring) but not including the costs of JEI mobilization and coordination or executive overview. • The assessment will distinguish the costs of the approach elements supported by JEI in Discovery Schools from the underlying or baseline costs of the ICT support for ERfKE schools. Much of these costs are not separately budgeted or attributed to individual schools—the baseline includes at least one computer lab, costs of EduWave, costs of ADSL, or broadband connections through Ministry arrangements with Jordan Telecom and the NBN. The assessment will include only the additional or incremental costs required for the e-learning approach, including the need for additional computer labs, wireless, laptops and datashow projectors, whiteboards (where provided), additional lab technicians, additional technical support, and additional training for teachers. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 28 of 29 • This part of the assessment will rely mainly on information available from JEI and proxy estimates of the costs of Ministry staff, interns, and other Jordanian involvement based on estimates obtained from the Ministry or other sources. Part Two: • On the basis of the Part Three report, estimates of the costs of additional hardware and software requirements to address shortcomings identified—caching and storage capacities, and virus protection. • Projections of the costs of maintaining the current technology support for the Discovery Schools—ongoing technical support, expected rate of replacement of laptops and desktops, ongoing e-learning training and pedagogic support for teachers, and supply of lab technicians. • This part of the assessment will require consultations with (in addition to JEI coordinators) the Ministry ICT Department and possibly the Training Department as well as with selected vendors and NGOs. Part Three: • Identify the major cost drivers for scaling up the e-learning technology for the other ERfKE schools. • Identify the major alternative configurations of technology deployment, use and support, and the cost implications for each. Factors to consider include the mix of laptops and desktops, the feasibility of greater use of whiteboards, the minimum requirements for computer labs and lab technicians, the use of media rooms, and arrangements for technical support and maintenance. • Construct at least two scenarios for scaled up technology supporting e-learning for consideration by JEI and the Ministry. Activities: • Meet with JEI Executive Director, PMO, technology track coordinator, and other JEI personnel as necessary to review findings of Part Three report, review approach obtain cost data and other specifics on the inputs provided by JEI partners and the ICT and e￾learning training provided to teachers, the ICT technicians and technical support for the computer labs, and the variance in such support among the Discovery Schools. Accomplished: At time of consultancy, PMO had recently resigned and the Executive Director was away until 11/18, with a meeting possible only at the end of the consultancy. Meetings with technology track coordinator provided valuable input, particularly from the site surveys and other specifics on the training provided to lab technicians and teachers, and related insights to the problems of technical support and maintenance. The project coordinator was very helpful in arranging other meetings and school visits on short notice. • With JEI assistance, arrange visits to a sample of JEI schools (3 or 4 representing different levels and sizes and different availability of labs and hardware (e.g. some have interactive white boards, some have more labs than others). Partially Accomplished: Visits were arranged to Janeel Shaker (boys’ secondary school and to Queen Rania (girls’ secondary school. World Links Director of Programmes accompanied on school visits. Report: Task 4 Cost Assessment of Technology Support for E-Learning April 17, 2008 Page 29 of 29 • Either as part of the school visits or separately, meet with a representative sample of principals and a representative sample of lab technicians to obtain views on management of the inputs, technical support and maintenance issues, and other views on the level and configuration of inputs. Partially Accomplished: Met with two principals, several teachers, and lab technicians at two schools. • Meet with Ministry ICT Department to obtain views on priorities and likely plans for technology provision, computer upgrades/replacement and technical support/maintenance, and cost estimates if possible. Accomplished: Very substantive meeting with Director of ICT Directorate, including plans and cost estimates. • With the assistance of JEI, meet with selected vendors and with firms or NGOs potentially capable of providing technology support services to obtain (nonbinding) estimates of likely costs. Partially Accomplished: Met with e-content partners Rubicon and Jordan Telcom/Integrated Solutions and with Rubicon but not with hardware or software vendors. • Review the estimates and scenarios developed under Parts One, Two, and Three of the approach described above with JEI and others as JEI may suggest. Accomplished: Draft report of Task Three reviewed in detail with team and feedback received from JEI. • Meet with USAID to review progress and findings. Accomplished: Met with Education Officer and CTO 11/18 • Prepare a draft report prior to departure. Partially Accomplished