., - ~nforriition Technology for Youth (IT4Youth In Rural Communities in the West Bank US& Cooperative Agreement # 294-A-00-00-00070-00 Summary able of Contents 11. Program evaluation methodology used for this report III. The impact of the IT4Youth program during its first year of implementation Al. Results achieved , . - ~nfp&&~kr~ . . . - Curriculum development . ~ ' -.~a~acity..buildin~ - . Household . survey ~. , . .. ~ ~2. R&ults planned but not achieved .- Infrastrbcture . . - capacity building . . COP HHS rR IT IT4Youth IYF JCSC MOE MOLG PMO USAID WA MC RITC International Youth Foundahh Annual Evduaiion Reporf, April I, 2001-Mmeh 31,2002 Abbreviations Chief of Party Household Survey Intermediate Results Information Technology Information Technology for Youth International Youth Foundation Joint Community Senices Council Minisw of Education Ministry of Local Government Program Management Organization United States Agency for International Development Welfare Association Management Committee Regional Information Technology Center Znternndbnnl Yo& Fowtdndbn Annual Evaluation Repo* ApriII, 2001-March 31,2002 IT 4 Youth Program lSt Annual Evaluation Report I. Summary A. Background. This report presents an assessment of the performance of the IT4Youth program during its first year of implementation (April 2001-March 2002). The program, developed in partnership by the International Youth Foundation (WF) and the Welfare Association (WA), is funded by the United States Agency for International Development. The IT4Youth program works with local partners to assist schools and community organizations in a selected cluster of villages in the northern region of the West Bank to provide access to and training in computer-based IT for youth. It is designed to promote IT use in the community for education, work, and recreation to enhance the learning skills and employability of its participants, leading to a better quality of life. The program targets an estimated 7,000 mal students and youth, 25 computer skills teachers, 50 non-IT teachers, and 300 hundred parents and adult community members in a cluster of eight villages and three hamlets (population 30,000) in adjoining areas of Nablus and Jenin districts: in Jenin, the villages of Silet ad￾Dhahr, Jaba', Fandaqomiya and al-Attara and the hamlet of A1 Assaa'ssa; and in Nablus, the villages of Burqa, Bizariya, Beit Imrin, Sebastiya and the hamlets of Nisf Jbeil and Ijnisinya. The program is implemented in cooperation with a wide range of local institutions including: the MOE of the Palestinian Authority, local village councils, schools, the ~uturekids~ Center, training institutions, and the youth center in Silet ad￾Dhahr, and reaches youth thtough two mechanisms: IT Labs in Schools Program: Through the construction/renovation of 14 computer labs in MOE schools, development of the IT curricula (Futurekids@ program), and training of IT teachers and non-IT teachers. Community Progmm: Through the establishment of a Regional Information Technology Center (RITC) in Silet ad-Dhahr, two mobile IT units, an out-of￾school IT program, and IT vocational training programs. Management of the IT4Youth Program is by the Program Management Council (PMC), comprised of three members from IYF and three fiom WA. The PAW meets 4 times a year to review progress, to consider issues raised by staff, to provide any needed guidance to the project team, to review the quarterly reports, and to discuss fkmcial and programmatic issues the reports raise. The PMC submits quarterly progress reports to USAID. It also submits to USAID requests for budget re-allocation, project enhancement and project expansion. Znternanbnal Youth Foundanbn Annual Evaluaabn Rep04 April I, ZOOI-March 31,2002 The implementation of the program is carried out by the Program Management Organization (F'MO). Program staff consists of a COP, three coordinators, a financial controller, an office assistant and a regional field IT administrator. At the co111111unity level, program implementation and supervision is carried out through two committees: the Joint Community Services Committee (JCSC), which consists of one representative fiom each local village council and assumes a supervisory role on pro& implementation, and a Management Committee (MC), which' consists of 16 youth who support the implementation of program activities and promotes the program at the community level. The program is evaluated by a team of three external evaluators. The team has: developed the tools and plan for a baseline household and community survey; planned and put in place a process for evaluating indicators based on the annual indicators submitted in the evaluation plan; worked with mouth staff to develop reporting formats and information; and made field visits with project staff to check on progress and talk informally with community leaders, school and government staff. In addition, the evaluation team assisted staff in the monitoring of the project performance and data generation fiom the field. B. - Main Findings. 1. A total of 14 labs have been constructedfrenovated. Each lab was provided with a package of furniture and equipment. This package included an average of 21 computers, computk tables, chairs, curtains, a scanner, a printer, networks, wiring, lighting requirements, Classnet sohare, and other technical needs. In light of the complicated political situation, the process of construction, renovation, and equipping of labs was remarkable and has been carried out according to plan. The technical problems that occurred involved a few computers and monitors that were not functional. 2. Cooperation agreements have been reached between the IT4Youth program, the MOE, and the ~uturekid? Center for piloting the Futurekids' program for grades 8-10. Model 1 bf the Futurekids@ curriculum was prepared and adapted to local needs based on feedback provided by the MOE. 3. A series of training courses was organized for Ilteachers and staff of the MOE. Training was designed to provide trainees with knowledge and skills to implement an improved IT program for students, to effectively manage the IT labs (set-up, maintenance, etc.), and to train other teachers. Training was supposed to be accompanied by field support and follow up by the Futurekids@ Center. However, due to closures and mobility restrictions, no field visits were able to be carried out. Instead, follow-up was carried out Iniernational Youih Foundnnbn Annual EvaIuntion Repo* April I, 2001-hh 31,2002 through telephone and e-mail, an arrangement that was understandably not satisfactory. 4. Two courses on basic computer skills were organized for 40 non-IT teachers (i.e. teachers of regular subjects). 5. The IT4Youth program gave special attention to building the capacity of the staff of the MOE. An average of six to ten staff members from the MOE have attended training courses. This was seen as an important step towards: building strong partnerships between the program and the MOE; improving the ability of the MOE to support and monitor the implementation of the project in their schools; and strengthening the capacity of the MOE to expand the project to other regions in future. 6. Several training courses were organized for members of the JCSC and MC (communication, voluntary work and fundraising). Members of the JCSC and MC have also benefited from the joint planning and implementation of program activities. 7. A number of activities were carried out to strengthen the capacity of IT4Youth staff members, including: a trip to the USA to visit effective IT programs and to attend an IYF workshop on sustainability; a visit to Cairo for training on USAD procedures; technical support from IYF staff and consultants; and close and regular follow-up and coordination with IW. 8. The implementation of the ~uturekids~ pro-has varied from one school to another. One out of the 14 schools has not implemented the program because of an absent teacher. 9. Some of the planned activities not implemented during this program year include: the establishment of an RITC, connecting schools to the Internet; and constructing a mobile IT unit. 10. The quantitative HHS has been completed and analyzed. The qualitative component has been delayed as a result of the escalation of violence. 11. Program evaluation methodology used for this report. This annual evaluation utilizes a variety of methodologies for data gathering and synthesis, including: A review of the four quarterly status reports; In-depth interviews with the IT4Youth program staff; A review of the results of individual and group interviews with JCSC and MC members in Jenin and Nablus, carried out in April 2002; Iniernaiional Youth Foundation Annual Evaluation Report, April I, 2001-March 31,2002 Interviews with the ~uturekids~taff; and Household survey results. lTI. The impact of the IT4Youth program during its first year of implementation. It is important to note at the outset that the continued and escalating violence and the disruptions and closures in the region have influenced the implementation of the program. Despite these challenges, the vast majority of planned output and outcome targets for the first 12 months, according to the Detailed Implementation Plan for fiscal year 2001, have been accomplished, especially in schools. The major exception is the implementation of a number of community-based activities that were postponed as a result of the delay in the construction of the RITC. The main assumptions underlying the Detailed Implementation Plan for fiscal year 2001 (Section 11) were no longer operative during this reporting period: + That people and goods will be able to move with relative ease to and within the West Bank, + That unemployment and poverty will not rise significantly above June 2000 levels; + That public sector institutions will be able to finance the intended services as planned; + That there will be no political obstacles hindering the continuance of the program; and + That project partners and participants will be able to continue to fdfill their roles and responsibilities in the program. Al. Results Achieved. Infrastructure. In order to increase the access of students in the cluster area to IT, the program has given priority to establishinglrenovating and equipping a computer lab in each school. The following has been achieved: A total of 14 computer labs have been constructed/renovated. Out of the total, four new labs have been constructed and ten have been renovated. Supported by the IT4Youth program, 13 labs were completed during the second quarter. The 14th laknstructed with the municipality of Silet ad-Dhahr-was completed during the third quarter. Each lab was provided with a package of furniture and equipment This package included an average of 21 computers, computer tables, chairs: curtains, scanner, a printer, networks, Classnet sohare, n&g, lighting requirements and other technical needs. Internntiond Youh Foundation Annd Evduotion Report, April I, ZOOI-Mmch 31,2002 Additional needs of some labs were also met. For example, air conditioning was installed in one of the labs, sunroofs were installed in two labs, and steel bars were installed in all labs that are located on the ground floor of the schools. IT4Youth staff installed software and networks (Windows XP and educational programs). The installation was expected to be complete before the beginning of the second school semester, which started on the 26' of January. However, as a result of a two- to three-week delay in receiving the computers due to a strike at the Israeli airport and customs department, there was a delay in the installation of computers and software and the completion of the preparatory phase. Observations. The implementation strategy and approach had a number of positive elements: According to the initial plan as recommended by the MOE, 13 PCs were to be provided to each school. This meant that the computer to student ratio would be 1:4, which is not acceptable according to international standards (UNESCO). However, the IT4Youth program upgraded the initial design to increase the computerlstudent ratio. Specs were changed slightly to save some money, which was used to buy more computers. As a result, each school now has a total of 21 computers. The computer to student ratio has become close to 1 :2. Such a ratio should increase the quality of student learning. The constructionlrenovation work was carried out through a decentralized model of implementation to overcome the problem of closures of the Palestinian territories by the Israeli army. This decision ensured the completion of work on time, created some jobs for youth in the target communities, and enhanced the community feeling of ownership of the project. The day-to-day site supervision was carried out by an engineering consulting firm A1 Hanbali Engineering Firm. Local councils nominated three bidders in each region who submitted their bids. A committee composed of the IT4Youth manager, engineers £?om the MOE, and A1 Hanbali Engineering Firm reviewed the bids and selected the winning bidders based on technical and hancial criteria. Computer technicians, IT teachers, MOE computer supervisors and IT4Youth staff worked together to install software and networking. This provided the IT teachers and the MOE computer supervisors with useful practical experience in setting up computer labs. Program implementation encountered some limitations, including a few shortcomings in the labs. Only three computers and monitors did not function Internadbnal Youth Foudinion Annual Evaluadbn Reporf April I, 2001-March 31,2002 well and some software was not installed in one lab on time. About five wmputers had problems with Classnet, but in general, the networks of wmputers functioned well. To overcome such problems, especially in the middle of the strict closure of the Palestinian tenitories and the lack of regular transportation system, it is rewmmended that: When the political conditions and mobility of people improve, all labs should be tested to identify and correct all operational problems. The technical section within the IT4Youth project should develop a lab manual that clarifies procedures and guidelines for the maintenance of labs. This document should be discussed and agreed upon by the IT4Youth project, the MOE and schools. The IT4Youth project should give special attention to continuing to build the capacity of local IT teachers and the IT department of the MOE to be able to cany out simple and routine maintenance of computers. It is highly rewmmended that IT4Youth organize an advanced training come on the maintenance of wmputer labs. This is especially important in light of the continued closure of the Palestinian tenitories and the inability of wmputer technicians to reach schools on regular basis. In addition, this step would ensure the technical sustainability of the program when it phases out. Curricuium development. Cooperation agreements have been reached between the IT4Youth program, the MOE and the Futurekids@ Center for piloting the Futurekids@ Program in grades 8-10. Model 1 of the Futurekids@ Curriculum was prepared to be taught during the second semester of the school year 2001i2002. Based on requests by the MOE, some adaptations were suggested to the cuniculum by the Futurekids@ Program to make it even more appropriate to local needs. Modifications were approved by the three partners. Twenty-two copies of both teacher guidebook and curriculum work papers were printed and distributed to IT teachers and MOE staff. The ~uturekids~ program focuses on building computer skills through practical, interactive exercises. In addition to basic computer operation and skills, Module 1 of the Futurekids@ Curriculum focuses on the Microsoft Tublisher" pro-cram. The Module consists of 7 classes (6 are mandatory and one is optional). One of the 6 mandatory classes includes an evaluation of the students' achievements. Capacity building. Capacity buildiig represents a core component of the IT4Youth program. Capacity building strategies include: classroom trainin=, 0. on￾the-job training; follow-up and support through field visits; joint planning; International Youth Foundation Annual Evaluation Report, April I, 2001-.Wareh 31,2002 monitoring and evaluation of activities, etc. Following is a description of the main activities carried out: A. Capacity Building of Schools. 1. IT teachers. Training of IT teachers has been aimed at providing them with the knowledge and skills to implement an improved IT program for students, to effectively manage the IT labs (set-up, maintenance, etc.) and to train other teachers. An assessment of training needs of schoolteachers (including IT teachers) was carried out during the first quarter. Based on this, a three-year training plan has been developed. During the first year, the IT4Youth program contracted a number of trainers to carry out a series of training activities aimed at upgrading the technical skills of IT teachers in schools. It is worth noting.that the IT teacher tra-g program represents an on-going activity that will be implemented over the coming two years. Capacity building of IT teachers included the following: a. Training courses. During the first year of the implementation of the project, a series of training courses for IT teachers was carried out Training targeted 13 IT teachers, who attended training courses on Windows 2000 Professional, Windows 2000 Server, set-up of computer labs parts 1 & 2, and the Classnet training course (see table below). A 9-day training of trainers course for 13 IT teachers on the implementation of the Futurekids'@ package was prepared and implemented by the ~uturekids'@ Center. The objective was to equip IT teachers with the knowledge and skills to implement the Futurekids' program in an effective manner. Internafional Youth Foundation Annual Evaluhn Rqort, April I, 2001-March 31,2002 Table 1: Training Courses Course title I Trainers 1 Duration Windows 2000 1 Amra 1 3 days Professional I Training - I (21 hours) Center server Training (14 hours) Center 'rovided to IT Teachers ant Participants 15 participants (1 1 IT teachers, 2 computer supervisors, 2 heads of technical departments at the MOE) 17 participants (1 1 IT teachers, 2 computer suoervisors, 2 heads of Staff of the MOE Notes Half of the trainees were not able to attend the first day of mining because of the closures. The first day's training ajas repeated the next day. A pre-test was carried out and showed that the performance of the majority of the participants was much lower than expected. rn This training represented part 2 of the previous training session. Trainiog was delayed mice due to Israeli closure of the territories. Part 1 I I te&nical dkarments at the MOE, 2 computer engineers I - ! from the MOE Set up of Staff of the 1 day 19 participants . The main objective was to comvuter labs IT4Youth omare lT teachen and MOE Set up of computer labs Part 2 Computer Department ' (12 IT teachers, 2 computer officials to manage and 1 supervisors, 2 heads of maintain the computer labs. 1 technical departments at the . Course n.as postponed as a MOE, 2 computer engineers result of closure of territories. from the MOE. 1 comDuter engineer 6om the General Technical De~ment ofthe MOE) Staff of the 3 hours per IT4y0uth school Computer Department 12 participants This mining was carried out during the set-up of each (12 IT teachers) computer lab and a hands￾on training session. All the IT teachers pamcipated in the set up of the computer lab. A few of the teachers could not participate due to closure of their areas. 21 participants 0 Training u~cluded theoretical and practical components. (13 IT teachers, 2 computer Each trainee was given a supervisors, 2 heads of package of the futurekids' technical departments at the resource materials to help in the MOE, 4 computer supervisors implementation of the program 60m the directorates of the in schools. MOE in Qalqilia, Tulkarem, Salfeet and Nablus. I Training of ; Futurekidsw : 9 days trainers on ~uturekids" Center . ~ Inmlrtional Poufh Foundation Annual Evaluahbn Report, April I, 2001-March 31,2002 b. Practical Training. Most IT teachers worked with the computer technicians during the installation of the computer networks and software. This step was seen as practical training for the IT teachers. It was agreed with the MOE to use the services of two of the trained IT teachers to train other non-IT teachers (see below). This will provide IT teachers with practical training experience and encourage them to train other teachers in their schools. c. Follow-up on-the-job training by ~uturekids~ Center. In addition to the training of IT teachers, and in accordance with the agreement behveen IT4Youth and the ~uturekids@ Center, staff eom the Futurekids" Center are to cany out monthly monitoring visits to each teacher to follow up on progress, assess the implementation process, and provide teachers xvith support and guidance. It has also been agreed that the ~uturekids~ Center will be involved in a comprehensive evaluation of the Futurekids" program at the end of each semester by helping to collect data for analysis by the IT4Youth evaluation team. The implementation of the Futurekids@ Program started in 12 out of the 14 schools. Later in the semester the other two schools implemented the program. The delay was due to lack of power in one village and lack of power in the computer lab in the other. The implementation in one of the schools was cancelled after the first week because, as a result of a very strict closure, the IT teacher was not able to reach the school (see below). Observations. All training courses were evaluated after the trainings. Results showed that there were an equal number of male and female participants in each training course and that the trainings included theoretical and practical skills. Pre-training assessment showed that the technical skills of teachers were not satisfactory. Post-tests showed varied degrees of improvement Most training courses were attended by 19 persons (13 IT school teachers', two heads of technical departments at the MOE, two computer supervisors at the MOE and two maintenance engineen at the MOE). The participation of heads of the technical departments, the IT supervisors and the maintenance engineers will assist in strengthening the capacity of those departments and improving their support and supervision of IT teachers in other regions. FGDs with IT teachers showed a high level of enthusiasm about the Number of teachers is 13 while the number ofschools is 14 because one of the teachers teaches in hso of the targeted schools. Znrenrational Youth Foundnfion Annual Evaluation Repori, April I, 2001-March 31,2002 training. Some trainees indicated that the training was very useful in developing their skills and in introducing them to more advanced and up￾to-date computer skills. Others reported that the training was useful for their self-development and will open new opportunities for them. They thought that the organization and effectiveness of the training were very good. Observations of participants regarding the trainings focused on the following: Trainings were very intensive and their duration were short (this was done in compliance with MOE's wishes); Transportation to training events were too costly and difficult; One lab was not fully functional at the time of training. A participant indicated that if the lab had been functional, the training would have had better effects because they would have been better able to apply what they had learned during the training courses. Main recommendations include: 0 More training during the holidays (being planned); More use of Arabic language during the training (more emphasis on Arabic, while maintaining the requisite content in English); and More training on networking and lab management (being planned). 2. Non-IT teachers. Two (3-day) basic computer training courses were organized for 40 computer-illiterate teachers (20 from Jenin and 20 f?om Nablus). The training was carried out by one of the IT teachers and one of the MOE computer supervisors. B. Capacity building of the MOE staff. The IT4Youth program has given special attention to building the capacity of the staff of the MOE. This was seen as an important step towards building strong parinership between the Program and the MOE, improving the ability of the MOE to support and monitor the implementation of the project in their schools, and strengthening the capacity of the MOE to expand the project to other regions in kture. Towards this end, an average of six to ten staff members from the MOE have attended training courses on: Windows 2000 Professional; Windows 2000 server; set up of computer labs parts 1 & 2; Classnet h-aining course; as well as the ~uturekids~ program. For more details please see Table 1 above. It is worth noting that the two IT supervisors in Jenin and Nablus and the two heads of Technical Departments in the two regions in addition to one of the International Yo& Foundaiion Annual Evaluaiion Report, April 1,2001-March 31,2002 IT4Youth staff, attended the training on ~uturekids~ program to become able to monitor the implementation in schools. C. Capacity Building of Members of the JCSC and MC. In the second quarterer the Joint Council Services Committee (JCSC) was formed. The JCSC consists of representatives of each village council in the project cluster. During the initial phase, members of the JCSC met on a monthly basis. However, due to the strict closure and the inability of members to meet on a regular basis, it was decided to split the committee into two (one for Nablus and one for Jenin) and to maintain coordination through the Coordinator of each committee. The Nablus JCSC was officially registered with the MOLG. The Jenin JCSC was registered before the initiation of the project. However, the registration does not include Jaba' village. An official application has been filed with the MOLG for the registration of Jaba' village with the Jenin JCSC. The commitment of members of the JCSCs remains an important tool for ensuring the sustainability of the project at the community level. Two Management Committees, one in Jenin and one in Nablus, were created during the third quarter. Local councils nominated the candidates of the committees in both areas according to a set of criteria developed by the JCSC and the IT4Youth staff. There are 16 members of the two MCs (nine males and seven females). Four sub-committees have been established within each MC. During the 3rd quarter the two MCs have undergone a reshucturing process because some of the members had to quit for personal reasons. Each member who left the committee was replaced by another member om the same village. The total number and gender balance was maintained. JCSC members and other MC members oriented the new members. Capacity building of members of the JCSC and MC was canied out through the following: 1. Training. A training plan aimed at developing the skills and knowledge of members of the JCSC and MC was developed during the visit of the IYF Director of Capacity Building Services. Special focus has been given to discussing roles and responsibilities, legal issues related to the constitution of the JCSC and MC, building effective team work, improving communication among members of the committees, improving the reporting abilities, etc. Identified training areas for JCSC members included: legal issues related to JCSC structure and function; skills of board members, maintenance and operation of infixstructure; basic computer skills; personal skills (communication skills and conflict management); and sustainability. Identified training areas for MC members include basic management and administration skills; financial management; management of meetings; Infernahbnal Youfh Foundaii'on Annual Evduotion Repori, April I, 2001-March 31,2002 event planning and management; computer skills; and personal skills (communication skills and conflict management). During the kt year, the following training courses were implemented: A training workshop on communication skills was organized for members of the JCSC; A training workshop on voluntary work was organized for members of the MC; A training workshop on fundraising, especially fundraising 6om the Welfare Association, was organized in November 2001 to provide capacity-building training and technical assistance to JCSC and MC members and local council members. Two workshops were organized (one for the seven members of the Jenin JCSC and one for the six members of the Nablus JCSC). 2. Joint Planning and Follow-up. The Youth and Community Coordinator (YCC) worked closely with members of the JCSC and MC to clarify roles and responsibilities and to develop plans for each committee. Special focus has been given to building mutual trust among members of the MCs and between those members and the YCC, as well as to developing effective communication systems and documentation of activities. Some simple adrninisbtion and communication procedures were discussed and agreed upon. It has been agreed that MCs will meet every two weeks to discuss progress and plan for the activities, in addition to a weekly telephone conference between the youth and community coordinator and the coordinators of the two MCs to update each other on new information and progress. With the support of the YCC, both MCs planned a community awareness and promotion campaigns. The campaign was not implemented in the Nablus district as a result of the closures and subsequent Israeli invasion of the district, but it was implemented in the Jenin district. The objectives of the campaign included: introducing the concepts and objectives of the IT4Youth project; mobilizing .the local community in support of the project; raising the awareness of students regarding TT, and promoting support of the project. Observations. Field monitoring showed that the roles, responsibilities and obligations of members of the JCSC and MC continue to be somewhat unclear to several members. Clarifymg such roles and responsibilities would International Youth Foundation Annual Evaluation Report, April I, 2001-Mmch 31,2002 improve the commitment of members, increase their participation in the supervision and implementation of the various community activities, and enhance their feeling of responsibility towards the program and its sustainability. Capacity-building activities for members of the JCSC and MC appear to need improvement. All training activities use off-the-shelf training courses and need to be better customized. More efforts should be made to increase the effectiveness of coordination and cooperation between the JCSC and MC. C. Capacity Building of IT4Youth Staff. During the first year of implementation, various activities were implemented to strengthen the capacity of the IT4Youth staff. Activities included: 1. An Information and Learning Exchange in the USA. During this visit the team was oriented to IYF in Baltimore, Maryland; attended a three-day sustainability workshop; visited USAID in Washington, D.C.; and visited information technology educational projects in Washington, D.C. and the San Francisco Bay Area, California. According to members of the team, the visit was very successful in introducing them to: key concepts related to sustainability; different models of IT training; exchanging information and experience with a number of IT training institutions; expanding the networking activities of the IT4Youth project; and discussing potential cooperation between the IT4Youth project and other projects in the USA, such as Schools Online. 2. Support from IYF Staff and Consultants. Throughout the year, IYF has maintained close contact with IT4Youth program staff members to strengthen their ability in implementing the project. Examples of this support include: IYF's Regional Director for AfXca and the Middle East maintained daily contact with the program COP and staff to follow up project implementation and provide various types of support. During the second quarter, IYF's Director of capacity Building Services visited Palestine, made site visits and worked with staff in assessing training needs of members of the JCSC and MC and in developing a capacity building plan for the two committees. IYF's Director of Technical Support Services visited Palestine, made site visits and worked with staff and the Field External Evaluator to review project progress, develop the annual Detailed Implementation Plan, prepare the expansion and enhancement proposals and provide Internntional Youth FoundazSon Annual Evaluation Repori, April I, ZOOI-March 31,2002 consultation and advice on various issues related to project implementation. During the first quarter, the Field External Evaluation team leader visited Palestine, made site visits and worked with IT4Youth staff and other members of the evaluation team to develop a detailed evaluation plan for the project. Throughout the year, the Evaluation Team maintained daily contact with IT4Youth staff to assist staff in the monitoring of the project performance and data generation from the field, assess project achievements, strengthen the capacity of staff in reporting, project monitoring, evaluation of training activities and provide consultation and advice related on issues related to monitoring and evaluation of project activities. When asked about the support of IYF, the COP and staff of the IT4Youth staff emphasized the importance of the direct and professional relationship between IYF and the program staff.2 According to the COP of the IT4Youth Program, IYF "...has been online. IYF staff and consultants have always been there when we needed them. Many innovative and useful ideas were provided to us. The only problem is the complications associated with the visits of expatriate staff and consultants to Palestine in the middle of a very complicated political situation. This has sometimes led to delays or cancellation of some visits, evacuation of some consultants, etc. In addition, the mles sometimes prevent consultants fium visiting Palestine when the situation is risky, and even if those staff or consultants are willing to take such risks. Therefore, and until the situation improves, it is recommended to use the services of local consultants." Internrrtional Youth Foundnrion Annual Evaluahbn Report, April I, 2001-March 31,2002 D. Capacity Building of Students. Following the completion of the Futurekids@ training of trainers course, IT teachers started the implementation of the program in their schools for grades 8-10. The total number of students in the 14 schools is 5,372 (3,090 males and 2,282 females), of whom 2,011 are students in grades 8-10. According to the agreement between the IT4Youth program and the Futurekids@ Center, staff from the Futurekids@ Center would visit each school once a month to follow up on project implementation, monitor progress; and provide consultation and support to IT teachers. However, due to the strict closure imposed on the tenitories, trainers of the Futurekids@ staff were unable to carry out any field visits to schools. Implementation was followed up through e-mail and phone contacts between IT teachers and staff of the Futurekids@. According to Futurekids@ Staff, program implementation has been as follows: Table 2: Imolementation of the ~uturekids~ momam . - in schools until the 28" of March 2002. Status 1 No. of schools Schools that had implemented the Futurekids@ curriculum 1 7 Schools that had provided a theoretical introduction to the ~uturekids~ I 3 curriculum but nit practical applications I 1 I Schools that had not implemented the program because of the inability of the teachers to reach the school due to the strict closure The implementation of the program was not assessed because 3 Futurekids@ staff could not get in touch with the IT teachers International Youth fiundation Annual Evalrmtion Report, April I, 2001-March 31,2002 3. Fandaqurniya Boys School 1. Jaba' Boys 2. Jaba' Girls -- Table : rota1 no. --- in sc M 564 - 4.Fandaqumiya Girls I - ] 6. Silet ad-Dhnltr Girls I .......... -. .., 5. Silct ad-Dl~ahr Boys 1 7. Scbastiya Boys .... DetSIed students ~1- - . F - 546 :sentation of the Stat No. of st grad1 M 136 163 176 149 ents in 1-10 F ...... 191 156 140 , of Program - Implementation in Schools Progress until 28 March 2002 PD ' According to 1:uturckids proyrnm stnlT, contact with the IT tcnchcrs was not possible. I W Until 28 March 2002, no pructicul upplicution was curried out bccuusc, uccording to the tcf~chcr, Clnssnct wns not insti~llcd. 1:utnrckid~ rcporkd [hut Clnssnct OD is not n rcquircmcnt ibr implcmcnting tl~c Futurekids Curriculum. Until 28 Mi~rcll 2002, no pructicnl upplicntion was cnnictl out hccuusc, the l~cnd tcacher tlccidctl not to use the computer lnb, l'l~is rccloircd 1110 intervention of tllc l'I'4Youtl1 program wit11 the MQE. The prnctical application started in April 2002. -~ - School - B. Sebastiya Girls 9. Beit Imrin Boys 10. Beit Imrin Girls 11. Bizariya Boys 13. Ihtrqa Boys 14. B~trqa Girls Total No. of school F 278 No. of students in "lci~cl~cr could not rcl~ch school due to closurc. "No clcctricity in the village due to rcl~ubili~ation ofthe clcctricily network. International Youth Foundation Annual Evaluation Report, April I, 2001-March 31,2002 - Progress until 28 March 2002 , , , . -. - . Unit 5 ... Unit 5 -. -- Unit 3 . - Unit 2 . All units ...... Unit 1 ... .- Unit 1 ~ ~ Unit 1 ... Program not implemented6 ~~ Program not implemented Program ......... not implcmcntcd Program not implemented7 Prog;am not . implcmcnted Program not implemented Unit 3 Unit 3 G? ' According to I~uturokids program staff, contact with tho IT tcacl~crs wns not possible. International Youth Foundation Annual Evaluation Report, April I, 2001-March 31,2002 International Youlh Foundnnbn Annual Evaluation Repori, April I, 2001-.+larch 31,2002 Observations. According to Futurekids@ staff, IT teachers who implemented the program were confident and capable of maintaining a good quality. Students were enthusiastic about the program and reacted positively. Monitoring of the implementation by Futurekids@ program staff, IT4Youth staff, and the MOE was not optimum. The implementation process was lei? to the initiative of each teacher resulting in differences in the pace and quality of instruction. Although the Futurekids@ program is based on practical application and skill development concepts, the implementation of the program in three schools was limited to a theoretical introduction without involvement of ITlYouth staff, Futurekids@ staff, or the MOE. Household Survey. In an effort to assess the level of knowledge, interest and use of computers in the project villages, a household survey (HHS) was designed by the Evaluation Team. Through a random sample of households, the HHS aimed to establish baseline information on knowledge, attitudes and practices of community members in relation to IT at an early stage of the mouth project's life span. This statistically valid survey will be supplemented by focus groups to develop quantitative and qualitative baseline data against which future impact evaluation will be compared. Information generated by the HHS will feed into the program planning process. Project activities can be more precisely planned based on the information generated by the survey. The HHS will be repeated in January 2003 and January 2004 to assess the achievements of the program, monitor change over the course of the program implementation, and evaluate the impact of the program. The quantitative survey was administered by the Palestinian Economic Policy Research Institute (MAS), a Palestinian research organization. Survey data was analyzed by the evaluation team. The qualitative survey was supposed to be implemented in August 2001. The process was delayed because the design, the bidding and the negotiation process with research institutions took longer than expected. In addition, the escalation of violence resulted in the delay in training of fieldworkers and initiation of fieldwork. The qualitative survey was delayed as a result of the massive Israeli invasion of the Palestinian territories in March 2002. The report that combines the results of the quantitative and qualitative surveys, scheduled to be finalized in May 2002, was also delayed. The survey included two questionnaires: International Youth Founddon Annual Evaluation Report, April 1,2001-Mmch 31,2002 1. A questionnaire aimed at gathering basic information on the household (socioeconomic, demographic and educational information). In addition, the questionnaire included basic information related to ownership of computers and intemet connection by the household. Total completed questionnaires: 1,008. 2. A questionnaire that targeted a randomly selected sample of youth 10-24 years of age and aimed at assessing their KAP on a range of indicators related to IT. Total completed questionnaires: 637. Following is a summary of the main findings of the HHS. The complete report of the qualitative survey will be completed at a later stage. n General information about the household: The average household size in the village clusters (7.9 persons per household) is larger than the average for other rural areas in Jenin and Nablus (5.9 persons per household9). Educational level of members of households is relatively low. Around 55% of family members 10 years and above have completed 6 years of education or less. Only 8% of family members 10 years and above have obtained a degree above high school. 0 Only 55.5% of participants in the labor force were employed/ working when the survey was carried out. This reflects the high level of unemployment in the villages. Only 46% of heads of households were employed. 6.5% of the households are headed by females. 12.7% of the households owned computers; 46.5% of which were used computers when they were purchased. 29.1% of the households have telephone lines. 0 0.9% of the households are connected to Intemet. b. Information about youth 10-24 years: 0 The percentage of youth 10-24 in the village clusters (34.1%) is slightly higher than the rest of rural areas in Nablus and Jenin (31.5%'4. Palestinian Central ~ureau of Statistics, Palestinian National Census, 1997. lo bid. 23 International Youth Foundation Annual Evaluation Reporf April I, 2OOI-Much 3I, 2002 78.6% of male youth 10-24 years and 66.6% of the females are students. 59.9% of male youth 10-24 years and 39.1% of female youth use computers. 52.6% of students use computers in comparison with 25.1% of out-of￾school youth. Out of those youth who use computers, only 26.6% have daily access to computers. 3.9% of youth 10-24 years of age use Internet, 88% of whom use internet for less than 7 hours a week. Out of the total number of youth who use Internet 40.8% have e-mail addresses and 47.1% know about chatting. A2. Results Planned but not Achieved. Infrastructure. Regional IT Center. The establishment of a Regional IT Center PC) represents a first step towards the implementation of community IT activities. The Center is envisioned as a base for computer training activities, vocational training program, out of school program, etc. The Center is supposed to serve not only residents of Silet ad-Dhahr but also residents of surrounding villages. During the third quarter, the construction of the Center began even though there is a shortfall of funds needed to complete the project that will be requested in the enhancement proposal or the reallocation of budget. The best four conttactors who participated in the renovation1 construction of school labs were invited to submit their bids for the establishment of the regional center according to the specifications that have been prepared by the project engineering consultant firm. Closed bids were received and reviewed by a committee of the IT4Y staff. The winning bidder was contracted. It must be highlighted that delay in establishing the RITC has led to a serious delay in implementing the vast majority of the community activities. Activities at RITC for the community at large (IT and local business growth); for the local council members; and for the JCSC had been postponed due to the delay in the establishment of the RITC. The following is a summary of activities not implemented: Hiring, orienting and training the manager and staff of the RITC; Znttrnational Youth Foundmion Annual EvaIuation Report, Aprill, 2001-March 31,2002 Hiring a youth specialist to assist the staff of the center in finalizing all the legal issues related to the establishment of the IT Youth Club, developing internal policy and program documents (mission, vision, goals, bylaws, etc.), development and implementation of DIF' and monitoring and evaluation; Implementing various training activities for out-of-school youth, municipal staff, adult community members, etc.; Implementing the vocational training program. Classroom Internet Connections. According to plan, all school labs and RITC should have been connected to the Internet by the end of year 1. In the absence of effective telephone networks, the IT4Youth program examined various options for installing Internet connections including wireless connections. This was not completed because it took more time than anticipated to complete the installation of the labs. It is therefore recommended that the program give more attention to this matter in the near future. Mobile Units. It has been planned to establish two computer mobile units. The objective of the establishment of those units was to increase the access of rural youth living in remote are& and smaller villages toIT. The establishment of the mobile units has been cancelled after consultation between the IT4Youth program staff, IYF, and the CTO of USAID for the following reasons: The current political complications have not only resulted in the isolation of Palestinian regions from one another, but also the isolation of each to~vn or village. Most of the program villages are isolated from one another by Israeli military check points. Operating the mobile units under such conditions would be an impossible task. Time and effort of coordinating with the Israeli military is too demanding and the security of the vans and safety of their drivers and staff can not be ensured. Most of the paved roads that connect some of the villages with each other were severed by the Israeli army. Existing roads are rough agricultural roads. If used on regular basis, the dust and roughness of the roads would damage the computers. h light of the above two factors and provided that there is a shortage of fund that delayed the construction of the RTIC, a more important componenf it was decided to cancel the mobile units project and redirect those funds (1) to the RTIC, and (2) increase the computer to student ratio in the school computer labs to internationally acceptable level to improve Student access and learning outcomes. Iniemational Youth FoundPhon Annual Evaluation Repori, April I, 2001-March 31,2002 A3. Pace of Implementation. In light of the complicated political situation, the pace of implementation of the project is remarkable, eipecially the construction/ renovation of computer labs. Nevertheless, some tasks were delayed: The equipping and setting up labs with furniture, computers, and the set up of labs, connections, and final checking of computers took more time than expected, resulting in a two to three week delay after the start of second semester before they became operational. The installation of software also took much longer than planned resulting in a two to three week delay after the start of the second semester. The investigation of the options for installing wireless Internet connections in school labs took more time than expected. B1. Unintended Results. A number of unintended results were identified: Positive: The project played an important role in constructing a road that connected Bazaria (Nablus cluster) with Silet ad-Dhahr (Jenin cluster) without passing by Israeli check points. PECDAR has agreed in principle to support a project to pave two kilometers of road between Bazaria and Silet ad-Dhahr. When implemented, this project will enhance access to the Regional IT Center. The project funded 13 computerslclassroom or a computer to student ratio of 1:4. However, computers were obtained at prices that allowed for more computerslclass room thereby reducing the ratio from 1 :2. Such a step should contribute to improving the quality of computer training. Based on a request from the IT4Youth project and the Evaluation team, some of the members of the MC were trained by MAS and participated as fieldworkers in the HHS. According to the four members of the MC who participated in the fieldwork, the experience has given them more information about the project, the current level of knowledge, attitudes and use of computers in their villages and helped them in establishing contacts with various community members. Such an experience would help them in implementing their tasks in future. Contact has been initiated with a USAID supported youth employment program, managed by PALTRADE. This project co-funds the salary of newly graduated students for 6 months. USAID covers 50% and the Palestinian company covers the other 50%, with a commitment f?om the Palestinian company to employ the person for at least one year. Members of the MC in Nablus and Jenin brought application forms from PALTRADE and Infernational Youth Foundation Annual Evaluation Report, April I, 2001-2Mmeh 31,2002 encouraged local youth to apply. Ten youth applied and four of them were short-listed and called for interviews. The project was temporarily frozen following the Israeli invasion of the Palestinian areas. Students were very interested in the new computer program. As a result, some of the IT teachers decided to teach additional computer classes &om 7:00 am to 8:00 am (before the school regular schedule begins) or after school hours or both. As a means to promote the awareness of private and non-profit organizations of the conditions in the targeted villages, each viIlage council started the preparation of a "Village Profile". The Profile will include information on location, area, classification of each village (areas A, B or C), population characteristics, local government unit, economic background, facilities, inffastructure and senices, community-based organizations, NGOs active in the villages, etc. Due to strict closures, the IT4Youth program decided to carry out the construction and renovation work through a decentralized approach. This included hiring construction contractors in both Nablus and Jenin rather than one contractor for bbth areas. This strategy has been extended to hiring local craftspeople that handle much of the site construction. The positive outcome was the increased feeling of ownership and local accountability and the provision of needed &nployment for construction workers who are unemployed due to the closures. Negative Nonenoted. C1. Factors that hindered program implementation. Several factors have hindered program implementation or limited its effectiveness. Most of those factors are external. Of primary concern is the current political situation between Palestine and Israel. Random and prolonged Israeli closures that prevent access to the geographical areas of the program, and the violent invasion of the cities and villages where IT4Youth staff and partners live and work and where the IT4Youth project is operating, have disrupted the implementation of the program. Since the initiation of the program, the political situation has been steadily worsening. This has made the movement of people and goods difficult and time consuming impacting most aspects of the program, especially communication and transportation. Following is a summary of difficulties that the IT4Youth has encountered during the implementation of the program: The closure of Palestinian areas and the isolation of various towns and villages &om one another have hampered the implementation of many the program's planned activities. For example, some of the IT4Y st& have not Zniernaiional Youih Foundation Annual Evaluation Reporf April I, 2001-March 31,2002 been able to reach their offices or move out of their houses. It became difficult for members of the JCSC and MC to meet on regular basis. Coordination mechanisms between the two JCSC and MC became difficult Movement of goods to the project became impossible. Training activities, fieldwork and follow up of the program were stopped. According to the agreement with the IT4Y project, one of the responsibilities of the Futurekids@ program is to cany out monthly monitoring visits to each teacher to follow up progress, assess the implementation process and provide teachers with support and guidance. This was not possible because of the Israeli closure and restrictions on movement, impeding program implementation. The delay of hiring a local contractor by the local council of Silet ad-Dhahr to complete the internal electricity networks has delayed the initiation of teaching in Silet ad-Dhahr boys school. The rehabilitation of the electricity network in al-Attarah village has led to a shortage of power for an extended period of time. This has lead to a delay in implementing the program in the village school. The delay in establishing the RITC in Silet ad-Dhahr has postponed the implementation of a number of community activities linked to it. Various activities are dependent on having that facility available and equipped. A number of training activities were not possible due to closure of territories. This has interrupted the planning process and delayed the implementation of activities. Contacts by IT4Youth program staff with members of the JCSC and MC reveal that some members continue to be unclear regarding the respective roles and responsibilities of the two committees. In addition, it became apparent that coordination mechanisms between JCSC and MC need further improvement. The restructuring of the MCs, especially the MC in Nablus, has interrupted the work of the committee. It will take time before the new members can carry their responsibilities effectively. The implementation of the program has been slowed down as a result of various bureaucratic regulations, especially during the first phase of implementation. Agreements that specifies scope of work, roles, responsibilities and obligations, coordination mechanisms should be signed between the IT4Youth program and the MOE. C2. Program areaslactivities that should be modified and why. The program has been successful in building an infrastructure and experience in schools and less successful at the community level due to the delay in establishing the RITC. During the second year, priority should be given to the following: International Youth Foundmion Annual Evaluation Repo* April I, 2OOI-Much 3I, 2002 Improving the capacity of IT4Youth staff, ~uturekids@ staff, and MOE staff to monitor and report on program implementation. Developing quality control policies and procedures. At present more effort needs to be expended to ensure quality control of program implementation. Developing a decentralized model of implementation. This is especially important in light of the continued closure, restrictions on mobility and isolation of areas. - Defining the various roles of the JCSC and the MC. Dl. Opportunities that have arisen. Following is a summary of the main opportunities that have arisen as a result of the implementation of this program: The program has led to more interest among local councils and local community to develop more activities for youth. One of the important ideas being explored by Silet ad-Dhahr Village Council is the possibility of establishing a branch of a1 Quds Open University in Silet ad-Dhahr. The opening of such branch would have important impact on the lives of youth, not only in Silet ad-Dhahr, but in all the surrounding villages. Youth, especially females, would expand their chances to pursue higher education. In addition, the presence of the university branch in the region would have positive effects on the project as the academic environment would add to the importance of the IT project. Negotiations have been temporarily frozen because of the current political crisis. The participation of women in the MC represents an important opportunity that should be expanded. The inclusion of women, which received some resistance fiom the local community in the beginning, then acceptance and support, would give more access to females in the villages. This approach would help consolidate and expand program benefits to more females. The involvement of the MOE computer teams in planning and implementing the project and the participation of those teams in training activities will help in building the capacity of the Ministry to implement the project effectively and to expand the project to other areas in the future." E. Main findings, lessons learned and recommendations. Following is a summary of the main findings, lessons learned and recommendations for next year: l' Though not a part of the original plan, the MOE has volunteered its staffand expressed enthusiasm for the program. This will build even more ownership for the IT4Youth initiative among the MOE and further build its capacity to manage similar programs. Znternahbnol Youih Foundonbn Annual Evaluation Report, April I, 2OOI-March 31,2002 1. Stabilization of the Implementation of the R4Youth Program in Schools. The program has been successful in laying out the foundations for the implementation of the program in schools (labs constructed, equipment provided, teachers trained, etc). Now as the program moves from inffastructure development to service delivery, more attention should be given to improving and upgrading quality of implementation and developing policies and procedures that would stabilize program implementation. The following is recommended: Continue to develop clearer and more effective coordination mechanisms with the MOE. This includes well-defined plans that specifies roles, responsibilities, obligations, and time frames, for each party etc. Improve the follow-up and field monitoring of the program in schools. This includes developing quality control policies and procedures related to the implementation of the ~uturekids~ cumculum, maintenance of equipment, training of teachers, field reporting, etc. 2. Decentralization. The achievements of the program have benefited from the decentralized implementation approach. The establishment of the JCSC and MC and decentralized bidding have all contributed to increased local participation and promotion of the project. Further steps will be needed in future to continue to increase a sense of local ownership to the project. In light of the continued closure of the Palestinian territories, restrictions on movement, isolation of villages and towns, more attention should be given to building local capacity in each village to implement the project effectively. 3. Strengthening the Community Component of the Program To overcome the delay in the community component of the program, special attention should be given to the following: Construction of the RITC; Capacity building of the JCSC and MC with special focus on clarifying roles, responsibilities and obligations, improving planning and implementation capacity, upgrading technical skills and knowledge; Training strategy and capacity building efforts should be upgraded based on task-oriented competency based training approaches. 4. Monitoring and Evaluation Building sound performance monitoring and evaluation systems at an early stage of program implementation has been very useful in improving the quarterly planning process and the utilization of information. Program should give special attention to improving the monitoring and evaluation capacity of partners (Futurekids@, MOE and JCSC/MC). International Youth Fowrdaiion Annual Evaluahbn Report, April I, 2OOI-MPrch 31,2002 Also, the program's monitoring and evaluation procedures between the evaluation team and IT4Youth program staff should continue to be improved in order to better capture data on progress and impact. IV. Financial Report (see attached) International Youth Foundaiz'on Annual Evaluation Report, April I, 2001-March 31,2002 Annex 1 Main Results of the Household Quantitative Survey Variable Information about the Household I Average household size 7.9 persons Ihterate Able to read and write Elemenmy school (Grades 1-6) Secondary school (Grades 10-12) More than secondary 1 8% I Labor force status of family members 10 years and above Fdy members 10 years and above in labor force 32.2% I Males 10 years and above in labor force 57.6% Females 10 years and above in labor force 4.7% Employed Labor force participants who are employed (n=1766) 55.5% Males in labor force who are employed (n=1643) 54.1 % Females in labor force who are employed (n=123) 74.6% Heads of households Females 6.5% Labor force status of heads of households Employed 46% 1 Unemployed 38.8% I Students 0.5% I Housewives 4% Others 10.6% Household and IT Households that own a computer 12.7% Condihon of the computer when purchased Used 46.5% Do not how 1% Households that own printers 32.3% Main reasons for purchasing a computer Children education 36.5% Children recreation 3.7% Internet 2.5% 1 1 ~ Adult education 33.3% I ! - Adult recreation il 7% I 1 Reasons for not purchasi mg a computer I Do not how to use the computer 30.5% Can not afford the cost 57.7% Time consuming 1.8% Useless 7.6% I I Damages the health (sight) 0.1% Znternahbnal Youth Foundation Annual Evaluation Repoe April I, 2001-Much 31,2002 18-22 employed 55.8% I 22-24 employed 1 26.8% ~ School attendance of youth 10-24 I Males in school 78.6% Females in schools 66.6% ! Male youth 10-24 years who use computers 51.9% Female youth 10-24 years who use computers 39.1% 1 Distribution of youth 10-24 who use computers by software used 1 Games 34.3% Educational 25.6% 1 Word processing 24.8% Financial 2.2% Spread sheets 0.7% Data bases 1.5% Internet 4.8% I Use of computer by age 10-14 years 41.7% 15-17 years 35.9% 18-24Years 65.1% Use of computer by youth 10-24 years by school attendance Students using computers 52.6% Out of school youth us in^ computers 25.1 % Have daily access to computers 26.6% Preferred screen interface English 11.6% Zntern&nal Youth Foundaiiin Annual Evaluation Report, April I, 2001-March 31,2002 Arabic Youth 10-24 who use internet by age 10-14 years 15-17 years Studies 1 43.8% 1 88.4% 1 3.8% 2.2% 4.4% 18-24Years Main reasons for using internet Work Increase general knowled~e Recreation I 8.3% 1 Other 11.5% Average weekly Internet use hours by youth who use internet 1 I Less than 7 hours 88% 7-20 hours 5.1% 20-40 hours 6.9% Availability of e-mail address among youth who use internet 40.8% 1 Knowledze of chatting among youth who use internet 47.1% 12.9% 1 I 3.8% 1 32.6% I Reasons for not using internet Internet use is costly 8.9% Accessible computer not connected to internet 15.5% Do not know how to use internet 67.3% Do not have free time 0.9% Not interested 4.2% Computer has advantages 81.3% 1 Computer has disadvantages 27.7% 1 Internet has advantages 46.1% I Internet has disadvantages 17.9%