Bibliographic Information: USAID Award Number: AID 294-A-12-00006 USAID Program Area and Element: Program Area A12/ Element A055 Education / Strategic Objective SO 13 Author: Leadership and Teacher Development Program Contractor Name: AMIDEAST- Leadership and Teacher Development Program Sponsoring USAID Agency Operating Unit: Education Development Office Date of Publication: November 1, 2015 Language of Document: English Contact Information: Dr. Said Assaf PhD , Chief Of Party : sassaf@amideast.org DISCLAIMER: This publication was produced for review by the United States Agency for International Development (USAID). It was prepared by the USAID Leadership and Teacher Development Program (LTD). The views expressed in this publication do not necessarily reflect the views of USAID or the United States Government. LEADERSHIP AND TEACHER DEVELOPMENT PROGRAM Leadership and Teacher Development Program (LTD) November 1, 2015 This publication was produced for the United States Agency for International Development. Evaluation Report Impact of the Leadership and Teacher Development Program on Improving the Quality of Education in the West Bank and Gaza Fiscal Year 2015 Evaluation Report Impact of the Leadership and Teacher Development Program on Improving the Quality of Education in the West Bank and Gaza Fiscal Year 2015 November 1, 2015 The Leadership and Teacher Development Program Cooperative Agreement No. 294-A-12-00006 DISCLAIMER This report is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The Contents are the responsibility of AMIDEAST and do not necessarily reflect the views of USAID or the United States Government. i CONTENTS Acronyms ...........................................................................................................................................................................................................ii Executive Summary......................................................................................................................................................................................... 3 Section 1: To what extent did LTD contribute to empowering principals to promote effective schools? ...........................10 I. Survey of Principal Effectiveness...............................................................................................................................................11 II. Results of the Teacher Survey of Principal Effectiveness ....................................................................................................17 III. PMP Indicator Results...................................................................................................................................................................19 IV. Mixed-Methods Study of School Improvement Teams........................................................................................................20 V. Recommendations for Improvement of Leadership Training ............................................................................................42 Section 2: To what extent did LTD contributed to improving the capacity of teachers to enact learner-centered approaches to teaching and learning? ......................................................................................................................................................44 I. Survey of Classroom Engagement.............................................................................................................................................45 II. Survey of Teacher Effectiveness ................................................................................................................................................ 53 III. Recommendations for Improvement of Leadership Training ............................................................................................63 Section 3: To what extent did LTD contribute to enhancing the capacity of principals and teachers to utilize technology for improving leadership, instruction, and professional development? .....................................................................65 I. Robust In-Kind Assistance for Technology Infrastructure ...................................................................................................65 II. Detailed Results..............................................................................................................................................................................65 Section 4: To what extent did LTD contribute to improvement in Student Achievement? .....................................................68 I. Results for Cohort I Post-Post Tests..........................................................................................................................................68 II. Results for Cohort II Post Tests ..................................................................................................................................................69 Section 5: To what extent did LTD contribute to sustaining the capacity of teacher educators to apply effective training approaches and practices? ..........................................................................................................................................................................71 I. Quality of Trainers’ Performance after TEEP ..........................................................................................................................71 II. Findings............................................................................................................................................................................................72 Section 6: To what extent did LTD contribute to building the capacity of leadership, instructors and students to engage in planning and action leading to improvement in organization functioning and pedagogical practice in the Faculty of Education of Al-Azhar University, Gaza? ................................................................................................................................................74 I. Strategic Plan ..................................................................................................................................................................................74 II. Teacher Educator Enhancement Program (TEEP)..................................................................................................................76 III. PCELT................................................................................................................................................................................................. 80 ANNEX A: Principal Effectiveness Survey (Principal Form) ..............................................................................................................87 Annex B: Teacher Effectiveness Survey (Teacher Form) ...................................................................................................................92 Annex C: Classroom Engagement Survey (Student Form) .................................................................................................................. 2 ii Acronyms Accreditation and Quality Assurance Commission AQAC America-Mideast Educational and Training Services AMIDEAST Assessment and Evaluation Department AED Automated Directives System ADS Chief of Party COP Directorate of Planning DP Directorate of Supervision and Qualifications DSQ District Leadership Team DLT Geospatial Management Information System Geo-MIS Leadership and Teacher Development LTD LTD/NIET Joint Working Group JWG Leadership Diploma Program LDP Middle East and North Africa MENA Ministry of Education and Higher Education MoEHE Model Schools Network MSN Monitoring and Evaluation M&E Monitoring and Evaluation Task Force METF National Institute for Educational Training NIET National Teacher Education Strategy TES Operational Plan OP Professional Certificate in English Language Teaching PCELT Quality Assurance Unit QAU School Improvement Team SIT Supervision Diploma Program SDP Teacher Educator Enhancement Program TEEP United States Agency for International Development USAID University of Massachusetts, Amherst UMass 3 Executive Summary Introduction This evaluation report presents findings of evaluation research aimed at describing evidence that LTD is contributing to improvement of the quality teaching and learning in the West Bank and Gaza among the 144 schools of Cohort II during the 2014-2015 school year, and improving the quality of pre-service teacher education at Al-Azhar University, Gaza. Six major research questions frame the research and analysis. What evidence do we have that LTD contributed to: 1. Empowering principals to promote effective schools? 2. Improving the capacity of teachers to enact learner-centered approaches to teaching and learning? 3. Enhancing the capacity of principals and teachers to integrate technology to improve leadership and classroom instruction. 4. Improving student achievement? 5. Sustaining the capacity of teacher educators to apply effective training approaches and practices? 6. Building the capacity of leadership, instructors and students to engage in planning and action leading to improvement in organization functioning and pedagogical practice in the Faculty of Education of Al-Azhar University, Gaza? Design and Method of the Study To investigate these questions, LTD used a mixed-methods (quantitative and qualitative), quasi￾experimental design that incorporates baseline and endline data from representative and random samples of LTD’s diverse groups of beneficiaries—principals, teachers, teacher educators, and students. Findings of the evaluation will be shared with AMIDEAST/LTD and with key partners in the Ministry of Education and Higher Education, and Al-Azhar University, Gaza, with the aim of improving and/or developing policies, strategies and approaches that will enhance the quality of professional development provided to principals, teachers, and teacher educators. FINDINGS 1. EMPOWERING PRINCIPALS TO PROMOTE EFFECTIVE SCHOOLS 1.1. The Leadership Diploma Program contributed substantially to the improvement of principals’ leadership effectiveness.  By the end of the Leadership Diploma Program, 82% of principals of Cohort II schools rated their leadership competencies as effective across all seven domains of principal effectiveness. Overall there was an impressive 30% improvement in principals’ reported competency levels in all domains by the end of the training program.  The most impressive improvement was in planning school improvement (72% change), followed by school/community relations (25% change), technology (25% change), instructional leadership (24% change), assessment (23% change), school environment (24% change), and management of resources (20%).  These findings are corroborated by results of teachers’ evaluations of their principals’ performance using the same tool. 4 1.2. LTD contributed to the capacity of SIT members, and in particular the performance of principals, to work collegially and collaboratively in developing their school’s vision and mission, establish strategic goals, and prepare an implementation plan.  Eighty-eight percent of SIT members surveyed rated as high the quality of their team’s performance of tasks required to develop their school’s vision and mission, establish strategic goals, and prepare an implementation plan.  Ninety percent of SIT members surveyed agreed that team members exercised cooperation and collaboration in developing in the school improvement team.  Ninety-four percent of SIT members surveyed agreed that the principal was effective in supporting and managing collaborative work among SIT members in developing the SIP. 1.3. LTD contributed to the efficacy of SIT members to work collegially and collaboratively in implementing their School Improvement Plans.  Seventy-seven percent of SIT members surveyed rated their performance in monitoring the implementation of their plans.  Eighty-four percent of SIT members surveyed rated their performance in deploying and managing material and human resources for the effective implementation of their plans.  Ninety-one percent of SIT members surveyed agreed that as a team they exercised cooperation and teamwork during implementation of the plan.  Ninety-three percent of SIT members surveyed agreed the principal exercised shared leadership in supporting and managing collaborative work among SIT members. 1.4. As a result of capacity-building for SITs and in-kind assistance to schools, LTD contributed to empowering the SITs to make substantive improvements to the overall learning environments of their schools by…  Creating a more child-friendly climate by making major improvements to the physical conditions of schools.  Fostering the widespread use of ICT in classroom instruction.  Strengthening internal and external relations among students, staff, and parents, especially by leveraging the evidence-based SIP to secure donations from the local community towards achieving targets for school improvement.  Aligning the school vision and mission to its goals and targets for enhancing students’ learning. 2. IMPROVING THE CAPACITY OF TEACHERS TO ENACT LEARNER-CENTERED APPROACHES TO TEACHING AND LEARNING 2.1. LTD contributed substantively to building the capacity of teachers to enact learner-centered approaches and strategies and to prepare their students with 21st century learning skills (critical thinking, creativity, communication, and collaboration).  LTD students are more likely to agree (65%) than non-LTD students (55%) that their schools provide a positive learning environment; non-LTD teachers, to the contrary, are slightly more likely (60%) than their LTD peers (55%) to think so. 5  LTD teachers (88%) and their students (68%) are more likely to describe their classroom environments as learner-centered than their non-LTD peers (81% and 54%) respectively.  LTD students (68%) are more likely to agree that their teachers are building their 21st century learning skills than their non-LTD peers (56%).  LTD teachers (87%) are more likely to agree that they are building their students’ 21st century learning skills than their non-LTD peers (80%). In particular, LTD students are more likely to make real-world connections to what they learn in class; do project-based learning; feel encouraged to offer their own opinions or ideas; and, participate in group work.  Students of LTD teachers are less likely than their non-LTD peers to engage in misbehavior than their non-LTD peers. 2.2. Through its technical support of NIET’s delivery of LTD’s training curriculum for teacher qualification, LTD contributed to the growth of teachers’ competencies across the seven domains of the Ministry’s standards for effective teaching. By the end of their LTD training, teachers in the program improved their capacity to:  Facilitate student-centered teaching and learning by 24%. LTD teachers design learner￾centered teaching and learning to foster students’ active engagement in meaningful learning and assessment activities.  Design effective educational materials (lesson plans) and resources by 27%. LTD teachers plan lessons and units of instruction that take into account varieties of resources both inside and outside the classroom to improve teaching and learning.  Create a safe and effective learning environment by 23%. LTD teachers create a classroom environment that is child-friendly, treats students equally, and fosters creativity.  Monitor and evaluate the teaching and learning process by 25%. LTD teachers use a variety methods for formative and summative assessments of student learning.  Provide guidance and direction for learners by 17%. LTD teachers are prepared to deal sensitively and appropriately in addressing students’ cognitive, physical, emotional, and social well-being and needs.  Seek continuous professional development by 26%. LTD teachers are self-directed in pursuing their own professional learning through reflective practice and inquiry, and by participating in professional learning communities.  Encourage cooperation with stakeholders in the community by 22%. LTD teachers develop partnerships with peers, families, and community organizations to provide students with authentic contexts in which to transfer their learning and to receive additional support for their learning needs.  LTD’s strong emphasis on technology in its teacher trainings is a contributing factor to improving teachers’ competencies. A statistically significant difference in the teacher competency scores was found between competencies most associated with teaching and learning and the teachers’ use of the Internet to search for teaching resources and for professional development. 6 2.3. LTD contributed to building the capacity of teachers and principals to promote the values and conditions that foster a child-friendly school. Based on a set of indicators measuring behavior, LTD students reported more positive school behaviors than their non-LTD peers.  Students of LTD teachers were less likely than their non-LTD peers to have hit a fellow student (34% vs. 49%)  Students of LTD teachers were less likely than their non-LTD peers to have been hit by another student (23% vs. 29%)  Students of LTD teachers were less likely than their non-LTD peers to have been hit by a teacher (43% vs. 64%)  Students of LTD teachers were less likely than their non-LTD peers to have skipped school (12% vs. 29%)  Students of LTD teachers were less likely than their non-LTD peers to have had their parents called to school because of misbehavior (14% vs. 20%). 3. ENHANCING THE CAPACITY OF PRINCIPALS AND TEACHERS TO INTEGRATE TECHNOLOGY TO IMPROVE LEADERSHIP AND CLASSROOM INSTRUCTION. 3.1. LTD’s provision of technology resources and training to both school leadership and teachers contributed to improvements in school effectiveness.  LTD contributed to a 27% growth in the principals’ capacity to use technology towards improving their leadership in school management, instructional supervision, and community relations.  Based on students’ assessment of their teachers’ classroom practices, LTD teachers are 10% more likely than non-LTD teachers to use technology in the classroom.  Teachers’ use of technology to search online for teaching resources grew by 24%, which is a strong indicator that their LTD training boosted their capacity to take responsibility for their own professional learning. 4. IMPROVING STUDENT ACHIEVEMENT 4.1. LTD contributed somewhat to improvements in student achievement in two three out of four core academic: Arabic, English, and science.  A post-post study in May 2015 of Cohort I schools found that LTD students scored higher in two out of four tests of achievement compared to controls. LTD scored substantially higher in English and a little higher in math, but scored slightly lower in Arabic and science. These results tend to mirror the results found one year earlier in May 2014.  A post-study in May 2015 of Cohort II schools that LTD students scored higher in three out of four tests of achievement compared to controls. LTD students scored higher in Arabic, English, and science and scored just slightly lower in Math than the controls. 7 5. SUSTAINING THE CAPACITY OF TEACHER EDUCATORS TO APPLY EFFECTIVE TRAINING APPROACHES AND PRACTICES 5.1. The performance of NIET’s trainers for the Leadership Diploma Program were evaluated by 144 in-service principals enrolled in the program. The results exceeded by 11% the benchmark for effective performance across six training competencies: using a variety of learning activities; balancing theory and practice; practicing active learning and learner-centered techniques; employing educational technology and new media; facilitating critical thinking; and helping trainees to transfer their learning to the real-world context of their workplace. 5.2. The performance of university consultants who trained in-service teachers for the Teacher Qualification Program delivered by NIET were evaluated by over 700 teachers enrolled in the program. The results exceeded by 8% the benchmark for effective performance across NIET’s six training competencies. 6. BUILDING THE CAPACITY OF LEADERSHIP, INSTRUCTORS AND STUDENTS TO ENGAGE IN PLANNING AND ACTION LEADING TO IMPROVEMENT IN ORGANIZATION FUNCTIONING AND PEDAGOGICAL PRACTICE IN THE FACULTY OF EDUCATION OF AL-AZHAR UNIVERSITY, GAZA 6.1. LTD contributed to the successful capacity-building of leadership of the Faculty of Education to carry out systematically the process of strategic planning based on international standards of best practice for higher education institutions.1 Members of the Strategic Planning Steering Committee of the Faculty of Education demonstrated their abilities to:  Collaborate in producing a viable and future-oriented vision, mission and goals.  Commit to shared leadership during all stages of the process.  Build a consensus among the Faculty leadership and teaching staff towards embracing the vision and mission of the Strategic Plan.  Devote the necessary scope and depth of analysis involved in the process.  Assess the relationship of the plan to budget, human capacity, and local realities.  Ensure the inclusion of all stakeholders in the development and implementation of the plan.  Commit to ensuring the ongoing review, evaluation and adjustment of the plan. 6.2. The TEEP program contributed substantively to building the capacity of 12 full-time members of the Faculty of Science to engage in planning and action leading to improvement in learner￾centered pedagogical practice in pre-service teacher education courses.  In a survey at the end of the TEEP training, the participants as a group rated the impact of the program on their teaching practices as 4.71 out of 5, which is the equivalent of a 94% approval rating.  TEEP enabled the participants to rely less on didactic instruction and more on student￾centered and active learning activities. 1 Adapted by Joseph B. Berger from A Guide to Strategic Planning for African Higher Education Academic Units by Fred M. Hayward and Daniel J. Ncayiyana with Jacqueline E. Johnson (2003), Center for Higher Education Transformation, Johannesburg, South Africa. 8  TEEP built the capacity of the participants to engage in individualized professional learning by practicing the inquiry cycle of action research, learning circles, and monitoring growth by keeping a portfolio of professional practice. 6.3. LTD contributed substantively to building the capacity of pre-service teachers of English to engage in planning and action consistent with learner-centered pedagogical methods and techniques. Based on their experiences during practice teaching and in the formal or non-formal contexts in which they are teaching school children in Gaza, PCELT graduates affirmed that the PCELT training enabled them to to:  Improve their perceptions of students and their learning: They believe that PCELT increased their sensitivity to the emotional, cognitive, and social needs of learners.  Adopt positive values and attitudes about teaching: They are convinced that PCELT developed their professionalism and attribute this to PCELTs’ emphasis on self-reflection, peer observation, and collective feedback focused on problems of practice.  Acquire a wide variety of useful and effective tools and approaches: They appreciate not only the richness of the PCELT toolkit but also for equipping them with a variety of strategies for selecting and applying the tools appropriately in different contexts with students.  See evidence of improved student learning as a result using PCELT methods and techniques for planning, instruction, and assessment: They credit PCELT for helping them to increase their students’ motivation to learn; improve their speaking fluency and listening comprehension; and engage them in collaborative activities resulting in larger participation and active learning.  Spread their PCELT experience and learning to others: They have been sharing their PCELT skills and materials with peers and other educators, including classmates in their pre-service program; cooperating teachers during their practice teaching in schools; and with relatives and friends. RECOMMENDATIONS FOR IMPROVEMENT OF LEADERSHIP AND TEACHER TRAINING 1. Based on the results of both quantitative and qualitative research on principal effectiveness, the Leadership Diploma Program delivered by LTD’s partner, NIET, the leadership effectiveness of principals can be further enhanced. For this to happen, the following recommendations are suggested: 1.1. Continue to build the principal’s capacity to involve parents, teachers, and other stakeholders of the school community in discourses and decisions on improving the school; in particular, attention should be given to learning how to improve the flow of information using technology to communicate with stakeholders. 1.2. Improve the principal’s capacity to involve parent members of the SITs in planning and collecting data for the school self-assessment. 1.3. Empower the principal with greater discretion to manage the workload and scheduling of SIT tasks and meetings so as to accommodate the limited free time that school staff and parents have during a typical workday. 1.4. Build the principal’s skills in giving constructive feedback and in managing differences of opinion in order to reach a consensus in decision-making. 9 1.5. Emphasize the responsibility of the principal and the SIT to systematically monitor and document the implementation of the SIP. 1.6. Continue to build the capacity of the principal and the SIT in managing resources intended to support the quality of teaching and learning and improve learning outcomes. 1.7. In light of the West Bank and Gaza’s chronically low-resourced education sector which, among other things, limits the capacity of a school to finance its annual SIP, the MoEHE should consider ways to empower principals to seek alternative revenue flows to fill budget deficits. 2. Based on the results of the principals’ assessment of their teachers’ performance on the Teacher Effectiveness Survey, the Teacher Qualification Training delivered by LTD’s partner, NIET, teacher can further enhance the quality of students’ learning. For this to happen, the following recommendations are suggested: 2.1. Facilitating student-centered teaching and learning would benefit by building teachers’ capacity to write learning outcomes that align with outcomes of the national curriculum and to involve students and other stakeholders in clarifying desired learning outcomes. 2.2. Designing educational materials and resources would benefit by building teachers’ capacity to improve creative learning and abilities by using a variety of teaching and learning resources; designing units and semester plans to make effective use of available teaching and learning resources inside the school and the larger community to support the curriculum. 2.3. Creating a safe and effective learning environment would benefit by building teachers’ capacity to involve students in the drafting school and classroom regulations. 2.4. Monitoring and evaluation of the teaching and learning process (i.e., assessment) would benefit by building teachers’ capacity to develop different assessment tools that fit the individual differences of students; to implement remedial learning strategies based on assessment results; to help students to use self-assessment; to design lesson plans to improve students' learning based on assessment results; and to use results of self-reflection to improve the process of teaching and learning. 2.5. Providing guidance and direction for learners would benefit by building teachers’ capacity to provide appropriate guidance in helping students to think about suitable career choices; consult with experts to find appropriate solutions to students with learning difficulties. 2.6. Seeking continuous professional development would benefit by building teachers’ capacity to take advantage of appropriate methods to achieve students' learning outcomes; to use evaluation results to identify training needs; and, to use action research to improve the teaching and learning process. 2.7. Encouraging cooperation with stakeholders in the community would benefit by building teachers’ capacity to encourage students to engage in local community-service learning; and to engage with families and community members to find appropriate solutions to learning difficulties facing students. 10 Section 1: To what extent did LTD contribute to empowering principals to promote effective schools? Introduction Enhancing the principals’ role in applying the values, strategies, and practices for shared leadership and supportive instructional supervision is a major goal of the LTD leadership training. By enacting the leadership competencies linked to the Ministry’s Standards for Effective Schools, the principal demonstrates his/her ability to plan and lead a participatory model of school improvement involving key stakeholders of the school community—teachers, students, and parents—as well as seeking supportive collaboration with other principals and with officials from the local directorate of education, particularly with members of the newly formed School Improvement Teams (DLTs). A mixed-methods design was adopted. Data were collected from principals, teachers, and parents using both survey and focus group methods. A survey of Principal Effectiveness, designed by NIET, was administered to all principals. So as not to rely on self-reported data from principals only, and to provide a more reliable assessment of school leadership, a random sample of LTD and non-LTD teachers filled out an identical survey to evaluate their principals’ performance. To better explore the question of shared leadership, LTD administered a survey to a random sample of school improvement teams (SIT) whose members include mainly principals, teachers, parents, and a smaller number of other school staff and members of local communities. Furthermore, LTD conducted focus groups with a purposeful sample of SIT members from both Cohort 1 and Cohort II schools, the aim of which was to explore the quality of leadership and teamwork of SIT members in planning and implementing their school improvement plans (SIP). Table 1. Data sources and collection methods to address research question #1 Data Collection Method Frequency of Data Collection Sample Survey: Principal Effectiveness (principal’s form) Baseline & Endline 106 Survey: Principal Effectiveness (teacher’s form) Baseline & Endline 616 School Improvement Team (SIT) Survey Baseline (Cohort I only) & Endline (Cohort I + Cohort II) Principals Teachers Parents 142 829 159 School Improvement Team (SIT) Focus Group Baseline (Cohort II only) & Endline (Cohort I + Cohort II) 25 23 19 11 I. Survey of Principal Effectiveness The survey asked the principals to rate the level of their competency in leadership based on the seven domains of effective schools: Planning, School/Community Relations, Managing Resources, Teaching and Learning, School Environment, Assessment, and Technology. The survey used a 4-point Likert scale the measures the extent that a competency was met, where 1 is “Not near expected level,” 2 is “Approaching…,” 3 is “Achieved…,” and 4 is “Exceeded the expected level.” Based on this scale, therefor, a mean of 3 would indicate an appropriate level of competency, while a 3.5 or higher suggests the level was surpassed. A. Detailed Results from the Principal’s Questionnaire By the end of their Leadership Diploma training, the principals reported they had met or exceeded the leadership competencies, making substantial improvement across all seven domains according to the Ministry’s Standards for Effective schools: Planning, School/Community Relations, Managing Resources, Teaching and Learning, School Environment, Assessment, and Technology (Figure 1). As seen in Figure 2, the domain of planning registered the most dramatic change, improving on average three times more than the others. KEY FINDINGS The Leadership Diploma Program contributed substantially to the improvement of principals’ leadership effectiveness. • By the end of the Leadership Diploma Program, 82% of principals of Cohort II schools rated their leadership competencies as effective across all seven domains of principal effectiveness. Overall there was an impressive 30% improvement in principals’ reported competency levels in all domains by the end of the training program. • The most impressive improvement was in planning school improvement (72% change), followed by school/community relations (25% change), technology (25% change), instructional leadership (24% change), assessment (23% change), school environment (24% change), and management of resources (20%). These findings are corroborated by results of teachers’ evaluations of their principals’ performance using the same tool. 12 2.02 2.96 2.92 2.92 2.97 2.94 3.13 3.48 3.70 3.63 3.62 3.69 3.62 3.77 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 Level of Leadership Competencies 72% 25% 25% 24% 24% 23% 20% 0 10 20 30 40 50 60 70 80 Percentage Change in Leadership Competencies Figure 1. Grand means for baseline and endline scores on the principal effectiveness survey Figure 2. Percentage change in competency levels at the completion of leadership training 13 B. Results of sub-competencies in each domain of leadership 1. Improvement Planning In assessing their leadership in planning school improvement (Table 2), the principals report big improvements in: involving parents (83% change) and teachers (54% change) in the improvement planning process; getting input from teachers (79% change) on developing the school’s mission and vision, and clarifying the mission to the school community (72% change); and, using data from action research (76% change) and the school self-assessment study (72% change) to improve the school’s performance. Table 2. Results for leadership in planning school improvement Planning Pre Post %Change q1.3 I involve parents in the school improvement planning. 1.71 3.13 83% q1.1 I involve teachers in the construction of the school's vision and mission. 2.04 3.66 79% q1.6 I use action research in the development of my work at the school. 1.87 3.30 76% q1.4 I clarify the vision and mission to the community. 2.06 3.55 72% q1.5 I build the school improvement plan based on results of the school self-assessment. 2.11 3.63 72% q1.2 I include teachers in the school improvement planning. 2.36 3.63 54% Grand Mean 2.03 3.48 72% 2. School/Community Relations In the area of school/community relations (Table 3), principals reported impressive growth in their capacity to involve stakeholders in decision making related to improving the school based on results (46% change); to encourage and model teamwork in school (39% change); and to engage the community in activities that support the teaching and learning processes (36% change). These results compare favorably with findings from the survey of School Improvement Teams (see page ___). Table 3. Results for leadership in school/community relations School/Community Relations Pre Post %Change q2.2 I involve stakeholders in decision making related to improving the school based on the results of the self-evaluation data. 2.48 3.61 46% q2.1 I encourage and model teamwork in school. 2.71 3.78 39% q2.4 I engage the community in activities that support the teaching and learning processes. 2.63 3.57 36% q2.5 I promote and model ethical behavior as expected by others. 2.85 3.67 29% q2.3 I communicate effectively with school staff. 2.97 3.79 28% q2.9 I encourage local organizations to support teaching and learning. 2.86 3.52 23% q2.8 I invite parents to visit the school to discuss their child's performance and progress. 3.10 3.64 17% q2.6 I show respect and appreciation for differences of members of the school community. 3.34 3.91 17% q2.10 I resolve conflicts between staff professionally. 3.17 3.69 16% q2.7 I treat school staff fairly (without bias. 3.50 3.82 9% Grand Mean 2.96 3.70 25% 14 3. Management of Resources Regarding their competency in managing the human (Table 4), material and financial resources of their school, principals reported big growth in their capacity to invest in the development of the skills and expertise of school staff (33% change); in identifying the needs of staff to support the teaching-learning process (30% change); and, in providing teachers opportunities for their professional development aimed at improving their teaching practices (24% change). Table 4. Results for leadership in managing school resources Managing Resources Pre Post %Change q3.1 I invest in the development of the skills and expertise of school staff. 2.76 3.67 33% q3.2 I work to identify the needs of staff to support the teaching-learning process. 2.83 3.67 30% q3.3 I provide teachers opportunities for their professional development aimed at improving their teaching practices. 3.06 3.80 24% q3.5 I support the use and maintenance of all learning resources at the school (library, sports rooms ...) 3.20 3.83 20% q3.7 I reach out to the local community to mobilize resources to help meet the needs of the school. 3.11 3.66 18% q3.4 I manage the school budget with transparency to address needs and priorities. 3.42 3.88 13% q3.6 I manage and accurately document the school's financial records. 3.54 3.88 10% Grand Mean 3.13 3.77 20% 4. Teaching and Learning In the area of instructional leadership of teaching and learning (Table 5), principals reported big growth in following up and supporting the professional development of teachers after the conclusion of trainings (31% change) and in being familiar with theories of teaching and learning and using this knowledge to help teachers improve their teaching (29% change). Competencies showing 25% or better improvement included the principal’s capacity to provide extra-curricular activities that support student learning per the improvement plan and curriculum; ensuring the positive development of students' attitudes and behavior; and supporting collaboration among subject teachers to work towards achieving integrated teaching and learning. Table 5. Results for leadership in the support of teaching and learning Teaching and Learning Pre Post %Change q4.6 I follow up and support the professional development of teachers after the conclusion of trainings. 2.75 3.61 31% q4.2 I am familiar with theories of teaching and learning and use this knowledge to help teachers improve their teaching. 2.66 3.42 29% q4.7 I ensure there are extra-curricular activities that support student learning per the improvement plan and curriculum. 2.88 3.63 26% q4.4 I ensure the positive development of students' attitudes and behaviors. 2.95 3.70 25% q4.10 I support the collaboration among subject teachers to work towards achieving integrated teaching and learning. 2.84 3.56 25% q4.1 I supervise the teaching-learning practices of teachers in their classrooms. 2.99 3.68 23% 15 q4.9 I implement policies that promote successful learning for all students, both strong and weak performers. 2.94 3.61 23% q4.3 I work with the school community to support student learning. 3.02 3.65 21% q4.8 I make available resources that to enable teachers to implement curricular and extra-curricular activities. 3.10 3.72 20% q4.5 I support the continuous professional development of teachers. 3.09 3.70 20% Grand Mean 2.92 3.63 24% 5. School Environment In the domain of creating a positive school environment (Table 6), four competencies stand out. Principals reported substantive growth in their capacity to involve teachers in decision making related to the school and its community (32% change); to delegate tasks to staff that are commensurate with their abilities (30% change); to develop policies that provide a safe and child-friendly school environment (30% change); and to encourage students to engage in volunteer and cooperative work (29% change). Table 6. Results for leadership in fostering a positive school environment School Environment Pre Post %Change q5.2 I involve teachers in decision making related to the school and its community 2.77 3.65 32% q5.1 I delegate tasks to staff that commensurate with their capacity. 2.77 3.59 30% q5.5 I develops policies that provide a safe and child-friendly school environment. 2.84 3.68 30% q5.3 I encourage students to engage in volunteer and cooperative work. 2.88 3.71 29% q5.8 I seek ways to reward and incentivize teachers in their work. 2.94 3.64 24% q5.6 I sets clear standards for the cleanliness of the school buildings and property. 2.97 3.65 23% q5.4 I build a learning environment that promotes respect and self￾esteem. 3.06 3.72 22% q5.7 I fully support the work of the school guidance counselor. 3.09 3.66 18% q5.9 I shows appreciation for the efforts of the school staff. 3.41 3.86 13% Grand Mean 2.97 3.68 24% 6. Assessment In the area of using student assessment data to improve teaching and learning (Table 7), three competencies showed strong growth. Principals reported improvements in ensuring the use of a variety of methods in the evaluation of students' performance (27% change); monitoring the variety of learning assessments used in helping students improve their learning (26% change); and, engaging teachers in decision-making based on the results of student learning assessments (25% change). 16 Table 7. Results for leadership in using assessment data to improve teaching and learning Assessment Pre Post %Change q6.2 I ensure the use of a variety of methods in the evaluation of students' performance. 2.85 3.62 27% q6.3 I monitor the variety of learning assessments used in helping students improve their learning. 2.82 3.54 26% q6.6 I engage teachers in decision-making based on the results of student learning assessments. 2.88 3.59 25% q6.5 I documents the results of performance evaluations of teachers with the aim of development and improvement. 2.95 3.60 22% q6.4 I provide feedback to teachers about their teaching practices with a view to continuous improvement. 2.97 3.61 22% q6.1 I notify parents of their children's assessment results in order to help them improve. 3.16 3.76 19% Grand Mean 2.94 3.62 23% 7. Technology Lastly, in the use of technology to enhance school management and instructional support (Table 8), the biggest growth was reported in the use of technology (laptops and WiFi) to search for his/her own professional development (29% change). Excellent growth (25% or better) was also reported in their use of technology to search for resources on teaching and learning; in encouraging teachers to develop their competency in using technology in teaching and learning; in facilitating administrative functions; and in communicating with teachers and school staff and stakeholders. Table 8. Results for leadership in utilizing technology for school management and instructional supervision Technology Pre Post %Change q7.7 I use technology in professional development 2.72 3.52 29% q7.6 I use technology to search for resources on teaching and learning 2.84 3.60 27% q7.2 I encourage teachers to develop their competency in using technology in teaching and learning. 3.01 3.78 26% q7.3 I use technology to facilitate administrative functions. 2.94 3.68 25% q7.5 I use technology to communicate with teachers and school staff and stakeholders 2.71 3.39 25% q7.1 I increase the teachers' use technology in their teaching. 3.05 3.76 23% q7.4 I follow up on the technical maintenance and repairs of all IT used in the school. 3.15 3.70 17% Grand Mean 2.92 3.63 25% C. Might Other Factors Explain the Results? Besides LTD’s leadership training, were there any statistically significant differences in the scores of the principals’ leadership competencies due to gender, years’ experience as an educator or administrator, or highest level of academic qualification? An analysis was done using both the independent samples t-test and one-way ANOVA. The results found that there were no differences in scores due to these other factors. In other words, other factors such as a principal’s gender or years as an administrator made no difference in the scores for leadership competencies. 17 The implication of these results is that LTD’s Leadership Diploma training is the major reason for improvements in principals’ leadership competencies. Although this claim cannot be verified beyond all doubt, as the sample did not include non-LTD principals for comparison, the following results from the teachers’ questionnaire on the survey strongly support the contention that LTD’s Leadership Training is responsible for big changes in the quality of school leadership. II. Results of the Teacher Survey of Principal Effectiveness A. Teacher Assessment of Principal Effectiveness The baseline score (i.e., grand mean) of the teachers’ assessment of their principals’ leadership competencies were virtually identical to the self-assessment by principals, and, as seen in Table 9, there was just a 7.53% difference in the endline scores of the two groups. Table 10 compares the teachers’ and principals’ ratings for the seven domains of principal’s leadership competencies. Although principals rated themselves higher than teachers, the differences as shown in the table are quite modest, ranging from roughly 6-10%. What these findings suggest then is that for the most part, both teachers and principals were in reasonably close agreement as to the changes observed in leadership competencies due to LTD’s Leadership Diploma Program. Table 9. Comparison of baseline and endline grand means for the principals’ and teachers’ questionnaires on the Principal Effectiveness Survey Baseline Grand Mean Endline Grand Mean Teachers 2.84 3.38 Principals 2.84 3.64 % difference 0.04% 7.53% Table 10. Comparison of principals’ and teachers’ endline grand means for the seven domains of effective leadership Endline Results Teachers Principals % difference Planning 3.20 3.48 8.3% School/Community Relations 3.38 3.70 8.8% Managing Resources 3.49 3.77 7.6% Teaching and Learning 3.41 3.62 6.1% School Environment 3.34 3.69 9.9% Assessment 3.39 3.62 6.6% Technology 3.43 3.63 5.8% B. Which Leadership Competencies Can Be Made Even Stronger? As the results have shown, both LTD and non-LTD teachers believe their LTD principals are demonstrating very positive levels of “expected” school leadership. On the assumption, however, that teachers are likely to be both more critical and more objective, do their evaluations suggest particular leadership competencies that principals might want to reflect on for future professional development? We investigated this question by taking the average endline scores for each of the seven domains on the 18 teachers’ questionnaire, and those that fell below the average are ones that principals might wish to consider giving extra attention. When we look at the lowest scoring sub-competencies in each of the seven domains in Table 11, four of the lowest are all related to the principal’s capacity to “involve” parents (Q1.3), teachers (Q5.2), and other stakeholders (Q2.2) in discourses and decisions on improving the school; likewise, improving flows of information (Q7.5) using technology to communicate with stakeholders is seen as a competency that principals—according to the teachers—may want to strengthen. Table 11. Endline Scores, Principal effectiveness survey (Teachers Questionnaire) Domain Questions Mean Planning (Avg. 3.2) q1.6 The principal uses action research in the development of my work at the school. 3.18 q1.3 The principal involves parents in the school improvement planning. 3.09 School/Community Relations (Avg. 3.4) q2.7 The principal treats school staff fairly (without bias) 3.34 q2.10 The principal resolves conflicts between staff professionally. 3.33 q2.2 The principal involves stakeholders in decision making related to improving the school based on the results of the self-evaluation data 3.27 Managing Resources (Avg. 3.5) q3.1 The principal invests in the development of the skills and expertise of school staff. 3.38 q3.2 The principal works to identify the needs of staff to support the teaching-learning process. 3.36 Teaching and Learning (Avg. 3.4) q4.2 The principal is familiar with theories of teaching and learning and uses this knowledge to help teachers improve their teaching. 3.32 q4.4 The principal ensures the positive development of students' attitudes and behaviors. 3.3 School Environment (Avg. 3.3) q5.1 The principal delegates tasks to staff that commensurate with their capacity. 3.24 q5.8 The principal seeks ways to reward and incentivize teachers in their work. 3.21 q5.2 The principal involves teachers in decision making related to the school and its community 3.18 Assessment (Avg. 3.4) q6.6 The principal engages teachers in decision-making based on the results of student learning assessments. 3.34 q6.3 The principal monitors the variety of learning assessments used in helping students improve their learning. 3.29 Technology (Avg. 3.4) q7.6 The principal uses technology to search for resources on teaching and learning 3.35 q7.7 The principal uses technology in professional development 3.34 q7.5 The principal uses technology to communicate with teachers and school staff and stakeholders 3.24 19 III. PMP Indicator Results A. Benchmark for Effectiveness Based on principals’ self-reporting, shown in Table 2, there was a 28.3% change in the baseline score of 2.84 for principal effectiveness compared to 3.64 at the endline. The weighted average of the combined scores (means) on the principals' forms (n = 106) and teachers’ forms (n = 616) of resulted in an endline score of 3.42 (Table 12). LTD, based on its Program Management Plan (PMP), considers this score the benchmark for LTD’s definition of “Effective Leadership.” B. Surpassing the Benchmark It was found that 82% of the principals scored 3.42 or better on the endline results of the principals’ self￾assessment survey. Thus, based on the PMP target of “80% of participating principals demonstrating effective school leadership according to the MoEHE’s Effective School Standards and Competencies, LTD met its target for Cohort II (Table 13). In sum, LTD exceeded its target of 80% for Indicator 2.1, “Participating principals demonstrating effective school leadership.” Table 12. Baseline/endline scores and weighted averages for combined principal and teacher scores Principal's Self-Evaluation N Minimum Maximum Mean Std. Deviation TOT_PR 106 1.69 3.66 2.8404 0.39425 TOT_PST 106 3.02 4 3.6448 0.25272 Teacher's Evaluation of Principal N Minimum Maximum Mean Std. Deviation TOT_PR 612 1 4 2.8393 0.5744 TOT_PST 616 1.41 4 3.3802 0.49807 Combined Weighted Averages of Principals plus Teachers Pre 2.84 Post* 3.42 Percentage change 20.41178 *This value is the mean score for "Effective Leadership” Table 13. Results for PMP Indicator 2.1 Indicator 2.1 Target Actual Percentage of participating principals (per cohort; at post measurement) demonstrating effective school leadership according to principals and teachers based on MoEHE’s Effective School Standards and Competencies 80% 82% 20 IV. Mixed-Methods Study of School Improvement Teams Introduction Every school principal enrolled in LTD’s Leadership Diploma Program is expected to form and lead a school improvement team to produce a school improvement plan (SIP). The planning process entails four phases: first, the formation of a team comprised of the principal and several teachers and parents, and possibly other staff or faculty as deemed necessary; next, completing a school-wide self-assessment framed by the Ministry of Education’s National Standards for Effective Schools; then, writing strategic goals and a work plan for the SIP, and presenting the plan for review by the local district and to AMIDEAST for procurement; and lastly, implementing and monitoring the plan. What is clear from these procedures is that the LTD approach is a consultative and participatory process designed to bring about a shared approach to school leadership. Shared leadership is understood as “the practice of governing a school by expanding the number of people involved in making important decisions related to the school’s organization, operation, and academics…[and] entails the creation of leadership roles or decision-making opportunities for teachers, staff members, students, parents, and community members.”2 Unlike to typical command-and-control model that typifies the principalship in schools in the West Bank and Gaza, this shared approach engages key stakeholders across the school community to collect data, identify needs, and implement strategic goals toward improving student learning and school management. 2 http://edglossary.org/shared-leadership/ KEY FINDINGS 1. LTD contributed to the capacity of SIT members, and in particular the performance of principals, to work collegially and collaboratively in developing their school’s vision and mission, establish strategic goals, and prepare an implementation plan.  Eighty-eight percent of SIT members surveyed rated as high the quality of their team’s performance of tasks required to develop their school’s vision and mission, establish strategic goals, and prepare an implementation plan.  Ninety percent of SIT members surveyed agreed that team members exercised cooperation and collaboration in developing in the school improvement team.  Ninety-four percent of SIT members surveyed agreed that the principal was effective in sorting and managing collaborative work among SIT members in developing the SIP. 2. LTD contributed to the efficacy of SIT members to work collegially and collaboratively in implementing their School Improvement Plans.  Seventy-seven percent of SIT members surveyed rated their performance in monitoring the implementation of their plans.  Eighty-four percent of SIT members surveyed rated their performance in deploying and managing material and human resources for the effective implementation of their plans.  Ninety-one percent of SIT members surveyed agreed that as a team they exercised cooperation and teamwork during implementation of the plan.  Ninety-three percent of SIT members surveyed agreed the principal exercised shared leadership in supporting and managing collaborative work among SIT members. 21 Design of the Research The research attempted to understand the extent that the school improvement team (SIT) contributes to a process of shared leadership in the development and implementation of a school improvement plan (SIP). A mixed methods design was used in which baseline and endline quantitative data were collected and then explained or elaborated using qualitative data from focus groups. Because the baseline research was conducted in [month] when most schools had just completed drafting their SIPs, the baseline research examined shared leadership and teamwork only with respect to the planning phase of SIP process. Conversely, when the research was repeated at the end of the school year, the research looked at how the process of shared leadership and teamwork was reflected in the implementation of the SIPs. In both instances, questionnaires were distributed to principals either at trainings or via their local district directorate. All focus groups were conducted at the National Institute for Educational Training (NIET), and the sample of SITs represented the geographic diversity of the school districts where is working. A. Results of the SIT Survey Research 1. Planning Phase of the School Improvement Process To investigate the planning phase of the SIP process, the survey focused on three major questions:  Task Performance: What was the overall quality of the SIT's role and tasks in developing the school’s vision and mission, establishing strategic goals, and preparing an implementation plan?  Teamwork: To what extent did the SIT members agree they exercised cooperation and collaboration in developing in the school improvement team (SIP)?  Principal Leadership: How effective was the principal in supporting and managing collaborative work among SIT members in developing the SIP? The survey instrument used a five-point Likert scale for quality of task performance (very low = 1 to very high =5) and agreement (1= strongly disagree to 5 = strongly agree) and included a set of open-ended questions to allow for additional comments. Thus a mean ranging from 3 to 3.5 indicates a satisfactory level of quality or agreement, and anything above 3.5 indicates a high level of quality or agreement. Thus if we first look at the big picture of the results, we see in Figure that the SIT members evaluated as quite high the quality of the performance of the SIT members in carrying out tasks related to the planning phase. 22 Were there any statistically significant difference in the means of these three indicators of SIP planning due to gender, position, or district? An analysis was done using both independent samples t-test and one-way ANOVA (see Annex ___). Gender explained the variance in the means of only “Teamwork” (exercising cooperation and collaboration in developing in the school improvement team. Directorate explained the variance in the means for all three—Task Performance, Teamwork, and Principal Leadership—with Qalqilia appearing to have the most influence on the variance. With regard to position, this variable explained the variation in means for Task Performance and Teamwork. 2. Task Performance during planning of the SIP What was the overall quality of the SIT's role and tasks in developing the school’s vision and mission, establishing strategic goals, and preparing an implementation plan? This section of the questionnaire used a five-point Likert scale for quality (1= very low to 5 = very high) The grand mean score for task performance by members of SITs is 4.09, indicating a high quality task performance in developing the school’s vision and mission, establishing strategic goals, and preparing an implementation plan. Table 14 ranks the scores in descending order, and we see that the members of SITs rated highest their performance in developing their school’s mission and vision (4.37 out of 5) , followed by setting clear goals for school improvement (4.33 out of 5). Table 14. Ranked scores for quality of the SIT's role and tasks in developing the school’s vision and mission, establishing strategic goals, and preparing an implementation plan. Question Principal Teacher Parent Other Grand Mean QA2. Develop the mission and vision of the school. 4.19 4.24 4.04 4.37 4.21 QA3. Set clear goals for school improvement. 4.16 4.12 4.04 4.33 4.16 QA4. Develop a strategic plan for school improvement. 4 4.04 4.06 4.16 4.07 QA5. Prepare a work plan for implementing the school improvement plan. 4.02 4.01 3.91 4.23 4.04 QA1. Collect data for the school self-assessment study. 4.2 3.88 3.72 4.09 3.97 Overall Grand Mean 4.11 4.06 3.95 4.24 4.09 4.12 4.11 4.23 4.06 4.16 4.21 3.95 4.01 4.08 4.24 4.09 4.22 3.50 3.60 3.70 3.80 3.90 4.00 4.10 4.20 4.30 Task Performance Teamwork Principal Leadership Key Indicators of the SIP Planning Process Principal Teacher Parent Other Figure 3. Three key indicators of SIT performance during the planning phase 23 Of the scores on the lower end of the ranking—below the grand mean of 4.09—we see that two tasks in particular scored least well: preparing a work plan for implementing the school improvement plan (QA5) and collecting data for the school self-assessment study (QA1). It would appear that these lower assessments are mostly attributable to parents (Table 15), of whom 67% evaluated their involvement as “high” in collecting data for the school self-assessment study compared to principals (98%), teachers (79%), and Others (93%). Likewise, 78% of the parents were in less agreement compared to the other members in regards to preparing a work plan for implementing the school improvement plan. Table 15. Level of quality of task performance during planning of the SIP Question Principal Teacher Parent Other QA1. Collect data for the school self-assessment study. Low 0% 2% 3% 0% Moderate 2% 19% 30% 7% High 98% 79% 67% 93% QA2. Develop the mission and vision of the school. Low 0% 1% 3% 0% Moderate 5% 9% 10% 5% High 95% 91% 87% 95% QA3. Set clear goals for school improvement. Low 0% 1% 2% 2% Moderate 2% 10% 19% 2% High 98% 89% 79% 95% QA4. Develop a strategic plan for school improvement. Low 0% 1% 2% 2% Moderate 11% 14% 9% 7% High 89% 84% 89% 91% QA5. Prepare a work plan for implementing the school improvement plan. Low 0% 2% 0% 2% Moderate 5% 15% 22% 5% High 96% 83% 78% 93% 3. Teamwork during planning of SIP: To what extent did the SIT members agree they exercised cooperation and collaboration in developing in the school improvement team (SIP)? This section of the questionnaire used a five-point Likert scale for agreement (1= strongly disagree to 5 = strongly agree). The grand mean score for teamwork for all groups is 4.09, indicating solid agreement among the different members on SITs. Table 16 shows the ranking order in descending order for the eight questions in the domain. Highest levels of agreement—those scores above the grand mean (QB5, QB2, QB6, QB7, and QB4) —indicated that the participants among the four groups of SIT members were committed to working together cooperatively, collaboratively, and with a good deal of mutual respect for differing viewpoints. 24 Table 16. Level of agreement among SIT members that they exercised cooperation and collaboration in developing in the school improvement team (SIP) Question Principal Teacher Parent Other Grand Mean QB5. There was a spirit of mutual cooperation and trust among the team members. 4.41 4.30 4.12 4.21 4.26 QB2. Team members were committed to participating in meetings. 4.11 4.26 4.05 4.19 4.15 QB6. Team members had positive attitudes towards their work in the school and with local organizations. 4.20 4.21 4.09 4.09 4.15 QB7. Team members were receptive to different viewpoints. 4.09 4.21 4.01 4.12 4.11 QB4. There was an agreed upon common approach to the work of the team. 4.02 4.14 4.24 4.02 4.11 QB3. Team members showed commitment in completing tasks entrusted to them during meetings. 4.07 4.21 3.91 4.12 4.08 QB8. Team members efficiently completed their tasks associated with planning. 3.93 4.02 3.83 4.02 3.95 QB1. The leadership of the school improvement team achieved the team's intended goals for planning. 4.02 3.91 3.87 3.95 3.94 Overall Grand Mean 4.11 4.16 4.02 4.09 4.09 For those that with agreement levels below the grand mean (QB3, QB8, and QB1, the implication is that the completion of tasks for some of the members may have been challenging at times, which in fact was an issue that both teachers and parents discussed in the focus groups, where both workloads and scheduling conflicts sometimes hindered the timely completion of tasks. 4. Principal Leadership during planning of SIP: How effective was the principal in supporting and managing collaborative work among SIT members in developing the SIP? This section of the questionnaire also used a five-point Likert scale for agreement (1= strongly disagree to 5 = strongly agree). The grand mean score for all groups is 4.19, indicating very solid agreement among the SIT members that the principal was effective in supporting and managing collaborative work among SIT members in developing the SIP (Table 17). Indeed, Figure 4 shows there was near universal agreement that the principal exhibited effective leadership during the SIP planning process. The ranking of the seven items in Table 17 in descending order suggests that if there is any room for improvement in this otherwise outstanding assessment, it would be in regard to two skills: offering constructive feedback to team members (QC6); and, helping the team to work through their differences (i.e., how to constructively manage conflicting views). Comments in focus groups made by parents and teachers (discussed below) tended to validate these two points. 25 Table 17. Level of agreement that the principal was effective in supporting and managing collaborative work among SIT members in developing the SIP. Question Principal Teacher Parent Other Grand Mean QC4. The principal encouraged team members to express their opinions. 4.34 4.31 4.25 4.33 4.31 QC3. The principal shared information about the planning process with the rest of the team. 4.3 4.24 4.12 4.28 4.24 QC2. The principal showed commitment in accordance with school rules and regulations. 4.23 4.29 4.12 4.21 4.21 QC5. The principal provided technical and administrative support to members of the team. 4.34 4.22 4 4.21 4.19 QC1. The principal supported the team members in completing their tasks in a timely manner. 4.16 4.22 4.09 4.26 4.18 QC6. The principal offered clear and specific feedback to team members. 4.19 4.13 4.03 4.16 4.13 QC7. The principal helped the team to work through their differences. 4.07 4.03 3.95 4.09 4.04 Overall Grand Mean 4.23 4.21 4.08 4.22 4.19 Figure 4. Percentage of SIT members who agreed/strongly agreed the principal was effective in supporting and managing collaborative work among SIT members in developing the SIP. 5. Implementation of the School Improvement Process To investigate the implementation phase, the survey explores four main questions:  Monitoring SIP implementation: What was the overall quality of the SIT's role and tasks in monitoring the implementation of the SIP? 70% 75% 80% 85% 90% 95% 100% Principal Teacher Parent Other The principal was effective in supporting and managing collaborative work among SIT members in developing the SIP. Agree/Strongly Agree 26  Deploying and managing resources: How effectively did the SIT deploy and manage material and human resources for effective implementation of the SIP?  Practicing teamwork and collaboration: How effectively did the members of the SIP exercise cooperation and teamwork during implementation of the plan?  Modeling shared leadership: How effective was the principal in supporting and managing collaborative work among SIT members? As with the results of the planning phase, Figure 5 clearly shows that the SIT members evaluated the overall quality of the SIT members’ performance across all four domains of implementing their school improvement plans (SIP) was well above 3.5 out of 5, that is, they assessed the quality of their work as quite high. Figure 5. Key Indicators of the SIP Implementation Process Were there any statistically significant difference in the means of the four indicators of SIP implementation due to gender, position, years at the school, and district? An analysis was done using both independent samples t-test and one-way ANOVA. Of the four dependent variables, a statistically significant difference in the mean of “Practicing teamwork and collaboration” due to gender only. Of the three other main variables—position, years at the school, and district—no statistically significant differences were found. Now, let’s take a look at the results in greater detail. a) Monitoring SIP implementation What was the overall quality of the SIT's role and tasks in monitoring the implementation of the SIP? This section of the questionnaire used a five-point Likert scale for quality (1= very low to 5 = very high), and the grand mean of 3. 87 for the three items comprising this domain reflects a relatively high level of quality in performing the roles and tasks necessary to monitor the implement the SIP (Table 18). As a group 77% of the SIT members collectively rated the quality of their performance as high in regard to 3.85 4.17 4.14 4.31 3.91 4.13 4.19 4.24 3.82 4.02 4.11 4.16 3.88 4.07 4.11 4.27 3.5 3.6 3.7 3.8 3.9 4 4.1 4.2 4.3 4.4 Monitoring SIP implementation Deploying and managing resources Practicing teamwork and collaboration Modeling shared leadership Key Indicators of the SIP Implementation Process Principal Teacher Parent Other 27 supervising, assessing, and reporting about the progress of implementing the SIP (Figure 6). It bears noting, however, that of these three reporting had the least high rating. In fact, 46% of the principals themselves rated the team’s performance in writing of progress reports as low to moderate, suggesting this is an area that may need strengthening (Table 19). Table 18. Quality of monitoring SIP implementation Questions Principal Teacher Parent Other Grand Mean QA1. Supervising the implementation of the school improvement plan 4.04 3.95 3.88 3.93 3.95 QA2. Assessing the progress of the plan 3.90 3.90 3.81 3.87 3.87 QA3. Preparing special reports on the progress and successes of the plan 3.62 3.90 3.76 3.85 3.78 Overall Grand Mean 3.85 3.91 3.82 3.88 3.87 Figure 6. Overall quality of performance in monitoring SIP implementation 2% 2% 4% 16% 19% 24% 82% 79% 71% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% QA1. Supervising the implementation of the school improvement plan QA2. Assessing the progress of the plan QA3. Preparing special reports on the progress and successes of the plan Quality of Performance in Monitoring SIP Implementation Low Moderate High 28 Table 19. Quality of performance in monitoring SIP implementation Monitoring SIP implementation Principal Teacher Parent Other Column N % Column N % Column N % Column N % QA1. Supervising the implementation of the school improvement plan Low 0% 2% 3% 0% Moderate 14% 14% 21% 19% High 86% 83% 76% 82% QA2. Assessing the progress of the plan Low 0% 2% 4% 2% Moderate 20% 19% 22% 17% High 80% 80% 74% 82% QA3. Preparing special reports on the progress and successes of the plan Low 2% 4% 7% 6% Moderate 42% 22% 19% 17% High 56% 75% 74% 78% b) Deploying and managing resources How effectively did the SIT deploy and manage material and human resources for effective implementation of the SIP? This domain comprised 10 items on the questionnaire and was measured based on a five-point Likert scale for quality (1= very low to 5 = very high), and we see in Table 20 that the grand mean for all groups is 4.02 out of 5, which indicates solid agreement among the SIT members that they were able to successfully deploy and manage material and human resources for effective implementation of the SIP (Figure 5). Table 20. Quality of performance in deploying and managing resources Principal Teacher Parent Other Grand Mean QB1. Managing the process of implementing the school plan 3.86 3.88 3.83 3.44 3.75 QB2. Effectively managing resources to ensure the quality of teaching and learning 4.19 4.07 3.89 3.78 3.98 QB3. Effectively managing teaching and learning with a high level of performance 3.88 3.94 3.91 3.78 3.88 QB4. Improving internal relations at the school 4.41 4.19 4.16 3.67 4.11 QB5. Improving the external relations of the school 4.22 4.13 4.10 3.67 4.03 QB6. Improving the school environment 4.28 4.28 4.15 3.89 4.15 QB7. Encouraging community participation 4.10 4.08 4.07 3.89 4.04 QB8. Mobilizing technology in teaching and learning 4.20 4.37 4.08 4.00 4.16 QB9. Mobilizing technology for school management 4.38 4.31 3.98 3.67 4.09 QB10. Mobilizing human and material resources of the school 4.18 4.09 4.04 3.78 4.02 Grand Mean 4.17 4.13 4.02 3.76 4.02 29 A closer examination of the means for the individual items points to noteworthy strengths as well as to areas that would benefit from further developed. Of the six items that scored at or above the mean of 4.02 (Table 20, above), four relate to enhancing relationships among stakeholders of the school community, both inside the school and with the local community (QB4, QB5, QB7, QB10), while the other two items highlight the importance of technology for improving teaching and learning and school management (QB8, QB9). These findings are confirmed in the focus group discussions with SIT members. In short, we are seeing evidence of the strong influence of LTD’s robust provision of technology resources (i.e., laptops, LCDs, and Internet connectivity) to support effective teaching and school management, plus its practical, school-based approach to professional development of the Leadership Diploma Program. Naturally, there is always room for improvement, and the findings also point to a number of areas, although rated quite high, that might deserve further development. Three of the 10 items scored below the grand mean of 4.02 (Table 20, above). As seen in Table 21, on the issue of managing the implementation process of the school plan about 1 in 5 of all principals (21%), teachers, parents and other school staff rated low to moderate the quality of managing the process of implementing the school plan (QB1), which appears to be validated by findings (discussed below) from focus groups that SITs experienced some difficulty in monitoring and reporting the progress of SIP implementation. Additionally, 26% of the principals and 20% of the teachers rated “low” to “modest” the level of the team’s performance in effectively managing teaching and learning with a high level of performance (QB3), while 16% of teachers, 27% of parents, and 22% of other school staff rated “low” to “modest” the quality of effectively managing resources to ensure the quality of teaching and learning. Table 21. Quality of performance in deploying and managing resources Deploying and managing resources Principal Teacher Parent Other QB1. Managing the process of implementing the school plan Low 0% 2% 4% 2% Moderate 23% 18% 19% 17% High 77% 81% 77% 82% QB2. Effectively managing resources to ensure the quality of teaching and learning Low 0% 1% 3% 2% Moderate 7% 15% 24% 20% High 93% 84% 73% 78% QB3. Effectively managing teaching and learning with a high level of performance Low 0% 1% 3% 0% Moderate 26% 19% 17% 25% High 74% 81% 80% 76% c) Practicing teamwork and collaboration: How effectively did the members of the SIP exercise cooperation and teamwork during implementation of the plan? LTD’s model of shared leadership puts a premium on teamwork among the members of the school improvement team. The development of the school improvement plan requires that members of a school 30 improvement team—the principal, teachers, parents and others from the school community—can work collaboratively and efficiently to plan, share ideas, and support each other. This section of the questionnaire used a five-point Likert scale for agreement (from 1= strongly disagree to 5 = strongly agree). The grand mean for all groups is 4.14 out of 5 (Table 22), indicating solid agreement among SIT members that they effectively exercised cooperation and teamwork during implementation of the plan. A closer examination of the means for the individual items points to noteworthy strengths as well as to areas that might be further developed. Table 22 points to four of the items scored above the mean of 4.14 (QC5, QC6, QC2, and QC4). What these items have in common is they are the necessary ingredients for shared leadership: mutual trust, cooperation, positive attitudes among stakeholders, commitment, and a common vision and methodology for working together. An impressive 91% of the respondents, on average, agreed that the members of the SIP practiced skills and attitudes contributing to effective collaboration and teamwork during implementation of the plan (Figure 8). Table 22. Practicing teamwork and collaboration Questions Principal Teacher Parent Other Grand Mean QC1. The leadership of the school improvement team achieved the desired goals during the implementation of the plan 4.03 4.03 3.96 3.93 3.99 QC2. The commitment of team members to participate in meetings 4.19 4.26 4.14 4.13 4.18 QC3. The commitment of team members to carry out tasks entrusted to them during meetings 4.13 4.21 4.09 4.04 4.12 QC4. The existence of a common vision and methodology among the team members 4.16 4.15 4.16 4.15 4.16 QC5. The existence of a cooperative atmosphere and mutual trust between team members 4.32 4.34 4.24 4.27 4.29 QC6. The existence of positive attitudes by team members to work in the school and with outside institutions 4.21 4.25 4.11 4.25 4.21 QC7. Acceptance by team members team of outside comments and opinions 4.04 4.21 4.11 4.07 4.11 QC8. The effective completion by team members of tasks and requirements associated with implementation 4.03 4.04 4.08 4.11 4.07 Grand Mean 4.14 4.19 4.11 4.12 4.14 31 Figure 7. Agreement on level of teamwork and collaboration While there is no doubting these impressively positive assessments, some room for improvement is suggested in the results on the matter of “accepting outside comments and opinions” (QC7). Table 23 shows that some 15% of the principals and 11% of the parents remained neutral on this item; moreover, of all the 10 items, only this one was found to have a statistically significant variance in the means based on gender, position, and directorate. What these statistics suggest, then, is that a modest number of SIT members—as a group—may have found some difficulty if being fully receptive to the views from other stakeholders in the school community. Table 23. Acceptance of outside comments and opinions QC7. Acceptance by team members team of outside comments and opinions Principal Teacher Parent Other Disagree .0% 1.2% 1.0% .0% Neutral 14.9% 6.4% 11.5% 11.1% Agree 85.1% 92.4% 87.5% 88.9% 88% 90% 94% 95% 91% 90% 90% 93% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% QC8. The effective completion by team members of tasks and requirements associated with implementation QC7. Acceptance by team members team of outside comments and opinions QC6. The existence of positive attitudes by team members to work in the school and with outside institutions QC5. The existence of a cooperative atmosphere and mutual trust between team members QC4. The existence of a common vision and methodology among the team members QC3. The commitment of team members to carry out tasks entrusted to them during meetings QC2. The commitment of team members to participate in meetings QC1. The leadership of the school improvement team achieved the desired goals during the implementation of the plan Practicing teamwork and collaboration Disagree Neutral Agree 32 d) Modeling shared leadership How effective was the principal in supporting and managing collaborative work among SIT members? Through its Leadership Diploma Program, LTD, aims to transform the principalship from a traditional command-and-control model to one that values and practices shared leadership. The SIT is a key structure in this transformation, for it is the space where the principal is expected to share leadership by engaging teachers, parents, and staff members or others from the local community in a participatory process of discourses and decision-making to advance ongoing school improvement. This section of the questionnaire also used a five-point Likert scale for agreement (1= strongly disagree to 5 = strongly agree), and the grand mean for all groups is 4.24 out of 5 (Table 24), indicating solid agreement among SIT members that the principal was effective in supporting and managing collaborative work among SIT members. Table 24. Agreement that the principal models shared leadership Questions Principal Teacher Parent Other Grand Mean QD1. The principal helped the rest of the members in the completion of tasks in a timely manner 4.20 4.21 4.20 4.26 4.22 QD2. The principal's commitment was consistent with the regulations and codes of the school 4.32 4.30 4.25 4.30 4.29 QD3. The principal shared information associated with the implementation process with the other members of the team 4.27 4.25 4.19 4.22 4.23 QD4. The principal encouraged the team members to express their views 4.47 4.36 4.16 4.33 4.33 QD5. The principal provided technical and administrative support to the team members 4.43 4.26 4.13 4.28 4.28 QD6. The principal provided clear and specific feedback to team members 4.29 4.12 4.09 4.26 4.19 QD7. The principal helped the team members to overcome the sources of conflict 4.22 4.16 4.07 4.24 4.17 Grand Mean 4.31 4.24 4.16 4.27 4.24 The four scores that are above the grand mean of 4.24 for how teachers and parents evaluated the principal’s performance in supporting and managing their collaboration on the SIT (QD2, QD, QD4, QD5), suggest the principal did an outstanding job of providing the needed structure, encouragement and information to allow SIT members to express their views and take action during the implementation of the SIP. The three items that scored below the mean (QD1, QD6, and QD7), point again to two challenges perceived by many SIT members, namely, insufficient time sometimes hindered the efficient completion of tasks and the principal’s efforts to manage conflicting views less effective than expected. 33 Table 25 points to a pattern in the levels of agreement by parents. Whereas teachers’ scores of agreement tend to remain in the 90 percentile range across all seven items in this domain but one—87% for giving feedback—parents scores dip into the mid-80 percentile range on four indicators of the principal’s support and management of collaboration: encouraging the expression of viewpoints (85%), providing technical and administrative support (84%), offering clear and specific feedback (86%), and helping the team members to overcome the sources of conflict (86%). It bears mentioning that the differences in the means for the principal’s capacity to manage conflict (QD7) was found to be statistically significant with regard to parents. In sum, while the results of the parents’ scores on these four indicators are still relatively quite high, the fact that they are lower than those of the teachers deserves consideration. 6. Results of open-ended questions on the surveys The questionnaire included a number of open-ended questions allowing the respondents an opportunity to identify what they believed were important achievements and challenges from their experiences implementing the SIP. Several thousand written entries were coded and then classified into the most frequently occurring comments pointing to either achievements or challenges. a) Major Achievements The principals, teachers, and parents indicated that the implementation phase of the SIP process was characterized by improvements in the school environment; increased use of technology in teaching and 95% 97% 94% 95% 95% 87% 91% 94% 94% 93% 85% 84% 86% 86% 75% 80% 85% 90% 95% 100% QD1. The principal helped the rest of the members in the completion of tasks in a timely manner QD2. The principal's commitment was consistent with the regulations and codes of the school QD3. The principal shared information associated with the implementation process with the other members of the team QD4. The principal encouraged the team members to express their views QD5. The principal provided technical and administrative support to the team members QD6. The principal provided clear and specific feedback to team members QD7. The principal helped the team members to overcome the sources of conflict Agreement that the principal practiced shared leadership Teacher Parent Figure 8. Percentage of teachers and parents agreeing that the principal practiced shared leadership 34 learning; enhanced internal and external relationships in the school community; and a deepened understanding of the importance of a school’s vision mission.  Child-Friendly Environment: Some 23% of participants noted that the school environment was greatly enhanced thanks to better school facilities that helped to create more active classrooms and produce a more child-friendly atmosphere throughout the school. Examples of changes included painting and decorating school walls with educational slogans and pictures; and the planting of flower gardens.  Spreading the Use of Technology: Some 24% indicated that the classroom use of technology increased in teaching and learning. Many of the schools focused their SIP budgets on purchasing laptops, LCD projectors, and other educational devices such as smart boards, that were installed classrooms, libraries, and computer labs. This infusion of technology took advantage of the training teachers received in how to make the most of technology for blended learning.  School-Community Relations: Some 33% of the respondents linked the SIP process to increased participation and cooperation of parents and community members. Participants noted that the strong level of cooperation among the SIT members themselves resulted in extending their school’s networking with different community members and local institutions. One school, for example, leveraged its improved community relations to win the support of the local village council to rehabilitate the school’s gym.  Valuing the Vision and Mission: Nearly 20% of the participants commented they have a better understanding of the school vision and mission. They noted the importance not only of the process of writing vision and mission statements, but also the importance of aligning the goals of the SIP with resources needed for their successful implementation. As one teacher put it, A “Planning is the secret of success for achieving goals.” a) Some Challenges along the Way: Along with the many accomplishments during the implementation phase, many of the respondents noted that managing the available financial and human resources was a recurring frustration. Some 53% of the participants, for example, stated that their school day workload, lack of available time, and limited financial resources tended to complicate their efforts during the implementation of their SIPs. 35 B. Focus Groups Results for School Improvement Team Introduction The M&E units of both the Leadership and Teacher Development Program (LTD) and its sister project, the School Support Program (SSP), jointly conducted three focus groups in November 2014 and six focus groups in May 2015 at NIET. The participants represented a purposive sample of SITs from both Cohort I and Cohort II schools. The participants were arranged into three groups according to their specific SIT roles—principal, teacher, or parent. Each session lasted 90 minutes and was audio recorded. Since none of the schools had completed the drafting of their SIPs, the discussion focused only on tasks leading up to the drafting of the SIP. The November interviews focused on the planning phase of the SIP process and were framed by the following core questions:  Think about your role during the SIP planning process. What specific tasks did you perform and what was most challenging for you personally?  Reflect on the quality of teamwork among the members of your SIT. Did you feel empowered to share your views, ideas, and contribute to decisions? Explain.  What barriers did the team face as a group while developing the SIP and how were they managed? The interviews in May explored the implementation phase of the SIP process and were framed by these core questions:  A key responsibility of the school development team is to implement the SIP. What has changed for the better in your school as a result of the implementation of the SIP? What contributed to this success?  Think about the obstacles your team faced during the implementation of the SIP. What internal and external factors contributed to the challenges?  All things considered, do you think that the SIT is an effective mechanism for shared leadership towards improving the school? How could it work better? KEY FINDINGS As a result of capacity-building for SITs and in-kind assistance to schools, LTD contributed to empowering the SITs to make substantive improvements to the overall learning environments of their schools by… • Creating a more child-friendly climate by making major improvements to the physical conditions of schools. • Fostering the widespread use of ICT in classroom instruction. • Strengthening internal and external relations among students, staff, and parents, especially by leveraging the evidence-based SIP to secure donations from the local community towards achieving targets for school improvement. • Aligning the school vision and mission to its goals and targets for enhancing students’ learning. 36 1. Planning Phase of the SIP Process a) Quality of interaction and collaboration Across the three groups, participants described positive impressions about the level of collaboration among members. Principals emphasized their role in facilitating discussions with team members to collectively identify and prioritize key ideas and issues for discussion. They felt it was their responsibility to minimize conflict by assigning tasks and responsibilities to specific individuals. Teachers claimed that the exchange of views and ideas was aided by teachers’ shared beliefs about the goals of the SIP, by the diversity of their areas of expertise, and by their willingness to share resources and tasks. A teacher described this dynamic: “From the very beginning, the roles of our team were distributed. For example, I was responsible for surveys and interviews with the parents; another teacher was responsible for the student survey, a third for the teacher survey, and a fourth did data entry. Then we worked together on the analysis and identified strengths and weaknesses and then prepared the strategic plan based on seven domains and each of us took one of these and discussed our work.” Parents described the collaborative nature of the meetings as smooth, acceptable, successful. As one parent recalled: “We developed the school improvement plan and set action points to carry out the plan. We cooperated at all stages and we held several meetings" In sum, for all SIT members, a major outcome of the planning process was the belief that it brought key stakeholders in the community closer together. b) Freedom to share and exchange ideas and opinions Overall, the three groups conveyed similar views. Principals spoke of the importance of mutual trust, respect for differing opinions, and the need to engage all team members in the process. These beliefs were echoed by teachers, who noted that in general they felt enabled to speak their mind and that the principal was receptive to their points of view and encouraged feedback from all during the SIP process. As one teacher explained: “Discussions were done cooperatively; when anyone had an idea, the others would chime in and the school administration was very helpful. When the principal had an idea she would offer it and he listened to others on the team.” On the other hand, they also observed that principals would try to avoid conflicting opinions in discussions, preferring sometimes to meet with the teachers in smaller groups or individually. Parents commented that team members listened to one another’s suggestions and this encouraged the exchange of viewpoints. In short, we see an improvement in communication and trust between the school principal and teachers, and better communication and networking between the school and parents and with the local community more broadly. A principal summed up this point succinctly: “We got the local community and parents involved; I mean [the SIT process] gave them a clearer picture of what the principal and teachers are doing—a much clearer picture.” 37 c) Participation in decision-making Somewhat contrary to the above description, the principals stated there were limits to their willingness to consult teachers and parents in decision making during the development of the SIP; instead, they sometimes found it easier to make the decisions unilaterally without engaging other stakeholders. This observation is consistent with the responses from teachers and parents. Teachers acknowledged being consulted for their feedback on the plan, but this usually happened after the plan had already been drafted by the principal. Nonetheless, the inclusiveness of the process is a great improvement, as one teacher noted, “Before the program, the principal used to write the plan himself and distribute copies for our feedback, but with the formation of the SIT (under LTD), this process became more inclusive.” Similarly, parents commented that the principal was the main decision maker; that is, they would tend to defer to his/her authority. One parent stated in this way: “Since the principal is the head of the SIT, he makes all the decisions and our role is to carry them out.” It would appear, then, that parents perceived their role during the planning phase as minimal and with little direct input in the drafting of the SIP document itself. They did, however, expect to have a bigger role during the implementation phase. In sum, the collective nature of the planning process brought the administration and teachers closer together and this, too, is reflected in the inclusion of community members in the planning process, a fact appreciated by the parents, though they would expect to be more involved in the implementation phase. As one parent put it: “The school improvement plan was completed because the members of the team cooperated and listened to each other’s suggestions.” d) Challenges during the planning phase of the SIP 1) Limited Time and Heavy Workload All groups commented that finding time to meet and carry out the many tasks of the process was their biggest challenge, and more so for the principals and teachers because of the heavy workload in their typical workday. Teachers stressed the difficulties they faced in trying to juggle their duties to the SIT with their primary responsibility of covering the curriculum they teach and with other obligations relating to school-wide development. A principal spoke of the pressure: “Principal: “From the start of the school year, I’m planning; I’m following up on budgeting and other issues. Sometimes when we want to meet with teachers, one of the SIT members is out of school (for other activities) and this creates a burden for everyone and it delays our submitting our plan to the school directorate.” Teachers added that trying to write and revise drafts of the SIP was further complicated because of the combination of time constraints and the difficulty of resolving individual scheduling conflicts. Parents observed that some parents missed meetings. As one teacher noted, “We don't often have free time between us, the members of the SIT. So we are forced to meet during our lunch break." 38 2) Coping with the Challenges Principals took a variety of steps to mitigate disruptions to the planning process caused by time and scheduling difficulties: meeting Friday afternoons after the Al-A’ser Prayer; shifting around teachers’ schedules to free up class time; meeting during breaks. One principal admitted taking the extreme measure of personally completing most of the tasks on his own. Some teachers freed up time during school hours by taking home more of their prep work and grading tasks; and some teachers simply met on their own without the principal whenever they had time. 2. Implementation Phase of the SIP Process The participants were asked to share and discuss what changed for the better in their schools as a result of the SIT’s efforts to implement the school improvement plan (SIP), and to consider what contributed to this success. Likewise, the participants also shared stories about challenges they faced and offered recommendations for improving the performance of the SIT as a mechanism for shared leadership. Since the development of the SIP is structured by the Ministry’s standards for effective schools, we used the same criteria to organize the analysis of the participants’ discussion:  School environment  Teaching and learning  Management of material and human resources to improve teaching and learning  The use of technology in teaching and learning  Internal and external relations  Connecting Students and Teachers to the School Community a) School environment All groups—principals, teachers and parents—explained that their teams focused a good deal of attention on renovating facilities such as the library, cafeteria, and bathrooms. They emphasized the importance of improving both the conditions and appearances of the school to create a healthier, more appealing, and more child-friendly learning environment. Specific examples included improved sanitation, water dispensers in corridors, outdoor canopies to provide shade, decorative and educational logos and images painted on walls, and renovations to gymnasiums. The SIT secured resources for these improvements mainly through AMIDEAST assistance but also through contributions solicited by the SIT from individuals, families, or businesses in the local community. As one parent explained: “There’s a paint factory in our town and it supplied paint free of cost which we used to paint and decorate the walls of the school. We managed to make improvements to the school environment at a savings of NIS 14000 ($3500).” 1) Teaching and learning The consensus heard from all three groups is that the SIP process cast a much stronger spotlight on supporting more effective teaching and learning. Principals expressed satisfaction in seeing and supporting teachers who are using new strategies such as integrated teaching and collaboration among teachers. 39 As one principal remarked: “I really feel happy when I enter a class and find the math and Arabic teachers together collaborating to improve the students’ understanding.” Teachers were unanimous in their praise for changes resulting from the influx of technology resources thanks to LTD. The availability of LCD projectors, laptops and improved computer labs have, in their view, empowered teachers to teach more outside the box, moving away from didactic instruction and embracing more innovative teaching strategies they learned in their LTD trainings that get students actively engaged with the curriculum. They also maintained that they feel supported in trying new methods, like integrated teaching and peer collaboration, to improve student learning. A teacher commented: “The program (LTD) provided us with LCDs, laptops, and screens. This really motivated us! We worked together to make effective use of the technology, and the students became more actively engaged in class.” Just as teachers are seeing a difference, so too are parents. They commented that the SIP process redoubled the school’s attention on improving the learning conditions for students. For example, they point to new programs such as rewarding students with certificates when for academic progress, providing learning support for struggling students, and making the school library more engaging and learner-focused as efforts that are making a difference for students. 2) Management of material and human resources to improve teaching and learning Members of the three groups repeatedly linked improvements in the physical and learning environments of the school to the SIP’s emphasis on effective management of resources. As an illustration, school gardens—a common SIP initiative—were planted not only to beautify the school, but also to create learning spaces where students could transfer their classroom learning to a real￾world context. In some cases, schools even used gardens to cultivate “cash crops” such as cotton that were later sold locally to support other school development projects. Teachers—a school’s most important human resource—observed that the school’s improvement plan put them at the center of the school’s development. As one teacher pointed out: “You could see the creativity of the teachers increasing during the LTD program and there was a huge increase in teachers’ self-motivation. We helped the teachers of 11th and 12th grade, the Tawjihi, to grow professionally.” This observation was affirmed by a principal: “It wasn’t just me ‘the principal’ or the technology teacher or the science teacher working alone. No. What developed was a spirit of cooperation and teamwork like we were one big team and whatever we decided to do had to be decided by all.” 40 3) The use of technology in teaching and learning The consensus among all groups is that LTD’s support to the SIP process by equipping schools with Wi-Fi access, laptops, and LCD projectors has transformed the schools. One principal said it all: “We use to have only one LCD projector and internet only in the lab, but now that the whole school is connected to the Internet and there are three LCDs, life in school became better for the teachers and the students.” Teachers observed that the improved availability of technology resources has sparked a healthy competition among teachers to integrate technology and new media like YouTube and Facebook to get students more excited and engaged in the learning process. Principals were excited to see teachers who were enrolled in the LTD trainings to be coaching other teachers in school on using technology in their classrooms. Parents, too, stepped up to support the spread of technology in schools. In just one example of many like it, a school’s SIT managed to leverage its AMIDEAST resources to secure additional resources from the local village council to equip a computer lab and this helped turn around reading and math scores for weak students and provided reinforcement for students preparing for the Tawjihi exam. 4) Internal and external relations: All groups were unanimous in their view that the SIT was a catalyst for deepening and extending the network of relationships between the school and the local community. This was a major outcome of the school self-assessment study that each SIT conducted in formulating needs and objectives for the school improvement plan (SIP), where the results were shared with teachers and staff and then with local municipal councils. Communicating evidence-based school needs helped to improve networking among groups inside and outside the school, which often translated into bigger commitments from the local community to pledge support. In one example, the local community aided the school in purchasing whiteboards for classrooms and supplies for painting the school. As one principal proudly boasted: “My school’s relations with the local community has improved tremendously. Honestly, whatever issue I raise, whether it’s for blackboards or security bars for windows, I can count on the local community. Our communications with the local community have become really strong.” Seeing community relations becoming stronger, many teachers noticed an increase in parents’ attendance at meetings to discuss their children’s progress. Teachers and parents also found common cause in supporting co-curricular learning activities in which students produced handcrafts or traditional soap to be sold locally, with the profits going to the school to help it meet other needs. The work of the SIT also breathed new life into the school’s relationship with the parent’s council. As one parent described: “Our meetings have seen an increase in the numbers of parents in attendance, and we see them asking more about their children’s progress in school. For example, we had a problem with unexcused absences and managed to work out new rules with the school to help reduce the problem.” 41 5) Connecting Students and Teachers to the School Community: All groups maintained that the SIP implementation process contributed to big improvements in how teachers and students perceive their responsibility for making their school a successful learning community. In some schools the SIP process has been a catalyst for closer, more respectful teacher￾student relations not only in the classroom but also as citizens of the school community. In one example, a teacher explained proudly: “We established a student council and it’s been a success. We’ve seen students’ attendance improve. Students have a role and voice in school affairs and offer their opinions about improvements. I’ve been teaching for 16 years and this is the first time that students have attended teachers’ meetings, and we really listened to what they had to say about making the school better.” As earlier mentioned, environmental clubs have helped to increase students’ appreciation for protecting and beautifying the school grounds, involving initiatives such as planting gardens and painting and decorating walls. Some principals link these activities what they describe as a dramatic decline in vandalism, littering, and violent behaviors in their schools. One principal recounted the impact the club had on the misbehavior of some students: “We had some students after school who would play around and wind up breaking windows or damaging trees. When I found out who they were, I had them join the environment club and they really took an interest in planting and tending the school garden. In fact one of the kids got his neighbors to donate some metal bars which he used as stakes to support saplings in the garden.” 6) Challenges during the implementation phase of the SIP a) Too little time and too little money Given the seriously low-resource environment that the West Bank and Gaza’s school system operates in, there was no surprise that all groups contended that lack of sufficient financial resources threatened the capacity of schools to accomplish the strategic goals and objectives of their SIP. An additional risk to the timely completion of goals was the fact heavy administrative and teaching workloads sometimes hindered principals and teachers from devoting sufficient time and effort to carrying out tasks. b) Bureaucratic Inertia Principals mainly, but also teachers and some parents, complained that precious time was often lost in completing tasks during implementation as a result of centralized bureaucratic inertia. Too often, implementing tasks such as refurbishing facilities or installing equipment required following multiple rules and regulations that required authorization at different levels of the district and Ministry bureaucracy. In other words, a lack of decentralized authority sometimes proved counterproductive to an SIT trying to implement time-bound improvements inside their school. 42 V. Recommendations for Improvement of Leadership Training A. Based on the results of both the quantitative and qualitative research on principal effectiveness, the following areas are suggested for improving the Leadership Diploma Program. 1. Continue to build the principal’s capacity to involve parents, teachers, and other stakeholders of the school community in discourses and decisions on improving the school; in particular, attention should be given to learning how to improve the flow of information using technology to communicate with stakeholders. 2. Improve the principal’s capacity to involve parent members of the SITs in planning and collecting data for the school self-assessment. 3. Empower the principal with greater discretion to manage the workload and scheduling of SIT tasks and meetings so as to accommodate the limited free time that school staff and parents have during a typical workday. 4. Build the principal’s skills in giving constructive feedback and in managing differences of opinion in order to reach a consensus in decision-making. 5. Emphasize the responsibility of the principal and the SIT to systematically monitor and document the implementation of the SIP. 6. Continue to build the capacity of the principal and the SIT in managing resources intended to support the quality of teaching and learning and improve learning outcomes. 7. In light of the West Bank and Gaza’s chronically low-resourced education sector which, among other things, limits the capacity of a school to finance its annual SIP, the MoEHE should consider ways to empower principals to seek alternative revenue flows to fill budget deficits. B. Focus Group participants suggested the following actions to improve the Work of the SIT 1. Reduce the work load for teachers working on the SIP. 2. Free up one or two class sessions per week for teachers to hold their meetings. 3. Motivate teachers by providing them with some kind of appreciation for their efforts. 4. Improve coordination for scheduling professional development trainings for principals and teachers to avoid situations where both are absent at the same time. This would increase the time they both have to work on the SIP. 5. Increase the level of authority of principals participating in the LTD program so they have more flexibility in managing their heavy administrative workload by the ministry and the districts offices. 6. Improve the level of coordination between the schools on the one hand and the district directorates and the central Ministry on the other in order to avoid having to produce two “identical” SIPs and budgets—one for LTD and the other for the ministry. 7. Conduct a mid-term formative assessment and final summative evaluation of the SIP implementation process and share the findings to help improve the performance of SITs. 8. Encourage exchange visits among SIT teams from different schools and set up a Facebook page so as to facilitate sharing of ideas and information. 43 9. Use public relations strategies to demonstrate the work and outcomes of the SIP process to the school community and to local organizations and institutions. 10. Each newly organized SIT needs to take account of achievements, processes and resources from previous years so that new initiatives avoid needless duplication. 11. The SIP planning phase should pay more attention to aligning the national curriculum with specific goals to improve teaching and learning, for example, by planning and supporting co￾curricular field trips that make real-world connections to what students are learning in the classroom. 12. Standardize the LTD process for developing an SIP to all schools in the West Bank and Gaza. 44 Section 2: To what extent did LTD contributed to improving the capacity of teachers to enact learner-centered approaches to teaching and learning? Introduction A key goal of LTD is to promote the development of high quality teaching and learning. To achieve this, LTD has worked to build the capacity of the National Institute for Educational Training (NIET) to deliver high quality in-service professional development to underqualified (non-certified) teachers leading to their obtaining the equivalent of a teaching diploma. LTD implemented two interventions designed to achieve goal. Firstly, it designed and delivered a training-of-trainer program, the Teacher Educator KEY FINDINGS LTD contributed substantively to building the capacity of teachers to enact learner-centered approaches and strategies and to prepare their students with 21st century learning skills (critical thinking, creativity, communication, and collaboration). • LTD students are more likely to agree (65%) than their non-LTD peers (55%) that their schools provide a positive learning environment; non-LTD teachers, to the contrary, are slightly more likely (60%) than their LTD peers (55%) to think so. • LTD teachers (88%) and their students (68%) are more likely to describe their classroom environments as learner-centered than their non-LTD peers (81% and 54%) respectively. • LTD students (68%) are more likely to agree that their teachers are building their 21st century learning skills than their non-LTD peers (56%). • LTD teachers (87%) are more likely to agree that they are building their students’ 21st century learning skills than their non-LTD peers (80%). In particular, LTD students are more likely to make real-world connections to what they learn in class; do project-based learning; feel encouraged to offer their own opinions or ideas; and, participate in group work. LTD contributed to building the capacity of teachers and principals to promote the values and conditions that foster a child-friendly school. Based on a set of indicators measuring behavior, LTD students reported more positive school behaviors than their non-LTD peers. • Students of LTD teachers were less likely than their non-LTD peers to have hit a fellow student (34% vs. 49%) • Students of LTD teachers were less likely than their non-LTD peers to have been hit by another student (23% vs. 29%) • Students of LTD teachers were less likely than their non-LTD peers to have been hit by a teacher (43% vs. 64%) • Students of LTD teachers were less likely than their non-LTD peers to have skipped school (12% vs. 29%) • Students of LTD teachers were less likely than their non-LTD peers to have had their parents called to school because of misbehavior (14% vs. 20%). 45 Enhancement Program (TEEP), which created a national cadre educational trainers at NIET who are experienced in learner-centered instruction and assessment. Secondly, LTD provided technical assistance to NIET in the design and development of a 9-module curriculum for teacher certification. LTD used three research activities to understand the extent that teachers of Cohort II have enacted standards and competencies aligned with learner-centered instruction (Table 32): 1) a baseline/endline survey with a random sample of teachers, principals, and students was conducted in cooperation with NIET; 2) a quasi-experimental survey of classroom engagement involving a random selection of teachers, students, and a control group of non-LTD teachers and students for comparison; and, 3) in cooperation with the Assessment and Evaluation Department of the MoEHE, LTD administered standardized tests of achievement to students using a quasi-experimental design to a sample of students of Cohort II schools (pre- and post-test design); additionally, LTD administered the tests to students of teachers from Cohort I (pre-, post-, and post-post design)—results of the tests are reported in Section 4 below. Table 25. Data collection sample and methods for evaluating the impact of teacher training on teachers’ competences Data Collection Method Frequency of Data Collection Sample Classroom Engagement Survey Baseline & Endline Students Teachers LTD Controls LTD Controls 1607 918 58 35 Teacher Effectiveness Survey Baseline & Endline Teachers Principals 304 41 Standardized Tests of Achievement Baseline & Endline LTD Controls Cohort I (post-post) 435 319 Cohort II (post) 1117 1470 I. Survey of Classroom Engagement Method of Data Collection For the endline evaluation of Cohort II schools, a random selection of 58 LTD teachers and 35 non-LTD teachers across the 6 districts of Cohort II yielded a sample of just over 1600 LTD and 900 non-LTD students who participated in the Cohort II survey of classroom engagement.3 Based on the subjects taught by teachers, 22% of the students were from math classes, 24% from science, 21% from Arabic, 13% from English, and 20% from English. The survey instrument contained a 20 item, five-point Likert “Classroom Engagement Scale” (Cronbach's Alpha = .918). These same 20 items were divided into four subscales representing 21st century learning skills: Critical Thinking, Creativity, Communication, and Collaboration. The instrument was filled out by the selection of teacher and by their respective students. These two sources of data for the same variables—along with the comparison schools—made for a robust analysis 3 Nine teachers from the baseline study either transferred to other schools or were on leave when LTD conducted the endline study; thus, there were 350 fewer students in the endline than the baseline study. 46 of the contribution of LTD’s teacher training to enhancing students’ classroom engagement, development of 21st century learning skills, and their social behavior. A. Satisfaction with School Environment The results in Table 25 show that LTD students were more satisfied with their schools’ learning environment compared to non-LTD students. On average, LTD students were more satisfied with the learning environment of their schools (65%) than their non-LTD peers (55%) (Figure 10). Interestingly, however, LTD teachers’ estimation of their students’ satisfaction was slightly lower than their non-LTD peers (Figure 11). The variation in the means between LTD teachers and the comparison group was statistically significant. Table 26. Mean scores for satisfaction with school learning environment and classroom engagement Questions Control Schools LTD Schools % Difference QA1. I feel that my school is preparing me to be successful in my future learning 3.85 4.00 4% QA2. I am happy to be a student at this school 3.46 3.75 8% QA3. I feel excited when I come to school 3.15 3.30 5% Figure 9. Percentage of students who strongly agree/agree they are satisfied with their schools 77% 67% 49% 71% 60% 46% I FEEL THAT MY SCHOOL PREPARED ME TO BE SUCCESSFUL IN THE FUTURE LEARNING I AM HAPPY TO BE A STUDENT AT THIS SCHOOL I FEEL EXCITED WHEN I COME TO SCHOOL Students' agreement their school provides a positive school learning environment LTD Control 47 Figure 10. Percentage of teachers who strongly agree/agree their students are satisfied with their schools B. Learner-Centered classroom engagement The scores in Table 26 show that overall both LTD teachers and their students rated their classroom environments as more learner-centered than their non-LTD peers. The variation in the means between LTD teachers and the comparison group is statistically significant. Table 27. Scores for teacher and student rating of learner-centered classroom indicators Learner-Centered Classroom % Difference LTD Non-LTD Teachers 4.25 4.01 5.8% Students 3.80 3.39 11.4% On average, LTD students were more likely to agree (68%) that their classrooms reflect a learner-centered environment compared to their non-LTD peers (54%). LTD teachers were more likely to agree (88%) that their classrooms reflect a learner-centered environment compared to their non-LTD peers (81%). 78% 53% 33% 83% 66% 31% THE SCHOOL IS PREPARING STUDENTS TO BE SUCCESSFUL IN THEIR FUTURE LEARNING THE STUDENTS ARE HAPPY TO BE AT THIS SCHOOL THE STUDENTS ARE EXCITED WHEN THEY COME TO SCHOOL Strongly Agree/Agree LTD Control 48 Table 28. Percentage of students who strongly agree/agree on learner-centered indicators Questions LTD Control QB1. My teacher encourages me to think and find answers and solutions 77% 69% QB2. I Participate in small cooperative groups in the classroom 68% 55% QB3. I express my opinion freely in the class 65% 52% QB4. My teacher's style makes the learning process interesting 68% 56% QB5. I participate in a variety of classroom activities 63% 55% QB6. My teacher helps me when I find some difficulty to understand the lesson 79% 66% QB7. My teacher cares about my suggested ideas 63% 47% QB8. My teacher encourages me to ask questions in classroom 73% 60% QB9. My teacher helps me stay actively engaged. 58% 32% QB10. I don’t spend most of the time just copying information 65% 40% QB11. In most lessons, the teacher checks my knowledge and understanding 69% 54% QB12. My teacher gives me time to debate what I have learned in the classroom 67% 55% QB13. My teacher encourages students to discuss and debate 68% 54% QB14. My teacher's style of teaching helps me to understand easily 70% 61% QB15. I participate in implementing projects 76% 58% QB16. My teacher gives me feedback about my performance on tests and assignments 68% 54% QB17. My teacher gives me feedback about my performance during lessons 63% 53% QB 18. I participate in interesting activities 59% 51% QB19. My teacher use technology in the class 57% 46% QB20. My teacher makes real-world connections to what we learn in class 76% 61% Average 68% 54% Table 29. Percentage of teachers who strongly agree/agree on learner-centered indicators Questions LTD Control QB1. I encourage students to think to find answers and solutions 98% 91% QB2. Students participate in small cooperative groups in the classroom 76% 71% QB3. Students express their opinions freely within the class 90% 86% QB4. My teaching style makes the learning process interesting 93% 94% QB5. Students participate in a variety of classroom activities 83% 77% QB6. I help students when they find any difficulties in understand the lesson 100% 89% QB7. I care a lot of suggested ideas by the students 95% 91% QB8. I encourage students to ask questions in class 98% 94% QB9. I spend part of the time in the reading and writing of information 76% 51% QB10. Students do not spend most of the time in copy information 93% 86% QB11. In most classes, I ask students to memorize facts and figures with checking the degree of their knowledge and understanding 93% 89% QB12. I give students time to discuss what they have learned in the classroom 86% 77% QB13. I'm talking less than the students in the class to allow students to debate 90% 71% 49 QB14. My teaching style helps students to understand easily 84% 89% QB15. Students participate in projects 85% 71% QB16. I give students feedback about their performance on tests and assignments 98% 89% QB17. I give students feedback about their performance during the lesson 97% 77% QB18. Students participate in interesting activities 64% 54% QB19. I use technology in the classroom 71% 71% QB20. I link between what the students learning and the daily life 100% 94% Average 88% 81% C. 21st Century Learning Skills Tables 29 and 30 show the results for the extent that students and teachers strongly agree/agree that indicators or 21st century learning skills are present in their classrooms. The indicators are grouped into for domains: critical thinking, creativity, communication, and collaboration. We find that LTD students rate each skill area higher (by about 11%) than non-LTD students (Table 29). The variation in the means between LTD students and the comparison group is statistically significant. We see, too, that LTD teachers’ scores were higher (by about 7%) than the comparison group of non-LTD teachers. The variations in the means of the two groups is also significant. Table 30. Student rating of indicators of 21st century skills in their classrooms 21th Century Learning Skills Control Schools LTD Schools % Difference Communication 3.60 3.99 10% Critical thinking 3.43 3.83 11% Collaboration 3.35 3.72 11% Creativity 3.39 3.78 11% Table 31. Teacher rating of indicators of 21st century skills in their classrooms 21th Century Learning Skills Control Schools LTD Schools % Difference Critical thinking 4.27 4.50 5% Creativity 3.94 4.17 6% Communication 4.09 4.34 6% Collaboration 3.72 4.08 9% Overall, LTD students and teachers are more likely to describe their teaching and learning as contributing to the development of 21st century learning skills. LTD students (68%) are more likely to agree that their teachers use learning activities that develop 21st century learning skills—critical thinking, creativity, communication, and collaboration—than their non-LTD peers (56%) (Figure 15). LTD teachers concur, and they are more likely to agree (87%) than their not-LTD peers (80%). 50 Figure 11. Percentage of students who strongly agree/agree their classrooms reflect indicators of 21st century learning skills. Figure 12. Percentage of teachers who strongly agree/agree their classrooms reflect indicators of 21st century learning skills. On the assumption that teachers are likely to overestimate the quality of their teaching skills and that students are more likely to be more conservative, it is illuminating to compare how students from the two groups (LTD and non-LTD) rate the individual indicators comprising 21st century learning skills. Table 31 ranks the items from largest to smallest percentage of agreement. What is particularly revealing is the percentage difference between the two groups on specific teaching techniques. We see that compared to their non-LTD peers, LTD students are more likely to make real-world connections to what they learn in class; do project-based learning; feel encouraged to offer their own opinions or ideas; and, participate in group work. 75% 68% 65% 67% 62% 56% 52% 55% CRITICAL THINKING CREATIVITY COMMUNICATION COLLABORATION Student Agreement: 21th Century Learning Skills LTD Control 98% 81% 93% 77% 93% 75% 84% 69% CRITICAL THINKING CREATIVITY COMMUNICATION COLLABORATION Teacher Agreement: Preparing Students with 21th Century Learning Skills LTD Control 51 Table 32. Comparison of student ratings of indicators of 21st century leaning skills Critical Thinking LTD Control Difference QB20. My teacher make real-world connections to what we learn in class. 76% 61% 15% QB6. My teacher helps me when I find some difficulty to understand the lesson. 79% 66% 13% QB8. My teacher encourages me to ask questions in classroom. 73% 60% 13% QB4. My teacher's style makes the learning process interesting. 68% 56% 12% QB1. My teacher encourages me to think and find answers and solutions. 77% 69% 8% Average 75% 62% 12% Creativity LTD Control Difference QB15. I participate in implementing projects 76% 58% 18% QB3. I express my opinion freely in the class 65% 52% 13% QB14. My teacher's style of teaching helps me to understand easily 70% 61% 9% QB 18. I participate in interesting activities 59% 51% 8% Average 68% 56% 12% Communication LTD Control Difference QB7. My teacher cares about my suggested ideas 63% 47% 16% QB16. My teacher gives me feedback about my performance on tests and assignments 68% 54% 14% QB3. I express my opinion freely in the class 65% 52% 13% QB12. My teacher gives me time to debate what I have learned in the classroom 67% 55% 12% QB17. My teacher gives me feedback about my performance during lessons 63% 53% 10% Average 65% 52% 13% Collaboration LTD Control Difference QB15. I participate in implementing projects 76% 58% 18% QB2. I Participate in small cooperative groups in the classroom 68% 55% 13% QB5. I participate in a variety of classroom activities 63% 55% 8% QB 18. I participate in interesting activities 59% 51% 8% Average 67% 55% 12% D. Student Behavior A key assumption of LTD’s theory of change is that if teachers and principals enact the methods and techniques for effective schools and learner-centered classrooms they learn from their leadership and teacher trainings, a more student-friendly learning environment will be reflected in the behavior of students. To measure student behavior, the Classroom Engagement Survey asked students to respond to five questions and indicate how often they had:  Hit a fellow student  Been hit by a fellow student 52  Been hit by a teacher  Skipped school  Misbehaved in a way that resulted in a parent being called to school On all indicators, LTD students reported more positive school behaviors than their non-LTD peers (Figure 17). The results show that students of LTD teachers were less likely than their non-LTD peers to have hit a fellow student (34% vs. 49%), been hit by another student (23% vs. 29%), been hit by a teacher (43% vs. 64%), skipped school (12% vs. 29%), and to have their parents called to school because of misbehavior (14% vs. 20%). Figure 13. Indicators of misbehavior as reported by students 34% 23% 43% 12% 14% 49% 29% 64% 29% 20% I HIT A STUDENT DELIBERATELY I WAS HIT BY A STUDENT DELIBERATELY I WAS HIT BY A TEACHER I SKIPPED SCHOOL MY PARENTS WERE CALLED TO SCHOOL BECAUSE OF MY MISBEHAVIOR Indicators of Misbehavior as Reported by Students LTD Control 53 II. Survey of Teacher Effectiveness A. Teacher Effectiveness Survey The teacher effectiveness survey used a 4-point Likert scale that measures the extent that teaching competencies were met, where 1 is “Below expected level,” 2 is “Approaching expected level,” 3 is “Achieved expected level,” and 4 is “Exceeded expected level.” Results in Table 33 show that teachers reported substantive development across all seven competency domains as a result of their monthly face￾to-face trainings and bi-monthly learning circles. The results are ranked by percentage of change from largest to smallest. LTD is confident that the strength of the observed percentage of change is largely attributable to the innovative content and experiential learning approach of the 9-module Teacher Education training curriculum. The learning approach is inquiry-based and job-embedded. Teachers learn new methods and techniques and then try them out in their classrooms. They use action research document their classroom interventions, and then reflect on the results with peers in communities of practice (learning circles). KEY FINDINGS Through its technical support of NIET’s delivery of LTD’s training curriculum for teacher qualification, LTD contributed to the growth of teachers’ competencies across the seven domains of the Ministry’s standards for effective teaching. By the end of their LTD training, teachers in the program improved their capacity to: • Facilitate student-centered teaching and learning by 24%. LTD teachers design learner-centered teaching and learning to foster students’ active engagement in meaningful learning and assessment activities. • Design effective educational materials (lesson plans) and resources by 27%. LTD teachers plan lessons and units of instruction that take into account varieties of resources both inside and outside the classroom to improve teaching and learning. • Create a safe and effective learning environment by 23%. LTD teachers create a classroom environment that is child-friendly, treats students equally, and fosters creativity. • Monitor and evaluate the teaching and learning process by 25%. LTD teachers use a variety methods for formative and summative assessments of student learning. • Provide guidance and direction for learners by 17%. LTD teachers are prepared to deal sensitively and appropriately in addressing students’ cognitive, physical, emotional, and social well-being and needs. • Seek continuous professional development by 26%. LTD teachers are self-directed in pursuing their own professional learning through reflective practice and inquiry, and by participating in professional learning communities. • Encourage cooperation with stakeholders in the community by 22%. LTD teachers develop partnerships with peers, families, and community organizations to provide students with authentic contexts in which to transfer their learning and to receive additional support for their learning needs. • LTD’s strong emphasis on technology in its teacher trainings is a contributing factor to improving teachers’ competencies. A statistically significant difference in the teacher competency scores was found between competencies most associated with teaching and learning and the teachers’ use of the Internet to search for teaching resources and for professional development. 54 The observed changes seen in Table 33 also reflect the great emphasis the trainings place on building the capacity of teachers to design learner-centered lessons and units of instruction. Teachers also learn and experiment using alternative and authentic assessment activities to increase the active engagement of students. The smallest amount of observed change isfor the domain of “Providing Guidance and Direction for Learners.” This result is not entirely unexpected since the content of the teacher qualification curriculum does not include theories or methods diagnosing or intervening with special needs students nor about guidance and/or career counseling. Table 33. Results for seven domains of teaching competencies Teacher Competency Domains Baseline Endline % Change 2. Designing educational materials and resources 2.6 3.29 27% 6. Seeking continuous professional development 2.69 3.4 26% 4. Monitoring and evaluation of the teaching and learning process 2.64 3.3 25% 1. Facilitating student-centered teaching and learning 2.78 3.44 24% 3. Creating a safe and effective learning environment 2.74 3.37 23% 7. Encouraging cooperation with stakeholders in the community 2.63 3.21 22% 5. Providing guidance and direction for learners 2.92 3.41 17% The following seven tables present the results and percentage of change for the individual items comprising the MoEHE’s seven standards of effective teaching. The results are ranked in descending order from largest to smallest percentage of change. When interpreting the results, it should be remembered that a mean value of 3 indicates a satisfactory level competency, while a 3.5 or higher indicates the competency level was surpassed. 1. Facilitating student-centered teaching and learning Results for Standard 1 (Table 34) provide further validation of the findings reported previously that classrooms of LTD teachers are more likely to reflect a learner-centered environment than those of non￾LTD teachers. Specifically, LTD teachers design learner-centered teaching and learning to foster students’ active engagement in meaningful learning and assessment activities. Table 34. Means and percentage of change for Standard 1 Standard 1. Facilitating student-centered teaching and learning Pre Post % Change 1.4 I write learning outcomes that align with outcomes of the national curriculum. 2.59 3.36 30% 1.5 I involve students and other stakeholders to discuss the desired learning outcomes. 2.63 3.34 27% 1.6 I implement activities that engage students in collaborative learning. 2.73 3.46 27% 1.1 I plan monthly and daily lesson plans that consider the different learning styles of students. 2.73 3.41 25% 1.7 I implement learning activities that are relevant to lesson content, and ask questions to assess students' understanding. 2.84 3.52 24% 1.2 I consider individual differences among students. 2.99 3.54 18% 55 1.3 I consider students' prior knowledge. 2.95 3.45 17% Grand Mean 2.78 3.44 24% 2. Designing educational materials and resources Results for Standard 2 (Table 35) show that LTD teachers improved markedly in planning lessons and units of instruction that take into account varieties of resources both inside and outside the classroom to improve teaching and learning. Table 35. Means and percentage of change for Standard 2 2. Designing educational materials and resources Pre Post % Change 2.3 I use community resources (material and human) to improve the learning process. 2.34 3.02 29% 2.6 I involve students in developing different learning resources. 2.59 3.32 28% 2.5 I improve students’ creative learning and abilities by using a variety of teaching and learning resources. 2.64 3.35 27% 2.1 I design my annual plan to develop teaching and learning materials and reference their sources. 2.5 3.17 27% 2.7 I use a variety of teaching and learning approaches to achieve the goals of the curriculum. 2.69 3.4 26% 2.2 I use ICT in teaching and learning. 2.7 3.39 26% 2.4 I use teaching and learning that meet students' needs. 2.76 3.36 22% Grand Mean 2.6 3.29 27% 3. Creating a safe and effective learning environment Results for Standard 3 (Table 36) indicate that LTD teachers improved their capacity to create a classroom environment that is child-friendly, treats students equally, and fosters creativity. Table 36. Means and percentage of change for Standard 3 3. Creating a safe and effective learning environment Pre Post % Change 3.3 I involve students in the drafting school and classroom regulations. 2.5 3.17 27% 3.2 I create a learning environment that encourages students to learn through trial and error. 2.66 3.37 27% 3.6 I create a learning environment that promotes creative and critical thinking. 2.69 3.36 25% 3.7 I assign tasks to students that enhance their self-confidence in taking responsibility for their learning. 2.81 3.49 24% 3.1 I encourage student participation in different classroom activities. 2.88 3.51 22% 3.4 I provide equal learning opportunities for all students. 2.8 3.36 20% 56 3.5 I create a safe and healthy learning environment for students. 2.86 3.35 17% Grand Mean 2.74 3.37 23% 4. Monitoring and evaluation of the teaching and learning process Results of Standard 4 (Table 37) show that LTD teachers strengthened their capacity to use a variety methods for formative and summative assessments of student learning. Table 37. Means and percentage of change for Standard 4 4. Monitoring and evaluation of the teaching and learning process Pre Post % Change 4.2 I use the results of self-reflection to improve the process of teaching and learning. 2.39 3.23 35% 4.1 I design lesson plans to improve students' learning based on assessment results. 2.39 3.12 31% 4.6 I develop different assessment tools that fit the individual differences of students. 2.62 3.36 28% 4.3 I implement remedial learning programs to meet the specific needs of students based on assessment results. 2.54 3.23 27% 4.8 I select assessment strategies appropriate to the learning needs of students. 2.64 3.3 25% 4.12 I encourage students to use self-assessment. 2.62 3.27 25% 4.4 I provide parents with reports about their children's academic achievement. 2.49 3.08 24% 4.9 I document assessment results to follow up on the progress of students. 2.73 3.35 23% 4.7 I reflect on my practices to guide my professional development. 2.87 3.51 22% 4.1 I give constructive feedback to students based on assessment results. 2.84 3.45 21% 4.5 I use the results of monitoring and evaluation to improve teaching and learning. 2.75 3.3 20% 4.11 I use the monitoring and evaluation as a strategy in teaching and learning. 2.81 3.37 20% Grand Mean 2.64 3.3 25% 5. Providing guidance and direction for learners Results for Standard 5 (Table 38) show that LTD teachers improved their capacity to deal sensitively and appropriately in addressing students’ cognitive, physical, emotional, and social well-being and needs. 57 Table 38. Means and percentage of change for Standard 5 5. Providing guidance and direction for learners Pre Post % Change 5.6 I consult with experts to find appropriate solutions to students with learning difficulties. 2.44 3.06 25% 5.4 I give student tasks and assignments connected to their daily lives in the real world. 2.91 3.44 18% 5.5 I provide appropriate guidance in helping student to think about suitable career choices. 2.84 3.33 17% 5.2 I follow the appropriate procedures to improve student behavior. 2.98 3.44 15% 5.1 I provide students with proper guidance about their everyday well-being (e.g., health, hygiene and public safety and self￾discipline). 3.15 3.56 13% 5.3 I cultivate positive values and attitudes in students. 3.2 3.59 12% Grand Mean 2.92 3.41 17% 6. Seeking continuous professional development Results for Standard 6 (Table 39) indicate that LTD teachers improved their capacity to self-direct their own professional learning through reflective practice and inquiry, and by participating in professional learning communities. Table 39. Means and percentage of change for Standard 6 6. Seeking continuous professional development Pre Post % Change 6.6 I use action research to improve the teaching and learning process. 2.3 3.28 43% 6.7 I keep a portfolio to document events and activities to aid my professional development. 2.67 3.45 29% 6.2 I apply what I learn in training to promote active learning in the classroom. 2.76 3.56 29% 6.3 I share experiences with colleagues to do collaborative teaching and projects. 2.81 3.56 27% 6.5 I take advantage of appropriate methods to achieve students' learning outcomes. 2.62 3.31 26% 6.1 I use evaluation results to identify training needs. 2.55 3.22 26% 6.4 I search the Internet for relevant teaching resources. 2.79 3.47 24% 6.8 Participate in trainings and study days to develop my performance. 3.07 3.56 16% Grand Mean 2.69 3.4 26% 7. Encouraging cooperation with stakeholders in the community 58 Results for Standard 7 (Table 40) indicate that LTD teachers improved their capacity to develop partnerships with peers, families, and community organizations to provide students with authentic contexts in which to transfer their learning and to receive additional support for their learning needs. Table 40. Means and percentage of change for Standard 7 7. Encouraging cooperation with stakeholders in the community Pre Post % Change 7.1 I encourage students to engage in local community-service learning. 2.5 3.17 27% 7.6 I use community-based resources to improve the teaching and learning process. 2.63 3.26 24% 7.5 I engage experienced colleagues to support learning activities and collaborative projects. 2.79 3.44 23% 7.2 I participate in finding appropriate solutions to community problems. 2.5 3.07 23% 7.3 I provide parents with reports on the results of their students' academic performance. 2.63 3.15 20% 7.4 I cooperate with parents to resolve problems facing their children (i.e., behavioral, learning and health). 2.71 3.19 18% Grand Mean 2.63 3.21 22% Results of Statistical Analysis Our analysis investigated whether there were any statistically significant relationships between variations observed in scores for teaching competencies and demographic variables: sex, age, marital status, years teaching, subject taught, hours of effort related to teaching tasks, highest degree, skill level using a computer, availability of Internet at home, searching the Internet for teaching resources, or, for professional development. The analysis found no statistically significant differences in the teacher competency scores based on all but three of the variables—hours devoted to preparing for teaching (Table 41), using the Internet to search for teaching resources (Table 42), and using the Internet for professional development (Table 43). These results indicate that putting technology in the hands of teachers is empowering. Although we cannot attribute causality between these findings and LTD’s provision of Internet connectivity and laptops to teachers, this is evidence, nonetheless, that LTD’s strong emphasis on technology in its teacher trainings is a contributing factor to improving teachers’ competencies. 59 Table 41. Hours devoted to preparing for teaching ANOVA Sum of Squares df Mean Square F Sig. Facilitating student￾centered teaching and learning Between Groups 4.144 4 1.036 5.585 .000 Within Groups 50.819 274 .185 Total 54.963 278 Designing educational materials and resources Between Groups 5.201 4 1.300 5.610 .000 Within Groups 63.505 274 .232 Total 68.706 278 Creating a safe and effective learning environment Between Groups 2.313 4 .578 3.362 .010 Within Groups 46.455 270 .172 Total 48.768 274 Monitoring and evaluation of the teaching and learning process Between Groups 2.040 4 .510 2.734 .029 Within Groups 50.349 270 .186 Total 52.389 274 Providing guidance and direction for learners Between Groups 1.915 4 .479 2.046 .088 Within Groups 63.639 272 .234 Total 65.553 276 Seeking continuous professional development Between Groups 3.283 4 .821 3.991 .004 Within Groups 55.948 272 .206 Total 59.231 276 Encouraging cooperation with stakeholders in the community Between Groups 2.790 4 .697 2.433 .048 Within Groups 77.409 270 .287 Total 80.199 274 Table 42. Using internet to search for teaching resources: ANOVA Sum of Squares df Mean Square F Sig. Facilitating student￾centered teaching and learning Between Groups 2.187 4 .547 2.931 .021 Within Groups 56.144 301 .187 Total 58.330 305 Designing educational materials and resources Between Groups 5.801 4 1.450 6.392 .000 Within Groups 68.286 301 .227 Total 74.087 305 Creating a safe and effective learning environment Between Groups 1.763 4 .441 2.555 .039 Within Groups 51.068 296 .173 Total 52.832 300 60 Seeking continuous professional development Between Groups 3.017 4 .754 3.753 .005 Within Groups 59.893 298 .201 Total 62.910 302 Table 43. Using internet for professional development: ANOVA Sum of Squares df Mean Square F Sig. Facilitating student￾centered teaching and learning Between Groups 4.089 4 1.022 5.673 .000 Within Groups 54.241 301 .180 Total 58.330 305 Designing educational materials and resources Between Groups 7.999 4 2.000 9.108 .000 Within Groups 66.088 301 .220 Total 74.087 305 Creating a safe and effective learning environment Between Groups 4.732 4 1.183 7.280 .000 Within Groups 48.100 296 .162 Total 52.832 300 Seeking continuous professional development Between Groups 3.514 4 .879 4.408 .002 Within Groups 59.396 298 .199 Total 62.910 302 Encouraging cooperation with stakeholders in the community Between Groups 1.444 4 .361 1.227 .300 Within Groups 87.135 296 .294 Total 88.579 300 B. Principal’s Questionnaire As with the Principal Effectiveness Survey, in addition to having teachers’ self-evaluate their teaching competencies, we asked LTD principals to evaluate the teachers. As seen in Table 44 the percentage difference between the two groups on the seven standards are relatively modest; thus we can assume that the teachers’ self-evaluations were, despite being slightly elevated, a relatively accurate self￾assessment. Table 44. Comparison of results for teachers’ and principals’ assessment of teachers’ performance Standard Teachers Principal % Difference 1. Facilitating student-centered teaching and learning 3.44 3.24 6% 2. Designing educational materials and resources 3.29 3.17 4% 3. Creating a safe and effective learning environment 3.37 3.23 4% 4. Monitoring and evaluation of the teaching and learning process 3.30 3.10 6% 5. Providing guidance and direction for learners 3.41 3.21 6% 6. Seeking continuous professional development 3.40 3.18 7% 7. Encouraging cooperation with stakeholders in the community 3.21 3.12 3% Grand Mean 3.34 3.18 5% 61 Room for Improvement in Using Assessment to Improve Teaching and Learning What competencies might teachers want to work on based on their principal’s evaluation of their performance? We investigated this question by taking the average of the means of the items comprising each of the seven domains (from the endline scores). Those competencies that fell below the average are ones that teachers may wish to further develop. The results are shown in Table 45. Of note, there are three items most related to teaching practices that fell below the “satisfactory” threshold of 3.00, and these are clustered in Standard 4, which deals with assessment. What these imply is that teachers feel they are not making effective use of student performance data to adjust their teaching strategies or techniques to meet the learning needs of students. Table 45. Competencies falling below the average of the endline score 1. Facilitating student-centered teaching and learning (Avg. 3.24 within the domain) 1.4 I write learning outcomes that align with outcomes of the national curriculum. 3.16 1.5 I involve students and other stakeholders to discuss the desired learning outcomes. 3.12 2. Designing educational materials and resources (Avg. 3.17 within the domain) 2.1 I design my annual plan to develop teaching and learning materials and reference their sources. 3.08 2.3 I use community resources (material and human) to improve the learning process. 3.01 3. Creating a safe and effective learning environment (Avg. 3.22 within the domain) 3.3 I involve students in the drafting school and classroom regulations. 3.07 4. Monitoring and evaluation of the teaching and learning process (3.10 within the domain) 4.6 I develop different assessment tools that fit the individual differences of students. 3.07 4.3 I implement remedial learning programs to meet the specific needs of students based on assessment results. 3.06 4.12 I encourage students to use self-assessment. 2.99 4.1 I design lesson plans to improve students' learning based on assessment results. 2.98 4.2 I use the results of self-reflection to improve the process of teaching and learning. 2.97 5. Providing guidance and direction for learners (Avg. 3.22 within the domain) 5.5 I provide appropriate guidance in helping students to think about suitable career choices. 3.13 5.6 I consult with experts to find appropriate solutions to students with learning difficulties. 3 6. Seeking continuous professional development (Avg. 3.22 within the domain) 6.5 I take advantage of appropriate methods to achieve students' learning outcomes. 3.16 6.1 I use evaluation results to identify training needs. 3.05 6.6 I use action research to improve the teaching and learning process. 2.95 7. Encouraging cooperation with stakeholders in the community (Avg. 3.12 within the domain) 7.1 I encourage students to engage in local community-service learning. 3.06 7.2 I participate in finding appropriate solutions to community problems. 2.95 62 C. Results for PMP Indicator 3.2 One of LTD’s performance indicators that it reports to USAID is the percentage of participating teachers applying effective teaching methods in their classroom. LTD’s PMP target for this indicator is 60%; that is, we expect that 60% or more of the teachers are practicing effective teaching based on the MoEHE standards (Table 46). LTD’s benchmark mean (based on a 4-point Likert scale) was calculated by taking the weighted average of the combined endline scores (means) on the teachers' forms (n = 304) and principal's forms (n = 301) of the Teacher Effectiveness survey, which was determined to be 3.26 out of 5. Compared to the baseline mean of 2.53 (Table 47), there was a 29% improvement in effective teaching of the teachers. It was found that 61% of the teachers scored 3.26 or better on the endline results. Thus, LTD met its target of 60% for Indicator 3.2, “Participating teachers applying effective teaching methods in their classroom.” Table 46. Results for PMP Indicator 3.2 Target Actual 3.2: Percentage of participating teachers applying effective teaching methods in their classroom 60% (all cohorts) 61% Table 47. Calculation of benchmark score for teacher effectiveness Teacher's Self-Evaluation N Minimum Maximum Mean Std. Deviation TOT_PR 304 1.34 4 2.7142 0.49781 TOT_PST 306 1.87 4 3.3433 0.40694 Valid N (listwise) 304 Principal's Evaluation of Teacher N Minimum Maximum Mean Std. Deviation TOT_PR 301 1 3.56 2.346 0.47508 TOT_PST 304 1.54 4 3.1804 0.43962 Valid N (listwise) 301 Combined Weighted Averages of Principals plus Teachers Pre 2.53 Post 3.26 Percentage change 28.89124 63 Furthermore, it bears mentioning that the principals of Cohort II schools observed marked improvement the performance of LTD teachers in their schools at the completion of NIET’s Teacher Qualification training (Figure). III. Recommendations for Improvement of Leadership Training Based on the analysis of data from the principals’ assessment of their teachers’ performance on the Teacher Effectiveness Survey, the following teacher competencies are suggested for improvement in Teacher Qualification Training delivered by LTD’s partner, NIET. 1. Facilitating student-centered teaching and learning would benefit by enhancing teachers’ capacity to write learning outcomes that align with outcomes of the national curriculum and to involve students and other stakeholders in clarifying desired learning outcomes. 2. Designing educational materials and resources would benefit by building teachers’ capacity to select or develop a variety of teaching and learning resources designed to stimulate students’ creative and critical thinking; and, by designing units and semester plans to make effective use of available teaching and learning resources found in the as well as the larger community. 3. Creating a safe and effective learning environment would benefit by building teachers’ capacity to involve students in the drafting school and classroom regulations. 4. Monitoring and evaluation of the teaching and learning process (i.e., assessment) would benefit by building teachers’ capacity to: develop a variety of appropriate assessment tools that fit the individual differences of students; implement remedial learning strategies based on assessment results; to help students to use self-assessment; to design lesson plans to improve students' learning based on assessment results; and to use results of self-reflection to improve the process of teaching and learning. 5. Providing guidance and direction for learners would benefit by building teachers’ capacity to provide appropriate guidance in helping students to think about suitable career choices; consult with experts to find appropriate solutions to students with learning difficulties. 23% 36% 0% 5% 10% 15% 20% 25% 30% 35% 40% Percent Change in Teachers' Competencies by End of LTD Training Reported by Teacher Reported by Principal Figure 14. Change in Teacher effectiveness as reported by teachers and principals 64 6. Seeking continuous professional development would benefit by building teachers’ capacity to take advantage of appropriate methods to achieve students' learning outcomes; to use evaluation results to identify training needs; and, to use action research to improve the teaching and learning process. 7. Encouraging cooperation with stakeholders in the community would benefit by building teachers’ capacity to encourage students to engage in local community-service learning; and to engage with families and community members to find appropriate solutions to learning difficulties facing students. 65 Section 3: To what extent did LTD contribute to enhancing the capacity of principals and teachers to utilize technology for improving leadership, instruction, and professional development? I. Robust In-Kind Assistance for Technology Infrastructure LTD sought to understand the extent to which principals and teachers use LTD-supplied technology resources to enhance their professional development and improve school effectiveness. By investing substantial resources toward guaranteeing all 300 LTD school buildings have Internet connectivity and Wi￾Fi accessibility, and by equipping principals and teachers with laptops and LCD projectors, LTD expects to enhance the quality of teaching and learning and the effectiveness of administrative performance. Findings reported in this section are based on the analysis of data extracted from the endline surveys for Principal Effectiveness, Teacher Effectiveness, and Classroom Engagement. II. Detailed Results The results of the three surveys (Principal Survey, Teacher effectiveness Survey, and Classroom Engagement Surveys) provide strong evidence that LTD contributed to the capacity of principals to embrace the use of technology towards improving their leadership in school management and instructional supervision. As seen in Figure 15, based on the average score of four indicators on the Principal Effectiveness Survey, the principal's use of Technology grew by an impressive 27%. Even more significant in terms of sustainability is the fact that the principals' use of technology for their own development grew by 29%. KEY FINDINGS LTD’s provision of technology resources and training to both school leadership and teachers contributed to improvements in school effectiveness. • LTD contributed to a 27% growth in the principals’ capacity to use technology towards improving their leadership in school management, instructional supervision, and community relations. • Based on students’ assessment of their teachers’ classroom practices, LTD teachers are 10% more likely than non-LTD teachers to use technology in the classroom. • Teachers’ use of technology to search online for teaching resources grew by 24%, which is a strong indicator that their LTD training boosted their capacity to take responsibility for their own professional learning. 66 Figure 15. Observed growth in principals’ use of technology Teachers reported a 25% increase overall in their use of technology towards enhancing teaching and learning, in researching subjects they teach, and in their professional development (Figure 16). Furthermore, teachers’ use of technology to search online for teaching resources grew by 24%, which is a strong indicator that their LTD training enhanced their capacity to take more responsibility for their own professional learning. Figure 16. Observed growth in teachers’ use of technology Q7.3. I use technology to facilitate administrative functions Q7.5. I use technology to communicate with teachers and school staff and stakeholders Q7.6. I use technology to search for resources on teaching and learning Q7.7. I use technology in professional development Pre 2.94 2.71 2.84 2.72 Post 3.68 3.39 3.60 3.52 % Change 25% 25% 27% 29% 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 Principal's Use of Technology Q2.2. I use ICT in teaching and learning. Q6.4. I search the Internet for relevant teaching resources. Pre 2.7 2.79 Post 3.39 3.47 % Change 26% 24% 1.00 1.50 2.00 2.50 3.00 3.50 4.00 Teachers' Competency Using Technology 67 Although teachers’ responses on the Classroom Engagement Survey point to no real difference between LTD and Control schools in their use of technology in the classroom, with just a 2% difference between the two groups, scores from the students’ responses to the same question indicate that LTD teachers are 10% more likely than non-LTD teachers to use technology in the classroom (Figure 17). What’s more, the students’ results were found to be statistically significant based on an independent samples t-test. Figure 17. Observed growth in teachers’ use of technology 4.03 3.94 3.42 3.10 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 LTD Control LTD Control Teachers (2% difference) Students (10% difference) Teacher's use of technology in the classroom 68 Section 4: To what extent did LTD contribute to improvement in Student Achievement? Introduction Over the past two years, AED has administered achievement tests in four subjects to two cohorts of students taught by LTD teachers. Students of Cohort I teachers took a pre-test at the start of their teachers' LTD training in October 2013 and a post-test at the end of the training in May 2014. A year later a new batch of students of the same Cohort I teachers took a "post-post" test in May 2015; this was done to explore the impact of teachers’ instructional practices a year after their training had ended. Similarly, students of Cohort II teachers took a pre-test in October 2014 and a post-test in May 2015. Table 48. Data collection sample for AED tests of student achievement Data Collection Method Frequency of Data Collection Sample Standardized Tests of Achievement Baseline & Endline LTD Controls Cohort I (post-post) 435 319 Cohort II (post) 1117 1470 I. Results for Cohort I Post-Post Tests As seen in Figure 18, LTD students in May 2014 scored higher than their non-LTD peers on the post-post tests in English and Math, but slightly lower in Arabic and Science. A year later in May 2015 (Figure 19) LTD schools compared to the controls scored substantially higher in English and slightly higher in math, while scoring lower in Arabic and science. KEY FINDINGS LTD contributed somewhat to improvements in student achievement in two three out of four core academic: Arabic, English, and science. • A post-post study in May 2015 of Cohort I schools found that LTD students scored higher in two out of four tests of achievement compared to controls. LTD scored substantially higher in English and a little higher in math, but scored slightly lower in Arabic and science. These results tend to mirror the results found one year earlier in May 2014. • A post-study in May 2015 of Cohort II schools that LTD students scored higher in three out of four tests of achievement compared to controls. LTD students scored higher in Arabic, English, and science and scored just slightly lower in Math than the controls. 69 Figure 18. Cohort I: post-test results May 2014—scores are out of 100 Figure 19. Cohort I: Post-Post Test results May 2015—scores are out of 100 II. Results for Cohort II Post Tests As seen in the chart below, LTD students scored higher in the post-test than their non-LTD peers in Arabic, English, and science, and scored just slightly lower in Math. When comparing these results with LTD schools from Cohort I, the notable difference is in the better performance of students in science. This difference might be the result of NIET’s efforts to improve both the training curriculum and the trainers’ performance. This explanation has merit since the baseline scores of the two groups were virtually equivalent, 22.22 for LTD students and 22.23 for controls. Further research needs to be done, however, to investigate the precise causes of the differences in all the scores. 51.12 58.25 26.05 33.29 48.89 50.26 23.65 33.80 ARABIC ENGLISH MATH SCIENCE LTD Cohort I: Post-Test Score Results May 2014 Post LTD Post Control 42.07 47.51 22.90 31.61 46.95 36.32 21.92 33.36 0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00 40.00 45.00 50.00 Arabic English Math Science Score Cohort I: Post-Post Test Score Results May 2015 Post-Post LTD Post-Post Control 70 Figure 20. Cohort II: post-test results May 2015—scores are out of 100. Discussion The results of the post scores for both cohorts suggest that LTD’s teacher training may have contributed to the difference in outcomes between students of LTD and non-LTD teachers. At the same time, however, the observed decline in the post-post scores observed for Cohort I students might suggest that teachers were unable to sustain their impact on student achievement one year later. But this would be a premature conclusion. In reality, there are many factors besides the teacher that contribute to a student's academic performance, including individual characteristics, quality of teaching from previous years, family influences, community factors, and so on. Moreover, judging teacher performance based on one or two years of test results is unreliable.4 Nonetheless, the research is very clear that of all the factors that impact student learning and achievement, it is the teacher that matters most. And based on the multiple sources of evidence provided in this report, LTD is confident in making that claim that its model of school-based reform has empowered LTD teachers, and the principals supporting them, to enhance the quality of learning for students. 4 The literature on the use of student achievement data in estimates of teachers’ “value-added effectiveness” is clear that it must be done across multiple years, not one or two. See for example: Steele, J. L., Hamilton, L. S., & Stecher, B. M. (2011). Using student performance to evaluate teachers. Santa Monica, CA: Rand Corporation. Also, Baker, E. L., Barton, P. E., Darling-Hammond, L., Haertel, E., Ladd, H. F., Linn, R. L., and Shepard, L. A. (2010). Problems with the Use of Student Test Scores to Evaluate Teachers. EPI Briefing Paper# 278. Economic Policy Institute. 46.34 38.18 24.77 37.08 40.96 32.14 25.59 35.84 5 10 15 20 25 30 35 40 45 50 Arabic English Math Science Score Cohort II: Post-Test Score Results Post LTD Post Control 71 Section 5: To what extent did LTD contribute to sustaining the capacity of teacher educators to apply effective training approaches and practices? Introduction As reported in last year’s Annual Report (FY2014), LTD achieved one of its four Intermediate Goals, the establishment of the National Cadre of Teacher Educators at the National Institute for Educational Training (NIET). TEEP was co-designed and co-delivered by AMIDEAST sub-contractor, the University of Massachusetts, Amherst. In all, some 25 NIET trainers and 19 faculty consultants underwent a rigorous 270-hour inquiry-based and job-embedded professional development program called the Teacher Education Enhancement Program (TEEP). The purpose of the TEEP program was to ensure NIET’s capacity to deliver high quality training to in-service principals enrolled in the Leadership Diploma Program and to in-service teachers enrolled in the Teacher Qualification Program (funded by the European Joint Financing Agreement). The TEEP program culminated in a graduation ceremony in June 2014 to award certificates of completion to the TEEP graduates and to mark their induction into NIET’s National Cadre of Teacher Educators. I. Quality of Trainers’ Performance after TEEP A key indicator used by NIET to assess quality are the scores obtained from evaluation surveys filled out by trainees typically after every three face-to-face trainings. Since the completion of the TEEP program, NIET, with occasional technical support from AMIDEAST, has taken responsibility for providing continuous professional development in order to monitor and maintain a high level of quality of the trainings being delivered to in-service principals and teachers. NIET’s training survey assesses trainees’ agreement with a set of statements grouped into six domains: learning outcomes; training content; training methods and activities; administrative matters; physical amenities; and assessments. NIET collects and enters the survey data and then shares the data files with Key Findings The performance of NIET’s trainers for the Leadership Diploma Program were evaluated by 144 in-service principals enrolled in the program. • The results exceeded by 11% the benchmark for effective performance across six training competencies: using a variety of learning activities; balancing theory and practice; practicing active learning and learner-centered techniques; employing educational technology and new media; facilitating critical thinking; and helping trainees to transfer their learning to the real￾world context of their workplace. The performance of university consultants who trained in-service teachers for the Teacher Qualification Program delivered by NIET were evaluated by over 700 teachers enrolled in the program. • The results exceeded by 8% the benchmark for effective performance across NIET’s six training competencies. 72 LTD’s M&E Department. Before conducting the analysis, LTD puts the files through a process of screening and cleaning to ensure data quality. For evaluation purposes, LTD focuses on the two domains (scales) that measure the quality of a trainer’s performance, namely, training methods and assessment methods.  Training Methods: Six variables comprise the scale of training methods and activities. These cover the following: using a variety of learning activities; balancing theory and practice; practicing active learning and learner-centered techniques; employing educational technology and new media; facilitating critical thinking; and helping trainees to transfer their learning to the real-world context of their workplace.  Assessment Methods: Five variables comprise the scale of training assessment methods, and these items cover: using a variety of assessment methods; selecting appropriate assessment activities; employing continuous assessment; providing continuous feedback; and allowing sufficient time to complete all assessment activities. NIET’s survey uses a 4-point Likert agreement scale (Strongly Agree, Agree, Disagree, and Strongly Disagree). Because the target value in LTD’s PMP is based on a 5-point scale, for reporting purposes LTD converted NIET’s 4-point scale to a 5-point scale. II. Findings LTD considers the target value of 3.67 on a 5-point scale to be the benchmark value indicative of an effective trainer according to NIET’s current set of competences for effective trainers. Evidence from the trainees’ evaluations of their trainers’ performance indicates that NIET’s national cadre of trainers continue to provide quality professional development for principals and teachers. Table 49 indicates that the mean scores for trainers of the Leadership Diploma Program ranges from 3.92 to 4.22, representing a weighted average, or grand mean, of 4.10.5 This score is approximately 11% higher than the minimal score 3.67 for effective training performance. Table 49. Results of trainer evaluation surveys District Mean # of Sessions Average # of Participant Hebron 3.92 6 17 Bethlehem 4.16 2 19 Ramallah 3.93 5 21 Salfit 4.22 9 22 Jerusalem Sub 3.96 4 17 Qalqilyia 4.22 8 22 Grand Mean (weighted) 4.10 5 The weighted average was calculated by multiplying of each component by the number of trainees in each subject specialization—Arabic, English, mathematics, science, and technology education. 73 Table 50 shows that the mean scores for trainers of the Teacher Qualification Program across the five subject specializations range from 3.88 to 4.07, representing a grand mean of 3.96. This score is 8% higher than the minimal score 3.67 for effective training performance. Table 50. Results of trainer evaluation surveys TRAINER EVALUATION (18 trainers) District Arabic English Math Science Technology Education Hebron 3.83 3.76 3.96 3.88 3.64 Bethlehem 4.01 4.13 3.84 4.00 4.21 Ramallah 4.05 4.08 3.90 3.84 4.00 Salfit 4.22 4.00 4.38 N/A 3.97 Jerusalem Sub 4.04 4.14 3.77 3.75 3.80 Qalqilyia 4.11 4.19 3.83 3.86 4.13 Average/Specialization 4.06 4.07 3.89 3.88 3.99 Grand Mean (weighted) 3.97 74 Section 6: To what extent did LTD contribute to building the capacity of leadership, instructors and students to engage in planning and action leading to improvement in organization functioning and pedagogical practice in the Faculty of Education of Al-Azhar University, Gaza? Introduction Since September 2013, LTD worked with the Faculty of Education of Al-Azhar University, Gaza, to achieve its goal of improving the quality of pre-service education in the Faculty of Education. This effort took a three-fold approach:  Strategic Planning to align curricular policies, structures, processes and practices through a process of strategic planning focused on learning  Teacher Educator Enhancement to harmonize faculty professional development with the strategic plan through action research in communities of practice to ensure all students learn  Professional Certificate in English Language Teaching to help future teachers of English build the professional habits of learner-centered teaching and professional development I. Strategic Plan A. Background Over a two-year period from 2013 to 2015, AMIDEAST, with technical support from UMass, delivered a capacity-building initiative to support the development of a 3-year strategic plan for Al-Azhar University’s Faculty of Education. As much a process as a product, the goal of the initiative was to define and align the vision and mission of the Faculty of Education with its priorities for improving the quality of its pre-service courses and enhancing the readiness of its graduates to meet the MoEHE's standards for learner-centered instruction. After forming a steering committee comprised of the heads of the Faculty’s four departments and chaired by the dean of the Faculty, the committee undertook an eight-stage strategic planning process that included the development of mission and vision statements, a SWOT and gap analysis, formulating strategic goals and objectives with well-defined targets and indicators, creating structures to monitor and KEY FINDINGS LTD contributed to the successful capacity-building of leadership of the Faculty of Education to carry out systematically the process of strategic planning based on international standards of best practice for higher education institutions. Members of the Strategic Planning Steering Committee of the Faculty of Education demonstrated their abilities to: • Collaborate in producing a viable and future-oriented vision, mission and goals. • Commit to shared leadership during all stages of the process. • Build a consensus among the Faculty leadership and teaching staff towards embracing the vision and mission of the Strategic Plan. • Devote the necessary scope and depth of analysis involved in the process. • Assess the relationship of the plan to budget, human capacity, and local realities. • Ensure the inclusion of all stakeholders in the development and implementation of the plan. • Commit to ensuring the ongoing review, evaluation and adjustment of the plan. 75 assess implementation, winning the plan’s approval by the university administration, implementing the plan, and institutionalizing the process for future strategic planning. The strategic plan received official approval on December 15, 2014. Educational technology is a major priority of the plan, and LTD has provided in-kind assistance for retrofitting three lecture halls with ICT technology to enhance innovative teaching and learning. In June 2015, LTD conducted an in-depth interview with the dean of the Faculty of Education and three heads of departments to understand what the Faculty gained from the planning process. B. Findings from In-Depth Interview 1. Ownership of the Process Overall, the committee members said that the process was very thorough and systematic, but in their view what distinguished it from past strategic planning was its inclusiveness. They commented that this was the first time in anyone’s memory that the head of departments and department faculty members were involved in every step of the process, from developing the mission and vision, to participating in the SWOT analysis, and to prioritizing needs to drafting the actual document itself. A strong feeling of ownership emerged from the process. They appreciated the structure and guidance provided by UMass, which they feel empowered them with skills they will use again in future strategic planning. The only regret they had was that UMass was unable to provide comments and feedback before the finalized plan was sent for printing.6 2. Cooperation and Teamwork The committee members appreciated the cooperative structure of meetings and workshops that created a respectful and collegial exchange of ideas and feedback. Terms of reference were developed through a consultative process so that tasks and responsibilities were clearly defined and distributed equitably. This created an inclusive and collaborative working environment unlike past strategic planning that was typically done exclusively by the dean. In contrast, this time the dean served as the chair of the committee but entrusted leadership responsibilities to the heads of the departments, who in turn brought other faculty members on board to give input. In sum, the members of the committee felt that the process encouraged multiple perspectives and an atmosphere of inclusiveness and collegiality that fostered consensus on goals, values and priorities that they and their respective units aspire to. 3. Crafting mission and vision statements The members noted that a lot of discussion and debate went into shaping the mission and vision statements of the Strategic Plan. They admitted they hadn’t realized how complicated the task would be, but they came to appreciate the importance of aligning the goals and priorities of each of the four departments with the mission and vision of not just the Faculty of Education, but also with the university as a whole. They concluded that the process of discussion, debate and consensus building enabled them 6 For budgetary reasons beyond the control of AMIDEAST, the sub-contract with UMass had to be discontinued, and this occurred several months before the Strategic Planning process was to be completed. “We are proud of the Strategic Plan because we wrote each word in it and now we are committed to making it a reality.” “It was no longer a one man show. We were all allowed to put our thoughts and state the needs to create a better learning environment for our students” 76 to focus productively on the development of goals, benchmarks, strategies and activities that will empower teachers and students to make the most of available resources. 4. Prospects for success The members anticipate that the new strategic plan will impact the quality of pre-service programs. In particular, they were excited by the priority given to the technology resources that LTD’s in-kind assistance made possible. Many of the inter-departmental goals of the Plan were designed to maximize support for the innovative uses of technology in teaching and learning. They added, however, that monitoring the progress of implementation is critical. In this regard, the Steering Committee appointed the heads of departments the responsibility of monitoring the implementation of activities in their respective departments, and to report monthly to the full committee. One of the lessons learned from the planning process was the importance that monitoring would play as a mechanism for ensuring the harmonization of the Faculty’s policies and resources with the vision, mission, and goals of each of the four departments. II. Teacher Educator Enhancement Program (TEEP) A. Satisfaction Survey At the completion of the TEEP program, the participants completed a 10-question satisfaction survey based on a 5-point scale of agreement. Eleven of the 12 participants returned the survey. One of the 10 questions—Question 8—was excluded in the calculation of the grand mean because it was not intended to measure satisfaction with the training content or learning goals. Table 51 presents the results ranked from largest to smallest: Table 51. Results of TEEP satisfaction survey Questions Mean Q4 The TEEP program was applicable across academic specializations. 4.91 Q10 I am convinced TEEP resulted in my students becoming more actively engaged in the classroom. 4.91 Q7 The idea of self-reflection on my teaching practices was made clear to me. 4.82 Q3 I benefited learning about Action Research. 4.80 Q2 The information provided was useful for my continuous professional development. 4.73 KEY FINDINGS The TEEP program contributed substantively to building the capacity of 12 full-time members of the Faculty of Science to engage in planning and action leading to improvement in learner-centered pedagogical practice in pre-service teacher education courses.  In a survey at the end of the TEEP training, the participants as a group rated the impact of the program on their teaching practices as 4.71 out of 5, which is the equivalent of a 94% approval rating.  TEEP enabled the participants to rely less on didactic instruction and more on student-centered and active learning activities.  TEEP built the capacity of the participants to engage in individualized professional learning by practicing the inquiry cycle of action research, learning circles, and monitoring growth by keeping a portfolio of professional practice. 77 Q6 I now have a better understanding of how to conduct action research to enhance my teaching practices. 4.73 Q1 There are aspects of my practice where I've seen improvement. 4.55 Q9 I believe that TEEP has contributed to changing my performance in classroom. 4.55 Q5 The practice of being a "critical friend" was clear. 4.44 Q8 I had some prior knowledge about many of the topics covered by TEEP. 3.82* Grand Mean 4.71 * The mean for Question 8 was excluded from the calculation of the grand mean. The results indicate that even though none of the participants had any background in education or pedagogy, they believe that the TEEP training substantially developed their pedagogical content knowledge and skills. Likewise the results indicate that the participants valued learning about and engaging in reflective practice and action research and in sharing their experiences with “critical friends.” Though the relatively short duration of the TEEP program did not permit the participants to engage repeatedly in action research projects, they did report however that they have begun to see improvements in their own teaching practices and increased engagement of their students. These findings are corroborated in the results of focus groups, which are presented in the following section. B. Impact of TEEP on Teacher Educators in the Faculty of Education and Faculty of Science Focus groups were conducted with 10 TEEP alumni—4 from the Faculty of Education, who were part of TEEP I that ran from August 2013 to May 2014, and 6 from TEEP II that went from February to August 2015. The main goal of TEEP was to help instructors of students in pre-service teacher programs to build their capacity to rely less on traditional lecturing and more on student-centered and active learning strategies. TEEP provided the participants an experiential and inquiry-based process in which they conducted action research, engaged in reflective practice and learning circles, and documented their professional growth using portfolios of professional practice. The main question of the focus groups asked the participants to discuss how TEEP made a difference in their attitudes, values, and practices as a teacher educator. Additional questions on the benefits of being a part of a professional learning community, and whether they could attribute changes in their students’ learning as a result of their participation in TEEP. Two 90-minute focus groups were moderated by El Wafa Company upon the invitation of AMIDEAST. El Wafa is a private company contracted by USAID to monitor and report on USAID’s funded-projects in Gaza. 1. TEEP’s impact on instructors’ attitudes, values, or beliefs as teacher educators The participants explained that the experience of reflecting on their own problems of practice by doing action research, by experimenting with new techniques, and by reflecting on their experiences with colleagues and in their portfolios increased their understanding their students as learners, not just pupils. Instead of seeing their students as passive learners simply soaking up knowledge in a “A teacher is like a mirror for his students. I’ve come to understand that as the performance of the teacher improves, the performance of the students improves.” 78 lecture class, they now view them as active learners who need to be engaged critically and collaboratively in what they are learning. Moreover, the participants claimed that TEEP helped them build closer professional relationships with colleagues as an interdisciplinary community of practice, something they felt had been missing in their professional development in higher education. Teachers highlighted that TEEP offered them the opportunity to see themselves as part of a community of practice—something quite new for them— that gave them a space for sharing and exchanging experiences and ideas among colleagues. 2. TEEP’s impact on instructors’ practice as teacher educators The participants agreed that TEEP improved how they think about and design their lessons. They liked the logic of “backward design” because it helped them develop learning outcomes from the perspective of their students’ prior knowledge, and to link outcomes to big ideas and essential questions in their course syllabus. They liked the strategy of planning a lesson by first clarifying what students were expected to know or be able to do and then to come up with assessment activities that would allow students to demonstrate their learning. TEEP helped them see that teaching is more than just lecturing about content and grading students on a midterm and final exam. They acknowledged that TEEP improved their capacity to engage students more actively through a variety of alternative assessment methods. They mentioned, for example, the use of pre-planned open-ended questions to stimulate critical thinking and discussion; or, the use of brainstorming, small group discussions, and problem solving to encourage collaborative learning. They remarked that many of these strategies were ones they picked up from the activities they did in the TEEP workshops and then applied to their own classroom instruction. 3. The impact of action research on identifying and resolving problems of practice The participants were unanimous in affirming that the process of doing action research enhanced the capacity to identify problems of practice and to experiment with different teaching methods to improve their students’ learning. They also observed that action research forced them to become more reflective about their teaching and its impact on their students’ learning. Some even went as far as adding action research to the skills that they wanted their students to learn, especially since all students have to do practice teaching in schools as part of their practicum requirement. The main challenge to doing action research, however, is class size. Some of the participants admitted being reluctant to use action research in courses with large student enrollments (anywhere from 100 to 250) because it was simply too challenging logistically on the one hand, and all but impossible to identify different learning needs among such large numbers of students. “TEEP provided us with a space to meet with other colleagues to discuss, reflect and share experiences. And it added to my professional career as a member of the administration of the Faculty of Education, and even had a positive impact on my life outside academia.” “After TEEP, I and my students become more interactive and engaged in a participatory way. Students started to be more engaged not as receivers but as learners and participants.” “As we learned to do action research, we tried to transfer this experience to our own students. We asked them to prepare their own action research and use it to improve their research skills.” 79 4. The impact of working with critical friends The participants described their enthusiasm for working with a critical friend. They said that the practice of developing a trusting, confidential and supportive relationship with a colleague solely for the purpose of professional growth was both a novel and transformative experience. The also appreciated that the structure of the TEEP workshops further reinforced critical friend relationships by frequently using small group and collaborative learning techniques. In this way, the workshops connected the pairs of critical friends into a larger professional learning community among members of the same department or among those with similar specializations. C. Challenges and recommendations  Although the participants welcomed the inquiry cycle of action research, reflective practice and sharing feedback with critical friends, they admitted that changing the culture of the university was easier said than done. The norm is for teachers to use very didactic, lecture-based instructional practices. Getting more members of the faculty to switch over to learner-centered approaches required the commitment of the university administration to provide the time, resources and incentives to make it happen. It was suggested that there needed to be an annual strategy for implementing recurring trainings and workshops to spread the TEEP approach to professional development.  The participants recognized the impact of their action research on the attitudes, behaviors, and performance of their students doesn’t happen overnight. It is an incremental process. Thus, they suggested there should be some way for tracking students over multiple semesters and providing their teachers with follow-up or refresher workshops to increase the sustainability of TEEP’s impact on the performance of both teachers and their students.  While there was broad agreement that reflective practice is aided by keeping a portfolio of professional practice, the participants recommended that if future workshops use them, there needs to be a more systematic process in place to facilitate follow-up and feedback on the evidence of professional growth being documented.  The participants recommended that Al-Azhar University should reach out to other universities and faculties in Gaza to share the TEEP approach to professional learning. “As I am teaching highly theoretical and abstract subjects, TEEP has had an impact on my teaching practices in terms of having critical friends among my colleagues with whom I can discuss challenges I face in teaching such abstract subjects.” 80 III. PCELT Introduction The Leadership and Teacher Development program (LTD), funded by USAID and implemented by AMIDEAST, offers the Professional Certificate for English Language Teachers (PCELT) to undergraduates in their final year of the pre-service English teaching program at Al-Azhar University. To learn about their teaching experiences since completing the course in applying their PCELT skills in whatever formal or non￾formal contexts they might be teaching in, LTD/Gaza conducted focus groups with PCELT graduates in November 2014 and again in June 2015. In all, the participants included 16 women and 13 men. In presenting the results, it is necessary to appreciate the challenging contexts in which PCELT was delivered. Firstly, AMIDEAST was prohibited from working directly with in-service teachers from government schools because of USAID compliance regulations. This required changes to the PCELT curriculum and its mode of delivery. Designed originally for in-service teachers, AMIDEAST recalibrated PCELT to accommodate 4th year students enrolled in the English pre-service teacher education program of Al-Azhar University, Gaza. Unlike in-service teachers who have actual formal teaching experience, the only teaching these student had was limited to practice teaching; hence, they had less of a professional knowledge based with which to make sense of their training. Secondly, in the normal context of a pre-service teacher education program, students would be expected to enter the teaching workforce almost immediately after graduation. This is not so for Gaza. With unemployment hovering at 45.5% and almost 80% of the population living well below the poverty line KEY FINDINGS LTD contributed substantively to building the capacity of students to engage in planning and action leading to improvement in learner-centered pedagogical practices in pre-service teacher education courses for teachers of English. PCELT enabled participating students to:  Improve their perceptions of students and their learning. They believe that PCELT increased their sensitivity to the emotional, cognitive, and social needs of learners.  Adopt positive values and attitudes about teaching. They are convinced that PCELT developed their professionalism, and attribute the change in large part to PCELTs’ emphasis on using reflection and feedback about their teaching practices and understand the value of learner￾centered instruction.  Acquire a wide variety of useful and effective tools and approaches: They appreciate not only the richness of the PCELT toolkit but also for equipping them with a variety of strategies for selecting and applying the tools appropriately in different contexts with students.  See evidence of improved student learning as a result using PCELT methods and techniques for planning, instruction, and assessment. They credit PCELT for helping them to increase their students’ motivation to learn; improve their speaking fluency and listening comprehension; and engage them in collaborative activities resulting in larger participation and active learning.  Spread their PCELT experience and learning to others. They have been sharing their PCELT skills and materials with peers and other educators, including classmates in their pre-service program; cooperating teachers during their practice teaching in schools; and with relatives and friends. 81 ($2.OO per day),7 there are few job opportunities in an already oversaturated education sector. Despite this situation, some college grads with education degrees can often find some self-employment either tutoring students or, if lucky, as a part-time instructor with local or international organizations offering non-formal education or training for youth. Despite these challenges, the focus groups were conducted with the aim of learning about the graduate’s experiences and to shed additional light on the information that World Learning/SIT collects from participants at the end of each PCELT course. A. Description of the Sample and Research Questions LTD/Gaza canvased by phone all PCELT participants who graduated from three intensive courses offered between January and June 2014 and found that thirteen—8 females and 5 males—were using their PCELT skills in a variety of contexts. Of this group, only two had found full-time jobs—one at a private schools and the other at a youth center. For the rest, five were co-teaching afterschool English lessons as volunteers to students at the Al-Azhar University’s American Corner, and six were tutoring at home to groups of students from their neighborhoods. These thirteen individuals accepted LTD/Gaza’s invitation to participate in two focus groups conducted at the LTD/Gaza office in November 2014. Each focus group lasted 90 minutes. The core questions were derived from a 2013 Survey of PCELT Graduates developed by AMIDEAST. Following are the core questions: 1) Would you say that PCELT changed your perception of students and student learning? How? 2) How have your values or attitudes about teaching changed because of PCELT? 3) Tell us about PCELT methods or techniques you’ve implemented with learners. Give specific examples. (e.g., for teaching listening, speaking, reading, writing, vocabulary and grammar) 4) What are some differences you’ve seen in the learners by using PCELT techniques? (Evidence of learning.) 5) Have you been able to share your PCELT experience with your colleagues, friends or others? Tell us how? 6) Have you kept in contact with your PCELT colleagues? Do you feel you are part of a “PCELT community”? 7) What are your plans for the future? (For example: professional development, school projects, initiating workshops, mentoring, etc.? The core questions framed the analysis of the responses. The following section presents the results. B. Results 1. Perceptions of Students and Their Learning The participants believe that PCELT increased their sensitivity to the emotional, cognitive, and social needs of learners. The word “empathy” was used repeatedly to describe this change. A particular PCELT activity that brought home this idea was when the trainer asked them to role-play students in a simulation of a foreign language class. This activity shocked them into greater consciousness about the huge impact that 7 These figures are reported by the UNPD: http://www.undp.org/content/dam/undp/library/corporate/fast￾facts/english/FF-about-gaza-2009-EN.pdf 82 teaching practices can have on both the affective and cognitive experiences of a language learner. Overall, the consensus of the participants was that PCELT helped them recognize that:  Every student can and will learn when teachers act to affirm the students’ capacity to learn.  No two students learn the same way, so it’s important for the teacher to be sensitive to variations in the learning styles among students.  Students are not to be blamed for making mistakes; rather, the teacher’s responsibility is to help students become more aware of their learning, for example by providing multiple opportunities for students to reflect on and correct their own mistakes.  When the teacher creates a supportive, friendly, and caring classroom environment, students are more inclined to become actively engaged in the learning process.  Earning the love and respect of the students is its own reward. 2. Values and Attitudes about Teaching Similar to their views about student learning, the participants believe that PCELT increased their professionalism, though not at the outset. Many admitted that at first they imagined that a PCELT certificate would be easy to earn and would be a nice addition to their resume. To their surprise, they found that PCELT’s emphasis on reflection and feedback forced them to reevaluate their emergent philosophy of teaching. They came to recognize that their professional responsibility was more than teaching content, assessing students, and dispensing grades. Quite the opposite. They said that PCELT’s approach taught them that effective language learning happens when the teacher uses meaningful, authentic communication tasks to motivate and engage students, not through intimidation, and certainly in the use of corporal punishment that is not uncommon in “traditional” classrooms. They credited this change in attitude in large part to PCELT’s method of engaging them in structured feedback sessions immediately after their practice teaching. In short, they came to appreciate how much the craft of teaching benefits from self-reflection, collaboration, trust, and mutual respect among colleagues. Additional thoughts from participants about what PCELT taught them about effective teaching include the following:  Effective teaching is the capacity of the teacher to enable learning to happen.  Effective teaching means making easier what students find hard to learn; this requires flexibility and creativity in selecting appropriate teaching strategies and techniques.  Effective teachers empathize with their students and remember that they, too, were once students. As one participant explained, “PCELT made us students and teachers at the same time.” “Before PCELT I used to think teaching is just a job: come to school, teach, and go home; but PCELT made me see the students like a little family—getting them interested, engaged and to like me. Before PCELT I didn’t realize how important creating a friendly environment was.” “It’s the teachers job to teach and if students didn’t’ understand the first time, then maybe I’m doing something wrong. I need to change my way and help them understand.” 83  Effective teachers reflect on their problems of practice and value feedback from peers to think more critically about improving their teaching. 3. Contexts for Implementing PCELT Methods and Techniques after the Course As mentioned above, only several of the participants were employed full-time as English teachers, while the majority were using their PCELT skills in a variety of non-formal situations, for example by tutoring groups of students at home or teaching students at an afterschool programs on their college campus or at local youth centers. Regardless of their situation, the participants were able to practice a wide variety of PCELT skills; in fact, they said the PCELT toolkit had equipped them so well that “if one technique doesn’t work, another one will be better.” Moreover, they felt empowered by learning PCELT’s approach to lesson planning for teaching and assessment using PPU and ECRIF. The following summarizes the methods and techniques they were practicing:  Teaching Listening: questions or tasks to focus listening  Teaching Speaking: songs; role-playing; games  Teaching Reading: skimming; scanning; concept mapping; read-look up-speak  Teaching Writing: question prompts  Teaching Vocabulary: pictures/drawings; making real-world connections; acting; strategic use of translation; concept mapping  Teaching Grammar: modeling; demonstrating; examples to produce inductive learning Other techniques mentioned included: warm-ups; seating students strategically; using gestures/body language; pausing to let students answer; calling on student to help another; giving clear instructions; constructive feedback; showing appreciation; closing with a wrap-up activity; pair-work/grouping; games; projects (e.g., producing a video); use of questions to assess students’ prior knowledge; being patient with students who don’t know the correct answer immediately: allowing time to respond; paraphrasing the question; allowing the student to pick a classmate to help her. Implementing PCELT techniques was not without its challenges, however.  A number of the participants experienced some resistance to their use of PCELT skills during their practicum teaching in public schools; they blamed the dominant culture of the teacher-centered classroom, which sometimes makes students—and some cooperating teachers—reject innovative methods/techniques as a threat to covering textbook content they need for tests.  For those PCELTers who tutored students in their homes, a few found that students often equated “learning” with doing exercises and copying—practices that reinforce memorization and test taking. Students would sometimes interrupt a learning activity and tell the PCELTer that it was unnecessary because their English teacher at school didn’t teach it that way. 4. Evidence of Student Learning Using PCELT techniques Despite facing challenges and regardless of the context in which they were teaching, the participants agreed overwhelmingly that their PCELT skills increased students’ motivation and improved their learning. “We gave them vocabulary to help them express their thoughts and feelings about the war in Gaza. They wanted to make a video of their presentations, and they are presenting these today at the American corner.” 84 They felt well-equipped with tools and approaches to create a friendlier, learner-centered environment where, for example, students were less afraid of making mistakes and felt encouraged to use English to communicate feelings and ideas linked to the real-world. Some of the big improvements observed in their students’ learning and their attitudes about learning English included the following:  More motivated to learn: Students showed more willingness to ask questions if they didn’t understand something; somewhat uncharacteristically, some students started arriving before the start of a lesson to practice what they’d learned or ask for more information.  Improved fluency: During speaking activities, students would try using new vocabulary and were more comfortable making mistakes, particularly when learning tasks allowed them to use English to talk about their lives in Gaza.  More attentive: The variety of learning activities kept students interested and focused, resulting in improved listening comprehension.  More eager to collaborate: Students responded enthusiastically to cooperative learning activities; stronger students were seen supporting weaker classmates by modelling and asking questions. PCELTers also spoke about some of the challenges they think prevented them from having a bigger impact on student learning.  Some students expect teachers to simply “cover content” and prepare them for taking tests; that is, they perceive English as academic subject only, not a living language for communication. For example, listening comprehension is not emphasized in the public school curriculum and this can create a challenge for teachers to keep students interested and engaged.  Teaching lower elementary students demands methods and techniques not addressed by PCELT.  Students expect teachers to translate everything; PCELTers would respond by using translation strategically, but sparingly.  PCELT methods and techniques are perceived as “wrong”—they go against the pedagogical norm of teacher-dominated classroom instruction and rote memorization. 5. Challenges: Implementing PCELT methods and techniques can be tough A number of the participants experienced some resistance to their use of PCELT skills.  This was common experience for all during their practicum teaching in public schools. They blamed this on the dominant culture of the teacher-centered classroom, which sometimes makes students—and some cooperating teachers—resist efforts by PCELTers to use innovative methods/techniques instead of simply covering textbook content they need to pass tests.  It was similar for those PCELTers who tutored students in their homes. They explained that students equate “learning” with doing exercises and copying—practices that reinforce memorization and test taking. Students would sometimes interrupt a learning activity and tell the PCELTer that it was unnecessary because their teacher at school didn’t teach it that way. 85 6. Sharing PCELT Experience with Others The participants described instances of sharing their knowledge and skills with other educators, including cooperating teachers who mentored them during practicum teaching; with classmates in the Faculty of Education; and with relatives and friends who are teachers.  Cooperating teachers sometimes expressed interest the innovative methods they saw PCELTers applying during their school practicum teaching and would ask them to share ideas or materials.  PCELTers coached fellow classmates not enrolled in PCELT in how to use the PCELT methods for planning lessons for their own practice teaching.  Many said that they shared contents of their portfolios with relatives and close friends who also teach English. One of the few participants who landed a full-time job said that some of her colleagues—teachers with far more years of experience—come to her for ideas to make their own lessons more innovative. In an interesting twist, one participant helped her sister, who is an Arabic teacher, to try some language games and other creative activities from the PCELT toolkit and apparently the lessons were a success.  PCELTers used Facebook to share ideas and resources with friends and fellow PCELT graduates, as well as to network professionally with PCELTers in the West Bank and Gaza and Gaza and other countries like Egypt and Morocco. 7. Life after PCELT: An Emergent PCELT Community Although communication via email and social media—Facebook primarily—and meeting up with partners with whom they co-teach in afterschool programs (for the few who do) provide opportunities for some professional exchanges, the participants admitted they don’t feel they belong to a well-networked professional community.  They perceived a nascent PCELT community growing in Gaza and the West Bank, but described it as fragmented by cohort and region. Whereas graduates of the same cohort (12 individuals) tend to stay in touch, little or no communication existed across the different cohorts.  A Facebook page set up by AMIDEAST for PCELTers in the West Bank and Gaza and another set up by an SIT trainer for PCELTers in Morocco are visited by some of the Gaza groups. But communication with communities outside Gaza is splintered, as noted above, because each cohort has its own Facebook page and this tends to isolate members rather than connect them to others beyond their own cohort. 8. Plans for the Future The participants’ number one priority is to find a job. In the face of widespread unemployment, their fear is that long-term joblessness will threaten their capacity to apply, sustain, and improve on what they learned in PCELT. Four of the participants used their own initiative and won approval from their university to start an afterschool English program for local school students. They do this on a voluntary basis. With the local labor market unlikely to improve anytime soon, some of the participants suggested they needed to create their own opportunities, for example by:  Forming a PCELT club to facilitate collecting and sharing information about teaching, job “I have the least experience in my school, but teachers were asking me, ‘What does it mean to demonstrate and how to demonstrate during a lesson? How can we teach a listening lesson?’ I know how to lesson plan using PPU and I taught this to my friends. One friend is a teacher and has no education background, so I’m teaching her what I learned. She’s my colleague, but I’m her mentor.” 86 prospects, and opportunities for professional development.  Starting up a for-profit center for English language learning. C. Recommendations for Improving PCELT Throughout the focus group interview the participants reflected on their experiences learning and applying the PCELT’s teaching philosophy, approaches, and practices both during the training period and after its completion. The following summarizes recommendations they offered for improving some of the technical and professional aspects of PCELT. 1. Technical  Methods and techniques for teaching literature: Textbooks for teaching English in the West Bank and Gaza’s schools include poems and short stories, but PCELT’s approach to the teaching of reading focuses on comprehension of short paragraphs. PCELT should also provide appropriate methods for teaching literature to English language learners.  Innovative ways to teach writing: The participants believe that for a variety of reasons, English teachers discount the importance of developing and evaluating students’ writing. Faced with large classes, a lack of training in teaching writing, and their own weak writing skills, many Palestinian teachers of English do little to motivate students to take writing lessons seriously. Because of this, PCELTers said they would like the course to offer more innovative ways to get students excited about writing. 2. Professional  How to give written feedback: PCELT trainers did an excellent job of helping the participants develop their capacity to give and receive oral feedback following peer observations of practice teaching. However, they would have appreciated more opportunities to develop the skill of giving written feedback.  All-PCELT website: A single Facebook page serving as a common landing spot for PCELTers everywhere would improve professional networking across the geographically dispersed communities of PCELT alumni.  Professional Association: The formation of a professional association for PCELT graduates in Gaza would improve communication among alumni of different cohorts and foster the exchange of information, ideas, and resources for teaching, as well as news and opportunities about employment and continued professional development.  Training Follow-up: Provide workshops or refresher courses to help alumni expand and deepen what was learned in the course. 87 ANNEX A: Principal Effectiveness Survey (Principal Form) ب هللا الرحمن الرحيم الكفايات القيادية عند مديري المدارس الفلسطينية في الضفة الغربية استبانة المدير عزيزي المشارك يسعى المعهد الوطني للتدريب التربوي إلى تطوير وبناء مقدرات موظفي وزارة التربية والتعليم بمختلف مستوياتهم ومسمياتهم الوظيفية من خالل برامجه المختلفة، وبرنامج القيادة المدرسية واحد من أهم البرامج التي ينفذها المعهد لتمكين مديري المدارس من تحسين مدارسهم بناء على المعايير الفلسطينية للمدرسة الفاعلة. وبعد أن شارف البرنامج على االنتهاء يعمل قسم الدراسات في المعهد الوطني على معرفة على ً فاعلية البرنامج في رفع كفايات المديرين وأدائهم في تطوير مدارسهم بناء معايير التحسين المدرسي. فإنه؛ يؤمل منك التعاون في تحديد مستوى معرفتك وتقدير مستوى كفاياتك في المحاور والمجاالت الوارده في الصفحات اآلتية قبل التحاقك بالبرنامج وبعده، وذلك باالستعانة بمقياس األداء المتدرج. معلومات عامة -القسم االول: الجنس ذكر أنثى سنوات الخبرة في التعليم أقل من 5 سنوات من 5 إلى أقل من عشرة من 10 إلى أقل من 15 أكثر من 15 سنة سنوات الخبرة في اإلدارة أقل من 5 سنوات من 5 إلى أقل من عشرة من 10 إلى أقل من 15 أكثر من 15 سنة المؤهل العلمي دبلوم بكالوريوس ماجستير دكتوراة المؤهل التربوي دبلوم تربية بكالوريوس تربية ماجستير تربية دكتوراة تربية المحافظة دولة فلسطين وزارة التربيـة والتعلـيم المعهد الوطني للتدريب التربوي State Of Palestine Ministry of Education National Institute for Educational Training 88 القسم الثاني: المعرفة قدر درجة معرفتك وفهمك في الموضوعات اآلتية )ضمن ياق المدرسي( قبل التحاقك بالبرنامج وبعده بت ّ الس قدير يترواح من 5-1 حيث-: .5 لدي معرفة وفهم كبير وعميق بالموضوع. .4 لدي معرفة وفهم بالموضوع . .3 أعرف عن الموضوع بعض الشيء. .2 معرفتي قليلة عن الموضوع. .1 ليس لدي أية معرفة بالموضوع. درجة المعرفة بالموضوع قبل البرنامج الموضوع درجة المعرفة بالموضوع بعد البرنامج 5 4 3 2 1 5 4 3 2 1 الة. ّ المعايير الفلسطينية للمدرسة الفع الرؤية والرسالة. التقويم الذاتي. بناء الفرق المدرسية. التخطيط المدرسي. العالقات ودورها في تحسين العملية التعليمية التعلمية. صناعة قرارات مبنية على البيانات. إدارة الموارد البشرية. إدارة الموارد المادية. البيئة المدرسية الداعمة للتعلم. ال. ّ مباديء التعليم والتعلم الفع متابعة تحسين التعليم والتعلم وتقويمه. دور المجتمع المحلي في التحسين المدرسي. دور التكنولوجيا في العملية التعليمية. األبحاث اإلجرائية. 89 الثالثالقسم : الكفايات القيادية عند مدير المدرسة فيما يأتي مجموعة من مؤشرات تقيس كفايات المديرين القيادية يرجى قراءتها ومن ثم ضع التقدير من -1 4 الذي ترى نفسك فيه بناء على مقياس األداء المتدرج، حيث: 1 ال يحقق مستوى ألداء المطلوب، 2يقترب من تحقيق المستوى المطلوب، :3 يحقق مستوى ألداء المجال المطلوب، :4 يتجاوز مستوى ألداء المطلو ّ ب ويتميز مستوى الممارسة قبل البرنامج التدريبي الممارسات مستوى الممارسة اآلن 4 3 2 1 4 3 2 1 التخطيط .1 أشرك المعلمين في بناء رؤية المدرسة ورسالتها. .2 أشرك المعلمين في بناء خطة المدرسة. .3 أشرك أولياء األمور في بناء خطة المدرسة. .4 أوضح رؤية المدرسة ورسالتها للمجتمع المحلي. .5 أبني الخطة المدرسية بناء على نتائج عملية تقويم ذاتي لواقعها. .6 أجري بأبحاث إجرائية لتطور عملي في المدرسة. العالقات .1 أشجع العمل الجماعي في المدرسة و أنمذجه. .2 أشرك المعنيين في صناعة قررات تتعلق بتحسين المدرسة مستندة الى .3 أتواصل بفاعلية مع الطاقم في المدرسة. بيانات نتائج التقويم الذاتي .4 أشرك المجتمع المحلي في نشاطات تدعم عمليتي التعليم والتعلم .5 أعزز أخالقيات المهنة و أنمذجها )أنمذج السلوكيات التي يتوقعها من .6 أظهر االحترام والتقدير ألفراد مجتمع المدرسة على اختالفهم. اآلخرين(. .7 يعامل طاقم المدرسة بعدالة ) دون تحيز ( .8 أدعو أولياء األمور لزيارة المدرسة ومناقشة أوضاع أبنائهم التعليمية .9 أستثمر المؤسسات المحلية في دعم عمليتي التعليم والتعلم. وتحسينها. 10 أحل الصراعات بين العاملين بمهنية. المجال مستوى الممارسة قبل البرنامج التدريبي الممارسات مستوى الممارسة اآلن 4 3 2 1 4 3 2 1 الموارد 1 أستثمر خبرات طاقم المدرسة في دعم أهدافها. 2 أحدد احتياجات الطاقم المدرسي التي تدعم العملية التعليمية التعلمية. 3 أسهل التحاق المعلمين ببرامج تطور مهني لسد احتياجاتهم وتحسين 4 أصرف م ممارساتهم التعليمية. يزانية المدرسة حسب احتياجاتها و أولوياتها. 5 أؤكد على استخدام مصادر التعلم في المدرسة ومراكزها التطويرية )المكتبة، غرف الرياضة......،( وصيانتها بإستمرار. 90 6 أحافظ على دقة المعلومات المالية للمدرسة و أحرص على توثيقها. 7 أتواصل مع المجتمع المحلي لزيادة موارد المدرسة وتفعيلها. التعليم والتعلم .1 أشرف على الممارسات التعليمية التعلمية للمعلمين في صفوفهم. .2 أثري معرفته بنظريات التعلم والتطور بهدف تحسين عمليتي التعليم والتعلم في مدرستي. .3 أتعاون مع المجتمع المدرسي لدعم تعلم الطلبة .4 أؤكد على تعزيز )بناء( شخصية الطالب بشكل متكامل .5 أدعم التطور المهني للمعلمين .6 أتابع تطور ممارسات المعلمين التعليمية نتيجة التحاقهم ببرامج التطور المهني. .7 أؤكد على النشاطات الالصفية التي تدعم تعلم الطلبة في خطة المدرسة وتكاملها مع األنشطة الصفية. .8 أقدم التسهيالت والتجهيزات حتى يتمكن المعلمو ن من تنفيذ االستراتيجيات التعليمية .9 أتبنى سياسة تعزيز النجاح والتعلم لجميع الطلبة ) بمن فيهم ذوي الصعوبات والتفوق االكاديمي( 10 أؤكد على تشارك )تعاون( معلمي المادة للعمل التعاوني نحو تحقيق األهداف التعلمية المقصوده. مستوى الممارسة قبل البرنامج التدريبي الممارسات مستوى الممارسة اآلن 4 3 2 1 4 3 2 1 البيئة المدرسية .1 أفوض العاملين في المدرسة بصالحيات تتناسب و مقدراتهم. .2 أشرك المعلمين في صناعة قرارات ذات عالقة بالمدرسة ومجتمعها .3 أشجع الطلبة على العمل التطوعي والتعاوني. .4 أبني بيئة تعلمية تعزز االحترام وتقدير الذات. .5 أطور سياسات توفر بيئة مدرسية آمنه تعزز تعلم الطالب وراحته. .6 أضع قواعد واضحه للحفاظ على نظافة المدرسة وممتلكاتها. .7 أعزز دور المرشد التربوي في المدرسة .8 أبحث عن األمور المحفزة للمعلمين على العمل و أعززها. .9 أقدر جهود العاملين في المدرسة. التقويم .1 أعلم أولياء األمور بنتائج تقويم ابنائهم بهدف التحسين. .2 أؤكد على استخدام اساليب متنوعة في تقويم أداء الطلبة. .3 أتبع اجراءات مختلفة في تقويم الطلبة لتحسين تعلمهم. .4 أقدم للمعلمين تغذية راجعه عن ممارساتهم التعليمية بهدف التحسين انجازاتهم )بهدف التطوير والتحسين(. .5 أوثق نت المستمر.ائج تقويم المعلمين و 91 .6 أشرك المدير المعلمين في صناعة قرارات تستند الي نتائج تقويم الطلبة. التكنولوجيا .1 أعزز استخدام المعلمين للتكنولوجيا في انشطتهم التعليمية. .2 أشجع المعلمين على تطوير مقدراتهم في استخدام التكنولوجيا في 3 أستخدم التعليم )بالذات الحاسوب( التكنولوجيا في تيسير العمليات اإلداريه. 4 أتابع صيانة األجهزه التكنولوجيه في المدرسه وتحديثها. 5 أستخدم التكنولوجيا في التواصل مع المعلمين والطاقم المدرسي و ذوي 6 أ العالقة ستخدم التكنولوجيا في البحث عن مصادر التعليم والتعلم 7 أستخدم التكنولوجيا في التنمية المهنية 92 Annex B: Teacher Effectiveness Survey (Teacher Form) ب هللا الرحمن الرحيم الكفايات التعليمية لدى معلمي المدارس الحكومية الفلسطينية استبانة المعلم عزيزي/تي المشارك/ة يسعى المعهد الوطني للتدريب التربوي إلى تطوير وبناء قدرات معلمي المدارس الفلسطينية في كافة المراحل من خالل برامجه المختلفة، ويأتي برنامج تأهيل المعلمين للصفوف من )10-5( في المرحلة الثانية بعد البدء بمشروع تأهيل المعلمين للمرحلة األساسية من )4-1( بالتعاون مع الجامعات الفلسطينيية. واآلن وبعد أن شارف البرنامج على االنتهاء، يعمل قسم الدراسات في المعهد الوطني على معرفة التغييرات التي أحدثها البرنامج التدريبي على كفايات المعلمين التعليمية، لذا نأمل منك التعاون في تحديد رأيك في مستوى كفاياتك التعليمية اآلن وقبل البدء في البرنامج في المحاور والمجاالت الوارده في الصفحات اآلتية، باالستناد إلى مقياس األداء المتدرج المرفق. عزيزي المعلم : تذكر، هذه االستبانة لغرض البحث فقط و ليس لتقييم المدرسة او آداء اعضاء الهيئة التدريسية حيث سيتم االحتفاظ بردودكم وبكل ما تقدموه من معلومات بسرية تامة وألغراض المشروع فقط معلومات يملؤها الباحث: رقم االستبانة اسم الباحث المديرية التي تقع فيها المدرسة رقم المدرسة الوطني دولة فلسطين وزارة التربيـة والتعلـيم المعهد الوطني للتدريب التربوي State Of Palestine Ministry of Education National Institute for Educational Training 93 القسم األول: الكفايات التعليمية للمعلم فيما يأتي مجموعة من المؤشرات التي تقيس كفايات المعلمي ن التعليمية يرجى قراءتها ومن ثم ضع تقدير ك لمستوى أدائك بناء على مقياس األ حيث أن: داء المتدرج :1 ال يحقق مستوى الكفاية المطلوب، :2 يقترب من تحقيق مستوى الكفاية المطلوب، :3 يحقق مستوى الكفاية المطلوب، :4 يتجاوز تحقيق الكفاية ويتميز. مستوى الكفاية قبل البرنامج التدريبي الكفاية االولى: تسهيل التعليم والتعلم المتمركز حول الطالب مستوى الكفاية بعد البرنامج التدريبي ) 1( ) 2 ( ) 3 ( ) 4 ( الكفاية الفرعية ) 1( ) 2 ( ) 3 ( ) 4 ( 1.1 أبني خططا يومية وفصلية تنسجم و أنماط تعلم الطلبة المختلفة. 1.2 أراعي الفروق الفردية بين الطلبة. 1.3 أراعي الخبرات السابقة الطلبة. 1.4 أبني مخرجات تعلم محددة تنسجم و مخرجات التعلم العامة لمنهاج المرحلة األساسية. 1.5 أشرك الطلبة وذوي العالقة في مناقشة مخرجات التعلم المرجوة. 1.6 أنفذ أنشطة مصاحبة للمنهاج تعزز التعلم التشاركي بين الطلبة. 1.7 أطبق أنشطة صفية مرتبطة بمحتوى الدرس، و أطرح اسئلة من مستوى مهارات التفكير المتنوعة للتأكد من تعميق الفهم. مستوى الكفاية قبل البرنامج التدريبي الكفاية الثانية: تصميم المصادر والمواد التعليمية والتعلمية مستوى الكفاية بعد البرنامج التدريبي ) 1( ) 2 ( ) 3 ( ) 4 ( الكفاية الفرعية ) 1( ) 2 ( ) 3 ( ) 4 ( 2.1 أضع خطة سنوية لتطوير مواد التعليم والتعلم ومصادرهما ) من، وكيف، ومتى، وأين، ولماذا؟(. ُم الطلبة. 2.2 أوظف تكنولوجيا المعلومات واالتصاالت في تعليم وتعل 2.3 أستعمل المصادر المجتمعية )كالرحالت والزيارات والمصادر البشرية( في تحسين عملية التعلم. 2.4 أوظف مصادر التعليم والتعلم التي تتناسب مع احتياجات الطلبة. ّ 2.5 أ ة اإلبداعية باستخدام مصادر التعليم والتعلم المختلفة. ّن من القدرات والمهارات التعلمي حس ّم متنوعة. 2.6 أشرك الطلبة في تطوير مصادر تعل 2.7 أوظف مصادر تعليم وتعلم متنوعة لتحقيق عناصر المنهاج. 94 ّ مستوى الكفاية قبل البرنامج التدريبي الكفاية الثالثة: المشاركة في توفير بيئة ه تعلمي فاعلة وآمنة مستوى الكفاية بعد البرنامج التدريبي ) 1( ) 2 ( ) 3 ( ) 4 ( الكفاية الفرعية ) 1( ) 2 ( ) 3 ( ) 4 ( ّ 3.1 أ ة المختلفة. عزز مشاركة الطلبة في األنشطة الصفي ُم من خالل المحاولة والتجريب. ّم تشجع الطالب على التعل 3.2 أستخدم بيئة تعل ّ 3.3 أ ة والمدرسية. شرك الطلبة في عمليات صياغة القواعد الصفي 3.4 أوفر فرص تعلم متساوية لجميع الطلبة. 3.5 أوفر بيئة تعلم صحية وآمنة تشجع الطلبة في عملية تعلمهم. 3.6 أوفر بيئة تعلمية جاذبة للطلبة تحفزهم على التفكير االبداعي والناقد. ًمهم. ّهم لمسؤولية جودة تعل 3.7 أكلف الطلبة بمهام تعزز ثقتهم بأنفسهم وتحمل مستوى الكفاية قبل البرنامج التدريبي الكفاية الرابعة: المتابعة والتقويم لعملية التعليم والتعلم مستوى الكفاية بعد البرنامج التدريبي ) 1( ) 2 ( ) 3 ( ) 4 ( الكفاية الفرعية ) 1( ) 2 ( ) 3 ( ) 4 ( ً 4.1 أصمم خططاً فردية لتحسين التعليم على عملية التقويم. بناء 4.2 أستخدم نتائج التأمل الذاتي لتحسين عمليات التعليم والتعلم. 4.3 أنفذ برامج تعلم عالجية تتالءم مع احتياجات الطلبة الخاصة باالستناد إلى نتائج التقويم. 4.4 أزود أولياء األمور بالتقارير حول النتائج الدراسية الخاصة بأبنائهم. ُم. 4.5 أستخدم نتائج المتابعة والتقويم لتحسين التعليم والتعل 4.6 أبني أدوات تقويم مختلفة تناسب الفروق الفردية عند الطلبة. 4.7 أقيم ذاتي من أجل التطور المهني. ُ 4.8 أختار ة. إستراتيجيات التقويم التي تتالءم مع حاجات الطلبة التعلمي ُ وثق نتائج التقويم الستخدامها في متابعة تقدم الطلبة. 4.9 أ 4.10 أقدم التغذية الراجعة المناسبة للطلبة بناء على نتائج التقويم. 4.11 أستخدم المتابعة والتقويم كإستراتيجية تعليم وتعلم. 95 4.12 أشجع الطلبة على إستخدام التقويم الذاتي. مستوى الكفاية قبل البرنامج التدريبي الكفاية الخامسة: اإلرشاد والتوجيه للمتعلمين مستوى الكفاية بعد البرنامج التدريبي ) 1( ) 2 ( ) 3 ( ) 4 ( الكفاية الفرعية ) 1( ) 2 ( ) 3 ( ) 4 ( ّ 5.1 أ ه الطلبة نحو السلوكيات اليومية ) مثل الصحة والنظافة الشخصية والسالمة العامة واالنضباط الذاتي...(. وج ّبع اإلجراءات المناسبة لتحسين سلوك الطلبة اليومي. 5.2 أت 5.3 أرفع من شأن القيم واالتجاهات االيجابية لدى الطلبة. 5.4 أكلف الطلبة بمهام أداء وواجبات ترتبط بواقع حياتهم. 5.5 أستخدم اإلرشاد والتوجيه المناسبين الختيار مسارهم المهني المالئم المكانياتهم. 5.6 أتواصل مع المختصين إليجاد أفضل الحلول لمعالجة صعوبات التعلم. مستوى الكفاية قبل البرنامج التدريبي الكفاية السادسة: السعي للتطور المهني مستوى الكفاية بعد البرنامج التدريبي ) 1( ) 2 ( ) 3 ( ) 4 ( الكفاية الفرعية ) 1( ) 2 ( ) 3 ( ) 4 ( 6.1 أستخدم نتائج التقييم في تحديد احتياجاته التدريبية. 6.2 أطبق الخبرات التدريبية التي أكتسبها في تفعيل األنشطة الصفية. 6.3 أتبادل الخبرات مع الزمالء الستخدام األساليب المشتركة في التعليم والمشاريع. 6.4 أجمع المصادر ذات الصلة بعمله بواسطة وسائل تكنولوجيا المعلومات واالتصاالت. 6.5 أستفيد من المنهجيات المالئمة للوصول الى مخرجات تعلم الطلبة. 6.6 أستخدم البحث اإلجرائي في تحسين عملية التعليم والتعلم. 6.7 أحتفظ بملف يحتوي الفعاليات واألنشطة المختلفة لمتابعة تطور ي المهني. 6.8 أشارك في الدورات واأليام الدراسية لتطوير أدائي. مستوى الكفاية قبل البرنامج التدريبي الكفاية السابعة: تفعيل الشراكة داخل المجتمع مستوى الكفاية بعد البرنامج التدريبي 7.1 أشجع الطلبة على االنخراط بالتجارب المجتمعية المحلية التي تدعم تعلمهم. 7.2 أشارك في وضع الحلول المناسبة للمشكالت المجتمعية. 7.3 أزود أولياء األمور بتقارير حول نتائج تحصيل أبنائهم. ُمية والصحية(. 7.4 أتعاون مع أولياء األمور لحل مشكالت الطلبة المختلفة )السلوكية والتعل 2 7.5 أشارك الزمالء بالخبرات التي تدعم عمليات التعلم والمشاريع المشتركة 7.6 أوظف المصادر المجتمعية في تحسين عملية التعليم والتعلم 1 القسم الثاني-: معلومات عامة .1 الجنس .1 ذكر .2 أنثى .2 العمر .1 أقل من 30 سنة .2 من 30 – 39 سنة .3 من 40 - 49 سنة .4 أكثر من 50 سنة .3 الحالة االجتماعية .1 أعزب/ عزباء .2 متزوج /ة .4 سنوات الخبرة في التعليم .1 أقل من 5 سنوات .2 من 5 إلى أقل من عشرة .3 من 10 إلى أقل من 15 .4 أكثر من 15 سنة .5 التخصص )المادة التي تدرسها هذا العام ( __________________________ .6 ما هو معدل عدد الساعات التي تمضيها/ تمضينها بالنشاطات التعليمة خالل أسبوع كامل )7 أيام( .1 أقل من 30 ساعة .2 من 30 – 34 ساعة .3 من 35 – 39 ساعة .4 من 40 – 44 ساعة .5 أكثر من 45 ساعة .7 المؤهل العلمي: .1 دبلوم .2 بكالوريس .3 ماجستير .4 دكتوراة .5أخرى ____________ .8 هل تعمل/ين حالياً للحصول على درجة علمية جديدة؟ .1 نعم .2 ال ً .3 جيد .4 متوسط .5 .9 ما درجة اتقانك الستخدام الحاسوب؟ .1 ممتاز .2 جيد جدا ضعيف أو معدوم .10 هل يوجد انترنت في البيت؟ .1 نعم .2 ال .11 أستخدم التكنولوجيا في البحث عن مصادر التعليم والتعلم 1( أوافق بشدة 2( أوافق 3( الى حد ما 4( ال أوافق 5( ال أوافق بشدة .12 أستخدم التكنولوجيا في التنمية المهنية 1( أوافق بشدة 2( أوافق 3( الى حد ما 4( ال أوافق 5( ال أوافق بشدة 2 Annex C: Classroom Engagement Survey (Student Form) ب هللا الرحمن الرحيم المشاركة الصفية استبانة الطالب عزيزي الطالب : تذكر، هذه االستبانة لغرض البحث فقط و ليس لتقييم المدرسة او آداء اعضاء الهيئة التدريسية حيث سيتم االحتفاظ بردودكم وبكل ما تقدموه من معلومات بسرية تامة وألغراض البحث فقط معلومات يملؤها الباحث: رقم االستبانة اسم الباحث المديرية التي تقع فيها المدرسة رقم المدرسة الوطني E في أي صف تم تعبئة االستبانة؟ .1الصف الخامس .2 الصف السادس .3الصف السابع .4 الصف الثامن .5 الصف التاسع .6 الصف العاشر في أي حصة تم تعبئة االستبانة ؟ .1الرياضيات .2 العلوم .3 اللغة العربية .4 اللغة االنجليزية .5 التكنولوجيا A الجزء األول: انطباعك عن مدرستك: الرجاء التفكير في مدرستك فقط أثناء اإلجابة عن األسئلة التالية: ما درجة اتفاقك مع العبارات التالية: ضع دائرة حول اإلجابة التي تراها مناسبة لكل جملة. .1 أشعر بأن مدرستي تقوم بإعدادي الكون ناجحا في تعلمي المستقبلي دولة فلسطين وزارة التربيـة والتعلـيم المعهد الوطني للتدريب التربوي State Of Palestine Ministry of Education National Institute for Educational Training 3 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .2 أنا سعيد لكوني طالب في هذه المدرسة 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .3 اكون متحمسا عند القدوم الى المدرسة 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة B الجزء الثاني: تجربتك داخل الصف/الحصة الرجاء التفكير في حصص مادة )مبحث هذه الحصة ( أثناء اإلجابة عن األسئلة التالية: ما مدى اتفاقك مع العبارات التالية: ضع دائرة حول اإلجابة التي تراها مناسبة لكل جملة. .1 يشجعني معلمي على التفكير اليجاد االجابات و الحلول 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .2 أشارك في مجموعات تعاونية صغيرة داخل الصف 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .3 أعبر عن رأيي بحرية داخل الحصة 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .4 يستخدم معلمي اسلوب يجعل عملية التعليم ممتعة 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .5 أشارك في أنشطة صفية متنوعة 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .6 يساعدني معلمي عندما اجد صعوبة في فهم الدرس 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .7 يهتم معلمي كثيرا بافكاري المطروحة 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .8 يشجعني معلمي على طرح األسئلة في الصف 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة 4 .9 يقضي معلمي معظم وقت الحصة في قراءة المعلومات وكتابتها 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .10 أقضي معظم وقت الحصة في نسخ المعلومات 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .11 في معظم الحصص، معلمي يطلب مني حفظ الحقائق واالرقام دون التحقق من درجة معرفتي و فهمي لها 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .12 يعطيني معلمي وقتاً لنقاش ما تعلمته في الصف 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .13 يتحدث معلمي اكثر من الطلبة في الحصة دون السماح للطالب بالنقاش 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .14 أسلوب معلمي يساعدني على الفهم بسهولة 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .15 أشارك في المشاريع 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .16 يعطيني معلمي مالحظات حول ادائي في اإلختبارات والواجبات 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .17 يعطيني معلمي مالحظات حول ادائي أثناء الدرس 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .18 أشارك في انشطة مثيرة لإلهتمام 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة .19 يستخدم معلمي ادوات ووسائل تكنولوجية في الحصة 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة 20 . يربط معلمي بين ما ما نتعلمه وبين الحياة اليومية 5 1( ال أوافق بشدة 2( ال أوافق 3( الى حد ما 4( أوافق 5( أوافق بشدة C الجزء الثالث: استخدام الحاسوب ضع دائرة حول اإلجابة التي تختارها لكل سؤال: .1 هل لديك جهاز حاسوب في البيت؟ 1( نعم 2( ال .2 هل لديك انترنت في البيت؟ 1( نعم 2( ال .3 أستخدم جهاز الحاسوب النجاز الواجبات البيتية 1( نادرا 2( احيانا 3( غالبا .4 أستخدم جهاز الحاسوب لزيادة المعرفة التي اكتسبتها داخل الصف 1( نادرا 2( احيانا 3( غالبا D الجزء الرابع: السلوك )خالل العام الدراسي الحالي( الرجاء التفكير في مدرستك .االسئلة التالية تطلب منك ذكر عدد المرات التي قمت بها باالمور التالية داخل المدرسة . تذكر ، ال أحد سيعلم انك من قمت باالجابة عن االسئلة : .1 ضربت احد الطالب متعمدا داخل المدرسة: a )وال مرة b )5-1 مرات c )10-6 مرات d )اكثر من 10 مرات .2 تعرضت للضرب المتعمد من قبل احد الطالب: a )وال مرة b )5-1 مرات c )10-6 مرات d )اكثر من 10 مرات 6 .3 تعرضت للضرب من قبل المعلم : a )وال مرة b )5-1 مرات c )10-6 مرات d )اكثر من 10 مرات .4 غادرت المدرسة بدون اذن : a )وال مرة b )5-1 مرات c )10-6 مرات d )اكثر من 10 مرات .5 احضرت ولي أمري الى المدرسة بسبب قيامي بأمر خاطئ : a )وال مرة b )5-1 مرات c )10-6 مرات d )اكثر من 10 مرات E الجزء الخامس: معلومات إضافية عني وعن عائلتي .1 ضع دائرة حول الجواب الذي ينطبق عليك؟ .1 أنثى أو .2 ذكر .2 كم سنة قضيت في هذه المدرسة بما فيها السنة الحالية؟ )أكتب عدد السنوات في المستطيل( _________ ضع دائرة حول االجابة التي تناسبك. .3 ما هي أعلى درجة علمية حصلت عليها والدتك/ولية أمرك؟ .1 أقل من ثانوي .2 الدراسة الثانوية .2 جامعي )بكالوريوس أو أكثر ( 7 .4 ما هي أعلى درجة علمية حصل عليها والدك/ولي امرك؟ .1 أقل من ثانوي .2 الدراسة الثانوية .2 جامعي )بكالوريوس أو أكثر ( نشكر لكم تعاونكم