The Performance Evaluation of USAID/Ethiopia-Funded International Foundation for Education and Self￾Help Capacity Building in Teacher Education Program This evalu ation repor t was prod uced for r eview by USAID as part of an evaluation conducted for Performance Evaluation of the USAID/Ethiopia funded Internationa l Foundation for Education and Self-Help (IFES H) Capacity Building in Teacher Education Program (CBTEP) under the Contract Agreement Order No. AID-663-O-11-00009 and REQ￾663-11-000022. It was prepared by PRIN Intern ational Consultancy & Research Services PLC. The views expressed in this report do not necessarily reflect the views of USAID. February 2012 Addis Ababa, Ethiopia 1 Contents Acknowledgements ...................................................................................................................... 4 List of Acronyms ......................................................................................................................... 5 Executive Summary .................................................................................................................... 6 1. Introduction ………………………………………………………………………………. 7 1.1. Background ................................................................................................................... 10 1.2. Objectives of the Evaluation ........................................................................................ 12 1.3. Scope of the Assessment .............................................................................................. 13 2. Technical Approach/Methodology ...................................................................................... 13 3. Limitations of the Evaluation .............................................................................................. 15 4. Major Findings ..................................................................................................................... 15 4.1. The Project: Initiation, Relevance, Partnership, and Support ..................................... 15 4.2. Assessment of Interventions: Practices, Achievements and Innovations ..................... 20 4.2.1. English language improvements ........................................................................... 20 4.2.2. Teaching and systems improvements .................................................................... 21 4.2.3. Educational resources solicitation and utilization ................................................. 25 4.2.4. Community services and cross-cutting issues ....................................................... 25 4.3. Summary of Emerging Impacts of the Program ........................................................... 27 4.4. Strengths, Weaknesses, Opportunities and Challenges ................................................ 28 4.4.1. Strengths ................................................................................................................ 28 4.4.2. Weaknesses ........................................................................................................... 29 4.4.3. Opportunities ......................................................................................................... 29 4.4.4. Challenges ............................................................................................................. 30 4.5. Best Practices and Lessons Learnt ............................................................................... 31 4.6. Sustainability ................................................................................................................ 31 5. Conclusion ............................................................................................................................. 32 6. Recommendations................................................................................................................. 32 Annex I: People Contacted/Interviewed .................................................................................... 35 Annex II: Key Informant Interview Guide ................................................................................ 38 Annex III: List of Major Documents Reviewed ....................................................................... 42 Annex IV: List of Volunteers List of IFESH Volunteers under CBTEP 2003/2004 - 2011/2012 A/Y ......................................................................................................................................... 43 Annex V: IFESH/ETHIOPIA Volunteers Present at IFESH Mid-Year Conference for Focus group discussion at Bishale Hotel, Addis Ababa, December 2011 ........................................ 46 Annex VI: IFESH-Ethiopia CBTEP Fund/Grant from July 2002 through July 2012 ................ 47 Annex VII: Table 6 CBTEP beneficiaries per year ................................................................... 32 Annex VIII: Statement of Work for Evaluation of the International Foundation for Education and Self-Help (IFESH) Capacity Building in Teacher Education Project (CBTEP) in Ethiopia .................................................................................................................................. 33 Annex IX: Addressing Cross-Cutting Issues at Hosanna TEC ................................................... 40 2 List of Tables Table 1: Education Indicators …............................................................................................ 10 Table 2: Type and number of trainees at BDU for the training delivered by AV, 2011/12… 21 Table 3: In-service training by local volunteers with the support of AV at BDU, 2010/11….24 Table 4: Training and workshop organized by volunteers at HCTE, 2010/11 ……………… 24 Table 5: Resources mobilized by volunteers ………………………………………………… 25 3 List of Figures Figure 1: No. of students attending English Language Tutorial Sessions in BDU, 2011/12 … 20 Figure 2: CBTEP beneficiaries per training areas at BDU, 2010/11 ………………………… 24 4 Acknowledgements The Consultant firm acknowledges that this evaluation report is a product of the concerted efforts of various organizations and individuals that were directly and indirectly involved at various levels whose names could not all be listed here. Thus, the Consultant would like to extend its heartfelt appreciation to those who contributed their part for the accomplishment of this report. Particularly, the Consultant is very grateful to the State Minister of General Education, the Director for Teachers and School Leaders Development Directorate, Director for Planning and Resource Mobilization Directorate, and concerned experts from MoE; Presidents/Vice-Presidents/Deans/Department Heads/Coordinators, instructors, and students from HLIs; principals and teachers from primary and secondary schools; Country representative and program director from IFESH/Ethiopia; and the current IFESH volunteers as well as ex-volunteers for lending their scarce time and for their willingness to share their deep knowledge regarding IFESH voluntarism in Ethiopia. The Consultant is also highly indebted to USAID/Ethiopia and its high calibre experts for building trust on the capacity of the firm in carrying out the job and giving it clear and concise direction to undertake the assignment. Finally and most importantly, the Consultant firm is very thankful to the members of the consulting team for their high sense of responsibility, dedication, cooperation and uncompromising stand for quality delivery of the outputs right from the beginning to the end of the project. 5 List of Acronyms AVs American Volunteers AVT American Volunteer Teachers BDU Bahir Dar University CBTEP Capacity Building in Teacher Education Program CPD Continuous Professional Development CPDP Continuous Professional Development Plan KTC Kotebe Teachers College CTEs College of Teacher Education DBCTE Debre Birhan College of Teachers Education DBU Debre Birhan University ELIC English Language Improvement Centre ELIP English Language Improvement Program ELTIP English Language Training and Improvement Program FGD Focus Group Discussion GEQIP General Education Quality Improvement Program GER Gross Enrolment Rate GOE Government of Ethiopia GPI Gender Parity Index GU Gondar University HCTE Hossana College of Teachers Education HDP Higher Diploma Program HIs Higher Institutes HLI Higher Learning Institute HTC Hossana Teachers College HU Haramaya University IFESH International Foundation for Education and Self-Help LV Local Volunteers MOE Ministry of Education NER Net Enrolment Rate NGO Non-Governmental Organization. PGHET Post graduate Higher Education Training SBCPD School Based Continuous Professional Development SIP School Improvement Program STDs Sexually Transmitted Diseases TDP Teachers Development Program TEIs Teacher Education Institutions TTC Teachers Training College TTI Teachers Training Institute USAID United States Agency for International Development VSO Volunteer Services Overseas 6 Executive Summary A century has gone since modern Western education was introduced to Ethiopia and decades have elapsed since some kind of teacher education started to produce local teachers for the schools/colleges. Ever since then there has been concerns with regard to the adequacy and quality of the teacher education sub-system. In all times, there have been efforts to support local initiatives with inputs from experts (volunteers or advisors) deployed by partner countries and/or organizations. One of the biggest partners that have been supporting MoE’s education strategy by deploying experienced American volunteers (AVs) is USAID/IFESH. It has been operating since 1996 by deploying 216 AVs who worked in various Universities, CTEs, or community schools. In order to see whether the program was successful or not in the last 10 years, USAID/Ethiopia commissioned PRIN International Consultancy & Research Services PLC to evaluate the Performance of the program with the ultimate purpose of determining the future direction in the strategies of USAID/Ethiopia’s Basic Education Office and the MoE for enhancing the quality of teacher education programs. PRIN constituting a team of four (4) experienced experts has started the work after the conclusion of formal contract agreement with USAID. As it has been clearly stipulated in the statement of work (SOW), the performance evaluation was conducted to achieve the following specific objectives:  To examine and verify the results and effectiveness (output and outcome) of the program since its inception guided by the project agreement, goal, objectives, and targets.  To examine whether the program activities, outputs, and outcomes have been integrated as appropriate into the initiatives of the host government and teacher training institutions.  To determine the level of satisfaction of the MOE, CTEs, universities, and USAID/Ethiopia.  To identify lessons learned including innovations.  To identify the opportunities and challenges of program implementation and how they have been addressed, including recommendations for appropriately addressing those that have not yet been sufficiently addressed.  To determine if the program is still relevant and desirable, and suggest options for future directions and sustainability beyond the end of the current grant period (July 31, 2012) In realizing the objectives of the Evaluation, the team started the Performance Evaluation with solid description of the project and conducted a comprehensive appraisal of the performance of the program at output and outcome levels as well as ascertaining the sustainability of the project in terms of continuity of the benefits associated with changes brought by the project. The team has also tried to examine the emerging impacts envisaged as the longer-lasting change(s) to individuals or institutions as a result of the project. This was carried out based on participatory approach involving all concerned stakeholders. In particular, the evaluation method used facilitated validation of the data collected through triangulation of the sources and methods. The team considered 9 HLIs as the institutional sample for the purpose of this Performance Evaluation. These were Debreberhan University (DBU), Debreberhan College of Teacher Education (DBCTE), Hossana College of Teacher Education (HCTE), Kotebe College 7 of Teacher Education (KCTE), Bahir Dar University (BDU), Haromaya University (HU), Gondar University (GU), Gondar College of Teacher Education (GCTE), and St. Mary’s University College (SMUC). The primary sources of data were two respondents from IFESH Country Office (the country representative and the program officer), 18 volunteers currently on duty, 6 ex-volunteers, State Minister of General Education (Federal Ministry of Education), Director of Teachers and Educational Leaders Development Directorate; Director of Planning and Resource Mobilization Directorate; two experts of teachers (and English Language) education (Federal Ministry of Education), Officials in host higher education institutions (Presidents, Vice￾Presidents, Deans; Department Heads and HDP/TDP coordinators as well as Assistant coordinators, Gender Officers). Besides, teachers and directors of local schools, graduating class students and HDP trainees who have significantly benefitted from the volunteers’ activities had served as sources of data (list of respondents is provided in Annex I). Though the plan assumed that an adequate number of volunteers would be actively working at the selected Universities/CTEs, it was observed that only one volunteer was actively employed in most of the institutions visited and there were places where no one was found (e.g., Adama University, KCTE, St. Mary’s University College). The plan also assumed that the assignment of volunteers was limited mainly to teacher education institutions and English language areas. However, there were instances where the assignment of volunteers was made to areas less directly related to these two core fields based on the requests from the universities (e.g. health and arts at Haromaya; health and technical Assistance at Adama). The respective respondents in each HLI were selected using simple random sampling, purposive sampling, availability sampling, and snowball sampling methods by considering their position in connection to the volunteers’ management and their activities. Snow-ball sampling was particularly useful when studying about the activities of the early volunteers during which someone who got interviewed first is requested to mention if he/she knows anybody who has worked with the volunteers. This has helped to overcome the problem of lack of adequate documentation about the activities of the volunteers and the ‘swift’ turn-over of officials at the host institutes. The secondary sources of data included project documents of IFESH, report of volunteers, ELIC centres and other materials produced by the volunteers or by those who have got guidance from the volunteers. The team was guided by an Evaluation framework including field report format, planned activity schedule, and data collection tools pertinent to the nature of the Evaluation. Project documents and sample materials produced by the volunteers as well as publications were reviewed. The team had also clustered the HLIs based on their geographical location and visited the entire sampled HLIs. Apart from meeting all the eighteen current volunteers for FGD at Beshale Hotel during the mid-year review meeting in Addis Ababa, the team also approached eleven (11) ex-volunteers through their e-mail addresses and got prompt responses from six (6) of them. Document review, key informants interview, FGD and observation were the data collection tools used for the evaluation. Data generated through these tools were analysed using both qualitative and quantitative methods. The findings of the Evaluation are itemised according to relevant themes as per the SOW. The results of the analyses revealed that IFESH had mobilized 82 volunteers in diverse areas of expertise in the last 10 years (2002/03—2011/12) with a grant released by USAID. The volunteers were assigned as pre-service and in-service teachers, trainers, mentors, coaches, policy advisors, and technocrats of various occupations in about 49 host institutions. It was observed that more than 40,000 personalities (instructors, school teachers, students, education 8 practitioners & others) have benefited from the value-adding services of IFESH volunteers since 2002/03. As whole, in running this program, 3,628,605 USD has been allocated (see Annex VI for details on this). In line with the ultimate purpose of the program, some emerging impacts were noted. Among these are, professional competence of teacher educators or instructors has improved over time which is reflected in classroom interactions and communications with their students which, in turn, is hoped to contribute to improvement of the quality of education in the institutions. Female students are actively participating in clubs, ELIC, and networks. Some encouraging signs are being observed regarding internal or local voluntarism and are being strengthened in order to ensure sustainability. Similarly, the work of the volunteers provided the locals with experiential learning and skills in preparing programs for workshops, seminars, reporting of learning events, development of proposals, follow up on observations and feedback, communication and interactions for sharing experiences and lessons. Co-workers also indicated that they have acquired such values as commitment to one’s purpose, punctuality and seriousness from the volunteers. Further, the results of the Evaluation had indicated that the Ministry and other hosting institutions have been quite satisfied with the performances of IFESH volunteers and the demand for their services is still surging. The Ministry is still desirable to have many more volunteers who are experts of English language and subject area pedagogy. Thus, based on the evidences it gathered so far, the consulting team had safely concluded that IFESH/USAID volunteerism program in Ethiopia has been highly successful. Such encouraging performances have been credited to the concerted efforts of highly committed and experienced volunteers, a great deal of cooperation and support by management and work colleagues in host institutions, a strong leadership from IFESH/Ethiopia country office, a high level support from the Ministry of Education and USAID/Ethiopia. It was also observed that though the program is facilitated by volunteers, it has entertained several challenges that might hinder its pace. The most pertinent and pressing ones include bureaucratic management on the part of some host institutions, language barriers, lack of clear-cut schedules (for the volunteers) to get started which resulted sometimes in excessive idling, and a tendency for the co-workers to leave everything to the volunteers instead of working closely and learning from them in the process. A huge gap between what the Ministry demands and what IFESH supplies also happens to be one of critical challenges of the program. IFESH was unable to meet the demands of the Ministry due to cost and other factors. There are several best practices and lessons of the programme that can be considered for further interventions. Some of the prominent practices that are recommended for the purpose of replication (scaling up) are the establishment of Active ELIC centre , frequent moderation of free talk, weekly debates, reading e-books, participating in “You Go Girls Club” and “Celebrating English Day” for the purpose of English language improvement, organizing of several training on essay writings for female university students and supporting primary school teachers, trying to make females active in IHL, the implementation of HDP for capacitating teachers and introducing them with reflective methods of teaching. The program is strongly aligned with the education policy & strategies of the country with its critical relevance particularly to both the general education quality education improvement program and the English Language Improvement Program. The program has also manifested 9 much strength that contributed for its success like the deployment of highly qualified and experienced volunteers, existence of strong IFESH country office to support the volunteers, system of placing volunteers based on the need of host institutes, and a culture of developing handover notes to successors to ensure continuity of activities. In general, the program was found to be highly successful and the team suggests that the program deserves continuation with practical emphasis and inclination on capacity building for the host institutions and co￾workers. The Performance Evaluation resulted in several recommendations for future interventions. These include the need for effective and efficient volunteer management system to be practiced at host institutions. This requires the need for well-organized ‘on-arrival orientation’ for volunteers to establish early familiarity with HLIs staff, the need for preparing clear job description and defining focus areas for the volunteers, the need to focus on subject area pedagogy and not limited to generic teaching methods. There is also a need to focus on capacity building rather than problem solving (service giving) on the part of the volunteers. To this end, it is recommended that the volunteers as well as their co-workers at host institutions should be adequately capacitated. The volunteers have to be encouraged to engage themselves in innovative projects through appropriate level of funding from USAID and host institutions. The MoE, USAID, and IFESH have to devise appropriate strategy to encourage and mobilize local volunteers efficiently and effectively. Finally, the MoE and/or the host institutions have to work hard towards creating a robust knowledge management system so that cherished wisdom and wealth of expertise from volunteers can be generated, stored, disseminated, retrieved, and transferred to relevant job incumbents. Best practices (outstanding performances) of volunteers need to be documented systematically and shared among host institutions as well as newly recruited volunteers. 10 1. Introduction 1.1. Background A century has gone since modern Western education was introduced to Ethiopia and decades have elapsed since some kind of teacher education started to produce local teachers for the schools/colleges. Ever since then there has been concerns with regard to the adequacy and quality of the teacher education sub-system. In all times, there have been efforts to support local initiatives with inputs from experts (volunteers or advisors) deployed by partner countries and/or organizations. One of the biggest partners that have been supporting MoE’s education strategy by deploying experienced American volunteers is USAID/IFESH. IFESH is a United States-based private, voluntary, non-profit and non-governmental organization (NGO). It was Rev. Sullivan who founded IFESH in 1981 being the author of Sullivan Principles, which was built on the principle of "self-help", endowing people with the tools to help themselves in overcoming barriers of poverty and oppression. As cited in the IFESH ‘The Bridge Newsletter’ (volume 8, issue 2, 2008), Rev. Dr. Leon H. Sullivan put his vision as, “I envision a bridge from America to Africa over which one day we will move from one side to the other and back again . . .” IFESH/Ethiopia established an Ethiopian office in 1996 based on an agreement between IFESH and the Ethiopian Ministry of Education. IFESH is currently operating in Ethiopia with a License No. 0898. It has been supporting the education sector in Ethiopia by deploying highly qualified professionals and experienced American Volunteer Teachers (AVT) to various teacher education and other higher education institutions (HEIs) with funds from USAID/Ethiopia. Since 2002/03, IFESH has been supporting the Government’s effort to improve the quality of education by supporting teacher’s capacity building program. The Ministry of Education conducted a study on the ‘Quality and Effectiveness of Teacher Education in Ethiopia’ in 2002. Based on the recommendations and indicative action plan presented in the study report, a task force was established to produce the ‘National Framework for the Teacher Education System Overhaul (TESO) Programme’. TESO consists of five priority programmes that have been implemented since 2003. These include:  PP1: Teacher Educators  PP2: Pre-service Teacher Training  PP3: In-service Teacher Training  PP4: Teacher Education System  PP5: Leadership and Management Programme (LAMP)  PP6: English Language Improvement Programme (ELIP) Starting in 2003-2004, a group of donors are jointly funding the activities related to teacher education under a Teacher Development Programme (TDP) that includes the components of the TESO, the English Language Improvement Programme (ELIP), and the Leadership and Management Programme (LAMP). USAID is one of the major donors that support the program. Table 1: Education Indicators (MOE , 2010/11 Annual Abstract Kindergarten GER 5.2% Net enrolment primary education 85.3% GPI at Primary Level (1-8) 0.94 Primary Pupil Teacher Ratios (Grades 1-8) 51:1 Qualified Primary Teachers(diploma) `1-4s 5-8 20.1% 83.3% primary Repetition Rate (1-8) 8.5% Drop-out Rate of Primary (Grades 1-8) 13.1% Water: 34.4% Libraries: 41.0% NER of Secondary First Cycle (Grades 9-10) 16.3% Population below poverty line (less than US$1 per day) 39% (2005) # of CTE 32 Enrolment in CTE 164,501(41% Females) Graduates of CTE 26802(40.3% Female Enrolment in Higher 11 In July 2002, USAID/Ethiopia awarded IFESH with the Capacity Building in Teacher Education Program (CBTEP) under grant number 663–G–00–02–00360-00 for a five-year period (from July 24, 2002 through July 31, 2007). Based on its satisfactory performance, continued relevance, and the need expressed by the Government of Ethiopia (GoE), the grant was extended in July 2007 for one year (from August 1, 2007 through July 31, 2008) under a bridge funding mechanism to deploy 10 volunteers in Teacher Education Institutions (TEIs). In 2008, IFESH was granted an extension of the program for another four years from August 22, 2008 through July 31, 2012 to bring additional 40 volunteers. Originally, the overall goal of the CBTEP was to help improve the quality of education through capacity building of teacher education institutions and in training teacher Colleges in alignment with the Ministry of Education’s teacher development program initiative with the following objectives: 1. To improve teaching of the English language; 2. To provide assistance to TTIs, TTCs and the Ministry of Education in order to improve the quality of primary education, especially as related to Teaching of the English language; and 3. To provide English language books, reference materials and other resource materials for schools and resource centres. In the second phase of CBTEP, the objectives were revised and explained in greater detail while maintaining the original goal as point of reference without any significant paradigm shift in its strategies. Thus, the objectives of CBTEP II (2009 - 2012) were: 1. To work toward the attainment of the UN Millennium Development Goals, particularly achieving universal primary education and combating HIV/AIDS, malaria and other diseases. 2. To assist the Ministry of Education in implementing its initiative in GEQIP, especially the Teacher Development Program (TDP), by providing technical assistance to the Ministry of Education faculties who train teacher trainers and teachers of TEIs, in order to improve the quality of education; 3. To assist in the establishment of Special Needs Education in the TEIs; 4. To assist the Ministry with strategic planning for its School Improvement Program (SIP); 5. To improve teaching of the English Language in teacher education institutions; and, 6. To initiate and implement HIV/AIDS education programs by integrating them with gender equality promotion, preventing harmful traditional practices and STDs In realizing these objectives, IFESH/CBTEP recruited and assigned well-qualified, experienced American Volunteers (AVs) to CTEs, education faculties in the universities and the MoE to improve the quality of education through capacity building of pre- and in-service teachers as well as educational administrators. The volunteers worked with Ethiopian counterparts under the Higher Diploma Program (HDP), served as advisors, and performed other teaching tasks. Some volunteer educators facilitated HIV/AIDS education and gender mainstreaming efforts and improved the teaching of English language in teacher training institutions and improved democracy and good governance through peace education; and fostering social and cultural relations between Africans and Americans regardless of race, colour, faith, or sex. IFESH was required to provide a total of 32 volunteers between 2002/3 and 2007, another 10 volunteers in 12 2007/2008, and 40 volunteers in the four years between 2008/9 and 2011/12, totalling 82 volunteers during the life of the project (i.e. an average of 10 volunteers annually). To facilitate the deployment of AVs in Ethiopia, USAID/Ethiopia had allocated about $3,628,605 (Three million six hundred twenty-eight thousand and six hundred five dollars) (See Annex VI for details on this). The contribution from the side of the Ethiopian Government, which is mainly in kind is not included into this amount. IFESH CBPTE programme has been implemented on the basis of collaboration and agreements between IFESH and the United States Agency for International Development (USAID) and the Government of Ethiopia/Ministry of Education. 1.2. Objectives of the Evaluation The main purpose of this evaluation was to assess the performance of the USAID/Ethiopia￾funded CBTEP implemented by IFESH and thereby provide USAID/Ethiopia and the Ethiopian MoE useful information and data for determining the future direction in the strategies of USAID/Ethiopia’s Basic Education Office and the MoE for enhancing the quality of teacher education programs. More specifically, the performance evaluation was conducted to achieve the following objectives:  To examine and verify the results and effectiveness (output and outcome) of the program since its inception guided by the project agreement, goal, objectives, and targets.  To examine whether the program activities, outputs, and outcomes have been integrated as appropriate into the initiatives of the host government and teacher training institutions.  To determine the level of satisfaction of the MOE, CTEs, universities, and USAID/Ethiopia.  To identify lessons learned including innovations.  To identify the opportunities and challenges of program implementation and how they have been addressed, including recommendations for appropriately addressing those that have not yet been sufficiently addressed.  To determine if the program is still relevant and desirable, and suggest options for future directions and sustainability beyond the end of the current grant period (July 31, 2012). In order to facilitate effective data collection and assessment, the consulting team identified the following themes:  Project initiation, partnership, relevance and support  Assessment of interventions: practices, achievements, and innovations  Beneficiaries’ satisfaction with the English language enhancement, general teaching and learning process improvement, educational resource solicitation and distribution, as well as system introduction and improvement activities carried out by the volunteers.  The emerging results of the program and significant changes observed in English Department, situation of colleges/universities, capacity of teachers and institutes.  Strengths, weaknesses, opportunities and challenges observed during program implementation.  Sustainability of the volunteers’ activities and outcomes of the intervention in the absence of the volunteers. 13 1.3. Scope of the Assessment The evaluation started with solid description of the project and conducted a comprehensive analysis of the project output, outcome and sustainability. The output and outcome are conceived as continuum of changes brought about the value additions of the program. It has also tried to see the emerging results of the project as envisaged longer-lasting change(s) to individuals or institutions. In general, the assessment covered very broad areas while trying to look into the results of the project which includes project inputs, strategies applied, efficiency, effectiveness, and sustainability. In addressing these key issues, the consulting team adopted indicators for measuring the outputs and outcomes of the program under consideration. Some of the key issues considered included ways that the project changed/transformed the understanding, attitudes and professional practices of the teacher educators and education experts, ways that the activities of the project transformed the institutional culture and introduced professionalism in teacher education, changes in understanding, disposition and practices created on the part of the teacher educators and education experts, how the volunteers were received by hosting institutions, institutional factors that promote/constrain the maximum use of the volunteers and challenges the volunteers faced during their assignment. The evaluation was carried out from November 21, 2011 to January 13, 2012. 2. Technical Approach/Methodology The evaluation was done based on participatory approach in which all concerned stakeholders were involved and they were free to comment on the program. Suitable evaluation tools were devised to collect data from the identified primary and secondary sources for the evaluation. The method of evaluation was designed with in-built mechanisms for the validation of qualitative data through triangulation. The primary sources of data were two respondents from IFESH Country Office (the country representative and the program manager), 18 volunteers currently on duty, 6 ex-volunteers, State Minister of General Education (Federal Ministry of Education), Director of Planning and Resource Mobilization Directorate, Director of Teachers and Leaders Professional Development Directorate; two experts of teachers (and English Language) education (federal Ministry of Education), officials of the hosting higher education institutions (viz., Presidents, Vice-Presidents, Deans; Department Heads; HDP/TDP coordinators & Assistant coordinators; Gender officers; & local counterparts of AVs). Besides, senior university students as well as teachers and directors of local schools who have significantly benefitted from the volunteers’ activities had served as sources of data (list of respondents is provided in Annex I). Though the plan assumed that an adequate number of volunteers would be actively working at the selected Universities/CTEs, it was observed that only one volunteer was actively employed in most of the institutes visited and there were places where none were found (e.g., Adama University and KCTE). The plan also assumed that the assignment of volunteers was limited mainly to teacher education institutions and English language areas. However, there were instances where the assignment of volunteers was made to areas less directly related to these two core fields based on the requests from the universities (e.g. health and arts at Haramaya; Health and Technical Assistance at Adama). 14 The secondary sources of data were the project documents (master documents, action plans, annual narrative reports of IFESH, report of volunteers, evaluation reports of HLIs about the volunteers’ performances, ELIC centres and other materials produced (see Annex III for list of the documents reviewed). It was realised that there is neither baselines nor clear indicators of performance in the project documents. For instance, the documents talk about improving English language skills. Yet there were no measurable indicators as to what improvements in language skills of the beneficiaries should be. Similarly, even though the project document mentions improving the teaching methods applied by the teacher educators, there was no indicators of targets and measurable indicators. Thus, considering the facilitating roles of the volunteers and through an open discussion of the most significant changes in areas related to the volunteers’ activities, attempt has been made to deduce the emerging results. In facilitating the performance evaluation, the team prepared an evaluation framework including field report format, data collection tools pertinent to the nature of the evaluation, reviewed the project documents, and sample materials produced by the volunteers and publications/newsletters from IFESH/Ethiopia country office. The evaluation team had also grouped itself to sample HLIs based on their geographical location and visited sample HLIs that have benefited from the program. Based on the discussions and decision reached with USAID, the team considered 6 Universities and 4 CTEs as samples of the study. This included Debreberhan College of Teacher Education (DBCTE), Hossana CTE, Kotebe CTE, Debreberhan University, Bahir Dar University, Adama University, Haramaya University, Gondar University, Gondar College of Teacher Education (GCTE), and St. Mary’s University College. Accordingly, data have been collected from MoE, hosting institutions and schools, beneficiaries (students, teachers, & administrative support staff), IFESH country office, volunteers and Ex-volunteers. Finally, the team collected the information from all officers and ex-officers whom the volunteers had/have worked with. Such respondents, depending on the specific situation of the HLI, included the Presidents of the Universities; Heads of the CTEs, Directors of Academic Programs; Deans of the Faculties/Schools of Education; Coordinators/leaders of HDP/TDP; Assistant Coordinators of HDP/TDP; VSO volunteers (co-workers); Head of University Clinic; Coordinator of HIV/AIDS Prevention Office; Heads of Gender Offices; Community service directors, teachers, ELIC coordinators and members. The respective respondents in each HLI were selected using purposive, availability (plus snow-ball) and simple random sampling methods by considering their position in connection to the volunteers’ management and their activities. Snow-ball sampling was particularly useful when studying about the activities of the early volunteers during which someone who got interviewed first is requested to mention if he/she knows anybody who has worked with the volunteers. This has helped to overcome the problem of lack of adequate documentation about the activities of the volunteers and the ‘swift’ turn￾over of officials at the host institutes. In this evaluation, both quantitative and qualitative data collection tools were employed. These included document review (which focused on analysis of relevant documents), key informants interview, FGD with students and eighteen volunteers (see Annex VI), questionnaire with a very few questions addressing the level of satisfaction about the program and finally observation on the overall campus environment, ELIC centres and the volunteers’ work stations. The instruments of data collection had passed through rigorous content validity assessment. The FGD Guideline developed by the team and the same was deliberated by the team prior to its administration (see Annex II). Moreover, the team was guided by the philosophy of ‘Appreciative Inquiry’ while collecting data from the respondents. 15 Both qualitative and quantitative methods have been applied to analyze the data collected from the various sources. Quantitative analysis involved summarizing data in tables and figures based on computations of counts, averages and percentages. Qualitative analysis involved classification/grouping of data, application of content analysis and interpretive analysis. 3. Limitations of the Evaluation While every possible effort has been made to carry out the evaluation in a professionally sound way, there are some reservations which the team of consultants would like to point out as possible limitations. The volunteers have varied and diverse expertise. Therefore, they are not expected to do one and the same thing. For instance, some are active only in teacher training and could devote their time in in-service teacher education in the host institutions and in the local formal schools. Some others are rather very active on cross-cutting issues such as gender and HIV/AIDS. Most such volunteers opt to work with female students within the host institutions. There are volunteers who are more interested in direct involvement in major academic activities such as teaching in post-graduate classes and advising students on their research and many more. This situation made using uniform sampling frame very difficult. As a result, the sampling frame was varied from one institution to the other. Nonetheless, whatever data acquired have been handled with utmost professional care. Besides, due to the shortage of documents on details of everyday activities of the ex-volunteers at the host institutions, it was very difficult to cross-check generic reports obtained from IFESH country office. This problem has been aggravated by the fact that some of the officers with whom the volunteers worked are not even in the institutions let alone on their posts. Nonetheless attempt has been made to overcome this by tracing the whereabouts of those officers. Besides, the e-mail contacts with the ex-volunteers have helped the evaluation immensely. 4. Major Findings 4.1. The Project: Initiation, Relevance, Partnership, and Support Program Initiation The program was initiated with the request of MoE and all of the respondents confirmed that it is aligned with the education program of Ethiopia. In this regard, the IFESH/Ethiopia country office had confirmed that the recruitment of AVs passes through four stages in a cycle: needs analysis, ranking the needs (in the light of order of priorities of educational institutions, relevance of the volunteers’ qualifications & other considerations), recruitment of the volunteers, and placement and orientation of the volunteers. But all of the respondents except senior management members in the host institutions reported that they were not aware of the procedures how the AVs were recruited. There is still a blurred expression about the acknowledgment of volunteer’s assignment in each institute as there are faculties (e.g. co￾workers at Haramaya University and DBCTE) which were not informed about the selection criteria and background of the volunteers. For instance, HDP coordinators (and Education Faculty Dean) at Haramaya University noted that they knew nothing about the recruitment of the volunteers; even they took no part in ‘on-arrival orientation’ for the volunteers. The same was reported from DBCTE. On the contrary, HPD coordinators at Adama University indicated that they attended on-arrival orientation workshops organized to introduce the volunteers to co￾workers and coordinators. Irrespective of such disarray, which may be partly attributed to the 16 very nature of the host institutions, higher level managements of the Universities reported that the IFESH Country Office invites the HLIs to make their requisitions based on the resumes of potential volunteers sent to the institutions for their review and approval. It is only when the credentials of the volunteers are approved by the receiving institutions that IFESH allocates the volunteers. Nonetheless, there were cases where volunteers appeared to be technically incompetent relative to the job requirements and in some cases could not fit the work environment as well as the cultural make-up of the locality. The result was that some of these volunteers left the country without doing any further. It was also reported that some colleges like Hossana CTE, did not get volunteers on special needs education irrespective of repeated requests. Program Relevance The HLIs, particularly the newly opened ones, are staffed with very young, less experienced but energetic instructors. These instructors require close support in the form of experience sharing, coaching and mentoring from experienced and expert professionals. The IFESH volunteers are fit to fill such gaps. Many of the HLIs are also staffed with less effective teachers in conveying the essence of the subject matter and practical life experience because of their limited exposure to modern (learner-centred) teaching methods (active learning methodologies) and lack of mastery of English language skills. Thus, it has been confirmed that the program responded to the challenges by providing technical and material support that expanded the horizons for the improvement of language teaching skills and methodologies, which is a significant contribution to the quality of education at primary and secondary schools as well as at HLI. A statement of the Academic President of St. Marry University College acknowledges this fact. He said, “We are lucky enough to get a volunteer who is experienced and willing to work with us. He served as CPD leader and was resourceful in organizing public seminars on inter-cultural issues”. Similarly, the President of Haramaya University expressed his due regard to the program as, The volunteers go out of their way to show us what we did not see. For instance, we did not have Guidance and Counselling service separately set up to take care of students and staff members who need such services. However, Dr. Augustine Sasey, the current IFESH volunteer is working on this very important element. Not only that, as you know, our teachers teach their classes as usual business and go to their home or they feel that they have done their entire job. The volunteers, however, are not like that. They give equal value to the informal and non-formal element of learning (for instance, the school clubs, private consultations, and other ways of engaging or making the learners active), which are very important in attitude formation as professionals. The President also noted that the volunteers from native speakers of English language are very useful for the following inter-related reasons: (i) English is a very important language to access knowledge and present day technological innovation and global competitiveness; and (ii) most of the students in our universities are ill-prepared in using the English language. The other very important element the President underscored is that because these volunteers come with experience, they help us overcome the weaknesses of our young instructors who are new both to the profession and not very familiar with application of their academic knowledge. All these opinions are shared by the former AVP of Adama University who gives high value to the volunteers’ presence as far as they are carefully used. It can be said that all the officers interviewed do agree to the above points and strongly underscore the relevance and appropriateness of the IFESH-Ethiopia’s activities. The volunteers on their side also confirmed 17 that the program is valuable and necessary for the Ethiopian Education system. But some volunteers warned IFESH to stick to the ‘Self-help’ principles of the Founder. Volunteers have also confirmed that volunteerism is one of the strategies for Ethiopia since it is a cost effective deployment of professionals. It is an asset and a big opportunity for Ethiopia to have experienced and practice-oriented professionals. IFESH Ethiopia’s Support All the Volunteers had expressed their satisfaction with the program management on the side of the IFESH. For the volunteers, IFESH has managed the program very well. The support the volunteers provided was rated to be very much welcoming. They (the volunteers) testified that IFESH is a well organized and professional organization that provided the leverage to manage the program successfully. A volunteer who had similar experience in another African country comparing the activities of IFESH Ethiopia to IFESH office in that particular country judged IFESH Ethiopia as much more competent in terms of effectively managing the program. For the volunteers, the country-level orientations they were provided on-arrival were also additional strengths of the management. The Evaluation team had learned from the documents and face-to-face interviews of IFESH program manager and the current volunteers that IFESH/Ethiopia country office makes periodic visits to host institutions and monitors the progress of the volunteers. The country office is also vigilant enough in tracing any difficulties/challenges that the volunteers are facing and acts quickly to amicably settle the problem. The documents reviewed from the country office also revealed that the office has an elaborate plan for monitoring and reviewing the volunteers’ progress and its report was found to be well-documented. Further, it has been learned that IFESH/Ethiopia has good partnership and working relations with HLIs. Particularly with Universities where the leadership is relatively stable during the last few years the relation is very strong. IFESH’s relation with University units and Departments is more through the volunteers themselves and dependent on the nature of management (decision making) the university/CTE employs. In those host institutes where the management of the volunteers is rather centralized; there is no particular contact IFESH makes directly with the units, ELIC centres, HDP coordinator and even departments. In such cases there is no means whereby IFESH gets information about the contributions (strengths and drawbacks) of the volunteers and the program itself from the units’ perspectives. A good example as mentioned above is Haramaya University. USAID’s Support With reference to USAID, even though the volunteers are aware that they are here with the financial support of USAID to Ethiopian Government; the FGD participants had the reservation regarding the visibility and close follow up of USAID. That means they expect equal visibility of USAID as IFESH in the operation of the programme. Lack of fund for some burning needs at the grass root level (which are connected to their activities) might have instigated the volunteers to develop such opinion of USAID. As a result the volunteers tend to feel that USAID has not done much in connection to the program apart from providing fund to IFESH for the operation of the major program. On the other hand, the FGD participants and individual volunteers suggested that USAID should work hard to ensure the coordination of US-funded voluntary efforts like IFESH and PEACE CORPS. It was suggested that USAID should take the lead in promoting IFESH volunteers’ management and creation of a strong positive image for IFESH – a tendency to expect also technical support. 18 Host Institutions’ Support The Deans and Vice-deans of the HLI indicated that they provided all the needed supports to the volunteers to enable them perform the activities they have planned. All the HLIs facilitated housing, budget support, transportation and assigned coordinator or focal person to support the volunteers in their work. The assignment of a focal person who assisted and accompanied the volunteers in their work facilitated integration and handling bureaucratic processes as well as dealing with cultural and social challenges in the community in which they worked and lived. The volunteers themselves have affirmed that they got appropriate institutional support from the HLIs. They perceived that their suggestions were well taken and all the necessary supports were provided for any initiative they come up with as far as there is no resource limitation. This has been an opportunity even for home institutions. According to the Dean of the Faculty of Education and Behavioural Sciences (HU), this is an opportunity the Faculty is enjoying as a result of the volunteers’ presence. He said, “They can knock every door and get it open. As a result, they are essential means for the Faculty to have broader influences.” This shows that the volunteers play important role in strengthening the visibility (and contributions) of Education Faculties at the host Universities. On the contrary, volunteers have condemned the ever endless meetings and unplanned operations of universities and colleges where they are unable to get coordinators. Apart from logistic and a ‘go on’ support on the personal initiatives the volunteers take, there is no particular guidance which the volunteers receive from any corner regarding their professional activities. On their own, the volunteers try to do many things (to address many outstanding problems in which they feel handicapped in terms of expertise and training received). For instance, the current volunteer at Haramaya University who is positioned as HDP advisor is also serving as ‘HDP leader’, ‘Post-graduate program theses supervisor’, ‘Head of Guidance and Counselling Unit’, ‘supports the clinic and the gender office, and also engage himself in editorial works.’ This is seen as work overload for one person given particularly the demanding nature of the HDP and PGHET. If at all the volunteer has to deal with all these businesses, he needs to have other experts who support him and his duty would rather be coaching. The same is true in DBU where Katheryn is expected to work on ELIC, CPD of linked primary schools, and support Gender office. The current volunteer at Bahir Dar University, Mary Lugton, was also found to be extremely overstretched because of her services’ demand from diverse sources including from the University staff (for academic and professional English support), from students (for developing the 4 skills in English), from gender office (for training girls in life skills and assertiveness), from primary and secondary schools (for continuous professional development of teachers), from women instructors club (for building their research skills), and from Regional Education Bureau and Woreda Education Offices (for training principals & education officers in mentoring & coaching). With only three months of her stay in Bahir Dar University, Mary has been bombarded with lots of assignments. Comparing the level of engagement and effectiveness of services rendered by IFESH volunteers with that of VSO, an observer from Harromaya had the following to say, The management gives the IFESH volunteers many additional works and they finish their terms without observable accomplishment in any one area. Therefore, when I compare them with VSO volunteers, the VSO volunteers could deliver effectively in their areas because they (VSO volunteers) are strict in terms of concentrating on areas to which they were assigned. (The former HDP coordinator at Haramaya University) 19 These instances indicate that some of the IFESH volunteers have been overstretched on their duties, which may cause them to be less effective when it comes to delivering quality results. Thus, a culture of joint planning as well as realistic goal setting has to be created in the host institutions in order to ensure optimum utilization of IFESH volunteers. On the other hand, there are cases where the volunteers reported that they are underutilized by host institutions (e.g. DBCTE) and that they initially disoriented and confused due to the lack of adequate institutional information. For instance, one volunteer remarked, “I was like a floating boat for about 3 months”. There were also cases where lack of continued commitment to support for the volunteers’ initiatives has been reported. Regarding this a volunteer said, “Verbally the officers are ready to help but in practice they lack determination to pursue on planned activities.” Such reflections once again remind all the concerned stakeholders that a system has to be established at host institutions to manage the volunteers in the most effective and efficient ways. Competence of volunteers The coordinators were asked to assess the competence of the volunteers on the following criteria (indicators):  English language enhancement  Improvement of general teaching–learning process  Engagement in co-curricular activities  Solicitation and distribution of educational resources  Introduction and improvement of systems  Readiness to share and receive knowledge  Participation in community support activities The assessment was to be done on a five-point scale ranging from ‘very-high’ to ‘very-low’ level of competence. In this regard, the coordinators at Haramaya and Adama universities were reserved as there were perceived individual differences among volunteers. With such proviso, they gave their general observation that most of the respondents agreed that their level of satisfaction with the competence of the volunteers was ‘High’. But a coordinator from Bahir Dar University described that all AVs that were/are assigned to the university except one were found to be very competent in their areas of assignment so far. Particularly, those that were assigned for HDP and the two of the three assigned for ELIC were found to be very competent and successful in their stay at the University. All the officers contacted in this study also reported that the volunteers are willing to support and share their experience with the local counterparts. However, they said that “the capacity of the locals to learn from and maximize benefits from the presence of the volunteers depends largely on interest, interaction skills, and the information seeking behaviour of the local teachers.” For example, the ELIC and HDP coordinators of Hosanna TEC agreed on the opinion that “Cultural and communication barriers limited the opportunities for the majority of the staff to interact with and learn from the volunteers”. The Drama Club coordinator expressed that “Teaching is both an art and a science that can be learned through practice and working with experienced colleagues. The volunteers can be a great resource for the transfer of tacit knowledge through active engagement with them.” According to the ELIC coordinator of Hossana, “The willingness and readiness to work together and learn new ways of doing things from the volunteers and information seeking and sharing behaviour of individual staff and attitudinal and cultural problem constrained the maximum utilization of the volunteers”. This 20 was supported by the observations of the volunteer placed at the MoE when he said, “Everyone is helpful and cooperative, but sometimes the capacity of some people is not up to my hopes.” The above observations suggest that AVs are creating great opportunities to learn and improve themselves for local counterparts at host institutions. However, great deals of local staff have not been benefitted because of their lack of efforts to grab the windows of opportunities opened to them. This may require devising appropriate knowledge management strategy in each hosting institution. 4.2. Assessment of Interventions: Practices, Achievements and Innovations The interventions of the project have been seen in terms of the four areas the project document identified earlier: English language improvement; teaching and systems improvement; educational resources supply and utilization; and community services and cross-cutting issues. The Consulting team had tried to examine the practices, achievements, and innovation attempts recorded and credited to any of the volunteers and/or to the host institutions in light of the above intervention areas. These are explained as under. 4.2.1.English language improvements As clearly stated in the CBTEP II Document, one of the objectives of the program is to improve teaching of the English Language in teacher education institutions. Again, one of the strategies proposed is the establishment of ELIC for ELIP. ELIP is doing well both in DBU, BDU, Gondar University, Haramaya University, Adama University, CTC, and HTC. The demand for English Language training is soaring from the students, instructors, and administrative staffs. Having a quick look at Figure 1 and Table 2 below, a case in BDU (2011/2012), provides sufficient evidence. Fig. 1: No of students attending English Language Tutorial sessions at BDU, 2011/2012 21 Table 2: Type and number of trainees attending at BDU The data presented in Figure 1 and Table 2, taking the example of BDU, depicts attempts that are under way in improving the English language proficiency of both students and teachers in HLI. According to the FGD held in BDU and DBU, ELIP has contributed in improving reading, communication skills as well as writing skills of the participants. There are debates, free talks, teaching grammar, exercise writing, leading/chairing meetings and other activities led by AVs, which help to build confidence to use English, positive feeling, etc. as some of the positive interventions intended to improve English Language skills of the beneficiaries. Volunteers are also helping the ELIP centre to be able to get more input supplies like English Language Kits, Cassettes, and aids by contacting different donors. On top of this, ELIC has been fully furnished with computers and audio-video cassettes with donations from IFESH/Ethiopia country office at BDU. All of the interviewees and FGD participants had noted that the demand for English language service is soaring from university staff (for academic language support service), from admin staff (for improving communication skills), from students (for developing writing, listening, speaking, and communication skills), and from school teachers (for enhancing effective teaching and communication skills). As a whole, there is a positive and green light in the ELIP, especially among students. Even the existence of a centre itself is valued and users of the ELIC are increasing from time to time. In almost all the institutes visited ELIC has Internet connections, show rooms, and mini￾libraries supported by native speakers where the HLI community is enjoying and getting the opportunity to communicate with them. The Ethiopian English teachers have been assigned there as committee members to sustain the program. The program is perceived to be good but the limited resources and (in a few cases) the upper management of the Universities/colleges were found to be less attentive to the centre mainly due to the several problems which compete for the limited resource. The demand for the service the centres provide is still there, hence the volunteers are over-stretched and this has created gaps in the level of satisfaction of the beneficiaries. 4.2.2.Teaching and systems improvements The respondents (e.g. from Hossana CTE & Haramaya University) agreed that the volunteers have changed the prevailing work culture and their presence was visible in the work of the units which they were supporting. For instance, the Dean of Hosanna CTE indicated that “The volunteers showed us the importance of things which we did not normally appreciate and that the solutions to our problems are within us. We learned that we can make a difference and started to look for solutions to problems within us.” The volunteers widened the thinking and Trainees M F T Students (Tutorial and academic purpose) 147 51 198 Teachers 28 9 37 Administrative staff 11 12 23 Total 186 72 258 22 perception horizons of the institutions to improve the learning system in their institutions. The ELTIP coordinator at the MoE believes that the presence of the volunteers contributed towards improving the work culture in the office. He said, We have learned how to organize our work and deliver results timely. Now, we are not accumulating work. For instance, I used to give a script to the secretary to type it for me. Now, following the interventions by AVs, I am writing it myself. The performance of our directorate is exemplary at the ministry level, and this is mainly due to the contribution and skill transfer of the volunteers. The volunteer at the MoE supported the Teachers and Leadership Development Core Process in developing competency standards of English teachers based on which competency tests were developed for both primary and secondary school English teachers. There are 32 testing centres in the country and the volunteer trained 130 examiners and 10 exam administrators in November and December 2011. The volunteer noted that the test construction and administration has improved a great deal since its inception in May 2011.The volunteer has a plan/mandate to continue working on the analysis of the test results and based on which would develop training modules to provide training for the English teachers nationally. At Haramaya, the volunteers got involved in a lot of system improvement and in introducing some innovations. For instance, an early volunteer (Sheetal Shah) introduced client information software to facilitate efficient service delivery for student patients; introduced the balcony approach to clinic service delivery; initiated the purchase of queue machine; and collection of opinions and queries from female students in a confidential way. Furthermore, an ex-volunteer at BDU (Dr Emile Boresma) had created an ICT club for the university community, which run effectively for three years but halted because of her absence. The same volunteer had also introduced a guideline for academic research, which is still used as the main reference for postgraduate students of the University. Similarly, volunteer Dr. Augustine Sasey had introduced Guidance and Counselling services. The volunteers largely devoted their time on putting in place a broad-based staff development system both in the area of teaching and English language. They also introduced (or enhanced) special support for females as a means to help them get empowered. The English language club as a means to help the improvement of the English language capacities of the learners has also been introduced by the volunteers almost in all Institutes. There are very notable volunteers who strengthened the foundation for ELIC centre in HLIs with the mobilization of resources, collecting books, audio materials. Helen in Hossana and Katheryn Craytonshay in DBU may be mentioned as exemplars. The HLIs are supporting the establishment of the English language resource centres and laboratories through providing classes, tables, chairs, dividers, adapters, operating systems and computers. The ELIC coordinators indicated that the establishment of the English language resource centres enabled English teachers to improve their teaching skills. It is integrated into the curriculum of the English Department as in the case of Hossana. Katheryn Craytonshay in DBU is highly recognized for the introduction of her experiences on pre-test and post-test approaches for all her interventions (trainings) where she measures what they had before and the progress brought as a result of the intervention. She strongly advocated on the utilization of time and its management as she values punctuality which has consequently favourably influenced her Ethiopian Colleagues. 23 In the area of HDP, many teachers have been trained. Officials from MoE and HLIs had confirmed that more than 4000 teachers have significantly changed their teaching styles as a result of HDP launched across the country. As it has been reported by the respondents about HDP, some behavioural changes have been observed in the teaching process by using the reflective approach (student-centred method) across faculties, a great shift from the traditional method of teaching to participatory methodology. Moreover, promising orientations are being witnessed among the university staff towards engaging themselves in Action Research, preparing SMART lesson plans, attempting their best to design CPDP and using continuous assessment of learning. The teachers are getting to teach individuals in class rather than the whole class and to differentiate among the students due to the sensitivity training and experiences acquired, mainly, from the volunteers. Volunteer Sheetal Shah (Haramaya) was a good example for introducing this. In her remedial English class for academically underperforming girls, Sheetal could identify students according to their abilities, interests and aptitudes and had private consultations with them. The Gender Office could buy this idea and make use of it in its training programs. There are testimonies from beneficiaries regarding the results (outcomes) of the HDP, wherein the volunteers played important roles. Here are selected testimonies from the teachers as presented verbatim: Frankly speaking, I was not a teacher before I got HDP training. I was a class monitor or just commander-in-chief of the class. After I got HDP training, I become a real teacher (i.e. a professional teacher) and came to recognize that I am responsible and accountable for the academic weakness of every student. It informed me to be a holistic teacher by capacitating myself to assume the role of a facilitator or a moderator. I am not happy with the interruption of HDP as there are teachers who should get the chance of similar training. I am absolutely sure that it can add value to the quality of education in our University. (An Instructor, BDU) Honestly speaking, HDP is a smart intervention to harness quality delivery of education. Through this program, I am able to conceptualize what a real teacher is! (An Instructor, BDU) I was resistant to what they called HDP initially. I used to take part in it only because I was responsible for coordinating other instructors. However, through time, as we debated with the volunteer (Joetta Brown), I came to learn that there are much more I had to learn to become more competent in teaching. I finally started preaching HDP. (Haramaya University) Helen (a volunteer in HCTE), has trained 20 in-service teachers in HDP in active learning, action research, continuous assessment and school placement. From the HDP participants, Helen identified the best performing teachers and further coached 2 in-service teachers to replace her as HDP coordinators and trainers. She made follow up, classroom observations on how teachers practiced interactive learning methods and provided feedbacks and refresher trainings to share experiences and lessons learned. The HDP and ELIC coordinators of HCTE indicated that the demonstration effect of what Helen has been doing was visible in the work of fellow teachers. Besides the HDP program, volunteers in collaboration with LV facilitated 24 several in-service trainings on academic and crosscutting issues. The data presented in Tables 3 and 4 from DBU and HCTE, respectively may illustrate this. Table 3: In-service training by Local Volunteers with the support of AV at DBU (2010/11) Training areas M F T SBCPD , policy preparation and implementation 45 13 58 Measurement and Evaluation in teaching 47 12 59 Student centred teaching methodology 47 12 59 Special needs Education 14 46 60 Total 153 83 236 Fig. 2: CBTEP beneficiaries per training areas in all institutions (2003-2010/11) When it comes to Hossana College of Teacher Education, the beneficiaries of training services rendered by an AV are displayed in Table 4 below. Table 4: Training and workshops organized by volunteers in HCTE (2010/11) Area of training Duration Modalities # of participants Target group Language skills 2 days In-service 160 Primary school teachers Continuous assessment 2 days In-service 100 Linkage school teachers CPD 2 days In-service 150 Linkage school teachers Gender/HIV/AIDS inclusive education 2 days pre- and in-service More than 150 TTC and linkage school teacher Active learning 2 days In-service 150 Linkage school teachers Library 1&half days In-service librarians 40 Linkage schools, high schools and TTC 25 As depicted in Tables 3 and 4, in-service trainings were also organized by the LV with close supervision of the volunteers. This entails, beyond the capacity building and introducing the culture of volunteerism, volunteers are working on the long-term knowledge management, skills transfer and finally the sustainability of the program. As a whole, IFESH volunteers have addressed a total of about 40,000 beneficiaries from 2003 to 2011 (Refer to Annex 5). Recognizing the contributions of AVs to the enhancement of teachers’ competences, a senior official from MoE had asserted that “the structural change that was observed in the area of teaching is significant and this is partly credited to the relentless efforts of IFESH volunteers.” (The Sate Minister for General Education) 4.2.3.Educational resources solicitation and utilization For the purpose of substantiating the classroom approaches of teaching English, volunteers have assisted in strengthening ELIC in each HLI and they are helping the ELIC to be able to get more input supplies like English Language Kits, Cassettes, and aids by contacting different donors. On top of this, ELIC has been fully furnished with computers and audio-video cassettes with donations from IFESH/Ethiopia country office and HLI community is using them as resource centre for the improvement of the language. Volunteers are donating their own personal books, collecting from friends and associations at their home country; even they are mobilizing local resources for the linked primary schools. DBU volunteers in collaboration with IFESH have tried to enrich the ELIC centre. Besides to this, they collected more than 2,900 books and 15 science kits which they have distributed to 15 linked primary schools which is thought to augment the capacity of the schools in supplementary books. This was also practiced in many other colleges and universities. Table 5 shows resource mobilization done by volunteers in HCTE. Table 5: Resources mobilized by volunteers Books/learning materials Quantity Expected number of beneficiaries Receiving institution Books 30,000 Over 10,000 students, teachers and community members Hadiya Education Office, public libraries, schools Audio-visual materials 15 2630 teachers and students ELIC The volunteers not only worried about the scarcity of resources but also the proper utilization of these dearth resources. In DBCTE, there are unused computers, television, cassette recorders, copiers, unused cassettes and headphones, and the like which are simply stored somewhere and wearing out. Thus, the volunteers are continuously facilitating trainings and orientations in connection to using such facilities. A volunteer in Gondar hospital trained nurses about how to care for patients and use laboratory kits wisely. 4.2.4.Community services and cross‐cutting issues Volunteers were observed to be very enthusiastic in supporting the community, especially in their link to nearby primary schools. They are participating in different trainings provided on 26 such essential issues as gender and HIV/AIDS. They organize co-curricular clubs and attend various occasions (such as Women’s day celebration, special conferences and research paper presentations) wherein they provide meaningful inputs. But they are facing barriers as the HLIs don’t yet have mechanisms to involve the volunteers in strategic planning and other related aspects. Volunteers were also engaged in the trainings of the HLI community about the use of Internet, sharing of documents with other departments; facilitating training on essay writing skills for female students, providing materials and technical supports for nearby primary schools as well as providing training on CPD. Thus, it has been reported that participation of teachers have increased. As a result, teachers are facilitating trainings using the resource centres and the Internet services as inputs. Yet, the volunteers reported that the change (as well as the initiative for change) on the part of the local staff is not as they expect it to be. The volunteers (e.g., Helen and Megan in HCTE, Katheryn Craytonshay in DBU) organized trainings and workshops to capacitate linkage with primary schools. They encouraged schools to increase library use rate of students and provided shelves and books for schools which demonstrated results in terms of more participation of students and teachers in using the library resources. The volunteer at Gondar University Community School organized the library in new form, established a book club, and identified students according to their learning level. Presently, she is giving tutorial sessions for those lower and medium achievers. Furthermore, the volunteer is providing special support for 3 (three) Special Needs children. The teachers in her school are learning from her experience. In fact this is a very essential area because in many schools, tutorial programs are provided (also as a requirement by the education offices) but it is only a kind of re-teaching. In most cases it does not focus on the learning difficulty of the learner. Gender Officers at BDU and Gondar University affirmed that they are getting extended support from IFESH volunteers, particularly from Ex-volunteer (Emille) and current volunteer (Marry) on HIV/AIDS, Gender and Equity related experience. As a result of the concerted efforts of the volunteers, a good number of female students are getting increasingly aware of gender-related issues and are becoming more confident about their education. It was reported that volunteers are also providing Tutorial Sessions for female students. Yet, it was commented that the number of volunteers is too limited to provide tutorial sessions to all girls that are registered in the Gender Office. It has also been reported in DBU that the University and the IFESH Volunteers have conducted needs assessment in 20 sample primary schools. Based on the findings of the assessment, they have prioritized the needs, developed proposals, secured funds from IFESH and are supporting them accordingly. Thus, the University has given trainings for 58 teachers selected from 16 schools within 6 districts of the North Shoa Zone. The training focused on continuous assessment techniques, management of large class size, action research, interactive teaching approaches, teaching methodologies, and special needs education. Thus, it has been reported in the post-testing review that 91.8% of the participants have expressed their satisfactions on the trainings offered by the volunteer. As part of their mission, IFESH volunteers have also been engaged in a number of community services around Bahir Dar and Gondar. Teachers at Tana Haik Senior Secondary School and Mekerem 16 General Primary School in Bahir Dar are getting more benefits from volunteers on CPD, especially to develop their professions continuously. This is done via Clusters at department level. The volunteers are supporting as facilitators so that real knowledge transfer is underway. The head master of Meskerem 16 General Primary School reported, “The 27 Volunteer (Mary Augstin) is very good. Most teachers in our school like her. She is helping us by capacitating our teachers. Specially, she (the volunteer) is helping teachers on how to teach English language in a simple and communicative way.” This is also practiced in three junior schools of Bahir Dar (FelegeAbay, EwuketFana, and Meskerem 16) and District Education Officers as well as High School Principals. Thus, participants have given positive feedbacks to the volunteers. Besides, an IFESH volunteer (Emille) has also organized ‘You Go Girls Club’ at Tana Haik Preparatory school in Bahir Dar. This club has demonstrated effectiveness in guiding seemingly hopeless girls to achieve high academic successes. The volunteer recruits 5- 6 orphaned/desperately poor girls attending Grade 11 capacitates them economically, emotionally, psychologically, and academically. She has been doing this project for the past couple of years and had achieved the result of 100% success rate in transition of students from secondary school to tertiary level. As a result, she is considered as a role model for the female students in the locality of Bahir Dar. At Haramaya, the activities of the volunteers is not limited to only those businesses which were mentioned in the project documents: teacher education, English language improvement, community services and cross-cutting issues (such as gender and HIV/AIDS). For instance, there were volunteers in the areas of health (clinic service as well as staff on Faculty of Health) just like volunteers in Gonder, pure science education (in the Faculty of natural sciences); theatrical art; guidance and counselling services and Teaching/advising in graduate classes. Some of these responsibilities, though given with the expressed need of the University’s top management, are not directly related to the four core themes of the project with ultimate goal of contributing to improvement of the quality of basic education in Ethiopia. Nonetheless, the IFESH/Ethiopia staff connect some of these responsibilities to the two major cross-cutting issues: HIV/AIDS and Gender. Besides, the understanding the University leaderships holds about quality education has its own influence on how they utilize the volunteers. For instance, at Haramaya there is a tendency to want to get volunteers in every area where the University management sees a need (a gap). Whereas, at Adama the volunteers are assigned to the core activities the program document already specified. 4.3. Summary of Emerging Impacts of the Program The ultimate purpose of the IFESH program is to bring significant change in capacity of teachers and institutes and thereby improve the quality of Education. The consulting team of this study had observed the glaring realities of some promising and positive changes that can contribute to the quality of Ethiopian Education have begun showing up in all the HLIs. To demonstrate these facts, the team has summarized the prominent observations as follows:  The professional competence of teacher educators or instructors has been improved over time and this is reflected in classroom interactions and communications with their students. This may in turn contribute to the overall improvement of the quality of the education system at national level.  It has been noted that university students and school teachers are becoming accustomed to speaking English openly without feeling shy in front of their colleagues.  Female students’ network is coming to be visible through such opportunities as ‘You Go Girls club’, English Language Club, ELIC membership, Tutorial and English Language Day. Many more members are joining and becoming role models to their successors.  There are some encouraging results to the extent that internal (local) voluntarism has to be initiated and strengthened in order to ensure sustainability. Some local professionals 28 seem to have been excited with such social services and become volunteers themselves in some localities (e.g. Debreberhan) especially in supporting linked primary schools.  There is much inspiration from the work of the volunteers that made the locals reflect on and often question their work culture.  A network of communication which is being strengthened between the CTEs and the nearby primary schools which facilitated experience sharing among teachers in the application of interactive teaching learning techniques, the role of teachers as facilitators in the teaching learning process, CPD, cooperative engagements with their colleagues and mainstreaming of co-curricular activities are valuable to support the learning process.  The fact that ELIC is strengthened and the libraries are better organized helped teachers in many cases to enhance their reading habits; use library resources and engage in personal development. This in turn gives teachers the confidence to use interactive learning methods that improved teacher-student interactions.  The work of the volunteers provides the locals with experiential learning and skills in preparing programs for workshops, seminars, report writing of learning events, development of proposals, follow up observations and feedback, communication and interactions for sharing experiences and lessons.  Co-workers value the lesson they learned from the volunteers with regard to– commitment to one’s purpose, punctuality and seriousness. These are essential traits which if inherited would ensure sustainable change in Ethiopian context.  The volunteers introduced some systems which improved work processes and quality of service delivery to the clients (the university/college students). Among this are the clinic service delivery and technique for assessing girls’ opinion at Haramaya University; the publication of local (in-campus newsletter) newsletter at Adama, and introduction of the ‘extra-formal’ learning opportunities such as clubs and important occasions (e.g. seminar on HIV/AIDS or girl students’ harassment). For instance, due to the activities of the volunteers in these ‘extra-formal’ activities, the local staff (instructors) could see that teaching to cover the academic course outline is not enough in educating professionals and that such ‘extra-formal’ activities are also equally essential particularly to form founded professional attitudes. 4.4. Strengths, Weaknesses, Opportunities and Challenges 4.4.1. Strengths These are the areas in which IFESH/Ethiopia program excels at and needs to capitalize on them so as to make the program more successful in the future. These are expected to help the program managers exploit opportunities, neutralize threats, and avoid weaknesses as much as possible in order to guarantee the future success of the program.  Involvement of highly qualified and dedicated professionals/volunteers  There is peaceful relation and communication, good feeling and awareness, the staff is ready to support, the volunteers are co-operative, housing and offices are good.  Attention to ELIC – an emerging center which every guest is ready to visit and the university is also regarding this as an image building point.  Availability of strong country office and placement of volunteers based on the needs of the host institutes,  Handover noted s to succeeding volunteers to ensure continuity of activities 29 4.4.2. Weaknesses These are areas of improvement that IFESH itself, volunteers, HLIs and other concerned stakeholders of the IFESH Program should take into account for the success of the project. a) IFESH  In Annual volunteers’ workshops, hosting universities are either not equally invited or not taking part even when invited. On-arrival orientation at host institute level is not accustomed. Lack of experience sharing program among hosting institutes and among the volunteers.  IFESH is good in assigning experts. But in some respects, the follow up, monitoring and support mechanism seem not as solid as expected particularly in those institutes where the management is a bit centralized. b) Volunteers  The volunteers plan to do many things in a very short time. For instance, a volunteer may plan to engage in teacher development programs, teach or supervise regular students, carryout some editorial works and more. This over stretches the volunteers and makes them concentrate on doing things (giving services) rather than transferring knowledge with a focus on certain areas. c) HLIs  The Volunteers were working for different fields in HDP, ICT, special needs education and ELIC due to the diversified needs of the institutes and this has made it quite difficult for them to focus on one or a few program (s) for long enough time to see its results  Lack of preparedness and poor documentation of the volunteers’ activities at host institutions– a problem of knowledge management.  Gap was observed between expectations of the volunteers and the responsiveness of the host institutions – which resulted in underutilization of the volunteers in a few cases.  Facilities and other resources are not readily at the disposal of the volunteers to carry out their planned activities. As a result there are times they fail to implement their plan.  Time wastage due to lack of responsiveness of the host institution in deployment of the volunteers: e.g. host institutions fail to prepare their annual academic calendar prior to the volunteers’ arrival.  Failure to communicate the job description and profiles of the volunteers to units where the volunteers would be assigned. Absence of well-structured knowledge management systems that are put in place in host institutions, especially to exploit the potentials volunteers have brought to Ethiopia. 4.4.3. Opportunities  English currently is being enforced as the medium of instruction in Ethiopian school system/HLIs  High demand for volunteers’ service due to rapid expansion of educational institutions in Ethiopia  Government’s emphasis on quality improvement (considering such strategies as GEQIP, ESDP IV, GTP) 30  The volunteers can serve as a link between their host institutes and Universities/Colleges in the US.  Introduction of massive reforms in Education system. Nowadays higher institutions in Ethiopia are in the era of change (institutional dynamism). They have done and are still doing Business Process Reengineering – that means the old ways of doing things is being questioned and a new experience is sought.  Excellent hospitality of community members/members of society  Willingness of work colleagues to work with volunteers.  Availability of resource persons in English Department who championed the usage of ELIC services  The existence of VSO and peace corps may help to get a different and yet external perspective.  Supportive institutional climate in such institutes as Haramaya.  Ministry of Education in earnest need of such supports provided to younger staff (academics)  Young staff eager to learn from the volunteers  A desire for change and an ongoing change process at the Universities which increases the need for inputs from experienced volunteers. 4.4.4. Challenges There are several challenges which the program implementation process encountered. Here are the most challenging ones:  Gap between MoE’s need and the capacity of IFESH (mainly in terms of budget)  Bureaucratic management skills at the end of host institutions,  Language barriers/ gaps ,  Volunteers’ lack of clear-cut schedules to started their work ones they arrive at host institution. Cultural variations and lack of proper orientation and background about the local cultures, language, lack of planning and organized protocols, frequent unplanned meetings , lack of understanding about time value among majority of people – both teacher trainees and teacher educators, wrong expectation and perceptions from Ethiopian counterparts to do their jobs and/or ease their workloads  Mismatch between budget allocation period of IFESH (Dec-January) and the academic calenderer of the university (Sept- June)  Most experienced staff members who are supposed to share experience from the volunteers and takeover the mission upon their evacuation are less interested to benefit from such learning and assume such responsibility.  There is a tendency from the co-workers to leave everything, or at least whatever they do, to the volunteers instead of working together and learning in the process.  The Universities have multiple problems which need the sort of expertise that can be gained from the volunteers. This makes the universities to push for whatever they give priority to instead of considering the basic tenets of the project. As a result, much of the volunteers’ time is spend on activities outside or only marginally related to the stipulated objectives of the program. 31 4.5. Best Practices and Lessons Learnt  ELIC centre has been strengthened and giving active services.  ELIC members are becoming able to communicate in English with volunteers and they are exercising Free talk, debate (weekly), reading e-books, participating in “You Go Girls Club and English day”. There are also activities that encourage internet access to all students, book lending and reading for fun  Training on essay writings for female university students and supporting primary school teachers  Trying to make females active in HLI  The HDP has been strengthened and the fact that the volunteers value and work on it has helped overcome some of the resistance to active learning from the part of experienced Ethiopian teachers. IFESH is well managed but needs better support to host institutions and contacts with units who directly work with volunteers. 4.6. Sustainability Programs are sustainable if they are planned and implemented with the community. Thus, IFESH Volunteers are working with the HI communities and they are capacitating and transferring the expertise they have. In line with this, volunteers are encouraged to be working directly with the local staff to ensure skills and knowledge transfer as way of building in the sustainability element into the program. However, whether the result of a volunteer’s activities is sustainable or not generally depends on several factors including:  The nature of the volunteers’ activities: if the volunteers focus on capacity building and helping local staff to use the experiences transferred, and then it is more likely that the sustainability objective will be realized.  The attitude of the local staff: where effort has been made to increase the receptivity of the local staff (e.g. value attached to language club as a means to improve English language competence) then the result will most likely be sustainable.  Local capacity: where there is capacity, there is a possibility to continue with some of the changes introduced by the project.  Institutional support: when the institution supports and internalizes the rationale of the project, then there will be a high probability that the benefits will continue to deliver long after the project closes. Based on these and other considerations, it can be said that some of the results of the volunteers’ activities would be sustainable while other may be less sustainable. The teaching￾learning process has been already undergoing change and there is no reason for teachers to revert back because institutionally that is the desired state of teaching and learning. HDP will continue to be functioning because that is desired both by the instructors and the Universities/the MoE. A look at these instances reveal that where there is capacity, institutional support and the local staff is committed, nothing can possibly hinder the sustainability of some of the project results. . 32 The challenges faced constrain sustainability of outcomes. There are trends of changes both in the attitudes and teaching skills of teachers but continuity might be limited to the units which worked directly with the volunteers. There are no efforts in the institutions visited to create learning and sharing systems at the institutional level. There are also no incentives to institutionalize the changes which put the issue of sustainability in eminent jeopardy. The HLIs need to ensure that ELIP continues even in the absence of volunteers as English language learning is one of the focuses of the government in improving English Language. The ELIC is already established and it is managed by the students and volunteers have already given out the experiences. But the withdrawal should be smooth and gradual. Schools are sharing experiences; facilitate TOT, and ELIC members are serving the HLI community by making e-books available. Dr Ousman, a volunteer in DBCTE had to say the following regarding the sustainability of the results of the program, “There is a bright possibility of sustaining the program - we are building the capacity, they have the resources, they can keep on and they are giving their ears and eyes, giving attentions”. If this is realized in all host institutions, there is no problem of sustainability of the results of the program. 5. Conclusion In this performance evaluation, an attempt has been made to assess the performance of the USAID/Ethiopia-funded CBTEP program implemented by IFESH and thereby provide USAID/Ethiopia and the Ethiopian MoE useful information and data for determining the future direction in enhancing the quality of education in Ethiopia in general and teacher education programs in particular. In doing so, analysis of the project input, output, outcome, and sustainability were the main focuses of the evaluation. A participatory approach has been used in collecting both the primary and secondary information. Data have been analysed on the basis of robust qualitative and quantitative analytical techniques. From analysis of the data and the results documented it can be concluded that irrespective of the several weaknesses and challenges the program is accomplishing its objectives– contributing to the improvement of quality of education in Ethiopia. The volunteers have been making considerable contributions in enhancing the capacities of instructors and school teachers in effective teaching and in English language improvement. Considerable number of students, administrative staffs, and education officers also got some support in English language skills improvement. The volunteers had also participated in system improvements and community support programs. Based on the analysis of the findings of the study, the Consulting team had suggested that the program should continue with more focus on capacity building in the area of English language, subject area pedagogy and cross-cutting issues. 6. Recommendations In light of the major findings and observations of the Performance Evaluation of IFESH Ethiopia volunteers in HLIs, it seems imperative to forward the following recommendations for the key stakeholders 6.1. USAID 6.1.1 While what has been achieved so far is encouraging and invaluable; it is far below satisfying the needs of the Ministry of Education. This is mainly due to 33 budget constraint. Therefore, let USAID consider this for the ensuing phase of the program. Irrespective of our limited knowledge of the internal policy of USAID, the consulting team feels that it would be much more beneficial if USAID considers providing technical support besides funding the program. 6.1.2 Volunteers would be much more encouraged and motivated if there is possibility for them to get a limited fund for solving some problems that emerge out of the work environment of the volunteers through action research or any other relevant professional exercise. Therefore, it is commendable that USAID consider additional direct funding (outside the program budget) for such mini￾projects. 6.2 IFESH 6.2.1 The program document should be strictly attended to while implementing the program. For that there is a need to aware the host institutions and if at all there is any change to the original document due to the specific situation of the host institutes, then the document needs to be amended. 6.2.2 The need is still there, particularly in the area of “Subject area pedagogy” and in “English language skills enhancement”. Therefore, these have to be the major focus areas in the next program phase. 6.2.3 The placement duration of the volunteers is too short to demonstrate results. It takes time for the volunteers to establish, understand the environment and integrate themselves with the community. In some cases the volunteers leave before they complete their planned activities. Therefore, IFESH/USAID need to consider the initial duration (may be increasing to at least two years) with the possibility for extension. 6.2.4 The volunteers need to be adequately oriented that they are here to build capacity rather than doing things for (in place of) the local staffs and giving services. They have to take the role of a coach and a mentor in their areas of expertise. 6.2.5 There is a need to consider discussing with VSO and Peace Corpus to reduce as much as possible duplication of efforts but rather create synergy and ensure sustainability. This needs to start from the country offices. The MoE and/or other stakeholders (funders & implementers) can discuss on this issue together (through common platform created by MoE) and come up with a comprehensive plan of action to deploy the volunteers efficiently and effectively. 6.2.6 There must be clear memorandum of understanding between IFESH and hosting institutions regarding the roles of each party in volunteers’ management. 6.2.7 Volunteerism has to be supported to continue in Ethiopia and IFESH/USAID need to consider mechanisms to encourage local volunteerism (whereby the local volunteers work with IFESH volunteers out of their site of work). In addition to volunteerism, IFESH also could try exchange program between Ethiopian HLIs and HLIs in USA. 6.3 MOE/Host Institutes 6.3.1 Assignment of volunteers should strictly be based on their area of maximum expertise with a focus also on thematic areas of the program. 6.3.2 On-arrival of the volunteers, university level orientation has to be carried out in the presence of all relevant staff members. The roles and responsibilities as 34 well as the working relations which should define the volunteers’ presence should be clearly established with the host unit of the host institute. 6.3.3 There has to be stage (forum) whereby the activities of the project and lessons learned can be discussed by the direct beneficiaries in the presence of the university management. 6.3.4 There is a huge need for host institutions to mainstream gender and HIV/AIDS properly to address diversity issue. A good example is the sexual harassment policy of St. Mary University. Therefore, the volunteers need to concentrate on building the capacity of the Universities and Colleges so that they ‘walk’ the ‘talk’. 6.3.5 Given the situation of gender mainstreaming reported from Hossana CTE (see Annex IX); it is commendable that there is a need to make sure the commitment of leadership and higher management to be able to institutionalize the gender mainstreaming activities; having baseline (e.g. gender audit for instance on political will) prior to commencement of such mainstreaming initiatives; and identify good practices and replicate (scale up) them in all linkage schools as learning models once the initiative implemented. 35 Annex I: People Contacted/Interviewed N O Name Position Tool administered Address 1 Tomas Tesfaw (3rd yr) Students /English /ELIC FGD DBU 2 Melese Degu (3rd yr) Students /English /ELIC FGD DBU 3 Alazar Kassaye (2nd yr) Students Computer science (ELIC) FGD DBU 4 Hamerenoh Zeleke (2nd yr) Students Computer science Dept (ELIC) FGD DBU 5 Dawit Kassahun(2nd yr) student Engineering /ELIC FGD DBU 6 Nesredin Yohanes (3rd yr) Students, Geography Dept /ELIC) FGD 7 Naod Tilahun (2nd yr) Student Engineering Dept /ELIC FGD DBU 8 Yared Dinku (2nd yr) Students IT Dept FGD DBU 9 Michael Emishaw (3rd yr) student, English Dept FGD DBU 10 Eyob Workineh (3rd ) student English Dept FGD DBU 11 Dejene Mulugeta Lecturer , English Interview DBU 12 Ato Azemeraw Ayehu Lecturer and Community Service directorate Interview DBU 13 Katheryn Caytonshay Volunteer (ELIP) Interview DBU 14 Dr Ellaline Roy Volunteer( Senior Advisor for Vice president for research and community services Interview DBU 14 Lemma Demessie Lecturer and Vice director for institute of Education Interview DBU 15 Dr Ousmane BA Volunteer /ELIP Interview DB CTE 16 Mamuye G/Hiwot Dean Interview DB CTE 17 Hiwot Beyene Instructor Interview KCTE 18 Zeleke Beyoro V/Dean Interview KCTE 19 Dr FirewTegenge, The Academic Vice President, Interview BDU 20 Dr AbiyYigzaw, Postgraduate research and community service coordinator Interview BDU 21 Mr MengistuMulu Community School Vice Principal Interview GU 22 Mr Dawit Amogne Coordinator of ELIC Interview BDU 23 Mr Tilahun---, Counterpart for ELIP Interview BDU 24 Ms Martha --, Gender Unit Officer Interview BDU 25 Mr TinsaeBerihun, Gender Office, Head; Interview GU 26 Mr Sirur --, Vice Principal, Interview Wogera High School, North Gondar Zonal Administration 27 Mr ZewgeTsegaye Principal, Interview Meskerem full cycle Primary School, Bahir Dar City 28 Emily Boersma, Ex-volunteer, Interview BDU 29 Mary Lugton, volunteer Interview BDU 30 Liz Rath, volunteer Interview GCTE 31 Shamekia Chandler, volunteer Interview GUcommunity school 32 Joan Denne Ex- volunteer Email 33 and Donald Thompson Ex- volunteer Email 34 Ericka Scott Ex- volunteer Email Adnew Ontoro Dean, Interview HCTE Tesfaye Academic Dean, Interview HCTE Aster Assistant Dean, Interview HCTE Elfenh, ELIC Coordinator , Interview HCTE Sekata English Drama, club Interview HCTE 36 coordinator Abera Guremo English teacher Interview HCTE Abera HDP Coordinator Interview HCTE Adem School linkage coordinator Interview HCTE Zeregaw English teacher Interview HCTE Nuria Gender focal person Interview HCTE Hanna HIV/AIDS officer Interview HCTE Solomon , Assistant director Interview Woldhana Primary school Dutamo,Wolde director , Director Interview Woldhana Primary school Redi English Teacher, Interview HCTE Ali English Teacher, Interview HCTE Alemayehu Teacher Interview linkage school Dinknesh Teacher, Interview HCTE Umer Teacher Interview linkage school Yakob Teacher Interview linkage school Sashank Lopez, IFESH Volunteer Interview HCTE Simon, VSO Volunter Interview HCTE Solomon, English language Teaching Improvement Program(ELTIP), Interview MOE Tewodros Chewhraga Director, Teacher and Educational leaders Development Directorate Interview MoE Mike IFESH volunteer Interview MOE Tedela Academic vice President, Interview St. Mary University Dr. Ayele Abebe Former AVP Adama Interview Ato Bogale Tesemma Former ELIC Head Adama Interview Ato Ahimed Siraj Former ELIC Head Adama Interview Ato TAye Geresu Head, ELIC Adama Interview Ato Eshetu Tekle Former HDP Leader Adama Interview Ato Gemechu Wirtu Former HDP Leader Adama Interview Ato Tolera Negasa Former Vice-Dean of Faculty OF Edcuation Adama Interview W/ Tezazew D/Head, Gender Offcie Adama Interview Dr. Aseffa Abahumna Admin Vice-President Adama Interview A group of 3 instructors who attended HDP in the former Volunteers’ classes Young instructors Adama FGD Prof. Belay Kassa, HU President Haramaya Interview Ato Dawit Negassa Dean, Faculty of Education and Behavioral Sciences Haramaya Interview Dr. Tesfahun Former Dean, Faculty of Education Haramaya Interview 37 Ato Abdella Coordinator, HDP Haramaya Interview Ato Tassew Mezgebu D/Coordinator,. HDP Haramaya Interview Ato Sirak Former Coordinator, HDP Haramaya Interview Ato Adnew Head, Department of English Language Haramaya Interview --- Former Head of ELIC Haramaya Interview, telephone --- University Chief Librarian Haramaya Interview W/ Emebet Head, Gender Office Haramaya Interview W/ Kidisina D/Head of Gender Office Haramaya Interview Ato Fayera Director, Academic Programs Haramaya Interview Mrs. Jane VSO Volunteer (HDP Advisor) Haramaya Interview Dr. Augustine Sasey IFESH Volunteer Haramaya Interview A group of six instructors Three attending HDP and another 3 attending PGTHE Haramaya FGD A group of 18 current volunteers On mid-term review From all over the country FGD 38 Annex II: Key Informant Interview Guide (Questions selected from this have also served to facilitate the FGD) Evaluation of the USAID/Ethiopia-funded International Foundation for Education and Self-Help (IFESH) Capacity Building in Teacher Education Program Leading Questions for Key Informant Interview (KII) IFESH has been supporting the education sector in Ethiopia by deploying highly qualified professionals and experienced American volunteer teachers to various teacher education and other higher education institutions with funds from USAID/Ethiopia. The program had started in 2002/3 and since then IFESH had deployed about 82 volunteers to various HLIs in Ethiopia. The purpose of this instrument is to gather information regarding the achievements and challenges of the IFESH/CBTEP and draw some lessons for future improvement of the program’s implementation scheme. The items incorporated into the instrument have been designed in a way they solicit interactive discussions among participants and cause generation of clear ideas that serve as the bases for identifying key areas of intervention in the future. Therefore, the participants are kindly requested to give their honest assessment of the performance of USAID/IFESH/CBTEP in the past 10 years and forward appropriate measures to be taken by relevant bodies in order to improve the future implementation of the program. (Instruction or Guide to FGD Facilitators/Interviewers: Any facilitator has to convey warm greetings whenever he/she meets the participants. He/she should clearly and concisely state the objectives of the research and explain the rights of the participants to give or not to give the information in relation to the subject under consideration. He/she must explain that the participation in this data collection process is purely on voluntary basis and all information is used for the purpose of the study only. No individual’s privacy is going to be affected as a result of the information provided to the facilitator and/or the implementing firm. All facilitators must be guided by the philosophy of “Appreciative Inquiry” while moderating the discussion and capturing relevant information.) I) Program relevance and national support (posed to MoE officials, University Presidents, TEC deans, IFESH, USAID) a. How was the program initiated? To what extent did the key stakeholders participate during the initiation & launching of the program? b. To what extent is the program aligned to the education goals and strategies of the country? How relevant is the program? (Establish concrete relations) c. How relevant is the program to enhance the capacities of education actors and contribute towards improvement of quality of education in Ethiopia? d. To what extent do policy-makers and strategists admire/appreciate the value of the program? (Give concrete examples) e. How supportive are the policy makers and other key stakeholders towards the success of the program?(Give concrete examples) f. Is the project up to its promises in terms of accomplishing what it envisioned to do? How effective was the project? II) Deployment and contributions of the Volunteers (posed to University Presidents, TEC deans, relevant department heads) a. When did you get volunteers for the first time in your institution? How many of them were deployed in this institution so far? What was your role in recruitment of the volunteers? b. How competent were the volunteers? c. What contributions have they made so far? (Please specify significant contributions the made)with regard to the following interventions as appropriate: i. English language improvement _______________ ii. Teaching and systems improvements __________ iii. Educational resources supplied and utilization ___ iv. Community services and cross-cutting issues _____ v. Others (please specify) _______________________ d. More specifically, how do you describe the behaviors displayed or contributions made by volunteers in terms of (get concrete examples or numbers): 39 i. Efforts they make to assimilate with institution’s community easily and their willingness to support the institution and work colleagues. ii. Activities organized by volunteers that benefited teachers and students in colleges/universities and students including HIV/AIDS education, gender and other co-curricular activities. iii. Activities organized and led by volunteers that benefited teachers and students in neighboring schools. iv. Manner in which the volunteers technically supported the institutions in development of strategies, guidelines, plans, etc. v. Trainings and technical assistance provided in the HDP of teacher development program or in English language departments in the colleges/universities. No Area of training Duration modalities (pre service, in service, ) No. Participants Target group 1 2 vi. Level of effectiveness of the volunteers in working with their counterparts and transfer of knowledge and skills to counterparts. vii. Resources (such as policy drafting, manuals, program design, modules, researches, education material, etc.) produced with the technical assistance of volunteers and continue to be in-use. viii. Innovations introduced by volunteers that continue to be in-use. ix. Type of support institution management received from volunteers. x. Volunteers’ activities level of response to the perceived needs of its beneficiaries. xi. How many books or materials have been donated by the volunteers No Books/learning materials # Expected No. of beneficiaries Receiving institution Remark 1 2 3 xii. What changes have been observed in the resource centre or library as a result of the materials donation? e.What is the volunteers’ activities relationship to the host government and colleges/universities plan and strategy? f.How sustainable are volunteer activities? III) Institutional and Community Support (Relevant department heads, teachers/work colleagues, IFESH) a) How were the volunteers received by hosting institutions? How interactive, cooperative, willing to share knowledge, and productive were the volunteers in their duty stations? b) What institutional factors promote/constrain the maximum use of the volunteers? What challenges did the volunteers face during their assignment? How were they rectified? c) How organized and systematic are hosting institutions in efficient and effective use of the volunteers (i.e. maximized use of the volunteers’ time during their tenure in colleges/universities)? d) What is the volunteers’ activities relationship to the host government and colleges/universities plan and strategy? e) How sustainable are volunteer activities? f) How are the co-curricular activities integrated in the institute teaching learning process?(in terms of English, teaching, club, community service and system) g) What is the level of satisfaction of the staff on the work of volunteers? (Beneficiaries) Key:VH-very high, H-High, M- medium, L- low, VL- very low, NA-not applicable Considering the assignments , trainings and facilitation of volunteers, how are you satisfied with the following milestones/areas: VH H M L VL NA English language enhancement General teaching learning process improvement Co-curricular activities engagement Educational resource solicitation and distribution 40 System introduction and improvement Readiness to share and receive Knowledge Participation on community support Others (please specify and rate) IV) Emerging results of the program (Presidents, deans, department heads, teachers) 1. What Significant changes are observed in 1.1. English Department, 1.2. Situation of colleges/universities , 1.3. Capacity of teachers and institutes , Intended or unintended results that is attributable to the interventions of the IFESH as perceived by stakeholders. 2. Is there any improvement in the competitiveness of teachers in English Language use? How? 3. Are there any changes that are considered in ELIP for Ethiopian Colleges and Universities? 4. How does volunteering add to skills and learning for the staff of institutes, primary school teachers and volunteers themselves and others? 5. What relationships, networks, bonds of trust between people and institutes are developed through volunteers’ involvement? 6. How does volunteering strengthen shared sense of culture including language referring to any of the aspects in the list below, what are the effects of volunteers on the institutes. 6.1. On the college/university as a whole 6.2. On the services that the College or University provides 6.3. On the staff 6.4. On the local community and/or wider community V) SWOT (Strengths, weaknesses, opportunities and challenges) Analysis(presidents, deans, department heads, Teachers, IFESH) 1. What (internal and external) factors do you think have contributed to the successful performance of volunteers? 2. What (internal) factors had constrained the institution from optimum use of the volunteer(s)? 3. What challenges did the volunteers face during their assignment? How were they rectified? VI) Sustainability (Presidents, Deans) 1. How sustainable are volunteers’ activities? 2. How can you sustain the outcomes of the intervention in the absence of volunteers ( systems, infrastructures, practices, resources( human, materials , financial ) 3. Are you ready to concentrate on transmitting knowledge and skills to the community, what is known as ‘Skill transfer’ or ‘knowledge management’? 1. The Way forward 1. How relevant and desirable do you think is the program? 2. What possible gray areas should be improved if the program persists? 3. What other thoughts or ways of doing the program efficiently and effectively? 4. Which area should it focus if it continues? 5. Any further suggestions or recommendations?_______________________________________________ 2.2. Leading Interview Questions for Volunteers 2. Can you state your background in brief? 3. How did you become a volunteer? 4. What is your impression about the overall management of the program (You may focus on the roles played by IFESH, USAID, and host institutions)? 5. What is your impression about the country, education sector, and work colleagues? (kindly describe all +ve& -ve aspects of each) 41 6. What are the specific contributions you made to the hosting institutions (IFESH, USAID, and institution of assignment)? (Quantify them if possible—give specific numbers here) 7. What progresses have you observed in relation to your mission? 8. What do you think of the best practices and lessons learnt during your volunteerism? 9. What are the specific challenges you faced and opportunities you exploited (or yet to be exploited) during your volunteerism? 10. How do you see the programs’ relevance, desirability, continuity and sustainability in Ethiopia? 11. Any further comments:_______________________ 42 Annex III: List of Major Documents Reviewed 1. International Foundation for Education & Self-Help IFESH/Ethiopia Reports ( biannual and special reports) 2. IFESH field monitoring reports 3. The Bridge , quarterly publication of IFESH/Ethiopia 4. Volunteers’ reports 5. Assessment of the Impact of the Capacity Building in Teacher Education Project (CBTEP): Assessment of program accomplishments and Impacts September, 2007 6. Capacity Building in Teacher Education Project Phase I: (CBTEP original), A Project to Provide Technical Assistance To Improve Teacher Training and Primary Education in Ethiopia, Original doc 7. Capacity Building in Teacher Education Phase II: (CBTEP Extension), Program Description to Extend the Capacity Building in Teacher Education Project in Ethiopia from June, 2008, through July 2012x 43 Annex IV: List of Volunteers List of IFESH Volunteers under CBTEP 2003/2004 - 2011/2012 A/Y yea r No Name Sex Qualification Assignment Site Responsibility 2003/2004 A/Y 1. 2. Anna Janne Hammernick F BA Adwa TTI Teacher Trainer English/Education 3. 4. Selamawit Sertsu F MA Dessie TTC Teacher Trainer English/Education 5. 6. Susan Ruth Yarger F MA Harar TTI HDL 7. 8. Wendie Ann Willis F MA Debre Berhan TTC HDL 9. 10. Dianne Cary F Ph.D Awassa TTC HDP 11. 12. Carolyn casale F MA Abi Adi TTC Teacher Trainer/ Deputy HDL 13. 14. Aundrey J. Somerville F MA Gondar TTC Teacher Trainer/ Deputy HDL 15. 16. Hellen Ann boxwill F MA Hosaina TTC Teacher Trainers/HDL 2004/2005 A/Y 17. 18. Rose C. Amazan F MA Hossaina TTI HDL 19. 20. Joan Denne F Med Awassa TTC HDL Returnee 21. 22. Donald Thompson M MA Awassa TTC HDL Returnee 23. 24. Carolyn Casale F MA Mekele TTC HDL Returnee 25. 26. Brenda Kaye Carter F BA Adwa TTI Teacher Trainer English/Education 27. 28. Susan Ruth Yarger F MA Debre Berhan TTI HDL Returnee 29. 30. Whitlow, Emery (Trey) M MA Educational Media Agency, MoE Technical Assistant Returnee 31. 32. Rebecca Eve Martin F MA Nazareth TTEC HDL 2005/2006 A/Y 33. 34. Rose C. Amazon F MA Hossaina TTC HDL Returnee 35. 36. Carolyn Casale F MA Mekele TTC Teacher Trainer English/Education 37. 38. Emily Boersma F MA Bahir Dar University HDL 39. 40. Allie Bak F MA Debub University HDL 41. 42. Teydin Erkkinen F MA Awassa TTC HDL 43. 44. Carlos Fernandez M MA Debre Berhan TTC Teacher Trainer/ICT 45. 46. Nicholas Dimma M Ph.D St. Mary’s College HDL 47. 48. Joetta Brown F MA Alemaya University HDL 2006/2007 A/Y 49. 50. Allie Bak F MA Adama University HDP Moderator 51. 52. Carlos Fernandez M MA D/Berhan TEVTC ICT Instructor 53. 54. Carlos Lewis F MA Mekele TTC HDP Moderator 55. 56. Emily Boersma F MA B/Dar University HDP Coordinator 57. 58. Nelicia Murrell F MA Hossana TTC Teacher Educator, Eng. Education 59. 60. Nicholas Dima M PhD St. Mary’s College HDP Leader 61. 62. Elizabeth Braden F MA Assela TTC ICT Instructor 63. 64. Peter Farber M MA Haramaya University HDP Leader 2007/2008 A/Y 65. 66. Donald Thompson M MA D/Berhan University Science/ Education 67. 68. Edona Alexandria F PhD Aksum University Human Dev’t & Family studies 69. 70. Emily Boersma F MA B/Dar University Education/ English 71. 72. Janice Heckler F MA MoE Education /ICT 73. 74. Joan Denne F MA D/Berhan University Education 75. 76. Katherine Simpkins F PhD MoE Special Education 77. 78. Kim Deprenger F MA Adama University Education & PTE 79. 80. Sapna Thakker F MA D/Berhan University Special Education 44 No Name Sex Qualification Assignment Site Responsibility 2008/2009 A/Y 81. Deborah Zinn F PhD HESC Technical Advisor 82. Katherine Simpkins F PhD MoE School Improvement Specialist 83. Melissa Gallagher F MA Hossana TTC Teacher Trainer 84. Tafaya Ransom F MA Deredawa University Teaching & Curriculum Development 85. Emily Springer F MA Deredawa University Technical Advisor 86. Robert Walker M PhD D/Berhan University HDP Leader 87. Laura Johnson F MA Adama University Teaching Guidance & Counselling for Post Graduate Studies 88. Adisa V. Beatty F MA Adama University Technical Assistant 89. Edona Alexandria F PhD Aksum University Technical Assistant to the President 90. Lucinda Acquaye F MA Bahirdar University Program Coordinator 91. Sheetal Shah F MA Haramaya University Health Education 92. Paul Roberst M MA D/Berhan University ICT Trainer 2009/2010 A/Y 93. Carolyn Smith F PhD Bahirdar University Education Faculty; Teachers Training & Counselling 94. Charlene Taylor F MA Assela School of Health Health & HIV/AIDS Teacher Trainer 95. Ericka Schott F MA Gondar University ELIP, Teacher Trainer 96. Jeremiah Walker M BA Debre Berhan CTE ICT Trainer 97. Katheryn Crayton-Shay F MA Debre Berhan University HDP Leader (came in the 1st quarter) 98. Kristin Juelson F MA Gondor community School & Sociology Dept In-service Teacher training (English) & Sociology Instructor 99. Laura Johnson F MA Adama University Teacher Education in Counseling 100 Patrick Hahn M PhD MoE then switched to Haramaya University Science Educator Ray Mobley M MA Debre Berhan University HDP Leader (Declined in the first week of his arrival and replaced by Katheryn Crayton-Shay) 101 Sandra Ross F PhD Axum University Teacher Trainer (declined end of the first quarter) 102 Sheetal Shah F MA Haramaya University Health & HIV/AIDS Educator 2010/2011 A/Y 103 Abebech Gutema Mura F MA Debre Berhan CTE Teacher Trainer /ELIP 104 Comesha Griffin F MA Finote Selam CTE Teacher Trainer (HDP Leader) 105 Katherine Simpkins F Ph.D. Kotebe CTE Teacher Trainer (Special Needs Edu.) 106 Katheryn Crayton-Shay F MA Debre Berhan University Teacher Trainer /ELIP 107 Megan Flowers F MA Hossana CTE Teacher Trainer /ELIP 2011/2012 A/Y 108 Bahereh T. Smith F MA Jimma University Teacher Trainer/Education Faculty 109 Carol Oconnor F Ph.D Hawassa CTE English Teacher Trainer 110 Derek Jefferson M MA Haramaya University English Teacher Trainer & Theatrical Art 111 Elizabeth Jean Rath F BA Gondar CTE English Teacher Trainer 112 Elizabeth Leu F Ph.D MoE Technical Advisor Ellein Carol McDaniel (returned home on Oct. 13 2011 due to health problem) F Ph.D Dire Dawa University HDP Leader 45 No Name Sex Qualification Assignment Site Responsibility 113 Kelly Anne Bodkin F MA Sebeta Special Education CTE Special Education Teachers Trainer 114 Kenyatta Albeny F MA (Ph.D candidate) Finoteselam CTE English Teacher Trainer 115 Mariama Ross F Ph.D Aksum University Teacher Trainer/Technical Advisor for A/V/President 116 Mary Lugton F M.Ed Bahir Dar University English Teacher Trainer 117 Michael Mark Martucci M MA MoE ELTIP Advisor 118 Ousmane Ba M Ph.D Debre Berhan CTE English Teacher Trainer 119 Sashenka Lopez F MSc Hossana CTE Library Science Specialist/English Reading Skill Dev’t Specialist 120 Shamekia Chandler F BA Gondar University Community School English Teacher Trainer/Library Science Specialist 121 Augustine Sesay M Ed.D Haramaya University HDP Leader 122 Katheryn Crayton-Shay F M.Ed Debre Berhan University Teacher Trainer (CPD & ELTIP) NO. OF VOLUNTEERS ASSIGNED AS OF 2011/2012 82 CBTEP TOTAL OBLIGATION 82 Total Volunteers at the End of the Project (July 31, 2012) will be= 82 46 Annex V: IFESH/ETHIOPIA Volunteers Present at IFESH Mid-Year Conference for Focus group discussion at Bishale Hotel, Addis Ababa, December 2011 No Name Sex Qualification Assignment Site Responsibility 1 Augustine Sesay M Ed.D,   Haramaya Unviersity HDP & PGHEDT   2 Bahereh Smith F MA,   Jimma University ELIC program; CPD and TA; 3 Carol Oconnor F PhD. M.Ed., TESOL;   Hawassa CTE ELP and Teacher trainer r   4 Elizabeth Rath F BA, TESL).    Gondar CTE ELP and CPD   5 Derek Jefferson M MA,   Haramaya Unviersity English & Art 6 Yousif Ahmed M BSc., Diploma Dire dawa university   Engineering   7 Katheryn Crayton‐ Shay F M. Ed.,  MA in English.   Debre Berhan University Train Teachers CPD, ELP   7 Kelly Bodkin F MA, Education;   Sebeta SN CTE SNCTE and  ELIC   7 Kenyatta Albeny F PhD (candidate), Comparative   Finoteselam CTE ELIP   10 Mariama Ross F Ph.D,   Aksum university TA to A/V/President office; HDP leader; and ELIC.   11 Mary Lugton F M.Ed,   Bahir Dar University ELIC and CPD.   12 Michael Mark Martucci M MA,   Ministry of Education TA in ELIP:   13 Ousmane Ba M Ph.D   Debre Berhan CTE ELIP   14 Sashenka Lopez F MSc,   Hossan CTE ELIP and Reading   15 Shamekia Chandler F BA,   Gondar University Community School English and CPD 16 Douglas Smith M BSc ; Post graduat Diploma Jimma university TA (technical assistant) 17   Berihun Mekonnen Mebrate   M PH.D GSSW/AAU Lecturer   18 Ellaline Macaualy   F PhD Debre Berhan university Advisor   47 Annex VI: IFESH-Ethiopia CBTEP Fund/Grant from July 2002 through July 2012 Start date End date No of volunteers Budget allocated in US$ Remark target actua l 1st July 27, 2002 July 31, 2007 32 32 $1270605 2nd Aug. 1, 2007 July 31, 2008 10 8 $300000 Assigned less by 2 3rd June 2008 Sep22/2008 July 31, 2012 40 12 10 5 15 $1200000 Plus $858000 adjustment Totalling $2050000 2 more assigned to compensate with 2007/2008 Total (from 2002 through 2012) 82 82 $3628605 Annex VII Table 6: CBTEP beneficiaries per year1 Years Pre-Service Teachers Trained In-Service Teachers Trained Administrativ e staff Policy/Gov’t Individuals Trained NGO/CBO Representativ es Other Beneficiaries Total M F T M F T M F T M F T M F T M F T M F T 2003/04 1809 1235 3044 476 132 608 32 26 58 0 0 0 39 41 80 173 141 314 2529 1575 4104 2004/05 665 1297 1962 265 80 345 25 24 49 0 0 0 22 7 29 20 30 50 997 1438 2435 2005/06 4561 3517 8078 484 71 555 13 38 51 0 0 0 1 3 4 880 467 1347 5939 4096 10035 2006/07 2269 2797 5066 1078 300 1378 94 46 140 0 0 0 3 10 13 241 218 459 3685 3371 7056 2007/08 1263 1588 2851 686 269 955 79 5 84 0 0 0 18 17 35 139 57 196 2185 1936 4121 2008/09 482 287 769 270 101 371 0 0 0 0 0 0 63 266 329 584 1217 1801 1399 1871 3270 2009/10 776 514 1290 257 81 338 0 0 0 28 46 74 33 30 63 535 645 1180 1629 1316 2945 2010/11 318 909 1227 582 236 818 0 0 0 18 24 42 2 1 3 64 39 103 984 1209 2193 Total 12143 12144 24287 4098 1270 5368 243 139 382 46 70 116 181 375 556 2636 2814 5450 19347 16812 36159 1 IFESH CBTEP Beneficiaries Statistic since 2003 33 Annex VIII: Statement of Work for Evaluation of the International Foundation for Education and Self-Help (IFESH) Capacity Building in Teacher Education Project (CBTEP) in Ethiopia 1. Title The evaluation of the USAID/Ethiopia-funded International Foundation for Education and Self-Help Capacity Building in Teacher Education Program. 2. Purpose The purpose of this Task Order is to provide a team to conduct a comprehensive evaluation of the USAID/Ethiopia-funded Capacity Building in Teacher Education Program implemented by International Foundation for Education and Self-Help (IFESH). The findings and recommendations of the performance evaluation will provide USAID/Ethiopia and the Ethiopian Ministry of Education with information and data for determining the future direction in the strategies of USAID/Ethiopia’s Basic Education Office and the Ministry of Education to enhance the quality of teacher education programs. 3. Background IFESH is a United States-based private, voluntary, non-profit and non-governmental organization (NGO). IFESH/Ethiopia established an Ethiopian office in 1996 based on an agreement between IFESH and the Ethiopian Ministry of Education. IFESH is currently operating in Ethiopia under Proclamation No 621/2009, registered as a Foreign Charity with License No.0898. IFESH has been supporting the education sector in Ethiopia by deploying highly qualified professionals and experienced American volunteer teachers to various teacher education and other higher education institutions with funds from USAID/Ethiopia. In July 2002, USAID/Ethiopia awarded IFESH with the Capacity Building in Teacher Education Program (CBTEP) under grant number 663–G– 00–02–00360-00 for a five -year period (from July 24, 2002 through July 31, 2007). During this period, IFESH assigned 8 volunteers each year, totaling 40 volunteers to Ethiopia. Based on its satisfactory performance, continued relevance, and the need expressed by the Government of Ethiopia (GOE), the grant was extended in July 2007 for one year (from August 1, 2007 through July 31, 2008) under a bridge funding mechanism to deploy 10 volunteers in Teacher Education Institutions (TEIs). In 2008, IFESH was granted an extension of the program for another four years from August 22, 2008 though July 31, 2012 to bring an additional 40 volunteers. 34 The overall goal of the IFESH/CBTEP is to help improve the quality of education through capacity building of teacher education institutions and in training teacher educators in alignment with the Ministry of Education’s teacher development program initiative. The specific objectives of the IFESH/CBTEP include the following:  Provide technical assistance to the Ministry of Education (MOE) in the implementation of the General Education Quality Improvement Program initiative, especially addressing the Teacher Development Program (TDP) in order to improve the quality of teacher education.  Provide technical assistance to establish Special Needs Education in the TEIs.  Technically assist the MOE in strategic planning for implementation of the School Improvement Program (SIP).  Improve teaching of the English language in the TEIs.  Initiate and implement HIV/AIDS education programs that integrate gender equity and prevention of harmful traditional practices (HTP) and sexually transmitted diseases (STDs).  Donate English language books and reference materials to schools and education resource centers. IFESH/CBTEP aims to recruit and assign well-qualified, experienced, American volunteers to Teacher Training Colleges, education faculties in the universities and the Ministry of Education to improve the quality of education through capacity building of pre- and in-service teachers as well as educational administrators. The volunteers are expected to work with Ethiopian counterparts under the Higher Diploma Program (HDP), serve as advisors, and perform other teaching tasks. Some volunteer educators are expected to facilitate HIV/AIDS education and gender mainstreaming and improve the teaching of English language in teacher training institutions. IFESH was required to provide a total of 32 volunteers between 2002/3 and 2007, another 10 volunteers in 2007/2008, and 40 volunteers in the four years between 2008/9 and 2011/12, totaling 82 volunteers during the life of the project (i.e. an average of 10 volunteers annually). Objectives of the Evaluation The evaluation team shall conduct a performance evaluation of the USAID/Ethiopia-funded IFESH/CBTEP. The following are the main objectives of the evaluation:  Examine and verify the results and effectiveness (output and outcome) of the program since its inception vis-à-vis the project agreement, goal, objectives and targets.  Examine whether the program activities, outputs, and outcomes have been integrated as appropriate into the initiatives of the host government and teacher training institutions.  Determine the level of satisfaction by the MOE, College of Teacher Education (CTEs), universities, and USAID/Ethiopia.  Identify lessons learned including innovations.  Identify the opportunities and challenges of program implementation and how they have been addressed, including recommendations for appropriately addressing those that have not yet been sufficiently addressed. 35  Determine if the program is still relevant and desirable, and suggest options for future directions and sustainability beyond the end of the current grant period (July 31, 2012). 4. Focus of the Evaluation- Questions and Issues The evaluation team shall address the following issues: h) The performance of IFESH/CBTEP.. i) Activities organized by volunteers that benefited teachers and students in colleges/universities and students including HIV/AIDS education, gender and other co-curricular activities. j) Activities organized and lead by volunteers that benefited teachers and students in neighboring schools. k) Manner in which the volunteers technically supported the institutions in development of strategies, guidelines, plans, etc. l) Trainings and technical assistance provided in the HDP of teacher development program in the colleges/universities. m) Level of effectiveness of the volunteers in working with their counterparts and transfer of knowledge and skills to counterparts. n) Resources (such as manuals, modules, researches, education material, etc.) produced with the technical assistance of volunteers and continue to be in-use. o) Innovations introduced by volunteers that continue to be in-use. p) Type of support institution management received from volunteers. q) Efficient and effective use of the volunteers (i.e. maximized use of the volunteers’ time during their tenure in colleges/universities). r) Volunteers’ activities relationship to the host government and colleges/universities plan and strategy. s) Volunteers’ activities level of response to the perceived needs of its beneficiaries. t) Constraints on volunteers’ activities and how they were rectified. u) Sustainable volunteer activities. v) Lessons learned from the volunteers contributions. 5. Evaluation Methods Both quantitative and qualitative methods will be required to collect data from a number of sources, including field visits and interviews/focus groups with representatives of the implementing partner, MOE, colleges, universities, Regional Sate Education Bureaus (RSEBs), schools. Techniques/instruments to be used to capture data may include structured questionnaires, direct project activity observations, interviews and other rapid appraisal methods. The Team is expected to interview MOE officials; presidents, deans, teachers and students of universities/colleges; school directors; teachers; students; and implementing partner’s staff in Addis Ababa as well as in regional offices, and others deemed necessary. 36 Evaluation methods include, but are not limited to:  Prepare evaluation framework including the methodology, questionnaires, and tools pertinent to the nature of the evaluation.  Review of the project agreement and determine if it has been followed.  Assess the extent to which all key counterparts working with IFESH volunteers have performed their expected roles and responsibilities under the program.  Review all internal performance and assessment reports, beneficiaries’/trainees’ reports and sample materials produced by the volunteers.  Review all program reports, publications such as quarterly newsletters, annual updates and related documents submitted to USAID/Ethiopia, and certify the reliability of the information provided.  Visit a sample of institutions that have benefited from the program and interview key stakeholders – heads of departments, Deans of Colleges/Universities, supervisors and counterparts of volunteers, and volunteers – to verify if program goals and objectives as well as expected results have been or are being achieved, and to determine the relevance and continued desirability of the program.  Meet GOE/MOE officials to determine the degree of satisfaction with the results of the program, fit of the program to the educational plan and strategies of the donor and the host government, and continued relevance and desirability of the program. Once selected, the Contractor (Assessor/Evaluator) will prepare a detailed evaluation framework including sample size and instruments which will be reviewed and approved by USAID/Ethiopia. The evaluation work will include field visits to a representative sample drawn from the following list of institutions from the various regions in Ethiopia (Tigray, Amhara, Oromia, SNNP and Addis Ababa) that have benefitted from the CBTEP: Abi Adi CTE Gondar CTE Dessie CTE Adwa CTE Dessie CTE Awassa CTE Debre Berhan CTE Gambella CTE Mekele CTE Adama University Mekele University Haramaya University Hossania CTE Axum University Bahir Dar University Debre Berhan University Gondar University St. Mary's University College (private) MOE, Teacher Education, Curriculum, Higher Education, etc. Departments 6. Report and Deliverables Schedule 37 The evaluation will be conducted not more than four week period. The first week will be devoted to the development of instruments, obtaining approval from USAID/Ethiopia, and reviewing various documents in the IFESH/Ethiopia office in Addis Ababa and other places. Then, the assessors will spend not more than a week and half in the field (outside Addis Ababa) to collect data and information. They will spend the last week in Addis Ababa to compile and analyze the data, prepare the draft report, and debrief USAID/Ethiopia. The Contractor will submit the final report to USAID/Ethiopia no later than ten days after the four-week duration of the evaluation work. Schedule/Timeline: tentative schedule of the evaluation will be as followed: No. Date Activity 1 November 4-6,2011 Framework of the evaluation, development of instruments, sampling size, and approval from USAID. 2 November 5-12, 2011 Review documents and collect data and information from field 3 November13- 15, 2011 Compile and analyze data, and draft report 4 November 18, 2011 Debriefing to USAID/Ethiopia 5 November29,2011 Submit final evaluation report Report writing: The Team shall provide to USAID and its stakeholders a first draft of the evaluation report for review and comments. The report (formatted in Times New Roman, with font size 12 and single space) should not exceed 25 pages. The first draft shall follow the general format as follows: Executive Summary Purpose of Evaluation Background Methodology Body of the report Findings Lessons learned Conclusion 38 Recommendations Appendices Discussion on the findings and draft report (One workday): The Team will have a half-day meeting with the USAID BES Office Team to discuss the findings and the draft report and a half-day debriefing session with the MOE, IFESH/Ethiopia and other stakeholders deemed necessary to discuss the findings, experiences, lessons learned, and recommendations. Relationships and Responsibilities: The Team will work under the technical direction of the BES Office Chief or designee. This evaluation shall be participatory and collaborative in nature, encouraging and relying on host country and IFESH Ethiopia Field Office personnel. 7. The Evaluation Team The evaluation will be conducted by a two-man team of independent assessors/evaluators. Requirements of each member of the team are as followed: Education Specialist: Team Leader The team leader should have a minimum of Master’s level degree in education with five years of experience in the developing world, Ethiopia or Africa in particular. He/she must have successful previous experience in teacher education development in Ethiopia and in leading evaluation teams. Excellent written and oral communication skills are essential. Knowledge of USAID programs, regulations and procedures will be an advantage. Evaluation Specialist Advanced degree (Masters) and four years of experience in conducting participatory evaluations, including facilitating group meetings and using rapid appraisal methods is necessary. The evaluation specialist must have successful previous experience in educational evaluation and familiarity with teacher education issues. 8. Terms and Conditions Duty Post: The duty post for this Delivery Order is Ethiopia. Language Requirements: Members of the evaluation team are required to have excellent oral and writing proficiency in English. All reports are to be submitted in English. Access to Classified Information: The team will not have access to classified information. 39 Logistical Support: The Evaluation Team has to be able to manage its logistics within the country. 9. Evaluation Criteria The criteria set forth below will serve as the basis for the technical and cost evaluation of the proposal to fulfill the tasks identified in the Statement of Work (SOW). 1. Clear, realistic evaluation plan and methodology which ensures that the evaluation SOW is well treated. (35 points) 2. Qualification of the evaluation team members: The team leader and the evaluation specialist to have a minimum of advanced degree (masters or PhD) in education, preferably in research methods and/or educational evaluation and successful previous experience in development and management of teacher education. (30 points) 3. Experience in conducting educational research and evaluation activities, especially in the developing world, Ethiopia or Africa in particular. (15 points) 4. Experience in education sector, particularly in teacher education, and familiarity with Ethiopian Ministry of Education teacher development initiative. (15 points) 5. Costs: The proposal will be evaluated, generally to determine that proposed costs are reasonable and realistic. (5 points) 40 Annex IX: Addressing Cross-Cutting Issues at Hosanna TEC The situation The Government of Ethiopia formulated gender and HIV/AIDS policies and established machineries at all levels of the administrative structure to ensure implementation of the polices. Gender and HIV/AIDS Desks are established at higher learning institutions to address gender stereotypes and practices that perpetuate the subordinate position of women in Ethiopia. The Gender and HIV/AIDS officers at Hosanna TTC organized various awareness raising and information sharing events to help students avoid risky sexual behaviours and develop social skills. They have established clubs and worked in collaboration with English language clubs (such as writing clubs, drama clubs and music clubs) to address gender and HIV/AIDS issues in an entertaining and educative way. According to the Gender officer, as is the case in other HLIs in Ethiopia, female students at Hosanna TTC face challenging situations. Nuria is a teacher at Hosanna TTC. She worked as Gender and HIV/AIDS focal person of the college. She noted that female students faced challenges as the college does not provide accommodation to students. It is the responsibility of each student to take care of themselves with food, accommodation and other basic needs with a monthly allowance of 350 birr. This particularly created challenges for female students. Due to their limited social exposure, female students are forced to get protection from male students who come from the same community and share accommodation with their fellow male students. As a result of this way of life, female students were expected to perform domestic work for their male counterparts. Due to their socialization, male students expected female students to prepare food and wash their clothes. They saw female students as care givers. This created huge economic and time burden for female students. The female students did not have enough time to study and developed submissive behavior in meeting their male counterparts’ needs and demands. In some cases, female students are asked to make sexual favors. There were reported cases of female students involved in transactional sex to supplement their living with additional incomes. The female students did not also have developed social skills and experience to avoid unwanted pregnancy, STI and HIV/AIDS. The gender perception and bias at the college was also not favorable for female students. Women are generally considered soft and weak needing support and protection from male counterparts. The response The Gender and HIV/AIDS officer of the college realized the challenges of the female students and organized different awareness raising, social skill development and experience sharing events to help female students cope up with the challenges they are facing. As a woman herself, she cares for the causes of female students. She holds discussions with individual female students about their life, experience in sharing accommodations and resources and mobility in town. The hardships of the female students did not give her peace of mind. She organized different counseling and advisory services for female students. The major Gender and HIV/AIDS activities included:  Organizing welcoming program to all female students. According to Nuria, the welcoming event provided female students a sense of inclusiveness and helped them connect one another emotionally and socially. Senior students shared their experiences and advice to new comers. The female students were encouraged to make friends with female students other than their own villages. This helped develop social networks and socialization of the female students.  Providing counseling service on how to manage life in shared accommodation and resource management. Female students were advised and made to share experiences on how they manage their study time and financial resources. This made the female students assertive in dealing with their male counterparts who shared resources.  Establishing Gender and HIV/AIDS clubs to create venue for discussion, debating and sharing personal testimonies and other skills. Participating female students have found club participation useful. They have developed social skills that helped them manage and overcome challenges. 41  Providing Gender and Assertiveness trainings with the support of volunteers. The trainings helped students question their own perceptions and biases.  Celebrating March 8 Women’s Day with Q&A and writing competitions. The result The KII with the Gender and HIV/AIDS officer revealed that the various interventions taken to address gender and HIV/AIDS issues have resulted in improvements in the social skills of female students. The trends of changes observed included:  Female students improved their social and networking skills  They reported cases of harassment and sought for advice and protection from the police  Many rape and gender based violence reported cases increased  Increased membership of Girls’ and HIV/AIDS clubs  Female students become assertive  Academic performance of female students improved as a result of tutorial classes, study skills trainings and support and advice among female students Sustainability There are “islands” of individual initiatives to address gender and HIV/AIDS issues in the college. However, there is no system that would sustain the process and results of the initiatives. Gender and HIV/AIDS issues are perceived as only the business of the officers. Gender officers are usually nick-named and discouraged for their work. Teachers ridicule the work of the Gender officer: “When is your holiday”, referring to the gender-related activities. Gender and HIV/AIDS activists have not also been supported by the management. Another important thing for sustainability and monitoring results is the need for baseline information. Way forward  Getting the commitment of leadership and higher management to able to institutionalize the gender mainstreaming activities  Having baseline (e.g., Gender Audit assessment) to assess the political will of management, gender competency of staff, the college’s working culture to determine the extent to which system, structure and mechanisms are gender sensitive to ensure accountability to incorporate gender in the teaching-learning process  Designing program intervention which will promote gender equality  Identifying good practice and replicating them in linkage schools as learning model