THE INTERNATIONAL FOUNDATION FOR EDUCATION AND SELF-HELP (IFESH) MID-TERM EVALUATION REPORT July 22, 2011 This publication was produced for review by the United States Agency for International Development (USAID). It was prepared by Morgan Borszcz Consulting, LLC. This document was produced for review by the United States Agency for International Development (USAID). It was prepared by Morgan Borszcz Consulting, LLC, under contract number AID-OAA-C-10-00119. Cover Photo: Exam time for students at Our Lady of Apostles (OLA) College, Cape Coast, Ghana. Photo, Ms. Yael Cohen, Morgan Borszcz Consulting, LLC, January 2011. - i - IFESH Mid-Term Evaluation Report Acknowledgements The Africa Bureau, Office of Sustainable Development, Education Division (AFR/SD/ED) in the U.S. Agency for International Development (USAID) requested that an evaluation of the International Foundation for Education and Self-Help’s (IFESH’s) American Educators for Africa (AEFA) program be conducted. Since then, the Evaluation Team interviewed countless stakeholders and compiled and analyzed the data gathered, all of which has culminated in this Report. The Evaluation Team would like to thank all of the stakeholders interviewed for their insights, candor, and time. In particular, we thank the staff at IFESH (both at Headquarters and in the field) for allowing us full access, and facilitating an introduction to the stakeholders we interviewed. The hospitality we received both in the U.S. and in Africa was tremendous, and greatly appreciated. Interviews conducted by the Evaluation Team included personnel at the USAID Missions, the Ministries of Education (MoEs), the host institutions where volunteers are placed, other organizations doing similar work such as the Peace Corps and World Cocoa Foundation, along with the current and former American Volunteer Educators (AVEs) and current Local Volunteer Educators (LVEs). The Evaluation Team acknowledges that this endeavor was not a traditional evaluation; we have provided targeted recommendations herein, along with best practices research. We hope that this Report will be of benefit to IFESH as the organization continues to meet the challenges of a changing environment and to other organizations, in particular in the areas of Monitoring, Evaluation, and Reporting (ME&R), and volunteer management. The Evaluation Team was comprised of the following individuals: Ms. Yael Cohen Ms. Gayla Cook-Mohajane Mr. Michael Matthews Mr. Obie Shaw - ii - IFESH Mid-Term Evaluation Report Contents Acknowledgements ..................................................................................................................................................................................................... i List of Acronyms ......................................................................................................................................................................................................... iv I. Executive Summary........................................................................................................................................................................................... 1 1.0 Background ................................................................................................................................................................................................. 1 2.0 Goals and Methodology ................................................................................................................................................................... 1 3.0 Current State ........................................................................................................................................................................................... 2 4.0 Findings ........................................................................................................................................................................................................ 2 5.0 Recommendations ............................................................................................................................................................................... 4 II. Introduction ........................................................................................................................................................................................................... 6 1.0 Background Information ................................................................................................................................................................... 6 2.0 Goals of Evaluation .............................................................................................................................................................................. 6 3.0 Approach/Methodology ................................................................................................................................................................... 7 4.0 Key Evaluation Questions ................................................................................................................................................................ 7 5.0 Use and Limitations of Data Collection Methods ............................................................................................................ 8 III. Evaluation Findings ............................................................................................................................................................................................. 8 1.0 The African Education Context ................................................................................................................................................... 8 2.0 Evolution of the USAID and IFESH Partnership ................................................................................................................ 9 3.0 Determinants of AEFA Success in Each Country .......................................................................................................... 10 4.0 Filling Gaps with a Purpose to Improve Teaching, Systems, and Education Resources ......................... 12 4.1 Current State: Teaching, Systems, and Education Resources ................................................................................. 12 4.2 Major Findings: Teaching, Systems, and Education Resources ............................................................................... 16 5.0 Strengthening LNGOs and Distribution of Small Grants .......................................................................................... 19 5.1 Current State: LNGOs and Distribution of Small Grants ......................................................................................... 19 5.2 Major Findings: LNGOs and Distribution of Small Grants ........................................................................................ 19 6.0 Enhancing ME&R ................................................................................................................................................................................ 20 6.1 Current State: ME&R ....................................................................................................................................................................... 20 6.2 Major Findings: ME&R ..................................................................................................................................................................... 22 7.0 Increasing Organizational Effectiveness ................................................................................................................................ 22 7.1 Current State: Organizational Effectiveness ....................................................................................................................... 23 7.2 Major Findings: Organizational Effectiveness ..................................................................................................................... 24 8.0 Strategic Implications for IFESH and USAID ..................................................................................................................... 27 8.1 Comparative Organizations......................................................................................................................................................... 27 8.2 Gap Analysis and Sustainability .................................................................................................................................................. 29 8.3 Achieving Greater Impact ............................................................................................................................................................. 29 - iii - IFESH Mid-Term Evaluation Report 8.4 Major Findings: Strategic Implications for IFESH and AEFA ..................................................................................... 30 IV. Recommendations .......................................................................................................................................................................................... 30 1.0 USAID Strategic Recommendations ...................................................................................................................................... 30 2.0 IFESH and AEFA Strategic Recommendations ................................................................................................................ 30 3.0 Teaching, Systems, and Education Resources Recommendations ....................................................................... 31 4.0 Education LNGOs and Distribution of Small Grants Recommendations ....................................................... 32 5.0 ME&R Recommendations ............................................................................................................................................................ 32 6.0 Organizational Effectiveness Recommendations ............................................................................................................ 33 7.0 Conclusion ............................................................................................................................................................................................. 36 Appendix A: List of Interviewees .................................................................................................................................................................... 38 Appendix B: Djibouti Case Study ................................................................................................................................................................... 44 Appendix C: Ethiopia Case Study .................................................................................................................................................................. 54 Appendix D: Ghana Case Study ..................................................................................................................................................................... 66 Appendix E: Liberia Case Study ...................................................................................................................................................................... 76 Appendix F: Malawi Case Study ...................................................................................................................................................................... 94 Appendix G: Non-Visit Countries Case Study .................................................................................................................................... 103 Appendix H: Best Practices for International Volunteer Management ................................................................................. 111 Appendix I: American Volunteer Educator Tables ........................................................................................................................... 129 Appendix J: IFESH Gap Analysis ................................................................................................................................................................... 131 Appendix K: Revising the Performance Monitoring Plan (PMP) ............................................................................................... 132 Appendix L: Work Plan ..................................................................................................................................................................................... 143 Appendix M: Conflict of Interest Statement ......................................................................................................................................... 156 - iv - IFESH Mid-Term Evaluation Report List of Acronyms Acronym Definition AED Academy for Educational Development AEFA American Educators for Africa AEI Africa Education Initiative AFR Africa Bureau AMEU African Methodist Episcopal University AVE American Volunteer Educator BPR Business Process Re-Engineering CBO Community Based Organization CEO Chief Executive Officer CESLY Core Education Skills for Liberian Youth CFPEN Centre de Formation des de l’Education Nationale (National Education Training Center) CFO Chief Financial Officer COO Chief Operating Officer CPD Continuing Professional Development (of Teachers) CR Country Representative CRIPEN Centre de Recherche, d’Information et de Production de l’Education Nationale (National Education Research, Information and Publishing Center) CTP Community Teacher Program DEO District Education Office ECCE Early Childhood Care and Education ED Education Division EFA Education for All EGMA Early Grade Math Assessment EGRA Early Grade Reading Assessment ELIC English Language Improvement Center ELIP English Language Improvement Program ESL English as a Second Language FACTS Foreign Assistance Coordination and Tracking System FCUBE Free Compulsory Universal Basic Education FFW Foundation for Women FTI Fast Track Initiative GDA Global Development Alliance GDP Gross Domestic Product GES Ghana Education Service GIZ German Society for International Development GoE Government of Ethiopia GORD Gouvernement de la République de Djibouti (Government of the Republic of Djibouti) HDP Higher Diploma Program HIV/AIDS Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome HQ Headquarters ICT Information and Communication Technology IQC Indefinite Quantity Contract - v - IFESH Mid-Term Evaluation Report Acronym Definition IEFA International Educators for Africa IELTS International English Language Testing System IFESH International Foundation for Education and Self-Help INSET In-Service Educational Training JICA Japanese International Cooperation Agency KPA Key Performance Area KRTTI Kakata Rural Teacher Training Institute LNGO Local Non-Governmental Organization LTTP Liberia Teacher Training Program LVE Local Volunteer Educator MDGs Millennium Development Goals MDGE Millennium Development Goal for Education ME&R Monitoring, Evaluation, and Reporting MENESUP Ministère de l'Education Nationale et de l'Enseignement Supérieur (Djibouti Ministry of Education) MIE Malawi Institute of Education MoE Ministry of Education MOEST Ministry of Education Science and Technology M&E Monitoring and Evaluation NGO Non-Governmental Organization OLA Our Lady of Apostles College OVC Orphans and Vulnerable Children PDOW Pre-Departure Orientation Workshop PEPFAR President Emergency Plan for AIDS Relief PMP Performance Monitoring Plan PPP Public/Private Partnership PTA Parent Teachers Association PVO Private Voluntary Organization RTTI Rural Teacher Training Institute SD Office of Sustainable Development SO Strategic Objective SOW Statement of Work STEM Science, Technology, Engineering, and Mathematics SWOT Strengths, Weaknesses, Opportunities, and Threats TESOL Teachers of English to Speakers of Other Language TFA Teachers for Africa TLMs Teaching and Learning Materials TOEFL Test of English as a Foreign Language TRC Teacher Resource Center TT Teacher Training TTC Teacher Training College TTI Teacher Training Institute UNDP United Nations Development Program UNESCO United Nations Education, Science and Cultural Organization UPE Universal Primary Education - vi - IFESH Mid-Term Evaluation Report Acronym Definition USAID U.S. Agency for International Development USG U.S. Government VSO Volunteer Service Overseas WAEC West African Examinations Council WCF World Cocoa Foundation YMCA Young Men Christian Association ZRTTI Zorzor Rural Teacher Training Institute - 1 - IFESH Mid-Term Evaluation Report I. Executive Summary On July 10, 2009, the U.S. Agency for International Development (USAID) awarded Cooperative Agreement No. RLA-A-00-09-00028-00 to the International Foundation for Education and Self-Help (IFESH) to strengthen basic education in select Sub-Saharan African countries. The Cooperative Agreement provides IFESH with $7,999,257 through July 10, 2012 for a program titled American Educators for Africa (AEFA). 1.0 Background Through the AEFA program, IFESH recruits, selects, and places American teachers in Sub-Saharan African colleges and universities (particularly Teacher Training Institutions (TTIs)), and other educational settings for a period of one academic year. Over the past two academic years through this Cooperative Agreement, IFESH has placed 74 American Volunteer Educators (AVEs) in eight countries: Djibouti, Ethiopia, Ghana, Kenya, Liberia, Malawi, Nigeria, and Senegal. IFESH is a venerable institution in the arenas of civil rights, and educational and economic advancement in the U.S. and Africa, founded by Reverend Leon Sullivan, a highly respected civil rights leader and social change innovator. The partnership between IFESH and USAID to support educational initiatives in Africa began in 1992 with the Teachers for Africa (TFA) program. Over the years, IFESH reports training more than 180,000 pre-service and in-service teachers in Sub-Saharan Africa previous to the current Cooperative Agreement, and that by training these teachers, more than three million primary school children have benefited from improved learning environments. Related IFESH initiatives have included the integration of HIV/AIDS messaging into primary and secondary curriculum and pedagogy, training for teachers in gender equity, and the prevention of sexual assault in the school environment. In addition, IFESH has built or refurbished 350 schools in six African countries. 2.0 Goals and Methodology The goals of the Mid-Term Evaluation were to review the effectiveness and efficiency of the AEFA program, and to provide IFESH with targeted and practical feedback. The scope was primarily based on determining the impact of the AEFA program activities in the field, the application of USAID’s education guiding principles, and if IFESH has a sustainable infrastructure in place to execute educational programming in Africa. A case study approach was used for the Mid-Term Evaluation. Specific countries were selected from eight countries where IFESH operates the AEFA program as representative of the operational and national educational dynamics. The five countries that were selected for in-person field visits to examine the varied communities and contexts were: Ghana, Liberia, Ethiopia, Djibouti, and Malawi. These field visits were conducted by the Evaluation Team between January and March 2011. The Evaluation Team also conducted telephone interviews with select stakeholders in the three other countries: Senegal, Nigeria, and Kenya. The Evaluation Team developed the following five key questions to guide the parameters of the Mid-Term Evaluation: 1. How has AEFA strengthened the capacity of teachers, administrators, and policymakers to enhance the quality of school teaching methodologies and pedagogical materials at the basic education level through technical aid and the training of teachers, using the main AEFA methods of teacher training, Teacher Resource Centers (TRCs), donated books, and serving as policy/management advisors? 2. How effective has AEFA been in providing technical assistance and knowledge to increase the capacity of Local Non-Governmental Organizations (LNGOs) to implement sustainable development programs, primarily in the area of education? 3. Does the IFESH field team – Country Representative, Field Office staff, and American and Local Volunteer Educators (AVEs and LVEs) – have the tools they need to succeed? Are the necessary - 2 - IFESH Mid-Term Evaluation Report organizational structures (people, processes, and technologies) in place to support and sustain AEFA? 4. Is IFESH using rigorous monitoring and evaluation systems that produce evidence of results and support program improvements and sustainability? Address Monitoring, Evaluation, and Reporting (ME&R) of the country program as a whole, and from stakeholder perspectives. 5. What are the successes, challenges, and opportunities of AEFA as an innovative, sustainable, and replicable model for accelerating education improvement? These five questions drove what data was collected, and ultimately how the findings and recommendations were organized. In collecting data on the AEFA program, the Evaluation Team focused on the three components of the AEFA program: 1) teacher training and policy/management support, 2) TRCs and donated books, and 3) education LNGO support. The Evaluation Team solicited input from USAID’s Africa Bureau, Office of Sustainable Development, Education Division (AFR/SD/ED), USAID Missions, and IFESH prior to finalizing the Report. 3.0 Current State AVEs are producing educational innovations in new course offerings, teacher training, and policy/management interventions that enhance basic education quality. However, many of these contributions are ad hoc and are not institutionalized, although they are related to meeting specific host institution needs. LVEs and CPD activities, intended to promote AEFA sustainability, are falling short in achieving this purpose as a result of insufficient planning and monitoring of performance. Book donation projects and TRC activities exhibit mixed results dependent upon how well they are planned and monitored for impact. Likewise for working with LNGOs and distribution of small grants, the intention for capacity building shows potential for success but falls short in strategically meeting AEFA objectives. The AVEs, IFESH HQ staff, and staff from the IFESH Field Offices devote considerable time to ME&R and provide the data required by the Cooperative Agreement. However, the ME&R system is not sufficiently tailored to the needs of all AEFA stakeholders. Nevertheless, IFESH is on target to meet or exceed most revised targets in its work plan. Improving performance to produce sustainable evidence based results of the AEFA program will need to be the focus of IFESH management and leadership to strengthen its operations. The considerable legacy of the IFESH name is being strongly challenged by competition, funding constraints, and the changing demands of partners for more focused, specialized program offerings from IFESH. 4.0 Findings The Evaluation Team analyzed and compiled data collected, and constructed findings in the following areas: 1) teaching, systems, and education resources, 2) LNGOs and distribution of small grants, 3) ME&R, and 4) organizational effectiveness. As the Evaluation Team interviewed stakeholders of the AEFA program in each country, the success of the program could be determined by the belief in the effectiveness of the program by the major stakeholders. In the Report, the Evaluation Team first discusses the five main determinants of successful execution of AEFA activities. These five success determinants, elaborated on in the Report, are as follows: 1) achieving USAID Mission objectives, 2) communication with USAID, 3) coordination between the Ministry of Education (MoE), host institutions, and IFESH, 4) volunteer usefulness to host institution, and 5) volunteer satisfaction. Based on the quantitative and qualitative data collected by the Evaluation Team, we assigned ratings for each of these success determinants, ranging from minimally acceptable/useful, to acceptable/useful, and to very acceptable/useful. The average rating overall for the five AEFA case study countries visited was rated as acceptable/useful. - 3 - IFESH Mid-Term Evaluation Report The Evaluation Team then presents the three types of AEFA volunteer assignments observed. This served as a backdrop to the identification of the current state and associated findings by area, which are summarized below. In the area of teaching, systems, and education resources, the major findings were:  Stakeholders affirm the quality of AVEs  Integrating crosscutting themes (Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome (HIV/AIDS) information, gender awareness, and community involvement) works  AVEs are the only source for Continuing Professional Development (CPD) for some teachers  AVEs placed in policy/management roles are few but have strong sustainable impact potential  AEFA exercises appropriate programmatic flexibility in the post-conflict environment of Liberia  Host institutions are requesting increasingly specialized AVEs for teaching and policy/management advisor assignments  Sustainable impact is more likely when AEFA activities complement those of the MoE and USAID  Stakeholders differ on the value of book donation utilization, appropriateness, and cost effectiveness  Some AVEs lecturing and providing technical assistance at universities are not supporting the AEFA goal of providing support to primary and secondary school teachers and education administrators – directly or indirectly  AEFA schedules are out of sync with some host institutions  IFESH work planning lacks continuity and burdens the resources of host institutions and IFESH  There is insufficient planning for continuity of policy/management support assignments  AEFA is inconsistent in constructing and monitoring a program for each country with outcomes and a work plan  Establishing public/private partnerships is progressing slowly as an IFESH mobilization strategy  The use of LVEs is not sufficiently contributing to building AEFA sustainability  Knowledge management and sharing are minimal The major findings for LNGOs and distribution of small grants are that, while the grants given serve an educational purpose, the process is not being implemented to strategically support AEFA objectives, nor is the grants process transparent. The Evaluation Team also found that the distribution of small grants is not solely dedicated to LNGOs, and that volunteer engagement with LNGOs varies. In ME&R, the major findings include that Teaching and Learning Materials (TLMs) and TRCs activities are valuable but need better monitoring to understand utilization and impact, and that the utilization and impact are not sufficiently addressed in book donation projects. Furthermore, IFESH is not using the monitoring feedback loop to improve program implementation, and not all elements of the Performance Monitoring Plan (PMP) have been implemented. For organizational effectiveness, the major finding is that current structures require greater efficiency and effectiveness, especially in relation to planning country programs, personnel functions and performance, and aspects of volunteer management processes. In terms of strategy and sustainability, it was found that the mission of IFESH remains relevant. However, the culture created by the founder of IFESH, Reverend Leon Sullivan, is facing major new demands for program refinement and impact in the face of stiff competition for shrinking resources. Based on an analysis of the information gathered and distillation of the major findings, the Evaluation Team answered the five key questions as follows: 1. The AEFA program is strengthening the local capacity to improve the quality of education in all countries, with AVEs introducing innovative products, techniques, and systems. However, in some - 4 - IFESH Mid-Term Evaluation Report cases, the improvements are ad hoc and not sustainable. There is an absence of guidance and monitoring of the use of LVEs to ensure ongoing program sustainability. 2. The intention to use small grants to support educational LNGOs has not been fully realized. In some cases, they have been instead directed towards a variety of non-strategic activities. 3. To build on isolated successes and the organization’s legacy, IFESH needs to adopt a systematic approach to performance oriented systems and staff accountability to be more competitive. 4. While considerable time is devoted to ME&R, the output is not sufficiently tailored to the needs of users nor does it result in program adjustments. 5. IFESH is a valuable well-known brand, by Africans in particular, and individual AVEs are well regarded relative to comparable organizations. MoEs, host institutions, USAID, and other potential partners require consistent, results oriented, increasingly specialized, and reliable partners to provide training and advisory services and transfer innovations in the field of education. IFESH provides a viable, replicable model for such services. However, in order to take advantage of existing opportunities and meet these demands to become a preferred service provider with a sustainable infrastructure in place, IFESH must deliver more customer focused, consistently reliable services, using more effective systems. 5.0 Recommendations The Evaluation Team makes recommendations that apply to the AEFA program overall as well as country￾specific recommendations in each of the five case studies for Djibouti, Ethiopia, Ghana, Liberia, and Malawi. The Evaluation Team recommends that if IFESH accepts the findings and recommendations, IFESH should undergo a major strategic thinking and change management exercise to reengineer its business processes to become a volunteer-centered, results-oriented organization. This will mean reviewing and making adjustments wherever necessary in staff, board functions, and internal and external systems. The Evaluation Team also recommends that IFESH develop and implement a business plan approach for the AEFA program and each AEFA country with outcomes, indicators, and targets. The business plan scope needs to examine what progress will have been made, with what AEFA inputs, in three years, five years, or ten years, and what the impacts will be. The business plan will identify why countries are selected and what types and numbers of volunteers are needed to accomplish the plan. Also, in line with the new USAID Education Strategy, which emphasizes selectivity and focus, IFESH should consider identifying education specialization – innovation areas – to become a leader in the field. Based on lessons learned and accumulated intellectual capital, IFESH might consider where there is comparative advantage, such as: reading, teacher/learner performance, distance education, Information and Communication Technology (ICT) in education, education in post-conflict settings, workforce development programs/technical schools, and professionalization of teachers/lecturers. For improving teaching, systems, and education resources, IFESH should consult with AVEs, Country Representatives, and Local Volunteer Educators (LVEs) on how to proceed with the CPD component in terms of execution and since targets are currently not being met. With plans to distribute more books in 2011-2012 through the AEFA program, IFESH should review the book donation operations including the use of previously donated books to date, the supply chain strategy, and security, and conduct a cost-benefit analysis. To achieve continuity, IFESH will need to consider assigning a Knowledge Management Coordinator at IFESH Headquarters (HQ) to archive the instruments, syllabi, proposals, TLMs, and policy documents that AVEs produce and establish a knowledge management system and protocols. IFESH should also revisit how to better utilize the valuable resources available through the collaboration with Bennett College for Women and Lincoln University. The Evaluation Team recommends that IFESH review its processes and experiences in making small grants to LNGOs thus far in AEFA countries to assess their strategic relevance to AEFA goals. Also, IFESH HQ - 5 - IFESH Mid-Term Evaluation Report management should clarify if AVEs should continue to engage with LNGOs. If so, the engagement should be documented in the AVE job description. ME&R is another area that requires attention. IFESH should revise the Performance Monitoring Plan (PMP) based on this Mid-Term Evaluation. AVEs and Country Representatives should act on ME&R issues such as resolving how to reconcile CPD and in-service training reporting. Easy-to-use instruments like sign-in rosters and feedback forms should be used to assess how TRCs are utilized and how they can be improved. Simple forms should be prepared for completion by LNGO grant recipients to provide data on beneficiaries (who, how they benefited, and how the number is calculated) for monitoring and reporting purposes. To improve organizational effectiveness in support of AEFA, the Evaluation Team recommends that IFESH conduct a review of the organization’s processes, technologies, and people, and a strategic planning exercise. IFESH will need to develop key performance areas, and adjust staffing as needed, including re-evaluating what skills/knowledge a Country Representative should have to successfully execute his/her duties. For IFESH HQ and Field Office staff, a Personnel Appraisal System should be instituted by IFESH HQ, in collaboration with the Field Offices, with common tasks and key performance areas for similar positions. To achieve effective and sustainable volunteer management, IFESH will need to examine and adjust its current practices. Recruitment should be a well-planned year-round process with performance targets and candidates should be selected as early as possible and given full knowledge of their assignment, including real-life information on the challenges and rewards of service. Year to year continuity should be improved and the use of the country planning approach would support this effort as would a formal process to transition outgoing volunteer work to inbound volunteers. The Pre-Departure Orientation Workshop (PDOW) in Scottsdale, Arizona and the in-country orientations should be revised to match the information desired by the volunteers as garnered from the interviews conducted for this Report. These orientations should be done in such a way as all volunteers arrive in-country on time to merge into country programs. Likewise, in-country training sessions should be instituted to reinforce policies and procedures (such as safety, evacuation, and medical) and provide training as needed. For volunteers, the performance appraisal form should be revised to elicit more useful feedback, and should be collected and discussed during the regular monitoring visits to host institutions. In times when less funding is available from donors, IFESH must demonstrate visible and measurable impact to justify its cost. IFESH should find a way to narrow the difference between its cost model and that of its competitors to ensure the efficient and effective use of funding. The findings and the recommendations contained within this Report require consideration at different levels within the organization to determine their feasibility and desirability. They are intended to provoke debate and motivation to make the work more gratifying for staff and volunteers, and more productive for IFESH and its stakeholders. Giving all children in Africa the opportunity for a decent education is the goal of IFESH and the dedicated staff, volunteers, and donors who support its programs. This Report salutes this laudable endeavor. - 6 - IFESH Mid-Term Evaluation Report II. Introduction 1.0 Background Information Helping Africa to achieve Universal Primary Education (UPE) is where the goals and missions of the U.S. Agency for International Aid (USAID) and the International Foundation for Education and Self-Help (IFESH) intersect. The IFESH strategy to accomplish this is through quality teacher education and training and education resources support. The Africa Bureau (AFR) of USAID bases its development strategy on effective partnerships, solid research, and best practices.1 Sub-Saharan Africa, where USAID invests a large proportion of Non-Critical Priority Country resources, is where IFESH operates. Both organizations focus strategies on achieving the Millennium Development Goal (MDG) 2 to provide universal access to primary education for all children by 2015, and to promote incorporating the crosscutting themes of strengthening the rights of vulnerable populations (particularly, women and children), Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome (HIV/AIDS) prevention and mitigation information, and community involvement and accountability into education.2 IFESH is a venerable institution in the arenas of civil rights, and educational and economic advancement in the U.S. and Africa founded by Reverend Leon Sullivan. As an acknowledgment of his leadership in creating effective vehicles for empowering individuals to better their lives through skills training programs, and influencing government policies for employment equity and funding for related interventions, Reverend Leon Sullivan was consulted by American Presidents and Captains of Industry from the sixties through the nineties. Success in the U.S. led to the expansion of his concepts to Africa, with support of African Presidents, civil society organizations, and the private sector. The Sullivan Principles that Reverend Leon Sullivan developed in 1977 promoted corporate social responsibility and racial equality, applying economic pressure on South Africa in protest of its system of apartheid; they were widely adopted by U.S.-based businesses operating in South Africa. In 1999, the Global Sullivan Principles were extended beyond apartheid to promote corporate advancement of human rights and social justice at the international level. IFESH reports training more than 180,000 pre-service and in-service teachers in Sub-Saharan Africa previous to the current Cooperative Agreement, and that by training these teachers, more than three million primary school children have benefited from improved learning environments. Related IFESH initiatives have included the integration of HIV/AIDS messaging into primary and secondary curriculum and pedagogy, training for teachers in gender equity, and the prevention of sexual assault in the school environment. In addition, IFESH has built or refurbished 350 schools in six African countries. Currently, IFESH has offices in Côte d'Ivoire, Djibouti, Ghana, Guinea, Ethiopia, Kenya, Liberia, Malawi, Nigeria, and Senegal, all of which are in Sub-Saharan Africa. On July 10, 2009, USAID awarded a three year Cooperative Agreement No. RLA-A-00-09-00028-00 to IFESH to implement the American Educators for Africa (AEFA) program aimed at strengthening basic education in select Sub-Saharan African countries. Through the AEFA program, IFESH assists African countries in attaining their Education for All (EFA) goals. The Cooperative Agreement requires IFESH to recruit and assign volunteers termed American Volunteer Educators (AVEs) over three year period to carry out training activities and provide other relevant support services in eight priority countries where USAID currently has strategic education objectives: Djibouti, Ethiopia, Ghana, Kenya, Liberia, Malawi, Nigeria, and Senegal. AEFA’s goal is to strengthen basic education of the target countries by enhancing their capacity to provide pedagogical support to primary and secondary school teachers and education administrators to enable them achieve the goals of EFA and the Millennium Development Goal for Education (MDGE). 2.0 Goals of Evaluation The goals of the Mid-Term Evaluation were to review the effectiveness and efficiency of AEFA, and to provide IFESH with targeted and practical feedback. The scope is primarily based on determining the - 7 - IFESH Mid-Term Evaluation Report success of the AEFA activities in the field, and application of USAID’s education guiding principles.3 The objective to evaluate AEFA was to:  Record insights into past successes and failures  Document performance  Review policies and procedures  Determine if policies are effective in the field  Gain knowledge on how to adjust programs and realign interventions to match U.S. Government (USG) and African priorities  Issue recommendations for long-term sustainability In essence, the purpose of the Mid-Term Evaluation was to examine whether AEFA is meeting the mission of USAID, and if IFESH has a sustainable infrastructure in place to execute educational programming in Africa. 3.0 Approach/Methodology A case study approach was used for the Mid-Term Evaluation. Specific countries were selected as representative of the operational and national educational dynamics of the AEFA program in the eight countries where IFESH operates. It was determined that this approach would work best because the AEFA program is implemented in countries with varied communities and contexts. Ghana, Liberia, Ethiopia, Djibouti, and Malawi were the five countries where the Evaluators conducted in-person field visits. The Evaluators conducted telephone interviews with select stakeholders in the other three countries of Senegal, Nigeria, and Kenya. The selected case study countries had a large enough beneficiary pool to collect data and represent all aspects of the AEFA program being implemented. From a stakeholder perspective, the case study approach explored the types of benefits for the various stakeholders and whether they focus on the direct beneficiaries, or the institution or system as a whole, and sustainability potential. Stakeholders interviewed included:  IFESH personnel (Headquarters (HQ) and Field Offices)  AVEs and LVEs  USAID Missions  Ministries of Education (MoEs)  Host institutions  Local Non-Governmental Organizations (LNGOs) The Evaluation Team4 developed Interview Guides prior to conducting interviews in the field. These Interview Guides were used in the field during each set of country visits. The Evaluators first visited Ghana and Liberia, and then went to Djibouti, Ethiopia, and Malawi. This approach enabled the Evaluators to use a range of tools to capture the data necessary for a successful evaluation in various country contexts. Using both qualitative and quantitative data provided the Evaluation Team with comprehensive information for analysis. 4.0 Key Evaluation Questions Prior to going into the field, the Evaluation Team developed key questions to guide the parameters of the Mid-Term Evaluation. The five overarching key questions were: 1. How has AEFA strengthened the capacity of teachers, administrators, and policymakers to enhance the quality of school teaching methodologies and pedagogical materials at the basic education level - 8 - IFESH Mid-Term Evaluation Report through technical aid and the training of teachers, using the main AEFA methods of teacher training, Teacher Resource Centers (TRCs), donated books, and serving as policy/management advisors? 2. How effective has AEFA been in providing technical assistance and knowledge to increase the capacity of Local Non-Governmental Organizations (LNGOs) to implement sustainable development programs, primarily in the area of education? 3. Does the IFESH field team – Country Representative, Field Office staff, and American and Local Volunteer Educators (AVEs and LVEs) – have the tools they need to succeed? Are the necessary organizational structures (people, processes, and technologies) in place to support and sustain AEFA? 4. Is IFESH using rigorous monitoring and evaluation systems that produce evidence of results and support program improvements and sustainability? Address Monitoring, Evaluation, and Reporting (ME&R) of the country program as a whole, and from stakeholder perspectives. 5. What are the successes, challenges, and opportunities of AEFA as an innovative, sustainable, and replicable model for accelerating education improvement? These five questions drove what data was collected, and ultimately how findings and recommendations were organized. In collecting data on the AEFA program, the Evaluation Team focused on the three components of the program: 1) teacher training and policy/management support, 2) TRCs and donated books, and 3) education LNGO support. 5.0 Use and Limitations of Data Collection Methods The data collection methods that were used throughout the Mid-Term Evaluation included in-person interviews, anonymous surveys, group interviews, and direct observations. On their own, each of these data collection methods have certain limitations such as with in-person interviews with interviewees who may not have a comfort level with the interviewer to be candid and state their true opinions during an interview. However, combining all of these data collection methods enabled the Evaluation Team to capture the data necessary for a robust evaluation. The data that was collected was both qualitative and quantitative. While in the field, the Evaluation Team adjusted the collection method to gather primarily qualitative data once it was determined that predominantly quantitative data was inadequate to obtain given the wide variance in-country contexts and difficulty in isolating variables. Although the Evaluation Team recognizes that quantitative data can help to understand what happened, qualitative data can describe why or how it happened and can provide a way to illustrate the stories behind the numbers. By intentionally using a mainly qualitative evaluation method in conjunction with the quantitative data available, the Evaluation Team was able to understand why certain results were achieved or not achieved, explain unexpected outcomes, and arrive at informed recommendations for USAID and IFESH that aim to mitigate challenges and propagate more successful outcomes and impacts. III. Evaluation Findings 1.0 The African Education Context World leaders established the MDGs in 2000 to guide efforts to significantly reduce poverty and its root causes by 2015. Deliberately ambitious, the MDGs have provided a global agenda that has galvanized international action towards agreed indices of change. In 2002, the World Bank together with development partners launched the EFA Fast Track Initiative (FTI), a global partnership to help low-income countries meet the education MDGs and the EFA goal that girls and boys complete a full cycle of primary education by 2015. A huge increase in primary school enrollment rates with near parity for girls is the MDGs success story for Africa. Not unsurprisingly, the perverse inverse relationship has resulted in a decline in quality in most - 9 - IFESH Mid-Term Evaluation Report countries with the influx of massive numbers of students overwhelming the capacity of classrooms, teaching materials, teachers, and management systems which were inadequate in all countries from the start. One of the main strategies of the MDGs approach was constant learning. From the continuous analysis of successes and failures by governments, the United Nations Development Program (UNDP), international and local civil society, and donors like the U.S., and African MoEs have responded with policy revisions and implemented changes with the help of international stakeholders like USAID and IFESH, always asking, which interventions make the most difference, with a sustainable impact, the most cost effective? The following chart compares the education and gender indicators of the MDGs for five of the countries where AEFA operates. The poverty level in each country is an approximate indicator of the financial capital and human capital available to the government for improving education. The youth population percentage represents a gauge for the social risk faced by the country if education does not improve. 2.0 Evolution of the USAID and IFESH Partnership USAID began supporting IFESH programs in 1992 with the Teachers for Africa (TFA) program. Over the years, USAID has continued to fund TFA. Initially the teacher-focused program was aimed at supporting grassroots education efforts in Africa with skilled African-American volunteers. As TFA became more oriented toward the formal education system instead of LNGOs, recruitment expanded to anyone who met the criteria, and it was renamed International Educators for Africa (IEFA). The current program funded by USAID is AEFA, a component of IEFA, where placed volunteers are referred to as American Volunteer Educators (AVEs). The former grassroots orientation is retained with the capacity-building component to support the education-related work of LNGOs; and now with a new small grants feature. Another new Education/Socio-Economic Indicators of Sub-Saharan Africa and Five Countries with AEFA Programming Indices Sub-Saharan Africa Djibouti Ethiopia Ghana Liberia Malawi Likely to achieve MDG 2 by 2015: Boys and girls alike will be able to complete a full course of primary schooling5 No Yes Yes No No No Net enrollment primary education (%)6 75.3% (2008) 44.4% (2009) 82.7% (2009) 75.9% (2009) 75.2% (2000) 90.8% (2009) Primary Completion rates7 63.9% (2008) 35.4% (2009) 55.2% (2009) 82.7% (2009) 57.6% (2008) 59.2% (2009) Literacy rate, youth total (% of people ages 15-24)8 72.1% (2009) 86% (2002)9 44.6% (2008) 80.1% (2009) 75.6% (2008) 86.5% (2009) Proportion of pupils starting grade 1 who reach last grade of primary, Both sexes10 - 64.3% (2008) 38.1% (2008) 73.2% (2008) 45.6% (2007) 41.9% (2008) Proportion of pupils starting grade 1 who reach last grade of primary, Boys11 - 64.1% (2008) 35.4% (2008) 75.4% (2008) 48.6% (2007) 42.1% (2008) Proportion of pupils starting grade 1 who reach last grade of primary, Girls12 - 64.5% (2008) 41.4% (2008) 70.9% (2008) 42.6% (2007) 41.7% (2008) Percentage of population under 15 years13 42.6% (2009) 36.1% (2009) 43.5% (2009) 38.4% (2009) 42.7% (2009) 46.2% (2009) Population below poverty line (less than US$1 per day)14 45% (2003) 74.4 % (2010)15 39% (2005) 30% (2006) 83.7% (2007) 73.9% (2004) Key: - = no data reported. - 10 - IFESH Mid-Term Evaluation Report feature is Local Volunteer Educators (LVEs). These are locally recruited educators intended to ensure sustainability of the work of AVEs. The President/Chief Executive Officer (CEO) of IFESH explained that the “shift in philosophy” to the current AEFA also embraced policy making in addition to teacher training and curriculum development. She explained that the program hallmarks maintain a belief in diversity among volunteers, and that the work of volunteers is led by host country needs as “they know more than an outsider.” Also, she said that IFESH recruits and screens for volunteers who can accept the desire for reforms of host countries. IFESH has worked in a changing context with USAID. As a USAID/Washington program, the TFA operated almost completely autonomously from USAID Missions, not unlike other USAID/Washington-funded programs. During funding agreements with IFESH under the Africa Education Initiative (AEI) of USAID, in response to feedback in AEI evaluations, USAID requested that IFESH improve communication with USAID Missions to promote mutual reinforcement among USAID investments. The Cooperative Agreement states that the AEFA program will work “in support of the goals and strategic objectives of USAID Mission-based bilateral education programs” to strengthen host country basic education. Further, there is a new USAID Education Strategy 2011-2015 that emphasizes a more narrow focus of USAID investments to achieve greater impact. The three-year Cooperative Agreement with IFESH was competitively awarded in July 2009. This marked the first competitive award; previous awards were sole sourced. USAID in Washington has consistently advised IFESH of the importance aligning program activities with USAID Missions, and monitoring and evaluating the impact of its activities in each country. As such, these are areas that the Evaluation Team examined during the Mid-Term Evaluation. 3.0 Determinants of AEFA Success in Each Country As the Evaluation Team talked to stakeholders of the AEFA program in each country, the success of the program could be determined by the belief in the effectiveness of the program by the major stakeholders: host institutions (and the teachers and in schools they work with), MoEs, USAID Missions, LNGOs, AVEs and LVEs, and the IFESH staff in Scottsdale, Arizona and the Field Offices. After an analysis of the data collected, the Evaluation Team found that five factors determine the extent to which the AEFA program is successful in the five case study countries that were visited, as perceived by the major stakeholders. The following are a description of each: 1. Achieving USAID Mission objectives – USAID strategy in each country is determined in conjunction with the host government. Coordination with host country priorities guides IFESH activities since each of the current eight country programs began with a Memorandum of Understanding (MOU) with the host government. Alignment with USAID objectives means that other U.S. Government (USG) resources are available and the area of intervention is not only a priority area but one where America has strategic advantages. In the chart below, three means that USAID perceives that IFESH and USAID objectives are directly aligned (IFESH working directly with or complementing USAID), a two means aligned most of the time, and a one means not at all. 2. Communication with USAID – Even if IFESH worked in the same priority areas as the USAID Mission, effective communication is important for coordination of the considerable resources of the USG, including the Peace Corps, and relationships with other contractors who have funding for related work, within which IFESH plays a complementary role. Excellent communication exemplified by regular and collaborative communication earns a three, satisfactory communication denoted by ad hoc but meaningful communication earns a two rating, and poor communication meaning irregular communication is a one. 3. Coordination between the MoE, host institutions, and IFESH – Although IFESH signs an MOU with each country’s MoE, there are variations in congruity between the views of the national MoE and - 11 - IFESH Mid-Term Evaluation Report the institutions where AVEs work. Good coordination promotes better use of the work of AVEs for strategic objectives and possible institutionalization. When the MoE and host institutions have the same understanding of the outcomes of the AEFA program, the rating is a three, some of the time earns a two rating, and not at all is a one. 4. Volunteer usefulness to host institution – Whether the AVE fits into a larger strategic program for USAID and the MoE or not, host institutions have their specific needs. How well the AVE fills these needs is measured against the extent to which the host institution perceives the AVE matches the requirements of the job description, and the burden on their resources. Host institutions ratings of AVEs as “very useful” is a three rating. Less than the highest rating means that there were AVEs who were rated as “useful” and/or there were issues related to host institution management of AVEs is a two. Situations where AVEs were not useful earn a one rating. 5. Volunteer satisfaction – The AVE is the core of the program. Their feeling that their work was appreciated and useful, and they were well supported by IFESH speaks directly to management and leadership from the Country Representative and Field Office staff, and then from HQ staff and leadership. Ratings flow from responses from AVEs about their volunteer experience, and IFESH Field Office and HQ staff, with three indicating that the majority of AVEs in the country felt favorably, two meaning not all gave the highest ratings, and one indicating most volunteers had concerns. The ratings are indicative and intended to show the interplay of the success of these categories in the five of the eight AEFA countries visited by the Evaluation Team. They are not scientific, especially with the small numbers of volunteers in each country. These categories could become performance criteria for quantitative measurement in the monitoring and evaluation system of AEFA; the data could be collected with simple surveys about satisfaction. The information in this chart is meant to prompt reflection, program and/or operational adjustments, and to inform decisions. The performance country average, based on the five countries the Evaluation Team visited, overall is a 2, and ranged from a 1.6 to a 2.8. The following are examples of the information used by the Evaluation Team to determine ratings by country by category. In Djibouti, for example, communication with the Mission had been poor, improved when USAID brokered a partnership plan between IFESH and Academy for Educational Development (AED) for interim IFESH oversight, and deteriorated again because IFESH Field Office and HQ staff initiated a change in the IFESH Statement of Work (SOW) with the MoE without coordinating with USAID/Djibouti; also how IFESH dealt with personnel issues without informing the Mission until the last minute. Following a period of strained relations between IFESH and MENESUP, the President/CEO of IFESH came to Djibouti and communication Performance Ratings of Five Countries Case Study Country Achieving USAID Objectives USAID Communi￾cations MoE-Host-IFESH Coordination Host Institution Usefulness Volunteer Satisfaction Country Average Djibouti 2 2 2 2 2 2 Ethiopia 3 3 3 2 3 2.8 Ghana 1 1 1 3 2 1.6 Liberia 3 2 2 2 2 2.2 Malawi 1 1 2 2 2 1.6 Category Average 2 1.8 2 2.2 2.2 2 Key: 1=minimally acceptable/useful, 2=acceptable/useful, and 3=very acceptable/useful. - 12 - IFESH Mid-Term Evaluation Report improved. In contrast, Ethiopia coordinates closely with the MoE, host institutions, and USAID Mission. Volunteers have clear roles and feel well-supported. In Ghana, IFESH does not coordinate or communicate productively with the USAID Mission about AEFA, nor with the MoE; however, host institutions find the work of AVEs work within their institutions very useful. In Liberia, the AVEs are working squarely within USAID objectives and are useful to host institutions. But one host Teacher Training Institution (TTI) has spent considerable time mediating interpersonal AVE issues. USAID/Liberia would like more useful program communications. Finally, in Malawi, both the Mission and some host institutions suggested that AVEs could be deployed more strategically. IFESH and USAID are not communicating effectively about this issue. The determinant of success ratings were derived from quantitative and qualitative data collected by the Evaluation Team. Each of the aforementioned categories is comprised of a variety of factors and data including how IFESH HQ manages the program, the effectiveness of the Field Offices, and the systems in place from the first contact with volunteers to alumni status, and the organizational foundation and strategic guidance set by the Board of Directors of IFESH. For background on each country and a full discussion, refer to the case studies in the Appendices B-F. The Evaluation Team conducted research entitled Best Practices for International Volunteer Management, which serves to reference and benchmark findings and recommendations for this Mid-Term Evaluation (refer to Appendix H). The review was compiled in parallel to field data collection and analysis and confirmed many of the findings and supported recommendations of the Evaluation Team. The Evaluation Team offers IFESH proven approaches to volunteer management that could be of particular benefit in areas in which stakeholder interviews revealed weaknesses. 4.0 Filling Gaps with a Purpose to Improve Teaching, Systems, and Education Resources In this section, the Evaluation Team examined the contributions of AEFA. The first section describes the current state, and the following section presents the Evaluation Team’s major findings. 4.1 Current State: Teaching, Systems, and Education Resources How has AEFA strengthened the capacity of teachers, administrators and policymakers to enhance the quality of school teaching methodologies and pedagogical materials at the basic education level through technical aid and the training of teachers, using the main AEFA methods of teacher training, resource centers, donated books, and serving as policy/management advisors? The Evaluation Team found that there three main types of AEFA volunteer assignments:  The Gap Filler  The Gap Closer  The Specialist Expert The Gap Filler is the AVE who is requested to teach courses at a university because a staff member is away for some reason or because there is a high demand for a subject. Usually the curriculum exists for the course so the contribution of the AVE to the course content is optional. The work would be done without the AVE, but perhaps with difficulty for the institution or the course might not be offered temporarily. This is the case, for example, with some AVE assignments in Ghana. In the case of the Gap Closer, the AVE is working him/herself out of a job because their role is to help coach/mentor their replacement or build or improve a program to the point that outside help will not be needed. This is the case with most AVE assignments in Ethiopia, Liberia, and Senegal because AVEs fit into specific programs that they are helping to strengthen. An AVE is also closing a gap when they produce a - 13 - IFESH Mid-Term Evaluation Report syllabus or curriculum, assembling the information and teaching materials that ostensibly can become part of the standard curriculum. The Specialist Expert is the AVE who must have a specific skill set that the tertiary institution or MoE needs for a specific assignment for a special purpose. This role fits the traditional Senior Advisor or Senior Consultant role that education systems in the U.S. and Africa require from time to time. This may be in some area of policy, it can be a technical area such as developing a curriculum policy for science and mathematics, for a special project such as to develop a new major area of study, or a school of education. AVE policy/management support usually involves managing processes to develop and implement the policy or to improve aspects of education management. In the case of Liberia, the Minister of Education said, “The AEFA contribution to capacity development [in staffing the research and drafting of the education reform policy] was the most important one that IFESH could make.” Since Liberia is a post-war recovering country, the need for Gap Fillers is understandable because of severe human resource shortages. But one could question if the placement of a volunteer in a university teaching general courses – not to pre-service or for in-service teachers – fulfills the objectives of AEFA. For each type of AVE assignment, an aspiration of the IFESH proposal was that the work of the AVE would have long-term effect: that is, change the way the system operates or be integrated into the system. Whether AVE contributions were sustainable or transient has to do with the intentions of the host institution when they engage with IFESH. There are examples of how IFESH contributions are institutionalized in some countries that should be models for other IFESH countries. In Djibouti, Senegal, and Ethiopia, for example, AVE duties include teaching English pre-service, developing curricula and resource centers, mentoring and coaching, and conducting teacher observations with feedback and other Continuing Professional Development (CPD) activities for in-service teachers. In each of those three countries, there is a national program for English improvement in schools, with policies and a systemic infrastructure. AVEs are working within these structures and systems. In each country, the work with English language improvement is a priority for the MoE and complementary to USAID interventions with the MoE. Sustainability is more elusive when an AVE is placed teaching a course that is isolated and unconnected with an institutional or system objective. As the supervisor of one AVE said emphatically, “The faculty here is not going to change the way they teach because of an IFESH volunteer,” while at the same time expressing strong appreciation for the “fresh ideas and methods” the volunteer brought to that particular science faculty. The Specialist Expert is the type of AVE who has the greatest opportunity to produce sustainable impact. At the MoE in Liberia an AVE with a solid background as a business analyst became the coordinator of the new Education Policy Reform Act, synchronizing research conducted by all the universities, working groups in all education sub-sectors, numerous consultative meetings to debate, revise, and ultimately, a National Conference on Education, for final discussions and inputs before legal promulgation. In this instance, through the policy/management support of the AVE, the impact of AEFA in Liberia is on the entire system, for years Djibouti Case Study Highlights  AVE Assignments: o Policy and management support to the National Teacher Education Training Center o Capacity development of the CFPEN’s M&E activities o In-service teacher training/classroom observations o Capacity development of the English Language Instruction Unit  LVEs: None  IFESH Aligned/Complementary to USAID and MoE: Yes  Coordination of Outcomes between MoE, Host Institutions, and IFESH: Medium  LNGOs: None  Donated Books: None  Small Grant Funds Distributed: None  Volunteer Satisfaction: Medium Refer to the Case Study for Djibouti in Appendix B. - 14 - IFESH Mid-Term Evaluation Report to come. The Minister of Education has high expectations for another equally talented policy advisor after two successful years. Not all AVEs playing policy/management roles were equally successful. An example is the TTI manager with whom an AVE was assigned to work who did not think he required management assistance, contrary to the view of the MoE. Sometimes these placements have resulted in under-utilized AVEs. In this case the AVE fully utilized his time providing in-service teacher development support. Several AVEs’ job descriptions have combined teaching with serving in a policy/management advisory capacity. Sustainability is also an issue in the area of providing educational resources, a key component of the AEFA strategy, through TRCs and donated books. AVEs have established numerous TRCs. The Evaluation Team visited a TRC established by a volunteer under the predecessor TFA program at a TTI in Ghana that was still flourishing for two reasons: 1) teachers required Teaching and Learning Materials (TLMs) before going out to do their mandatory practice teaching, and 2) the TTI allocated staff to manage the center in lieu of part of their teaching load. Some TRCs the Evaluation Team visited face the question of who will sustain it after the AVE departs. Book donations have their own sustainability issues. A book that gets into a child’s hands at a school can produce benefits indefinitely. The challenge is being sure the book is not sitting unused on a shelf. IFESH best practice for book donations appears to be when there has been active involvement of beneficiaries before books are shipped. This was the case in Ethiopia and Nigeria. In Liberia, where the U.S. government made a commitment to provide millions of books because of the huge TLMs scarcity after the war, there are large variations in how well recipient schools utilized the books, according to Evaluation Team interviews and site observations. In Ethiopia, a new university that worked with the IFESH Field Office to specify the books they needed says they are fully utilizing the donated books. Nigeria reports that they formed committees representing parents, receiving schools, and administrators to coordinate the titles and distribution of the donated books, which contributed to a satisfactory outcome. 4.1.1 AVEs and Innovation AVEs are producing three types of education innovation in their assignment workplaces, products, processes, or organizational innovations that are new to their school or institutional setting.  Product Innovation – In the education sector, a product innovation can be a new or significantly improved curriculum or new educational software.  Process Innovation – Process innovation involves a new or significantly improved production or delivery method. This method can be a new or significantly improved pedagogy or other educational technique. Ethiopia Case Study Highlights  AVE Assignments: o Advisors to English Language Improvement Program (ELIP) o Community schools projects; girls, reading, and CPD o Staff science and health teaching positions at universities o Advisor to final stage of establishing new School of Social Work  LVEs: Yes  IFESH Aligned/Complementary to USAID and MoE: Yes  Coordination of Outcomes between MoE, Host Institutions, and IFESH: High  LNGOs: Yes  Donated Books: None  Small Grant Funds Distributed: o Rural schools for desks, science kits and books o Urban schools for income generating projects to feed OVC or keep them in school o A university to fund LVEs to conduct CPD workshops to improve quality of surrounding schools  Volunteer Satisfaction: High Refer to the Case Study for Ethiopia in Appendix C. - 15 - IFESH Mid-Term Evaluation Report  Organizational Innovation – This innovation involves introducing a new organizational method in business practices, workplace organizations, or external relations. In education, this method can be, for example, a new way of organization of work between teachers, or organizational changes in the administrative area.16 Examples of AVE innovations at their assigned work places include:  Compiling new syllabi and curricula – All AVEs reported that they developed the syllabi for the courses that they teach in Liberia, identifying information sources including textbooks, and creating accompanying learning materials. In Nigeria, at Sa’adatu Rimi College of Education an AVE worked in collaboration with the LVEs to develop programs on skills for teaching reading, writing, and math which includes an in-service teacher training workshop, development of instructional materials, a training manual, and a teacher’s handbook. This new material supported the National Curriculum, which is important for sustainability.  Establishing academic offerings – In Nigeria, an AVE assembled an Early Childhood Care and Education (ECCE) syllabus, using a template developed by a U.S.-based IFESH partner, Bennett College. It is being adapted for appropriate culture content, to then be used in Nigerian Primary Schools. At African Methodist Episcopal University (AMEU) in Liberia, one of the main assignments of one AVE is to help establish the College of Education, researching and helping to write the business plan. Helping to create the program major in English was one responsibility of an AVE at Cuttington University, also in Liberia. In support of the National Curriculum, an AVE at the University of Liberia has brought new techniques in production of TLMs to the standard course that is mandatory for all education majors. In Ethiopia, AVEs over a period of years have assisted in establishing the Department of Social Work at the University of Addis Ababa. At another Ethiopian University, an AVE is assisting peers to create a peer-reviewed public health journal.  Devising diagnostic tools – A classroom observation tool was developed by an AVE in Ethiopia who aims to build a scientific classroom observation/feedback system within the MoE to improve the quality of English teachers. In Liberia, the standard university admissions tests based on Test of English as a Foreign Language (TOEFL) and the International English Language Testing System (IELTS) have proven inappropriate due to the country’s history. An AVE developed a modified version following the structure of those tests, to be tested, modified, and then used to provide writing support at a TRC established by IFESH volunteers.  School management practices – At the Kakata Rural Teacher Training Institute (KRTTI) Demonstration School, AVEs have worked with management and teachers to change sign-in and reporting procedures, reduce teacher absenteeism, and improve accountability.  Monitoring and Evaluation (M&E) mapping tool – A tool was developed for the regional teacher training center in Senegal with the aim of highlighting the efforts that are the most effective and those that need adjustment. Ghana Case Study Highlights  AVE Assignments: o Teacher training o Staff science and English teaching positions at universities o TRC and TLMs support  LVEs: None  IFESH Aligned/Complementary to USAID and MoE: Mixed  Coordination of Outcomes between MoE, Host Institutions, and IFESH: Low  LNGOs: None  Donated Books:14,122 to schools, libraries, and training colleges  Small Grant Funds Distributed: None  Volunteer Satisfaction: Medium Refer to the Case Study for Ghana in Appendix D. - 16 - IFESH Mid-Term Evaluation Report  Assembling diversified TLMs – Student teachers in the Instructional Materials and Production course at the University of Liberia produced educational portfolios that they could carry with them into Liberian schools, consisting of a variety of instructional materials including assessments, tests, quizzes, games, and West African Examinations Council (WAEC) Exams. “The AEFA volunteer taught [university] students to create materials for primary and secondary school classes so that teachers could teach, even without books,” praised a Dean at the University of Liberia. 4.2 Major Findings: Teaching, Systems, and Education Resources These are the major findings that arose after an analysis of the data collected through stakeholder interviews: 1. Stakeholders affirm the quality of AVEs – IFESH volunteer competency is affirmed by the views of host institutions and the Missions. In the field, interviewees from MoEs and host institutions were most familiar, apart from IFESH, with volunteers from Peace Corps, Volunteer Service Overseas (VSO), and Japanese International Cooperation Agency (JICA). With very minor exceptions, in comparing the volunteers, they said that IFESH volunteers were the most highly qualified and had the most experience. “AEFA is Peace Corps for professionals,” observed a USAID official in Malawi. This finding substantiates IFESH priding itself on recruiting highly qualified volunteers. As the AVE chart for 2010-2011 shows (refer to Appendix I) an impressive 73% of AVEs have Masters or Doctoral level degrees and 30% of the 41 volunteers hold Doctoral degrees. As for experience, 51% of AVEs had more than ten years teaching experience. Even though the Cooperative Agreement was signed close to the beginning of the school year in 2009, the figures were about the same for AVE qualifications although the proportion of volunteers with more than ten years teaching experience was a little less at 48%. 2. Integrating crosscutting themes works – Gender equity, HIV/AIDS prevention and mitigation, and community involvement have been a part, in some way, of the work of nearly every AVE, especially gender equity and HIV/AIDS. Since many AVEs lecture at the university level, most do not interact with Parent Teachers Associations (PTAs). Most of the gender and HIV/AIDS work has been in the form of workshops. In a country like Liberia with extreme gender imbalances in education access and the teaching corps, there needs to be special gender interventions, which IFESH could possibly address. Also, if a more comprehensive plan were put in place in each country, AVEs could work with MoE officials to see if these topics could be integrated into the teacher training curriculum. 3. AVEs are the only source for CPD for some teachers – Even though the numbers are not major in the context of all the teachers in an entire country, for many teachers in most AEFA countries, the CPD training would not have been provided had AVEs not been there. Liberia Case Study Highlights  AVE Assignments: o Teacher training o Staff math and English teaching positions at universities o Policy support at the MoE o TRC and TLMs support  LVEs: Yes  IFESH Aligned/Complementary to USAID and MoE: Yes  Coordination of Outcomes between MoE, Host Institutions, and IFESH: Medium  LNGOs: Yes  Donated Books: 20,319 to schools, universities, and training colleges  Small Grant Funds Distributed: o Women’s literacy o Support community schools o Agricultural support project at TTI  Volunteer Satisfaction: Medium Refer to the Case Study for Liberia in Appendix E. - 17 - IFESH Mid-Term Evaluation Report 4. AEFA has exercised programmatic flexibility in the post-conflict environment of Liberia – IFESH programming has been flexible, which is appropriate, in dealing with the changeable environment of post-conflict Liberia. For example, not all AVEs in Liberia are directly engaged in teacher training at TTIs, and not all AVEs at the three Liberian universities teach education majors. However, because of the dire human resources shortage in this post-war recovering country, AVEs are helping to produce the graduates who are in short supply in all fields; some of them may go on to staff the education faculties. This is also the justification for working with private universities. 5. Host institutions are requesting increasingly specialized AVEs for teaching and policy/management advisor assignments – As one MoE official said, “Just because you speak English doesn’t qualify you as an English teacher.” Proficient use of English has become central to economic development plans in many African countries, in addition to being the foundation for academic performance in other subjects. Several host institutions identified the need for AVEs with specific qualifications and experience in English instruction, reading and writing. For the policy/management advisors, AVEs were being called upon to bring practical as well as theoretical knowledge of, for example, assessment of teacher and student learning in Djibouti. 6. Sustainable impact can only occur when AEFA activities are coordinated with and complement those of the MoE and USAID – Many AVEs offer workshops that have excellent content and respond to teacher needs, such as a series constructed in Malawi this academic year. However, attendance is optional and there is no surety of institutionalization within the teacher training programs at the host institutions. The more desirable setting is like that in Senegal this year, where the AEFA program complemented the in-service teacher training program of one university to deliver ten mandatory workshops for 493 newly recruited teachers trained in classroom management, evaluation, and lesson planning. 7. AVEs placed in policy/management roles are few but have strong sustainable impact potential – Matching the prior expertise of an AVE to the assignment with a clear scope of work and a supervisor who wants the policy/management support appears to determine the success of the AVE in navigating the constraints of being “an outsider” to take advantage of opportunities to influence systemic and sustained impact through transformed policies and/or management practices. Systemic transformation is a primary objective of AEFA, as stated in their technical application. 8. Stakeholders differ on the value of book donation utilization, appropriateness, and cost effectiveness – Sentiment among IFESH staff, AVEs, and USAID officials is that the utility of book donations in some cases is that they are not useful, and not always subject/age appropriate. Said one USAID official, “I wonder whether the benefits of donated books are worth the taxes and the time spent on them as compared to developing local capacity.” This sentiment was echoed in more than one country by USAID officials as well as IFESH Country Representatives. In both Ethiopia and Nigeria, however, the IFESH Country Representatives worked out selection and distribution plans with beneficiaries so that recipients secured books that they needed. The coordinator of the massive Malawi Case Study Highlights  AVE Assignments: o Domasi College of Education o Chancellor College o Malawi Institute of Education o CPD Workshops  LVEs: None  IFESH Aligned/Complementary to USAID and MoE: Mixed  Coordination of Outcomes between MoE, Host Institutions, and IFESH: Low  LNGOs: None  Donated Books: None  Small Grant Funds Distributed: None  Volunteer Satisfaction: Medium Refer to the Case Study for Malawi in Appendix F. - 18 - IFESH Mid-Term Evaluation Report university expansion program in Ethiopia found the new books that the universities received very useful. 9. Some AVEs lecturing and providing technical assistance at universities are not supporting the AEFA goal of providing support to primary and secondary school teachers and education administrators – directly or indirectly – The goal of AEFA is to provide support to primary and secondary school teachers and education administrators. AVEs in some AEFA countries are working in universities lecturing in departments that do not contribute directly or indirectly to the development of the basic education system, though they may be helping to strengthen aspects of tertiary education. 10. AEFA schedules are out of sync with some host institutions – Two out of three volunteers working at the two Rural Teacher Training Institutes (RTTIs) in Liberia have been unable to serve as trainers in the Liberia Teacher Training Program (LTTP) as was planned because they could not undergo the mandatory training program due to the time they arrive in-country. Fortunately, they have been involved in CPD and one-on-one coaching on critical school management issues like absenteeism at the demonstration schools attached to the institutions. Similarly, at Cuttington University in Liberia and Cape Coast University in Ghana, the AVEs also did not arrive at the beginning of the school year. 11. IFESH work planning lacks continuity and burdens the resources of host institutions and IFESH – Many of the innovations by AVEs described above clearly required more than one year to execute. Fortunately, six of the eight AVEs in Liberia returned for a second year, along with many who renewed in other countries. A repeated request from host institutions was that IFESH plan for multiple-year projects, and the volunteers come for two-year stints, closer to the norm for other volunteer organizations. An AVE who continues is cost efficient on many fronts; for example, less recruitment and training resources expended by IFESH making AEFA more cost effective. There is also less effort for the host institution where housing is a scarce commodity that the host institution must allocate carefully. Competition over housing with local staff and other volunteers was so intense at one host institution that the administrator declared he would prefer to deal with one volunteer organization, rather than all, to reduce the management burden on him. 12. There is insufficient planning for continuity of policy/management support assignments – Apart from if the assigned AVE returns, there is no evidence of instructive documentation or steps to hand over efficiently to a successor AVE. In this type of support, a break in service can inhibit the success of a specific project or slow down implementation. 13. AEFA is inconsistent in constructing and monitoring a program for each country with outcomes and a work plan – The AEFA country programs in Ethiopia, Liberia, Nigeria, and Djibouti, to varying degrees, have objectives and outcomes for the combined efforts of some or all of the AVEs over a period of one or more years associated with coordination with the USAID Mission and the MoE. In contrast, Malawi, Ghana, and Kenya AEFA country programs are not planned around strategic outcomes. Tangentially, a function that appears to be missing in AEFA is a person and a system to oversee and facilitate such program coherency and continuity. 14. Establishing public/private partnerships is progressing slowly as an IFESH mobilization strategy – Other than in Nigeria with Intel Corporation, this strategy has not borne fruit in any IFESH country. Even in a tight economy, corporations are operating in Africa and funding programs. This is a strategic challenge in terms of corporate image, program focus and performance, fundraising strategy, and other factors. 15. The use of LVEs is not sufficiently contributing to building AEFA sustainability – After a slow start, four AEFA country programs have begun to use LVEs to assist with delivering CPD training. In most cases, they are instructors for workshops who receive a stipend. While capacity is sometimes being built in local institutions to continue this type of training, in some cases it already exists so that the only input is financial. - 19 - IFESH Mid-Term Evaluation Report 16. Knowledge management and sharing are minimal – There is no visible system to gather, store and disseminate the innovative products, processes and organizational changes from the labor of the talented AVEs that AEFA assembles each year, interacting with their African counterparts. On a practical level this information continuity is needed for the next AVE, the host institution, and the education system as a whole to build on as well as to contribute to the body of knowledge about education in Africa. 5.0 Strengthening LNGOs and Distribution of Small Grants How effective has AEFA been in providing technical assistance and knowledge to increase the capacity of Local Non-Governmental Organizations (LNGOs) to implement sustainable development programs, primarily in the area of education? In this section, the Evaluation Team focuses on the contributions of AEFA to strengthening LNGOs and distributing small grants. The first section describes the current state, and the following section presents the Evaluation Team’s major findings. 5.1 Current State: LNGOs and Distribution of Small Grants In Senegal, a $3,000 grant was administered to the Committee Against Violence on Women for building training and sensitivity workshops. This is educational though does not address academic or vocational skills as is the case in other grants. Each Country Representative individually makes decisions on the allocation of small grants. There are no common formal guidelines or monitoring systems. It is not evident if these country by country practices are in line with the spirit of the Cooperative Agreement. Nearly every AVE worked with a LNGO in 2009-2010. This year, 2010-2011, some understood the IFESH Program Director to say to minimize outside activity and concentrate on teacher training. This instruction needs to be clarified before AVEs are assigned for next year. There is minimal feedback on the range and types of experiences AVEs have had and to what extent their activities have built capability in LNGOs. Also unanticipated is the fact that the grants are being made to academic institutions for costs related to CPD training programs. 5.2 Major Findings: LNGOs and Distribution of Small Grants These are the major findings that arose after analyzing data from interviews with IFESH staff and LNGO beneficiaries, and reviewing records on small grants. 1. Volunteer engagement with LNGOs varies – Although AVEs are advised at the Pre-Departure Orientation Workshop (PDOW) in Scottsdale, Arizona that they should all be engaged in a LNGO activity, messaging in the field is not consistent. For example, AVEs in Liberia were advised to reduce their LNGO level of effort. 2. Small grants distribution is not solely dedicated to LNGOs – Most small grants appear to go to schools and tertiary institutions for a variety of types of projects, rather than to educational LNGOs, which was articulated in the IFESH proposal within the Cooperative Agreement. Grants to tertiary institutions are usually for the training costs for workshops, including paying their staff to serve as LVEs. Sometimes IFESH has no direct role in the workshops other than funding. 3. The small grants component is neither transparent nor strategic – Since there are no formal guidelines, the small grants process is controlled almost exclusively by the Country Representative in each country. The process is not systematic and does not appear to consistently support AEFA’s objectives. - 20 - IFESH Mid-Term Evaluation Report 6.0 Enhancing Performance ME&R Is IFESH using rigorous monitoring and evaluation systems that produce evidence of results and support program improvements and sustainability? Address Monitoring, Evaluation, and Reporting (ME&R) of the country program as a whole, and from stakeholder perspectives. In this section, the Evaluation Team focuses ME&R activities conducted by AEFA. The first section describes the current state, and the following section presents the Evaluation Team’s major findings. 6.1 Current State: Performance ME&R AVEs are responsible for collecting data and reporting the collected data on a quarterly basis to the Field Office. One Field Office staff member, or the M&E/AVE, is responsible for collecting the data and compiling it. The data is not reviewed. The Country Representative may review the compiled report but does not conduct any analysis of the data. The Country Representative submits the report to IFESH HQ via email. The data is not verified at the country or the HQ level. Through a review of the Quarterly Reports, it is apparent that the information being gathered has changed from one academic year to another as the format and content is not consistent across reports. There is little synthesis or analysis of the data, nor does it address IFESH’s progress in achieving its AEFA targets. ME&R should be placed within the context of an overall country by country framework and outcomes within IFESH specialties that complement MoE and USAID objectives. For example, if AVEs are called on to help with English language improvement in Ethiopia or Senegal, then the outcomes for which they are responsible need to be specified and measured. In that sense, AEFA is like an Indefinite Quantity Contract (IQC) or an umbrella contract, in that each separate country (like a task or project), has its own outcomes and expected impact, in addition to those of the overall contract. This missing element is a large part of the reason for the lack of satisfaction with the ME&R system by stakeholders in fulfilling their needs. Apart from the mainly output indicators that are in the Performance Monitoring Plan (PMP), it needs to measure progress and success in improving recruitment (refer to Section 7.0 Organizational Effectiveness), knowledge management, and the satisfaction of host institutions, USAID, and volunteers. IFESH is on target to meet or exceed most revised targets set in its work plan (refer to the Performance Summary chart), except for training of education administrators, in-service teacher development, and half￾day CPD days. AEFA is not claiming credit for policy initiatives and reforms undertaken. As stated earlier, there are good reasons to review the targets, indicators, and definitions in the context of reviewing whether the definitions were correct or if they should be altered due to the real conditions that exist in AEFA countries. A good example is CPD training workshops in that the number of in-service teachers trained contradicts the data that no CPD training workshops have taken place. This is a situation where a rapid appraisal is in order. Periodic review of ME&R issues can be done by the IFESH HQ M&E staff together with the M&E/AVE in each country. These issues can be addressed in group discussions at the in-country mid-semester AVE workshops. Ideally, this type of review should be joined by relevant stakeholders from the MoE, host institutions, and USAID. This would be an opportunity to discuss the targets that are not being met, the reasons, and develop a plan of action for the appropriate party: AVEs or IFESH HQ M&E staff might examine if the issue is with the indicator definitions, with AVE execution, or if there are other reasons. Minor corrections would be to add the word “trained” to the Education Administrators and LNGO indicators, as described in USAID performance measures used by IFESH. - 21 - IFESH Mid-Term Evaluation Report Another target to query is book donations. As an input, it speaks to progress in replenishing TLMs in IFESH countries in classrooms. However, it does not speak to how the books are being utilized or contributing to learner performance. Pre-post test score improvement is another performance measurement method to be queried. Improvements show subject knowledge gain but not necessarily changes in attitude or behavior. Such changes can be gauged through classroom observations, and ultimately through application of practices that translate into better student performance. Also, taking an average of increases in pre-post scores is not meaningful because of the differences among the tests. AVEs have been advised to devise other methods if not using pre￾post tests. What is useful with pre-post tests is that AVEs are measuring changes in student performance which gives the instructor feedback on the effectiveness of the teaching methods, and students can see what knowledge they have gained. The strategy of having an AVE assigned to ME&R has worked well in that they play a major role in compiling Quarterly Reports for IFESH HQ, where they are packaged together for USAID. In reviewing the Quarterly Reports in the context of this Mid-Term Evaluation, one appreciates that indicators were adjusted at the beginning of Fiscal Year (FY) 2011. How to systemize the transfer of knowledge from the current M&E/AVE to their successor is a concern. A review of the current PMP for the AEFA program reveals that it is not fully being followed, and may be too complex. In Appendix K, the Evaluation Team presents a suggested revision of the PMP that is simplified, and incorporates the performance measures recommended in this Mid-Term Evaluation report. Although the current framework is consistent with the proposal accepted by USAID for the Cooperative Agreement, and the indicators are as prescribed by USAID, the current state indicates that the types of corrections suggested above could improve current monitoring and reporting. Performance Summary for Eight AEFA Countries17 2009-2010 2010-2011 Total* Target** AVEs 33 41 74 115 LVEs 69 111 180 42 CPD Training Workshops - 22 22 100 Teacher Resource Ctrs. - 24 24 28 Subgrants to NGOs/CBOs - 12 12 28 Books Distributed 128,641 134,354 262,995 90,000 Pre-Svc. Teachers Trained M: 4,264 M: 2,165 M: 6,429 F: 4,198 F: 705 F: 4,903 18,302 T: 8,462 T: 2,870 T: 11,332 In-Svc. Teachers Trained M: 3,568 M: 2,894*** M: 6,462*** F: 1,008 F: 934*** F: 1,942*** 15,876 T: 4,576 T: 3,828 T: 8,404 Ed. Administrators M: 248 M: 440 M: 688 F: 97 F: 225 F: 322 4,095 T: 345 T: 665 T: 1,010 LNGO/CBO Representatives M: 822 M: 306 M: 1,128 F: 1,571 F: 131 F: 1,702 5,593 T: 2,393 T: 437 T: 2,830 Users of Resource Ctrs. - M: 537 M: 537 - F: 22 F: 22 5,800 - T: 559 T: 559 Key: M= Male, F= Female, T=Total, - and = no data reported. * Total = The sum of the 2009-2010 and 2010-2011 columns. ** Target = 2009-2010 actuals + 2010-2011 target + 2011-2012 target. For indicators that were not used in 2009, Target = 2010-2011 + 2011-2012 only. *** = the lack of male/female data in the AEFA January-March 2011 Quarterly Report for In￾Service Teachers Trained in Senegal has made these figures estimates. - 22 - IFESH Mid-Term Evaluation Report 6.2 Major Findings: Performance ME&R These are the major findings that arose after an analysis of the data collected through stakeholder interviews, a review of IFESH reports and documents, and referencing pertinent USAID policies and guidelines. 1. TLMs and TRC activities are valuable but need better monitoring to understand impact – The production of TLMs and creation of TRCs are central to the IFESH Education Model. Malawi is the only country with virtually no activity reported in this area. The TRC model also calls for a local committee to be formed for each TRC that addresses space requirements, security, staffing and oversight, and other resources required, which should address sustainability. However, it does not appear that the formation and operations of these committees are monitored. 2. Utilization and impact are not sufficiently addressed in book donation projects – Book donations are output indicators. With book donation projects there is anecdotal evidence that they are effective but there is no other, more substantial evidence. 3. IFESH is not adequately using the monitoring feedback loop – A performance measurement framework is a management tool. The purpose of the indicators is for stakeholders to monitor, analyze variances, and make appropriate adjustments. For example, IFESH Quarterly Reports showed that at the end of the first year the CPD target was not being met in most countries. Other than to report the numbers, there does not appear to be any verification of the data or active engagement with the PMP data by all stakeholders (host institutions, AVEs, Country Representatives, USAID Missions, and IFESH HQ staff) to interpret what they mean and devise a program response. For example, while AEFA is meeting most targets, there are countries like Kenya that are underperforming so there should be a discussion among stakeholders to devise a response. Also, AVEs complain that they never receive feedback on the reports they submit. Looking at data across countries also allows Country Representatives and AEFA program staff to gauge productivity from AVEs. For example, how many workshops should an AVE reasonably plan and execute in a year? Obviously many factors come into play, but if one AVE can do two two-day workshops in a year and another can do 12, there should an examination of the planning process for AVE workloads. In Ethiopia, even though the overall objective is coordinated with the MoE and the Mission, at the beginning of each assignment year, AVEs conduct needs assessments and from those results prepare work plans. Although most performance indicators and targets are set by the PMP, there must be a plan to execute activities to meet the targets. AVEs also need feedback to manage their work towards required outputs of their assignment. 4. AEFA staff and volunteers are not implementing some elements of the existing PMP – PMP provisions that have not been implemented (that could help to improve program performance) include: customer satisfaction feedback, a review of authoritative country education reports by Country Representatives and M&E/AVEs to inform AVE assignments, evaluation of in-country training sessions of AVEs, and forms to capture policy and management technical assistance. Innovations such as products and processes that AVEs introduce are buried in their narrative reports, and are sometimes repeated in subsequent reports. This indicates that the PMP has too many data collection requirements, the data collection and reporting processes can be improved, and/or the volunteers and staff are not able to execute their ME&R duties. 7.0 Increasing Organizational Effectiveness Does the IFESH field team – Country Representative, Field Office staff, and American and Local Volunteer Educators (AVEs and LVEs) – have the tools they need to succeed? Are the necessary organizational structures (people, processes, and technologies) in place to support and sustain AEFA? - 23 - IFESH Mid-Term Evaluation Report The Evaluation Team examined the organizational structure and operations of IFESH as they relate to the AEFA program. The first section describes the current state, and the following section presents the Evaluation Team’s major findings. Organizational effectiveness has to do with how well it performs in achieving the outcomes the organization intends to produce. To achieve outcomes, an organization must have a strategic plan to connect its mission with strategic goals and outcomes, and performance goals and indicators that drive the operations of the organization. Operationally, organizational effectiveness consists of the effective and efficient application of human resources leveraging effective processes and enabling technology. This Mid-Term Evaluation report assesses the operational categories – processes, technology, and people – that impact the AEFA program, derived from feedback to the Evaluation Team from AVEs (current and alumni), LVEs, USAID Missions, host institution representatives, and IFESH staff. The scope of this assessment did not encompass a complete assessment of IFESH; however, since AEFA is the largest program and funding source for IFESH, the Evaluation Team examines the overall strategy, management and operation of IFESH from the perspective of AEFA. 7.1 Current State: Organizational Effectiveness As founder of IFESH, the legacy of the late Reverend Leon Sullivan continues to dominate the operational environment of IFESH. For example, in asking about the absence of a personnel performance appraisal system, the often-repeated response to the Evaluation Team was, “this is the way we’ve always done it.” Such a hold on the organization is akin to the strengths and weaknesses used to characterize family-owned businesses.18 One feature of family-owned businesses is that the staff may not be aligned with the skills/knowledge requisite to successfully perform the responsibilities of the position.19 Staff performance issues relate to management concerns raised by one Mission Director who said that even though she liked the quality of the volunteers, “...I don’t have to be involved with other organizations to the degree that I have to with AEFA.” This contrasts with another senior Mission official who said, “The Government here really likes IFESH volunteers because of their experience…the Country Representative is an institution…AEFA is ready to expand to the next level.” These uneven satisfaction levels indicate lack of quality control by IFESH management. The overall success of the organization depends, to a large extent, upon institutional leadership, including both the President/CEO and the Board of Directors. It was reported that a strategic planning process was conducted three years ago; however, the implementation of that plan is not evident either at the IFESH HQ level or in the field. This indicates that the organization’s leadership including the Board, and the CEO/President has not made the implementation of the strategic plan a priority. Sustainability of AEFA and IFESH is a major preoccupation of the President/CEO, as USAID is currently the primary funder of AEFA. IFESH needs to be engaged with potential donors on a regular basis. It has been suggested – in IFESH’s own Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis – that the location of IFESH in Scottsdale, Arizona inhibits its ability to develop and harness relationships in Washington, D.C., where development work is central. Successful implementation of AEFA is necessary to attract funders for IFESH. For resource development and fundraising, there is a plan to meet the $300,000 benchmark in unrestricted funding (the amount is determined by the Chief Financial Officer (CFO)). The plan as it exists includes targeting large donors, old friends, and smaller donations through past volunteers. It is admirable that IFESH reports that they have been able to cover a $2 million deficit over the past two years. According to the Institute for Volunteer Research,20 the key to the success of an international volunteer organization is proper volunteer management. It provides the foundation for successful volunteer performance and high satisfaction among all stakeholders. Since IFESH’s core mission is the placement of - 24 - IFESH Mid-Term Evaluation Report volunteers into the field, the Evaluation Team focused a great deal of attention in this area. The research maintains that volunteer management practices must be consistently applied throughout the volunteer lifecycle with policies and procedures for each stage. The following are the stages of the lifecycle: 1. Recruitment 2. Volunteer Appointment21 3. Orientation and Training (including In-Service) 4. Deployment 5. Field Support 6. Reporting 7. Performance Evaluation 8. Retention 9. End of Service Refer to Best Practices for International Volunteer Management, Appendix H, for a more detailed discussion of each of these stages. The Evaluation Team does not present findings for Reporting (#6 above) in this section as they have been discussed in the ME&R Section. 7.2 Major Findings: Organizational Effectiveness 7.2.1 Processes A. IFESH strategic guidance and management of AEFA are not overarching priorities 1. There is no strategic plan in place to improve and sustain the AEFA program. This indicates that the President/CEO and the Board are not providing strategic leadership to make AEFA more efficient, effective, and sustainable. 2. Country programs are not developed consistently. It is a commonly accepted practice to have an overall plan for the work of a development organization in a country. There is no standard process and format for developing this plan in AEFA. Some countries have the plan as a result of close alignment with the Mission because of the particular Mission’s planning process for design and implementation. Some volunteers conduct needs assessments and then build work plans upon arrival, others do not. As a result, the productivity of IFESH in each country, and of individual volunteers, varies greatly and is difficult to measure and report. B. Lack of structure and performance goals impair volunteer management efficiency 1. Recruitment 1.1 Most AVEs interviewed found out about AEFA through casual word-of-mouth or online through their own search for a volunteer experience. Few found out about AEFA through any outreach activity conducted by IFESH. 1.2 The current recruitment process as viewed by AVEs does not cast the recruiting net widely enough. 1.3 No AVE alumni are formally involved with the recruitment process with IFESH HQ; however, some AVEs have referred candidates to apply. 1.4 IFESH did not produce recruitment documentation that demonstrated a year round cycle of implementation and results. 1.5 The stated IFESH recruiting process that includes interviews and vetting is not being followed consistently. Several AVEs said they were not interviewed or had brief telephonic interviews, and did not go through a panel selection process, contrary to the stated procedures of IFESH. 2. Volunteer Appointment - 25 - IFESH Mid-Term Evaluation Report 2.1 Volunteer placement success is uneven. According to host institution respondents, IFESH sometimes places candidates who are not good matches for their specifications, even though the AVE is well-qualified. Other times, placements of AVEs are ideal. 2.2 If applicants were better vetted during the recruitment process, they could be appointed prior to expending resources to bring them to Scottsdale, Arizona for the PDOW. 3. Orientation and Training 3.1 Apart from an in-country orientation, each country had a mid-term workshop that most AVEs found useful. Feedback is that those sessions could be even more useful in reinforcing policies and procedures that protect and support AVEs, and for actively assessing impact and training including the mechanical aspects of ME&R. 3.2 AVEs requested more detail about the assignments at the PDOW in order to conduct research and arrange TLMs to take with them. Lack of continuity or handing over responsibilities from one AVE to the next was also cited as an issue by both AVEs and supervisors. 3.3 AVEs are not adequately prepared for the field. Some AVEs said that the orientation materials and later stages did not give them a full picture of what to expect in Africa or from their assignments, this was particularly so for the post-war conditions of Liberia. Other AVEs said that nothing could prepare one for working in Africa. Others said their backgrounds – one had been in Peace Corps and another had been a U.S. Marine – or their temperaments enabled them to by-pass the shock that some applicants would later experience. To a very limited extent, but of concern to host institutions, some AVEs in francophone countries did not have suitable French proficiency to execute their assignments. The majority opinion (of AVEs and host institution representatives) was that the PDOW developed and executed by IFESH HQ personnel is inadequate in preparing volunteers for the multi-cultural working environment in Africa. The lack of local language training was also mentioned. 3.4 Neither IFESH HQ nor any of the Field Offices have established a knowledge management system. 4. Deployment 4.1 AVEs were satisfied with the logistical support provided in getting them to their assignments. Most AVEs were also satisfied with the way they were accompanied by IFESH Field Office staff to their assignment. Some AVEs found a discrepancy between their understanding of their assignment and that of their host institution supervisor. 4.2 Field Office staff have not communicated a need for a support resource at host institutions. As such, when AVEs arrive, their experiences in acclimating to their new environment vary widely. 5. Field Support 5.1 Some AVEs described the Country Representative and Field Office staff as their main support resource. Others had this relationship with someone at their host institution. Frequent communication between these parties was always the main factor in the comfort level of the volunteer, mainly by mobile phones because email is slow and expensive in all AEFA countries. IFESH provides mobile phones to all AVEs. 5.2 In all countries, AVEs are registered with the American Citizen Services to be contacted in the case of emergencies and for security alerts. However, some AVEs were unaware of a formal contingency plan for emergencies. The Evaluation Team found that in countries with more security issues – Liberia, Djibouti, and Ethiopia – there was a more heightened security consciousness, which gave the perception of a different security policy toward volunteers. 5.3 Situations requiring mediation and leadership by the Country Representative occur infrequently. When it does, it is usually related to housing issues or work relationships with AVE supervisors. - 26 - IFESH Mid-Term Evaluation Report The Evaluators found that AVEs were generally satisfied with incident reporting and grievance procedures and that unsuccessful mediation was the exception. 5.4 Housing is a difficult problem for many of the host institutions and they may also be faced with providing housing for a VSO, IFESH, JICA, and/or Peace Corps volunteer. 5.5 There have been some cases where the AVEs received their stipends late. 5.6 A main complaint from AVEs was the lack of feedback after preparing their reports which are rolled up into the Quarterly Reports to USAID. 5.7 IFESH does not create final reports on AVEs. 6. Performance Evaluation 6.1 Generally, AVEs reported that their host institution supervisors were too busy to provide detailed performance feedback. 6.2 Host institution supervisors evaluate AVEs on a quarterly basis by completing a form. The form developed by IFESH HQ contains ratings but does not provide an opportunity for host institution supervisors to provide detailed assessments. 7. Retention 7.1 Six AVEs did not complete their contracts in 2009-2010. For the 2010-2011 period underway, two AVEs have left early; and one did not arrive at post in September 2010 due to illness.22 (Note: Two of the six departures in 2009-2010 were in Djibouti, directly or indirectly related to a tragic car accident where two AVEs were seriously injured.) 7.2 While the majority of AVEs are viewed as highly professional by host institutions, there were instances where AVEs lacked professional discipline; in one instance one AVE did not complete grading exams. Even though rare, these occurrences mar the reputation of IFESH with host institutions. 8. End of Service 8.1 IFESH does not have a formal end of service process (e.g., a workshop that includes debriefing AVEs on what they have learned, thanking them for their service, and inviting their future participation). Peace Corps, in contrast, has an end of service component; research shows that post-program debriefing and follow-up helps volunteers. C. Related IFESH processes and systems do not consistently support improving AEFA performance 1. Some Field Offices were found to be very effective in networking and coordinating activities with other organizations in the field, others were not. While IFESH has name recognition, many host institutions wanted to know more about what IFESH does. 2. While there are a few examples of formal documented procedures for IFESH HQ and Field Offices staff (e.g., an accounting manual and employee handbook), there is varied awareness of the contents and execution. 3. Although AVEs are supplied with a formal IEFA Handbook, there is varied awareness of its contents. Furthermore, each country has developed its own country handbook that varies widely in containing information relevant to volunteer assignments. 4. Field communication with the primary funder of AEFA, USAID, varied greatly. Lack of regular and meaningful communication poses negative implications for USAID Mission support of AEFA. 5. There is no formal employee performance appraisal procedure in place for IFESH Field Office staff except in Ethiopia. 7.2.2 Technology A. Insufficient use of technology within IFESH HQ and its Field Offices - 27 - IFESH Mid-Term Evaluation Report 1. Most AVEs submit their quarterly reports through email. However, there is no use of online technology for reporting or conducting satisfaction surveys for monitoring purposes or for assessments of AVEs or other IFESH personnel. 2. The only technology consistently being used is Skype. 3. IFESH has a blog on their website but it is not regularly used. B. Basic database management not in use 1. There are inconsistencies in records on small grants and the numbers of LVEs between IFESH HQ and records in the Field Offices. 7.2.3 People 1. Some Field Office staff are concerned with the lack of benefits – The benefits received by Field Office staff varied from country to country. 2. Employees do not have performance standards to know what is expected of them and how those expectations will be measured during performance evaluations – Formal performance evaluations are not in place at IFESH HQ. Country Representatives may or may not informally evaluate his/her staff in the field. 3. Responsibilities for the Cooperative Agreement are not appropriately divided and are being overseen by the President/CEO – The Vice President for Research, Program Development, and M&E, the Director of Education Programs, and the Vice President of Finance are all involved in aspects of Cooperative Agreement, and in at least one case share responsibility of the TRCs. This arrangement appears to produce a lack of clarity and incomplete record keeping. More importantly, there is no individual with education expertise to enable comprehensive monitoring of progress towards achieving the goals stated within the Cooperative Agreement and supporting AVEs in the field. 4. The President/CEO is overburdened with the day-to-day functioning of the organization – This micromanagement responsibility impedes her ability to effectively plan for the future. There is no full-time Chief Operating Officer (COO) in place. It is certainly the time for the IFESH Board and President/CEO to strategically address organizational structure effectiveness, harnessing the strengths and overcoming the weaknesses that have been identified. 8.0 Strategic Implications for IFESH and USAID What are the successes, challenges, and opportunities of AEFA as an innovative, sustainable, and replicable model for accelerating education improvement? This section presents a synthesis of comparative organizations to IFESH. The Evaluation Team also provides a Gap Analysis (refer to Appendix J), and addresses the implications for IFESH and USAID, and concludes with major findings. 8.1 Comparative Organizations With a diverse field of volunteer organizations operating in Africa, IFESH must define and market its comparative advantage. According to the Missions and host institution representatives interviewed, they reported that IFESH places more experienced and higher degreed volunteers than does its main competitors (VSO, Peace Corps, JICA, and German Society for International Development (GIZ)). It is important to note that these comparative organizations are much larger and place more volunteers than IFESH. Peace Corps is most immediately looked at as the leading U.S. comparative organization to IFESH. The perception may be affected because the comparison is between an organization that places approximately - 28 - IFESH Mid-Term Evaluation Report 40 volunteers per year and one that places 4,000. The majority of Peace Corps volunteers working in the education field are typically recent college graduates with limited teaching experience. These volunteers are assigned to teach subjects in their specialties (English, Mathematics, Biology, and Science) at the junior and high school level. However, Peace Corps also recruits a limited number of university teachers for work in Teacher Training Colleges (TTCs) and universities. These more senior volunteers are fewer in number. In addition, Peace Corps operates a “Masters International Program” through relationships with a number of universities in the U.S. that permit Master candidate volunteers to serve with Peace Corps while earning a Masters Degree. A recently announced “third-year fellowship” program with a Non-Governmental Organization (NGO), International Relief & Development (IRD), will serve as a model for professionalizing the service of Peace Corps volunteers wishing to extend their tour of duty for a third year of service. The third year Peace Corps volunteers will be assigned to the partner organization serving as a member of the organization’s staff. In turn, the NGO will share in the cost of the volunteer’s third year of service. Peace Corps has numerous partnerships with organizations that can be on a national project level or an international sector level. Recently, for example, Peace Corps signed an MOU with USAID. Peace Corps Response programs most resemble the IFESH model in the field. Peace Corps Response has the ability to operate in any country where a Peace Corps country office exists. In some circumstances, Liberia being an example, Peace Corps Response entered the country as a stand-alone program. All Peace Corps Response volunteers are recruited through a dedicated office at Peace Corps HQ. The Peace Corps Response assignments are determined at the country level between the Peace Corps Country Director and host institution making the request. Only those Returned Peace Corps volunteers who served successfully (or those who are granted eligibility due to certain circumstances), having completed their initial 27-month tour of duty, are eligible to apply for Peace Corps Response positions. Peace Corps Response positions are usually six months to one year assignments and are tailored to the specific needs of the requesting institution. These assignments typically require the volunteer candidate to have a specialized skill and are at a graduate or Ph.D. level. The volunteer would also be expected to possess the requisite language skills. Japan International Cooperation Agency’s (JICA) mission in Africa is based around the concept of "human security." JICA country programs provide assistance to ensure health care, elementary education and drinking water. In addition, JICA places an emphasis on economic growth in Africa. To that end, volunteers work to improve basic industrial infrastructure such as roads and power development, promoting trade and investment, and improving agricultural productivity as well as to promote partnerships with the private sector. Education is a major component of JICA’s overall technical assistance in Africa. It covers primary and secondary math and science teachers as well as teacher training. Volunteer Service Overseas (VSO) works in education in 19 countries in Africa, Asia, the Pacific and the Caribbean. The organization works to strengthen education systems and improve the quality of teaching. Volunteers support improvements in education by working in (TTCs) and with groups of schools on developing teaching methods. They also work within the mainstream education system to overcome the barriers facing marginalized groups and by improving classroom techniques so that lessons are enjoyable. VSO also works with local government offices and MoEs in areas such as assessment, strategic planning, national curriculum development, monitoring and evaluation and national quality standards. German Society for International Development (GIZ, formerly GTZ) is a German Government owned enterprise, established in 2011, that brings together under one roof the German Development Service, the German Technical Cooperation, and Capacity Building International, Germany. GIZ supports the Government in achieving its objectives in the field of international cooperation for sustainable development, and international education work around the globe. GIZ operates in more than 130 countries worldwide with 1,135 development workers, 700 local experts in partner organizations, and 850 volunteers. - 29 - IFESH Mid-Term Evaluation Report 8.2 Gap Analysis and Sustainability To be competitive with other organizations that are doing similar work, IFESH will need to strengthen its operations and management. Those areas needing improvement are depicted in the Gap Analysis found in Appendix J include: volunteer management, procedures, fundraising, and outreach efforts. The Gap Analysis demonstrates where IFESH is currently operating and where it should be to achieve optimal performance. In addition, the Evaluation Team examined the cost per volunteer as one indicator of competiveness. The per-head volunteer cost for IFESH and Peace Corps were calculated. The costs associated with fielding a volunteer involves a variety of factors such as staffing (both at the Headquarter level and field level), stipends, and programmatic costs. For IFESH, the estimate for per volunteer cost is approximately $69,558.23 In contrast, the estimate to field a volunteer at Peace Corps is approximately $42,000.24 IFESH volunteers receive a monthly stipend of $850 (may be between nine and 11 months, depending on length of program in-country), and Peace Corps volunteers receive $6,075 at the end of service plus a monthly living allowance to cover expenses in-country (the amount varies by country, and by location within a country). Although neither estimate is scientific, fielding volunteers through IFESH is more expensive.25 Whether it is economies of scale that drive the lower cost of Peace Corps Volunteers, it may be the overhead costs that make IFESH a pricier option. Should IFESH field more volunteers, the per-head volunteer cost would decrease assuming staffing and other costs remained the same. In times when less funding is available from donors, IFESH must demonstrate visible and measurable impact to justify its cost. IFESH should find a way to narrow the difference between its cost model and that of its competitors to ensure the efficient and effective use of funding. 8.3 Achieving Greater Impact USAID recently released an Education Strategy for 2011-2015. The Africa Bureau has also spelled out the guidelines for its continued investments.26 IFESH will need to examine its operations to ensure alignment as there are key areas that should drive how IFESH conducts its business going forth. One USAID official was emphatic that “IFESH needs to examine what they do, who their customers are, and how to engage more effectively at a local level…the USAID trend is toward strong in-country involvement using host country organizations with strongly evidence-based programs.” This official also expressed that the AEFA program was “too small” in a country where the shortfall of trained teachers was 30,000. In that same country, MoE representatives who were familiar with IFESH said that the volunteers through the AEFA program “trained people better” than other programs; however, they said that going forward, IFESH needed to be more targeted and should coordinate better to “fit into the agenda of the MoE.” More than one USAID Mission felt that they “were not maximizing the use of AEFA volunteers” because of insufficient planning and program alignment between IFESH, USAID, the MoE, and the host institutions. To achieve impact and scale, IFESH may need to select highly-specialized qualified candidates and in larger numbers. Also, IFESH may want to focus on fewer countries, re-examining the way it has spread volunteers among countries. The new USAID Education Strategy 2011-2015 says that USAID seeks broad based impact, going to national scale from local or regional levels. Currently AVEs are dispersed to different host institutions in a country, with perhaps two at one host institution. If IFESH is to show impact, it may need to look at a different approach. For example, IFESH works with demonstration schools at two RTTIs in Liberia. It may want to focus the efforts of two or more AVEs on a school to demonstrate impact using the many innovative techniques the AVEs use. Another option is to spread out to more schools with many more volunteers. The objective is to clearly demonstrate the learning improvement outcomes. The USAID Education Strategy 2011-2015 also says that a country’s capacity and commitment is “absolutely essential” in order to drive impact and expansion to scale. Again, IFESH currently is working with MoEs with such commitment – Djibouti, Senegal, and Ethiopia – in education activities that lend themselves to scaling up. - 30 - IFESH Mid-Term Evaluation Report IFESH is also well-positioned because it is already in countries which fit the criteria of “relative need” in terms of marginalized populations, gender disparities and post-conflict settings. One decision might be, if there are resources to field larger numbers due to need in a given country, to decide if IFESH should be working in fewer countries. 8.4 Major Findings: Strategic Implications for IFESH and AEFA These are the findings from a review of comparable organizations, conducting a Gap Analysis related to sustainability, and looking at factors affecting greater impact for AEFA and IFESH. 1. The competitive advantage of IFESH is under pressure – In many ways, as the Peace Corps seeks greater diversity and creates new programs and partnerships, AEFA differentiation could lessen. 2. The current AEFA design and strategy is too broad – Environmental factors all indicate the need for more focus to produce better, precisely measured impacts. 3. The cost per volunteer, volunteer management, procedures, and outreach efforts impede AEFA/IFESH sustainability prospects – These are the priority performance areas that are weak in the AEFA program. IV. Recommendations These recommendations apply to the AEFA program overall. There are country-specific recommendations in each of the five Case Studies for Djibouti, Ethiopia, Ghana, Liberia, and Malawi, presented in Appendices B-F. IFESH HQ should determine which recommendations to implement that serve their best interests relative to resource and time constraints, in concert with the Field Offices. 1.0 USAID Strategic Recommendations 1.1 USAID might consider having all the volunteer organizations working with USAID (Peace Corps/VSO/IFESH) to form a single council that USAID Missions can buy into. USAID/Ethiopia has this type of model. To do this, USAID could facilitate an Intel Global Development Alliance (GDA) that includes several countries. In line with the Paris Declaration, this coordination could enlist other donors as well. 2.0 IFESH and AEFA Strategic Recommendations 2.1 If IFESH accepts the findings and recommendations in this Report, IFESH should undergo a major strategic thinking and change management exercise to reengineer its business processes to become a volunteer-centered, results-oriented organization. IFESH could leverage the knowledge and experience of MBA interns; e.g., graduate students at the Thunderbird School of Management to assist in such endeavors. IFESH should address the way: a. Employees are managed on a performance basis. b. Financial data is maintained to achieve easy accessibility and transparency. c. Communication with USAID is conducted in a thorough, concise, and informative manner. 2.2 IFESH HQ and Field Offices should prepare annual communications plans to improve organization outreach to accomplish organizational goals (like recruitment and establishing public/private partnerships). IFESH should determine what clear messages need to be conveyed that are program-related. As a part of outreach, in one IFESH country, for example, a host institution representative suggested an annual fair to better inform all existing and potential partners about IFESH. Another idea is to engage the local media and promote the good works IFESH is accomplishing. 2.3 Develop and implement a business plan approach for AEFA and each AEFA country, with outcomes, indicators, and targets. The business plan scope should examine what progress needs - 31 - IFESH Mid-Term Evaluation Report to be made, with what AEFA inputs, in what timeframe, and the desired impact. The approach should go beyond volunteers to project the total resources required. The business plan should identify which volunteers are needed to accomplish the plan. In considering impact, assess which countries and the number of volunteers needed. In other words, revisit the AEFA approach based on this Mid-Term Evaluation. The approach would include a process for the AVE to sit with the Country Representative and the host institution to review the business plan and the work plan. Such a process would also help to minimize communications gaps. 2.4 IFESH HQ and Field Office staff should work in concert with other USAID implementing partners and USG organizations, and coordinate assignments of AVEs where feasible. IFESH could follow other organizations, for example, by arranging a program partnership with the Peace Corps through a formal mechanism such as an MOU. 2.5 In line with the new USAID Education Strategy 2011-2015 which emphasizes selectivity and focus, IFESH should consider identifying education specialization – innovation areas – to become a leader in the field. Based on lessons learned and accumulated intellectual capital consider where there is comparative advantage in: reading, teacher/learner performance, distance education, Information and Communication Technology (ICT) in education, education in post-conflict settings, workforce development programs/technical schools, and professionalization of teachers/lecturers. 2.6 To build competitive sustainability, IFESH should review their cost per volunteer model. 3.0 Teaching, Systems, and Education Resources Recommendations 3.1 In addition to the recruitment of AVEs with specific knowledge and experience in areas of needs for the host countries and institutions, IFESH needs to closely match AVE skills to host institution specifications. 3.2 AVEs should not be placed at universities if the assignments do not further the goal of AEFA, which is to support strengthening the basic education system. 3.3 IFESH should revisit how to utilize the valuable materials developed by Bennett College for Women and Lincoln University and resources available through this collaboration. For example, Bennett College for Women might be a resource for establishing the College of Education at AMEU in Liberia, where an AVE is assigned. 3.4 As soon as possible, based on the lessons learned, IFESH should consult with AVEs, Country Representatives, and LVEs on how to proceed with the CPD component in terms of execution and targets. 3.5 The arrival of the AVEs needs to sync with their host institution academic calendars. The AVEs should start the school year with their peers, and be able to receive any mandatory training required for their placement. 3.6 Pair an AVE with a host country national to enhance sustainability prospects; in some settings, this could help mediate multi-cultural patterns (e.g., gender, business practices, and cultural practices). 3.7 IFESH needs to plan for multiple-year projects by better coordinating the transitions between AVEs. 3.8 Develop a transition protocol between outgoing and incoming volunteers to close the gap between July and September, especially for policy advisor volunteers, so that they can have a hand off, preferably face-to-face and if not possible via a Skype video call. 3.9 Based on lessons learned thus far, develop training for the PDOW and a handbook on roles and responsibilities of the AVE policy/management advisor assignment. Ideally solicit the assistance of an AVE with the experience in this area. 3.10 To improve its branding, IFESH should formalize materials for host institutions that include: - 32 - IFESH Mid-Term Evaluation Report a. Options of how the AVE can serve, e.g., as a Senior Lecturer or Staff Adviser to a Designated Administrator or Department, and b. Emphasis on the concept of helping to bring effective (“modern”) methods and pedagogy into the classroom, institution, and community. 3.11 IFESH should provide oversight and contribute to the refinement of volunteer work plans, implement regular monitoring and feedback to volunteers about their work plans and workload for the year. 3.12 IFESH should prepare an attractive certificate template, signed by the President/CEO of IFESH and to be signed by a representative of the host institution, to present to trainees upon completion of training courses designed and presented by AVEs. 3.13 Review the book donation operations, including the use of previously donated books to date, the supply chain strategy, security, and a cost-benefit analysis. With plans to distribute large amounts of books in 2011-2012 through the AEFA program, IFESH should develop a rationale and plan for any program continuation that takes into consideration these Mid-Term Evaluation findings, and consider whether books should be distributed or not. 3.14 The IFESH Board of Directors and senior management should accelerate public/private partnerships as an innovative resource mobilization strategy. 3.15 IFESH needs to develop a more systematic approach to the use of LVEs, improve collaboration between AVEs and LVEs, and institutionalize CPD training activities to enhance sustainability of the AEFA program. 3.16 IFESH should encourage host institutions to institutionalize TRC efforts through assignment of a TRC Director of a permanent staff member, and with the creation of a student committee. 3.17 Create a knowledge management system to gather, store, and disseminate the innovative products, processes, and organizational changes resulting from the work performed by the AVEs. 3.18 IFESH needs to review the job descriptions of AEFA personnel to ensure inclusion of qualified education specialist skills. This skillset is necessary to establish a system to oversee and facilitate program coherency, continuity, and serve as a resource to Country Representatives and volunteers. 3.19 Develop the job description for policy assignments (either as all or part of a volunteer’s assignment) as fully as possible. Ensure that the counterpart(s) who the volunteer will work with are part of this process. 4.0 Education LNGOs and Distribution of Small Grants Recommendations 4.1 IFESH management should clarify if AVEs should continue to engage with LNGOs, and at what level of commitment. IFESH should document LNGO engagement in AVE job descriptions. 4.2 A task force should be formed to review the processes and experiences in making small grants to assess their strategic relevance to AEFA goals. The task force should include IFESH staff, Country Representatives, and AVEs. Simple written guidelines for soliciting, awarding, and monitoring the awards should be prepared. They should address who reviews and approves grant applications to ensure transparency and competitive consideration of the best use of the funds in each country. 5.0 ME&R Recommendations 5.1 IFESH should structure rapid appraisals of book donation ordering, delivery, and distribution and the CPD component. Country Representatives and AVEs should compare experiences in book donation and CPD efforts so that successful components are replicated in all countries. The outcome should be improved procedures, monitoring and reporting. 5.2 Revise the PMP based on this Mid-Term Evaluation; refer to Appendix K for suggested revisions. 5.3 IFESH should undertake ME&R tasks in the PMP that have been overlooked such as: customer satisfaction surveys, a review of authoritative country education reports by Country - 33 - IFESH Mid-Term Evaluation Report Representatives and M&E/AVEs to inform AVE assignments, evaluation of in-country training sessions of AVEs, and forms to capture policy and management technical assistance. Verification procedures need to be developed, including a reporting mechanism. 5.4 IFESH should consider using email and/or simple online techniques for conducting satisfaction surveys with the Missions and host institutions, and possibly, for assessing satisfaction with volunteers. This would enable IFESH HQ to directly monitor key stakeholders’ satisfaction levels. IFESH HQ would then liaise with the Country Representatives on how to address the feedback. 5.5 Consider establishing a ME&R committee comprised of ME&R AVEs who are returning, newly designated ME&R AVEs, and ME&R staff at the PDOW. This committee would be held responsible for setting a plan to execute and improve ME&R and knowledge management efforts during the coming year. 5.6 Conduct training of AVEs and Country Representatives on ME&R issues identified in this Mid￾Term Evaluation jointly with ME&R staff at the PDOW. Training should include a walk-through of how to prepare Quarterly Reports, and an open discussion on ME&R non-performance issues and exploration of potential solutions. Invite former M&E/AVEs to share experiences. 5.7 Produce useful required instruments: Easy-to-use instruments like sign-in rosters and feedback forms should be drafted with AVEs at the upcoming PDOW to assess how TRCs are utilized and how they can be improved. A simple instrument to capture information for reporting on CPD sessions should also be designed. If small grant awards continue, forms should be prepared for completion by LNGO grant recipients to provide data on beneficiaries (who, how they benefited, and how the number is calculated) for monitoring and reporting purposes. The same instrument could have baselines, targets, and milestones so that there is some sense of the use of the small grant funds before the final report. Each form should include definitions and instructions. 5.8 The monitoring system needs to be improved to gauge an AVE’s suitability to a host institution, how well they are working out in their assignments, and if they are fulfilling responsibilities right up to the end of their assignments. 5.9 Improve the ME&R framework, definitions, and reporting. First clarify the main objectives of AEFA in the particular country. The M&E framework should be constructed around this model. Then match volunteers to specify learning outcomes/deliverables – what it affected. This would accommodate larger numbers of AVEs. 6.0 Organizational Effectiveness Recommendations 6.1 Review the organization’s structure – processes, technology, and people – against this Mid-Term Evaluation report and the Gap Analysis therein, deciding if IFESH wishes to become volunteer￾centered and results-oriented, and take the appropriate steps. 6.2 Conduct a strategic planning exercise, review the organizational chart of AEFA, develop key performance areas, and adjust staffing as needed. 6.3 Re-evaluate what skills/knowledge a Country Representative should have to successfully execute his/her duties. 6.4 Use a country planning approach to develop the country priorities with the Mission, the MoE, and host institutions (preferably at a workshop but also see how others do it), identify the appropriate host institutions for the plan, and then develop job descriptions. Ideally, current volunteers each year should contribute to new descriptions as well as thoughtfully re-write those for returning AVEs. 6.5 Make recruitment a continuous year-long process with a plan open to stakeholders that is monitored regularly. 6.6 Ensure that the vetting process is thorough, including conducting a panel or other in-person interview, and checking references. - 34 - IFESH Mid-Term Evaluation Report 6.7 IFESH needs to create and maintain a recruitment database to properly facilitate better recruitment and matching. 6.8 Consider having each current volunteer prepare a three to five minute You-Tube type video where they answer a series of rapid fire questions such as: “My name is…, I’m from…, My background is…, I’m a volunteer at…, My main job is…, My target is to achieve…, Where I live is…, What I don’t have is…, How I manage is…, My greatest challenge this year has been…, My greatest achievement this year…, My support system is…, My advice to you is…” to inform prospective candidates about life as an AVE. A collage of photos would also be good if available. 6.9 Mobilize alumni to assist with recruiting and advocacy efforts following their service. 6.10 Appoint AVE candidates as early as possible and give them their assignment, which specifies needs. This will enable AVEs to better prepare, and to possibly mobilize contributions for books and supplies ahead of arriving at their assignment. 6.11 IFESH should advertise more widely. Refer to Appendix H for targeted suggestions. 6.12 Begin setting expectations during the recruitment process, and continue to manage expectations as volunteers start their term and during their field experience. 6.13 Include a FAQ sheet with one question illustrating benefits IFESH provides to volunteers and profiles of AEFA alumni in the Recruitment Package. 6.14 Construct the PDOW and in-country orientations based on a survey of AVEs on what should be covered and what was more or less useful to them at both; include returning AVEs and alumni in orientation planning and execution. Depending on feedback from the survey, consider having a shorter PDOW orientation and using the Internet wherever possible. In addition to preparing volunteers for the field, use the orientation for working groups like re-designing how to do in￾service training, changing CPD metrics, writing small grants guidelines, and reviewing how LVEs are working. Be sure to have the knowledge management packages of what the AVE did the previous years, preferably on flash drives. 6.15 At orientation, AVEs should be briefed on the new USAID Education Strategy 2011-2015, USAID Evaluation Policy, the Early Grade Reading Assessment (EGRA) and Early Grade Math Assessment (EGMA), and their country’s MoE and Mission priority education objectives. IFESH should also brief AVEs on all policies and procedures required during their term of service. 6.16 Address the AVE evaluation feedback received during the current evaluation process that states that: a. The PDOW does not adequately prepare volunteers for life in Africa, and b. In-country orientation does not include a local language training specific to volunteer placement. 6.17 Tell volunteers as much as possible about their assignments, the challenges, and rewards of being an AVE, about being a role model, and about living conditions, housing, health, and security considerations in their country of assignment, at both the PDOW and in-country orientations. Also, include as much of this information as possible in the country handbook; ideally it should be provided to volunteers electronically or manually in Scottsdale, Arizona, ahead of their arrival in￾country. 6.18 The USAID Mission, the MoE, and the host institutions should be part of the in-country orientation. 6.19 IFESH should be more proactive with communications, for example: a. Acknowledge receipt of volunteer application during recruitment, b. Follow up with alumni after their volunteer service has ended, c. Acknowledge and respond to quarterly reports from AVEs, and d. Keep open lines of communication with USAID/Washington and the Missions. 6.20 Final reports on AVEs available for reference purposes should include whether they completed the assignment satisfactorily or not, and be certified by the host institution. - 35 - IFESH Mid-Term Evaluation Report 6.21 Adopt best practices to ensure continuity from one AVE to the next (i.e., keeping a site log that is passed from volunteer to volunteer). That gives a continuum of what was done at a site, which are great and horrible contacts, what the background of the site is, efforts that worked or failed, and organizational mapping exercises. A simple survey of demographics and site information is included as part of the introduction to the community and life in the site. 6.22 Establish a formal process to transition outgoing volunteer work to inbound volunteers, including institutionalizing materials that volunteers created so that new ones do not have to reinvent the wheel. AVEs should be reminded about this early as part of the knowledge management system. 6.23 IFESH may want to consider instituting a formal end of service process such a workshop, to include: a reminder of their final appraisal that will be approved by their supervisor, and about an end of service survey that they will complete which will be part of the AEFA performance measurement system. The objective is to ensure that they maintain their professionalism and complete all assignments. IFESH may also want to consider revising the volunteer contract to reflect end of service language. 6.24 Assign a Knowledge Management Coordinator at IFESH HQ. Archive the instruments, syllabi, proposals, TLMs, and policy documents that AVEs produce. Establish a knowledge management system and protocols. In the AVE contract, require that a hard and electronic record of the assignment is left in the Field Office for handing over to the AVE successor and for forwarding to IFESH. Make dissemination of AEFA innovations and lessons learned other IFESH volunteers and programs a task for someone on the AEFA staff. 6.25 The IFESH volunteers in the RTTIs could serve as mentors to the Peace Corps volunteers in the schools since Peace Corps volunteers typically work in junior and high schools (some Peace Corps volunteers also work in the RTTIs). This should be explored during the country planning phase. 6.26 Make contact with Peace Corps in Washington. IFESH and Peace Corps Country Directors have had a close relationship in the past in many posts in Africa. There are a number of synergies between which should be exploited. 6.27 Continue to monitor that AVEs are paid on time. 6.28 IFESH should establish in-country team activities to encourage professional collaboration. 6.29 Consider engaging the host institution in developing a welcoming packet and assigning a “Welcoming Coordinator” to facilitate a volunteer’s transition to the field. 6.30 Utilize the in-country workshops to reinforce policies and procedures (such as safety, evacuation, and medical) and provide training that will have been identified through an assessment of AVE and program needs, such as training on the ME&R system, need for rapid appraisals for monitoring feedback, to support policy advisory work, to reinforce alignment of the IFESH program with MoE and USAID objectives, and to plan for the following year. This is assuming workshops occur during each semester. One workshop might be in conjunction with an IFESH fair to publicize the AEFA program and other IFESH activities. Invite host institutions to attend the IFESH fair to learn more about what IFESH does. 6.31 Policies, procedures, and program aspects should be consistent throughout AEFA countries, using best practices that have evolved, except for country-specific exceptions. For example, there should be a checklist of what all in-country orientations cover, such as registration with American Citizen Services at the U.S. Embassy in-country. 6.32 A single, real-time database containing all records on small grants, book donations, AVEs, and LVEs needs to be managed and kept current. One IFESH HQ staff member needs to be assigned ownership of this database and be held accountable for its accuracy. 6.33 The IFESH Ethiopia Handbook should be a model for other countries, and likewise their evacuation booklet. - 36 - IFESH Mid-Term Evaluation Report 6.34 IFESH should join the Overseas Security Advisory Council (OSAC), and coordinate the involvement of the Country Representatives. All Country Representatives should also approach the U.S. Embassy to convey information about the Warden System to AVEs upon their arrival. 6.35 Review the dispute resolution process, articulating how to record the incident and actions taken. 6.36 Include multi-cultural conflict resolution for AVEs and Country Representatives at the Scottsdale, Arizona and in-country orientations, and training to sensitize recruits to their status as role models. Another option is to role play conflict scenarios at orientations to facilitate appropriate solutions from management and AVEs. 6.37 It is imperative that IFESH volunteers arrive in-country on time to merge into country programs. They will not be able to serve as teachers in the RTTIs if they arrive in-country late. IFESH needs to keep in mind that the comparative organizations (e.g., Peace Corps volunteers) get to country on time, serve for two full years, and may be as qualified as the IFESH volunteer. 6.38 IFESH has to make the case that their volunteers are worth the expense and in some cases inconvenience of requiring housing from host institutions. 6.39 IFESH should provide guidance to Field Offices on how to provide comparable institutional support for AVEs as received by volunteers at Peace Corps, JICA, and VSO. Hosting a volunteer should help reduce the burden of a school director, not cause additional work. Address this with the Country Representatives during the PDOW in Scottsdale, Arizona. 6.40 Review Field Office structures and include a Volunteer Coordinator position, especially in countries with a large volunteer group. 6.41 All information on the AEFA program, from recruitment to end of service, needs to be up to date and readily accessible. 6.42 Explore possible ways to provide basic benefits to Field Office staff. 6.43 For volunteers, the performance appraisal form should be revised to elicit more useful feedback, and should be collected and discussed during the regular monitoring visits to host institutions. 6.44 For IFESH staff, a Personnel Appraisal System should be instituted by IFESH HQ, in collaboration with the Field Offices, with common tasks and key performance areas for similar positions. IFESH HQ’s Human Resources should conduct a 360 degree review to solicit input from the volunteers and other staff members for each staff member’s evaluation. 6.45 IFESH should consider providing professional development opportunities such as training courses to enhance the skills/knowledge of its Field Office staff. 6.46 IFESH could use a full-time COO to provide some management relief to the President/CEO and managing the functioning of the organization. 7.0 Conclusion These recommendations require consideration at different levels within the organization to determine their feasibility and desirability. They are intended to provoke debate and motivation to make the work more gratifying for staff and volunteers, and more productive for IFESH and its stakeholders. Giving all children in Africa the opportunity for a decent education is the goal of IFESH and the dedicated staff, volunteers, and donors who support its programs. This Report salutes this laudable endeavor. 1 Bureau for Africa Vision Statement. Retrieved from http://www.usaid.gov/locations/sub-saharan_africa/vision.html. 2 USAID Education Strategy 2011-2015. Washington, DC: USAID, 2011. Print. 3 Please refer to Appendix L for the Scope of Work (referred to as the Work Plan by the Evaluation Team) dated November 16, 2011. - 37 - IFESH Mid-Term Evaluation Report 4 Please refer to Appendix M for a Conflict of Interest Statement as disclosed by the Evaluation Team. 5 Tracking the Millennium Development Goals (n.d.). United Nations online MDG Monitor. Retrieved from http://www.mdgmonitor.org/country_progress.cfm?c=DJI&cd=262. 6 Education and Population Data (n.d.). World Bank online. Retrieved from http://data.worldbank.org. 7 Ibid. 8 Ibid. 9 Population, Health, and Human Well-Being—Djibouti (2003). EarthTrends. Retrieved from http://earthtrends.wri.org/pdf_library/country_profiles/pop_cou_262.pdf. 10 Ibid. 11 Ibid. 12 Ibid. 13 Ibid. 14 Millennium Development Goals Indicators (2010). UN online. Retrieved from http://mdgs.un.org/unsd/mdg/Data.aspx. 15 Country Cooperation Strategy for Djibouti (2010). World Health Organization online. Retrieved from http://www.who.int/countryfocus/cooperation_strategy/ccsbrief_dji_en.pdf. 16 Innovation: the OECD Definition (n.d.). OECD online. Retrieved from http://www.oecd.org/document/10/0,3746,en_2649_35845581_40898954_1_1_1_1,00.html. 17 IFESH Revised Work Plan; IFESH Quarterly Reports; IFESH AVE Distribution Tables; IFESH AEFA Book Distribution Tables. Received from IFESH HQ. 18 Strengths and Weaknesses of Family Businesses: How Does Yours Measure Up? Retrieved from http://www.familybusinessconsulting.com/resources/strengths.shtml. 19 Overcoming Obstacles in a Family Owned Business (Pros and Cons). Retrieved from http://www.evancarmichael.com/Women￾Entrepreneurs/2923/Overcoming-Obstacles-in-a-Family-Owned-Business-Pros-and-Cons.html. 20 Please refer to Appendix H. 21 For purposes of this Mid-Term Evaluation, the Evaluation Team adjusted the Hiring term to Volunteer Appointment. 22 The early terminations are reported in the AEFA January-March 2011 Quarterly Report. 23 To reach this estimate, the total value of the Cooperative Agreement was divided by the number of volunteers fielded each year: $7,999,257/(33+41+41). The exact line items are in the Cooperative Agreement on page 7. 24 To reach this estimate, the budget for FY2009 was used divided by two cycles of volunteers: $336.1million/(4,000+4,000). The FY2009 budget was used as it is more representative than the spike in the FY2010 budget and increased number of fielded volunteers. The budget number is found on page 3 of the FY2011 Congressional Budget Justification. 25 We were unable to obtain line items from Peace Corps to obtain a more realistic cost to field volunteers in the Africa region. We did receive a programmatic cost per volunteer in the Africa region; the average is $34,000 per volunteer (in FY2011). It would drop to approx. $30,000 per volunteer if Guinea was not included. Guinea is high because of the evacuations. The low country is $15,000 and one of the higher countries is $50,000. Rwanda, for example, is $27,000 per volunteer, and they rely heavily on PEPFAR funding. (Note: The EMA region is higher per volunteer.) U.S. direct hires in the field are paid out of HQ budget. This average also does NOT include the end of service stipend. This equals $6075 per volunteer if 27 month commitment is completed. This would raise the average cost per volunteer to approximately $40,000. This average of approximately $40,000 does NOT include the cost to recruit/place volunteers, which we were not able to obtain. 26 Sub-Saharan Africa (n.d.). USAID online. Retrieved from http://www.usaid.gov/locations/sub-saharan_africa/. - 38 - IFESH Mid-Term Evaluation Report Appendix A: List of Interviewees Stakeholder interviews were conducted in person by the Evaluators, unless denoted by * to indicate a telephone interview. IFESH Headquarters U.S. Based Date Name Title Organization 13 December 2010 Dr. Julie Sullivan President/CEO IFESH 13 December 2010 Mr. Momodou Mambouray Director, Education Programs IFESH 13 December 2010 Mr. Santry Elmer HR, Insurance, Procurement, Security IFESH 14 December 2010 Mr. Chris Marsh Vice President, Finance IFESH 14 December 2010 Mr. Nephi Bushman Assistant Vice President for Finance IFESH 14 December 2010 Ms. Caroline Anderson M&E/Senior Program Officer IFESH 14 December 2010 Ms. Barbara Ewing Program Officer/Recruitment IFESH 14 December 2010 Dr. Emma Ojamueraye Vice President, Research, Program Development, and M&E IFESH 14 December 2010 Mr. Alan Detheridge Board Member IFESH 14 December 2010 Dr. Eamon Kelly* Board Chair IFESH Date Name Title Organization 13 January 2011 Mr. Charles Feezel Education Program Director WCF 18 January 2011 Mr. Ibrahima Ba* Country Representative (Senegal) IFESH 18 January 2011 Ms. Lucy Kithome* Education Specialist USAID/Kenya 21 January 2011 Mr. Chege Waruingi* Country Representative (Kenya) IFESH 27 January 2011 Mr. Pape Sow* Education Team Leader USAID/Senegal 28 January 2011 Ms. Nafisa Ado* Country Representative (Nigeria) IFESH 8 March 2011 Ms. Elice Browne Former AEFA Volunteer IFESH 16 March 2011 Mr. Randy Adams Supervisor, Training and Evaluation Unit Peace Corps 16 March 2011 Ms. Tanya Gipson-Nahman Education and Youth Specialist Peace Corps 16 March 2011 Ms. Eleanor Shirley Program Analyst, M&E Peace Corps 16 March 2011 Ms. Yanick Douyon* Former AEFA Volunteer IFESH 18 March 2011 Mr. Earl Yates Associate Director, Office of Management Peace Corps 29 March 2011 Mr. Larry Blake Chief Administration Officer, Africa Region Peace Corps 29 March 2011 Mr. Brock Brady Consultant Peace Corps - 39 - IFESH Mid-Term Evaluation Report Ghana Date Name Title Organization 24 January 2011 Ms. Cheryl Anderson Mission Director USAID 24 January 2011 Mr. Bob Davidson Mission Education Team USAID 24 January 2011 Mr. Emmanuel Mensah-Ackman Mission Education Team USAID 24 January 2011 Mr. Edwin Afari M&E Team USAID 24 January 2011 Mr. Michael Koffman Country Director Peace Corps 24 January 2011 Mr. Jorge Delgado RSO U.S. Embassy 24 January 2011 Mr. Kwesi Dzidzienyo Country Representative IFESH 25 January 2011 Mrs. Cecilia Koomson Principal Bishop Lemaire Primary School 25 January 2011 Sister Beatrice Hammond Principal Pere Planque Primary School 25 January 2011 Sister Elisabeth Amoako Ahren Principal Our Lady of Apostles College 25 January 2011 Mr. Callistus Mbano AVE Our Lady of Apostles College 25 January 2011 Ms. Cheryl Mbano AVE Our Lady of Apostles College 26 January 2011 Dr. Kweko Monney Head of Department of Entomology and Wildlife University of Cape Coast 26 January 2011 Dr. Patrick Hahn AVE University of Cape Coast 27 January 2011 Mr. Emmanuel Kaba Assistant Registrar Catholic University College of Ghana 27 January 2011 Ms. Cecilia Quansah Principal St. Joseph’s College of Education 27 January 2011 Mr. Isaac Adset-Mensah Vice Principal St. Joseph’s College of Education 27 January 2011 Mr. Raphael Owusu TRC Director St. Joseph’s College of Education 29 January 2011 Ms. Pamela Fisk AVE Catholic University College of Ghana 31 January 2011 Ms. Margaret Odotei Education Program Technical Coordinator WCF 31 January 2011 Mr. Joseph Baffo Accountant IFESH 31 January 2011 Mr. Seth Koranteng CTP Coordinator IFESH 31 January 2011 Ms. Shirley Lamptey Administrative Secretary IFESH 31 January 2011 Ms. Salomey Asirifi Secretary IFESH 31 January 2011 Mr. Emmanuel Asare Acting Director, Teacher Education Ghana Education Service 29 March 2011 Mr. Andrej Kolaja Director of Placement and Data Analysis Peace Corps 5 April 2011 Ms. Cecilia Darkoh* Former AEFA Volunteer IFESH 4 May 2011 Ms. Sandy Ojikutu* Former Mission Education Team in Nigeria USAID - 40 - IFESH Mid-Term Evaluation Report Date Name Title Organization 31 January 2011 Ms. Evelyn Oduro Manager, Teacher Education, In-Service Educational Training (INSET) Ghana Education Service Liberia Date Name Title Organization 2 February 2011 Ms. Teddy Bryan Mission Director USAID 2 February 2011 Ms. Miriam White Mission Education Team USAID 2 February 2011 Ms. Luann Gronhovd Mission Education Team USAID 2 February 2011 Ms. Julia Richards Mission Education Team USAID 2 February 2011 Mr. Othello Gongar Minister MoE 2 February 2011 Dr. Jean Bell Manning President African Methodist Episcopal University 2 February 2011 Mr. Zayzay Miller Program Assistant Peace Corps 2 February 2011 Mr. Alonso Porte Training Manager Peace Corps 2 February 2011 Mr. Chris Gillis Regional Security Officer U.S. Embassy 3 February 2011 Mr. Samuel Sagbeh Principal United Methodist School – Book Donation Site 3 February 2011 Mr. Matthew Rogers Program and Administrative Assistant IFESH 3 February 2011 Mr. John Freeman Accountant IFESH 3 February 2011 Mr. Edward Gboe National General Secretary YMCA of Liberia 3 February 2011 Mr. John Ford AVE University of Liberia 3 February 2011 Ms. Asta Kaba AVE University of Liberia/ African Methodist Episcopal University 4 February 2011 Mr. John S. Plackie Vice Principal of Administration William V. S. Tubman High School – Book Donation Site 4 February 2011 Ms. Emily Gurgbeh Peal CEO Foundation for Women 7 February 2011 Mr. Dennis Chattah LVE IFESH 7 February 2011 Ms. Maryella Matthews AVE Cuttington University 7 February 2011 Mr. Andrew Johnson AVE Cuttington University 8 February 2011 Mr. Daniel Yekeh Vice Principal/ Instruction and Teacher ZRTTI Demonstration School 8 February 2011 Mr. Lawson D. Kpaiwolo Registrar and Teacher ZRTTI Demonstration School 8 February 2011 Mr. Jerome Huey AVE ZRTTI 8 February 2011 Mr. Johnson S. Beyan Director ZRTTI 8 February 2011 Mr. Justin M. Kanneh Academic Dean ZRTTI 9 February 2011 Dr. Frederick Gbegbe Provost Cuttington University 9 February 2011 Mr. John M. Sellu Director KRTTI - 41 - IFESH Mid-Term Evaluation Report Date Name Title Organization 9 February 2011 Mr. Martin L. Poquie Academic Dean KRTTI 9 February 2011 Ms. Margaret (Peggie) Scott AVE KRTTI 9 February 2011 Ms. Vera (Adisa) Beatty AVE KRTTI 10 February 2011 Professor Euphemia Abdullai Dean of Teacher’s College University of Liberia 10 February 2011 Ms. Cecelia Cassey Teacher University of Liberia 10 February 2011 Mr. Richmond Draper Country Representative IFESH 12 February 2011 Ms. Lonette Williams AVE MoE 14 February 2011 Dr. Evelyn Kandakai LVE IFESH Djibouti Date Name Title Organization 6 March 2011 Dr. Harriett Nettles AVE CFPEN 7 March 2011 Ms. Stephanie Funk Mission Director USAID 7 March 2011 Mr. Alpha Diallo Mission Education Team USAID 7 March 2011 Ms. Saada AbdillahiIdriss Mission Education Team USAID 7 March 2011 Ms. Sierra Hutchinson Yessoufou Country Coordinator/AVE IFESH 7 March 2011 Mr. Randall Martin Chief of Party Project Aide 8 March 2011 Mr. Mahdi Isse Secretary General/ Executive MENESUP 8 March 2011 Mr. Soulimain Adoul-Aziz In-Service Training Coordinator CFPEN 8 March 2011 Mr. Ali Abdillahi Guelleh Basic Education Coordinator CFPEN 8 March 2011 Mr. Ali Abdi Director and Supervisor CFPEN 8 March 2011 Mr. Ibrahim Saleh English Pedagogical Advisor MENESUP Ethiopia Date Name Title Organization 10 March 2011 Mr. Jason Fraser Deputy Mission Director USAID 10 March 2011 Ms. Allyson Wainer Mission Education Team USAID 10 March 2011 Mr. Befekadu Gebretsadik Mission Education Team USAID 10 March 2011 Ms. Nwando Diallo Country Director Peace Corps 10 March 2011 Mr. Mamo Mengesha Country Representative IFESH 10 March 2011 Ms. Megan Flowers AVE Hossanna College of Teachers 10 March 2011 Ms. Katheryne Shay AVE MoE 10 March 2011 Mr. Yasabu Berkneh TDP Expert MoE 10 March 2011 Mr. Solomon Worku TDP Expert MoE 10 March 2011 Mr. Theodros Shewarget Teachers and Education MoE - 42 - IFESH Mid-Term Evaluation Report Date Name Title Organization Leaders Development Core Process Owner 10 March 2011 Mr. Berhanu Alebachew Head, Higher Education Expansion and Strengthening Department MoE 11 March 2011 Ms. Ericka Scott AVE Gondar University 11 March 2011 Ms. Meneber Enyew Director Gondar Community School 11 March 2011 Dr. Desseleyn Mengesha Vice President Gondar University 11 March 2011 Mr. Thomas Syre AVE Haramaya University 11 March 2011 Dr. Aklilu Hailemichael Vice President Aksum University 11 March 2011 Mr. Siyoum Adamu Lecturer and Coordinator of the Teacher Development Workshop with LVEs Teacher Development Workshop 11 March 2011 Ms. Birzaf Abebe Principal/School Director Bazen School 11 March 2011 Dr. Tena Alamirew Assistant Professor Haramaya University 11 March 2011 Ms. Jenny Conway VSO Haramaya University 11 March 2011 Mr. Augustine Sesay AVE Haramaya University 11 March 2011 Dr. Abdella Yuya Director Haramaya University 12 March 2011 Mr. Soulimayne Harraye LNGO Yenet Tesfaye 12 March 2011 Dr. Andergachew Getu Department Head Gondar University 12 March 2011 Mr. Alayawe Mamo LNGO Kindu Trust 14 March 2011 Ms. Lillian Gebrewold Program Officer IFESH 14 March 2011 Mr. Assefa Berhanu Principal/School Director Tinsae Birhan Primary School – Recipient of small grant 14 March 2011 Ms. Zewdie Mekonnen Principal Finfre School 14 March 2011 Ms. Aster Kefale Teacher and Chairperson Charity Club, Finfre School – Recipient of small grant 14 March 2011 Ms. Magere Akele Student recipient of small grant to Charity Club Finfre School 14 March 2011 Ms. Seble Yabil Student recipient of small grant to Charity Club Finfre School 14 March 2011 Mr. Berihun Mekonnen AVE IFESH Malawi Date Name Title Organization 16 March 2011 Ms. Selena Mposa Country Representative IFESH 17 March 2011 Mr. Curt Reitsma USAID Mission Director USAID 17 March 2011 Mr. Craig Anderson Deputy Chief of Mission U.S. Embassy 17 March 2011 Dr. Dixie Maluwa-Banda Director of Higher Education MOEST 17 March 2011 Dr. John Collins Mission Education Team USAID - 43 - IFESH Mid-Term Evaluation Report Date Name Title Organization 17 March 2011 Mr. Ramsey Sosola Mission Education Team USAID 17 March 2011 Mr. Chikondi Maleta Mission Education Team USAID 17 March 2011 Mr. Achisomo Kabvala Program Assistant Peace Corps 18 March 2011 Dr. Chris Kalumegna Principal Chancellor College 18 March 2011 Ms. Pamela Fontenot AVE Chancellor College 18 March 2011 Dr. Elizabeth Jackson* AVE Chancellor College 18 March 2011 Dr. Bretta Blanton* AVE Chancellor College 18 March 2011 Ms. Jacynth Clark AVE Malawi Institute of Education 18 March 2011 Mr. Mahtarr Jallow AVE Domasi College of Education 18 March 2011 Mr. George Khaki Assistant Director Malawi Institute of Education 18 March 2011 Dr. William Susuwele-Banda Director Malawi Institute of Education 18 March 2011 Ms. Elias Charewera Principal Domasi College of Education - 44 - IFESH Mid-Term Evaluation Report Appendix B: Djibouti Case Study 1.0 Summary International Foundation for Education and Self-Help (IFESH) is emerging from a tumultuous period following a tragic automobile accident involving two American Volunteer Educators (AVEs) and one Djiboutian counterpart in February 2010. The U.S. Agency for International Development (USAID)/Djibouti Representative and some of the volunteers had to play a significant role in ensuring that the volunteers received the proper medical care and evacuation. In the aftermath of the accident, other issues related to the management of the country program led Djibouti’s Ministère de l'Education Nationale et de l'Enseignement Supérieur (Ministry of Education, (MENESUP)) to question IFESH’s continued presence in the country.4 Findings of the ensuing investigation into the accident combined with USAID/Djibouti and MENESUP’s concerns resulted in a number of significant programmatic and organizational changes for the IFESH program in Djibouti. The USAID/Djibouti Representative worked hard to ensure that MENESUP allowed IFESH to remain in￾country. In a concerted effort to rehabilitate the IFESH brand with MENESUP, host institutions, and USAID/Djibouti, IFESH Headquarters (HQ) implemented recommendations from USAID/Djibouti and established a partnership with the Academy for Educational Development (AED) under their Project AIDE (Assistance Internationale pour le Développement de l’Education). Through this partnership, IFESH now collaborates with AED on shared objectives of supporting the nation’s education reform and meeting specific technical needs expressed by MENESUP and USAID/Djibouti.5 AVEs have played a strong role in the past in English language instruction and Teaching and Learning Materials (TLMs) development at the University of Djibouti, and assisting the English pedagogical unit of the MENESUP. In order for IFESH to make a notable difference and to help USAID/Djibouti and MENESUP meet their strategic objectives, the leadership, organizational structure, and systems of IFESH must be improved, as well as at IFESH HQ, especially in: communication between key stakeholders; recruitment to match Djiboutian requirements; properly orienting and training volunteers; monitoring an AVE’s suitability for meeting a host institution’s needs, how well AVEs are working out in their assignments, and if they are fulfilling responsibilities right up to the end of their assignments. 2.0 Country Context In 1999, the Government of the Republic of Djibouti (GORD) embarked on stabilizing the national economy in order to create the funds and resources necessary for a national campaign to eradicate poverty. An important element of the national strategy was to focus resources on basic education. This need was partly met through the initiation of a national education reform movement.6 Although Djibouti is still classified in the United Nations Development Program’s (UNDP) Human Development Index as one of the world’s poorest nations, ranked 147 out of 169 countries,7 current indicators suggest that Djibouti is on track to achieve the Millennium Development Goals for Education (MDGE) for boys and girls by 2015.8 As Djibouti: Education/Socio-Economic Indices1 Likely to achieve MDG 2 by 2015: Boys and girls alike will be able to complete a full course of primary schooling Yes Net enrollment primary education (%) 44.4% (2009) Primary Completion rates 35.4% (2009) Literacy rate, youth total (% of people ages 15-24) 86% (2002)2 Proportion of pupils starting grade 1 who reach last grade of primary, Both sexes 64.3% (2008) Proportion of pupils starting grade 1 who reach last grade of primary, Boys 64.1% (2008) Proportion of pupils starting grade 1 who reach last grade of primary, Girls 64.5% (2008) Percentage of population under 15 years 36.1% (2009) Population below poverty line (less than US$1 per day) 74.4% (2010)3 - 45 - IFESH Mid-Term Evaluation Report of 2008, Djibouti allocated 8.3% of Gross Domestic Product (GDP) to education.9 According to MENESUP, the primary gross enrollment rate is now 80% while the completion rate is estimated at 72.7%.10 2.1 MoE Priorities Djibouti’s national development framework focuses on the importance of education in directly addressing the country’s other broader development goals. Specifically, every field of intervention centers on five strategic objectives aimed at improving and strengthening education access and reducing disparities, quality of education, institutional capabilities, managerial capabilities, and partnerships. While USAID/Djibouti and MENESUP’s priorities are closely aligned through their collaboration with Project AIDE, MENESUP did express additional priorities in the field of English education which are not directly supported by USAID/Djibouti. Previous AVEs had been serving as English language instructors at the University of Djibouti and as advisors with CRIPEN (Centre de Recherche, d’Informationet de Production de l’Education Nationale). These assignments were curtailed following the retooling of the program and subsequent move to Project AIDE’s management oversight. The USAID Country Representative stated that their education priorities were focused and it was preferable that AVEs work toward achieving the Mission’s strategic education goals rather than having numerous U.S. Government (USG) funded projects that try to be all things to all people. In a report on the Information and Communication Technology (ICT) development in Djibouti, it was noted that MENESUP has made capacity-building a priority for teachers in the use of ICT through the National Education and ICT project.12 In higher education, a key focus has been on producing skilled teachers and encouraging out-of-school youths to get vocational training. MENESUP expressed their intention to continue the development of their distance education capabilities and requested future IFESH support in this specialized field. Currently, one AVE with an ICT background has been collaborating with counterparts, according to the Country Coordinator, at the Dubai Care Center for Digital Pedagogical Resources as well as working with CFPEN AEFA Djibouti Performance Summary11 2009-2010 2010-2011 Total* Target** AVEs 6 3 9 14 LVEs 0 0 0 / CPD Training Workshops - 0 0 / Teacher Resource Ctrs. - 12 12 / Subgrants to NGOs/CBOs - 0 0 / Books Distributed 0 0 0 0 Pre-Svc. Teachers Trained M: 942 M: 0 M: 942 F:1,356 F: 0 F: 1,356 3,738 T: 2,298 T: 0 T: 2,298 In-Svc. Teachers Trained M: 463 M: 82 M: 545 F: 230 F: 51 F: 281 500 T: 693 T: 133 T: 826 Ed. Administrators M: 0 M: 177 M: 177 F: 0 F: 72 F: 72 50*** T: 0 T: 249 T: 249 LNGO/CBO Representatives M: 0 M: 175 M: 175 F: 0 F: 0 F: 0 0 T: 0 T: 175 T: 175 Users of Resource Ctrs. - M: 0 M: 0 - F: 0 F: 0 400 - T: 0 T: 0 Key: M= Male, F= Female, T=Total, - = no data reported, and / = no data available. * Total = The sum of the 2009-2010 and 2010-2011 columns. ** Target = 2009-2010 actuals + 2010-2011 target + 2011-2012 target. For indicators that were not used in 2009, Target = 2010-2011 + 2011-2012 only. *** = The AEFA January-March 2011 Quarterly Report states 400; however to remain consistent with all of the other target figures, 50 was taken from the Revised Work Plan. - 46 - IFESH Mid-Term Evaluation Report (Centre de Formation des de l’Education Nationale) on a limited basis. The Director of CFPEN stressed that building capacity in the distance education area would help them in meeting their mission of providing training to teachers in the field while at the same time respecting the broader decentralization efforts currently underway.13 2.2 USAID Priorities USAID/Djibouti has focused its efforts on increasing access to and quality of basic education, training teachers, strengthening Parent Teachers Associations (PTAs), and developing the information, management, and planning capacity of MENESUP. To improve teacher training, USAID/Djibouti has employed a cascade approach where master trainers are trained, and they in turn train primary teachers and directors throughout the country. Additionally, USAID/Djibouti’s support to MENESUP has resulted in the establishment of a nationwide set of quality and access standards. USAID/Djibouti’s contribution to teacher training began with the provision of technical and material support to CFPEN’s pre-service training program for primary school teachers, and with grants to UNICEF for the development and production of training modules and primary teaching/learning materials through CRIPEN. Through Project AIDE, USAID/Djibouti has been able to extend support to the development of Djibouti’s comprehensive in-service teacher training program, which is now successfully underway with the full involvement of all MENESUP institutions responsible for providing technical support for the improvement of teacher performance at the school and classroom levels.14 2.3 IFESH Alignment with the MoE and USAID In concert with MENESUP, USAID/Djibouti, and Project AIDE, three AVEs directly support MENESUP in the implementation of decentralized primary level in-service teacher training. These AVEs have introduced education innovations in syllabi and programs, and how classes and schools are managed. In addition, the AVEs collaborate with CFPEN’s technical groups and Project AIDE personnel to enrich training module content and delivery for primary school directors, pedagogical advisors, Teacher Resource Center (TRC) staff, and CFPEN’s new Evaluation Team. In the past, AVEs worked with CRIPEN to develop and design English textbooks and teaching materials. IFESH’s contribution to the development of English as a Second Language (ESL) materials and training of English Language trainers and teachers was also noted by MENESUP officials. From the viewpoint of the IFESH Field Office, the visit of IFESH HQ staff to Djibouti helped to articulate IFESH objectives to CFPEN (who had not been included in the initial AED/IFESH partnership discussions) and MENESUP. These discussions touched on the difference in approaches between AED and IFESH. The direct communications between IFESH HQ staff and MENESUP and CFPEN also complicated the task of aligning the IFESH program with USAID/Djibouti. The meetings between MENESUP and IFESH seemed to have resulted in a general agreement between the two that the IFESH role could be expanded. It was assumed by the two parties that the IFESH scope could be broader than that of Project AIDE and USAID/Djibouti’s strategic goals, but still within that of USAID, an idea not shared by USAID/Djibouti. The USAID/Djibouti Representative has pushed hard for IFESH to be part of the Implementing Partners’ Groups that meets every two weeks and for them to contribute to USAID/Djibouti objectives. MENESUP made reference to the fact that they view this year as an “evaluation” year for IFESH. They also stated their expectation that moving forward volunteers placed with them will have been vetted to the same extent as one would expect of a World Bank or other professional placement. 3.0 Improving Teaching, Systems, and Education Resources How has AEFA strengthened the capacity of teachers, administrators, and policymakers to enhance the quality of school teaching methodologies and pedagogical materials at the basic education level through technical aid and - 47 - IFESH Mid-Term Evaluation Report the training of teachers, using the main AEFA methods of teacher training, Teacher Resource Centers (TRCs), donated books, and serving as policy/management advisors? 3.1 Teacher Training In 2010-2011, one AVE was assigned to work in teacher training focused on pedagogical advice and self￾evaluation training. Another AVE was tasked with teacher training focused on ICT; unfortunately, this volunteer was out of the country and not available to be interviewed by the Evaluation Team. With the approval of the Ministry, the AVEs are working with the Pedagogical Advisor for English. The AVEs have helped to conduct a four-day training program for middle school teachers, including classroom observations in Djiboutiville, where they evaluated 35 teachers. 3.2 TRCs and Book Donations One AVE is listed as the TRC support for CFPEN by IFESH HQ. AVEs developed and administered two TRC training sessions through their association with Project AIDE. There have been no book donations to Djibouti under AEFA. 3.3 AVE Policy/Management Support One volunteer was recruited to help with the establishment and capacity building of monitoring and evaluation within CFPEN. The AVE has not been successful in this position due in part to language constraints, which resulted in difficulties building cohesive working partnerships with the individuals overseeing the project and other international expert consultants. The AVE expressed her belief that as a female she experienced a lack of respect working in a traditionally male dominated society. This sentiment was not shared by another female AVE who felt the real problem was more the AVE’s lack of language skills. In her role of providing expert policy support, even with a Ph.D., the AVE did not satisfy MENESUP’s need for practical development and application of multi-level assessment methods and analyses and has since terminated her tour of service returning to the U.S. early. 3.4 IFESH and USAID Crosscutting Themes: Gender Equity, HIV/AIDS, and Community Involvement There are no AVEs currently working on issues related to HIV/AIDS. Community involvement is limited to the AVEs’ primary job assignment. On the other crosscutting theme of gender equity the scope of work of the AVEs is entirely dependent upon the efforts of Project AIDE, their managing partner. One AVE has worked with trainers on gender awareness looking at strategies in integrating gender when constructing modules, which can then be “Djiboutinized.” 3.5 Use of LVEs In line with the spirit of the AED/IFESH partnership and the wishes of USAID/Djibouti, IFESH has not engaged Local Volunteer Educators (LVEs) in Djibouti. 3.6 Use of Small Grants In line with the spirit of the AED/IFESH partnership and the wishes of USAID/Djibouti, there are currently no small grants in Djibouti. 3.7 Mobilizing Public/Private Partnerships In line with the spirit of the AED/IFESH partnership and the wishes of USAID/Djibouti, there are no public/private partnerships in place or being planned by IFESH. - 48 - IFESH Mid-Term Evaluation Report 4.0 Strengthening Education LNGOs How effective has AEFA been in providing technical assistance and knowledge to increase the capacity of Local Non-Governmental Organizations (LNGOs) to implement sustainable development programs, primarily in the area of education? In line with the spirit of the AED/IFESH partnership and the wishes of USAID/Djibouti, IFESH is not engaged in this area in Djibouti. 5.0 Organizational Effectiveness Does the IFESH field team – Country Coordinator, Field Office staff, and American and Local Volunteer Educators (AVEs and LVEs) – have the tools they need to succeed? Are the necessary organizational structures (people, processes, and technologies) in place to support and sustain AEFA? 5.1 Volunteer Recruitment, Performance, and Support 5.1.1 Host Institution Identification/Volunteer Job Descriptions and Placement Process/Host Institution Resource Allocation and Support USAID/Djibouti expressed satisfaction with the recruitment process of the three most recent AVEs. However, upon further reflection it is evident that candidates for service in Djibouti must possess high proficiency in French language skills and should be vetted for this skill prior to being offered as candidates for Djibouti. Arriving in-country in September 2010, the three AVEs had a long lead-in time from September to November as they supported the AED capacity development contract and learned the Djiboutian education system and the workings of CFPEN. The AVEs shadowed a Project AIDE staff member to become integrated into the project. By January 2011, AED was pleased with the integration of the AVEs but was surprised when an official at MENESUP suggested that the new arrangement with IFESH was not working and questioned what value the AVEs were adding to the Djiboutian education system. The MENESUP official softened a bit when the Project AIDE Director and the USAID/ Djibouti Education Program Manager, assured him that the AVEs were indeed following their Statement of Work (SOW) faithfully. After this meeting, MENESUP appeared to be satisfied that the AVEs were working in a more satisfactory way than before. Still, the Ministry official said that “helping out is not good enough” and wanted the SOW for the volunteers to be redone. The Project AIDE Director and IFESH Country Coordinator worked on this, meshing the SOW with the USAID/Djibouti focus on primary schools, capacity development, and teacher training. The AVEs’ host institution, CFPEN, seemed pleased with the possibility that the AEFA program may not be managed by Project AIDE in 2011-2012 because they could see greater value with tasks that they could not do within the current SOW. Subsequently, the IFESH Country Coordinator drafted the revised SOW, which was then reviewed by Project AIDE. To ensure that all partners had a clear understanding of the changes in the AVEs’ SOW, the Project AIDE Director took the unusual step of getting MENESUP and CFPEN to not just review, but to stamp them. The revised SOW is complementary to Project AIDE’s efforts.15 5.1.2 Orientation and Training AVEs found the U.S. orientation too long and felt as though it was a “cookie-cutter” type program repeated from year-to-year with very little effort put into tailoring the training to the specific needs of the volunteer group. The same feeling held true for the initial in-country training they received; in 2010/2011, the in￾country orientation was “abridged” due to an appointment of a Country Coordinator to fill the Country Representative vacancy. Having found the orientation lacking, AVEs suggested that prospective volunteers should be provided with as much information as possible about their assignments: the challenges and rewards of being an AVE, about being a role model, and about living conditions, housing, health, and security - 49 - IFESH Mid-Term Evaluation Report considerations. AVEs suggested that they should receive training on the Djiboutian education system, the French roots, the reform progress, structure of the MENESUP, the USAID/MENESUP partnership history, and AVE assignments and objectives for past and new AVEs. 5.1.3 Volunteer Monitoring and Performance Assessment The methods for AVE assessment are limited to self-review, host institution supervisor review, and Quarterly Reports. IFESH HQ does not require any reports or input from Project AIDE for their own reporting. Performance appraisals of AVEs are mainly anecdotal and qualitative. CFPEN would prefer volunteers with two-year contracts (assuming that they are successful their first year). It was expressed that one year assignments are too labor intensive for the host institution. This was a general theme of all of the host institution meetings in Djibouti. Another request was made to establish clear lines of authority for the IFESH volunteers. Currently, there is no formal “Note de Service” for the volunteers. It is not clear as to whose authority the volunteers operate in the country. Moving forward, it is imperative that this situation be clarified. The AEFA program and the different partners’ roles and responsibilities need to be formalized. It was also noted that CFPEN found the attitude of IFESH a bit “off putting.” When a staff member from IFESH HQ came to Djibouti to post the new volunteers, it was reported that he came into the CFPEN office and barked orders at the staff insisting that an office be provided for the AVEs, and equipped with Internet. This was a bit strange as they felt as if they were helping IFESH by providing the office space. CFPEN had not actively sought out IFESH as a partner and were only informed of their presence by MENESUP. 5.1.4 Safety and Security AVEs interviewed were familiar with IFESH policies and procedures regarding safety and security, and emergency evacuation. AVE involvement in rural areas is rare in Djibouti, unlike in other AEFA countries. Therefore, due to security concerns, travel outside of Djiboutiville is restricted to Embassy/USAID/IFESH vehicles and all must travel on official mission orders. 5.1.5 Office Support to Volunteers (Medical, Stipends, and General) Volunteers were familiar and satisfied with the medical insurance and receipt of their stipends. 5.2 Communications Internal – The IFESH Country Coordinator maintains daily contact with AVEs. They feel supported by USAID/Djibouti and Project AIDE staff. External – Lacking a Country Representative, IFESH is formally represented by Project AIDE in dealings with MENESUP, and USAID/Djibouti. However, the IFESH Country Coordinator and the Project AIDE Director collaborated on project documents which were submitted to USAID and MENESUP in the name of IFESH. Additionally, the IFESH Country Coordinator attended all implementing partner and decision-making meetings. 5.3 Knowledge Management and Sharing/Use of ICT There was very little evidence of knowledge management and sharing efforts on the part of the Field Office. Paper files are stored, and the office operates with one email address. 5.4 Field Office Operations IFESH is currently recruiting for a full-time Country Representative. In the interim, an AVE has been designated as the Country Coordinator. She is being financially compensated for this additional duty. IFESH HQ and USAID/Djibouti are collaborating on the recruitment and screening of applicants. USAID/Djibouti - 50 - IFESH Mid-Term Evaluation Report has questioned some aspects of the hiring process for a past Country Representative as large discrepancies between the two organizations on the scoring of applicants have occurred as well as what could be construed as preferential treatment for some applicants (e.g., flying one into Scottsdale, Arizona for an interview while other equally qualified candidates are not accorded the same opportunity). The Field Office staff comprises the Country Coordinator, an Accountant/Administrative Officer, a Driver, two Security Guards, and a part-time Office Cleaner (twice weekly). Office space is provided by MENESUP on the CFPEN grounds. This has provided closer access to the AVEs’ counterparts. Additional space is provided at the Project AIDE office. 5.4.1 Financial Management The Office Accountant/Administrative Officer maintains the accounts and records and follows established procedures laid out in the accounting manual for preparing budgets and financial reports. Additional responsibilities include communications support between IFESH HQ, IFESH Field Office, partner organizations, and host country nationals. The Country Coordinator has signatory authority for the financial books. 5.4.2 Staff Performance Areas/Assessment Staff performance appraisals are not currently being conducted. 5.4.3 Staff Recruitment/Training With a small staff, turnover is very low in the Field Office. When a need to hire staff arises, the Country Coordinator/Representative will seek high caliber staff through advertising in the paper and talking to sister organizations for any recommendations. Candidates are required to submit an application that includes a CV and references. If approved, the candidate is invited in for an interview. Once staff is hired, there is no training beyond on-boarding the new employee. This process consists of explaining the mission of IFESH and what he/she must perform to get the job done. There is no professional development of staff. 5.4.4 Operational Issues Field Office staff have national medical coverage which includes three months of maternity leave and two weeks of paid medical leave annually. Additionally, IFESH does pay into the national social security scheme. 6.0 Performance ME&R Is IFESH using rigorous monitoring and evaluation systems that produce evidence of results and support program improvements and sustainability? Address Monitoring, Evaluation, and Reporting (ME&R) of the country program as a whole, and from stakeholder perspectives. Monitoring of IFESH is to a large extent a joint undertaking with USAID/Djibouti, MENESUP, and Project AIDE. There was general agreement on the part of the partner institutions that there is a management issue with IFESH. Partners believe that IFESH has a great mission and they support it, but the ability to see what needs to be done is not there. A very frank assessment of the work of AVEs was offered by one of the host institutions. They point out that there have been a number of good AVEs in the past and present who are truly interested in the program and working well with their institution. At the same time, there have been occasions when they had to stop and ask themselves, “why is this volunteer here?” or “how did this person get recruited? What are the actual IFESH recruitment procedures? Are these volunteers vetted before they are sent to country?” It was also stated that on occasion, some of the volunteers have brought more personal problems with them than solutions to their issues. The official stressed the importance of language skills and their being demonstrated experts their specialty areas. He encouraged IFESH to be more robust in their vetting process. - 51 - IFESH Mid-Term Evaluation Report 7.0 Implications for IFESH, USAID, and Djibouti What are the successes, challenges, and opportunities of AEFA as an innovative, sustainable, and replicable model for accelerating education improvement? While the AEFA program in Djibouti has encountered serious challenges in recent times, the program has survived through hard work and aid from supporters on the ground. The liaison with AED, with the intervention of USAID/Djibouti, has provided IFESH a much needed reprieve and has allowed the program to begin the process of refocusing and retooling. With the proper leadership at the country level, IFESH would be well positioned to take advantage of the comparative edge they enjoy in the country. Having relatively few international volunteer organizations operating in the country, IFESH would be able to reasonably expand its footprint in Djibouti, assuming availability of funding and host country and USAID/Djibouti concurrence. 7.1 Comparative Organizations There are not many comparative organizations operating in Djibouti. The Japanese International Cooperation Agency (JICA) is in the process of constructing a center that will serve as an organizational focal point for all of the TRCs in the country. It was mentioned by an official at MENESUP that “in comparison to the JICA volunteers, the IFESH volunteers seem to be of a certain age and as such bring some additional credibility to their positions.” Peace Corps has received a request from the GORD to place volunteers but is not currently planning to open a program in the country. 8.0 Recommendations The following recommendations are specific to the AEFA program in Djibouti. The Evaluation Report has other recommendations applicable to all AEFA countries. The recommendations should be discussed and implemented in coordination with IFESH HQ. Organizational Effectiveness 1. IFESH must develop a consistent, transparent, and respected hiring process that allows strategic partners (funders, e.g., USAID) some involvement in the selection of a new Country Representative. 2. Country Representative candidates should have demonstrated experience/understanding of how to work with USAID. The new Country Representative will need to continue the work to repair the IFESH brand in Djibouti through active engagement with the MENESUP, CFPEN, Project Aide, and USAID/Djibouti. 3. In terms of the Human Resources system, there is no performance assessment system nor is there evidence of a system that would provide documented feedback to a staff member or AVE who is not performing. A performance assessment system for AVEs must be put into place to provide real data for either improving performance or making personnel decisions, thereby providing due process to the scrutinized individual and accountability to the host institution. 4. Review the dispute resolution process, articulating how to record any action decisions taken. Volunteers should not be put in the position of enforcing administrative action against another volunteer. 5. IFESH also needs to be able to provide a comparable level of comprehensive institutional support for their volunteers as do similar organizations (e.g., Peace Corps, JICA, and Volunteer Service Overseas (VSO)), especially when there is a serious incident requiring immediate attention. Recruitment and Placement 6. Recruitment of volunteers for Djibouti must place an emphasis on French language skills. 7. Determine a way to facilitate continuity of volunteer work from one year to the next. - 52 - IFESH Mid-Term Evaluation Report 8. Prospective volunteers should be made aware of as much information as possible about their assignments: including the challenges and rewards of being an AVE, about being a role model, and about living conditions, housing, health, and security considerations. Orientation and Training 9. AVEs should receive training on the Djiboutian education system, the French roots, the reform progress, structure of the MENESUP, the USAID/MENESUP partnership history, and AVE assignments and objectives for past and new AVEs at either the PDOW or in-country orientation. ME&R 10. The monitoring system needs to be improved to gauge an AVE’s suitability to a host institution, how well they are working out in their assignments, and if they are fulfilling responsibilities right up to the end of their assignments. Implications for IFESH, USAID, and Djibouti 11. On its face, an association with CRIPEN to support English language training – for which there is a severe shortage of staff – seems to be a good match for AVEs. While working with CRIPEN at the middle and high school levels does not support the specific USAID/Djibouti mission goals, it would be in line with broader USAID education objectives. 9.0 Methodology and Information Sources The Evaluation Team interviewed two AVEs while in Djibouti (a third AVE was on travel out-of-country). The Evaluation Team interviewed the IFESH Country Coordinator, USAID/Djibouti Representative, USAID/Djibouti Education Program Manager and Education Team member, personnel at MENESUP, and representatives of host institutions. Names for all interviewees are found in Appendix A. The following documents and publications were reviewed for this case study: IFESH. (2009). AEFA – IFESH Technical Application. IFESH: Scottsdale, Arizona. IFESH. (2009). AEFA Quarterly Report, July-September 2009. IFESH: Scottsdale, Arizona. IFESH. (2009). AEFA Quarterly Report, October-December 2009. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, January-March 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, April-June 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Annual Report and Quarterly Report, July-September 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, October-December 2010. IFESH: Scottsdale, Arizona. USAID. (2009). Cooperative Agreement Between USAID and IFESH. USAID: Washington, D.C. USAID. (2009). Djibouti Assistance to Education Evaluation. USAID: Washington, D.C. Government of the Republic of Djibouti (GORD). (2008). National Initiative for Social Development (INDS). Djiboutiville, Djibouti. 1 Tracking the Millennium Development Goals. (n.d.). United Nations online MDG Monitor. Retrieved from http://www.mdgmonitor.org/country_progress.cfm?c=DJI&cd=262. Education and Population Data (n.d.). World Bank online. Retrieved from http://data.worldbank.org. Millennium Development Goals Indicators (2010). UN online. Retrieved from http://mdgs.un.org/unsd/mdg/Data.aspx. - 53 - IFESH Mid-Term Evaluation Report 2 Population, Health, and Human Well-Being—Djibouti (2003). EarthTrends. Retrieved from http://earthtrends.wri.org/pdf_library/country_profiles/pop_cou_262.pdf. 3 Country Cooperation Strategy for Djibouti (2010). World Health Organization online. Retrieved from http://www.who.int/countryfocus/cooperation_strategy/ccsbrief_dji_en.pdf. 4 “Conventions with IFESH” Letter from MENESUP to IFESH HQ, April 2010. 5 AEFA Annual Report and Quarterly Report. (July – September 2010). Scottsdale, Arizona: IFESH. 6 Djibouti: Poverty Reduction Strategy Paper (July 2009). IMF Country Report. Retrieved from http://www.imf.org/external/pubs/ft/scr/2009/cr09203.pdf. 7 International Human Development Indicators (n.d.). UNDP online Human Development Report. Retrieved from http://hdrstats.undp.org/en/countries/profiles/DJI.html. 8 The World Factbook (April 2011). CIA online. Retrieved from https://www.cia.gov/library/publications/the-world￾factbook/geos/dj.html. 9 World Development Indicators (2010). World Bank online. Retrieved from http://data.worldbank.org/country/djibouti. 10 The Evaluation Team conducted an interview of officials at MENSEUP in Djibouti on 8 March 2011. 11 IFESH Revised Work Plan; IFESH Quarterly Reports; IFESH AVE Distribution Tables; IFESH AEFA Book Distribution Tables. Received from IFESH HQ. 12 Excerpted from InfoDev's Survey of ICT and Education in Africa (Volume 2):53 Country Reports (as cited by Hare, 2007). ICT in Education in Djibouti. Retrieved from http://www.infodev.org/en/Publication.398.html. 13 Ibid. 14 Djibouti Assistance to Education Evaluation (2009). USAID Report. USAID: Washington, D.C. 15 The Evaluation Team conducted an interview with an official at Project AIDE in Djibouti on 7 March 2011. Project AIDE was recruited to work with IFESH in 2010-2011 as a way to ease IFESH back into acceptance at the Ministry. - 54 - IFESH Mid-Term Evaluation Report Appendix C: Ethiopia Case Study 1.0 Summary The International Foundation for Education and Self-Help (IFESH) has traditionally worked closely with the U.S. Agency for International Development (USAID); therefore its country program is closely aligned with the Ministry of Education (MoE) as well, because of their close partnership with USAID/Ethiopia. A best practice and sustainability example observed by the Evaluation Team is the Higher Diploma Program (HDP) where IFESH volunteers initially headed these programs at Universities and Teacher Training Colleges (TTCs), or worked alongside Ethiopian counterparts, until outside support was no longer needed. A similar model was begun with the English Language Improvement Program (ELIP). According to the 2010 Millennium Development Goals (MDGs) Report Card compiled by the Overseas Development Institute, Ethiopia “provides key lessons in improving access to education, having raised primary school enrollment to 15.5 million, an increase of over 500 percent.”2 But the MoE realizes that the overall increase in the enrollment ratios has come at the expense of educational quality and has embarked on a program to improve education quality since 2009. The foundation of the quality improvement program is to enhance teacher skills and develop more qualified teachers, along with putting appropriate Teaching and Learning Materials (TLMs) in the hands of teachers and students. The American Volunteer Educators (AVEs) are an important resource for the MoE in strengthening teachers’ Continuing Professional Development (CPD) including workshops in: continuous assessment; reading skills development; conducting classroom observations and feedback; and adding topics like integrating HIV/AIDS and gender into the curriculum and gender practices to the classroom. AVEs greatly augment the MoE pre-service and in￾service training delivery, reaching many teachers who would not have otherwise received the training due to lack of staff. IFESH is successful due to a combination of factors: the MoE is clear about what its goals are and the value IFESH can add, as is USAID/Ethiopia, and the IFESH Country Representative and Field Office staff, and organizational structures are effective. The way that IFESH systems are implemented, along with the ones Field Office staff in Ethiopia has created – such as staff assessment – are models for other IFESH operating units. 2.0 Country Context “Best practice to address the challenges of achieving the MDGs” is how the United Nations Development Program (UNDP) describes how the Government of Ethiopia has invested in both physical and human capital formation in line with the objective of poverty eradication and bringing about social development in Ethiopia. “By spending more than 60% of its total expenditure on poverty oriented sectors, such as agriculture, education, health, water and road development during the last seven years, the government has maximized its efforts and shown the highest level of dedication to bring about pro-poor economic growth.”3 Admirably, Ethiopia is well on track to achieve universal primary education, with achievements in terms of higher gross enrollment ratios, as well as increases in the total number of primary and secondary schools in the country. The Gross Enrollment Rate (GER) for primary school (grades 1-8) reached 95.9% (93% for Ethiopia: Education/Socio-Economic Indices1 Likely to achieve MDG 2 by 2015: Boys and girls alike will be able to complete a full course of primary schooling Yes Net enrollment primary education (%) 82.7% (2009) Primary Completion rates 55.2% (2009) Literacy rate, youth total (% of people ages 15-24) 44.6% (2008) Proportion of pupils starting grade 1 who reach last grade of primary, Both sexes 38.1% (2008) Proportion of pupils starting grade 1 who reach last grade of primary, Boys 35.4% (2008) Proportion of pupils starting grade 1 who reach last grade of primary, Girls 41.4% (2008) Percentage of population under 15 years 43.5% (2009) Population below poverty line (less than US$1 per day) 39% (2005) - 55 - IFESH Mid-Term Evaluation Report female and 98.7% for male) during 2009-2010.4 The Net Enrollment Rate (NER) stood at 89.3% (86.5% for female and 87.9% for male) for the same year (2009-2010).5 According to the 2010 MDGs Report Card compiled by the Overseas Development Institute, “Ethiopia provides key lessons in improving access to education, having raised primary school enrollment to 15.5 million, an increase of over 500 percent.”6 Achieving the gender equality MDG also appears to be within grasp, but with areas of concern for education planners. The GER for boys increased from 74.6% to 98.7% in 2002-2003 to 2009-2010 while GER for girls at primary level increased from 53.8% to 93% during the same period.7 Due to prior structural history, however, the gender disparity gets wider at higher levels of the educational system. Many more girls enroll at secondary and tertiary education levels now that they are achieving gender parity at the primary level. Rural areas and specific disadvantaged groups have lower enrollments.8 2.1 MoE Priorities The Government of Ethiopia (GoE) is focused on the importance of education in indirectly addressing the other development goals, and makes the issue of quality of paramount importance. The overall increase in the enrollment ratios though has come at the expense of educational quality: the completion rate for first cycle and second cycle primary education is currently at 74% and 46% respectively during 2009-2010. Therefore, in 2009, the GoE initiated a General Education Quality Improvement Program. Objectives included upgrading teacher’s quality and increasing the number of teachers through on the job training and summer training, as well as reducing the pupil to teacher and pupil to textbook ratios. As a result, the percentage of certified teachers in primary education has improved and the share of female teachers is higher than male teachers in both cases. Another education challenge is the nutritional status of many Ethiopian children. The government has implemented a program to support food insecure households with food security and productive safety net programs. This program is an effort to address malnutrition so that it does not affect the performance of children, and to ensure the achievement of universal access to education. The GoE has embarked on an ambitious and radical expansion of its higher education sector: from two federal universities to 22 in just over a decade, with ten regional universities already open or in various stages of construction. Even with this huge expansion of student numbers, the percentage of the available cohort that attends higher education is still low at about 3%, compared with a Sub-Saharan average in 2007 of 6%, according to the United Nations Education, Science, and Cultural Organization (UNESCO).9 The Ethiopian government sees higher education as an important plank in its strategy for social and economic development and one of the ways of spearheading regional identity and autonomy in a country with over 80 languages and dialects. The plan has created an equally huge need for qualified university staff in all fields and especially in the growth plan sectors of agriculture, industry (construction and manufacturing), service sectors, and the social service support sectors of health and education. 2.2 USAID Priorities The portfolio of USAID in Ethiopia is one of the largest and most complex in Africa, for a transforming country that is central to regional stability in the Horn of Africa. Consequently, Ethiopia is one of the top recipients of assistance. USAID/Ethiopia capitalizes on partnership with the Ethiopian Government to tackle poverty and deliver basic public services, especially health.10 USAID/Ethiopia works with the MoE to raise the quality of primary education by improving teacher skills and methods, providing English textbooks, strengthening school-community cooperation, decentralizing planning and management, and extending civic education. In 2009, USAID/Ethiopia helped enroll 1.6 million children in schools countrywide, and trained more than 8,500 primary school principals and community education and training board members in educational planning and management. Programs also focus on increasing access for girls and the disadvantaged and on improving schools in largely Muslim areas. - 56 - IFESH Mid-Term Evaluation Report USAID/Ethiopia’s basic education strategic objective is improving delivery of and access to quality primary education, with an emphasis on early grade reading in target areas. USAID/Ethiopia has identified five objectives in order to achieve its overall strategic objective: 1) improving capacity of teachers in content and methodology; 2) improving management and planning capacity at school at the local, regional and Ministry of Education level; 3) developing and distributing high quality low cost textbooks and other learning materials; 4) enhancing community involvement in delivery of quality education; and 5) increasing access to quality basic education to children and adults in marginalized communities. Also, according to the Fiscal Year (FY) 2010 Congressional Appropriation, continued emphasis remains on monitoring and evaluating progress of the programs.12 2.3 IFESH Alignment with the MoE and USAID Three AVEs were placed in Ethiopia in year 2009-2010, though five slots were available; one of the AVEs left two months early. For the current program year, five AVEs are in place. AVEs are an important resource for the MoE in strengthening their CPD including workshops in: continuous assessment; reading skills development; conducting classroom observations and feedback; and adding topics like integrating HIV/AIDS and gender into the curriculum and gender practices in the classroom. AVEs greatly augment the MoE pre-service and in-service plans which are conceptualized, but not fully actualized, providing services that would not reach many teachers due to lack of staff. “Ethiopia has its plan, they know how they want to get there and what they want their donors to do,” stated the USAID Education Leader in Addis Ababa to the Evaluation Team. Given this clear stance of the Ethiopian government, there is no policy advisory role for AEFA in Ethiopia at the national level. AVE policy contributions feed in for a specific program or subject area such as the English Language Improvement Program. 3.0 Improving Teaching, Systems, and Education Resources How has AEFA strengthened the capacity of teachers, administrators, and policymakers to enhance the quality of school teaching methodologies and pedagogical materials at the basic education level through technical aid and AEFA Ethiopia Performance Summary 11 2009-2010 2010-2011 Total* Target** AVEs 3 5 8 14 LVEs 16 47 63 / CPD Training Workshops - 2 2 / Teacher Resource Ctrs. - 1 1 / Subgrants to NGOs/CBOs - 4 4 / Books Distributed 0 5,950 5,950 20,000 Pre-Svc. Teachers Trained M: 351 M: 314 M: 665 F: 114 F: 126 F: 240 1,905 T: 465 T: 440 T: 905 In-Svc. Teachers Trained M: 449 M: 348 M: 797 F: 73 F: 169 F: 242 1,000 T: 522 T: 517 T: 1,039 Ed. Administrators M: 87 M: 3 M: 90 F: 43 F: 2 F: 45 300 T: 130 T: 5 T: 135 LNGO/CBO Representatives M: 200 M: 34 M: 234 F: 145 F: 34 F: 179 300 T: 345 T: 68 T: 413 Users of Resource Ctrs. - M: 0 M: 0 - F: 0 F: 0 400 - T: 0 T: 0 Key: M= Male, F= Female, T=Total, - = no data reported, and / = no data available. * Total = The sum of the 2009-2010 and 2010-2011 columns. ** Target = 2009-2010 actuals + 2010-2011 target + 2011-2012 target. For indicators that were not used in 2009, Target = 2010-2011 + 2011- 2012 only. - 57 - IFESH Mid-Term Evaluation Report the training of teachers, using the main AEFA methods of teacher training, Teacher Resource Centers (TRCs), donated books, and serving as policy/management advisors? 3.1 Teacher Training An innovative Ethiopian strategy is to make new universities in each region responsible for improving teacher performance. This in essence leverages public infrastructure and human capacity investment at these institutions. USAID Mission personnel observe that an area of potential policy assistance is to help with monitoring, evaluating and improving this policy; for example, what the student performance consequences are of sending the worst teachers (10th grade dropouts) to teach first and second grade, rather than those more highly trained. With the changes in teacher training over the years, teachers lack pedagogical training, which HDP was created to correct. All AVEs are assigned to universities and TTCs, and are involved in teacher development and improving schools in the area, English language improvement, or teaching and curriculum development in critical areas like health. Sometimes the universities provide transportation, other times AVEs use local transportation, like bicycle-powered taxis. AVEs advise graduate students and lecture, strengthening the departments where they are placed. At Graduate School of Social Work of Addis Ababa University (AAU), an AVE served as coordinator of the Ph.D. program. The first Bachelor of Social Work graduation is in 2012 and the school anticipates having sufficient graduate degree holders to be self-sufficient. Another AVE at the AAU, in the School of Journalism in the past academic year, advised graduate students, supported plans to upgrade Information and Communication Technology (ICT) facilities and the library, and also helped to coordinate expatriate staff. The Asella School of Agriculture is host to an AVE who is engaged in teaching, technical assistance, and helping build capacity for community-based programs. The AVE taught courses for the English Language Improvement Center (ELIC) including ICT training to students at the Asella School of Health, as well as delivering a course to administrative staff (secretaries and accountants). The course helped to strengthen English communication, writing, and comprehension skills; all participants received certificates of completion. There were other ELIC activities like mock trial examples of job interviews. In addition she provided pedagogy and methodology books and materials to Asella TTC so that teachers and teacher trainees will benefit from the materials donated to the school. Strengthening the ELIP now takes precedence for AVEs over the better established HDP, which used to be the primary vehicle for improving the pedagogical skills of students and in-service teachers. HDP trains teachers in modern active, learner-centered techniques. AVEs formerly were key personnel in establishing the HDPs. AVEs sometimes served as the HDP Leader or Co-Leader at an institution, reaching out to surrounding schools to provide in-service professional development for teachers, coaching Ethiopian personnel. These HDPs and university outreach programs function like satellite programs, extending to dozens of schools within a driving radius. An example of extracurricular but learning related activities is the expansion of a book club at a rural elementary school near Gondar University. An example of technical assistance is writing for and advising on the publication of a university manual on thesis preparation and helping to develop and serving as Editor-in-Chief of the university’s peer reviewed Health Education journal at Haramaya University. “We try not to use AVEs instead of Ethiopians” for core functions, stated MoE teacher training managers to the Evaluation Team. They said that they used AVEs to “help achieve goals, to assist and coach to build capacity so that the work should not stop when the AVE leaves.” The extracurricular work was dependent - 58 - IFESH Mid-Term Evaluation Report on AVEs, but not the teacher training activities, though AVE participation sometimes helped to accelerate or expand activities. Since the HDPs and ELICs are national policy and strategy, there are built-in Monitoring and Evaluation (M&E) mechanisms where stakeholders can check progress and adjust techniques and strategies, the primary ones being national workshops for the HDP and the ELIP. Usually the work the AVEs do continue for about three years, from one AVE to the next or with a returning AVE. AVEs work at different levels in the ELIC, currently at Teacher Training Institutes (TTIs), another at the federal level where she works on one of the teams. Also two AVEs and the Country Representative are members of the ELIC Team. The work extends beyond tertiary lecturers to having impact on all primary and secondary teachers: they develop modules and assess levels of teachers. 3.2 TRCs and Book Donations Unlike in other AEFA countries, the Evaluation Team observed in Ethiopia that TRC efforts were directed at providing funding only. Funds distributed through the AEFA program were for buying a limited number of books and science kits for the TRCs located at primary schools. In Ethiopia, for book donations, IFESH signed agreement with the universities to be consignees (Debre Berhan and Gondar); this is possibly a best practice for other countries where issues have arisen over customs clearance and duties. The Director of Higher Education Advancement recalled the contribution of books from IFESH and observed that they were well-selected, to the MoE’s satisfaction, “unlike other donors.” They were distributed to all universities, especially new ones. The perspective of the USAID Education Team was that book donations have “minimal impact for lots of work.” They contrast book donations with a preferable model, the Teaching and Learning Materials Program (TLMP) which develops books locally, and the capacity to do so rather than importing books. 3.3 AVE Policy/Management Support Technical support has been limited and is according to the needs assessments conducted by AVEs at their assigned host institution. An example was organizing a strategic planning workshop for Aksum University management. A frequently requested type of technical assistance is proposal and fund development. At Aksum University, an AVE prepared a proposal for unspecified donors for an FM radio station for community information such as HIV/AIDS and agriculture extension; it has not been funded. It is the view of the USAID Mission that the MoE could use more technical assistance as they are overstretched with responding to the backlog of needs, while at the same time implementing new programs. 3.4 IFESH and USAID Crosscutting Themes: Gender Equity, HIV/AIDS, and Community Involvement Every AVE has reported addressing gender issues and HIV/AIDS prevention and mitigation through their courses, in-service professional development workshops, extracurricular activities and/or engagement with Local Non-Governmental Organizations (LNGOs). An example is training for students and residents in sexual rights of women in prevention of sexually transmitted infections and HIV/AIDS, facilitated by a male AVE, which in itself was innovative. There appears to be no assessment or reports of the results of workshops like one on sexual harassment and HIV/AIDS for female students at Asella University or discrete activities like beginning a Female Association between the female students at the School of Agriculture and the School of Health. So while - 59 - IFESH Mid-Term Evaluation Report the activities are laudable and the reaction to the workshops was reported as very favorable, there should be some thought on how to look at their impact for the participants and the AEFA program. 3.5 Use of LVEs sand Small Grants In Ethiopia, Local Volunteer Educators (LVEs) are faculty at universities who are hired to conduct CPD training for teachers. They are paid stipends or half salary out of the Small Grants budget. The LVEs are therefore selected by the universities and have no direct connection to IFESH, nor has there been AVE involvement in these CPD activities. There were 52 LVEs at four sites as of the time of this Report. At Aksum University, for example, two dozen LVEs were engaged to train in-service teachers on assessment, active learning, learning styles, and special education needs. The comprehensive workshop series stretching over several weeks trained 212 school teachers during the 2009-2010 school year. Based on feedback about the first session the workshop was offered again beginning in April 2011. The assessment indicator was improvement in student performance after the teacher attended the workshops. For universities like Aksum University, these types of training workshops have represented their most significant responses to their charge to develop the schools in the surrounding areas. There is also a sustainability aspect since the LVEs have all done the HDP which includes active learning and teaching of pedagogy, which they will then impart to teachers at the workshops. The small grants awarded to Aksum University also included desks for one school, books for TRCs, and science kits for some schools participating in the workshops. In the case of Debre Berhan University, there is funding for a TRC. At Bazen School, where the desks were donated, the Principal told the evaluator that the results are that now all 90 Grade 7 students sit at desks. The Principal was proud to report that there were no drops outs this year, as previous years. Rapid infrastructure expansion has meant new schools that are not always equipped. The construction of the Library at Hossana is another small grant recipient. Other grants have gone directly to schools, either for books and a science kit, or for income-generating activities to support needy students. Also, it has been an ongoing process to get across that M&E (including pre- and post-tests) is not just for USAID, it is for the AVEs and their counterparts – an important skill for education and other development workers. Therefore, for the M&E needs of USAID/Ethiopia, IFESH, and AVEs, this area needs more clarity and training, especially as regards disadvantaged girls, orphans, and vulnerable children. Recipients showed the evaluator records accounting for the funds and Parent Teacher Associations (PTAs) appear to be involved in oversight for the grants. IFESH Field Office staff monitor the use of the funds and receive interim reports from recipients; these documents provide the beneficiary data for Quarterly Reports to IFESH Headquarters (HQ) and USAID. This is a better practice for IFESH to share internally, since the IFESH Field Office received no interim reports for small grants, only at the end. 3.6 Mobilizing Public/Private Partnerships IFESH HQ has engaged PepsiCo and Starbucks in discussions to form public/private partnerships to support AEFA activities in Ethiopia. 4.0 Strengthening Education LNGOs How effective has AEFA been in providing technical assistance and knowledge to increase the capacity of Local Non-Governmental Organizations (LNGOs) to implement sustainable development programs, primarily in the area of education? Only two AVEs worked with groups outside of school settings. Both worked with women’s groups helping them with income-generating projects and business development skills. Documentation was via their Quarterly Reports. - 60 - IFESH Mid-Term Evaluation Report 5.0 Organizational Effectiveness Does the IFESH field team – Country Representative, Field Office staff, and American and Local Volunteer Educators (AVEs and LVEs) – have the tools they need to succeed? Are the necessary organizational structures (people, processes, and technologies) in place to support and sustain AEFA? 5.1 Volunteer Recruitment, Performance, and Support 5.1.1 Host Institution Identification/Volunteer Job Description and Placement Process/Host Institution Resource Allocation and Support In the first year of the AEFA program, only three AVEs were placed and one had to leave two months early. This was unfortunate for a country where the MoE and USAID welcome AVEs. The Evaluation Team was told that the MoE would like more volunteers. Ethiopians look to AVEs for skills and experience that they don’t have and are becoming increasingly selective. As one official commented, “Just because you speak English doesn’t mean you’re an English instructor.” They would like to have AVEs specifically qualified for assignments. The model AVEs who enjoyed their work and were valued by their host institutions included two ends of the spectrum. There were highly experienced academics or educators who had been principals or lifelong teachers who were approaching retirement or were retired. There were also educators or otherwise qualified younger graduates, with pertinent training, who wanted exposure in Africa before continuing graduate work. The feeling was that there were many Americans with superior educational background, skills and experiences who could fill AVE positions. In contrast to one AVE who was interviewed near her hometown by an alumnus in the area, another AVE had no interview and/or vetting, to their knowledge, before being offered positions. Even though the interview process varied, it was consistently found that most AVEs thought their position descriptions had been very vague and did not fully convey their assignments. 5.1.2 Orientation and Training There was a nearly unanimous view among the AVEs in Ethiopia that while the Pre-Departure Orientation Workshop (PDOW) in the U.S. was good for networking, it should have been more practical. Most want more practical implementation components in the U.S. including pre-reading, and to have had a real sense of what the work and living environment would be like and about their assignment. Also repeated was a desire for longer in-country language training. All AVEs praised the in-country orientation, indicating it should become a model for other countries, or that the items included should constitute a checklist. It included briefings on the strategic plans of the MoE and USAID/Ethiopia by their respective staffs, about cultural issues in Ethiopia by a consultant from Addis Ababa University, and basic Amharic lessons. The revised M&E reporting formats were introduced and discussed. There were also practical exercises on how to prepare pretesting, administer, record, and re-administer as post-tests after teaching, as well as how to compute results. At the end of the orientation, volunteers were accompanied to their respective sites to be introduced to their supervisors, be briefed about their assignment and housing arrangements, and to discuss expectations. IFESH Field Office staff in Ethiopia also provide a mid-year workshop. This serves as an opportunity for refresher training and to address any relevant topics that arise, and allows for volunteers to collaborate. 5.1.3 Volunteer Monitoring and Performance Assessment The methods for AVE assessment include self-review, academic peer review, supervisor review, and Country Representative review. The Field Office staff try to visit volunteers each month. - 61 - IFESH Mid-Term Evaluation Report 5.1.4 Safety and Security All AVEs were familiar with IFESH policies and procedures regarding safety and security, and emergency evacuation. 5.1.5 Office Support to Volunteers (Medical, Stipends, and General) Volunteers were familiar and satisfied with the medical insurance and receipt of their stipends. AVEs feel very supported by the IFESH Field Office staff. 5.2 Communications Internal – Talking to volunteers every day is key job of staff. AVEs feel supported and know they can call staff 24 hours a day. They were familiar with the warden system of the U.S. Embassy. One AVE stationed at Haramaya University serves as the regional warden. External – IFESH meets regularly with representatives from Volunteer Service Overseas (VSO) and Peace Corps to discuss how to create linkages among voluntary agency organizations in Ethiopia, and utilize all the volunteers’ expertise to maximize impact on the education sector. 5.3 Knowledge Management and Sharing/Use of ICT One AVE pointed out the need to hand over an assignment from one AVE to the next, including providing hard and electronic copies. 5.4 Field Office Operations Although five AVEs are funded through the AEFA program, and another volunteer funded by the Community Teacher Program (CTP) of USAID/Ethiopia, in the field they work in the same way. The staff comprises the Country Representative, a Program/Administrative Officer, an Accountant, an Office Assistant, a Driver, and three security guards. 5.4.1 Financial Management The Field Office staff follows the established procedure laid out in the accounting manual for preparing budgets and financial reports. On a monthly basis they distribute stipends, pay bills, and petty cash. 5.4.2 Staff Performance Areas/Assessment Individual Field Office staff members are assessed annually using a form created by the Country Representative. The review is conducted personally, measuring the individual against their job description to discuss the employee’s strengths and weaknesses. 5.4.3 Staff Recruitment/Training There is a set system for recruiting Field Office staff. First the Country Representative sets the criteria. Then the position is advertised in newspapers. As candidates apply, all are reviewed by the Review Committee, which includes staff. The committee establishes criteria for selection. There is a 45-day probation period. This system is structured and holistic in vetting and selecting personnel to hire. For the Program Officer position, two AVEs participated in the interviews, and the candidate had to take a written exam and a computer test. The Program Officer found it intense and challenging. Field Office staff receive training, apart from that provided by the Country Representative on office policies and procedure, on an as-needed basis. 5.4.4 Operational Issues Field Office staff have no medical or life insurance. It appears that they once had it but that due to budget constraints, these benefits were discontinued. - 62 - IFESH Mid-Term Evaluation Report 6.0 Performance ME&R Is IFESH using rigorous monitoring and evaluation systems that produce evidence of results and support program improvements and sustainability? Address Monitoring, Evaluation, and Reporting (ME&R) of the country program as a whole, and from stakeholder perspectives. Monitoring is to a large extent a joint undertaking with USAID/Ethiopia and the MoE. The Technical Working Group (TWG) meeting of all USAID/Ethiopia partners is a quarterly event where each partner reports their own performance, best practices, beneficiary priorities, and action plans. Each organization’s performance is assessed jointly with the MoE and Regional Education Bureaus. Also, observation visits by IFESH and USAID/Ethiopia personnel took place together to institutions in Gondar, Bahir Dar and Finote Selam in April 2010 to observe IFESH program activities and obtain feedback from both institutions on the educators’ performance and priorities for placement the following year. The USAID Mission Education staff however expressed dissatisfaction with the utility of AEFA Quarterly Reports. The issue seems to be aligning AVE outcomes with those of the USAID Mission in particular (since USAID is in partnership with the MoE). USAID/Ethiopia has a target number of students whose education quality should improve. IFESH should review its related numbers and its indicators and targets to confirm that they are useful and well-defined. It should then be possible to lay them out in a framework that includes the objectives of the MoE and USAID/Ethiopia. USAID/Ethiopia Education staff want to see IFESH impact in the context of their overall indicators. IFESH should have their indicators from which USAID/Ethiopia can extrapolate, and over time be able to show impact. IFESH emphasizes the use of pre-post tests as a primary tool to measure teacher training performance. While this technique may be relevant for the individual workshop or classroom, what is missing is a strategic objective that it is part of. Also, when AVEs work as part of a capacity building program, the indicators should relate to the capacity building progress and objectives, which should exist already; if not, IFESH Field Office staff should help to develop them with the MoE. 7.0 Implications for IFESH, USAID, and Ethiopia What are the successes, challenges, and opportunities of AEFA as an innovative, sustainable, and replicable model for accelerating education improvement? The MoE is goal oriented and able to use additional human capacity to realize its plans. At USAID/Ethiopia, they would like to see 20 AVEs placed in Ethiopia. The GoE also wants to spread the AEFA model to other sectors. “IFESH is ready to grow to meet these new demands,” said the USAID Education Team Leader. “We (USAID) want AVEs on teacher development but the MoE wants other things.” USAID/Ethiopia indicated that going forward, there may be re-focusing, such as including higher education, in line with the new USAID Education Strategy 2011-2015. USAID/Ethiopia is currently in discussion with IFESH about the way forward since the ten years of continuous funding for the IFESH Community Teachers’ Program will be up next year. IFESH is trying to arrange a public/private partnership with a major U.S. donor, a Global Development Alliance (GDA) with PepsiCo. Dr. Julie Sullivan, President/CEO of IFESH, was in Ethiopia in January 2011, and spoke to other teams about crosscutting volunteers where each office would contribute. The biggest interest was by the Health Team wanting social workers for orphans and vulnerable children; health has the largest program. There was also interest from the Agriculture Team. The impact of AEFA is seen as being constrained by the small number of volunteers. This indicates the need for IFESH, USAID/Ethiopia and USAID to jointly develop a way forward. USAID/Ethiopia is willing to be flexible in letting host institutions decide what they need most working with AVEs. USAID/Ethiopia wants better tools to measure impact and indicate that improved impact data from IFESH could lead to more partnership opportunities with IFESH.USAID/Ethiopia negotiated with Peace Corps on geographical areas to - 63 - IFESH Mid-Term Evaluation Report cluster volunteers, considering community needs and security concerns. They then develop a detailed work plan with expected outcomes, over a period of time with all stakeholders. The good practice to be noted by IFESH is that the work plan should proceed and inform the current process of each host institution writing Statements of Work (SOWs), with USAID/Ethiopia and MoE approval, volunteers are placed at the requesting host institution. 7.1 Comparative Organizations The Peace Corps program in Ethiopia was re-opened in October of 2007, placing 43 Health Sector volunteers in the country, funded through the President’s Emergency Plan for AIDS Relief (PEPFAR). The current program now consists of an Environment Sector and a new Education Sector. Both Environment and Education volunteers are funded through a Memorandum of Understanding (MOU) with USAID/Ethiopia. This is a fairly unique arrangement in the Peace Corps world. Most countries operate their programs exclusively off of appropriated Peace Corps agency funds. The Peace Corps Education Sector in Ethiopia currently stands at 40 volunteers but will increase to 70 by May of 2012. Most volunteers are working in the primary schools. A strategy of the sector is to strengthen the primary schools’ linkages with the Teacher Colleges. USAID/Ethiopia is coordinating the efforts of Peace Corps, VSO, and IFESH in the country to better address USAID goals as well as those of the MoE. Currently, Peace Corps has in place five Teacher Trainers Education volunteers with Masters Degrees or higher at Teacher Colleges across the country. The other 35 Generalist Education Volunteers are working in the primary schools. The primary school teacher volunteers are also working to energize local PTAs, and introducing gardening in the schools. Because of the relationship between the Peace Corps, PEPFAR, and the USAID Mission, they have the financial resources to offer more training to their volunteers and counterparts than what might be typical in a country where only appropriated funds are being used. There are also a number of Master’s International Peace Corps volunteers in-country (earning their degrees while serving overseas). Informants thought their quality is similar to VSO and that IFESH is on par or higher than other volunteers. 7.2 Stakeholder Viewpoints The “wish list” of the Vice President of Aksum University is indicative of further assistance the MoE would like from IFESH:  Academic staff to replace staff being groomed  Academic staff to coach/mentor at the new universities – this is more important to them than AVEs being in the classroom  Placement of volunteers in key fields: English Language Instruction, Technology, ICT, E-Library, and Tele-Education (Indiana University is a partner with Aksum University)  E-books 8.0 Recommendations The following recommendations are specific to the AEFA program in Ethiopia. The Evaluation Report has other recommendations applicable to all AEFA countries. The recommendations should be discussed and implemented in coordination with IFESH HQ. Improving Teaching, Systems, and Education Resources 1. IFESH should identify how to assist the MoE with M&E of the overall system and its innovations. For example, its approach to improving teachers’ subject matter mastery with a third pedagogical year: Is it working? Also the innovative Ethiopian strategy to strengthen subject matter mastery using - 64 - IFESH Mid-Term Evaluation Report university departments, and then to provide an extra year of pedagogical studies to become teachers (e.g., no school of education at Aksum University). 2. Pair an AVE with a host country national to enhance sustainability prospects; in some settings, this could help mediate multi-cultural patterns (e.g., gender, business practices, and cultural practices). Strengthening Education LNGOs 3. Working with LNGOs needs to be reviewed in the Ethiopian context. Questions to be considered include: Are the desired outcomes and intended impact being met? Does the Ethiopian experience require a complete re-definition of this component or is it a variation? 4. Small grants award policies and procedures and materials need to be reviewed and revised for: consistency, accountability, whether AVEs are involved, management of expectations, and impact consistency with the Cooperative Agreement. Organizational Effectiveness 5. The IFESH Handbook for Ethiopia and the evacuation booklet are good models for other IFESH countries. ME&R 6. Improve the ME&R framework, definitions, and reporting. First clarify the main objectives of AEFA in Ethiopia. The M&E framework should be constructed around this model. Then match volunteers to specify learning outcomes/deliverables – what it affected. This would accommodate larger numbers of AVEs. 7. Improve tools and methods of data collection, such as client satisfaction surveys for host institution and MoE staff. Implications for IFESH, USAID, and Ethiopia 8. Use a business plan approach – outcomes, benchmarks, and what will be accomplished. The business plan should scope needs and resources to look at three years, five years, ten years – what progress will AVES have made, what impact? Identify what volunteers need to accomplish it. 9.0 Methodology and Information Sources The Evaluation Team interviewed five AVEs and one LVE, and principals, teachers, and PTA representatives while in Ethiopia. Separately, two former volunteers responded to an on-line survey. The Evaluation Team interviewed the IFESH Country Representative, Field Office staff, the USAID Education Team staff and Team Leader, the USAID Deputy Mission Director, personnel at the MoE, representatives of host academic institutions, and representatives at LNGOs working with AVEs and/or that are receiving small grants. Names for all interviewees are found in Appendix A. The following documents and publications were reviewed for this report: IFESH. (2009). AEFA – IFESH Technical Application. IFESH: Scottsdale, Arizona. IFESH. (2009). AEFA Quarterly Report, July-September 2009. IFESH: Scottsdale, Arizona. IFESH. (2009). AEFA Quarterly Report, October-December 2009. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, January-March 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, April-June 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Annual Report and Quarterly Report, July-September 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, October-December 2010. IFESH: Scottsdale, Arizona. - 65 - IFESH Mid-Term Evaluation Report USAID. (2009) Cooperative Agreement Between USAID and IFESH. USAID: Washington, D.C. 1 Tracking the Millennium Development Goals (n.d.). United Nations online MDG Monitor. Retrieved from http://www.mdgmonitor.org/country_progress.cfm?c=DJI&cd=262. Education and Population Data (n.d.). World Bank online. Retrieved from http://data.worldbank.org. Millennium Development Goals Indicators (2010). UN online. Retrieved from http://mdgs.un.org/unsd/mdg/Data.aspx. 2 Comprehensive new reports show progress on Millennium Development Goals (2010). ODI Report. Retrieved from http://www.odi.org.uk/news/details.asp?id=196&title=comprehensive-new-reports-show-progress-millennium-development￾goals. 3 Ethiopia: 2010 MDGs Report – Trends and Prospects for Meeting MDGs by 2015. UNDP Report (2010). Retrieved from http://www.undp.org/africa/documents/mdg/ethiopia_september2010.pdf. 4 Ibid. 5 Ibid. 6 ODI Report (2010). 7 UNDP Report (2010). 8 Millennium Development Goals in Ethiopia (2010). UNDP. Retrieved from http://www.et.undp.org/index.php?option=com_content&task=view&id=30&Itemid=113. 9 Excerpted from Daily Ethiopia online (as cited by Ashcroft, 2011). Ethiopia: Expanding and Improving Higher Education. Retrieved from http://www.dailyethiopia.com/index.php?aid=842. 10 Sub-Saharan Africa (n.d.). USAID online. Retrieved from http://www.usaid.gov/locations/sub￾saharan_africa/countries/ethiopia/. 11 IFESH Revised Work Plan; IFESH Quarterly Reports; IFESH AVE Distribution Tables; IFESH AEFA Book Distribution Tables. Received from IFESH HQ. 12 Congressional Budget Justification (2010). USAID online. Retrieved from http://www.usaid.gov/policy/budget/cbj2010/2010_CBJ_Book_2.pdf. - 66 - IFESH Mid-Term Evaluation Report Appendix D: Ghana Case Study 1.0 Summary Statistics in Ghana reveal a comparatively bright picture for education in Ghana. The country has made great strides in improving access and quality education. Volunteers fielded by the International Foundation for Education and Self-Help (IFESH) through the American Educators for Africa (AEFA) program are contributing to enhancing the quality of education in Ghana in a positive albeit limited manner at the tertiary level. The effectiveness of the AEFA program is restricted to the stated satisfaction with volunteer work by host institutions. The operations of IFESH and the sustainability of the AEFA program, however, demonstrate a limited impact. In the first year, two American Volunteer Educators (AVEs) served in Ghana, and both returned the following year and were joined by two new AVEs. The AEFA program volunteers have helped train approximately 1300 pre-service and in-service teachers since its inception. Volunteers are also filling identified gaps by serving as instructors at the tertiary level. The AEFA program in Ghana has distributed approximately 14,000 books in its first year. Although the Education Team at the U.S. Agency for International Development (USAID) Mission in Ghana is not very familiar with the AEFA program, they are familiar with the work of IFESH through its other projects being implemented in the country. The Ministry of Education (MoE) has had along working relationship with IFESH in the past but is not very familiar with the current AEFA program; they welcome closer coordination. Coordination with other stakeholders such as the Peace Corps is limited. While the AEFA program is not directly aligned with the MoE and USAID/Ghana priorities, from the perspective of host institutions, AVEs are filling needs in Ghana and bring fresh perspectives to education peers in their respective fields. IFESH communication, development of a country plan that better aligns the common areas of interest of the Ghana Education Service (GES) and USAID/Ghana, and a related Monitoring, Evaluation, and Reporting (ME&R) plan, are candidates for improvement to enable AVEs to provide a greater impact in Ghana. 2.0 Country Context According to the April 2010 Ghana Millennium Development Goals (MDGs) Report, Ghana is making progress towards achieving the Millennium Development Goal for Education (MDGE) of universal primary education for girls and boys by 2015. Great strides have already been made in the area of net enrollment rates with the primary level enrollment jumping from 69.2% in 2005-2006 up to 83.7% in 2007-2008.2 Additionally, primary completion rates have steadily improved over the past decade increasing from 63% in 1999 up to 88% in 2008.3 Part of this overall success can likely be attributed to the opening of over 400 new primary schools across the country between 2007 and 2008 as well as a reduction in the pupil to teacher ratio. A significant decline Ghana: Education/Socio-Economic Indices1 Likely to achieve MDG 2 by 2015: Boys and girls alike will be able to complete a full course of primary schooling No Net enrollment primary education (%) 75.9% (2009) Primary Completion rates 82.7% (2009) Literacy rate, youth total (% of people ages 15-24) 80.1% (2009) Proportion of pupils starting grade 1 who reach last grade of primary, Both sexes 73.2% (2008) Proportion of pupils starting grade 1 who reach last grade of primary, Boys 75.4% (2008) Proportion of pupils starting grade 1 who reach last grade of primary, Girls 70.9% (2008) Percentage of population under 15 years 38.4% (2009) Population below poverty line (less than US$1 per day) 30% (2006) - 67 - IFESH Mid-Term Evaluation Report in the poverty rate has been achieved over the last decade as well. In 1991-1992, the poverty rate was at 51.7%, and has almost been reduced to half at 28.5% as of 2005-2006.4 Ghana is well on its way to achieving Goal 2 of the MDGs even though the country may fall short, and is making progress with respect to Goal 3 in promoting gender equality and women’s empowerment. 2.1 MoE Priorities The key policy objectives cited in the Ghana MDGs 2010 report include increasing access to, and participation in education and training, with greater emphasis on gender and geographical equity; improving the quality of basic education; and enhancing the delivery of education services. In order to reinforce the attainment of universal primary education, a number of policies were carried out in 2008. These policies include the construction/rehabilitation of classrooms; strengthening the capitation grant initiative; expanding coverage of the school feeding program; enforcing laws that support the implementation of Free Compulsory Universal Basic Education (FCUBE); expanding non￾formal education in partnership with community groups, Non￾Governmental Organizations (NGOs), and private providers; and developing a national policy on distance learning.6 According to the GES, the district level will be critical in carrying forth educational initiatives as teacher training initiatives are being transferred to the district level.7 Despite the advances made in the area of universal primary education, the MoE has also identified a number of challenges to achieving this goal by 2015. These challenges include the problem of teacher posting and retention; the decline in quality of education; an inadequate infrastructure; a low level of teacher commitment; a low accountability to parents and students; falling quality of science and technology; and the high cost of education.8 AEFA Ghana Performance Summary5 2009-2010 2010-2011 Total* Target** AVEs 2 4 6 11 LVEs 0 0 0 / CPD Training Workshops - 0 0 / Teacher Resource Ctrs. - 2 2 / Subgrants to NGOs/CBOs - 0 0 / Books Distributed 14,122 30,000 44,122 20,000 Pre-Svc. Teachers Trained M: 0 M: 786 M: 786 F: 956 F: 246 F: 1,202 2,405 T: 956 T: 1,032 T: 1,988 In-Svc. Teachers Trained M: 32 M: 34 M: 66 F: 15 F: 14 F: 29 200 T: 47 T: 48 T: 95 Ed. Administrators M: 0 M: 25 M: 25 F: 0 F: 25 F: 25 200 T: 0 T: 50 T: 50 LNGO/CBO Representatives M: 0 M: 0 M: 0 F: 6 F: 0 F: 6 200 T: 6 T: 0 T: 6 Users of Resource Ctrs. - M: 0 M: 0 - F: 0 F: 0 400 - T: 0 T: 0 Key: M= Male, F= Female, T=Total, - = no data reported, and / = no data available. * Total = The sum of the 2009-2010 and 2010-2011 columns. ** Target = 2009-2010 actuals + 2010-2011 target + 2011-2012 target. For indicators that were not used in 2009, Target = 2010-2011 + 2011-2012 only. - 68 - IFESH Mid-Term Evaluation Report 2.2 USAID Priorities The U.S. Government Basic Education assistance to Ghana through USAID seeks to expand access to basic education as well as to improve the quality of primary education. The Education program places emphasis on increasing the percentage of school enrollment and completion, especially for girls; helping to ensure that children who complete primary school are able to read at grade level; improving the management and accountability of school systems; and increasing community involvement in schools and education.9 According to the Education Team at the USAID Mission, they have been focusing on basic through secondary level, along with engaging in efforts to enhance the quality of teacher training. Based on the interview with the USAID Education Team in Ghana, efforts have been focused mainly on kindergarten through senior high school (also referred to as secondary). Additional efforts have also centered on strengthening the capacity of Teacher Training Institutes (TTIs) to deliver higher quality training to teachers. One such program that USAID funded that was being executed by IFESH is the Community Teacher Program (CTP), which aims to support teacher certification. The program was being phased out as a result of a change in teacher certification policy of the MoE as of February 2011. 2.3 IFESH Alignment with the MoE and USAID In Ghana, IFESH has fielded a total of six AVEs during the past two academic years; two AVEs in 2009-2010 and four in 2010-2011. Of the current four AVEs, one teaches courses at University of Cape Coast, one teaches courses at Catholic University College of Ghana in Fiapre, and the other two are involved in teacher training, Textbook and Learning Materials (TLMs), and Teacher Resource Centers (TRC) at Our Lady of Apostle College (OLA) in Cape Coast. The two AVEs from 2009-2010 were also involved in teacher training, TLMs, and TRC efforts at OLA. IFESH’s efforts through the AEFA program are not fully aligned with those of USAID. IFESH volunteers are providing tertiary support primarily, while USAID/Ghana has been focused on supporting efforts aimed at kindergarten through senior high school levels. However, IFESH is aligned with USAID/Ghana’s priority to strengthen the capacity of TTIs to deliver increased quality training by teachers. GES is working across all levels, and encourages that IFESH coordinate with district level administration so as to ensure alignment and to assist in achieving educational goals. Among needs they specified were TLMs for science and technology and attrition rates for disadvantaged groups. Despite that gender equity, HIV/AIDS awareness, and community involvement are major crosscutting themes for both USAID/Ghana and IFESH, the AVE role on the ground in the execution of these themes is limited. There is no consistent AVE focus on any of these themes as observed in the field through interviews with AVEs and respective host institutions. 3.0 Improving Teaching, Systems, and Education Resources How has AEFA strengthened the capacity of teachers, administrators, and policymakers to enhance the quality of school teaching methodologies and pedagogical materials at the basic education level through technical aid and the training of teachers, using the main AEFA methods of teacher training, Teacher Resource Centers (TRCs), donated books, and serving as policy/management advisors? Overall, the work of four AVEs in Ghana reveals a limited albeit positive impact. The host institutions reported satisfaction: volunteers are filling vacancies and enriching teaching practice at their institutions. They are largely disconnected from each other without unified objectives for the IFESH country program. 3.1 Teacher Training In terms of teacher training, only one AVE truly supports this effort in Ghana. The sole AVE conducts courses at OLA for future teachers in the areas of mathematics and sciences. This Teacher Training College - 69 - IFESH Mid-Term Evaluation Report (TTC) is female only, and as such the volunteer is also contributing to gender equity efforts. The other two AVEs teach courses at the university level but their students are not teachers in training. One teaches in the sciences at the University of Cape Coast, and the other volunteer teaches at the Catholic University of Ghana, Fiapre in the English Department. They are all supporting academic needs, in particular in mathematics and sciences. The one volunteer not involved with teaching is tasked with creating and conducting workshops at OLA. But only one workshop was documented over a span of two years. In a group interview with teachers who had participated in the workshop their assessment was that the workshop on TLMs development suited the needs of some, but not others. However, they and the GES representative present attested to a strong need for Continuing Professional Development (CPD) and specified several areas where it was needed. A needs assessment should inform future workshops, they said. The host institutions are very satisfied with the efforts of the volunteers and express that their presence has had a positive impact on their students, universities, and surrounding communities. According to one representative at a host institution, volunteers bring in fresh ideas especially on design and more modern tools. Curriculum should incorporate instructional design and assessment, which is lacking with the local professors. Global trends, according to the representative, show that professors should be facilitating learning instead of lecturing. Volunteers therefore bring a “fresh injection of methodology and pedagogy.” Although the volunteers are not supporting strategic needs in building teacher capacity they are fulfilling ad hoc needs of host institutions in Ghana. AVEs are maintaining education quality by filling in a gap caused by regular staff not being available and/or existing staff being overloaded. As one representative at a host institution expressed, “….without the AVE the situation would have been difficult.” Another representative shared IFESH “opened her eyes” to creativity and materials. At this level, the teacher training has been a success, and volunteers have filled ad hoc needs. 3.2 TRCs and Book Donations Only one AVE in Ghana is involved with TRC efforts. The one volunteer at OLA was engaged with an assessment of TRC needs; this included assessment of its current state, development of goals and objectives for the Center, and a plan to achieve them.10 Efforts continued through the AVE’s assignment, and it was noted that policies and inventory procedures for the TRC were developed between October and December 2010 as reported by the Quarterly Report for that time period. In January 2011, the Evaluation Team observed the TRC, which is still a work in progress as it was recently moved to a new location (a building that is still under construction). A student committee was established to assist with the TRC and to ensure continuity once the AVE departs. While it is evident that OLA appreciates the efforts and teachers are beginning to reap its contents, the actual impact will only be determined after the departure of the AVE. Although the involvement with TRCs is extremely limited in Ghana, efforts with TLMs are more evident across all AVEs. All AVEs, which was echoed by each of their respective host institutions, expressed that their work with TLMs added value to the classroom as students in Ghana are not familiar with similar quality teaching materials. The new approaches and methodologies for providing visual presentations of learning material is making an immediate impact as discussed by workshop attendees and other teachers at host institutions. If the students and colleagues are incorporating these innovations in their daily teaching efforts, it is then that the larger impact of AEFA’s role can be determined. Lastly, in terms of book donations, Ghana has received a total of 14,122 books and these were distributed during the 2009-2010 academic year.11 The Evaluation Team did not visit the donation recipients. However, there is the common sentiment from IFESH Field Office staff and AVEs that book donations are not useful; one interviewee even countered that the money should be used to fund local publishers instead.12 3.3 AVE Policy/Management Support AVEs are not providing this type of support in Ghana. - 70 - IFESH Mid-Term Evaluation Report 3.4 IFESH and USAID Crosscutting Themes: Gender Equity, HIV/AIDS, and Community Involvement Despite that gender equity, HIV/AIDS awareness, and community involvement are major crosscutting themes for both USAID and IFESH, the AVE role in these themes is limited. The volunteers at OLA conducted workshops that incorporated gender equity, and focused on empowering women in mathematics, science and technology.13 Other than this one noted effort, there is no consistent AVE focus on either of these other themes as documented in the Quarterly Reports or observed in the field through interviews with AVEs and respective host institutions. 3.5 Use of LVEs IFESH has not engaged Local Volunteer Educators (LVEs) in Ghana. The Country Representative told the Evaluation Team that IFESH is in discussions with the GES about how to field LVEs in the next quarter for in-service teacher training. 3.6 Use of Small Grants IFESH has not identified or applied for small grants in Ghana. 3.7 Mobilizing Public/Private Partnerships IFESH has not engaged in identifying or mobilizing public/private partnerships in Ghana. 4.0 Strengthening Education LNGOs How effective has AEFA been in providing technical assistance and knowledge to increase the capacity of Local Non-Governmental Organizations (LNGOs) to implement sustainable development programs, primarily in the area of education? Currently, IFESH is not engaged in activities with LNGOs in Ghana. 5.0 Organizational Effectiveness Does the IFESH field team – Country Representative, Field Office staff, and American and Local Volunteer Educators (AVEs and LVEs) – have the tools they need to succeed? Are the necessary organizational structures (people, processes, and technologies) in place to support and sustain AEFA? 5.1 Volunteer Recruitment, Performance, and Support Successful volunteer recruitment begins with the identification of criteria and characteristics needed for assignments at host institutions. Although the recruitment occurs at IFESH HQ, the Country Representative plays a role in matching the volunteer with the host institutions. While there is evident consultation with the Colleges of Education with whom the volunteers are placed, there is little collaboration with either the MoE or the USAID Mission on vetting volunteers for certain positions. 5.1.1 Host Institution Identification/Volunteer Job Descriptions and Placement Process/Host Institution Resource Allocation and Support Representatives at host institutions expressed the desire for communication regarding job descriptions and placements of volunteers to begin sooner. In addition, they do not receive follow up reminders and entry points into the host institutions are inconsistent. Neither the MoE nor the Mission are involved in vetting competing host institution needs or the actual job descriptions to align priorities among stakeholders. Representatives at host institutions are clearly appreciative of receiving volunteers but continue to stress the need for volunteers with Information and Communications Technology (ICT) and Science, Technology, Engineering, and Mathematics (STEM) expertise. Also, placements include some private institutions, which - 71 - IFESH Mid-Term Evaluation Report do not appear to be a consideration when placing volunteers. Placements are also not fully aligned with country and USAID priorities as discussed in Section 2.3. 5.1.2 Orientation and Training The volunteers expressed that the training they are currently receiving is not beneficial. The Pre-Departure Orientation Workshop (PDOW) in Scottsdale, Arizona is too long, and in-country orientation is not long enough. The AVEs did not feel prepared for what to expect when living in the field. Volunteers also identified a need for specific training such as lesson planning as it would be valuable for those who need a refresher or have not had that particular training. According to volunteers, improvements at both orientations are needed to make them more valuable. Volunteers also stated that they do not receive any ongoing training, and that they would benefit from specialized training. 5.1.3 Volunteer Monitoring and Performance Assessment Formal monitoring of volunteers varies. One volunteer receives performance evaluations from the Dean as well as the students. Another volunteer receives quarterly reviews from the Principal. Performance assessments are completed by supervisors at host institutions but the results are not explicitly discussed with the AVEs. IFESH Field Office staff members do not observe or regularly monitor volunteers but do conduct site visits with host institution supervisors and to check in on the well-being of AVEs. 5.1.4 Safety and Security Volunteer knowledge and awareness of safety and security policies and procedures varies even though all are briefed on safety and security at the in-country orientation upon arrival. One volunteer, for example, was not aware of any formal safety and security plan. Others stated that there is a plan. All volunteers were informed not to travel at night, and consistently reported that safety rule. Volunteers are not aware if an emergency evacuation plan is in place but did state that they are registered with the U.S. Embassy. Despite the variance, all volunteers shared that the Country Representative would contact them should there be any necessary communications concerning safety and security, and likewise, they would contact the Country Representative if there was an issue. There are written guidelines for if a civil disturbance or natural disaster occurs but this document was not referenced by volunteers. 5.1.5 Office Support to Volunteers (Medical, Stipends, and General) The volunteers receive medical support and monthly stipends from IFESH. If a medical issue arises, the volunteer is to pay the medical costs out of pocket upfront, keep all receipts, and then get reimbursed. Additionally, the volunteers are offered Life Insurance. Regardless of country or rural/urban location, all volunteers receive a monthly stipend of $850. This stipend is taken from the IFESH HQ budget but is distributed locally. All volunteers are required to have a cell phone, and IFESH will pay for the first month. No other support such as materials or money for copies is provided to AVEs. 5.2 Communications Internal communication between volunteers and the Field Office are adequate, according to both parties. However, there is rarely communication among the volunteers, and some expressed that it would be beneficial along with a newsletter in Ghana. In terms of external communication, this is an area that is neither adequate nor satisfactory among stakeholders interviewed. Representatives at host institutions expressed satisfaction with IFESH Field Office staff. However, there was an instance of crossed wires in which a host institution believed that a volunteer was coming for the 2009-2010 academic year, but did not arrive until the following year. It appears to have been a lack of communication between the Registrar and the Academic Department that would host the - 72 - IFESH Mid-Term Evaluation Report volunteer. Communication with host institutions is limited except during soliciting interest and committing to hosting volunteers. Communication with USAID/Ghana about AEFA is virtually non-existent; but there is some communication about CTP. When a program is funded in Washington, according to USAID/Ghana, they have minimal involvement. 5.3 Knowledge Management and Sharing/Use of ICT There is virtually no use of demonstrated knowledge management or ICT within the Field Office. Paper files are stored, and the office operates with two office email addresses; they are not specific to an individual. All communications go through the Country Representative. The use of cell phones is more common than the use of email. According to staff, they are in the process of creating a database; however, neither the purpose of the database nor the intended content was made clear. 5.4 Field Office Operations Although manuals and documented policies and procedures exist, awareness and execution of them vary across topics and by individual. There is a Policies and Procedures Handbook for Ghana, an Employee Handbook, and an International Educators for Africa (IEFA) Handbook. None of these were referenced by staff or by volunteers. The size of the volunteers at the time of the field visit in January 2011 was eight (four AEFA volunteers, and four volunteers serving under a different IFESH funded program), and the size of the Field Office staff was eight (six permanent positions and two temporary positions funded by CTP that ended in February 2011). 5.4.1 Financial Management The budget process appears to function, and QuickBooks is the financial tool that the Field Office uses. In terms of stipends for AVEs, there were no complaints of late receipt. 5.4.2 Staff Performance Areas/Assessment IFESH Field Office staff members are not being regularly assessed as reported by interviewed staff. Formal employee evaluations are not taking place even though it is mentioned in the Employee Manual. 5.4.3 Staff Recruitment/Training Staff turnover is low in the Field Office. When a need to hire staff arises, the Country Representative will seek high caliber staff through advertising in the paper and talking to sister organizations for any recommendations. Candidates are required to submit an application that includes a curriculum vitae (CV) and references. If approved, then the candidate is invited in for an interview. Once staff is hired, there is no training beyond onboarding the new employee. The onboarding consists of explaining the mission of IFESH and what he/she must perform to get the job done. There is no professional development of staff. 5.4.4 Operational Issues While there are handbooks and manuals, it is evident that they are not referenced. Formal assessments are not taking place, and 13th month payments are not being made even though both are mentioned in the Employee Manual. 6.0 Performance ME&R Is IFESH using rigorous monitoring and evaluation systems that produce evidence of results and support program improvements and sustainability? Address Monitoring, Evaluation, and Reporting (ME&R) of the country program as a whole, and from stakeholder perspectives. Overall, the success of IFESH rests in the positive impacts that its volunteers are making at their host institutions. The satisfaction with the performance of volunteers is not a concern; however, the performance - 73 - IFESH Mid-Term Evaluation Report of IFESH personnel as perceived by its clients is an area that does require improvement. From the stakeholder perspective, IFESH is not successful in communicating evidence based results, nor is it sustainable. Although there is a Monitoring and Evaluation (M&E) Manual and that volunteers receive some instruction during the PDOW, M&E efforts are not strategic. Volunteers may or may not be diligent in their reporting as there is no verification of the data collected. Furthermore, there is little review of the reporting that the volunteers submit to the Field Office. The Country Representative is responsible for submitting the data then to IFESH HQ where data is compiled into a Quarterly Report that is submitted to USAID. Once again, there is no verification of data, and little analysis is conducted. Instead the primary focus is the compilation of the data from the various countries into one report. The Quarterly Reports provide limited analysis of IFESH’s progress towards meeting any AEFA targets. A simple number of the number of beneficiaries trained in a given quarter does not present a holistic picture of the status of training in any given country. At a high level, the host institutions are satisfied with IFESH as related to the work of the volunteers that they host. AVEs, like other instructors are assessed by their students. All supervisors interviewed reported that they discussed those results with the volunteers. However, they are not receiving the “results” of what the AVEs are doing from a reporting perspective. While this is not required and may occur on an informal level, performance ME&R is limited. Satisfaction as expressed by USAID/Ghana is mixed. The Education Team at the Mission is very familiar with IFESH’s CTP work but knows very little about the AEFA program and the work of AVEs. While it may be a result of AEFA being U.S. Government funded, both IFESH and the USAID Mission could benefit from further collaboration about AEFA. 7.0 Implications for IFESH, USAID, and Ghana What are the successes, challenges, and opportunities of AEFA as an innovative, sustainable, and replicable model for accelerating education improvement? Although IFESH’s AEFA program model could be replicated, improvements are necessary for the organization to enable continued innovation and to ensure sustainability. Its current efforts are not integrated enough to ensure sustainability after the departure of a volunteer but it is clear that teaching innovations from the U.S. are being shared with future teachers, students, and staff at host institutions in Ghana. The value of this exchange of information is undeniable but IFESH should be mindful that there are other organizations that field volunteers and provide similar services in Ghana. The Japanese International Cooperation Agency (JICA), Volunteer Service Overseas (VSO), and Peace Corps are organizations that host institutions and the USAID Mission name them as comparative to IFESH. All of these organizations have large volunteer numbers compared to IFESH. There is also the sentiment that IFESH primarily operates at the tertiary level and at TTIs, which is not common for the other organizations. Although IFESH’s strategic advantage has been in training untrained teachers, ultimately there is the sentiment that “any Private Voluntary Organization (PVO) in Ghana could do the same things as IFESH in training teachers if you gave them the money.”14 IFESH will continue to face competition by these other known organizations if they do not continue to differentiate themselves and grow to demonstrate greater strategic impact. 8.0 Recommendations The following recommendations are fairly specific to the AEFA program in Ghana. However, the Evaluation Report has other recommendations applicable to all AEFA countries. The recommendations should be discussed and implemented in coordination with IFESH HQ. Improving Teaching, Systems, and Education Resources - 74 - IFESH Mid-Term Evaluation Report 1. Volunteers should continue to incorporate fresh methodologies (e.g., TLMs) in their teaching since this aspect of quality is a demonstrated need in Ghana. 2. IFESH should field more volunteers in TTIs rather than tertiary host institutions so that teacher quality and capacity can be addressed in greater quantity. 3. IFESH should encourage host institutions to institutionalize TRC efforts through assignment of a TRC Director of a permanent staff member, and with the creation of a student committee. 4. To improve its branding, IFESH should formalize materials for host institutions that include: a. Options of how the AVE can serve, e.g., as a Senior Lecturer or Staff Adviser to a Designated Administrator or Department, and b. Emphasis on the concept of helping to bring effective (“modern”) methods and pedagogy into the classroom, institution, and community. 5. IFESH should provide oversight and contribute to the refinement of volunteer work plans, implement regular monitoring and feedback to volunteers about their work plans and workload for the year. Strengthening Education LNGOs 6. IFESH should seek specific NGOs to support in order to contribute to local capacity building and to diversify its portfolio of activities in Ghana. Organizational Effectiveness 7. IFESH should review how it prepares job descriptions, recruits, and prepares AVEs for assignments in order to ensure that an AVE will make a good fit in his/her role. 8. IFESH should begin the process of identifying needs with host institutions sooner. It should be an iterative and collaborative process that results in job descriptions that are clearer and more accurate. 9. IFESH should employ a formal process to assess its Field Office staff, and consider providing professional development opportunities such as training courses to enhance the skills/knowledge of its Field Office staff. ME&R 10. A review of ME&R objectives and training are needed. 11. Creation and adoption of M&E tools beyond an Excel file is needed to improve the ME&R system. Implications for IFESH, USAID, and Ghana 12. Greater coordination between these parties is necessary to achieve alignment with priorities. 13. IFESH should consider not fielding volunteers in Ghana as positive impact is limited by the scope and purpose of placed volunteers. 14. IFESH should examine the reasons communication with USAID/Ghana is ineffective to develop a constructive communications strategy. AEFA is not particularly known with the Mission; so IFESH should determine what clear messages need to be conveyed that are program-related, and how they can be reinforced (e.g., supportive printed material, and monitoring results). 9.0 Methodology and Information Sources The Evaluation Team interviewed four AVEs while in Ghana. The Evaluation Team interviewed the IFESH Country Representative and four Field Office staff, the USAID Education Team Leader and members of the USAID Education Team, the USAID Mission Director, personnel at the Ministry of Education, representatives of host academic institutions, and WCF. Names for all interviewees are found in Appendix A. The following documents and publications were reviewed for this case study: - 75 - IFESH Mid-Term Evaluation Report IFESH. (2009). AEFA Quarterly Report, July-September 2009. IFESH: Scottsdale, Arizona. IFESH. (2009). AEFA Quarterly Report, October-December 2009. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, January-March 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, April-June 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Annual Report and Quarterly Report, July-September 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, October-December 2010. IFESH: Scottsdale, Arizona. USAID. (2009). Cooperative Agreement Between USAID and IFESH. USAID: Washington, D.C. 1 Tracking the Millennium Development Goals (n.d.). United Nations online MDG Monitor. Retrieved from http://www.mdgmonitor.org/country_progress.cfm?c=DJI&cd=262. Education and Population Data (n.d.). World Bank online. Retrieved from http://data.worldbank.org. Millennium Development Goals Indicators (2010). UN online. Retrieved from http://mdgs.un.org/unsd/mdg/Data.aspx. 2 ODI Report (2010). Comprehensive new reports show progress on Millennium Development Goals. Retrieved from http://www.odi.org.uk/news/details.asp?id=196&title=comprehensive-new-reports-show-progress-millennium-development￾goals. 3 UNDP Report (2010). 2008 Ghana Millennium Development Goals. Retrieved from http://www.undp￾gha.org/docs/GhanaGhanaMDGReport-2010.pdf. 4 Ibid. 5 IFESH Revised Work Plan; IFESH Quarterly Reports; IFESH AVE Distribution Tables; IFESH AEFA Book Distribution Tables. Received from IFESH HQ. 6 UNDP Report (2010). 7 The Evaluation Team conducted an interview of with GES officials in Ghana on 31 January 2011. 8 UNDP Report (2010). 9 Program Overview – Education (n.d.). USAID online. Retrieved from http://www.usaid.gov/gh/programs/education/. 10 AEFA Quarterly Report (January-March 2010, p. 18). Scottsdale, Arizona: IFESH. 11 AEFA Book Distribution Tables (Ghana) file (2011). Received from IFESH HQ. 12 The Evaluation Team conducted an interview of IFESH Field Office staff in Ghana on 31 January 2011. 13 AEFA Quarterly Report (April-June 2010, pp. 19-21). Scottsdale, Arizona: IFESH. 14 The Evaluation Team conducted an interview of personnel at the USAID Mission in Ghana on 24 January 2011. - 76 - IFESH Mid-Term Evaluation Report Appendix E: Liberia Case Study 1.0 Summary By fielding highly experienced volunteers as peers, mentors, and supervisors, the International Foundation for Education and Self-Help (IFESH) American Educators for Africa (AEFA) Program has helped train and upgrade skills of about 3,000 teachers in 20 months of operation in Liberia, helping rehabilitate an education system hard-hit by war. American Volunteer Educators (AVEs) have introduced education innovations in syllabi and programs, how classes and schools are managed, and coordinated education reform policy design and business process reengineering tasks at the Ministry of Education (MoE). While teaching practice and mentoring have been their primary occupation, they have actively participated in their academic and wider communities, have produced and mobilized Textbooks and Learning Materials (TLMs) often at their own expense, helped distribute 20,000 donated books, and have raised funds for projects at their host institutions, as well as with Local Non-Governmental Organizations (LNGOs) where they have served a capacity-building advisors. Minister of Education Gongar stated emphatically, “The IFESH contribution to capacity development is the most important thing the organization can do to improve Liberian education.” He underscored the value of IFESH-supplied policy support in the short-staffed MoE, to help establish fundamental frameworks to guide all the education sub-sectors – work that otherwise would have been delayed. The eight AVEs for each of the past two academic years is the largest cohort among the eight IFESH countries where AVEs are fielded in the 2010-2011 academic year. The performance of the eight AVEs in their host institutions and the MoE, aided by the outreach of the Country Representative with U.S. Agency for International Development (USAID)/Liberia and other donors, has earned the IFESH AEFA recognition as a contributor to education in Liberia; AEFA could be seen as a type of pilot that can be scaled up. However, the resources provided through the AEFA program could add more value if there were closer alignment and coordination between USAID, the MoE, IFESH, and other partners like Peace Corps and the Liberia Teacher Training Program (LTTP). Impact could be greater if there was more strategic, outcomes-based AVE focus, such as conducting flexibly structured in-service training, while the education system is still in flux; IFESH has been responsive but on an ad hoc basis. AEFA work has illuminated large human capacity gaps in the education sector that AEFA could fill with many more skilled AVEs at the school, county and national levels. To do so requires that IFESH satisfactorily address operational improvements in recruitment, orientation and management specified in this Mid-Term Evaluation; also, to undertake joint planning of priorities, outcomes and indicators for three to five years with USAID, MoE, and other partners. USAID and USAID/Liberia need to determine if they want to leverage both their investment thus far in IFESH, and IFESH’s experience and goodwill in Liberia. The incoming Mission Director emphasized that Liberia has a high-risk profile; while there is room for improvement, nonetheless the AEFA program is performing effectively in this difficult environment. Liberia: Education/Socio-Economic Indices1 Likely to achieve MDG 2 by 2015: Boys and girls alike will be able to complete a full course of primary schooling No Net enrollment primary education (%) 75.2% (2000) Primary Completion rates 57.6% (2008) Literacy rate, youth total (% of people ages 15-24) 75.6% (2008) Proportion of pupils starting grade 1 who reach last grade of primary, Both sexes 45.6% (2007) Proportion of pupils starting grade 1 who reach last grade of primary, Boys 48.6% (2007) Proportion of pupils starting grade 1 who reach last grade of primary, Girls 42.6% (2007) Percentage of population under 15 years 42.7% (2009) Population below poverty line (less than US$1 per day) 83.7% (2007) - 77 - IFESH Mid-Term Evaluation Report 2.0 Country Context Despite improvement in enrollment, it is unlikely that Liberia will achieve the Millennium Development Goal for Education (MDGE) of universal primary education for girls and boys by 2015, according to the 2010 Millennium Development Goals (MDGs) Report for Liberia, Achieving 2015: Progress, Prospects, Constraints.”2 This is all the more tragic in a country once known for its academic institutions.3 Liberia is still in “recovery” from 14 years of civil war that ended in 2003. “The status of Liberia’s achievements towards 2015 MDGE goals should be viewed within the context of the country’s fragile yet emergent state,” according to this report prepared by the MoE. Recent immigration of refugee families with children fleeing unrest in Ivory Coast further strains the education system. Education consumes nearly 12% of the state 2009-2010 budget.4 It is one of many priorities of the Government, which simultaneously focuses on: peace building, rule of law, governance, restoration and expansion of infrastructure, and creating conditions for economic growth and delivery of health and other social services to reduce poverty and hunger. Further burdening the education system, over 40% of Liberians are under 15 years old. 2.1 MoE Priorities The MoE identifies teacher quality and compensation, access, rural-urban disparity, income inequality, and culture and tradition that undervalue formal education, especially for girls, as the challenges to achieving universal primary education by 2015. More than 80% of teachers are under-qualified, according to the Minister of Education. The policy objectives of the Education Sector Plan 2010-2020 cited in the MDGs 2010 report include ensuring that all children (especially girls, individuals with special needs, street children and children with “manageable” emotional behavioral disorders and learning disabilities) start school at a developmentally appropriate age and complete primary level education of a minimum stipulated quality. Secondly, that no individual is unable to access primary education because of socio-economic status. Thirdly, that there is greater efficiency, cost effectiveness and accountability in primary schooling. Lastly, that the school environment is conducive for all students to feel safe and at ease, especially girls. 2.2 USAID Priorities Liberia has the second-largest USAID program in Africa, with U.S. bilateral assistance of almost $230 million in Fiscal Year (FY) 2010. The Government of Liberia signed a $15 million Threshold Program in 2010 with the Millennium Challenge Corporation to strengthen indicators in land reform, girl’s education, and trade. USAID's post-conflict rebuilding strategy focuses on reintegration and, eventually, a longer-term development focus5 which hopefully will reduce aid dependency. Qualified teachers, financing, management capacity, and infrastructure were the major challenges for education in Liberia identified by USAID/Liberia in its 2009-2011 strategy, with major investments in education averaging over $20 million per year. USAID/Liberia Education Sector programs have been focusing on filling gaps in human resource capacity at tertiary level institutions and the MoE, upgrading teachers, infrastructure, and learning materials support. Major USAID/Liberia partnerships with the MoE include supporting the LTTP that addresses education quality with fast-track teacher training in new child-focused methodologies and student monitoring techniques. Core Education Skills for Liberian Youth (CESLY) condenses six years of primary education into three, including integrating life skills, geared toward some 67,000 over-aged and out-of-school youth who missed out on formal schooling. The Ambassadors’ Girls Scholarship Program (AGSP) enabled some of the most at-risk girls and boys to stay in school, though its funding ends this year. Adult literacy, delivery of one million new and used books - 78 - IFESH Mid-Term Evaluation Report to 1,700 school libraries with training on their use, and development of TLMs were other components. The Early Grade Reading Assessment Program (EGRA) is showing results in improving reading achievement. 2.3 IFESH Alignment with the MoE and USAID The deployment of AVEs in Liberia aligns directly with three MoE education objectives in which USAID is making major investments: teacher training to replenish the depleted ranks of qualified teachers, improving the quality of education through better sector planning and restructuring, and the quest to produce sufficient TLMs. Guided by talks and a Memorandum of Understanding (MOU) signed with the MoE in 2009, IFESH has fielded 16 AVEs, eight each for the academic years 2009-2010 and 2010-2011 – the most of any AEFA country. The MoE request was for expertise to assist with the large numbers of teachers receiving fast-track training at rural teacher training institutions, and at the University of Liberia, teacher training personnel as well as teachers, and for the ministry itself as it developed policies to guide and regulate the rebuilding system. Private tertiary host institutions were also approached or approached IFESH for volunteers. It was often stated that all of the public and private tertiary education institutions are working toward the same goal: developing human capital to reconstruct Liberia. The host institutions that approached IFESH emanates from relationships that stretch back to the sixties between Liberians and IFESH sister organizations. Several people interviewed, including IFESH staff, had known Reverend Leon Sullivan or worked in the International Opportunities Industrialization Center (IOIC) skills training program or the predecessor of AEFA, the Teachers for Africa (TFA) program. All of the 16 AVEs over the past two academic years except one assigned to the MoE, have been a lecturer or trainer at Zorzor Rural Teacher Training Institute (ZRTTI), Kakata Rural Teacher Training Institute (KRTTI), the University of Liberia, Cuttington University, or the African Methodist Episcopal University (AMEU); the latter two host institutions are private and church governed. While their primary task has been to train teachers or lecture university students (many of whom will become teachers), nearly all AVEs have been involved in mentoring or developing faculty at the institutions. AEFA Liberia Performance Summary6 2009-2010 2010-2011 Total* Target** AVEs 8 8 16 24 LVEs 1 2 3 / CPD Training Workshops - 2 2 / Teacher Resource Ctrs. - 2 2 / Subgrants to NGOs/CBOs - 4 4 / Books Distributed 20,319 0 20,319 20,000 Pre-Svc. Teachers Trained M: 357 M: 532 M: 889 F: 148 F: 121 F: 269 1,505 T: 505 T: 653 T: 1,158 In-Svc. Teachers Trained M: 637 M: 483 M: 1,120 F: 112 F: 147 F: 259 700 T: 749 T: 630 T: 1,379 Ed. Administrators M: 26 M: 46 M: 72 F: 2 F: 8 F: 10 500 T: 28 T: 54 T: 82 LNGO/CBO Representatives M: 65 M: 0 M: 65 F: 628 F: 0 F: 628 200 T: 693 T: 0 T: 693 Users of Resource Ctrs. - M: 0 M: 0 - F: 0 F: 0 400 - T: 0 T: 0 Key: M= Male, F= Female, T=Total, - = no data reported, and / = no data available. * Total = The sum of the 2009-2010 and 2010-2011 columns. ** Target = 2009-2010 actuals + 2010-2011 target + 2011-2012 target. For indicators that were not used in 2009, Target = 2010-2011 + 2011-2012 only. - 79 - IFESH Mid-Term Evaluation Report Production of teaching and learning materials, including teaching aids for teachers and students, diagnostic instruments and forms, or manuals and multi-media materials has been a task for every AVE. Apart from the efforts of AVEs in producing TLMs, IFESH has arranged for the importation and distributions of about 20,300 donated books7 ; AVEs had the main responsibility for planning and executing distribution. AVEs have developed two Teacher Resource Centers (TRCs), one at the IFESH office in Monrovia, and one at Cuttington University in Bong County. In the area of policy and education management support, one AVE was assigned to the MoE where she managed the development of the new education policy reform legislation, and related policies and processes for budgeting and personnel. Half of the other AVEs had substantial policy and management assignments in addition to their teaching, related to: student affairs and administration, disciplinary procedures for students and teachers, and examination administration. In addition, almost every AVE worked with an LNGO or university extracurricular organization to build their capacity. These areas comprise the IFESH components for improving education performance in Liberia: teacher training and staff development; developing TLMs, establishing TRCs and distributing donated books; serving as policy and management advisors; and advising education-related LNGOs. 3.0 Improving Teaching, Systems, and Education Resources How has AEFA strengthened the capacity of teachers, administrators, and policymakers to enhance the quality of school teaching methodologies and pedagogical materials at the basic education level through technical aid and the training of teachers, using the main AEFA methods of teacher training, Teacher Resource Centers (TRCs), donated books, and serving as policy/management advisors? 3.1 Teacher Training AVEs demonstrate good teaching practice, with the up-to-date interactive student-centered style, with continuous assessment and feedback, which is common practice in the U.S. In Liberia, the norm has been that students try to memorize what the teacher says or writes on the board, with virtually no books or teaching aids. The result has been unacceptably low reading performance by Liberian primary students.8 Pre-Service and General Education One or two AVEs have been placed at three Liberian universities. At Cuttington University, the University of Liberia, and AMEU (2nd year only), AVEs taught classes where either there was no instructor available, or where overcrowding prompts splitting the class (usually more than 70 students). The subjects included: Mathematics, Instructional Materials and Production (required for education majors at University of Liberia), Introduction to European History (required for all students at Cuttington University), Gender Studies, Women and Social Change, Introductory English Composition, Fundamentals of Literature, and Ethics (team-taught). AVEs have not had opportunities to use or adapt the HIV/AIDS Education and Early Childhood and Elementary Education Training Modules developed by Bennett College for Women and Lincoln University. Pre-service teacher training takes place at KRTTI and ZRTTI and in the Colleges of Education at the Universities. The specially-designed fast-track teacher training program operates at ZRTTI and KRTTI. Trainers for the LTTP must undergo a two to four week course (longer for those without education backgrounds). The course is usually scheduled in the late summer before AVEs arrive. At ZRTTI, one of the most remotely located host institutions, one of the AVEs attended the LTTP training. At KRTTI, where there have been two AVEs for two years, they were scheduled to undergo the LTTP training but have not which means they could not be trainers in the LTTP. - 80 - IFESH Mid-Term Evaluation Report However, because most AVEs could not serve as instructors in the LTTP, the demonstration schools have profited at both ZRTTI and KRTTI. AVEs have taught at both of these demonstration schools and at those close to Cuttington University. In-Service Professional Development The IFESH application envisaged half-day Continuing Professional Development (CPD) workshops for working teachers. The reality has been that a comprehensive in-service professional development program for teachers is not yet in place in Liberia and resources are focused on pre-service training to get enough teachers into primary schools; as this need begins to be satisfied, secondary teachers are the next target. Consequently, professional development of teachers by AVEs has taken many forms, such as one-on-one sustained coaching and feedback. At the demonstration schools AVEs have offered various types of in￾service training such as: short weekend workshops, and other in-service series of workshops stretching over several weeks in topics such as: teaching methods, classroom management, positive behavior support, and learning support. AVEs have been responsive to the scheduling realities at the Rural Teacher Training Institutes (RTTIs) and host institutions, responding to needs that they have identified through their observations and consultations with colleagues. One of the most frequent uses of AVEs was in conducting classroom observations of pre-service and in-service teachers and providing feedback. AVEs felt that this important technique for teacher improvement needed to be conducted more systematically. Classroom and School Management Support An example of how AVEs interface with classroom and school management is around the issue of cheating. As one AVE put it, “Early on in classroom observation, it was clear teachers were desensitized to the culture of rampant cheating existent in overcrowded classrooms. With classes averaging 70+, teachers ignored students’ propensity to talk, look onto one another’s papers, openly use classroom notes from copybooks, and pass cheat sheets during exams.” At KRTTI Demonstration School, the AVE suggested concrete steps for managing exams, and a new policy of transparent posting of grades, without names, to shed light on performance. According to one AVE, the cultural shift met resistance that AVE and colleagues worked through, but were ultimately rewarded, by discernable improvement toward students’ positive attitudes in taking class time more seriously and constructing study aids for review. At KRTTI, an AVE coordinated regular staff meetings with the principal administrators to strategize the standardization of processes for management and operations, including academic year planning, and a disciplinary protocol for teachers as well as students. At the University of Liberia, an AVE had ongoing meetings with Political, Spiritual and Student Government Officers to coach and mentor on budget, procedural challenges, and producing official documents. Almost every AVE was requested to assist with drafting funding proposals for infrastructure or programs at the host institution. Some had “special projects” in their job description but few understood that proposal writing would be part of their work; AVEs wanted more preparation for this task. Education Innovations New products and practices introduced into the Liberian education system by AVEs include:  Compiling new syllabi – All AVEs reported that they developed the syllabi for the courses that they teach in Liberia, identifying information sources including textbooks, and creating accompanying learning materials.  Creating culturally sensitive TLMs – An AVE worked with KRTTI trainees to transcribe, illustrate, and created lesson plans using traditional Liberian folk tales.  Growing academic departments – The College of Education at Cuttington University is new, and currently there are less than 50 students. At AMEU, one of the main assignments of the AVE is to - 81 - IFESH Mid-Term Evaluation Report help establish the College of Education, researching and helping to write the business plan. Helping to create the program major in English was one responsibility of an AVE at Cuttington University.  Constructing diagnostic tools – Standard university admissions test based on Test of English as a Foreign Language (TOEFL) and International English Language Testing System (IELTS) have proven inappropriate due to the Liberian context. An AVE developed a modified version following the structure of those tests. It was to have been tested at the beginning of the 2010-2011 school year but was not because the arrival of the AVE was delayed to well beyond the start of the semester.  Assembling diversified TLMs – Student teachers in the Instructional Materials and Production course at the University of Liberia produced educational portfolios that they could carry with them into Liberian schools, consisting of a variety of instructional materials including assessments, tests, quizzes, games, and West African Examinations Council (WAEC) exams. When AVEs were not taking the lead, they are usually serving as peer mentors, working alongside Liberian colleagues as co-directors or co-teaching courses. Most AVEs were reported to be totally immersed in their universities and training institutions, serving on staff committees, advising supplementary learning opportunities like the veteran Social Studies instructor who helped to advise the Social Studies Club at the University of Liberia, bringing his own TLMs accumulated over many years. At least three AVEs have been invited into discussions on the LTTP and the new education reform revisions. One AVE, in recognition of his work, was asked to helped conduct a two-week European Commission Support to Education in Liberia (ECSEL) teacher training program at the University of Liberia. The AVE served as a co-host, presenter, moderator, and facilitator for the intensive program enabling administrators to learn in greater detail the objectives of the MoE strategy with the LTTP and solicit their input on more ways to improve teacher education and address the critical shortage of qualified teachers within Liberia. Challenges to AVE Teaching Effectiveness AVEs face many of the same barriers to effective teaching as their Liberian colleagues. At universities, like the schools they visited, the Evaluation Team was struck by the absence of books and other TLMs. Another major challenge is the under-preparedness of a significant proportion of university students and teacher trainees. As part of the ceasefire ending the war, some combatants received free university tuition as their exit settlement, a chance to earn a future. The other reason is the 14 year disruption in social services, where many fled the country, and schooling that was operational was piecemeal. Other challenges common to all teaching institutions in Liberia include: lack of water, interruptions to electricity supplied by generators, insufficient or inadequate faculty housing, and limited Internet access. The Evaluation Team observed an LTTP training session at ZRTTI of 50 students, who were grouped to discuss handouts and prepare responses. Apart from allowing trainees to use verbal and visual learning as a way to supplement reading large amounts of material very quickly, one could observe how stronger students facilitated learning by students without the same reading or speaking competencies. To assess changes in knowledge levels, most AVEs used pre-post tests where they were appropriate, usually for the university courses, and for the in-service workshops. For most other in-service professional development methods, like classroom observations, one-on-one coaching, and working with policies and procedures at the demonstration schools and universities, the documentation was mainly in the form of the quarterly reports, with anecdotal assessment feedback. Ongoing Potential Roles for AVEs In recognizing the mentoring/supervisory role of AVEs, university administrators proposed that they continue and develop more structured staff development programs for the rapidly growing tertiary institutions. While there is a shortage of instructors in the LTTP, a role more AVEs could fill if they underwent the training, there is equal demand for providing in-service professional development at - 82 - IFESH Mid-Term Evaluation Report universities and RTTIs, delivering courses where no staff members are available, structuring required courses, or starting new education faculties. University and RTTI administrators have requested more AVEs to serve as English, math, and science instructors at training institutions, demonstration schools, and other short￾staffed primary and secondary schools in the area. The latter is the focus of the Peace Corps; IFESH and Peace Corps are communicating and should continue to do so to coordinate their efforts. The lack of Information and Communications Technology (ICT) facilities and instructors was a recurring theme, for improving the capacity of faculty and administrators, as well as students. 3.2 TRCs and Book Donations “Without textbooks or computers, I have had to expend personal resources for photocopying instructional materials for students,” said one AVE, reflecting the experience of most. Some AVEs arranged for donations of books and materials from their own collections, from their churches, and other groups in the U.S. One AVE who arranged for 200 new and used dictionaries found that most of the university students receiving them had never used one before. IFESH Field Office staff have reported establishing two TRCs. The first is at the IFESH office in Monrovia. It provides Internet service, books and research materials to IFESH volunteers, and their university and LNGO counterparts. By the end of the first year, it was reported to be having 15 users per day; the categories of users are not specified. The other TRC is at the Cuttington University, and opened in April 2010. It serves pre-service and in￾service teachers throughout Bong County; it also supports literacy and fluency initiatives on the campus. Two AVEs staff the TRC, along with several volunteers. Since opening, the TRC has served approximately 200 university students, university faculty, pre-service teachers, and in-service teachers. Faculty and in-service teachers come for the teaching resources, including textbooks, teaching materials, supplies, and the computer. Students and pre-service teachers come for direct tutoring and teacher training. Sustainability is the challenge for the TRC at Cuttington University TRC. A Cuttington University alumnus co-directs the TRC but may not be able to continue because no remuneration has been forthcoming. Good students were identified to tutor voluntarily. One AVE is transferring his expertise in tutoring pedagogy. There is insufficient evidence other than anecdotal to assess the impact of the TRC. Another TRC planned for the University of Liberia is pending, awaiting the repeatedly delayed move to the new Fendell Campus, a 30 minute drive outside of Monrovia. This is the second year in Liberia for the AVE responsible for this project; it will be important that this work continues even if the AVE does not renew for a third year. An AVE based at Cuttington University worked with an IFESH Monrovia Local Volunteer Educator (LVE) to plan and distribute 20,000 donated books in 2009. They took the 2007 school census and distributed books according to the types of schools and institutions in the area, and the population distribution, targeting a ratio of four students per student. Each recipient signed a register. The books included: dictionaries, language arts, social studies, science, mathematics, vocation skills, health, and general topics. In spot visits to book donation recipients at schools during this evaluation, there were mixed results. It appears that donated books were distributed according to IFESH records. At one school that had been an American-funded model school before the war, there was adequate shelf space for the books – mainly textbooks – and a staff member said students made use of them. At another school some books were on shelves; the others were still in boxes seen by the Evaluation Team through the windows of a storeroom. A good practice was at a school where the supplementary reading books that had been donated by IFESH were kept on shelves in the Principal’s office where borrowers – teachers and students – signed them in and out. - 83 - IFESH Mid-Term Evaluation Report New books were not being utilized at the ZRTTI Demonstration School where the Evaluation Team talked to administrators and teachers and observed classes. Typically, there were no books evident in the classrooms. New textbooks (produced by the MoE with USAID support, for various subjects) were in boxes in the principal’s office. Delayed distribution, we were told, was because enrollment had increased since the books had been ordered the previous semester, and they were awaiting consultation with the Parent Teachers Association (PTA) on an accountable way to allocate the books since each child would not have one as intended. “We don’t have a culture of using books, we must reintroduce it,” was the observation of Professor Abdullai, of the University of Liberia, an authority on education, who has worked at all levels of the system. With book donations as an ongoing feature of AEFA, there is a need to review the effectiveness of this component. 3.3 AVE Policy/Management Support Minister of Education Gongar stated emphatically, “the IFESH contribution to capacity development is the most important thing the organization can do to improve Liberian education.” He underscored the value of IFESH-supplied policy support in the short-staffed MoE, to help establish fundamental frameworks to guide all the education sub-sectors – work that otherwise would have been delayed. Minister Gongor is referring to one AVE with 15 years of experience as a program analyst in the government bureaucracy in California, who was assigned full-time to the Ministry of Education; this is her second year. Her assignments have evolved over her two tours, based on building a trust relationship with supervisors who knew what they wanted who were respectively, the planning head and policy advisor, who became the Minister. In 2009-2010, she worked on the assessment of the education sector, helping to coordinate with university officials and other stakeholders (they were experts for each sub-sector – e.g., early childhood, primary, vocational). That work culminated in the Law Reform Act. She provided daily support convening meetings, worked on the research with teams, chaired the management section of the report, and edited the entire document. The Evaluation Team attended the final consultative conference for stakeholder input at Cuttington University on February 3-5, 2011, before submission to become law. In 2010-2011, priorities shifted and the AVE’s assignments become more business process re-engineering included a salary/remuneration study where they investigated ghost employees and why employees were not getting paid, and personnel efficiency. Her main constraint, though initially an advantage, has been her ambiguous role in the Ministry. She has earned respect over time. The biggest results indicator, she said, was that Ministry staff and others “actually attend her meetings” and there was greater cooperation when they didn’t perceive her as a threat and understood her role. Building relationships was a challenge – there was no incentive for people to cooperate initially. She commanded no resources; they then saw that senior management allocated resources that she needed, like transportation. Eventually they came together as a team, with her mentoring the “quite capable young people in the office.” A Scott Family Liberia Fellow had a similar function within the Ministry (refer to Section 7.1 Comparative Organizations). IFESH saw support to policymakers as a way to make systemic improvements in the delivery and quality of education, with AVEs working alongside counterparts. However, like in the area of CPD, IFESH could not foresee the specific policy areas where the education system needed assistance. Consequently, AVEs have been resourceful: one conducted her own informal rapid appraisal at the KRTTI and developed a responsive work plan, in consultation with the administration. The result was workshops in some areas and one-on-one coaching with teachers to regulate grading, absenteeism, and classroom management policy and procedure. This same AVE’s experience highlights the relationship between policy development and policy implementation. Her work plan included ensuring that the grading and curriculum guidelines already established by the MoE and/or Margibi County were followed with greater accuracy among in-service teachers. - 84 - IFESH Mid-Term Evaluation Report In contrast, the AVE at ZRTTI made little headway with the advisory portion of his assignment: the administration did not see the need. The AVE fully applied himself to teaching at the ZRTTI Demonstration School and later became the only AEFA LTTP-certified trainer. Monitoring policy/management advisor assignments, as shown by these examples, is important because AEFA managers need to know early whether what the challenges are. At the same time, the policy/management advisor job description tasks can often be general because the advisor helps to define them. In other USAID funded policy advisor assignments, one of the main monitoring and evaluation tool is client satisfaction forms. The success of AEFA in providing a policy analyst to the MoE exerts pressure on AEFA to locate an equally qualified successor and have a smooth transition without losing the momentum. The Minister of Education is one of the most powerful advocates of IFESH in Liberia. The Minister now wants an expert AVE to help establish the curriculum design framework in preparation for an institute while also continuing the business process re-engineering (BPR) tasks. An important lesson learned from the successes of the policy advisor assignments – as well as the advisor who was “not needed” – point to describing the assignments now that there has been good experience, including that the AVE will need to evolve their work plan. Another lesson is that in recruitment, it is not only skills that are important; the incumbent must also have the temperament and attitude conducive to the bureaucratic environment. Experience also points to a two year work plan to accomplish impact. 3.4 IFESH and USAID Crosscutting Themes: Gender Equity, HIV/AIDS, and Community Involvement The very fact that the majority of AVEs are women makes them role models for gender equality, observed the Minister of Education when meeting with the Evaluation Team. The MoE, USAID and host institution representatives interviewed named the large gender imbalance in favor of men, as teachers and administrators, as a major challenge. They supported AVEs providing workshops and including in their courses training about the educational disparities that exist in the classroom between boys and girls and providing tools to teachers to improve learning outcomes for girls. Mobilizing community support as an important element is recognized by the MoE as a component in programs they are implementing with USAID. Since AVEs function at the teacher training level, their opportunities for promoting community involvement are mainly limited to the demonstration schools at KRTTI and ZRTTI. Gender awareness and HIV/AIDS prevention are integrated into the LTTP curriculum. There was little familiarity by the MoE or host institutions of training by IFESH or other service providers on HIV/AIDS prevention and testing, and creating learning environments that are respectful of people living with HIV/AIDS or affected by HIV/AIDS (e.g., orphans). Half of AVE reports recorded that the AVE had addressed gender in their work. This indicates the need for IFESH to identify the reasons for those who did not and to address this in both the Pre-Departure Orientation Workshop (PDOW) and in-country orientation, and perhaps through ongoing training. 3.5 Use of LVEs IFESH Quarterly Reports are largely silent on the work of the two LVEs in Liberia, and inconsistent in reporting when LVEs began. There are different understandings on the definition of an LVE between an AVE, and the IFESH HQ and IFESH Field Office. An AVE reports having “local volunteer educators” working at the TRC at Cuttington University but they are not counted as LVEs for monitoring purposes. The two LVEs in Liberia identified by the Country Representative were interviewed by the Evaluation Team. One described his work as: to provide logistical support to AVEs, support book donation distribution and - 85 - IFESH Mid-Term Evaluation Report TRCs, and assist the Country Representative in communicating with potential new host institutions and in representation generally. His background includes teaching and senior NGO leadership roles. He works on a flexible part-time basis of 20 hours per week. He seems to function as a program assistant, not unlike the staff member with the title Administrative and Program Assistant. The other LVE understood her engagement to be on a retainer basis, functioning as an as-needed adviser. She served as Minister of Education during the war and is a Ph.D. holder with extensive experience in all aspects of education including teacher development. Terms of reference for the two LVEs on record were not available for the Evaluation Team while in Liberia. At the beginning of the 2010-2011 academic year, the IFESH Annual Report stated that it anticipated a substantial increase in the engagement of LVEs in Liberia to raise the number of teachers receiving CPD training. However, during the visit of the Evaluation Team in February 2011 LVEs were not conducting or planning CPD training. 3.6 Mobilizing Public/Private Partnerships No public/private partnerships were in process to the knowledge of all parties interviewed. 4.0 Strengthening Education LNGOs How effective has AEFA been in providing technical assistance and knowledge to increase the capacity of Local Non-Governmental Organizations (LNGOs) to implement sustainable development programs, primarily in the area of education? 4.1 LNGO Capacity Building Activities IFESH envisaged supporting education by increasing the capacity of LNGOs and education stakeholders to implement sustainable development programs. When beginning implementation, this was a mandate for every AVE and a change from the IFESH original proposal assigning one AVE in each country to this task. The most dynamic LNGO engagement is that of the AVE assigned to the MoE. Because of her background in microfinance, the AVE identified the Foundation for Women (FFW), which provides training workshops for women. Workshops include developing businesses, managing loans, and addressing basic literacy training. FFW mobilized around policy changes so that market women can receive social security and use their cellphones to make payments, providing new advantages to rural women. The AVE has also been helping them to develop a simple monitoring and evaluation system to document performance for their donors, and to address the negative impact of early pregnancy on girls’ lives, especially in rural areas. Other AVEs have or are working with the Young Men Christian Association (YMCA), We Care (Bong County), and extracurricular programs at AVE institutions. In the first year, nearly all AVEs had an LNGO activity. Some were quite involved while others were less so, due to their workload or a protracted time to assess needs and develop a plan. This academic year they were instructed by IFESH to focus on teacher support. 4.2 Use of Small Grants Three small grant awards have been approved according to interviewees, though records from IFESH only specify one. The first grant was provided to the Friends of Liberia to build a poultry house for the chickens and ducks at KRTTI to generate supplemental income to support student activities including establishment of a children’s reading program and a media lab at KRTTI. Another objective was to enhance the scope and capacity of KRTTI's program by providing agriculture and animal husbandry experiences to complement the curriculum. - 86 - IFESH Mid-Term Evaluation Report The Country Representative approved a grant for the YMCA for their community schools. A third grant has been approved for the Foundation for Women (FFW) in Liberia to assist with their adult literacy program, paying teachers’ salaries in community schools, scholarships for ICT in universities, and for scholarships for females. They are awaiting disbursement; each grant is $3,000. Documentation was unavailable on how the number of beneficiaries of support to LNGOs is calculated. There is no monitoring data since the only report required is at the end of the project. 5.0 Organizational Effectiveness Does the IFESH field team – Country Representative, Field Office staff, and American and Local Volunteer Educators (AVEs and LVEs) – have the tools they need to succeed? Are the necessary organizational structures (people, processes, and technologies) in place to support and sustain AEFA? There was a view that if IFESH is to grow, it should align guidelines from one country to the next – using the best practices from them all: as in the “A Big Mac in Ethiopia should taste the same as a Big Mac in Liberia” theory of standardization. This Report conveys such feedback to IFESH management for consideration. 5.1 Volunteer Recruitment, Performance, and Support 5.1.1 Host Institution Identification/Volunteer Job Descriptions and Placement Process/Host Institution Resource Allocation and Support Host institutions are asked annually to complete job description forms to request an AVE. Most host institutions interviewed by the Evaluation Team requested a longer turnaround time than two to four weeks allowed this year for receipt and completion of the request. This is due to the need to compile input from various departments and identify housing resources. Most interviewees found the job descriptions vague or too general and commented that they did not spell out what they would be doing. It was suggested that if the program was more tailored, job descriptions could be more defined. They spoke about wanting a work plan before they arrived on site; however, this could not be easily done in advance. Housing in rural areas is a scarce commodity, especially with accompanying services of water, electricity, and sanitation. The rural host institutions provide housing as their in-kind contribution, but expressed the pressure they are under as their regular faculty size continues to grow and they must make decisions in allocating housing between permanent faculty and staff, IFESH, and other volunteers from the Peace Corps and Volunteer Service Overseas (VSO). In Monrovia, housing is also scarce and accommodations for AVEs are rented on the open market, which is tight. In general the host institutions accord AVEs the rights and privileges of visiting lecturers, apart from salary. KRTTI senior management has had to be involved, along with IFESH Field Office staff, in adjudicating conflict between AVEs over living together. Both RTTIs indicate that a shared house is what they are able to allocate for IFESH volunteers, requiring that AVEs arrive prepared to share. 5.1.2 Orientation and Training Over half of volunteers said the PDOW in Scottsdale, Arizona could be improved to make better use of the time. They said they had not been fully prepared for their assignments and living in Liberia by the orientation in Scottsdale, and to a lesser extent, by the in-country orientation. They wanted more time with the Country Representative, and wished they had known more about:  Their assignment, who they would be working with, and which materials including TLMs they could bring  The LNGO they would be working with - 87 - IFESH Mid-Term Evaluation Report  Health facilities and what it would be like to go to the hospital  Their new home and how to buy groceries  Language training, about Pele with English as a second language, and the several dialects of English  Differences in the education system, from pre-school to university level  Sexual harassment Most AVEs reported that they had received training in Scottsdale, Arizona at the PDOW, upon arrival in￾country, and at a workshop in October, which was mainly about reporting. 5.1.3 Volunteer Monitoring and Performance Assessment IFESH Field Office staff visit volunteers at host institutions on a monthly basis, using the opportunity to liaise with the host institution programmatically. Telephone contact is frequent for nearly all volunteers, except at the ZRTTI where telephone reception is poor. There was conflict about work and living together between two AVEs reported at one host institution. This situation was protracted and involved repeated intervention by the host institution and Field Office staff. The supervisor of each AVE completes a performance appraisal form on a quarterly basis, which the volunteer usually submits with his/her quarterly report, which is easier for the supervisor than submitting the assessment directly to the IFESH Field Office due to ICT limitations. During the evaluation, while every AVE was rated as performing their duties at a useful or very useful level, supervisors and management provided nuances about the performances of AVEs that were not evident on the performance appraisal forms. These responses suggest that the different method should be used for AVE performance appraisal. 5.1.4 Safety and Security The Evaluation Team was told that the U.S. mission in Monrovia has been drawn-down (to order the departure of all non-essential personnel and family members) more times than any in the world. As such, safety and security is a critical aspect for volunteers placed in Liberia. Policies and procedures for ensuring personal safety and communicating in the case of emergencies were known by all volunteers and staff. Two volunteers were not clear about emergency evacuation. The fact that most volunteers had received this information indicates that the information should be repeated often due to whatever reason an AVE might have missed or forgotten it. The Regional Security Officer (RSO) at the U.S. Embassy clarified that AVEs are considered private U.S. citizens. If registered with the embassy, they do receive embassy warden messages that are sent out to the registered American community residing in Liberia. American Citizen Services is the main U.S. mission contact for private U.S. citizens in Liberia. If registered, the volunteers would receive the warden messages via email. In the event of a crisis (assuming they are registered) they would be counted in the embassy’s general country evacuation plan. Wardens are assigned on a country level. 5.1.5 Office Support to Volunteers (Medical, Stipends, and General) Most volunteers have had no complaints with the use of the medical insurance benefit and the medical consultant provided through an arrangement with a local doctor. An exception was for two AVEs who experienced a long delay in receiving compensation for medical care, until IFESH HQ got involved. Interestingly, AVEs interviewed said there were no benefits from IFESH other than medical insurance, not recognizing as benefits provisions such as vacation leave, life insurance, and vision/dental if injured. Consistent late payment of stipends was disquieting for AVEs. The Country Representative attributed it to cash flow issues. - 88 - IFESH Mid-Term Evaluation Report 5.2 Communications Cell phones are the most common means of communication in Liberia and the way that everyone communicates: AVE to AVE, AVEs to IFESH Field Office staff, staff to host institution and MoE staff. Internet is available but generally it is slow. It is IFESH policy that AVEs do not communicate directly with USAID and the MoE, but do so only through the Country Representative. The education team leader thought this policy was unnecessarily restrictive. The exception is the policy adviser AVE who works directly with the MoE, and as a result, also communicates with USAID directly. The IFESH Country Representative liaises regularly with the LTTP, the Peace Corps, and other Non￾Governmental Organizations (NGOs) and donors. He was invited to a two-day presentation of the LTTP to talk about the involvement of AVEs at the RTTIs. Because of the profile of the Country Representative and the long history of IFESH in Liberia, IFESH enjoys recognition, the AEFA program less so. 5.3 Knowledge Management and Sharing/Use of ICT There was no evidence of archiving or sharing AVE innovations other than through Quarterly Reports with success stories submitted to USAID. 5.4 Field Office Operations 5.4.1 Financial Management The main issue uncovered in interviews with AVEs was the regular late payment of stipends; this was the case for both program years. Field Office staff indicated that it was due to cash flow problems because of late receipt of budgeted funds from the U.S. 5.4.2 Staff Performance Areas/Assessment/Training Field Office staff were clear about their job descriptions. There appeared to be overlap between the duties of the Program and Administrative Assistant and one of the LVEs. There was no formal performance appraisal system other than informal feedback. All staff orientation and training was performed by local or IFESH HQ staff. 5.4.3 Operational Issues A question was raised by the Field Office Accountant about whether there should be deductions from staff for the National Social Security Welfare Corporation contributions. He stated that they will register for and begin the deductions at an unspecified time. Otherwise, the Field Office functions effectively to the extent that the Evaluation Team could see. 6.0 Performance ME&R Is IFESH using rigorous monitoring and evaluation systems that produce evidence of results and support program improvements and sustainability? Address Monitoring, Evaluation, and Reporting (ME&R) of the country program as a whole, and from stakeholder perspectives. IFESH is on target to meet all of the targets set in its work plan (refer to AEFA Performance Summary chart), except for in-service teacher development and half-day CPD days. As indicated above, there are good reasons to review these targets and indicators, in the context of reviewing what makes sense given the “catch-up” mode that currently characterizes teacher training in the country. The reporting on the number of books donated does not speak to how the books are being utilized or contributing to learner performance. Another example of how indicators and reporting need to be improved is pre-post tests. Changes in pre-post test scores show subject knowledge gain but not necessarily changes in attitude or behavior. Taking an average of changes in pre-post scores is not meaningful because of the differences in subject matter and among the tests. What is useful is that AVEs are measuring - 89 - IFESH Mid-Term Evaluation Report performance and the importance of that feedback to them as the instructor, and most importantly to the students. AVEs at the PDOW were advised to devise other methods if not using pre- and post-tests. The strategy of having an AVE assigned to monitoring and evaluation has worked well in Liberia in that she plays a major role in compiling Quarterly Reports for IFESH HQ, where they are packaged together for USAID. In reviewing the Quarterly Reports in the context of this Mid-Term Evaluation, one appreciates that indicators were adjusted at the beginning of FY2011. How to transfer knowledge and the system from the current Monitoring and Evaluation (M&E) AVE to her successor is a concern. From the perspective of host institutions, they are generally pleased with the volunteers and their performance. However, when pressed, some host institution representatives voiced concerns about some of the volunteers’ social skills and adaptability. It was suggested several times that some of the volunteers, although experienced and well-qualified in their fields, have had trouble adjusting to living conditions in Liberia. USAID/Liberia is not satisfied with the IFESH work in Liberia and the reporting is doing little to change that feeling. There is a prevalent feeling among the USAID Mission staff, even the Mission Director, that IFESH is unconnected with the education program’s objectives. If there were clear program outcomes that all stakeholders agreed to, and a rigorous monitoring and evaluation system were in place and being used, then IFESH would be equipped to show the USAID Mission what progress is being made and how that ties into the USAID education strategy. The concerns USAID/Liberia and the host institutions have with the program or individual volunteers, are not being relayed back to IFESH. This suggests that this information is, in fact, not being collected and reported to IFESH for adjustments to the program. ME&R is not placed within the context of an overall framework for the AEFA program in Liberia that clarifies the outcomes of AVE activities that complement MoE and USAID/Liberia objectives. For example, performance baseline and outcomes do not always exist for AVEs who are called on to function as LTTP trainers, or to fill in for missing instructors at universities where they will be expected to demonstrate better teaching methods. Nor are clear outcomes always evident when AVEs are asked to set up new programs or innovations. 7.0 Implications for IFESH, USAID, and Liberia What are the successes, challenges, and opportunities of AEFA as an innovative, sustainable, and replicable model for accelerating education improvement? 7.1 Comparative Organizations It was commonly agreed by USAID/Liberia and host institutions during interviews that IFESH volunteers in Liberia have a comparative advantage over Peace Corps, VSO and the Japanese International Cooperation Agency (JICA), in being more experienced and highly qualified. The Peace Corps Response Volunteers and the Scott Family Liberia Fellows also provide the highly specialized policy and management advisors that AEFA sees as its competitive edge. Scott Family Liberia Fellows9 are country-focused, while the Peace Corps Response volunteers have the advantage of larger systems and budget support. 7.2 Gap Analysis and Sustainability USAID/Liberia believes that there are opportunities to collaborate with IFESH but there appear to be communications and program barriers. Certainly, Liberian education needs are huge and IFESH can source the appropriate human resources and provide the administrative systems. The communications barriers are both between USAID and USAID/Liberia and USAID/Liberia and IFESH. This appears to be due to the AEFA program status as a Washington-funded program which means that USAID/Liberia does not exercise full control. The Mission Director and Education Team members, however, stay abreast of the activities of AEFA and see it as a valuable resource for meeting its objectives. - 90 - IFESH Mid-Term Evaluation Report USAID Education Team staff does not find meetings with IFESH and its reports especially useful. This suggests that there should be better alignment of IFESH resources with the priorities and targets USAID/Liberia has with the MoE. This process has begun with the LTTP. It is clear to the MoE that larger numbers of IFESH volunteers could be utilized to meet many of the human capacity deficits in the education system. But to do this requires a more thorough and comprehensive planning process of all the partners, to identify those specific priorities and targets that best use the strengths and accumulated experience of IFESH in Liberia. The increasingly competitive funding environment also calls for this higher level analysis and planning 8.0 Recommendations The following recommendations are fairly specific to the AEFA program in Liberia. However, the Evaluation Report has other recommendations applicable to all AEFA countries. The recommendations should be discussed and implemented in coordination with IFESH HQ. Training and Innovations 1. IFESH should conduct a rapid appraisal to decide if AVEs should be trainers in the LTTP at the RTTIs (only one has been certified thus far) – with appropriate planning made for them to undergo the prerequisite training – or if they are better deployed to in-service training and working with the demonstration schools. 2. As soon as possible, IFESH should develop a strategy of how to expand AVE and LVE provision of In-Service Professional Development based on the lessons learned thus far. This could take the form of a workshop in corroboration with current AVEs, the MoE, host institutions, USAID/Liberia, and other knowledgeable parties such as the LTTP. The current LVE with expertise in this area should play a coordinating role and be tasked with following through on the strategy, working with the IFESH Field Office in Liberia. 3. As an indicator, the definition and targets of “CPD Half-Days” need to be revisited to reflect the realities experienced at Liberian tertiary institutions and the strategy of the MoE. 4. AVEs have not had opportunities to use the materials developed by Bennett College for Women and Lincoln University. IFESH should revisit how to utilize the valuable resources available through this collaboration. For example, Bennett College might be a resource for establishing the College of Education at AMEU to which an AVE is assigned. 5. In view of the finding that AVEs are developing syllabi and helping to develop an English major program and a department/school of education, the orientation of volunteers should include an overview of the Liberian education system and curriculum development guidelines. 6. IFESH should develop a strategy and plan to share and archive the innovative processes and products produced by the AVEs, for wider use in Liberia, and for contribution to the field of international education generally, to maximize the investment made. TRCs and Donated Books 7. Review the book donation operations, including the use of previously donated books to date, the supply chain strategy, security and a cost-benefit analysis. With plans to distribute another 20,000 books in 2011-2012 through the AEFA program, IFESH should develop a rationale and plan that takes into consideration the review findings. 8. The question of sustainability of TRCs through ultimate provision of a stipend or staff time allocation to manage the TRC should be part of discussions between the host institution and the Country Representative and/or IFESH HQ. The discussions should inform plans for any ongoing program. AVE Policy/Management Support - 91 - IFESH Mid-Term Evaluation Report 9. When an AVE is assigned to a policy/management support role, training at the PDOW and in￾country orientation should include a briefing on the current status of policy development and implementation in Liberia. Training should also address the opportunities and constraints of serving as an advisor. 10. Develop a transition protocol between outgoing and incoming volunteers to close the gap between July and September, especially for policy advisor volunteers, so that they can have a face to face hand off. For every position where continuity is important (most of them) have a face to face hand off, and if not in person, then via a Skype video call. Support to LNGOs 11. The processes and experiences in making small grants to LNGOs from Liberia should be reviewed and compared with other AEFA countries, to prepare simple written guidelines for soliciting, awarding, and monitoring the awards. The review task force should include IFESH HQ staff, Country Representatives, and AVEs. They should address who reviews and approves grant applications to ensure competitive consideration of the best use of the funds in each country. 12. IFESH HQ management should clarify if support to LNGOs continues to be an AVE responsibility. If so, it should be part of the orientation with host institutions and should be part of the AVE job description. Recruitment and Placement 13. Tell prospective volunteers as much as possible about their assignments, the challenges, and rewards of being an AVE, about being a role model, and about living conditions, housing, health, and security considerations in Liberia especially, at both the Scottsdale and in-country orientations. Also, include as much of this information as possible in the country handbook; ideally it should be provided to volunteers electronically or manually in Scottsdale, ahead of their arrival in-country. Organizational Effectiveness 14. Review the dispute resolution process, articulating how to record and action decisions taken. 15. It is imperative that IFESH volunteers arrive in-country on time. They will not be able to serve as teachers in the RTTIs if they arrive to country late. IFESH needs to keep in mind that their competition Peace Corps volunteers get to country on time, serve for two full years, and may be as qualified as the IFESH volunteer. 16. Housing is a very difficult problem for many of the host institutions and when faced with providing housing for a VSO, IFESH, JICA, and/or Peace Corps Volunteer. IFESH therefore has to make the case that AVEs are worth the expense and in some cases inconvenience. 17. IFESH needs to be able to provide the same level of institutional support for their volunteers as received by volunteers at Peace Corps, JICA, and VSO. Hosting a volunteer should help reduce the burden of a school director, not cause additional work. 18. Review Field Office structures and include a Volunteer Coordinator position, especially in countries with a large volunteer group size. 19. Assign a Knowledge Management Coordinator at IFESH HQ. Archive the instruments, syllabi, proposals, TLMs, and policy documents that AVEs produce. Establish a knowledge management system and protocols. In the AVE contract, require that a hard and electronic record of the assignment is left in the Field Office for handing over to the AVE successor. Assign a staff member to disseminate AEFA innovations and lessons learned. 20. Include conflict resolution for AVEs and Country Representatives at the PDOW and in-country orientation, and training to sensitize AVEs to their status as role models. Also consider role playing such scenarios at orientations to facilitate appropriate solutions from IFESH HQ staff and Country Representatives, and AVEs. - 92 - IFESH Mid-Term Evaluation Report ME&R 21. Easy-to-use instruments, like sign-in rosters and feedback forms should be used to assess how TRCs are utilized and how they can be improved. 22. Very simple forms should be prepared for completion by LNGO grant recipients to provide data on beneficiaries (who, how they benefited, and how the number is calculated) for monitoring and reporting purposes. The same instrument could have basic milestones so that there is some sense of the use of the funds before the final report. 23. Establish a formal performance appraisal process: a. For IFESH staff, a Personnel Appraisal System should be instituted by HQ, in collaboration with the Field Offices, with common tasks and key performance areas for similar positions. b. For volunteers, the appraisal form should be revised to elicit more useful feedback, and should be collected and discussed during the regular monitoring visits to host institutions. Implications for IFESH, USAID, and Liberia 24. Establish a communications/management protocol between USAID and USAID/Liberia, including who is the point of contact and the frequency and purpose of the communications. 25. IFESH and the Peace Corps Country Director have had a close relationship in the past. There are a number of synergies between the two groups in Liberia which should be exploited. The Country Representative was scheduled to meet with the VSO Director and the Peace Corps Country Director to discuss placement strategies of their volunteers. 26. The AVEs in the RTTIs could serve as mentors to the Peace Corps volunteers in the schools since Peace Corps volunteers typically work in junior and high schools (some Peace Corps volunteers also work in the RTTIs). 27. IFESH needs to look at how it can capitalize on its relationship with the President and long history in Liberia, and physical infrastructure associated with IFESH in the past. 9.0 Methodology and Information Sources The Evaluation Team interviewed eight AVEs while in Liberia, and two LVEs. The Evaluation Team interviewed the IFESH Country Representative and office staff, Education Office staff and Education Team Leader at USAID, the USAID Mission Director, personnel at the MoE, representatives of host institutions, and representatives at LNGOs working with volunteers and/or that are also receiving small grants. An AVE from the previous year was interviewed by telephone. Names for all interviewees are found in Appendix A. The following documents and publications were reviewed for this case study: Government of Liberia. (2010). Liberian 2010 MDGs Report – Achieving 2015: Progress, Prospects, Constraints. UNDP. http://www.undp.org/africa/documents/mdg/liberia_september2010.pdf. IFESH. (2009). AEFA – IFESH Technical Application. IFESH: Scottsdale, Arizona. IFESH. (2009). AEFA Quarterly Report, July-September 2009. IFESH: Scottsdale, Arizona. IFESH. (2009). AEFA Quarterly Report, October-December 2009. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, January-March 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, April-June 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Annual Report and Quarterly Report, July-September 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, October-December 2010. IFESH: Scottsdale, Arizona. USAID/Liberia. (2008). Investing in People: Education. Monrovia, Liberia. http://liberia.usaid.gov/sites/default/files/USAID%20Liberia%20Education%20Briefer.pdf. - 93 - IFESH Mid-Term Evaluation Report USAID. (2009) Cooperative Agreement Between USAID and IFESH. USAID: Washington, D.C. U.S. Government. (2011). Background Note Liberia. http://www.state.gov/r/pa/ei/bgn/6618.htm. 1 Tracking the Millennium Development Goals (n.d.). United Nations online MDG Monitor. Retrieved from http://www.mdgmonitor.org/country_progress.cfm?c=DJI&cd=262. Education and Population Data (n.d.). World Bank online. Retrieved from http://data.worldbank.org. Millennium Development Goals Indicators (2010). UN online. Retrieved from http://mdgs.un.org/unsd/mdg/Data.aspx. 2 Background Note: Liberia (2011). State Department online. Retrieved from http://www.state.gov/r/pa/ei/bgn/6618.htm. 3 IFESH AEFA Book Distribution Tables. The 2009/2010 shipment for Liberia was 521 boxes with an average 39 books each (2011). Received from IFESH HQ. 4 Achieving 2015 Progress, Prospects, Constraints: Liberia’s Progress towards the Millennium Development Goals (September 2010). Liberia 2010 MDG Report. Retrieved from http://www.undp.org/africa/documents/mdg/liberia_september2010.pdf. 5 Background Note: Liberia. 6 IFESH Revised Work Plan; IFESH Quarterly Reports; IFESH AVE Distribution Tables; IFESH AEFA Book Distribution Tables. Received from IFESH HQ. 7 AEFA Book Distribution Tables (Liberia) file. 8 EGRA Plus: Liberia – Quarterly Progress Report: October-December 2008 (January 2009). USAID/Liberia. Retrieved from http://pdf.usaid.gov/pdf_docs/PDACO430.pdf. 9 Scott Family Liberia Fellows Program Description (March 2007). Center for Global Development. Retrieved from http://www.cgdev.org/doc/about/Scott_Family_Fellows.pdf. - 94 - IFESH Mid-Term Evaluation Report Appendix F: Malawi Case Study 1.0 Summary By most indicators, Malawi is a country that has experienced a dramatic improvement in its overall economic development since 2000. The GDP growth rate is up almost three-fold to 4.73% which outpaces the annual population growth rate of 2.8%.1 Similarly, Malawi has seen some significant improvements in the education sector with enrollment rates, completion rates, and literacy rates. The country was primed for technical assistance to help increase the number of trained teachers. The strategic goals of the Ministry of Education, Science, and Technology (MOEST) and U.S. Agency for International Development (USAID)/Malawi are well-aligned, thus making it quite possible for the International Foundation for Education and Self-Help (IFESH) to define volunteer assignments and place them so as to make progress towards these goals. However, it is unfortunate that IFESH has not implemented the American Educators for Africa (AEFA) program in this manner. USAID/Malawi therefore questions the usefulness of AEFA in Malawi. There have been positive IFESH accomplishments in the country. Most of the host institutions are happy with the volunteers and the Country Representative enjoys a good relationship with the ministry. With little interaction from USAID/Malawi, IFESH placed nine American Volunteer Educators (AVEs) at three higher education institutions to meet specific education goals of each institution. The MOEST leaves placement decisions to the host institutions. Some of these volunteers have taken the initiative and created Continuing Professional Development (CPD) workshops that have been well received and gone far to help IFESH meet one of its Cooperative Agreement goals. The unfortunate nature of this success is that it has little to do with institutionalized policies or organizational direction but instead with the hard work of the individual volunteers. In order for IFESH to make a notable difference in Malawi and to help USAID/Malawi and the MOEST meet their aligned strategic goals, a deliberate planning process with them is required. This process would produce a country plan with specific outcomes to be agreed upon among all stakeholders, including host institutions. This planning would then direct commensurate changes in preparation of job descriptions, orientation of volunteers, and concrete outcomes for a monitoring and evaluation system that would be more useful to all stakeholders. 2.0 Country Context Malawi has made drastic improvement in increasing the net enrollment rates as well as primary completion rates according to the 2010 Malawi Millennium Development Goals Report prepared by the Ministry of Development Planning and Cooperation. However, it is unlikely Malawi will meet the targets of Goal 2 of the Millennium Development Goals (MDGs): achieving universal primary education. Failure to achieve this target is not due to lack of or failure of effort, but that the country started off with a very low baseline compared to other United Nations countries. It is likely, however, that Malawi will meet most of the other MDGs. Malawi: Education/Socio-Economic Indices2 Likely to achieve MDG 2 by 2015: Boys and girls alike will be able to complete a full course of primary schooling No Net enrollment primary education (%) 90.8% (2009) Primary Completion rates 59.2% (2009) Literacy rate, youth total (% of people ages 15-24) 86.5% (2009) Proportion of pupils starting grade 1 who reach last grade of primary, Both sexes 41.9% (2008) Proportion of pupils starting grade 1 who reach last grade of primary, Boys 42.1% (2008) Proportion of pupils starting grade 1 who reach last grade of primary, Girls 41.7% (2008) Percentage of population under 15 years 46.2% (2009) Population below poverty line (less than US$1 per day) 73.9% (2004) - 95 - IFESH Mid-Term Evaluation Report While free primary education was implemented in 1994, pupils still must buy books and materials (if they are available). The government recently rescinded the requirement of school uniforms. Primary education is eight years and pupils enter at seven years old. Secondary education is four years, but an adequate number of schools do not exist and fees inhibit many from enrolling. Malawi’s rural population accounts for 85% and most are subsistence farmers or work on large plantations.3 With the persistence of poverty, the majority of households are unable to meet their food requirements.4 Malawi was ranked 153 out of 169 countries in the 2010 United Nations Development Program (UNDP) Human Development Reports.5 2.1 MoE Priorities The MOEST created a strategic plan for 2009-2017. This plan was built upon three pillars: 1) access and equity, 2) quality and relevance, and 3) governance and management. These priorities traverse primary, secondary and higher education. The problem appears worse in higher education because the universities are small and while they have the desire to expand, they do not have the resources to do so. They also want to improve gender equity in higher education. Currently, the tertiary level female population is 40% and the desired goal is an even split. The MOEST proposes to better link basic and higher education. To accomplish this proposal, improved teacher education is a must. Primary school teachers are trained in four public Teacher Training Colleges (TTCs), with the aid of some private institutions. Secondary teachers are trained in three public institutions which are part of the tertiary system. The total output has remained at less than 3,000 for primary school teachers and about 400 for secondary teachers annually. The rapid expansion of primary and secondary education has increased the demand for trained teachers. The present supply is not enough to meet this demand. MOEST plans to open five new higher education institutions in the next five years. They recognize the need that must be filled. 2.2 USAID Priorities In the 2010 U.S. Congressional Budget Justification, USAID/Malawi was allocated $8 million in funding to improve basic education. This overall foreign assistance budget focuses on country-driven plans with an emphasis on the quality and relevance of basic education, serving the education needs of out-of-school youth and measure learning outcomes and results. It should also be noted that special emphasis was placed on the requirement that all programs will be monitored and reported on using the basic education indicators in the Foreign Assistance Coordination and Tracking System (FACTS) managed by the Office of the Director of U.S. Foreign Assistance. Therefore, any expenditure made toward the basic education goals must be measureable and reported to the overall system. USAID/Malawi states that their strategy is aligned with the MOEST’s strategic priorities as set out in its 2006 – 2011 Malawi Growth and Development Strategy and it incorporates major U.S. initiatives such as the Africa Education Initiative and the Millennium Challenge Account. USAID is responding to priority areas of need as set out by the MOEST. These needs include strengthening basic education, building and reinforcing education professional development skills and capacity, providing technical assistance to help streamline and strengthen education management information systems, promoting the role of communities and their abilities to mobilize resources for improvement, and supporting the decentralization process within the education sector. 2.3 IFESH Alignment with the MoE and USAID The AVEs placed in Malawi do very little to support the joint goals of USAID and the MOEST. In the first year of the program, with little interaction from USAID/Malawi or the MOEST, IFESH placed four volunteers at host institutions to meet specific education goals of each institution. The selections were made in coordination between the Country Representative and the individual host institutions. However, in the second year of the program, requests for volunteers were channeled through the MOEST, with - 96 - IFESH Mid-Term Evaluation Report USAID/Malawi being informed at each step. Each host institution, whether it is Chancellor College or Domasi College of Education, chose volunteers that could fill their institution’s specific needs. At Chancellor College, these placements tended to be “gap fillers” (teaching university classes where they needed teachers), but not making a systemic change nor addressing the combined strategy of USAID and the MOEST. At Domasi College of Education, volunteers have been placed to train teachers but with little consistency. While IFESH Quarterly Reports document that some of the volunteers conduct pre-service training for teachers, both primary and secondary, there is little evidence of the impact of this training. This is not to say that IFESH or the volunteers are not liked. Both IFESH Headquarters (HQ) and the Country Representative enjoy a good rapport with MOEST and that working relationship is much appreciated and respected. Further, Chancellor College values highly the volunteers placed within the ranks of their faculty. The volunteers bring skills and experience few others at the institution hold and there is some tangential training of fellow teachers. However, this training is informal and neither expansive nor systematic. There are also some activities being conducted by the volunteers, which are self-driven and not part of their assignments but are addressing some of the needs, e.g., one AVE’s work in conflict resolution. These good works are focused primary on a single task and assignment and do not align with the programmatic developmental goals of either MOEST or USAID/Malawi. IFESH states in its 2010 Annual Report that the country program’s main focus is basic education through teacher￾training in improved teaching methodologies and the development of pedagogical materials. The following sections analyze the accomplishments and challenges of the AEFA. 3.0 Improving Teaching, Systems, and Education Resources How has AEFA strengthened the capacity of teachers, administrators, and policymakers to enhance the quality of school teaching methodologies and pedagogical materials at the basic education level through technical aid and AEFA Malawi Performance Summary6 2009-2010 2010-2011 Total* Target** AVEs 4 5 9 14 LVEs 0 1 1 / CPD Training Workshops - 0 0 / Teacher Resource Ctrs. - 1 1 / Subgrants to NGOs/CBOs - 0 0 / Books Distributed 0 0 0 0 Pre-Svc. Teachers Trained M: 312 M: 176 M: 488 1,5125 F: 200 F: 88 F: 288 T: 512 T: 264 T: 776 In-Svc. Teachers Trained M: 167 M: 211 M: 378 F: 17 F: 87 F: 104 200 T: 184 T: 298 T: 482 Ed. Administrators M: 8 M: 75 M: 83 F: 4 F: 37 F: 41 50 T: 12 T: 112 T: 124 LNGO/CBO Representatives M: 487 M: 0 M: 487 F: 736 F: 0 F: 736 300 T: 1,223 T: 0 T: 1,223 Users of Resource Ctrs. - M: 0 M: 0 - F: 0 F: 0 400 - T: 0 T: 0 Key: M= Male, F= Female, T=Total, - = no data reported, and / = no data available. * Total = The sum of the 2009-2010 and 2010-2011 columns. ** Target = 2009-2010 actuals + 2010-2011 target + 2011-2012 target. For indicators that were not used in 2009, Target = 2010-2011 + 2011-2012 only. - 97 - IFESH Mid-Term Evaluation Report the training of teachers, using the main AEFA methods of teacher training, Teacher Resource Centers (TRCs), donated books, and serving as policy/management advisors? 3.1 Teacher Training The numbers being reported by IFESH in the September 2010 Annual Report, 422 pre-service teachers training out of the target of 500, shows that they are close to meeting this component. However, if one looks behind the statistics, it seems that most of these numbers are being generated by the individual effort of volunteers and not driven by the program. The volunteer at the Malawi Institute of Education (MIE), who is on her second term and is much liked by the host institution, has worked with AVEs at Chancellor College to design several Continuing Professional Development (CPD) workshops that have been received with praise. These workshops have created much of the success for this component. According to the IFESH Quarterly Report January-March 2011, five CPD workshops have been conducted, two of which were hosted at the Chancellor College. However, the problem remains on how to institutionalize this progress. Much of the other work is extraneous to this, or any of the other components. 3.2 TRCs and Book Donations AVEs are not providing support for these efforts in Malawi. 3.3 AVE Policy/Management Support AVEs are not providing this type of support in Malawi. 3.4 IFESH and USAID Crosscutting Themes: Gender Equity, HIV/AIDS, and Community Involvement All three crosscutting themes are addressed in the CPD workshops conducted by the three volunteers at the Malawi Institute of Education and Chancellor College. These in-service training workshops were focused on student management, however these issues were covered. Gender equity is especially relevant since the female volunteers themselves struggle with transferring knowledge because of the less than receptive nature of their male colleagues. Further, most of the volunteers have become active in their community, not through any program assignment, but on their own accord and initiative. 3.5 Use of Local Volunteer Educators There are no Local Volunteer Educators (LVEs) in Malawi. 3.6 Mobilizing Public/Private Partnerships The Country Representative states that she has been working to build a public/private partnership with other education organizations but to no avail. 4.0 Strengthening Education LNGOs How effective has AEFA been in providing technical assistance and knowledge to increase the capacity of Local Non-Governmental Organizations (LNGOs) to implement sustainable development programs, primarily in the area of education? There has been no support of LNGOs since their work with Domasi Youth Organization three years ago. 5.0 Organizational Effectiveness Does the IFESH field team – Country Representative, Field Office staff, and American and Local Volunteer Educators (AVEs and LVEs) – have the tools they need to succeed? Are the necessary organizational structures (people, processes, and technologies) in place to support and sustain AEFA? - 98 - IFESH Mid-Term Evaluation Report 5.1 Volunteer Recruitment, Performance, and Support 5.1.1 Host Institution Identification/Volunteer Job Descriptions and Placement Process/Host Institution Resource Allocation and Support Host institutions say they are presented with a list (or collection of resumes) of potential volunteers from which they could chose and then those volunteers are placed if housing is available. Job descriptions are written by the host institution. They are vague and often the actual work differs greatly from the original description. Aside from housing, very few other resources are made available to the volunteers by the host institution. 5.1.2 Orientation and Training A consistent comment the Evaluation Team heard, both from host institutions and from the AVEs, was that the volunteers are not well prepared for life in Malawi. The Pre-Departure Orientation Workshop (PDOW) does little to prepare the volunteers and the in-country orientation is too short. Further, the MoE and host institutions are not part of the in-country orientation so that relationship does not begin until volunteers begin their work. USAID/Malawi participates in the in-country orientation to brief AVEs. 5.1.3 Volunteer Monitoring and Performance Assessment Volunteers receive performance evaluations on an individual, ad hoc basis. Some receive evaluations by their supervisors at the host institutions while others do not. There are no formal assessments conducted by either IFESH HQ or the Country Representative. 5.1.4 Safety and Security Safety and security does not appear to be a concern in Malawi. Plans are in place for security issues if they arise and volunteers are provided with guidelines they should follow to protect their safety. Many of these rules though such as not driving at night and not taking public transportation, are regularly disregarded by volunteers. 5.1.5 Office Support to Volunteers (Medical, Stipends, and General) Medical coverage and other support policies are in place. Stipends are distributed by the Country Representative without incident. 5.2 Communications Communication is an issue in Malawi. The Country Representative has good communication with MOEST, talking with the Director of Higher Education on a regular basis. But communication with other parties – host institutions, USAID/Malawi, other policymakers, and officials – is lacking. A complicating factor is the relationship between IFESH and USAID/Malawi under the previous Education Team Leader. The Country Representative was given orders at that time to no longer communicate directly with the MOEST. The Education Team Leader wanted all communications with MOEST and the host institutions to be coordinated through that office. This restriction severely handicapped the Country Representative’s effectiveness in managing programmatic issues. Host institutions, USAID, and MOEST are not in communication during the volunteers’ term to gauge progress and align work toward combined goals. Further, even though IFESH HQ directs the Country Representative to visit each volunteer monthly, this connection is usually made via telephone and not on a monthly basis. IFESH would also be well served to improve external communications within the country by holding an “Open Day” and engaging the media. - 99 - IFESH Mid-Term Evaluation Report 5.3 Knowledge Management and Sharing/Use of ICT Internet is slow and expensive in Malawi so Information and Communication Technology (ICT) lags behind some other countries in Africa. Until the new ICT infrastructure is in place, which is being donated and installed by the Indian government, using these modern technologies will prove difficult. In the meantime, IFESH does need to establish a knowledge management system to collect data and accomplishments at the end of service terms and establish protocols for sharing these successes with other stakeholders and across Africa. 5.4 Field Office Operations The IFESH Field Office is small with only the Country Representative and a Receptionist/Accountant. There is no performance assessment process in place nor are there regular staff meetings, established hiring processes, or staff training plans. One could argue that none are needed with such a small staff. The Country Representative is well versed in the financial management of operating the country program. The hindrances appear to be created by dictates from IFESH HQ and not created in-country. For example, a yearly budget is created and then the budget is reviewed monthly and often that budget is adjusted downward. Not only does the monthly review create extra work for Field Office staff, that staff had budgeted for a year based on the process only to have fewer funds available. Another item that should be noted is the location of the office. There may not be an ideal location since the host institutions are spread out across the country. But currently, with the volunteers in Zomba and Domasi and the USAID Mission and MOEST are in Lilongwe, the office location in Blantyre is a problem. It is a great expense in time and travel costs for the Country Representative to visit the stakeholders and the volunteers. 6.0 Performance ME&R Is IFESH using rigorous monitoring and evaluation systems that produce evidence of results and support program improvements and sustainability? Address Monitoring, Evaluation, and Reporting (ME&R) of the country program as a whole, and from stakeholder perspectives. The Country Representative does use the established procedures to collect data and feedback from host institutions. This data is rolled up to the Quarterly Reports submitted to USAID. While the numbers of beneficiaries is good to know, it does not go far enough to produce evidence of any results or sustainability. A common criticism, from both USAID/Malawi and the host institutions, is continuity. While there may be many reasons for this response – terms of volunteer service, changing of host institutions, shifting of USAID priorities – a functioning monitoring and evaluation system would help provide these stakeholders with evidence of IFESH success. Without it, as Malawi is now, none of the stakeholders are aware of any long￾term, systemic changes occurring. These changes could be happening but there is no process to document, track, and conclude that they are in fact occurring. From the perspective of host institutions, they are generally pleased with the volunteers and their performance. However, when pressed, some host institution representatives voiced concerns about some of the volunteers’ social skills and adaptability. It was suggested several times that some of the volunteers, although experienced and well-qualified in their fields, have had trouble adjusting to living conditions in Malawi. USAID/Malawi is not satisfied with AEFA in Malawi and the reporting is doing little to change that feeling. There is a prevalent feeling among the USAID Mission staff, even the Mission Director, that IFESH is unconnected with the education program’s objectives. If there were clear program outcomes that all stakeholders agreed to, and a rigorous monitoring and evaluation system were in place and being used, then - 100 - IFESH Mid-Term Evaluation Report IFESH would be equipped to show the USAID/Malawi what progress is being made and how that ties into the USAID Education Strategy 2011-2015. The concerns that the host institutions have with the program or individual volunteers are not being relayed back to either the IFESH Field Office or IFESH HQ. USAID/Malawi has expressed its concerns with the AEFA program to both the IFESH Field Office and IFESH HQ; however, they have not been satisfied by the responses from IFESH. This suggests that IFESH is not collecting pertinent feedback about the program from stakeholders and making commensurate program adjustments. 7.0 Implications for IFESH, USAID, and Malawi What are the successes, challenges, and opportunities of AEFA as an innovative, sustainable, and replicable model for accelerating education improvement? There is a potential for growth for IFESH in Malawi. Most host institutions and the MOEST still see value in IFESH and the volunteers they place. With a new USAID Education Team Leader arriving soon, IFESH may be given a chance to realign and reconnect itself with USAID/Malawi. There are some significant challenges to expanding the AEFA program and many of those are not within the control of IFESH. The geographic expanse of the higher education institutions and lack of ICT and infrastructure will continue to be a hindrance to Malawian education and those organizations attempting to provide development assistance. As MOEST adds new higher education institutions and expands the teacher base for secondary schools, there are opportunities for IFESH. Peace Corps has begun to send volunteers to Malawi, who are providing teacher training and are posted to institutions very similar to those of IFESH – similar education and teaching experience. At the MIE, Volunteer Service Overseas (VSO) is similar, with the exception that MIE pays them a stipend in addition to providing the accommodation, security, furniture, and electricity. At Domasi College of Education, Japanese International Cooperation Agency (JICA) volunteers are similar and seem to have better reputations with the Principal. In addition, they serve a full two-year tour of service (as do the Peace Corps volunteers). 8.0 Recommendations The following recommendations are fairly specific to the AEFA program in Malawi. However, the Evaluation Report has other recommendations applicable to all AEFA countries. The recommendations should be discussed and implemented in coordination with IFESH HQ. Placements 1. Placements should not be made only to fill gaps but where the volunteers can have the greatest impact in forwarding USAID/Malawi goals. Communications 2. A communications plan needs to be developed that includes improving relations with USAID/Malawi and building on the existing good communication between the Country Representative and MOEST. 3. IFESH needs to engage the media and promote the good works IFESH is accomplishing. IFESH Malawi would also be well served to improve external communications within the country by holding an “Open Day” and engaging the media. Orientation 4. Volunteers need to be better prepared for living in Malawi. The action needs to happen both during recruiting (weeding out those who will not be able to make the adjustments) and at the - 101 - IFESH Mid-Term Evaluation Report PDOW and in-country orientation. Real life training needs to be provided so the culture shock is not too overwhelming for the volunteers resulting in early terminations. 5. USAID/Malawi, MOEST, and the host institutions should be part of the in-country orientation. ME&R 6. The Monitoring and Evaluation (M&E) framework needs to address relationships, processes, and program performance against outcomes developed jointly with USAID, MOEST, and host institutions. Organizational Structure 7. IFESH, USAID/Malawi, and MOEST need to work together to develop a work plan that will include the job description for the volunteers and address the programmatic goals of USAID/Malawi and the MOEST. This work plan should come from a five year perspective in order to ensure continuity of work as volunteers change every year. 8. In times when less funding is available from donors, IFESH must demonstrate visible and measurable impact to justify its cost. In the first two years of the AEFA program, IFESH placed nine volunteers and impacted just over 3,000 people in a country of over 15 million. 9. IFESH needs to establish a knowledge management system to collect data and accomplishments at the end of service terms and establish protocols for sharing these successes with other stakeholders and across Africa. 9.0 Methodology and Information Sources The Evaluation Team interviewed five AVEs (three in person, and two by phone) while in Malawi. The Evaluation Team also interviewed the IFESH Country Representative, the USAID Education Office staff and Education Team Leader, the USAID Mission Director, personnel at the MOEST, and representatives of host institutions. Names for all interviewees are found in Appendix A. The following documents and publications were reviewed for this case study: Ministry of Education, Science and Technology. (2008) National Report of Malawi for UNESCO. http://www.ibe.unesco.org/National_Reports/ICE_2008/malawi_NR08.pdf. UNPD. (2005) Human Development Report. http://hdr.undp.org/en/media/HDR05_complete.pdf. World Bank. (2010) World Development Indicators database. http://data.worldbank.org/data￾catalog/world-development-indicators/wdi-2010. IFESH. (2009). AEFA – IFESH Technical Application. IFESH: Scottsdale, Arizona. IFESH. (2009). AEFA Quarterly Report, July-September 2009. IFESH: Scottsdale, Arizona. IFESH. (2009). AEFA Quarterly Report, October- December 2009. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, January-March 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, April-June 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Annual Report and Quarterly Report, July-September 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, October-December 2010. IFESH: Scottsdale, Arizona. USAID. (2009). Cooperative Agreement between USAID and IFESH. USAID: Washington, D.C. - 102 - IFESH Mid-Term Evaluation Report 1 World Development Indicators (2010). World Bank online. Retrieved from http://data.worldbank.org/country/malawi. 2 Tracking the Millennium Development Goals (n.d.). United Nations online MDG Monitor. Retrieved from http://www.mdgmonitor.org/country_progress.cfm?c=DJI&cd=262. Education and Population Data (n.d.). World Bank online. Retrieved from http://data.worldbank.org. Millennium Development Goals Indicators (2010). UN online. Retrieved from http://mdgs.un.org/unsd/mdg/Data.aspx. 3 About Malawi (n.d.). UNDP Malawi online. Retrieved from http://www.undp.org.mw/index.php?option=com_content&view=article&id=57&Itemid=213&58930effcf1f629ed28c9ca6846cfdcd= 9aabc51cb0a244b61705fa507324f444. 4 Ibid. 5 International Human Development Indicators (n.d.). UNDP online Human Development Report. Retrieved from http://hdrstats.undp.org/en/countries/profiles/MWI.html. 6 IFESH Revised Work Plan; IFESH Quarterly Reports; IFESH AVE Distribution Tables; IFESH AEFA Book Distribution Tables. Received from IFESH HQ. - 103 - IFESH Mid-Term Evaluation Report Appendix G: Non-Visit Countries Case Study 1.0 Summary The Evaluation Team did not visit Kenya, Nigeria, and Senegal. As such, the countries these three countries were not given the same level of intense scrutiny as the countries that were visited. The research was limited to telephone interviews held with the Country Representative and U.S. Agency for International Development (USAID) Education Team Leader for each respective country. This case study combines the findings and recommendations from all three countries. It is important to note for Kenya, that the Country Representative had only been in his position for two months so he had little to offer in historical context. 2.0 Country Context The following are brief descriptions of each of the three countries: Kenya Since free primary education was introduced in 2002, the net enrollment rate rose to 86.5% by 2006. Kenya has done a fair job at addressing gender parity, and reached close to even in 2004. Gender equity in primary enrollment however continues to experience sharp regional disparities, being particularly low among girls in arid and semi-arid regions. In addition, output and quality assessment studies reflect problems of quality in teaching and learning. In 2009-2010, there were two American Volunteer Educators (AVEs) working in Kenya and one was placed at the Kenya Education Staff Institute (KESI) in Nairobi, Kenya and the other was placed at the Kenya Institute of Education (KIE) also located in Nairobi, Kenya. Both of the AVEs focused on teacher training, education management, and policy at the Ministry of Education (MoE). For the 2010-2011 year, the number of AVEs doubled to four. Three of them are located at KESI and the fourth is located at KIE. All four are focusing on education management, teacher training, and Teacher Resource Centers (TRCs). There are no Local Volunteer Educators (LVEs) working in Kenya for either the 2010-2011 year or the 2009-2010 year. Nigeria Currently, 88.8% of school-aged children are enrolled in school. Nevertheless, regional differences are stark. State primary completion rates range from 2% to 99%. In particular, progress needs to be accelerated in the north of the country if the Millennium Development Goals (MDGs) target is to be met. Low completion rates reflect poor learning environments and point to the urgent need to raise teaching standards. The government is also focusing efforts to accelerate progress and reduce regional disparities. In 2009-2010, there were two AVEs working in Nigeria and they both focused on teacher training and book donations at Sa’adatu Rimi College of Education located in Kano, Nigeria. For the 2010-2011 year, the number of AVEs increased to five. One AVE is located at the State MoE in Kano, Nigeria focusing on teacher training and MoE Policy. Another AVE is located at the USAID/Northern Education Initiative Project in Bauchi, Nigeria and is focusing on teacher training. Two other AVEs are located at Sa’adatu Rimi College of Education with both focusing on teacher training and one of them also working with teacher resource centers and on education management. The fifth AVE is focusing on TRCs, and education management at Kano State University of Science and Technology in Wudil, Nigeria. There are four LVEs working in Nigeria for the 2010-2011 year, and there were six for the 2009-2010 year. Senegal Over the past decade, Senegal has made strides to achieve universal access to education. The primary school enrollment rate has risen from 69% in 2000 to 92.5% in 2009. But the gap between boys (82.4%) - 104 - IFESH Mid-Term Evaluation Report and girls (77.3%) remains significant. Senegal’s own goal of providing ten years of quality basic education, especially to girls and vulnerable children, requires additional attention and approaches to address this concern. In 2009-2010, there were six AVEs working in Senegal and they were all placed at different learning institutions in Dakar, Thies, St. Louis, Fatick, Longa, and Kaolack. All of the AVEs worked on teacher training, and for two of them that was their only type of work. The other four AVEs also focused on MoE Policy and Non-Governmental Organizations (NGOs). For the 2010-2011 year, there are seven AVEs who similarly to the 2009-2010 year are located in Thies, St. Louis, Fatick, and Longa as well as learning institutions in Kolda and Tambacounda. In addition to teacher training, MoE Policy, and working with Local Non-Governmental Organizations (LNGOs), three of the seven AVEs are also focusing on TRCs this year. There are 57 LVEs working in Senegal for the 2010-2011 year and there were 46 for the 2009-2010 year. 2.1 MoE Priorities All three countries have developed, within the past decade, ten year plans to establish universal primary education within the near future. Within these plans is a focus on improving access and quality, training teachers, distributing learning materials, and eliminating gender and regional disparities. All three countries also aim to provide more quality education both formal and non-formal in secondary vocational technical and higher education and also training opportunities, especially in under-served areas and to disadvantaged groups and girls. 2.2 USAID Priorities In the 2010, U.S. Congressional Budget Justification, USAID/Kenya was allocated $6.3 million in funding to improve basic education, USAID/Nigeria $14 million, and USAID/Senegal $9.7 million. This overall foreign assistance budget focuses on country-driven plans with an emphasis on the quality and relevance of basic education, serving the education needs of out-of-school youth and measure learning outcomes and results. It should also be noted that special emphasis was placed on the requirement that all programs will be monitored and reported on using the basic education indicators in the Foreign Assistance Coordination and Tracking System (FACTS) managed by the Office of the Director of U.S. Foreign Assistance. Therefore, any expenditure made toward the basic education goals were to be measureable and reported to the overall system. Each of the USAID Missions in these countries is responding to the priority areas as stated by the respective MoEs. These needs include strengthening basic education, building and reinforcing education professional development skills and capacity, providing technical assistance to help streamline and strengthen education management information systems, promoting the role of communities and their abilities to mobilize resources for improvement, and supporting the decentralization process within the education sector. 2.3 IFESH Alignment with the MoE and USAID The International Foundation for Education and Self-Help (IFESH) in Kenya is in transition, having just hired a new Country Representative in November 2010. This new Country Representative is working to rebuild the country program and realign the AEFA program to the priorities of the ministry of education and USAID. The interview with USAID/Kenya revealed that the mission is familiar with the work IFESH is doing in teacher training, curriculum development, and policy development and they expressed this work is very helpful. In Nigeria, IFESH has a strong Country Representative who maintains close communications with the USAID mission and works to align the work of the AVEs to the mission goals. These supported goals include book donations, teacher resource centers, and curriculum development (especially in non-Science, Technology, Engineering, and Mathematics (STEM) subject matter). The AEFA program in Nigeria has also been able to integrate the crosscutting themes of gender equity and community involvement. - 105 - IFESH Mid-Term Evaluation Report Senegal has a very active and large IFESH presence – second only to Liberia– in AVEs placed and far more LVEs (45) than any other country. This country programs enjoys strong support from the USAID mission and has a well-adept Country Representative to manage the program. The program is so well liked in Senegal that the mission would like to see IFESH branch out of education and provide similar volunteers in other development areas. The interview with USAID/Senegal also revealed that the mission is familiar with the work IFESH is doing in teacher training, developing teaching materials, and policy assistance with the regional directorates. The mission describes itself as the glue between IFESH and the MoE. - 106 - IFESH Mid-Term Evaluation Report AEFA Performance Summary1 Kenya Nigeria Senegal 2009-2010 2010-2011 Total* Target** 2009-2010 2010-2011 Total* Target** 2009-2010 2010-2011 Total* Target** AVEs 2 4 6 6 2 5 7 14 6 7 13 19 LVEs 0 0 0 / 6 5 11 / 46 57# 103 / CPD Training Workshops - 1 1 4 - 3 3 10 - 14 14 / Teacher Resource Ctrs. - 0 0 1 - 3 3 3 - 3 3 / Subgrants to NGOs/CBOs - 0 0 3 - 2 2 3 - 2 2 / Books Distributed 0 0 0 0 94,200 98,400 192,604 20,000 0 0 0 0 Pre-Svc. Teachers Trained M: 72 M: 0 M: 72 290 M: 1,756 M: 104 M: 1,860 4.887 M: 474 M: 253 M: 727 F: 18 F: 0 F: 18 F: 1,211 F: 31 F: 1,242 F: 195 F: 93 F: 288 2,109 T: 90 T: 0 T: 90 T: 2,967 T: 135 T: 3,102 T: 669 T: 346 T: 1,015 In-Svc. Teachers Trained M: 109 M: 0 M: 109 400 M: 132 M: 1,102 M: 1,234 1,000 M: 1,579 M: 634*** M: 2,213*** F: 92 F: 0 F: 92 F: 142 F: 336 F: 478 F: 327 F: 130*** F: 457*** 1,500 T: 201 T: 0 T: 201 T: 274 T: 1,438 T: 1,712 T: 1,906 T: 764 T: 2,670 Ed. Administrators M: 95 M: 88 M: 183 200 M: 2 M: 10 M: 12 300 M: 30 M: 16 M: 46 F: 44 F: 67 F: 111 F: 2 F: 13 F: 15 F: 2 F: 1 F: 3 200 T: 139 T: 155 T: 294 T: 4 T: 23 T: 27 T: 32 T: 17 T: 49 LNGO/CBO Representatives M: 0 M: 52 M: 52 100 M: 0 M: 20 M: 20 300 M: 70 M: 25 M: 95 F: 0 F: 67 F: 67 F: 0 F: 15 F: 15 F: 56 F: 15 F: 71 100 T: 0 T: 119 T: 119 T: 0 T: 35 T: 35 T: 126 T: 40 T: 166 Users of Resource Ctrs. - M: 0 M: 0 200 - M: 537 M: 537 400 - M: 0 M: 0 - F: 0 F: 0 - F: 22 F: 22 - F: 0 F: 0 400 - T: 0 T: 0 - T: 559 T: 559 - T: 0 T: 0 Key: M= Male, F= Female, T= Total, - = no data reported, and / = no data available. * Total = The sum of 2009-2010 and 2010-2011 columns. ** Target = 2009-2010 actuals + 2010-2011 target + 2011-2012 target. For indicators that were not used in 2009, Target = 2010-2011 + 2011-2012 only. *** = There is no male/female breakout in the AEFA January-March 2011 Quarterly Report; therefore, the Evaluation Team calculated a 5:1 ratio of male to female based on historical data. # =The AEFA January-March 2011 Quarterly Report states 56; however, to remain consistent with all of the other Target figures, the Evaluation used the figure from the Revised Work Plan. - 107 - IFESH Mid-Term Evaluation Report 3.0 Improving Teaching, Systems, and Education Resources How has AEFA strengthened the capacity of teachers, administrators, and policymakers to enhance the quality of school teaching methodologies and pedagogical materials at the basic education level through technical aid and the training of teachers, using the main AEFA methods of teacher training, Teacher Resource Centers (TRCs), donated books, and serving as policy/management advisors? 3.1 Teacher Training Teacher training plays a key role in the impact being made by AVEs in all three countries. In both STEM and non-STEM subject matters, the AVEs are working in support of MoE goals of improving the quality of education of the teachers being placed in the schools. 3.2 TRCs and Book Donations There are no donated books in Senegal and Kenya but in Nigeria 94,200 book donations have been made.2 There is support for teacher resource centers in both Senegal and Nigeria. However, there is no support for teacher resource centers in Kenya. 3.3 AVE Policy/Management Support There are no AVEs placed directly with the MoEs in any of these three countries but some do work on policy via curriculum development and pedagogy within their placements at host institutions. 3.4 IFESH and USAID Crosscutting Themes: Gender Equity, HIV/AIDS, and Community Involvement All three crosscutting themes are addressed in the Continuing Professional Development (CPD) workshops conducted by the AVEs in the three countries. Some are pre-service and some are in-service training workshops and they are focused on pedagogy, but these issues were covered. 3.5 Use of LVEs The LVE program is well utilized in Nigeria, and especially in Senegal. The LVEs provided workshops on teaching skills in literacy and mathematics in Nigeria. In Senegal, LVEs worked closely with the AVEs in providing teacher training and in holding workshops on classroom management and lesson planning. There are no LVEs in Kenya. 3.6 Use of Small Grants All three countries do administer small grants but only one of these grants addresses these crosscutting themes. In Senegal, a $3,000 grant was administered to the Committee Against Violence on Women for building training and sensitivity workshops. 3.7 Mobilizing Public/Private Partnerships The Country Representative in Nigeria has built a public/private partnership with Intel to promote school interventions, support these programs financially, and build a framework for sustainability. Neither Kenya nor Senegal have identified or established a public/private partnership to date. 4.0 Strengthening Education LNGOs How effective has AEFA been in providing technical assistance and knowledge to increase the capacity of Local Non-Governmental Organizations (LNGOs) to implement sustainable development programs, primarily in the area of education? Through small grants and workshops, IFESH in Nigeria and Senegal appear to be providing technical assistance to LNGOs in areas of need. However monitoring feedback is not available to corroborate this. In Kenya, a $4,000 grant was given to Chogoria Community Library Service Group to renovate their library - 108 - IFESH Mid-Term Evaluation Report and train staff. In Nigeria, three grants were administered to local institutions to support their development work. One was not specifically in education but in sanitization. In Senegal, a second grant to the one noted above was given to a local organization to help improve student living and learning environments. The LNGO small grant targets have been met in Nigeria and Senegal but not in Kenya. 5.0 Organizational Effectiveness Does the IFESH field team – Country Representative, Field Office staff, and American and Local Volunteer Educators (AVEs and LVEs) – have the tools they need to succeed? Are the necessary organizational structures (people, processes, and technologies) in place to support and sustain AEFA? 5.1 Volunteer Recruitment, Performance, and Support 5.1.1 Host Institution Identification/Volunteer Job Descriptions and Placement Process/Host Institution Resource Allocation and Support The process for placing AVEs in Kenya, Nigeria and Senegal was not discussed during the telephone interviews. 5.1.2 Orientation and Training A consistent theme during the interviews was that the volunteers are not well prepared for life in Africa. The Pre-Departure Orientation Workshop (PDOW) does little to prepare the volunteers and the in￾country orientation is too short. Further, the host institutions and USAID Missions do not appear to be part of the in-country orientation on a consistent basis. 5.1.3 Volunteer Monitoring and Performance Assessment There are no formal assessments of volunteers conducted by either IFESH Headquarters (HQ) or the Country Representative. 5.1.4 Safety and Security Nigeria and Senegal have plans are in place for security issues if they arise and volunteers are provided with guidelines they should follow to protect their safety. The new Country Representative in Kenya is currently developing a safety and security plan. 5.1.5 Office Support to Volunteers (Medical, Stipends, and General) Medical coverage and other support policies are in place. Stipends are distributed by the Country Representative and have been conducted without incident. 5.2 Communications There appear to be good, open communications between the IFESH Country Representatives and host institutions, the USAID Mission, and MoE. In Nigeria and Senegal, the Country Representatives visit volunteers and host institutions regularly, and seem to be closely aware of the progress of the volunteers and the satisfaction of the host institutions in this work. The new Country Representative in Kenya does not currently have a communications plan. 5.3 Knowledge Management and Sharing/Use of ICT There is no evidence of any knowledge management or information sharing being conducted in any of the three countries, other than the IFESH Quarterly and Annual Reports. 5.4 Field Office Operations There were no major issues discovered with the operations of the Field Offices during the phone interviews. All financial issues, human resources, communications, and security issues fall within the normal range of what was found in other countries. The three Country Representatives, including the new hire in - 109 - IFESH Mid-Term Evaluation Report Kenya, appear to be equipped to manage their respective programs as best they can under the restriction of their country context and IFESH HQ. 6.0 Performance ME&R Is IFESH using rigorous monitoring and evaluation systems that produce evidence of results and support program improvements and sustainability? Address Monitoring, Evaluation, and Reporting (ME&R) of the country program as a whole, and from stakeholder perspectives. IFESH in Kenya, Nigeria, and Senegal do use the established procedures to collect data and feedback from host institutions. This data is rolled up to the Quarterly Reports submitted to USAID. While the numbers of beneficiaries is good to know, it does not go nearly far enough to evidence any results or sustainability. In Senegal, the USAID Mission expressed that, compared to similar organizations, IFESH provides experienced professionals, a new perspective, and “new blood to the system.” 7.0 Implications for IFESH, USAID, and Host Country What are the successes, challenges, and opportunities of AEFA as an innovative, sustainable, and replicable model for accelerating education improvement? Based on the limited information gained from the telephone interviews, Nigeria and Senegal all seem to have successful elements to their AEFA Program. The Country Representatives have established close relations with the USAID Missions, the MoEs, and host institutions. IFESH in Nigeria and Senegal have incorporated AVEs, LVEs, and LNGOs to fulfill the goals established within the Cooperative Agreement. The two countries have focused on the primary goal of both USAID Missions and the MoEs. Using the AVEs, workshops and support of local partners, they are addressing the main challenges to quality education as stakeholders see them. While they may not be meeting the beneficiary number goal as noted in the IFESH Annual Report submitted in September 2010, they are making progress toward meeting the goal of improving education Nigeria and Senegal. There was a replacement of the Country Representative in Kenya at the beginning of the academic year 2010-2011. As evidenced in the IFESH Quarterly Reports, the AVEs indicate constraints in implementing their activities. Individual AVEs report some execution of training workshops. There has also been work reported with LNGOs and small grants. It is not evident, however, how these activities are strategically related and assist the MoE in achieving educational goals. 8.0 Recommendations Given the limited scope of these three countries, the Evaluation Team is not in a position to offer specific targeted recommendations. All three countries and IFESH HQ should review the recommendations found in the main body of the Report. 9.0 Methodology and Information Sources The Evaluation Team interviewed the IFESH Country Representative and USAID Education Team Leader only within each of the three countries. The names of all interviewees are found in Appendix A. The following documents and publications were reviewed for this case study: Republic of Kenya, Ministry of Planning and National Development. (2005). MDGs Status Report for Kenya. http://www.undg.org/archive_docs/6585-Kenya_Second_MDG_Report_-_Report.pdf. Republic of Kenya, Ministry of Education, Science and Technology. (2010). Kenya Education Sector Support Programme 2005 – 2010. http://chet.org.za/manual/media/files/chet_hernana_docs/Kenya/National/Kenya%20Education%20Sect or%20Support%20Programme%202005-2010.pdf. - 110 - IFESH Mid-Term Evaluation Report Government of the Federal Republic of Nigeria. (2010). Nigeria Millennium Development Goals Report. http://www.ng.undp.org/mdgs/Final-MDG-report-2010.pdf. République du Sénégal. (2010). Objectifs du Millénaire pour Développement: Progrès Réalisés Perspectives. World Bank. (2010) World Development Indicators database. http://data.worldbank.org/data￾catalog/world-development-indicators/wdi-2010. IFESH. (2009). AEFA – IFESH Technical Application. IFESH: Scottsdale, Arizona. IFESH. (2009). AEFA Quarterly Report, July-September 2009. IFESH: Scottsdale, Arizona. IFESH. (2009). AEFA Quarterly Report, October-December 2009. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, January-March 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, April-June 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Annual Report and Quarterly Report, July-September 2010. IFESH: Scottsdale, Arizona. IFESH. (2010). AEFA Quarterly Report, October-December 2010. IFESH: Scottsdale, Arizona. USAID. (2009). Cooperative Agreement Between USAID and IFESH. USAID: Washington, D.C. 1 IFESH Revised Work Plan; IFESH Quarterly Reports; IFESH AVE Distribution Tables; IFESH AEFA Book Distribution Tables. Received from IFESH HQ. 2 AEFA Book Distribution Tables (Nigeria) file (2011). Received from IFESH HQ. - 111 - IFESH Mid-Term Evaluation Report Appendix H: Best Practices for International Volunteer Management 1.0 Introduction Fulfilling the mission of an International Volunteer Organization (IVO) depends heavily upon how it recruits trains, deploys, and supports its volunteers. Volunteer management is a principal determinant of success in the field. Ensuring the proper staff and policies are in place to administer well-intentioned individuals of all ages from all walks of life is critical. Simply having a vision or mission to redress an identified need is not enough in practice. Other components and structures that constitute an organization must be put into place, including a balanced board of directors, effective staffing, strategic planning (including programming, liaising with the host institution on needs and content, execution, the way forward), communications (with stakeholders), funding and fundraising (how to interact with donors, keep them apprised of what’s happening), financial management, human resources, field offices and management, crisis management, change management, and knowledge management and transfer.1 Proper volunteer management provides the foundation for successful volunteer performance and high satisfaction among all stakeholders, and is consequently an international volunteer organization’s biggest key to success. 2.0 Volunteer Management What is volunteer management? The Institute for Volunteering Research defines the practice accordingly: “Managing volunteers [consists of] a number of activities that include recruiting, coordinating, leading, supporting, administering, and organizing volunteers and/or having responsibility for strategic planning for volunteering.”2 Generally speaking, volunteer management is functionally very similar to paid staff management. The caveats which distinguish volunteers from paid staff are that volunteers are not remunerated and they are serving principally out of belief and conviction in the organization’s cause. In the case of International Foundation for Education and Self-Help (IFESH), volunteers are paid a monthly stipend to cover living expenses during their service. These volunteers need to be valued and assessed, the same way as paid staff. It is the work of the volunteers that the customers see therefore it is important to ensure quality performance from these representatives. Further, it is just as important that the volunteers are made to feel part of the organization and take ownership of their work and progress. Volunteer management practices must be consistently applied throughout the volunteer lifecycle. The rest of this paper will examine the different stages of a volunteer’s term of service from initial contact to end of service. At each stage, the organization’s responsibilities to the volunteer will be addressed with an explanation of the multiple approaches available toward resolving that stage, while ensuring the organization’s mission is accomplished and the volunteer’s needs and expectations are met. A successful volunteer experience from the perspective of the volunteer, the organization, and the beneficiaries/ stakeholders relies upon effective management of the volunteer service lifecycle. This lifecycle includes the following stages: 1. Recruitment 2. Hiring3 3. Orientation and Training 4. Deployment 5. Field Support 6. In-Service Training 7. Reporting 8. Performance Evaluation 9. Retention 10. End of Service - 112 - IFESH Mid-Term Evaluation Report Policies and procedures should be established around each of these stages to ensure compliance and the best performance possible from the volunteers. 3.0 Managing and Optimizing the Volunteer Lifecycle 3.1 Recruitment Volunteer recruiting is the first point of contact with the future volunteer. As such, organizations should strive to make a good first impression and a strong sales pitch for the vision it is trying to accomplish. Organizations that do not develop a sound recruitment strategy and invest time and money into its implementation will either be faced with critical shortages of competent volunteers, or can be saddled with a pool of volunteers who are not good fits for the program and the organization. Volunteers from all age groups and levels of experience each have their own skills to offer and their own motivations or reasons for donating their time. The challenge is to develop a sufficiently refined process for efficiently and effectively culling those whose motivations, interests and enthusiasm to closely match what the Local Non￾Governmental Organization (LNGO) is trying to achieve on the ground. 3.1.1 Implementing a Targeted Recruitment Approach Broadly speaking, there are two different recruitment strategies that an organization can use to develop its hiring pool: general (or “warm body”) and targeted. Warm body recruitment is most often used when the organization wants to enlist large numbers of volunteers in short order, often for short one-off campaigns, with lower skill requirements, and with the expectation that the volunteer will be satisfied with simply having donated their time. Conducting a general recruiting campaign in this vein usually involves broad, indiscriminate outreach across whole populations, “General recruitment involves reaching mass audiences through media and public outreach programs such as public events, public service announcements on television and radio stations, billboards … This is the most common recruitment method, but it is the least focused.”4 General recruitment can be effective as a stand-alone strategy for many organizations with particular objectives, such as cleaning up trash in a park or serving soup in the local church. The drawback of this recruitment strategy for a technical, long-term service program is that it can result in numerous unsuitable applicants leading to ample time and energy wasted in the interviewing and hiring processes. Moreover, as industry expert Linda Graff notes, “The currently observable decline in volunteer numbers, combined with predictable demographic shifts,” that is, the aging baby boomer population, “suggests a looming shortage of volunteers. As a result, recruitment must be both more deliberate and more effective in today’s increasingly competitive market.”5 Consequently, to achieve complex goals requiring skilled volunteers, general recruiting will not suffice as the sole approach. The nature of international volunteerism with its lengthy deployments in often difficult settings and in service of a mission that requires a modicum of technical expertise suggests that general recruiting would be an inadequate method for IFESH. It should be at the very least used in concert with targeted recruiting or discarded as a recruitment strategy altogether.6 Targeted recruiting, on the other hand, is the practice of identifying and hiring volunteers who fit a specific profile suited to a particular task or project. These tasks can require a degree of technical expertise or an academic or professional background that meets a minimum competence and achievement threshold. For instance, Médecins Sans Frontiers seeks health professionals such as general practice doctors, nurses, and surgeons, ideally with experience working in different cultural contexts. Meanwhile, Peace Corps pursues candidates with strong academic credentials and a demonstrated ability to adapt, improvise, and innovate in a variety of situations. Targeted recruiting can help organizations to locate and enlist volunteers with the proper background, interests, and expertise for the project at hand. While this recruitment method requires more time and effort at the front end to implement, it can pay off in the long run with improved volunteer performance, enthusiasm, and retention. The key to this targeted approach for IFESH is to establish specific criteria that will meet the needs of the host institutions where volunteers are to be placed. Given the specificity of the needs of the host institutions, a general recruitment approach discussed above is insufficient for IFESH. - 113 - IFESH Mid-Term Evaluation Report 3.1.2 Recruiting For an organization to be successful in attracting ideal candidates for a particular job or posting, the recruitment effort should be treated the same way as it would be for an ideal paid staff position. This approach will demonstrate to the candidate the quality of the organization, its commitment to fulfilling its mission, and willingness to give the volunteer full support throughout his/her service. Potential candidates will be able to understand and visualize how the skills and time they are contributing can make a real impact. With those caveats in mind, a successful targeted recruiting message consists of the following components:  A clearly defined mission statement, including what need the organization is filling, who the beneficiaries are, and by which methodology the organization will achieve its goals.  A carefully articulated position description, including the position title, expected duties, technical requirements, term length, location, and to whom the volunteer will be reporting.  A narrative of what the volunteer can expect out of the experience, and what support the volunteer can expect from the organization. The former includes skills development, meeting new people and professional networking, the opportunity to learn about new cultures and issues, among other things. The latter, meanwhile comprises skills training (both pre-service and in-service) and development, cultural and language training, housing, stipends, and other site- or project-specific support.7 This message and position description can and should be developed in concert with program staff, the field office, and the hosting entity. After all, they will have first-hand knowledge and insight into the technical requirements, and will be the ones working side by side with the new volunteer. Some organizations engage current and past volunteers in the process, which can allow for added nuance in the description, more thoroughly capturing the nature of day-to-day activities in the field. While soliciting recruiting assistance from current volunteers may not be feasible in cases where their location and engagement in their work is prohibitive, past volunteers could certainly be tapped for gained knowledge and experience. 3.1.3 Developing the Candidate Pool Once the approach has been developed and honed, the next step is to get the message out to the general candidate pool and begin identifying potential targets for interviewing. Not having an effective recruitment plan or failing to skillfully implement the outreach effort can result in a decided lackluster crop of applicants. It must be a comprehensive endeavor, using specific approaches and carefully chosen venues and contexts to maximize the odds of locating the volunteer who best fits the position and the organization. A skilled recruiting staff should be in place with the ability to identify and reach out to top candidates. There are many avenues recruiters can take to uncover and recruit future volunteers including:  Workshops, trade shows, and conferences with relevant content and audiences. An IVO specializing in education should look at the event calendar for Teachers of English to Speakers of Other Language (TESOL) for prospective recruiting efforts. For example, IFESH could consider: o The Annual TESOL Convention o Teach For America Summit o The National Service Learning Conference o The National Association of Teacher Educators o The National Council of Teachers of Mathematics o The National Association of Biology Teachers  Target teachers, for example at Education Career Fairs, who are considering a sabbatical since IFESH’s volunteer commitment is one year of service. - 114 - IFESH Mid-Term Evaluation Report  Use volunteer alumni for leads on candidates. Current and past volunteers in the field often interact with peers from other organizations who could be candidates for recruitment. Additionally, alumni in their post-service careers will frequently work in industries related to their volunteer experience – they may have co-workers and contemporaries seeking a new chapter in their lives. Also, volunteer alumni can contribute by writing for trade journals and local newspapers to publicize their work and the IFESH brand (and IFESH would review prior to publication).  Engage relevant audiences through social media such as Facebook, LinkedIn, YouTube, and blogs.  Use volunteer matching search engines such as volunteermatch.org and idealist.org.8  Use diversity liaisons to reach out to minority-serving institutions and organizations.  Advertise on English as a Second Language (ESL) websites and education publications such as Teachers of Color Magazine.  Evaluate where the organization’s most successful volunteers have come from in the past and mine those sources for new candidates and leads.9 Some volunteer organizations supplement these recruitment practices by partnering with other like-minded organizations to share candidate pools and leads. Memorandums of Understanding (MOUs) between two or more such entities often include the maintenance of centralized resume repositories and candidate databases. These arrangements can also help volunteers who are at the end of their service period to find a new, similar position with another organization. For example, Teach For America and the Peace Corps, organizations with related programming and similar technical requirements use an MOU to exchange expertise and facilitate recruiting. Additionally, the recruitment program can be more efficiently executed and more precisely targeted by recruiting year-round for specific projects as they appear on the planning horizon. Once- or twice-yearly “volunteer drives” are perhaps easier to prepare and implement and can yield a promising cohort of potential volunteers. On the other hand, this unbalanced approach also means that the recruitment team experiences heavy workloads for two to three months at a time, and lays fallow the rest of the year. Additionally, new projects or crisis situations that arise during a gap in the organization’s recruitment calendar can be much more difficult to rectify with new, skilled volunteers from the outside. By utilizing a year-round approach to recruiting, it is more feasible to staff a permanent recruitment team and there are no gaps in project coverage and implementation as volunteers can be brought into the fold on an as￾needed basis.10 It should be noted, however, that each of these two approaches are organization and programming specific. In any case, at the point of recruitment (when initial contact is made with a candidate), whether it’s over the phone, at a conference, or at a recruiting event, it is essential to obtain vital information about the prospective volunteer. At a minimum, make sure to obtain his/her contact information, jot down a brief list of specific, relevant skills on offer, and capture the volunteer’s interest in the organization’s mission and the work to be performed. This volunteer data should be maintained in a regularly-updated database for use in future recruitment cycles. Recruiting specialists should be prepared to provide a brief synopsis of the organization’s mission and to describe the application process.11 Finally, it pays to be frank and truthful about what the position and organization do, and do not make any promises to the candidate. Otherwise, volunteer disillusionment can become a problem, performance suffers, and the organization is faced with an unnecessary and avoidable headache. The recruitment period is when expectations should begin to be set. 3.2 Hiring 3.2.1 The Application The next step in the volunteer lifecycle is the application and hiring process. Ideally, the application tool will incorporate pre-screening rubric which can determine in advance which candidates are potentially good fits - 115 - IFESH Mid-Term Evaluation Report for the organization. This can take the form of simple eligibility questions or more in depth requests, such as asking the applicant to respond to a series of technical questions that can reveal the level of expertise he or she carries. Another option is for the candidate to simply submit a resume and a cover letter outlining his/her qualifications and interests. Any of these methods can be useful, but careful consideration should be given to which approach would fit the organization’s needs. On the one hand, requiring only a cover letter and resume can make the application process less onerous on the candidate, and can encourage greater numbers to apply for the position. On the other hand, this type of application may not lend itself well to an automated pre-screening tool and can also lead to an applicant pool with numerous unsuitable candidates who are just “throwing their resume out there to see what sticks.” Meanwhile, asking eligibility questions or technical questions can help the hiring staff hone in with more precision on ideal candidates to interview; but the extended application process may discourage potential volunteers who have other constraints on their time. 3.2.2 The Interview The interview is the hiring staff’s best opportunity to get to know the potential volunteer. Beyond asking the standard interview questions such as “Why do you want to work for us” and “What are your greatest strengths/weaknesses,” consider asking the applicant for his/her feelings on a technical or programmatic aspect of the organization’s mission. IFESH should be probing a volunteer’s commitment to working in challenging environments, and should provide an example of an everyday situation encountered in the field and inquire how the applicant would address the issue or resolve the problem. Indeed, scenario interviewing can provide a valuable window into the candidate’s thought processes and thinking under pressure. This approach is particularly useful to judge who may be best suited for more difficult or dangerous placements, such as a teacher trainer in rural Liberia versus one in urban Ghana. Combining scenario interviewing with behavioral interviewing techniques (e.g., assessing adaptability, communication, decision making, ambition, and integrity) allows for further refinement of the candidate’s skill set, interests, and suitability for the position.12 Other assessment tools can include personality tests (e.g., the Myers-Briggs) and career aptitude tests, and other organizations use these to gauge certain criteria. These do incur extra costs, and volunteers may not be willing to commit the time; others may feel that these more esoteric tools are too personal in nature. Using such test is not necessarily a recommendation but merely a suggested tool for screening candidates for compatibility with the harsh nature of some placements. The ideal volunteer for a skilled position will boast equal levels of technical expertise, commitment, enthusiasm, willingness, and ability to work with peers within the organization and beneficiaries (or stakeholders) in the field. It bears repeating that not every applicant will be suited for the organization. It is better to be short-handed for a little while longer while lining up a more appropriate candidate than it is to put time and money into a volunteer who is not a good fit for the placement.13 As with the recruitment stage, it is important to convey to the candidate precisely what support he or she can expect from the organization and the program staff. These benefits run the gamut from training, professional support, a stipend, medical coverage, and post-service support (such as adjustment allowances and alumni network membership). IFESH should create such a message for its potential volunteers so that expectations are conveyed during the recruiting phase. This message should be reinforced throughout the on-boarding process, including during the interview, in the offer letter, and during orientation and pre￾service training. 3.2.3 The Hire Once the volunteer manager and, ideally, several team members have completed the rounds of interviews, the team should convene to discuss what they have learned about the candidate. Assess how the interviews have demonstrated the candidate’s suitability for international volunteer service. Will the candidate be able to work well with the organization and his/her team? Is personality and aptitude a good fit for work in the field? Can the candidate be trusted to uphold the organization’s mission and successfully perform the tasks - 116 - IFESH Mid-Term Evaluation Report in the field? These are all general questions to be answered that can predict how effectively the candidate will perform and how likely the candidate is to complete his/her service term. Further, all the information gathered during this process should be documented and recorded for future reference. In addition to these generalities, specific placement issues need to be addressed. Where are the organization’s needs? And where can the candidates’ skills fill those needs? Numerous factors play into the placement decision including technical expertise, language skills, personality, cultural background and ethnicity, previous volunteer experience, and stated interests. Once the location and length of placement to be offered has been determined, a final consideration is whether to recommend a probation period before deployment. Probation periods are especially useful for less experience candidates, as it will allow the IVO a length of time during which to further evaluate and train the volunteer before sending him/her overseas. Additionally, this period can be used by the newly hired volunteer to further reflect upon the assignment and ensure that he is fully committed to the program.14 Once these details have been sorted the offer is made and, if accepted, the candidate is a new volunteer. 3.3 Orientation and Training 3.3.1 Orientation/Pre-Service Training Orientation serves a dual purpose, to introduce the organization to the volunteer and to on-board the volunteer and make him/her feel like a part of the team and prepare for the field. As urbanministry.org, a volunteering resource clearinghouse, puts it, “Treating volunteers as part of your organization’s staff helps them feel they are part of a team and fosters commitment and retention. Volunteers, as staff members, help represent your agency to the public. The more they know and understand about the nature of your operations and your cause, the more they can contribute to public relations, marketing, and advocacy.”15 It is therefore important that orientation be a well thought out event that helps to indoctrinate the new volunteer into the organization’s mindset. At the same time, attention must be paid to ensuring that the volunteer understand how the various support structures offered by the organization function. Reporting, evaluations, communications, supplies, crisis contingency plans, lines of authority, and other such components should be thoroughly reviewed. The more rigorously structured and consistently applied the orientation session, the more effective and efficient it will be at bringing the new volunteer into the fold. By adopting a standard orientation schedule containing a regular series of activities, costs can be minimized, paperwork can be streamlined, and the staff and facilitators can deliver a standardized message honed through repetition. A good way to codify the information presented during orientation is to provide each volunteer with a volunteer handbook, articulating the organization’s vision, mission, policies, procedures, and program/country-specific information. This handbook has the added advantage of being a ready resource for reference through the volunteer’s deployment.16 One industry-wide survey determined that only one in four regular volunteers do not receive training for their roles and responsibilities. There are numerous reasons why a volunteer might not have the opportunity to take a skills development course, with the three most commonly cited being “cost to the organization,” “time taken to attend training,” and “lack of cover for work.”17 Each of these obstacles can be preempted by offering pre-service training as part of the orientation process. In doing so, the marginal cost of training is lower given that training staff may already be present for orientation activities and the courses can take place in the same space, and volunteers who have not yet begun to work would not have to balance day-to-day duties against attending the courses. The availability and quality of pre-service training can be important determinants in the success of the volunteer’s deployment and work in the field. With the administrative and organizational tasks and information taken care of by the orientation, training should be dedicated to technical skills development, cultural sensitization, language acquisition, and other pertinent contextual matters. If the organization has - 117 - IFESH Mid-Term Evaluation Report enough training resources available, highly targeted training covering the day-to-day activities and specific situations and contexts that may present themselves in the field can help the volunteer to quickly and comfortably adapt to their posting. The Peace Corps have found that tailoring a portion of the training sessions to the country which the volunteer will be deployed, and particularly to the region of that country (e.g., Djibouti or rural Senegal), is particularly effective in preparing the volunteer’s mindset for the work ahead. Other more general competencies that form an important part of any training session include leadership, communication, empathy, decision making, and basic project planning or management. These are applicable in nearly any context, are highly transferable, and are typically valued by volunteers seeking to further their own development in the course of their service. For international postings, cultural sensitization is of paramount importance. Volunteer placements often require significant cultural adaptation and adjustment to different and sometimes difficult work and conditions. Going into the volunteer experience with a basic understanding of the host country’s languages, traditions, mores, politics, dress, gender relationships, religion, history, and other indigenous conventions and practices can allow for smooth assimilation and avoid awkward pitfalls which can endanger the organization’s mission. Proper conduct in the field will go a long way toward encouraging beneficiaries to accept the assistance that the organization is trying to provide. In light of cultural sensitization’s determinant role in the success of the initiative, some organizations have codified cultural training in the form training workbooks.18 Volunteers will be deployed to the field to work on the site of a hosting institution or entity. Therefore, time must also be taken in training to introduce and describe the hosting institution. Discuss their practice, their stakeholders and beneficiaries, and their needs which have led to their request for volunteer support. If possible, provide volunteers with a pamphlet containing the aforementioned along with the location, points of contact including photos (if desired) for easy identification, a narrative of the host’s activities and history, and any other notable information. Such a welcome packet would provide insights and context for the volunteer as they begin to visualize their placement. Finally, volunteers should be trained in using the organization’s reporting and knowledge management (knowledge management) tools. These enable volunteers to capture and share the work, outcomes, and stories from the field. Reporting and knowledge management are discussed in a separate section below. In administering this training, consider varying the types of activities to keep the material fresh, particularly if the workshop runs a full day or more. To sustain the interest of the participants in the matter at hand, try getting away from a single speaker at a podium into more engaging session types such as panels, round table discussions, break out groups, and role-playing, among others. The format chosen should serve as an effective vehicle for the content. For instance, breakout sessions by country can encourage participants to learn and share insights about what pertains to their specific posts. Meanwhile, other topics such as medical evacuation which are valid for all participants are better addressed in a general session. An overview of about how to submit reports to the field office is probably best suited to a lecture-style presentation to keep the message clear and concise for all participants. Conversely, a session on independent decision￾making might benefit from a role-playing group exercise with report outs at the close of the activity. Generally speaking, policies and procedures should be presented in a straightforward manner, while skills development and familiarization with programmatic content and concepts are areas better suited to a more creative approach. The ideal length of the pre-service training is highly specific to the organization, its needs and mission, and the particulars of the volunteer’s job description. Overseas deployments, however, with their cultural and language requirements on top of programmatic and administrative matters, will require more intensive training. Furthermore, some IVOs find that implementing some amount of pre-service training in-country can accelerate the immersion process and ensure a smooth start to the volunteer’s work. - 118 - IFESH Mid-Term Evaluation Report 3.3.2 Volunteer Work Plan While orientation and pre-service training should help the volunteer to understand the scope and execution of their tasks and responsibilities, it should also be an opportunity to encourage them to set appropriate, yet ambitious goals for their term of service. For instance, if the position description lists teaching primary school girls in a Ghanaian village how to read in English, then work with the volunteer to make 100% English literacy in that class the target by the end of service. These goals can help make responsibilities appear less abstract by attaching a tangible outcome toward which to work. Reporting and evaluation activities also benefit with measurable components to track over time. Also, it can facilitate assessments of what is working for the volunteers and beneficiaries, and where more support or different tactics may be required. Once finalized, the goals should be wrapped up into the volunteer work plan. The volunteer work plan is an agreement between the volunteer and IFESH which incorporates the volunteer’s job description, activities, reporting requirements, and objectives (or goals). Ideally, the plan will also be developed in concert with the field office and include input from the hosting institution, if possible. This document should be clear and concise, it should contain an evaluation schedule and it should define measures of success for evaluation purposes. Additionally, the plan should identify lines of authority and communication, and identify the degree of latitude the volunteer has in accomplishing his/her tasks and taking initiative. The work plan need not be overly precise about the volunteer’s day-to-day activities; rather, it should allow for flexibility to the extent that the situation on the ground may demand. Finally, the volunteer work plan should be constructed and treated as a living document to be revised during evaluations and as events warrant. 3.4 Deployment Once the pre-service training is complete and the work plan is in place, the volunteer should have the necessary basic skills, knowledge, and guidance to deploy to the field. The actual placement location should have been determined during the hiring stage, and pre-service training should have prepared the volunteer for the cultural context in which he/she will be serving. The most efficient way to conduct deployment is for it to be incorporated directly after the conclusion of pre-service training. Consequently, volunteers should arrive at training with their personal effects and be mentally prepared to leave the country within a few days. The IVO should assemble a deployment package for each volunteer containing, but not limited to, the following items:  Airline tickets – if at all possible, these should be round-trip as many countries’ customs officials will not allow entry without proof of a return ticket. Volunteers should also be given a letter from IFESH stating the purpose and period of service. Depending on the country, visas may also need to be obtained.  Funds – before departure, ensure that the volunteer has sufficient funds for travel and the first few weeks of service. Volunteer stipends should also be provided on time and other assistance that volunteers may need, e.g., banking, should be arranged by the organization’s staff.  Immunizations and prophylaxes – It is absolutely incumbent on the IVO to disclose any and all medical risks that volunteers may encounter during their service. Consequently, the organization should either provide a list of immunizations and prophylaxes that the volunteer can obtain at his/her travel clinic, or arrange for a nurse to attend orientation and administer the medication on the spot just prior to deployment. The former option requires advance communication about these needs and will usually expect the volunteer to assume the costs. For long term deployments, many organizations use the latter approach as a small, but appreciated service to the volunteer.  Information packet –Supplement the organization handbook passed out during orientation with an information packet including (but not limited to): field office address and points of contact, host institution address and points of contact, address of the local clinic, country security officer phone - 119 - IFESH Mid-Term Evaluation Report number, related Ministry address (e.g., Ministry of Education for volunteer teachers), U.S. Embassy address and phone number, and locations and contact information for fellow volunteers serving locally as well as maps of the country and capital city, if possible, and information on appropriate office attire for the local context. In addition to providing the volunteer with a deployment package, headquarters will need to communicate to the field office that the volunteer’s arrive is imminent. The field office should be prepared to receive the volunteer and to give him/her direction on next steps. Upon arrival at the field office, the field office should have the volunteer register or check in, and staff must report his/her successful arrival to headquarters. A brief orientation at the field office can be valuable, consisting of an overview of country operations, introduction to the staff, a review of the volunteer’s job description and expectations, and a discussion of the support services the office will provide. The volunteer should also receive any additional information available on recent, pertinent local and national events and situations as well as a heads up on local cultural and political peculiarities that he may experience. Once the orientation is complete, provide the volunteer with any additional supplies deemed necessary (such as a mobile phone) and provide transport to the posting location. 3.5 Field Support Support of volunteers does not end with deployment – volunteering can be a physically, mentally, and emotionally taxing endeavor. Continued commitment and success depends on the organization providing the volunteer with the tools to succeed. Both headquarters and the field office have a responsibility to assist those who are serving in the field. The most common support resources include:  Communications – Keeping the volunteer consistently informed and in the loop is vital for making him/her feel valued for the role he plays in the organization. This is particularly true when the volunteer is the only one from the IVO working at his/her location; reaching out and drawing in these individuals will make them feel a part of a larger team and can better connect him/her to his/her peers and the overarching mission at hand. With the pervasiveness of the internet throughout the globe, it is easier and less expensive than ever to stay in contact with volunteers in the field. The field office should send out regular emails pertaining to the local conditions in which the volunteer serves. These could include updates on the political, cultural, medical, and financial climate in the country and region. From headquarters, if the organization publishes a regular newsletter, it should include volunteers on the electronic distribution list.  Supplies – Nearly all volunteer programs will require supplies of some sort to properly carry out their tasks, be they simple office materials or more specialized tools such as textbooks. These should be supplied to the volunteer through the field office. This will allow for faster and more responsive distribution, especially valuable when time is often of the essence. Moreover, the field office will be able to more accurately account for supplies by tracking durable goods as they go out and are returned at the end of the project; and tracking the burn rate of consumables. Volunteers should be informed prior to departure of any supplies they may need to furnish on their own.  Training and mentoring – In-service training is an absolute necessity for any long-term assignment, and is discussed in detail in the next section. Mentoring comprises not just training elements, but also general professional and personal support. Assigning each volunteer in the field to a mentor who works onsite or close by can ensure that the volunteer can seek guidance on a variety of matters, whether directly related to the work or not. Having a mentor to provide advice and encouragement can help maintain enthusiasm and commitment to the organization’s cause.  Mediation and Leadership – Sometimes a volunteer will come up against a problem that he is not empowered to or capable of resolving. For example, the principal of a primary school in which the volunteer serves refuses to allow her to teach boys because of her gender. This is a situation that can get prickly quickly and inflame the host and volunteer alike. IFESH Field Office staff need to be - 120 - IFESH Mid-Term Evaluation Report prepared to step in and resolve the predicament before any permanent damage to the program occurs. The volunteer should be able to seek leadership support from the IVO to help bring the matter to the attention of the right people.  Feedback and evaluations – Regular assessments of the volunteer’s activities can ensure that the volunteer is confident in his/her performance, that beneficiary’s needs are being served, and that the organization’s mission is well represented. This is discussed in more detail below. While each of these components of volunteer support can be administered from headquarters, Field Offices will usually provide the most effective and timely implementation. Field office staff can provide a human touch to the program and help expedite the resolution of problems. Volunteers will need more than just their pre-service training to succeed; they also need ongoing support from each other and the organization to deal with the challenges of volunteering. Organizations should consider innovative ways to offer additional support such as support groups, mentorships between new and returning volunteers, group￾building activities for volunteer cohorts, and participation in cultural activities outside the scope of the program. As one long-time volunteer management consultant puts it, “Volunteers who feel a positive sense of connection with the staff and volunteers of their agency will tend to feel good about the experience and will want to continue to volunteer.”19 3.6 In-Service Training Training should not end at the pre-service stage; it forms an important component of field support. As the volunteer begins work and familiarizes himself/herself with the task, responsibilities and the context, he/she may find that more technical training is needed. As it would with paid staff, training enhances the ability of volunteers to perform effectively and improves their impacts. Organizations should plan to offer sessions at least once yearly, and make available year-round online training (where internet is available), books, and other skills development tools. Annual training should include:  Refresher training for volunteers who decide to renew their contract for another year of service as an IFESH volunteer. Over time, adherence to policies and procedures, reporting requirements, and particular roles and responsibilities can waver. A refresher training course is useful to ensure that volunteers are reminded of the organization’s mission, obligations, and principles. Doing so will ensure that the volunteer’s efforts retain their efficacy and the beneficiaries and stakeholders expectations are being met.  Skills development training for all volunteers. When volunteers are deployed in a dynamic environment, they will benefit from learning new skills and refining pre-existing ones. A volunteer teacher in Djibouti may find that his/her hosting school is experiencing staffing issues – training in staff management can allow the volunteer to expand his/her role beyond the classroom and help his/her school solve its problems. Meanwhile, some other skills are always a work in progress no matter the volunteer. For example, Teach For All has discovered that leadership is an essential skill that is useful and highly valued in all quarters – honing volunteers’ leadership proficiency over time is a process that can produce large impacts.  Continuing to adapt to a new sociopolitical context (postings in an unstable country or region can be susceptible to events beyond the volunteer’s control and the IVO’s control). Security communications and evacuation plans should be in place prior to placing volunteers in such situations. From time to time, other opportunities for in-service training may arise, particularly workshops and conferences hosted by other organizations and institutions. If a workshop is taking place in-country (or in the same region) that addresses subject matter relevant to the volunteer’s work, it could be worthwhile to sponsor his/her participation. Meeting with leaders and subject matter experts in the field can be an invaluable experience and help to open new doors in the volunteer’s thinking and activities. Collaborating - 121 - IFESH Mid-Term Evaluation Report with practitioners and peer volunteers from across the discipline can improve the volunteer’s performance upon returning to his/her posting and help develop intangible skills such as teamwork, empathy, and critical thinking. Finally, organizational change may necessitate learning new policies and procedures. Volunteer organizations will sometimes experience extensive changes in its business processes or operating practices due to donor pressure, streamlining, new leadership and so on. In the event these changes lead to revamped policies and procedures, the organization should consider convening volunteers and paid staff alike for discussion and training. It is important to keep volunteers well-informed of meaningful organizational events to make them feel as if they are part of the team and that their input and efforts matter. This form of integration can help retention in a time of change when consistency in the field particularly cherished. Generally speaking, training should be conducted in the field (if possible) to take full advantage of the cultural and geographical context. For those organizations whose activities are spread across multiple countries, it may make more sense to convene a regional training session to minimize staff travel and costs. Also, if the organization is a member of a LNGO network that facilitates information sharing, for example, this can be a good way to supplement the organization’s existing training program with access to training courses and materials provided by other organizations. This capability, however, should not be viewed or used as a substitute for the organization’s in-house training offerings.20 Additionally, training can serve as a motivator and boost volunteer enthusiasm and retention, such as sending a volunteer to a conference or a workshop on a topic directly or even indirectly related to work. This tactic can improve motivation and retention.21 Similarly, for IVOs with stretched budgets training can be a tangible way of recognizing and showing appreciation for volunteers.22 3.7 Reporting Apart from the service on behalf of the organization for the beneficiaries and host institutions, organizations should also require regular reporting from the volunteer. Reporting is an integral piece of the field work because it helps the organization to understand the progress being made and to make course corrections should conditions change. While staff in field offices should be expected to have a consistently firm grasp of how the program is performing at the country or regional level, it is up to the volunteers to provide the most granular observations. Volunteers are on the front line and experience first-hand impacts of the organization’s mission and activities on the beneficiaries. These are all outcomes that cannot be tracked easily from abroad at the headquarter level – organization officials and benefactors rely upon the field office staff who in turn rely upon the volunteers. The content of these communiqués will vary widely depending on the organization and the sector in which it specializes and operates. Even so, the art of reporting generally follows the same broad outlines. A good reporting culture is that volunteers generate reports that are detailed, accurate, unbiased, timely, and consistent. Volunteers should be able to submit narratives with corroborating data that is linked to and supports their objectives laid out in the work plan. There should also be a fixed schedule for submitting their write-ups – monthly is a common practice, and one that appears to be working for IFESH, along with end of semester activities that coincide with the quarterly reporting schedule. The organization can help volunteers produce and submit quality, timely reports by deploying a standardized system that everyone in the organization uses and references. Reporting systems can either be designed in￾house with some database expertise, contracted out to a development reporting organization such as SageFox (which can come with dedicated monitoring and analysis support), or purchased right out of the box from a software developer. The first option is the likely the cheapest, but requires the capability to build the software and support implementation. The second may be more expensive, but the value is in the added services. The third option is perhaps the quickest to implement on a rudimentary level, but may not come with any support. The choice of reporting systems is dependent on the organization’s needs and - 122 - IFESH Mid-Term Evaluation Report resources. The key to the success of any reporting system regardless of the origin is to standardize and codify reporting requirements for all volunteers (and paid staff, too). The robustness of the software or whatever M&E support might be included will mean nothing if the reporting is inconsistent or non-existent. Regardless of the system, it is imperative that volunteers are trained on what to collect and how to report the data collected. 3.8 Performance Evaluation Any long-term deployment must incorporate regular assessments of the volunteer’s performance into its programming. Given that the volunteer is providing many of the same services a paid staff member would, it is only natural to provide them with the same feedback structure as well. Performance evaluations serve a variety of important functions including:  Ensuring the goals are being achieved.  Ensuring the volunteer’s needs are being met. This aspect of the evaluation takes into account training needs, the quality of field office support, whether or not the organization’s policies and procedures are effective and suited for the volunteer’s duties. This is also an opportunity for the organization to gauge the effectiveness of its pre-service training, particularly the language and cultural preparation, and make revisions for future volunteers if necessary.  Assessing whether programmatic changes are needed. The organization should review the programming for that country or region to ensure the work being done is still closely aligned to stakeholder needs.  Increasing volunteer commitment and enthusiasm. Regular assessments can make the volunteer feel a valued member of the greater team. By pointing out the amazing things that he/she has accomplished in the duration, and by identifying new challenges and goals, the volunteer is likely to feel more motivated to continue succeeding and serving the organization and beneficiaries to the best of his/her abilities. Integrate a self-evaluation component as well as an evaluation from the hosting institution.23 Supplement these sources with program records, data and outcomes, and the volunteer’s in-service reporting. Taken together, these elements will allow for a holistic assessment of the volunteer’s accomplishments, capabilities, areas for improvement, and the upcoming year’s expectations. As for the evaluation process itself, it generally follows the same structure regardless of the type of service or the posting location. At a minimum, evaluations should include:  Reviewing and recognizing the volunteer’s accomplishments.  Identifying areas for improvement. While the volunteer is donating his/her time and effort to the cause, that person must be open to constructive criticism, just like any other work environment. Indeed, the volunteer is seeking the best for the mission and is likely also seeking substantive professional development out of his/her time in the field. o Review the volunteer’s work plan and goals, and determine where his/her impact may not be to the level desired. o Discuss what the mitigating factors are – is this a product of the volunteer’s work style? Is the host institution or entity not cooperative? Are there supply shortages? Are local politics at play? Or were the goals set out in the work plan unrealistic? Does the volunteer require additional training to meet new technical challenges?  Revising the volunteer work plan and looking forward. Does the work plan still reflect the realities on the ground? Are the goals set out in the plan still valid? Change the work plan and set new goals if needed.  Address volunteer questions and concerns. Volunteering is a two way street and it is critical that their needs, uncertainties, and worries be engaged in an empathetic manner. - 123 - IFESH Mid-Term Evaluation Report A successful evaluation cycle relies upon effective communication and understanding and, properly handled, should result in shared expectations, commitment, and enthusiasm between the volunteer and the organization going forward. 3.9 Retention Applying emphasis and resources to volunteer retention can pay dividends. Volunteers renewing their commitment for another service cycle are a cost, time, and administrative savings over recruiting a new volunteer to take the retiring one’s place. Generally speaking, “Recruiting volunteers is an expensive and time-consuming job, so charities like to maximize retention.”24 Assuming the posting location is unchanged or substantially similar, and the technical requirements are similar, orientation and pre-service training are likely not needed. Additionally, placement costs are reduced and administrative and preparative efforts are much less onerous. IVOs should be aware, however, that typically retained volunteers will expect a period of leave and travel arrangements to return home before beginning the next cycle. Just as with the other aspects of volunteer management already discussed in the paper, retention is a component that should be conscientiously addressed. Organizations should articulate a retention strategy to maximize the number of volunteers sticking with the mission and program it into the volunteer management duties. A recent study by the Urban Institute found that “two-fifths of volunteers have stopped volunteering for an organization at some time because of one or more poor volunteer management practices.”25 This same paper identifies a pair of key components to an effective retention strategy, including “investing in recognizing volunteers, providing training and professional development for them, and screening volunteers and matching them to organizational tasks.”26 Ultimately, the lynchpin of volunteer retention is effective volunteer management. If the organization provides a clear, compelling mission, recruits intelligently, provides the volunteer with training, field support, and positive feedback, and gives the volunteer a chance to serve to the best of his/her abilities, those volunteers will be much more likely to return for another year of service. Even if a volunteer does not decide to return for another year, IFESH should have a continuity strategy in place to ensure a smooth departure of the outgoing volunteer and the arrival of an inbound volunteer. 3.10 End of Service Researchers have linked post-program debriefing and follow-up for former volunteers to positive outcomes for volunteers.27 Effective volunteer management includes tool to bring closure to the service and prepare these volunteers for the next stage. It is in everyone’s interest, the organization, and the volunteer, to create a smooth transition as they leave service. Many volunteers will feel a sense of loss as they part from the structure and mission of their volunteer life. Adequate time must be allowed for the volunteer to prepare themselves. An organized closing ceremony is an effective way to provide volunteers a formal way to end their service. This ceremony would also be a good place to present any awards or recognition the organization may find useful. Providing volunteers with assistance in taking steps towards their post-volunteer life will minimize any troubled emotional attachment. An exit interview is commonly used to bring closure to the person’s service and provide valuable feedback on both the program and the individual. Having that individual prepare a written evaluation of the program is a good way to document any challenges or strengths that volunteer managers may not see. With programs that continuously place new volunteers every year, replacing old ones, it is important to transition from one group to the next, and provide real life experience for the new volunteers to draw upon and help avoid potential mistakes. This transition of information can be electronically with video or audio tapes. Each local volunteer manager could keep a binder that holds personal letters from former volunteers to new ones, telling them how their service went and offering advice. Some organizations build a - 124 - IFESH Mid-Term Evaluation Report booklet of collected experiences, and advice gathered over years of managing volunteers and provide that booklet to entering volunteers. There are several positive reasons to maintain strong connections with past volunteers. They can help recruit new volunteers, share their knowledge at orientation or other workshops and development functions, offer networking opportunities as the next group of volunteers leave service, and provide historical perspective and context for the program as a whole. 4.0 Board of Directors Another important aspect to an organization is its board of directors. The board is responsible for two fundamental types of work: governance and support. In the governance role, the board acts as one “body,” whereas in the support role, the board functions as a collection of individual volunteers. As a governing body, the board is responsible for:  Financial oversight  Determining mission, vision, and overall strategies, policies, and priorities  Monitoring and evaluating program performance and impact  Developing and monitoring the effectiveness of the fundraising strategy  Selecting, evaluating, and terminating the Executive Director/CEO  Ensuring compliance with legal and contract requirements Governance is essential for nonprofits, and Executive Directors should work with the board to improve its ability to govern. It is important to note that governance is not management. Management of the organization resides with the Executive Director. In their support role, board members as individuals act as “champions” and provide support to their organization by:  Making financial contributions  Leveraging personal and professional connections for financial contributions, media coverage, and political contacts  Providing advice and technical assistance in their areas of expertise  Acting as ambassadors of the organization in their community One way to improve governance is to openly discuss the issue with the board. Governance should be an explicit recurring item on board meeting agendas, and the board chair and Executive Director should foster a culture that is open to discussing difficult issues that often come with governance. Nonprofits, and in turn, its board of directors, are officially governed by its bylaws. The board should prepare, review, and revise its bylaws on a semi-regular basis, e.g., every three years. The Executive Director should be responsible for tracking all revisions to bylaws, as board terms can make it difficult for the board itself to keep track of historical revisions. Another way to ensure each board member understands his/her responsibility is to develop a board member contract or agreement. Board contracts/agreements are not meant to serve as legal documents; rather, they are meant to clearly outline the responsibilities of the board and the responsibilities of the organization to the board. The contract should communicate core values, foster a culture open to debate and disagreement, demonstrate the board’s accountability to the nonprofits constituents and the public, and set expectations for board-board and board-staff relationships. As a governing body, the board of directors is responsible for monitoring and evaluating the performance of the organization. In many nonprofits, the leadership struggles to determine who is responsible for the performance of the board itself. While shared responsibility may come naturally when the nonprofit has a - 125 - IFESH Mid-Term Evaluation Report strong Executive Director and a strong board, the true responsibility for the board’s performance lies with the Executive Director. This may seem counterintuitive given that the board is responsible for evaluating the Executive Director, but as the leader of the organization, the Executive Director is ultimately responsible for all aspects of the organization, including the board. In fact, part of the board’s evaluation of the Executive Director can be the performance of the board. If the board is underperforming, the Executive Director should be held responsible. Board members should be constantly engaged with strategic efforts that directly impact its constituents. Framing itself in this way, the board should ask, “What does the organization need to accomplish this year?” and “What can I do as a board member (or what can we do as the board) to help it accomplish this goal?” Volunteers can often be overlooked by nonprofit boards. Nonprofit leaders, including board members, need to invest time and attention to engaging, managing, and retaining volunteer talent. One of the best ways to ensure the board properly engages volunteer involvement is to make volunteerism a regular agenda item for all board meetings. Topics can range from recruitment, to training, to volunteer morale, to retention. 5.0 Fundraising Strategies Non-Governmental Organizations (NGOs) across the spectrum heavily rely upon fundraising to support their operations, including recruitment and deployment of volunteers, and fund its direct and indirect costs. Without proper fundraising planning or implementation, the organization runs the risk of short-changing its beneficiaries and endangering its volunteer corps through spending cuts and reducing necessary resources. To ensure ample, stable, dependable cash flow from donors, NGOs should consider the following in planning their fundraising efforts:  Diversify funding sources among various donors such as individuals, government grants and contracts, for-profit businesses in a related industry, and foundations. Doing so will insulate the organization against the capriciousness of any one donor, or against donor fatigue.  Develop and maintain strong relationships with donors. These benefactors have successfully been captured and cultivating a solid rapport can lead to consistent contributions. The organization can then concentrate its fundraising development on new, prospective donors. 28  Involve the organization’s Board of Directors in fundraising. The board with its standing, influence, and peer networks should be able to help secure many of the organization’s largest funders. Delegating fundraising activities solely to the staff can hinder efforts to land important donors.  Align the fundraising campaign with the organization’s mission. If the mission and donor’s priorities do not match, the odds of raising funds are significantly reduced. 29  Be transparent on how funds are being used. Prospective donors are more likely to give if they know how their money will be spent. Meanwhile, larger institutional and government donors will require such transparency as a minimum condition of awarding grants.  Demonstrate the results. Throughout the fundraising campaign, the organization should take care to refine its fundraising message to clearly and concisely express the outstanding, high-impact results that are delivered for the money raised. This will reassure new and existing donors alike of the organization’s effectiveness and sustainability.  Consider employing a development officer to enhance fundraising operations. These officers devise near- and long-term campaigns to maximize fundraising from all available potential sources, coordinate the fundraisers’ efforts, and provide guidance throughout the campaign. A caveat: fundraising should not be incumbent on the development officer. They should be expected to provide the tools, guidance, and framework for implementation, but the actual fundraising should be done by an organization’s board and staff – those who know, work, and live the organization’s mission.30 - 126 - IFESH Mid-Term Evaluation Report  Take full advantage of fundraising resources. In addition to enlisting a development officer, organizations have at their disposal other tools and resources to enhance fundraising efforts. These include, but are not limited to: o Networks and associations, such as the Association of Fundraising Professionals and the Better Business Bureau Wise Giving Alliance; o Grants repositories and support, such as the U.S. governmental Grants.gov (http://www.grants.gov) and The Foundation Center (http://www.foundationcenter.org); o Fundraising expertise and guidance providers such as The Grantsmanship Center (http://www.tgci.com); and o Fundraising rules and regulations to ensure compliance at http://www.irs.gov/charities/index.html. Given the importance of donors and grantors to the organization’s success in the field, fundraising should be a year-round endeavor. To avoid always pulling from the same pool of potential and existing donors, organizations should consider using and rotating through a few different approaches to fundraising. These approaches can include seeking grants, seeking major gifts, hosting special events, offering memberships and conducting membership drives, and general annual appeals through email, social networking, traditional media (print, television, and radio), mail, and telephone. Each of these will reach different audiences and will ensure that the organization’s revenue streams are sufficiently diversified and stable.31 In a tight economy, diversification of funders becomes an even more critical tactic to ensuring the survival of the organization and its mission. 5.0 Conclusion This compilation of best practices has focused on areas within volunteer management. While not all practices are suited for all organizations, certain practices have been highlighted for IFESH to consider adopting. For volunteer management to be successful, the resources (processes, technology, and people) within IFESH should also be examined and strengthened. A recent publication set forth the following characteristics that donors look for in nonprofits when making contributions to educational efforts in developing countries:  Concrete plan and deliverables  Reasonable administrative costs  Strong track record  Cultural fit  U.S. based staff  Relative needs and opportunities  Good communications in communities  Contacts and connections for the company  Capacity to expand and scale good ideas  Understanding business culture32 While this list may not be exhaustive, it demonstrates the need for an organization to have effective and efficient management and operations in place, and an understanding of the countries that they work in. Moving forward, IFESH should strive to achieve these characteristics in luring future donors. In working with donors, an organization may have the opportunity to inform a corporate donor of the challenges and realities in implementing educational initiatives in developing countries. A donor and recipient relationship may be considered hierarchical but a strong one is an exchange of information that ultimately benefits the local communities in reaching educational targets. - 127 - IFESH Mid-Term Evaluation Report 6.0 List of International Volunteering Organizations http://www.volunteersforprosperity.gov/index.php with more organizations listed at http://www.volunteersforprosperity.gov/vol/educ.php and other subsites http://www.vso.org.uk/ http://www.peacecorps.gov http://www.unv.org/ http://www.winrock.org/ http://www.acdivoca.org/ http://www.volunteerinternational.org/ 1 These structural components are all integral to a successful IVO, but fall outside the scope of this Mid-Term Evaluation. 2 Valuing Volunteer Management Skills (n.d.). Institute for Volunteering Research online. Retrieved from http://www.ivr.org.uk/projects/Current+Projects/Valuing+Volunteering+Management+Skills. 3 For purposes of this Mid-Term Evaluation, the Evaluation Team adjusted the Hiring term to Volunteer Appointment. 4 Goodman, Denise Ph.D. (1999). Targeted Recruitment: The Most Effective Strategy. Nacaconline. Retrieved from http://www.nacac.org/adoptalk/recruitment_strategy.pdf. 5 Graff, Linda L. (2002). “Emerging Trends and Issues in Volunteerism and Volunteer Program Management.” e-Volunteerism, Vol. 3 Issue 1, p. 11. 6 Maryland Advisory Committee on Volunteerism, “Best Practices for Developing a Volunteer Program” (2002). Retrieved from http://www.urbanministry.org/best-practices-developing-a-volunteer-program. 7 Targeted Recruitment (n.d.). Volunteer Centres Kent online. Retrieved from http://www.volunteercentreskent.org.uk/Default.asp?page=136. 8 Fixler, J. et al (n.d., p. 75). Boomer Volunteer Engagement: Collaborate Today, Thrive Tomorrow. 9 Williams, Emilio N. (2000). Recruitment and Retention: A Cross-Stream Challenge. The Resource Connection, Volume 4 Number 1, p. 1. Retrieved from http://www.nationalserviceresources.org/filemanager/download/637/rcv4n1.pdf. 10 Maryland Advisory Committee on Volunteerism. 11 Ibid. 12 Williams, p. 5. 13 Maryland Advisory Committee on Volunteerism. 14 Williams, p. 5. 15 Maryland Advisory Committee on Volunteerism. 16 Maryland Advisory Committee on Volunteerism. 17 Valuing Volunteer Management Skills. 18 Refer to Peace Corps “Culture Matters” publication as an example of a cultural sensitization handbook. Retrieved from http://www.peacecorps.gov/wws/educators/enrichment/culturematters/index.html. 19 Lynch, Rick (2000). Volunteer Retention and Feelings of Connection. e-Volunteerism, Vol. 1, Issue 1, p. 1. Retrieved from http://nc.casaforchildren.org/files/public/community/programs/ProgramResources/Volunteer_Retention_Lynch.pdf. 20 Valuing Volunteer Management Skills. 21 Maryland Advisory Committee on Volunteerism. 22 Volunteerism and Development (2003). World Volunteer Web online. Retrieved from http://www.worldvolunteerweb.org/fileadmin/docs/old/pdf/2003/essentials_vol.pdf. - 128 - IFESH Mid-Term Evaluation Report 23 Sherraden, Margaret, et al (2008). Impacts of International Volunteering and Service: Individual and Institutional Predictors. Center for Social Development, pg. 6. Retrieved from http://www.forum-ids.org/resources/papers/Center-for-Social￾Development-Impacts-of-International-Volunteering-Service-2008.pdf. 24 Volunteer Management Practices and Retention of Volunteers (2004, p. 9). Urban Institute online. Retrieved from http://www.urban.org/url.cfm?ID=411005. 25 Ibid. 26 Ibid. 27 Sherraden, Margaret et al, p. 6. 28 North Carolina Center for Nonprofits (2007). “Fundraising: Good Strategies, Best Practices, and Useful Resources.” Retrieved from http://www.volunteer-connections.org/express/nccenteronfundraising.pdf. 29 Ibid. 30 Poderis, Tony (n.d.). “Who Should Raise the Money from within Your Organization?” Retrieved from http://www.raise￾funds.com/2004/who-should-raise-the-money-from-within-your-organization/. 31 North Carolina Center for Nonprofits. 32 Van Fleet, J. (2011). A Global Education Challenge: Harnessing Corporate Philanthropy to Educate the World’s Poor. Center for Universal Education, Brookings Institute, Working Paper 4, pp. 35-36. - 129 - IFESH Mid-Term Evaluation Report Appendix I: American Volunteer Educator Tables 1.0 American Volunteer Educators (AVEs) for 2009-20101 Djibouti Ethiopia Ghana Kenya Liberia Malawi Nigeria Senegal TOTAL Percent of Total: Total Number of AVEs for 2009-2010 6 3 2 2 8 4 2 6 33 100.0% Male AVEs for 2009-2010 2 1 1 1 3 2 0 2 12 36.4% Female AVEs for 2009-2010 4 2 1 1 5 2 2 4 21 63.6% Number of AVEs with Bachelors Degree (BA or BFA) 4 0 1 0 1 2 0 1 9 27.3% Number of AVEs with Masters Degree (MA, MS, MFA, MPH, or M.Ed.) 1 1 1 0 7 1 1 3 15 45.5% Number of AVEs with Doctorate Degree (Ed.D., Ed.S. or Ph.D.) 1 2 0 2 0 1 1 2 9 27.3% Number of AVEs with 0-5 Years Teaching Experience 3 1 0 0 4 1 0 3 12 36.4% Number of AVEs with 6-10 Years Teaching Experience 2 1 0 1 3 1 1 1 10 30.3% Number of AVEs with 11-20 Years Teaching Experience 0 1 2 0 1 1 1 0 6 18.2% Number of AVEs with More Than 20 Years Teaching Experience 1 0 0 1 0 1 0 2 5 15.2% Number of AVEs Who Completed Contract 2 3 2 2 8 4 0 6 27 81.8% Number of AVEs Who Did Not Complete Contract* 4 0 0 0 0 0 2 0 6 18.2% * = AVEs who did not complete his/her contract due to either early termination or late arrival. - 130 - IFESH Mid-Term Evaluation Report 2.0 American Volunteer Educators (AVEs) for 2010-20112 Djibouti Ethiopia Ghana Kenya Liberia Malawi Nigeria Senegal TOTAL Percent of Total: Total Number of AVEs for 2010-2011 3 5 4 4 8* 5 5 7 41 100.0% Male AVEs for 2010-2011 0 2 2 2 3 1 1 1 12 29.3% Female AVEs for 2010-2011 3 3 2 2 5 4 4 6 29 70.7% Number of AVEs with Bachelors Degree (BA or BFA) 0 1 1 0 1 2 1 2 8 19.5% Number of AVEs with Masters Degree (MA, MS, MFA, MPH, or M.Ed.) 2 2 2 0 7 1 4 3 21 51.2% Number of AVEs with Doctorate Degree (Ed.D., Ed.S. or Ph.D.) 1 2 1 4 0 2 0 2 12 29.3% Number of AVEs with 0-5 Years Teaching Experience 1 1 0 0 4 0 0 5 11 26.8% Number of AVEs with 6-10 Years Teaching Experience 1 2 0 2 2 1 0 1 9 22.0% Number of AVEs with 11-20 Years Teaching Experience 0 0 2 1 2 3 1 0 9 27.3% Number of AVEs with More Than 20 Years Teaching Experience 1 2 2 1 0 1 4 1 12 36.4% * = Two AVEs terminated their contracts in Liberia since conducting the interviews in February 2011 per the January-March 2011 Quarterly Report. 1 Figures received from IFESH HQ. 2 Ibid. - 131 - IFESH Mid-Term Evaluation Report Appendix J: IFESH Gap Analysis Definitions: 5 – Leading: The organization has differentiated itself and quickly provides innovative solutions at the enterprise level. 4 – Optimizing: The organization has not only developed capabilities but also actively integrates them into its daily operations. 3 – Practicing: The organization has some capabilities in place and is utilizing past feedback and experiences to improve operations. 2 – Developing: The organization exhibits conceptual knowledge of desired capabilities and recognizes the benefits. 1 – Aware: Organization lacks capabilities and exhibits limited knowledge and execution. - 132 - IFESH Mid-Term Evaluation Report Appendix K: Revising the Performance Monitoring Plan (PMP) The Evaluation Team recommends that the International Foundation for Education and Self-Help (IFESH) make adjustments to the existing Performance Monitoring Plan (PMP) contained in the American Educators for Africa (AEFA) Program Monitoring & Evaluation Manual that was previously revised in September 2010. As such, the Evaluation Team presents a revised PMP. We consolidated the original seven Strategic Objectives (SOs) into five SOs; of these five, we developed one new objective (numbered SO 5). The Evaluation Team eliminated some indicators to simplify the PMP. We left in the useful monitoring methods that have not been utilized by IFESH such as surveys to measure the satisfaction of host institutions, Ministries of Education (MoEs), and the U.S. Agency for International Development (USAID); also instruments to measure usage of Teacher Resource Centers (TRCs). The Evaluation Team added new indicators that were commended in this Mid-Term Evaluation Report, including monitoring AVE applications received year round, surveys to measure the quality of volunteer management as perceived by American Volunteer Educators (AVEs), feedback on AEFA personnel performance, and milestones (to be established) in a new strategic plan for AEFA sustainability. Lastly, the activities and indicators in SO4 related to Local Non-Governmental Organizations (LNGOs) and education stakeholders have been adjusted to be more inclusive of educational stakeholders apart from LNGOs since they have been found to be the major recipients of small grants. The Evaluation Team connected each category with the number of program objectives for SO5. This practice aligned elements such as the frequency and indicators of each program objective within the SO. If IFESH decides to re-visit their revised PMP, not only should they consider the revisions presented herein, but should also create clarity among the categories and program objectives for all other SOs. - 133 - IFESH Mid-Term Evaluation Report Goal: To strengthen basic education of the target countries by enhancing their capacity to provide pedagogical support to primary and secondary school teachers and education administrators to enable them to achieve the EFA and MDGE goals. SO3 Integrate HIV/AIDS education and gender awareness into the educational curriculum in the TTIs as well as in primary and secondary school and at the community level to engage and strengthen community participation in efforts to reduce HIV/AIDS infection and manage its impact and promote gender equality. SO 5 Implement AEFA through the provision of high quality volunteer support with adequate health care, safety, and security arrangements, proficient internal and external communications, and effective administration of resources. Regularly monitor performance, report to stakeholders, and act on feedback to achieve sustainability. SO2 Provide technical assistance and training to pre-and in-service teachers and education administrators to improve teachers’ content mastery and classroom instructional practices leading to measurable increases in pupil achievement and overall education quality that they can sustain. Support will be in the areas identified by MoEs, host institutions and USAID, to include: pedagogy, curriculum, early childhood education, public health, education management, strategy and policy development, educational research, impact evaluation procedures, and classroom assessment. Devise innovative approaches to support teachers, including Teacher Resource Centers, donated books, Public￾Private Partnerships, and technology. SO1 Recruit and deploy a diverse cadre of experienced and committed American educators who provide the skills African institutions want to improve their educational systems. SO 4 Increase capacity of local nongovernmental organizations, community-based groups and education stakeholders to implement sustainable development programs which support education. - 134 - IFESH Mid-Term Evaluation Report Performance Monitoring Plan for IFESH‐AEFA (Rev 9‐2010) PROGRAM OBJECTIVE PERFORMANCE INDICATOR INDICATOR DEFINITION AND UNIT OF MEASUREMENT DATA SOURCE METHOD OR APPROACH OF DATA COLLECTION DATA COLLATION, VALIDATION AND ANALYSIS REPORTING Schedule Frequency Responsible Party Schedule By Date Responsible Party & Recipient/Rpt GOAL: To strengthen basic education of the target countries by enhancing their capacity to provide pedagogical support to primary and secondary school teachers and education administrators to enable them to achieve EFA and MDGE goals. Client satisfaction with IFESH‐AEFA activities Changes in key country education statistics Client satisfaction: the degree to which MoEs, TTIs, & Missions rate program performance (quantitative) Their views of how IFESH is most helpful and what they want for the future (qualitative) Questionnaires completed by MoEs, TTIs, & Missions or telephone interview forms AEFA Country Plan with Goals MoE & DEO UNESCO EFA Monitoring Reports Conduct survey then follow up with telephone interviews Review reports to inform AEFA assignments Prepare AEFA Country Plan Annually – start early to refine process Ongoing – monitor info as it becomes available HQ M&E Staff CR & M&E/AVE Quarterly Annually, mid‐ year, to inform assignments HQ M&E Staff →  USAID Quarterly & Annual Reports & at Orientation CR & M&E/AVE → USAID Annual Report & at Orientation PROGRAM OBJECTIVE PERFORMANCE INDICATOR INDICATOR DEFINITION AND UNIT OF MEASUREMENT DATA SOURCE METHOD OR APPROACH OF DATA COLLECTION DATA COLLATION, VALIDATION AND ANALYSIS REPORTING Schedule Frequency Responsible Party Schedule By Date Responsible Party & Recipient/Rpt SO1‐ Recruit and deploy a diverse cadre of experienced and committed American educators who provide the skills African institutions want to improve their educational systems. ACT 1. Recruit, orient, & deploy highly qualified AVEs to train pre‐service teachers at TTIs, in‐service No. of applications received‐target: 50 per quarter (year round process) Appropriate qualifications: recruits match requirements set in country job AVE applications Post‐Training Assessment Tabulations of applicant data Survey of host institutions, Quarterly ‐ applications Annually ‐ #s of Recruiter & AEFA Manager CR & M&E/AVE Educator data: annually, in Dec. Survey: in time HQ M&E Staff CR & M&E/AVE - 135 - IFESH Mid-Term Evaluation Report teachers through CPD activities & support MoE/DEO in education governance issues. 2. Recruit counterpart LVEs to work with AVEs to ensure sustainability and country ownership. 3. Identify appropriate placements with MoEs and host institutions. No. of AVEs & LVEs recruited & deployed by assigned country , gender, with appropriate qualifications, & experience Appropriate institutional assignments developed Educator ratings of U.S.& in‐country orientations description Appropriate placements: they align with country needs to meet EFA & MDGE goals Orientation ratings: U.S. and country orientations achieve key training objectives (i.e., Educators understand the objectives of IFESH‐ AEFA, their assignment is clear, they know how to execute key components of their assignment, they know M&E & reporting expectations) form Customer satisfaction surveys from MoE, host institutions & Missions MoEs, and USAID & follow up calls Forms should rate different components of orientation so feedback can be utilized for improvement AVEs Quarterly Dir of Research, AEFA Dir, M&E Staff, M&E Consultant to have data for QR Annual Orientation Assessment by HQ M&E Staff Country Assessment by CR and M&E/AVE TO: CRs & Orientation Assessments to CEO & AEFA Dir for action All reports used in USAID Quarterly & Annual Reports HQ M&E Staff & Consultant → CEO & USAID Quarterly & Annual Reports PROGRAM OBJECTIVE PERFORMANCE INDICATOR INDICATOR DEFINITION AND UNIT OF MEASUREMENT DATA SOURCE METHOD OR APPROACH OF DATA COLLECTION DATA COLLATION, VALIDATION AND ANALYSIS REPORTING Schedule Frequency Responsible Party Schedule By Date Responsible Party & Recipient/Rpt SO2 ‐ Provide technical assistance and training to pre‐and in‐service teachers and education administrators to improve teachers’ content mastery and classroom instructional practices leading to measurable increases in pupil achievement and overall education quality that they can sustain. Support will be in the areas identified by MoEs, host institutions and USAID, to include: child‐centered pedagogy, curriculum, early childhood education, public health, education management, strategy and policy development, educational research, impact evaluation procedures and classroom assessment. Devise innovative approaches to support teachers, including Teacher Resource Centers, donated books, public/private partnerships, and technology. Use knowledge management to enhance teacher training effectiveness and sharing with others. Assess interventions for effectiveness. ACT - 136 - IFESH Mid-Term Evaluation Report Pre‐Service: 1. Deploy AVEs to TTIs to teach the above themes. 2. Educators use IFESH TT took‐kit as appropriate. 3. Bennett College & Lincoln U. compile TT tool‐kit with input from past & former educators. Phase all sections in over 3 years based on feedback about needs. No. of pre‐service (new) teachers trained (m/f) No. of TLMs produced Ratings by AVEs of utility of IFESH TT tool‐kit & other materials prepared by Bennett College and Lincoln University Teacher trained unit = person completing semester course>24 hours long A TLMs title is each product developed by Educator, e.g., a booklet, a poster Toolkit comprises components compiled for CD during 2009 orientation but other resource material from relevant sources Course records Post‐training self‐assessment forms Pre‐post test results Practice teaching observation results Tabulations from course records Tabulations of assessment forms summarizing types of skills gained & types of attitudes changed, TLMs source List TLMs developed (title & category) Quarterly Baseline: Most student teachers start with 0 skills & some incorrect attitudes AVE & LVE (per course & summary) CRs (country summary) Dir. Of Ed. Bennett College & Lincoln University Proj. Directors Quarterly HQ M&E Staff In‐Service: 1. Hold group training CPD workshops for in‐ service teachers & school administrators on the SO themes and other cross‐ cutting issues. Emphasize modern relevant education themes consistent with No. of teachers trained (m/f) Trainees acquire new skills & change attitudes (specify both) No. of TLMs titles developed Teacher trained unit = person completing semester course>24 hours long No. of TLMs developed by Educator No. of follow up classroom observations, feedback sessions & Course records Post‐training self‐assessment form Form for follow up classroom observations, feedback sessions & technical assistance Tabulations from course records Tabulations of assessment forms summarizing types of skills gained & types of attitudes changed, TLMs source Quarterly Baseline: Pre‐ test of knowledge levels & attitudes AVEs & LVEs (per course & summary) CRs & M&E/AVEs (country summary) Quarterly (CPD results) HQ M&E Staff HQ M&E Staff USAID Quarterly & Annual Reports - 137 - IFESH Mid-Term Evaluation Report host country reforms. 2. Promote support to training including classroom observation & feedback. technical assistance sessions Toolkit comprises components compiled for CD during 2009 orientation plus other resource material from relevant sources sessions Training assessments by participants Training evaluation results Innovative Approaches: 1. Establish or upgrade TRC & teaching outposts in rural & educationally disadvantaged areas. 2. Establish “teaching social networks” for knowledge sharing among teachers including through the use of mobile phones. 3. Provide books & TLMs to school, TRC & libraries. 4. Establish No. of TRCs & rural teaching outposts established No. of teachers & school administrators using the TRC to upgrade their competencies (m/f) No. & financial value of donated books & TLMs provided to assist teachers No. of schools (& teachers) participating in the CPD monthly half‐ No. of TRCs IFESH personnel establish or upgrade The value of donated books is as stated by the supplier and the no. is how many are distributed to beneficiaries An innovation can be a product (a curriculum, a course), a process, or an organizational method Each PPP formed is a unit Quarterly Reports of AVE & LVE Survey Report School Reports Publications of MoE/DEO Extraction from Reports Surveys Quarterly CRs Quarterly HQ M&E Staff ALL reported in Quarterly & Annual Reports to USAID - 138 - IFESH Mid-Term Evaluation Report Teacher Exchange Magazines or Journals at national and/or district levels. 5. Support or train uncertified teachers to obtain qualifications to join the teaching professional cadre. days training activities No. of innovations introduced at school, district or MoE level No. of PPPs formed Policy: Execute assignments (policy research, analysis, etc.), provide technical assistance, & training as required. List and descriptions of policy engagements No. of trainees (for training sessions) Assessments of training Policy engagement: each time an Educator is assigned a task. Training = one‐on‐ one or groups Policy engagement report form Training assessment forms Summarize on Quarterly Report Form Educator prepares form for each policy engagement Educator administer training form after each Quarterly Baseline: no engagements unless assignment is a continuation of another AVE AVEs & LVEs compile summary Dir. of Ed. Quarterly HQ M&E Staff Knowledge management: Design and implement system. System design and usage integrated into AEFA materials and training Satisfaction ratings on usage Usage rating scale defined in satisfaction survey AVEs, host institutions, MoEs, & USAID Survey of AVEs Survey of host institutions, MoEs, & USAID Semi‐Annually – to be part of other data collection HQ M&E Staff Semiannually HQ M&E Staff Assess impact of Some evidence of improvement used Any accepted Training and technical Pre‐post tests, training Quarterly AVEs in quarterly Quarterly CR→ HQ M&E Staff → AEFA - 139 - IFESH Mid-Term Evaluation Report interventions. in at least 60% of interventions assessment method assistance recipients evaluation forms, group feedback sessions reports Managers →  USAID PROGRAM OBJECTIVE PERFORMANCE INDICATOR INDICATOR DEFINITION AND UNIT OF MEASUREMENT DATA SOURCE METHOD OR APPROACH OF DATA COLLECTION DATA COLLATION, VALIDATION AND ANALYSIS REPORTING Schedule Frequency Responsible Party Schedule By Date Responsible Party & Recipient/Rpt SO3‐Integrate HIV/AIDS education and gender awareness into the educational curriculum in the TTIs as well as in primary and secondary school and at the community level to engage and strengthen community participation in efforts to reduce HIV/AIDS infection and manage its impact and promote gender equality. ACT 1. Training of teachers, school administrators, & LNGOs on HIV/AIDS awareness and integration into the education curriculum. 2.Formation of HIV/AIDS and Gender Equity clubs. 3.Support of PLWHA and OVCs. Percentage of AVE activities that incorporate HIV/AIDS or gender awareness. Target: 50% AVEs will report on community involvement but no target is set because it is not possible with some AVE assignments Person trained: participant in any specific workshop Adaptation of HIV/AIDS Curriculum: When an institution adds it as an offering LNGOs assisted: those that receive TA or training from IFESH Post‐training assessment forms Or Group Report Form (for mass meetings or low literacy audiences) TA Report Form (Provide a box to tick HIV/AIDS or gender aspect for all activities) Quarterly Reports of AVE & LVE Quarterly Extraction from Reports Surveys CRs Quarterly HQ M&E Staff All reported in Quarterly & Annual Reports to USAID PROGRAM OBJECTIVE PERFORMANCE INDICATOR INDICATOR DEFINITION AND DATA SOURCE METHOD OR APPROACH OF DATA COLLATION, VALIDATION AND ANALYSIS REPORTING - 140 - IFESH Mid-Term Evaluation Report UNIT OF MEASUREMENT DATA COLLECTION Schedule Frequency Responsible Party Schedule By Date Responsible Party & Recipient/Rpt SO 4‐Increase capacity of local nongovernmental organizations, community‐based groups and education stakeholders to implement sustainable development programs which support education using technical assistance and small grants. ACT 1. Identify & establish relationship with LNGOs & other education stakeholders that can sustain AEFA objectives. 2. Conduct competency assessment & needs surveys. 3. Design and deliver capacity building interventions to address needs and capacity gaps. 4. Develop guidelines for transparent solicitation, award & monitoring of small grants. No. LNGOs & other education stakeholders trained in design & implementation of sustainable education support programs No. (and financial value) of sub‐ grants provided to LNGOs & education stakeholders No. of beneficiaries of educational support programs of assisted & trained stakeholders No. of LVEs involved Status of applications, LNGOs assisted: those that receive TA or training from IFESH Quarterly Reports of AVE and LVE School Reports Quarterly Reports of AVE or Country Rep if no AVE is involved Baseline: Capacity level of beneficiaries based on assessment CRs Quarterly HQ M&E Staff ALL reported in Quarterly & Annual Reports to USAID - 141 - IFESH Mid-Term Evaluation Report 5. Develop toolkit for use by AVE/LVEs working with LNGOs & small grants. awards & monitoring process PROGRAM OBJECTIVE PERFORMANCE INDICATOR INDICATOR DEFINITION AND UNIT OF MEASUREMENT DATA SOURCE METHOD OR APPROACH OF DATA COLLECTION DATA COLLATION, VALIDATION AND ANALYSIS REPORTING Schedule Frequency Responsible Party Schedule By Date Responsible Party & Recipient/Rpt SO 5‐Implement AEFA through the provision of high quality volunteer support with adequate health care, safety, and security arrangements, proficient internal and external communications, and effective administration of resources. Regularly monitor performance, report to stakeholders, and act on feedback to achieve sustainability.*Volunteer management life cycle: Recruitment, Volunteer Appointment, Orientation and Training, Deployment, Field Support, Reporting, Performance Evaluation, Retention, and End of Service. ACT 1. Implement the volunteer management life cycle.* 2. Use the AEFA M&E system to monitor & manage program performance & report to key stakeholders. 3.Design/monitor communications plans in field & HQ (can be part of work plan). 4. Design and use 1. Satisfaction ratings by AVEs, host institutions, MoEs, & USAID 2. Higher than average performance for all AEFA performance objectives 3. Plan milestones executed & any results 4. Most AEFA personnel perform better 1. Scale to be determined by M&E staff 2. Indicators are as set above in this PMP 3. Defined in the communications plans 4. Appraisal system to be determined by IFESH HR department 5. Strategic plan goals and indicators to be developed by IFESH Board and senior 1. Surveys of host institutions, MoEs, USAID, & AVEs 2. As specified for each SO above 3. Personnel activity reports 4. Personnel appraisals 5. New AEFA funding sources, AEFA 1. Surveys & telephone calls 2. As specified for each SO above 3. Activity report form 4. Staff appraisals of all AEFA staff 5. To be determined as part of strategic plan 1. Tri‐annually or semi‐ annually 2. Quarterly 3. Quarterly 4. Tri‐annually 5. TBD in strategic plan 1. M&E staff & CRs 2. CRs, AVEs, M&E staff, AEFA managers 3. CRs & AEFA managers 4. Supervisors of CRs, AEFA managers, & staff 5. TBD in strategic plan 1. Plan before AVEs go to the field 2. According to each SO above 3. Beginning of each quarter 4. Set out start & completion dates when system is instituted 5. TBD in strategic plan 1. AEFA managers & USAID 2. AEFA managers→ host institutions & USAID 3. AEFA managers & CEO & USAID 4. AEFA managers, CEO & IFESH Board 5. IFESH Board & - 142 - IFESH Mid-Term Evaluation Report a personnel appraisal system that links key performance areas (KPAs) of IFESH staff to program KPAs. 5. Develop and implement a strategic plan for sustainability of AEFA. than average on most KPAs 5. Board, CEO, and senior management monitor strategic plan regularly and make progress toward plan goals management recognition, PPPs, etc., to be identified in new Strategic Plan CEO - 143 - IFESH Mid-Term Evaluation Report Appendix L: Work Plan The Evaluation Team produced a Work Plan that was approved by USAID on November 16, 2010. Since its development, adjustments to the contents based on field visits and the timeline were made as agreed to by USAID. The following include the Work Plan and Work Plan Addendum dated November 16, 2010. - 144 - IFESH Mid-Term Evaluation Report - 145 - IFESH Mid-Term Evaluation Report - 146 - IFESH Mid-Term Evaluation Report - 147 - IFESH Mid-Term Evaluation Report - 148 - IFESH Mid-Term Evaluation Report - 149 - IFESH Mid-Term Evaluation Report - 150 - IFESH Mid-Term Evaluation Report - 151 - IFESH Mid-Term Evaluation Report - 152 - IFESH Mid-Term Evaluation Report - 153 - IFESH Mid-Term Evaluation Report - 154 - IFESH Mid-Term Evaluation Report - 155 - IFESH Mid-Term Evaluation Report - 156 - IFESH Mid-Term Evaluation Report Appendix M: Conflict of Interest Statement Prior to the initiation of the Mid-Term Evaluation, all four members of the Evaluation Team were asked to disclose any conflict(s) or potential conflict(s) of interest with undertaking the evaluation of the International Foundation for Education and Self-Help (IFESH). Ms. Yael Cohen, Mr. Michael Matthews, and Mr. Obie Shaw disclosed no conflict or bias. Ms. Gayla Cook-Mohajane disclosed that she had been previously contracted by IFESH to provide Monitoring, Evaluation, and Reporting (ME&R) guidance but maintained this previous work would not inhibit her objectivity or independence in conducting the evaluation of IFESH. None of the Evaluators encountered any conflicts during the course of the evaluation. Morgan Borszcz Consulting, LLC, reports that no violation took place during the evaluation period as signed by Ms. Yael Cohen, on behalf of all of members of the Evaluation Team. Signature: Date: July 19, 2011 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov