USAID Pakistan March 14, 2011 This report was produced for review by the United States Agency for International Development by International Business & Technical Consultants, Inc. (IBTCI) under the Evaluation Services IQC, AID-RAN-I-00-09-00016, REQ￾LAC-10-000019. Its authors are Frank Dall, Jeffrey Tines, Ana Cristina Accioly de Amorim, Sheere Kadez Brooks, Thomas D. Tilson, and Luis Enrique Medrano. EVALUATION Evaluation of the LAC/RSD Regional Education Program Volume II. Annexes March 14, 2011 This report was produced for review by the United States Agency for International Development by International Business & Technical Consultants, Inc. (IBTCI) under the Evaluation Services IQC, AID-RAN-I-00- 09-00016, REQ-LAC-10-000019. Its authors are Frank Dall, Jeffrey Tines, Ana Cristina Accioly de Amorim, Sheere Kadez Brooks, Thomas D. Tilson, and Luis Enrique Medrano. Evaluation of the LAC/RSD Regional Education Program Volume II. Annexes Evaluation of the LAC/RSD Regional Education Program Volume II. Annexes March 14, 2011 Prepared by: Frank Dall (Team Leader) Jeffrey Tines Ana Cristina Accioly de Amorim Sheere Kadez Brooks Thomas D. Tilson Luis Enrique Medrano The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Evaluation of LAC/RSD Regional Education Program: Volume II International Business & Technical Consultants, Inc. Table of Contents Annex I: Statement of Work .......................................................................................................................................... 1 Annex II: Team Members .............................................................................................................................................. 12 Annex III. Final Work Plan ............................................................................................................................................. 13 Annex IV: Data Collection Instruments ..................................................................................................................... 31 Annex V: List of People Interviewed .......................................................................................................................... 61 Annex VI: Online Survey Questionnaire Results and Analyses ............................................................................ 67 Annex VII: Findings Matrices ......................................................................................................................................... 74 Annex VIII. Bibliography .............................................................................................................................................. 160 Evaluation of LAC/RSD Regional Education Program: Volume II 1 International Business & Technical Consultants, Inc. Annex I: Statement of Work Terms of Reference and Scope of Services USAID/LAC BUREAU Statement of Work Evaluation of LAC-/RSD Regional Education Program 1. Purpose of the Evaluation The purpose of the evaluation is to assess the impacts and comparative advantages of Latin America and the Caribbean (LAC) Regional Education programming in order to inform the design of new USAID/LAC regional education interventions. LAC’s aim is to understand the types of programs best conducted regionally rather than bilaterally and the factors to consider in determining program organization and management. The main objectives of the evaluation are to evaluate the benefits and challenges of supporting different models for regional programming, including the advantages and disadvantages of working regionally versus bilaterally to address the education development challenges, and contrasting the different models for program implementation at the USAID and implementing partner level as discussed in the background section; identify the broader impacts of the regional programs; and consider the value-added of technical assistance provided to implementing partners through USAID/Washington-based contracts and cooperative agreements. The evaluation will also provide recommendations for the design of future regional assistance programs in the education sector. 2. Background The USAID Latin American and Caribbean Bureau has taken a two-pronged regional approach to address challenges in the education sector. The first focuses on improving the quality of education by creating effective models for change. Through the hemispheric Centers of Excellence for Teacher Training (CETT), LAC has developed an effective model for improving the teaching of literacy in the early grades of primary education. CETT has also worked to strengthen educational evaluation assessment in the region. To date, CETT has trained over 31,200 teachers and benefited over 875,000 children. The second prong involves improving the quality and relevance of policy dialogue around education reform, both as a means of strengthening regional capacity to implement reforms, as well as a way to build political support for improving educational quality. Over the past 14 years, the LAC Bureau has worked to create more informed policy dialogue by reaching out to both government and non-governmental actors across the region. A key aspect of LAC’s efforts to Evaluation of LAC/RSD Regional Education Program: Volume II 2 International Business & Technical Consultants, Inc. strengthen the regional policy dialogue has been an emphasis on improving educational accountability, including support for the development and dissemination of reports on the quality of national and sub-national education systems. 2.1. Teacher Training: The Center of Excellence for Teacher Training (CETT) In 2002, USAID established three regional Centers of Excellence for Teacher Training (CETT) in Latin America and the Caribbean, a Presidential Initiative announced at the 2001 Summit of the Americas. The goal of the three Centers was to address the high rates of illiteracy and school underachievement through improved reading instruction in the early grades (1-3). Each regional CETT developed and implemented a model for providing effective in-service training to teachers to strengthen the literacy instruction skills of teachers. Training was accompanied by teacher circles and in-classroom facilitation and support. During the final years of implementation, each regional CETT refined and systematized the program. As the Centers of Excellence for Teacher Training graduate from LAC funding in 2010, partnerships with Ministries, the private sector, nongovernmental organizations, and other donors have been developed to support sustainability of impact and ongoing use of the model. Partnerships with local and international companies, including Scholastic, are improving reading instruction for primary school children by contributing in-kind and monetary resources. Three regional CETTs were implemented, each with a unique model of program management. The Caribbean CETT (CCETT) was implemented through a cooperative agreement with the Joint Board of Teacher Education (JBTE), which implemented the program across the English￾speaking Caribbean. CCETT was implemented in Jamaica, St. Lucia, St. Vincent and the Grenadines, Belize, Guyana, and Dominica with USAID funding. Trinidad and Tobago and Grenada self-financed implementation with technical support from the CCETT. (In the case of Grenada, reprogrammed USAID funds provided minimal supplementation to the self-financing.) In late FY 2009, five additional countries signed Memoranda of Understanding to self-finance implementation of the program with technical assistance from the JBTE. These countries are: St. Kitts and Nevis, Antigua and Barbuda, Anguilla, Montserrat, and the British Virgin Islands. The CETT in Central America and the Dominican Republic (CARD CETT) was implemented through a cooperative agreement with the Universidad Pedagógica Nacional (UPN) in Honduras, which worked in consortium with institutions in Guatemala, Nicaragua, El Salvador, and the Dominican Republic to develop and implement the program in these countries. Each institution led development of a separate component of the program. Honduras was responsible for teacher training, Guatemala for testing and evaluation, the Dominican Republic for materials development, El Salvador for partnerships and sustainability, and Nicaragua supported validation of all materials and components. Mexico provided technical assistance to the program, focusing on the use of information and communication technologies. In the Andes, CETT was implemented through a cooperative agreement with the Universidad Peruana Cayetano Heredia (UPCH) in Peru. UPCH worked collaboratively with institutions in Ecuador and Bolivia to develop and implement the training. Development teams cross-cut the country teams, with a specialist from each country contributing to the development of the Andean CETT program components. Evaluation of LAC/RSD Regional Education Program: Volume II 3 International Business & Technical Consultants, Inc. To build local capacity and strengthen program implementation, USAID/Washington provided technical assistance to the CETT implementing partners through various mechanisms. Examples of the assistance provided include: (1) hemispheric workshops organized to bring together staff from all three CETTs to share information as well as learn about and address timely themes and issues; (2) regional workshops on relevant topics targeted to a single regional CETT; (3) exchange trips/study tours; (4) expert technical assistance on topics such as best practices in literacy instruction, test development and analysis, development of assessment or evaluation tools, planning for sustainability, partnership development, and other pertinent topics; and (5) mid-term evaluations designed to identify program strengths and weaknesses to inform implementation. 2.2. Policy Reform Activities 2.2.1. Partnership for Educational Revitalization in the Americas (PREAL) The Partnership for Educational Revitalization in the Americas (PREAL) is a regional program working with a network of public and private organizations in Latin America that share a common interest in promoting educational reform in the region. Created in 1995, PREAL is jointly managed by the Inter-American Dialogue in Washington, DC and the Corporación de Investigaciones para el Desarrollo (CINDE), in Santiago, Chile. PREAL is implemented through a cooperative agreement with the Inter-American Dialogue. PREAL collaborates with a broad range of public and private sector stakeholders, including governments, NGOs, and international organizations. The program seeks to improve the quality and equity of education by promoting and encouraging the implementation of better education policies. Mechanisms used to carry out this task include regional working groups, conferences and workshops, publications, and electronic communications networks, including internet websites. PREAL’s activities extend throughout the entire region through extensive distribution of numerous publications and participation in education policy-related events in various countries. PREAL has considerable engagement in countries producing national education report cards and establishing national and business partnerships. While PREAL’s geographic scope is broad and extends to non-USAID education presence countries, PREAL is particularly active in those countries where USAID missions have bilateral education programs, including El Salvador, Guatemala, Honduras, Nicaragua, the Dominican Republic, and Peru. 2.2.2. Civic Engagement for Education Reform in Central America (CERCA) In addition to PREAL, LAC/RSD Education supported a second regional policy reform activity, Civic Engagement for Education Reform in Central America (CERCA), from 2002-2006. Complementing work undertaken by PREAL, CERCA worked to build sustained, bottom-up support for improving education quality in five countries: Guatemala, Honduras, El Salvador, Nicaragua and Dominican Republic. The program aimed to create conditions for parents and local communities to engage actively in supporting schools and in holding schools and policy makers accountable for improving the quality of basic education. Evaluation of LAC/RSD Regional Education Program: Volume II 4 International Business & Technical Consultants, Inc. 2.3. Prior and Ongoing Evaluations In order to strengthen its programming, LAC/RSD Education has funded several evaluations of its programs over the past 10 years. Evaluations to assess program progress and impact over time of PREAL were completed in 20011[1] and 2006. Evaluations of CETT include a midterm evaluation (2004) of teacher performance, a review of the professional development and materials (2006), and an assessment of the private partnership component (2005). The first year of an impact evaluation has been completed; the evaluation assesses the impacts of the CETT training program on teacher knowledge and practice, as well as on student performance. LAC/RSD is currently completing the second year of this evaluation, as well as several focused assessments on specific topics related to CETT implementation. Additionally, USAID/Peru is in the process of completing an evaluation of bilateral and regional basic education activities, including CETT, and USAID/Guatemala is completing an evaluation of CETT regarding its impact on student learning and active classroom management. 2.4. USAID Program Management CETT. While funding and reporting of the hemispheric CETT program were the responsibility of the LAC/RSD Education Team, responsibility for day-to-day management and oversight of the country-based implementing institutions partners was the responsibility of the field Missions. The Agreement Officer’s Technical Representative (AOTR) for each CETT cooperative agreement was based in the respective USAID Mission office in the country of lead institutions. This arrangement had the advantage of facilitating strong oversight of program leadership and implementation. Coordination between CETT and the bilateral education program (and other bilateral activities) was also readily facilitated within each focus country. At the same time, the arrangement presented major challenges relating to flow of information, coordination, and decision-making. For example, tensions arose around communications between the CETT implementing partners, local government officials, and USAID field offices in some of the CETT countries. Coordination with bilateral programs in the other countries was a challenge. Finally, the quality of reporting and data provided to LAC/RSD presented an ongoing challenge. 1[1] Full report available at: http://pdf.usaid.gov/pdf_docs/PDABU827.pdf PREAL. AOTR responsibility for PREAL resides in LAC/RSD together with funding and reporting responsibilities. Oversight of communications and coordination with USAID field offices is the responsibility of LAC/RSD. 3. Statement of Work 3.1. Key Questions In undertaking the evaluation, the following key questions must be considered in addition to other relevant questions the offeror may propose. In addressing these questions, use the “natural experiment” inherent in the different models for program coordination, management, and implementation (outlined in background section). For example, each regional CETT program is structured and managed differently, and the USAID bilateral education landscape also differs between regions. Dimensions to consider include: administrative and technical Evaluation of LAC/RSD Regional Education Program: Volume II 5 International Business & Technical Consultants, Inc. coordination and management of the program by USAID and between USAID offices; implementing partner administrative and technical management and coordination; types of intervention; and USAID bilateral presence. a. What are the benefits and challenges of supporting regional program models, such as are used in CETT and PREAL? Benefits and challenges • What were the benefits of supporting regional education programs?/What was gained by working regionally? • What were the challenges of supporting regional education programs? How could those challenges have been reduced? • What was sacrificed/lost by working regionally? • What were the advantages and disadvantages of the different models of program management and coordination (outlined in the background section) – by and for USAID? By and for implementing partners? Coordination and synergy • What worked, and what didn’t work, in the coordination of each regional education program between USAID/W, USAID missions, program partners, contractors, and other key stakeholders? • How closely did the regional programs complement and support bilateral mission activities? • How have program partners supported inter-program/project synergies with bilateral programs? • Was there sufficient coordination between the CETT and PREAL regional programs and mission bilateral education programs? • What factors limited coordination and synergy between the regional education programs and the bilateral education programs? • What factors facilitated coordination and synergy between the regional education programs and the bilateral education programs? • How could the CETT and PREAL regional education programs have worked better with bilateral education programs? • What synergies and coordination were there between CETT and PREAL? • What factors (administrative, technical, etc.) limited coordination and synergy between the regional education programs? • What factors (administrative, technical, etc.) improved coordination and synergy between the regional education programs? • (other relevant questions proposed by the offeror) b. What are the broader impacts of the regional education program? Evaluation of LAC/RSD Regional Education Program: Volume II 6 International Business & Technical Consultants, Inc. • What are the key contributions of the regional education program to improving education quality in the region as well as in individual countries? Attitudes, perceptions, and policy • Considering different education stakeholder groups, including government, public, business sector, etc., as appropriate, to what extent have the regional programs changed attitudes, as well as the visibility of technical education issues, such as early grade literacy, public-private partnership, etc.? Offeror to provide concrete examples. • To what extent have the regional programs influenced educational policies? Offeror to provide concrete examples. • To what extent have ministries of education actively incorporated components of the regional programs into national educational policy or interventions? • What factors facilitated/inhibited ministries’ adoption or adaptation of components of the regional programs? Capacity and sustainability • In what ways have CETT and PREAL strengthened capacity in partner institutions? • In what ways have CETT and PREAL strengthened capacity at various levels within ministries of education? • To what extent are partner institutions capable of and interested in carrying on the technical components of CETT and PREAL? • To what extent do ministries of education feel capable of carrying on technical components of CETT and PREAL? Provide examples of specific components to which ministries have committed. • Have the CETT and PREAL regional programs sufficiently built capacity such that technical interventions can be adequately sustained and/or replicated or scaled up? • What factors of the regional programs have contributed to replication and sustainability? In addition to other factors, consider the roles, contributions, and limiting factors of public private partnerships. • To what extent has there been continuation/attrition of personnel trained under CETT and PREAL programs in ministries and partner institutions? (brain drain factor) • (other relevant questions proposed by the offeror) c. What is the value-added of providing technical assistance to implementing partners? • How has technical assistance to implementing partners in the region added value to the CETT and PREAL programs? Consider technical assistance provided through USAID/Washington-based contracts as well as through the cooperative agreements. • What types of technical assistance provided to implementing partners in the region were most useful? • (other relevant questions proposed by the offeror) 3.2. Methodology Evaluation of LAC/RSD Regional Education Program: Volume II 7 International Business & Technical Consultants, Inc. The evaluation team will be expected to propose an overall evaluation approach that includes specifying innovative techniques for data collection and analysis. The use of comparison groups, where feasible and appropriate is encouraged, especially for the purposes of comparing sub-regional experiences as well as the experience of countries with different funding streams as in the case of Grenada and Antigua. Data collection methods may include the following: • Document review • Key informant interviews • Structured survey interviews • Focus group discussions • (other relevant methods proposed by the offeror) A number of data sources should be used including the following: • Key documents such as program annual reports and evaluation reports. • Interviews or focus group discussions with USAID Education Team members in relevant missions and the LAC Bureau; ministry of education officials; private sector partners; civil society partners; implementing partners; beneficiaries; donor partners; and others as relevant in person, by phone, and/or using email questionnaires. • (other relevant sources proposed by the evaluation team and approved by USAID) 3.3. Key Recommendations Based on the principal findings, the evaluation team will propose key recommendations for designing future assistance programs for education in the LAC region. The following questions must be considered in addition to other relevant questions the offeror may propose. • What types of interventions are most appropriate to be implemented at a regional level? • Are there ways to better link regional education programs? • Are there ways to better link regional education programs with bilateral education programs? • What lessons have been learned about working across countries regionally? • What lessons have been learned about how to engage successfully with the private sector in education? • How can lessons generated from the regional programs contribute to national policies on education? • (other relevant questions proposed by the offeror) 3.4. Target Countries Eight countries have been identified as representative of a cross-section of implemented activities and issues of concern and will be the focus of the evaluation. It is expected that Evaluation of LAC/RSD Regional Education Program: Volume II 8 International Business & Technical Consultants, Inc. evaluation teams will visit these eight countries. It is anticipated that key persons in other countries in the region will be contacted through telephone or electronic means. Jamaica – The CCETT lead institution is based in Jamaica, and USAID/Jamaica held management responsibility for the CCETT cooperative agreement. PREAL has recently begun national education report card activities in Jamaica, and evaluators can also ascertain the impact in Jamaica of PREAL publications and participation in policy dialogue. USAID/Jamaica implements a bilateral education program. Eastern Caribbean – Three countries in the Eastern Caribbean represent different stages and approaches to implementation of CCETT. While PREAL does not have report card or other direct activities in the countries, the evaluation team also should consider the impact of PREAL publications, research, and participation in policy dialogue. Additionally, coordination with the Caribbean Regional USAID office is a concern. USAID has no bilateral education activities in the Eastern Caribbean. • St Lucia (Eastern Caribbean) – CCETT implemented in country since 2002. • Grenada (Eastern Caribbean) – Self-financed implementation of CCETT in country beginning in 2005. • Antigua (Eastern Caribbean) – In 2009, the Government of Antigua signed a Memorandum of Understanding to begin self-financed implementation of the CCETT program with technical support from the Joint Boards of Teacher Education during 2009, providing another angle on the impact of CETT in country. Dominican Republic (DR) – CARD CETT is implemented in the DR. PREAL has extensive activities in the DR, including being in the process of completing a second country report card for the DR (the first was published in 2006), and having established a national partnership and an active business alliance. USAID/DR implements a bilateral education program. CERCA implemented school report cards and other activities. Honduras – The CARD CETT lead institution is based in Honduras, and USAID/Honduras held the management responsibility for the CARD CETT cooperative agreement. PREAL has worked extensively in Honduras, including supporting publication of two national report cards (2002, 2005), and the establishment of a national partnership and business alliance. USAID/Honduras implements a bilateral education program. CERCA implemented school report cards and other activities. Peru – The Andean CETT lead institution is based in Peru, and USAID/Peru held the management responsibility for the Andean CETT cooperative agreement. PREAL has produced two national report cards in Peru (2003, 2006) and has established a national partnership. USAID/Peru implements a bilateral education program. Ecuador – The Andean CETT is implemented in Ecuador. PREAL supported production of a national report card in Ecuador (2006) and established a national partnership. USAID/Ecuador does not have a bilateral education program. 4. Deliverables Evaluation of LAC/RSD Regional Education Program: Volume II 9 International Business & Technical Consultants, Inc. The evaluation team will deliver the following products to the LAC/RSD Education Team: 4.1. Work Plan: The work plan should describe in detail the organization of the evaluation, methodology to be used, data analysis procedures, schedule of evaluation activities, and schedule of key interviews and country travel, if necessary. The draft work plan should be submitted to USAID prior to the pre-trip planning meeting. The final work plan must be submitted for approval before data collection begins. 4.2. Data Collection Tools: Prior to data collection, the evaluation team must submit copies of instruments or tools to be used to the LAC/RSD Education Team for review, input, and approval. 4.3. Pre-Trip Planning Meeting (one-day): The evaluation team (U.S. members) will meet with the LAC/RSD Education Team prior to travel to present and discuss plans and data collection tools and clarify expectations. 4.4. Draft Report: Within one week of return from field, the evaluation team should submit a draft report to the LAC/RSD Education Team. The draft report will follow the format below: ƒ Table of Contents ƒ Executive Summary ƒ Introduction ƒ Background ƒ Evaluation Questions ƒ Methodology ƒ Principal Findings ƒ Recommendations and Future Program Considerations ƒ Annexes (including scope of work, evaluation team composition, final work plan, data collection tools, data, list of interviewees, and other relevant information) 4.5. Debrief: Following submission of the draft report, the lead evaluator(s) must present and debrief the LAC/RSD Education Team on the key findings and recommendations. The evaluation team must allow for discussion and must coordinate the schedule for the debriefing with the LAC/RSD Education Team. The evaluation team may provide the debriefing through a teleconference, as appropriate to reduce expenses. The debriefing must be provided no later than October 8, 2010 (depending on award). 4.6. Final Report: The evaluation team will submit a final report of approximately 50 pages in length in English incorporating USAID’s comments and suggestions, within one week after USAID submits written Evaluation of LAC/RSD Regional Education Program: Volume II 10 International Business & Technical Consultants, Inc. comments on the draft report. Two printed and one electronic version of the complete report must be submitted to USAID. 5. Performance Period The evaluation will take place over a period of eight to ten weeks. The evaluation team will begin to carry out activities immediately upon award. The proposal should include a draft work plan in the annex outlining estimated schedule and deliverable due dates. 6. Key Personnel and Proposed Level of Effort The offeror will assemble a team of at least four education evaluation experts with demonstrated management skills and appropriate technical expertise, experience and educational training in evaluation design and methods. The team will comprise two sub-teams each to also include a local expert in addition to the U.S. experts, for a total of six team members. References must be provided. The offeror should propose a team with an appropriate mix of the following skills: education evaluation, education policy reform, teacher training, capacity development of partner institutions, and private sector engagement with education. The evaluation team leader is responsible for the final product. USAID expects that measures will be taken to ensure consistency in data collection; for example, by the full team traveling to the Dominican Republic to ensure consistency of data collection, followed by one team traveling to Honduras and the Andean countries and one to the English-speaking Caribbean. An illustrative evaluation schedule appears below. Six-day weeks are authorized while in-country. Illustrative Evaluation Schedule Week 1: Review reports and other preparation Week 2: Dominican Republic – full team Week 3: Honduras (team one) / Jamaica (team two) Week 4&5: Peru, Ecuador (team one) / Eastern Caribbean (team two) Week 6: Follow up and report preparation Draft report & debrief due to USAID Week 7/8: USAID review draft report and provide feedback Week 9: Finalize report Final report due to USAID Team Evaluation Specialist Team Leader Co-Team Leader/Education/Evaluation Specialist Evaluation of LAC/RSD Regional Education Program: Volume II 11 International Business & Technical Consultants, Inc. Two Education Evaluation/Education Specialists Two in-country research assistants Logistics Assistant/Interpreter Evaluation of LAC/RSD Regional Education Program: Volume II 12 International Business & Technical Consultants, Inc. Annex II: Team Members Dr. Frank Dall: Evaluation Specialist Team Leader Dr. Dall was responsible for the management and coordination of the overall evaluation process and its deliverables as outlined in section four of the RFTOP, including overall development and compilation of the draft and final work plans, and draft and final evaluation reports, as well as coordination of team member responsibilities and workflow. He was the principal interlocutor between the team and USAID, the lead author on the work plan and report, and the lead presenter at all briefings to USAID. He also led sub-team one to Honduras and Ecuador. Dr. Jeffrey Tines - Co-Team Leader/Education/Evaluation Specialist Dr. Tines backstopped the team leader in all management and coordination roles, and in submitting deliverables. He co-authored the work plan and evaluation report, and served as co-presenter at all briefings to USAID. He also led sub-team two to Jamaica and the Eastern Caribbean. Ana Cristina Accioly de Amorim Education Evaluation/Education Specialists Ms. Accioly provided key education, evaluation and regional expertise, including education evaluation, education policy reform, teacher training, capacity development of partner institutions, and private sector engagement with education. She wrote and edited key pieces of the work plan and evaluation report. Dr. Tom Tilson - Education Evaluation/Education Specialist Dr. Tilson provided key education, evaluation and regional expertise, including education evaluation, education policy reform, teacher training, capacity development of partner institutions, and private sector engagement with education. He wrote and edited key pieces of the work plan and evaluation report. Dr. Sheere Brooks In-country Research Specialist Dr. Brooks provided key local support in the Caribbean evaluation focus regions, and key analysis regarding the education and political context in these regions. They conducted fieldwork preparation (provide names/contact information of key informants and subject matter experts for interviews; suggests/arranges site visits), contributed to drafting of work plan and draft/final report, and helped deliver presentations as needed. Lic. Luis Medrano - In-country Research Specialist Dr. Medrano provided key local support in the Andean evaluation focus regions, and key analysis regarding the education and political context in these regions. He conducted fieldwork preparation (provide names/contact information of key informants and subject matter experts for interviews; suggests/arranges site visits), contributed to drafting of work plan and draft/final report, and helped deliver presentations as needed. Patricio Crespo – Project Director (home office-based) Mr. Crespo provided key M&E, writing/editing, workflow, and coordination support to Team from Washington, DC. Evaluation of LAC/RSD Regional Education Program: Volume II 13 International Business & Technical Consultants, Inc. Annex III. Final Work Plan A. Pre- trip and Field Work 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 S S M T W T F S S M T WTFSSM TWT F S S M T W T F S S Team (Home) Desk Research (10/18 - 11/19) Work Plan (10/18 - 11/19) Data Collection Tools Pre-trip Planning Meeting (10/18 - 11/19) Team 1 (Field) Travel to the Dominican Rep (11/20) Field w ork in Dominican Rep.(11/22-11/26) Travel to Honduras (11-27) Field w ork in Honduras (11/29 - 12/3) Travel to Ecuador (12/4) Field w ork in Ecuador (12/6 - 12/10) Travel to Peru (12/11) Field w ork in Peru (12/13 - 12/17) Travel to Home Base (12/18) Team 2 (Field) Travel to the Dominican Rep (11/20) Field w ork in Dominican Rep.(11/22-11/26) Travel to Jamaica (11/27) Field w ork in Jamaica (11/29 - 12/3) Travel to Grenada (12/4) Field w ork in Grenada (12/6 - 12/8) Travel to St Lucia (12/9) Field w ork in St. Lucia (12/10 - 12/14) Travel to Antigua (12/15) Field w ork in Antigua (12/16 - 12/17) Travel to Home Base (12/18) Team (Home) Online survey administration (11/20-12/18) TL completes D.C. Interview s (12/15-12/17) Note: Team Leader departed Peru for the U.S. Nov. 9 OCT. - NOV. NOVEMBER DECEMBER 10 / 18 - 11/ 19 Evaluation of LAC/RSD Regional Education Program: Volume II 14 International Business & Technical Consultants, Inc. B. Detailed Description (day-by-day) for the Field Work DOMINICAN REPUBLIC Date Place Activities 10/18/2010 to 11/19/2010 U.S.A. and Home offices Desk review; Reading background documents Meeting and Logistics planning for field work; Preparation of work plan (schedule and action plan) and data collection tools (evaluation methodology and instruments); Pre-trip planning meetings with USAID LAC/RSD; and Country clearances. Saturday 11/20/2010 Travel Team members travel from their respective home bases to the Dominican Republic. Monday 11/22/2010 Dominican Republic IBTCI submits final work plan and evaluation methodology. IBTCI Team reviews progress of desk research, discusses work plan, vets data collection tools, and assigns writing responsibilities for the end of evaluation debrief and final report (in accordance with the final report outline provided in the RFTOP SOW). Team discusses uniform protocol for administering the data collection tools (key Informant Interviews, Focus Groups, and On-Line Survey) and analyzing the data. Team 1 (T1) and Team 2 (T2) field test interview protocols in Santo Domingo and work separately to finalize their schedules and meetings for DR, and their respective countries. Participant lists are categorized by KIIs, Focus Groups and On-Line Survey Instruments. Local Admin assists with logistics of the On-line Survey to be distributed and Focus Group meetings planned for Thursday. Tuesday 11/23/2010 Dominican Republic T1 and T2 continue field tests and revise protocols based upon feedback from field tests. IBTCI submits final data collection tools (Six KIIs, Two Focus Group Questionnaires, and one On-line Survey) Evaluation of LAC/RSD Regional Education Program: Volume II 15 International Business & Technical Consultants, Inc. DOMINICAN REPUBLIC Date Place Activities Wednesday 11/24/2010 Dominican Republic KII interviews will be held with USAID/DR mission staff in the morning. Wednesday PM KII interviews with key stakeholders from CETT, PREAL and CERCA projects in Santo Domingo. Focus Group with public sector key stakeholders conducted in Santo Domingo. T1 and T2 Team Leaders coordinate with IBTCI home office to finalize next country meetings (Jamaica and Honduras) based on USAID guidance. (IBTCI Home Office staff is already in touch with USAID Jamaica and Honduras). Note, at least one key person (Dall, Tines, Accioly, or Tilson) will attend all KIIs conducted. The Research Assistants (Brooks and Hermoso) will also be provided the opportunity to be part of the two person IBTCI team conducting KIIs to ensure they have a sufficient grasp of the methodology which will be applied to carry out subsequent research responsibilities during country visits in Latin America (Team 1) and the Caribbean (Team 2). Thursday 11/25/2010 Dominican Republic Thursday AM Private Sector Focus Group will be held in Santiago. KII interviews will be held with key stakeholders in Santiago on Thursday AM and Thursday PM. One member each from the TI and T2 team e-mails the online-survey to relevant DR contacts (and periodically follows up to ensure greater participation for this instrument.) Friday 11/26/2010 Dominican Republic Friday AM KII interviews will be held with key private and public sector stakeholders. Friday PM Team will conduct a de-briefing with USAID. Friday PM the necessary time will be set aside to complete key stakeholder interviews. Time will be devoted working to bring the team on board concerning data analysis and presentation techniques. Teams will begin work in the analysis of data collected in the DR during the week and to process and store this data for eventual inclusion in the final draft report. Relevant findings will be organized in line with the final report outline and stored on the IBTCI Google Portal. Saturday 11/27/2010 Travel from the Dominican Republic Each team will travel to their respective next assignments - Honduras for T1, and Jamaica for T2. Upon arrival teams will meet to plan and coordinate the next week’s activities. Evaluation of LAC/RSD Regional Education Program: Volume II 16 International Business & Technical Consultants, Inc. HONDURAS Date Place Activities Saturday 11/27/2010 Travel Team Members travel from Dominican Republic to Tegucigalpa Honduras. Monday 11/29/2010 Honduras Team will meet with the USAID Mission in Honduras at 9.30 am to discuss work schedule and to interview key respondents. Team discusses uniform protocol for administering the data collection tools (key Informant Interviews, Focus Groups, and On-Line Survey) and analyzing the data. By telephone, or email, Team 1 finalizes its schedule and meetings for Honduras. Participant lists are categorized by KIIs, Focus Groups and On-Line Survey Instruments. Local Admin assists with selecting venues and the logistics of the Focus Group meetings planned for Thursday. Tuesday 11/30/2010 Honduras Two members of the Team in Tegucigalpa begin KII interviews with the responsible for managing and coordinating CARD-CETT at the Pedagogical University of Honduras, The local admin. asst. tracks down and finds senior PREAL and CERCA stakeholders and confirms by time, date and place relevant KII meetings. The third member of the team e-mails the online-survey to relevant Hondurans contacts (and periodically follows up to ensure greater participation for this instrument.) Local Admin assists with the logistics of transportation for the Team interviews with KIIs. Evaluation of LAC/RSD Regional Education Program: Volume II 17 International Business & Technical Consultants, Inc. HONDURAS Date Place Activities Wednesday 12/01/2010 Honduras. Wednesday AM, KIIs continue with the local interviewees responsible for managing and coordinating, PREAL and CERCA. Wednesday PM, designated team members conduct KIIs at USAID with the Mission’s resident education team. The Team Leader coordinate with IBTCI home office to finalize next country meetings (Ecuador) based on USAID’s recommendations and guidance. (IBTCI Home Office staff is already in touch with USAID Honduras). Note, at least one key person (Dall or Accioly) will attend all KIIs conducted. The Research Assistant (Hermoso) will also be part of the two person IBTCI team conducting KIIs to ensure he has sufficient grasp of the methodology which will be applied to carry out subsequent research responsibilities during country visits in Ecuador and Peru. Thursday 12/02/2010 Honduras Separate Focus Groups for Public and Private Sector will be held in Tegucigalpa. Depending on potential participants’ response. The IBTCI Team with the help of the admin. asst., will select an appropriate site for the Private sector Focus Group meeting. A hotel conference room may act as a back-up site. Friday 12/03/2010 Honduras Half day will be reserved for follow up of USAID KIIs and any remaining KIIs. The rest of the day will be devoted to working on the analysis of data collected in Honduras during the week, and on processing and storing this data for eventual inclusion in the final draft report. Relevant findings will be organized in line with the final report outline and stored on the IBTCI Google Portal. Evaluation of LAC/RSD Regional Education Program: Volume II 18 International Business & Technical Consultants, Inc. ECUADOR Date Place Activities Saturday 12/04/2010 Travel Team Members travel from Honduras to Quito, Ecuador .Upon arrival the team will meet to coordinate the next week’s activities. Monday 12/06/2010 Ecuador The Team will meet with the USAID Mission education team in Quito to coordinate work schedule and to carry out key informant interviews. Team discusses uniform protocol for administering the data collection tools (key Informant Interviews, Focus Groups, and On-Line Survey) and analyzing the data. Team finalizes its schedule and meetings for Ecuador. Participant lists are categorized by KIIs, Focus Groups and On-Line Survey Instruments. Local admin asst helps with the organization and logistics of the Focus Group meetings planned for Thursday. Possible venues and a list of provisional FG participants will be completed. Tuesday 12/07/2010 Ecuador Two members of the Team in Quito begin KII interviews with the responsible for managing and coordinating Andean CETT at the Maria Maestra Bolivar University, while Dr. Hermoso e-mails the online-survey to relevant Ecuadorean contacts (and periodically follows up to ensure greater participation for this instrument.) The local admin asst will help with logistical and transportation needs for the Team members carrying out KIIS interviews. Evaluation of LAC/RSD Regional Education Program: Volume II 19 International Business & Technical Consultants, Inc. ECUADOR Date Place Activities Wednesday 12/08/2010 Ecuador Wednesday AM, KIIs continue with the remaining local stakeholders responsible for managing and coordinating the Andean CETT and PREAL programs. Wednesday PM, designated team members conduct KIIs at the USAID Mission. The Team Leader coordinate with IBTCI home office to finalize next country meetings in Peru based on USAID recommendations and guidance (IBTCI Home Office staff is already in touch with USAID Peru). Note, at least one key person (Dall or Accioly) will attend all KIIs carried out in Ecuador. The Research Assistant (Hermoso) will make up part of the two person IBTCI team conducting KIIs Thursday 12/09/2010 Ecuador Separate Focus Groups for the Public and Private Sector will be held in Quito. Subject to potential participant response, the IBTCI Team will select an appropriate site for the Focus Group. Friday 12/10/2010 Ecuador Half day will be reserved to follow up on any remaining USAID or any other KIIs. In line with previous coordination with the USAID LAC Bureau and the COTR in Washington, DC, a mid evaluation review will be held in via teleconference, in the hotel, and will include members of the both Teams. T2 will be patched into this review meeting, from their hotel venue in St. Lucia. The remainder of the day will be devoted to beginning work on the analysis of data collected in Ecuador during the week and to processing and storing this data for eventual inclusion in the final draft report. Relevant findings will be organized in line with the final report outline and stored on the IBTCI Google Portal. Saturday 12/11/2010 Travel from Ecuador The Team then travels to Lima, Peru. Upon arrival team members will meet to plan and coordinate the subsequent week’s activities. Evaluation of LAC/RSD Regional Education Program: Volume II 20 International Business & Technical Consultants, Inc. PERU Date Place Activities Saturday 12/11/2010 Travel Team Members travel from Quito, Ecuador to Lima, Peru Monday 12/13/2010 Peru The team will meet the USAID Mission’s education team in Lima to discuss the schedule and to key informant interviews with the relevant USAID program staff in Lima. By telephoning and emailing, the Team will finalize its schedule of meetings for the Peruvian segment of the evaluation. Participant lists will be categorized by KIIs, Focus Groups and On-Line Survey Instruments. A local member of the Team will help with the logistics and organization of the two Focus Group meetings planned for Thursday. Tuesday 12/14/2010 Peru Two members of the Team in Lima begin KII interviews with those responsible for managing and coordinating the Andean CETT HQs at the Cayetano Heredia University, in Lima. The third member of the team, will e-mail the online-survey to relevant Peruvian contacts (and periodically follows up to ensure greater participation for this instrument.) With local help, logistical and transportation will be provided to assist the Team to carry out KIIs Meetings in a timely and efficient way. Wednesday 12/15/2010 Peru Wednesday AM, KIIs continue with the local interviewees responsible for managing and coordinating, PREAL. Wednesday PM, designated team members conduct KIIs at the USAID Mission, in Lima. The Team Leader coordinates with IBTCI home office to finalize the field trip, and wrap – up field activities in Peru. Note, at least one key person (Hermoso, or Accioly) will attend all KIIs conducted. The Research Assistant will also be part of the two person IBTCI team conducting KIIs. Evaluation of LAC/RSD Regional Education Program: Volume II 21 International Business & Technical Consultants, Inc. PERU Date Place Activities Thursday 12/16/2010 Peru Separate l Focus Group meetings for the Public and Private Sectors will be held in Lima, at suitable venues. Friday 12/17/2010 Peru A half day will be set aside to follow up of USAID KIIs, or to carry out remaining KIIs. The rest of the day will be devoted to beginning work on the analysis of data collected in Honduras during the week and to processing and storing data for eventual inclusion in the final draft report. Findings will be organized appropriately in line with the final report outline, and stored on the IBTCI Google Portal. Saturday 12/18/2010 Travel from Peru The Team travels to Home Base in the USA. Upon arrival teams will meet to plan and coordinate the final weeks’ activities. Evaluation of LAC/RSD Regional Education Program: Volume II 22 International Business & Technical Consultants, Inc. JAMAICA Date Place Activities Saturday 11/27/2010 Travel Team Members travel from Dominican Republic to Kingston, Jamaica. Monday 11/29/2010 Jamaica. Monday AM IBTCI Team conducts KII interview with USAID staff. Monday PM conducts KII interviews with selected public and private sector key informants. Monday PM the Team works to finalize their schedules and meetings for Kingston. Participant lists are categorized by KIIs, Focus Groups and On-Line Survey Instruments. Logistics of the Interviews and Focus Group meetings for the week Wednesday and Thursday will be arranged. Tuesday 11/30/2010 Jamaica Tuesday AM and PM IBTCI Team members continue with KII interviews in Kingston with those responsible for managing and coordinating CCETT. The third member of the team e-mails the online-survey to relevant Jamaican contacts (and periodically follows up to ensure greater participation for this instrument.) Wednesday 12/01/2010 Jamaica. Wednesday AM, IBTCI Team will conduct public sector FG in Kingston with key public sector stakeholders responsible for managing and coordinating CCETT. Wednesday PM, designated team members will conduct additional KIIs with key public and private stakeholders. The Team Leader coordinate with IBTCI home office and key informants in Jamaica to finalize next country meetings (Grenada, St. Lucia and Antigua) based on USAID guidance. Evaluation of LAC/RSD Regional Education Program: Volume II 23 International Business & Technical Consultants, Inc. JAMAICA Date Place Activities Thursday 12/02/2010 Jamaica Thursday AM IBTCI Team will conduct private sector Focus Group. Thursday PM IBTCI Team will conduct additional KII interviews with key public and/or private sector informants. Thursday PM on-line survey form will be forwarded to specific key informants. Friday 12/03/2010 Jamaica Friday AM IBTCI Team will provide USAID/Jamaica with a de-briefing of interviews, focus groups and on-line survey findings in Jamaica. Friday PM IBTCI Team will conduct additional KII interviews in Kingston. Friday PM IBTCI Team will devote time to the analysis of data collected in Kingston during the week and to process and store this data for eventual inclusion in the final draft report. Relevant findings will be organized in line with the final report outline and stored on the IBTCI Google Portal. Saturday 12/04/2010 Travel from Jamaica Team travels to Grenada. Upon arrival teams will meet to plan and coordinate the next week’s activities. Evaluation of LAC/RSD Regional Education Program: Volume II 24 International Business & Technical Consultants, Inc. GRENADA Date Place Activities Saturday 12/04/2010 Travel Team Members travel from Jamaica to Grenada. Monday 12/06/2010 Grenada Monday AM visits the local CETT coordinating entity and the GoG section for overseeing CETT and PREAL activities. Monday AM the Team works to finalize their schedules and meetings for Grenada Participant lists are categorized by KIIs, Focus Groups and On-Line Survey Instruments. Monday PM IBTCI Team conducts KII interviews with selected public and private sector key informants. Logistics for the public and private sector Focus Groups and KII are arranged. Tuesday 12/07/2010 Grenada Tuesday AM and PM IBTCI Team members continue with KII interviews in with those responsible for managing and coordinating CCETT The third member of the team e-mails the online-survey to relevant Grenadian contacts (and periodically follows up to ensure greater participation for this instrument.) Logistics of transportation for the Team interviews with KIIs. Wednesday 12/08/2010 Grenada Wednesday AM, IBTCI Team will conduct public sector FG with key public sector stakeholders. Wednesday PM, IBTCI Team will conduct private sector FG with key private sector stakeholders. The Team Leader coordinate with IBTCI home office and key informants to finalize next country meetings (St. Lucia and Antigua) based on USAID guidance. Interviews will continue with KIIs at the local offices of the CETT coordinating agency. Wednesday PM IBTCI Team will devote time to the analysis of data collected in Grenada during the week and to process and store this data for eventual inclusion in the final draft report. Relevant findings will be organized in line with the final report outline and stored on the IBTCI Google Portal. Evaluation of LAC/RSD Regional Education Program: Volume II 25 International Business & Technical Consultants, Inc. GRENADA Date Place Activities Thursday 12/09/2010 Travel from Grenada Team Travels to St. Lucia. Upon arrival teams will meet to plan and coordinate the next week’s activities. Evaluation of LAC/RSD Regional Education Program: Volume II 26 International Business & Technical Consultants, Inc. ST. LUCIA Date Place Activities Thursday 12/09/2010 Travel Team Members travel from Grenada to St. Lucia. Friday 12/10/2010 St. Lucia Friday AM visits the local CETT coordinating entity and the GoSt. Lucia section for overseeing CETT and PREAL activities. Friday AM the Team works to finalize their schedules and meetings for St. Lucia. Participant lists are categorized by KIIs, Focus Groups and On-Line Survey Instruments. Friday PM IBTCI Team conducts KII interviews with selected public and private sector key informants. Time will be set aside for the Team to participate in a review meeting linking T1, T2 IBTCI and USAID Washington by teleconference. Hotel conferencing facilities will be used for this purpose. Saturday 12/11/2010 St. Lucia Team finalizes its schedule and meetings for St. Lucia. Participant lists are categorized by KIIs, Focus Groups and On-Line Survey Instruments. IBTCI Team reviews progress of desk research, discusses work plan, vets data collection tools, and assigns writing responsibilities for the end of evaluation debrief and final report. Monday 12/13/2010 St. Lucia. Monday AM and PM IBTCI Team members continue with KII interviews in with those responsible for managing and coordinating CCETT. The third member of the team e-mails the online-survey to relevant St. Lucia contacts (and periodically follows up to ensure greater participation for this instrument.) Logistics of transportation for the Team interviews with KIIs. Evaluation of LAC/RSD Regional Education Program: Volume II 27 International Business & Technical Consultants, Inc. ST. LUCIA Date Place Activities Tuesday 12/14/2010 St. Lucia Tuesday AM, IBTCI Team will conduct public sector FG with key public sector stakeholders responsible for managing and coordinating CCETT. Tuesday PM, IBTCI Team will conduct private sector FG with key private sector stakeholders. The Team Leader coordinate with IBTCI home office and key informants to finalize next country meetings in Antigua. Interviews will continue with KIIs at the local offices of the CETT coordinating agency. Tuesday PM IBTCI Team will devote time to the analysis of data collected in St. Lucia during the week and to process and store this data for eventual inclusion in the final draft report. Relevant findings will be organized in line with the final report outline and stored on the IBTCI Google Portal. Wednesday 12/15/2010 Travel from St. Lucia Team travels to Antigua. Upon arrival teams will meet to plan and coordinate the next week’s activities. Evaluation of LAC/RSD Regional Education Program: Volume II 28 International Business & Technical Consultants, Inc. ANTIGUA Date Place Activities Wednesday 12/15/2010 Travel Team Members travel from St. Lucia to Antigua. Thursday 12/16/2010 Antigua Thursday AM visits the local CETT coordinating entity and the GoA section for overseeing CETT activities. Thursday AM the Team works to finalize their schedules and meetings for Antigua Participant lists are categorized by KIIs, Focus Groups and On-Line Survey Instruments. Thursday PM IBTCI Team conducts KII interviews with selected public and private sector key informants Logistics of transportation for the Team FGs and interviews with KIIs. Friday 12/17/2010 Antigua Friday AM, IBTCI Team will conduct public sector FG with key public sector stakeholders responsible for managing and coordinating CCETT. Friday PM, IBTCI Team will conduct private sector FG with key private sector stakeholders. Friday PM IBTCI Team will devote time to the analysis of data collected in Antigua during the week and to process and store this data for eventual inclusion in the final draft report. Relevant findings will be organized in line with the final report outline and stored on the IBTCI Google Portal. Team wraps up its field activities by holding a final review meeting to see if each team‘s member’s written contributions completed and ready for submission to the overall Team Leader at IBTCI. Saturday 12/18/2010 Travel from Antigua. Team travels to Home base Evaluation of LAC/RSD Regional Education Program: Volume II 29 International Business & Technical Consultants, Inc. C. Post-field Work Plan with First Project Extension 15 16 17 27-30 31 3-7 10 11 12 13 14 17 18 19 20 21 24 2 5 26 27 2 8 3 1 1 2 3 4 7 8 9 10 11 14 15 W T F M-T F M-F M T W T F M T W T F M T W T F M T W T F M T W T F M T Review Draft and Decide on Final Outline Send Final Outline to USAID COTR Interviews by Phone/In Person - Local and Long Distance Team members complete their written assignments Review material written by team and merge into draft report Analysis of on-line survey results and merge into draft report Complete writing the draft final report Deliver Draft Final Report to USAID Prepare the debrief Deliver the debrief Comments on Draft and debrief are provided by USAID Review comments from USAID Prepare the Second Draft Deliver Second Draft to USAID USAID reviews the draft and prepares comments USAIDdelivers comments Team prepares the final report Team Delivers the Final Report DECEMBER JANUARY FEBRUARY Evaluation of LAC/RSD Regional Education Program: Volume II 30 International Business & Technical Consultants, Inc. D. Post-field Work Plan with Second Project Extension 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 0 2 1 2 2 2 3 2 4 2 5 2 6 2 7 2 8 2 9 3 0 3 1 1 2 3 4 5 6 7 8 9 10 11 # 13 14 15 16 17 18 19 2 0 2 1 S SMTWT F S SMTWT F S SMTWT F S SMTWT F S SMTWT F S SMTWT F S SMT WT F S S M 1. Review Draft and Decide on Final Outline 2. Send Final Outline to USAID COTR 3. Interview s by Phone/In Person - Local and Long Distance 4. Team members complete their w ritten assignments 5. Review material w ritten by team and merge into draft report 6. Analysis of on-line survey results and merge into draft report 7. Complete w riting the draft final report 8. Deliver Draft Final Report to USAID 9. Prepare the debrief 10. Deliver the debrief 11. Comments on Draft and debrief are provided by USAID 12. Review comments from USAID 13. Prepare the Second Draft 14. Deliver Second Draft to USAID 15. USAID review s the draft and prepares comments 16. USAID delivers comments 17. Team prepares the final report 18. Team Delivers the Final Report TASKS & FEBRUARY DELIVERABLES JANUARY Evaluation of LAC/RSD Regional Education Program: Volume II 31 International Business & Technical Consultants, Inc. Annex IV: Data Collection Instruments INSTRUMENT FOR USAID EDUCATION OFFICIAL KEY INFORMANT INTERVIEWS COUNTRY _____________________ VENUE __________________________ DATE (day/month/year) ___________________ PERSON INTERVIEWED: ______________________________________________________ POSITION: __________________________________________________________________ Interviewer/s’ name/s __________________________________________________________ Time started: ____________Time ended: ___________ Total time: ________________ BACKGROUND INFORMATION: USAID has contracted IBTCI to assess three education programs in Latin America and the Caribbean (CERCA, CETT, AND PREAL) that have been implemented in several countries. Of special interest are the benefits and challenges of working with a USAID/Washington-based program, both in terms of project impact and project management in the participating countries. USAID is particularly interested in what has worked well and not so well. In addition, they would like guidelines for possible future regional programs. It should be noted that these three regional programs were developed and have been supervised by USAID/Washington, DC. We greatly appreciate your participation in the interview. Your responses will be kept in total confidence. Notes for the Interviewer: We need information about each specific program, e.g., CERCA, CETT and PREAL. In the first question you will find out the programs that are familiar to the Interviewee. If the person knows about more than one of the programs, then be sure to ask about EACH program in the following questions as appropriate. An important part of the dynamics of the Washington-based programs relates to interactions and complementary with the missions own local programs in education. Therefore, we ask in question 2 for information on the mission’s education programs. I. The Broader Benefits and Challenges of a USAID/Washington-based Program Please tell me which program/s you have worked with: CETT, PREAL and/or CERCA? Program Role or involvement in the program CERCA CETT PREAL Because of possible interactions between the Washington-based education programs and the mission’s own education programs, could you please summarize the mission’s own education programs and your involvement? Program Description Role or involvement in the program If the respondent is familiar with more than one of the regional programs, go to question 3. Otherwise, proceed to question 5. Were there any synergies among CETT, CERCA and PREAL? a. If yes: what were they? b. If no: why not? How would you assess the coordination among these three regional programs? In your opinion, were there any benefits of working with the Washington-based education programs? a. If yes: what benefits? b. If no: why not? Did you encounter challenges in working with the Washington-based education programs? If yes: could those challenges have been reduced, and how? What were the interactions between the Washington-based programs and your own mission programs in education? Evaluation of LAC/RSD Regional Education Program: Volume II 32 International Business & Technical Consultants, Inc. (Possible follow-up probing questions: Did they complement each other in terms of project focus or partners who were involved? Did they conflict with each other? Were there any opportunities or efforts to harmonize these programs? Were there any benefits or difficulties for the mission in having both Washington-based and local education programs in your country?) In your opinion, did these programs support the USAID country bilateral education program(s)? a. If yes: to what extent? b. If no: why not? What is your assessment of the effectiveness of the Washington-based programs in addressing educational issues? That is, to what extent did the Washington-based programs prove to be effective in terms of addressing key educational issues in your country? What is your assessment of the efficiency of the Washington-based programs in addressing educational issues? That is, to what extent were the Washington-based programs efficient in terms of working in and attaining the project goals in your country? II. Program Coordination and Synergy Overall, to what extent was there coordination between Washington and the mission regarding the implementation of the regional program(s) in this country? What were the factors (administrative, management, technical) that tended to: Facilitate coordination between the mission and Washington? Limit coordination between the mission and Washington Was the mission involved in coordination of the regional programs in this country with the Ministry of Education, other partners in the country, the contractor, or technical assistance? If so, please describe the role of the mission. III. Recommendations for the design of future regional assistance programs What are the major lessons learned from these three Washington-based programs? Would you recommend that USAID consider future Washington-based programs in education? If so, what types of programs would be most appropriate for your country? What factors need to be considered when designing a project that will ensure success of a regional program? End of Interview Express appreciation for participating in the interview. (Perhaps, if there is anything else that they would like to add as an afterthought, leave a phone number and email address for contacting the interviewer.) Evaluation of LAC/RSD Regional Education Program: Volume II 33 International Business & Technical Consultants, Inc. INSTRUMENT FOR GOVERNMENT (MOE) KEY INFORMANT INTERVIEWS Date: (day/month/year) ______________ COUNTRY________________________ VENUE_____________________________ Name of Interviewer: __________________________ Name of person being interviewed: ___________________________ Present position and organization: _________________________________________ Beginning time for interview ________ Ending time __________ Total time _________ BACKGROUND INFORMATION: USAID has contracted IBTCI to assess three education programs in Latin America and the Caribbean that have been implemented in several countries. Of special interest are the benefits and challenges of working with a regional program, both in terms of project impact and the management of the projects across the participating countries. USAID is particularly interested in what has worked well and not so well. In addition, they would like guidelines for possible future regional programs. It should be noted that these three regional programs were developed and have been supervised by the USAID offices in Washington, DC. We greatly appreciate your participation in the interview. Your responses will be kept in total confidence. On which USAID regional project has the interviewee been involved? Please include the position and dates of involvement: Please explain that a “regional” program is one that is developed and implemented by USAID/Washington and it involves the participation of multiple countries. Project Position – Role and responsibility Dates of involvement CERCA PREAL CETT In addition to regional programs, please ask if the person is familiar with any other education programs sponsored by the local USAID mission, so-called bilateral programs. Name of Project Position – Role and responsibility Dates of involvement If the person being interviewed is familiar with more than one USAID regional program, then be sure to clarify in each of the questions below, as appropriate, which of the programs is being discussed. Try to solicit information for each of the regional programs. I. Broad Benefits and Challenges of regional Program Models In which of the USAID regional programs has the MOE been involved? CERCA, PREAL, and CETT? What has been the role of the Ministry of Education in each/any of the regional programs? Who are/were the major partners in your country working with USAID regional program(s)? (List the of public and private partners and summarize the role of each) Name of partner Role of partner In your opinion, are there any benefits from working with a USAID regional program(s)? Yes or no; please elaborate. Have there been any challenges working with a regional program(s)? If yes, how were these been addressed? What might have been done better to overcome the challenges? Challenge How addressed What might have been done better Evaluation of LAC/RSD Regional Education Program: Volume II 34 International Business & Technical Consultants, Inc. How was the USAID regional program(s) in your country coordinated? (If necessary, probe with the following institutions: Role of a partner university within the country, role of a partner organization in another country, role of the MOE, role of USAID/Washington, role of local USAID mission, other) Can you give instances of where good and poor coordination took place: Among the institutions within your country working on the regional program(s); e.g., universities, MOE, USAID, other partners Between different regional programs within your country (if more than one project), e.g., between CETT and PREAL Between your country and the other participating countries on the regional program(s) Between the USAID regional program(s) in your country and the other education programs sponsored by the local USAID mission (if any) With USAID What suggestions do you have for improving the coordination among partners that we have just talked about? To what extent has the Ministry tried to incorporate the main activities of the regional program(s) into its own plans and activities? Which activities and how has this been done? In your opinion, has the USAID regional program(s) made any significant contributions to improving educational quality in your country? If yes: please tell me the contributions. If no, why not? Has the regional program(s) had any influence—positive, negative, or mixed--on national educational policies? Please elaborate. If the Ministry has incorporated any aspects of the regional programs, would it like to continue to support these programs? What programs or activities would the Ministry like to support? Would the Ministry like to expand or scale-up the programs? Please elaborate. If the Ministry has not incorporated or continued with activities of the regional program(s), please explain why? Does the Ministry have sufficient skills and resources to sustain and/or expand the program(s)? If not, what would they need? Has the regional program(s) strengthened the capacity of the Ministry in any ways or in any areas? If so, please describe how the Ministry’s capacity has been strengthened? Value-added of technical assistance Has the Ministry of Education received any form of technical assistance through the program? Elaborate on the type of TA and its contribution. Type of TA Contribution (little Æ much) Is there any technical assistance that the Ministry would have liked to have, but did not receive? How might this addition of TA strengthen the Ministry? Conclusions and recommendations Evaluation of LAC/RSD Regional Education Program: Volume II 35 International Business & Technical Consultants, Inc. If you have had experience with both USAID regional and USAID local mission programs, can you tell me which approach seems to be most useful and why? In thinking about possible new USAID-sponsored regional programs, what should be included in order to help ensure that it would be successful in your country? Consider factors such as program focus, management issues including coordination and communication, technical assistance, other) End of Interview Express appreciation for participating in the interview. (Perhaps, if there is anything else that they would like to add as an afterthought, leave a phone number and email address for contacting the interviewer.) Evaluation of LAC/RSD Regional Education Program: Volume II 36 International Business & Technical Consultants, Inc. INSTRUMENT FOR CERCA KEY INFORMANT INTERVIEWS COUNTRY: _____________________ VENUE: __________________ DATE: (day/month/year)_______________________ Name of interviewer: __________________________ Name of person being interviewed: ______________ Present position and organization: ______________________________________ Beginning time for interview _________ Ending time ________ Total time _________ BACKGROUND INFORMATION: USAID has contracted IBTCI to assess three education programs in Latin America and the Caribbean that have been implemented in several countries. Of special interest are the benefits and challenges of working with a USAID/regional program. A “regional” program means that is was developed, implemented, and financed by USAID/Washington and there were multiple countries participating. We are interested in both project impact and the management of the projects within and across the participating countries. USAID is particularly interested in what has worked well and not so well. In addition, they would like guidelines for possible future regional programs. Today we will focus on just one of these programs, the CERCA project. We greatly appreciate your participation in the interview. Your responses will be kept in total confidence. On which project has the interviewee been involved? Please include the position and dates of involvement. Please explain that a “regional” program is one that is developed and implemented by USAID/Washington and it involves the participation of multiple countries. Project Position – Role and responsibility Dates of involvement CERCA PREAL CETT In addition to regional programs, please ask if the person is familiar with any other education programs sponsored by the local USAID mission, so-called bilateral programs. Name of Project Position – Role and responsibility Dates of involvement I. Benefits and Challenges of CERCA’s Regional Model What institutions were involved in CERCA in your country and what was the role of each? (Explore the range of public and private partners; two are listed as a start.) Institution Role of the Institution USAID Ministry of Education Private Sector University Local Government How good was the CERCA coordination with each of the institutions just mentioned? How might the coordination among CERCA’s partners have been strengthened? What were the key contributions of CERCA to improve education quality in your country? Did CERCA result in any new programs, activities or procedures in your country? If so, what were they? Interviewer’s note: As a follow-up prompt, you might refer to the following program components: Development of community participation to support schools “Report cards” Evaluation of LAC/RSD Regional Education Program: Volume II 37 International Business & Technical Consultants, Inc. Case studies of schools with good community participation Training of teachers or administrators To what extent was the Ministry of Education involved in these activities? Did the Ministry incorporate any aspects of the CERCA program into its own programs? If so, which ones? Are any of these activities still being implemented by the Ministry? If so, which ones? If none of the CERCA activities are continuing, why not? Did CERCA built sufficient capacity in the Ministry and/or other partner institutions to ensure the sustainability of the major program activities? Were there any special benefits because CERCA was a regional program involving several countries? Were there any challenges or difficulties relating to CERCA because it was a regional program? Have you heard about the other USAID-regional programs CETT and PREAL? If so, did CERCA coordinate at all with these programs? If you are familiar with any USAID education programs supported by the local USAID mission – so-called bilateral programs, can you tell me about any advantages or disadvantages of a regional program compared to a bilateral program? II. The Value-added of Technical Assistance Did your organization receive any technical assistance from CERCA? Assistance could include consultants, workshops, publications, etc.) If yes, can you recall any specific technical assistance that was provided? b. How useful was it? III. Conclusions and Recommendations Are there any important lessons you learned from CERCA about what makes a good education project? If yes, what are they? (Possible probing suggestions: effective regional participation, management and coordination, addressing high priority needs of many countries, overcoming some of the challenges that they faced, etc.) Do you think there are aspects of the CERCA management that should have been improved? Why and how? (Possible probing suggestions: management, coordination, communication, information dissemination, and technical assistance,). Is there anything else that you would like to recommend or tell me about CERCA that we have not discussed? End of Interview Express appreciation for participating in the interview. (Perhaps, if there is anything else that they would like to add as an afterthought, leave a phone number and email address for contacting the interviewer.) Evaluation of LAC/RSD Regional Education Program: Volume II 38 International Business & Technical Consultants, Inc. INSTRUMENT FOR CETT KEY INFORMANT INTERVIEWS COUNTRY: _____________________ VENUE: __________________ DATE: (day/month/year)_______________________ Name of interviewer: __________________________ Name of person being interviewed: ______________ Present position and organization: ______________________________________ Beginning time for interview _________ Ending time ________ Total time _________ BACKGROUND INFORMATION: USAID has contracted IBTCI to assess three education programs in Latin America and the Caribbean that have been implemented in several countries. Of special interest are the benefits and challenges of working with a USAID/regional program. A “regional” program means that is was developed, implemented, and financed by USAID/Washington and there were multiple countries participating. We are interested in both project impact and the management of the projects within and across the participating countries. USAID is particularly interested in what has worked well and not so well. In addition, they would like guidelines for possible future regional programs. Today we will focus on just one of these programs, the CETT project. We greatly appreciate your participation in the interview. Your responses will be kept in total confidence. On which project has the interviewee been involved? Please include the position and dates of involvement. Please explain that a “regional” program is one that is developed and implemented by USAID/Washington and it involves the participation of multiple countries. Project Position – Role and responsibility Dates of involvement CERCA PREAL CETT In addition to regional programs, please ask if the person is familiar with any other education programs sponsored by the local USAID mission, so-called bilateral programs. Name of Project Position – Role and responsibility Dates of involvement I. Organization, Management, Coordination and Synergy of the program Which institutions were involved in CETT in your country and what was the role of each? (Explore the range of public and private partners; two are listed as a start.) Institution Role of the Institution USAID Ministry of Education Universities Private Sector Local Foundations Can you say something about the leadership of CETT in your country? What went well? What might have been better? Can you say something about the leadership of CETT in the region? What went well? What might have been better? What were the main issues within your country regarding the management of the project? What can you tell me about the coordination of project activities with partners such as the Ministry of Education the universities USAID (include USAID/Washington as well as the local mission, if possible) - other partners Evaluation of LAC/RSD Regional Education Program: Volume II 39 International Business & Technical Consultants, Inc. Can you tell me about any issues of project management due to the participation of several countries in the region? 6a. What went well? 6b. What were the problems? 6c. How did they get resolved? Were there any special benefits to your country as a result of the participation of other countries in the region? What difficulties were encountered as a result of working on a regional basis? Was anything lost as a result of having to work under a regional program? Did the benefits outweigh the difficulties that were encountered? Do you know anything about the CERCA and PREAL projects? 14 a. If yes, was there any coordination of CETT with these projects? 14b. If yes, what was the coordination? If you are familiar with any USAID education programs supported by the local USAID mission – so-called bilateral programs, can you tell me about any advantages or disadvantages of a regional program compared to a bilateral program? II. Information about the CETT program Did CETT address important national education issues in your country? 13a. If yes, which ones? 13b. If not, why not? What was the single most important benefit of the CETT program? Has CETT had an impact in your country in terms of: 15a. Programs of the Ministry of Education 15b. Quality of teaching in primary schools (a few schools or many schools) 15c. Policies of the Ministry of Education 15d. Other Is CETT now operating in your country? 16a. If yes, who is administering it? 16b. If yes, is the entire program being implemented or only some components? If CETT is not operating in your country now, why not? Has CETT developed sufficient capacity to sustain the program: 18a. In the Ministry of Education? 18b. In other partner institutions? III. The value-added of technical assistance Did CETT provide technical assistance to your country in support of the program? (Technical assistance could include consultants, workshops, materials and publications, etc.) Evaluation of LAC/RSD Regional Education Program: Volume II 40 International Business & Technical Consultants, Inc. 19a. If yes: can you describe what technical assistance was provided? 19b. Which type of technical assistance was the most useful? How important were the regional workshops and meetings to CETT’s activities? 23a. Which ones? 23b. Why or why not? IV. Conclusions and recommendations What are the major lessons learned from the CETT program about characteristics of an effective program to improve education? (e.g., pluses and minuses of working through a regional, focusing on teaching reading and teacher training, involvement of public and private partners, etc.) What recommendations can you make that would have made the CETT project stronger and more effective? (e.g., management, leadership, program focus, technical assistance, communication, other) If you were planning a regional education program, what factors would you take into account to ensure success? (e.g., management, coordination, communication, partnerships, others.) Is there anything else that you would like to tell me about CETT that we have not discussed? END OF INTERVIEW Express appreciation for participating in the interview. (Perhaps, if there is anything else that they would like to add as an afterthought, leave a phone number and email address for contacting the interviewer.) Evaluation of LAC/RSD Regional Education Program: Volume II 41 International Business & Technical Consultants, Inc. INSTRUMENT FOR PREAL KEY INFORMANT INTERVIEWS COUNTRY: _____________________ VENUE: __________________ DATE: (day/month/year)_______________________ Name of Interviewer: ________________________________________ Name of person being interviewed: ___________________________________________ Present Position and organization: ____________________________________________ Beginning time for interview _________ Ending time ________ Total time _________ BACKGROUND INFORMATION: USAID has contracted IBTCI to assess three education programs in Latin America and the Caribbean that have been implemented in several countries. Of special interest are the benefits and challenges of working with a USAID/regional program. A “regional” program means that is was developed, implemented, and financed by USAID/Washington and there were multiple countries participating. We are interested in both project impact and the management of the projects within and across the participating countries. USAID is particularly interested in what has worked well and not so well. In addition, they would like guidelines for possible future regional programs. Today we will focus on just one of these programs, the PREAL project. We greatly appreciate your participation in the interview. Your responses will be kept in total confidence. On which project has the interviewee been involved? Please include the position and dates of involvement. Please explain that a “regional” program is one that is developed and implemented by USAID/Washington and it involves the participation of multiple countries. Project Position – Role and responsibility Dates of involvement CERCA PREAL CETT In addition to regional programs, please ask if the person is familiar with any other education programs sponsored by the local USAID mission, so-called bilateral programs. Name of Project Position – Role and responsibility Dates of involvement Organization, Management, Coordination and Synergy of PREAL program What is the relationship of your organization with PREAL? What can you tell me about the coordination between your organization and PREAL? Are there other major partners of PREAL in your country? (Explore, in particular, public and private partners. What have been their roles?) To what extent has the Ministry of Education been an active partner of PREAL including the use of the PREAL documents? What can you tell me about the coordination among PREAL’s partners? Do you think PREAL needs to strengthen coordination with its partners? If yes, how? Have you heard about the other USAID regional programs like CETT and CERCA projects? If so, has PREAL coordinated at all with these projects? Benefits and Challenges of PREAL program What have been the major benefits or advantages of PREAL given that it works in many countries? What difficulties have been encountered as a result of PREAL working on a regional basis in many countries? Did the benefits outweigh the difficulties that were encountered? Evaluation of LAC/RSD Regional Education Program: Volume II 42 International Business & Technical Consultants, Inc. If you are familiar with any USAID education programs supported by the local USAID mission – so-called bilateral programs, can you tell me about any advantages or disadvantages of a regional program compared to a bilateral program? What has been accomplished as a result of the activities of PREAL and its partners? (e.g. workshops, publications, etc.) What are the key contributions of PREAL to improving education quality in: your country? the region? To what extent has PREAL had an influence on the development or change of national education policies in your country? What would you suggest that would increase the influence of PREAL on national education policy in your country or , more generally, to improve the education system? Among PREAL’s policy instruments (research, publications, workshops, etc.) which are the most useful ones for your country? Among PREAL’s policy instruments which are the least useful ones for your country? Have PREAL’s instruments affected the activities of your organization? How? Have PREAL’s instruments affected the activities of the Ministry of Education? How? Technical Assistance Has your organization received any technical assistance from PREAL? (Technical assistance could include consultancies, study tours, workshops, publications, etc.) If yes, what kind of technical assistance? (Be as specific as possible about examples of technical assistance.) How useful has each of the examples of technical assistance been? Conclusions and recommendations What are the most important lessons you learned from PREAL in terms of an effective program to improve education? (e.g., effective regional participation, management and coordination, addressing high priority needs of many countries, overcoming some of the challenges that they faced.) What recommendations can you make that would make the PREAL program stronger and more effective? (e.g., management, leadership, program focus, technical assistance, communication, research, publications, workshops, etc.) Is there anything else that you would like to recommend or tell me about PREAL that we have not discussed? End of Interview Express appreciation for participating in the interview. (Perhaps, if there is anything else that they would like to add as an afterthought, leave a phone number and email address for contacting the interviewer.) Evaluation of LAC/RSD Regional Education Program: Volume II 43 International Business & Technical Consultants, Inc. QUESTIONS FOR PRIVATE SECTOR FOCUS GROUPS Facilitator’s Instructions: This document is a focus group guide; it is not a structured survey questionnaire. The questions below are examples of the kind of questions for group discussions on the topic of, for example, ” Benefits and Challenges.” This format will help to keep the session on the general topic while giving participants enough flexibility to provide information that may be new to the topic under discussion. The facilitator begins by explaining the objectives of the session and the reason for the Focus Group meeting. The Focus Group meeting will start with the following steps: 1. Presentation of the tasks that the groups will be required to carry out (5-10 minutes). 2. Break the Focus Group into 4-6 working teams and assign each team defined tasks (10 minutes). 3. Each team will select a team leader and reporter prior to starting the session. Team leaders will lead the group in discussing the questions provided, while the reporter will prepare the groups’ answers for presentation during the final plenary. 4. Allow teams time for discussion, to formulate answers, and to write them on flip charts (40 minutes). 5. Request each team to present the results of their discussion to the rest of the participants during a plenary session (20 minutes). 6. Conduct an open-forum discussion of the presentations mediated by the facilitator (15 minutes). 7. Thank participants for coming, and close the Focus Group meeting (Facilitators will then collect all flip-chart notes for typing up and analysis). Facilitator/s’ name/s:______________________________ Focus group date: ___________________ Location (country, city):_________________________ Focus group participants: Name Affiliation _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ BACKGROUND INFORMATION 1. What is or has been the type of involvement of the organization that you represent with the CETT, CERCA and/or PREAL programs? Please explain. Team 1. BENEFITS AND CHALLENGES 2. From the perspective of your institution, have you had any direct or indirect benefits from the program? Yes or no? SKIP CODE: If yes, go to 2a. If no, go to 2c. 2a. If yes, what were the benefits? Who benefited? How did they benefit? 2b. In your opinion, what aspect of the program proved to be the most effective in improving educational performance? 2c. If no, why do you believe that there were no benefits to your institution or your beneficiaries? 3. From the perspective of your institution, did you face any challenges implementing any aspects of the program? 3a. If yes, what types of challenges did you face? What did you do to resolve these challenges? 3b. If no, why do you believe that you did not face any challenges? Team 2. Broader Effect of Program Impact 4. In your opinion, did the program have a negative impact on the private-sector educational system? Evaluation of LAC/RSD Regional Education Program: Volume II 44 International Business & Technical Consultants, Inc. 4a. If yes, what was the impact? 4b. If no, why was there no impact? 5. Did the program contribute to the development of the educational capacity of private-sector educational institutions? 5a. If yes, what type of capacity was developed? What specific activities contributed to capacity building? 5b. If no, why wasn’t capacity developed? What was necessary to develop capacity to be built? Team 3. Sustainability 6. If the program contributed positively to your institution, do you believe that the improvement/s will be sustained in the future? 6a. If yes, how will the developments be sustained? 6b. If no, why can’t the developments be sustained? What should have been done to ensure that the developments were sustained? 7. If the program contributed positively to private-sector institutions in general, do you believe that the improvement/s will be sustained in the future? 7a. If yes, how will the developments be sustained? 7b. If no, why can’t the developments be sustained? What should have been done to ensure that the developments were sustained? Team 4. GENERAL COMMENTS/RECOMMENDATIONS 8. What has been the most important lesson that you have learned from this program? How will you use this lesson in the future? 9. What has been the most important lesson that your institution has learned from this program? 9a. How will your institution use this lesson in the future? 10. If this program were to be replicated in the future, what specific recommendation(s) can you make so that private-sector institutions would get the maximum benefit from the program? Facilitator Instruction: Please thank the group for their participation and state that their inputs will better focus the evaluation and lead to possible future USAID-sponsored education initiatives. Assure them that their shared information and statements are anonymous and will not be attributed to individual participants. Evaluation of LAC/RSD Regional Education Program: Volume II 45 International Business & Technical Consultants, Inc. QUESTIONS FOR PUBLIC SECTOR STAKEHOLDERS FOCUS GROUPS Facilitator’s Instructions: This document serves as a focus group guide; it is not a structured survey questionnaire. The questions below serve as examples of the kind of questions for group discussions on the topic of, for example, ” Benefits and Challenges.” This format will help to keep the session on the general topic while giving participants enough flexibility to provide information that may be new to the topic under discussion. The facilitator begins by explaining the objectives of the session and the reason for the Focus Group meeting. The Focus Group meeting will start with the following steps: 1. Presentation of the problem and the tasks that groups will be required to carry out (5-10 minutes). 2. Break the Focus Group into 4-6 working teams and assign each team defined tasks (10 minutes). 3. Each team will select a team leader and reporter prior to starting the session. Team leaders will lead the group in discussing the questions provided, while the reporter will prepare the groups’ answers for presentation during the final plenary. 4. Allow teams time for discussion, to formulate answers, and to write the answers on flip charts (40 minutes). 5. Request each team to present the result of their discussion to the rest of the participants during a plenary session (20 minutes). 6. Conduct an open- forum discussion of the presentations mediated by the facilitator (15 minutes) 7. Thank participants for coming, and close the Focus Group meeting (Facilitators will then collect all flip-chart notes for typing up and analysis.) Facilitator(s’) name(s)______________________ Focus Group date__________________ Location (country, city)___________________ Focus-group participants: Name Affiliation _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ _________________________ Team 1. Background Knowledge Assessment What is your organization’s level of involvement in the three programs – CETT, CERCA, and PREAL? In which CETT, CERCA or PREAL programs were/are you, or your organization involved ? What was/is your level of involvement in your CETT, CERCA and PREAL programs ? What did your CETT, CERCA or PREAL program contribute to education in your country? Was your organization’s experience with any of the three programs above, a positive one? Why? Team 2. Benefits and Challenges of Regional Program Models and Activities Were there any benefits from working with USAID/ multi-country education programs? If there were benefits, please list the 4 major positive outcomes known to your organization. Were there any challenges of working with USAID/ multi-country education programs? Please list the 4 major problems. Evaluation of LAC/RSD Regional Education Program: Volume II 46 International Business & Technical Consultants, Inc. If there were challenges, please discuss the specific tasks or activities that the programs implemented or did not implement that made USAID/ multi-country education programs challenging. How effective are USAID/multi- country program models in addressing educational issues? How efficient are USAID/multi-country program models in addressing educational issues? Team 3. The Broader Effects of a USAID/Washington-based Education Program Have these programs strengthened teaching capacity in public schools? How, and why? Did these programs strengthen or build any national capacities? Who benefitted, and how? Have these programs strengthened capacity to deliver better public-education services in your country? How? Which public-sector education institutions in your country were able to adopt and implement the teaching-improvement components of these programs? To what extent have these programs been able to sustain teaching improvements in public schools in your country? Have these programs affected attitudes on reading and writing issues in the public-education sector? Please provide examples. Team 4. The Value-added of Technical Assistance Has USAID-sponsored classroom management training through these programs improved management practices in your public￾sector schools? How? Please give examples. What USAID-supported assistance in teaching materials development did not work? Please explain. Have private-sector partnerships improved the quality of national public education in your country? What was their contribution? Do you know of any public schools that may have benefitted from services provided by these USAID-sponsored programs? Which public schools, and how did they benefit? Plenary: Recommendations for the Design of Future Washington-based A Assistance Programs What conclusions can you draw about what works in providing technical assistance to education in your country through Washington-based programs like CETT, CERCA or PREAL? What conclusions can you draw about what does not work in providing technical assistance through Washington-based programs like CETT, CERCA and PREAL? Are there better ways of using the technical resources and skills provided through Washington-based programs, to improve public education in your country? Please explain. Should your country continue to depend on international aid and expertise to improve education services? Yes or no, and why? Facilitator Instruction: Please thank the group for their participation and state that their inputs will better focus the evaluation and lead to possible future USAID sponsored education initiatives. Assure them that their responses are anonymous and that shared information and statements will not be attributed to individual participants. Draft November 25 Dr Evaluation of LAC/RSD Regional Education Program: Volume II 47 International Business & Technical Consultants, Inc. 1CERCA ONLINE SURVEY QUESTIONNAIRE SURVEY BACKGROUND: USAID has contracted IBTCI to assess education programs in Latin America and the Caribbean. The main objectives of the evaluation are to: Evaluate the benefits and challenges of USAID/Washington to design and administer education programs in multiple countries taking into account: the potential benefits and liabilities in terms of program impact, and the special challenges of managing field programs under USAID/Washington oversight. Contrast the different models in terms of program implementation at the USAID and implementing-partner level. Identify the broader impacts of the program operated in multiple countries. Consider the value-added of technical assistance provided to implementing partners through USAID/Washington-based contracts and cooperative agreements. Provide recommendations for the design of future Washington-based assistance programs in the education sector. The results of this evaluation will provide important information on what has worked well and what has not worked so well with the CERCA, PREAL, and CETT projects. This online survey focuses only on the CERCA project. The information you provide will lead to recommendations for possible future USAID/Washington-based projects in the region. We would very much appreciate your response within five days of receiving this document. The questionnaire is divided into the following six sections: Background information Project organization and structure Project information and impact Technical Assistance Conclusions and recommendations Special questions for USAID officials Your participation in this survey will remain confidential and any response you provide will not affect your work or standing. The information you share will be used for the evaluation only and not for any other purpose. Any information that may be quoted from responses to this questionnaire will not be attributed to any person completing this survey. Background Information Date: DD, MM, YYYY Your Name: Your position: Title: Name of Institution: Number of years in this position: Contact Information: Telephone: Email: Are you familiar with the USAID CERCA program? Yes No If “no,” there is no need for you to complete this survey. If yes: in what years were you involved with the program? Briefly describe your role and activities under the CERCA program. Project organization and structure Please describe the extent to which the CERCA program has been incorporated into the Ministry of Education. Please list the public and private partners in your country that have had a significant role in the implementation of CERCA. For each partner institution, please comment briefly on its role. Evaluation of LAC/RSD Regional Education Program: Volume II 48 International Business & Technical Consultants, Inc. Institution Role of the Institution USAID Ministry of Education Universities Private Sector Local Foundations The team members in the various participating countries worked well together under the program. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know In the early stages of the project In the latter stages of the project Were there any problems that occurred in how the countries worked together and how issues were resolved? Yes No Please describe. There was good communication within the project. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know Among partners within your country Between your country and other country partners With USAID If you disagree or strongly disagree with any type of communication listed in Question 12 please describe the problems with communications and the efforts made to improve the situation. Did CERCA collaborate with the PREAL or CETT projects? Yes No If so, please describe. What is your opinion about how USAID/Washington managed CERCA? Program Information The CERCA program provided strong benefits to the education system in your country in terms of strengthening the skills of local community members to support for basic education. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know Comments. Evaluation of LAC/RSD Regional Education Program: Volume II 49 International Business & Technical Consultants, Inc. The CERCA program provided strong benefits to the education system in your country in terms of developing more commitment and involvement by local community members to support for basic education. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know Comments. The CERCA program responded to high-priority needs for educational reform in your country. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know Comments. The following CERCA activities have made a significant contribution to strengthening education in your country. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know Local community involvement Use of “report cards” Strengthening teacher training Case study research of selected schools Social mobilization campaigns Other (explain) The project benefited significantly from the participation of other countries. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know If the project benefited from the participation of multiple countries, please summarize the benefits and the factors that lead to these benefits. What has been single most important benefit of the CERCA project? The participation of multiple countries did not benefit your country. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know If, overall, the participation of multiple countries did not benefit your country, please explain why. D. Sustainability Is the CERCA program still operating in your county? Yes No If yes, what specific activities are continuing? If no, why hasn’t the program continued? Did CERCA develop sufficient capacity in the following institutions to sustain the program? Yes No Ministry of Education Other public partners Evaluation of LAC/RSD Regional Education Program: Volume II 50 International Business & Technical Consultants, Inc. Local community organizations Private partners Other Please elaborate. Are there sufficient funds available to sustain the CERCA program? Yes No Please summarize any source of funds. Does the Ministry of Education plan to continue to support CERCA? Yes No Comments. Does the Ministry have plans to scale-up the CERCA program? Yes No Comments. Technical Assistance CERCA’s technical assistance significantly benefitted the project. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know What types of technical assistance were most useful? The regional CERCA workshops, conferences, and seminars made a significant contribution to the project. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know If you Disagree or Strongly Disagree with Question 33, please describe how these meetings have or have not made a significant contribution to the project. The CERCA program used technology to enhance communication. To what extent did these technologies make a significant contribution to the project? Very little contribution Some contribution Significant contribution Email Project websites Teleconferences Other (describe) If the technologies made very little contribution, how could they be used more effectively in the future? Please comment on the strengths and weaknesses of the use of these technologies in CERCA. Conclusions and recommendations From your perspective, what are your most important lessons learned from working with CERCA? (You may comment on management, program design or impact, capacity building, other.) Evaluation of LAC/RSD Regional Education Program: Volume II 51 International Business & Technical Consultants, Inc. If you could contribute to the design of a new USAID/Washington-based program in education like CERCA, what would you include to help ensure the success of the program? Overall, would you recommend that in the future USAID use a Washington-based approach to address educational needs? Yes No Please explain. Please add any final comments. Questions for USAID officials If you are a USAID mission official, please answer the following questions: Please summarize the extent to which you were involved in managing the CERCA project. How important has the CERCA project been in terms of meeting the need for a USAID education program in your mission? Please explain. Are there any advantages for your mission due to the fact that CERCA is a USAID/Washington: initiated project? managed project? funded project? Are there any disadvantages for your mission due to the fact that CERCA is a USAID/Washington: initiated project? managed project? funded project? Please summarize any ways in which the USAID/Washington-based management of CERCA could have been improved. Are funds allocated through USAID/Washington (versus through individual missions) an effective way to use limited funds for education? Please comment. Thank you very much for participating in this survey. The information you shared will help in the evaluation and will provide useful information in the design and implementation of future programs. Evaluation of LAC/RSD Regional Education Program: Volume II 52 International Business & Technical Consultants, Inc. CETT ONLINE SURVEY QUESTIONNAIRE SURVEY BACKGROUND: USAID has contracted IBTCI to assess education programs in Latin America and the Caribbean. The main objectives of the evaluation are to: Evaluate the benefits and challenges of USAID/Washington to design and administer education programs in multiple countries, taking into account: the potential benefits and liabilities in terms of program impact, and the special challenges of managing field programs under USAID/Washington oversight. Contrast the different models in terms of program implementation at the USAID and implementing-partner level. Identify the broader impacts of the program that operated in multiple countries. Consider the value-added of technical assistance provided to implementing partners through USAID/Washington-based contracts and cooperative agreements. Provide recommendations for the design of future Washington-based assistance programs in the education sector. The results of this evaluation will provide important information on what has worked well and what has not worked so well with the CERCA, PREAL, and CETT projects. This online survey focuses only on the CETT program. The information you provide will lead to recommendations for possible future USAID/Washington-based projects in the region. We would very much appreciate your response within five days of receiving this document. The questionnaire is divided into the following six sections: Background information Project organization and structure Project information and impact Technical Assistance Conclusions and recommendations Special questions for USAID officials Your participation in this survey will remain confidential; your responses will be anonymous and will not affect your work in any way. The information you share will be used for the evaluation only and not for any other purpose. Any information that may be quoted from responses to this questionnaire will be anonymous. 2. Background Information Background Information Date: DD, MM, YYYY Your Name: Your position: Title: Name of Institution: Number of years in this position: Contact Information: Telephone: Email: Are you familiar with the USAID CETT program? YES NO If “no,” there is no need for you to complete this survey. If yes: in what years have you been involved with the program? Briefly describe your role and activities under the CETT program. Project organization and structure Evaluation of LAC/RSD Regional Education Program: Volume II 53 International Business & Technical Consultants, Inc. Hosting the project in a university was an effective approach for project implementation. (In Jamaica, include the role of the Joint Board of Teacher Education as well as UWI.) Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know What have been the principal advantages and disadvantages of hosting the project in a university? The team members in the various participating countries worked well together. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know Please describe any problems that occurred as a result of having CETT located in several countries that were expected to work together How were the difficulties overcome? There was good communication in the project. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know Among partners within the country Between your country and other country partners With USAID Please describe any problems with communications. Was there any collaboration with the PREAL or CERCA projects? If so, please describe. Please summarize the involvement of public and private partners in implementing the project. Institution Role of the Institution USAID Ministry of Education Universities Private Sector Local Foundations What is your opinion about how USAID/Washington managed CETT? Program Information The CETT program has provided strong benefits to the education system in your country in the following ways: Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know Improvement of teaching reading and writing Evaluation of LAC/RSD Regional Education Program: Volume II 54 International Business & Technical Consultants, Inc. Improvement of teacher training Improvement of classroom instruction and management Please explain or give examples. The CETT program responded to high-priority needs for educational reform in your country. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know Please elaborate. The participation of multiple countries strengthened the implementation of the project in the following areas: Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know Curriculum design Teacher training Management Education policy Other (please describe) If the project benefited from the participation of multiple countries, please summarize the benefits and the factors that lead to the benefits. What has been single most important benefit of the CETT project? The participation of multiple countries did not benefit your country. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know If, overall, the participation of multiple countries did not benefit your country, please explain why not. Is the CETT program still operating in your county? Yes No If yes, are both the reading-instruction and teacher-training components still operating? Yes No Please explain. Has the Ministry of Education included CETT as one of its own programs? Yes No If yes, explain CETT’s position or role in the Ministry. If not, why not? Has CETT developed sufficient capacity in the following institutions to sustain the program? Yes No Ministry of Education Evaluation of LAC/RSD Regional Education Program: Volume II 55 International Business & Technical Consultants, Inc. Other public partners Private partners Other Please elaborate. Are there sufficient funds available to sustain the CETT program? Yes No Please summarize the source of the funds. Are there plans to scale-up the CETT program in your country? Yes No Technical Assistance CETT technical assistance has provided significant benefit to the project. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know What types of technical assistance have been the most useful? The regional CETT workshops, conferences, and seminars have made a significant contribution to the project. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know Please describe how these meetings have or have not made a significant contribution to the project. The CETT project has used some technologies to enhance communication and, in some cases, to provide support or training to staff and teachers. To what extent have these technologies made a significant contribution to the project? Very little contribution Some contribution Significant contribution Email Project websites Teleconferences Online courses Other (describe) Please comment on the strengths and weaknesses of using these technologies. How could these or other technologies be used more effectively? Conclusions and recommendations What are the most important lessons learned from working with CETT about the characteristics of an effective regional education program? (You may comment on management, program design or impact, capacity building, other.) If you could contribute to the design of a new USAID/Washington-based program in education, what would you include to help ensure the success of the program? Overall, would you recommend that in the future USAID use a Washington-based approach to address educational needs? Yes No Please explain. Evaluation of LAC/RSD Regional Education Program: Volume II 56 International Business & Technical Consultants, Inc. Please add any final comments. Questions for USAID officials If you are a USAID mission official, please answer the following questions: Please summarize the extent to which you were involved in managing the CETT project. How important has the CETT project been in terms of meeting the need for a USAID education program in your mission? Please explain. Have there been any advantages for your mission due to the fact that CETT is a USAID/Washington: initiated project? managed project? funded project? Have there been any disadvantages for your mission due to the fact that CETT is a USAID/Washington: initiated project? managed project? funded project? Please summarize any ways in which the USAID/Washington-based management of CETT could have been improved. Are funds allocated through USAID/Washington (versus through individual missions) an effective way to use limited funds for education? Please comment. Thank you very much for participating in this survey. The information you shared will help in the evaluation and will provide useful information in the design and implementation of future programs. Evaluation of LAC/RSD Regional Education Program: Volume II 57 International Business & Technical Consultants, Inc. PREAL ONLINE SURVEY QUESTIONNAIRE SURVEY BACKGROUND: USAID has contracted IBTCI to assess education programs in Latin America and the Caribbean. The main objectives of the evaluation are to: 1. Evaluate the benefits and challenges of USAID/Washington to design and administer education programs in multiple countries taking into account: a. the potential benefits and liabilities in terms of program impact, and b. the special challenges of managing field programs under USAID/Washington oversight. 2. Contrast the different models in terms of program implementation at the USAID and implementing-partner level. 3. Identify the broader impacts of the program that operated in multiple countries. 4. Consider the value-added of technical assistance provided to implementing partners through USAID/Washington-based contracts and cooperative agreements. 5. Provide recommendations for the design of future Washington-based assistance programs in the education sector. The results of this evaluation will provide important information on what has worked well and what has not worked so well with the CERCA, PREAL, and CETT projects. This online survey focuses only on the PREAL project. The information collected will lead to recommendations for possible future USAID education projects in the region. We would very much appreciate your response within five days of receiving this document. The questionnaire is divided into the following seven sections: 1. Background information 2. Project organization and structure 3. Project information and impact 4. Technical Assistance 5. Conclusions and recommendations 6. Special questions for USAID officials Your participation in this survey will remain confidential; your responses will be anonymous and will not affect your work in any way. The information you share will be used for the evaluation only and not for any other purpose. Any information that may be quoted from responses to this questionnaire will be anonymous. 2. Background Information Background Information Date: DD, MO, YYYY Your Name: Your position: Title: Name of Institution: Number of years in this position: Contact Information: Telephone: Email: Are you familiar with the USAID PREAL program? YES NO If “no,” there is no need for you to complete this survey. If yes: in what years have you been involved with the program? Briefly describe your role and activities under the PREAL program. Project organization and structure Evaluation of LAC/RSD Regional Education Program: Volume II 58 International Business & Technical Consultants, Inc. Was there any collaboration with the CETT or CERCA projects? Yes No If yes, please describe. Please summarize the involvement of public and private partners in administering or contributing to the project. Institution Role of the Institution Public Ministry of Education Other ministries Public universities Research Institutes Communications centers Regional think tanks Bilateral organizations Multilateral organizations Other Private NGOs Private universities Private companies Multinational companies Cooperative agencies Other What is your opinion about how USAID/Washington has: - supported PREAL? - managed PREAL? - coordinated PREAL? Program Information What are the major PREAL activities in your country? In general, the PREAL program has provided strong benefits to the education system in your country through the following activities: Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know Report cards Policy dialogue Research studies Publications Media campaigns Workshops Other Please summarize the most important impacts of PREAL in your country. Evaluation of LAC/RSD Regional Education Program: Volume II 59 International Business & Technical Consultants, Inc. In general, the PREAL program has provided strong benefits to the education systems in participating Latin America and the Caribbean countries. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know Please comment. Has the Ministry of Education incorporated information generated by PREAL, or used data analyzed and collected by PREAL￾sponsored institutions? Yes No Please explain. Technical Assistance Technical assistance provided by PREAL has significantly benefitted your national education program. Strongly disagree Disagree Neutral Agree Strongly Agree Don’t know What types of technical assistance have been the most useful? The PREAL project used some technologies to enhance communication and the sharing of documents. Have these technologies added value to the project? Very little contribution Some contribution Significant contribution Email Project websites Teleconferences Websites Technical blogs Other (describe) Please comment on the strengths and weaknesses of the use of these technologies. How could these or other technologies be used more effectively? Conclusions and recommendations What are the most important lessons you learned from working with PREAL regarding characteristics of effective regional education programs? Overall, would you recommend that in the future USAID use a Washington-based approach to address educational needs? Yes No Please explain. If you could contribute to the design of a new USAID/Washington-based education assistance program, what would you include to help ensure the program’s success? Please add any final comments. Questions for USAID officials Evaluation of LAC/RSD Regional Education Program: Volume II 60 International Business & Technical Consultants, Inc. If you are a USAID mission official, please answer the following questions. Please summarize the extent to which you were involved in managing the PREAL project. How important has the PREAL project been in terms of meeting the need for a USAID education program in your mission? Please explain. Have there been any advantages for your mission due to the fact that PREAL is a USAID/Washington: initiated project? managed project? funded project ? Have there been any disadvantages for your mission due to the fact that PREAL is a USAID/Washington: initiated project? managed project? funded project ? Please summarize how USAID/Washington’s management of this program could be improved. Are funds allocated through USAID/Washington (versus through individual missions) an effective way of distributing and using limited funds for education? Please comment. Thank you very much for participating in this survey. The information you shared will help in the evaluation and will provide useful information in the design and implementation of future programs. Evaluation of LAC/RSD Regional Education Program: Volume II 61 International Business & Technical Consultants, Inc. Annex V: List of People Interviewed A. CARIBBEAN SITE VISITS ANTIGUA Name of Respondent Position Institution Date of Interview LAC Regional Program(s) Ms Emily Victoria Edghill Director, Language Arts, Ministry of Education Ministry of Education (MoE) 12/16/2010 C-CETT Ms Doristene Estinoff Assistant Director of Planning Ministry of Education (MoE) 12/16/2010 C-CETT Mrs. Jacintha Pringle Director of Education Ministry of Education (MoE) 12/16/2010 C-CETT Mrs. Paula Spencer Lecturer Antigua State College 12/16/2010 C-CETT GRENADA Name of Respondent Position Institution Date of Interview LAC Regional Program(s) Dr Martin Baptiste Former Chief Education Officer Ministry of Education (MoE) 12/21/2010 C-CETT Mrs. Pearl Belfon Early childhood officer Ministry of Education (MoE) 12/08/2010 C-CETT Mrs. Gloria Bonaparte Education Officer Ministry of Education (MoE) 12/08/2010 C-CETT Dr Jeffrey Britton Principal Marryshaw Community College 12/07/2010 C-CETT Alisha Durrant Administrative secretary, CETT Ministry of Education (MoE) 12/06/2010 C-CETT Ms Patricia Felix Deputy Chief Education Officer Ministry of Education (MoE) 12/06/2010 C-CETT Mrs. Elizabeth Forsyth Reading Specialist Marryshow Community College and Ministry of Education (MoE) 12/07/2010 C-CETT Mrs. Bernadette John Early childhood officer Ministry of Education 12/08/2010 C-CETT Ms Peron Johnson Project Officer Ministry of Education (MoE) 12/07/2010 C-CETT Mrs. Alexis-Jones Teacher Trainer, Marryshow Community College 12/07/2010 C-CETT Anne-Marie Marcheau Permanent Secretary Ministry of Education (MoE) 12/08/2010 C-CETT Ms Yolande Noel Early childhood officer Ministry of Education 12/08/2010 C-CETT Mr. Michael Philbert Book shop owner (private sector) The Book shop 12/08/2010 C-CETT Mr. Timothy Scott C-CETT ICT Specialist C-CETT and Ministry of Education 12/07/2010 C-CETT Ms Sandra Thomas Curriculum Development Specialist Ministry of Education (MoE) 12/23/2010 C-CETT Evaluation of LAC/RSD Regional Education Program: Volume II 62 International Business & Technical Consultants, Inc. JAMAICA Name of Respondent Position Institution Date of interview LAC Regional Program(s) Mr. Delroy Alleyne Former and final Regional Coordinator of C-CETT prior to end of project Joint Board of Teacher Education (JBTE), University of the West Indies, Kingston 11/29/2010 C-CETT Mr. Mansfield Blackwood Senior Technical Specialist USAID Mission, Barbados and the Eastern Caribbean 12/03/2010 C-CETT Mrs. Yvonne Coore-Johnson Former AOTR USAID Mission, Jamaica 12/13/2010 C-CETT Ms Sandra Falconer Former Director of Communications Air Jamaica 12/03/2010 C-CETT Mrs. Winsome Francis Reading Specialist St Joseph Teachers’ College 12/02/2010 C-CETT Mr. James Goggin Director USAID Mission, Barbados 12/03/2010 C-CETT Dr Winsome Gordon Exec. Dir. Jamaica Teacher’s Council Ministry of Education (MoE) 11/30/2010 C-CETT Mrs. Millicent Graham Principal/Teacher Trainer Alpha Primary School 12/03/2010 C-CETT Professor Stafford Griffith 2nd Director of PIU/C￾CETT JBTE, UWI, Kingston 12/02/2010 C-CETT Mr. Luke Jackson ICT Specialist PIU/JBTE, UWI, Kingston 01/04/2011 C-CETT Dr Clement Lambert Teacher Trainer School of Education, UWI, Kingston 11/29/2010 C-CETT Mr. Jasper Lawrence Former Chief Education Officer Ministry of Education (MoE) 12/01/2010 C-CETT Professor Errol Miller 1st Director PIU/C-CETT JBTE, UWI, Mona 12/03/2010 C-CETT Ms Sharon Neita Director Scholastic Books 12/01/2010 C-CETT Dr Sonjai Reynolds-Cooper Former AOTR USAID Mission, Jamaica 12/27/2010 C-CETT Mrs. Claire Spence AOTR Education Officer and Project Officer USAID Mission, Jamaica 11/30/2010 C-CETT Mrs. Joan Spencer￾Hernandez Assessment and Diagnostic Specialist, C￾CETT School of Education, UWI, Kingston 11/30/2010 C-CETT Dr Marcia Stewart Deputy Director/PIU/UWI JBTE, UWI, Kingston 12/06/2010 C-CETT Mr. Robert Wright Caribbean Director INMED 12/03/2010 C-CETT ST. LUCIA Name of Respondent Position Institution Date of Interview LAC Regional Program(s) Dr Martina Augustin Dean, Academic Programmes Sir Arthur Lewis Community College (SALCC) 12/14/2010 C-CETT Evaluation of LAC/RSD Regional Education Program: Volume II 63 International Business & Technical Consultants, Inc. Ms. Elizabeth Bissette Teacher Educator in Language Arts Sir Arthur Lewis Community College (SALCC) 12/10/2010 C-CETT Mrs. Christella Duplessis￾Charles Reading Specialist Ministry of Education (MoE) 12/10/2010 C-CETT Ms Marietta Edward Deputy Chief Education Officer for Instruction Ministry of Education 12/14/2010 C-CETT Mr. Millines Herman C-CETT ICT Specialist Sir Arthur Lewis Community College (SALCC) 12/10/2010 C-CETT Mrs. Augusta Ifill Chief Education Officer Ministry of Education 12/13/2010 C-CETT Ms Esther Joseph Basic Education Enhancement Programme Ministry of Education (MoE) 12/14/2010 C-CETT Mrs. Caterina Mark MOE Testing and Evaluation Officer Ministry of Education 12/14,2010 C-CETT Ms. Nadia Maxwell Teacher Educator in Language Arts Sir Arthur Lewis Community College (SALCC) 12/10/2010 C-CETT Ms Desiree Montoute Former Administrative Secretary C-CETT 12/14/2010 C-CETT Ms. Angelina Polius Teacher Educator in Language Arts Sir Arthur Lewis Community College (SALCC) 12/10/2010 C-CETT Mrs. Sonia Severin Acting Dean Sir Arthur Lewis Community College (SALCC) 12/13/2010 C-CETT B. CA-RD SITE VISITS DOMINICAN REPUBLIC Name of Respondent Position Institution Date of Interview LAC Regional Program(s) Lesbia Brea De Castillo Education Adviser Falcondo Foundation 11/25/2010 CERCA Dr. Miguel J. Escala Instituto Tecnologico de Santo Domingo Rector 11/22/10 PREAL Aida Consuelo Hernandez EDUCA Directora Ejecutiva 11/23/10 PREAL Erika Morales Huriado Coordinator of Dissemination and Community Relations Escuelas Efectives program Pontificia Universidad Catolica Madre y Maestra, PUCMM 11/27/2010 CETT Erika Morales Hurtado CETT, Community Relations Coordinator Pontificia Universidad Catolica Madre y Maestra, PUCMM 11/26/2010 CETT Norma Mena Jaque, CETT School Activites Coordinator Pontificia Universidad Catolica Madre y Maestra, PUCMM 11/26/2010 CETT Sarah Gonzales Lora Vice Rector of International Relations Pontificia Universidad Catolica Madre y Maestra, PUCMM 11/26/2010 CETT/PREAL Radhames Mejia Vice Rector and Director Center for Education and Human Development, 11/24/2010 CETT/PREAL Evaluation of LAC/RSD Regional Education Program: Volume II 64 International Business & Technical Consultants, Inc. DOMINICAN REPUBLIC Name of Respondent Position Institution Date of Interview LAC Regional Program(s) PUCMM Ancell Sheker Mendoza MOE , Director General Education Assessment 11/25/2010 CETT Dra Liliana Montenegro CETT Director Pontificia Universidad Catolica Madre y Maestra, PUCMM 11/26/2010 CETT Dr. Fernando Ogando Secretaria de Estado de Educacion, Coordinador Ministry of Education - International Development Department 11/25/2010 CETT /PREAL Madga Pepén Peguero FLACSO Facultad Latinoamericana de Ciencias Sociales Investigadora Asociada 11/25/10 PREAL Cesar Nicolas Penson Banco Central de la Republica Dominicana Miembro de la Junta Monetaria 11/25/10 PREAL Marina Teveras Education Specialist USAID Mission 11/24/2010 CETT Jana Wooden Education Officer USAID Mission 11/24/2010 CETT HONDURAS Name of Respondent Position Institution Date of Interview LAC Regional Program(s) Mireya Batres Education Officer and CETT AOTR USAID, Tegucigalpa, 11/29/2010 CETT Cynthia Chassy Education Officer USAID, Tegucigalpa 11/29/2010 Dra. Elia del Cid Vice Minister of Education, Ex-CETT professional Ministry of Education, Tegucigalpa 12/02/2010 PREAL/CETT Dr. Carleton Corrales Ex- CETT Director, Rector of a private polytechnic univ. University of Honduras, Tegucigalpa 12/01/2010 CERCA/ CETT/ PREAL Lic. Lea Cruz The Rector UPNFM, Tegucigalpa 11/30/2010 CETT/PREAL Lic. Gloria Gamero Education Coordinator Ministry of Education, Tegucigalpa 11/30/2010 CETT Lic. Alba Luz Director CARE International CARE International 12/03/2010 CETT/ CERCA/ PREAL Lic. Daniel Medina Researcher, CETT Tegucigalpa 12/03/2010 CETT Dr. Marco Tulio Mejia CETT Director (retired) UPNFM, Tegucigalpa 11/30/2010 CETT/ PREAL Lic. Edwin Moya Director of MIDEH USAID/AED Project, Honduras 12/01/2010 CETT/ PREAL Srta, Bessy Pacheco Director of Concorde NGO network Tegucigalpa 12/02/2010 CETT Josefina Gamero Pinel Technical Director FEREMA, Tegucigalpa 11/29/2010 CERCA / PREAL Evaluation of LAC/RSD Regional Education Program: Volume II 65 International Business & Technical Consultants, Inc. Dr. Renan Rapalo Senior Researcher, Education National Pedagogic University (UPNFM) 11/30/2010 CETT/ CERCA Karla Yaneth Raudales Program Coordinator FEREMA, Tegucigalpa 11/29/2010 PREAL Lic. Karla Ruiz Manager Honduran Talent, Member of the Chamber of Commerce 12/04/2010 PREAL/ CETT Mr. Ned Van Steenwyk Ex- USAID COTR CERCA (retired) Tegucigalpa 12/03/2010 CETT/ CERCA/PREAL Dra. Gloria Ulloa Director, CETT UPNFM, Tegucigalpa 11/30/2010 CETT/ PREAL C. ANDEAN SITE VISITS ECUADOR Name of Respondent Position Institution Date of Interview LAC Regional Program(s) Ms. Ana Alvarado Auditor for CETT program office in the USAB USAB 12/07/2010 CETT Orazio Belletini Drector Grupo Faro 12/08/2010 PREAL María Cecilia Bustamante Now Evaluation Consultant for USAID, UNICEF and CARE. Movimiento Fe y Alegría 12/08/2010 CETT Mr. Mario Cifuentes Director of CETT implementation in Ecuador Director Academico del Area de Educacion USAB 12/07/2010 CETT Ms. Helen Conefrey Agregada de Cooperación Comisión Europea 12/10/2010 CETT Ms. Patricia Medina CETT materials developer at the UASB UASB 12/07/2010 CETT Ms. Soledad Mena Coordinadora Pedagogica UASB 12/08/2010 CETT Sr. Juan Ponce Sub-Director Académico FLASCO Facultad Latino Americana de Ciencias Sociales 12/08/2010 PREAL Sr. Fernando Unda Coordinador de Proyectos Educación Cooperativa de Ayuda y Remesas al Ecuador – CARE 12/10/2010 CETT Ms. Sofia Villalba Program Development Specialist USAID Mission 12/07/2010 CETT PERU Name of Respondent Position Institution Date of Interview LAC Regional Program(s) Patricia Arregui Working Group coordinator GRADE Grupo de Analisis para el Desarrollo 12/13/2010 PREAL Evaluation of LAC/RSD Regional Education Program: Volume II 66 International Business & Technical Consultants, Inc. Fernando Bolaños Esp. Educacion USAID 12/13/2010 CETT Dr. Jorge Cobian Director Educac. Primaria - Minedu 12/14/2010 CETT Eduardo Arrarte Fedler Director CETT - INMED 12/15/2010 CETT Richard Goughnor Jefe de Mision USAID 12/15/2010 Ariana Mantilla Jefe de assuntos corporativos Natura 12/14/2010 Peregrina Morgan Presidenta - IPEBA Instituto Peruano de Evaluacion, Acreditacion y Cerificacion de la Calidad de la Educacion Basica 12/14/2010 PREAL Patricia Osorio Especialista OEI Organizacion de Estados Iberoamericanos 12/14/2010 CETT Dra. Juana Pinzas JEFA ESTUDIO Coordinador of CETT Evaluation PUCP Pontificia Universidad Catolica de Peru 12/16/2010 CETT Dra. Miriam Ponce Director BASIC Education 12/15/2010 CETT Paola Robles Project Coordinator Empresarios por la Educacion 12/16/2010 CETT Nelson Soto Gerente MA Security PLUSPETROL 12/17/2010 CETT Luis Miguel Starke Co-director Empresarios por la Educacion 12/16/2010 CETT Idel Vexler Vice-Ministro de Gestión Pedagogica MINEDU 12/16/2010 PREAL/CETT Cecilia Zevallos Gerente Desarrollo Sostenible Fundac. Backus 12/17/2010 CETT D. ADDITIONAL PEOPLE INTERVIEWED Name of Respondent Position Institution Date of Interview Barbara Knox-Seith Education Lead USAID/ LAC Bureau 1/6/ 2011 David Evans Retired Education Team Leader USAID/ LAC Bureau, 12/22/2010 Cheryl Kim Former COTR/AOTR, CETT and CERCA USAID/LAC Bureau 1/10/2011 (Truncated phone call) Alicia Slate COTR, Education USAID/Nicaragua 2/2/2011 (Telephone call) Jeff Puryear Director PREAL 10/22/2010 Ana Flores CERCA Field Coordinator AED 1/4/2011 Roger Rasnake Carol Da Silva Felipe Milan Mirka Tvaruzkova Rebekah Levi Vice-President CETT M&E team Aguirre Division of JBS International 10/25/2010 Evaluation of LAC/RSD Regional Education Program: Volume II 67 International Business & Technical Consultants, Inc. Annex VI: Online Survey Questionnaire Results and Analyses Introduction Apart from key informant interviews, focus groups, and document reviews, the LAC evaluation also administered an online survey to collect information from respondents who are or have been involved with any of the three education programs but could not be personally reached by the evaluation team. The survey collected a sample of 76 respondents; the Table below provides a brief profile of the respondents. The sample size exceeded the targeted number of 65 respondents.1 Table 1: Distribution of Respondents Number of Respondents Percentage share Distribution of Respondents by Program Affiliation CETT 55 72% PREAL 14 18% CERCA 7 9% Total 76 Distribution of Respondents by Current Occupation Education Specialists, Advisors 22 29% Management of Education Projects or Programs 14 18% Director for Education Programs 11 14% Teacher Trainer 7 9% Academia 3 4% President or Vice-President of Education Programs 3 4% Teachers 3 4% Independent Consultants 2 3%                                                             1 As stated in the Methodology document, the sample size was determined based on a population list of 200 people, with a 95% confidence level and 10% confidence interval. Evaluation of LAC/RSD Regional Education Program: Volume II 68 International Business & Technical Consultants, Inc. M&E Specialists 2 3% Other Unclassified 9 12% Total 76 Methodology The survey was designed and uploaded into the Survey Monkey website, which allowed the evaluation team to distribute the instrument electronically. Each education program had its own survey instrument but all three surveys consisted of questions that cover the following general areas: Program Organization and Structure Program Information Sustainability Technical Assistance Conclusions and Recommendations A field test was conducted on December 1, 2010 to assess the overall effectiveness of the survey to achieve valid outcomes. Results revealed the need for formatting changes to improve ease of use and reduce the length of time to complete. The survey was launched on December 9, 2010 and was kept online until January 5, 2010. English and Spanish versions were available to give respondents a choice on which survey to use. Email invitations were sent to potential respondents and follow up messages were sent every week to those who have not replied. The collection, management, and analysis of results were done in two applications. The first one was the MaxQDA software, which is a qualitative data analysis package that helps classify and identify trends in responses. The analysis of the survey’s descriptive statistics was done on Microsoft Excel. Methodological Considerations As with many remotely-administered surveys, the LAC Online Survey had to consider several factors in carrying out and managing surveys. Non-random: The survey aimed to gather information from individuals with knowledge of one or more programs so that the information will be useful and relevant. Thus, a purposive sampling method was applied, which is a non-random form of sampling that selects respondents with known or demonstrable knowledge of the subject matter. So the respondents included administrators, senior professionals, trainers, and relevant key respondents involved in the three programs at all functional levels. Because the method was non-random, findings are reported in Evaluation of LAC/RSD Regional Education Program: Volume II 69 International Business & Technical Consultants, Inc. the context of the survey sample only and not of the entire population of those involved with CETT, CERCA, and PREAL. Qualitative findings: MaxQDA is a qualitative data analysis software that allows the user to create a classification or taxonomy of the responses so that significant trends and outcomes can be identified within the data. Because the analysis is qualitative, the strengths of the findings cannot be determined with quantitative statistical techniques (e.g. significance testing). Contact information: The survey team experienced challenges in reaching some respondents because the contact information was either outdated or inactive. In Universidad Nur, for example, the email systems for 40 people rejected the online survey because the addresses were inactive. The survey team was able to obtain current addresses after one week and the email messages were successfully delivered. Only one C-CETT survey: There was only one respondent for the Caribbean-CETT survey. The survey was included in the analysis of an aggregated set but it was clearly not sufficient to report findings specific to a C-CETT subsample. CERCA findings: Of all the LAC programs, CERCA had the shortest period of operation and the least number of participants in terms of local staff. Main Findings A. Technical Assistance Figure 1 0% 10% 20% 30% 40% 50% Strongly disagree Disagree Neutral Agree Strongly agree Don't know No Response The program's technical assistance provided significant benefits CETT CERCA PREAL ` For CETT respondents, 71 percent at least agreed that technical assistance, such as trainings, workshops, and seminars, provided significant benefits to the local education program. Sixty￾four percent of PREAL respondents and 61 percent of CERCA respondents gave the same Evaluation of LAC/RSD Regional Education Program: Volume II 70 International Business & Technical Consultants, Inc. view. Respondents stated that a major benefit of technical assistance was it provided teachers with a program to improve their instruction skills. CETT, for example, had the physical resources and technical staff to enhance teaching skills through training. Another benefit of technical assistance was that local education professionals were provided with a framework that could monitor and improve the progress of education programs. B. Program Perceptions Figure 2 0% 10% 20% 30% 40% 50% 60% Yes No Don't Know No Response Would you recommend Washington-based approaches in the future? CETT CERCA PREAL As seen from Figure 2 above, 53 percent of CETT respondents and 57 percent of CERCA respondents would recommend a Washington-based approach for future education programs. However, only 43 percent of PREAL respondents agreed (although 21 percent did not provide any preference). But an examination of the qualitative responses provides a more qualified or restricted response. Generally, respondents would recommend a Washington-based program in the future but it must be cognizant of different realities, needs, and priorities among countries and/or ministries. CETT respondents saw how the program deferred to local governments and offices in many situations but also called for an increase in the responsibilities of local officials in the program. Some CERCA respondents stated the need to intensify the program’s links with education ministries, universities, local governments and other institutions. Evaluation of LAC/RSD Regional Education Program: Volume II 71 International Business & Technical Consultants, Inc. C. Communication and collaboration within CETT and CERCA projects Table 2: There was good communication and collaboration within projects. Strongly disagree Disagree Neutral Agree Strongly agree Don't know CETT Among partners within your country 0% 2% 12% 35% 45% 6% Between your country and other country partners 0% 2% 14% 37% 41% 6% With USAID 2% 4% 12% 24% 48% 10% CERCA Among partners within your country 17% 0% 0% 33% 33% 17% Between your country and other country partners 17% 0% 17% 33% 33% 0% With USAID 20% 0% 20% 20% 20% 20% Table 2 shows most CETT and CERCA respondents agreed or strongly agreed that there was communication within their programs and with relevant organizations. An important factor in this outcome was the application of information technology (IT) to help facilitate communication between parties. In Guatemala, CERCA teleconferences were conducted to link institutions and individuals within the program. In CETT, the use of IT communications made the exchange of information more fluid and effective. Table 3: The team members in various participating countries worked well together. Strongly disagree Disagree Neutral Agree Strongly agree Don't know No Response CETT 0% 4% 9% 39% 31% 13% 4% CERCA 15% 0% 8% 0% 46% 23% 8% CETT and CERCA respondents also agreed that program members in each country worked well together. In CERCA, its thrust of civic engagement achieved a significant level of community participation at all levels including teachers, students, parents (mostly mothers), and community organizers. In certain cases, there was also the involvement of key local government officials like the vice-mayor. Evaluation of LAC/RSD Regional Education Program: Volume II 72 International Business & Technical Consultants, Inc. D. Public Private Partnerships The survey also explored respondents’ perceptions on public-private partnerships that have played a significant role in the implementation of the education program. Listed below are the general findings by group that was drawn from qualitative responses in the survey. USAID: The Agency was largely seen as a source of financial support and technical assistance. The Reporte Escolar in Guatemala, for example, was recognized as a valuable technical resource for the implementation of community-based programs to increase the quality of education. Because it was also recognized as the lead and driving force of all education initiatives, USAID was also seen as a provider of support in program management and facilitation. Ministry of Education: To the respondents, education ministries viewed themselves as local overseers of education initiatives, regional or otherwise. On one hand this was beneficial. They encouraged collaboration and cooperation with the various agencies like USAID and the universities. They also exercised regulatory authority over public schools, ensuring the teaching methods were consistent with national policies (e.g. to improve reading comprehension). On the other hand, some ministries determined their level of engagement depending on their own plans. The involvement of the MINED in El Salvador, for instance, ranged from neutral to active participation and their engagement was highly dependent on the current administration’s education policies and its ability to influence CETT programs. Universities: Higher-level educational institutions provided technical and methodological expertise in program assessment. Universities also became a forum for policy dialogue on education issues. However, in some instances, universities appeared to be disconnected from local education ministries and its initiatives. A key element to their successful contributions was the formation of consortia for regional education. The consortia allowed universities to gain regional perspectives on education issues while also contributing their local expertise to the discussion of various education issues. Private sector: The involvement of the private sector was largely in the area of training and program support but was very limited in terms of funding. The expertise of private sector groups and NGOs contributed to the success of programs like Roundtable for Reading. In some areas, local entrepreneurs and information technology (IT) companies explored the idea of partnering with or supporting local education initiatives but nothing was formally established. Local foundations: Most of the respondents stated there were not many local foundations involved in regional education programs. The main reason was that there were no groups that had the expertise and local capacity to partner with or execute local education plans. The most notable exception was FEPADE in El Salvador. It was recognized as the only local foundation capable of executing details of the CETT program. Evaluation of LAC/RSD Regional Education Program: Volume II 73 International Business & Technical Consultants, Inc. E. Incorporation into national education programs/Ministry of Education Most of the respondents that provided information to this question stated that only selected components of the programs have been incorporated into national education programs (by the local ministry). In Guatemala, for instance, the ministry established the Education Reform Project, which had some elements of the CETT program but not those that included strategies to train technicians and teachers. However, none of the respondents were able to conclude whether the approach had positive or negative consequences on the national program as a whole. Evaluation of LAC/RSD Regional Education Program: Volume II 74 International Business & Technical Consultants, Inc. Annex VII: Findings Matrices CARD-CETT Matrix Based on Field Interviews, Focus Groups and Online Survey Data, Collected (November-December, 2010) SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey What are the benefits and challenges of supporting regional program models, such as are used in CETT? Benefits and challenges What were the benefits of supporting regional education programs? What was gained by working regionally? ●From the USAID DC viewpoint coordinating multiple needs and demands in many countries presented a logistical and management challenge. ●This was partly offset because CETT was a White House initiative and had direct access to Presidential funds which were quickly accessed and disbursed. ●CETT leaders in Honduras and the DR noted that USAID leadership in DC started well, but with several changes in leadership and DC management styles, became more challenging. ●Both DR and Honduran CETT leaders attested to there being a 2 year hiatus at the start of CETT, before the program was able to get started. ●Cause was assigned to weak leadership at the UPN and USAID coordination level in Honduras. ●Undefined technical roles for participating CARD institutions created conflict and confusion. ●Ministries of education gained indirectly, by becoming more aware that their own national education shortcomings were not isolated, or unique, but shared by other countries in their region. ●MoEs learned that sharing ideas and comparing solutions was both cost-effective and beneficial both nationally and to others in the region. ●Where national problem solving resulted in a working solution, or model, like CETT- national prestige could be enhanced and international attention could attract additional technical and financial ●The IBTCI team was unable to assess how regional programs reviewed in the countries visited in the CARD-CETT area had benefited the private sector. ●In only one country, not visited , had the private sector under a local PS consortium called FEPADE had any impact on the implementation of CARD-CETT. ●In Guatemala, under the leadership of the UVG, a private university, CETT activities made some impact. ●There, the UVG held community based popular reading programs and awareness raising ● In the DR CETT challenged the MINERD to be more critical about the way teachers were being trained and USAID funding and TA helped the government begin important reforms in classroom teaching methods. ● We gained a lot from meeting and working with other educators from CARD countries. That taught teamwork, problem sharing, and how to provide technical support to others in need. ● The DR learned to apply a new classroom centered model for teacher training that had trainers mentoring teachers in situ. A unified, multisectoral commitment to foster change in teaching Results-based changes in performances of primary-level students. Evaluation of LAC/RSD Regional Education Program: Volume II 75 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey support. campaigns to encourage public interest in reading and writing, with some success. What were the challenges of supporting regional education programs? How could those challenges have been reduced? ●Most respondents claimed that benefits outweighed challenges. ●DR PUCMM leaders claimed the main challenge was agreeing to a common vision and mission from the start. Each country wanted to do its own thing. The DR’s already developed PUCMM linguistic model eventually provided conceptual leadership, for CARD. ●In Honduras, the main problem was weak UPN leadership, from the start. When a respected & experienced leader/ manager was found, things began to happen quickly. An implementation plan was drafted, each partner was assigned clear technical tasks and responsibilities, budgets were developed and funds disbursed compliant with USAID rules, etc. ●MoE officials interviewed claimed that there weren’t any challenges, except the usual ones of needing to better coordinate the many competing international donor programs vying for the MoE’s attention, all at once. This was especially true in Honduras, where recent budget cuts had removed key posts from the ministry. ●None evident. ●Education in the DR is a highly politicized sector and it is almost impossible to bring about real change without changing work attitudes and the culture in the ministry. ●In the DR CETT training was often wasted because those trained couldn’t apply what they learned, in their jobs. USAID/W may not fully understand the local context and therefore might just have a supervisory role. Mission-level exchange and collaboration might be better. What was sacrificed/lost by working regionally? ●Institutions playing a lead role in the CARD-CETT sacrificed privacy and anonymity but, UPN, PUCMM, UVG, FEPADE, all claimed they gained recognition, funding, professionalism and TA at levels not experienced before. ●Ministries were not fully included in CETT activities so they had nothing to gain, or lose. The MoE in Honduras was now forced by CETT’s closure, to face its own teacher training problem, but was unable to respond due to the current public sector funding ●Since there seemed to have been very little or no, private sector support for CETT in most CARD countries, the IBTCI team was unable to assess this aspect, fairly ●None evident There were no perceived opportunity costs from working regionally. Evaluation of LAC/RSD Regional Education Program: Volume II 76 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey problem. What were the advantages and disadvantages of the different models of program management and coordination (outlined in the background section) – by and for USAID? By and for implementing partners? ●The CARD-CETT model hub–spoke model worked well, once the hub had established a clear modus operandi under stronger leadership. ●The 4 spoke institutions only began to contribute after they were given a clear mandate for specific technical elements and activities. ●The main disadvantage of this model was coordinating and synchronizing activities to coincide with each country’s education needs. ●From the outset, and through the program’s life span, the MoE’s absence meant that CARD-CETT’s impact on national policy reforms was delayed. . ●The MoE officials interviewed didn’t have much to say about the regional aspect of the CARD-CETT model, but commented on the importance of sharing problems and solutions regionally with neighboring CARD countries. ●A regional program like CARD-CETT, funded by USAID, provided the resources needed to encourage this to happen. ●In both the DR and Honduras participants claimed the main CETT advantages were learning to apply a new methodology and teacher training model to the problem of reading and writing. ● USAID funded TA was shared by all CARD members who learned to work together on common regional problems, for the first time in their experience. Management systems promoted communication and coordination among local offices, countries/missions, and Washington Coordination and synergy What worked, and what didn’t work, in the coordination of each regional education program between USAID/W, USAID missions, program partners, contractors, and other key stakeholders? ●USAID DC’s access to Presidential funds and White House decision making meant that when CETT partners needed financial or technical assistance, they quickly received it. ●DC’s authority and leverage from the WH, helped to provide high quality TA through DC based private sector partners like Aguirre (GEM I), Creative (BEST), AED and others, in a timely manner. ●Where USAID Missions, in the DR, Honduras, Guatemala, Nicaragua, and El Salvador were engaged in CETT, some local USAID program funding was used to incorporate selected CETT activities into ●The MoE’s in both Honduras and the DR were not very aware of the difference between USAID regionally funded and run programs and USAID bilaterally funded programs. ●Both countries claimed to have excellent relations with the local USAID mission and were happy to be still receiving TA and ●FG groups were not aware of this aspect. USAID/W was a source of financial and managerial support but was also seen as a distant entity. In countries with missions, USAID was invaluable in terms of coordination and technical support. Respondents did not Evaluation of LAC/RSD Regional Education Program: Volume II 77 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey the USAID country program. ●The absence of USAID country program managers with knowledge of past CETT activities, made finding hard evidence challenging. funding, from USAID. ●They both emphasized the need for more support, especially for education, which was now a national and regional priority. ●Senior staff in both ministries drew attention to the importance of technically proficient education support from the local USAID office. When this happened, quality technical solutions helped to resolve problems quickly. have much input on program partners, contractors, and other key stakeholders. How closely did the regional programs complement and support bilateral mission activities? Was there sufficient coordination? What factors limited and facilitated coordination? How could the CETT program worked better with the bilateral programs? ●In Honduras, where CETT & PREAL activities had been coordinated out of the local USAID mission, regional programs were considered supplementary to the local USAID mission’s own programs, but never fully part of the country program. ●Both CETT and PREAL were considered DC funded and owned. One senior Honduran deputy education program head, who was funded out of DC, was tasked to coordinate CARD-CETT activities throughout the region. ●Informants claimed that her role was unclear, and sometimes conflictive and controversial. ●In the DR, where both the MoE and PUCMM worked closely with the local USAID mission, there was, under the leadership of a previous USAID education program chief, some effort to try to harmonize regional education programs with the national USAID education program. ●Less evidence of this kind of cooperation was observed in the Coordination and communication efforts were mostly within the country. Evaluation of LAC/RSD Regional Education Program: Volume II 78 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey ●Little, or no, memory of CERCA existed within the current mission staff in either Honduras or the DR. Even a retired senior USAID program officer, with some responsibility for CERCA, could not remember the project in any detail. ●When CETT CARD started a multilateral agreement was signed between USAID DC and USAID country Directors in the DR, Honduras, El Salvador and Guatemala to support and implement CETT in the region. Honduras USAID mission, even though USAID Washington had put the CARD￾CETT USAID coordination office, in Tegucigalpa. What synergies and coordination were there between CETT and PREAL? What factors (administrative, technical, etc.) limited or improved coordination and synergy between the regional programs? ●The senior educator responsible for initiating PREAL and CETT that the original intention was not to run these two programs together. In fact PREAL preceded CETT by a number of years, and will continue to receive USAID funding, till 2014. ●USAID DC’s intention was to develop and run complementary programs that would each tackle specific aspects of the LAC education problem separately. ●A prime USAID objective was to encourage support for locally generated research and problem solving initiatives, and eventually less dependence foreign assistance. ●In both the DR and Honduras there was no evidence of synergies between CETT and PREAL, except in so far as CETT and MoE leadership received PREAL publications and attended PREAL meetings and conferences, from time to time. ●Senior informants stated that PREAL was an information sharing and more research oriented academic program that generated publications and other documents which were frequently shared with those in the PREAL network. Sometimes, PREAL sponsored and held international ●Participants were not aware of any synergies between CETT and PREAL in either Honduras, or the DR. Approximately 63% of the respondents stated either: 1) there was no coordination with PREAL, or 2) they did not know of any. Evaluation of LAC/RSD Regional Education Program: Volume II 79 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey meetings. What are the broader impacts of the regional education program? What are the key contributions of the regional education program to improving education quality in the region as well as in individual countries? ●CARD-CETT’s main contribution according to informants, was to begin a professional dialogue between neighboring CARD institutions for the first time. ●A clearly defined set of common education needs in the field of reading, writing and teacher training were identified. ●The well developed common CETT approach, with well designed and jointly produced teaching/learning materials, and a clear teacher training methodology, contributed significantly to primary education quality improvement, in the CARD region. ●In the DR, Honduras, Nicaragua , Guatemala, and to a lesser extent in El Salvador, CETT prodded governments to adopt aspects of the CETT model in their national primary education reform programs, with mixed results. ●In the DR senior MoE officials praised CETT’s contribution to the regional education dialogue, and thought that regional workshops, conferences and meetings funded by USAID, through CETT & PREAL, had made an important contribution both to the internal as well as the regional policy dialogue. ●Governments in the CARD region are now obliged to be more critical about primary education achievement levels, national standards and teacher performance levels in classrooms. ●CETT partner institutions like the PUCMM in the DR and the UPM Honduras, continue to urge their MoE’s to incorporate CETT findings, especially teacher training and inter-active ●In Honduras, CARE International claimed they had adopted the CETT model regionally and were using CETT materials to train teachers in several LAC countries including Nicaragua, El Salvador, Belize and Ecuador , with positive results. ●The regional contribution participants in the FGs lauded was the opportunity to visit and learn from other countries how to tackle common problems. This had never happened before. Tangible improvements in reading and writing from students in Grades 1 to 4. Evaluation of LAC/RSD Regional Education Program: Volume II 80 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey reading and writing learning methods, into their national primary curricula.. Attitudes, perceptions, and policy Considering different education stakeholder groups, including government, public, business sector, etc., as appropriate, to what extent have the regional programs changed attitudes, as well as the visibility of technical education issues, such as early grade literacy, public￾private partnership, etc.? (Provide concrete examples). ●Leaders interviewed in CARD-CETT coordinating institutions in both the DR and Honduras, claimed that there was consistent evidence that CETT had made a real impact on the public policy dialogue and indeed, on public polices in the education field, in both countries. ●In the DR pronouncements over the media by the President suggested that education and education reform had now become a national development fiscal priority, and that education would receive 4% of GNP, in subsequent national budgets. ●The occurrence of five cases of cholera in Santo Domingo, while the IBTCI team was there, led the MoH to campaign for the need to improve health education both in schools and publicly, to curtail a cholera pandemic. ●In both MoE’s all informants were aware of the importance of carrying out national reading, writing and teaching reforms beginning at the pre￾school and through to the secondary levels. ● In both countries recent education reform documents reflected this need and in the case of the DR, the CETT model is recommended and adopted as the national model for training teachers how to teach reading and writing. ●There was little evidence in either Honduras, or the DR, that regional programs like CETT or PREAL were receiving any significant help from the private commercial sector, per se. ●Informants attested to a weak private sector, and no strong tradition of private sector involvement in education, in both countries. ●The opposite was experienced in El Salvador, where the coordinating group was a private sector consortium called FEPADE, which took a lead role in promoting and implementing CETT activities, in that country. ●Participants from both the NGO and public sector, in the DR, attested CETT having transformed the way 1- 4th g teachers were being trained; the way teachers were now being helped in class, the way teachers were encouraged to do action research and in the way teachers now worked together to resolve common problems. USAID provided financial and technical assistance in implementing projects, which greatly benefited ministries and, to some extent, universities. However, respondents had little awareness of the involvement of the private sector and local foundations. To what extent have the regional programs influenced educational ●CARD-CETT has had some impact on education policy & practice in: ●In El Salvador, FEPADE’s community engagement activities, ●In the DR, CETT influenced the national adoption of the CETT There was moderate agreement from respondents about the Evaluation of LAC/RSD Regional Education Program: Volume II 81 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey policies? (Provide concrete examples). In the DR, the CETT Model is now incorporated into the national primary education reform and CETT training methods and materials are being reproduced and applied by the ministry in selected training centers schools run by the MoE. In Honduras, some aspects of the CETT model are being incorporated into the training of trainers’ model which is being introduced by the UPN to INCE, the National Training Institute. INCE trained trainers will go out to Normal Schools and training centers to train teachers in the new CETT methods. In Nicaragua, the government has agreed to incorporate the CETT teacher training model into the national teacher training curriculum which will be applied through the “Escuelas Normales” and by teacher supervisors, at the school level. In Guatemala, the UVG will apply CETT methods and materials through appropriate modules in UVG’s teacher education degree courses. The Team is not aware of CETT having had a major impact on national policies, in that country. In El Salvador, where ongoing disagreements between FEPADE and the ministry is preventing the full adoption of the CETT model. Some success was achieved when the MoE adopted CETT’s community participation approach for their national decentralization initiative, which is now part of the current education reform. moved the MoE to adopt a more decentralized approach for managing schools and education, at the local Education Executive Council (CDE)level. teacher training model incorporation of education programs within their Ministries and national policies and some are in place (e.g. Agreement with Universidad Pedagocica in Honduras). A perceived drawback from some respondents was the lack of local consultation Evaluation of LAC/RSD Regional Education Program: Volume II 82 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey To what extent have ministries of education actively incorporated components of the regional programs into national educational policy or interventions? What factors facilitated/inhibited ministries’ adoption or adaptation of components of the regional programs? What factors of the regional programs have contributed to replication and sustainability? ●The DR CETT leadership at the Catholic University (PUCMM), has committed to working with the MoE to incorporate the CETT Teacher Training model into the national education reform. This partnership was formally between the university’s senior administration, and the MoE. ●Two senior CETT /PUCMM staff had recently joined the MoE. One was now Deputy Minister and the other Director of the National Education Assessment Center. Their influence in senior positions has helped to promote CETT within MoE ranks. ●Both the ministry’s recent adoption of the CETT teacher training model and materials, and the implementation of the “Escuelas Efectivas” program, which has adopted the CETT approach in selected schools throughout the DR. ●The absence of international funding will determine how fast EE schools can be implemented, and sustained, in future. ●In Honduras, there was a willingness to adopt CETT methods and materials at the MoE level, but the current funding problem caused by the political crisis meant there wasn’t funding to implement new activities. In fact, Honduras was forced to turn to other donors and projects for assistance. ●In Honduras leading NGOs like Care International, FEREMA , Save the Children, and smaller local NGOs, have adopted elements of the CETT training model and are applying it in selected schools where they are mandated to work. ●They considered their contribution to be important because of the current national budget crisis & lack of public funding. ●In the DR The Catholic University PUCMM was actively engaged in training teachers and working with schools in selected districts to introduce CETT methods and materials, with considerable success. ● We did not find evidence of strong private sector support for CETT in either country. ●FG participants in the DR were not aware of the regional dimensions of the CETT program, but were well informed about CETT activities and achievements in their country. ●Honduran FG groups did not know much about CETT and similar activities in other CARD countries. An important component in the adaptation of regional programs was the participation of other sectors, particularly universities, government leaders, and NGOs. Capacity and sustainability Evaluation of LAC/RSD Regional Education Program: Volume II 83 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey In what ways has CETT strengthened capacity in partner institutions including various levels within ministries of education? Has CETT built sufficient capacity such that technical interventions can be adequately sustained and/or replicated or scaled up? To what extent has there been continuation/ attrition of personnel trained under CETT? ●Leaders and ex-leaders from the CETT coordinating institutions, in the DR and Honduras, testified to the importance of CETT’s having built institutional capacities in: Financial management & accountability; the management of complex activities with clear objectives & outcomes; the writing of fund raising proposals; the development of outreach and promotional activities; the ability to engage in, and coordinate team activities, both nationally and internationally. ●An important benefit for all participants, was the chance to develop individual abilities and technical skills that would not have been possible without the CETT’s support. ●Some senior DR educators in the MoE who were trained by CETT, prior to joining public service, attested to the effectiveness of the training they received through CETT. ●From the beginning, in all countries except Nicaragua, CETT programs had not been set up to incorporate direct inputs from the MoE. ●Respondents, did not think CARD-CETT had been able to build sufficient capacity within the two ministries visited to sustain what was now being done mainly through partner institutions and local NGOs. . ●In Honduras, public sector officials shared similar views, except that in their case, CETT had started by training a corps of national trainers which had made a mark initially. ●Poor MoE leadership, Outside of strengthening the capacity of ministries and universities, CETT has not been involved with other partner institutions in terms of education program development. Evaluation of LAC/RSD Regional Education Program: Volume II 84 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey lack of funding and weak follow up had dissipated the gains achieved by this and other training received through CETT. . To what extent are partner institutions including ministries of education capable of and interested in carrying on the technical components of CETT? Provide examples of specific components to which ministries have committed. ●USAID local missions in the DR and Honduras were each aware of the contribution made by CETT , but were either unable to act because of the deteriorating political situation and a embargo on USAID and other donor funding (Honduras)., OR, because the USAID education budget had been severely reduced and the demand for dwindling resources on the existing country program, inhibited making any major investment in CETT related activities, beyond those already made, (i.e. in the DR USAID funding had already been committed to the EE project) ●The DR’s PUCMM Center for Excellence in Language Learning which had been recently set up by the MoE and PUCMM, had recently received international support from UNESCO and other donors to continue the work started by CETT. ●PUCMM was committed to continuing teacher training using the CETT model and materials, and to helping the MoE implement CETT nationally through the Moe and USAID, EE project. ●In Honduras, an interview with the Deputy Minister of Education revealed a willingness to adopt aspects of CETT, but the lack of international funding and other In the DR the MoE, the PUCMM and USAID have recently signed an agreement which will adopt CETT materials and methods for “Escuelas Efectivas,” nationally from 2009-2014. In Honduras, in 2009 the Minister of Educ. Signed an agreement with the UPNFM and INCE to train INCE staff to begin national teacher training using the CETT model and materials. In Nicaragua, the MINED has agreed to adopt all the major elements of the CETT training model in their national primary teacher training curriculum and started the process by training an initial 120 multi￾grade teachers using CETT interactive reading and writing ●Both, PUCMM in the DR, and the UPNFM in Honduras, expressed a strong willingness to continue research and TA to CETT both in country and regionally, but with one caveat, the need for more funding. ●Both institutions spoke of a promising new PREAL/ D.C. supported RICETT initiative which would provide funding and support to enable CETT partner institutions to continue meeting regionally, for the next 4 years. ●FG participants in Honduras were aware of private sector contributions from NGOs like CARE and FEREMA, but were not aware of other sources of private sector support, or funding. ●In the DR, the contribution of the PUCMM, a private sector institution, was well known and respected. They cited the university’s work in teacher education, languages and reading and writing. Private sector participation has been limited or at least respondents have not been aware of any. Evaluation of LAC/RSD Regional Education Program: Volume II 85 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey obstacles meant that this could not be done. ●The MoE will, use other sources of international funding to implement parallel primary quality improvement programs like the math learning program currently funded and promoted by the JICA. methods. In El Salvador, with FEPADE, a private sector entity, encouraged the MoE has adopted CETT’s in-class training of teachers approach as the basis of its teacher supervision reform. In Guatemala, the UVG mobilized parental and community support for the improvement of reading and writing skills at the primary level, through its “Creative Classrooms” project which received some support from the MoE. What have been the roles, contributions, and limiting factors of public private partnerships ●Apart from NGO involvement in both countries visited, and the leadership played in El Salvador by a private sector consortium, the team was unable to uncover any consistent long-term private sector support for education during, each short one week visit. ●MoE officials who discussed private sector support claimed they encouraged this, but had not, in the case of the DR, had much success in attracting any real PS support for education. In the DR there wasn’t a strong public-private sector tradition for support to social programs. ●In Honduras, the response was similar, ●In both countries visited, informants testified to not having much private sector involvement in education , because, either there wasn’t a tradition for this (Honduras) , or because the private sector gave only in a token manner to gain public favor for their companies , or products (The DR). ●None that they were aware of. Participation has been mostly limited to ministries and universities. Evaluation of LAC/RSD Regional Education Program: Volume II 86 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey but there the MoE had been successful in attracting local and international NGO support for education. ●International NGOs with international donor funding were now actively engaged in improving primary education quality in a small number of rural, or marginalized urban schools. What is the value-added of providing technical assistance to implementing partners? How has technical assistance to implementing partners in the region added value to the CETT program? Consider technical assistance provided through USAID/ Washington-based contracts as well as through the cooperative agreements. What types of technical assistance provided to implementing partners in the region were most useful? ●In the DR senior managers and professionals lauded the TA received from Washington through Aguirre and Creative Associates, but thought that this had not been frequent enough, or focused on real national needs. ●Very little TA was received through the USAID Country Offices, for CETT activities. ●In the DR, some financial USAID mission support was received for teacher training workshops carried out by national CETT professionals, from the PUCMM. ●On 3 occasions, the CARD-CETT M&E Training Team from the UVG, in Guatemala, did come to carry out training workshops, or to do evaluations, in the ●From the outset, the MoE’s in countries visited were never direct beneficiaries of the regional CARD￾CETT program in their respective country, so they did not receive direct TA from USAID/ Washington. ●In the DR, because of strong prior linkages with PUCMM who led CETT there, the MOE did receive some training assistance through workshops funded by USAID/ Washington funding. ●None evident. ●The FG participants in all four FGs held in the two countries visited, attested to having benefitted from CETT / CERCA training, either through teacher training courses, or through technical workshops carried out by national CETT coordinating entities, ●Some training and international TA was provided by USAID, through Washington based companies like Aguirre, Creative Associates, and AED. Specific workshops such as the teachers' education circles, study circles, and Teaching Assistance helped considerably. Constant monitoring and supervision of teachers and programs also helped maintain continuity, such as those done in Honduras and Guatemala. Teachers were very satisfied with the experience Evaluation of LAC/RSD Regional Education Program: Volume II 87 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey DR. ●In Honduras, informants could not remember who carried out international TA for CETT, but some officials did remember participating in CETT workshops run by US based international trainers. ●In Honduras, the UPN CETT team provided the MoE teacher supervisors with training in CETT methods. Benefits received from such training were never fully taken advantage of by the ministry, resulting in insignificant impact on teaching and learning in most schools. Evaluation of LAC/RSD Regional Education Program: Volume II 88 International Business & Technical Consultants, Inc. C-CETT Matrix: Based on Field Interviews, Focus Groups and Online Survey Data, Collected (November-December, 2010) SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey What are the benefits and challenges of supporting regional program models, such as are used in CETT? Benefits and challenges What were the benefits of supporting regional education programs? What was gained by working regionally? Many small islands without the population and capacity to have expertise in all areas Drawing on experiences and synergies across the region to support efforts in other countries. Universal opinion that opportunities to share ideas among professionals in different countries is very valuable Possible advantages of a Washington￾based project include (a) funding that would otherwise not be available and (b) outreach to countries that could a not be included in a mission-based project A major benefit was the agreement of standards in the teaching of reading and teacher training across the countries in the region. The new literacy standards became a “Bible” for many teachers. “Strengths in unity.” Broke down barriers. There were several references to useful visits to other countries, for example, teachers from Grenada who were introduced to CETT by visiting St. Lucia. Ministry personnel spoke enthusiastically about the benefits of sharing information with colleagues in other countries Some Reading Specialists continued to communicate with each other through emails. When Specialists from other islands visited and observed classes, they often had helpful comments. At the Awards Ceremony there was a good exchange of ideas and resources. In general, they also valued the quality of the CETT program in terms of helping to improve teacher skills and reading for the students ICT component benefited from the regional program, as well as providing benefit for the program – there were inter-island links, although some islands (A) Potential to create a collaborative relationship between the private sector and educational programs (B) Capacity building of teachers who worked in literacy programs for children in grades 1, 2 and 3. Unfortunately, the potential for a mutually beneficial relationship was never exploited. (C) Developing PPP at a regional level with multi-nationals is a much better approach for a regional program than establishing PPP’s at a national level with each private sector company. (D) PPP’s with multi￾nationals is better JAMAICA – The following benefits were identified by the focus groups: (a) availability of resources; (b) provision of training to teachers, (c) improved literacy levels of students in Grades 1 to 3; (d) provision of a framework for parents to get involved in the education of their children; (e) integration of technology into the classroom; (f) enhanced learning environment; (g) opportunity to share ideas, successes and failure with other countries in the region; (h) teacher performance was recognized; (i) teachers were trained to used assessment procedures and enhance their teaching strategies; (j) provision of in-service training and supervision of teachers’ performance by a literacy specialist. Evaluation of LAC/RSD Regional Education Program: Volume II 89 International Business & Technical Consultants, Inc. But these types of visits were limited. “Shared experiences across the countries were extremely important and helped make CETT stand out as a special project.” Many Reading Specialists from different countries who trained together kept in contact through email benefits more than others. In the early years of CETT in Grenada, teachers were exposed to the best practices in St Lucia. Grenada also reported that they benefited from Jamaican experience in training “The project brought together some of the best minds on literacy and education in the region.” “The meeting of the minds with people and experts of different countries was very beneficial to the CETT Model in Grenada.” developed at a regional level and relationship development should be addressed at the regional offices. The CETT Program addressed one of the important issues in Jamaica - the lack of resources in the education system. It improved the literacy level nationally and it improved student performances on the national exams. GRENADA The following benefits were identified by the focus groups: (a) an improvement in reading levels for children from kindergarten to grade 3; (b) improved use of diagnostic/assessment instruments to determine reading levels of students; (c) enhanced and more parental involvement; (d) enhanced collaboration among teachers, schools and education officers; (e) helped teachers to implement a more holistic approach to language arts; (f) empowered teachers and provided them with more teaching and resource materials. The CETT program contributed to more emphasis on classroom transformation, exposure to a wider variety of Evaluation of LAC/RSD Regional Education Program: Volume II 90 International Business & Technical Consultants, Inc. teaching/learning materials, improved school/community relationships. It provided training on best practices in literary, and the sharing of CETT practices with non-CETT schools. ST. LUCIA - The following benefits were identified by the focus groups: (a) contributed important teaching and resources materials; (b) developed resources centers at the Sir Arthur Lewis Community College, class libraries in CETT schools and internet services (c) developed reading expertise by training reading specialists and literacy coordinators;(d) increased interest in reading and parental involvement, (e) increased teachers’ capacity by teaching them how to use new teaching approaches and provided them with diagnostic/assessment tools; (f) provided specialized reading materials to students; (g) provided gender- and culturally-appropriate materials; (h) helped . students develop a love for reading. Evaluation of LAC/RSD Regional Education Program: Volume II 91 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey What were the challenges of supporting regional education programs? How could those challenges have been reduced? First Director recognized importance of full participation at all levels – universities (including UWI/Cave Hill), teacher’s colleges, and schools. Teachers were given recognition and ownership Project must be based in a respected institution Needs to be full-time person in each country responsible for the project; in some countries Reaching Specialists had this role. Suggestion that there be a teacher training specialist in each country. Each country with Steering Committee chaired by Ministry representative Technologies had important role, but not fully developed. In some countries, e.g., Grenada, St. Lucia, and Antigua, the system never worked “Rich interaction among the countries including a better understanding of the respective small variations in education and culture. For example, Jamaica has a slightly different system for early education than the other countries.” “One size does not fit all.” Major difficulty was that the countries are spread out and traveling is costly. The communication system helped out, but did not reach its potential. The distances, especially to the Eastern In general, ministry personnel said they appreciated and valued the communication with staff at JBTE, but there were some exceptions especially in Santa Lucia and Antigua. Not many teachers got the opportunity to visit other countries Reports varied on the turnaround time of the student test data sent to Jamaica. Some said they received results in a few weeks and others reported that it was often some months before they received the information. Some minor concerns expressed about variations among the countries that were difficult to accommodate under a centralized program. Perhaps one example of variation could have been the development of an in￾service rather than pre￾service teacher training model for St. Lucia. Although Jamaica has a different early education structure than other countries. (A) Private-public partnership development never functioned well. The UWI did a very poor job of communicating with the private sector, informing the private sector of project activities and their respective progress. (B) USAID and the leadership of the CETT project did a poor job of communicating with the private sector sponsors. The PPP relationship was not well organized or managed. Private Sector contributions were not sufficiently recognized by the project leadership. Over time the PPP relationship worsened due to poor communication and collaboration. (C) The JBTE never designed or considered the need for a sustainability plan for the PPP component of the JAMAICA. The following challenges were identified by the focus groups: (a) teachers were called away on short notice and had to leave their classrooms, unattended: (b) some reading material was culturally in appropriate, (c) the re-structuring of the CETT’s school program impacted negatively in the early phase of the program; (d) the Reading Specialist provided more support to some schools than to others due to proximity. GRENADA The following challenges were identified by the focus groups: (a) problems in the delay of administering tests and receiving diagnostic test results. This impacted negatively on the timely use of teaching interventions; (b) a lack of resource materials necessary to effectively transform the classroom so that the CETT Model could be implemented appropriately; (c) communication equipment Evaluation of LAC/RSD Regional Education Program: Volume II 92 International Business & Technical Consultants, Inc. Caribbean, provided additional challenges to communication, especially because the wireless ICT system never worked. Also, there seemed to be a lag in perception regarding the status on the project, for example, in Jamaica we heard that St. Lucia has a “start cluster,” but the reality is the project there is not being continued due to changes in key Ministry personnel. USAID/mission. Existing assumption that regional entities have the same vision, goals and objectives is not necessarily true. The Caribbean is a diverse region – culture, language and needs. USAID/mission. The USAID Mission in Barbados was marginalized and never became sufficiently engaged, which would have been important to the successful sustainability of the CETT program in the Eastern Caribbean. USAID/mission. Inability to meet the demands for expansion to additional schools within specific countries; e.g. In Jamaica, where the former government wanted to add 200 schools in FY2007, but this could not be accommodated within the timeframe required. Also, time lost in travelling especially between Eastern States and the timely distribution of the Scholastic books. Although the ICT did provide many benefits in terms of communication and, to some extent, training, there were also several comments that the technology was underutilized in. In addition, some countries receive no benefit Grenada where technical issues were not resolved, in St. Lucia where equipment never reached the college, and in Antigua where the equipment arrived at the end of the project’ and no installation or training was provided. Given the recent advances with the Internet, a new program could do much more. CETT provided computers, and in some cases, communication technologies at the school level. Although this worked fine in some locations, in other places the telecommunications were not used and even the use of computers for data input and storage tended to fall off and, eventually, dropped completely as the project ended and because of the turnover of trained personnel. In some countries (e.g., Grenada and St. Lucia), alternative materials were introduced such as “Jolly CETT project. and network never was adequately installed and made operational; (d) difficulty in transforming the classroom and preparing the teaching/resource materials needed to implement the CETT model appropriately; (e) not enough close follow-up of the implementation of the CETT model in the schools administratively and technically; (f) lack of available resources to implement CETT model appropriately. ST. LUCIA - The following challenges were identified by the focus groups: (a) A major challenge was the amount of time and effort required of each teacher to implement the CETT model appropriately; (b) the lack of materials and equipment that were needed but never arrived; (c) the diagnostic/assessment aspect of the CETT Model did not work well because the pre-test results were not communicated in a timely manner to the teachers so that they could adapt their teaching efforts accordingly; (d) the turnaround time for test scores impacted negatively on the effective Evaluation of LAC/RSD Regional Education Program: Volume II 93 International Business & Technical Consultants, Inc. Phonics” and there may have been some confusion regarding the options for teaching literacy, especially as the project ended. There were some indications of some monitoring challenges in the Eastern Caribbean where technology equipment went missing (Grenada and St. Lucia) and professional books stolen from the resource room (St. Lucia). Although there were successful efforts to get agreement across countries on standards for reading, these changes were not accepted in the Eastern Caribbean (OECS) where the curriculum and exams are controlled by UWI in Cave Hill. This had a direct impact on CETT not being well incorporated in the colleges. The distances between Jamaica and the Eastern Caribbean created challenges in communication. In addition to programmatic and ICT issues, one example may be the still outstanding financial issues between the college in St. Lucia and UWI. In Grenada, it was reported implementation of the program; (e) initially there was resistance toward adopting a student-centered approach to teaching; (f) training activities were important but disruptive to the educational process; (g) the CETT program was not implemented in an efficient manner. Evaluation of LAC/RSD Regional Education Program: Volume II 94 International Business & Technical Consultants, Inc. that some promises were made by UWI to the schools but were not kept. The Education Officers were not able to obtain the student assessment data. It took a long time for the schools to receive the technical support from computers and other equipment. The equipment aspect was very problematic. Geographic distance between the EC states and Jamaica was a problem – sometimes correspondence was not attended to quickly. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey What was sacrificed/lost by working regionally? Little lost or sacrificed Some needs of individual countries not recognized On balance, benefits far outweighed difficulties Haiti was not able to benefit from the C￾CETT regional program Antigua did not participate in the CETT initiative even though they were given promises of participating. The problem seemed to be a combination of insufficient funds and poor management/administration of the CETT program at a regional level. Also, officials said that instead of “giving so much to so few, less could be given to more.” The Ministry in Antigua feels it has good capacity and that it can even administer the test, provided that they get permission (and software?). (A) Communication with USAID/Washington was limited and not effective with the private sector. (B) USAID/Washington and USAID/Jamaica as well as the JBTE at UWI should have developed a better communication, branding and marketing strategy for the project. Many private sector companies were not sure of the “value￾added” of their contributions to the Evaluation of LAC/RSD Regional Education Program: Volume II 95 International Business & Technical Consultants, Inc. They feel the need for some initial technical support from the CETT office in Jamaica. It appears that there was not USAID money during the CETT years to provide assistance to Antigua and that the Ministry had neither the resources to carry out the project on its own. There is lacking even a basic planning process under which the Ministry and College could lay out what it wants to do and the necessary support from within and outside the country. In St. Lucia, there was a feeling that CETT was a Jamaica program and that CETT would have been better accepted if it had been a country-specific initiative. This resulted in lack of ownership. In Grenada, there was feeling that it received lass financial support because the program was regional. CETT project. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey What were the advantages and disadvantages of the different models of program management and Management structure of project was good Good leadership team in the field With only one exception of the countries visited, the ministries were not the key stakeholders, which hindered sustainability. It (A) There could have been more buy-in with the CETT program if USAID/Washington and UWI had been Evaluation of LAC/RSD Regional Education Program: Volume II 96 International Business & Technical Consultants, Inc. coordination (outlined in the background section) – by and for USAID? By and for implementing partners? USAID/mission. Management of C￾CETT by a field-based USAID official contributed tremendously to the successful implementation of the program. A field-based AOTR had a better understanding of the dynamics, synergies and needs of the region and how best to promote and support the successful implementation of the project. should be noted that in one country where the ministry took the lead role (St. Lucia), the Ministry (based on the appointment of a new Permanent Secretary) decided not to sustain CETT. Almost universal initial complaint of teachers was the amount of work required to implement CETT, however, after becoming familiar with the program and also seeing the benefits in the classroom, most complaints went away. Nevertheless, this issue did come up, especially organizing separate group work, as an important sustainability issue. The management team at the colleges worked well – Reading Specialist, ICT Specialist, and Administrative Assistant more effective in their efforts to market the program. (B) USAID/Washington did not adequately raise awareness and market the CETT program. (C) InMed, the USAID contractor for private￾public partnerships, did not effectively communicate with the private sector and engage them more in the CETT program. (D) USAID/Washington did not adequately supervise the work of InMed. (E) JBTE did not effectively communicate with the private sector and thus did not foster their participation. Coordination and synergy SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey What worked, and what didn’t work, in the coordination of each regional education According to project senior staff, initially there were good relationships with USAID/DC, but this deteriorated significantly at the end of Phase 1 with The ministries had little or no contact with USAID personnel, except for the involvement of the mission Evaluation of LAC/RSD Regional Education Program: Volume II 97 International Business & Technical Consultants, Inc. program between USAID/W, USAID missions, program partners, contractors, and other key stakeholders? the change of personnel in DC. Difficult Issues included changing reporting requirements, funding and budgeting procedures, and the development of tests to measure impact. (If the impact evaluation had been designed at the beginning of the project, a different experimental design could have been created that would work with small populations.) Some project staff said that USAID/ DC was too distant and not sufficiently informed about circumstances in the field Ongoing good relations with the mission in Jamaica. The project senior staff felt that it would have been better to delegate more responsibility for the project to the mission. USAID/Barbados upset by lack of their involvement in project Coordinator thought that USAID/Barbados should have had a major role. Some efforts to tie CETT together with bilateral programs in Jamaica, but not too successful No bilateral education programs in countries visited other than Jamaica Other than the top project officials, the perception was that the local mission was not much involved in the project. Remaining financial issues between JBTE and USAID Good relations between the ministries in Jamaica. In St. Lucia, the project did not follow protocol re. visits in Phase 2 In St. Lucia and Grenada, there was little integration of CETT in the college programs Reading Specialists were highly valued in all countries. One comment from a Dean in St. Lucia was that the project would been better conceived as an in-service initiative rather than a pre￾service component in the colleges. Most important benefits included teaching children to read and enhancing children’s interest in books. Others reported strong benefits for teachers Including teachers spending more time with students on an individual basis. There have been clear advantages to hosting the project at UWI and through the JBTE to the teacher training colleges. However, by excluding the ministries as key stakeholders, sustainability has been impacted. Even within the colleges, CETT has not Evaluation of LAC/RSD Regional Education Program: Volume II 98 International Business & Technical Consultants, Inc. and the teacher training colleges, with the exception of Grenada where the Ministry has the lead role and the relationship the college seems dysfunctional. Senior project staff member said that it use a mission not about technically, but did review work plans and deliverables. Initially the project work more with USAID/DC, but by the second phase interaction was primarily with the mission. USAID/mission. Regional projects should be managed and lead by individuals who are based in the region so that stronger and closer relationships can be developed with the regional implementing partners. USAID/mission. Regional program models can provide a significant amount of support and guidance to bilateral programs that are implemented in the same region. USAID/mission. Regionally-based education programs are more effective and efficient than Washington-based programs because those who are responsible for implementing the regional programs are more knowledgeable of the reality on the ground and are better prepared to ensure that such programs are implemented well. Better coordination and communication with key stakeholders because of proximity. Greater opportunities for the development of beneficial synergies with been well integrated into the college programs. CETT materials continue to reside in locked resource rooms at the colleges in St. Lucia in Grenada, and there has been no formalization of a turnover of these materials, supposedly to the colleges. Even when CETT was active, there were problems in some of the colleges in terms of sharing materials and resources with staff members and students at the college in St. Lucia and Grenada. Some hostility among the Language Arts staff in the Curriculum Unit in St. Lucas, supposedly, because they were never involved in the project. In Grenada, MoE benefitted from both USAID’s bilateral program (school reconstruction) and regional program (CETT). There was good coordination. In Grenada, in the beginning the design of the CETT project was perceived as excellent, especially the integrated approach to teaching literacy,; assessment approaches/techniques, Evaluation of LAC/RSD Regional Education Program: Volume II 99 International Business & Technical Consultants, Inc. bilateral programs. USAID/mission. Placing the primary responsibility for managing the C-CETT program in the Jamaica Mission created resentment in the Barbados Mission Likewise, when USAID Washington funded regional educational activities to the Barbados Mission, the Jamaica Mission became upset. The regional offices should be involved in the development of any regional strategic approach such as C-CETT. There needs to be clear delineation of roles and responsibilities between the USAID Washington Bureau and the USAID Mission. USAID/mission. Felt the leadership of CETT had high standards and Errol Miller was able to forge a relationship with CARICOM in persuading governments to sign on to the project. and Action Research. The project was able to mobilize support within the Ministry of Education for the project under the leadership Professor Miller. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey How closely did the regional programs complement and support bilateral mission activities? Was there sufficient coordination? What factors limited and facilitated coordination? How could the CETT program worded better with the bilateral programs? According to project senior staff, initially there were good relationships with USAID/DC, but this deteriorated significantly at the end of Phase 1 with the change of personnel in DC. Difficult Issues included changing reporting requirements, funding and budgeting procedures, and the development of tests to measure impact. (If the impact evaluation had been designed at the beginning of the project, a different experimental design could have been created that would work with small populations.) In Jamaica the lessons learned from the CETT program enhanced the design of the bilateral programs both within and across countries. JAMAICA – The CETT approach has contributed positively to the Ministry of Education’s Literacy 1,2 and 3 in coordination with USAID’s bilateral program. The regional CETT project has impacted positively on the approaches used in bi-lateral programs in primary education as well as programs that focus on numeracy. Evaluation of LAC/RSD Regional Education Program: Volume II 100 International Business & Technical Consultants, Inc. Some project staff said that USAID/ DC was too distant and not sufficiently informed about circumstances in the field Ongoing good relations with the mission in Jamaica. The project senior staff felt that it would have been better to delegate more responsibility for the project to the mission. USAID/Barbados upset by lack of their involvement in project Coordinator thought that USAID/Barbados should have had a major role. Some efforts to tie CETT together with bilateral programs in Jamaica, but not too successful No bilateral education programs in countries visited other than Jamaica Other than the top project officials, the perception was that the local mission was not much involved in the project. Remaining financial issues between JBTE and USAID Good relations between the ministries and the teacher training colleges, with the exception of Grenada where the Ministry has the lead role and the relationship the college seems dysfunctional. Senior project staff member said that it use a mission not about technically, but did review work plans and deliverables. Initially the project work more with GRENADA The CETT program was supported by the USAID school rehabilitation program that took place after Hurricane Ivan. The CETT schools that were damaged structurally or had their equipment/furniture damaged by the Hurricane but were re-equipped and/or re-habilitated with additional USAID funds so that the schools would be able to continue with the implementation of the CETT program. The regional CETT project and the school rehabilitation project were coordinated very well so that the CETT program could be implemented. Evaluation of LAC/RSD Regional Education Program: Volume II 101 International Business & Technical Consultants, Inc. USAID/DC, but by the second phase interaction was primarily with the mission. USAID/mission. Regional projects should be managed and lead by individuals who are based in the region so that stronger and closer relationships can be developed with the regional implementing partners. USAID/mission. Regional program models can provide a significant amount of support and guidance to bilateral programs that are implemented in the same region. USAID/mission. Regionally-based education programs are more effective and efficient than Washington-based programs because those who are responsible for implementing the regional programs are more knowledgeable of the reality on the ground and are better prepared to ensure that such programs are implemented well. Better coordination and communication with key stakeholders because of proximity. Greater opportunities for the development of beneficial synergies with bilateral programs. USAID/mission. Placing the primary responsibility for managing the C-CETT program in the Jamaica Mission created resentment in the Barbados Mission Likewise, when USAID Washington funded regional educational activities to the Barbados Mission, the Jamaica Mission became upset. The regional offices should be involved in the development of any regional strategic Evaluation of LAC/RSD Regional Education Program: Volume II 102 International Business & Technical Consultants, Inc. approach such as C-CETT. There needs to be clear delineation of roles and responsibilities between the USAID Washington Bureau and the USAID Mission. USAID/mission. Felt the leadership of CETT had high standards and Errol Miller was able to forge a relationship with CARICOM in persuading governments to sign on to the project. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey What synergies and coordination were there between CETT and PREAL? What factors (administrative, technical, etc.) limited or improved coordination and synergy between the regional programs? No PREAL activities No PREAL activities What are the broader impacts of the regional education program? SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey What are the key contributions of the regional education program to improving education quality in the region as well as in individual countries? Training and support of teachers key to improved quality USAID/mission. CETT has impacted significantly on the skills of teachers and students and this is reflected in the significant improvement in reading. Teachers have been able to many students from being ‘at risk’ to reading at or above their grade levels. CETT Improved reading of children using a new approach including such things as making use of data for decision-making at policy and implementation levels, competency-based promotion for students, accountability within the ministries, improved school JAMAICA – Some of the positive impacts that the CETT regional program has had are: (a) Improved teaching skills for teachers; (b) increased the importance of the role of monitoring teacher performance; (c) improvement in the Evaluation of LAC/RSD Regional Education Program: Volume II 103 International Business & Technical Consultants, Inc. students are beginning to ‘out-perform’ students at other schools in a number of independent national measures of student literacy and achievement. leadership, and some influence on policy and teacher training initiatives. Continued sharing of ideas among teachers within and across countries. Especially in St. Lucia there were efforts to extend CETT to other schools. The project had an impact on teaching reading. Teachers developed new views on the struggling readers and switched from just calling them “lazy” to students who were "at risk” and they discovered ways to address the challenges of working with underachieving students. Infused classrooms with libraries The ongoing training and the interaction and support of the Reading Specialist were very important, what one person called “on the ground school-based support.” CETT was able to demonstrate positive test results on CETT tests, but not on the national tests, at least in St. Lucia. Highlighted the importance of in-service teacher training and school leadership skills of school principals; (d) schools have an improved “look” and an enhanced image at the community level; (e) the literacy levels at a national level have improved in Grades 1 to 3 as well as the national GSAT test scores; (f) teaching ability of teachers in CETT schools has been impacted positively; (g) CETT Model is being implemented in Ministry of Education schools as well as by the teacher training institutions GRENADA - Some of the positive impacts that the CETT regional program has had are; (a) The CETT model has strengthened the teaching capacity in public schools as a result of the teacher training initiatives and the introduction of standardized evaluation. (b) the teacher training initiatives have encouraged more collaboration between literacy specialist and teachers as well as between teachers themselves; (c) .the Ministry of Education and public sector institutions were positively impacted because many schools Evaluation of LAC/RSD Regional Education Program: Volume II 104 International Business & Technical Consultants, Inc. performance monitoring In Grenada, classrooms were transformed and teachers learned that local materials could be used to make teaching aides. Schools were equipped with reading rooms. Parents were involved too as ideas were learnt from them about literacy and especially their input in helping to design the reading rooms. A special benefit of CETT is that teachers are empowered to use strategies to address the problem of illiteracy taking into account that children differ in their learning capabilities and may take different routes to learning. Teachers are more compelled to explore and devise different ways of helping children to learn to read. Encouraged sharing and communication among teacher and across schools. adopted and implemented the CETT principles such as classroom management, classrooms became active learning centers, and teachers became better prepared to plan educational activities. ST. LUCIA - Some of the positive impacts that the CETT regional program has had are: (a) strengthened the capacity of the Permanent Secretary and the Chief Educational Officers; (b) . teachers are better trained and are supported by a Reading Specialist in literacy-related issues; (c) the diagnostic/assessment system that was introduced helped teachers better evaluate student performance in literacy; (d) the management information/communication system that was such an integral part of the CETT program had limited impact because it never became operational and never reached its potential. Evaluation of LAC/RSD Regional Education Program: Volume II 105 International Business & Technical Consultants, Inc. Attitudes, perceptions, and policy SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey Considering different education stakeholder groups, including government, public, business sector, etc., as appropriate, to what extent have the regional programs changed attitudes, as well as the visibility of technical education issues, such as early grade literacy, public-private partnership, etc.? Offeror to provide concrete examples. Within the ministries, the project heightened the awareness of strong reading programs including the need for teacher training, the use of test scores for designing implementation strategies, etc. There seemed to be no impact in other public or private sector entities. There was little gained in heightening awareness about public-private partnership. The project had impact on such things as diagnostic testing and use of data for teaching strategies, systematic monitoring of interventions, and the need for ongoing teacher training and support. In Jamaica, the pending new licensing requirements for teachers will require portfolios to demonstrate good practices plus classroom observation by principals. The CETT program has strengthened these requirements. In spite of considerable efforts of the project to encourage sustainability, the project is not being sustained in any of the countries visited. Basically, only remnants of the project remain among trained teachers and principals in some of the schools. The listening centers and book corners in the classrooms and the use of books have been The contributions through Scholastic, Pearson, and Book Merchant was approximately $1.2 million. In addition, these organizations supported the participation of master primary school teachers at the annual IRA meeting, but there was disappointment in the feedback from the participants to the project. In the end, some of the master teachers were given books and $1000 cash instead of being sent to the annual meeting. DHL provided free shipping in the beginning Air Jamaica gave a break on shipping materials and for transporting staff Alcoa aluminum JAMAICA – Principals, teachers and students who participated in the CETT program continue to use the CETT approach even though the CETT program is officially finished. Principals still support and teachers still use the active learning approach, the resource and learning materials and students still visit the reading rooms in order to read books. As a result, reading/writing skills continue to improve. The CETT Model was very effective in demonstrating its effectiveness towards enhancing the educational system however very little was achieved vis-à-vis influencing national educational policy. ST. LUCIA - The CETT Model changed the attitudes and perceptions of school principals and teachers regarding how best to influence reading/writing skills of students. Also, the CETT Model influenced positively teachers’ attitudes towards Evaluation of LAC/RSD Regional Education Program: Volume II 106 International Business & Technical Consultants, Inc. sustained. The use of the testing and the grouping of students in individualized groups have probably been the least sustained elements of the program. Some comments that the project could’ have done a better job in demonstrating impact on achievement, especially to ministries that were considering the continuation of the program. Many respondents commented on the changed atmosphere in CETT classes, which were described as more dynamic and exciting than regular classes. provided financial contribution Digicel may have provided telecommunication benefits BP provided some financial support Cable and Wireless provided computers Overall, the project did not do a good job in maintaining communication with the donors As had been promised in memorandum of understanding. The annual meeting of donors was not sufficient. The second COTR not as active as the first. In the second phase of the project, very little effort was invested in developing private-public sector partnerships to ensure that the CETT project was sustained in country. a student-centered approach to education. Nevertheless, since the program has ended there has been a problem trying to maintain a high level of motivation amongst teachers. The lack of financial support and incentives has made it difficult to motivate teachers to continue with the CETT Model. The fact that the MoE has not developed a national policy that endorses the CETT Model has made the sustainability of the CETT principles difficult. The technical capacity of principals and teachers was enhanced but a systematic approach to sustainability was never embraced or adopted by the national government. Likewise, no private/public partnership was ever developed so that the CETT Model could be continued. Evaluation of LAC/RSD Regional Education Program: Volume II 107 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey To what extent have the regional programs influenced educational policies? Offeror to provide concrete examples. Little or no evidence of influence on educational policies Perhaps some influence on the Ministry in Jamaica relating to teaching of literacy in the early primary grades There is no evidence in any of the countries of changes in policy as a result of CETT. Although, perhaps not a policy, in Jamaica the Ministry recognizes the importance of monitoring and helped to shape their own literacy program and the need to hire “Literacy Specialists.” The system for observing and evaluating teachers may become part of the new certification process being developed by the Ministry in Jamaica. In Grenada, the MoE increased the number of hours to the teaching of language arts in schools. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey To what extent have ministries of education actively incorporated components of the regional programs into national educational policy or interventions? What factors facilitated/inhibited Cost, especially for large scale implementation, has been major constraint for ministries considering implementation Perhaps ministries could have considered expansion only to +/- 30% of There is only some evidence of impact on ministry programs. For example, the Ministry in Jamaica has incorporated some of the ideas from CETT into their new Literacy 1,2, 3 program Evaluation of LAC/RSD Regional Education Program: Volume II 108 International Business & Technical Consultants, Inc. ministries’ adoption or adaptation of components of the regional programs? What factors of the regional programs have contributed to replication and sustainability? the most needy schools One suggestion was that the ministries should have been obligated initially, and more-so as the years went by, to make a financial contribution to the project. JBTE learned how the project could be expanded at a lower cost through more efficient use of computers, more focused teacher training, etc. Some senior project officials felt that USAID was hostile towards expanding the project to new countries. With relatively small amounts of money, the project might have made greater progress in expanding the program within the existing CETT countries and to new countries. From the project perspective, CETT more than attained its goals of reducing the number of “at risk” students in literacy skills. The Coordinator reported that after three years, the number of “at risk” students was reduced by 70-75%. According to the Director, St. Lucia was one country where there were strong gains in achievement. (It is interesting to note, however, that there were no benefits of the project in terms of achievement based on the Ministry’s tests.) In some countries, key individuals made a difference. For example, in Grenada Mr. Baptiste, the Ministry representative, was an enthusiastic supporter of the project, and when he left, support declined. As another example, in St. Lucia, the initial Permanent Secretary was a strong supporter, but when this including the new position of Literacy Coordinator. Inhibiting factors include not involving the ministries as key stakeholders, other than St. Lucia, and potential costs for implementation, especially on a large scale. Associated costs include ongoing training and support of teachers, data collection and analysis (student test scores). JBTE develop proposals for sustaining the projects in Jamaica, St. Lucia, and in Grenada but none of the countries, even after extensive negotiations and cost reductions accepted the proposals. The cost was often cited as the major constraint, but there may be other issues as well. For example, we were told that the Permanent Secretary in St. Lucia was not convinced of the benefits of the program based on their own literacy tests given at the end of grade 2 and grade 4. It is not clear to what extent the Ministry test was an appropriate assessment of the effectiveness of CETT. Some CETT teachers have gone for further study at UWI, and reported that as a result of their experience Evaluation of LAC/RSD Regional Education Program: Volume II 109 International Business & Technical Consultants, Inc. person was replaced, interest in the project at that level died. Several individuals commented that “everyone needs to be on board.” A senior project official liked the idea of conducting a study to assess the current status of CETT implementation in the various countries. with the project, they have done better than other teachers. In the Eastern Caribbean, the curriculum for primary education comes out of UWI/Cave Hill, which is different than what emanates from Jamaica. The CETT curriculum was never formally integrated into the Eastern Caribbean curriculum and, thus, there has been reluctance in the colleges to incorporate the CETT program because it is not aligned with the tests that students must take. Use OECS guidelines. Although Antigua was promised support beginning in 2004, there has been almost no progress. Only in 2009 did some materials and equipment arrive (but not installed) and in early 2010 one orientation training for teachers was held, but nothing further. CETT methodology has application not only in the domain of literacy but could be used across other areas such as numeracy session. In Grenada, “nobody is saying anything about CETT at the MoE or in the schools now.” The Ministry is not collecting data or monitoring as it did under Evaluation of LAC/RSD Regional Education Program: Volume II 110 International Business & Technical Consultants, Inc. CETT. Some concerns expressed about MoE micro-management of the project and disappointment when officials didn’t see more financial benefits. Also reported poor management within Grenada after Mr. Baptiste left and between Grenada and Jamaica – inadequate communication, slow distribution of materials, reluctant principals and teachers, lack of implementation plan, college never engaged (seen as a conceptual flaw), delay in results from tests in Jamaica (negatively impacted on instruction), ICT delayed and insufficient training and the network never operated, and delays in transfer of funds. One comments that it might have been better to have the program managed from Barbados Capacity and sustainability SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey In what ways has CETT strengthened capacity in partner institutions including various levels within ministries of education? Has CETT Project staff believes that there is sufficient capacity within the ministries to sustain the program. USAID/mission. Efforts to sustain the program as defined in the Sustainability Although the data is spotty, there is some evidence that ministry personnel have been trained sufficiently to carry out the CETT (A) USAID/Washington and Jamaica and the UWI did not sufficiently engage the private sector (which JAMAICA – Most teachers who worked in the CETT schools are still using the CETT strategies and materials in their classrooms and continue Evaluation of LAC/RSD Regional Education Program: Volume II 111 International Business & Technical Consultants, Inc. built sufficient capacity such that technical interventions can be adequately sustained and/or replicated or scaled up? To what extent has there been continuation/ attrition of personnel trained under CETT? Plan were successful and are being continued until at least by FY2012 in all participating states. In the long-term, this will be continued through the Teacher Training Institutions that will offer Associate and Degree Programs in Reading using the CETT Methodology…a serious Policy breakthrough, accepted by CARICOM and in keeping with the IRA Standards. (Note: this success in sustainability was not strongly supported in the CETT countries visited by the Evaluation Team.) program if it so desired. Change of personnel in some countries has been an issue, everything from high turnover of teachers and the change of senior personnel including the Permanent Secretary. One example of building capacity among the teachers is carrying out action research. In St. Lucia, they produced an impressive publication “Action Research and Dissemination Reports” in 2005 that includes action research of many of the teachers. A Reading Specialist in Jamaica reported that over three years there was a 40% turnover of teachers in her schools. Although all was not lost with these teachers because they took with them new skills, but the high turnover clearly had a negative impact. School principals learned to take a more active role in supervising and supporting teachers Although the CETT program was implemented through the colleges, in general, CETT was not well integrated into the college is very interested in issues of literacy). (B) USAID needs to develop a different approach to that used in the CETT program to more successfully engage the private sector. WI never focused on the development of a sustainability plan that promoted to maintain a print-rich classroom environment. Even though the CETT program has officially ended, teachers who developed an enhanced capacity to provide literacy programs are maintaining the CETT principles. As a result, many students are still benefitting from the CETT program unofficially. Unfortunately, the sustainability component of the CETT program was poorly conceptualized and coordinated. Evaluation of LAC/RSD Regional Education Program: Volume II 112 International Business & Technical Consultants, Inc. programs and curriculum In Grenada, one person commented that the capacity of the MoE would need to be further developed in how to manage, supervise and evaluate such an approach. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey To what extent are partner institutions including ministries of education capable of and interested in carrying on the technical components of CETT? Provide examples of specific components to which ministries have committed. Although lack of finances was often cited by the Ministry for not sustain the program, it was also noted by senior project staff that there is a culture within ministries not to sustain projects that have been externally funded. This was also a period of acute financial problems within the ministries and, thus, not a good time for new investments. Likewise, the private sector was not in much better shape financially to contribute. A UK produced package called Jolly Phonics has been introduced into several of the countries; it seems to be well received. In general, the ministries have the capacity to implementation CETT, perhaps with minimal outside support initially, if they so decided. In all the countries visited, the ministries lacked the will to sustain the program. Antigua was a special case, because the program never started even though participation began in 2004. The Ministry seemed to lack the will and necessary funds, and the project could not provide adequate funding and, in fact, only provided minimal support near the end of the project. No communication from 2005-2008. The project visited Antigua and belatedly provided some equipment and books, but nothing other than an orientation meeting for 25- JAMAICA – The technical components of the CETT program are being carried out within the MoE schools based on the initiative of individual principals and teachers. The MoE has not developed a ministerial policy that promotes the integration of the CETT program technical components across all schools or across different subject areas, e.g., mathematics, numeracy. Those individuals who are informally continuing with the CETT Model have become convinced of the value-added of the technologically introduced and the assessment procedures. Such individuals are encouraging that the CETT Model be expanded to other subject areas. The CETT Program was Evaluation of LAC/RSD Regional Education Program: Volume II 113 International Business & Technical Consultants, Inc. 30 teachers in early in 2010 was conducted. Antigua received $20,000 in telecom equipment, but never installed. UWI/JBTE developed proposals to provide support to ministries, but this initiative not successful. The most optimistic continuation of CETT is Grenada, where some components are still being implemented in some schools. 20 schools have been given CETT school materials and the Ministry official is helping to train them on how to use the CETT materials. This is more than the original number of schools that participated in the CETT program. Yet, even here, the Ministry has not taken the lead to promote CETT and some of the schools have reverted to the “whole classroom” approach CETT model with individualized groups. “There is a need at the national level to find a “champion” of the CETT model so that it continues both at the college and MoE.” an eye opener for the Government. The Government started dialogues around and implementation of the Literacy 1,2,3 program which will soon be expanded to Literacy 4,5 and 6. Nevertheless, the MoE did not fully support the CETT program. They did not participate in the training workshops and did not fully support the program. The MoE presently has Literacy Specialist employed in each region of the country. These individuals can provide technical support for those schools that have decided to continue with the CETT Model unofficially. ST. LUCIA - The MoE is not continuing with the CETT Model officially. Those principals and teachers who were trained in the Model and implemented it in their respective schools are continuing to implement it but the MoE has not adopted it officially as a pedagogical approach that should be implemented across all schools. Such individuals are encouraging that the CETT Model be expanded to other subject areas. Evaluation of LAC/RSD Regional Education Program: Volume II 114 International Business & Technical Consultants, Inc. The practitioners believe that the action-based research component of the CETT Model helped make it more effective because it allowed teachers to experiment with various approaches that the reading specialist proposed in order to enhance the literacy skills of targeted students. Additionally, the Division of Teacher Education at Sir Arthur Lewis Community College has incorporated certain components of the CETT Model into their teacher training curricula and program. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey What have been the roles, contributions, and limiting factors of public private partnerships? Initially, there was good support from the private sector through the initial work of the contractor in Washington INMED?? Who made arrangements with Scholastic Inc. to provide 400,000? books to the schools over several years. In addition, peers and Publishing also provided books as did the local company Book Merchant in Jamaica. The books were highly appreciated in all locations and made a major contribution to the project. Nevertheless, there is need to develop indigenous reading materials In addition, there were contributions from other companies including Air The ministry personnel at all levels universally acknowledged the benefits of the books contributed to the schools and the teacher training colleges. There was some concern about the cultural appropriateness of the books, since they came from the United States, but overall this did not seem to be a large factor and, in fact, the more recent books seem to be more appropriate given greater GRENADA The private sector has provided funds for reading corners in classrooms as well as contributed additional materials to the schools. The private sector has also provided support in fund raising activities that have been organized by the schools. Evaluation of LAC/RSD Regional Education Program: Volume II 115 International Business & Technical Consultants, Inc. Jamaica, DHL, Illumet, etc. These were useful and much appreciated. Overall, other than the books, the involvement and contribution of the private sector was not substantial and certainly only a small fraction of the initially anticipated contribution of $20 million. UWI made efforts to get private sector support, but acknowledged that this type of effort was not a familiar area to the University. It was also reported that the individual assigned to this task should have been full-time, but in any case, may not have been the appropriate person. USAID/mission. JBTE invited several companies to support the program and each pledged a certain sum of money to the project. The target of $450,000 was far surpassed and some companies pledged support up until 2011. The sustainability plan had to be approved by LAC/W, but the first draft was not accepted, as USAID/Washington was not convinced the plan was realistic – they found the initial plan to be ‘confusing.’ LAC/Washington asked for additional evidence to confirm the project could be sustained such as pictures and receipts of books. They felt the local USAID mission and JBTE could not take anything for granted. representation of minorities. One suggested in Jamaica there was to include the National Parent Teachers Association. The private sector could help finance programs if they had been better integrated into CETT. The private sector should not be perceived as a “cash cow,” as they have their own ideas to solve problems and could have been used to advise on management and mobilizing communities. Regarding the wireless WAN in 2005, the ICT team submitted an application to obtain a frequency license from Extel (St Lucia) and NTRC (Grenada). The application required the signature of the MoE and the application was presented to the PS at the MoE and to another senior MoE official. The application sat in the MoE for 4 months and was not signed. The project literally died from there on. In the meantime the private sector (LIME and Digicel) had offered support to the project by allowing CETT to use their communication towers at a reduced cost. Based on the lack of cooperation from the MoE in obtaining the license, the project lost a valuable Evaluation of LAC/RSD Regional Education Program: Volume II 116 International Business & Technical Consultants, Inc. opportunity in getting the private sector on board. Although there was almost universal appreciation of the contribution of Scholastic books, there was the following comment from Grenada: It may have been better to work with a national company for the provision of books. There are many national textbook companies eager to work with the public school system. However, the MoE and/or the national CETT manager never approached them. As a result, when Scholastics stopped providing materials, all support in the area of materials also stopped. In Grenada, “There were problems throughout the project getting the Private Sector involved. The only private sector entity that remained involved and contributed positively to the CETT project was Scholastic Books What is the value-added of providing technical assistance to implementing partners? SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Group Online Survey How has technical assistance to implementing partners in The project staff did not recall much in the way of technical assistance, although they made reference to a very The ministry staff expressed great appreciation for the JAMAICA – Teachers became very motivated by the improved results that Evaluation of LAC/RSD Regional Education Program: Volume II 117 International Business & Technical Consultants, Inc. the region added value to the CETT program? Consider technical assistance provided through USAID/ Washington-based contracts as well as through the cooperative agreements. What types of technical assistance provided to implementing partners in the region were most useful? useful workshop on brain-based teaching and learning, evaluation and testing (of marginal value), and efforts (unsuccessful) to write funding proposals. One Reading Specialist also recalled teaching a second language to children and using music to teach phonics. They highly valued the interregional meetings that involve all three of the CETT programs. These were well run and provided an opportunity for shared learning experiences and to work out some of the issues relating to USAID. technical assistance provided by the project, especially the training of the Reading Specialists and, through them, training of the college personnel and the classroom teachers and principals. There was little recollection of international consultants. The annual meeting of teachers was highly appreciated. One respondent said that the most useful assistance was the delivery of training modules on literacy and technology-related technical assistance. One respondent said that the EMIS system was the most useful despite mismanagement and flaws in the system. In Grenada, “Most useful was the workshops and consultants from UWI/JBTE (Ernandez-Spencer and Warrican). Publications were good and non-CETT schools also received some books. Workshops held in the other countries were the most useful; felt they got to learn how the CETT model was to be administered from St Lucia… diagnostic testing was the most useful were generated for students who participated in the CETT program. The CETT program provided the principals, education officers and teachers with the technical skills needed to improve educational performance. Teachers were motivated by the benefits of CETT to students who achieved better results. GRENADA The technical assistance provided to the implementing partners improved the capacity of the school management and technical services in such areas as educational planning and system of accountability related to student learning. At the same time, the technical assistance provided teachers with enhanced skills on how to use differentiated instructional approaches based upon the needs of the students. The technical assistance also helped teachers in their classroom management techniques as well as time and resource management in the classroom. ST. LUCIA - The technical assistance provide by the CETT Evaluation of LAC/RSD Regional Education Program: Volume II 118 International Business & Technical Consultants, Inc. technical assistance as the results of this test could be used to improve children’s reading and writing. “ program enhanced management practices used by public education institutions. The schools are using improved grouping strategies, better data collection techniques as well as instructional interventions and classroom management techniques. Questions for USAID officials only What are the major lessons learned from the C-CETT regional program? Regional programs have tremendous potential when they address common regional and/or sub-regional problems such as those related to education and literacy. When designing a regional or sub-regional program, one must carefully analyze and study the similarities across the region or sub-region Numerous bilateral programs across different countries cannot address regional problems. hat can be addressed by a regional approach. It is better to address common problems across a region with a regional approach. Even though one contrary example of this approach is PREAL, which is a regional initiative, but it has done very little in the area of policy dialogue for the Caribbean. Nevertheless, a regional approach is potentially much more effective and efficient in bringing about regional change that numerous bilateral initiatives. Would you recommend that USAID consider future Washington-based programs in education? If so, what types of programs would be most appropriate for your country? The regional approach employed in the C-CETT program is the best way to implement a regional program. Bilateral programs can benefit more from regional programs that are implemented in the same region than other bilateral programs. Regional and bilateral programs that are jointly based in the same Mission, e.g., USAID Mission Jamaica create an opportunity to enhance communication, synergies and coordination, which is what happened in Jamaica. What factors need to be considered when designing a project that will ensure success of a regional program? The manager and overseer of such a regional program should be based in the region and should have the opportunity to work in an environment where similar bilateral programs are being implemented in the same technical areas – such as literacy. Also, the manager and or overseer of a regional project should be in the same country where the regional implementing partner is located as in Jamaica – C-CETT USAID program manager in the same country as UWI-Mona. Evaluation of LAC/RSD Regional Education Program: Volume II 119 International Business & Technical Consultants, Inc. Andean-CETT Matrix: Based on Field Interviews, Focus Groups and Online Survey Data, Collected (November￾December, 2010) SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey What are the benefits and challenges of supporting regional program models, such as are used in CETT? Benefits and challenges What were the benefits of supporting regional education programs? What was gained by working regionally? Ecuador did not have a state￾backed educational model. In Peru, the National Education Project had no political or technical support. The Centro Andino project put in the public agenda the issues of policies, models and national educational curriculum in basic education, the importance of training and the quality of classroom teaching, the definition of new standards and indicators to measure the quality of literacy teaching and learning, the importance of support, discussion and information to improve awareness and decisions for educating the poor. The transfer of management skills and learning to the classroom is of great value and enhanced through the exchange of experiences. The agreements between the MoEs in Bolivia and Peru, which formally established the recognition of the Reading Schools and Centro Andino as the focal points of teacher training and literacy learning. The involvement of families as allies, especially in Ecuador where mothers were the main vehicle for social recognition and the growth of learning. The MOE of Peru and Ecuador recognized the CETT project’s guidance on the core issues that The Ministries of Education of Ecuador and Peru won the opportunity to work on the education component with a holistic perspective that included policies, methods of teaching reading and writing, assessment, training, certification, accreditation of the teaching profession, management and administration of regional and local educational development. The “School for All” model is endorsed by the official program policies. It goes beyond just improving the processes of reading progress and writing; it seeks to improve the classroom environment, classroom setting, teaching methods, discipline, evaluation, learning, working with parents--everything is for everyone including the schools’ link with business. It is really a model of the ‘school for all,’ to promote development. Understanding that the sustainability of changes in the classroom and teacher performance are not achieved in the ministries but in the classroom through systematic presentations and evaluations. Overall Centro Andino achieved: the first benefit was to refine the methodology of teaching reading and writing in the first 3 years of There is evidence that the private sector has gained awareness and "bought" the educational theme, inserting it in corporate social responsibility policies; organized, invested resources; and managed high-impact social and educational programs in many communities, especially the isolated rural poor. For the 3 countries, the awareness approach strategy impact in the private sector was successful. In Peru CONFIEPP strengthened relations with the mining sector. The strategy has positioned itself and has been recognized. The strategy involved bringing the company to schools to see the achievements; you could see that companies need to understand that the change will promote the qualifications and education of the future working force. Training programs, With the educational sector, the FG in Ecuador and Peru revealed an incredible motivation, willingness and ability to work of teachers and specialists, even beyond the terms set forth in the CETT model. There was no impact on the private sector because CETT was only for the public sector, but teachers in public schools already knew other teachers and took CETT's model to private schools, such as the German School that requested this model. Congresses, open courses, and education meetings between teachers of public and private schools made CETT known. In Phase 2 of CETT with the support of the sustainability component it could bring outside speakers to give courses and a good number of private sector teachers were The most evident benefit of CETT was the training and development provided to teachers. This in turn led to improved performance among students and a change in attitude and perceptions. Evaluation of LAC/RSD Regional Education Program: Volume II 120 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey comprised the technical consulting units. In Ecuador it was clear the learning-model fir reading, writing and reading comprehension, not only as a result of spontaneous actions but as a fundamental part of an affective-emotional process and constant encouragement to the child, in which the alphabetic codes have meaning. The school should complement the knowledge learned or not learned at home. Bolivia’s NUR University and CETT, at the end of 2008, certified 584 trained teachers through the program “Read and write in the first cycle of primary school,” from 60 education units from different municipalities and districts of Santa Cruz city, including El Torno, Pailón, Warnes, Yapacaní, Montero, and city Districts I and II who completed a training process to develop skills for the successful teaching of literacy in lower primary school. primary school, overcoming the technical and methodological differences between teams from the three countries. The second benefit: achieve leadership in the model and methodology of training and performance in the classroom. The difference caused a shift in how to assess and strengthen the performance of teachers in the classroom. The challenge consisted in working hard to break the culture of traditional model-based teaching and the type of supervision, changing that for support for the classroom teacher, tracking, observations, meetings with teachers to discuss the new challenges and resolve needs to adapt the methodology to the new requirements--this was the difference. We worked through shared learning groups, with modeling in the classroom to relearn how to conduct the class sessions. There were meetings to exchange experiences between the teams of the 3 countries. Adjustments were made, improving the guides and materials for teachers, to work with them. The benefits in terms achievements were: In Ecuador there was not a plan or methodology for training or teaching reading and writing before CETT. With CETT, municipalities were strongly engaged in some rural locations, such as Otavalo with a program for training teachers and supported by the AECI and FEPP. In total 617 principals activities, meetings and workshops for teachers, educational materials, exhibitions, reading contests, and writing were activities in which many private companies collaborated and continue doing so, using the CETT methodology. able to attend. Evaluation of LAC/RSD Regional Education Program: Volume II 121 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey were trained, 2,284 teachers and 69,108 children. In Bolivia, children learn the alphabet in their first semester and by the year’s end they are reading and writing short texts. In the second and third year they have higher levels of reading comprehension and produce different kinds of texts; they have also developed their speaking skills. Teachers who participated in the program say they have renewed their teaching practices and have gained awareness of the need for lifelong learning. In Bolivia, by the end of 2009, the teaching skills strengthening program “Read and Write” in primary school’s first cyc involved two types of training modes: classroom and online distance training through the Internet. The overall results in 6 years are: 581 directors and heads of schools have benefited, 3,187 teachers have been trained and 105,369 children improved literacy and reading comprehension. In Peru UPCH has had a working relationship with MINEDU since 1996, assisting in the processes and methodology of teacher training for primary and secondary through PLANCAD, National Teacher Training Plan. CETT successfully developed between 2003-2009 the reading and writing skills in a wide range of children in schools in rural, urban and poor urban areas of Lima, Callao, Piura, Cusco and Ucayali. 163,809 children benefited, 5,777 school teachers Evaluation of LAC/RSD Regional Education Program: Volume II 122 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey were trained and 983 school principals. Training activities were in two modes, classroom and online mediated by ICTs. What were the challenges of supporting regional education programs? How could those challenges have been reduced? Initially, UPCH, UASB and NUR member Universities went through a long process of technical discrepancies. They disagreed with the methodology. Harnessing the skills and experiences related to education, universities created effective tools for improving the quality of the teacher training processes and enhancing the quality of literacy teaching in basic education in rural and marginal urban areas. The struggle and thorough discussion on the particularities of each country, the country's vision, a regional ideology, social development goals, problems and priorities in education as part of the model intended to be unique for the three countries. Although the project has impact at the teacher level, schools, children, communities, not all [other] sectors have been covered, also more experience in classroom training has been achieved. Much more could be achieved if it first extends through the use of ICTs and if a complete CETT model would have been developed in an UGEL or a Canton and then evaluated and replicated. Recovered and validated the basic model that is fundamental in teacher training and in learning to read and write: moving from understanding the knowledge of The biggest challenge was managing the tensions and methodological differences between specialists in the 3 countries. No consensus was reached on the approach and pedagogical model for the model for the 3 countries elaborated by the UPCH. This forced the hiring of an international consulting firm to select experiences in LA and the world that endorse the strength of the proposed design. This situation led to resignations of specialists in UPCH and forced changes in team specialists. A lack of broader involvement of ministries was a weakness due to some subjective factors, specifically the political alignment of the proposed regulations or lack of clarity in the instruments. The ability to listen, and mutual respect for national identity and shared learning were able to reduce tensions and advance in the national perspective but not the regional. Every country made its own CETT model.. In Ecuador there was a need for research and creating a research unit in the UASB was proposed with the idea of systematizing and producing knowledge, but a greater involvement of the Ecuadorian MOE will be needed. According to IPEBA, after the CETT experience certifying the How to maintain dynamic technical relationships and involve the MOE; in the case of Peru and taking the experiences of the private sector as developments that can be validated as contributions to educational development. Natura and PlusPetrol sent their materials designed in the dynamics between teachers and families with children to enter and validate the content with the policies and pedagogical guidelines of the EBR program, but the MOE never understood the logic of consultation and disseminated materials between specialists without a goal. The FG with other technical specialists from the MOE of Ecuador and Peru revealed critical aspects of internal project management in CETT, conflicts with the participation of technicians and teachers, problems of coordination and inter￾sectoral coordination and weakness in the management of regional and local educational development. From one private sector respondent: The program was oriented towards teachers and was well-designed. But it had challenges among the leadership at the national and local levels. Evaluation of LAC/RSD Regional Education Program: Volume II 123 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey "doing" and practices: the daily events, the culture as a knowledge base. After the meeting between the 3 countries in Santa Cruz, Bolivia, in which a single training methodology model was not accepted, the roles were clear. A functions and responsibilities manual was made and technical country and regional responsibilities were clearly established. Technical responsibilities were defined: Bolivia would be responsible for developing materials and strategies to use new ICTs for teacher distance training, Ecuador had the evaluation of learning and applied research and Peru the training and methodological framework for reading and writing and overall project coordination. competence of teachers and educational institutions must continue, also common standards, accreditation to increase the quality of education, teacher performance and quality in the management of educational institutions. How to empower individuals and educational institutions with the capacity and leadership to achieve progress and legitimize change and contribute to their sustainability. USAID projects should aim to integrate into the MOE culturally and technically to meet a common goal and not in practice differentiate by its brand ... "this is the from the USAID project, this is not" What was sacrificed/lost by working regionally? In June 2003, an historic meeting was held between the three countries in Santa Cruz, Bolivia, on the training component. The result: the evaluation matrix was not approved, with the resulting model that every country would define its own. That was a big turning point in the regional coordination of CETT-Andino The leaders did not use all their available resources for better integration with policies. Excessive bureaucracy and adherence to the rules did not provide a flexible and comprehensive framework to build a national and regional model. Many private initiatives have had an impact on a small scale and have achieved their own financial sustainability, building their own capacity and making their own initiatives through agreements with the Centro Andino as in the cases of Peru and Reading Schools in Ecuador. These models are not fully known by the MOEs and not linked to the synergy of the project. There were no stated losses from working regionally. What were the advantages and Universities have opted to move beyond the empty areas or lack of coordination with the MOEs in In Peru and Ecuador the low technical capacity of specialists to understand the model and In Peru and Ecuador, the groups of teachers and specialists have One advantage was that different Evaluation of LAC/RSD Regional Education Program: Volume II 124 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey disadvantages of the different models of program management and coordination (outlined in the background section) – by and for USAID? By and for implementing partners? Ecuador and Peru; things went well and the achievements and impacts on small scale were visible and significant qualitative changes replicated in models of teacher training, teaching of literacy and reading comprehension. their role, and changes in practice by the persistence of vertical memorization learning methods, repetitive classroom teaching, lack of follow up training, without an objective and reasonable assessment methodology, without clearly setting out in the classroom teaching skills, and without support in the early years was an obstacle to progress in terms of institutional achievements and further refinement of the model. The directors of the Ministries bureaucratized the model and participation of their representatives. A way to evade responsibility was the typical message: consult Escuela Lectoras / Centro Andino, they know. taken advantage of the benefits of the training processes. Opportunities for the production of educational materials appropriate to each cultural context have been created. Follow-up teacher performance pilot programs have been put into operation and proposals have been designed for that purpose that still have to be validated. management models gave autonomy in addressing designing programs issues (e.g. teacher training). Coordination and synergy What worked, and what didn’t work, in the coordination of each regional education program between USAID/W, USAID missions, program partners, contractors, and other key stakeholders? Having a methodological and financial instrument such as CETT to manage changes in basic education and meet the expectations to improve the quality of learning in reading, writing, reading comprehension and teacher training were the major aspects that worked that supported most of the achievements.. Ecuador remained active in the CETT project without an agreement between USAID / WDC and USAID / Quito and without a local USAID specialist. Develop CETT’s role as coordinator and facilitator for the proper entry of the project in each ministry. Maintain a dynamic cash flow and In USAID / W and USAID / Peru both the technical and financial assistance of USAID has had good acceptance, there is a tradition of long years of bilateral cooperation with USAID / WDC. They are considered partners, but there is no awareness of the differences and what it takes to have different forms of cooperation In Ecuador CETT activities were coordinated from Lima (UASB and UASID / P) and UASB felt the delay in funds from WDC; when cash flow came from USAID / Peru the situation improved, but the technical assistance was always absent. Constant communication among managers at all levels was very helpful in scenarios (e.g. periodic evaluation in Ecuador, Peru, and Bolivia) From one respondent only: the different people involved made it difficult to accommodate the various perceptions and ideas. Evaluation of LAC/RSD Regional Education Program: Volume II 125 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey accompanying public investment programs, especially if finances were managed from WDC, which improved when funds from USAID￾Peru were used from Lima for UASB- Quito. For USAID / W and USAID / Peru, the possibility of being a bridge of synergies and the focal point and convener for regional or national dialogue was lost. WDC USAID pressure to follow CETT’s original design was strong. Ecuador negotiated that variations specific to the country were accepted and this was understood for Peru and Bolivia. Bulletins from Aguirre / JBS consultants have supplied in great part this knowledge, the understanding and exchange of information on progress, and challenges in progress of the CETTs. How closely did the regional programs complement and support bilateral mission activities? Was there sufficient coordination? What factors limited and facilitated coordination? How could the CETT program worked better with the bilateral programs? In the case of Ecuador the CETT was coordinated out of the Quito USAID mission; there was no agreement between USAID/WDC and Ecuador. Technical operations and financing were through USAID / Peru. In Peru the first 18 months of CETT were directed from WDC. In 2001 the education section was initiated; in 2002, CETT was based on the establishment of a Bilateral Agreement signed between UPCH and MINEDU without USAID / Peru; and in 2003 an education officer was hired to coordinate CETT Peru , Ecuador and Bolivia.. None of the three countries managed to have the MOE to work closely with the USAID missions. The alliances were political and USAID / WDC and USAID / Peru respected the decisions of the universities and approved developments in the context of national characteristics and priorities; it can be said that they built and advanced on a mutual basis of trust and respect. The working model included strategies involving municipalities, and regions. Lima Province and Piura were covered. C.A. was meeting with regional directors to select areas where no one had worked before, the poorest, and a link was established with the regions, then with the UGEL and municipalities. "In Paita, Piura, no educational interventions had been implemented for over 20 years, when we went there, schools were abandoned." With the regional government of Piura, UGEL of Sullana, Paita, UGEL 04 North Lima, Coronel By the year 2007 in the second phase of CETT in USAID/Peru was looking to give the project sustainability. The term usually refers to the continuous flow of the benefits resulting from the development activities towards the target population beyond the term of a project, including challenges to obtain and manage resources at the same time they reduce their dependency on a single funding source. USAID raised the need to seek local partners, to decentralize the management and hiring a senior technical assistance local and regional, but this did not succeed. USAID did succeed in finding local partners in the private sector with similar goals that came forward to Participants noted that many relationships between USAID, UASB, UPCH had a conflict and dispute tone. This came from not very clear roles and interpretations without objectivity of the agreement. In some cases the presence of a lawyer was required to advise the institutions and clarify the scope and purposes of the rules. Though communication was generally good, there were bilateral mission activities that had challenges because of the lack of communication between/from USAID and the countries. Evaluation of LAC/RSD Regional Education Program: Volume II 126 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey Portillo, in Ucayali Sullana Municipality, smaller district municipalities like Colan in Paita, municipality of La Huaca, the Bureau of Consultation Puente Piedra, Lima. It was envisioned that the way to make a viable Project was by inviting MINEDU. All specialist in UGELES were invited to workshops, those from the central area did not attend due to several reasons--the ministry had no funds for travel expenses, the ministry always said their role was as “watchmen” over the project and not as participants, also many specialists had been changed and were not aware of CETT and were not invited. C.A. has been involved with the UGELES. In Sullana, Piura they went along with the specialist, they really felt involved. finance the project. In Peru the local partners were Plus Petrol, Natura and Employers for Education; in Ecuador it was the Swiss Cement Company. What synergies and coordination were there between CETT and PREAL? What factors (administrative, technical, etc.) limited or improved coordination and synergy between the regional programs? Most responses showed knowledge from the leaders, officials, teachers, and specialists and knowledge of links between the two projects but there were clear differences between them. But Raquel Villsana ,the Andean CETT Executive Coordinator in Peru, Ecuador and Bolivia remarked that in Piura a Canadian university was promoting a training school for teachers. Centro Andino shared all their reading and writing methodology and all materials with them. Peru and Ecuador did not have synergy between the two projects; each identified their own role but both valued the status reports produced by the Faro Group and GRADE, and in the case of Peru helped to create standards for the curriculum. No known synergies between these projects; PREAL has USAID assistance and CETT has ended. Participants from Ecuador and Peru were unaware of and did not mention PREA-CETT relations. Forty seven percent of the respondents stated not knowing of any coordination with PREAL while 52 percent said there were none. What are the broader impacts of the regional education program? What are the key The method of support in Ecuador The teacher training programs In Peru, private groups Participants reported that The Reporte Escolar was Evaluation of LAC/RSD Regional Education Program: Volume II 127 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey contributions of the regional education program to improving education quality in the region as well as in individual countries? ensures the quality of learning and teaching performance to the extent that the trainers live in the communities doing follow-up for 3 months and developing demonstration classes. This also assures the achievement of quality standards.. In Ecuador the recent Education Act as part of the country's new constitution integrated bilingual education with Spanish and indigenous education, in accordance with the characteristics of the country. Reading Schools in Ecuador became state policy for teaching literacy. At UASB certificate programs have been created in areas such as language and communication, management of reading texts and writing quality was recognized by the Ecuadorian MOE. Ecuador transferred the responsibility of intercultural training to the DINEIR of the MOE. Inside the MOE a bilingual education area has been created as a response to Reading Schools. The literacy teaching process was changed from a oppresive system in which the child “did not learn” to a creative, affective , stimulating and motivational process for the children. In Peru Centro Andino began working in a differentiated, focalized way with teachers from the lower grades and has accelerated the programs in these levels. They now work with budgets based on their results, there is an improvement of the initial learning in the first 5 years. The budget of the Republic (Ministry of Economy and Finance) of the last two years has reserved an amount to support basic education. It is an achievement raised to public policy by the work of the Centro Andino. PELA-Special Program Learning Achievement--which works on communication, is based on the training strategies of Centro Andino. Also on the same basis teacher training, PRONACAB, was created. There is a second training program in mathematics and communication, and the Centro Andino has been hired recently by the MINEDU to use the CETT methodology and finance activities in selected poor communities in the coastal mountains and rainforest. In Peru's Amazon communities, PlusPetrol grants scholarships for the training of native teachers using the CETT methodology. the shared learning between teachers, fairs and competitions to integrate families and communities and to include new skills in the educational profile were excellent contributions. During the Ecuador CETT, trained school teachers with training strategies considered valuable and innovative had no MOE supervisors or managers present. The curriculum model was included in the updated curriculum in language. an important tool in improving teaching skills and techniques and the results showed in better student performance. Evaluation of LAC/RSD Regional Education Program: Volume II 128 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey offer the program in 4 semesters. 210,450 primary school teachers attended Attitudes, perceptions, and policy Considering different education stakeholder groups, including government, public, business sector, etc., as appropriate, to what extent have the regional programs changed attitudes, as well as the visibility of technical education issues, such as early grade literacy, public￾private partnership, etc.? (Provide concrete examples). In Bolivia CETT conducted campaigns to sensitize the different sectors of society about the importance of acquiring the skills of reading and writing at an early age,including: Annual meetings for teachers, with the aim of creating an update and exchange of experiences with new teaching strategies for literacy and related issues; “Consensus building for Reading and Writing,” Santa Cruz and Yacapani, composed of members of educational institutions, municipalities, NGOs, cultural, civil and private companies to join efforts to promote reading and writing in their areas of influence. Encouraging the support of parents through meetings, talks and workshops between teachers and parents, places for consultation and cooperation, a mutual learning process to help a child succeed accompanied by supporting materials: posters and leaflets, information material and other audiovisual resources. Network of Reading Schools, by 2008 20 reading schools were located in the municipalities of Yapacani, Warnes, El Torno, La Guardia, Montero, Saavedra, Cotoca and Education Districts 2 and 3 in the city of Santa Cruz de la Sierra. In Peru UPCH launched higher What has more impact on the experience in the three countries is that the CETT model is not known through conventions or bureaucratic rules. CETT is practical in terms of the needs and problems and to acquire visibility and confidence, in a logic of adding successive impacts from the lowest levels to the highest, in a spiral of learning experiences, creative innovators in qualification, literacy, technical training, education management, production of materials, etc., having educators in the poorest and forgotten areas of the countryside and perhaps forgotten by the official system, the most important actors, “the voices of the forgotten are the true actors of change.” The groups of teachers and highly involved specialists are those that design and disseminate in their work encouraging messages, more than the bureaucrats and administrators, government officials or coordinators in bureaucratic positions, positive signals of change as much in the qualification model as in the definition of standards and indicators to measure the quality of the educational performance and what has been learned. In the Ecuadorian MOE national political will is needed, so they Private companies in Peru , Ecuador and Bolivia are progressively developing interest in helping support education through their national forums. Investments in educational programs, supporting teacher training, educational material production, competition among children and youngsters in reading and writing reveal a growing link between educational policies and economic development. Participants pointed out that changes and long lasting transformations require a longer time span so the training cannot only consist of short workshops, they must use all of the CETT strategies, training, demonstration lessons and guides, with a minimum term of 2 or 3 years with the same people in the same locations. The ministry has really changed, that change is occurring at the lower levels and teachers are demanding more and more, the search for training and lifelong learning. There was an increase in consciousness among teachers to be more reflective or aware of their duties. CETT Peru for example followed a plan to assess, analyze, and reflect on what occurred during classroom instruction and apply change if needed. Professionals who have participated in the program have demonstrated an improvement in teaching performance. Some have even organized the "Movimiento Pedagogico." Evaluation of LAC/RSD Regional Education Program: Volume II 129 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey courses in pedagogy. Classroom teachers trying to replicate the experience have formed movements “Teaching Movement for Reading and Writing,” include/understand the importance of the change, often they do not want to accept the change, a policy is needed for the MOE to accept the change. To what extent have the regional programs influenced educational policies? (Provide concrete examples). There is a good number of experiences, all of them are verifiable: The practice of support to achieve new standards of quality in teaching and teacher training and indicators for assessing performance in the classroom, where the essence of the role of the teacher is defined. The teaching of reading and writing became state policy based on the methodology of Reading Schools. Books and other materials have been reproduced The training model has been incorporated into some private schools in Ecuador and in general incorporated as a methodology in educational reform processes in the three countries. In Peru the CNE, the National Council of Education, shared the training methodology, seeking to influence through the MINEDU to formalize and recognize it as an effective training program for teachers, recommended it but did not get incorporated as part of the public policy. The MINEDU training programs became more systematic. C.A. enablers have been called to work and are recognized by the MINEDU. The program Teach Peru has C.A. teachers. It has been effective in the learning progress of children as evidenced by national tests. The link with the private sector has been based on a proven systematic awareness and advocacy. The teachers feel that the project was not a small project, how to receive training and performance evaluation was new. Having a model school with visible results was the best selling point of the project. An entrepreneur in Piura (gas company) said: "I want to make our school like the School of Our Lady of Mercy" in a poor neighborhood in Piura. Performance evaluation, performance assessment work was not only for the children but for the teachers. It was noted that In Peru, based on strategic alliances, Natura company with the program “See to Believe” in partnership with USAID developed CETT training in schools in poor urban neighborhoods, companies such as Rimac, Ventanilla, and PlusPetrol, based on a agreements with CETT, provide support in teacher training for bilingual native Amazonian communities of the Lower Urubamba, and adaptation of curriculum and cultural content moving beyond reading and writing including in math; Business for Education, taking social responsibility and offering education financing CETT Puno, Ica and Arequipa. Backus Group contracted UPCH for training in literacy and so did the Minera Antamina. In Bolivia indigenous associations used the model. Cement companies in Ecuador supported the intercultural bilingual The sample of responses to this question was not sufficient to have any statistically significant finding. Evaluation of LAC/RSD Regional Education Program: Volume II 130 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey these had improved performance; as they progressed further in their training and support they had better performance too. Classroom observations helped in a significant improvement of performance. education. In Ecuador, financing was given by the Holstein company, that has been able to sustain long-term project support working with a NGO and a low budget, the activities can be reduced to a single event and it is not possible to measure the real impact of the activity. To what extent have ministries of education actively incorporated components of the regional programs into national educational policy or interventions? What factors facilitated/inhibited ministries’ adoption or adaptation of components of the regional programs? What factors of the regional programs have contributed to replication and sustainability? The teacher training model was recognized by the MOE Ecuador. Ecuador has managed to transfer the responsibilities of bilingual intercultural training to DINEIR MOE. The MOE has created an area of bilingual education as a response to “Reading Schools”. The teaching of reading and writing became state policy based on the methodology of “Reading Schools”. The training model has been incorporated into some private schools in Ecuador and incorporated as a general methodology in educational reform processes in the three countries and led to advances in the private sector of education and possibly to teacher training in mathematics. Books and other materials have been reproduced. While private initiatives include integrating the CETT methodology in social responsibility activities for education, they do not necessarily go through a strict coordination with the MOE. Focus groups in Quito and Peru, specialists, trainers, participating teachers were unaware of many aspects of the regional nature of the CETT . Reports for USAID established for this purpose did not have readership at that level. Universities in Quito and Bolivia were an information source, but such is not the case in Peru where bureaucratic bias was observed in the handling of information in the MOE. The sample of responses to this question was not sufficient to have any statistically significant finding. Capacity and sustainability In what ways has CETT strengthened capacity in partner institutions including various levels within ministries of education? Has CETT Those responsible for the CETTs-- teachers, specialists, trainers, supervisors--have expressed their firm conviction that the project changed the way basic education is managed--using a new model, setting targets and real targets for decentralization by establishing Some specialist teachers who were trained by CETT were integrated in various organizations including the MOE, Reading Schools, Centro Andino, OEI, GRADE, PlusPetrol. All of them reported that their skills and expertise grew with the training Centro Andino was responsible for promoting private participation in the three countries in order to provide sustainability which was the central responsibility of U. Cayetano in the The virtual training enabled the program to reach teachers in remote locations around the country. Teachers participated in the provinces of Carchi, Imbabura, Esmeraldas, The sample of responses to this question was not sufficient to have any statistically significant finding. Evaluation of LAC/RSD Regional Education Program: Volume II 131 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey built sufficient capacity such that technical interventions can be adequately sustained and/or replicated or scaled up? To what extent has there been continuation/ attrition of personnel trained under CETT? bases of the program, developing firm links with private sector partners and cooperation with common goals in basic education, promoting dialogue, research and planning, problems of teacher training and teaching, developing educational materials, all as never been done before for any project. No project ever aroused so much interest and was able to call on teachers to be trained to the point that in Ecuador for example, they speak of "educational activism," a call to everyone. It speaks of a cultural change to the extent that many trained indigenous teachers are incorporated into teacher training and are fulfilling their role with mixed groups. This would not have been possible without CETT. and the experience they gained is being used in their current responsibilities from different sectors. The MoEs’ distance from this experience and in their receptive role was a negative point due to the lack of leadership in education policy authority. In the second part of the 4th year, steps toward more secure sustainability were taken: USAID suggested the continuity of approach, the plan was made for each country, there was a balance although USAID did not attend, because it was an area of C.A. and partners focused on the sustainable model. Partnerships with private sector, the C.A. got positioned, events were organized in many places, hotels, businesses, municipalities, and government spaces. Formalization of the teaching movement for Reading and Writing. It is a movement that has autonomy in the field of reading and writing, creating games, contests, parades, UGELES mobilization. This is the focal point of the movement linked with the education sector. Technical sustainability, tested model, systematic. Materials in module format in the second phase: technical sustainability for the sector. Social sustainability through mechanisms of integration into regional and local policy roundtables for reading and writing. Institutional sustainability with universities that have incorporated second period, 2007- 2009. In none of the countries of Peru, Bolivia, or Ecuador did the private sector participate in public education but awareness as part of the strategy has led entrepreneurs to visit schools. Reading Schools is the product and has led to direct action of many companies. Progress in raising awareness on the importance of the model has been made. The agreements have provided space for corporate volunteer training: in Natura for example many workers have been involved and more workers are being trained as trainers. Pluspetrol has worked in teacher training, Machiguengas in Lower Urubamba for 4 years, the agreement works, the money ($ 150,000 annually) of the agreement is used to finance the activities, pay the salary of the team that lives in the area, there are 4 teachers, transportation, materials, logistics costs, organizing workshops. Reaching primary and secondary school language, mathematics and health education. Involving 24 communities across the Napo, Cotopaxi, especially in rural areas and urban and rural Pichincha. Virtual training took two forms: fully online (e-learning) and mixed (b-learning). In the latter case, the main component was support in the classroom demonstration classes and shared learning groups. In both cases, teachers received most of their training through the project’s virtual classroom. Evaluation of LAC/RSD Regional Education Program: Volume II 132 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey C.A. program professionals into the technical teams of each university, the Andina and UNUR. In the case of U. Cayetano in addition to the above, basic education continues with the reading and writing certificate; in 2010, 57 graduated. Annual audits were made by USAID, and a long closing audit without any observations. This generates a lot of confidence and credibility in the management of l UPCH. With regard to contracts with PlusPetrol, Companies for Education, Natura, as long they were in other USAID-assisted project audits, but after closing they are no longer audited. It is a college program, they go through the internal and external audit of the U. Cayetano. The MINSA accepted the model and "recommended" it but nothing more and now asks for support, as they do not feel they have the capacity to implement the process developed by UA; MINEDU is not ready. The Ministry has asked USAID for a pilot in Lima, to revive the model, apparently with a more integrated approach. Fernando Bolaños knows this and informed us about it. Lower Urubamba. The aim more clearly implemented in the second stage was to encourage active learning methodology using methods and materials for children, parents and businesses, participating in trade fairs and visits to schools so that all learned, including private companies that were invited. Building a global movement and schools are sustainable and are used as a benchmark for other schools that are not in the process. "Schools that learn, grow and share." To what extent are partner institutions including ministries of education capable of and interested in carrying on the technical components of CETT? Provide examples of specific components to which Any intervention project in public schools in urban areas or areas of rural poverty should be more integrated, not only must it promote improved components limited to reading and writing, or influence the first two or three grades, it should rather cover all primary school with comprehensive improvement and in all courses, "comprehensive school" with educational The MOE of Ecuador has taken DINEIR CETT methodology for training teachers in intercultural bilingual education and has a training offer that is exceeded by the high demand for teachers. In this institutionalization process in the CETT methodology DINEIR-MOE has a person responsible for Reading Schools who coordinates the bilingual No further references were obtained from the groups. The sample of responses to this question was not sufficient to have any statistically significant finding. Evaluation of LAC/RSD Regional Education Program: Volume II 133 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey ministries have committed. components, leadership development, improved working environment and educational management. Primary school teachers teaching mathematics, social sciences, environmental sciences, as well as communication, reading and writing, have been trained for this education and these capabilities must be strengthened as their teaching is weakened and educational quality suffers if there is not training in other areas. They do not feel pressure to achieve results. That work must not only have to do with educational goals. There are other lines that address training, management, relations, including discipline, links with families, community, public safety, school climate in schools, community development, emotional development issues with students. Other factors should be considered that are important in assessing the quality of education and of course the quality of the project. area. In the new Constitution of Ecuador bilingual education is integrated with education for Hispanic and indigenous groups and this is based on experience with CETT Reading Schools is an education policy for teaching literacy in MOE-Ecuador. In Peru, Centro Andino has promoted a masters at UPCH specializing in reading and writing, the Center awarded grants to Andean specialists MINEDU. In Ecuador, Peru and Bolivia, the creation of teaching materials and instructional booklets allowed them to work with their own training material with teachers and children in the classroom. Adopting a new approach and method of teaching writing and reading, sponsored by the Ecuadorian and Peruvian CETT, from the teaching of text-letter codes per se to the inclusion of them in a teaching model based on affective components and encouragement to the child as a basis for learning. What have been the roles, contributions, and limiting factors of public private partnerships. In both Peru and Ecuador (Bolivia was not visited) there were growing relations between private employers and CETT. In Peru's case because of the growing market expansion private investments that have generated surpluses and the strength of corporate social responsibility policy. In Bolivia, the private sector showed no great interest for political reasons and did not In Ecuador there were laws that promoted tax benefits to private sector investment in social development, although there were small but had positive reception. In Bolivia, there were no favorable conditions, it was difficult to create linkages and confidence in business, raise funds, plus broken relationships No evidence for knowledge of the subject in the public sector. Participants such as PlusPetrol, Banco Continental, telephone, EBEL Corporation, Natura, Employers for Education. INMED INMED-Andes (Partnerships for Children), between 2003- 2005 was commissioned USAID mostly provided the financing, design, and technical assistance of various projects with public universities being the main beneficiaries. Within the private sector, there was some financial support and funding in limited amounts for training. There was also Evaluation of LAC/RSD Regional Education Program: Volume II 134 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey promote CETT approaches. between the government of President Morales and the U.S. government. In Bolivia there was a withdrawal of the private sector, however the MOE funded educational interventions with public money allocated to local governments and municipalities that have managed their own budgets to work in public schools. With the financial base to train teachers hired in schools to develop materials, produce teaching guides, etc. In Peru, the dealings were directly with Centro Andino without MINEDU intervention. Some factors helped: the Peruvian economic growth has given companies surpluses in parallel with the growth of corporate social responsibility policy. The intervention of the CONFIEPP (Peruvian Confederation of Private Employers) provided the breakthrough, also Business for Education, with whom Centro Andino signed an agreement that is currently in effect. That agreement is based on the experience of the product Reading Schools with the name of "Read to Grow" initiative, now a in Peru for program partnerships with the private sector in collaboration with the CETT UPCH and strategic alliances with private sector companies (Pluspetrol and former South American Bank) in the public schools of North Lima, Piura, Ucayali, Ica and Cusco. assistance provided to the Roundtable for Reading. Other than those, private sector involvement was limited. Evaluation of LAC/RSD Regional Education Program: Volume II 135 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey major impact on the private sector to support basic education in Peru. What is the value-added of providing technical assistance to implementing partners? How has technical assistance to implementing partners in the region added value to the CETT program? Consider technical assistance provided through USAID/ Washington￾based contracts as well as through the cooperative agreements. What types of technical assistance provided to implementing partners in the region were most useful? There were workshops for discussion of the SOW, proposals and other initiatives, evaluation of training activities, budgets, books, etc. all of a routine nature. In Ecuador, although in the early stages and given the nature of contractual relationships, with the project managed from USAID / WDC, then from USAID / Lima, there were financial delays to initiate activities. USAID funds were always available to support activities in the 3 countries. Some respondents said they lacked political will and commitment to the Peruvian MOE to request further technical assistance and get the most out of the convention. A bureaucratic bias in the Peruvian MOE’s choice of representatives limited the participation of many specialists with experience and ability in the forums of discussion, and experience in exchange groups and workshops. Neither attended visits to the classroom. The argument given is that the MOE had no budget. Note: MOE specialists focus group in Peru emphasized that for the first time they were called to speak openly of the CETT, offer their insights, constructive criticism, ideas and visions for better use of experiences. The MOE specialists in Peru said that it has squandered its potential and expertise gained in the CETT training. The language specialists received grants from UASB for training. But visits to schools in early 2009 did not organize a work plan and no methodology was defined, no one knew how and who paid visits, specialists were not there because of a lack of funds. Opened by the UASB but lacking the commitment and organization of MINEDU. Technical assistance was provided in terms of training, methodology, and the development of a methodological framework. The experts were very knowledgeable. But perhaps additional assistance could have been beneficial in the quantitative assessment and monitoring and evaluation of the program. Conclusions, recommendations and lessons learned from the overall coordination of the project UPCH Lima-Andean CETT? 1. Every program should begin investigating everything done before, lessons learned, etc. 2. We must build a baseline that responds to real needs. 3. If you think there is a national program to strengthen local capacity that has no central vision, we must advise each region to create their own strengths, they should grow in autonomy and develop their own efforts and experiences. 4. The selection of the team of trainers with skills to create, manage conflicts and tensions, knowing the community, development management, role of the private sector in education is essential. You need multi-national teams with vision, technical, political and educational, to appreciate the wealth of learning and Evaluation of LAC/RSD Regional Education Program: Volume II 136 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Focus Groups Online Survey to advance on potential pitfalls. 5. Regional programs are interesting, are important for the vision of Latin America, learning and exchange between teachers from different areas, of mutually reinforcing everyone, especially school teachers and rural poor by enabling them to cover gaps in their training and skills, and have a better awareness of their realities, needs and priorities. 6. From the start develop the capacity for autonomy, in teachers and public authorities or private. This is key to the sustainability of any intervention. 7. The issue of accountability can gain ground in many ways, the results are evidence that inspire confidence at all levels. 8. You need a monitoring system that supports the teacher trainers to provide feedback for the process, not to punish. We must make the players feel that they are moving, make self-criticism, see how it grows, and give trust. Trainers and teachers have been through this, they felt valued, had autonomy, felt together. Evaluation of LAC/RSD Regional Education Program: Volume II 137 International Business & Technical Consultants, Inc. CERCA Matrix: Based on Field Interviews, Focus Groups, and Online Survey Data, Collected (November – December 2010) Leadership Role USAID Role Central Public Sector role Parents Association & Community Roles Municipalities Role Private Sector Involvement Focus Groups Survey Response Benefits & Challenges ●CERCA was setup by USAID in response to an AED proposal, in 2002. ●The inaugural CERCA meeting held in Puerto La Ceiba, in Honduras, in 2002, and organized by USAID Washington and AED, launched the program. ●The AED team managing CERCA provided two CERCA coordinating private sector foundations; in the RD (FALCONDO) & in Honduras (FEREMA ), training and resources to implement CERCA community activities in each country. ●In the DR, the start was slow and picked up only after some local municipalities became involved. ●The five respondents who knew about CERCA, in the DR and Honduras, did not know of USAID’s role in CERCA. ●USAID Washington’s role was indirect and was carried out by AED under the leadership of Dr. Carmen Siri with strong field support from Ana Florez and other members of the AED team. ●A retired USAID PSC program manager attested to CERCA being a Washington driven project and that it didn’t seem to be getting the direction and support it needed from DC to succeed. It came to a premature halt when DC funding ceased. ●All five respondents in RD and Honduras, who provided information on CERCA attested to CERCA not having been initially an MoE supported project. ●In both countries, MoE’s role was allowing CERCA to access selected pilot schools. ●Where CERCA seemed to have an impact, CERCA was adopted by local municipalities, in both Honduras and the DR. ●An important CERCA initiative was the “Teacher Initiative” groups which brought the teachers’ unions in line with the need to bring communities and parents into schools. The TIs allowed teachers to ●PTAs and local communities in selected pilot schools received CERCA TA training and played important roles in identifying school needs and developing school report cards. ●In the DR and Honduras, Municipalities played a key role in CERCA activities through Municipal schools which piloted school report cards (SRCs). ●Very little evidence was forthcoming of any significant private sector support for CERCA schools in the DR or Honduras, beyond that provided by the two local NGOs involved in organizing CERCA activities. ●Only three FG participants in Honduras had heard of CERCA. ●No one in either FG in the Dominican Republic, had heard of CERCA. ●Only one public sector participant in the DR knew of CERCA, but couldn’t describe what it did. The community participation increased the interest and involvement (or accountability) of the area. The School Report Card (Reporte Escolar) provided a measure of the school's accountability in the community Evaluation of LAC/RSD Regional Education Program: Volume II 138 International Business & Technical Consultants, Inc. Leadership Role USAID Role Central Public Sector role Parents Association & Community Roles Municipalities Role Private Sector Involvement Focus Groups Survey Response voice their own ideas and opinions about how to improve quality in schools. Coordination & Synergy ●Two NGOs, FALCONDO in the DR & FEREMA in Honduras, played key roles coordinating activities between AED, pilot schools and municipalities, in pilot areas. ●The USAID Washington LAC Bureau coordinated most CERCA activities via contracts with AED and MSI. ● A retired USAID program coordinator in Honduras remembered CERCA and attested to having had little to do with CERCA. The project was funded out of DC and managed and run by AED, in all 5 participating countries. ●AED respondents testified to the difficulty AED had in getting permission from local USAID missions to approach ministries of education and other local beneficiaries. USAID mission attitudes delayed starts in Honduras, the DR and Nicaragua. ●In the two countries visited, government ministries did not play a significant role in supporting CERCA activities. ●In El Salvador and in Guatemala, where AED’s prior project implementation activities, had already established good relations with the MoEs, MoE commitment to CERCA activities was strong. ●In Honduras coordination with the MoE was weak and commitment to CERCA activities was mainly through municipalities. ●In the DR, where USAID’s resistance to CERCA delayed the pilot program’s ●For PTAS and communities already actively empowered to play a role in school improvements, CERCA provided training, guidance and funding. ●CERCA enabled participating schools to implement well planned community supported activities to promote better quality education in their schools. ●In the DR, private sector support was received through foundations and NGOs like Falcondo. ●In Honduras FEREMA, another local foundation, provided local support for CERCA school- based activities. ●FG participants who worked with local NGOs like FALCONDO and FEREMA attested to their organizations having played an important coordination and management role in support of CERCA. activities, but couldn’t provide details. CERCA made communicatio n between individuals and institutions easier. In Guatemela, there were teleconferenc es every Monday to exchange ideas and information. Evaluation of LAC/RSD Regional Education Program: Volume II 139 International Business & Technical Consultants, Inc. Leadership Role USAID Role Central Public Sector role Parents Association & Community Roles Municipalities Role Private Sector Involvement Focus Groups Survey Response implementation, CERCA never impacted the MoE. Ministry of Education ●None of the senior MoE managers interviewed in both countries could remember CERCA. ●In both the DR and Honduras public sector involvement was negligible. In El Salvador and Guatemala, the public sector was more committed to supporting and encouraging CERCA activities. ●Ministries of education IN EL Salvador and Guatemala encouraged parental and community involvement in schools via the SRC development process. In the DR, CERCA had permanent representation in the MOE. The School Report Card was also implemented in the northern region of the DR, Guatemala, and El Salvador (Plan Escolar Anual (PEA) and Proyecto Educativo Institucional (PEI)). Policy & Perceptions ●In Honduras, FEREMA leadership had played an active role in CERCA, & some of what CERCA had achieved with communities may have influenced later attempts by the government to decentralize education services to the local municipality and community levels. ●USAID mission managers interviewed thought that just as CERCA seemed to be making an impact, lack of DC support brought the project to a premature halt. This resulted in no long term impact. ●In 2004, without an ●No one in the MoE in Honduras, or the DR, remembered CERCA. ●The IBTCI team was not able to visit impacted schools to meet with PTAs, because this was not in our USAID SOW. ●We were unable to visit municipalities that had worked with CERCA, because of the tight schedule. ●Two FG participants, one each from FEREMA and FALCONDO, thought that CERCA’s limited policy success may have had an impact on MoE CERCA's focus on civic involvement improved the perception and participation of individuals and consequently the community. In Evaluation of LAC/RSD Regional Education Program: Volume II 140 International Business & Technical Consultants, Inc. Leadership Role USAID Role Central Public Sector role Parents Association & Community Roles Municipalities Role Private Sector Involvement Focus Groups Survey Response However this hasn’t made any observable impact on education decentralization, yet. ●The FALCONDO senior staff interviewed, also thought that the MoE’s recent education decentralization reforms, may have benefitted from lessons learned through CERCA. independent evaluation, Washington funding for CERCA stopped, and AED thought the residue of funds were transferred to support PREAL activities. education decentralization reforms, but they weren’t sure. the DR, a notable initiative was the Programa Escuelas Para Padres de Familia. Capacity & Sustainabilit y ●FEREMA attested to the impact on their own staff, made by AED through well conducted training workshops and regional meetings. ●In both countries visited , those who had worked with CERCA, praised Dr. Carmen Siri and her team for the work they did mobilizing support for CERCA in their own and other participating countries. ●Exposure to the problems experienced by other countries in the education quality field helped Honduras to be more critical about quality issues in their own ●Both Washington, and the local USAID Missions, didn’t play a hands on role, beyond providing funding and technical support, through DC based contractors like AED & MSI. ●CERCA was difficult to sustain and made little long term impact because it was cut short by USAID Washington, before CERCA achievements could be properly consolidated and evaluated. ●CERCA managers in AED had argued for extending CERCA ●CERCA activities and SRCs may have been able to continue in Guatemala, El Salvador and Nicaragua had CERCA funding not been halted prematurely, by Washington. ●If there was a long term impact, that may well have resulted from the training and support provided to school PTAs through AED TA. ● Reports and formative evaluations shared by AED on the impact of AED training and other CERCA activities, are a measure of the short￾term impact of CERCA activities centered around the design, use and impact of school report cards. ●Informants in Honduras and the DR attested to the role played by municipalities in providing active support to CERCA activities, especially where municipal schools were involved. ●Two CERCA leaders in the DR thought that what CERCA had achieved would have been sustainable, because for the first time, municipalities had been actively brought in to play a role in promoting ●NGOs which participated in CERCA, benefitted from training which they received with AED assistance. ●AED training showed participants how to work with local communities and schools in teams. ●Through AED organized Regional meetings and workshops, participants from the five CERCA countries learned that their own school quality problems No information provided by respondents Evaluation of LAC/RSD Regional Education Program: Volume II 141 International Business & Technical Consultants, Inc. Leadership Role USAID Role Central Public Sector role Parents Association & Community Roles Municipalities Role Private Sector Involvement Focus Groups Survey Response schools. beyond the initial 4 year period, by at least 1-2 more years. ●The decision cut CERCA funding in DC was made to transfer funds to PREAL which needed USAID support to get started. ●AED also made a case, unsuccessfully, for joining forces with PREAL to transfer CERCA experiences and provide PREAL with an activities based implementation capability. better quality education, in local schools. were common problems, shared by others in neighboring countries. Technical Assistance ●Cooperating NGOs attested to the importance of training and other support provided by AED, with USAID Washington funding. ●MoE Informants said no significant local USAID TA, or other support, was received for CERCA, in the two countries visited. ●Conflicts resulted between USAID and AED over AED’s focus on process rather than results. ●Too much of what AED was doing was focused on the how rather than the what ●MoEs in the two countries visited, played a largely passive role and had not provided any funding, or TA support to CERCA. ●AED informants claimed that In Guatemala, Nicaragua and El Salvador, where AED was already well established with the local ministries, the MOEs played an ●CERCA actively supported and provided training in the following areas: School report card development; community mobilization activities; teacher initiative groups; monitoring and evaluation; & some community action research. ●FG participants with some recollection of CERCA, attested to having received some training from AED during the early phase of CERCA. Technical assistance - monitoring activities, forums, workshops, and assessments - strengthened national networks. Evaluation of LAC/RSD Regional Education Program: Volume II 142 International Business & Technical Consultants, Inc. Leadership Role USAID Role Central Public Sector role Parents Association & Community Roles Municipalities Role Private Sector Involvement Focus Groups Survey Response and tangible outcomes. ●FALCONDO & FEREMA coordinators did acknowledge benefitting from invitations to attend sporadic CERCA regional meetings organized by AED. active role in CERCA. Areas for Improvement for Future Regional Projects ●FALCONDO advised,that if CERCA were to be continued the international agency providing support needed to set clearer benchmarks from the outset, to hold local communities more accountable for their actions. ●In the DR support from the USAID mission was negligible because the local Mission considered CERCA to be an intrusive Washington program. ●The development of school report cards was the most effective aspect of CERCA, but this activity didn’t receive support from the governments till late in ●Projects that set out to bring about change at the local level need to be managed and run locally, either by the USAID Mission, or by surrogate entity like an NGO, or a government agency. ●For CERCA to have gone to scale, in any of the five countries involved, a lot more time and funding would have been required. ●The AED supported CERCA program was well implemented by AED and national partners, but couldn’t be sustained for want of long-term support from USAID, Washington. ●Should USAID support ●For CERCA to have made a long term sustained impact, the MoEs in each country would have had to play a more direct role in the implementation, management and promotion of the program, from the beginning. ●Too few schools were involved in CERCA, so the impact was never significant. The program in the DR was more akin to a pilot program and never stood a chance of making any significant contribution toward quality improvement nationally. ●After CERCA established a presence in schools through the report card process, schools needed to be more frequently visited and supported by CERCA TA, for activities to have been sustained. ●No mechanism was ●If municipalities are to play a leading role in implementing report cards and in supporting other CERCA activities , then more training needs to be given to administrators and local government managers and leaders. ●The bottom up approach developed in both countries by CERCA was the right approach, but it wasn’t funded long enough to succeed. ●All projects working with marginalized communities can’t go any faster than ●Private sector involvement should only be encouraged in countries where the private sector already has a committed role in promoting national education & social development. ●Donors need to follow-up and evaluate project impacts, especially in countries like Honduras and the DR. ●In the DR, communities working with international funding should be held more accountable for the funding they receive. Honduras has a history of wasted resources and unsustainable projects. Maintain the civic- and community￾oriented character but expand the program's reach to address issues related to education such as continued funding and labor market needs. Evaluation of LAC/RSD Regional Education Program: Volume II 143 International Business & Technical Consultants, Inc. Leadership Role USAID Role Central Public Sector role Parents Association & Community Roles Municipalities Role Private Sector Involvement Focus Groups Survey Response the project cycle. ●There was insufficient follow up to insure that CERCA generated activities were properly carried out, in each school. ●Many of the gains made through CERCA were not sustainable because some MoEs were never full CERCA partners, till late in the process. This was especially true for CERCA activities in Honduras and the DR. ● Regional projects like CERCA should not be driven by contractors. They need to be well managed and ownership transferred as quickly as possible to beneficiaries on the ground. another CERCA type program, in future, more attention should be paid to the regional dimension which wasn’t fully exploited. ●Too few regional meetings or workshops, were held and not enough attention paid to sharing country experiences between the five countries which were eventually involved. ●There was no evidence of any CERCA evaluations other than self￾administered formative evaluations carried out by AED. ●During the period when CERCA should have been evaluated, Tobias the USAID Administrator, had put a halt to support activities like project evaluations. ●Both USAID missions commented on the need to have projects managed out of DC run by managers who are created to provide ongoing long term support to expand the SRC process, because CERCA, as implemented, was only a pilot project. poor communities are capable of absorbing innovation and change. Evaluation of LAC/RSD Regional Education Program: Volume II 144 International Business & Technical Consultants, Inc. Leadership Role USAID Role Central Public Sector role Parents Association & Community Roles Municipalities Role Private Sector Involvement Focus Groups Survey Response field experienced and understand the dynamics of running programs in complex cultural contexts where governments still play an important legitimizing and controlling role, in education. NOTA BENE: The analysis above is taken from interviews held with four informants each claiming significant experience with the CERCA program in two countries; Honduras and the Dominican Republic. Three Focus Group members also claimed to have some knowledge of CERCA, but their testimony was less reliable since FG participants could not remember any CERCA details. An important interview conducted with two AED CERCA informants in AED, Washington DC, on the 3rd of January 2011, provided additional and valuable information about the CERCA regional program. Two respondents in the DR belonged to one foundation, FALCONDO, which had played an active leadership role in implementing CERCA in that country. Only two Honduran respondents were active in providing support or oversight for CERCA, in that country, one from a local foundation’s perspective and the other from the point of view of a local USAID Mission program manager. Out of the nearly 40 interviews carried out in 2 CARD countries, the DR and Honduras, 4 informants and 3 Focus Group participants knew of, or could share information about CERCA. A significant amount of useful data on CERCA was obtained from the Washington,DC‐based AED team that managed the USAID / AED CERCA contract. Evaluation of LAC/RSD Regional Education Program: Volume II 145 International Business & Technical Consultants, Inc. PREAL Matrix: Based on Field Interviews, Focus Groups, and Online Survey Data, Collected (November – December 2010) SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Online Survey 1. What were the benefits of supporting regional education programs? What was gained by working regionally? - Most of PREAL’s publications, conferences, electronic communications and information sharing are designed to improve knowledge￾sharing and collaboration within the region. - PREAL’s Central American and Latin American Report Cards, for example, provide some country-specific data, but prioritize comparative analyses on countries at the sub￾regional and regional level. - PREAL’s Best Practices program, its research program and working groups, and its Business -Education programs also are designed and best suited for implementation at the regional level. - Prior projects which included non-USAID funding, such as the Study Tours (Pasantías) program, research on child labor through Primero Aprendo project, and school violence are also designed to foster collaborative regional efforts to solve common problems. HO: - Sharing experience among countries. - Countries in the region know what is going on in education in their neighboring countries. DR: PREAL has promoted a permanent space for education policy dialogue at national and regional level. - The program is a knowledge center that provides research and publications on education themes. PREAL has been recognized as the main reference for policy discussions and for scholars doing research in education. In Guatemala, it is seen as resource in the design and implementation of education policies. 2. What were the challenges of supporting regional education programs? How could those challenges have been reduced? HO: - PREAL is no longer sustaining dialogue among Central American partners (Associasiones Basicas). The program suspended regional Committee meetings. There is no more opportunity to share experiences among them. - Working groups are concentrated in South America, not in Central America. DR: PREAL is no longer sustaining dialogue among Central American partners (Associasiones Basicas). There is not much coordination and/or articulation among partners. - The amount of money available for PREAL’s Nothing stated Evaluation of LAC/RSD Regional Education Program: Volume II 146 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Online Survey activities is not enough. -Study tours are few and isolated. The experience is not incorporated into country’s reality. - The number of publications is not enough. 3. What was sacrificed/lost by working regionally? Nothing Nothing Nothing stated 4. What were the advantages and disadvantages of the different models of program management and coordination (outlined in the background section) – by and for USAID? By and for implementing partners? N/A Nothing stated 5. (a) What worked, and (b) what didn’t work, in the coordination of each regional education program between USAID/W, USAID missions, program partners, contractors, and other key stakeholders? (a) PREAL makes a concerted effort to coordinate with national partners, USAID missions, international organizations and its region-wide network of contacts in developing and carrying out its activities. PREAL’s advisory board and Central American task force, composed of distinguished experts and civil society leaders from the region, have provided important input on policy priorities, although both groups are currently being revamped. - A Central American Coordinator provides a regular bridge between program staff, partners and missions, visiting countries frequently. - Our national partners cooperate with mission staff on areas of mutual interest from co￾organizing events to special briefings for mission staff. - PREAL’s Business Education Alliance HO: PREAL needs to increase follow up and evaluation of partners’ activities. There is lack of evaluation and long-term technical coaching. DR: There is a very good coordination with PREAL and between PREAL’s partners. Partners share expenses to develop their activities. EC: There is barely any coordination with other PREAL’s partner. The many countries that PREAL covers might be a call for a less centralized approach. The activities are diverse among countries. USAID/W was responsible for (preparing proposals for) funding while USAID Missions coordinated activities with PREAL associated centers in each country. USAID/W appears distant. Evaluation of LAC/RSD Regional Education Program: Volume II 147 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Online Survey program has successfully coordinated the work of national business associations through the provision of technical information, best practices, study tours, private consultancies, and conference planning. - Working Group coordinators have also played a pivotal role in connecting regional experts around specific topics. - In addition, PREAL has brought together groups of national partners (e.g. report card team leaders, strategic partners, Central American Coordinating Committee?) on different occasions to discuss common challenges and activity strategies, - PREAL co-directors are in frequent contact with representatives and leaders of international organizations and governments. (b) Prior to establishing the Central American Coordinator position, PREAL did not have a system in place that provided sufficient interaction with USAID missions, and we still occasionally have issues with delayed communications due to heavily booked staff commitments on both ends. - Other problems in program planning arose as a result of delayed delivery of anticipated USAID funds, which in turn made it difficult for us to renew agreements with our partners/working groups on a timely basis. - PREAL’s work has largely focused on influencing opinion leaders (including the business sector), and has not been as successful at reaching out directly to parents or private citizens (though that is part of PREAL’s overall objective). - Despite considerable effort, it has not found a productive approach to working with teachers’ Evaluation of LAC/RSD Regional Education Program: Volume II 148 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Online Survey unions. Nor has it made a systematic effort to fully capitalize on synergies with international organizations, despite good relations with all of them. 6. How closely did the regional programs complement and support bilateral mission activities? Was there sufficient coordination? What factors limited and facilitated coordination? How could the CETT program worked better with the bilateral programs? - PREAL regularly shares information, expertise and publications with a variety of organizations (bilateral, multilateral and private) working on education in Latin America and seeks synergies on topics of mutual interest. - PREAL leadership is in regular communication with the leadership at the OEI, OAS, World Bank, Inter-American Development Bank, Central American Coordination for Culture and Education (CECC), and others. - PREAL is about to sign agreements with UNESCO and the CECC to undertake joint activities. - Coordination between all PREAL programs has been a strong point of its various programs, as each has tended to rely upon and benefit from tried-and-true conceptrual frameworks, program models, collaborative activities and multiple perspectives on complex education policy topics. PREAL complements and supports bilateral programs by sharing information about programs’ themes. Even though the USAID/W was distinct from USAID missions (See Question 5), it was always supportive of bilateral mission activities. 7. What synergies and coordination were there between CETT and PREAL? What factors (administrative, technical, etc.) limited or improved coordination and synergy between the regional programs? - While no formal collaboration has occurred, PREAL experts have written about CETT’s program model as well as consulted informally with CETT national and regional staff. HO: There was no coordination between the two programs; however, teacher training is one of the subjects addressed by PREAL. DR: The partner knows CETT, but there is no coordination of activities with it. EC: Partner does not know CETT. Not much coordination with other organizations except the one international seminar organized by the working group on decentralization (c/o Margarita Pena). Evaluation of LAC/RSD Regional Education Program: Volume II 149 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Online Survey 8. What are the key contributions of the regional education program to improving education quality in the region as well as in individual countries? - PREAL’s experience to date suggests that regional programs and related activities can have an important effect on national-level education reform in the following ways: - Influencing policy is easier when networks of people work together- sharing information, reinforcing each-others work, and providing comparison points. - Regional networks also help create a “critical mass” of individuals dedicated to improving education both within countries and regionally. This is particularly important since translating good ideas into practice may require sustained and consistent messages over several years. - Reformers must be both active and patient, devising sound recommendations and positioning themselves to provide answers when those in power are ready to ask questions. - Regional publications can help create demand for technically sound, easily digestible information by making comparisons and showing what is possible. - Stakeholders, particularly in ministries or the private sector, want to know not only what’s wrong, but what to do about it. PREAL’s regional publications help bridge the gap between researchers and practitioners by providing user-friendly information on problems, progress, best practices and new ideas. - Working with national partners and connecting them with each other over the long term brings multiple benefits. - PREAL’s work with national partners to strengthen their analytical and communications skills and provide them with access to cutting PE: One of the PREAL’s most important contributions was the promotion of educational standards and evaluation. The MOE is adapting it to the national curriculum and evaluation. HO: - Instilled the theme of education reform for analysis and regional comparison. - Knowledge sharing of key education issues through high quality research and publications focused on improving quality of education and equity. - Production of instrument to assess performance of education systems (Report Cards - RC). The RC is widely used by MOEs, academia, and NGOs. - Knowledge dissemination on education among decision-makers, specialists and academia DR: PREAL has promoted: (a) a permanent space for education policy dialogue at national and regional level; (b) a national network of people involved in education; (c) knowledge sharing of education policy reform issues. - Study tour was important but it was not integrated to our country’s reality -Decision-makers in the MOE use RC - Regional Reports uses easy language that helps non-technical audience - Working groups with themes of common regional interest (standards and evaluation, teaching, etc) - Knowledge sharing of key education issues through high quality research and publications focused on improving quality of education and equity. EC: RC allows the nation to know the status of education in the country. RC helps increasing policy dialogue among education stakeholders in Ecuador. For instance, after releasing the 2010 RC the Catholic University promoted debate to education stakeholders discuss its results. The regional nature of the research helped level the uneven capabilities among countries and reduced gaps in knowledge. PREAL was a luxury in San Salvador and Tegucigalpa because of the excellent researchers from Argentina, Chile, and Uruguay. Evaluation of LAC/RSD Regional Education Program: Volume II 150 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Online Survey edge information beyond national borders helps make them “go-to” resources for policy planning. - At the same time, strong civil society partners provide continuity by speaking beyond one administration, and they act both as a stimulus to change—by pressing governments for reform—and as political allies in efforts to improve. - Decision-makers read RC and act upon its results. For instance, after reading the 2008 RC ministry of finance’s officials reduced MOE’s budget because, according to RC, the later was not spending its budget efficiently. 9. Considering different education stakeholder groups, including government, public, business sector, etc., as appropriate, to what extent have the regional programs changed attitudes, as well as the visibility of technical education issues, such as early grade literacy, public￾private partnership, etc.? (Provide concrete examples). - PREAL has formulated a series of fundamental reforms needed to overcome the failings of Latin America’s educational system. These include: 1. setting learning standards and evaluating performance toward meeting them; making schools accountable—to parents, employers and citizens; improving the quality of teaching; and investing more money in preschool, primary and secondary education. 2. For example, more than five years ago, when PREAL began systematically promoting the idea of accountability in its conferences and publications, the issue was completely absent from education policy discussions. There was not even an appropriate word in Spanish. -Today, accountability is a standard topic in speeches by the region’s ministers of education, and several countries have introduced accountability mechanisms as part of their reform packages. - Another example is PREAL’s Working Group on the teaching profession, which has worked extensively with programs in several countries to recognize teaching excellence and to foster informed discussion throughout the region on effective teaching. - Awards, sponsored by business and non- HO: Through best practices and seminars PREAL informed business communities how private sectors can have positive impact on education. - PREAL has helped empresarios por la education identifying possible allies. - In 2008, the Ministry of Education declared publicly that the RC was his compass. - Publications like standards, evaluation and teacher performance have been widely used at the MOE. The presence of PREAL increased the participation of different stakeholder groups but still within the context of how they saw their role in the local education sector. The government and public sector, for example, was more involved as beneficiaries of the program. Universities and research institutes participated more within their roles as contributors of information and knowledge. Communication centers amplified their role as the forum for policy dialogue with the presence of PREAL. Evaluation of LAC/RSD Regional Education Program: Volume II 151 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Online Survey governmental organizations with support from ministries of education, have created an innovative vehicle for discussing teacher quality, formation and training, instituted positive incentives for good performance and established a database of “best practices” used by the award-winning teachers. - PREAL has also helped advance the issues of accountability through its publications and international conferences (in Mexico and Jamaica) on the topic, as well as the topic of public-private partnerships, for which PREAL has written a book, created a best-practices database, and coordinates activities via the PREAL Business – Education Alliance program. 11. To what extent have the regional programs influenced educational policies? (Provide concrete examples). - After more than a decade of activity, PREAL has achieved a number of notable successes. It has become one of the region’s leading private organizations dedicated to changing the way public and private leaders in Latin America think about schools and education. - It is a credible source of information, analysis and policy recommendations for governments from across the political spectrum as well as civil society leaders. Some examples of policy influence include: - Peru’s vice minister of education, long a skeptic of education standards, decided to begin a program to establish national learning standards after repeated exposure to the arguments of PREAL’s working group on education standards and assessment. - With support from PREAL, Honduran national partner FEREMA has been a major source of information and influence on national education HO: PREAL study on early childhood development inspired the Ministry of Education to implement it as a national policy. DR: MOE uses one of the materials produced, textos integrados Substantial references to PREAL have been made in various education policies (e.g. the use of the Education Progress Report by the MOE in Peru). Evaluation of LAC/RSD Regional Education Program: Volume II 152 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Online Survey policies, including through the use the first two national report cards and their marked impact on national strategic plans, as well as EFA planning and support for municipalities (through the PREAL – FEREMA Strategic Partnership). - After a PREAL-sponsored visit to successful business-education initiatives in Colombia, three private sector leaders from El Salvador persuaded the minister of education to establish a business advisory council to channel business ideas directly to the ministry, modeled on a similar institution operating in Colombia. - PREAL worked during nearly two years to develop and encourage Central American ministers of education to jointly adopt a set of modern education reform measures. In 2007, the ministers (and subsequently, Central American presidents) approved the Education Decalogue 2021, which contains all the key PREAL recommendations. - The minister of education in El Salvador incorporated three key PREAL policy recommendations into a multi-year plan for education (Plan 2021)—a year after PREAL’s co-director suggested them at a conference— and publicly acknowledged PREAL’s influence on her decision. - After PREAL helped produce two national and six state-level education report cards in Colombia, national organizations expanded the state-level report card initiative to include 16 state reports and now produced a new national report periodically, entirely with local funding. 12. (a) In what ways has PREAL strengthened - PREAL’s experience suggests that its core activities and approach are effective in helping HO: (a) Program’s partner reputation and prestige has increased due to partnership with PREAL’s impact has been more on the policy Evaluation of LAC/RSD Regional Education Program: Volume II 153 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Online Survey capacity in partner institutions including various levels within ministries of education? (b) Has PREAL built sufficient capacity such that technical interventions can be adequately sustained and/or replicated or scaled up? leaders recognize problems, understand their causes, identify solutions, generate support for reforms, implement them and evaluate the results; informing stakeholders so they can provide political support for reformers; identifying and communicating successful efforts and innovations elsewhere; and building and sustaining networks of concerned, informed and active reformers. - Prior evaluations in 2003 and 2006 have shown considerable impact in all of these areas—with long-term contributions to creating a stronger demand for education improvement. - More recent PREAL interventions, such as the Strategic Partnership projects in Central America and the Caribbean have focused on specific interventions designed to help governments address concrete management challenges. -For example, successful efforts by PREAL and national partners in the Dominican Republic to help school leaders in large urban school districts design long-term quality management plans are now being scaled up with ministry of education support and USAID financing. PREAL. (b) PREAL built capacity through programs like strategic project but technical coaching, monitoring and evaluation was not provided. DR: Echoing the experience of Honduras about strategic projects, DR also lacked support to successfully finish its project. Due to lack of funds they could not publish and disseminate a research project. PREAL has served as a model to the partner institution interviewed. By its example, it helped the organization systematizes its own activities’. - PREAL has strengthened partner’s capacity through publications and workshops. - PREAL’s publications are used as instructional material for education graduate programs. - Week newsletters from PREAL keep tertiary education institutions abreast on education policy issues. EC: PREAL has been an inspiration for the Ecuadorean partner. As a young organization, it has used PREAL’s policy instruments (specially research and publications) as guidance in the area of education. debate in education (through research) than on technical interventions. 13. To what extent are partner institutions including ministries of education capable of and interested in carrying on the technical components of CETT? Provide examples of specific components to which ministries have committed. N/A NA Evaluation of LAC/RSD Regional Education Program: Volume II 154 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Online Survey 14. What have been the roles, contributions, and limiting factors of public￾private partnerships? - PREAL’s principal work on public-private partnerships has been through its business leadership program. Activities in this area aim to mobilize business leaders in specific countries, press for critical changes in educational policy and practice, and put resources into innovation and reform at the community and school level through business￾education “partnerships”. - To date, PREAL has developed a network of business leaders concerned with promoting education reform, established business￾education initiatives in five countries (Guatemala, Panama, El Salvador, Honduras and the Dominican Republic) and regularly provides advice and assistance to groups in Nicaragua. - We have helped these groups develop strong programs, connected them with similar groups in other countries, and worked to place education on the agenda of the Corporate Social Responsibility movement in Latin America. - The objective now is to help existing groups become more effective and authoritative actors in education policy and to expand our network to several countries where we are not currently active. - The role of PPPs in Latin America as a sustainability mechanism has tended to be in providing independent support for and perspectives on education that can extend beyond political timelines and horizons, as well as to improve the efficiency and quality of education. HO: Today the political situation is not favorable to such alliance. However, with PREAL’ support, some organizations in the private sector (including the interviewed one) are preparing together an education program to be implemented next year. DR: Most public-private partnerships (PPP) are limited to provide financial resources for school infra-structure and not to improve quality of education. Alliances have been established to make private companies ‘adopt’ schools located near their business establishments. - PPPs are not organized to support mayor national education programs. - There is lack of knowledge of PREAL’s knowledge sharing activities in the area of business and education alliances. Exposure to PREAL’s initiative in this area was limited to participation in a study tour in Colombia. There was no further engagement form PREAL’s local partners to follow up with people who participated in the study tour. Various private sector foundations have participated in PREAL reports and studies because of their knowledge and expertise. Examples include Fundación Gabriel Lewis Galindo, Fundación de Chile, and COSPAE. Other organizations also supported PREAL based on their expertise, like communication groups supporting the MEDUCA education portal and COSPAE website. Public sector agencies, notably public universities, provided technical input in the development of various resources like the First and Second Report Card in the DR, 15. How has technical PREAL has not provided technical assistance Technical assistance has Evaluation of LAC/RSD Regional Education Program: Volume II 155 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Online Survey assistance to implementing partners in the region added value to the CETT program? Consider technical assistance provided through USAID/ Washington-based contracts as well as through the cooperative agreements. What types of technical assistance provided to implementing partners in the region were most useful? to its partners. not been provided because most of the work has been policy-oriented (e.g. Peru) and outcomes would be harder to identify and measure (i.e. more long-term and less discrete). 16. What has been accomplished as a result of the activities of PREAL and its partners? (e.g. workshops, publications, etc) - PREAL has provided a host of interventions that are most easily categorized into the following programs areas: - Publications and conferences: Produces a series of events and accessible documents in a variety of formats in order to broadly disseminate cutting edge ideas throughout the region. - National Partnerships (Central America): Provides formal collaboration with national organizations devoted to education reform via collaborative events, the distribution of PREAL publications, and several “strategic partnerships” designed to influence education policy. - Monitoring program: Assesses the state and progress of education systems and particular policies in Latin America at the national, sub￾regional and regional levels and makes recommendations for change. - Business Education Alliance: Engages with business leaders at the national and regional level to mobilize private sector support for education reform. HO: - RC and best practices videos for teachers and principals, - Consultant Committees to support education authorities to understand and implement pre￾school policies. The initiative had a positive impact in the in the Ministry of Education. - Supported study on education reform in Honduras in 2005. - Provided information and persuaded Camara de Comercio to do something for Honduran education. DR: Publication on national education reform (1992) and quality of education in DR. EC: Policy dialogue. After publication of the 2010 RC education stakeholders met to discuss RC’s results. The research and information provided by PREAL publications and reports were recognized as the most significant accomplishments. Another significant accomplishment is the establishment of systems for monitoring and assessing the progress of education within a country such as the Education Progress Reports and the Working Group on Standards and Assessment. Evaluation of LAC/RSD Regional Education Program: Volume II 156 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Online Survey - Best practices program: Identifies and summarizes successful education policy reform initiatives in and outside of the region and archives them online. - Working Groups: Collect, provide and disseminate focused and expert analyses on specific topics; current WG’s are based around on the teaching profession, standards and evaluation, and school management. 17. To what extent has the MOE been an active partner of PREAL, including the use of the PREAL policy instruments? - Peru’s vice minister of education, long a skeptic of education standards, decided to begin a program to establish national learning standards after repeated exposure to the arguments of PREAL’s working group on education standards and assessment. - After a PREAL-sponsored visit to successful business-education initiatives in Colombia, three private sector leaders from El Salvador persuaded the minister of education to establish a business advisory council to channel business ideas directly to the ministry, modeled on a similar institution operating in Colombia. - PREAL worked during nearly two years to develop and encourage Central American ministers of education to jointly adopt a set of modern education reform measures. In 2007, the ministers (and subsequently, Central American presidents) approved the Education Decalogue 2021, which contains all the key PREAL recommendations. -The minister of education in El Salvador HO: The MOE uses RC and publications. HO: - Partner sends publications to the MOE, from which 50% are distributed to municipal and district localities. - Director of evaluation uses documents related to evaluation. - Ministers of education utilize the RC. For instance, data to formulate the 2010-1014 Education Plan were extracted directly form the 2009 RC. DR: PREAL needs to provide decision-makers at the MOE with more up-to-date RC. When the instrument is ready the data is already obsolete. In Peru and other countries the MOE used the Educational Progress Report to discuss local education issues. Evaluation of LAC/RSD Regional Education Program: Volume II 157 International Business & Technical Consultants, Inc. SOW Question Project Leadership Ministry of Education and Other Public Sector Private Sector Online Survey incorporated three key PREAL policy recommendations into a multi-year plan for education (Plan 2021)—a year after PREAL’s co-director suggested them at a conference— and publicly acknowledged PREAL’s influence on her decision. 18. (a) Among PREAL’s policy instruments, which are the most useful ones for your country? (b) Which are the least useful ones for your country? HO: (a) RC (b) The working group DR: (a) RC. Many decision-makers use this instrument because it is easy to read and understand. NA Lessons Learned Recommendations Honduras - There are not many institutions focused on improving quality of education. - Through PREAL the partner has increased the number of education stakeholders it works with. - Through PREAL the partner has learned to work with network system. - PREAL taught private sector’s partner the importance of engagement with education. - Through best-practice private sector partner realize what can be achieved in improving quality of education. - The annual funds provided by partnership is not enough, It should be increased. - To be more present through technical assistance and evaluation. - Promote regional activities to allow sharing of experience. - RCs need to be prepared more often. Today it is prepared every five years. - Coaching for education officials program should restart. - Working groups should focus more on Central America. - Constant communication and support from PREAL in order to carry on with private sector education project. Evaluation of LAC/RSD Regional Education Program: Volume II 158 International Business & Technical Consultants, Inc. Lessons Learned Recommendations Dominican Republic - Partnership between regional program like PREAL and local NGOs is important in the process of education reform. - PREAL is a meaningful instrument to inform about education issues and best practices. - Regional experience should be increased. The MOE and the partners would benefit a lot from it. - RC takes too long to be published. The program needs to be more efficient in providing up-to-date RCs. -The program should increase incidence in the media. - PREAL should increase number of national research in education. - Dissemination of PREAL’s program and its policy instruments should be more intense within the business community. The work of partners disseminating the program sometimes is not enough. Ecuador - PREAL is an important education policy program for the country but it needs to be more active and disseminated. - The program should promote more policy dialogue events at the national level. - PREAL should instill the theme of public-private partnership in Ecuador to inform the business community about how they can impact quality of education in the country. - RC should be more flexible and consider other variables that are relevant to the country being analyzed. Peru - PREAL needs to increase dissemination and promotion of RC. Partner needs more funds to do it.- PREAL - Networks make a difference. Connecting people working in different countries on similar issues over time strengthens their resolve, expands their knowledge and enhances their impact. The process of sharing experience and ideas generates synergy, often producing insights and recommendations beyond those produced by any single network member. - Civil society can influence education policy. PREAL’s experience clearly shows that politicians, decision makers, and opinion-shapers are amenable to influence, as are the policies and initiatives they support. They respond not only to pressure but also to good ideas, information, and recommendations from reliable sources. Leaders often move from civil society roles into government, carrying ideas acquired in civil society activities with them. Evaluation of LAC/RSD Regional Education Program: Volume II 159 International Business & Technical Consultants, Inc. Lessons Learned Recommendations -In Central America, tackling many education policy issues with a regional perspective is advantageous because of the similarity in trends, challenges, and institutions working together or on education policy reform issues. - Looking at regional impact for PREAL programs requires both a long-term perspective and a sense of the “big-picture” shifts in policy debate as much as national-level reforms or changes. These include PREAL’s success with the subjects of accountability and standards and evaluation. Evaluation of LAC/RSD Regional Education Program: Volume II 160 International Business & Technical Consultants, Inc. Annex VIII. Bibliography Andean CETT Annual Reports. (2004 – 2009). Andean CETT Work Plan. (2004 – 2007). Ano Academico 2010-2011, UASB, Ecuador, (2010). Bernbaum, M., Herrera R. J., Schiefelbein, E. (April 2010). Evaluation of USAID/Peru’s Education Program: AprenDes and CETT Andino: Final Evaluation Report. The Mitchell Group. C-CETT Annual Reports, Project Implementation Unit. (2002 – 2008). C-CETT Milestones: Year One. (October 2002 – September 2003). C-CETT Quarterly Reports. (October – December 2003). C-CETT Work Plan. (April – September 2009). C-CETT Year One: First Quarter. (October 2002 – December 2003). C-CETT Year One Work Plan. (October 2002 – September 2003). CARD-CETT Annual Reports, Project Implementation Unit. (2003 -2009). CARD-CETT Executive Committee Meeting Minutes. (2003 – 2010). CARD-CETT Technical Committee Meeting Minutes. (2003 – 2010). CARD-CETT Work Plan. (March 2002). CERCA Report on Project Results. (2004 – 2006). CERCA Workplan. (September 2002 – March 2006). CETT- CARD Informe Final: Aprendizaje Efectivo de la Lectoescritura, ILCE, Mexico for USAID, June, (2010). CETT: A Performance Assessment. O’Brien and Associates International, Inc. (February 2005). CETT Information Packet. (June 2004). Evaluation of LAC/RSD Regional Education Program: Volume II 161 International Business & Technical Consultants, Inc. CETT MOBIS Evaluation and Technical Support to LAC/RSD/HER. Aguirre Division/JBS International. (2007 – 2010). Chesterfield, R., Abreu-Combs, A., Culver, K. and Alvares de Azevedo, T. (November 2009). CETT Impact Study Report. Aguirre Division of JBS International, Inc. Chesterfield, R. and Abreu-Combs, A. (June 2010). CETT Two-Year Impact Study Report Aguirre Division of JBS International, Inc. Chesterfield, R., Enge, K., Linan-Thompson, S., and Thorne, C. (October 2004). Reflective Study of Peru CETT Schools. Aguirre International. Chesterfield, R.; Culver, K.; Hunt, B. C.; Linan-Thompson, S. (April – November 2004). Reflective Study of CETT: Final Report. Aguirre International. Chesterfield, R.; Culver, K.; Hunt, B. C.; Linan-Thompson, S. (April – November 2004). Reflective Study of CETT: Executive Report. Aguirre International. Crespo, C.; Culver, K.; Hunt, B. C.; Rasnake, R. (November 2004). Reflective Study Bolivia CETT Schools. Aguirre International. Culver, K.; Hunt, B. C.; and Linan-Thompson, S. (July 2006). CETT Professional Development Review: Final Report. Aguirre Division of JBS International, Inc. Culver, K.; Chesterfield, R.; Crespo, C.; Grenas de Contreras, L.; and Linan-Thompson, S. (October 2004). Reflective Study Guatemala CETT Schools. Aguirre International. Dainton, V.; Linan-Thompson, S.; Romero-Contreras, S.; and Thorne, C. (May 2006). CETT Professional Development Review: Bolivia CETT. Aguirre International. Educar Para Vivir, FEREMA, Honduras, (2009). Encuentro de Redes de Ninas y Ninos Creativos, FEREMA, Honduras, (2009). Enge, K., Gamero, G., Hunt, B. C., and Linan-Thompson, S. (October 2004). Reflective Study Honduras CETT Schools. Aguirre International. Ensayo de Reporte Escolar, CERCA, AED-WDC, (2006) (obtained from FALCONDO in the DR). Escuelas Lectoras : Una Propuesta Para EL Cambio, UASB, Ecuador, November, (2009). Escuelas Lectoras: Programa de Capacitacion para Educacion Basica: Lectura y Escritura. UASB, Ecuador, (2009). Evaluation of LAC/RSD Regional Education Program: Volume II 162 International Business & Technical Consultants, Inc. Gamero, G.; Hunt, B. C.; Marquez, A.; and Rodriguez, L.; (June 2006). CETT Professional Development Review: Guatemalan CETT. Aguirre International. Hunt, B. C.; Gamero, G.; and Marquez, A. (April 2006). CETT Professional Development Review: Dominican Republic CETT. Aguirre International Hunt, B. C.; Grant, N.; and Lambert, C. (June 2006). CETT Professional Development Review. Aguirre International. Hunt, B. C.; Chesterfield, R.; Linan-Thompson, S.; Crespo, C. (August 2004).Reflective Study Ecuador CETT Schools. Aguirre International. Hunt, B. C.; Culver, K.; Gamero, G.; and Louzano, P. (May 2006). CETT Professional Development Review: Honduran CETT. Aguirre International. Hunt, B. C.; Lambert, C. (June 2004). Pilot of Reflective Study Methodology C-CETT. Aguirre International. Hunt, B.; Tvaruzkova, M.; Levi, R.; and Millan, F. (December 2010). CETT White Paper Series; Paper Five: Paradigm Shift. Aguirre Division of JBS International, Inc. Informe Annual Fundacion FALCONDO, Republica Dominicana, (2009). Knox-Seith, B., and CLK. (October 2006). CETT Sustainability Draft Concept Paper. Knox-Seith, B. (October 2004). C-CETT Notes. La Educacion Pre-Basica Cambia La Vida, FEREMA, Honduras, (2008). Leer y Escribir: Competencias Para La Vida, Centro de Excelencia para la Capacitacion de Docentes, UCMM, Republica Dominicana, (2009). Linan-Thompson, S.; Romero Contreras, S.; and Thorne, C. (May 2006). CETT Professional Development Review: Peru CETT. Aguirre International. Lopez, G. G., Hunt, B. C., Sandoval, L. R., Schwarzer, D. (October 2004).Reflective Study of Nicaragua CETT Schools. Aguirre International. Navarro, R; and Skelton, A. (January 2011). CETT White Paper Series: Paper One: Regional Nature. Aguirre Division of JBS International, Inc. Evaluation of LAC/RSD Regional Education Program: Volume II 163 International Business & Technical Consultants, Inc. Outstanding Practices Related to Teachers, In UNESCO –Hamdan Bin Rashid Al-Maktoum Award, (2009-2010). (CARD CETT received this prize from UNESCO in Paris for the excellence of its work done for regional teacher training). Participacion Comunitaria, CERCA, AED- Washington, D.C., April, (2006). Planes Operativos de Capacitacion, CETT CARD, PUCCM the DR & UPNFM Honduras. March, (2010). Que La Calidad de La Educacion Mejore: Implicaciones de los Hallazgos de Una Investigacion de Casos, CERCA, AED – Washington, D.C., (2006). Reporte Escolar, CERCA, AED – Washington, D.C., (February, 2006) (obtained from FALCONDO in the DR). Valverde, G.; Wolfe, R.; and Roncagliolo, R. (June 2010) CETT White Paper Series: Paper Two: Testing. Aguirre Division of JBS International, Inc. Weekly Update of Aguirre Activities for the LAC Team. (2006 - 2010). Zardini, I. E. and Zardini, P. J. (June 2010). CETT White Paper Series: Paper Three: Cost￾effectiveness. Aguirre Division of JBS International, Inc. Zardini, P. J.; Zardini, I. E. (September 2010). CETT White Paper Series: Paper Four: Sustainability. Aguirre Division of JBS International, Inc. U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov It