USAID/PVC Matching Grant Evaluation Series: Woman/Child Impact-II Matching Grant CA FAO-A-00-96-00003-00 between SAVE THE CHILDREN-USA and USAID/PVC February, 2002 Conducted under USAID/Evaluation Indefinite Quantity Contract # AEP-I-00-00-00024-00 Task Order No. 1 Evaluation team members: Diana Myers (Save the Children-USA) Dr. William Millsap (MSI, Team Leader) ACKNOWLEDGEMENTS The evaluation team wishes to express its gratitude to the management in Save the Children’s Headquarters for their assistance and their candid responses to our many questions as we conducted the evaluation. But in particular, the team is extremely grateful to the staff in the Field Offices visited—they gave freely of their time, talked openly and honestly about their successes and the shortcomings experienced with the WCI-2 Grant. Visits to partners were arranged where the team was well-received and we learned first-hand how much effort it takes to establish viable partnerships. In carrying out the evaluation, the team has gained valuable insights from working with a fine set of professionals in Save the Children Headquarters and in its Field Offices. C:\12-20-working\New Folder\SAVE evaluation final report.doc ii Evaluation of “WOMAN/CHILD IMPACT-II” Matching Grant CA FAO-A-00-96-00003 between SAVE THE CHILDREN-USA and USAID/PVC TABLE OF CONTENTS 1.0 Executive Summary............................................................................................................................. 1 1.1 Overview 1 1.2 Key Findings, Conclusions and Recommendations.............................................................. 1 2.0 Evaluation Methodology and Team Composition .......................................................................... 4 3.0 Matching Grant Background............................................................................................................. 7 3.1 Historical & Technical Context and Partners ........................................................................ 7 3.2 Project Goal, Objectives, and Major Hypotheses to be Tested............................................. 9 4.0 Purpose of the Evaluation.................................................................................................................13 5.0 Program Implementation Evaluation Questions.......................................................................... 15 5.1 The Detailed Implementation Plan.......................................................................................15 5.1.1 DIP completion and DIP accuracy...................................................................... 15 5.1.2 Quality of DIP and degree of success in implementation ................................16 5.1.3 Familiarity with DIP and design.......................................................................... 17 5.1.4 Major successes and shortfalls in implementation ...........................................17 5.1.5 Impact Results........................................................................................................19 5.2 Assessment of project model and hypotheses...................................................................... 19 5.2.1 Project hypotheses articulated in CA.................................................................. 19 5.2.2 Replication and scale-up of approaches in project area or elsewhere...........20 5.3 Advocacy under the project .................................................................................................. 21 5.3.1 Advocacy activities and impact............................................................................21 5.3.2 Partner/PVO roles in advocacy............................................................................21 6.0 Partnership Questions.......................................................................................................................23 6.1 Analysis of WCI Partnership Schemes.................................................................................23 6.2 Measuring Institutional Capacity..........................................................................................26 6.3 Constraints to Partnership ..................................................................................................... 26 6.4 Information Technology........................................................................................................27 6.5 Use of local networks and service organizations................................................................. 27 7.0 Program Management ......................................................................................................................28 7.1 Strategic Approach and Program Planning .......................................................................... 28 7.2 Country Initiatives .................................................................................................................28 7.3 Conflict Management............................................................................................................29 7.4 Monitoring and Evaluation....................................................................................................29 7.5 Overall Management .............................................................................................................32 7.6 Sustainability..........................................................................................................................33 7.6.1 Overall sustainability survey................................................................................33 7.7 Financial Management .......................................................................................................... 33 7.7.1 Effectiveness of financial management...............................................................33 7.7.2 Leveraging other donor funds..............................................................................34 C:\12-20-working\New Folder\SAVE evaluation final report.doc iii 7.7.3 Cost effectiveness of technical approach............................................................35 7.7.4 Repercussions of “matching” requirement on program.................................. 35 7.8 PVO’s Information Management..........................................................................................35 7.9 Logistics 35 7.10 Project Supervision................................................................................................................35 7.11 USAID Management .............................................................................................................36 8.0 Conclusions and Recommendations................................................................................................37 Annex A: Key Events Timeline .............................................................................................................42 Annex B: Detailed Implementation Plan Table (derived from WCI Results Framework).........44 Annex C: Statement of Work................................................................................................................47 Annex C-1: List of Documents Reviewed............................................................................................55 Annex C-2: Evaluation Questionnaires...............................................................................................59 Annex D: Advocacy Implementation Chain .......................................................................................67 Annex E: Partnership Table, by Country Program Visited.............................................................70 Annex F: Sustainability Analysis.......................................................................................................... 74 Annex G .....................................................................................................................................................79 Annex G2 ...................................................................................................................................................83 Annex H1: Turnover Report..................................................................................................................85 Annex H2: Save the Children Diversity Patterns ...............................................................................86 Annex H2: Save the Children Diversity Patterns ...............................................................................87 ANNEX H2: Save the Children Diversity Patterns............................................................................89 ANNEX I....................................................................................................................................................91 Appendix 1: Realigned WCI Grant...................................................................................................... 93 Appendix 2: WCI Intermediate Results..............................................................................................95 List of Tables 3.1 Project Hierarchy of Objectives 5.1 Major Successes and Shortcomings in Implementation 7.1 Donor Funds Leveraged During the WCI Grant Funding Period 8.1 HQ & Field Staff Assessments of the WCI-2 Grant C:\12-20-working\New Folder\SAVE evaluation final report.doc iv List of Acronyms Used ARH Adolescent Reproductive Health BFO Bangladesh Field Office-Save the Children BHR Bureau of Humanitarian Relief CA Cooperative Agreement CB Capacity Building DIP Development Implementation Plan ECD Early Childhood Development EFO Ethiopia Field Office-Save the Children ESFO El Salvador Field Office-Save the Children FACETS Family and Community Empowerment Training System FO Field Office HFS Health Financing System HQ Headquarters IP International Programs IR Intermediate Result ISPM Instituto Salvadorena de Proteccion al Menores LNGO Local non-governmental organization M&E Monitoring and Evaluation MG Matching Grant MOU Memorandum of Understanding MSI Management Systems International NGO Non-governmental organization PME Planning, Monitoring and Evaluation PD Project-Program Director PLG Program Learning Group PM Project-Program Manager POP Program Operation Plan-Save the Children PVC Private and Voluntary Cooperation PVO Private Voluntary Organization (usually U.S.) RF Results Framework SC Save the Children SO Strategic Objective TA Technical Assistance USAID United States Agency for International Development WCI-2 Women-Child Impact-II Project C:\12-20-working\New Folder\SAVE evaluation final report.doc v Evaluation Identification Sheet PVO name Save the Children-USA Matching Grant Title Woman/Child Impact-II Cooperative agreement number CA FAO-A-00-96-00003 Amount of Grant $4,250,000 Period of Grant October 1, 1996 – September 30,2001 Any (cost/no cost) extensions? 6 month no cost extension, October 1, 2001– March 31, 2002 Current status of MG Midway through extension period USAID/PVC Grant Officer (s) Martin Hewitt Technical area of grant Cross-sectoral women/child impact with emphasis on EO, HPN, ED, and Humanitarian Response Date of the evaluation October – December 2001 Countries of program activity Bangladesh, El Salvador, Ethiopia, and Mozambique* Country programs evaluated Bangladesh, El Salvador, Ethiopia Evaluation Team Members (organization) Diana Myers, Save the Children-USA William Millsap, Ph.D., Management Systems International * Mozambique discontinued its participation in the WCI-2 grant program in February of 2000. C:\12-20-working\New Folder\SAVE evaluation final report.doc 1 Evaluation of “Woman/Child Impact-II” Matching Grant FAO-A-00-96-00003-00 between Save the Children-USA and USAID/PVC 1.0 EXECUTIVE SUMMARY 1.1 Overview The current matching grant, Woman/Child Impact-2 (WCI-2), was funded in the amount of $4.25 million dollars through a follow-on Cooperative Agreement between USAID/PVC and Save the Children (SC). The life of the grant was from October 1996 through September 2001. The goal of the grant was to: “contribute to sustainable human development through the systematic economic and social empowerment of children, women and other disadvantaged groups by expanding their participation in decision making at all levels of society. And that empowerment is achieved through collaboration with local organizations committed to democratic pluralism, gender equity, and economic growth benefiting disadvantaged women and children.” To contribute to this overarching goal, the grant planned a wide spectrum of capacity-building activities in SC Headquarters and in four Hub country settings (Bangladesh, El Salvador, Ethiopia and Mozambique) that included ongoing promotion of gender equity programs in all of SC’s programming efforts, continuing development of monitoring and evaluations systems for the Home Office and for the Field Offices, and establishing four strategic partnerships in the designated Hub countries. By way of contrast, the first Woman/Child Impact grant was a five-year grant funded for $5 million dollars between the years 1991–1996. The objectives of the first grant were to assist SC Headquarters in strategic planning and providing technical support for offices in Bangladesh, Bolivia, Haiti and Mali to design programs with greater emphasis on and sensitivity to the needs of women. Evaluation Purpose and Methods Management Systems International (MSI) conducted this final evaluation under contract to USAID/PVC. The final evaluation of the WCI-2 grant assesses its impact in the field and at SC’s headquarters (Westport and Washington) and its compliance with the requirements of the matching grant (MG) program. The overall purpose of the evaluation was to determine how the grant has strengthened the technical and organizational capacity of Save the Children and, through its local partnerships, has enabled local organizations to achieve increased capacity. A two-person team carried out the evaluation: MSI’s evaluation expert and a senior manager in SC’s Washington Office, familiar with the WCI grant. Semi-structured questionnaires were developed for collecting information from SC Headquarters staff, SC field staff and partner representatives. The evaluation team visited three countries - Bangladesh, El Salvador and Ethiopia - where the team interviewed key SC field staff, visited with partners, reviewed documents produced under the grant and observed the programs that the Field Office was supporting. During these visits, the team observed the types of programs that SC was operating along with those of its partners (e.g., Early Childhood Health in Bangladesh, the Protection of Minors in El Salvador and Adolescent Reproductive Health in Ethiopia). The evaluation was carried out between mid-September 2001 and January 2002. 1.2 Key Findings, Conclusions and Recommendations Findings The WCI-2 DIP submitted to USAID in 1997 was poorly designed and not capable of being implemented. PVC failed to conduct a formal review of the DIP; subsequently, after three years of confusion and C:\12-20-working\New Folder\SAVE evaluation final report.doc 2 uncertainty, the DIP was realigned to be consistent with USAID’s Result Framework format. Despite early problems, SC Field Offices achieved demonstrable results in implementing gender equity programs, establishing viable and sustainable partnerships and, to a lesser extent, creating operating M&E systems for their programs. SC HQ has also adopted and advocated policies supportive of gender equity and partnerships. The implementation of an M&E system at SC Headquarters has proceeded much slower than anticipated and much remains to be done before this is accomplished. Conclusions The WCI grant has added value to SC operations—both in the field and at the HQ offices in Westport and Washington, DC: · SC has integrated gender equity policies throughout the organization and worked hard to promote these policies with its international and national partners and through numerous international fora on women and children’s issues; · SC has strengthened the planning and monitoring of its field programs; programming procedures have been enhanced in HQ operations, and Program Learning Groups are carrying out important training and TA roles; and · SC Field Offices have increased their program outreach capabilities through devising strategic partnerships and have adopted more rigorous M&E approaches to assessing these efforts. Notwithstanding some shortcomings, USAID funds strengthened SC as an organization and its capacity to carry out its mission consistent with the goals of the matching grant program—USAID has received a fair return on its investment. Recommendations With the ending of the WCI-2 grant, the following macro recommendations are made: 1. SC should continue its advocacy of gender-sensitive policies in all of its operations and its interactions within the Save Alliance, and with other international and national partners. 2. SC should continue its support for FOs to establish strategic partnerships. Sharing resources achieves savings and more importantly institutional bridges are built with national and local NGOs for serving the needs of women and children. 3. Now that WCI-2 resources have been expended, SC should pursue less costly means for training and TA to its Field Offices (for example, gender relations analysis); Program Learning Groups could be used extensively to conduct some of this training. 4. Hub country Field Offices (FOs) have done an excellent job of developing their M&E systems, especially the ability to collect baseline data for key projects that is time-consuming and labor intensive. However, the FOs should focus on more cost-effective approaches to data collection such as rapid assessment strategies and employing sampling techniques to reduce the population universe. 5. FOs must recognize that M&E is not an end in and of itself; once data are collected and analyzed, the findings should be leveraged in search of other resources to sustain the projects/programs being monitored and evaluated. M&E should always be employed in an “applied context.” 6. Should SC receive future grants such as WCI, more monies should be allocated directly to field budgets for management of country projects and programs. The financial management of WCI seemed unduly complicated and at times managerially confusing to both the FOs and SC Headquarters. 7. Lastly, SC should continue striving to develop an organization-wide M&E system; but in striving to do so, SC may be over-reaching. The PME working group needs to focus on the core ingredients of any M&E system; what information is needed, at what organizational level, by whom, for what, and what level of burden (resources and time) can be absorbed by SC for C:\12-20-working\New Folder\SAVE evaluation final report.doc 3 collecting data? Too much energy is being expended in identifying “performance indicators;” more focus should be given over to figuring out what data are required to carry out SC planning and programming and then determine what data elements can serve as indicators of performance (proxy or otherwise). As important, SC’s senior management needs to integrate field-generated M&E data into its decision-making about resource allocation, fundraising, program focus and program content to make this organization-wide system a worthwhile investment. In summation, this evaluation has documented the gains made by SAVE as a result of the PVC Matching Grant. The organization has made considerable progress in promoting partnerships as a way to better leverage its resources in the field and in accordance with the intent of the grant. SAVE has also established an enviable record in making gender equity a high priority internally and externally. However, in the final analysis, many of the problems discussed in this report are directly attributable to SAVE’s poorly designed DIP. While PVC failed to provide SAVE with a formal review, there were certainly enough warning signs from the field offices that the DIP, as originally designed, was incapable of being implemented. Earlier attention to these issues would have facilitated a smoother implementation of the MG rather than making adjustments so late in the life of the grant. With respect to designing a functioning M&E system for documenting its performance against stated objectives, SAVE has simply taken too long to address this issue. Of course, this is not a problem unique only to SAVE; other PVOs are also struggling with how to measure their performance. Both USAID and their PVO partners need to focus more attention on what is doable and practical in the areas of evaluation and performance assessment. Lastly, the problems that took place with the MG probably could have been ameliorated had there been a better dialogue between SAVE and PVC. Both organizations now need to reflect on the “what went right” with the WCI-2 grant as well as “what went wrong” so as to avoid these mistakes in the future. C:\12-20-working\New Folder\SAVE evaluation final report.doc 4 2.0 EVALUATION METHODOLOGY AND TEAM COMPOSITION The Statement of Work (SOW-see Annex C) called for adopting evaluation methods consistent with assessing how the Woman/Child Impact (WCI) grant has enabled Save the Children-USA to strengthen the technical and organizational capacity of the organization. The evaluation consisted of six operational tasks as described in the Evaluation Workplan provided to PVC. These tasks included: 1. Carrying-out a thorough review of Save the Children (SC) documents and operations; 2. Developing semi-structured questionnaires for conducting interviews with key SC Headquarters staff and key field staff in the countries to be visited; 3. Carrying out interviews with SC Headquarters staff in Westport, CN and Washington, D.C.; 4. Conducting site visits to three of four Hub Countries where the WCI activities were centered over the life of the grant; 5. Processing and analyzing collected data on SC Headquarters and Field operations; and 6. Submitting a draft and final evaluation report to the evaluation Project Manager and Management Systems International for subsequent submission to USAID/BHR/PVC. Team Composition Two persons carried out the evaluation: MSI’s (Management Systems International’s) evaluation expert and a senior manager for Save the Children in the Washington, D.C. office familiar with the WCI grant. Consistent with the cooperative nature of PVC program, this evaluation represents a joint effort of two professionals regarding the successes, shortfalls, and overall lessons-learned from the WCI-2 grant to Save the Children-USA. · William Millsap, Ph.D.—served as the Evaluation Team Leader. Dr. Millsap is an evaluation specialist who has participated in numerous evaluations of USAID projects—including final assessments of PVC matching grants to Mercy Corps and the Salesian Missions. Most recently, he designed and conducted baseline surveys of civil society projects in Albania and Kosovo. · Diana Myers is an Associate Vice President for Save the Children International Development Programs. She has worked with SC for five years and is the resident expert in developing country-based networks with PVOs and NGOs. Evaluation Schedule The activities of this evaluation were done over a four-month period from September through December 2001. Although an initial team-planning meeting was held in mid-September, no evaluation activities were carried out until October. These activities were as follows:1 Sept. 3-11 Basic evaluation planning, including developing the SOW, is carried out by the SC team representative and the Team Leader; 1 Delays were encountered in carrying out the original schedule set forth in the SOW due to the unfortunate events of September 11, 2001. Travel planned for October was put off due to a travel embargo implemented by USAID until security procedures could be reviewed worldwide. C:\12-20-working\New Folder\SAVE evaluation final report.doc 5 Sept. 11-19 Team planning meetings were conducted and visits were made to the SC Headquarters Office in Washington, D.C.; Oct. 1-13 Interviews with SC Headquarters (HQ) Staff were conducted and program/operations documents available in the HQ Offices were reviewed; Oct. 15-19 The SC team representative visited the SC Field Office in El Salvador;2 Nov. 1-17 The evaluation team visited SC Field Offices (FO) in Ethiopia (Nov. 3-10) and Bangladesh (Nov. 11-17); Dec. 9-14 The Team Leader visited to the SC Field Office in El Salvador; and Dec. 15-31A draft final evaluation report was prepared and submitted to the evaluation contractor for initial review and comment; and Jan. 15-31 Comments from the contractor were received and incorporated into the final evaluation report for submission to USAID’s Office of Private and Volunteer Cooperation. Evaluation Methods The evaluation team first developed a SOW detailing the steps to carry out the WCI evaluation. Once accepted by USAID’s Office of PVC, the team began collecting and reviewing the most relevant documents covering the activities of WCI-2. These documents and other materials viewed during the SC Field Office site visits are cited in Annex C-1. The team expected to interview many individuals during the course of conducting this evaluation and decided that the best approach to conducting these interviews would be to develop semi-structured open￾ended questionnaires. This approach would allow the team to capture information with greater structure while at the same time permitting respondents to add greater depth to their answers during the interview sessions. The team developed questionnaires tailored to SC Headquarters staff, SC Field Staff, and for Partner representatives (See Annex C-2). These questionnaires were administered to key personnel knowledgeable about the WCI grant in Westport, CN and Washington, D.C. Similarly, the evaluation team had questionnaires completed by key field staff and SC partners in each of the WCI countries visited by the team. The evaluation team visited three countries; Bangladesh, El Salvador, and Ethiopia.3 During these visits, the evaluation team visited site settings where SC was operating many its own programs along with those of partners (e.g., Early Childhood Health in Bangladesh, the Protection of Minors in El Salvador, and Adolescent Reproductive Health in Ethiopia). The team also conducted unstructured interviews with a variety of beneficiaries participating in SC country programs. Reporting 2 Mission approval was not forthcoming for the USAID representative of the team to visit El Salvador with the SC counterpart; since arrangements had already been made, it was agreed that the SC representative should go ahead and visit the site. The USAID representative would conduct the site visit when Mission approval could be arranged. 3 Under the original grant design, four “Hub” countries were identified where SC would focus WCI activities (e.g., Bangladesh, El Salvador, Mozambique, and Ethiopia). However, the Mozambique Field Office withdrew from participation in the WCI grant in early 2000 as a result of the floods that ravaged much of the country in February 2000. C:\12-20-working\New Folder\SAVE evaluation final report.doc 6 The findings gleaned from the aforementioned primary and secondary data sources form the basis of the WCI-2 grant evaluation for USAID/PVC. The resulting report has been prepared in accordance with a standardized evaluation format developed by MSI. This format should achieve improved consistency among the final grant evaluations being carried out by MSI under contract to USAID/PVC. C:\12-20-working\New Folder\SAVE evaluation final report.doc 7 3.0 MATCHING GRANT BACKGROUND 3.1 Historical & Technical Context and Partners Save the Children Federation, Inc. (SC) is a nonprofit, nonsectarian organization working to improve the quality of the life of disadvantaged children and their communities in over 30 countries, including the United States. SC’s programs are designed and implemented according to the organization’s program principles -- child-centeredness, gender equity, sustainability, empowerment, scaling-up and measurable impact -- which were developed with assistance from the first USAID/BHR/PVC-funded Woman/Child Impact grant (WCI-I). This grant also led SC to articulate and define its five implementation strategies: direct implementation, community mobilization, partnership, institutional development and advocacy. 4 By 1996, the Woman/Child Impact (WCI) framework was firmly established in SC’s international program activities: economic opportunities, education, health, and humanitarian response. However, SC staff felt that more needed to be done in terms of: 1) developing staff skills and available tools to increase program sensitivity to the challenges faced by women; 2) designing and implementing a system that could monitor the impact of programs on women’s status and children’s well being; 3) understanding the nature of partnerships; and 4) seeking to influence national policies affecting women and children. Thus, a second grant application was submitted. The current grant, Woman/Child Impact-II (WCI-2), is funded through a follow-on cooperative agreement between PVC and Save the Children. Signed October 4, 1996, the agreement was to conclude September 29, 2001. The goal of the grant was “to contribute to sustainable human development through the systematic economic and social empowerment of children, women and other disadvantaged groups by expanding their participation in decision making at all levels of society. And that empowerment is achieved through collaboration with local organizations committed to democratic pluralism, gender equity, and economic growth benefiting disadvantaged women and children.” To contribute to this overarching goal, the program planned a wide spectrum of activities at both the organizational and program levels: Organizational-level activities included: · Developing organizational capacity for impact evaluation, emphasizing cost-effectiveness and sustainability; · Continuing to strengthen the Economic Opportunities Office started under WCI-I; · Promoting self-reliance by strengthening local staff and local organizations’ capacity to represent themselves and their communities nationally and internationally, through skills building and strategic networking such as that undertaken through the FACETS (Family and Community Empowerment Training Systems)5 initiative; · Strengthening SC’s program management structure, further defining and utilizing SC’s program 4 The first Woman/Child Impact grant was a five-year award for $5 million between the years 1991 – 1996. The objectives of this grant were to assist SC Headquarters in strategic planning and providing technical support for field offices in Bangladesh, Bolivia, Haiti and Mali to design programs with greater emphasis on and sensitivity to the needs of women. The program principles and strategies articulated under WCI-1 now constitute the fundamental framework for SC/US programming. Only Bangladesh continued as a “Hub” country in WCI-2. 5 FACETS teams were formed under WCI-I in Latin America, Africa, South Asia and Southeast Asia. Team members attended two annual skill-building meetings under the first WCI grant; three additional annual meetings under WCI-II focused on continued training in how to conduct and analyze the results of country-level gender relations analyses. C:\12-20-working\New Folder\SAVE evaluation final report.doc 8 principles through out the organization; and · Strengthening collaboration between organizations committed to democratic pluralism, gender equity and economic growth benefiting women and children and establishing and institutionalizing gender policies and practices. Program-level activities during the grant’s first three years included: · Expanding the impact of mutually reinforcing programs through multi-level strategic partnerships; · Documenting four strategic partnership models, one in each Hub country: Ethiopia, Bangladesh, El Salvador, and Mozambique; · Measuring the cost-effectiveness and sustainability of the four WCI strategic partnership models; · Developing analytic tools and methodologies to assess and increase cost-effectiveness and sustainability; · Measuring program impact with respect to economic and social empowerment of children, women and other disadvantaged groups; and · Influencing national policies that affect the lives of women and children. By the grant’s third year, SC staff at HQ and in the field had experienced many difficulties in implementing the DIP. With PVC’s participation, SC therefore developed a Results Framework for the WCI-2 grant6 which: · Restated the grant’s objectives in a more concise way; · Linked WCI grant objectives and components to a broader capacity-building results framework for the International Programs Department of SC; and, · Shifted emphasis among the objectives to better reflect current judgments about the most urgent capacity-building needs within International Programs. Although the intent of the grant remained the same, and the major elements as originally written are still reflected in the intended results, the focus of the grant’s final two years became: 1) Increasing SC’s involvement in promoting policies and practices that positively affect women and children; 2) Increasing capacity in planning, impact monitoring and evaluation; 3) Fostering, managing and learning from partnerships; and 4) Further incorporating gender perspectives into SC’s program practices. Undertaking this shift was deemed critical to effect and sustain the changes WCI had initiated or intended to initiate within programs around the world. Notwithstanding the changes implemented in response to the midterm evaluation, the final evaluation focuses on assessing the overall results of WCI-2 over the entire five years of the grant. 6 More will be said about this “realignment” of the grant in the management section of this report. C:\12-20-working\New Folder\SAVE evaluation final report.doc 9 3.2 Project Goal, Objectives, and Major Hypotheses to be Tested The project goal in SC’s grant application is restated as the goal in the Detailed Implementation Plan (DIP) submitted to USAID in March 1997 (see Table 3.1 below).7 Hypotheses in the application are not explicit, but can be inferred as being: 1) Strengthened partnerships will increase the developmental reach and cost-effectiveness of SC programs; 2) Integration of the Woman/Child Impact approach within SC’s evolving management structure will enhance program cost-effectiveness and impact; and 3) Partnerships with grassroots, national and international organizations can achieve more and better positive impact for women and children. Table 3.1 also lists the hierarchy of objectives presented in the DIP. These objectives are proposed for the life of the project 1997-2002. Home Office objectives are listed as well as the objectives for each Hub country at the area, national and community level. 7 USAID Matching Grant Application - FY 1996, by Save the Children, Westport, CT. Table 3.1 Project Hierarchy of Objectives WCI Project Goal Sustainable development and the empowerment of women, children, and other disadvantaged groups through collaboration among local and national NGOs, coalitions, governments, and private organizations working in support of gender equity, democratic-pluralism, and economic growth. C:\12-20-working\New Folder\SAVE evaluation final report.doc 10 WCI Project Objectives at Headquarters and Hub Countries Headquarters and Worldwide 1-Stimulate and provoke debate on theories and practices that impact women and children at multiple levels across sectors; 2-Strengthen FACETS teams by actively participating in and contributing to at least two international fora; 3-Support innovations in cost-effective programming, measurement and scaling-up; and 4-Raise SC’s standard of engagement in policy and global dialogue on issues concerning woman and children. 5-Develop at least four intersectoral program models of structured partnerships with NGOs and multilateral organizations, one in each Hub Country; 6-Further incorporate WCI technical services to integrate focused sector strategies into program vision and management of systems and structures; and 7-Establish and institutionalize a gender policy. Bangladesh/Area 1-Support evolving SC area structure; 2-Strengthen Hub Country capacity to have impact on at least two other countries in the South Central Asia Area; and 3-Strengthen child focus of WCI in Bangladesh and South and Central Asia Area. Bangladesh/National Level 1-Actively enter into partnerships with local NGOs and research bodies, international organizations and government for programming, research and advocacy; 2-Document SC’s experience with structured partnership; 3-Foster Early Childhood Development; and 4-Test, refine and expand Health Financing System (HFS). Bangladesh/Community Level 1-Improve status of women and increase women’s participation and decision-making at the household, village and local political levels. El Salvador/Area 1-Support the evolving SC area structure; and 2-Increase the coordination among the Hub countries in order to strengthen staff capability and program impact. El Salvador/National 1-Identify, formalize and implement structured partnerships; 2-Demonstrate the reach of program impact; 3-Evaluate the effectiveness of structured partnerships; 4-Document experiences in the programming of structured partnerships; and 5-Incorporate the Articles of the Convention of the Rights of the Child into SC program activities. C:\12-20-working\New Folder\SAVE evaluation final report.doc 11 El Salvador/Community 1-Establish programs that systematically address the issues of girls, boys and adolescents to facilitate their positive integration into El Salvadoran society; 2-Review, redesign and reinforce the CENET programs to improve maternal and child health and stimulate the integrated development of children 0-6 years; and 3-Develop and implement methods to work with adolescents focusing on reproductive health, life education, and vocational education. Ethiopia/Area 1-Support evolving SC area structure; and 2-Strengthen area (Hub) capacity to have impact on at least two other countries in Hub support area. Ethiopia/National 1-Identify, formalize and implement structured partnership models; 2-Demonstrated increased program impact from structured partnerships; 3-Demonstrate the viability of structures partnership model; and 4-Document SC program experience with structured partnership. Ethiopia/Community 1-Increased women’s decision-making and status at the household level; 2-Increased alternatives for children’s growth and well-being and adolescents awareness of and ability to act on life options; and 3-Increased participation of women in political economic and social policies and process. Mozambique/Area 1-Support evolving SC area structures; and 2-Strengthen area (Hub) capacity to have impact on at least two other countries in hub support area. Mozambique/National 1-Build or strengthen field capacity to monitor and evaluate program impact; 2-Identify the partnership model in each hub within first two years of grant (i.e. written agreement exists); 3-Demonstrate the use of a structured partnership as a means for scaling up impact; 4-Integrate UN Convention of the Rights of the Child within field office and partner agencies; and 5-Document field experience, training, system with impact on women and children. Mozambique/Community 1-Increased women’s decision-making and status at the household level; 2-Increased alternative for children’s well being, and adolescents awareness of and ability to act on life options; and 3-Increased participation of women in economic and social policies and processes. C:\12-20-working\New Folder\SAVE evaluation final report.doc 12 As noted above, SC WCI Managers in consultation with the USAID/PVC Manager decided in 1999 to realign the WCI grant to be congruent with USAID’s Results Framework approach, therein focusing more on results rather than program activities or processes, with clear indicators of performance for measuring grant outcomes. The resulting realignment of the WCI grant is displayed in Appendix 1.8 Chapter 5 provides further discussion on the inherent difficulties with the WCI-2 DIP and the subsequent advantages to adopting the Results Framework. 8 Source: WCI Grant Realignment Documents, Save the Children, Washington, DC, March 2000. 13 4.0 PURPOSE OF THE EVALUATION USAID/BHR/PVC commissioned this final evaluation of the WCI-2 grant to assess its impact in the field and at SC’s headquarters (Westport and Washington) and to fulfill the requirements of the Matching Grant (MG) program. The purpose of the evaluation is to assess how the grant has strengthened the technical and organizational capacity of Save the Children and through its local partnerships has enabled local organizations to achieve sustainable service delivery. The evaluation will articulate achievements and lessons learned in order to provide recommendations to improve the programming of future Matching Grants for BHR/PVC, as well as to help SC continue enhancing its capacity for continuous program/impact improvement. Potential stakeholders of this evaluation, in addition to USAID/BHR/PVC and SC, include local and international partners of SC as well as the women and children who have been affected by this grant over the past five years. SC Managers expect to share the evaluation findings with the general management of the organization, and very broadly with international program staff in the Field and Headquarters. Field staff in Hub Countries will in turn share relevant findings with NGO colleagues in their strategic partnership initiatives. Dissemination will occur over the course of the next six to eight months, through email, routine meetings, program learning groups and other gatherings already planned. Specifically, the findings will be used to inform SC and USAID Managers regarding: 1. The formation of a planning, monitoring and evaluation function within the Executive Office; 2. SC’s implementation of the plan for gender mainstreaming, consistent with the International Save the Children Alliance gender policy; 3. Continuing worldwide program improvement through tools created with WCI support; 4. The updating of three-year strategic plans and one-year operational plans (area, country and technical office) in the course of the coming year; 5. The Every Mother, Every Child High Impact Initiative, a decade-long campaign for which SC has articulated ambitious program and advocacy goals; and 6. A general assessment of the direct and indirect impacts of the WCI-2 grant. This final evaluation fulfills the requirements of the USAID/BHR/PVC Matching Grant Program. PVC will use this information in its annual Results Report, the review of any follow-on proposals and in distilling “learning” for a broader application. 14 Evaluation Issues The evaluation team focused on identifying major successes, constraints, lessons learned and unanticipated effects encountered by SC in implementing and managing the WCI-2 Grant. In particular, the evaluation assessed (A) program implementation and (B) program management efforts to achieve the goal(s) of the program. The following major issues were assessed in each of these areas. A. Program Implementation 1) Progress achieved in each major objective cited in the logframe or program planning matrix; 2) Extent to which progress has been made in achieving sustainability; and 3) Status of local partnerships with NGOs, community based organizations or local level government. B. Program Management 1) SC’s management capacity and its partnerships to carry out the activities called for under the grant; 2) USAID’s oversight and backstopping of this cooperative agreement; and 3) Important management lessons learned and recommendations for improving future CAs. 15 5.0 PROGRAM IMPLEMENTATION EVALUATION QUESTIONS This chapter discusses a range of issues that transpired during the implementation of the second WCI grant. Like the first WCI grant, the overall purpose of the follow-on award was to support efforts to empower women in having control over decisions directly affecting their lives. The second grant was intended to improve on the successes of the first grant through the continuation a gender equity approach in all of SC’s programs; implementation of FACETS to sensitize field offices in designing projects inclusive of women’s needs as well as those of children; ongoing use of SC’s program principles (child￾centeredness, gender equity, empowerment, sustainability, scaling-up and measurable impact) in strategic planning and programming; and, lastly, improving the organization’s efforts to document its project/program results. 5.1 The Detailed Implementation Plan The DIP submitted by SC in March 1997 was intended to provide an in-depth schematic of how the grant was to be carried out by SC Headquarters and the Field Offices participating as Hub countries in WCI-2. Unfortunately, there was a considerable disconnect between the WCI-2 MG proposal and the implementation details provided in the DIP, necessitating a realignment of the grant in early 2000. The shortfalls in the DIP are discussed below along with the steps taken by SC to correct perceived deficiencies. The revised DIP is attached as Annex B. 5.1.1 DIP completion and DIP accuracy Findings: (a) The original DIP for the grant was never completely implemented and its inadequacies were discussed during a conference on Strategic Partnerships held in Guatemala in late 1998, the start of the grant’s third year.9 (b) The grant’s midterm review in Spring 1999 reinforced this thinking and invited SC to proactively realign the grant and grant-related private resources in order to achieve the highest-priority results. In addition, the high turnover of grant-funded staff and the number of staff who had joined SC since the grant began provided further impetus to realign the grant and commit to revised results. (b) Thus, relatively late in the life of the grant, SC made changes in the project design. These changes were achieved through the “WCI Forum” process, which took place between May and October of 1999.10 These intensive staff consultations, in the Home Office and the field and with BHR/PVC, resulted in the development of a Results Framework (see Figure 3.1) that took the place of the original DIP. Conclusions: (a) Despite concerns about the DIP raised in November 1998 at the WCI Inter-Hub Country Workshop in Guatemala and other issues raised in the Midterm Evaluation (April 1999), it took SC almost a full year (March 2000) to complete the realignment of the WCI grant—also complicated by USAID’s slow response. (b) The shift to a RF framework in early 2000 is much closer to the actual activities taking place in 9 See Strategic Partnerships Development, Management and Measurement, WCI Inter-Hub Country Workshop, Antigua, Guatemala, November 6-9, 1998. At this workshop, participants expressed considerable dissatisfaction with the management and implementation of WCI-2 grant, the nebulous aspects of the objectives laid out in the 1997 DIP and the lack of program continuity due to staff turnover. The Evaluation Team confirmed these concerns in interviews with key staff at Headquarters and field staff in the fall of 2001. 10 WCI Realignment Documents, Save the Children, Washington, DC, March 2000. 16 the Hub countries. Recommendations: · Greater attention should be paid to the original proposal and the subsequent design laid out in the DIP. 5.1.2 Quality of DIP and degree of success in implementation Findings: (a) As listed in Table 3.1, the original WCI-2 DIP listed an extensive set of objectives. However, the majority of these objectives should have been categorized as “activities.” Indeed, they failed to meet the basic definition used by USAID/PVC regarding what constitutes an objective11: “An objective is a statement of what the program plans to achieve during the life of the project; it is the highest level result that a program can materially affect with its efforts, and objectives should be clear, precise, objectively measurable, and plausibly linked to a program goal.” Most of the DIP objectives were not clearly linked to program/project results, and were imprecise and not capable of being objectively measured. Throughout most of 1999, SC managers had an ongoing dialogue about the specious nature of the DIP and SC’s ability to accomplish its many and varied objectives/activities.12 (c) The structure of the DIP was overly complex with little likelihood that the cited objectives/activities could be objectively measured. SC HQ and Field Offices in the Hub countries attempted to collect performance data on their respective program objectives and project outcome targets; however, given the generally poor delineation in the DIP between objectives and activities, the data collected were essentially little more than qualitative narrative that failed to address the issue of performance vis-a-vis DIP objectives.13 Virtually no quantitative data was in evidence in the WCI-2 Annual Reports. (b) No evidence was found that USAID had ever conducted a formal DIP review and shared written comments with SC; apparently, USAID only informally provided approval of the WCI DIP;14 and (c) The Midterm Evaluation alerted SC managers and their USAID counterparts that project implementation was proceeding slowly and with difficulty but failed to provide any comments on the DIP. Conclusions: (c) The original DIP was overly ambitious and poorly designed, and lacked sufficient rigor to objectively measure project outcomes and impact. (d) Despite each Field Office providing the country sections of the DIP, field staff had considerable doubts about whether country activities could validate the general hypotheses laid out in the DIP.15 Indeed, field staff indicated that the grant objectives were unclear and needed to be revised, and the expectations in the DIP for collecting information on cost-effectiveness left 11 See USAID/PVC Matching Grant Application Instructions, August 1999, 2000, pp. 26-27. 12 Interestingly, the WCI Midterm Evaluation only briefly discusses the DIP (p. 14). No mention was made of the degree of dissonance that prevailed among SC HQ managers and field staff regarding the extensive and complex list of objectives. 13 This finding is based on a review of the WCI-2 Annual Reports (1996-98, 1998-99, and 1999-2000). 14 Personal communication with the Midterm Evaluator of the WCI-2 grant. A search of SC’s WCI documents did locate some internal memos on the illogic and complexity of the DIP design, the elaborate list of objectives and the general difficulty of measuring these objectives but these views did not surface in the Midterm Evaluation nor SC HQ responses to the Midterm Evaluation. 15 Personal communications and information provided in the field questionnaires. 17 field staff uncertain how to carry out this activity.16 (e) USAID was derelict in failing to provide formal comments to SC on the quality of the DIP and suggesting changes to improve its design logic; and Recommendations: (a) USAID must provide formal and timely feedback by way of a DIP review if it is to hold NGOs accountable with respect to implementing their projects/programs in accordance with their DIPs; and (b) Problems in DIP design and implementation issues should be identified by NGO staff and USAID staff within 12-18 months of MG funding; and (c) Both USAID and SC were lax in coming to a conclusion that the WCI grant needed to be realigned—the realignment should have occurred much earlier in the life of the grant. 5.1.3 Familiarity with DIP and design Findings: (a) Hub country staff were quite familiar with the DIP and its design for their country since they were involved in developing their own country sections. However, field personnel were generally confused over the hypotheses stated in the DIP having to do with rates of return, women achieving influence in decision-making and measuring project results. (b) Field staff appeared to be far more comfortable with the realigned DIP in the RF format; and (c) SC HQ staff were vague in their recall of the original DIP and like their field counterparts were more knowledgeable about the WCI RF. Conclusions: (a) Comments from both SC field and HQ certainly indicated that the DIP was confusing to readers; despite these concerns, the DIP was sent on to USAID without a thorough internal review; and (b) Relationships between the WCI-2 manager at SC and the MG manager at USAID were too cordial and allowed for a degree of informality that precluded a thorough and critical review of the DIP. Recommendations: (a) USAID should insure that each MG manager fully understands the DIP and its design logic; this should also be extended to situations where new MG managers replace departing managers; and (b) For future grant applications, SC should designate a professional capable of properly reviewing proposals and the development of any Detailed Implementation Plan with respect to design rigor and measurability. 5.1.4 Major successes and shortfalls in implementation Table 5.1 Major Successes and Shortcomings in Implementation Implementation Experience at a Glance Major Successes Major Shortcomings 1. Promulgating gender equity as a policy 1. DIP was unclear and nebulous 2. Three FOs established viable partnerships 2. Staff turnovers created delays 16 Source: questionnaires administered to SC FO staff, October 2001. 18 3. WCI-2 supported many TA workshops 3. Realignment into RF took too long 4. Hub countries established M&E systems 4. HQ still has no M&E system in place Findings: (a) Based on visits to the Hub countries, each Field Office has adopted gender equity policies for office operations and each FO has also sought to promulgate these policies to their partner organizations; HQ staffing has also become more sensitive to gender in its staffing patterns. (b) Each Hub Country was successful in developing viable partnerships with local and/or national level NGOs—ranging from “Adolescent Reproductive Health” in Ethiopia, the “Protection of Minors” in El Salvador, and “Early Childhood Development” in Bangladesh;17 (c) WCI funds supported numerous training activities not only for Hub country staff but also staff from other SC countries. These workshops addressed issues affecting women and children such as violence, empowerment, and gender mainstreaming. Technical workshops on strategic planning and monitoring & evaluation were also supported. (d) Each of the Hub countries visited developed viable M&E systems, albeit at different levels of sophistication and utilization, and used WCI funds to support a M&E staff person; and (e) SC HQ has been far less successful in establishing an operating M&E system; while WCI funds were used to support an M&E expert, this position continued to have turnover problems.18 In the first WCI grant, the evaluator noted that no M&E system was in place to adequately measure the impact of SC programs,19 this continues to be the case at SC HQ— although some sector programs have done a good job in developing performance indicators (e.g., health, economic opportunities and education). Conclusions: (a) SC as an organization has made considerable strides since WCI-1 in advocating the “six principles” as organizing concepts in SC planning and programming; (b) SC Field Offices demonstrated initiative and creativity in identifying potential partners and establishing viable partnerships in accordance with the WCI-2 goal of leveraging resources and achieving sustainable programs after the completion of the MG; and (c) SC HQ continues to have difficulty in discerning its M&E needs and acquiring the necessary expertise to implement a data-driven system. HQ managers appear to be emphasizing planning and programming as a surrogate for a reliable data based performance monitoring and evaluation system. Planning is inextricably linked to M&E but cannot stand as a solitary exercise. Recommendations: (a) SC should continue to promote the incorporation of its “six principles” as organizing concepts in programming; (b) Given the successes with partnership demonstrated under WCI-2, SC should expand its efforts to establish viable partnerships that are capable of being sustained and expanded; (c) SC must given greater attention to establishing a PME system capable of providing managers with critical information for planning and operating their programs; and 17 Although a designated Hub country, Mozambique’s FO partnership was not assessed since the country discontinued its participation in the WCI grant in February 2000. 18 Over a five-year period, four different individuals filled this position. The position has remained unfilled since June 2001. 19 See Final Evaluation, Woman/Child Impact Grant-1, 1991-1996, USAID and Save the Children Federation, Phil Boyle, Automation Research Systems, December 15, 1995, pp. 4-5. 19 (d) SC should also provide technical assistance to FO managers on how to use M&E information for leveraging resources and scaling-up their projects and programs. 5.1.5 Impact Results The original DIP listed a host of indicators for assessing the impact of WCI-2. Unfortunately, as noted earlier, these indicators were actually activities and not reliable impact indicators. Moreover, the WCI-2 Annual Reports provided little to no data on these indicators. But with the realignment of the DIP into the Results Framework, performance indicators were developed for demonstrating the impact of the MG. The impact of WCI-2, according to SC’s final report, noted, On an organizational level, WCI has had an indelible imprint. A key outcome of SC’s strategic planning in 2000 is a set of four High Impact Initiatives that embody the program goals, strategies and standards that WCI has promoted for 10 years.20 In the Final Report on WCI-2, extensive discussion is provided on the impacts of the MG—although hard data are lacking. 5.2 Assessment of project model and hypotheses 5.2.1 Project hypotheses articulated in CA Findings: (a) The WCI-2 program strategy sought to empower women and children by collaborating with local/national NGOs, coalitions, governments and private organizations working in support of gender equity, democratic pluralism and economic growth. The underlying hypothesis of the WCI project presented in the MG proposal was “ . . .the empowerment of disadvantaged groups strengthens their capacity to influence decisions that directly affect them;” (b) However, the more general hypothesis of the MG proposal was refocused in the DIP to that of building the capacity of national level NGOs to implement and sustain programs that benefit women and children. Emphasis was directed at increasing the rate of return for each development dollar. The restated hypothesis became: “the rate of return per dollar invested at the national and higher levels is significantly higher than the rate of return per dollar invested in expanding impact through local capacity-building.” Continuing, the DIP provided formulas for measuring the results of the foregoing hypothesis arguing that “any combination of relevant sector interventions plus any kind of structured partnership results in greater change in a dependent variable (e.g., women’s empowerment) across multiple levels, than sector interventions without structured partnerships;” 21 (c) The hypothesis dealing with “rate of return for dollars invested” is actually rather interesting and capable of being tested. The intent was to assess the cost-effectiveness of conducting more of SC’s operations through the establishment of partnerships. However, according to the Midterm Evaluation (p. 13), it was assumed that the WCI Impact Evaluation Specialist would be responsible for this component of the grant. Little guidance was provided by SC HQ on this area of inquiry and when the DIP was recast into a RF format, cost-effectiveness was dropped altogether; 22 20 See Draft Final Report, Woman/Child Impact-2 Grant (1996-2002), Save the Children, Washington, DC, 2002, p. 1. 21 See the Detailed Implementation Plan, Woman/Child Impact Grant 1997-2002, Save the Children, Cooperative Agreement No. FAO-A-00-96-00003, pp. 6-10. 22 See “WCI Realignment Documents,” Save the Children, Washington, DC, March 2000; also Figure 3.1 in Chapter 3 of this document. 20 (d) The supporting hypothesis of asserting “any combination of relevant sector interventions . . . etc.” is well, silly. Simply put, if you do anything by way of a structured partnership to help women, it will enhance their empowerment as opposed to carrying out interventions without a partnership. In sum, the statement is a poorly stated hypothesis at best not capable of being adequately tested, and at worst sheer babble; and (e) Notwithstanding the hypotheses presented in the DIP, SC Field Offices did manage to establish viable partnerships at local and national levels that are proving to be sustainable entities. More will be said about these partnerships in Chapter 6 of this report. Conclusions: (a) The hypotheses stated in the original DIP were discarded as part of the realignment of the WCI grant into a Results Framework document; In retrospect, these hypotheses were unintelligible, poorly understood by SC HQ or FO personnel, and not easily measurable—if at all; and (b) USAID failed to review or call into question the specious nation of the DIP hypotheses—in short, they ended up as window dressing with no utilitarian value to carrying out the WCI project. Recommendations: (a) In subsequent MG applications, USAID and the respective PVO should closely scrutinize whether stated hypotheses are testable and/or falsifiable, and if the proposed budget is adequate to collect the data needed to validate a given hypothesis. 5.2.2 Replication and scale-up of approaches in project area or elsewhere Findings: (a) As defined by SC, scaling-up refers to the use of strategies that have far reaching impact on larger numbers of women and children—essentially expanding programs while at the same time maintaining high program quality.23 In general, the concept is understood by SC staff but in practice remains rather elusive in terms of discernible outcomes. Certainly, SC will continue to advocate gender equity in a broader context along with establishing partnerships but the quantity and quality of M&E systems capable of assessing the efficiency and effectiveness of SC programs is still uncertain. (b) SC will continue establishing and expanding partnerships in its programs in the future— indeed, the WCI draft final report notes that SC has been moving away from direct service delivery to working more through partnerships. In fact, “no major programs in SC’s portfolio are implemented by SC alone;” and 24 Conclusions: (a) SC has been successful in promulgating to staff throughout the organization the appropriateness of gender equity as an organizing principle in all of its programs. The first WCI grant clearly articulated the need and WCI-2 has achieved organizational acceptance of the approach—which was practically applied in the Hub countries visited. Both grants helped to introduce practical skills in gender relations analysis; and (b) Partnership will be a major vehicle of intervention for many future SC programs. 23 Save the Children’s Key Program Principles, undated memo copy; scaling up is discussed as the process of carrying out partnerships, promoting advocacy and achieving more focused programs with larger impact. 24 Draft Final Report, Woman/Child Impact-2 Grant (1996-2002), Save the Children, Washington, DC, 2002, p. 19. 21 Recommendations: (a) SC should continue focusing on improving its organizational systems such that M&E plays a more vital role in scaling-up country programs—and in making corporate decisions about resource allocation, fundraising and program priorities. 5.3 Advocacy under the project 5.3.1 Advocacy activities and impact Findings: (a) While the term “advocacy” was never used in the WCI-2 grant proposal, the grant nonetheless sought to influence national policies on women and children and advocacy was therefore implicit; (b) SC’s current Strategic Plan features advocacy as an integral and critical ingredient to the organization’s worldwide efforts to promote programs aimed at improving the lives of women and children; and (c) The realigned RF identifies “External Policy Involvement” as an Intermediate Result (IR); in fact, “twelve of 14 responding field offices observed an increase in SC’s involvement in policy dialogue over the past three to five years (two were uncertain).”25 Conclusion: (a) Advocacy, under the auspices of the WCI-2 grant, has become an integral activity in all aspects of SC’s program implementation—ranging from gender mainstreaming in SC program planning and hiring policies, to promulgating gender equity with international and national level partners with which SC is working. 26 Recommendation: (a) SC should continue its advocacy efforts to improve policies affecting women and children. 5.3.2 Partner/PVO roles in advocacy Findings: (a) SC policy is to advocate its policies (e.g., attention to gender issues, women to be viewed in concert with programs for children, the advantages of partnering and the need for accountability through M&E efforts) in choosing and working with partners in the field. All local partners interviewed during the site visits stated they were now involved in advocating policies within their own organizations, consistent with SC policies. (b) In adopting SC policies, partners have often improved their own capacities, thereby leading to gains in funding and more importantly giving local partners an expanded profile from which to advocate for their particular projects or programs. As one local NGO put it, “We have shifted from being a micro-level player to that of a national player in providing services to women and children.”27 Conclusion: (a) Advocacy has become an inherent part of SC’s practices for improving the lives and women and children. In SC’s efforts to promote local/national partnerships, a natural transcendent effect has taken place among partners in which greater gender sensitivity is occurring along with acceptance of the importance of partnerships to maximize the use of available 25 Ibid., p. 8. 26 See the discussion on advocacy in the draft Final Report, Woman/Child Impact-2 Grant (1996-2002), Save the Children, Washington, DC, 2002, pp. 8-10. 27 Personal communication with local NGO partner in Bangladesh. 22 resources. Recommendation: (a) SC should continue its advocacy efforts with its partners and in turn support the advocacy efforts of these partners in those programs that complement and/or reinforce SC’s programs for women and children. 5.4 Implementation Lessons Learned Through implementing this matching grant, SC learned the following lessons: (a) Partnerships invariably result from personal relationships between individuals rather than the signing of formal agreements between institutions—it is important that personal contacts be maintained and nurtured to insure the continued viability of partnerships; and (a) Developing viable partnerships in Third World settings is a lengthy and complex process requiring patience and personal interaction in deference to more formalized approaches in the West; and (b) The expectation of establishing a global PME system proved to be unrealistic due to reasons of proper planning, implementation design and cost. These lessons will be addressed in greater detail in the management sections of this report. 23 6.0 PARTNERSHIP QUESTIONS During a decade of PVC support for two WCI grants, the concept of partnership—and moving away from direct service delivery—has been thoroughly accepted throughout Save the Children.28 WCI-1 (1991-96) helped SC articulate its five strategies (direct implementation, community mobilization, partnership, institutional development, and advocacy) and WCI-2 focused more attention on advocacy, gender equity issues and the formation of local/national partnerships. For the second WCI grant, SC placed greater emphasis on the creation of strategic partnerships29 defined as “concurrent institutional relationships formed at community, national, or international levels in order to increase impact around a clearly defined policy or practice that incorporates two or more program areas of Save the Children.” Subsumed within this definition was the expectation that “binding written agreements” would solidify the partnering experience. The underlying purpose of this approach was to increase the capacity and scale of SC programs. In fact, in a recent baseline end of grant survey of SC Field Offices (August 2001) conducted by SC HQ, almost all FOs reported they were engaged in partnership arrangements of some type.30 With respect to the WCI-2 grant, each Hub country was to design and initiate one primary partnership scheme with a local/national NGO or governmental entity. While each country established more than one partnership, only the primary schemes are discussed below for each of the Hub countries with the exception of Mozambique that had to withdraw from WCI participation in February of 2000. 6.1 Analysis of WCI Partnership Schemes Bangladesh In 1997, the WCI grant enabled the Bangladesh Field Office (BFO) to create an Early Childhood Development (ECD) Unit based in Dhaka in partnership with an NGO (Plan International), a national NGO31 and a local level NGO (Phulki) working in the field. Although SC/US has operated ECD programs in rural areas of Bangladesh since 1994 – long before the strategic partnership model developed – this model enabled the BFO to further promote ECD in urban areas and to advance the “state of the art” of ECD programs nationally and regionally. The project rationale rested on three basic premises: 1) the acknowledged value of early childhood education, 2) the need to develop greater specialization in ECD within the NGO sector and 3) consolidating existing ECD activities through capacity-building efforts among several implementing organizations. The key objectives of the project were to: · Develop, produce and disseminate a range of ECD materials for training trainers, supervisors, teachers and center staff; · Develop training modules and a training team able to provide effective ECD training support to continually strengthen their partners’ ability to implement a range of ECD activities; and · Establish an ECD unit to serve as a resource for NGOs and government organizations for the purpose of strengthening their institutional capacity in ECD. The first formal agreement was drawn up between Plan International and SC/US; this permitted the 28Partnership is defined as “a type of institutional relationship in which two or more organizations work together to achieve mutually defined goals on mutually accountable terms;” see Partnerships and Institutional Relationships: A Resource Book for Save the Children (draft), November 2001, p. 6. 29Originally referred to as “structured partnerships” in the Matching Grant. 30Nineteen of 20 countries as reported in “Baseline Survey Questionnaire Responses,” August 21, 2001. 31Gono Shahajjo Sangstha (GSS), a large Bangladeshi education-focused organization. 24 transfer of funds from Plan to SC for operating the ECD Unit. Two-year agreements were then entered into with national and local NGOs by SC—it was envisioned that the ECD Unit would become part of GSS at the end of this period but for reasons of poor leadership and the lack of funding, this transition never occurred. The original strategic partnerships under WCI-2 were dissolved in early 2001. As a result of the ECD partnership, many national and international organizations in the country have adopted the BFO’s approach to parenting and care-giving practices. The development of a strong group of skilled ECD trainers in both the BFO and its partner organizations will help ensure the continuity of ECD programming and training. The BFO estimates that the ECD program has to date reached approximately a million mothers either directly or through its partners. There continues to be widespread support in local communities for ECD activities. With the completion of WCI, ECD staff and activities have become closely integrated with other SC programs—notably the Nutrition and Child Development Program. El Salvador The focus of SC’s El Salvador Field Office (ESFO) has been on the promotion of non-violence and gender, respect for children’s rights and access to basic education and health care for children. WCI funds enabled the ESFO to leverage additional funds to support new initiatives in these areas. During the first phase of the WCI grant’s implementation (1997-99), progress was enhanced by outside funding from the Merrill Lynch Foundation (and other sources) to conduct workshops for parents on changing attitudes on “gender and violence” through early childhood education. These workshops were incipient steps that led to a partnership with Plan International in 1998 to continue supporting programs in early childhood education on national and regional levels. This partnership evolved into a strategic relationship formalized by a written agreement among the three parties (SC, Plan International and the Salvadorean Institute for the Protection of Minors [ISPM]). Through this partnership, the ESFO has worked with local news media (TV, radio and newspapers) to address the needs of women and children on the issues associated with ECD. It has contributed to major changes impacting children and women on a community level and has promoted internal (SC’s International Programs) and external policy and program changes in early childhood education on national, regional and international levels.32 The ongoing partnership has proven to be rich and productive for all of the partners. The ESFO has documented the impact of the partnership. Training has been provided to technical and support staff in all three partner institutions, including training for 90% of the mother caregivers in the country (approximately 750), reaching over 7,000 children under age seven and training for SC staff in Nicaragua and Honduras seeking to develop regional standards for early childhood education programs. The partnership has also been instrumental accomplishing the following:33 · Establishing minimum quality and safety standards for government-supported early childhood programs that also promote inclusion of more disabled infants and children and the participation of parents in a Head Start type of setting; · Raising awareness among 99 mayors and 350 community leaders of the importance of parental involvement in early childhood development and its relationship to violence reduction and increased well being; and · Promoting networking among more than 30 national institutions representing various facets of early childhood development and education, including the health and communications sectors. 32 Personal communication with ISPM and ESFO Managers, along with data derived from completed field questionnaires. 33 Strategic Partnerships Development, Management and Measurement, WCI Inter-Hub Country Workshop, Antigua, Guatemala, November 6-9, 1998. 25 Ethiopia During the first three years of the WCI-2 grant (1997-99), the Ethiopia Field Office (EFO) initiated two programs as potential candidates for application of the strategic partnership model: Harmful Traditional Practices (HTP) and Adolescent Reproductive Health (ARH). At the end of 1999, the ARH program was chosen to test the strategic partnership model. This project now operates in 25 government high schools in Addis Ababa to prevent, and mitigate the impact of, reproductive health problems. The partners include school administrators, adolescents, the Ministries of Education, Health and Labor and Social Affairs, the Institute for Curriculum Development and Research, national, regional and zonal population offices, Addis Ababa University, the Psychology Association, the Ethiopian Women Lawyer’s Association and other appropriate NGOs. The purpose of the ARH program was to promote safe sexual and reproductive health behavior in the slums of Addis Ababa — targeting high school youth with information about preventing unwanted pregnancies, reducing maternal death, eliminating unwanted abortions, and reducing the growing incidence of HIV/AIDS and STDs. The EFO utilized WCI funds in 1997 to start a pilot project in a local high school. Assisted by other local NGOs working in health and education, the pilot proved to be successful and led to securing a three year grant (1997-2000) from the Packard Foundation for expanding the project into an additional 24 schools. Working through local networking committees (with representatives from the community, the schools and government), information is shared on training needs, experiences and project results. A two-day workshop in early 2001 on project accomplishments supported the concept of partnerships and has led to increasing involvement on the part of the Ministries of Education, Health and Child Health to take greater ownership for the school-based ARH programs.34 The ARH project has accomplished much over the last few years; all stakeholders report a feeling of ownership and local-level committees are making decisions on implementation. Knowledge of ARH issues has extended to the national level and is influencing long-term educational trends and teaching practices. When the ARH project started in 1997, reproductive health issues were quite new to most people in Ethiopia. Currently, however, both governmental and nongovernmental organizations are scaling up ARH programs for adolescents in private schools and out of school. The EFO is sharing its experiences at various forums and is getting requests for technical assistance from many different organizations. Findings: (a) All of the Hub Countries discussed above entered into partnerships of varying duration and types for promoting special initiatives of interest to each SC Field Office; (b) Entering into “strategic partnerships” allowed each FO to initiate projects and achieve results on a broader scale and with improved efficiencies than the FO would have been able to accomplish on its own; (c) Each FO experienced difficulties (albeit for different reasons) in seeking to require partners to enter into written agreements, discretion and patience needs to be employed by FOs as they move to this stage of the relationship;35 (d) Each FO carried out monitoring and evaluation (M&E) activities of their respective projects with varying degrees of success; the BFO felt more TA should have been provided by SC HQ, whereas 34 See Partnerships and Institutional Relationships: A Resource Book for Save the Children, Volume 2: Case Studies and Other Resource Materials, November 2001 (draft), Washington, DC. 35 These difficulties included foot dragging, outright refusal to sign a binding agreement, disputes between partners over subagreements, and differing interpretations as to whether agreements were legal documents or less formal Memoranda of Understanding (MOUs), detailing the responsibilities of each partner in carrying out project activities. 26 the ESFO and EFO both did extensive M&E with EFO making far better use of the results; and (e) All FOs felt that partnering with national and regional NGOs tended to make better use of resources for carrying out projects; however, there was an observed tendency among local NGOs (and sometimes, regional and national NGOs) to view partnering only as an opportunity to secure outside funds. Conclusions: (a) WCI funds were an important source of undesignated funds for FOs to hire staff to identify initiatives and potential partners leading to “strategic partnerships” in keeping with the grant design; (b) FOs agreed that “partnering” permitted adopting program strategies for partners to share resources (financial, technical expertise and human) to increase program scale and subsequent sustainability. However, no data were available on the cost-effectiveness of partnerships;36 (c) SC Headquarters was generally of the opinion that written agreements had to be in place to validate partnerships and then viewed them as legal contracts; the field experiences indicate that partnerships are more akin to a courtship or a kabuki dance in which partnerships are an ongoing process and not an end in and of themselves. The resulting written agreements are generally viewed as MOUs by partners rather than contractual agreements; and (d) The key to successful partnerships is the successful management of different individuals and different institutional cultures with different reward structures — it is crucial that strong relationships exist between the institutions’ directors, but equally true that leadership is a shared enterprise and not singular to one partner. Recommendations: (a) Further studies should be conducted on the cost-effectiveness of entering into partnerships and the efficiencies to be derived under a partnership structure; and (b) Patience should be shown as FO Directors think about partnerships; flexibility needs to be the order of the day at SC HQ as FOs move towards developing written agreements with partners. 6.2 Measuring Institutional Capacity The WCI-2 grant made no provisions for making subgrants to partners37—so no attempt was made by FOs to measure the institutional capacity of partners beyond informal assessments based on the local reputations of these partners. 6.3 Constraints to Partnership The main constraint acknowledged by FOs was the time required by the FO Director and other staff to enter in successful partnerships capable of being sustained. Partnerships are arrangements of mutual benefit and not just convenience. Patience and care have to be exercised by all the parties involved as the partnering process transpires. Too often, there was a feeling on the part of managers that SC HQ saw partnerships as legal handshakes between organizations rather than more personal and cultural interactions taking place among different institutions in Third World settings — in effect being far more informal in the structural context.38 If partnering is viewed as the means to achieving greater capacity for 36 With the realignment of the WCI-2 grant in March 2000, FOs were no longer required to attempt assessing cost￾effectiveness of partnerships; however, prior to this change, almost no data had been collected on this issue. 37 The Cooperative Agreement was amended on August 27, 1999, to permit subgrants up to $124,000; only $3,000 was ultimately dispersed to a partner in Ethiopia. 38 SC is now completing a Partnership Resource Book as one of the last WCI products. This resource book provides information to SC staff on how to go about the partnering process, stressing that principles such as mutual trust, respect and influence on determining the goals of a collaborative activity are more important to the “partnership” 27 carrying out tasks at the field level, then Field Managers need greater flexibility to enter into arrangements of varying rigor with organizations without written agreements that always pass legal muster in different cultural settings. To be sure, managers must also insure that the potential liability or exposure of SC (and indirectly, USAID) is safeguarded. Moreover, each partner must have access to a game plan detailing the responsibilities of each partner but this need not be a binding contract unless each partner is amenable to this type of agreement. 6.4 Information Technology With respect to the primary partnerships established in each of the Hub Countries, SC Field Offices were more advanced than their partners in the area of information technology.39 All FOs were internally networked and using e-mail to communicate with SC HQ. Each office had up to date computer equipment (CPUs, printers, etc.) and contemporary software. The burden of M&E fell to SC staff; in at least two of the FOs (El Salvador and Ethiopia); statistical packages (Epi-Info and SPSS/W, respectively) were being used to process and analyze collected data. Upgrading information technology with current or new partners will be important to SC in the future if these partners are to achieve a greater level of parity for sustaining partner relationships. 6.5 Use of local networks and service organizations All Hub countries were able to establish viable partnerships based on the personal contacts of SC staff with local/regional and/or national/international NGOs. In the case of Bangladesh, Save the Children’s ECD program is now being promoted as part of UNICEF’s efforts to strengthen this important sector. UNICEF, in coordination with the Government of Bangladesh, continues to work with international and national NGOs to insure the continuation of ECD efforts. The El Salvador and Ethiopia Field Offices continue to carry out their respective partnerships (protection of minors in El Salvador and adolescent reproductive health programs in Ethiopia) through active networks of local, regional and national organizations. result than whatever written agreement may be developed. Examples of agreements that can be used for partnerships and a half-dozen case studies are also included; three of the case studies document WCI-funded strategic partnerships. 39 Based on personal communication with SC Field Office M&E Specialists. 28 7.0 PROGRAM MANAGEMENT The purpose of PVC’s Matching Grant Program is to strengthen the technical and organizational capacity of a given PVO receiving a grant award. This Chapter examines how Save the Children as an organization has been strengthened by the WCI-2 grant. 7.1 Strategic Approach and Program Planning Findings: (a) The WCI-2 grant was designed as a cross-cutting approach to continue the efforts of the first WCI grant—advocating programs for the empowerment of women and children, institutionalizing the program principles (see Section 3.1 of this report) and supporting ongoing efforts to improve and strengthen the capacity of SC; (b) Four Hub countries were identified to participate in WCI-2 (Bangladesh continued from WCI-1, and three new countries, El Salvador, Ethiopia and Mozambique). Bangladesh choose to work in the area of credit and basic education, El Salvador focused on basic education and health, Ethiopia decided to pursue reproductive health for youth and education, and Mozambique emphasized programs in credit and basic education;40 (c) The WCI-2 grant continued to promote empowerment but by specifically adopting a strategy of establishing vertical and horizontal structures within country settings. Horizontal structures were to be multisectoral and synergistic, whereas vertical structures were to operate at multiple levels in each country through various types of partnerships.41 In short, each Hub country was to experiment with intersectoral programs in concert with in-country partners at the local and national level; and (d) At the HQ level, SC staff participated in the development of long-term strategic plans, the production of key planning documents and the delivery of international workshops promoting policies and practices positively impacting women and children. Conclusions: (a) WCI-2 resources enabled SC to carry out important strategic planning activities, supporting FOs in the design and establishment of partnerships and promoting the ongoing development of improved mechanisms to monitor and evaluate the performance of SC projects and programs. (b) The WCI-2 grant contributed to visible progress in terms of setting SC’s strategic program directions (which, e.g., now formally incorporate advocacy for the first time), discussing the need for and testing various approaches to an organization-wide impact monitoring and evaluation system, and institutionalizing capacities in training, economic opportunities programming and gender relations analysis. 7.2 Country Initiatives Findings: (a) Initiatives carried out under the grant were the targeted intersectoral partnerships each Hub Country developed with local and/or national level NGOs — “Adolescent Reproductive Health” in Ethiopia, the “Protection of Minors” in El Salvador, and “Early Childhood Development” in Bangladesh; (b) Each Field Office collected data on its respective initiatives and, while the quality of these data 40 WCI Annual Report, 1996-98, and Midterm Evaluation, Woman/Child Impact Grant-2, Save the Children, Shirley Buzzard, with Rani Parker and Anthony Dewees, April 9, 1999. 41 See the SC Matching Grant Proposal of 1996 and SC Matching Grant DIP submitted in March 1997. 29 varied, the impacts of these programs were assessed and the results were shared with the USAID Mission; and (c) Lessons learned from these partnership initiatives have been summarized as case studies in SC’s Final Report on WCI-2; additionally, partnerships in three Hub countries supported by the MG are learning experiences for other Field Offices for establishing partnerships as described in SC’s Partnership Resource Book now in draft.42 Conclusion: (a) Each of the WCI countries with the exception of Mozambique carried out partnership initiatives and the lessons learned have been documented by SC. Recommendation: (a) Standardized approaches should be designed by the SC Home Office to monitor and evaluate country initiatives. 7.3 Conflict Management All of SC’s Field Offices are required to have security/contingency plans that spell out steps to be taken to ensure the security of staff and property in the case of a natural or man-made emergency/disaster. A number of Field Offices in disaster-prone countries have conducted preparedness training, and have demonstrated their capacity to respond programmatically to sudden turns of events (for example, Bangladesh, El Salvador, Pakistan/Afghanistan). An emergency response toolkit (“office in a box”) and the use of Personal Digital Assistants for rapid collection and central analysis of data are among the initiatives underway to improve field capacity. Disaster preparedness training has also been requested by a number of field offices under recently developed regional 3-year strategic plans. 7.4 Monitoring and Evaluation M&E Findings: (a) The Final Report of WCI-1 found that the “objective measurement of program impact has been problematic in all core countries.” While acknowledging that certain sectors (e.g., health, nutrition and credit groups) were collecting data on their programs, it was observed that few data were being collected to assess the “integrated impact” of SC programs;43 (b) The WCI-2 grant statement of purpose noted the need for “measurable, cross-sectoral models in four Hub Countries . . .strengthened sectoral models . . .and increased structural impact within SC and policy impact at the national, regional, and international levels using woman/child centered models and principles.”44 Certainly, SC recognized that it needed to continue improving its M&E capabilities over the life of the grant and anticipated using grant resources to accomplish this goal; (c) From the start of the second WCI grant, SC made a serious effort to develop an integrated M&E system for the organization. A summary of planning, monitoring and evaluation activities carried out under WCI-2 is presented in Annex G-1 along with the major documents generated under the grant (see Annex G-2). WCI resources were used to support an M&E Specialist, initially at SC's office in Washington, DC. Yet the Midterm Evaluation observed, “While some steps have been 42 See the draft Final Report, Woman/Child Impact-2 Grant (1996-2002), Save the Children, Washington, DC, 2002, and Partnerships and Institutional Relationships: A Resource Book for Save the Children, Volume 1: Practical Guidelines and Principles of Good Practice and Volume 2: Case Studies and Other Resource Materials, Nov. 2001, Washington, DC (these materials are still in draft). 43 Final Evaluation, Woman/Child Impact Grant-1, 1991-1996, USAID and Save the Children Federation, Phil Boyle, Automation Research Systems, December 15, 1995, pp. 4, 31-34. 44 USAID Matching Grant Application-FY 1996 by Save the Children, Westport, CT, p. 2. 30 made in the long road towards a functioning institution wide monitoring and evaluation for SC, the progress on this activity has not been on target with the grant schedule. A great deal remains to be done in the last half of the grant to achieve the stated objectives.” Although the M&E Specialist position was then moved to Westport, progress remained slow; (d) The Planning, Monitoring & Evaluation (PME) working group activated in June 2001 (when the last M&E Director left SC) has made clear progress in grappling with institutional issues such as consistent M&E terminology, tools and beneficiary counting; and (e) In point of fact, the FOs have accomplished much in the area of M&E. In each Hub country, WCI resources were used to hire individuals to develop and carry out M&E functions for the respective FO and its particular partnership initiative. Summaries of these M&E efforts for each Hub country are presented below:45 Status of FO M&E Systems Bangladesh With the help of HQ technical offices, BFO staff developed indicators to assess their Early Childhood Development Partnership, the Health Financing System being piloted in the impact area and other programs (e.g., health, education, gender empowerment, etc.) being run by the BFO. Questionnaires and focus groups were used to assess project processes and outcomes. The BFO has also assisted in developing the M&E capacity of its partners and making them sensitive to the need for ongoing documentation of project performance. WCI supported the establishment of an Organizational Learning Section (now in its third year of operation) in Bangladesh to document all BFO activities including case studies, baseline surveys and operations research of SC projects either completed or underway. El Salvador This FO recruited a skilled M&E person fluent in quantitative data collections techniques as well as qualitative approaches to M&E. Process indicators were identified for measuring changes in the knowledge, attitudes and practices of mother caregivers, technical staff in partnering institutions and participants in 27 communities where SC was carrying out early childhood development and education programs. In 1997-98, as part of the M&E effort, an extensive baseline was conducted of the target area that included 4,678 families with a total population of 24,734 persons,46 enabling the ESFO and its partners to measure the results of program interventions as they occur and most importantly the overall impact of the ECD intervention. Ethnographic data are also being collected on target communities by local fieldworkers. Ethiopia The Ethiopia Field Office has developed the most comprehensive M&E Unit of the three Hub countries. With the use of WCI funds, the EFO hired an Evaluation Specialist and a Documentation Specialist to assist in producing reports and archiving materials produced in the Unit. Structured questionnaires have provided baseline data in 25 schools.47 The EFO conducted training with its partners in participatory M&E to enhance their capacity for collecting data as well as identifying grounded indicators for measuring program impact. All data are being processed with SPSS/W (Version 10), demonstrating a relatively high level of sophistication for analyzing data and presenting data results. The EFO has been especially supportive of the M&E Unit; encouraging the publication of evaluation reports and using M&E findings to leverage other grant resources. 45 These data are derived from personal interviews conducted in the field and the SC WCI Final Report now in draft. 46 Estudio de Base del Area de Impacto do San Martin, Save the Children/US, El Salvador Field Office, Enero 1999 (summary of studies of 27 individual communities in Cuscatlan). 47 Baseline Survey Report on Adolescent Reproductive Health (ARH) in Government High Schools of Addis Ababa, prepared by the M&E Unit, Save the Children/US, Ethiopia Field Office, Oct. 2000. 31 Status of HQ M&E Systems SC Headquarters has not established a functioning M&E system, although efforts were made to use WCI funds in support of an M&E Unit. Over a five-year period, four separate M&E Specialists/Directors were hired with varying degrees of success. SC senior managers feel that the people who were hired had the academic experience but lacked the practical field (and organizational skills) to develop a viable M&E system for SC. HQ M&E staff did not succeed in devising consistent indicators for WCI, partnerships or sector programs. After the November 1998 Cyprus workshop, the field was asked to adopt a core indicator for each of its programs in an effort to standardize reporting and reduce reporting burdens – but this was not supported by adequate training and the effort was abandoned in 2001. At present, the field is asked to choose from comprehensive lists of indicators for each sector area, choosing what each FO feels is applicable. There were also the ongoing issues of where the WCI Office was to be placed in the SC organization and to whom M&E activities would be reported. The Midterm Evaluation suggested that the M&E person report directly to the VP International Programs in Westport. This change was made but then the person left in May 2001; the M&E position has been vacant since this time. Subsequently, funds for the HQ M&E position were shifted to support M&E field personnel in El Salvador and Ethiopia. HQ has used WCI funds to support a number of M&E workshops, training events and consultancies for Field Offices — including Hub countries — as well as the development and publication of useful materials on workshops and other M&E documents (see Annexes G-1 and G-2 and Annex C-1, Documents Reviewed). M&E Conclusions: (a) The Hub FOs have done a good job of developing M&E Units—overcoming certain obstacles to be sure—but nonetheless putting systems in place capable of designing evaluations, collecting and analyzing data and reporting findings. The ability to use data results for leveraging additional funds has been mixed; (b) Indeed, WCI has stimulated an increasing interest in M&E and the adoption of some type of data collection technique(s) in 15 of 20 SC FOs responding recently to a follow-up baseline survey;48 and (c) SC Headquarters has failed to create a comprehensive M&E system—despite ten years of support by two WCI grants. As noted above, some of the sectors do have adequate M&E systems in place (notably health and economic opportunities), but the ability to agree on common performance indicators for SC as an organization has not been accomplished. Also, FOs are required to prepare annual Program Operational Plans (POPs) and reports – but the review process at HQ is inconsistent; the data is often activity-level; and the use of this data is unclear. There appears to be a fair amount of confusion over what constitutes M&E. In reviewing many of the HQ documents, more discussion is given over to planning and programming than M&E.49 M&E Recommendations: (a) Greater emphasis should be given by FOs to developing evaluation designs that emphasize the collection of quantitative data in preference to process data, with special attention given to project/outcome data; (b) Every large FO should emulate the Bangladesh FO and establish an Organizational Learning Section; moreover, the statistical/software expertise of the Ethiopia FO could well serve as a data 48 “WCI-II Baseline Survey Responses,” Follow up to Baseline Questions on Realigned Results Framework IRs (2000), Save the Children, Washington, DC, September 2001, p. 4. 49 See the SC WCI Final Report (draft), pp. 14-19. 32 processing/analysis center in the short term while other centers are established. The current Hub countries could certainly all serve this function; and (c) SC HQ should determine exactly what kind of information it desires from an M&E system, who the data are for and how data will be used and what levels of resources are to be provided to carry out the M&E enterprise. It may well be that a global M&E system is too ambitious, but SC HQ managers need to get beyond the talking stage to that of designing and implementing even a modest M&E system capable of providing critical information to senior managers. Two initial steps would be a tighter review process for annual program plans from the field (POPs) and clear interest from SC’s senior management in using data from the field for corporate decision-making. 7.5 Overall Management Findings: (a) There is ongoing communication between SC HQ and FOs. Sound management systems are in place for planning and development programs and managing financial resources. Every FO does a Program Operational Plan (POP) for what it plans to do in the coming year as well as a closeout Annual Report of prior year activities;50 (b) The SC Organization Chart appears to be uncomplicated—divided between domestic and international operations with divisions into regional and activity sectors. For carrying out the WCI evaluation, documents were easy to access; and (c) The WCI DIP anticipated some focus on improving policies and procedures related to gender diversity, such as establishing and implementing a gender policy within SC and engaging in efforts to influence external (e.g., national and international) policies that affect the lives of women and children. The Midterm Evaluation reported on the high staff turnover at SC HQ and in the field, as well as noted problem areas in the gender/diversity of SC staff51 - and called upon SC to take steps to address these issues. The WCI grant can be credited directly with the drafting of a gender policy (adopted in 1999 by all members of the International Save the Children Alliance). The grant also stimulated the formulation of an SC/US Gender Statement that affirms the importance of proactively seeking female and minority candidates for job openings and new “family-friendly” work/life policies that took full effect in February 2001. (d) Data provided by the SC Human Resources office revealed that the overall turnover rate of Westport-paid staff (working in the United States and overseas) dropped from 30.5 percent in FY 1997 to 22.2 percent in FY 2001. The turnover rate for international field staff, mostly local hires, also dropped from 42 percent to 28 percent (see Annex H-1). Human Resources staff attribute these changes to the adoption of the policies mentioned above, the hiring of an Employee Relations Manager and the involvement of senior managers from around the world in strategic planning processes between 1999-2001. SC has also applied the notion of gender equity to its own hiring practices and has accomplished more balanced gender ratios in various staff settings (see Annex H￾2). Conclusions: (a) Management operations were found to be relatively sound without any obvious problems; however, data quality between program sectors is quite uneven—greater rigor is needed to insure management has reliable data for making sound decisions; and (b) SC senior managers were aware of the criticisms about staff turnover and gender diversity and have been addressing these concerns at least since the midterm review of the WCI-2 grant. Recommendation: 50 The preparation of the Annual Report requires data on indicators identified in the POP, but as noted above the indicators are not always well understood by the field and many assess only activities. 51 WCI Midterm Evaluation, April 1999, p. 22. 33 (a) SC Headquarters needs to complete its planning efforts for the implementation of a sound M&E system, and commit to using that data in organizational decision-making. 7.6 Sustainability As used by Save the Children, “sustainability implies lasting positive change in institutions, behaviors, or policies affecting human well-being. Sustainability involves enabling individuals, communities and institutions to promote change and to adopt new behaviors and systems that endure during and after SC’s involvement.”52 Further, SC defines sustainability as being comprised of four dimensions; institutional, financial, behavioral and policy. 7.6.1 Overall sustainability survey Findings: (a) Institutional: Programs initiated by the partnerships established in the Hub countries are all continuing, either directly with the same participants or through integration into the partners’ own programming; (b) Financial: no cost-recovery programs were observed for the partnerships, however, the Health Financial Scheme in Bangladesh has developed a cost-recovery program to support a microcredit group from which the profits (i.e., interest on small loans) are being used to pay for local clinics to serve women and children; (c) Behavioral: Baseline data have been collected on partner programs in El Salvador and Ethiopia but until follow-up data are collected, no conclusions on behavioral change can be drawn; and (d) Policy: Gender equity as an operating policy has been adopted by all of the partners who have interacted with SC/US during the implementation of WCI-2. Additionally, most of these partners have accepted the importance of establishing M&E in their planning and programming for assessing the performance of their programs. Conclusions: (a) Hub country FOs have been successful in sustaining the basic tenets of the WCI grant: gender equity in program operations, the advantages of partnering and the importance of M&E; and (b) SC Headquarters achieved its overall goal of expanding the organization’s capacity to implement gender-sensitive programs throughout SC/US —especially in the areas of external policy involvement, advancing gender equity and implementing field programs that achieve increased outreach to women and children. Much remains to be done in strengthening SC HQ capabilities to monitor the sustainability of its various field operations. Recommendation: (a) The FOs should continue their efforts to promote the tenets of WCI and SC HQ should continue its efforts to improve HQ M&E systems. 7.7 Financial Management 7.7.1 Effectiveness of financial management Findings: (a) Examination was made of the WCI financial management of the grant at the FO level. Each FO was able to provide a summary of its WCI grant expenses through the end of the grant on September 30, 2001. Of course, since the grant completion date was extended by USAID (to March 30, 2002), leftover monies were being reallocated by SC HQ and remain to be accounted for once 52 Save the Children’s Key Program Principles, undated memo copy. 34 the grant is closed out; (b) From a review of the WCI Annual Reports, SC carefully tracks expenditures of the WCI grant in the aggregate. However, given the cross-cutting nature of the WCI grant and with a relatively high percent of monies being allocated to SC HQ,53 the Annual Reports failed to identify exactly how these resources were used throughout SC HQ operations;54 and (c) Once Mozambique discontinued its role as a Hub country under WCI, the balance of its funds were reprogrammed to fund an Africa M&E regional position and associated expenses. Conclusion: (a) SC HQ has sound management systems in place but, in the case of a grant like WCI which was designed to serve a wide host of needs and multiple interests, the ability to earmark and justify expenditures was a difficult task and it is unclear where the oversight was vested in making expense allocation decisions. Recommendation: (a) SC HQ should devise better tracking and oversight procedures to financially manage a cross￾cutting, multisectoral grant. 7.7.2 Leveraging other donor funds Finding: (a) The monies leveraged during the five years of the WCI grant in the SC Hub countries are presented in Table 7.1 Table 7.1 Donor Funds Leveraged During the WCI Grant Funding Period Hub Country WCI Grant Budget No. of Grants Awarded Amount Awarded Bangladesh $400,000 16 $1,304,196 El Salvador $350,000 12 $3,098,515 Ethiopia $200,000 51 $24,545,358 Totals: $950,000 79 $28,948,069 Source: SC Headquarters, Grants FY 97-01; see Annex I. Conclusion: (a) The WCI grant did not directly result in all of the monies leveraged by the Hub FOs, but according to the FO Directors the grant was very influential in their donor-seeking activities. Accordingly, the WCI grant has been a very good investment with respect to the amount of funds leveraged during the five-year life of the grant. Recommendation: (a) Based on the experiences of the Ethiopia FO, SC HQ and other FOs should utilize their M&E findings in a more pragmatic way to leverage donor funding for projects. 53 Based on the 5-year budget provided in the DIP, the total WCI-2 grant was $4.24 million dollars, $1.3 million allocated to the Hub countries and $2.95 million to SC HQ. The field received 30.6 percent of the grant with the balance of 69.4 percent going to HQ—this also includes a 14.6 percent indirect cost category. 54 For example, the costs of workshops and associated travel/per diem were covered directly by the SC HQ WCI budget even though they were field expenses. Transfers of funds to the field also supported special projects (up to $100,000 annually of Program Innovation Funds). 35 7.7.3 Cost effectiveness of technical approach Finding: (a) All of the FO Directors expressed satisfaction with their efforts to achieve partnerships and felt there were economic savings gained by working with these partners. However, no data was collected or provided on cost effectiveness per se; given the differing levels of sophistication among most partners, it proved unduly difficult and costly to obtain cost data. Eventually, with the realignment of the grant, cost effectiveness was no longer considered a reliable performance indicator. Conclusion: (a) Assuming the data can be collected at a reasonable cost, cost effectiveness can be a useful data set for comparing different approaches to a particular issue. But to achieve this end result requires a great deal of rigor in the data collection process from all of the entities involved, something that was not achieved under WCI-2 due to cost and practical considerations. In retrospect, this was an unrealistic expectation of the WCI proposal and DIP. 7.7.4 Repercussions of “matching” requirement on program (a) No discussions were held with SC managers on this issue; the budget information reviewed focused only on the USAID grant contribution to the WCI program. However, the WCI Annual Report did cite the SC match vis-a-vis the USAID MG expenses; and (b) No conclusions were drawn on this issue. 7.8 PVO’s Information Management Finding: (a) SC HQ and the FOs visited have up-to-date information management systems with contemporary software. The widespread use of e-mail has certainly improved communication between SC HQ and the FOs. Conclusion: (a) SC has made considerable investments in information resources for managing its projects and programs. Recommendation: (a) For SC to continue its support of developing an organization wide M&E system, it should invest in statistical software and training for key personnel. 7.9 Logistics The Evaluators did not collect any information on logistical issues. 7.10 Project Supervision Findings: (a) Supervision on the WCI proved to be a difficult issue over the life of the grant. First, there were staff turnovers owing to some of the problems previously discussed; (b) There was ongoing tension on where WCI as an office should report in the SC management hierarchy; (c) At the time of the midterm review, the PD was about to transfer to an offshore site but wanted still to run the project; co-directors ran the project for years three and four; and 36 (d) The USAID Project Manager retired in the third year of the grant, and a new USAID manager took over the project in October 1999. Conclusions: (a) Given the complexity of the WCI grant and all that it entailed, SC senior managers failed to resolve management issues that were apparent early in the grant; (b) Notwithstanding HQ management and supervision problems, the Hub countries carried out WCI activities as designed although often without promised TA and other support from WCI staff at HQ. Recommendation: (a) The supervision of the grant should have been worked out in the original proposal—especially the reporting line of authority issue. 7.11 USAID Management Findings: (a) The USAID Project Manager (PM) failed to provide SC with formal comments on the DIP design submitted by SC; (b) The relationship between the USAID PM and SC’s PD was too cozy—in that too many decisions on the WCI-2 grant were made informally without oversight at the SC HQ level; 55 (c) It appears that the WCI-2 grant was allowed to run on autopilot with little oversight on PVC’s part. The original USAID PM was very committed to the WCI goals and apparently was less concerned about the theoretical rigor of the DIP design and its implementation; and (d) By the time a new PM was appointed at USAID (Oct. 1999), there were many concerns about the amorphous nature of WCI and what was actually being accomplished and how. Conclusions: (a) With the appointment of a new PM at USAID, it was apparent that realignment of the WCI-2 grant had to take place in order to measure the results and outcomes of the grant. Recommendations: (a) SC senior managers should have required formal feedback from USAID on the DIP along with formal minutes or reports from the WCI PD on any meetings held with the USAID PM. (b) USAID/PVC must insure that formal reviews of DIPs occur and are given to PVOs. 55 Personal communication. 37 8.0 CONCLUSIONS AND RECOMMENDATIONS Conclusions Difficulties in program implementation, partnership issues, and management problems have been addressed in earlier sections of this report. Now that WCI-2 is coming to a close, there are two critical questions that must be answered in the final analysis of this evaluation; 1) What were the value-added attributes of WCI-2 to Save the Children as an organization, and 2) Did USAID get its money’s worth? In terms of gains made under the grant: SC staff generally felt that the WCI grant has added value to SC operations—both in the field and at HQ offices in Westport and Washington, DC. Interviews and questionnaires administered to staff confirm these views and demonstrate divergent opinions between SC HQ and Field Offices.56 The evaluation team administered questionnaires to HQ and field staff to capture their perceptions on the general results of WCI-2. Staff were asked to rate the grant’s key results per the realigned “Results Framework”. Table 8.1 presents a summary of these ratings. Table 8.1 HQ & FIELD STAFF ASSESSMENTS OF THE WCI-2 GRANT* SAVE Management Hub Field Countries Westport Wash, D.C. Bangla El Sal Ethiopia Non￾Hub (n=3) (n=7) (n=4) (n=3) (n=6) (n=3) Achieved Intermediate Results a. Promotion of policies with positive impacts on women 2.17 2.36 2.00 2.00 2.50 2.50 b. Expanded Capacity for M&E 2.67 2.64 2.50 2.30 1.50 2.50 c. Improved Org. Capacity 2.30 2.64 1.75 1.00 1.80 2.50 Reached WCI-2 Objectives 2.42 2.75 1.88 2.00 1.90 2.50 Extent M&E Measured WCI-2** NA NA 2.25 1.67 2.30 NA * Presented in mean scores on a five point scale: 1=Best; 5= Very Poor ** Asked only of Field Managers given their implementation responsibilities Source: Structured Questionnaires completed by SC Managers; see Annex C-2 Questions 1 and 3. Policy Promotion: Overall, the SC staff felt WCI helped in the promotion of policies supportive of women and children—especially gender issue; many mentioned the internalization of SC gender equity policies and even how these policies had been extended to partners. However, it was also acknowledged that the WCI grant was perceived initially as being pretty nebulous in its focus until the realignment of the grant objectives took place. This perception was more evident in SC HQ—although Ethiopia respondents were even more skeptical. Given the broader scope of WCI-2 compared to the first WCI grant, many staff thought the grant’s goals were vague and the objectives difficult to achieve. Even after the realignment, these attitudes persisted, although to a lesser extent. 56 Admittedly, the numbers involved are very small and are not intended to serve as a survey but rather as a means to validate and reinforce the qualitative data gathered in interview sessions with SC staff by the Evaluators. 38 M&E capacity: All Hub country field staff were more positive than HQ staff about their expanded and improved capacity to conduct M&E activities. This was certainly confirmed in an examination of M&E materials produced and efforts underway in the field. The office that was most efficiently using its M&E findings to leverage donor funds was the most positive about its expanded M&E capabilities. SC HQ staff (Westport and Washington) apparently are still dubious about the organization’s M&E abilities. This no doubt reflects ongoing efforts to come to grips with what type of M&E system is needed by SC and concern as to whether senior leadership is serious about using those data for decision-making. Organizational Capacity: Field staff also felt more positive than HQ staff about local organizational capacity being improved—ranging from the increased skills of SC staff to reaching more beneficiaries through partnering efforts. The successful partnering efforts in El Salvador are reflected in their assessment of organizational capacity. Despite setbacks, the FOs felt they had achieved WCI-2 objectives—at least per the realigned RF format. SC HQ staff were less convinced of whether objectives had been achieved and this undoubtedly is associated with the vagueness of the grant’s objectives and a failure to build a coherent and strong constituency for the program in spite of its cross-cutting approach. In retrospect, designing a program that is multisectoral and overarching has the potential for strong program gains but has high risks for accomplishing efficient and effective implementation. But in the main, SC as an organization has benefited greatly from WCI-2. Secondly, did USAID get its money’s worth? This is not an easy question to answer. The WCI-2 grant, as originally designed, was lacking in focus with objectives and proposed outcomes very difficult to measure. But to SC’s credit, it made real structural changes in the grant design in the grant’s third year resulting in the complete realignment of the grant. These results are now fairly discernible (see Figure 3.1, Results Framework). · SC has fully integrated gender equity policies throughout the organization and has worked hard to promote these policies with its international and national partners along with participating in numerous international forums on women and children issues; · SC has strengthened the planning and monitoring of its field programs, programming procedures have been enhanced in HQ operations, and Program Learning Groups are carrying out important training and TA roles; and · Hub country Field Offices have increased their program outreach capabilities through devising strategic partnerships and have adopted more rigorous M&E approaches to assessing these efforts. SC achieved most of it objectives; perhaps, not as well as originally intended and with problems during the first few years, but the organization addressed these problems to its credit and did accomplish a great deal. The performance outcomes were uneven—especially between the field offices and the SC HQ offices—but the overall numbers and quality of products produced under WCI-2 are substantial. Notwithstanding some shortcomings, USAID funds strengthened SC as an organization and its capacity to carry out its mission consistent with the goals of the Matching Grant program. In sum, USAID received a fair return on its investment. Recommendations Specific recommendations have already been provided on a range of topics related to program implementation, partnering activities and management. Save the Children is a moderately centralized organization capable of reacting to issues quickly and the organization is well managed and interested in improving programmatic or operational deficiencies when identified. However, as macro recommendations, the following thoughts are presented on SC as an organization and ongoing areas that SC should address. 39 1. SC should continue its advocacy of gender-sensitive policies in all of its operations and its interactions within the Save Alliance, and with other international and national partners. 2. SC should continue its support for FOs to establish strategic partnerships. Sharing resources achieves savings and more importantly institutional bridges are built with national and local NGOs for serving the needs of women and children. 3. Now that WCI-2 resources have been expended, SC should pursue less costly means for training and TA to its field offices (for example, in gender relations analysis); the PLGs could be used extensively to conduct some of this training. 4. Hub country Field Offices have done an excellent job of developing their M&E systems, especially in collecting baseline data for key projects that is time-consuming and labor intensive. However, the FOs should focus on more cost-effective approaches to data collection such as rapid assessment strategies and employing sampling techniques to reduce the population universe. 5. FOs must recognize that M&E is not an end in and of itself; once data are collected and analyzed, the findings should be leveraged in search of other resources to sustain the projects/programs being monitored and evaluated. M&E should always be employed in an “applied context.” 6. Should SC receive future grants such as WCI, more monies should be allocated directly to field budgets for management of country projects and programs. The financial management of WCI seemed unduly complicated and at times managerially confusing to both the FOs and SC Headquarters. 7. Lastly, SC should continue striving to develop an organization-wide M&E system; but in striving to design a global system, SC may be over-reaching. The PME working group needs to focus on the core ingredients of any M&E system; what information is needed, at what organizational level, by whom, for what, and what level of burden (resources and time) can be absorbed by SC for collecting data? Too much energy is being expended in identifying “performance indicators;” more focus should be given over to figuring out what data are required to carry out SC planning and programming and then determine what data elements can serve as an indicator of performance (proxy or otherwise). As important, SC’s senior management needs to integrate field-generated M&E data into its decision-making about resource allocation, fundraising, program focus and program content to make this organization-wide system a worthwhile investment. Closing Remarks In summation, this evaluation has documented the gains made by SAVE as a result of the PVC Matching Grant. The organization has made considerable progress in promoting partnerships as a way to better leverage its resources in the field and in accordance with the intent of the grant. SAVE has also established an enviable record in making gender equity a high priority internally and externally. However, in the final analysis, many of the problems discussed in this report are directly attributable to SAVE’s poorly designed DIP. While PVC failed to provide SAVE with a formal review, there were certainly enough warning signs from the field offices that the DIP, as originally designed, was incapable of being implemented. Earlier attention to these issues would have facilitated a smoother implementation of the MG rather than making adjustments so late in the life of the grant. With respect to designing a functioning M&E system for documenting its performance against stated objectives, SAVE has simply taken too long to address this issue. Of course, this is not a problem unique only to SAVE; other PVOs are also struggling with how to measure their performance. Both USAID and their PVO partners need to focus more attention on what is doable and practical in the areas of evaluation and performance 40 assessment. Lastly, the problems that took place with the MG probably could have been ameliorated had there been a better dialogue between SAVE and PVC. Both organizations now need to reflect on the “what went right” with the WCI-2 grant as well as “what went wrong” so as to avoid these mistakes in the future. DIP Table ANNEXES Annex A: Key Events Timeline......................................................................................................................................................................... 42 Annex B: Detailed Implementation Plan Table (derived from WCI Results Framework).................................................. 44 Annex C: Statement of Work.............................................................................................................................................................................. 47 Annex C-1: List of Documents Reviewed.................................................................................................................................................... 55 Annex C-2: Evaluation Questionnaires......................................................................................................................................................... 59 Annex D: Advocacy Implementation Chain ............................................................................................................................................... 67 Annex E: Partnership Table, by Country Program Visited.................................................................................................................. 70 Annex F: Sustainability Analysis...................................................................................................................................................................... 74 Appendix 1: Realigned WCI Grant.................................................................................................................................................................. 93 Appendix 2: WCI Intermediate Results......................................................................................................................................................... 95 DIP Table ANNEX A: KEY EVENTS TIMELINE FY # : 1-Oct-96 to 30-Sep-97 FY # : 1-Oct-97 to 30-Sep-98 FY # : 1-Oct-98 to 30-Sep-99 Date Event Date Event Date Event Q 1 (M) CA signed Q 1 (M) IP orientation/training manual produced/disseminated Staff of umbrella grants in NIS request/receive training on Program Principles Q 1 (M) Breakthroughs for Children (1999- 2003) – new SC strategic plan EO Office becomes independent of WCI funding Q 1 (A) GRA trng/Nepal Q 1 (A) M&E workshop/Cyprus Ptnship baseline survey GRA/West Bank/Gaza Agricult. projects Landmine advocacy Q 1 (A) Strat. Ptnrship Workshop/ Guatemala FACETS V/Mozambique GRA trng/Pakistan/Afghanistan Q 2 (M) Q 2 (M) Q 2 (M) Midterm evaluation Q 2 (A) GRA trng/Malawi Rep at 42nd session of Comm. on Status of Women Q 2 (A) Haiti sponsorship rev/significant benefits screen FACETS IV/Malawi Q 2 (A) Q 3 (M) Q 3 (M) Q 3 (M) Midterm evaluation J. Bryson Strategic Planning methodology introduced throughout IP Q 3 (A) DIPs prepared for grant and Hub countries GRA trng/Nepal, South Central Asia Q 3 (A) Q 3 (A) Q 4 (M) IP handbk revised Core indicators adopted Gender Resource Biblio published; FACETS coordination function shifts to field Q 4 (M) Q 4 (M) WCI Realignment/RF developed along with RF for capacity building in IP Q 4 (A) FACETS III/DC; GRA trng/South East Asia Q 4 (A) Leadership for Social Change course MOU signed Q 4 (A) WCI Forums/Westport, DC, Ethiopia FO – Field Office; GRA – Gender Relations Analysis; PLG – Program Learning Group; PME – Planning, Monitoring and Evaluation DIP Table FY # : 1-Oct-99 to 30-Sep-00 FY # : 1-Oct-00 to 30-Sep-01 FY # : 1-Oct-01 to 30-Mar-02 Date Event Date Event Date Event Q 1 (M) Q 1 (M) IP ldrship strat plnng II/Norwalk – updated 10-yr plan and developed high impact initiatives Q 1 (M) FINAL EVALUATION Steps to mainstream gender incorporated into 02-04 strategic/operational plans Q 1 (A) Ptnrship workshop/Nepal; doc. of CHAPS/Malawi FACETS VI/Sri Lanka; Himalayan FO gender manual Baseline survey (1998-99) conducted Q 1 (A) Core indicator approach reviewed CDROM of key plnng, monitoring, reporting guidance disseminated Q 1 (A) Q 2 (M) Alliance Gender Policy adopted Q 2 (M) Policy/Advocacy Steering Committee Formed Ann van Dusen named COO SC Work/Life Policies fully in effect Q 2 (M) PROJECT ENDS Q 2 (A) IP Leadership strat plnng I/Westport GRA/Burkina Faso Q 2 (A) FACETS survey; FACETS PLG/Egypt GRA/Philippines Q 2 (A) Partnership Resource Book disseminated Q 3 (M) Mozambique withdraws as Hub country First State of the World’s Mother’s reported issued Rep at Beijing+ 5 Q 3 (M) Advocacy staff hired/DC Gender mainstreaming approach adopted Q 3 (M) Q 3 (A) WCI Planning Summit/ Ethiopia GRA/Ethiopia Q 3 (A) Ptnrship survey Revised PME guidance to field Q 2 (M) Q 4 (M) WCI Hub Meeting/Ethiopia focuses on grant deliverables IP capacity bldg. RF revised 1 st SC gender award/Pak/Afghanistan Q 4 (M) PROJECT NO-COST EXTENSION REQUESTED Q 2 (A) Q 4 (A) InterAgency M&E Wksp/ DC, PME Working Group formed Partnership Rpt/Philippines GRAs/Mali, Uganda Q 4 (A) Partnership workshop/Philippines End-of-grant survey (2000-01) conducted Q 3 (M) DIP Table ANNEX B: DETAILED IMPLEMENTATION PLAN TABLE (DERIVED FROM WCI RESULTS FRAMEWORK) WCI Grant Goal: Strengthened capacity of Save the Children to Increase its Development Impact Goal/Objective/ Activity Indicator End of Project Target Accomplishment Data verified?57 Explanation for Change in Indicator or Target, if any58 Target Met? WCI Grant Strategic Objective: Improved capacity to implement gender-sensitive, quality programs which increase scale and sustainability Design/implement SC PME system Organizational systems in place to assess program quality and impact on women and children Organization-wide M&E system Planning, monitoring, evaluation working group active; planning tools being posted on SaveNet - Key metrics for SC strategic plan under discussion Y Original DIP was revised as Results Framework in March 2000 N Improve SC programs with respect to scale and sustainability # of new programs which meet quality indicators in respective sector(s) of implementation No targets set N/A Original DIP was revised as Results Framework in March 2000 N/A Design/develop improved planning systems for SC Decisions regarding SC involvement in policy dialogues (representation and resource allocation) coordinated between departments Systems in place for decision making Advocacy staff hired Policy Advocacy Steering Committee functioning Y Original DIP was revised as Results Framework in March 2000 Y IR 1 External Policy Involvement: Increased contribution to and promotion of policies and practices which positively impact women and children Support forums where positive policies affecting women and children are promoted % increase in number of international forums in which SC staff participate intended to positively impact women and children No targets set WCI-2 supported 85 forums on women and children issues over 5 years; 60 gender related events, 10 PLGs for technical offices, and 15 TA and strategic planning workshops Y N/A 57 Enter “Y”, if data were verified by evaluators and “N” if it was not possible for evaluators to substantiate PVO data. Items to be reviewed will be based on 1/5 random review of objectives/activities. 58 Enter any revisions to indicators during the life of the project as well as any changes in targets for indicators. Describe approval process for change. If necessary, comment on changes in M&E section. DIP Table Goal/Objective/ Activity Indicator End of Project Target Accomplishment Data verified?57 Explanation for Change in Indicator or Target, if any58 Target Met? Promote SC’s gender mainstreaming policies within SC Alliance # of documented instances of improved gender integration within SC Alliance member organizations as a result of adoption of SC Alliance Gender Policy SC gender policies adopted by SC Alliance members Common gender policies adopted by Alliance members Y Proved to be an unreliable indicator and will be changed Y IR 2 Organizational Systems: Strengthened capacity to plan, and support the implementation and monitoring of sustainable field programs focused on the empowerment of women and children Design PME framework PME framework agreed on at HO Global M&E system in place PME framework still being discussed Y N Establish PME Work Group PME Working Group formed and active Active work groups Established and functioning Y Y Develop PME tools Preliminary PME tools prepared PME tools in place for use by FOs&HQ PME tools under development Y N SC international (IP) programs adopt results framework (RF) IP capacity building, RFs reviewed every six months IP reviews routine All IPs now reviewed annually Y Review will become an annual exercise rather than every six months Y WCI staff involved with full range of SC activities % of PLGs where WCI staff and/or FACETS participate WCI staff fully integrated into sectoral programs At the end of FY 2001, this was 50 percent Y N Support development of sustainability/exit plans for all IPs # of new programs with defined sustainability/exit plan or strategy All FOs establish exit strategies for their programs with projected sustainability index Data unavailable—not yet collected Y N Ensure the inclusion of SC Program Principles in all Program Operation Plans (POPs) POP process includes criteria ensuring that programs use program principles (PPs) SC PPs integrated in all POPs All SC countries have now incorporated PPs in their annual POPs Y Y DIP Table Goal/Objective/ Activity Indicator End of Project Target Accomplishment Data verified?57 Explanation for Change in Indicator or Target, if any58 Target Met? IR 3 Program Implementation: Increased program outreach to women and children through improved organizational capacity at FO level Expand programs capable of reaching women and children # of women and children reached through SC programs Expanded base of women and children being reached by SC programs The Hub Countries directly reached 1,679,641 beneficiaries and 2,704,616 indirect beneficiaries on an annual basis (source: 2002 POP reports) Y This indicator will be changed to reflect percent of change against baseline data Y SC field offices adopt RF in all programming efforts # FOs using RFs for project and country planning in FY 2001 All FOs are using RFs in their programming Out of Sc 30 countries, 27 have now adopted RFs as part or their program designs Y Achieved 90% of target WCI Documents Reviewed ANNEX C: STATEMENT OF WORK SAVE THE CHILDREN WOMAN/CHILD IMPACT PROGRAM INSTITUTIONAL STRENGTHENING GRANT 1996 – 2001 FINAL EVALUATION--SCOPE OF WORK SEPTEMBER 20, 2001 MATCHING GRANTS PROGRAM OFFICE OF PRIVATE AND VOLUNTARY COOPERATION BUREAU FOR HUMANITARIAN RESPONSE U.S. AGENCY FOR INTERNATIONAL DEVELOPMENT Headquarter QS I. PROGRAM IDENTIFICATION PVO name: Save the Children-USA Cooperative agreement number: FAO-A-00-96-00003 Date of the evaluation: September-October 2001 Country programs evaluated Bangladesh, El Salvador, Ethiopia II. PROGRAM BACKGROUND History of the Program Save the Children Federation, Inc. (SC) is a nonprofit, nonsectarian organization working to improve the quality of the life of disadvantaged children and their communities in 50 countries, including the United States. SC’s programs are designed and implemented according to the organization’s program principles-- child-centeredness, gender equity, sustainability, empowerment, scaling-up and measurable impact – which were developed with assistance from the first USAID/BHR/PVC-funded Woman/Child Impact grant (WCI-I). SC’s Woman/Child Impact (WCI) framework, now firmly established in international program activities, addresses children’s needs and rights by focusing on four core areas: economic opportunities, education, health, and humanitarian response. The WCI approach increases women’s choices and opportunities in these areas and emphasizes demonstrable positive impact on women’s status and children’s well being. The current grant, Woman/Child Impact II (WCI II), is funded through a follow-on cooperative agreement between PVC and Save the Children. Signed October 4, 1996, the agreement concludes September 29, 2001. The goal of the grant was “to contribute to sustainable human development through the systematic economic and social empowerment of children, women and other disadvantaged groups by expanding their participation in decision making at all levels of society. And that empowerment is achieved through collaboration with local organizations committed to democratic pluralism, gender equity, and economic growth benefiting disadvantaged women and children.” To contribute to this overarching goal, the program planned a wide spectrum of activities at both the organizational and program levels: Organizational-level activities included: ÿ Developing organizational capacity for impact evaluation emphasizing cost-effectiveness and sustainability; ÿ Continuing to strengthen the Economic Opportunities office started under WCI I; ÿ Strengthening SC’s program management structure, further defining and utilizing SC’s program principles through out the organization; ÿ Promoting self-reliance by strengthening local staff and local organizations’ capacity to represent themselves and their communities at nationally and internationally, through skills building and strategic networking such as FACETS; and Headquarter QS ÿ Strengthening collaboration between organizations committed to democratic pluralism, gender equity, and economic growth benefiting women and children and establishing and institutionalizing gender policies and practices. Program-level activities included: ÿ Expanding the impact of mutually reinforcing programs through multi-level strategic partnerships; ÿ Documenting four strategic partnership models, one in each hub country: Ethiopia, Bangladesh, El Salvador, and Mozambique; ÿ Measuring the cost-effectiveness and sustainability of the four WCI strategic partnership models; ÿ Developing analytic tools and methodologies to assess and increase cost-effectiveness and sustainability. ÿ Measuring program impact with respect to economic and social empowerment of children, women and other disadvantaged groups; and ÿ Influencing policies that affect the lives of women and children. These activities were the focus during the grant’s first three years. During this time, WCI resources contributed to visible progress in terms of setting SC’s strategic program directions (formally incorporating advocacy for the first time), defining and testing an organization-wide impact monitoring and evaluation system, and institutionalizing our capacities in training, economic opportunities programming, and gender-relations analysis. In late 1998, the start of the grant’s third year, a conference on Strategic Partnerships was held in Guatemala. Here, it became apparent that some expectations of the grant were extremely complex, and even not realistic. The grant’s mid-term review in Spring of 1999 reinforced this thinking and invited SC to proactively realign the grant and grant-related private resources in order to achieve the highest-priority results. In addition, the high turn-over of grant-funded staff and the number of staff who had joined SC since the grant began provided further impetus to realign the grant and commit to revised results. This reaffirmation was achieved through the “WCI Forum” process, which took place between May and October of 1999. These intensive staff consultations, in the home office and the field and with BHR/PVC, resulted in the WCI Results Framework (see Appendix 1), which: · Restates the grant’s objectives in a more concise way; · Links WCI grant objectives and components to a broader capacity-building results framework for the International Programs Department of SC; and, · Shifts emphasis among the objectives to better reflect current judgments about the most urgent capacity-building needs within International Programs. Although the intent of the grant remained the same, and the major elements as originally written are still reflected in the intended results, the focus of the grant’s final two years became: Headquarter QS 5) Increasing SC’s involvement in promoting policies and practices that positively affect women and children; 6) Increasing capacity in planning, impact monitoring, and evaluation; 7) Fostering, managing, and learning from partnerships; and 8) Further incorporating gender perspectives into our program practices. Undertaking this shift was deemed critical to effect and sustain the changes WCI had initiated or intended to initiate within programs around the world. Notwithstanding the changes implemented in response to the midterm evaluation, the final evaluation shall focus on assessing the overall results of WCI II over the entire five years of the grant. Current Implementation Status Guided by the Results Framework, grant-funded activities – most notably the development of a written Planning, Monitoring and Evaluation framework and a Partnership Resource Book -- are winding up. Organizational changes related to the grant – e.g., field-level partnerships, advocacy work, program learning based on concrete data, and implementation of a new gender mainstreaming policy and plan continue. Fulfillment of grant deliverables is being supported by US-based staff who are partly funded by WCI. August and September are focused on the two major deliverables noted above and preparation for the final evaluation and writing the final report. Program Planning Matrix Exhibit 1 presents WCI-II’s overall strategic objectives, performance indicators, intermediate results and associated measurement indicators. III. PURPOSE OF THE EVALUATION USAID/BHR/PVC commissions this final evaluation of the WCI-II grant to assess its impact in the field and at SC’s headquarters (Westport and Washington) and to fulfill the requirements of the Matching Grant (MG) program. The purpose is to assess how the grant has strengthened the technical and organizational capacity of Save the Children and through its local partnerships has enabled local organizations to achieve sustainable service delivery. The evaluation will articulate achievements and lessons learned in order to provide recommendations to improve the programming of future Matching Grants for BHR/PVC, as well as to help SC continue to enhance its capacity for continuous program/impact improvement. Potential stakeholders of this evaluation, in addition to USAID/BHR/PVC and SC, include local and international partners of SC as well as the women and children who have been affected by this grant over the past five years. Save the Children also expects to share the evaluation findings with senior management of the organization, and very broadly with international program staff in the field and home offices. Field staff in hub countries will in turn share relevant findings with NGO colleagues in their Strategic Partnership initiatives. Dissemination will occur over the course of the next 6-8 months, through email, routine meetings, program learning groups, and other gatherings already planned. Specifically, the findings will be used to inform SC and USAID Managers regarding: 7. The formation of a planning, monitoring and evaluation function within the Executive Office; 8. SC’s implementation of the plan for gender mainstreaming, consistent with the International Save the Children Alliance gender policy; 9. Continuing worldwide program improvement through tools created with WCI support; 10. The updating of 3-year strategic plans and 1-year operational plans (area, country, and technical office) in the course of the coming year; Headquarter QS 11. The Every Mother, Every Child High Impact Initiative, a decade-long campaign for which we have articulated ambitious program and advocacy goals; and 12. A general assessment of the direct and indirect impacts of the WCI-II grant. IV. EVALUATION ISSUES The final evaluation is expected to identify major successes, constraints, lessons learned and unanticipated effects encountered by SC in implementing and managing the WCI II Grant. Accordingly, the evaluation shall assess (1) program implementation and (2) program management efforts to achieve the goal(s) of the program. A. Program Implementation 1. Access progress towards each major objective as cited in the logframe or program-planning matrix. · The evaluators will assess the progress made towards achieving each specific objective in terms of what had been proposed in the program planning matrix or logframe, and what had been accomplished by and since the midterm evaluation (see revised IRs and associated issues in Appendix 2). Review the project’s “baseline” and end-of-project data from which assessments can be made about the achievement of project objectives. · Determine if the project had a detailed implementation plan and the familiarity of field staff with the project design, implementation plan and monitoring and evaluation plan and data. · Assess effectiveness of models, approaches or logical assumptions of the project; is there evidence that these have been replicated elsewhere in country or in other countries? · Document how SC advocated policy changes to impact woman and children in each respective hub country and the effects, · Discuss what SC and its local partners have “learned” implementing this project. 2. Access progress towards sustainability. · Describe the program elements, financial or operational, that are intended to be sustained (objectives); the means for judging if the sustainability objectives have been achieved (indicators); and sustainability achievements and prospects for post-grant sustainability. · Determine if the project has any cost-recovery mechanisms, i.e., local level financing or approaches to generate resources to support project operations. Describe the achievements of these mechanisms and provide an estimate of the magnitude of the system, for example, provide a ratio of costs recovered to operational expenses. 3. Access the status of local partnership(s) with NGOs, community based organizations or local level government. · Provide a chart that categorizes local level partners by type (i.e., NGOs, affiliates of the PVO, private or commercial groups, cooperatives, community-based organizations, regional or local governments or intermediate service organizations); identify the type of mechanism employed with each partner (i.e. MOU, sub-grant, contract). a) Outline the role, responsibilities and decision-making responsibility of the partners; b) Assess the fiscal autonomy and amount of grant funds directly managed in past year; c) Assess the process that the PVO used to build and maintain local partnerships; d) Describe the partnership policy of SC and its approach to assessing potential partners; Headquarter QS e) Review whether SC conducted formal assessments of local partner capacity and developed plans to build their capacity; and f) Provide a discussion on what elements were crucial in building viable partnerships with respect to the SC experience. · Document change in local partner capacity and the major constraints to effective partnerships? Has the project increased the local partners’ access to information technology? · Assess the local level partners satisfaction with the partnership as well as SC’s and their local NGO partners involvement in local networks or with intermediate service organizations (e.g., PVO networks). B. Program Management 1. Assess SC’s management capacity and its partnerships to carry out the activities called for under the grant. Management elements should include the following: Strategic Approach and Program Planning—review what changes have occurred in SC Headquarters’ capacity to manage this type of project to: · Manage the planning process --- program renewal, strategy integration, project design. · Address over-arching program issues of replicability, scale-up, sustainability. · Use performance data to forecast emerging trends and develop strategic plans. Country Level Initiatives--identify SC’s contributions to PVO cooperation and coordination with the USAID mission and other development partner programs assessing: · The level of coordination and cooperation that PVO has with in-country development organizations and with the Mission. · The level of coordination and cooperation that PVO has with in-country development partners. Has PVO used the project data for advocacy with the public sector or consistently shared lessons learned with other PVOs in-country or with non-partner NGOs? Does the PVO provide the Mission with results data? · The PVO capacity to address emergency civil or natural disasters or food security. Is there a history of civil or natural disasters or food security issues in the project area? Does the project have contingency plans to address these issues? Monitoring and Evaluation—assess whether the project implemented a process and put into place a sustainable system to monitor project performance and collect outcome and impact data. Provide evidence that the project established results-oriented objectives and valid indicators for the technical intervention and capacity building components in the project; and how SC acted on recommendations from mid-term evaluation. Determine how SC: a) Improved the knowledge and skills of field staff on how to measure performance and analyze data. b) Transferred monitoring and evaluation skills to local partners; c) If local partners increased M&E in their own activities (non-PVC-funded programs) as a result of skills gained through this project. d) Used the MG to develop a sustainable capacity at headquarters and in the field offices to monitor project performance and measure effects and impact. Has the PVO headquarters: · Fostered analysis and self evaluation in country programs, or conducted quantitative or qualitative analysis to refine interventions; · Conducted periodic review of performance data by project personnel and taken actions as a result of review; Headquarter QS · Institutionalized performance monitoring and impact evaluation systems developed with MG funds into other non-PVC grant funded programs; and · Integrated MG activities with other BHR/PVC grants such as the Food for Peace Program. Financial Management—the evaluation will determine if adequate financial systems are in place to verify program revenue expenditures of the grant, the overall adequacy of the financial monitoring systems in place, whether additional resources have been leveraged, and the cost-effectiveness of the program. Information Management—the evaluation will comment on the utility and timeliness of SC reports from the field to Headquarters and from the HQ to USAID/BHR/PVC; how these reports were used in ongoing programming by SC HQ staff, and the role, if any, of information technology (e.g., the Internet, etc.). Logistics—comment on the adequacy and timelines of SC material inputs. Supervision/Human Resources Development—assess if there were sufficient staff with the appropriate technical and management skills to oversee program activity at both headquarters and in the field program. USAID Management--comment on USAID’s oversight and backstopping of this cooperative agreement. 2. Cite the major management lessons learned and provide recommendations as appropriate in the final evaluation report. V. EVALUATION METHODOLOGY The evaluation methodology should, at a minimum, consist of secondary analysis of reports from both headquarters and field level implementation, analysis of other documentation, interviews and discussions with SC staff (both those closely related to the grant and others not directly involved), WCI staff, partner organizations, and, if appropriate, beneficiaries of the strategic partnership models. In addition, the evaluation team is expected to develop and deploy other appropriate methodologies relevant to carrying out the final evaluation. As a result of the mid-term evaluation and events described in Section II above, the WCI program underwent a realignment process and modified the grant’s planning/monitoring structure. It is recommended that the evaluation team use the WCI Results Framework (RF) in Appendix 2 and hub country RFs as its primary reference point, and refer as appropriate to the DIP, annual reports, and other documents provided to review the periods before and after realignment. VI. TEAM COMPOSITION AND PARTICIPATION The Evaluation Team will consist of a Senior Evaluation Specialist with the requisite technical expertise in evaluation, country experience, and management/writing skills for conducting the SC final evaluation. Additionally, SC will identify a counterpart professional fully familiar with the WCI II project to assist the team in accessing documents, identifying critical persons to interview, and to accompany the team on field site visits. VII. SCHEDULE* The Evaluation timetable is projected as follows from September through December 2001. Headquarter QS · Sept. 4-10 Plan and initiate evaluation, including preparation activities; · Sept. 11; 19 Team Planning Meetings and visits to SC office in Washington, D.C.; · Sept. 17-Oct. 13* Conduct site visits to SC projects in Bangladesh, Ethiopia, and El Salvador; · Oct. 15-26* Prepare Draft Evaluation Report; · Oct. 29* Submit Draft Evaluation Report to MSI; · Nov. 9* MSI submits Draft Evaluation Report to PVC/PVO-SC; · Nov. 30 PVC/PVO-SC returns review comments; · Dec. 14 Final Evaluation Report submitted. *Now revised due to unforeseen restrictions on official travel imposed by USAID; the evaluation field site visits are now likely to take place in October with the subsequent dates for deliverables adjusted accordingly. VIII. REPORTING AND DISSEMINATION REQUIREMENTS The Draft and Final Evaluation Report of Save the Children’s WCI II grant will be submitted to the designated Evaluation contractor—Management Systems International—per the above deliverable schedule. Upon its internal review, the contractor shall distribute draft and final reports to SC and USAID/BHR/PVC. Headquarter QS ANNEX C-1: LIST OF DOCUMENTS REVIEWED Save the Children-Westport, CT, Washington, DC and Field Offices Headquarters: Westport, CT/Washington, DC Breakthroughs for Children: Strategic Directions 1999-2003, Save the Children, Washington, DC, 1998. Detailed Implementation Plan, Woman/Child Impact Grant 1997-2002, Save the Children, Cooperative Agreement No. FAO-A-00-96-00003. Defining Women's Empowerment & Improvement of M&E Systems in Bangladesh/Ethiopia, WCI Consultant Report, Jake Pfohl, September 10, 1999. Draft Final Report, Woman/Child Impact-2 Grant 1996-2002, Save the Children, Washington, DC, 2002. Empowerment through People, Programs and Institutions: A Report of the FACETS Phase III. Workshop, Washington, DC, June 10-20, 1997. Final Evaluation, Woman/Child Impact Grant-1, 1991-1996, USAID & Save the Children Federation, Phil Boyle, Automation Research Systems, December 15, 1995. Gender Equity Policy, International Save the Children Alliance, London, England, 1999. Institutional Strengthening Grant FY-1996, Woman/Child Impact-2, submitted to USAID/BHR/PVC, December 1995. Gender Mainstreaming: A Resource Guide, Save the Children, Washington, DC, May 2000. Inter-Program Learning Group Communication, “An Indicator Opportunity,” July 6, 1997. Midterm Evaluation, Woman/Child Impact Grant-2, Save the Children, Shirley Buzzard, with Rani Parker and Anthony Dewees, April 9, 1999. Partnerships and Institutional Relationships: A Resource Book for Save the Children, Volume 1: Practical Guidelines and Principles of Good Practice, Nov. 2001 (Draft), and Volume 2: Case Studies and Other Resource Materials, Nov. 2001 (Draft), Washington, DC. “Road Map to Success: Measuring Program Impact”, A Report of a Save the Children Workshop, Larnaca, Cyprus, November 17-21, 1997, Save the Children, Washington, DC. Save the Children’s Key Program Principles, undated memo copy. State of the World’s Mothers 2000, Save the Children, Westport, CT, 2001. Strategic Partnerships Development, Management and Measurement, WCI Inter-Hub Country Workshop, Antigua, Guatemala, November 6-9, 1998. “WCI Realignment Documents,” Save the Children, Washington, DC, March 2000. “WCI-II Baseline Survey Responses,” Follow-up to 2000 Questions on Realigned Results Framework IRs, Save the Children, Washington, DC, September 2001. Headquarter QS Woman/Child Impact Grant-2 Annual Report, Year IV, July 1, 1999–June 30, 2000, Save the Children, Washington, DC. Annual Report, Year III, July 1, 1998–June 30, 1999, Save the Children, Washington, DC. Woman/Child Impact: A Five-Year Matching Grant, Oct. 1, 1996–June 30, 1998, Save the Children, Washington, DC [Annual Report]. Bangladesh Annual Report, FY 2001, Save the Children/US, Bangladesh Field Office, September 2000. “A Study on Prevalence of RTI/STDS in a Rural Area of Bangladesh,” Save the Children/US Bangladesh Field Office and DIPHAM Research and Service Center, 1996. “Assessment of Village Education Resource Center (VERC) ECD Program,” Manikganj District, August 1998-September 2000; a joint study by VERC and Save the Children/US, January 2001. Baseline Survey of Home Based Early Learning Opportunity (HBELO), Save the Children/US, Bangladesh Field Office, January 2001. Case Study of the "Strategic Partnership" of Bangladesh Field Office, Nasmeen Ahmed, Nazma Kabir, and Helen Gallagher, Save the Children/US, Bangladesh Field Office, April 2001. Career Development and Gender Training, Save the Children/US, Bangladesh Field Office, July 2000. Gender Relations Analysis of Homestead Production Program (HPP), Save the Children/US, Bangladesh Field Office, Dhaka 2001. Health Financing Scheme: Working Paper, Save the Children/US, Bangladesh Field Office June 1999. Health Financing Scheme: Dharmandol, Save the Children/US, Bangladesh Field Office, March 1999. Program Operational Plan (POP) FY02, Save the Children/US, Bangladesh Field Office, October 2001. Program Strategy Paper (PSP) FY 00-02, Save the Children/US, Bangladesh Field Office, January 2000. Programming Towards Empowerment, Report 1, Process and Outcomes, M. Shameem Siddiqi, Consultant, Save the Children/US, Bangladesh Field Office, August 2001. Project Proposal on Social Mobilization for Arsenic Mitigation in Nasirnagar, Brahmin Baria, NGO Forum for Drinking Water Supply and Sanitation, July 2001. Reading for Children: Action Research for a Post-Literacy Intervention, Talat Mahmud and Tahsinah Ahmed, Save the Children/US, Bangladesh Field Office, September 2001. RTI/STDs and Risky Sexual Behavior in a "Conservative" Society, Richira Tabassum Naved, Save the Children/US, Bangladesh Field Office, July 1996. Saving Newborn Lives: Bangladesh Strategic Plan, Save the Children/US, August 2002. Save the Children/US, Bangladesh Field Office, Proposal to USAID: "KAISHAR" Knowledge and Attitude Improvement of Sexual Health for Adolescents' Responsibility, September 2001. Headquarter QS Workshop Report on Bangladesh Field Office as a Learning Organization, Save the Children/US, Bangladesh Field Office, October 2000. Women's Savings Group of Save the Children/US, Bangladesh Field Office, Review Report, February 1999. Women's Savings Group of Save the Children/US, Bangladesh Field Office: A Background Document, Rosalind Hawlader, Consultant, December 23, 1998. El Salvador "Acuerdo de Cooperacion, Plan International," Save the Children/US, El Salvador Field Office, 1999. "Acuerdo de Cooperacion, ASAPROSAR," Save the Children/US, El Salvador Field Office, 2000. Changing Gender and Violence Paradigms Through Early Childhood Education, Final Report to the Merrill Lynch Foundation, Save the Children/US, El Salvador Field Office, August 2000. "Desarrollo Juventil Comunitaro: Taller de Planificacion PPB," Save the Children/US, El Salvador Field Office, June 1997. El Salvador Strategic Alliance Results Framework: Save the Children, Plan International, and ISPM 2001, Save the Children/US, El Salvador Field Office, 2001. Estudio de Base del Area de Impacto do San Martin, Save the Children/US,El Salvador Field Office, January 1999 (summary of studies of 27 individual communities in Cuscatlan). Informe Preliminar del II Congreso Nacional de Educacion Parvularia, "Percepciones, Creencias y Practicas Relacionadas con la Equidad de Genero y el Desarrollo Social Saludable de Ninos y Ninas Escolares, segun Maestros y Maestras del Nivel de Parvularia," Save the Children/US, El Salvador, y Ministerio de Educacion, Govierno de El Salvador, 1999. La Mujer Trabajadora del Sexo del Municio San Martin, San Salvador, Save the Children/US, El Salvador Field Office, 2000. "Mapa de la Ruta del Exito: Midiendo el Impacto Programatico," Save the Children/US, El Salvador Field Office, November 1997. Manual de Monitero y Evaluacion, Save the Children/US, El Salvador Field Office, 2000. Plan General Proyecto de Evaluacion de Impacto en Programas, Dirigidos a Mujeres, Ninos y Ninas WCI, Save the Children/US, El Salvador Field Office, June 1998. "Primer Taller de Educacion Inicial: Sensibilizacion hacia General y Prevencion de Violencia," Save the Children/US, El Salvador Field Office, August 1998. Plan Operativo Programatico (POP), Ano fiscal 2001, Save the Children/US, El Salvador Field Office, September 2000. Save the Children/US, El Salvador Field Office, WCI Final Reports, 1999, 2000. Headquarter QS Ethiopia Adolescent Reproductive Health (ARH): Empowerment, Definition and Measurement Workshop Report, Save the Children/US, Ethiopia Field Office, February 18-20, 2000. Adolescent Reproductive Health (ARH): Annual Report Oct. 1, 2000-Sept. 30, 2001, Save the Children/US, Ethiopia Field Office, February 12, 2001. Adolescent Reproductive Health (ARH): Knowledge and Practice: Baseline Survey Report, Save the Children/US, Ethiopia Field Office, February 1999. Annual Report, Oct. 1, 2000-Sept. 30, 2001, Save the Children/US, Ethiopia Field Office, October 2001 (including Human Resources Department: Analysis of Staffing Patterns). Baseline Survey Report on ARH in Government High Schools of Addis Ababa, prepared by the M&E Unit, Save the Children/US, Ethiopia Field Office, October 2000. Committee Awareness Workshop, ARH Project, Save the Children/US, Ethiopia Field Office, April 11, 2001. Focus on Adolescent Reproductive Health (ARH): Exhibition & Panel Discussion Proceedings, Save the Children/US, Ethiopia Field Office, June 2000. High School Teachers Workshop on Revising Textbooks with Reference to Adolescent Reproductive Health, Nazareth Rift Valley Hotel, Save the Children/US, Ethiopia Field Office, September 6-9, 2001. Parents Orientation Workshop, ARH Project, Save the Children/US, Ethiopia Field Office, January 28, 2001. Proceedings of Staff Gender Awareness Training (September 1999), Save the Children/US, Ethiopia Field Office, October 13, 1999. Proceedings of the WCI Workshop, Save the Children/US, Ethiopia Field Office, September/October 1999. Proceedings of the Workshop on HIV/AIDS Networking Among Government and NGOs, Addis Ababa Office of HIV/AIDS Prevention and Control in collaboration with Save the Children/US, Save the Children/US, Ethiopia Field Office, October 5-6, 2000. Report of ARH Regional Workshop and Exhibitions, Save the Children/US, Ethiopia Field Office, June 3, 1999. Training Report on Gender Relations Analysis, Save the Children/US, Ethiopia Field Office, June 20, 2000. Youth Friendly Service Workshop Proceedings, Save the Children/US, Ethiopia Field Office, December 2000. Headquarter QS ANNEX C-2: EVALUATION QUESTIONNAIRES Headquarter QS USAID/PVC EVALUATION SAVE THE CHILDREN WOMEN/CHILD IMPACT 2 GRANT PVC/SAVE THE CHILDREN HQ QUESTIONNAIRE PERSON INTERVIEWED:___ ____________________________________________ POSITION IN SC: _____________________________________________________ LOCATION: HQ-DC____ HQ-SC____ Other ____ DATE OF INTERVIEW: _______________ TYPE OF INTERVIEW: Face to Face _____ TIME: _______________ Phone _____ General Issues 1. On a scale of 1-5 (1=best, 2=good, 3=moderate, 4=poor, 5=very poor); please provide your assessment of the performance of the WCI-2 grant in terms of the following Intermediate Results a. Promotion of internal/external policies and practices positively impacting women/children; b. Expanded capacity to plan/support the implementation/monitoring of sustainable field programs; ____ c. Improved organizational capacity to conduct outreach to women/children at the field level; 2. How has Save the Children directly benefited from the WCI-2 grant? Please cite at least two benefits by order of priority. a)_______________________ b) ______________________ 3. Using the same scale in question 1, rate the extent to which you believe SI reached its stated objectives in WCI-2. ____ 4. What factors intervened to promote or mitigate achieving WCI-2 objectives? HQ Level: COUNTRY: ANSWER THE FOLLOWING QUESTIONS TO THE EXTENT YOU CAN: Policy Issues 1. What specific contributions were made under the WCI-2 grant promoting policies and practices that affected women and children? ? Please cite at least two contributions by order of priority. Headquarter QS a) _________________________________ b) _________________________________ 2. How has WCI strengthened/promoted “gender integration” within SC and its Alliance member organizations? 3. To what extent has WCI been able to influence women & children policies outside of SC? Provide examples of policy changes and on what organizational entity. 4. What are the major differences of WCI-2 with that of the original WCI-1 grant? Strengthening Systems 5. What M&E systems have been developed to support “integrated programming and planning” as opposed to single sector PME? 6. What skills do you feel best represent the gains made under the WCI-2 grant? a)____________________ b) ____________________ c) _______________________ Please discuss: Field Questions USAID/PVC EVALUATION SAVE THE CHILDREN WOMEN/CHILD IMPACT 2 GRANT PVC/SAVE THE CHILDREN FIELD QUESTIONNAIRE PERSON INTERVIEWED: _____________________________________________________ POSITION IN SC: _____________________________________________________ LOCATION: HQ-DC____ HQ-SC____ COUNTRY: El Sal ___ Ethiopia ___ Bangladesh ___ DATE OF INTERVIEW: _______________ TYPE OF INTERVIEW: Face to Face _____ TIME: _______________ Phone _____ General Issues On a scale of 1-5 (1=best, 2=good, 3=moderate, 4=poor, 5=very poor); please provide your assessment of the performance of the WCI-2 grant in terms of the following Intermediate Results a. Promotion of internal/external policies and practices positively impacting women/children; ____ b. Expanded capacity to plan/support the implementation/monitoring of sustainable field programs; c. Improved organizational capacity to conduct outreach to women/children at the field level; 2. How has Save the Children directly benefited from the WCI-2 grant? Please cite at least two benefits by order of priority. a)_______________________ b) ______________________ 3. Using the same scale in question 1, rate the extent to which you believe SI reached its stated objectives in WCI-2. ____ 4. What changes occurred in SI organizational structure and leadership as a result of implementation of WCI-2? HQ Level: COUNTRY: 5. What factors intervened to promote or mitigate achieving WCI-2 objectives? HQ Level: COUNTRY: Field Questions ANSWER THE FOLLOWING QUESTIONS TO THE EXTENT YOU ARE ABLE: Strengthening Systems 1. What skills do you feel best represent the gains made under the WCI-2 grant? a)____________________ b) ____________________ c) _______________________ 1. How is SC/_____________ and its partners using these skills gained through this project in other programs? 2. What effect did participation in the Strategic Partnership have on the operational or technical capacity of the local partner? 3. Was your country’s Strategic Partnership a formal relationship or more that of carrying out a process? Please discuss. 4. In strengthening its organizational capacity, how does SI coordinate resources from different grants, in particular, the Food for Peace program? Field Program Implementation 1. Is your country using PME systems to assess program/project performance? Please describe the nature of these systems. 2. To what extent have these PME systems been able to measure the results of the WCI-2 grant? Rate on a scale of 1-5 (1=best, 5=very poor). ______ 3. How are indirect beneficiaries of WCI-2 identified and enumerated in contrast to direct beneficiaries? 4. In what ways have WCI’s Strategic Partnerships increased field capacity to implement partnership models? 5. Were sustainability plans integrated into WCI-supported SC programs? How effective were these plans? 6. How has your country leverage WCI-2 grant monies? 7. How effective has the Economic Opportunities program been in increasing outreach to women in Field Questions need? Has was this program integrated with other SC resources? 8) What degree or level of integration existed between the WCI-2 project and country level food security programs—if at all? Partners USAID/PVC EVALUATION SAVE THE CHILDREN WOMEN/CHILD IMPACT 2 GRANT PARTNER QUESTIONS PERSON INTERVIEWED: ___________________________________________________ POSITION/ORGANIZATION: ________________________________________________ INTERVIEW LOCATION: ___________________________________________________ Date of Interview: ________________________ Time of Interview: ___________________ 1) How would you describe your relationship with Save the Children? (What kind of relationship is it?) To your knowledge is there a written agreement? 2) What are the strong points of the relationship? 3) What are the weak points of the relationship? (What things do you wish were different?) 4) What are the most important things you/your organization have gained under the[name of partnership project]? a)_____________________b)____________________c)_____________________ 5)What effect did participation in the [name of partnership] have on your organization’s operational or technical capacity? 6) How are you using the skills gained through this project in your other programs or work? 7) On a scale of 1 to 5 [1=best and 5=very poor], please assess the [partnership project’s] performance in terms of: a) ____ reached its goals; b) ____ improved local partner capacity; and c) ____ quality of the partnership. WCI-2 SOW 66 USAID/PVC EVALUATION SAVE THE CHILDREN WOMEN/CHILD IMPACT 2 GRANT BENEFICIARIES FOCUS GROUP QUESTIONS 1) How long have you (speaking to the group) been involved with this Early Childhood Center/Health Program of Save the Children/______________? 2) How did you get involved? 3) During the time you have been associated with Save the Children\__________, what practices have changed that directly affect women and children? 4) What do you now do differently as a result of participating in this program? 5) How does your husband and/or mother-in-law feel about this program? 6) What aspects of the Save the Children project do you expect to continue? 7) How have YOU benefited from participating in the Save the Children program? a) ______________________________________________________________ b) ______________________________________________________________ d. How has YOUR family benefited from participating in the Save the Children program? c) ______________________________________________________________ d) ______________________________________________________________ 67 ANNEX D: ADVOCACY IMPLEMENTATION CHAIN (NB: Examples cited are only those that relate to WCI-funded activities) 1. Country 1 Program: BANGLADESH Responsible Organizatio n (Type) 59 Advocacy Capacity Improvement Activities60 Organization( s) targeted for improvement Advocacy policy Targets and (institution targeted) Actual Advocacy Events Policy changes realized Comments National Level Partnership/ ECD Unit (2 PVOs, 4 local NGOs) NA Local NGOs Goal: recognition of importance of quality ECD programs and trained caregivers Outreach, production and dissemination of publications, production of training materials, training courses UNICEF continues advocacy with government World Bank sought partner input for current 5- year education plan Local Level ß ß ß 59 Describe in parentheses as USPVO, LNGO, Gov’t., business, etc. 60 List as separate rows advocacy activities that an organization does directly by itself (such as a USPVO advocating directly with government or establishing policy forums), and tactics where an organization strengthens another organization to perform the advocacy (such as when a USPVO strengthens LNGO capacity to conduct advocacy.) 68 2. Country 2 Program: EL SALVADOR Responsible Organizatio n (Type) Advocacy Capacity Improvement Activities Organization( s) targeted for improvement Advocacy Targets (policies) and institution targeted Actual Advocacy Events Policy changes realized Comments National Level USPVO (SC/US) NA Local NGO, INGO and government programs for young children Goal: recognition of role that quality ECD programs play in reduction of violence within families and in society at large; chiefly, the Instituto Salvadoreno para la Protection de los Minores (ISPM) Two national workshops, awareness-raising for municipal leaders, training for preschool and mother caregivers ISPM now actively promotes ECD programs and nationwide use of SC-developed training curriculum; universities have modified curriculum and service deliverers are using it Partnership continues; media, Supreme Court, first lady actively involved. SC/US programs in region also embrace ECD. ß Local Level ß ß ß 69 3. Country 3 Program: ETHIOPIA Responsible Organizatio n (Type) 61 Advocacy Capacity Improvement Activities62 Organization( s) targeted for improvement Advocacy policy Targets and (institution targeted) Actual Advocacy Events Policy changes realized Comments National Level USPVO (SC/US) NA National, zonal, regional and local (woreda) committees that oversee high schools in Addis Ababa Goal: recognition of importance of providing adolescent reproductive and sexual health (ARSH) information to high school students; Ministries of Health, Education and Labor and Social Affairs (various levels), National Office of Population and national Curriculum Institute Outreach, production and dissemination of training materials, training for committee members, school administrators and teachers ARSH now seen as critical school-based activities; Ministries of Health and Education budgeting for continuation and expansion; Curriculum Institute revising school texts SC/US has been asked to develop a model for delivering ARSH services in rural areas USPVO (SC/US) NA Local NGOs Goal: acceptance of community-managed primary schools as legitimate means of achieving EFA Starting up coalition of local NGOs as Basic Education Network (BEN) Government accepts model BEN functions actively (SC/US is observer) Local Level ß ß ß 61 Describe in parentheses as USPVO, LNGO, Gov’t., business, etc. 62 List as separate rows advocacy activities that an organization does directly by itself (such as a USPVO advocating directly with government or establishing policy forums), and tactics where an organization strengthens another organization to perform the advocacy (such as when a USPVO strengthens LNGO capacity to conduct advocacy.) 70 ANNEX E: PARTNERSHIP TABLE, BY COUNTRY PROGRAM VISITED The following tables summarize the relationships between the various Partners in the project, as they relate to the USPVO. Arrows indicate sources of funding emanating from the MG. Dotted lines indicate funding from other sources. The following key indicates shape corresponds to which kind of organization. Table E 1: Key to Partnership Tables NB: Not applicable to Save the Children partnerships (see descriptions in Table E1 below). USPVO Sub-sub grantee Affiliate/ Independent CBO Primary Grantee Key to Symbols Implementation Partners that don’t get MG funds Funding/TA 71 Table E 1: Country 1 Partnership Table: BANGLADESH Level Organizations Organization Type63 Agreement type64 Role/Responsibility65 Funding level, source, and autonomy66 Quality and outcome of Partnership67 Primary Local Partners68 Plan International*, Gonjo Shahajjo Sangstha, Phulki, Grameen Shikha, VERC INGO, local NGOs MOUs Service delivery, training materials development (Plan and GSS also shared costs) Successful in developing model materials still in use, attracting positive attention from World Bank/UNICEF/government; SC/US and Plan programs continue but ECD unit no longer exists. Partners that are “subs” to primary partners69 Gonjo Shahajjo Sangstha Local NGO Service delivery (expansion) Subgrant from SC/US for program implementation GSS no longer operating Implementation partners that do not receive funds Phulki, Grameen Shikha, VERC Local NGOs Service delivery Programs continue Customer partners *Plan International and SC/US shared the investment costs of the ECD unit on a 50:50 basis. Table E 2: Country 2 Partnership Table: EL SALVADOR 63 For example, LNGOs, affiliates of the PVO, private or commercial groups, cooperatives, community-based organizations, regional or local governments or intermediate service organizations. Add any characterization that is useful to your analysis (for example, environmental, anti-corruption, new organization, etc.) 64 For example, MOU, sub-grant, contract 65 For example, service delivery, aggregation of interests, target of institutional strengthening, or others. 66 USD equivalent of funds provided through the grant and degree of autonomy that organization has over expenditure. Indicate source of funds (USPVO, affiliate, LNGO, or sub-awardee, or other support organization) for funds trickling down from MG. 67 Was Partnership successful? Is it still ongoing? Growing? Has it terminated as planned? What are organizations’ (USPVO and other organizations’) perception of the quality of the Partnership? 68 Receiving funds directly from the USPVO. 69 Organizations that receive funds that are re-granted from TNC, through and intermediate primary partner. 72 Level Organizations Organization Type Agreement type Role/Responsibility Funding level/ autonomy Quality and outcome of Partnership Affiliates/Other Independent Partners Plan International INGO MOU Service delivery NA Perceptions positive; collaboration continue Primary Local Partners Institute for the Protection of Minors government MOU Policy development and oversight NA Very positive; collaboration continues. National network of organizations concerned with ECD active. (SC obtained USAID and World Bank funding for further work in ECD.) Partners that are “subs” to primary partners ASPROSAR Local NGO MOU Service delivery Subgrant from SC/US for service delivery Very positive; relationship continues Implementation partners that do not receive funds Customer partners 73 Table E 3: Country 3 Partnership Table: ETHIOPIA Level Organizations Organization Type Agreement type Role/Responsibility Funding level/ autonomy Quality and outcome of Partnership Affiliates/Other Independent Partners Primary Local Partners Addis Ababa high schools Public sector NA Provide teachers’ time, space for ARSH clubs NA Very positive Partners that are “subs” to primary partners Implementation partners that do not receive funds Ministries of Health, Education, Labor and Social Affairs, National Office of Population; others Government MOU Support programming, provide coordination among ministries NA Very positive; demands for expansion continue Customer partners 74 ANNEX F: SUSTAINABILITY ANALYSIS Table F 1: Country 1 Sustainability Analysis: BANGLADESH Item Supporting factors Inhibiting factors Conclusion Political Support of World Bank, UNICEF and other prominent donors Need for more engagement of Ministry of Education Institutional Increased capacity for and long￾term commitment to quality ECD programming exists within Plan, SC/US, Phulki, Grameen Shikha, VERC Funding to produce materials, conduct training, expand service delivery Financial Plan and SC/US have private funding to support continuation of ECD programming SC needs to more effectively leverage public/other private funds to support ECD/M&E and increase impact; not clear that local NGOs have funding to support such programs 75 Table F 2: Country 2 Sustainability Analysis: EL SALVADOR Item Supporting factors Inhibiting factors Conclusion Political ISPM feels “full ownership” of new policy – and there is increasing recognition in El Salvador of violence/gender as a national issue Violence/gender are still not at top of donor agendas Institutional ISPM is well-funded, technically able and likely to continue promoting ECD Competition of post-disaster basic human needs (e.g., shelter); fragility of mother caregiver networks (more staff are needed) SC has long-term commitment to ECD and staff skilled in this area Financial SC need to leverage additional funds to retain staff and continue programming/advocacy/M&E in ECD 76 Table F 2: Country 3 Sustainability Analysis: ETHIOPIA Item Supporting factors Inhibiting factors Conclusion Political National, zonal, regional and woreda committees feel ownership of ARSH programs and see them as umbrella for HIV/AIDS and other efforts Government budgets are very limited; large donors need exposure to this program model Institutional Committees are technically capable of maintaining coordination, management and advocacy roles Committee members have many other responsibilities SC staff members with personal networks remains on staff Staff members could leave Financial Packard Foundation supported model program SC needs to leverage additional funds to retain staff and expand programming/M&E in rural areas Government is now budgeting for ARSH program costs Not clear what the costs of a rural model will be 77 Table F 3: Headquarters Continuing Effort to Support Changes Item Supporting factors Inhibiting factors Conclusion Political Advocacy SC Board has accepted this key element of strategic plan. Post￾September 11 environment is one where the needs of women and children can be promoted. Humanitarian needs of Afghanistan may be seen as politically paramount; US foreign aid budget could take this and related support from other foreign aid programs Partnership Concept and practice are accepted within SC SC needs to seek opportunities to work with large national NGOs; funding is not always conducive to partnership M&E Remains high on SC’s agenda for implementing internal organizational change and improving program impacts; donors also interested Many donors continue to have specific requirements for reporting that make creating consistent systems challenging. Institutional Advocacy HQ advocacy staff hired and field offices now hiring staff who can speak with government/donors Staff are stretched with other responsibilities. Partnership Partnership Resource Book will be available in early 2002. Staff need training in the “nuts and bolts” of partnering and support from senior leadership in taking risks associated with partnership M&E Most staff see their job as including responsibility for planning, monitoring and evaluation; new COO has this as a high priority. SC must decide what data it needs centrally and at field level; staff are stretched with other responsibilities – and must be trained in new system/skills. Item Supporting factors Inhibiting factors Conclusion Financial Advocacy Private funds secured to increase Need continued funding, and 78 capacity of field and HQ staff as spokespersons, mobilize US constituents. new funds to document success stories in a marketable form Partnership Many donors pushing this approach in addition to internal momentum Cost-effectiveness not clear; donors not always willing to pay for “process” time M&E Finding public or private funding for this function has been difficult (expected to be a core function). 79 ANNEX G MAJOR PLANNING, MONITORING, AND EVALUATION ACTIVITIES OVER THE 5 YEARS AT SAVE THE CHILDREN FEDERATION (FY97-FY01) Event Report/Document Date Content Conference on Sponsorship focusing on Child Benefits in 9/96 (Kent, CT), followed by regional workshops Policy Paper on Child Benefits 3/97 · Organizational framework for sponsorship programming established · Framework includes program principles · IP program strategy with the WCI approach SC’s Sponsorship Management Policies & Procedures Manual 10/97 Provides organizational framework for sponsorship programming, including program principles and WCI approach Consultation by John Bryson Memo of Oct. 3, 1997 from Gary Shaye to all IP staff Suggestions regarding approach to strategic planning at the departmental level, leadership development Workshop on Measuring Program Impact (Cyprus) Road map to Success: Measuring Program Impact 11/97 · Review of program planning and common definitions · Project monitoring system · SC new impact monitoring system · Partnerships · Implementation plan for new framework · Matrix of core indicators What is a Strategic Opportunity Revised 3/98 Improved screening process to ensure those opportunities funded reflect programmatic, geographic, and/or financial scale – or clearly have potential for achieving scale Overview of IP Strategic Planning Process 4/98 Established planning process linking strategic plans of the agency to those of IP, the Areas, and the Field Offices IP Meetings in 2/98 (Pocantico, NY); IP divisional retreats in spring and summer 1998 Breakthroughs for Children, IP’s Strategic Plan for 1999-2003 2/98 and beyond · External environment analysis · IP program strategy: goals, program focus, program principles, program strategies, WCI, sector strategies · Strategic Objectives, implementation plan · M&E plan, updated matrix of core indicators IP Area Meetings (around the world) 4/98-5/98 · Roll-out of IP’s Strategic Plan, linking to Area and Field Office 3-year strategic plans · Roll-out of new impact monitoring system, with core 80 Event Report/Document Date Content indicators and beneficiary counts Development Programs’ Strategic Plan for 1999-2001 8/98 · Overall trends and vision · PSPs of individual sectors Tools and Techniques for Program Quality 8/98 · Issued by the WCI Office · Tools for conducting program principle analysis on programs IP Meeting with all Directors, AVPs, VP (Washington, DC); workshops with other SC staff (Westport) Training Modules for “Breakthroughs for Children” 1. Generic content 2. Specific to each dept. 10/98 · Review of key points of IP strategic plan · Development of specific plans for achieving the Strategic Objectives A Review of the External Evaluations of Development Projects Implemented by Save the Children 3/99 · By Bradshaw Langmaid · Covering 15 years · Summary: SC successfully broadened programmatic reach without losing focus and core values. Noted innovation. Difficulties arose when undertook activities outside core program areas without first building capacity. IP Area Meetings (around the world) 4/99-5/99 · Area and Field Office strategic & operational planning · Review of the impact monitoring system, in particular core indicators · Introductory session to Results Frameworks IP Meeting: Managing for Growth, Quality & Impact Notebook with compilation of conference documents 10/99 · Planning discussions · Results Framework training by 2 people from USAID Significant and Beyond: A Point of Departure for Discussion 1999/2000 Linking significant benefits for sponsored children to program quality – programs that are life-changing or life￾saving (child-focused), sustainable (empowerment), and strategic (complementing IP program strategies and program principles) Planning Conference Presentation to all staff on 2/2/00 1/00 Plan for developing a 3-year plan that identifies key strategic priorities for focused program, policy, communication and fundraising efforts Norwalk I Leadership Conference Notebook with compilation of conference documents 4/00 Agency-wide strategic planning IP Area Meetings (around 4/00-5/00 Area and Field Office strategic and operational planning, 81 Event Report/Document Date Content the world) linkages to preliminary directions of the organizational strategic plan, sessions on Results Frameworks Addition of position: Planning & Impact Evaluation Specialist 10/00 Based in the field (Ethiopia) Development of PME training materials Survey of field offices about the POP guidelines 11/00 Conducted by Kate Robins, WCI PIE Director Norwalk II Leadership Conference Notebook with compilation of conference documents 11/00 · Agency-wide strategic planning, development of the High Impact Initiatives · Discussion group on PME led by Director for Planning & Impact Evaluation SC’s Core Documents on CD￾ROM 11/00 Compiled by the WCI team Workshop on Developing a Common Approach to Sponsorship-Funded Programming Sponsorship Strategic Plan Implementation Summary – Strategic Priority V. 3/01 · Plan for improving the quality, deployment, and cost￾effectiveness of sponsorship-supported programming · Determination of the template of community programs (school health, ECD, etc.) · For the future: establish requirements for measuring impact for each program area. Revised POP and PSP guidelines 3/01-8/01 · Field planning within the context of the new agency strategic plan and High Impact Initiatives PME Working Group revived, meeting monthly Checklist for review of program plans in field offices’ POPs 7/01 Also working on defining the PME Framework, etc. Developing the implementation plan for SC’s Breakthroughs for Children strategy with High Impact Initiatives 1/01-4/01 IP Area Meetings, mini￾workshops for WWO staff 4/01-5/01 · Training in implementation of the new organizational strategic plan with a focus on the High Impact Initiatives · Linking strategic and operational planning throughout the agency IP Area Directors Meeting Facilitation Guide for SC’s new strategic plan and High Impact 7/01 · PSP guidelines · POP review process 82 Event Report/Document Date Content Initiatives · System of management reviews · Post-WCI support to PME · PME framework · Annual Area reports August 14, 2001 83 ANNEX G2 SELECTED WCI-II PUBLICATIONS/PRODUCTS (1996-2002) The Emerging Global Consensus on Gender: An Analysis of Recent UN Conference Commitments. Catherine Wiesner. 1997. Empowerment Through People, Programs and Institutions: A Report on the FACETS Phase III Workshop. 1998. FACETS Annual Meeting (Sri Lanka, December 1999). 2000. Gender Equity Policy: The International Save the Children Alliance. English, French and Spanish. 2000. Gender Mainstreaming: A Resource Guide. 2000. Gender Relations Analysis: A Guide for Trainers. A. Rani Parker, Itziar Lozano and Lyn A. Messner. French and Spanish versions (1995 English publication). 1998. The Human Rights of Women and Children: Challenges and Opportunities. 1998. International Programs Handbook. Save the Children, 1997-98. Key Save the Children Documents (CD-ROM). 2000. Partnership and Institutional Relationships: A Resource Book for Save the Children. Volume I: Principles of Good Practice/Annotated Bibliography and Volume II: Case Studies. Forthcoming (Spring 2002). A Planning, Monitoring and Evaluation Framework. Save the Children. Forthcoming (Spring 2002). Preventing Family Violence: A Manual for Action. Josephine Warrior. 1999. Program Exchange: Save the Children’s International Programs Newsletter. 18 issues. 1996 - 2001. Road Map to Success: Measuring Program Impact (Cyprus, November 1998). 1998. Save the Children: Key Program Principles. 1997. Strategic Partnerships: Development, Management and Measurement (WCI Inter-Hub Workshop, Guatemala, November 1999). 1999. Surviving Widows (VIDEO). 1999. Towards Community Empowerment for Gender Equity in Inheritance Practices: Action Research. 2000. Training Resources for SC Program Principles: An Annotated Bibliography. 1997. 84 WCI Hub Summit Report (Addis Ababa, May 2000). 2000. NB: In addition to the list above, numerous workshop reports, gender relations analyses findings and other resource materials were prepared with support from the WCI-II grant and are available in the field as well as Save the Children’s Washington Office. 85 ANNEX H1: TURNOVER REPORT 86 SC Staff Turnover, FY 1997 to FY 2001 HQ-paid Permanent staff only hris 09/27/01 Combined HQ-paid Staff: FY 1997 FY 1998 FY 1999 FY 2000 FY 2001 1st 3Q FY01 1st 3Q FY00 Average Employment 380.0 372.5 327.0 355.5 400.5 399.0 348.0 Turnover 30.5% 42.4% 32.7% 25.6% 22.2% 16.5% 16.7% Remarks: From FY97 to FY01 Combined HQ-paid Staff increased 5.4%. FY01 turnover was 22.2%, lowest of the five-year period. By Population 1. Westport/DC Staff FY 1997 FY 1998 FY 1999 FY 2000 FY 2001 1st 3Q FY01 1st 3Q FY00 Average Employment 186.0 203.5 209.0 222.0 242.0 241.5 213.5 Turnover 18.8% 24.1% 34.5% 27.5% 18.6% 14.1% 17.5% Remarks: From FY97 to FY01 Westport/DC Staff increased 30.1%. FY01 turnover was 18.6%, lowest of the five-year period. 1a. Westport Office Staff FY 1997 FY 1998 FY 1999 FY 2000 FY 2001 Average Employment -- -- 179.5 177.0 189.5 Turnover -- -- 38.4% 28.8% 15.8% Remarks: From FY99 to FY01 Westport Staff increased 5.6%. FY01 turnover was 15.8%, lowest of the three-year period. 1b. D.C. Office Staff FY 1997 FY 1998 FY 1999 FY 2000 FY 2001 Average Employment -- -- 29.5 37.0 52.5 Turnover -- -- 10.2% 21.6% 28.6% Remarks: From FY99 to FY01 D.C. Office Staff increased 77.9%. FY01 turnover was 28.6%, highest of the three-year period. 2. USP Field Office Staff FY 1997 FY 1998 FY 1999 FY 2000 FY 2001 1st 3Q FY01 1st 3Q FY00 Average Employment 101.0 86.0 33.0 41.0 41.5 41.5 39.5 Turnover 41.6% 82.6% 30.3% 13.2% 26.5% 16.9% 7.7% Remarks: From FY97 to FY01 U.S. Programs Field Staff decreased 58.9%. FY01 turnover was 26.5%, second lowest of the five-year period. 3. IP Field Office Staff FY 1997 FY 1998 FY 1999 FY 2000 FY 2001 1st 3Q FY01 1st 3Q FY00 Average Employment 93.0 83.0 85.0 103.5 117.0 116 97 Turnover 41.9% 41.0% 32.9% 26.1% 28.2% 21.6% 21.3% Remarks: From FY97 to FY01 International Programs Field Staff increased 25.8%. FY 2001 v. FY 2000: Comparison of 1st Three Quarters 87 ANNEX H2: SAVE THE CHILDREN DIVERSITY PATTERNS As of 6/30/97 Worldwide Staff Job Category Total Employees Percent Minority Percent Non￾Minority Percent Male Percent Female Officials & Managers 227 39% 61% 52% 48% Professionals 866 76% 24% 58% 42% Technicians 166 98% 2% 67% 33% Office & Clerical 396 77% 23% 41% 59% Other* 505 89% 11% 90% 10% Totals 2160 77% 23% 62% 38% IP Field Staff Job Category Total Employees Percent Minority Percent Non￾Minority Percent Male Percent Female Officials & Managers 136 53% 47% 68% 32% Professionals 751 82% 18% 63% 37% Technicians 165 98% 2% 67% 33% Office & Clerical 314 90% 10% 50% 50% Other* 501 90% 10% 90% 10% Totals 1867 85% 15% 69% 31% HQ-DC Staff Job Category Total Employees Percent Minority Percent Non￾Minority Percent Male Percent Female Officials & Managers 62 16% 84% 37% 63% Professionals 68 18% 82% 26% 74% Technicians 1 0% 100% 100% 0% Office & Clerical 60 15% 85% 10% 90% Other* 4 25% 75% 50% 50% Totals 195 16% 84% 26% 74% US Field Staff 88 Job Category Total Employees Percent Minority Percent Non￾Minority Percent Male Percent Female Officials & Managers 29 24% 76% 14% 86% Professionals 47 62% 38% 17% 83% Technicians 0 0% 0% 0% 0% Office & Clerical 22 55% 45% 0% 100% Other* 0 0% 0% 0% 0% Totals 98 49% 51% 12% 88% *Other – Sales Workers, Craft Workers, Operatives, Laborers, and Service Workers Note: For purposes of this diversity report, all employees (domestic & overseas) have been categorized as per the US Government definition of minority (Black, Hispanic, Asian, Native American) 89 ANNEX H2: SAVE THE CHILDREN DIVERSITY PATTERNS As of 6/30/01 Worldwide Staff Job Category Total Employees Percent Minority Percent Non￾Minority Percent Male Percent Female Officials & Managers 405 56% 44% 58% 42% Professionals 714 59% 41% 48% 52% Technicians 537 66% 34% 56% 44% Office & Clerical 569 75% 25% 49% 51% Other* 751 68% 32% 85% 15% Totals 2,976 65% 35% 60% 40% IP Field Staff Job Category Total Employees Percent Minority Percent Non￾Minority Percent Male Percent Female Officials & Managers 298 69% 31% 66% 34% Professionals 576 66% 34% 54% 46% Technicians 536 66% 34% 56% 44% Office & Clerical 519 78% 22% 53% 47% Other* 748 68% 32% 85% 15% Totals 2,677 69% 31% 64% 36% HQ-DC Staff Job Category Total Employees Percent Minority Percent Non￾Minority Percent Male Percent Female Officials & Managers 90 13% 87% 32% 68% Professionals 124 23% 77% 23% 77% Technicians 1 0% 100% 100% 0% Office & Clerical 39 38% 62% 5% 95% Other* 3 0% 100% 67% 33% Totals 257 21% 79% 24% 76% US Field Staff Job Category Total Employees Percent Minority Percent Non￾Minority Percent Male Percent Female 90 Officials & Managers 17 59% 41% 41% 59% Professionals 14 71% 29% 21% 79% Technicians 0 0% 0% 0% 0% Office & Clerical 11 64% 36% 9% 91% Other* 0 0% 0% 0% 0% Totals 42 64% 36% 26% 74% *Other – Sales Workers, Craft Workers, Operatives, Laborers, and Service Workers Note: For purposes of this diversity report, all employees (domestic & overseas) have been categorized as per the US Government definition of minority (Black, Hispanic, Asian, Native American) 91 ANNEX I 92 93 APPENDIX 1: REALIGNED WCI GRANT C:\12-20-working\New Folder\SAVE evaluation final report.doc 94 Appendix 1 WCI Grant Strategic Objective Intermediate Results 1) External Policy Involvement Increased contribution to and promotion of policies and practices which positively impact women and children 2) Organizational Systems Strengthened capacity to plan, and support the implementation and monitoring of sustainable field programs focused on the empowerment of women and children 3) Program Implementation Increased program outreach to women and children through improved organizational capacity at FO level Indicators •Organizational systems in place to assess program quality and impact on women and children •# of new programs which meet quality indicators in respective sector(s) of implementation •Decisions regarding SC involvement in policy dialogues (representation and resource allocation) coordinated between departments •% increase in number of international forums in which SC staff participate intended to positively impact women and children •# of documented instances of improved gender integration within SC Alliance member organizations as a result of adoption of SC Alliance Gender Policy •Planning, Monitoring and Evaluation (PME) framework agreed on at HO •PME Working Group formed and active •Preliminary PME tools prepared •IP Capacity Building RF reviewed and revised every six months •% of PLGs where WCI staff and/or FACETS participate •# of [new] programs with defined sustainability/exit plan or strategy •POP process includes criteria for ensuring that programs incorporate PPs •Field TA program planned and agreed on •# of new programs designed which explicitly incorporate all PPs in their design •# of women and children reached through SC programs •# FOs using RFs for project and country planning in FY2001 Indicators Improved Capacity to Implement Gender-sensitive, Quality Programs which Increase Scale and Sustainability 95 APPENDIX 2: WCI INTERMEDIATE RESULTS External Policy Involvement IR1: Increased contribution to and promotion of policies and practices which positively impact women and children ÿ How effective was SC’s participation/contribution in promoting policies and practices that positively affect women and children? ÿ How has WCI increased woman and child advocacy and participation - staff as individuals, and as an agency? ÿ What has been WCI’s impact within the Save the Children Alliance? ÿ How has WCI strengthened/promoted gender integration within SC and its Alliance member organizations? ÿ How has this in turn influenced gender mainstreaming and consequent implementation of programs of SC partners and community groups? ÿ How effective were the mechanisms to achieve policy influences outside of SC? ÿ What was the value added of the Leadership for Social Change course for SC? How effective was the course in reaching immediate and extended beneficiaries? ÿ In what ways is the course influencing current and future policy makers? Strengthening Systems IR2: Strengthened capacity to plan, and support the implementation and monitoring of sustainable field programs focused on the empowerment of women and children ÿ What tools and systems have been developed: a) to lead/support intersectoral programming/planning; and b) for improved agency-wide monitoring and evaluation? ÿ How are these being integrated into ongoing programming and operations? ÿ Are SC and its partners using skills gained through this project in other programs? ÿ In what ways have WCI’s Strategic Partnerships increased field capacity to implement partnership models? ÿ What effect did participation in the Strategic Partnership have on the operational or technical capacity of the local partner? What would make the partnership more effective? Cite the major implementation lessons learned and recommendations such alternative approaches to the current partnership models. ÿ How has WCI affected the development and adoption of gender-related policies within SC? ÿ How will gender equity considerations be integrated into programming and management beyond the end of the Grant? ÿ How has WCI increased organizational learning and capacity within International Programs? Field Program Implementation IR3: Increased program outreach to women and children through improved organizational capacity at FO level 96 ÿ How has WCI promoted the use of Planning, Monitoring, & Evaluation (PME) tools or systems in the field? To what extent were these tools used in the field to measure the application/integration of the Program Principles? Specifically, how effective were the tools in promoting quality programming? ÿ How effective has the Economic Opportunities program been in increasing outreach to women in need? How has Save the Children leveraged the resources provided by WCI? ÿ What are the implications, lessons learned and recommendations from the WCI Strategic Partnerships? ÿ To what extent was partnership used as a key implementation strategy in other programs of hub countries (other than the SP models)? in non-hub countries? Was this the result /influence of the WCI program? ÿ Were sustainability plans integrated into WCI-supported SC programs? How effective were the plans? What were the concrete lessons learned with regard to the quality, scale, and sustainability of programs in the field? ÿ What has been the impact of the GRA exercises in terms of delivering programs with higher positive impact on women and children? What are the primary lessons learned and how are they being applied? ÿ To what extent do field programs generally conform to the requirements of the gender-related policies? ÿ How has the Program Innovation Fund effected positive change for women and children? What are examples of its use? How were promising innovations documented and disseminated? Other (for all IRs) ÿ Were activities carried out under the different IRs appropriate and adequate to achieve the intended results? ÿ What changes have occurred in organizational structure and leadership as a result of implementation of WCI? ÿ How can evaluation findings help further incorporation/institutionalization of the WCI approach within SC, with specific emphases on gender, partnership, and (PME)? Grants FY97 - 01 (Bangladesh, El Salvador, and Ethiopia) 18-Dec-01 GL # FO Donor Name Number Amount Begin End Match ICR Current Obligation Amount Match Total Estimated AF Ethiopia 10128 Ethiopia AID Title II FY 96 663-0802-G-00-6052 10/1/95 9/30/98 634,088 0 634,088 0 22.06% 10140 Ethiopia AID Monetization 97 10/1/96 9/30/97 1,101,476 0 1,101,476 0 24.95% 10155 Ethiopia AID CS 13 Ethiopia FAO-A-97-00054 9/24/97 9/29/01 1,000,000 333,300 1,000,000 333,300 16.98% 10159 Ethiopia AID Monetization 98 10/1/97 2,252,086 0 2,252,086 0 16.98% 10165 Ethiopia AID OFDA Dollo Emg Asst AOT-G-00-98-00059 1/28/98 1/31/99 204,266 0 204,266 0 24.95% 10179 Ethiopia AID Monetization 99 10/1/98 9/30/99 1,375,295 0 1,375,295 0 24.95% 10192 Ethiopia AID Cross Border Pilot 663-A-00-99-00330 9/24/99 9/30/01 199,993 0 199,993 0 16.98% 10193 Ethiopia AID Cross Border Animal Healt 663-A-00-99-00329 9/20/99 12/31/02 449,463 0 449,463 0 16.98% 10196 Ethiopia AID Monetization 2000 10/1/99 9/30/00 595,432 0 595,432 0 16.98% 10203 Ethiopia AID OFDA Emg. Response AOT-G-00-00-00068 2/16/00 1/31/01 1,312,441 0 1,312,441 0 16.98% 10215 Ethiopia AID Pastoralist Education 663-A-00-00-00360 9/26/00 9/30/03 265,000 117,280 307,480 136,078 16.98% 10221 Ethiopia AID Monetization 2001 10/1/00 9/30/01 1,255,296 0 1,255,296 0 16.98% 10222 Ethiopia AID Dollo Bay Emergency 663-G-00-01-00320 3/8/01 4/30/01 3,645 0 3,645 0 0.00% 10228 Ethiopia AID High Risk Corridor-HIV Pre 663-A-00-01-00350 7/2/01 9/30/03 500,000 196,000 1,905,211 196,000 16.98% 10229 Ethiopia AID Drought Recovery HAD-G-00-01-00078 5/5/01 4/4/02 476,136 0 476,136 0 16.98% 10230 Ethiopia AID Title II Dev't Ass't 663-G-00-01-00363 6/9/01 9/30/02 335,185 92,593 335,185 92,593 16.98% 30173 Ethiopia AID Title 2 SCT 202E FFP-G-00-97-00045 10/1/96 9/30/99 216,317 0 216,317 0 24.95% 30174 Ethiopia AID 202e FFP-G-00-98-00004 10/1/97 9/30/03 1,384,207 327,400 1,884,207 327,400 16.98% 30177 Ethiopia AID Inland Transport FFP-G-00-99-00083 10/1/98 12/31/00 120,000 0 120,000 0 0.00% 30178 Ethiopia AID ITSH 2001 FFP-G-00-01-00056 10/1/00 12/31/02 196,200 0 196,200 0 0.00% 50007 Ethiopia Bodman Founda Ethiopia Emergency 9/18/00 9/30/01 15,000 0 15,000 0 16.98% 66911 Ethiopia Cansave Yifat Na Timuga 9/27/89 8/31/97 54,283 0 54,283 0 0.00% 44653 Ethiopia Christian Aid Liben Project 9/1/98 8/31/99 100,000 0 100,000 0 14.00% 44687 Ethiopia Christian Aid Early Warning/Emg 5/3/00 5/7/01 41,935 0 41,935 0 12.00% 55001 Ethiopia Citigroup Fdn Emergency Drought Relief 8/1/00 365,000 0 365,000 0 16.28% 44571 Ethiopia CRDA Small Business for DS007/94/7 4/4/94 8/31/97 4,298 0 4,298 0 10.00% 44629 Ethiopia CRDA Savings and Credit EX 583/97/6-2 4/1/97 3/31/99 14,085 9,125 14,085 9,125 10.00% 44635 Ethiopia CRDA Urban Health EX587-97-13-4 7/2/97 7,463 0 7,463 0 10.00% 31407 Ethiopia CRS JEOP 2/1/00 12/31/00 3,406,121 3,406,121 0.00% 31418 Ethiopia CRS JEOP 2001 FFP-A-00-01-00014 1/1/01 10/31/01 2,001,340 0 2,001,340 0 0.00% 37408 Ethiopia DOS Jigjiga Refugee Children PRM 11098 7/1/01 6/30/02 251,659 0 251,659 0 16.98% 44541 Ethiopia Future Forests Future Forests 10/1/92 10/1/96 99,390 0 99,390 0 5.00% 44621 Ethiopia Future Forests Bette Valley 97 1/1/97 3/12/99 70,077 0 70,077 0 5.00% Subtotals are on Obligated Amounts 1 GL # FO Donor Name Number Amount Begin End Match ICR Current Obligation Amount Match Total Estimated 50005 Ethiopia Gates Fdn Drought Emg 5/22/00 11/22/00 300,000 0 300,000 0 14.80% 50022 Ethiopia Gates Fdn Drought-Somali Region 11457 5/24/01 5/23/02 500,000 0 500,000 0 16.98% 50012 Ethiopia Lurie Fdn Community Schools 10/1/00 10/31/02 598,871 0 598,871 0 16.98% 44682 Ethiopia Netaid Emergency Drought Relief 2/21/00 8/31/00 75,000 0 75,000 0 20.00% 44673 Ethiopia Packard Fdn Maternal & Child Health 99-9439 9/24/99 9/23/02 353,128 0 353,128 0 14.80% 66980 Ethiopia Redd Barna Rights of Women 4/1/97 7,503 0 7,503 0 15.00% 66988 Ethiopia Redd Barna Supplementary Feeding - D 12/17/97 14,910 0 14,910 0 0.00% 67013 Ethiopia SC Netherlands Gode Emergency 5/1/00 1/31/01 437,000 0 437,000 0 4.76% 67025 Ethiopia SC Netherlands Drought Recovery & Mitigat ECHO/ETH/210/2001/ 2/1/01 12/31/01 642,780 0 642,780 0 4.75% 44546 Ethiopia Stichhiting Kin Stichhitng Kin BH91/581 SG 1/27/93 2/28/97 12,000 0 12,000 0 0.00% 80223 Ethiopia UNHCR Dollo Rehab 8/12/98 4/30/99 75,000 0 75,000 0 15.00% 80235 Ethiopia UNHCR Jigjiga Social Mobilization 99/AP/ETH/CM/200© 6/1/99 12/31/99 129,494 0 129,494 0 15.00% 80245 Ethiopia UNHCR Refugee Camps - Jigjiga 00/AB/ETH/CM/200c 1/1/00 12/31/00 433,206 48,726 433,206 48,726 5.00% 80254 Ethiopia UNHCR At-Risk Refugee Children 00/AB/ETH/CM/256 1/1/00 12/31/00 44,223 0 44,223 0 0.00% 80271 Ethiopia UNHCR Environmental Ed for Refu 01/AB/ETH/LS/450c 1/1/01 12/31/01 42,050 0 42,050 0 5.00% 80272 Ethiopia UNHCR Jigjiga- Somali Refugees 01/AB/ETH/CM/200c 1/1/01 12/31/01 430,204 0 430,204 0 5.00% 80130 Ethiopia UNICEF Street Children 2/8/93 17,911 0 17,911 0 0.00% 80259 Ethiopia UNICEF Separated Children 12/5/00 12/4/01 124,900 0 124,900 0 12.50% 51 24,545,358 AP Bangladesh 03108 Bangladesh **FFS Misc FFS Misc 1/1/97 34,487 0 36,487 0 0.00% 44622 Bangladesh Actionaid ECD Parental Ed 1/1/97 12/31/97 35,388 0 35,388 0 10.00% 44640 Bangladesh Actionaid REFLECT 1/1/98 9/30/99 43,012 17,237 43,012 17,237 10.00% 44611 Bangladesh Citicorp GGLS 7/1/96 50,000 0 50,000 0 25.20% 39704 Bangladesh DFID Reading for Children ED2000/117 6/1/00 12/31/01 29,915 0 29,915 0 4.55% 44587 Bangladesh Hunting Tech Homestead Prod 320757/1/H2/2/114 3/8/95 2/28/97 95,788 0 95,788 0 25.26% 44626 Bangladesh Hunting Tech Homestead Prod A8.8.1/02 4/9/97 9/30/99 106,680 0 106,680 0 22.96% 44655 Bangladesh Hunting Tech/DF Emg. Flood Relief 10/5/98 119,524 0 119,524 0 0.00% 31327 Bangladesh Pathfinder Family Planning DK2/2144-1 6/1/95 5/31/97 188,766 0 188,766 0 22.06% 44667 Bangladesh PLAN ECD 1/1/99 6/30/00 78,494 0 78,494 0 0.00% 44692 Bangladesh PLAN ECD 7/1/00 6/30/01 60,000 40,000 60,000 40,000 0.00% 44609 Bangladesh Plan Int ECD Training 9/1/96 12/31/98 68,414 0 68,414 0 0.00% 66995 Bangladesh Radda Barnen Post Flood Nutrition 23346 9/1/98 7/31/99 200,000 0 200,000 0 0.00% 80157 Bangladesh UNICEF Early Childhood 10/1/94 6/30/97 98,000 0 98,000 0 28.85% 80200 Bangladesh UNICEF Nutrition Project 10/1/96 9/30/98 41,000 0 41,000 0 25.26% 31415 Bangladesh World Vision Polio Eradication HRN-A-00-98-00053-0 5/1/00 4/30/02 54,728 4,562 91,213 4,562 16.98% 16 1,304,196 Subtotals are on Obligated Amounts 2 GL # FO Donor Name Number Amount Begin End Match ICR Current Obligation Amount Match Total Estimated LAC El Salvador 03160 El Salvador **FFS El Salv M Children w/ Disabilities 1434/2000 9/1/00 11/30/00 63,560 0 63,560 0 0.00% 03129 El Salvador **FFS FEPADE Education 4/8/97 12/31/97 49,897 0 49,897 0 0.00% 03115 El Salvador **FFS MOE Mined Project 10/1/96 73,611 0 73,611 0 15.00% 03127 El Salvador **FFS MOE MINED II 879-OC-ES 2/7/97 65,789 0 65,789 0 0.00% 10218 El Salvador AID Early Childhood Ed 519-A-00-00-00105 11/8/00 11/8/02 245,772 82,500 245,772 82,500 16.98% 10219 El Salvador AID Emg. Assistance-Earthqua 519-G-00-01-00067 2/17/01 3/31/01 25,000 0 25,000 0 16.98% 10220 El Salvador AID Temporary Shelter 519-A-00-01-00084 3/7/01 7/31/01 749,195 0 749,195 0 16.98% 10232 El Salvador AID Permanent Housing 519-A-00-01-00224 9/28/01 5/27/02 1,191,534 206,164 1,191,534 206,164 16.98% 50015 El Salvador Gates Fdn El Salvador Earthquake 6903 1/31/01 8/30/02 500,000 0 500,000 0 14.40% 39705 El Salvador GOES Youth Participation Rural D 01-05/99 11/24/00 11/23/01 24,507 2,450 24,507 2,450 0.00% 44671 El Salvador PLAN Training to Caregivers 5/10/99 9/30/99 49,800 7,450 49,800 7,450 0.00% 67027 El Salvador SC Netherlands Earthquake Response 8/13/01 59,850 0 59,850 0 5.00% 12 3,098,515 79 28,948,069 Subtotals are on Obligated Amounts 3