PRESIDENT’S EMERGENCY PLAN FOR AIDS RELIEF TRACK 1.0 ORPHANS AND OTHER VULNERABLE CHILDREN (OVC) PROGRAM EVALUATION December 2008 This publication was produced for review by the United States Agency for International Development. It was prepared by DeeDee Yates, Christopher O’Connell, Nicky Davies, and Muyiwa Oladosu through the Global Health Technical Assistance Project. PRESIDENT’S EMERGENCY PLAN FOR AIDS RELIEF TRACK 1.0 ORPHANS AND OTHER VULNERABLE CHILDREN (OVC) PROGRAM EVALUATION DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 09-001-129) is available in printed or online versions. Online documents can be located in the GH Tech web site library at www.ghtechproject.com/resources.aspx. Documents are also made available through the Development Experience Clearinghouse (www.dec.org). Additional information can be obtained from The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-00. TRACK 1.0 OVC PROGRAM EVALUATION iii ACKNOWLEDGEMENTS The evaluation team particularly thanks Colette Bottini, the OVC Technical Advisor at the Office of HIV/AIDS in USAID, for her gracious assistance with coordination of the evaluation. The team also thanks the USAID Missions and partners in the field, who were very generous with their time and knowledge at a particularly busy time of year. Without their support, the work would have been seriously hindered. Finally, to those community members who shared their insights and experiences in working with children, we express our admiration and respect. You represent the safety net for these children in need. DeeDee Yates (Team Leader) Christopher O’Connell Nicky Davies Muyiwa Oladosu iv TRACK 1.0 OVC PROGRAM EVALUATION TRACK 1.0 OVC PROGRAM EVALUATION v ACRONYMS AIDS Acquired immune deficiency syndrome AM Activity Manager APS Annual program statement AVSI The Associazione Volontari per it Servizio Internazionale CA Cooperative agreement CAFO Church Alliance for Orphans CBCO Community-Based Care for Orphans and Vulnerable Children program CBOs Community-based organizations CCF Christian Children’s Fund COP Country Operational Plan COPE Community-based Orphan Care, Protection, and Empowerment program CRS Catholic Relief Services CSSG Community self-help savings group CTO Cognizant technical officer DHS Demographic and Health Survey ECD Early childhood development FBO Faith-based organization FHI Family Health International GH Tech Global Health Technical Assistance Project GFATM Global Funds for AIDS, TB and Malaria GS&L Group saving and loan HIV Human immunodeficiency virus HWW Hope World Wide (Renamed “Olive Leaf Foundation”) IRCU Inter-religious Council of Uganda ISD Implementation Support Division LOA Life of activity M&E Monitoring and evaluation MDG Millennium Development Goals NGO Non-governmental organization OGAC Office of the Global AIDS Coordinator OHA Office of HIV/AIDS OI Opportunity International OVC Orphans and other vulnerable children PCI Project Concern International PI Plan International PSS Psychosocial support RAAAP Rapid assessment, analysis, and action planning vi TRACK 1.0 OVC PROGRAM EVALUATION REPSSI Regional Psychosocial Support Initiative RFA Request for applications ROSI Regional OVC Organizational Support Initiative within Olive Leaf SCinUG Save the Children in Uganda SLA Savings and loan association SO Strategic objective TWG Technical Working Group UNAIDS United Nations Programme on HIV/AIDS UNICEF United Nations Children’s Fund USAID United States Agency for International Development USG United States Government VCT Voluntary counseling and testing WC World Concern WFP World Food Program WSN Weaving the Safety Net program TRACK 1.0 OVC PROGRAM EVALUATION vii CONTENTS ACKNOWLEDGEMENTS .................................................................................................................................................. iii ACRONYMS............................................................................................................................................................................ v EXECUTIVE SUMMARY......................................................................................................................................................ix BACKGROUND..............................................................................................................................................................ix FINDINGS ..........................................................................................................................................................................ix RECOMMENDATIONS ................................................................................................................................................xii SECTION 1. INTRODUCTION....................................................................................................................................... 1 BACKGROUND............................................................................................................................................................... 1 AWARDS............................................................................................................................................................................ 2 CURRENT STATUS......................................................................................................................................................... 3 SECTION 2: EVALUATION METHODOLOGY........................................................................................................... 5 SECTION 3: FINDINGS ....................................................................................................................................................... 9 ACHIEVEMENTS OF TRACK 1.0 OVC PROGRAMS............................................................................................ 9 MANAGEMENT ISSUES............................................................................................................................................... 16 TRANSITION TO ENSURE CONTINUITY OF SERVICES............................................................................... 22 PROGRAMMATIC CHALLENGES ........................................................................................................................... 24 SECTION 4: RECOMMENDATIONS............................................................................................................................ 31 FUTURE MECHANISMS .............................................................................................................................................. 31 TRANSITION AND CONTINUITY........................................................................................................................ 33 OVC PROGRAM STRATEGIES GENERALLY....................................................................................................... 35 CONCLUSION.............................................................................................................................................................. 37 APPENDICES APPENDIX A. SCOPE OF WORK................................................................................................................................. 39 APPENDIX B. PERSONS CONTACTED .................................................................................................................... 47 APPENDIX C. SUMMARY OF OVC TRACK 1.0 AWARDS.................................................................................. 53 APPENDIX D. COMPARISON OF SERVICES............................................................................................................ 57 APPENDIX E. PARTNER SUMMARIES......................................................................................................................... 63 APPENDIX F. REFERENCES ............................................................................................................................................ 93 FIGURES FIGURE 1: DISTRIBUTION OF TRACK 1.0 OVC AWARDS................................................................................... 3 FIGURE 2: DIRECT SERVICES TO OVC...................................................................................................................... 12 FIGURE 3: AGES OF OVC REACHED DIRECTLY BY TRACK 1.0 PROGRAMS ............................................ 12 FIGURE 4: BENEFICIARIES BY AGE COMPARED TO A PROPORTIONAL BREAKDOWN..................... 13 viii TRACK 1.0 OVC PROGRAM EVALUATION TABLES TABLE 1 ES. NUMBERS EVER SERVED BY A TRACK 1.0 OVC PROGRAM....................................................... x TABLE 1: SUMMARY OF ORIGINAL AWARDS.......................................................................................................... 2 TABLE 2: SUMMARY OF KEY INFORMANTS.............................................................................................................. 6 TABLE 3: ACHIEVEMENTS CITED BY SPECIFIC GROUPS...................................................................................... 9 TABLE 4: NUMBERS OF OVC EVER SERVED BY A TRACK 1.0 OVC PROGRAM....................................... 10 TABLE 5: OVC TRACK 1.0 BUDGETS AND TARGETS REACHED COMPARED TO COUNTRY OVC BUDGETS AND TARGETS REACHED......................................................................................... 10 TABLE 6: PARTNERSHIP MODELS ............................................................................................................................... 15 TABLE 7. PARTNER INTERNAL MANAGEMENT STRENGTHS AND WEAKNESSES............................... 18 TABLE 8: USG INTERNAL MANAGEMENT STRENGTHS AND WEAKNESSES........................................... 20 TABLE 9. USG/PARTNER RELATIONSHIP STRENGTHS AND WEAKNESSES.............................................. 21 TABLE 10. PARTNER PLANS FOR CONTINUITY.................................................................................................. 23 TABLE 11. TRANSITION TIMELINE PHASE: NOW TO JUNE 2010.................................................................. 33 TRACK 1.0 OVC PROGRAM EVALUATION ix EXECUTIVE SUMMARY BACKGROUND The OVC Track 1.0 program of the President’s Emergency Plan for AIDS Relief (Emergency Plan) is a centrally funded USAID initiative announced in November 2003 that aims in part to help rapidly scale up support to orphans and vulnerable children (OVC) affected by HIV/AIDS in designated Emergency Plan focus countries. Fifteen organizations have received awards under this program, which ends in 2010. In October 2008 USAID contracted with the Global Health Technical Assistance Project (GH Tech) to conduct an external evaluation “to ascertain the collective impact, strengths, and weaknesses of the Track 1.0 OVC portfolio.” The three evaluation objectives are to 1. Evaluate the achievements of the Track 1.0 OVC portfolio based on the collective and individual experience and accomplishments of the cooperative agreements. 2. Assess and document management of the OVC Track 1.0 portfolio and of the individual OVC cooperative agreements. 3. Identify and recommend strategies and priorities for the future direction of Emergency Plan OVC programming and mechanisms. This evaluation, undertaken by a team of four independent consultants, took place October– December 2008, with field work November 1–21, 2008, in Kenya and Uganda (Team 1) and Namibia and Zambia (Team 2). The teams visited 82 organizations or agencies; more than 300 people participated in interviews or meetings. The teams provided a debriefing to in￾country Missions, in-country partners, and representatives of the US government (USG) and partners in Washington. This document reports the team’s findings on achievements of the Track 1.0 OVC program, challenges for OVC programming, management of Track 1.0 OVC programs, and transition plans to ensure continuity of service. It contains recommendations for OVC programming, mechanisms for the future, and steps for the transition. FINDINGS Achievements • Of 141 • Sixty-nine percent of prime partners and 73% of subpartners interviewed in the four countries visited considered “increased awareness of the needs of OVC” to be a specific achievement of the portfolio. prime partners, 12 are on track to achieve or exceed their targets (with varied attention to sustainable service provision) despite challenges posed by the OVC Track 1.0 mechanism that resulted in more than 1.2 million beneficiaries receiving services as of March 31, 2008. 1 These 14 are all the prime partners awarded Track 1.0 OVC agreements except for Save the Children, whose agreement ended in February 2007; it was not included in the analysis of achievements, which was based on desk research and field visit interviews and observations. x TRACK 1.0 OVC PROGRAM EVALUATION • Moreover, 78% of prime partners and 50% of Mission staff considered capacity￾building of subpartners and implementing community groups (and sometimes prime partners themselves and local government groups) as an achievement. TABLE 1 ES. NUMBERS EVER SERVED BY A TRACK 1.0 OVC PROGRAM Prime Partner OVC Target, Start of Project to March 31, 2008* Actual Achieved: Start of Project to March 31, 2008* On Track to Achieve Targets by End of Project** Africare 137,500 181,914 Yes AVSI 11,136 12,522 Yes CARE 61,000 66,744 Yes Christian Aid 29,375 35,991 Yes CCF 46,600 43,757 No CRS 100,370 111,306 Yes FHI 77,5002 15,131 Yes Olive Leaf (formerly HWW) 140,085 85,464 No OI 48,103 47,963 Yes Plan International (PI) 131,645 139,163 Yes Project Concern International (PCI) 144,749 236,308 Yes Project Hope 75,000 39,987 Yes Salvation Army 57,551 57,016 Yes World Concern 150,500 132,326 Yes Total 2,205,592 * Reported by partners in semiannual reports for 2008. ** Reported by partners in interviews with the evaluation team. The targets set for OVC Track 1.0 programs seem to be higher, with a lower cost per child, than bilateral OVC programs. For instance, activity managers report that Track 1.0 represents • 29% of the Zambia OVC budget, to reach 40% of Zambia’s OVC target • 7.62% of Kenya’s OVC budget, to reach 15.73% of Kenya’s OVC target 2 FHI’s original targets, given here, were adjusted down in line with changes in FHI’s Scope of Work and the role of FHI subpartner, CAFO, so that their targets will be met. TRACK 1.0 OVC PROGRAM EVALUATION xi • 14.6% of Namibia’s OVC budget, to reach 9% of Namibia’s OVC target The evaluation team is of the opinion that the initial push for numbers of children reached resulted in some partners offering the services with the least cost attached or to the most accessible audiences. The relatively small amounts of money available for program implementation after management and other costs were deducted from the budget was an issue repeatedly raised by subpartners, though not primes. The Track 1.0 OVC programs represent an impressive network of local organizations with the awareness and skills needed to reach out to vulnerable children in the community. For example, one prime partner in Kenya works with 80 community groups, a subpartner in Namibia provides assistance to 180 local community projects, and another prime partner works through 81 local partners across Uganda, Rwanda, and Kenya. Management • Centrallydesigned and managed programs that lack significant Mission input may make it more difficult for the US government to pursue its aim of increasing host government input and responsibility. • Centrally funded programs add a number of layers to decision making and communicating, which makes problem-solving more complex. • The multicountry nature of the Track 1.0 programs engaged partner regional and headquarters offices in a unique way that encouraged cross-country learning and sharing within a partner’s organization. • Centrally funded mechanisms can alleviate some of the administrative burden on countries where the USAID Mission is small. Transition to Ensure Continuity • The significant progress and network of partners is at risk of being lost due to uncertainty and lack of clarity on transition from current agreements. Clear communication and signals are needed first from USAID/Washington and then from the Missions. • Partners in Washington and nationally within the countries are more confident about current strategies for continuing the programs than local community organizations. • All stakeholders are actively exploring possible avenues for continuation of services to the children in their care. Specific for Track 1.0 OVC • The interpretation of the Emergency Plan OVC Guidance and its communication to partners have not been systematic or consistent. This was magnified in Track 1.0 OVC programs. • Integration with country programs: xii TRACK 1.0 OVC PROGRAM EVALUATION – Geography: Track 1.0 OVC programs were designed based on assessed ability to move quickly to scale up and meet high targets rather than on Mission country strategy or government plans. – Partner selection: The partners awarded a Track 1.0 OVC cooperative agreement would not necessarily have been the partners chosen by a Mission, for a variety of reasons. This had both positive and negative implications. – Government: When the programs began, some Missions were not able to integrate the Track 1.0 OVC partners with the USG response that had been coordinated with the host government. – Linkages: In each country OVC partners try to play complementary roles and leverage resources, but there is still underutilized potential for linking prevention, care, and treatment programs with education and microfinance work within both a single partner and a given locality. General for all OVC Programs • A strict interpretation of the definition of an OVC within the OVC Guidance— especially the age of the child and the links to HIV—has in some cases undermined program strategies. • The vast majority of programs rely heavily on volunteer efforts. Overworking volunteers may threaten sustainability. More work is needed on identifying strengths of and opportunities for older children, young people, and guardians to engage more directly with programs. RECOMMENDATIONS Future Funding Mechanisms 1. Where requested by Missions, future OVC service delivery agreements should be competed, awarded, and managed bilaterally rather than through a central mechanism. 2. Administrative and technical management support for service delivery agreements will be necessary only when requested by Missions or for nonpresence countries. 3. Facilitation of learning and sharing across countries and partners and external technical assistance for Missions on request will be necessary to improve the quality of program interventions and management. Transition for Continuity 1. Countries must absorb the program assets of the Track 1.0 investment. 2. The USG should continue to provide funds to cover continuity of services to current beneficiaries, and the funding must be shifted from USAID/Washington to the field Missions and designated for such use: 3. USG headquarters should clearly explain to Mission offices each step in the flowchart to relieve uncertainty about program transitions and continuity. TRACK 1.0 OVC PROGRAM EVALUATION xiii Clarifications about next steps should be conveyed through multiple channels— phone calls, emails, and personal visits—as each case may require. 4. Mission offices (especially those with many Track 1.0 programs) need to be informed of what additional funding they might be able to get to continue services. 5. Communication between USG headquarters and Mission offices about transition should include detailed explanations of what Mission offices need to put in place and transition guidance on budgetary issues, i.e., whether monies will be set aside for continuity, and whether Missions or Washington will manage these monies. 6. USAID/Washington should have stakeholder meetings with the headquarters staff of prime partners to discuss aspects of transition plans that concern them. The Mission offices on their part should discuss transition plans with in-country partners (including government officials) and subpartners. General OVC Program Recommendations 1. Future OVC programs need to focus more on reinforcing systems at different levels: government, community, and household. 2. Targets should emphasize support for households served in order to minimize stigma; address children under five years and young people; support families as the primary givers of care and services; and minimize the need for volunteers while still allowing projects to reach large numbers of children. 3. Rather than using cost per child to judge a program’s efficiency, it might be more helpful to look at what percentage of funds reaches local partners as a capacity building activity, a subgrant, or direct material support. xiv TRACK 1.0 OVC PROGRAM EVALUATION TRACK 1.0 OVC PROGRAM EVALUATION 1 1. INTRODUCTION The Emergency Plan OVC Track 1.0 program is a centrally funded USAID initiative announced in November 2003 and ending in 2010 that aims in part to help scale up support to orphans and vulnerable children (OVC) affected by HIV/AIDS in the 15 Emergency Plan focus countries. Program awards were granted to 15 organizations. In November 2008 USAID contracted with the Global Health Technical Assistance Project (GH Tech) to conduct an external evaluation “to ascertain the collective impact, strengths and weaknesses of the Track 1.0 OVC portfolio.” The three evaluation objectives are to 1. Evaluate the achievements of the Track 1.0 OVC portfolio based on collective and individual experience and accomplishments of the cooperative agreements. 2. Assess and document the management of the OVC Track 1.0 portfolio and of the individual OVC cooperative agreements. 3. Identify and recommend strategies and priorities for the future direction of Emergency Plan OVC programming and mechanisms. This evaluation report is organized according to those objectives. Section 1 gives the background to the Track 1.0 OVC awards and Section 2 presents the methodology for the evaluation. Section 3 deals with the findings related to achievements, management, transition issues, and programming challenges. Section 4 presents recommendations for mechanisms for future awards, how the transition can be managed, and program strategies for OVC programs. BACKGROUND The main objectives of the OVC Track 1.0 program as presented in the Annual Program Statement (APS) solicitation first issued in November 2003 are to • Provide comprehensive and compassionate care to improve the quality of life for orphans and other vulnerable children, and • Strengthen and improve the quality of OVC programs through the implementation, evaluation, and replication of best practices. Additional objectives address U.S. government (USG) priorities of sustainability, capacity￾building, and institutional strengthening for public and private partners, including community and faith-based organizations (CBOs and FBOs) working in this vital area. The APS sought proposals to increase care and support to OVC and adolescents affected by HIV in two or more Emergency Plan focus countries. In the five years since the program began, the awardees have adapted their original programs to meet changing guidance and programmatic priorities and needs defined by both the Office of the Global AIDS Coordinator (OGAC) and the in-country teams. 2 TRACK 1.0 OVC PROGRAM EVALUATION AWARDS Between February 2004 and August 2005, USAID competitively awarded 15 cooperative agreements in response to an APS on Support to OVCs Affected by HIV/AIDS (APS-M￾OP-04-813). Awards ranged from three to five years, though most (12) were for five years. Table 1 presents the amounts awarded. (Appendix C contains a complete list of awards, funding, countries, and targets.) TABLE 1: SUMMARY OF ORIGINAL AWARDS Total Average Total amount $140 million $9.3 million per award Federal share $107 million $7.2 million per award Match $32 million 30% Years Between 2004–10 5 years each Countries 13 3 per award Targets 1,981,016* 50,795* per country Annual funding per country, per award $667,778 Track 1.0 OVC Projects In-Country 45** 3.5 Track 1.0 partners per country * Does not include Salvation Army or Opportunity International.3 3 Cooperative Agreement documents supplied to the evaluation team for Salvation Army and Opportunity International do not include targets. ** CRS added Botswana and AVSI added Côte d’Ivoire after the initial awards were made. Though the average number of awards per country is 3.5, the awards were not evenly distributed by country. Nine countries had three or fewer Track 1.0 partners, and four countries (Kenya, Zambia, Uganda and Mozambique) had over 50% of the individual Track 1.0 in-country projects. Vietnam was not included in the APS because it was not named a focus country until June 2004. No awards were made to programs in Guyana. TRACK 1.0 OVC PROGRAM EVALUATION 3 FIGURE 1: DISTRIBUTION OF TRACK 1.0 OVC AWARDS CURRENT STATUS One of the Track 1.0 programs, Save the Children, closed out in 2007. The rest were originally set to expire between February 2009 and August 2010. In an effort to coordinate the transition of these awards, 13 have been extended until June 2010. Family Health International (FHI) will keep its original August 2010 end-date. 4 TRACK 1.0 OVC PROGRAM EVALUATION TRACK 1.0 OVC PROGRAM EVALUATION 5 2. EVALUATION METHODOLOGY This evaluation used a participatory approach as the overarching methodology. It encompassed • a series of meetings with USG staff in Washington to identify areas of emphasis in the evaluation, which were then factored into the design of the instruments used • a review of background materials, including annual program statements, cooperative agreements, the 2006 OGAC OVC Guidance, semiannual and annual reports, and partner brochures (details in Appendix F) • key informant interviews face-to-face and by phone • site visits to community projects and activities supported by the program • briefing and debriefing with country Missions, partners, and subpartners • briefing and debriefing for USG and partner headquarters. The evaluation team consisted of four consultants whose expertise included program design, child and youth development, HIV/AIDS, project management, and monitoring and evaluation (M&E). The four were paired in two teams, each of which visited two countries selected from the 13 countries with Track 1.0 OVC programs. Team one visited Uganda and Kenya and team two Zambia and Namibia. The countries were chosen for convenience, funds available for the evaluation, and availability of key Mission staff, and because they had not been involved in the Track 1.0 Abstinence, Being Faithful Prevention for Youth program evaluation. The evaluation was designed to ensure that all 14 prime partners (and key or representative subpartners) within the multicountry OVC Track 1.0 portfolio were visited at least once. Informants were selected based on their different roles in Track 1.0 mechanism, design, and implementation: coordinators from USG headquarters or Missions, program implementers in-country, or policy makers in the participating country’s government. A total of 317 informants were interviewed from 82 different agencies or institutions (see the summary in Table 2 and detailed descriptions in Appendix B). 6 TRACK 1.0 OVC PROGRAM EVALUATION TABLE 2: SUMMARY OF KEY INFORMANTS Type Description # of Org. # of Ind. USG Washington Cognizant Technical Officers (CTOs), Office of the Global AIDS Coordinator (OGAC), OVC Technical Working Group (TWG), Others 1 11 RHAP Regional OVC Senior Advisor; Human Capacity Development Advisor Southern Africa 1 2 Country Missions Activity Managers/OVC Advisors, Team Leaders, M&E Officers, Emergency Plan Coordinators, and other key personnel (South Africa Mission by phone and Haiti by questionnaire) 6 17 Government Personnel Uganda: Ministry of Gender, Labour and Community Development Kenya: OVC Secretariat, Ministry of Gender and Children Affairs Zambia: Ministry of Sport, Youth, and Child Development Namibia: Chairperson of National AIDS Council, Zambia; Ministry of Gender Equality & Children Welfare 5 11 Partner Headquarters Hope WW, PCI, World Concern, CRS, FHI, Project HOPE, CCF, CARE, Salvation Army, AVSI, Africare, PI, OI 13 18 Partners In-country Hope WW; PCI; CRS; FHI Namibia, FHI Zambia, Project HOPE, Christian Aid, CCF, CARE International, Salvation Army, AVSI, Africare, PI (World Concern & Opportunity International do not have in-country offices in Zambia) 13 114 Subpartners CRWRC, CETZAM, Habitat, Diocese of Solwezi, Diocese of Mongu, ECR, Nazarene Church Mission Zambia, Bwafwano, Church Alliance for Orphans (CAFO), Positive Vibes, BIDII, K-REP Microfinance, PACT, Inter-Religious Council of Uganda, Save the Children in Uganda, Pathfinder, Kenya 15 48 Community schools/teachers groups/vocational centers/early childhood development (ECD) Church committee/religious leaders Branch offices Savings and loan associations/Village health banks Community-based organizations Subtotal 9 3 2 4 10 26 21 26 4 31 14 96 Total 82 317 The three main objectives of the evaluation were the basis for design of a logical framework to guide data collection and analysis. Indicators for each objective were identified and means of verification ascertained. Questionnaires were then designed to capture information on the qualitative aspects of the indicators; other (mostly quantitative) indicators were verified through document review. Four sets of questionnaires were drafted to elicit information on knowledge and understanding of the purpose of the Track 1.0 OVC portfolio, major achievements and best TRACK 1.0 OVC PROGRAM EVALUATION 7 practices, technical issues with implementation, fit of Track 1.0 with other OVC programming, and wrap-around and linkages with other Track 1.0 programs. The questionnaires also asked about Track 1.0 management, mostly with respect to strengths and weaknesses in the relationship between USAID/Washington, Mission offices, and partners and local recipients of funds. Specific management issues probed on the centrally funded mechanism were M&E, research, financial management, supervision and coordination, communication, human capacity development, and planning. Other topics addressed in the instruments were capacity building and continuity of services to the OVC served, and recommendations for future strategies and programming. The content of the four questionnaires was generally the same except for areas that needed emphasis based on the target audience. The data collected were summarized in Microsoft Excel. Analysis included textual classifications of the responses and highlighting of emerging themes. Three to four themes were identified as major findings in each area of the evaluation. The themes are interpreted and reported as major findings in this report. The limitations of the methodology have implications for generalization of the findings. The countries selected were not based on the characteristics of USAID in-country missions. For example, some countries have many Track 1.0 partners, others only a few. Countries with fewer Track 1.0 partners may produce different results than those with many. Of the four countries evaluated, three had six or more partners, but overall nine of the 13 Track 1.0 OVC countries have four or fewer programs. The countries selected were all in Eastern or Southern Africa. An implication of focusing on fewer countries is that there will be more country-specific than general conclusions about the entire Track 1.0 OVC mechanism. Also, while the team tried to review as many documents as possible, some day-to-day process documents that might have provided vital information may have been inadvertently omitted. In some cases the team found it hard to attribute to Track 1.0 OVC programs innovations or strategies used by partners and subpartners due to their engagement with other programs. Preliminary findings were presented to USAID field offices and in-country partners and subpartners. More complete evaluation results were presented to the Emergency Plan, USAID Washington, and partner headquarters staff. The evaluators circulated a full draft report to USAID and partner annexes to the relevant partners. Their comments and suggestions inform this report. 8 TRACK 1.0 OVC PROGRAM EVALUATION TRACK 1.0 OVC PROGRAM EVALUATION 9 3. FINDINGS The findings are presented in four categories: achievements, management, transition, and programmatic challenges. ACHIEVEMENTS OF TRACK 1.0 OVC PROGRAMS Key Achievements • Of the 144 • Sixty-nine percent of prime partners and 73% of subpartners interviewed in the four countries visited volunteered “increased awareness of the needs of OVC” as a specific achievement of the portfolio. prime partners, 12 are on track to achieve or exceed their targets (with varied attention to sustainable service provision), despite those challenges posed by the OVC Track 1.0 mechanism. Just over 1.2 million beneficiaries had received services through March 31, 2008. • Moreover, 78% of prime partners and 50% of Mission staff mentioned capacity building of subpartners and implementing community groups (and in some cases prime partners themselves and local government groups) as an achievement. These three were the achievements most often identified by those interviewed. Other achievements that were expressed less consistently but were significant to particular subgroups interviewed are summarized in Table 3. TABLE 3: ACHIEVEMENTS CITED BY SPECIFIC GROUPS Interviewee Subgroup Achievements of the OVC Track 1.0 Portfolio CTOs and Mission staff • Introduced Missions to new OVC non-governmental organization (NGO) partners (particularly international NGOs) and program models that have now been tested and constitute an asset to the Missions • Allowed a scale-up in OVC programming in countries where Missions have limited staff capacity Prime partners and subpartners • Allowed organizations that did not have relationships with Missions or capacity to mobilize resources at the country level to compete for OVC Track 1.0 funding through their headquarters office • Made it easier for OVC to access a more comprehensive range of services either directly or indirectly through program efforts • Made possible sharing of program learning and problem-solving across countries with prime partners, through the regional aspect of program designs, and between OVC Track 1.0 partners, particularly at the headquarters level 4 These 14 prime partners constitute all those awarded Track 1.0 agreements except for Save the Children, whose agreement ended in February 2007 and which was not included in the analysis of achievements, which was drawn from desk research and field visit interviews and observations. 10 TRACK 1.0 OVC PROGRAM EVALUATION Meeting Targets The achievement almost unanimously identified in interviews was that OVC Track 1.0 partners were likely to meet the high targets set for them. The total target for OVC reached for 13 of the 15 agreements is 1,981,016, 5 TABLE 4: NUMBERS OF OVC EVER SERVED BY A TRACK 1.0 OVC PROGRAM an average of 50,795 per country (see Table 4). OVC Target: Start of project to March 31, 2008* Actual OVC Served: Start of Project to March 31, 2008* On Track to Achieve Total Targets ** Africare 137,500 181,914 Yes AVSI 11,136 12,522 Yes CARE 61,000 66,744 Yes Christian Aid 29,375 35,991 Yes CCF 46,600 43,757 No CRS 100,370 111,306 Yes FHI 77,5006 15,131 Yes Olive Leaf (formerly HWW) 140,085 85,464 No OI 48,103 47,963 Yes PI 131,645 139,163 Yes PCI 144,749 236,308 Yes Project Hope 75,000 39,987 Yes Salvation Army 57,551 57,016 Yes World Concern 150,500 132,326 Yes Total 2,205,592 * Reported by partners in the Semiannual Reports for 2008. ** Reported by partners in interviews with the evaluation team. The targets set for OVC Track 1.0 programs seem to be higher (with a lower cost per child) than bilateral OVC programs (see Table 5). TABLE 5: OVC TRACK 1.0 BUDGETS AND TARGETS REACHED COMPARED TO COUNTRY OVC BUDGETS AND TARGETS REACHED Country Track 1.0 OVC Budget as % of Total Country OVC Budget Track 1.0 OVC Targets Reached as % of Total Country OVC Targets Reached Kenya 7.6% 15.7% Namibia 14.6% 9% Uganda Awaiting information Awaiting information Zambia 29% 40% Source: 2008 information from Missions 5 Excluding Opportunity International and Salvation Army 6 FHI’s original targets, given here, were adjusted down in line with changes in FHI’s Scope of Work and the role of FHI’s subpartner, CAFO, which means that their targets will be met. TRACK 1.0 OVC PROGRAM EVALUATION 11 While one partner’s reason for getting off track on meeting the target had to do mainly with internal issues, the other partner reported that it had to adjust the targets downward because of changes in program design to ensure more longer-term, comprehensive service delivery in line with their interpretation of the OVC Programming Guidance issued by the Emergency Plan in July 2006. Although reaching high numbers of OVC will have improved the lives of these children and their caregivers, the initial emphasis in the Emergency Plan agenda of 2-7-107 7 “2-7-10” refers to the goals of the President’s Emergency Plan in the focus countries to prevent 7 million new HIV infections, treat at least 2 million HIV-infected people, and care for 10 million HIV-affected individuals and AIDS OVC. on quantity rather than quality of programs, and a lack of rigor in reviewing and reinforcing initial program design, has meant that some partners paid inadequate attention to quality, sustainability, capacity building, and strengthening of public/private partnerships (other program objectives outlined in the APS). For example, lack of attention to parity across ages (see Figures 2 to 4) has resulted in a focus on primary school children at the cost of children under 5 and sometimes of children over 12. Lack of attention to quality and sustainability is now having repercussion not only for the transition and longevity of services that USAID is currently providing through OVC Track 1.0, but also for community empowerment and ownership of OVC challenges. These limitations are unlikely to be limited to OVC Track 1.0 programming. Figure 2 presents the services received by OVC through all OVC Track 1.0 partners in each of the four countries visited. Figure 2 shows the age distribution of children who have been directly reached through all Track 1.0 partners in the four countries visited. In all cases psychosocial support (PSS) is the most frequent service, followed by health care (except in Zambia) (see Figure 2). It is the evaluation team’s opinion that because of the initial push for numbers of children reached some partners offered the services with the least cost or services to the most accessible audiences. For example, addressing primary school learners with life skills (health) or a self-esteem building program (PSS) is a relatively cost-effective way to reach a large number of children, rather than the more expensive options associated with secondary school education or early childhood development. Figures 3 and 4 show that more beneficiaries from certain age groups, particularly those above age 5, are being reached with services than would be expected from a purely statistical proportional breakdown. 12 TRACK 1.0 OVC PROGRAM EVALUATION Figure 2: Direct Services to OVC Figure 3: Ages of OVC Reached Directly by Track 1.0 Programs TRACK 1.0 OVC PROGRAM EVALUATION 13 Figure 4: Beneficiaries by Age Compared to a Proportional Breakdown Although the Emergency Plan is tracking provision of particular services, the data are difficult to analyze because in many countries there are no minimum standards for service delivery. 8 Thirteen prime partners listed two or more of the following challenges posed by the OVC Track 1.0 mechanism as hindering their potential to achieve their targets The original suggested APS indicators included quality measures; these were later amended to comply with OGAC-determined indicators, which were (i) number of OVC reached and (ii) caregivers trained. Although quality indicators were not reintroduced, quality of services was addressed to some extent in the OVC Guidance and emphasized in the Quality Improvement initiative in participating countries. However, the evaluation team’s impression is that often quality was lost in favor of quantitative indicators. 9 • Initial and continuing delays in work plan approval delayed implementation. : • Late and imbalanced funding obligations made long-term planning difficult, caused tension between prime partners and subpartners, and required some primes to redraft annual work plans. • There were conflicting interpretations of the OVC Guidance and whether primes should be implementing services not available to wrap-around, and there was no formal process to ensure that primes and mission staff interpreted the Guidance as intended. • The problem-solving structure is weak because OVC Track 1.0 management (both for USAID and prime partners) and power is separated from the problems faced in- 8 See Appendix D for a table of different inputs that partners are counting for different services. 9 Although some of these challenges are not unique to the OVC Track 1.0 portfolio, these points were raised in the context of the central funding mechanism exacerbating these challenges due to weak management – many of these issues are explored in more detail in Section 3.3. 14 TRACK 1.0 OVC PROGRAM EVALUATION country, such as creation of efficient M&E tools, linkages to wrap-around services, and resolving problem between primes and subpartners. • Additional staff time is needed to report to both USAID Washington and local Missions, which use different reporting guidelines, as is volunteer time to collect monitoring data to match reporting guidelines within the OVC Guidance. For instance, Salvation Army-trained teachers and home-based care volunteers must now regularly monitor 57,000 OVC, which is affecting their time and motivation to provide PSS. The relatively small amount of money available for program implementation after management and other costs were deducted was an issue repeatedly raised by subpartners, though less often by primes. Headquarters involvement has been beneficial in most cases; however, where international NGOs and their affiliates are subpartners, the result seemed to be multiple overhead charges or expensive operational costs being extracted before funds reach communities. The evaluation team heard from a number of partners and sub-partners about the cost per child, and subpartners often calculated their coverage and activities based on such a formula. Local community organizations may receive as little as US$20 per child to deliver services. Local subpartners expressed difficulty in determining how to spread their limited resources given the huge apparent needs. This is especially true for educational expenses, such as uniforms, fees and supplies. Block grants to schools, assistance to community schools, and links to government support mechanisms are all strategies partners have used to address the high demand and high cost of education in a relatively sustainable way, however, continued funding is still necessary to support such high-cost services. Increased Awareness of the Needs The impressive network of community organizations established or assisted by the Track 1.0 OVC program are the primary reason for this achievement. Nine prime partners reported using the best practice resource Journey of Life (developed by the Regional Psychosocial Support Initiative [REPSSI]) to raise community awareness of the needs of children. The high number of CBOs supported through Track 1.0 OVC programs resulted in considerable reach and impact. For example, CCF works through 80 community groups in Kenya, FHI’s subpartner CAFO in Namibiaprovides assistance to 180 community projects, and AVSI works through 81 local partners in Uganda, Rwanda, and Kenya. Track 1.0 OVC partners have also worked on national policy development. Three of the six partners in Zambia and all the partners in Namibia supported their relevant ministries as they drafted OVC standards and guidance. Subpartner Capacity Building The APS specified “sustainability, capacity-building and institutional strengthening across public and private sector partners, including community and faith-based organizations that are working in this vital area” as an important program objective. Although an achievement for many partners, capacity building has been a weak area for some due to their original program design. OVC Track 1.0 primes seem to have taken three types of approaches to program design (see Table 6). The different approaches have a significant impact on a prime’s ability to build the capacity of local partners for the longevity and continuity of services beyond the life of the program. TRACK 1.0 OVC PROGRAM EVALUATION 15 TABLE 6: PARTNERSHIP MODELS Program Design Approach Prime Partners Working solely through local affiliates or international NGOs who are subpartners, with little or no direct in￾country presence Opportunity International, World Concern Working through indigenous intermediary NGOs, that are responsible for building the capacity of local CBOs Christian Aid, FHI, CRS, PCI Working directly with a large number of small community responses (with a relatively large field office presence) PCI, CARE, Africare, AVSI, PI, Salvation Army, CCF, Olive Leaf Foundation *Some partners operate in more than one way. Evidence of capacity building of subpartners or local implementing partners was observed for 12 out of 14 partners. Working through a local indigenous intermediary NGO is a model typically used by international NGOs that concentrate on building capacity to deliver services rather than delivering services directly. Among the advantages and disadvantages, a key advantage is that by investing in an established indigenous intermediary NGO, the prime can focus on building its capacity in terms of quality of new and current program interventions, M&E and follow-up, and organizational development, including planning and resource mobilization. This intermediary NGO can then support CBOs and communities in future through support from other donors or the host government. This allows the prime to both fulfill its program objectives and leave a viable vehicle in place for continued or expanded service delivery. For this model to be effective, the intermediary organization must be capable of providing quality mentoring and training to smaller local organizations. Working directly with a large number of small community entities requires intensive engagement by field office staff in building the capacity of CBOs to access resources and support future service delivery through the local government, wrap-around services, etc., once the program ends. CCF, which works with 80 community organizations in Kenya, is using its relatively large staff to actively build the capacity of these CBOs to mobilize resources and integrate their programs into local government structures and work plans. However, CCF informally estimates that only 40% to 50% of them will be able to sustain services after the program ends. Although they are well linked to government structures, the human and financial resources available are too limited to support future service delivery. Much like Africare in Uganda, CCF has to intensify its work on capacity building to try and ensure continuity of services. Without a local intermediary organization, it must rely on them to mobilize resources directly, which is often unrealistic. Clear examples of intentional capacity building of local government structures were observed in 8 out of 14 partners (such as support for local government planning, strengthening of multisectoral committees run by local government officials, and advocacy for local policy implementation). This was an intentional sustainability strategy some prime partners used to complement their program design approach and development philosophy. However, some missions said that bilateral OVC programs are more systematically building local government capacity (e.g., through APHIA II in Kenya). Among the primes, 58% mentioned that their own capacity had been built through OVC Track 1.0 programs. Specific aspects of increased capacity that were observed included 16 TRACK 1.0 OVC PROGRAM EVALUATION sounder processes for OVC selection and monitoring (all primes to a varying degree) and more established in-country program management. Generally, although capacity has clearly been built in the vast majority of programs, it seems that in most cases there was no clear capacity development plan from the start, capacity assessments were rare, and there has been little acknowledgement of existing capacity on the ground and the achievements and challenges relating to this, e.g., less staff turnover and better local knowledge and resource mobilization. MANAGEMENT ISSUES Key Findings • Centrally designed and managed programs that lack significant Mission input may make it more difficult for the USG to pursue its aim of increasing host government input and responsibility. • Centrally funded programs add a number of layers to decision making and communicating, which makes problem solving more complex. • The multicountry nature of the Track 1.0 programs engaged partner regional and headquarter offices in a unique way that encouraged greater cross-country learning and sharing within a partner’s organization. • Centrally funded mechanisms can alleviate some of the administrative burden on countries where the USG presence is small or nonexistent. Because it is a centrally funded program managed by USAID, solicitation, selection, and management of OVC Track 1.0 awards are coordinated by USAID/Washington, where both the CTO and the Agreement Officer reside. The programs are coordinated in-country through Emergency Plan country teams, who assign activity managers (AMs) to act as technical liaisons. All but one partner received multicountry awards, with each covering an average of three countries. This evaluation reviewed both the administrative management of Track 1.0 awards and the technical management and direction given to the partners. In most cases, the same individuals and the same communication processes were used for both administrative and technical management. As a result, from a management and communications perspective, some challenges applied to both types of issues. The evaluation reviewed technical and administrative management of the Track 1.0 awards at three different levels: headquarters (Washington or other international offices outside the region), regional, and in-country offices. It also reviewed partner internal management, USG internal management, and partner/USG interactions at all three levels. Host Government Engagement: The intention of centralizing design and management of Track 1.0 OVC programs may have been to facilitate rapid scale-up of programming. However, this structure exists largely outside the processes that in-country USG teams use to engage host governments in taking the lead in planning and coordinating a national response. At the beginning of the Track 1.0 programs, many host government institutions and plans were not in place or did not have the capacity to coordinate the response. However, the USG teams TRACK 1.0 OVC PROGRAM EVALUATION 17 actively engaged host governments, and in many cases assisted efforts to increase the host government role in planning and coordinating the OVC response. Though host governments may not themselves design or manage individual partner interventions, programs designed and managed outside this structure may complicate the in-country USG team’s efforts to increase the role of the host government. Levels of Management Headquarters Headquarters were primarily involved with administrative management, including financial management, contracts, and compliance. The primary day-to-day people engaged were the USAID CTO in Washington and partner headquarters staff. These offices also had a key role in planning and reporting, but partners managed planning in a variety of ways. Some headquarters offices took the lead in planning and reporting, but with other partners country offices had that responsibility while headquarters staff compiled the documents and provided feedback and comments. Finally, headquarters staff was also responsible for troubleshooting management issues and providing program management backstopping to field offices. Regional Offices Some partners reported that their regional offices had an increased role in Track 1.0 because the programs were multicountry, though how and to what degree these offices were engaged varied between partners. Of the partners that used regional offices, some took on many of the responsibilities that might have otherwise been managed by an international headquarters office, such as coordinating planning and reporting. Some regional managers doubled as a country technical officer. In a few cases, partners attempted to manage programs from a regional office, but this proved challenging, and they eventually chose to hire in-country managers. Perhaps the most successful role of regional offices was to provide technical support and program management backstopping to in-country offices. USAID also has a regional structure that is sometimes used to provide administrative and technical management support. However, the involvement of the USAID regional mission in the Track 1.0 program varied depending on the country. Lack of involvement may have been in part due to staff and budget pressures on these offices. For example, the position of Regional OVC Advisor in the Southern African Regional USAID Mission was not created until January 2008, three or four years after the program began, and was never part of the original management strategy for the Track 1.0 program. Country-Level Roles Each country was asked to assign a Track 1.0 activity manager, who was primarily responsible for day-to-day program oversight, working with the partners in country to put program design, plans and OVC Guidance into the local context, handling M&E, and managing staff and financial resources on the ground. Both USG and partner in-country staff worked to engage the host government, USG staff primarily with national officials and partners with local officials. 18 TRACK 1.0 OVC PROGRAM EVALUATION Partner Internal Management Partners used a variety of models to implement their programs, with notable variations in the use of regional offices and in how subpartner and community groups were engaged, with resultant strengths and weaknesses (see Table 7). For example, for some partners regional offices were primarily technical liaisons, while others used them for more direct program management. The different models used to engage community groups, such as direct engagement vs. using an intermediary indigenous NGO, will have long-term impact on capacity building and sustainability. TABLE 7. PARTNER INTERNAL MANAGEMENT STRENGTHS AND WEAKNESSES Strengths Weaknesses • Cross-country learning and sharing of staff closely involved in program implementation • Technical and program management backstopping from regional and headquarters • Cost savings/economies of scale allowed hiring of more qualified individuals at regional level to support multiple programs. • Additional overhead costs • Monitoring data and analysis not always flowing back to implementers • Projects and partnerships designed at headquarters often difficult to tailor at country level • Multilayered approach could slow communication Partner Learning and Sharing • Some partners considered internal learning and sharing across countries to be a unique strength of the Track 1.0 program. for example: − One partner described an exchange visit in which its staff noted how a sister program in another country had been engaging local government officials in a way that improved the program’s linkages to other services in the area. Upon return, the partner increased its own engagement with local officials. During the evaluation visit, a local official praised this partner for its efforts at engaging the local government and linking with other service providers—a direct result of the exchange visit. − One partner shared program innovations, including a particularly original approach to its economic empowerment model and M&E database, with each of its country programs. • Several partners also cited learning and sharing between partners at the headquarters level as a unique strength of this program. Reportedly, USAID facilitated the initial interactions through OVC Track 1.0 partners’ meetings, but the partners themselves continued to collaborate regularly without USG facilitation. • Learning and sharing between partners at the country level seemed to be a missed opportunity in some countries, where it was observed that partners had been working independently on similar interventions or M&E solutions, but had not shared their experiences. The extent of interaction at the country level seemed to depend upon how active the USG Mission was in coordinating these activities. Some TRACK 1.0 OVC PROGRAM EVALUATION 19 partners suggested that it would be easier to get together if the USG Mission coordinated the effort, perhaps due to the competitive climate within the country. Regional/Headquarters Technical Assistance and Support • Partners that structured their program team to include technical support from the regional or headquarters office found this assistance to be beneficial to in-country staff in terms of facilitating cross-country dialogue, backstopping, and clarifications of official guidance. For example: − One country program reported that having regionally based technical assistance was useful because it helped consolidate experiences from other country programs and provided regular updates on OVC technical information, including promising practices and changes to the OVC Guidance. − One partner noted that it had received regular technical advice, research, and best practice documents from its headquarters, which was participating in monthly OVC-related forums in Washington and passing information on to the country office. − One partner noted that its regional technical person participated in monthly OGAC-sponsored OVC TWG conference calls, so it was able to get updates on various OVC-related issues, such as the child status index, and engage country offices in discussing these issues. Another partner noted that its regional officer was a conduit for experience-sharing between countries. Regional Cost Savings/Economies of Scale • Some partners suggested that the multicountry structure of their Track 1.0 programs resulted in cost savings through economies of scale. For example: − Two partners noted that they were able to hire more experienced people as regional program managers who provided technical support and program management backstopping to their country programs. − One partner hired a country director who doubled as a regional technical advisor, thus saving costs. USG Internal Management Management processes and procedures at each level within the USG are very structured and well-defined. However, the communication lines and division of responsibilities between different levels—e.g., between USAID/Washington and the Missions—are not as well defined. Thus, it was largely left to the individual USAID/Washington Track 1.0 CTOs to try to build relationships with activity managers at each Mission and establish roles and responsibilities. The nature of this structure inherently provided some management challenges, as did realities on the ground, such as high staff turnover, tremendous workloads, and limited funds for travel and in-country staffing. 20 TRACK 1.0 OVC PROGRAM EVALUATION TABLE 8: USG INTERNAL MANAGEMENT STRENGTHS AND WEAKNESSES Strengths Weaknesses • Administrative management burden alleviated from missions • Unclear lines of communication and unclear division of roles and responsibilities between CTOs and activity managers • High number of agreements managed by each CTO (and in some cases AMs) • Lack of funding for countries to pay for staff to provide support to partners CTO/AM Communication and Coordination One set of challenges that was both reported and observed at both the headquarters and in￾country offices was communication and coordination between CTOs and AMs. While responsibilities had been delineated on paper, the real challenge may be the structure itself. The CTO’s role carries with it a legal obligation to ensure that USG-funded programs within their purview are meeting the conditions of the award. Therefore, while CTOs can seek advice and assistance from AMs, ultimately it is up to them to ensure that legal obligations are met. The AM, however, does not report to the CTO but up the chain of command in the Mission. This structure inherently has the potential to put the CTO and the AM in difficult situations, and appears to have caused some problems early in the implementation of some Track 1.0 programs. High staff turnover of AMs also contributed to the challenges. Other USG Internal Management Issues Many of other USG internal management issues observed are not necessarily unique to the Track 1.0 program. For example: • The process for development, interpretation, and implementation of guidance: Individuals at all levels saw the value and importance of guidance, but there were different levels of understanding about both the content and the function of these documents. Over time, it seems, efforts have been made to improve communications about these documents, but there was some confusion as Track 1.0 was being implemented. • Country and funding agency reporting: Each Emergency Plan country has independently developed its own processes for gathering reports and data from partners. For partners operating in several countries, this can be a challenge. For example, the M&E system that one partner had in place in one country was hard to implement in another because the reporting requirements were different. • Administrative support for limited and nonpresence countries: Though the team visited only countries that had USAID Missions, the evaluation explored what impact centrally funded mechanisms might have on limited or nonpresence countries through interviews with USAID regional and field staff that have had experience with programs in such countries. The general impression they had was that centrally funded programs provided valuable management support to programs in countries with limited or no USAID presence. TRACK 1.0 OVC PROGRAM EVALUATION 21 USG/Partner Relationship In-Country The most significant differences observed among the countries visited stemmed from how each Mission chose to engage Track 1.0 partners. Because some Missions did not have the staff or time to engage with partners, they let them run with little direct oversight. Other Missions engaged the partners early and went through a process of reworking program designs to better fit country priorities. Countries with fewer Track 1.0 partners reported less difficulty folding these partners into their existing plans, but for those with more partners it took more effort to align. In general, partners that were engaged and folded into country programs early seem to have had a much slower start-up, but as the awards come to an end, they seem to be in a better position to be transitioned to bilateral mechanisms. Some partners that were not engaged early with Missions report being able to start up quickly. A few partners embraced this opportunity to implement and innovate quickly. However, they often later had to try to adjust programs mid-course once the Missions were staffed up and able and willing to become more involved. Programs that have still not been folded into a Mission’s strategy may face more difficulties in securing future funding because the Missions simply do not have mechanisms or staff in place to fund and manage stand￾alone activities. Headquarters Level The interactions between USAID/Washington CTOs and their counterparts in partner organizations alleviated administrative management burdens from the field. In addition, the interactions between the OGAC-sponsored OVC Technical Working Group and the partners allowed partners to discuss issues related to OVC activities. Partner field offices report this interaction helped them improve programs and better understand issues related to the OVC Guidance. TABLE 9. USG/PARTNER RELATIONSHIP STRENGTHS AND WEAKNESSES Strengths Weaknesses • Annual meetings allowed for learning and sharing. • Having a USG point person to interact with headquarters and country offices can help engage the partner at different levels. • People making decisions are separated from the problems. • In some countries partners did not collaborate or share experiences with each other. • There was duplication in M&E. • It was difficult for the Mission to ensure compliance (for instance with VAT exemption and M&E quality). Decision Makers Separated from Problems A centralized management structure has inherent challenges resulting from the fact that the people with the responsibility for making decisions are not on the ground. This affected 22 TRACK 1.0 OVC PROGRAM EVALUATION Track 1.0 programs primarily by small administrative delays. The communications process on both partner and USG sides, for example, would often take a day or two to secure approvals. While each issue alone might not have been a problem, the cumulative effect on the field staff was often a feeling of disempowerment. One extreme scenario was observed where a particular partner had several different management issues arise all at once. While this would be considered difficult to deal with under any circumstance, having the partner, subpartner, and USG decisionmakers all outside the country may have prolonged the time before the parties could reach a satisfactory resolution. Monitoring and Evaluation • Two partners were observed to have independently built solid M&E systems in the same country, but had never shared them with each other. • One Mission M&E staff person said there was no time to work with Track 1.0 partners because the bilateral partners were working within a very specific framework and it would take considerable effort to bring the Track 1.0 partners in line. • Partners were required to submit performance reports both to the in-country Missions and to the CTOs in Washington. The CTOs created reporting guidelines that were intended to ease the challenge of providing separate country reports. The partners were instructed to simply attach the country reports and provide a summary with aggregated data. In at least one case, however, the country’s web-based reporting tools were not conducive to developing a report that could easily be shared with the CTO, so the partner had to create a separate report for each country. TRANSITION TO ENSURE CONTINUITY OF SERVICES Key Findings • The significant progress and network of partners is at risk of being lost due to uncertainty and lack of clarity about the future. Clear communication and signals are needed first from USAID/Washington and second from the Missions. • Partners in Washington and nationally within the countries are more confident about the current strategies for continuity of services than local community organizations. • All stakeholders are actively engaged in exploring avenues for continuation of services to the children in their care. Continuity of services is a crucial aspect of OVC programming, especially at it relates to vulnerable children who are at important development and educational stages in their life. Continuity of services is defined here as current plans for follow-on made by stakeholders, including USG head office and missions, partners, and local organizations to ensure that services provided to OVCs and their families are continued when Track 1.0 OVC programs close. If this is to happen, a smooth transition is needed at the close of the Track 1.0 OVC program in June 2010. A critical aspect of continuity is ensuring that the requisite funding will be available to serve those in need irrespective of implementing partners. TRACK 1.0 OVC PROGRAM EVALUATION 23 At the USG level (in both headquarters and Missions), continuity means determining whether future funds are available for transition and continuity, and communicating with implementing partners on next steps. At the partner level (both at headquarters and in￾country), continuity implies leveraging resources, sourcing other funds, and building the capacity of local partners to be independent. For local partners, continuity means having the resources they need, human and financial, to continue to offer services. Current State of Transition Understanding of next steps and what Missions and partners must do to ensure continuity of services varies widely. USAID/Washington has communicated to Missions and partners that they need to begin planning for transition but has given no clear guidanceon what exactly that planning should include, or what, if any, funding might be available in the future. While almost all Missions and partners are doing something, none of the activities are coordinated, and there is a lack of clarity about the transition responsibilities of USAID Mission and headquarters staff. As noted in the achievements section, significant assets have been built up at the community level over the past four years. A number of stakeholders in the USG, partners, and communities are concerned about the relatively short time until the Track 1.0 program ends in June 2010 in which to complete the steps necessary to ensure a smooth transition and continuity of services. Lack of clarity about next steps is leading some partner staff to leave their positions, which will make the transition even more difficult. Further, the lack of clarity is leading some Mission staff to propose simply letting Track 1.0 run out, without putting effort into ensuring continuity of services to the beneficiaries the program has been serving. Plans for Continuity The evaluation team asked partners and USG staff at all levels about their plans to continue services to children after the end of the Track 1.0 OVC award. TABLE 10. PARTNER PLANS FOR CONTINUITY Plans to Ensure Continuity Partner Headquarters (n = 11) Partner In-Country (n = 12) Local Partner (n = 15) Accessing other funding 73% 92% 53.3% Relying on income￾generating activity 91% 75% 33% Using the capacity of local structures 91% 58% 67% Exploring support from Mission 55% 42% 6% Note: n = Total number of interviews conducted for partners and local partners. An important theme derived from the data is the improved ability of local partners to mobilize funds from other sources (see Table 10). The majority of key informants at prime partner headquarters (73%) reported having other sources of funding or making plans to do 24 TRACK 1.0 OVC PROGRAM EVALUATION so, and a higher proportion of their in-country offices reported the same (92%). However, only 53% of subpartners in-country reported having other sources of funding or having plans to do so. Participation of beneficiaries in income-generating and economic empowerment activities is another theme drawn from the data. This has important long-term implications for the ability of OVC and their families to continue to access services on their own. The majority of key informants at prime partners’ headquarters (91%) reported that beneficiaries were engaged in such activities or intend to do so, as did prime partners in-country (75%) and subpartners (20%). The types of income-generating and economic empowerment activities reported for beneficiaries included small-scale gardening, animal husbandry, credit and loan schemes, group savings, and internal lending programs. Another key indicator of continuity of services is the existence of local community structure and agencies to sustain provision of services. Most headquarters prime partner key informants (91%) stated that local community structures are now in place, as did in-country prime partner (58%) and subpartner (67%) informants. However, the responses from prime partners at all levels may be reflecting intentions rather than actuality since they are not as close to the situation on the ground as subpartners. The majority of the local partners in all four countries visited perceived that there were community structures for providing sustainable services in their area. They cited national task forces, OVC forums, linkages to local and municipal government offices, and village OVC committees and groups. The evaluation found that Mission offices were already providing some form of support to local partners in terms of (1) identifying local partners that are doing well, (2) involving more partners in Country Operation Plans (COPs), and (3) making plans for graduation mechanisms and linkages to absorb OVCs and their families currently served into other programs, e.g., through issuing a new APS, folding OVC Track 1.0 programs into other Mission programs (comprehensive models or OVC specific programs), and linking subpartners to other funding mechanisms supported by the mission. The partner responses may reflect a lack of clarity and communication about these plans. Of the four Missions visited, three were already involved in some form of support to at least some of the OVC Track 1.0 partners directly to continue to provide services. The general impression in all four countries visited is that Mission offices want more control of funds in order to continue reaching OVC currently being served and in some cases scale up program interventions. The Missions favor mechanisms rooted in buy-in or bilateral agreements where they retain decision-making power on agreements with partners in their countries. PROGRAMMATIC CHALLENGES Issues presented in this section are not confined to Track 1.0 OVC programs or partners. They cut across many USAID/Emergency Plan programs that support OVC. Given the commonality of these concerns across the wide range of Track OVC partners surveyed in this evaluation, the points below may merit attention in future OVC programming. TRACK 1.0 OVC PROGRAM EVALUATION 25 Findings Specific for Track 1.0 OVC • The interpretation of the OVC Guidance and its communication to partners was not systematic or consistent. This was magnified in Track 1.0 programs. • Integration with country programs: − Geography: Track 1.0 OVC programs were designed based on assessed ability to move quickly to scale up and meet high targets, rather than on Mission country strategy or government plans. − Partner selection: For a variety of reasons the partners awarded a Track 1.0 agreement would not necessarily have been the partners the Mission would have chosen. This had both positive and negative implications. − Government: When Track 1.0 OVC began, some Missions were not able to integrate the partners with the USG response that had been coordinated with the host government. − Linkages: OVC partners in country try to play complementary roles and leverage resources, but too little attention has been given to prevention for vulnerable children and to fully exploiting potential links with education and microfinance programs. In some cases there is underutilized potential for linking prevention, care, and treatment programs within both a single partner and a given locality. Interpretation of the OVC Guidance The Emergency Plan OVC Guidance was provided to Track 1.0 partners well into the second year of the program period. Especially at the outset it was, in some cases, used as an edict, not as guidance. Understanding and implementation of the guidance varied across countries. The Missions interpreted and contextualized it for national partners as best they could. Track 1.0 OVC partners were sometimes included in this and at other times depended on information from their head office about its interpretation and implementation. Track 1.0 OVC partners reported trying to offer or at least to monitor more services than had been in their original cooperative agreement in order to count each child as a direct primary recipient. The Guidance reporting guidelines demanded a much more robust monitoring system to distinguish between primary and supplementary beneficiaries. Partners were able to meet the challenge, but the potential value of the monitoring systems has not been fully realized. The Guidance encouraged partners to look beyond Track 1.0 OVC targets and to focus more on outcomes. However, due to the lack of minimum quality standards in some countries and to varying interpretations, the guidelines were often applied mechanically with more attention still on counting than on quality. The aggregation of the data on essential services may hide more than it reveals, and the pressure to count services may overshadow the importance of outcomes (see Appendix D). Integration within the Country Geography: Three Missions mentioned that the geographic spread of Track 1.0 OVC partners was not ideal. According to the interviews, partners based their decisions on where to work on where they already had a partner or program in order to facilitate a quick start-up to meet targets. Many partners chose to work near the main transport routes because 26 TRACK 1.0 OVC PROGRAM EVALUATION working in isolated, remote areas has substantial transportation costs and may have made it hard to reach the targets set. In the majority of cases, though not all, this did allow for a reasonably quick and smooth start-up; however, it did not take into sufficient consideration the Missions’ own strategic plans or bilateral agreements or the geographical gaps identified by the host government. In Zambia the mission asked one Track 1.0 OVC partner not to operate in a particular area because it duplicated work by another OVC program. The Kenyan mission asked two partners (CARE and CCF) to modify the districts in which they operate and work closely with existing partners in those districts. Partners: The partners that were awarded Track 1.0 OVC cooperative agreements did not necessarily reflect the choice of the missions. Some of the Track 1.0 OVC partners had little or no presence in-country. In two instances Missions reported having difficulty even finding the partner. In other instances the partners were already engaged with the Mission. In a number of cases the choice of subpartners also proved problematic for Missions because the subpartners were already engaged with the Mission through other mechanisms, and Missions prefer to consolidate their interaction with a particular organization. Two Missions commented, however, that they were exposed to some new and strong partners through Track 1.0. Government: At first some Missions were not able to coordinate the USG response to the host government because of the plethora of independent Track 1.0 partners. In formulating their programs the majority of Track 1.0 OVC partners did not appear to have consulted the national plan of action on children or HIV strategic plans where they existed, though partners did mention choosing countries based on HIV prevalence rates and the numbers of orphans in that country. In Kenya, unlike other bilateral OVC partners, Track 1.0 partners had not been trained to report through the government system. One host government official explained that in his country the USAID bilateral program provided a more cohesive and comprehensive HIV/AIDS program response that was well linked to the government. Recently more and more Track 1.0 OVC partners are participating at district or regional level in OVC Forums or District AIDS Committees, through which they provide data to government. At the national level coordination of the response for OVC requires more technical assistance and support. Program Linkages: In the majority of instances, partners—both international and local— are involved in other HIV/AIDS-related activities, such as palliative care, prevention messaging, voluntary counseling and testing (VCT), and treatment adherence. The same partners that are Track 1.0 OVC partners may be subpartners in another USAID agreement. The forums or platforms created by the Missions had the potential to bring OVC partners together and provided opportunities for them to locate complementary services, especially education, food and nutrition, economic strengthening, and housing. There is some underutilized potential for linking prevention, care, and treatment programs within both a single partner and a given locality; the ability of partners to integrate prevention and other services into their OVC work varies. Just as the OVC Guidance emphasized a holistic approach to a child, the Emergency Plan may need to consider a holistic approach to its partners, so that partners can visualize their work as part of a unified program rather than as separate pieces along the prevention, care, treatment, and impact mitigation spectrum. It is possible that such an approach may be TRACK 1.0 OVC PROGRAM EVALUATION 27 hampered by the need to submit a request for exceptions any time one partner receives more than 7% of the portfolio. General for all Emergency Plan-Funded OVC Program • A strict interpretation of the definition of an OVC from the OVC Guidance— especially the age of the child and the links to HIV—has in some cases undermined program strategies. • The vast majority of programs rely heavily on volunteer efforts. Overworking volunteers may threaten sustainability. More work is needed on identifying strengths and opportunities for older children, young people, and guardians to engage more directly with programs. Targets and Definitions Although partners recognize the importance of working with the whole family, and many interventions target a household—for example, provision of a home or involvement in a microfinance/savings and loan association—the emphasis is still on counting a child served. The team thinks this is due to the emphasis on meeting targets of number of children served compounded by the need to count particular services. The focus on the family and community, very clear in the OVC Guidance, was not translated into clear strategies or targets in Track 1.0. The team also encountered some anecdotal evidence that particular children in households were being singled out for services, such as school fees or clothes or food. Such an approach can easily lead to stigmatization of a child within the household. The definition of an OVC—especially the age—in the OVC Guidance proved problematic for many partners. Nine out of 14 partners at country level cited the OVC definition, especially the age limit, as a concern, as did 8 of 15 subpartners. Some partners and some missions were more flexible and innovative than others in addressing the age restriction on who qualifies as a child. The definition, in line with country definitions and the Convention on the Rights of the Child, states that a child is someone under the age of 18; however, organizations felt that the definition restricted their ability to respond to the needs of families and households. All the organizations in Namibia and Zambia said they were not allowed to provide services to children over 18. Many organizations did manage to source other support for these children, especially to complete their schooling, to attend vocational training, or to run their households. It is understood from the interviews that some Missions allow partners to work with vulnerable youth who are over 18 but count them as caregivers. In Kenya partners appeared more likely to do this or to overlook the age of the child. The interpretation of vulnerability due to HIV in the countries visited was not considered a priority problem by most partners. This may be because all the countries visited were high￾prevalence counties with generalized epidemics, or because by the time the evaluation was undertaken in November 2008, the Missions had constructed and communicated working guidelines for this. 28 TRACK 1.0 OVC PROGRAM EVALUATION Volunteers All projects work with community groups, such as schools, savings and loan groups, church committees, or volunteer child care organizations. The groups usually receive training on the psychosocial needs of children, financial management, and how to identify vulnerable children within the community. These groups are generally volunteer-led and managed and are often a positive, active force for children in their communities. Though both the APS and the OVC Guidance mention young people and older children as agents for change in the communities, there was little evidence that they participate in these groups. The team believes more emphasis is needed on the role of parents, guardians, and older youth in a household. The community groups rely on trained volunteers to visit children and households. A consistent theme in interviews with local partners and subpartners was the problem of volunteer attrition. Possible reasons are the pull of incentives from other projects and the push of burn-out due to excessive demands. In a number of instances volunteers were said to be serving up to 30 households through home visits, although the organizations stated that they would ideally like to see volunteers serving only six to eight households. Volunteers were also expected to undertake a multitude of tasks for a large number of children. Programs still rely excessively on volunteer labor to deliver and monitor services. The need to count children has been devolved to volunteers, who are now responsible not only for providing compassionate care but also for monitoring exactly what that care entails. Volunteers must now note which services are given to a child, for example, a prayer (PSS); help with homework (education); a talk on abstinence (health and prevention). This undermines the whole point of the OVC Guidance, which calls for holistic comprehensive care. Many volunteers are also the prime data capturers and with increasing demands for more detailed monitoring are being asked to record minutiae. Volunteer attrition threatens the sustainability of many programs. Promising Practices The evaluation team identified potentially promising practices in each country—innovative responses to some of the technical challenges identified. Promising practices have been shared at the yearly Track 1.0 Partner Meetings held in Washington DC and at the regional meetings of some partners, but it is not clear that best practices had been shared within a country with other OVC program. Not all the successes described below can be attributed the Track 1.0 program alone, however; many of the partners received support and technical assistance from other agreements. Subgranting A number of partners make subgrants to local NGOs, which in turn subgrant to communities. This practice pushes decision making and authority down to the appropriate level: Communities are best placed to identify the most vulnerable children and families and to allocate resources accordingly. The intermediary partner thus empowered to make subgrants is trained in the necessary financial management and accountability skills. The local community partner also builds skills in managing finances, reporting, and monitoring. This has been shown to provide a good basis for future fundraising and thus a solid foundation for the continuation of services to children. Some local partners visited make subgrants ranging from US$50 to US$60,000. TRACK 1.0 OVC PROGRAM EVALUATION 29 Economic Strengthening Adapting the CARE model, Christian Aid uses savings and loan associations (SLAs) as a sustainable source of funding to meet the needs of the OVC they care for and as an entry point to provide services to other OVC in the community. The SLAs are made up of OVC caregivers, including older orphans who are heads of households. They are organized, trained, and monitored by indigenous subpartners. In addition to economic strengthening, the SLAs have other tasks, such as conducting regular OVC household monitoring visits, and managing food and nutrition self-help projects funded and implemented by SLA members. The SLAs also help identify the neediest OVC in the community and provide the necessary resources to allow them to access education. Over time, these SLA caregivers have proven to be the most motivated members of local OVC committees, which are otherwise made up of community leaders, often men. In Namibia Project HOPE has married a similar microlending program using solidarity groups with health and parenting education, a further positive innovation. A number of other partners are now using similar approaches. School-Based Educational Support A number of partners provide block grants to a school instead of individual bursaries to children. This supports improved education for all vulnerable children in a community. A block grant is a resource exchange strategy in which a school receives a cash transfer in exchange for exempting OVC from school fees. The grant can be used for infrastructure improvement or buying textbooks or other learning aids that benefit all the children in the school. School personnel are also trained in proposal writing and PSS for OVC. A study by Africare/Uganda showed there were substantial savings in using a block grant rather than a tuition payment scheme. In one school, to enroll 100 children would have cost US$100,000, while the same children were accommodated for a block grant of US$2,000. The strategy also helps reduce stigma by not targeting an HIV-affected child orphan directly, while increasing teachers’ knowledge of HIV/AIDS and the psychosocial needs of affected children. Other programs have also effectively used general support to a school rather than tuition for specific children. In two of the countries visited programs helped community schools to reach the standards required for government accreditation. In the process OVC who would otherwise not be attending school were accommodated in accredited schools. The One-Stop Shop Small organizations initiated and led by the community can grow into well-resourced and mature institutions to support OVC. Bwafwano in Zambia, a Track 1.0 OVC subpartner of PCI and a subpartner within the RAPIDS program, offers a number of services to the children in its catchment area, among them early childhood development programs, schooling, counseling skills, visits at the attached clinic, VCT, and legal protection. All the services are offered on site, some being provided by Bwafwano and some by other organizations. In all, 1,300 OVC have been able to access services of various types. Between 60% and 70% access medical care and support, including deworming, immunization, etc.). Bwafwano has provided counseling to over 150 children aged 6–16. Through the PCI/BELONG program, between October 2007 and November 2008, over 7,000 new OVC enrolled with Bwafwano. The organization has increased in geographical scope and set up many sites in three main catchment areas. Bwafwano is now mentoring FBOs and CBOs. 30 TRACK 1.0 OVC PROGRAM EVALUATION TRACK 1.0 OVC PROGRAM EVALUATION 31 4. RECOMMENDATIONS Mirroring the findings, the recommendations are categorized into recommendations for OVC programming, for management mechanisms, and for the transition. FUTURE MECHANISMS OVC Service Delivery 1. Future OVC service delivery agreements should be competed, awarded, and managed bilaterally where requested by a Mission, rather than through a central mechanism. Though certain administrative burdens were alleviated by the central mechanisms, the benefits did not outweigh the problems inherent in centrally designed and managed programs. Therefore, where in-country Missions determine that they have the administrative and technical management capacity to award all their OVC service delivery programs themselves, it would be more efficient to use bilateral mechanisms. However, a benefit of the Track 1.0 program was the cross-country learning and sharing that was facilitated within partners. This learning and sharing was facilitated in part because of the unique way in which the partner headquarters and regional offices were engaged in these multicountry programs. To preserve this important element, Missions are encouraged to allow partners to include specific measurable cross-country learning and sharing activities as part of their program expenses. Partners must then demonstrate the value of these exchanges, and the Mission must hold partners to these commitments. 2. Administrative and technical management support for service delivery agreements should be necessary only when requested by Missions or for nonpresence countries. The Emergency Plan II is likely to have a much wider geographical scope, and not every country that will need OVC programs will have the management capacity to make such awards. Therefore, USAID/Washington’s Implementation Support Division (ISD), in consultation with OGAC, is encouraged to coordinate the development of future mechanisms to facilitate service delivery in those countries that seek to use a central mechanism. In addition, future central mechanisms should engage Missions more proactively at every stage from APS design and partner selection through implementation, and Missions should have ultimate approval of agreements that are made. To coordinate this effort the ISD may need to undertake a process to estimate future OVC services needs in Emergency Plan II countries by overlaying Mission locations, regional Mission coverage, Mission workload, and a country’s OVC service needs. This process will identify gaps in service delivery that the ISD can use in drafting develop a strategy to provide administrative support either through central or regional programs. These mechanisms should still allow for country-level technical design, direction, and management, while attempting to alleviate the administrative burden. 32 TRACK 1.0 OVC PROGRAM EVALUATION The ISD in USAID/GH/OHA should consult regional USAID Missions and in-country Mission staff about the potential role of regional missions in facilitating and managing OVC service delivery for countries in their region that require support. If they are to provide support for OVC service delivery, regional Missions need to be adequately resourced in terms of both human resource capacity and funding. Learning and Sharing and Technical Assistance 1. Facilitation of learning and sharing between countries and partners and provision of technical assistance to Missions on request will be necessary to improve the quality of program interventions and management. The USG may need to estimate future needs of OVC learning and sharing and technical assistance by surveying Missions and estimating needs in nonpresence countries. The USG should develop a mechanism for facilitating OVC learning and sharing and provision of technical assistance for country and region missions to buy into. The scope of work for this mechanism might be • OVC cross-country and regional learning and sharing of promising practices and program management approaches; activities might include − managing regional operations research projects − organizing and facilitating regional lessons-sharing events (for partners and Missions) − organizing and facilitating cross-country exchange visits (for partners and Missions) − conducting country-specific or regional OVC evaluations • Provision of technical assistance to Missions (country and regional) to support their own OVC technical needs or those of their NGO/FBO or government partners; activities might include: − providing technical input to strengthen and align Mission, host government, and NGO/FBO partner OVC M&E tools and processes The USG may also want to consider mechanisms to help facilitate cross-country learning and sharing between Missions and country programs. Unfortunately, neither of the two groups (the OVC TWG and regional USAID Missions) that are in a position to help facilitate cross-country learning and sharing has the budget, staff, or mandate to do so. A possible solution may be found by consulting the OVC TWG, regional USAID Missions, and in-country staff about potential strategies for facilitating learning and sharing and providing technical assistance to in-country teams that require support. For this to be successful, the effort will require a budget and staff. It may be easier and more cost-effective for regional Missions to achieve this either through buying into an established mechanism as described or issuing a region-specific mechanism for OVC learning and sharing and technical assistance. TRACK 1.0 OVC PROGRAM EVALUATION 33 However, if regional Missions are to help facilitate OVC learning and sharing and technical assistance, they need to be adequately resourced in terms of both human resource capacity and funding. TRANSITION AND CONTINUITY 1: Countries must absorb program assets of the Track 1.0 investment. Track 1.0 programs have built significant assets, among them community organizations and programs that are currently delivering services to OVC. Many of the community structures built through Track 1.0 programs may require reinforcement or assistance in coordination with government. Further analysis is needed to outline what program assets (i.e. community groups, people trained, and specific OVC projects) align with in-country programming priorities and might benefit from continued funding. This analysis can then be compared to other in-country funding and program options to give a clear picture of where continuing support might best be applied. If countries are to successfully absorb these assets, a number of decisions and actions need to take place in the next 18 months. Coordination and communication among all the stakeholders will be critical throughout. The evaluation team recommends that the ISD at USAID/Washington take a lead role in coordinating the transition, although Missions must be responsible for making all decisions about future OVC programming in their countries. The ISD team is well positioned to ensure that communication with the field and partners is timely. It can also provide technical support to Missions as needed on design of follow-on mechanisms. Table 11 lays out steps in the transition over the next 18 months. ISD can use a similar plan to help clarify responsibilities and keep the transition on track. TABLE 11. TRANSITION TIMELINE PHASE: NOW TO JUNE 2010 Tasks Estimated Timeline Key Outcomes USAID and OGAC Future Funding Decision Beginning February 2009 • Begin discussing future funding decisions with OGAC. • Formulate Transition Guidance for Missions on funding for transition. USAID/Washington Communication March 2009 • Communicate transition strategy, including Transition Guidance on funding. • Clarify responsibilities. Mission Decisions March–May 2009 • Missions review current programs and make decisions about future programming. • Missions communicate decisions to USAID/Washington. 34 TRACK 1.0 OVC PROGRAM EVALUATION TABLE 11. TRANSITION TIMELINE PHASE: NOW TO JUNE 2010 Tasks Estimated Timeline Key Outcomes Partner Coordination June 2009 • USAID/Washington, partners, and activity managers begin coordinating the transition program by program, based on Mission programming decisions. • Partners with projects being phased out are asked to stop reaching out to new beneficiaries, and estimate time and cost to phase out current beneficiaries. Mission Follow-On Solicitations June 2009–March 2010 • Missions that need to issue solicitations (APS or RFA, for example) to fund follow￾on programming have about nine months to design, issue, and make awards. • USAID ISD could provide technical assistance to missions if necessary to help design the APS solicitations. Handover March 2010–June 2010 • If any programs are awarded to new partners, a three-month window is available for new partner start-up and handover from current partners before the award ends. 2: The USG should continue to provide funds to cover continuity of services to current beneficiaries, and the funding must be shifted from USAID/Washington to field Missions and designated for OVC use. If country Missions are asked to fund services to current Track 1.0 beneficiaries through their budgets, some programs may need to be cut—either Track 1.0 or other programs. If funds are shifted into the general Emergency Plan budgets for each country and not specifically designated to fund continuity of services to Track 1.0 beneficiaries, there is a risk that the funds could be used to fund other elements of the country programs. Therefore, without continued commitment of funds at or near current levels, the USG will likely have to make decisions about cutting services. Many of the services being provided are having a profound impact on communities and represent a valuable investment that can continue to pay dividends if support is continued until the programs reach maturity. Therefore, the USG should continue to provide funds to cover continuity of services to current beneficiaries, funding should be moved to Mission budgets beginning in June 2010, and those funds should be designated to cover current services. Funds should be used to cover services, but not necessarily through the same prime partners. The time and funding commitments necessary to begin any USG procurement and competitive process require swift action if continuity of services is to be preserved. To do so, it is critical that USAID/Washington work with OGAC to draft a policy in time for the 2010 Country Operational Plans, because Missions and partners need to know what whether the USG intends to fund these activities before next steps can be taken in transition TRACK 1.0 OVC PROGRAM EVALUATION 35 planning. However, if future funding decisions cannot be made promptly, the ISD should draft Transition Guidance to help the Missions begin their planning. To ensure continued services, • USG headquarters should clearly explain to Mission offices each step in the flowchart to relieve uncertainty about program transitions and continuity. Clarifications about next steps should be conveyed through multiple channels—such as phone calls, emails, and personal visits—each case may require. • Mission offices, especially those with many Track 1.0 programs), need to know if service continuation will be funded, so that they can define and fine-tune strategies open to them to reach OVC who are currently being served. Communication between USG headquarters and Mission offices about transition should include detailed explanations on what Mission offices need to put in place, and Transition Guidance on budgetary issues, such as whether monies will be set aside for continuity, and whether Missions or Washington will manage them. • USAID/Washington should have stakeholders meeting with headquarters staff of prime partners to discuss how transition plans concern them. On their part, Mission offices should discuss transition plans with in-country partners, including government officials, and subpartners. OVC PROGRAM STRATEGIES GENERALLY The commonality of many of the challenges observed during the evaluation has led to these recommendations that apply equally to Track 1.0 and other OVC programs. 1. Future programs in support of OVC need to emphasize reinforcement of systems at different levels. Government There are gaps in the training of government personnel to provide much-needed leadership and coordination for the national response on OVC. Relevant government ministries and agencies need better platforms for coordination at all levels—national, regional, state/provincial, and local. The majority of informants interviewed at government entities mentioned too few skilled staff and attrition of the few available as a major challenge in performing their role effectively. A more robust emphasis is needed on building up institutions like schools, religious institutions, government departments responsible for child welfare, national and district AIDS committees, and regional forums. This would include policy and human capacity development plans for management staff and for staff that interact directly with children, such as early childhood, health care, social, church, and youth group workers as well as teachers. Community Programs should expand their focus on awareness and community capacity, emphasizing the direct involvement of parents, guardians, young people, and children. Community capacity should be defined at the local level, indicators agreed upon, and regular assessments done. Issues to be covered include: 36 TRACK 1.0 OVC PROGRAM EVALUATION • Child protection: identifying abuse • Legal protection: will writing, inheritance, monitoring of status of recently orphaned children • Access to key documents: birth certificates, death certificates • Access to government services: education, health, legal; exemptions; social assistance grants Households OVC are cared for in families. The households may be headed by a single parent, a grandparent, or a youth. New approaches to reaching households are needed. An approach to caring for OVC that focuses on the guardians, including single parents (the concept of the single orphan is unhelpful, further marginalizing the role of the surviving parent), elderly members of the extended family, and young people would relieve the need for so many volunteers, so many home visits, and so much transport. Training guardians, the primary caregivers, may well have the most impact on the life of a child because the guardian has more contact with the child than any volunteer ever could. The work of the volunteer that is still essential should be done by local paid staff. Additional work with schools, early childhood development centers, and other institutions would also alleviate the need for a huge cadre of volunteer workers. A package of support for households based on an assessment of what they require to continue to care adequately for all the children for whom they are responsible is necessary. Such an approach, with more direct training and support to parents, grandparents, guardians, and young people from within or outside the household as appropriate will alleviate the need for a host of volunteers. Using paid staff instead of relying solely on volunteers, and when volunteers are used, ensuring that the demands placed on them are realistic, could be very productive. 2. An indicator is needed that emphasizes the households served. This will minimize stigma; address the children under 5 and young people in the household; support families as the primary givers of care and services; minimize the need for volunteers; and still allow projects to reach large numbers of children. Rather than using a cost-per-child formula to judge a program’s efficiency, it might be more helpful to look at what percentage of funds reaches local partners either through capacity building, as a subgrant, or as direct material support. For the Emergency Plan, targets drive a program. The high targets of Track 1.0 OVC programs determined where partners would work, with whom they would work, what sort of services they could offer, and how they would evaluate a service. Unfortunately the OVC Guidance fed into this syndrome, adding a level of detail (services) that partners then started to chase. The very sound principles and examples in the Guidance may have been somewhat lost in the deluge of information on monitoring primary or supplementary services. Targets should be set in consultation with local partners and government. TRACK 1.0 OVC PROGRAM EVALUATION 37 Using a cost-per-child formula is possible but the number of caveats and subcalculations required mitigate its meaningfulness, may quash innovation, and ultimately may not reflect the situation well enough to provide useful information.Caveats include the number of standardized services being provided, rural or urban location (concentration of households with OVC), cost and quality of other programming activities, and the amount of additional resources leveraged for the same activities. A change in emphasis from numbers of children served to numbers of households reached will obviate some of the problems cause by the current definition of an orphan and vulnerable child. Young people in the household are also in need of services, and programs must at least ensure that a child or young person will be able to complete the educational cycle. More focus on older children and young people will fit in well with an emphasis on prevention and may already be accommodated in the proposed definition. Active engagement of young people in addressing their own situation helps both the household and the young people by building confidence and community connectivity. Future programs should use behavior change communication for young people and children, which should receive more attention in any future programs. Any definitions should be closely aligned to national definitions of OVC even if they may not include an HIV clause, to avoid stigma while ensuring that HIV-affected children are not marginalized. CONCLUSION In the four years since the inception of the Track 1.0 OVC programs, the 15 partner organizations have built an impressive set of programs that have significantly contributed to the Emergency Plan’s response by bringing much-needed OVC services to hundreds of communities affected by the HIV/AIDS pandemic. These programs have raised awareness of the needs of OVC and improved the ability of caregivers and communities to give them care and support. The programs represent a significant asset, but they still need support to truly become sustainable. As the Emergency Plan II evolves from an emergency response to a more sustainable, locally driven approach, so too must the Track 1.0 OVC programs. It is the evaluation team’s recommendation that USG offices in Washington and in the field work together to transition these programs to local control while retaining the strengths of the programs. The team further encourages partners and the USG to learn from the wealth of program experiences of Track 1.0 partners to improve OVC programs everywhere. By learning from these experiences to improve future programming and funding mechanisms, the USG will help enable its host government partners to reach their goals to provide comprehensive and compassionate care to orphans and other vulnerable children affected by the HIV/AIDS epidemic. 38 TRACK 1.0 OVC PROGRAM EVALUATION TRACK 1.0 OVC PROGRAM EVALUATION 39 APPENDIX A. SCOPE OF WORK I. PURPOSE This request sets forth guidelines for an external evaluation of the PEPFAR Track 1.0 Orphans and Vulnerable Children (OVC) portfolio of programs to inform the USAID Office of HIV/AIDS (OHA) and the Office of the Global AIDS Coordinator (OGAC) on future priorities for OVC programs and mechanisms. The primary focus of the evaluation is to ascertain the collective impact, strengths, and weaknesses of the Track 1.0 OVC portfolio. While this is not an evaluation of individual Track 1.0 OVC implementing partners, it is anticipated that the evaluation team will draw from the breadth and depth of the various Track 1.0 OVC partner experiences and achievement of results. As such the evaluation will focus on the following objectives: 1. Evaluate the achievements of the Track 1.0 OVC portfolio based on the collective and individual experience and accomplishments of the Track 1.0 cooperative agreements. 2. Assess and document the management of the OVC Track 1.0 portfolio and of the OVC cooperative agreements individually. 3. Identify and recommend strategies and priorities for the future direction of PEPFAR OVC programming and mechanisms. It is expected that the evaluation will begin in early autumn 2008 and be completed by the end of 2008. II. BACKGROUND: THE PEPFAR TRACK 1.0 OVC PROGRAMS The goals of the President’s Emergency Plan for AIDS Relief (PEPFAR) include care for 10 million HIV/AIDS affected individuals, including orphans and vulnerable children (OVC). In November 2003 and again in March 2004, USAID issued an Annual Program Statement (APS) “To Provide Support to Orphans and Vulnerable Children Affected by HIV/AIDS” to expand and strengthen care and support efforts under PEPFAR. Specifically, it asked for proposals to increase care and support to orphans and other vulnerable children (OVC) and adolescents affected by HIV in two or more of the focus countries under the Presidential Initiative. Fifteen cooperative agreements (CA) were awarded that implemented programs in 14 of the 15 PEPFAR focus countries. The key objectives of these OVC Track 1.0 programs were: 1. To provide comprehensive and compassionate care to improve the quality of life for orphans and other vulnerable children. 2. To strengthen and improve the quality of OVC programs through the implementation, evaluation, and replication of best practices in the area of OVC programming. Additional important program sub-objectives address U.S. government priorities of sustainability, capacity-building, and institutional strengthening across public and private sector partners, including community and faith-based organizations that are working in this vital area. 40 TRACK 1.0 OVC PROGRAM EVALUATION The APS notes the following approaches to addressing strategic areas of OVC programming: • Strengthening the coping capacity of families • Mobilizing and strengthening community-based responses • Increasing the capacity of children and young people to meet their own needs • Ensuring that governments develop appropriate policies, including legal and programmatic frameworks, as well as essential services, including basic social services, for the most vulnerable children • Raising awareness within societies to create an environment that enables support for children affected by HIV/AIDS • Developing, evaluating, disseminating, and applying best practices and state-of-the art knowledge in the area of quality OVC programming • Comprehensive programming and linkages with other HIV/AIDS program areas • Fostering strong partnerships with local in-country organizations • Creating public-private alliances The OVC Track 1.0 cooperative agreements are scheduled to end between mid-fiscal year 2009 and the end of fiscal year 2010. To account for funds spent and to inform future PEPFAR OVC programming, there is a need to understand the strengths and weaknesses of the overall portfolio and individual agreements, as well as the suitability of the managerial mechanisms employed. III. EVALUATION OBJECTIVES AND ILLUSTRATIVE QUESTIONS The evaluation team will assess the progress made to date by the Track 1.0 OVC programs in achieving the specific objectives of their cooperative agreements and review the programmatic and technical strengths and weaknesses of the Track 1.0 OVC program portfolio with special attention to recommendations of strategies and priorities for future PEPFAR OVC programs and mechanisms. Evaluation objectives and illustrative questions are as follows. During the team planning meeting, the evaluation team will define and prioritize questions from the scope of work in accordance with assignment objectives. 1. Evaluate the achievements of the OVC Track 1.0 portfolio based on the collective experience and individual strengths and weaknesses of the Track 1.0 agreements. A. Summary Achievement of Results • What did the portfolio collectively achieve as far as outputs and outcomes for children in quantitative and qualitative terms? • What were the individual contributions of the different Track 1.0 agreements in quantitative and qualitative terms? TRACK 1.0 OVC PROGRAM EVALUATION 41 • What were the main successes and challenges, if any, to accomplishment of the planned results? Please describe. • Overall, were the issues facing OVC correctly identified, and were the interventions implemented appropriate responses to these? • How have plans for accomplishing planned outcomes changed during the life of the projects? Why? • What are the overall strengths/weaknesses of a targeted OVC portfolio? • What were the results, if any, of the OVC partner’s own internal and external evaluations? B. Assess and document the overall Track 1.0 portfolio (as well as highlights of the individual Track 1.0 projects) in terms of the following programmatic aspects: • Targeting, e.g.: Did the targeting respond to trends in the epidemic and to specific vulnerable groups? How well were gender differentials taken into account? Did programs respond to the needs of children at different ages (from 0–17)? Are partners and sub-partners working in geographic areas considered appropriate by USAID Missions? • Program approach and interventions, e.g.: Were the programs adaptive to local circumstances? Were program interventions appropriate to identified target audiences? Were family-centered approaches used whenever possible? Were there any significant gaps in programming? Were the interventions based on standards of good practice? • Participation, e.g.: Did partners at all levels successfully involve local communities and beneficiaries (including children) in design and implementation? What, if any, best practices were employed to involve local communities and beneficiaries? • Integration and wrap-around, e.g.: Did partners integrate their Track 1.0 activities with other field activities (including and especially HIV-related and MCH services) and the PEPFAR country-level OVC portfolio? How have partners combined Track 1.0 resources with wrap-around funding? What were the strengths/weaknesses of USAID support globally and in country Missions in terms of enabling wrap-around? • Sustainability and continuity of care, e.g.: What are USAID Missions currently planning for follow-on or transition of these Track 1.0 OVC programs? What will these Track 1.0 OVC programs leave behind, especially in terms of local partner capacity￾building and sustainability? What has resulted in terms of continuity of care for program beneficiaries in cases where Track 1.0 OVC programs have already ended? 2. Assess and document the successes and shortcomings of management by USAID headquarters CTOs, USAID Mission activity managers, and OVC partners and sub-partners. • What were the strengths/weaknesses in terms of communication and reporting, timeliness of deliverables, supervision, monitoring and evaluation, financial and 42 TRACK 1.0 OVC PROGRAM EVALUATION procurement systems, including sub-granting, and, human resources, including human capacity development? 3. Identify and recommend strategies and priorities for the future direction of OVC programming, for example: • How can future programming best ensure continuity of services to OVC and their families? • What project activities or accomplishments have led to implementation of best practices? Describe those best practices. • What recommendations for technical approaches and strategies may be best applied in follow-on programming? • What management strategy and mechanisms are recommended to address future partner interactions with USAID Missions (including inclusion in Mission portfolio and activities) and USAID/OHA? E.g.: What USAID management issues need to be considered moving forward? What are the lessons learned for centrally funded programs and how these programs are or are not responsive to field needs? IV. METHODOLOGY The evaluation team is expected to propose a detailed work plan for collecting the necessary information and data. This should include a description of how the work plan responds to the above tasks and questions, and from whom and how the data will be collected and analyzed. The work plan should be collaborative and participatory, including plans for conducting interviews with implementing partners and key stakeholders at both the local and national levels. The plan should also include a full review of background materials provided, such as the Annual Program Statement, cooperative agreements, and semiannual and annual performance reports. In order to examine the above issues, the following methodology is suggested to be considered. 1. Team planning meeting (TPM): A two-day team planning meeting will be held in Washington, DC, to (a) share background, experience, and expectations for the assignment; (b) formulate a common understanding of the assignment; (c) review the background of the Track 1.0 OVC portfolio and its current status; (d) identify partners and key informants involved in the task, develop a common understanding of their relationships and interests, and agree on an approach to working with these groups and individuals; (e) define and agree on the roles and responsibilities of the team leader and team members; (f) agree on the objectives and desired outcomes of the assignment; (g) develop a realistic work plan; (h) orient the team to the report guidelines and financial forms; and (i) discuss all relevant administrative procedures. The initial two days will be very important for laying the groundwork for the evaluation. The team will be briefed by USAID staff at the beginning of the evaluation assignment in order to prepare them for key informant interviews and site visits and clarify issues. Within two days of the end of the TPM, the team will share the work plan and solidify the plan for completion of the evaluation with USAID/OHA. TRACK 1.0 OVC PROGRAM EVALUATION 43 2. Data collection: The evaluation team will review the various project documents and reports, including the APS related to the OVC portfolio, proposals, work plans, annual reports, internal and external evaluation reports, OGAC Orphans and Vulnerable Children (OVC) Programming Guidance, and other relevant materials. 3. Interviews and consultation meetings: The team will also conduct interviews and consultation meetings with stakeholders and key informants, including but not limited to USAID/OHA CTOs and Prevention Managers; USAID Mission staff (including Assignment Managers); and OVC Track 1.0 partners and sub-partners. 4. Field visits: The team will undertake a multi-country qualitative sample of OVC CA partners, sub-partners, beneficiaries, and USAID Mission staff. In order to accomplish these visits, it is likely that the team will need to split into two smaller teams for the field visit portions. The choice of sites within countries visited should reflect the diversity of partners, populations, interventions, and environmental contexts that comprise the Track 1.0 agreements. It is hoped that each of the 14 Track 1.0 OVC partners will be visited in at least one of the countries in which they work, and at least 1–2 partners be visited in all countries in which they work. It is anticipated that each member of the team will visit approximately 2–3 countries so that every OVC partner field site is visited by a member of the team. Field visits to be confirmed by USAID. In making the final selection of countries to be visited, USAID/OHA will ensure that the USAID Missions in those countries are aware of the purpose of the evaluation and timing of the country visits. 5. Field visit debrief: Approximately one working week following field visits, the team will report orally (and with slides) on initial top-line findings to both USAID and Track 1.0 partners. It is anticipated that this session will help the team to clarify any questions or issues that came up during the field visits and to solicit support for any gaps in information. Please note that this is in addition to the final USAID Washington debrief mentioned in section VI, “Deliverables,” below. V. TEAM COMPOSITION The evaluation team will consist of 4–5 persons—the Team Leader, two OVC Advisors, the Evaluation Advisor, and the internal GH Tech Logistics Coordinator. Collectively the team members should have strong backgrounds to comprehensively cover OVC programming and implementation in the HIV/AIDS context, and monitoring and evaluation of USAID portfolios (and programs) at international, national, and community levels. Please see the description below and the list of specific tasks to be accomplished by the Team Leader in conjunction with the other team members. An estimated level of effort for each task for the Team Leader is listed below. VI. DELIVERABLES Work Plan: During the TPM, the team will prepare a detailed work plan, which shall include the methodologies to be used in this assessment. The work plan shall be shared with USAID/OHA for approval no later than two days after the conclusion of the TPM. Preliminary Report: The team will submit a preliminary report, including findings and recommendations, upon completion of the field work. This report will highlight achievements and best practices as well as shortcomings and lessons learned. The report should include a 1–2 page brief for each Track 1.0 partner summarizing highlights of the 44 TRACK 1.0 OVC PROGRAM EVALUATION specific project, key results and recommendations (as annexes), and an overall general set of recommendations. A standardized format for these partner-specific annexes will be developed by the team during the TPM. The preliminary report should not exceed 30 pages in length (not including annexes, lists of contacts, etc.). This draft will include findings and recommendations for USAID/OHA and USAID Mission review. The partner annexes (in draft form) will be disseminated by USAID to the respective implementing partners. Partner organizations will have one week for review and factual corrections. USAID will have approximately three weeks to provide comments and suggestions to GH Tech for forwarding to the evaluation team, which shall be addressed in the final report. USAID Washington Debrief: The team will present the major findings to a USAID/OHA audience through a PowerPoint presentation. This debrief will include a discussion of past achievements and issues, as well as any recommendations the team has for future programming. Final Report: The team will submit the final report to GH Tech on or about January 30, 2008 (revised end date). GH Tech will review this report and send it to USAID/OHA. Due dates will be finalized with OHA during the TPM. This report should not exceed 35 pages (not including appendices, lists of contacts, etc.). The format will include executive summary, table of contents, findings, and recommendations. The report will be submitted in English, electronically. GH Tech and OHA will reach agreement on the details of report formatting/branding once the final report content has been approved. It will be a 508- compliant document. Any potentially procurement-sensitive information will be excluded from the report and will instead be included in a separate internal USAID memo for dissemination within USAID. The report will be disseminated within USAID and among implementing partners and stakeholders and will be made available for general dissemination. The final report document will be edited/formatted by GH Tech and provided to USAID/OHA approximately one month after USAID/OHA has reviewed the content and approved the final unedited content of the report. The final unedited report content can be used as a working document while final report editing/formatting is in process by GH Tech. GH Tech will provide five hard copies of this final version of the report to USAID/OHA. VII. LOGISTICS & ESTIMATED TIMELINE/LOE USAID/OHA will provide overall direction to the evaluation team, identify key documents and key informants, and liaise with USAID Missions to ensure logistical support for field visits prior to the initiation of field work. USAID/OHA personnel shall be available to the team for consultations regarding sources and technical issues before and during the evaluation process. USAID/OHA Point of Contact: Colette Bottini USAID – Office of HIV/AIDS TRACK 1.0 OVC PROGRAM EVALUATION 45 APPENDIX B. PERSONS CONTACTED USAID WASHINGTON John Crowley Chief, Implementation Support Division. Office of HIV/AIDS Cognizant Technical Officers (CTOs) Colette Bottini OVC Technical Advisor Pamela Wyville-Staples Senior Health Development Officer Christian Fung Health Science Specialist Rebekah Krimmel Public Health Advisor Megan Petersen New Partner Advisor/Outreach Andrea Halverson CTO Technical Working Group Gretchen Bachman OVC Senior Advisor (OHA) Beverly Nyberg OGAC Senior Technical Advisor on OVC Renee de Marco OVC Senior Advisor (AFR) Kirk Felsman Senior OVC Advisor - Regional Karin Turner HRC Advisor - Regional KENYA USAID Country Mission Warren Buckingham Kenya PEPFAR Coordinator Kate Vorley OVC Program Specialist (Office of Population and Health) Maurice Maina HIV Care and Support Specialist (Office of Population and Health) Jeniffer Wasianga Program Development Assistant (Office of Population and Health) Washington Mowomo Strategic Information / M&E Partners 1. Christian Aid (Program title: CBCO program) Karl Hughes Chief of Party, CBCO Jane Machira Senior Program Officer Sarah Okwaare Program Manager for Kenya Christian Aid Subpartner, BIDII Margaret Kisilu Program Director Edward Mururi CBCO Coordinator Nancy Mumbua Accountant/Data Tracking Julianna Makao CBCO Project Officer Ezekiel Dodo Field Officer 46 TRACK 1.0 OVC PROGRAM EVALUATION 2. Christian Children’s Fund (CCF) (Program title: Weaving the Safety Net [WSN]) William Oscar Fleming HIV/AIDS Program Specialist (CCF headquarters) Dennis O’Brian CCF Kenya National Director Daniel Kinoti WSN Program Manager Rose Kerubo WSN Program Coordinator Wilson Sarich Finance Officer, WSN Ancelim Gituma Education Officer, WSN Eunice Kilundo Youth Officer - Kiambu District, WSN Joan Nganga Youth Officer - Thika District, WSN Michael Otieno Database Officer, WSN Mugita Geshogo M&E National Coordinator (CCF Kenya country office) Jerusha Chege Community Development Facilitator - Ruiru Division, WSN Esther Chege Community Development Facilitator - Lari Division, WSN CCF Subpartners K-REP Microfinance Anne Gathuku General Manager, FAHIDA Project (HIV programme) John Kibua Chief Accountant, KREP Pathfinder International, Kenya Linda Cassey Kenya Country Representative Irene Mwaponda Key contact for WSN CCF Sub-subpartners (nongrantees) Ruiru AIDS Awareness Group John Mbugua CBO Coordinator Mary Njeri Gachuma Community Mobilizer Supervisor, ECD Teacher Mary Wanjiku Kinuthia Community Mobilizer Supervisor, PSS Tutor, ECD Teacher Edward Kimiti Community Mobilizer, trained paralegal, youth tutor Michale Mugo Paralegal, Community Mobilizer Nyaga Polytechnic College in Kiambu - OVC vocational training Hellen Wanjiku Beneficiary of vocational training who now owns hair salon 3. CARE International in Kenya (Local Links) Bill Philbrick CARE US, Acting Director, HIV/AIDS unit (headquarters) Pascal Malisa Project Manager, Local Links, CARE International in Kenya Rosemary M. Mbalwe Sector Manager, HIV and AIDS and Civil Society Strengthening, CARE International in Kenya Stephen O’Kello Project Officer, Local Links, CARE International in Kenya Sub-subpartners, CARE International Charles Ogutu HAKISHEP Mary Mairimu Hands of Love Society Charles Mutunga Barak Za Ibrahim Children’s Centre Issa Kassim Kenya Organisation of People Living with AIDS Joseph Kulongo Kibera Slum Education Programme TRACK 1.0 OVC PROGRAM EVALUATION 47 Jacob Kabayo Kibera Post-Test Clubs Network Electine Malesi Kibera Post-Test Clubs Network Government Officials Mary Mbuga National Coordinator, OVC Secretariat, Ministry of Gender, Children and Social Development Daniel Musembi M&E coordinator, OVC Secretariat, Department of Children’s Services, Ministry of Gender, Children and Social Development Joseph Kajwang Training, Operations, and Monitoring and Information System, OVC Secretariat, Department of Children’s Services, Ministry of Gender, Children and Social Development Dr. Bett Kiambu East District Medical Officer UGANDA USAID Country Mission Elise Ayers Team Leader for Strategic Objective 8, Department of Health, HIV/AIDS, and Education Herbert Mugumya Activity Manager, USAID Uganda Seyoum Dejene Advisor for Care and Treatment, USAID Uganda Partners 1. Salvation Army Holly Christofferson Technical Advisor for Microfinance Irene Akena Program Manager for Uganda Fred Sibulo Project Accountant Rebecca Kwagale M&E Specialist Janet Etou Abaka Regional Coordinator, Central and Western District, Salvation Army Mbale District Salvation Army Field Office Michael Eberu PSS and Counselling Assistant Coordinator Stephen Kurenu Assistant Regional Coordinator Salvation Army Subpartner: PACT, WORTH component Jane Klasuyana PACT, WORTH Coordinator Dorothy Leyombya Nakwaku Assistant Coordinator for WORTH Agnes Ntutonti Assistant Coordinator for WORTH 2. AVSI Lucia Castelli Program Manager Marco Trevinau Program Coordinator Charles Ddlamulira Program Officer James Mugabi Program Officer Laura Miguerina Junior Program Officer 48 TRACK 1.0 OVC PROGRAM EVALUATION AVSI Sub-subpartners Kamwokya Christians Caring Community Magdalene Ndagire Social worker Joseph Kakooza Social worker and counsellor Meeting Place International (financed sub-grantee) Businsye Manager 3. Africare Jacqueline Gayle Programs manager, headquarters Abdalla Meftuh OVC Track 1.0 Programme Coordinator, headquarters Bueno Dickens Sande Program officer Gad Tukamushaba Program coordinator Innocent Atukunda RM&E officer Two people from Africare Kampala office 4. Plan International (Program title: Breaking Barriers) Helen Monteil OVC Technical Advisor and Program Manager Jim Gibson Program Support Manager Kumukunda Marget Senior Project Coordinator Walter Obwola Grants Manager Gilbert Matabe M&E Advisor Immaculate Nakitso Tororo Breaking Barrier Program Manager Jacquie Muyama Tororo District Community Development Officer Doreen Nyadoi Tororo District Community Development Facilitator Vincent Omoding Tororo District Community Development Facilitator Plan International Sub-partners Inter-religious Council of Uganda (IRCU) Charles Serwanja M&E Specialist, IRCU (Plan International) Solomon Asaba Program Assistant, IRCU (Plan International) Save the Children in Uganda (SCinUG) Hadijah Nandyose Project Manager for Education, SCinUG Jonathan Gamusi Design & M&E, SCinUG (Plan International) Government Officials H. Willie Otim Commissioner for Youth and Children’s Affairs, Ministry of Gender, Labour & Social Development George Beekunda Acting Director, Social Development, Ministry of Gender, Labour, and Social Development George Ejidra Probation Welfare Officer, Nakawa Division in Kampala, Ministry of Gender and Community Development (Salvation Army contact) Susan Alamai Government Probation Officer, Tororo district TRACK 1.0 OVC PROGRAM EVALUATION 49 NAMIBIA USAID Country Mission Sangita Patel Deputy Director, USAID Namibia (Activity Manager) Gabriel Kalungi OVC Advisor (Activity Manager for CAFO NPI) Debra Mosel Assistant Mission Director Partners 1. Family Health International/FABRIC Lucy Steinitz Regional Advisor, Windhoek, Namibia Marika Matengu Technical Officer for OVC, Namibia Subpartners Positive Vibes David Lush Director Church Alliance for Orphans (CAFO/FHI Namibia) Henry Platt Director Nicolette Bessinger Program Manager 2. Project HOPE (Namibia) John Bronson Director of Income Generation Programs Hector Jalipa Africa Regional Director and Acting Country Director Faith Mandizvidza North Central Regional Supervisor Taimi Amukwaya Health Supervisor Mary Elago Loan Supervisor Joseph Luchenta Economic Strengthening Manager Eneas Emvula Business Skills Development Officer Ongwediva Open Market Konga Ekwatho Village Health Fund (10 women) ZAMBIA USAID Country Mission Kasese Bota Mwaba HIV OVC and Youth Advisor Rene Berger Team Leader HIV/AIDS Carl Henn HIV/AIDS Food and Nutrition Advisor, CTO for RAPIDS Jennifer Shields Strategic Information Officer Partners 1. Family Health International/FABRIC Mo Schroeder-Sanai Senior Program Officer Global Operations, Arlington VA Catherine Mukwakwa FABRIC Project Director, Zambia Kapanga Kasongo FABRIC Finance and Administration Officer, Zambia 50 TRACK 1.0 OVC PROGRAM EVALUATION Subpartners Expanded Church Response Wilfred Manda M&E Officer Yvonne Pande Program Officer Marjorie Chileka National Program Coordinator Bishop Joshua Banda Board Chairperson Rev. Raymond Nyirenma Acting Executive Director Sub-subpartners Oasis of Love Project of the Evangel Assembly Church, Chingola (of Expanded Church Response) John Lumbwe Pastor Phiri Numel Chairperson of Committee David Chawa Financial Secretary Yumbe Kalabo Committee Member and Volunteer Caregiver Jennipher Mwanza Committee Member and Volunteer Caregiver Sylvia Siame Committee Member and Volunteer Caregiver Jessie Chikoko Committee Member and Volunteer Caregiver Gertrude Ngosa Committee Member and Volunteer Caregiver Susan Chileshe Committee Member and Volunteer Caregiver Miluba-Mfiwa Widows and Orphans Community School, Chingola (of Expanded Church Response) Batalion Nyimbili Coordinator Clement Chileshe Volunteer Teacher Mary Chanda Volunteer Teacher Dominic Chisala Volunteer Teacher Mr. Kangwa Financial Officer Mrs. Kangwa Volunteer Caregiver 2. World Concern David Eller Executive Director Bethany Baxter Program Coordinator Beki Mukwakwa Moyo HIV/AIDS Coordinator, Zambia Mwila Bwalya M&E Officer, Zambia Paul Chinyawa Field Officer, Zambia Eli Toribia Chief of Party/Country Team Leader Namani Hamangaba Data Entry Encoder 3. Opportunity International Subpartners Habitat Daison Mbewe OVC Project Officer Martin Ketongo Grants Manager Victor Malilwe Assistant Accountant, Ndola Affiliate Waveson Hamuchanfilwi Field Manager Matambo Milner Salvation Army District Coordinator TRACK 1.0 OVC PROGRAM EVALUATION 51 CETZAM Kembo Ng’ona HIV/AIDS Coordinator Fred Chaatila Regional Operations Manager Dyson Mandivenga Chief Executive Officer Evans Mbew Ndola Branch Manager Obby Matafwali Ndola Branch Loan Officer Anthony Musonda Ndola Branch Loan Officer David Kayaya Ndola Assistant Branch Manager 4. Project Concern Clara Eder Director of M&E Kelly Skrable Documentation and M&E Technical Officer Rajesh Singh Deputy Country Director, Zambia Grant Mulenga National M&E Manager, Zambia Precious Nkhata M&E Officer, BELONG Zambia Precious Ngoma Project Officer Namnye Jemma Nakanyika Project Officer, HBC Amy Tiwai OVC Under-5 Advisor Chiluba Mumba Project Officer Benny Njobru DOD Program Manager Beatrice Simasasa OVC Coordinator, ZDF Knox Kalubi Project Officer for Kafue District Makangwe Community School – Kafue (PCI Partner) Subpartners Bwafwana Home-Based Care Organization (PCI Sub-Partner) Naommy Zulu Nutritionist Beatrice Chola Executive Director Eddie Muswa OVC Coordinator Duncan Sodala M&E Officer Kataso Bambala Education Officer Loveness Maseko ART Clinic Officer Phiri Isaac Program Officer 5. Olive Leaf Foundation, (Formerly Hope World Wide) Portia Nkosi Fikile Dlali Cleopatra Muma Program Coordinator Jennipher Milambo OVC Service Delivery Facilitator Wilson Banda Finance Officer 6. Catholic Relief Services Dorothy Brewster Lee Chief of Party, Track 1.0 OVC Program Exhilda Siakambo Project Officer James Campell M&E Coordinator Paul Macek Country Representative Cecilia Adalla Head of Programming 52 TRACK 1.0 OVC PROGRAM EVALUATION N-Lupasa Wa Kabonda Project Accountant Subpartners Catholic Diocese of Solwezi (CRS CHAMPS partner) Martha Mwale Site Coordinator Susan Manda Assistant Coordinator Batilnle Soneka Accountant Abel Kamau Program Coordinator Tendai Nhandu M&E Officer Vicar General Philip Phiri Director of Social Programs Catholic Diocese of Mongu Beatrice Simona Coordinator, CHAMPS Government Officials Nicholas Banda Chief Child Development Officer, Ministry of Sport, Youth and Child Development, Zambia Bishop Joshua Banda Chairperson, National AIDS Council TRACK 1.0 OVC PROGRAM EVALUATION 53 APPENDIX C. SUMMARY OF OVC TRACK 1.0 AWARDS The following is a detailed summary of each of the Track 1.0 OVC awards and achievements to date. The funding amounts and “Targets in Agreement” columns represent the original funding and targets in the original cooperative agreement. The achieved targets represent the cumulative targets achieved through March 31, 2008, as reported by the partners in the March 08 Semiannual Performance Reports, unless otherwise noted. Because the evaluation team did not have access to agreement modifications or buy-in agreements, this summary does not reflect any changes in targets or funding levels. Partner Total Award Federal Share Match (%) Award Dates Africare $10,457,473 $9,999,970 4.6% Mar 2005-2010 Targets in Agreement Achieved (Mar ‘08) Note Mozambique 240,000 83,327 Rwanda 161,440 15,104 Tanzania 240,800 63,360 Uganda 220,000 20,123 Total 862,240 181,914 AVSI $15,135,544 $7,211,884 109.9% Apr 2005-2010 Targets in Agreement Achieved (Mar ‘08) Note Kenya 3,243 3,479 Rwanda 2,500 2,419 Uganda 6,737 6,624 Côte d’Ivoire Added post-award Total 12,480 12,522 Care $5,680,558 $5,225,197 8.7% Apr 2004-2009 Targets in Agreement Achieved (Mar ‘08) Note Kenya 20,000 28,312 South Africa 35,000 38,432 Total 55,000 66,744 54 TRACK 1.0 OVC PROGRAM EVALUATION Partner Total Award Federal Share Match (%) Award Dates Catholic Relief Services $9,003,682 6,950,883 29.5% Feb 2004-2009 Targets in Agreement Achieved (Mar ‘08) Note Haiti 15,000 18,358 Kenya 20,000 37,500 Rwanda 3,250 9,233 Tanzania 9,500 23,814 Zambia 17,500 19,931 Botswana 2,470 Added post-award Total 65,250 111,306 Christian Aid $7,594,958 $5,894,958 28.8% Apr 2005-2010 Targets in Agreement Achieved (Mar ‘08) Note Kenya 2,625 6,206 Nigeria 2,497 Target not specified in award Uganda 6,500 12,409 Zambia 15,250 14,879 Total 24,375 35,991 CCF $4,864,549 $3,183,965 52.8% Mar 2005-2010 Targets in Agreement Achieved (Mar ‘08) Note Kenya 38,325 43,757 Total 38,325 43,757 FHI $9,261,181 $9,261,181 0% Aug 2005-2010 Targets in Agreement Achieved (Mar ‘08) Note Namibia 28,000 2,376 Agreement targets were changed South Africa 27,000 8,156 Zambia 22,500 4,599 Total 53,200 15,131 TRACK 1.0 OVC PROGRAM EVALUATION 55 Partner Total Award Federal Share Match (%) Award Dates Olive Leaf Foundation (formerly Hope World Wide) $8,600,137 $8,190,607 5% Mar 2005-2010 Targets in Agreement Achieved (Mar ‘08) Note Botswana 5,533 Targets in agreement not divided by country Côte d’Ivoire 31,185 Kenya 1,200 Nigeria 6,002 South Africa 28,600 Zambia 12,944 Total 146,000 85,464 Opportunity International $10,290,453 $5,090,089 102.2% Feb 2004-2009 Targets in Agreement Achieved (Mar ‘08) Note Mozambique 12,109 Targets in award not readable Uganda 23,135 Zambia 12,791 Total 47,963 Plan International USA $12,658,089 $8,000,000 58.2% Apr 2005-2009 Targets in Agreement Achieved (Mar ‘08) Note Kenya Targets in agreement not divided by country March 2008 semiannual report not provided to evaluation team Uganda Zambia Total 150,000 Project Concern $10,461,066 $8,507,770 23% Mar 2005-2010 56 TRACK 1.0 OVC PROGRAM EVALUATION Targets in Agreement Achieved (Mar ‘08) Note Ethiopia 161,775 Targets in agreement not divided by country Zambia 74,533 Total 193,000 236,308 Partner Total Award Federal Share Match (%) Award Dates Project Hope $9,699,623 $8,606,213 12.7% Apr 2005-2010 Targets in Agreement Achieved (Mar ‘08) Note Mozambique 50,000 31,823 Namibia 25,000 8,164 Total 75,000 39,987 Salvation Army $6,938,112 $5,894,769 17.7% Apr 2005-2010 Targets in Agreement Achieved (Mar ‘08) Note Tanzania 37,485 Team did not receive the original project description. Uganda 19,531 Total 57,016 Save the Children $5,877,000 $5,877,000 0% Feb 2004-2007 Targets in Agreement Achieved (Mar ‘07) Note Ethiopia 64,303 87,925 Save the Children’s agreement ended earlier than the others. The achieved numbers represent the data reported on the latest performance report as of March 2007. Mozambique 91,343 78,229 Total 156,646 166,104 World Concern $13,544,108 $9,913,708 36.6% Sep 2004-2009 Targets in Agreement Achieved (Mar ‘08) Note Haiti 22,500 21,208 Kenya 60,000 49,590 Zambia 68,000 61,528 Total 150,500 132,326 TRACK 1.0 OVC PROGRAM EVALUATION 57 APPENDIX D. COMPARISON OF SERVICES COMPARISON OF TRACK 1.0 OVC SERVICE DESCRIPTIONS AS REPORTED IN THE 2008 SEMIANNUAL REPORT (MARCH 2008) Service Project HOPE FHI-Namibia World Concern Olive Leaf Foundation Plan Food and nutrition Community gardens; growth monitoring; food parcels Food parcels; soup kitchen garden; short￾term food supplementing Received 1 food contribution, agricultural contribution, or basic nutrition training Food parcels (no indication of how many) Provision of food, nutrition, education on child school feeding Shelter and care Community rehabilitation/ received a blanket Renovations, integration into old or a new family Strengthening caregivers Clothing and bedding Protection Provision of birth and death certificates Protection from child abuse; removal of children for placement in temporary shelter; will writing; access to birth certificates Training in succession planning or memory boxes Referral to authorities for abuse; training in child rights Advocacy campaigns; birth registration; education on child abuse Health care Educational activities with caregiver; help to access health services HIV prevention education; monitoring immunization; referrals to clinics Received training in hygiene; received bed￾nets Referrals to medical care; prevention training; primary health care training for caregiver Treatment of common ailments and opportunistic infections; health education; provision of information, education, and communication materials; participation in meetings; school clubs Psychosocial support Educational activities with caregiver Holiday camps; counseling programs; kids’ clubs Received 1 home visit or attended 1 special event Participation in a kids club Counseling and guidance; life skills; writing wills 58 TRACK 1.0 OVC PROGRAM EVALUATION Service Project HOPE FHI-Namibia World Concern Olive Leaf Foundation Plan Education and vocational training Scholastic material School registration; advocacy for fee reduction; homework support; monitoring school attendance and performance bursaries for tertiary education; provision of school supplies Provision of school supplies and uniforms or fees Provision of uniforms and fees Support for education materials and provision of levies to attend vocational training Economic strengthening Child living with a caregiver who participates in a village health bank Accessing social grants; financial management for older OVC, income￾generating activities Training in basic business skills Income￾generating activities; food production Service PCI OI CRS CARE Food and nutrition Supplementary nutrition and food support to malnourished and food-insecure OVC Monitoring OVC to receive government food basket and nutrition services Feeding programs and food supplements in primary schools and early childhood development centers Shelter and Care Shelter, water and sanitation, and personal hygiene improvements at the household or school level Construct and renovate houses; provide mosquito nets Improvements or access to shelter and institutional placements; OVC households given blankets and tarpaulins during post-election emergency response Protection Legal/administrative support in child protection, property rights; community sensitization on child rights, sexual abuse, early marriage, gender, stigma and discrimination Inheritance plans; households insured, including those whose houses were constructed in previous years Monitoring that OVC received adequate protection from their caregivers and communities Training of teachers, CBO staff, youth group members, and caregivers on child protection and first aid; application for birth certificates TRACK 1.0 OVC PROGRAM EVALUATION 59 Service PCI OI CRS CARE Health care Securing free medical certificates, referral support, and follow￾up to ensure medical support is provided; reimbursement of medical expenses Provision of sanitary, healthy latrines Monitored that OVC received the services provided by the government Referral to medical support including HIV/AIDS, sexual health, and health education programs; OVC households given kitchen sets, treated mosquito nets, hygiene sets during post-election emergency response Psychosocial support Grief counseling; recreation participation; memory work, including succession planning and individual and group counseling Youth leaders trained to organize and train OVC, but most left their localities in search of employment Train and support CBO staff and teachers to counsel OVC including stigma reduction, HIV prevention, abuse/referrals to government departments, adoption and fostering, and recreational/cultural activities Education and vocational training Tutorial support; provision of education materials and uniforms; waiver of school fees Youth Apprenticeship Program for employment or starting their own businesses Support for preschool children; vocational training was not successful. OVC given school uniforms, learning materials; teachers trained; includes assistance with admission in schools, and school fee negotiations Economic strengthening Train and support caregivers in forming savings-led self-help groups, income￾generating activities, and identifying/ addressing OVC issues Microcredit to caregivers to increase capacity to care for OVC; train caregivers in business and financial management Training caregivers of OVC on the group savings and loans approach and establishing income- generating activities Service AVSI Africare Christian Aid CCF Food and nutrition Short-term food supplementation to malnourished or HIV-infected children and their caregivers Training families on nutrition; seeds for OVC community gardens; caregiver clusters trained to sustain backyard gardens and pool resources to buy seeds and establish home gardens Households with OVC given improved seed and livestock; Caregivers supported to establish kitchen gardens, and trained in improved security; exceptionally destitute households provided with direct food aid Short-term food support and Unimix 60 TRACK 1.0 OVC PROGRAM EVALUATION Service AVSI Africare Christian Aid CCF Shelter and care Supports institutions where children are living temporarily Materials for constructing OVC houses; community members helped with construction; provided soap bars and petroleum jelly Bedding and clothing Protection Recreational outing, music, dancing, and sport activities Succession planning and identification documents provided to OVC; local FM radio talk show on child protection, succession planning, birth registration OVC monitored by trained mentors weekly; OVC guardians trained in child protection; minor abuse cases identified and resolved; serious abuse cases identified and referred Training on children’s rights and addressing protection issues of OVC by paralegals; paralegal training Health care Agreement with hospitals, health care centers, and health insurance to provide health care for OVC and their families Peer education; theatres, dances, and painting of murals with HIV messages; home visits with education on health; peer educators given information on HIV prevention; provided water guard for cleaner drinking water, insecticide￾treated bednets, health insurance OVC participated in the weekly life skills sessions including HIV related training; OVC received complementary HIV training; OVC given direct medical support by project medical staff; others with serious ailments referred to local health care facilities; medical expenses paid for some OVC Home-based care; insecticide-treated bednet provision; health education; minor treatment and deworming Psychosocial support Material support, follow-up visits, and counseling OVC participate in life skills activities and PSS; teachers and community volunteers trained in PSS Kids Clubs and life skills sessions; OVC guardians trained in PSS and received one-on-one counseling support Training in PSS, Journey of Life counseling services Education and vocational training Providing school fees and scholastic material School kits and uniforms; OVC linked to vocational scholastic materials and support through block grant exchange program for secondary schools Secondary school and vocational skills training; primary school fees; provision of uniforms; address nonfinancial barriers. Scholastic support and start-up kits for youth TRACK 1.0 OVC PROGRAM EVALUATION 61 Service AVSI Africare Christian Aid CCF Economic strengthening Business skills training and income-generating activities for OVCs and their families Training and support to caregivers from the CCCs and associations supporting OVC; pig rearing; training in entrepreneurship; goats to child-headed households SLA component: loans for income￾generating activities; guardians trained in business planning and management OVC residing in households supported with microcredit 62 TRACK 1.0 OVC PROGRAM EVALUATION TRACK 1.0 OVC PROGRAM EVALUATION 63 APPENDIX E. PARTNER SUMMARIES 1. Africare 2. AVSI 3. CARE 4. Catholic Relief Services 5. Christian Aid 6. Christian Children’s Fund 7. Family Health International 8. Olive Leaf Foundation (Formerly Hope World Wide) 9. Opportunity International 10. Plan International 11. Project Concern International 12. Project HOPE 13. Salvation Army 14. World Concern 64 TRACK 1.0 OVC PROGRAM EVALUATION AFRICARE CTO Colette Bottini Program Community-Based Orphan Care, Protection and Empowerment project for Children Affected by AIDS (COPE for CABA). Key Partners Deloitte Emerging Markets Group (for technical expertise in income-generating activities, vocational training for youth and families, and microcredit), and Boston University's Center for International Health (for M&E plans, baseline, midterm and final evaluations, and operations research). The Population Council also provided initial technical assistance. Countries Uganda, Tanzania, Mozambique and Rwanda (in-country staff in Uganda interviewed; no field visits were conducted) Dates March 18, 2005–March 17, 2010 Funding Central funding: US$9,999,970; matched funding: US$457,503 Subpartners Africare is starting to more systematically build the capacity of local organizations that support OVC. Evaluation Team Visit Summary The evaluation team members, Nicky Davies and Christopher O’Connell, interviewed Africare headquarters staff member Abdalla Meftuh, in Washington. In Uganda the team interviewed three Uganda COPE staff and one headquarters staff member (Jacqueline Gayle) who was visiting at the same time. The team met two Africare country office staff briefly during the partner debriefing meeting. No field visit was conducted for logistical reasons. Program Description and Key Results LIFE OF ACTIVITY (LOA) PROGRESS TRACKING TABLES FOR OVC SERVED Reporting Period: April 1, 2005– March 31, 2008 Mozambique Planned for LOA Mozambique Achieved to Date Uganda Planned for LOA Uganda Achieved to Date Tanzania Planned for LOA Tanzania Achieved to Date Number of OVC ever served by an OVC program 60,000 83,327 20,000 20,123 37,500 63,360 LOA PROGRESS TRACKING TABLES FOR OVC SERVED Reporting Period: April 1, 2005–March 31, 2008 Rwanda Planned for LOA Rwanda Achieved to Date Totals Planned for LOA Totals Achieved to Date Program Target Total Number of OVC ever served by an OVC program 20,000 15,104 137,500 181,914 137,500 Source: 2008 March semiannual report. TRACK 1.0 OVC PROGRAM EVALUATION 65 The goal of COPE is to reduce the socioeconomic impact of HIV/AIDS on a total of about 137,500 OVC and their caregivers. The five strategic objectives (SO) of COPE are SO 1: Enhanced district/community capacity to coordinate care and support services for OVC and caregivers; SO 2: Increased access to life skills training, peer education and psychosocial care and support to OVC and their families; SO 3: Increased access to educational support services for OVC; SO 4: Increased access to healthcare and nutritional support (including nutrition education and food for OVC and caregivers); and SO 5: Increased access to income-generating opportunities for OVC and caregivers. Highlights and Promising/Best Practices • The use of block grants and resource exchange is an innovative approach that has emerged in response to the difficulties associated with direct assistance (payment of individual school fees). In a block grant initiative, a school receives a lump sum payment at the beginning of a school term, which it is then able to spend on books, uniforms, school refurbishments, etc. In exchange, the school agrees to admit a predetermined number of OVC who are exempted from paying school fees. Africare has chosen this strategy because it believes that it will contribute to a significant reduction in stigma and discrimination toward children perceived to be receiving assistance from the implementing agency. • Reaching higher markets with income-generating activities by assessing the “value chain,” for example adding value to bananas by drying them. Linking income-generating activities to large, viable markets so that they can make a significant difference to the quality of lives of OVC and their caregivers, for example, income-generating activities linked to Fruits of the Nile, which exports dried fruits. • Cost-effective use of the regional aspect of program design: A full-time Regional Technical Manager based in Dar es Salaam is exclusively focused on COPE. She provides technical back-stopping to all four countries and conducts quarterly site visits. This cuts down on more expensive technical support from headquarters and provides a daily link to a technical expert in the region. Challenges • Africare was initially driven to achieve high numbers of OVC “due to the emergency nature of the OVC Track 1.0 program.” Some of the strategies used to reach OVC with specific services (especially in terms of education) are not sustainable. Africare has received an additional $1 million from the Uganda Mission to strengthen the capacity of local partners and linkages to improve sustainability and retain continuity of services to OVC currently being served. • More systematic mapping of services (public and private) in a geographical area is needed to ensure coordination, but this is time consuming and difficult to keep current. Evaluation Follow-up Recommendations • Africare did a baseline study but has not conducted a mid-term evaluation. • It will be important to followup to ensure that the “costed extension” achieves its sustainability/continuity of service goals; there seems to be some tension within Africare about the approach to take. 66 TRACK 1.0 OVC PROGRAM EVALUATION THE ASSOCIAZIONE VOLONTARI PER IT SERVIZIO INTERNAZIONALE (AVSI) CTO Rebekah Krimmel Program Increased Access to Care and Support for OVC in the Great Lakes Region Key Partners No key partners; only subpartners at the country level Countries Kenya, Rwanda, Uganda, and now Côte D’Ivoire (visited in Uganda) Dates April 4, 2005–April 2, 2009 Funding Central funding: US$7,211,884; matched funding: US$7,923,660 Subpartners AVSI works through an operative network of 82 local partners across all the countries. Evaluation Team Visit Summary Evaluation team members Nicky Davies and Christopher O’Connell interviewed AVSI headquarters staff member Jackie Aldrette in Washington before traveling to meet the regional and Uganda country team in Kampala. The evaluation team was then taken on a field visit to Kamwokya Area of Kampala to visit grantees Kamwokya Christians Caring Community (clinic, youth center and microfinance institution) and Hands of Love (ECD and vocational learning center). The team then visited Meeting Point International, a network/self-help group of HIV-positive women supporting OVC in the Acholi quarter, Kireka area of Kampala. Program Description and Key Results Source: 2008 March semiannual report. AVSI has a two-pronged approach to holistic service provision that is focused on (a) the needs of the individual person and (b) supporting direct service providers that are deeply embedded in their communities. The entire program combines indirect and direct forms of assistance. Direct assistance, provided through qualified local partner organizations, provides for school attendance, learning materials, after-school programs, vocational training, health care, and recreational and emotional support. Indirect assistance consists of support to quality education, promotion of income-generating activities, community projects and sensitization, and family support. Training and consultations are provided for individual partners and local networks to address institutional and LOA PROGRESS TRACKING TABLE FOR OVC SERVED Reporting Period April 4, 2005 – March 31, 2008 Uganda Planned for LOA Uganda Achieved to Date Rwanda Planned for LOA Rwanda Achieved to Date Kenya Planned for LOA Kenya Achieved to Date Number of primary direct OVC ever served by an OVC program 6,011 6,624 2,231 2,419 2,894 3,479 LOA PROGRESS TRACKING TABLE FOR OVC SERVED Reporting Period April 4, 2005–March 31, 2008 Totals Planned for LOA Totals Achieved to Date Total program Target Number of primary direct OVC ever served by an OVC program 11,136 12,522 12,000 TRACK 1.0 OVC PROGRAM EVALUATION 67 operational weaknesses and to improve capacity, efficiency, and quality. For each child enrolled in the project, AVSI makes an intervention plan with specific activities with two main aims: education of the child and promotion of self-reliance for the child and the family. Highlights and Promising/Best Practices • To protect against the discontinuation of funds during an education cycle, AVSI is carefully selecting new OVC (to reach final targets) that will be able to finish an educational cycle before the program ends in June 2010. Although this gives a bias to the OVC selection, those selected are still vulnerable children. • AVSI has used the requirements of OVC Track 1.0 (including the Emergency Plan OVC Guidance) as an opportunity, rather than an imposition, to strengthen their own, and their subpartners, project management systems. This attitude allows AVSI to lead by example with subpartners reaping the benefit of this professional approach. • AVSI has a firm commitment to building partner capacity to become more independent (five partners have graduated to be funded externally) through rigorous planning, resource mobilization, M&E (including analysis of data captured as a self-monitoring tool), financial management, and reporting. • Although a child sponsorship organization, AVSI supports the whole household to address core needs, encouraging ownership of challenges and working towards self-reliance. For instance, caregivers are linked to microfinance and income-generating activities so that they can increasingly meet the needs of OVC themselves, such as educational costs. • Effective use of the regional aspect of the program design for cost-effective learning and sharing across the region; including employment of a regional staff person (Lucia Castelli) rather than replicating her roles for each country. This regional program manager provides technical management and country oversight; another staff member at headquarters provides more process-oriented management support (evaluations, work plans, etc). This seems to be a different and effective program management model. • AVSI has created a directory of services for each location to increase service linkages and wrap-around. Challenges • The comprehensive range of high-quality services provided by AVSI for OVC (including paid social workers, etc.) means that AVSI’s cost per child looks high. AVSI is concerned that any cost-per-child analysis take quality of services provided into account. • The OVC Guidance age limit on OVC (up to 18) does not take into account children who start schooling late. Some Guidance indicators are difficult to measure, such as direct and supplementary nutrition support. Evaluation Follow-up Recommendations • Lucia Castelli provides a considerable level of technical back-stopping and project management support to AVSI in the region (Kenya, Rwanda, Uganda and now Côte D’Ivoire) and is also an active voice from the field for the OGAC OVC TWG. USAID should preserve her role in the region if possible. 68 TRACK 1.0 OVC PROGRAM EVALUATION CARE CTO Christian Fung /Andrea Halverson Program Local Links for OVC Support Key Partners No key partners; only subpartners at country level Countries Kenya and South Africa (visited in Kenya) Dates April 30, 2004–May 31, 2009 Funding Central funding: US$5,225,197; matched funding: US$455,361 Subpartners Kenya: 14 community-based organizations, 23 faith-based organizations, 20 primary schools and 15 youth groups South Africa: 11 implementing partners in Limpopo and the Free State Evaluation Team Visit Summary Evaluation team members Nicky Davies and Christopher O’Connell interviewed Bill Philbrick on October 30, 2008, by telephone. On Monday, November 10, they team met with the CARE Local Links staff in Kenya, among them Pascal Masila and Rosemary Mbalwe, the HIV/AIDS and CSO Strengthening Officer. Pascal gave a detailed presentation of the Local Links program, which was followed by a brief discussion. The evaluation team then visited the Local Links field office in Kibera slum to meet with the three field officers and interview representative of six CBOs; Hands of Love ECD and vocational training centers; and one group savings and loan (GS&L) that had been trained by the Kibera Organization of People Living with HIV/AIDS. The field visits and the journey back to Nairobi provided extra time to discuss the program with the field officers and Pascal Masila. On Tuesday, November 11, the team interviewed Pascal Masila, Rosemary Mbalwe, and Stephen O’Kello for 2.5 hours using the prepared questionnaire. Program Description and Key Results The Local Links for OVC Support project is implemented by CARE in one informal urban setting in Kenya (Kibera) and two rural districts in South Africa (Limpopo and Free State.) Local Links uses the SLA model to provide economic strengthening to vulnerable families and community members. It also provide income-generating training and mentoring, and capacity building to local community￾and faith-based organizations, as well as training on PSS, child protection issues, and health. TRACK 1.0 OVC PROGRAM EVALUATION 69 LOA PROGRESS TRACKING TABLE FOR OVC SERVED Reporting Period: (February 2004 OR date of signed agreement – March 31, 2007) Kenya Planned for LOA Kenya Actual to Date Uganda Planned for LOA Uganda B Actual to Date Totals (A+B+…n) Planned for LOA Totals (A+B+…n) Achieved to Date Total Program Target Number of (OVC) ever served by an OVC program 26,000 28,312 35,000 38,432 61,000 66,744 55,000 Source: 2008 March semiannual report. Highlights and Promising/Best Practices • GS&L for OVC caregivers, without injecting any external funding, creates true ownership of the GS&L scheme. It also provides a social security network and safe space for problem solving, referrals, and input into the Kibera Organization of People Living with HIV/AIDS and its activities, which is not so feasible through individual microfinancing. • Linking caregivers to ECD services for children 0-8 years old builds their understanding of children’s developmental needs. • Reaching children 0-3 years old through home-based care activities and through establishment of quality-controlled home-based ECDs is commendable. In both cases caregivers are encouraged to engage in activities to improve their understanding of the needs of children and how to respond. • Linking ECD as an income generation strategy increases the availability of this service and earns income for caregivers. • Local and national level advocacy for the rights and needs of the urban poor is a useful model. • CARE now has increased experience of programming for the urban poor and a more child￾centered approach to OVC programming. Challenges • It is necessary to design OVC selection and monitoring tools. • Because of stigma and the challenge of doing no harm when trying to focus on those most vulnerable and affected by AIDS and when making referrals, the program needs proxy indicators for OVC affected by AIDS rather than proof (particularly for the Emergency Plan’s scholarship program, which identifies OVC using the death certificates of parents, which often do not mention HIV or AIDS). • CARE noted that without the umbrella coordination of Local Links (the regional aspect of the program), some of the program linkages and quality may be eroded over time. Evaluation Follow-up Recommendations • Ensure that the Local Links South Africa program is evaluated. 70 TRACK 1.0 OVC PROGRAM EVALUATION • Ensure that CARE Kenya understands the role that the Kenya Mission is trying to play in relation to management of the OVC Track 1 program and is responsive to this (with the encouragement of the CTO). CATHOLIC RELIEF SERVICES CTO Colette Bottini Program Orphans and Vulnerable Children (OVC) Program Key Partners No key partners, only subpartners at the country level Countries Haiti, Kenya, Rwanda, Tanzania, Zambia, Botswana (visited in Zambia) Dates February 20, 2004–February 19, 2009 Funding Central funding: US$6,950,883; matched funding: US$2,052,799 Subpartners Zambia: Catholic Dioceses of Solwezi and Mongu Tanzania: Catholic Dioceses Evaluation Team Visit Summary The evaluation team of DeeDee Yates and Muyiwa Oladosu met some of the CRS in-country Zambia team first at the joint consultative meeting of all partners in Lusaka and then during their own group interview. During the group interview, the team met with the CRS country representative, head of programming, M&E advisor, program coordinator, and other staff of the OVC team. Some of the CRS Lusaka staff introduced the team to their subpartner in Solwezi in the northwestern part of the country. During the visit to Solwezi, the team interviewed the Solwezi Diocese OVC coordinator, assistant OVC coordinator, accountant, M&E officer, and field site coordinator, as well as the OVC coordinator for Mongu Diocese, who was visiting Solwezi at the time. The team also made courtesy calls on the Bishop of Solwezi and the director of social programs. Program Description and Key Results CRS countries participating in the Track 1.0 OVC are Haiti, Botswana, Kenya, Rwanda, Tanzania, and Zambia (Botswana is relatively new, about a year). LOA PROGRESS TRACKING TABLE FOR OVC SERVED Reporting Period (February 2004 OR date of signed agreement – March 30, 2008) Haiti Planned for LOA Haiti Achieved to Date Botswana Planned for LOA Botswana Achieved to Date Kenya Planned for LOA Kenya Achieved to Date Number of OVC ever served by an OVC program 17,500 18,358 4,620 2,470 37,500 37,500 TRACK 1.0 OVC PROGRAM EVALUATION 71 Source: CRS Mid-Term Report March 2008 The table shows that compared to LOA, CRS achieved their targets of serving OVC in all countries except Botswana, which has a shortfall perhaps due to its being new. CRS provides care and support in the following core areas: • Education and vocational training: Ensures that OVC are able to access formal education or acquire the skills they need for gainful employment • Health care: Provides access to primary care, immunization, and treatment for acute illnesses, antiretroviral treatment for children with AIDS, and prevention • Psychological support: Provides a positive approach to children’s challenges through support groups and peer-led youth groups • Food and nutrition: Provides food for OVC through linkages to the CRS Food aid program, the World Food Program, and Food for Peace (most CRS food programs have ended) • Protection: Deals with the issue of stigma and discrimination; access to basic rights, including birth registration; inheritance claims; and sibling unification according to the host country’s norms and laws • Shelter and care: Ensures that all children live in safe abodes with access to clean, safe water • Economic Strengthening: Supports vocational training, income generation, savings and loan methodology as means to economic independence Highlights and Promising/Best Practices from Zambia • Capitalization on the large grassroots structures of the Catholic dioceses • Ability to provide the compassionate aspect of their objectives through strong integration with the Catholic Church • Added advantage of synergy and fit with programs like SUCCESS and RAPID • Provides a broad range of services to OVC through local partners • A cost-effective way of running the four regional offices in West, East, Central, and Southern Africa: Regional office staff sit with in-country staff, allowing more involvement of regional staff in local program implementation. • Decentralization, with regional offices having roles such as financial decision-making LOA PROGRESS TRACKING TABLE FOR OVC SERVED Reporting Period (February 2004 OR date of signed agreement – March 30, 2008) Rwanda Planned for LOA Rwanda Achieved to Date TZ Planned for LOA TZ Achieved to Date Zambia￾Planned for LOA Zambia￾Achieved to Date Total Achieved to Date Number of OVC ever served by an OVC program 3,250 9,233 20,000 23,814 17,500 19,931 111,306 72 TRACK 1.0 OVC PROGRAM EVALUATION Key Lessons Learned • Use of the quality-of-life tool to enhance the collection of process data on the quality of care based on a study of OVC perceptions of what affects their quality of life • Development of a comprehensive service card that is useful for referring OVC to other services to address gaps in the CRS program Challenges • There were major lapses in communication, mainly concerning finances, during the project. • Resources were stretched thin during the course of the project. • There was no capacity-building of subpartner staff so they can be more independent. • Volunteer attrition was a major challenge mentioned. Evaluation Follow-up Recommendations • Strengthen communication at all levels for more efficient project implementation. • CRS has funds from other sources but may need more to respond to increasing demands for services. • CRS should strengthen the capacity of their subpartners so they can transition to more independence by sourcing funds and even subgranting to others. CHRISTIAN AID CTO Megan Peterson Program Community-Based Care for Orphans and Vulnerable Children (CBCO) Key Partners No key partners; only subpartners at country level Countries Kenya, Nigeria, Uganda, and Zambia (visited in Kenya) Dates April 11, 2005–April 10, 2009 Funding Central funding: US$5,894,958; matched funding US$1,700,000 Subpartners Kenya: BIDII and IDCCS Uganda: YWAM, ACET and CPA Nigeria: GHADS and ADDS Zambia: CDN, CHEP, ADL and FHT Evaluation Team Visit Summary No headquarters interviews were conducted before the field visit. Evaluation team members Nicky Davies and Christopher O’Connell visited Christian Aid in Kenya on Wednesday, November 5, 2008. Karl Hughes and Jane Machira from the Christian Aid Kenya office joined the team on a site visit to meetings of two projects supported by a subpartner, BIDII, in the rural Machakos District. The first was a youth group meeting observed at a school, and the second was a meeting of an SLA made up of OVC guardians and older orphans. Informal discussions were held with both groups about the support they had received from the CBCO Program. TRACK 1.0 OVC PROGRAM EVALUATION 73 Program Description and Key Results Source: 2008 March semiannual report. The CBCO program uses SLAs as an entry point to provide services to OVC. The SLAs are made up of OVC caregivers, including older orphans who are heads of households. They are organized, trained, and monitored by Christian Aid’s indigenous subpartners. In addition to economic strengthening, the SLAs have several other tasks related to orphan care, such as conducting regular OVC household monitoring visits, and managing food and nutrition self-help projects, funded by the SLA and implemented by SLA members. The SLAs help identify the neediest children for educational support. Christian Aid and its subpartners also train older OVC to form and lead kids and youth clubs that provide health, HIV prevention, and PSS to OVC. Highlights and Promising/Best Practices • Christian Aid’s approach of using the SLAs as an entry point to other OVC services seems very promising and sustainable. Their initial approach was to build OVC Support Committees, of which the SLAs were an offshoot; however, observing that the SLAs were much stronger structures, they moved the OVC monitoring from the Support Committees to the SLAs and report great success. • Christian Aid’s M&E system is very strong. One promising practice of this system is its ability to track each service down to the specific subactivity provided to the beneficiary. Instead of simply marking that an OVC received economic strengthening support, for example, the system notes on a monthly basis what exact subactivities were provided, such as participation in the SLA, received a loan, or receive business training. Challenges • Christian Aid has had limited staffing capacity to ensure quality control of programs across all four countries. It has moved actively to address this, but it reflects a lack of field program start-up/design experience on the part of Christian Aid and possibly not enough advice from USAID at the program design stage. • Christian Aid’s HIV/AIDS knowledge and awareness program seems to be having limited impact based on its own M&E data. Christian Aid said that it will review this program component and make any necessary alterations. LOA PROGRESS TRACKING TABLE FOR OVC SERVED Reporting Period: April 11, 2005, to March 31, 2008 Kenya Planned for LOA Kenya Actual to Date Uganda Planned for LOA Uganda Actual to Date Zambia Planned for LOA Zambia Actual to Date Nigeria Planned for LOA Nigeria Actual to Date Totals (A+B+…n) Planned for LOA Totals (A+B+…n) Achieved to Date Total Program target Number of OVC ever served by an OVC program 2,625 6,206 7,600 12,409 15,250 14,879 5,000 2,497 30,475 35,991 25,000 74 TRACK 1.0 OVC PROGRAM EVALUATION Evaluation Follow-up Recommendations • The Mission and the partner need to harmonize their OVC definitions. Any OVC currently outside the Mission’s definition will need to be carefully handled. • BIDII is reportedly one of the strongest indigenous subs. It appeared to be a strong organization though still needing more training to sustain its programs without regular monitoring and support from Christian Aid. • BIDII participates in local government forums. Christian Aid should ensure that it is also engaged with government structures at other levels. • Any future work with Christian Aid should ensure that their program design builds on gaps identified by the national strategic plan and other national data sources, and that the partner is therefore used to best effect. • There could be more effective use of the regional aspect of the program design for cost￾effective learning and sharing across the region, including employment of a regional staff person (Karl Hughes), rather than replicating his roles for each country. This regional program manager role represents technical management for this program and country oversight. A full case study of the benefits of the regional aspect of the program design has been done. CHRISTIAN CHILDREN’S FUND CTO Colette Bottini Program Weaving the Safety Net (WSN) Program for Orphans and Vulnerable Children and Youth Affected by HIV/AIDS in Kenya Key Partners K-REP and Pathfinder International. Contracting TA partners: AED for training on ECD and mother mentoring, and REPSSI on training PSS, conference assistance, and sharing best practices. Countries Kenya (although initially applied for Ethiopia and Uganda as well) Dates March 18, 2005–March 17, 2010 Funding Central funding: US$3,183,965; matched funding US$1,680,584 Subpartners 80 local implementing partners Evaluation Team Visit Summary William Fleming was interviewed by evaluation team members Nicky Davies and Christopher O’Connell on October 30, 2008, in Washington. The team visited Christian Children’s Fund in Kenya November 6–7, 2008. On November 6 the team was briefed at the WSN office by seven of the 21 WSN staff led by Daniel Kinoti (WSN Manager) and Rose Kerubo (WSN Coordinator). Rose Kerubo joined the team on site visits to four projects and to meet one beneficiary. The two WSN youth officers joined the team at various points and the site Community Development Facilitator was present at each project site visited. On November 7 the evaluation team visited WSN subpartners Pathfinder Kenya and K-REP to conduct interviews; no WSN staff were present. Finally, the evaluation team met with Daniel Kinoti and Dennis O’Brian at the Kenya CCF Country Office. TRACK 1.0 OVC PROGRAM EVALUATION 75 Program Description and Key Results CCF’s Weaving the Safety Net (WSN) Project is a five-year program with the goal of reducing the impact of HIV/AIDS on 63,325 orphans, vulnerable children, and adolescent youth in Thika and Kiambu districts in Kenya’s Central Province. To meet this goal, CCF interventions integrate direct child support, assistance to families/caregivers, and government and community support systems. WSN’s activities in support of OVC and their caregivers include community mobilization and the following program interventions: educational support, vocational training, child protection, PSS, home-based care (HBC) for people living with HIV/AIDS, microcredit support to OVC caregivers and vulnerable households, health care, shelter, food/nutrition, and advocacy and policy influence. Source: 2008 March semiannual report. Mid-term results (March 2008) indicated that WSN had served 63% of the total OVC targets to be served within the five-year period and surpassed the target for caregivers trained. Highlights and Promising/Best Practices • Building the capacity of local CBOs/FBOs and local government structures, particularly Area Advisory Councils, to ensure longevity of programs and implementation of national policies • CCF’s commitment to using strategies that ensure sustainability as much as possible, such as the selection of implementing CBOs/FBOs using capacity assessment tools with the direct involvement of the local District Development Office. The engagement of local government officials aims to ensure sustainability of services by building trust, mutual understanding, and respect between local government and CBOs/FBOs. • WSN support for vocational training for young people including PSS activities for all pupils attending the school, not just those sponsored by WSN, and provision of toolkits to help graduates start their own business. • CCF’s early attention to building the confidence and skills of CBOs and FBOs to mobilize their own resources through fundraising, creating partnerships and linkages, and mobilizing material donations. Challenges • There were indications that microfinancing through K-REP may not be the most appropriate economic strengthening approach for the WSN program. An SLA approach may be a better starting point for rural caregivers. • WSN will continue to build the capacity of local partners to maintain services for OVC using external resources—this is still work in progress. LOA PROGRESS TRACKING TABLE FOR OVC SERVED Reporting Period: March 2005 or date of signed agreement – March 31, 2008 Kenya Planned for LOA Kenya Achieved to Date Total Program Target Number of orphans and vulnerable children (OVC) ever served by an OVC program 46,600 43,757 63,325 76 TRACK 1.0 OVC PROGRAM EVALUATION Evaluation Follow-up Recommendations • The Mission and WSN need to harmonize their OVC definitions. Any OVC currently outside the Mission’s definition will need to be carefully handled. • If the OVC currently being served by WSN are folded into APHIA II, USAID should try to use CCF’s experience of implementing a range of services for OVC and give them a greater role in APHIA II implementation—particularly as it is child-focused and therefore more likely to innovate in this area. Family Health International CTO Christian Fung/Andrea Halverson Program Faith-Based Regional Initiative for Vulnerable Children (FABRIC) Key Partners No key partners; only subpartners at country level Countries South Africa, Zambia, Namibia (visited in Namibia and Zambia) Dates August 22, 2005–August 21, 2010 Funding Central funding: US$9,261,181; matched funding: 0 Sub-Partners FABRIC works through three umbrella FBOs: First, the Church Alliance for Orphans (CAFO) in Namibia, and now Positive Vibes (both visited in Namibia); Expanded Church Response (ECR) (visited in Zambia); and the Southern African Catholic Bishops’ Council (SACBC) in South Africa Evaluation Team Visit Summary Evaluation team members DeeDee Yates and Muyiwa Oladosu visited FHI in Zambia and Namibia. In Zambia the team met with Catherine Mukwakwa, the FABRIC Project Director, who also serves as the country technical officer. They then interviewed a team from local subpartner ECR in Lusaka. In Chingola in the Copper Belt Province, the team met with two community groups that are subgrantees of ECR: the Oasis of Love project of the Evangel Assembly and the Muliba Mfiliwa Community School. They also met for discussions with local committee members, many of whom also serve as volunteer caregivers. In Namibia the team interviewed the new FHI technical officer for OVC, Marika Matengu, and met with the previous local partner, CAFO, and the new local partner, Positive Vibes. Lucy Steinitz, previous OVC senior technical officer, sent a written response. Program Description and Key Results The FABRIC program is providing care and support to OVC in Namibia, South Africa, and Zambia. FHI has made subgrants to three faith-based umbrella organizations: CAFO in Namibia, ECR in Zambia, and SACBC in South Africa. FABRIC builds the capacity of these large organizations to manage small grants programs that fund congregations and member organizations to provide OVC care and support activities. FABRIC is in its third year. It is partnered with 29 subrecipients (five less than in the second year) through the three major partners. Of these, six are in Namibia, eight in South Africa and 15 in Zambia. FHI/FABRIC has provided training to its three primary subpartners, which it calls implementing agencies. The three are all established church development agencies who make subgrants to local community groups who support OVC. The groups may be parish-based or inter-denominational, TRACK 1.0 OVC PROGRAM EVALUATION 77 but in all cases they serve children without regard to religious affiliation. The subrecipients have been trained in counseling, financial and grant management, M&E, and community support for OVC. All are volunteer organization with no paid staff. FHI/FABRIC has not reached its original agreement targets, which were renegotiated with Washington and country Missions due to a shift in the scope of work and role of FHI’s partner, CAFO. It is reaching the new targets. Source: 2008 March semiannual report Highlights and Promising/Best Practices • The local implementing agencies seem solid, focused, and able to continue the work with little outside assistance. CAFO is now a new partner and is no longer receiving funds from FABRIC. ECR is part of the RAPIDS bilateral OVC program in Zambia, and SABC has received funds directly from the field office for prevention work. It is not possible to attribute all the success of the local partner organizations to FHI because all the local partners received substantial support from other donors, including other USG programs. But FHI/FABRIC did help build the capacity of the organizations to make sub-grants. FABRIC aims to help Positive Vibes be independent within the short Track 1.0 time remaining. • Positive Vibes is a young organization specializing in enhancing the leadership of people living with and affected by HIV and AIDS, including children. It has piloted tools to improve the self-esteem and self-expression of children, allowing them to speak to caregivers and decision makers about their issues. • The local organizations and committees have tried to address volunteer attrition challenges by raising funds and material resources from other sources. Many report substantial successes in fundraising. Challenges • FHI/FABRIC considers volunteer attrition to be a key concern. This represents a loss of assets for the local organizations and brings into focus questions on sustainability. However, the committee members who have been trained expressed their satisfaction with what they had learned from the umbrella organization. • FHI reports, and the evaluation team also heard elsewhere, that subrecipients—church and ecumenical committees—receive very little funds given the need and demand for their services. The range of the sub-grants is US$1,500 to US$10,000. LOA PROGRESS TRACKING TABLES FOR OVC SERVED Reporting Period (October 2005 – March 31, 2008) RSA Planned for LOA RSA Achieved to Date Totals Planned for LOA Totals Achieved to Date Program Target Total Number OVC ever served by an OVC program 27,000 8,156 77,500 15,131 53,200 78 TRACK 1.0 OVC PROGRAM EVALUATION Evaluation Follow-up Recommendations • Prepare organizations in all three countries to transition to other funding sources to allow for continued support to community groups caring for children. • Help Positive Vibes to distill and document the essential ingredients and approaches needed to give children and youth a voice. Share these widely. OLIVE LEAF FOUNDATION (FORMERLY HOPE WORLD WIDE) CTO Pamela Wyville-Staples Program The Africa Network for Children Orphaned and at Risk (ANCHOR) Key Partners Rotarians for Fighting AIDS (RFFA); Coca Cola Africa; Emory University School of Public Health and Nursing Countries Botswana, Côte d’Ivoire, Kenya, Nigeria, South Africa, Zambia (visited) Dates March 18, 2004–March 17, 2010 Funding Central funding: US$8,190,607; matched funding: US$409,530 Subpartners Affiliates of the key partners are involved at country level. Evaluation Team Visit Summary Evaluation team members DeeDee Yates and Muyiwa Oladosu spoke to Portia Nkosi, the ANCHOR Regional Manager, and Fikile Dlali, who are based in South Africa. The team met also interviewed three Olive Leaf Foundation Zambia staff in Lusaka. Program Description and Key Results LOA PROGRESS TRACKING TABLES FOR OVC SERVED Reporting period: Oct 07 – Mar 08 Botswana Planned Botswana Actual Côte d’Ivoire Planned Côte d’Ivoire Actual Kenya Planned Kenya Actual Nigeria Planned Number of OVC ever served by an OVC program 10,750 5,533 31,000 31,185 25,300 1,200 23,100 LOA PROGRESS TRACKING TABLES FOR OVC SERVED Reporting period: October 2007– March 2008 Nigeria Actual South Africa Planned South Africa Actual Zambia Planned Zambia Actual Total Planned Total Actual Number of OVC ever served by an OVC Program 6,002 30,000 28,600 19,935 12,944 140,085 85,464 Source: 2008 March semiannual report. Olive Leaf Foundation works through country offices to reach community and faith-based organizations. The level of involvement of each partner varies from country to country, with Olive TRACK 1.0 OVC PROGRAM EVALUATION 79 Leaf being the lead and RFFA active in all countries. Coca Cola Africa primarily provides funds and material support. The team found no evidence of the involvement of Emory University. ANCHOR works through new or established community groups, schools, and support groups. These organizations are given training on PSS, leadership, governance, and microenterprise development as well as some material support. The list below suggests how many such organizations were engaged with ANCHOR in 2008: • Botswana: 20 schools and ECD centers and 4 clubs • Côte d’Ivoire: 38 local organizations • Kenya: 89 CBOs • Nigeria: 12 CBOs • South Africa: 55 schools and local organizations • Zambia: 32 schools and 18 CBOs Training is done in collaboration with the Regional OVC Organizational Support Initiative (ROSI) within Olive Leaf. Highlights and Promising/Best Practices • Olive Leaf partners in-country are able to leverage private sector funding for food and material assistance. • ANCHOR has an impressive network of schools and community organizations in the six countries. These seem to function best when they are part of or are established organizations or institutions, such as schools or PTAs. Local organizations are helped to develop financial and M&E systems. Partnerships with schools and local government structures offer possibilities of sustainability. • The establishment, with SIDA funding, of the ROSI offered additional resources and expertise for community capacity development. • Olive Leaf receives support from the South Africa Mission for its work with Kid’s Clubs at several sites. Challenges • Olive Leaf did not begin with a local capacity development approach and has since found that local organizations may not be in a position to implement due to leadership problems and planning and financial management constraints. • Establishing community child care forums or their equivalent is time-consuming and most involve all local stakeholders. The committees may understandably think that since they were formed by Olive Leaf, Olive Leaf has some responsibility to maintain them and provide continuing financial support. • There does not appear to be a clear plan for building up community ability to continue the care provided with ANCHOR. Many of the country programs complain of lack of volunteer commitment and dependence on Olive Leaf or partners for material support. These 80 TRACK 1.0 OVC PROGRAM EVALUATION complaints point to an underdeveloped approach to community mobilization and capacity building. Evaluation Follow-up Recommendations • Country-level Olive Leaf programs need to take a more systematic approach to community development. OPPORTUNITY INTERNATIONAL CTO Christian Fung/Andrea Halverson Program Sustainable Income and Housing for OVC in Africa Key Partner Habitat for Humanity Countries Mozambique, Uganda, Zambia (visited) Dates February 20, 2004–June 30, 2010 Funding Original agreement ceiling: US$5,090,089; matched funding: US$5,200,364: Final ceiling: $5,789,749 Subpartners Zambia: Christian Enterprise Trust of Zambia (CETZAM), and Habitat for Humanity, Zambia Mozambique: Banco Oportunidade de Mozambique and Habitat for Humanity Mozambique Uganda: Uganda Agency for Development, and Habitat for Humanity Uganda Evaluation Team Visit Summary Opportunity International (OI) does not have offices in-country but works with local organizations as partners. Evaluation team members DeeDee Yates and Muyiwa Oladosu visited CETZAM and Habitat for Humanity, Zambia, who are OI’s subpartners and local representatives in Lusaka and other parts of Zambia. Members of the CETZAM staff in Lusaka interviewed were the chief executive officer, regional operations manager, and HIV/AIDS coordinator. Present at the CETZAM interview were also key staff of Habitat, including the OVC project managers and the grants manager. The evaluation team visited Ndola in northwestern Zambia, where at the CETZAM Ndola branch they interviewed the branch manager, assistant branch manager, and two loan officers. The Habitat staff interviewed in Ndola were the field manager and assistant accountant; a Salvation Army district coordinator was also present. Program Description and Key Results OI countries participating in the Track 1.0 OVC are Mozambique, Uganda, and Zambia. LOA PROGRESS TRACKING TABLES FOR OVC SERVED Reporting Period: (February 2004 – March 31, 2008) Mozambique Planned for LOA Mozambique Achieved to Date Uganda Planned for LOA Uganda Achieved to Date Number of OVC ever served by an OVC program 10,355 12,109 25,877 23,135 TRACK 1.0 OVC PROGRAM EVALUATION 81 LOA Progress Tracking Tables for OVC Served Reporting Period: (February 2004 – March 31, 2008) Zambia Planned for LOA Zambia Achieved to Date Total Planned for LOA Total Achieved to Date Number of OVC ever served by an OVC program 11,871 12,719 48,103 47,963 Source: 2008 March semiannual report. Compared to LOA, OI exceeded its targets for serving OVC in two of the three countries and came very close in the third. OI provides unique services through its in-country subpartners in the following areas: • Microfinance: Ensures that OVC caretakers have access to microcredit, savings, and insurance to enable them to generate more income to take care of their children. • Secure and healthy housing: Provides healthy housing to families in communities adversely affected by HIV/AIDS. It specifically targets OVC caregivers to enable them to renovate or build houses. • Capacity building for caregivers: Provides HIV/AIDS prevention and care training to caregivers to enable them to give better care to OVC. • Capacity building of OVC: Trains youth in the three countries in house construction and maintenance, thus providing ready labor for construction in the community and increasing the economic viability of OVC trained and thus families served. Highlights and Promising/Best Practices from Zambia • The working relationship between Habitat and CETZAM has helped transform the lives of many families through the combination of shelter and economic empowerment. • CETZAM’s three grades of loan benefactors and the training and monitoring schemes have enabled communities to be more responsive to meeting their own needs. Lessons Learned • The synergy between CETZAM and Habitat for Humanity makes Zambia an interesting model for building local partnerships. • The graduated loan scheme of CETZAM was reported to have benefited petty traders, especially women, and could be scaled up to reach other women. Challenges • Providing a home to a family caring for OVC is relatively expensive compared to other interventions to support shelter but may have multiple benefits that are not always monitored or captured. • The way to deal with the sickness and death of a loan recipient or rights of ownership of homes built for a diseased caregiver is not clear. 82 TRACK 1.0 OVC PROGRAM EVALUATION • OI’s distance from the field has created challenges for communications between USG Mission offices and local partners. • Estimating housing costs should be more participatory to include inputs from the beneficiaries, especially in terms of the type of house and how it should be built. PLAN INTERNATIONAL (PI) CTO Pamela Wyville-Staples Program Breaking Barriers (BB) Key Partners World Conference of Religions for Peace, Save the Children US, and their in￾country affiliates Countries Kenya, Uganda, Zambia (visited in Uganda) Dates April 4, 2005–April 3, 2009 Funding Central funding: US$8,000,000; matched funding: US$4,658,089 Subpartners Key partner in-country affiliates Evaluation Team Visit Summary: Evaluation team members Nicky Davies and Christopher O’Connell interviewed Helene Montiel, Plan USA, on October 30 in Washington. They interviewed two staff from the Inter Religious Council of Uganda (IRCU) and two from Save the Children in Uganda (SCinUG) on November in Kampala. On November 18 they visited Breaking Barriers work in the Tororo and conducted interviews with Tororo District PI staff, 10 teachers trained by PI through BB, and 15 religious leaders trained by IRCU. The next day the team interviewed three Uganda PI staff in Kampala. Program Description and Key Results Breaking Barriers was spearheaded by Hope for African Children Initiative (HACI), which has since closed down, in collaboration with partner organizations. In Uganda, among these partners were Save the Children US, PI Uganda, and IRCU. Plan offices in Kenya and Zambia took over coordination of the BB projects in their countries. The project’s mandate is to expand sustainable, effective, quality programs in education, PSS, and community-based care for OVC and families affected by HIV and AIDS. This is implemented through both formal and informal school networks and religious institutions as a coordinated platform for rapid scale-up and scale-out. The initiative was a response to the high prevalence of OVC, mainly due to the proliferation of HIV and AIDS, and the high incidence of poverty in the project countries. TRACK 1.0 OVC PROGRAM EVALUATION 83 LOA PROGRESS TRACKING TABLES FOR OVC SERVED February 2006 or date of signed agreement – March 31 Kenya Planned for LOA Kenya Achieved to Date Uganda Planned for LOA Uganda Achieved to Date Zambia Planned for LOA Zambia Achieved to Date Number of OVC ever served by an OVC program 41,672 46,412 67,493 76,322 22,480 16,429 LOA Progress Tracking Tables for OVC Served February 2006 or date of signed agreement – March 31 Totals Planned for LOA Totals Achieved to Date Total Program Target Number of OVC ever served by an OVC program 131,645 139,163 150,000 Source: 2008 March semiannual report. Highlights and Promising/Best Practices • The SCinUG CHANCE informal school approach has promise. • There is an extensive list of highlights that can be reviewed from the Breaking Barriers mid￾term evaluation. Challenges • Closure of HACI in 2007 resulted in both funding constraints, because HACI was to provide a 50% cost share, and also planning, supervision, and coordination challenges. • Because of inadequate time for the project design, the triangular approach did not work in Uganda because (a) SCinUG was not willing to start programming in the same areas as Plan; (b) a capacity assessment might have identified weaknesses within IRCU that have proven problematic; and (c) a Uganda Government regulation prevented NGOs working with formal and informal schooling from operating in the same area. • The flow of funding to partners, in terms of both amounts and timing, was poor. • All three partners had inadequate capacity to deliver the program as designed, not having budgeted for enough personnel, support staff, and vehicles. • BB has suffered from having to overcome the expectation within the community that PI will provide all the resources and solutions. • Too little attention has been paid to sustainability or community ownership of programs; in particular, economic empowerment strategies were omitted. • IRCU has not implemented the advocacy component of this project adequately. • There is poor communication and coordination of activities as all levels: at the community level, where IRCU trains religious leaders and PI trains teachers, they work separately and do not communicate even about monitoring. 84 TRACK 1.0 OVC PROGRAM EVALUATION Evaluation Follow-up Recommendations • Section 9.5 of the BB mid-term evaluation points to specific areas of sustainability that the OVC Track 1.0 evaluation team agree need to be addressed urgently: – Giving more prominence to advocacy – Enhancing community capacity for self-reliance – Engaging more with local CBOs – Among other mechanisms, perhaps matching communities and schools with alternative partners; initiating school-based child-to-child campaigns; supporting school gardening to boost feeding programs; and giving support to public health facilities among other ways to strengthen sustainability aims1 • Each partner is addressing service continuity issues in its own way: IRCU aims to use its current Mission funding to continue service delivery in Uganda. SCinUG is trying to register its CHANCE informal schools with the Ministry of Education. PI will use its sponsorship funds. Funds to provide follow-up training to ensure the quality of PSS support provided by teacher and community caregivers is most at jeopardy when the program finishes. PROJECT CONCERN INTERNATIONAL CTO Rebekah Krimmel Program Better Education and Life Opportunities for Vulnerable Children through Networking and Organizational Growth Key Partners Pact and World Food Program Countries Ethiopia and Zambia (visited Zambia) Dates March 18, 2005–March 17, 2010 Funding Central funding: US$8,507,770 ; matched funding: US$1,953,296 Subpartners Zambia: Bwafwano, CBO Community School, CBOs affiliated with Christian Children Fund (CCF), HBC/ART CBOs, HBC CBOs Ethiopia: HIV/AIDS Prevention, Care & Support Organization (HAPSCO) Evaluation Team Visit Summary Evaluation team members DeeDee Yates and Muyiwa Oladosu visited the PCI office in Lusaka, Zambia, and interviewed the deputy country director, national M&E manager, the M&E officer, the BELONG project officer, the HBC project officer, the OVC under-5 advisor, project officer, DOD program manager, and the OVC coordinator – ZDF. The team also visited Bwafwano, one of the umbrella subpartners of PCI, where the project coordinator, M&E officer, accountants, and other project staff were interviewed after the team had been introduced to the project director. The team also visited Makangwe community school, managed by a subpartner of PCI, and met with the project officer for Kafue District who oversees the school. 1 See Mid-term Evaluation of the Breaking Barriers Project, Final Report, April 2008. TRACK 1.0 OVC PROGRAM EVALUATION 85 Program Description and Key Results PCI countries participating in the Track 1.0 OVC are Ethiopia and Zambia. Source: 2008 March semiannual report PCI achieved its target of serving OVC in Ethiopia though not in Zambia, but surpassed their LOA combined figures for the two countries. In Zambia PCI provides services in a range of areas through a home-based platform and a school-based platform. The Community Home-Based Platform is implemented by Bwafwano, a local organization formed in response to the rising number of HIV/AIDS cases and OVC in Chipata district in Zambia. The services it provides include nutritional support, PSS, child protection, educational support, health care, and shelter renovation. PCI’s BELONG project supports Bwafwano with financial and human resources. The PCI school-based platform works through grassroots CBOs that provide educational services to children who have difficulty accessing basic education. In Zambia BELONG partners with 26 community schools to provide educational and nutritional support, water sanitation, recreational services, PSS, health care (First Aid Training, Access to Health Care, HIV prevention, and malaria integration). PCI supports these community schools. Also, PCI has involved community leaders in the governing bodies of the schools. Highlights and Promising/Best Practices from Zambia The PCI operation • Provides educational support through both informal and formal schools, such as community colleges. It helps the informal educational structures to meet government standard by involving government personnel in reviews of their performance. • Uses schools and community-based platforms to reach OVC and their families. • Works through formal and informal structures to meet the needs of OVC, strengthening community structures in the process. Lessons Learned • The approach of strengthening community structures by working directly with grassroots CBO is unique. • Capacity-building of local teachers enables them to become more professional through exposure to standard materials and techniques. LOA PROGRESS TRACKING TABLE FOR OVC SERVED Reporting Period (March 18, 2005 – March 31, 2008) Ethiopia Planned for LOA Ethiopia Achieved to Date Zambia Planned for LOA Zambia Achieved to Date Totals (Ethiopia+ Zambia) Planned for LOA Totals (Ethiopia+ Zambia)- Achieved to Date Number of OVC ever served by an OVC program 15,599 161,775 129,150 74,533 144,749 236,308 86 TRACK 1.0 OVC PROGRAM EVALUATION • A referral system allows school children to access other services, thus making the school a focal point for access to comprehensive services. Challenges • How to maintain quality of service while expanding coverage of services over many schools • How to provide close M&E while rapidly expanding coverage of schools • How to retain volunteer teachers that have attained professional standards (they seem to be using the community schools as a launch pad to greener pastures). Evaluation Follow-up Recommendations • PCI needs to be more strategic in providing services through the schools. It may be more realistic and cost-effective to use umbrella organizations like Bwafwano to work with smaller grassroots CBOs rather than working directly with them. • There is a need to incorporate into the programming graduation schemes for volunteer teachers to become paid teachers so that community schools can retain qualified teachers. PROJECT HOPE CTO Rebekah Krimmel Program Sustainable Strengthening of Families of Orphans and Vulnerable Children in Mozambique and Namibia Key Partners None Countries Mozambique and Namibia (visited in Namibia) Dates April 4, 2005–April 2, 2010 Funding Central funding: US$8,606,213; matched funding: US$1,093,410 Subpartners None Evaluation Team Visit Summary Team members DeeDee Yates and Muyiwa Oladosu spoke to John Bronson, director of income generation projects at the head office, and met with the Project HOPE team in northern Namibia, which included the regional director, who is acting country director. While there they also visited a microlending group consisting of 10 women in Ongwediva. Program Description and Key Results Project HOPE works directly with guardians and parents of OVC, providing them with small loans and business training through a Village Health Bank group model. The loan is accompanied by a 10- session health and parenting course (18 sessions in Namibia) that is delivered every two weeks at the same time as loan repayments are made. Program targets were reduced after discussion with Washington based on the need to meet new standards of care in-country and a revised health curriculum. TRACK 1.0 OVC PROGRAM EVALUATION 87 LOA PROGRESS TRACKING TABLE FOR OVC SERVED Reporting Period (April 4, 2005 – March 31, 2008) Mozambique Namibia Total Program Planned for Target Total LOA Achieved to Date Planned for LOA Achieved to Date Planned for LOA Achieved to date Number of OVC ever served by an OVC program 50,094 31,823 17,750 8,164 75,000 39,987 75,000 Number of providers/ caretakers trained in caring for OVC 13,125 5,069 5,379 2,251 21,737 7,320 Source: 2008 March semiannual report. Highlights and Promising/Best Practices: • The combination of a solidarity group, microfinancing, and a health and parenting curriculum seems to draw on the best from many models. Working directly with parents and guardians has somewhat alleviated the need for volunteers to deliver services for children. The health and child development curriculum has potential to build families’ capacity to address the emotional and physical needs of children in their care. At a Village Health Bank meeting several women offered stories of how the training helped them deal with difficult or withdrawn children in their families. The importance of this simple but fundamental type of family support cannot be overstated. • Project HOPE demonstrates flexibility and responsiveness to the country Missions. In Namibia Project HOPE also receives support from the Mission and agreed with the Mission on reducing its Track 1.0 targets in order to meet the draft Namibian Quality Standards first presented in 2007. • The program is also responsive to local needs and has made many adaptations during the project. For example, Project HOPE created a claim form so volunteers can help families obtain certain essential services and be reimbursed for any costs. • Project HOPE has very strong monitoring systems that USAID has used in Mozambique. In Namibia it undertook baseline research on the situation of children in households in the target area. It has adapted its monitoring system to focus on collecting essential data that can be used to guide the program. • Project HOPE has noticed that the economic strengthening project design relies on retail businesses—80% of those taking loans use them for retail activities. To address this Project HOPE has hired a consultant to undertake a business opportunity assessment. This is a good opportunity to diversify. 88 TRACK 1.0 OVC PROGRAM EVALUATION Challenges • Measuring the change in a household due to improved economic activity of the caregiver has been a challenge. Project HOPE has improved its M&E tools and the use of volunteers to help households. • The purpose of using volunteers in a program that targets guardians and parents is not always apparent. The volunteer’s main role is to monitor and support household implementation of the health and development lessons. It is unclear whether the volunteers have the capacity to do this or are needed. • Project HOPE has not worked with a local microfinancing partner, though links have been made with Koshi-Yomuti in Namibia and with ProCredit in Mozambique. Evaluation Follow-up Recommendations • Project HOPE could receive funds from the Missions directly and could also find a local partner to take on some of the lending responsibilities. In Namibia it is looking to become a locally registered lending agency. It is also using a grant from HIVOS to explore options for sustainability. • Continue refining the M&E system by adding the child status index and share it with other USAID partners • Explore how the activities could be adapted for a male audience now that Project HOPE is moving toward more work with youth-headed households. SALVATION ARMY CTO Colette Bottini Program Sustainable Community Support for Orphans and Vulnerable Children Key Partners PACT Uganda for technical assistance for the WORTH component Countries Tanzania and Uganda (visited in Uganda) Dates April 4, 2005–April 2, 2010 Funding Central funding: US$5,894,769; matched funding US$1,043,343 Subpartners None Evaluation Team Visit Summary Evaluation team members Nicky Davies and Christopher O’Connell interviewed Holly Christofferson, Salvation Army headquarters, by telephone on October 30. In Uganda, they interviewed three staff members from the Salvation Army OVC project and a Kampala government probation welfare officer from the Ministry of Gender and Community Development on November 14 in Kampala. On November 17 they visited the Salvation Army office and OVC project sites in Mbale, including an SLA association with associated income-generating activities and discussions with a young PSS community mobilizer/supporter. TRACK 1.0 OVC PROGRAM EVALUATION 89 Program Description and Key Results The program goal is to improve the physical, economic, and psychosocial well-being of 65,000 OVC and their households in Tanzania and Uganda by strengthening the capacity of faith-based and community organizations and networks to provide vital safety nets of support. The objectives are to 1. Strengthen communities to construct community-based responses to meet the needs of OVC and others affected by HIV/AIDS. 2. Provide psychosocial support for OVC. 3. Improve the economic security of OVC. Source: 2008 March semiannual report. Highlights and Promising/Best Practices • Coordination and planning has greatly improved over time to become a good example; quarterly and monthly meetings ensure that “real” work plans are made and monitored and problems solved. • The “community conversation” approach ensures that community members identify their own coping strategies in caring for OVC. • The integration of adult literacy and income generation/business management training has made the WORTH program for increased confidence and economic empowerment of vulnerable caregivers very successful. • The project learned from Tanzania about the importance of linking with government for sustainability and then implemented that approach in Uganda. The Salvation Army in Uganda has quickly developed extremely close links with local government officials that are mutually beneficial. Challenges • Changing guidance on what data to collect (monitoring now requires tracking services provided and quality of services) compounds the problem of monitoring activities that is already made difficult by the distance between OVC project sites. LOA PROGRESS TRACKING TABLE FOR OVC SERVED Reporting Period (February 2004 OR date of signed agreement – March 31, 2008) Uganda Planned for LOA Uganda Achieved to Date Tanzania Planned for LOA Tanzania Achieved to Date Totals Planned for LOA Totals (Achieved to Date) Total Program Target Number of OVC ever served by an OVC program 20,000 19,531 37,551 37,485 57,551 57,016 65,000 90 TRACK 1.0 OVC PROGRAM EVALUATION • Data quality is a continuing issue. Uganda had a data quality assessment early on that was used to make changes. Tanzania has just had an evaluation by UNIDEX and MEASURE that identified much to be done to improve quality. That work has started. • Although relationships with USAID/Washington have been very collegial, there has been limited effort to develop the capacity of Salvation Army to respond to changes in guidelines and M&E. • Targets were already high before the OVC guidance implied the need for a comprehensive service approach (Salvation Army said that Uganda MEEP staff implied that the guidance should be considered a rule). Salvation Army feels that it has been encouraged, for sustainability reasons, to provide material inputs which it would not normally provide. A recent evaluation in Tanzania, however, showed that this material support was greatly appreciated by the community. This raises a programming dilemma. Evaluation Follow-up Recommendations The Salvation Army believes it can access funding through the national civil society strengthening granting mechanisms that supports CBOs and NGOs—it thinks it can be considered indigenous. The Uganda Mission staff are not convinced that the program design will make it eligible for civil society funding. WORLD CONCERN CTO Pamela Wyville-Staples Program Hope for Children Affected by HIV/AIDS Key Partners No key partners, only country-level subpartners Countries Kenya, Zambia, Haiti (visited in Namibia and Zambia) Dates September 30, 2004–September 29, 2009 Funding Central funding: US$9,913,78; matched funding: US$3,630,400 Sub-Partners Kenya: World Relief (in-country lead), Nazarene Compassionate Ministries, Medical Assistance Program, Food for the Hungry Kenya, Christian Reformed World Relief Committee, World Concern Zambia: Christian Reformed World Relief Committee (CRWRC; in-country lead), World Hope International Zambia, Operation Blessing International, and Nazarene Church Mission Zambia Haiti: World Concern (in-country lead), CRWRC, World Hope International, World Relief, Salvation Army, and Operation Blessing International Evaluation Team Visit Summary Evaluation team members DeeDee Yates and Muyiwa Oladosu visited the program office in Zambia in the offices of CRWRC. They also met with local subpartner Nazarene Church Mission Zambia. At both a number of personnel were present. Program Description and Key Results World Concern is the lead agency for a consortium of nine members of the Association of Evangelical Relief and Development Organizations, which implements projects to support OVC affected by HIV and AIDS. In each of the three countries a different member acts as in-country TRACK 1.0 OVC PROGRAM EVALUATION 91 lead. Each member has its own local subpartners—churches, CBOs, and FBOs through which it works. Source: 2008 March semiannual report. World Concern is on track to meet its targets. PSS services (described as at least one home visit or attendance at one special event in the six-month reporting period) have the largest number of children, 24,948, followed by food and nutrition. The vast majority of children reached in all three countries are in the 5–17 age group. Highlights and Promising/Best Practices • This program was able to reach remote areas by working through the consortium with actual churches/parishes. In Zambia the program used church leadership structures to decide which communities to approach. This helped build a sense of local control and ownership. • As one subpartner in Zambia explained, “Church activities for children may be compromised when the program stops, but they will not cease.” This seems to sum up the primary strength of this project: its linkages with local organizations. Challenges • External factors, such as the devaluation of the dollar and post-election violence in Kenya, are a challenge to the program in all countries, as are certain programmatic issues. In Zambia, the high number of children who could be eligible for the program is a challenge given the limited financial and human capacity of the partners. This is a consistent message from partners in many counties. • More work is needed on economic strengthening, such as small-scale income-generation (poultry, pigs, and sewing). • The number of international partners within the World Concern program may inhibit the focus on local partners. Issues of local partners of other international partners may not be internalized by the in-country lead partner, which has its own local partners to attend to. LOA PROGRESS TRACKING TABLES FOR OVC SERVED Reporting Period (February 2004–March 31, 2008) Zambia Planned for LOA Zambia Achieved to Date Kenya Planned for LOA Kenya Achieved to Date Number of OVC ever served by an OVC program 68,000 61,528 60,000 49,590 LOA PROGRESS TRACKING TABLES FOR OVC SERVED Reporting Period (February 2004–March 31, 2008) Haiti Planned for LOA Haiti Achieved to Date Totals Planned for LOA Totals Achieved to Date Program Target Total Number of OVC ever served by an OVC program 22,500 21,208 150,500 132,326 150,500 92 TRACK 1.0 OVC PROGRAM EVALUATION • The number of local partners, while offering promise of continuity of services, also becomes a capacity development challenge. In Zambia the team heard that the project moves out of a community after one year in order to move on to another community, so as to reach the target number of children. This may compromise the ability of a community to continue to offer quality services. • High staff turnover within the in-country lead partner may cause capacity problems and inability to give sufficient technical assistance to local partners. • Volunteer attrition is a challenge for in-country partners and subpartners. Evaluation Follow-up Recommendations • Consider ways to link other aspects of an organization’s work with its Track 1.0 OVC work. Some of the partners are also New Partners Initiative partners working on prevention. • Concentrate in the remaining life of the project on strengthening partner communities to continue to offer support to OVC and to identify resources in and around their locality that could be mobilized for that purpose. TRACK 1.0 OVC PROGRAM EVALUATION 93 APPENDIX F. REFERENCES USAID WASHINGTON DC • USAID. Presidents Emergency Plan for AIDS Relief. Annual Program Statement. March 11, 2004 • USAID. Presidents Emergency Plan for AIDS Relief, (PEPFAR). Programming Guidance. July 2006. • USAID. Presidents Emergency Plan for AIDS Relief (PEPFAR). Update. • USAID. Cooperative Agreement Administration Authorities. September 2008. • USAID. Track 1.0 Semiannual and Annual Reporting Requirements for the President’s Emergency Plan for HIV/AIDS Relief. • National Academy of Sciences. PEPFAR Implementation: Progress and Promise (Free Executive Summary). http://www.nap.edu/catalog/11905.html. KENYA • CARE. Local Links for OVC Support. • CARE. Year 4 Budget. October 2007–September 2008. • CARE. FY07 Semiannual Progress Report October 2006–March 2007. • CARE. Annual Progress Report FY 2007. Local Links Support to Orphans and Vulnerable Children (OVC) Affected by HIV in Kibera, Kenya, and Limpopo and Free State, South Africa. October 2006–September 2007. • CARE. Semiannual Progress Report FY 2008. Local Links Support to Orphans and Vulnerable Children (OVC) Affected by HIV in Kibera, Kenya, and Limpopo and Free State, South Africa. October 2007–March 2008. • USAID. Support to Orphans and Vulnerable Children (OVC) Affected by HIV/AIDS Cooperative Agreement Award GPO-A-00-04-00018-00. July 2004. • Christian Aid. CBCO PEPFAR Application. April 2005–March 2010. • USAID. Christian Aid Cooperative Agreement # GPO-A-00-05-00021-00. Community Based Care of Orphans and Vulnerable Children (CBCO). April 2005. • Christian Aid. Kenya Budget Details. October 2007–September 2008. • Christian Aid. Community Based Care of Orphans and Vulnerable Children (CBCO). FY 2008 Workplan. October 2007–September 2008. • Christian Aid. Community Based Care of Orphans and Vulnerable Children (CBCO). FY 2008 Annual Report. October 2007–September 2008. • Christian Aid. Community Based Care of Orphans and Vulnerable Children (CBCO). Semiannual Report FY 2008. October 2006–March 2007. • Christian Aid. Community Based Care of Orphans and Vulnerable Children (CBCO). Semiannual Report 2008. October 2007–March 2008. • Christian Children’s Fund. FY08 Budget. October 2007–September 2008. 94 TRACK 1.0 OVC PROGRAM EVALUATION • Christian Children’s Fund. Weaving the Safety Net. Annual Workplan and Budget (Year 4). October 2007–September 2008. • Christian Children’s Fund. Weaving the Safety Net. Annual Report (Year 3). October 2006– September 2007. • Christian Children’s Fund. Weaving the Safety Net. Semiannual Report (Year 3). October 2006–March 2007. • Christian Children’s Fund. Weaving the Safety Net Semiannual Report (Year 4). October 2007–March 2008. • USAID. Cooperative Agreement # GPO-A-00-05-00016-00. Weaving the Safety Net. March 2005. NAMIBIA • FABRIC. Narrative Justification for the Early Funding Request. October 2007. • FABRIC. FY 08 Budget. October 2007–September 2008. October 2007. • FABRIC. CAFO—FABRIC MER PLAN COP 07 (October 2007–September 2008), October 2007. • FABRIC. Revised Workplan. Support to Orphans and Other Vulnerable Children Affected by HIV/AIDS. January 2008. • FABRIC. Annual Report. October 2006–September 2007. Submitted November 2007. • FABRIC. Semiannual Report. October 2006–March 2007. Submitted April 2007. • FABRIC. Semiannual Report. October 2007–March 2008. Submitted April 2008. • USAID. Cooperative Agreement N o. GPO-A-00-05-00028- 00. Support to Orphans and other Vulnerable Children Affected by HIV/AIDS for Family Health International. August 2005. • USAID. Cooperative Agreement N o. GPO-A-00-05-00019- 00. Sustainable Strengthening of Families of Orphans and Vulnerable Children in Mozambique and Namibia. April 2005. • Project Hope. Project Hope Proposal. February 2007. • Project Hope. Workplan: October 2007–September 2008. Sustainable Strengthening of Families of Orphans and Vulnerable Children in Mozambique and Namibia. September 2007. • Project Hope. Annual Report: October 2006–September 2007. Sustainable Strengthening of Families of Orphans and Vulnerable Children in Mozambique and Namibia. Submitted November 2007. • Project Hope. Semiannual Report: October 2006–March 2007. Sustainable Strengthening of Families of Orphans and Vulnerable Children in Mozambique and Namibia. Submitted May 2007. • Positive Vibes. FGD Analysis Hand Exercise Addendum to FGD Analysis. • Positive Vibes. Focus Group Discussion. • Positive Vibes. Report: Children’s Voices Phase II. November 2007. TRACK 1.0 OVC PROGRAM EVALUATION 95 UGANDA • USAID. Cooperative Agreement # GPO-A-00-05-0001500 with AFRICARE. Scaling up the Community-Care Based Protection and Empowerment (COPE) Model. March 2005. • Africare. FY08 Annual Workplan. Community-Care-Based Protection and Empowerment (COPE). February 2008. • Africare Headquarters Budget Details. Community-Care-Based Protection and Empowerment (COPE), October 1, 2007–September 30, 2008. February 2008. • Africare Community-Care Based Protection and Empowerment (COPE) Workplan Approval Letter. February 2008. • Africare. Annex 1: Annual Report October 2006 to September 2007. Community- Based Orphan Care, Protection and Empowerment (COPE), Mozambique. November 2007. • Africare. Annex 2: Annual Report October 2006 to September 2007. Community- Based Orphan Care, Protection and Empowerment (COPE), Tanzania. November 2007. • Africare. Annex 3: Annual Report October 2006 to September 2007. Community- Based Orphan Care, Protection and Empowerment (COPE), Uganda. November 2007. • Africare. Annex 4: Annual Report October 2006 to September 2007. Community- Based Orphan Care, Protection and Empowerment (COPE). Rwanda. November 2007. • Africare. Annual Report October 2006 to September 2007. Community-Based Orphan Care, Protection and Empowerment (COPE). November, 2007. • Africare. Semiannual Report October 2006 to March 2007. Community-Based Orphan Care, Protection and Empowerment (COPE). April, 2007. • Africare. Cover letter for Semiannual Report October 2007 to March 2008. April 2008. • Africare. Semiannual Report October 2007 to March 2008. Community-Based Orphan Care, Protection and Empowerment (COPE). May 2008. • AVSI. FY08 Annual Workplan Outline.Cooperative Agreement GPO-A-00-05-00020-00. Increased Access to Care and Support for Orphans and Vulnerable Children (OVCs) in the Great Lakes Region. November 2007. • AVSI. Five-Year Budget. November 2007. • AVSI. FY08 Kenya Workplan Matrix. December 2007. • AVSI. FY08 Rwanda Workplan Matrix. December 2007. • AVSI. FY08 Uganda Workplan Matrix. December 2007. • AVSI. Annual Report, October 2006 to September 2007. Increased Access to Care and Support for Orphans and Vulnerable Children (OVCs) in the Great Lakes Region. November 2007. • AVSI. List of Implementing Partners with AVSI in Uganda and Their Profile. April 2008. • AVSI. Profiles of Local Implementing Partners AVSI in Kenya. April 2008. • AVSI. The PEV Report Submitted by AVSI Kenya to USAID Kenya. April 2008. • AVSI. Profile of Partners in Rwanda. April 2008. • AVSI. Semiannual Report October 2007–March 2008. April 2008. 96 TRACK 1.0 OVC PROGRAM EVALUATION • USAID. Cooperative Agreement GPO-A-00-05-00020-00. Increased Access to Care and Support for Orphans and Vulnerable Children (OVC) in the Great Lakes Region. April 2005. • Plan International. Detailed Workplan. October 2007–September 2008. Breaking Barriers: Ensuring the Future of Orphans and Other Vulnerable Children Through Education, Psychosocial Support and Community-Based Care (Kenya, Uganda, Zambia). September 2007. • Plan International. Breaking Barriers Annual Budget FY08. September 2007. • Plan International. Annual Report. October 2006–September 2007. Breaking Barriers Program: Ensuring the Future of Orphans and other Vulnerable Children through Education, Psychosocial Support and Community-Based Care (Kenya, Uganda, Zambia). November 2007. • Plan International. Semiannual Report. October 2006–March 2007. Breaking Barriers Program: Ensuring the Future of Orphans and other Vulnerable Children through Education, Psychosocial Support and Community-Based Care (Kenya, Uganda, Zambia). May 2007. • Center for Research Communication & Gender in Early Childhood Development (CRECHE). Mid-Term Evaluation of the Breaking Barrier Project: Final Report. April 2008. • USAID. Cooperative Agreement GPO-A-00-05-00017-00. Breaking Barriers Program: Ensuring the Future of Orphans and Other Vulnerable Children Through Education, Psychosocial Support and Community-Based Care. April 2005. • Salvation Army. Budget Details. October 2007–September 2008. September 2007. • Salvation Army. Annual Workplan. Track 1.0 OVC October–September 2008. June 2008. • Salvation Army. Annual Report—Year Three (October 2006–September 2007). Sustainable Community Support for Orphans and Vulnerable Children in Tanzania and Uganda. November 2007. • Salvation Army. Semiannual Report—Year Three (October 2006–March 2007). Sustainable Community Support for Orphans and Vulnerable Children in Tanzania and Uganda. May 2007. • Salvation Army. Semiannual Report—Year Four (October 2007–March 2008). Sustainable Community Support for Orphans and Vulnerable Children in Tanzania and Uganda. May 2008. • USAID. Cooperative Agreement GPO-A-00-05-00018-00. Sustainable Community Support for Orphans and Vulnerable Children in Tanzania and Uganda. April 2005. • Save the Children. Consolidated SCALE-UP HOPE Report (October 2006–February 2007). • Save the Children. Semiannual Program Report (October 2005–March 2006). SCALE-UP HOPE Program (International Save the Children Alliance-Expanded Unified Partnership with the Hope for African Children Initiative). May 2006. • Save the Children. Letter on Five-Month Workplan for SCALE UP HOPE Program (International Save the Children Alliance–Expanded Unified Partnership with the Hope for African Children Initiative) in Ethiopia and Mozambique. September 2006. TRACK 1.0 OVC PROGRAM EVALUATION 97 • Save the Children. Five-Month Workplan (October 2006–February 2007). SCALE UP HOPE Program (International Save the Children Alliance–Expanded Unified Partnership with the Hope for African Children Initiative) in Ethiopia and Mozambique. October 2006. • Save the Children. Final Workplan (October 2006–February 2007). SCALE-UP HOPE Program. September 2006. • Save the Children. Budget for Final Obligation of SCALE-UP HOPE OVC Program (October 2006–September 2007). September 2006. • USAID. AID No. GPO-A-00-04-00003-00. Support to Orphans and Vulnerable Children Affected by HIV/AIDS (OVC). February 2004. ZAMBIA • Hope Worldwide, South Africa. Workplan Period: October 2007–September 2008. African Network for Children Orphaned and at Risk (ANCHOR) Project in Botswana, Côte d’Ivoire, Kenya, Nigeria, South Africa, Zambia. • Hope Worldwide South Africa. Semiannual Report (October 2006–March 2007). African Network for Children Orphaned and at Risk (ANCHOR). May 2007. • Hope Worldwide, South Africa. Annual Report (October 2006–September 2007). African Network for Children Orphaned and at Risk (ANCHOR). November 2007. • Hope Worldwide, South Africa. Semiannual Report (October 2007–March 2008). African Network for Children Orphaned and at Risk (ANCHOR). May 2008. • USAID. Cooperative Agreement GPO-A-00-05-00014-00 with Hope Worldwide. African Network for Children Orphaned and at Risk (ANCHOR). March 2005. • USAID. Cooperative Agreement GPO-A-00-04-00001-00. Support to Orphans and Vulnerable Children Affected by HIV/AIDS (OVC). February 2004. • Opportunity International. Sustainable Income and Housing for Orphans and Vulnerable Children in Africa: Tier Two Proposal. January 2004. • Opportunity International. Semiannual Workplan: October 7, 2009–February 19, 2009. • Opportunity International. Semiannual Report (October 2006–March 2007). Sustainable Income and Housing for Orphans and Vulnerable Children (OVC) in Africa. May 2007 • Opportunity International. Semi- Annual Report (October 2006–September 2007). Sustainable Income and Housing for Orphans and Vulnerable Children (OVC) in Africa. November 2007. • Opportunity International. Semiannual Report (October 2007–March 2008). Sustainable Income and Housing for Orphans and Vulnerable Children (OVC) in Africa. May 2008. • Project Concern International. BELONG: Summary Table of PEPFAR Targets. September 2007. • Project Concern International. BELONG: Detailed Workplan. Country: Zambia. Fiscal Year (October 2007–September 2008). September 2007. • Project Concern International. BELONG: Detailed Workplan. Ethiopia. Fiscal Year (October 2007–September 2008). September 2007. • Project Concern International. BELONG: Project Results Framework. September 2007. 98 TRACK 1.0 OVC PROGRAM EVALUATION • Project Concern International. BELONG: International Travel Matrix. September 2007. • Project Concern International. BELONG: FY08 Final Budget. September, 2007 • Project Concern International. BELONG: Project Staffing Chart. September 2007. • Project Concern International. Final CV. Zambian Country Director and Africa Regional Director. September 2007. • Project Concern International. Final CV. Project Manager. September 2007 • Project Concern International. Final CV. Senior Technical Officer for HIV/AIDS. September 2007 • Project Concern International. BELONG: Project Conceptual Diagram. September 2007. • Project Concern International. Potential Training Curricula and Manuals to be Used Under BELONG. September 2007. • Project Concern International. 12 Month Workplan. (October 2007–September 2008). Better Education and Life Opportunities for Vulnerable Children Through Networking and Organizational Growth (BELONG). September 2007. • Project Concern International. Appendix A. BELONG: Project Results Framework. October 2007. • Project Concern International. Appendix B: BELONG Essential Services. November 2007. • Project Concern International. Appendix C: BELONG Reporting Matrix for Reporting Period (October 2006 – September 2007). Country: Ethiopia. October 2007. • Project Concern International. Appendix D: BELONG Project Staffing Chart November 2007. • Project Concern International. Appendix E: Potential Training Curricula and Manuals to be Used Under BELONG in Ethiopia. October 2007. • Project Concern International. Appendix F: Healthcare Quality Improvement Verification Checklist. November 2007. • Project Concern International. Appendix G: Shelter/Protection and Care Quality Improvement Verification Checklist. November 2007. • Project Concern International. Appendix H: Strengthening Capacity/Economic Empowerment Quality Improvement Verification Check List. November 2007. • Project Concern International. Psychosocial Quality Improvement Verification Checklist. November 2007. • Project Concern International. Coordination of Care Quality Improvement Verification Checklist. November 2007. • Project Concern International. Educational and Vocational Training Quality Improvement Verification Checklist. October 2007. • Project Concern International. Food and Nutrition Quality Improvement Verification Checklist. November 2007. • Project Concern International. Annual Report (Ethiopia) Better Education and Life Opportunities for Vulnerable Children Through Networking and Organizational Growth. (BELONG) (October 2006–September 2007). November 2007. TRACK 1.0 OVC PROGRAM EVALUATION 99 • Project Concern International. Ethiopia Project Results Framework. April 2007. • Project Concern International. Appendix B: Essential Services. BELONG Ethiopia Semiannual Report. May 2007. • Project Concern International. Appendix C: BELONG Ethiopia Reporting Matrix. Semiannual Report (October 2006–March 2007). April 2007. • Project Concern International. Appendix D: Staffing Chart. Semiannual Report: (October 2006 to March 2007). May 2007. • Project Concern International. Appendix E: Potential Training Curricula and Manual to be Used Under BELONG in Ethiopia. Semiannual Report (October 2006–March 2007). April 2007. • Project Concern International. Appendix F: BELONG Project Ethiopia Quarterly Reporting Form by Intermediate Result (for IR2 Partners). Semiannual Report (October 2006–March 2007). May 2007. • Project Concern International. Appendix G: BELONG Ethiopia Project Monitoring Checklist (October 2006 to March 2007). May 2007. • Project Concern International. Letter on BELONG Semiannual Report. (October 2006– March 2007). May 2007. • Project Concern International. Semiannual Report (Ethiopia) (October 2006–March 2007). May 2007. • Project Concern International. BELONG Zambia Semiannual Report. (October 2006– March 2007). May 2007. • Project Concern International. Appendix A: Zambia Project Framework. May 2007. • Project Concern International. Appendix B: Essential Services. Zambia Project Annual Report. May 2007. • Project Concern International. Appendix C: BELONG Zambia Reporting Matrix. May 2007. • Project Concern International. Appendix D: Staffing Chart Annual Report: Zambia May 2007. • Project Concern International. Appendix E: Potential Training Curricula and Manual to be Used Under BELONG in Zambia (October 2006–March 2007). April 2007. • Project Concern International. Appendix F: BELONG Zambia Service Record. May 2007. • Project Concern International. Appendix G: BELONG Zambia Goods Received Note. May 2007. • Project Concern International. Appendix H: BELONG Zambia Distribution List. May 2007. • Project Concern International. Appendix I: BELONG Zambia Caretaker Training Sign-in Form Appendix. May 2007. • Project Concern International. Appendix J: BELONG Zambia Activity Plan/Monthly Report. May 2007. • Project Concern International. Appendix K: BELONG Zambia Monthly Service Records. May 2007. 100 TRACK 1.0 OVC PROGRAM EVALUATION • Project Concern International. Appendix L: BELONG Quarterly Service. May 2007. • Project Concern International. Appendix M: Worth Member Caring for OVC. May 2007. • Project Concern International. Semiannual Report BELONG Zambia (October 2006– March 2007). May 2007. • Project Concern International. Appendix A: Results Framework. BELONG Zambia Project. October 2007. • Project Concern International. Appendix B: Essential Services. BELONG Zambia Project. October 2007. • Project Concern International. Appendix C: Reporting Matrix. BELONG Zambia. October 2007. • Project Concern International. Appendix D: Staffing Chart. BELONG Zambia. October 2007. • Project Concern International. Appendix E: Potential Training Curricula and Manuals to be Used Under BELONG in Zambia. October 2007. • Project Concern International. Appendix F: Zambia Worth Member Caring for OVC. BELONG. November 2007. • Project Concern International. Appendix G: Worth Survey Quantitative Data Analysis. BELONG Zambia. November 2007. • Project Concern International. Appendix H: BELONG Zambia Project Monitoring Checklist. November 2007. • Project Concern International. Appendix I: BELONG Zambia District Monitoring, Kalomo and Kafue Districts. October 2007. • Project Concern International. Appendix J: BELONG Zambia District Monitoring Result Summary and Action Plans, Kalomo and Kafue Districts. October 2007. • Project Concern International. Appendix K: BELONG Zambia OVC Movement Form. November 2007. • Project Concern International. Appendix L: BELONG Zambia Annual Report OVC Enrolment Form. November 2007. • Project Concern International. Annual Report Annual Report BELONG Zambia (October 2006–September 2007). November 2007. • Project Concern International. Appendix A: Results Framework. BELONG Ethiopia Semiannual Report. May 2008. • Project Concern International. Appendix B: Essential Services. BELONG Ethiopia. Semiannual Report (October 2007–March 2008). May 2008. • Project Concern International. Appendix C: Reporting Matrix. BELONG Ethiopia Semiannual Report (October 2007–March 2008). May 2008. • Project Concern International. Appendix D: Staffing Chart. BELONG Ethiopia Semiannual Report (October 2007–March 2008). May 2008. TRACK 1.0 OVC PROGRAM EVALUATION 101 • Project Concern International. Appendix E: Potential Training Curricula and Manual to be Used Under BELONG in Ethiopia Semiannual Report (October 2007–March 2008). May 2008. • Project Concern International. Semiannual Report, BELONG Ethiopia (October 2007– March 2008). May 2008. • Project Concern International. Semiannual Report, BELONG Zambia (October 2007– March 2008). May 2008. • Project Concern International. Appendix A: Results Framework. BELONG Zambia Semiannual Report (October 2007–March 2008). May 2008. • Project Concern International. Appendix B: Essential Services. BELONG Zambia Semiannual Report (October 2007–March 2008). May 2008. • Project Concern International. Appendix C: Detailed Workplan. BELONG Project Semiannual Report. (October 2007–March 2008). May 2008. • Project Concern International. Appendix D: Staff Structure. BELONG Zambia Semiannual Report. (October 2007–March 2008). May 2008. • Project Concern International. Appendix E: Potential Training Curricula and Manuals to be Used in BELONG Zambia. May 2008. • USAID Cooperative Agreement No. GPO-A00-05-00013-00. Scaling up Orphans and Vulnerable Children Support in Ethiopia and Zambia. March 2005. • Catholic Relief Services. FY08 Workplan. Support to Orphans and Vulnerable Children Affected with HIV/AIDS in Botswana, Haiti, Kenya, Rwanda, Tanzania and Zambia (October 2007–March 2008). May 2007. • Catholic Relief Services. Semiannual Report. Support to Orphans and Vulnerable Children Affected with HIV/AIDS in Botswana, Haiti, Kenya, Rwanda, Tanzania and Zambia (October 2007–March 2008). May 2007. • Catholic Relief Services. Annual Report Support to Orphans and Vulnerable Children Affected with HIV/AIDS in Botswana, Haiti, Kenya, Rwanda, Tanzania and Zambia (October 2007–September 2008). May 2007. • Catholic Relief Services. FY2008 Budget Support to Orphans and Vulnerable Children Affected with HIV/AIDS in Botswana, Haiti, Kenya, Rwanda, Tanzania and Zambia. November 2007. • Catholic Relief Services. Semiannual Report. Support to Orphans and Vulnerable Children Affected with HIV/AIDS in Botswana, Haiti, Kenya, Rwanda, Tanzania and Zambia (October 2007–March 2008). May 2007. • USAID. Cooperative Agreement No. GPO-A-00-04-00002-00. Support to Orphans and Vulnerable Children Affected by HIV/AIDS (OVC). February 2004. • Catholic Diocese of Solwezi. Strategic Plan 2008–2012. Solwezi, Zambia. March 2008. • Catholic Commission for Development. Policy for Development. Zambia Episcopal Conference. November 1997. • Catholic Relief Services. CHAMP OVC Service Card. Mingu, Zambia. 102 TRACK 1.0 OVC PROGRAM EVALUATION • World Concern Development Organization. Semiannual Report (October 2007–March 2008). Hope for Children Affected by HIV/AIDS in Kenya, Zambia, and Haiti. May 2008. • World Concern Development Organization. Annual Report (October 2006–September 2007). Hope for Children Affected by HIV/AIDS in Kenya, Zambia and Haiti. November 2007. • World Concern. Association of Evangelical Relief and Development Organizations HIV/AIDS Alliance. Cooperative Agreement # GPO-512-A-00-06-00013-00. April 2008. • World Concern Development Organization. Hope for People Affected with HIV/AIDS in Haiti. Annual Report October 2006–September 2007. • World Concern Development Organization. Hope for People Affected with HIV/AIDS in Haiti. PEPFAR FY 07 Program Level Indicators. Annual Report October 2006–September 2007. • World Concern Development Organization PEPFAR Program Level Indicators. (April 2008 Version). April 2008. • World Concern Development Organization. Annual Report (October 2006–September 2007). Hope for Children Affected by HIV/AIDS in Kenya, Zambia and Haiti. October 2007. • World Concern Development Organization. PEPFAR FY07 Program Level Indicators (October 2006–September 2007). • World Concern Development Organization. Semiannual Report (October 2006–March 2007). Hope for Children Affected by HIV/AIDS in Kenya, Zambia and Haiti. May 2007. • USAID. Cooperative Agreement No. GPO-A00-04-00023-00. Support to Orphans and Vulnerable Children Affected with HIV/AIDS (OVC). September 2004. • World Concern Development Organization. Fourth Year Budget. Hope for Children Affected by HIV/AIDS in Kenya, Zambia and Haiti. • World Concern Development Organization. Year 4 (FY08) Annual Workplan. Total (Excel Spreadsheet). Hope for People Affected with HIV/AIDS in Kenya, Zambia and Haiti. September 2007. • World Concern Development Organization. Emergency Plan Annual Workplan Logic Matrix. Fiscal Year 2008. (October 2007–September 2008). September 2007. • World Concern Development Organization. Annual Plan Year 4. (FY08)(October 2007– September 2008). Hope for People Affected with HIV/AIDS in Kenya, Zambia and Haiti. September 2007. For more information, please visit http://www.ghtechproject.com/resources.aspx Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com