MIDTERM EVALUATION REPORT: PEPFAR CARIBBEAN REGIONAL PROGRAM ______________________________________________________________________________________ MARCH 2013 This publication was made possible through the support of the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) through the U.S. Agency for International Development under contract number GHH-I-00-07-00059-00, AIDS Support and Technical Assistance Resources (AIDSTAR-One) Project, Sector I, Task Order 1. MIDTERM EVALUATION REPORT: PEPFAR CARIBBEAN REGIONAL PROGRAM To us, country ownership in health is the end state where a nation’s efforts are led, implemented, and eventually paid for by its government, communities, civil society and private sector. To get there, a country’s political leaders must set priorities and develop national plans to accomplish them in concert with their citizens, which means including women as well as men in the planning process. And these plans must be effectively carried out primarily by the country’s own institutions, and then these groups must be able to hold each other accountable. Secretary of State Hillary Rodham Clinton, June 1, 2012 (PEPFAR n.d.) The authors' views expressed in this publication do not necessarily reflect the views of the U.S. Agency for International Development or the United States Government. AIDS Support and Technical Assistance Resources Project AIDS Support and Technical Assistance Resources, Sector I, Task Order 1 (AIDSTAR-One) is funded by the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) through the U.S. Agency for International Development (USAID) under contract no. GHH-I-00–07–00059–00, funded January 31, 2008. AIDSTAR￾One is implemented by John Snow, Inc., in collaboration with BroadReach Healthcare, EnCompass LLC, International Center for Research on Women, MAP International, mothers2mothers, Social and Scientific Systems, Inc., University of Alabama at Birmingham, the White Ribbon Alliance for Safe Motherhood, and World Education. The project provides technical assistance services to the Office of HIV/AIDS and USG country teams in knowledge management, technical leadership, program sustainability, strategic planning, and program implementation support. Recommended Citation Franco, Lynne Miller, Lyn Messner, Carl Browne, Gillian Mason, and Clinton Sears. 2013. Midterm Evaluation Report: PEPFAR Caribbean Regional Program. Arlington, VA: USAID’s AIDS Support and Technical Assistance Resources, AIDSTAR-One, Task Order 1. AIDSTAR-One John Snow, Inc. 1616 Fort Myer Drive, 16th Floor Arlington, VA 22209 USA Phone: 703-528-7474 Fax: 703-528-7480 E-mail: info@aidstar-one.com Internet: aidstar-one.com iii INDEPENDENT EVALUATION QUALITY REVIEW STATEMENT This evaluation was subject to a meta-evaluation, to judge the quality of the evaluation process and product. Below is the statement from Dr. Robin Lin Miller, Meta-evaluator. To help readers determine whether the evaluation described in the current report is of adequate quality and its findings trustworthy, this evaluation was independently assessed against generally accepted international standards for evaluation quality. Just as an evaluation speaks to the quality of a program or policy, audits of this kind speak to the quality of an evaluation. Establishing the degree to which the current evaluation produced findings that should be used speaks to the importance of this evaluation in the life span of the PEPFAR Caribbean Region Program. Evaluation in contexts like those of the PEPFAR Caribbean Region Program is a particular challenge because the program entails a set of dynamic multisector initiatives operating at multiple levels—individual, organizational, societal, federal, and regional—and proposes a heterogeneous set of outcomes which in combination are expected to lead to reductions in new HIV infections and premature mortality. To the extent that evaluations are both process and product, their ultimate success in improving policies and programs rests on the quality of their execution, the trustworthiness of the information that is produced, and their ability to influence the thinking and actions of those who are in positions to move forward needed changes. An adequate evaluation of PEPFAR’s regional efforts would simultaneously provide an accurate picture of the initiative’s complexity and offer key actors a credible basis for decision making within the context of this dynamic and emergent effort. Stakeholders would be engaged in the evaluation as it unfolded, helping to shape the evaluation’s direction and engage in deliberative discussion of its findings. International standards for evaluation quality set benchmarks for how evaluations are conducted, the reliability and validity of the evidence that they produce, and the strength of reporting, dissemination, and use of evaluation evidence. Prior to conducting my review, I had no prior relationship with the PEPFAR Caribbean Regional Program or the AIDSTAR-One project. To perform this review I had complete access to all evaluation documents and data and to members of the PEPFAR Caribbean Regional Program and evaluation teams. I directly observed evaluation-related activities from the inception of the process through to the drafting of this final report. I observed data collection in two countries and multiple meetings between the evaluators and stakeholders and the evaluators and PEPFAR Caribbean Regional Program Team members, including meetings during which the evidence was discussed and conclusions and recommendations were revised and refined. Through my review, I am able to assess the evaluation’s findings and the means by which those findings were generated. iv In my opinion, the midterm evaluation of the PEPFAR Caribbean Regional Program admirably met the challenge of carrying out evaluation in the complex and less-than-ideal environment of real￾world evaluation. By comparison to the Organisation for Economic Co-Operation and Development’s Development Assistance Committee quality standards (OECD 2010) and in light of the modest budget and compressed timeframe over which the findings had to be produced, the evaluation provides a sound aggregate view of how country, regional, and technical assistance partners view the program’s performance to date and of their hopes for PEPFAR’s legacy in the region. The evaluation incorporated a variety of best practices to facilitate use and systematically collected and analyzed a considerable amount of qualitative data on a tightly scheduled timeline without undermining the integrity of the work. The final report provides a clear window into the breadth and depth of opinion and experience of the program and captures the reflections of the PEPFAR Caribbean Regional Program and stakeholders on the findings. The methods for generating evidence are generally well described and most significant limitations are acknowledged. Though not perfect, as no evaluations ever are, the findings, recommendations, and process that produced them are solid. Having closely observed all stages of the evaluation process, examined the data and data analytic procedures, and reviewed the draft reports, I believe the evaluation is trustworthy and provides the PEPFAR Caribbean Regional Program and PEPFAR Caribbean Regional Program partners a credible basis for considering the best ways to move their partnerships forward in unity. Robin Lin Miller, Ph.D. Professor, Ecological-Community Psychology Michigan State University CONTENTS Independent Evaluation Quality Review Statement.................................................................................................. iii Acronyms........................................................................................................................................................................... vii Executive Summary .......................................................................................................................................................... ix Part 1: Evaluation Purpose, Background, and Methods............................................................................................ 1 Part 2: Evaluation Findings............................................................................................................................................... 7 Part 3: Conclusions and Recommendations for the Way Forward .................................................................... 25 References......................................................................................................................................................................... 35 Appendix A: PEPFAR Caribbean Regional Program Goal Areas ......................................................................... 37 Appendix B: Comments and Feedback from Stakeholders................................................................................... 39 Appendix C: Terms of Reference Midterm Evaluation of the PEPFAR Caribbean Regional Program, January 2012...................................................................................................................................................................... 41 Appendix D: Detailed Evaluation Methodology: Data Collection, Analysis, and Limitations........................ 49 Appendix E: Data Collection Tools............................................................................................................................. 55 Appendix F: Documents Reviewed ............................................................................................................................. 63 Appendix G: List of Agencies and Technical Assistance Partners Active and Present in the 12 Countries .............................................................................................................................................................................................. 67 v vi vii ACRONYMS AIDS acquired immunodeficiency syndrome APR annual performance report CARICOM Caribbean Community and Common Market CDC U.S. Centers for Disease Control and Prevention CHAA Caribbean HIV/AIDS Alliance CHART Caribbean HIV/AIDS Regional Training Network CHLI Caribbean Health Leadership Initiative CHRC Caribbean Health Research Council COP country operational plan CRP Caribbean Regional Program CRSF Caribbean Regional Strategic Framework on HIV/AIDS CSO civil society organization DOD U.S. Department of Defense FSW female sex worker FY fiscal year Global Fund Global Fund to Fight AIDS, Tuberculosis and Malaria HAPU HIV/AIDS Project Unit HHS U.S. Department of Health and Human Services HIV human immunodeficiency virus HRSA Health Resources and Services Administration (U.S. HHS) I-TECH International Training and Education Center for Health MSM men who have sex with men NGO nongovernmental organization NSP national strategic plan OECS Organization of Eastern Caribbean States OGAC Office of the U.S. Global AIDS Coordinator PAHO Pan American Health Organization PANCAP Pan Caribbean Partnership against HIV/AIDS PEPFAR United States President’s Emergency Plan for AIDS Relief viii PF Caribbean Regional HIV/AIDS Partnership Framework PFIP Caribbean Regional HIV/AIDS Partnership Framework Implementation Plan PHCO PAHO HIV Caribbean Office ROP regional operational plan STT senior technical team TA technical assistance TAG technical advisory group UNAIDS Joint United Nations Programme on HIV/AIDS USAID U.S. Agency for International Development USG U.S. Government UWI University of the West Indies ix EXECUTIVE SUMMARY EVALUATION PURPOSE AND CONTEXT The United States President’s Emergency Plan for AIDS Relief (PEPFAR) Caribbean Regional Program seeks to help countries achieve sustainable and epidemic-relevant national HIV programs in the region. This midterm performance evaluation explores the Caribbean Regional Program’s processes and progress toward country ownership and sustainability, and is designed to feed into the Caribbean Regional Program’s, its country partners’, and its technical assistance partners’ planning for the remaining two years to strengthen these efforts. The evaluation team used the emerging conceptualization of country ownership’s four dimensions under PEPFAR II: political ownership and stewardship, institutionalization and community ownership, technical and managerial capability, and mutual accountability. These emerging concepts are ones to which Caribbean Regional Program implementers and partners alike are adjusting. The number and diversity of stakeholders involved in implementing the Caribbean Regional Program’s Partnership Framework sets it apart from other PEPFAR programs. With 12 country governments and two regional entities, the Caribbean Regional Program’s Partnership Framework has the largest number of signatories of any Partnership Framework: Antigua and Barbuda, the Bahamas, Barbados, Belize, Dominica, Grenada, Jamaica, St. Kitts and Nevis, St. Lucia, St. Vincent and the Grenadines, Suriname, Trinidad and Tobago, as well as the Caribbean Community’s Pan Caribbean Partnership against HIV/AIDS (PANCAP) and the Organization of Eastern Caribbean States’ (OECS) HIV/AIDS Program Unit. The Program also engages six U.S. Government (USG) agencies and departments (Centers for Disease Control and Prevention [CDC], U.S. Department of Defense [DOD], Health Resources and Services Administration [HRSA], Peace Corps, U.S. Department of State, 1 and the U.S. Agency for International Development [USAID]), each with its own mandate and technical assistance mechanisms. EVALUATION METHODS The evaluation design is grounded in systems thinking, appreciative inquiry, utilization-focused, participatory, and developmental approaches. The evaluation team conducted a document review, a group data collection session at the June 2012 PEPFAR Portfolio Review Meeting (with 25 country counterpart and technical assistance partners), and 153 semi-structured interviews. Interviews were conducted during field visits to five countries: Antigua and Barbuda, Barbados, Jamaica, the Bahamas, and Trinidad and Tobago, and by phone with additional stakeholders from the remaining seven Partnership Framework countries, PANCAP, OECS, Caribbean Regional Team members, international agencies, and regional and international technical assistance partners. 1 Including six U.S. Embassies, led by the U.S. Embassy in Barbados. x FINDINGS AND CONCLUSIONS RELEVANCE AND RESPONSIVENESS TO COUNTRY AND REGIONAL PARTNERS’ NEEDS The collaborative approach used in developing the Partnership Framework provided a strong start in fostering all four dimensions of country ownership by engaging key stakeholders at all levels of the response (mutual accountability, institutional and community ownership), aligning the Framework with the Caribbean Regional Strategic Framework on HIV and AIDS and national strategic plans (political ownership and stewardship), and identifying gaps not filled by other donors (capabilities). All countries and the Caribbean Regional Program remain committed to the Partnership Framework. Stakeholders recognize PEPFAR’s important contributions to their ability to respond to the epidemic, specifically by increasing their knowledge of the epidemic; building laboratory capacity; providing management and leadership training with CHART and CHLI (Caribbean HIV/AIDS Regional Training Network and Caribbean Health Leadership Institute); reaching key populations; and reducing stigma and discrimination. The Framework includes many of the areas that countries feel are priorities for greater sustainability and ownership of their HIV response, and even most “additional” priority areas identified by stakeholders, such as integration, are already included under the Partnership Framework. The evaluation findings indicate some disconnect between country and the Caribbean Regional Team perspectives about how their partnership can play out. PEPFAR, as a U.S. funding mechanism, has operational constraints in certain areas that limit how funds can be spent, and considerable flexibility in other areas. There is limited understanding at the country level about how much flexibility exists for country counterparts to discuss, amend, or change the annual work plans to meet their priority needs. More communication is needed to address the incongruity between countries’ impressions of PEPFAR as predetermining activities and management without adequate country input, and the Caribbean Regional Team’s desire for countries to take the lead, and to use Caribbean Regional Program resources to best support country efforts. BUILDING COUNTRY AND REGIONAL CAPACITY FOR SUSTAINABILITY AND OWNERSHIP The Caribbean Regional Program has brought concrete technical assistance and capacity building to the region, and enhanced countries’ abilities to manage their HIV programs and resources through training and technical assistance in the areas of prevention, strategic information, laboratories, and health systems strengthening. This capacity enhancement crosses multiple sectors: government (ministries and defense forces), civil society, nongovernmental organizations (NGOs), and to a lesser extent, the private sector. Stakeholders recognize that training alone is insufficient to ensure sustainable capacity. The dimensions of institutional and country ownership and mutual accountability call for more organizational capacity building of civil society and mentorship of government to allow them to play their roles more effectively. The Caribbean Regional Program currently lacks both a shared articulation of country ownership among all partners, and a method for monitoring and measuring progress toward ownership and sustainability. Building capacity for greater country ownership and sustainability is not a quick process, and requires clear goals, strategies, and metrics to stay focused. The Partnership Framework Implementation Plan and country work plans can play a critical role in this process as a road map for xi transition—they are intended to be flexible, responsive documents, and there is space and time to modify them to meet countries’ and regional partners’ changing needs for capacity building in service of country ownership and sustainability. All countries are concerned about sustainability. While countries are taking steps to sustain their HIV response, these steps are not always systematically developed, or clearly linked to support which they do or could get from the Caribbean Regional Program. The success of these emerging measures and systems is important for the development of political ownership and sustainability. Many respondents across stakeholder groups suggested that the Caribbean Regional Program should conduct focused discussions with governments to develop transition plans in a way that builds political will and takes realistic steps within the current economic climate. However, the term “transition” is often construed to relate only to funding issues. APPROPRIATENESS OF THE CARIBBEAN REGIONAL PROGRAM’S “REGIONAL” APPROACH The Caribbean Regional Program’s regional approach, encompassing country-specific efforts as well as economies of scale and regional goods and services, is very appropriate for the Caribbean. Yet the regional nature of the Partnership Framework and the country specificity it allows are not always implemented as intended, nor are these always clear to all stakeholders. The Caribbean Regional Team needs to articulate adequately the regional approach by explaining to countries, regional entities, and technical assistance partners where and how they fit in the regional model and, more importantly, their contributions to building country ownership and sustainability. Each country wants to get the maximum benefit and to have the approach tailored to its own situation. The Caribbean Regional Program allows some access for a wider range of smaller countries that would not necessarily have been afforded an individual PEPFAR program, and has generally avoided an approach that is uniform across countries and regional entities. CARIBBEAN REGIONAL PROGRAM COORDINATION AND COLLABORATION FOR COUNTRY OWNERSHIP The Caribbean Regional Program’s structure and operation have both facilitated and hindered its ability to work effectively to build capacity and sustainability for country ownership at the country and regional levels. It is a complex program, requiring coordination and collaboration across six agencies, 22 technical assistance partners (averaging 15 per country), 12 countries, and two regional entities. Given this context, effective coordination, both internal and external, is critical to achieving mutual accountability and effective functioning. Currently, coordination at the country level among country partners, between USG and country partners, and across implementing agencies is not yet optimal. USG agencies have operated in a “silo” approach, contributing to a lack of cohesion among USG agencies and technical assistance partners and resulting in some confusion and feelings of disempowerment among many country and regional partners. However, countries, technical assistance partners, and the Caribbean Regional Team all expressed the need and desire for a “One PEPFAR” approach. Current Caribbean Regional Program structures provide a frame for coordination and collaboration, but there is a need for more streamlined management and more accountability for unity. There is a need to strengthen country-level coordination and for each country to take greater leadership in coordinating its own efforts, including decisions about how Caribbean Regional xii Program resources are used, so that countries can take advantage of flexibility where it exists and collaborate with USG agencies to determine how best to work within the Program’s limitations. There appears to be a desire among the Caribbean Regional Team to improve working relationships, to engage with greater respect and improved dynamics, all of which are key to moving forward. The activation of the Senior Technical Team and the arrival of the U.S. Ambassador to Barbados augur well for improvement. These developments are too nascent for the evaluation to assess, but they are steps in the right direction. The Caribbean Regional Program has not been able to make full use of the stakeholder structures, the Steering Committee, and the Technical Advisory Group, and engagement among members appears limited. However, these structures offer potential for fostering greater ownership and sustainability. A final major issue is the considerable under-utilization of funds. PEPFAR and countries both have a role to play in removing bottlenecks to ensure greater impact on the HIV response. RECOMMENDATIONS Recommendations in this report provide specific guidance to the Caribbean Regional Team, other PEPFAR actors, participating national and regional partners, and technical assistance partners to accelerate efforts to reach sustainable, country-owned, epidemic-relevant country and regional HIV responses in the Caribbean. The translation of this guidance into action plans remains the responsibility of the Caribbean Regional Team and its partners. While the full report contains many recommendations, those of the highest priority are summarized here (in priority order): • The Caribbean Regional Program should clearly articulate and communicate the Caribbean Regional Program’s “regional” approach and theory of action for country ownership and sustainability. The Program should create a framing document (with visuals) to convey clearly what the “regional” approach means for the Program, how all the Program parts fit together, and the roles and responsibilities of countries, regional entities, and technical assistance partners within this approach. This articulation must also include a theory of action that delineates the pathways through which program actions lead toward country ownership and sustainability. • The Caribbean Regional Program and countries should develop country-specific and regional entity-specific ownership/sustainability strategies and plans. Countries need to develop clear plans that move them along the trajectory toward greater country ownership and sustainability, and these plans need to be based on agreement on definitions and dimensions of country ownership. These plans should include specific country-ownership short-term, medium￾term, and long-term goals; country-led strategies for achieving those goals; PEPFAR-funded support to achieve country ownership goals; other (non-PEPFAR) support to achieve these goals; and metrics for monitoring progress. The Caribbean Regional Team, with country and regional partners, should build aspects of these ownership/sustainability plans into the fiscal year (FY) 2013 and FY 2014 operational plans. • The Caribbean Regional Team should create a “One PEPFAR” voice and branding, and conduct joint agency visits whenever possible. Discussions with country and regional partners should clearly articulate what is PEPFAR-funded and what is not, and these discussions should examine the possibility of synergies that are not yet operationalized. xiii • The Caribbean Regional Program, countries, and regional entities should regularly assess government and Caribbean Regional Program progress toward commitments under the Partnership Framework and its Implementation Plan, and determine actions needed in case of non-compliance. • The Caribbean Regional Program should intensify opportunities to shift technical assistance responsibilities to regional and country-level technical assistance partners, while providing capacity building as needed to facilitate this transfer. • The Caribbean Regional Program should reshape the current portfolio to provide stronger support to civil society organizations (CSOs) so that they can play a greater role in national and regional program decision making and implementation within the HIV response. This includes examination of the Program’s arsenal of mechanisms and partners to provide more capacity building to CSOs for organizational strengthening and policy advocacy, with the result that members of key populations are present at key decision-making fora and their needs included in national HIV/AIDS policy agenda. • The Caribbean Regional Program, countries, and regional entities should jointly take responsibility for developing annual operational plans, with countries leading the identification of priorities. Quarterly country work-plan review meetings, facilitated by the Caribbean Program Team, should maximize in-person or virtual platforms (such as Adobe Connect). These meetings should be focused, with a substantive agenda, and provide a safe space for open discussion about successes, challenges, and action steps to address challenges and build on successes. • The Caribbean Regional Program and countries should require all external technical assistance partners to develop explicit capacity building plans related to country-level activities to ensure skill and capacity transfer to host-country nationals. xiv 1 PART 1: EVALUATION PURPOSE, BACKGROUND, AND METHODS INTRODUCTION EVALUATION PURPOSE AND OBJECTIVES The PEPFAR Caribbean Regional Program2 commissioned this midterm performance evaluation to “identify areas in which the initiative shows a strong likelihood of building a sustainable HIV-related infrastructure in the region, areas in need of improvement, and factors that are contributing to or impeding progress…Of particular interest is the identification and description of factors promoting country ownership and sustainability” (PEPFAR CRP 2012). This evaluation serves both accountability and learning functions, with the following specific objectives: • Ascertain if the PEPFAR Caribbean Regional Program3 is helping countries reach sustainable, epidemic-relevant national AIDS program models, • Identify key factors contributing to or impeding program results, • Make recommendations for program adjustments. This evaluation focused on program-wide efforts related to fostering country ownership and sustainability. It was not commissioned nor designed to evaluate specific country-level efforts or specific results within the program’s technical goal areas. The midterm evaluation results are intended for use primarily by the PEPFAR Caribbean Regional Program and its national and regional stakeholders and technical assistance partners. The timing for this evaluation was planned to coincide with the 2013 Regional Operational Plan (ROP) and to inform the remaining two years of the Caribbean Regional HIV/AIDS Partnership Framework. CARIBBEAN REGIONAL PROGRAM CONTEXT AND DESCRIPTION This evaluation explores the processes and progress toward country ownership and sustainability, and readers should consider the evolving conceptualization of country ownership within PEPFAR itself when examining the evaluation results. Accelerating local ownership of HIV/AIDS programs is the central goal of PEPFAR II, “to promote country ownership, and increase sustainability of national health programs” (PEPFAR 2011b). The focus on country ownership reflects a shift from an emergency response orientation under PEPFAR I. PEPFAR II emphasizes sustainability, which is directly linked to country ownership (PEPFAR 2009b), seeking to create programs that are 2 Funding for this evaluation was channeled through the AIDSTAR-One project, which is funded by PEPFAR through USAID. 3 “PEPFAR” refers to the global program; “Caribbean Regional Program” refers to PEPFAR-funded projects and activities in the region. 2 country owned and country driven, addressing HIV/AIDS within the broader health and development context, and building on PEPFAR strengths and increasing efficiencies. Country ownership is both an iterative process and an evolving concept. Supporting country ownership includes country-level change to local institution leadership, joint conceptualization in program design and planning, and clear measurements for this change over time. The Caribbean Regional HIV/AIDS Partnership Framework development process—focusing on country ownership, sustainability, and support for country coordination of resources and USG interagency collaboration (PEPFAR 2009a)—sets the stage for partnership and mutual commitments. While country ownership has always been a key concept of PEPFAR II, PEPFAR did not outline its key dimensions of country ownership until it issued its FY 2012 Country Operational Plan (COP) Guidance (PEPFAR 2011b); these dimensions are described in Box 1 below. Box 1. PEPFAR’s Four Dimensions of Country Ownership* 1. Political ownership and stewardship: Government has a vision for HIV support, and works with civil society, the private sector, and other funding partners to clearly articulate its priorities and plans for program development, and has oversight of specific activities conducted by all stakeholders. 2. Institutional/community ownership: Local institutions (e.g., governmental and civil society) own final decisions for each program stage, and have responsibility for programs and fund management. 3. Technical and managerial capability: Country leadership has the technical and management capabilities to oversee programs, and make adjustments and shifts over time. Planning must be deliberate, in order to provide ample opportunities for local capacity to perform activities. 4. Mutual accountability, including finance: A strong sense of defined accountability exists between governing bodies and the citizens it serves, as well as between country leadership and donors for measurable results over time. Information and processes are transparent, and civil society has mechanisms for providing input and feedback. Roles and responsibilities are explicit. * Adapted from PEPFAR 2011b. The Caribbean Regional Program implementers and partners alike are adjusting to and grappling with these emerging concepts, in the face of resource constraints among many country partner governments. In PEPFAR’s conception, country ownership does not equate with financial ownership, although financing is included in the fourth dimension. The USG has been active in the Caribbean region for several decades, but PEPFAR began investing funds in FY 2010 under the PEPFAR Caribbean Regional HIV/AIDS Partnership Framework (henceforth referred to as the Partnership Framework) (2010-14). The Partnership Framework is an agreement between the USG and Caribbean partners “to establish strategic alignment in the fight against HIV/AIDS through joint decision-making in setting programmatic priorities and partner contributions” and by combining USG and Caribbean regional and national resources (PEPFAR CRP 2009). The Partnership Framework was developed through a participatory process that included individual country consultations with all signatories: 12 participating countries4 and two regional entities. 5 The Partnership Framework was designed to be fully aligned with the existing Caribbean Community (CARICOM) PANCAP Caribbean Regional Strategic Framework on HIV 4 Antigua and Barbuda, the Bahamas, Barbados, Belize, Dominica, Grenada, Jamaica, St. Kitts and Nevis, St. Lucia, St. Vincent and the Grenadines, Suriname, and Trinidad and Tobago. 5 The Caribbean Community on behalf of its Pan Caribbean Partnership against HIV/AIDS (PANCAP) and the OECS on behalf of its HIV/AIDS Program Unit (HAPU). 3 and AIDS 2008-2012 (CRSF) (CARICOM 2008), and to focus on areas complementary to those addressed by other HIV/AIDS donor support in the region.6 The Partnership Framework provides for technical assistance to national and regional entities in HIV prevention, strategic information, laboratory strengthening, and health systems strengthening to improve the capacity of governments and civil society to implement an effective and sustainable HIV response (see Appendix A). These goal areas are further delineated in the Partnership Framework Implementation Plan (PFIP), which outlines specific priorities, approaches used, funding levels, and indicators of success (PEPFAR CRP 2010). Caribbean countries cooperate in many regional fora and organizational structures, while retaining individuality and independence. Figure 1 presents a map of the 12 Partnership Framework signatory countries. The two regional signatories, PANCAP and the OECS HIV/AIDS Project Unit (HAPU), are based in Guyana and St. Lucia, respectively. Further adding to the complexity is the range across countries in terms of size, cultures, economies, capacities, and HIV epidemics. Figure 1. Countries Participating in the PEPFAR Caribbean Regional Program in FY 12 6 The Global Fund to Fight AIDS, Tuberculosis and Malaria (Global Fund), the World Bank, the Joint United Nations Programme on HIV/AIDS (UNAIDS), the William J. Clinton Foundation (Clinton Foundation), the German Development Bank (KfW), and the Pan American Health Organization HIV Caribbean Office (PAHO/PHCO) all have activities in the countries that receive PEPFAR funding or technical assistance. 4 The intricacy in the number and diversity of stakeholders involved in implementing the Partnership Framework sets it apart from other regional PEPFAR programs. The Caribbean Regional Partnership Framework has the largest number of signatories: 12 country governments, two regional entities, and six USG agencies. Each USG agency has its own mandate and its own mechanisms for providing technical assistance, as follows: • The U.S. Department of Health and Human Services (HHS) Centers for Disease Control and Prevention (CDC) has the largest team among USG agencies. CDC works through cooperative agreements with ministries of health and PANCAP, and provides technical assistance through its staff and partners. CDC has staff in Barbados, Trinidad and Tobago, and Jamaica. 7 • The U.S. Agency for International Development (USAID) has teams in Barbados and Jamaica, and works through technical assistance partners with a combination of centrally funded mechanisms and bilateral projects, including a cooperative agreement with the Ministry of Health in Jamaica and PANCAP. • The Peace Corps provides human resources and small grants for HIV activities through its Volunteers in nine countries (three OECS country programs and Suriname will be closed soon), has a Regional HIV Coordinator based in Antigua and Barbuda, and offices in Belize, Jamaica, and Suriname. • The U.S. Department of Defense (DOD) has four dedicated staff, two of whom are based in the region (Barbados and Jamaica), works only with the defense forces, and provides technical assistance through partners. • The HHS Health Resources and Services Administration (HRSA) has no permanent staff in the region, uses regional and international technical assistance partners, and has a cooperative agreement with the University of the West Indies (UWI). • The U.S. Department of State (State) has embassies in six of the 12 countries,8 a three-person Coordinator’s Office in Barbados, and provides technical assistance through staff and small grants. EVALUATION DESIGN AND METHODOLOGY This midterm evaluation was designed as a learning and strategic planning tool for the Caribbean Regional Program and its stakeholders. The evaluation design is grounded in systems thinking, appreciative inquiry, utilization-focused, participatory, and developmental approaches: • A logical, stepwise process with built-in checkpoints for using initial findings to inform subsequent data collection and processing; • A highly participatory process, with regular consultation and communication with the Caribbean Regional Team and its stakeholders to ensure transparency and engagement; • A mixed-methods approach including a desk review, key stakeholders interviews, visits to five countries, and group data collection during the June 2012 Portfolio Review Meeting 7 CDC currently has a vacant position in the Bahamas. 8 The Embassy in Barbados serves the OECS countries as well, but only Grenada has a consulate. 5 • An appreciative inquiry approach that identified aspects of the Caribbean Regional Program in which alignment of goals and efforts brought significant benefit to country and regional HIV responses. The evaluation team was comprised of two international and two Caribbean evaluators who brought significant expertise and knowledge of PEPFAR and the region. The findings, conclusions, and recommendations represent the full views of all evaluation team members. EVALUATION DESIGN PROCESS The evaluation team engaged the Caribbean Regional Team and other key stakeholders throughout the evaluation process to ensure that the evaluation findings and report would be as useful as possible. The design phase included a one-day meeting in Barbados (May 9, 2012) with the Caribbean Regional Team to clarify and focus the evaluation, phone, and email interviews with additional Caribbean Regional Team members, and phone interviews with five Technical Advisory Group (TAG) members representing the range of stakeholder groups (governments, regional entities, civil society). Regular updates were provided to the Caribbean Regional Team through four conference calls, from June to August, and an in-person interim debriefing (June 22, 2012). The evaluation team organized opportunities for the Caribbean Regional Team to provide feedback on the draft report by phone (October 5, 2012) and in person in Barbados (October 18, 2012). For the broader stakeholder community, the evaluation team presented results at the PANCAP Annual General Meeting in Belize (October 26, 2012) and facilitated a virtual meeting using Adobe Connect software (December 7, 2012) with representative stakeholders who reviewed the draft report to discuss their feedback. Stakeholder comments additional to those already captured in the report text can be found in Appendix B. EVALUATION THEMES AND KEY QUESTIONS This midterm evaluation, conducted between May and November 2012, focused on the first two and a half years of the Caribbean Regional Program (FY 2010-12) to assess achievements and the potential to build capacity for sustainability and country and regional ownership. Table 1 outlines the key themes and original evaluation questions (as they are stated in the Terms of Reference [see Appendix C] and the evaluation design), as well as the refocused questions used in presenting the findings. Country ownership is a cornerstone of PEPFAR II, but definitions and dimensions have been evolving globally, and currently there is no guidance on metrics. The Caribbean Regional Program did not have a conceptual framework from PEPFAR for country ownership when the Partnership Framework was developed in 2010. With no formulated development hypothesis or logic model for country ownership for the Caribbean Regional Program, the evaluation team developed a framework for this evaluation by integrating the information on country ownership in the FY 2012 COP Guidance (PEPFAR 2011b) with the Caribbean Regional Program goals. 6 Table 1. Evaluation Themes and Questions Evaluation Theme Original Evaluation Question Refocused Evaluation Question Meeting the needs of country and regional partners In what ways is the PEPFAR Caribbean Regional Program helping country and regional partners advance toward the objectives in the Caribbean Regional Strategic Framework on HIV and AIDS? How relevant and responsive is the content of the Partnership Framework (as it is being implemented) to the needs of country and regional partners in achieving progress in combating their HIV epidemics? Building country and regional capacity and ownership To what extent is the PEPFAR Caribbean Regional Program helping countries understand and “own” their HIV epidemics and develop and sustain relevant and cost￾effective programs? How does the Caribbean Regional Program approach to building capacity and fostering ownership move the region closer to sustainability post-PEPFAR? Appropriateness of the regional model How can the regional approach be improved? Is the Caribbean Regional Program’s regional model appropriate and functional for creating synergies and economies of scale in the region? PEPFAR Caribbean Program functioning What organizational aspects of the Program are well implemented and which require improvement? How are the Caribbean Regional Program structures and processes for coordination and collaboration facilitating or hindering the achievement of country ownership? DATA COLLECTION, SAMPLING, AND ANALYSIS The evaluation team collected primary qualitative data using a variety of methods: document review, semi-structured interviews, and a group data collection session held at the June 2012 Caribbean Regional Program Portfolio Review Meeting. The evaluation team conducted semi-structured interviews with 105 individuals during field visits to 5 of the 12 Partnership Framework countries: Antigua and Barbuda, Barbados, Jamaica, the Bahamas, and Trinidad and Tobago. Phone interviews were conducted with 48 additional stakeholders, including representatives from the seven Partnership Framework countries not visited, PANCAP and OECS, Caribbean Regional Team members, international agencies, and regional and international technical assistance partners. Twenty-five country counterparts and technical assistance partners participated in the June 2012 Portfolio Review Meeting group data collection session. Content and thematic analyses were done using Dedoose software, version 4.3 (2012). Details on data collection processes, the sample, and the analysis approach can be found in Appendix D. Data collection tools are available in Appendix E, and a list of documents reviewed in Appendix F. To protect confidentiality, names of interviewees and specific references to individual countries are not included, and all interviewees were informed that they might be quoted in the report, but without identifiable information. LIMITATIONS TO THIS EVALUATION The evaluation encountered the following limitations: lack of baseline information on the level of country ownership and sustainability before the Caribbean Regional Program; possible civil society bias due to limited interviews with CSOs outside of those that are participating or have participated in the U.S. Ambassadors Small Grants Program; the difficulty some respondents had in differentiating between PEPFAR-funded and other donor-funded activities; and the possible reticence of some individuals to speak out fully for fear of reprisal. More details on limitations are available in Appendix D. 7 PART 2: EVALUATION FINDINGS This section presents the evaluation findings, organized around the four main evaluation questions (or lines of inquiry). These findings are discussed in terms of PEPFAR’s four dimensions of country ownership, but no baseline information existed to measure changes related to country ownership and sustainability. Findings are presented in relation to explicit expectations laid out in PEPFAR documents, and implicit expectations shared by respondents. Quotes are used to illustrate key points and messages, reflecting a range of respondents and frequently stated views, unless otherwise noted. The key findings reflect common themes across countries, unless otherwise noted. RELEVANCE AND RESPONSIVENESS OF THE CARIBBEAN REGIONAL PROGRAM TO COUNTRY AND REGIONAL PARTNER NEEDS The central focus of the Partnership Framework is to contribute to the region’s vision of substantially reducing the spread and impact of HIV as articulated in the CRSF. This will be achieved by providing technical assistance to expand partner countries’ and regional entities’ capacity to plan, oversee, and manage national responses to HIV so they can deliver quality services in partnership with civil society, the private sector, and other entities. Box 2 summarizes findings regarding the relevance and responsiveness of the Caribbean Regional Program to the needs of countries and regional partners detailed in this section. Box 2. Key Messages: Partnership Framework Responsiveness and Program Relevance to the Needs of Country and Regional Partners Responsiveness of Partnership Framework/Partnership Framework Implementation Plan • The Partnership Framework is effectively aligned with the CRSF and countries’ HIV National Strategic Plans; the development process was consultative, and all countries remain committed to the Partnership Framework goals. • Countries perceive the PFIP process as predetermined and not allowing for sufficient country specificity; many countries are not using the PFIP as a management and planning tool for their activities. • While countries may not recognize the flexibility of the Partnership Framework and PFIP, these documents were intended to be revisited annually to ensure alignment with country-specific goals and objectives. Relevance of the Caribbean Regional Program to Country Needs and Remaining Priorities • Stakeholders recognize the important contributions of the Caribbean Regional Program to their ability to respond to their epidemic: enhanced knowledge of the epidemic; strengthened laboratory capacity; management and leadership training; improved outreach to vulnerable populations; and reduced stigma and discrimination. • Additional priorities cited by stakeholders are already included in the Partnership Framework, and indicate yet-unaddressed components. 8 RESPONSIVENESS OF THE PARTNERSHIP FRAMEWORK AND PFIP TO COUNTRY AND REGIONAL NEEDS AND ALIGNMENT WITH THE CRSF AND NATIONAL STRATEGIC PLANS Fostering political ownership and stewardship requires shared purpose and vision. The Partnership Framework development process was designed to create this. The evaluation document review and interviews indicate that the Partnership Framework was developed through an extensive consultative process with high-level government representatives, PANCAP and OECS officials, and regional technical assistance partners. At least one Ministry of Health respondent from each country affirmed their country’s support to the Partnership Framework and to fulfilling their negotiated responsibilities outlined therein. It appears, however, that some benefits of that consultative process have been lost. Approximately 20 percent of respondents reported that they had not seen the Partnership Framework and PFIP, or had little or no knowledge of their existence. Among this cohort, lack of knowledge was highest among regional technical assistance partners and CSOs, followed by international technical assistance partners and government employees (see Figure 2). Several respondents remarked that many who had participated in the consultations were not involved in implementation, or had exited the system without adequately briefing their replacements. As a result, the documents did not filter down to those at the operational level, leading to an inconsistent understanding of the Partnership Framework and PFIP by national and regional partners. The data indicate, however, that staff turnover is unlikely to be the main explanatory factor for this gap in understanding. Figure 2. Awareness of the Partnership Framework and PFIP among non-USG Respondents An interactive process, similar to that used in developing the Partnership Framework, was used for developing the PFIP, with key administrative and technical ministry of health staff and CSOs, via national consultations convened in each participating country. Many key country representatives remarked that they felt the scope of the PFIP was largely predetermined by PEPFAR, and they did not feel that its objectives always coincided with national priorities then and now: They already had a grid for what the USG is doing and who would do it. They just wanted to hear that we would do our part. We never went into a philosophical conversation about where we are as a country in our HIV response, what we need and where we want to go. —Government Respondent #15 9 Some government officials remarked that descriptions of USG contributions outlined in the PFIP are not specific enough, and therefore created a level of uncertainty about the activities supported. National AIDS Program Managers reported that in many cases, they were unsure about the precise activities being undertaken on behalf of or in their countries and which organizations were performing the work. The Partnership Framework deliberately built on the CRSF, and respondents reported a high level of alignment between the Partnership Framework’s strategic objectives and priority technical areas of the preexisting CRSF. The Caribbean Regional Program originally intended the Framework to be updated as needed. However, very few country respondents suggested revising it or the PFIP, even though many countries and USG agencies stated that countries’ and regional partners’ capacity and focus have shifted to some degree since the Partnership Framework and PFIP were developed. When we wrote that document, we assumed it met country needs, but now there may be shifting priorities in countries. The people involved in developing the Partnership Framework may no longer be there. We may need to shift the goalposts. —PEPFAR Respondent #5 RELEVANCE OF THE PARTNERSHIP FRAMEWORK TO COUNTRY AND REGIONAL NEEDS Figure 3. Significant Changes Reported (from Most- to Least-Mentioned) Most respondents (85 percent), when asked about the most significant changes in their country or region regarding HIV/AIDS over the past two years, spoke of positive changes. Figure 3, above, presents the most frequently cited significant changes, across all stakeholder groups, in descending order. Those speaking of negative changes referred mostly to lack of adequate progress in meeting the needs of key populations and other vulnerable groups. In terms of the Caribbean Regional Program’s specific contributions, all national and regional stakeholders spoke of increasing capacity in reducing the spread of HIV and mitigating the impact of AIDS. Overall, the Caribbean Regional Program’s efforts to align HIV responses with the epidemic stand out, especially the focus on key populations and effective strategies to reach them. As discussed below, respondents most frequently cited the following as important Caribbean Regional Program contributions to these changes: helping countries better understand their epidemic; strengthening laboratories; training health providers in leadership and management; reaching key populations; and reducing stigma and 10 discrimination. In addition, many interviews spontaneously brought up the need for more attention to gender inequities. Helping Countries Better Understand Their Epidemic: Evidence-based understanding of the nature of the HIV epidemic in each country has been a gap in the region. Several government representatives remarked that the Caribbean Regional Program has provided equipment, training, staff support, and technical assistance to strengthen national health information systems as well as supported studies9 that help define the characteristics of the country’s HIV epidemic. This support has enhanced data gathering and analysis, providing better strategic information to guide evidence￾based interventions. Over the past two years, there has been a very significant re-definition of [our] health information system that allows for gathering strategic information on HIV. I think that we are in a better position to define the epidemiological features of HIV and so streamline our responses in a more effective way. —Government Respondent #8 Strengthening Laboratories: Government officials in all 12 countries reported that the Caribbean Regional Program played a critical role in strengthening country-level laboratories and related services and developing a functional regional reference laboratory. Specific areas cited include training of laboratory technicians, provision of equipment and training in related maintenance, provision of supplies, infrastructural development, and technical assistance in improving laboratory standards toward accreditation. Barbados presents the most striking example of support for laboratory strengthening in the region. Construction of a new laboratory is planned, staff support and training are being provided for existing laboratories, and technical assistance is being extended for achieving international levels of accreditation. [The] lab strengthening contribution to [the] national government is tremendous, and it does not focus only on HIV…We have been able to strengthen labs generally—we now serve others in the region in molecular diagnosis for TB and early infant diagnosis of HIV. —Government Respondent #2 Training Health Providers in Leadership and Management: Although not always identified immediately as a Caribbean Regional Program–supported activity, in almost every case, government officials hailed the Caribbean HIV/AIDS Regional Training Network (CHART) and the Caribbean Health Leadership Initiative (CHLI) projects as meeting key capacity building needs in the region. This sentiment was particularly emphatic among OECS countries, which have less internal capacity and opportunities for capacity development than larger countries. Reaching Vulnerable Populations: One of the biggest needs in the Caribbean is improving access to HIV prevention, care, and treatment for key populations. The Caribbean Regional Program, based on the HIV epidemic in the region, focuses its prevention efforts on men who have sex with men (MSM), female sex workers (FSWs), and the military. In many countries, the Caribbean Regional Program is the sole entity providing consistent support to these populations. A wide cross section of respondents described how Caribbean Regional Program support has helped countries to increase or improve their ability to reach and involve key populations, increase access to rapid testing and counseling, and increase awareness of key populations among the general population. 9 Several respondents noted delays in getting these studies started, and not always understanding all the factors behind the delays. Nevertheless, five studies in Jamaica and two in Barbados have been completed. 11 PEPFAR has helped us to redefine the vulnerable groups—MSM, FSWs, poverty and responsible sexual behavior—and a greater sense of empowerment among vulnerable groups. For example, [one key CSO in our country] is now asking to be at the table. Before they did not want to come forward, but empowerment fueled and nurtured these groups…PEPFAR resources that have helped create this enlightened environment. —Government Respondent #9 Regional entities, government officials in Barbados, and all (without exception) OECS countries spoke of the Caribbean HIV/AIDS Alliance’s (CHAA’s) contribution to reaching vulnerable populations, increasing access to services, creating activities with key populations, and helping Ministries of Health bridge the gap in reaching these groups. At least one non-OECS country asked for assistance from CHAA. Our government does not have the moral fortitude to say, “Let’s address this.” So CHAA has really been able to get MSM to get tested and then access to services if needed…In fact, more MSM were tested and knew about HIV and AIDS than heterosexual men! —Government Respondent #28 Many government and civil society respondents felt that their priority vulnerable populations include youth and migrants, and expressed a desire for PEPFAR-funded support to these populations. While Caribbean Regional Team members may agree that youth and other groups are key vulnerable populations and have in some cases petitioned to expand their projects to these groups, use of PEPFAR funds in the Caribbean remains restricted to MSMs, FSWs, and the military. Addressing Gender Inequity: The evaluation questions and interview guides did not ask directly about the PEPFAR Gender Strategy (PEPFAR 2011a). However, it is mandatory for all PEPFAR programs to integrate gender equity, and many respondents spoke spontaneously of unaddressed gender norms and inequities that impede the HIV response. The majority of comments (29 percent) were from government representatives, followed closely by civil society respondents (27 percent). The comments fell into three areas: gender-based violence, reaching men, and cervical cancer: It is a triple whammy: sex work, HIV, squatter. Women may be prone to violence from their husbands and the police. Gay men migrate to urban areas for anonymity and become homeless. Migrant plus MSM, sex worker, drug user is much more difficult. A sex worker who has been battered can’t go to the police to report that her client beat her and took her money. So it is layered. —CSO Respondent #1 Reducing Stigma and Discrimination: Respondents spoke of the Caribbean Regional Program’s contributions to reducing stigma and discrimination, which remain key barriers to the HIV response in the region. These contributions included supporting the defense forces’ use of alternative testing strategies that allow for confidentiality. All respondents remarked that much still needs to be done to remove HIV-related stigma and discrimination and that they envisage continued and enhanced assistance from the Caribbean Regional Program in this area. We still have a long way to go…Stigma and discrimination has reduced, it isn’t as bad as it was, but it’s still there. PEPFAR has helped….Now that [those infected or at risk to be] see there is genuine help they are becoming more open. —Government Respondent #14 Some respondents noted that countries could address stigma and discrimination by moving to a rights-based approach and strengthening advocacy and policy initiatives to influence the policy and legislative environment, such as laws related to workplace policies, sodomy, and others that affect key populations and people living with HIV (PLHIV). Respondents also felt that PEPFAR could facilitate movement in that direction. Civil society respondents concerned with both the inadequate HIV response and lack of progress changes to policy and legislation. 12 Technocrats are clear where the vulnerable populations have to be placed. Above the [technocrats], however, there is a glass ceiling. A lot of what needs to be done is still stopped at that level. Policy and legal issues need to be dealt with. Talk as much as you want to, but we are not going anywhere with the epidemiology if we do not deal with the vulnerable populations. —CSO Respondent #1 Remaining Priorities: During interviews, respondents were asked directly whether there were other priorities the Caribbean Regional Program should address and about their wishes for the Program. In response, government officials mostly mentioned areas that were already included in the Partnership Framework, but not yet a focus of Caribbean Regional Program projects and activities: strengthening the multi-sectoral response with greater emphasis on civil society and private sector involvement; strengthening human resource capacity of ministries of health to manage programs; costing of National Strategic Plans (NSPs) including care and treatment plans; and strengthening advocacy and policy initiatives. BUILDING COUNTRY AND REGIONAL CAPACITY FOR SUSTAINABILITY AND OWNERSHIP Technical assistance to Partnership Framework signatories aims to foster country ownership by expanding government capacity to plan, oversee, manage, deliver, and finance national HIV/AIDS programs. This section examines the evaluation findings with respect to the Caribbean Regional Program’s contributions to strengthening country and regional capacity building and fostering ownership needed for a sustainable HIV response summarized in Box 3. Box 3. Key Messages: Building Capacity and Fostering Ownership for Sustainability Post-PEPFAR Building Capacity for Ownership and Sustainability • Important but diverse efforts to build civil society and government capacity in the technical aspects of HIV programs and services and Caribbean Regional Program support of civil society actions are greatly appreciated by both government and civil society. • Government, civil society, and regional entities have capacity needs that require technical assistance post-2014. • Actions that address financial sustainability and other areas of health systems strengthening are important and merit greater focus going forward. Current Efforts and Commitment to Ownership and Sustainability • All stakeholders are thinking and concerned about sustainability and transitions post-2014 related to changes in PEPFAR and other HIV donor support. • Countries are taking steps to sustain the HIV response, but these are not always systematic or clearly linked to Caribbean Regional Program support. • There is a lack of a common “language” for talking about country ownership, as well as a lack of explicit monitoring of country progress. 13 BUILDING CAPACITY FOR OWNERSHIP AND SUSTAINABILITY The Caribbean Regional Program’s capacity building activities focus primarily on two dimensions of country ownership: (1) technical and managerial capacity and (2) institutional and community ownership. Building Government Capacity: Respondents across all stakeholder groups noted the range of training activities that the Caribbean Regional Program is supporting to strengthen government capacity (such as CHART and CHLI), strategic information (surveillance, studies), and laboratories. The remarks of a few respondents, shown below, exemplify the range of capacity strengthening areas. PEPFAR is making us see who are our true drivers of the infection…This is helping us to be more self￾sufficient. When you find where your problems are, with scarce resources, you can be more targeted. — Government Respondent #18 I would even say that there is now a greater appreciation of the value of operating accredited labs than ever before and this augurs well for improving diagnosis and treatment. —Regional Entity Respondent #2 In a few cases, the Caribbean Regional Program has begun to shift management and delivery of training from international to regional partners. The transition of funding and implementation responsibility for CHART, a long-standing USG-funded project that was operating before PEPFAR funds came to the region, directly to UWI has led the way, but there are still concerns about sustainability. We’ve transitioned quite a bit of I-TECH’s work to UWI as recipient of those funds. A lot of the CHART work needs to transition to the Ministry of Health. We’ve built capacity in a handful of Ministries of Health to implement training programs. That capacity is there, but the sustainability or ownership, I do not know. —International TA Partner Respondent #4 Some Caribbean Regional Team members were concerned that this kind of handover to regional partners is important but needs to be aligned with the capacity of those partners. In contrast, government and regional stakeholders have called for this process of channeling more responsibility and funding within the region to go further. Despite appreciation for training and organizational strengthening, substantial concern exists among all stakeholder groups about progress toward increasing, sustaining, and retaining human capacity due to limits in staff availability and frequent turnover. Several individuals, including some Chief Medical Officers and Permanent Secretaries, are in acting positions, and systems are not in place to pass relevant information to incoming staff. My biggest concern is that the people we are working with now will not be there in five years or beyond. You spend so much time and energy and money in training people and putting responsibility on their shoulders and they leave for a variety of reasons, institutional memory is lost, and things fall apart. —International TA Partner Respondent #2 Sustainability is predicated on understanding where HIV funding is coming from and where it is being used. Several respondents across all stakeholder groups remarked that Caribbean Regional Program support for national health accounts assessments have helped countries prioritize their needs for improved planning and sustainability. Government, regional entities, and technical assistance partners all stressed the importance of expanding this assistance to all countries in the region. 14 Building Civil Society Capacity: The capacity of CSOs varies across the region. Those that exist do not yet have access to financial resources to provide sustained support to key populations, nor do they have the resources to advocate effectively for these groups. The Caribbean Regional Program uses several mechanisms to increase civil society capacity to engage actively in national and regional HIV responses.10 These mechanisms require stronger internal governance and transparency in accounting. One respondent noted that such requirements can improve competitiveness and increase access to other funding, which is true for both the government and civil society. Government and civil society respondents spoke of the extent to which the Caribbean Regional Program has increased the prominence and capacity of CSOs working with key populations. Through these various mechanisms, CSOs receive funds to carry out activities as well as some organizational strengthening. There are differences in grants, with some directly building capacity (e.g., Peace Corps Volunteer Activities Support and Training [VAST] grants) and others (e.g., the State Department U.S. Ambassador’s Small Grants program) provide one-time support without an emphasis on sustainability. Respondents questioned the lasting merits of one-off grants and suggested longer-term investments and support that will allow for building civil society networks that increase independence and sustainability. I’m not saying [my CSO] should continue to get funds. However, if PEPFAR has seen the value in what [we] do, but cannot continue to fund it, how can we partner with PEPFAR to secure funds to assure the sustainability of what we started with PEPFAR? — CSO Respondent #3 COUNTRIES’ EFFORTS AND COMMITMENT TOWARD SUSTAINABILITY Sustainability of the HIV response is foremost on all stakeholders’ minds. Local governments are already carrying large social sector expenditures as a percentage of their gross domestic product, and international support for HIV and AIDS in the region is waning. All signatories to the Partnership Framework spoke of their approaches to sustaining their HIV response in general and beyond the Caribbean Regional Program, but there is variation in their progress and in how clearly their visions are articulated or incorporated into operational plans. Prior to the Partnership Framework, some countries had supported important staff positions with grants or loans that have now expired, and have since lost the capacity they had developed. In several instances, those with responsibility for the HIV program are also responsible for other projects and programs. While there is variation in country capacity and leadership, the global financial crisis is a reality that has posed additional challenges to countries that were already resource-constrained. The global economic recession directly affects the speed with which Caribbean countries will be able to absorb costs currently supported under the Partnership Framework or those funded through Global Fund grants or World Bank loans. The situation varies across the region, and some countries are suffering more than others. I am concerned about sustainability—we’ll always need a ‘godfather.’ We are small countries and our industries are very fickle. HIV can affect tourism, [crops] in the Windward Islands have been severely challenged. Financial commitment to the HIV response to equal or sustain what we have now by PEPFAR and PANCAP is unrealistic unless the world economy turns around. — Government Respondent #26 10 Mechanisms include the State Department’s U.S. Ambassador’s small grants program, Peace Corps Volunteer Activities Support and Training (VAST) grants, and cooperative agreements with CHAA and World Learning. 15 Numerous respondents spoke of the need for key decision makers (e.g., political ministers) to demonstrate willingness to ensure program continuation. Some country programs are making more progress than others because of this factor. This has happened because the Brigadier General supports the program; leadership from the very top. Having the buy-in from the highest level makes all the difference. — PEPFAR Respondent #7 Everything went dormant [after a World Bank project ended] and I am afraid that the same thing will happen to PEPFAR-supported activities…Government has to show more leadership in this area but I am not optimistic, money is a problem in the country. — Government Respondent #27 Strategies for Sustainability The three most common strategies respondents described for maintaining current programs are (1) decentralization and integration of HIV into primary health, (2) establishment of a multi-sectoral plan for HIV programs, and (3) and identification of other funding. These strategies are in line with the PFIP health systems strengthening intentions, although they are not a prominent focus of the Caribbean Regional Program’s activities to date. Decentralization and Integration of HIV: Some countries have started to, or plan to, integrate HIV into the primary health system, such as in sexual and reproductive health or other programs focused on chronic but non-communicable diseases. Governments prefer this approach because they can more easily assume costs of HIV goods, services, and support systems if these expenses are integrated into the general health budget. Several respondents noted that some Caribbean Regional Program laboratory and strategic information activities are contributing to countries’ integration efforts. The health information system including HIV surveillance was set up with an element of sustainability in that it was integrated into the general health care system from the beginning. —Government Respondent #8 Some respondents, across most stakeholder groups, wished for the Caribbean Regional Program to lend greater support to decentralization and integration, and identification of alternate, efficient health care models. Establishing a Multi-Sectoral Approach: The CRSF and the Partnership Framework speak explicitly about civil society and private sector engagement. Some countries are taking a multi￾sectoral approach to their HIV response as a means toward mutual accountability, but others are inadequately engaging multiple sectors to bring different perspectives to the overall effort. Countries vary in their multi-sectoral approaches: some of the approaches include involvement of multiple governmental ministries, formation of technical working groups comprised of personnel from different organizations, and creation of oversight committees comprised of government personnel, civil society, and the private sector. Some respondents are concerned that meaningful participation will be difficult if the government and private sector dominate to the detriment of civil society. We have to learn to use all the resources that are available in the country. Sometimes I get the impression that the government thinks that it can do it without the involvement of CSOs but that is a big mistake. — Government Respondent #29 Identifying Other Funding Modalities: Some respondents across stakeholder groups spoke of the importance of identifying other sources of funding to support the HIV response, such as government resources, international agencies, private sector, and existing or to-be-developed 16 national health insurance funds. Some countries, with the financial capacity, have started absorbing costs and personnel, or establishing strategies to absorb aspects currently funded by the Caribbean Regional Program, such as some laboratory-related costs. Countries vary widely in their ability to assume a greater share of costs. Other mechanisms for funding, besides increased government spending, private sector engagement, and international donors, need to be explored, as these sources are all limited. The Caribbean Regional Program has planned some activities (e.g., workshops) to help countries plan for the imminent end of their World Bank loans or Global Fund grants. Level of National and Regional Commitment and Ownership The Caribbean Regional Program is a partnership between PEPFAR, 12 countries, and 2 regional entities designed to move toward country ownership and sustainability of national and regional HIV responses. While there appears to be commitment to the Partnership Framework’s goals, conceptions about what “partnership” and “country ownership” mean vary. Several respondents, mostly from the Caribbean Regional Team, commented on shifts in the way countries and the PEPFAR agencies interact. They feel that a partnership is forming, but that this adjustment has taken time, and that room exists for better understanding of what “partnership” can and does mean for both sides. Respondents on both sides spoke of the desire for countries to have greater leadership within the partnership. The Caribbean Regional Team conveyed its expectations that countries will come forward to tell the Team how and where PEPFAR support can best be directed in support of countries’ plans, while also fulfilling country commitments under the PFIP. While we talk about country ownership and the Partnership Framework, countries do not quite get the control they need to have in moving the Partnership Framework forward and determining how it gets implemented. — PEPFAR Respondent #2 The Caribbean Regional Team spoke of their difficulty in understanding countries’ seeming reluctance to take further leadership. Countries remarked that governments have few staff to do the work required to ensure accountability of donor funding and measure its impact. Except in cases where cooperative agreements exist, country respondents frequently considered that the PFIP budget was managed externally and perceived most PFIP activities as coordinated and executed by external organizations. While in some cases this perception reflects reality, many Caribbean Regional Team members noted that flexibility exists in all areas of the PFIP, and that countries need to be more active in requesting specific work plan priorities. The need for a coherent process through which governments can align their Caribbean Regional Program work plans with their national HIV strategic plans is evident, but the requests must be consistent with the priorities outlined in the Partnership Framework and with the limits of PEPFAR funding. This incongruity of perceptions about the roles and responsibilities for the Partnership Framework and PFIP, and even power between countries and the Caribbean Regional Program, are not always well articulated. Some countries have made requests, for changes or specific types of technical assistance, which have not been accommodated. In other cases, the Caribbean Regional Program has provided assistance requested or identified additional technical assistance resources in response to requests. Currently there is no common language around or consistent monitoring approach for tracking progress toward country ownership. Caribbean Regional Team members spoke often of having a general sense of countries’ progress. Some informal monitoring is done through quarterly country work plan reviews. A few Caribbean Regional Team members spoke of the need to have established guidelines. 17 In terms of monitoring, PEPFAR [country ownership] indicators are a work in progress. We need some qualitative indicators, not all quantitative, such as integrating PEPFAR efforts into the National Strategic Plans. — PEPFAR Respondent #10 As highlighted earlier, the expected result of capacity building is that countries will assume greater ownership of their HIV response. A few government respondents challenged this assumption, viewing other factors as necessary for country ownership. I am not sure in what way the PEPFAR support to [my country] was intended to build ownership. I know that we have benefited in training at different levels which has helped to build capacity to deliver services but I am not sure about how to answer the ownership question. —Government Respondent #27 This sentiment goes beyond the basic economic factors that could affect a country’s ability to sustain the benefits of the training, and raises the need for more deliberate joint planning between the Caribbean Regional Program and policy makers for fiscal planning and relevant legislation. One area of weakness has been in terms of changing policies, working with policy makers to effect meaningful change. There is a disconnect between policy and policy makers and drivers and PEPFAR programs. I do not know how often PEPFAR management meets with policy makers. I know they meet with those at clinical and program level but don’t know about the policy level. — Government Respondent #9 These sentiments reinforce the wish expressed by some countries and regional entities for the Caribbean Regional Program to play a more active role in transition planning, including being a strategic technical partner for countries to help them develop clear plans. APPROPRIATENESS OF THE CARIBBEAN REGIONAL PROGRAM’S “REGIONAL” APPROACH Many historical antecedents of regional efforts exist in the Caribbean with the term “regional” used to refer to joint efforts and member organizations. From the perspective of the Caribbean Regional Program, it is regional in that it covers 12 countries. While many regional efforts in the Caribbean start from the regional level and flow down, the Caribbean Regional Program is a collection of 12 specific countries—a structure that allows for some economies of scale—as well as a set of regional goods and services deployed through regional entities. This section examines the appropriateness and economies of scale of the Caribbean Regional Program’s regional approach summarized in Box 4. Box 4. Key Messages: Appropriateness of “Regional” Model • The ability of the Caribbean Regional Program’s regional approach to encompass country-specific efforts, economies of scale, and regional goods and services is very appropriate to the Caribbean. • The relationship between a regional Partnership Framework, country-specific plans, and country ownership is not clearly articulated by the Caribbean Regional Team, or well understood by stakeholders. Respondents among all stakeholder groups deemed the Caribbean Regional Program’s regional approach appropriate to the unique needs of the Caribbean because it allows for coverage of many 18 small island nations that are too small for individual investment but are interlinked in terms of systems, movement, and trade and provides opportunities for economies of scale. I do not think anybody in the Caribbean will have a problem with the model. If I may say so, this model addresses country needs directly while at the same time allows for gaining from economies of scale through regional collaboration. The model is a good one, I wouldn’t change it. —Government Respondent #22 Respondents across the region spoke of CHART and CHLI, as prime examples of benefits of the regional approach. Organisation of Eastern Caribbean States country respondents unanimously spoke of the benefits of the PEPFAR-supported, upgraded, subregional Barbados National Laboratory, which allows for economies of scale. I think that the sharing of resources is a practical approach to providing quality services in the smaller countries which have limited capacity. Standards matter, and the only way that can be achieved is by having a regional collaborative approach. — Private Sector Respondent #1 Notwithstanding the endorsement of the regional approach by respondents at all levels, many spoke of a lack of clear vision and strategy for implementation of the regional approach. Even when USG agencies and countries appear to have a good understanding of the approach, they remain unclear about its mandate, structure, and operations. The PEPFAR regional program needs to do a better job at explaining our purpose, structure, mandate, and charter. — PEPFAR Respondent #3 Many respondents from ministries and regional technical assistance partners did not understand or agree with how PEPFAR resources are distributed among countries and regional entities, although there was no consensus among stakeholders on what the right balance should be. In responding to questions about the appropriateness of the regional approach, respondents, especially national representatives, cautioned against the arbitrary transfer of findings and lessons learned to all countries in the Caribbean Regional Program. While Caribbean countries have many similarities and commonalities, they are also unique and their uniqueness must be factored into the design and execution of support. Respondents did not say that the Caribbean Regional Program was applying a “one size fits all” approach, but many spontaneously cautioned against it. Caribbean Regional Team respondents appeared well aware of the diversity in the region and the need to tailor responses to country specifics. 19 CARIBBEAN REGIONAL PROGRAM COORDINATION AND COLLABORATION TO ACHIEVE COUNTRY OWNERSHIP PEPFAR is a global USG initiative involving six agencies that bring varied skills, constituencies, and resources to bear. This diversity is one of PEPFAR’s strengths, but also represents one of its challenges. Creating a synergistic, coordinated program across such different agencies requires commitment and investment of time and resources. The Caribbean Regional Program, in many instances, built on agencies’ activities and relationships that predated the arrival of PEPFAR resources in the region. This section examines how coordination and collaboration among the various agencies and their partners are working to advance country and regional ownership and sustainability summarized in Box 5. COORDINATION AND COLLABORATION Countries participating in the Caribbean regional program have an average of 15 technical assistance partners funded through PEPFAR, with a range of 10–18 per country. Appendix G presents which USG agencies have a presence and which technical assistance partners are active in each country. Within this complex web of agencies and partners, respondents spoke of several examples of effective collaboration. These include collaborative efforts of HRSA, CDC, and USAID on chart; CDC and USAID in the roll-out of rapid test kits and training in HIV counseling and testing; using Peace Corps Volunteers to provide human resources to support other agency interventions; and laboratory strengthening (CDC and DOD). Collaboration and Coordination with Countries and Technical Assistance Partners Country partners and regional and international technical assistance partners varied in their feedback about collaboration among USG agencies. Several countries specifically noted that they had no problems engaging with a variety of USG agencies, but did not feel they were able to access all that might be available to them. Countries with Cooperative Agreements with the CDC and major USAID projects were more likely to feel the lack of coordination between USG agencies, and were quite open about it. One challenge I have is dealing with so many USG entities, each with their own culture. They have not been doing a stellar job with coordinating. Sometimes CDC does not know what USAID is doing. This is the weakest aspect of the program. — Government Respondent #2 Respondents from several smaller countries, with limited coordination capacity due to few staff, focused more on difficulties dealing with a plethora of USG agencies and technical assistance partners coming and going. Many government and nongovernment officials expressed concern about the number of technical assistance partners involved in executing some aspects of the Partnership Framework, often with only minimal involvement from government or civil society actors. I do not like consultancies. We have had the assessment, but no one was trained on how to do it, so when we want to do it again, how do we do it? We have an excellent document, but there is no national who knows how to do it and we will need to do it again in the next five years. — Government Respondent #3 20 Several countries and technical assistance partners remarked that having the same technical assistance partners and agencies before and during the Caribbean Regional Program has resulted in confusion about what is and what is not PEPFAR-funded. Countries and technical assistance partners alike mentioned that they did not see clear mapping of activities across agencies and technical assistance partners, and this made it difficult to know who was doing what, when, and how. National AIDS Control programs do not always have the big picture of the Caribbean Regional Program activities in their country, with each USG agency and technical assistance partner working independently. It would make sense for [all the technical assistance partners] to sit together with the Ministry of Health and talk about what we’re doing, plan activities, not step on each other’s toes, timing…To be quite honest, at the Portfolio Review Meeting I was shocked that five agencies (CDC, USAID, Peace Corps, HRSA, DOD) worked in [the country I work in]. —International TA Partner #2 I did not know what the [large amount] was for or how it was used…We do not have condoms, lubricants, cannot do as much outreach… Then I learned I have spent [amount] and I do not know in what… I would have used those funds differently. — Government #3 In several countries, those in charge of HIV and AIDS activities in the Ministry of Health were not fully aware of what DOD was funding for the defense force, or had little or no knowledge of some activities conducted through technical assistance partners. Several countries mentioned a need for themselves to take a greater role in coordinating Caribbean Regional Program actors. We see the lack of coordination across USG and they need to acknowledge it. But, we also can take the lead on making this coordination an imperative.— Government Respondent #2 Box 5. Key Messages: Program Functioning in the Service of Country Ownership and Sustainability Strengthening Coordination and Collaboration to Foster Greater Country Ownership • The breadth of the Caribbean Regional Program has translated to 10–18 technical assistance partners per country— a substantial number to manage, especially for small countries. • Countries appreciate the respect they get from the Caribbean Regional Program but also want to have more of a say and a better understanding of what the Program can provide them. • Coordination at the country level is not optimal among country partners, between USG agencies and country partners, and across implementing organizations. • Although USG agencies have operated as “silos,” there is a desire to work together in a more unified manner as “One PEPFAR.” Caribbean Regional Program Structures in the Service of Country Ownership and Sustainability • Current Caribbean Regional Program structures provide a framework for coordination and collaboration but require more streamlined management and accountability for unity. • Stakeholder structures (the Steering Committee and Technical Advisory Group) are not used effectively. Spending • There is considerable under-utilization of funds. Both the Caribbean Regional Program and countries have a role to play in removing bottlenecks to ensure a greater impact of the HIV response. 21 The disperse nature of implementers poses challenges, but Caribbean Regional Team members individually appear to have developed productive relationships with their country partners. Although the sample is small, 12 government respondents present at the Portfolio Review meeting were asked what they valued most about working with the Caribbean Regional Program, and half spontaneously spoke to the personal characteristics of the Caribbean Regional Team: listening, respect, and flexibility. [There is] not a feeling that they want to force you into their own agenda, but [are] listening to countries. — Government Respondent #24 The positive comments during the Portfolio Review meeting reflect efforts by the Caribbean Regional Team members to engage with countries and their desire for greater planning and coordination of specific technical assistance partner activities with countries. Stakeholder respondents also spoke of a need for greater understanding of Caribbean culture by some technical assistance partners and Caribbean Regional Team members. Internal Coordination and Collaboration Both Interagency competition and the lack of a unified “One PEPFAR” approach within the Caribbean Regional Program have a significant effect on the level of collaboration and coordination. Some Caribbean Regional Team members were aware that their partners perceived the competition and discord within the team, but other team members were not aware. However, government and technical assistance partners mentioned the lack of unity and the competition among USG agencies, especially between USAID and CDC, in their interviews, almost as often as USG respondents did. Several USG and technical assistance partner respondents cited CDC, without prompting, as working as a separate entity. Interviews and document review revealed a series of factors that contribute to agencies’ potential to work at cross-purposes. These include the following: • Lack of a unified approach to the Caribbean Regional Program activities, which results in working in silos and within specific agency mandates (noted by USG, technical assistance partners, and governments). • Structural factors within PEPFAR that lead to competition for scarce resources among USG agencies (particularly mentioned by USG respondents). • Differences in agency culture and focus that create differences of opinion on the best approaches. This is compounded by inadequate efforts to embrace the strengths of different approaches to achieving the common goal of country ownership (mentioned particularly by USG respondents but also by technical assistance partners and government respondents). Manifestations of the above factors have included disrespectful interaction and behavior toward, or comments about, other agencies and their work (noted by USG respondents but also by several technical assistance partner and government respondents). Even with these factors pulling them apart, USG respondents across all agencies spoke often during their interviews of a desire to improve and work toward a common goal. Caribbean Regional Team members, government focal persons, and technical assistance partners called for a more holistic approach. Our government partners are smart and they see the lack of unity. We need to make sure we are presenting a unified PEPFAR program—that we work together! We need to have a unified strategy moving forward in how we brand ourselves. — PEPFAR Respondent #3 22 UNDER-UTILIZATION OF FUNDING The Caribbean Regional Program has contributed to strengthening capacity in the region and improving access for key populations. Yet many government, Caribbean Regional Team members, and technical assistance partners (regional and international) stated that the burn rate of funds has been low, with adverse implications for program implementation. Several countries pointed to the difficulties of navigating the complex financial procedures under Cooperative Agreements, the time￾consuming nature of the technical and financial review processes, and onerous reporting requirements as reasons for under-utilization of resources. Members of the Caribbean Regional Team and country respondents alike expressed some frustration with countries’ failure to consume resources available to them because of the cumbersome national bureaucratic systems. For example, financial decisions are often made within Ministries of Finance and Planning, in addition to Ministries of Health. THE CARIBBEAN REGIONAL PROGRAM STRUCTURES The Caribbean Regional Program has both internal and external coordinating mechanisms. Internal mechanisms include the Coordinator’s Office, the Embassy, the Senior Technical Team (STT), and technical working groups. The Steering Committee and Technical Advisory Group (TAG) are external mechanisms. This subsection presents findings on how well these mechanisms are fostering coordination and collaboration. Coordinating Structures within the Caribbean Regional Program Coordinator’s Office: The PEPFAR Caribbean Regional Program Coordinator is responsible for managing the operational and strategic planning processes, convening meetings of the STT, facilitating meetings of the technical working groups, and ensuring appropriate reporting to and communication with the Office of the U.S. Global AIDS Coordinator (OGAC). Working with six USG agencies with staff in multiple countries, geographic dispersal of countries within the region, the high cost of travel, and lack of video-conferencing facilities reportedly make coordination a challenge, especially with a staff of two. Respondents from several agencies and countries commented positively on the performance of the Coordinator’s Office, including examples of when the Coordinator has helped solve problems and address issues. Others (almost exclusively USG respondents) expressed frustration with communication mechanisms and management of last￾minute requests from OGAC. Work planning and coordination are done through monthly or quarterly calls with countries and agencies. Respondents reported that these calls were held inconsistently, had too many (or too few) participants, were not substantive enough, and did not sufficiently address coordination and collaboration across entities. The STT was formed at the June 2012 Portfolio Review Meeting11 and began regular meetings in July 2012, so it is too soon for the evaluation team to assess its effectiveness in improving collaboration and coordination. U.S. Ambassadors: A key issue raised by many USG, country, and technical assistance partner respondents is the lack of an in-country USG “PEPFAR Coordinator or Focal Point.” They expressed the need for someone who can speak for the Caribbean Regional Program as a whole (not from the perspective of an individual USG agency) and who would have the larger picture of all Caribbean Regional Program activities in a given country. As one USG respondent mentioned, 11 The STT is comprised of the Caribbean Regional Program Coordinator, senior management representation from CDC, USAID/Eastern Caribbean, USAID/Jamaica, DOD, Peace Corps, and HRSA. This structure is described as the PEPFAR Interagency Management (PM) in the U.S. Caribbean Regional PEPFAR Partnership Framework Management Operations Manual (March 2011, v.4). 23 I am [USG Agency] but I am PEPFAR—people use the words interchangeably. I also do other [USG Agency] activities besides PEPFAR. Sometimes I am asked to speak for PEPFAR but I am really just [USG Agency]. — PEPFAR Respondent #10 The U.S. Embassy in Barbados is designated as the focal point for administration of the Regional Partnership Framework, with the U.S. Ambassador to Barbados and the Eastern Caribbean as executive. This position was vacant from the start of the Partnership Framework until June 2012. The Ambassador’s presence and therefore ability to fulfill his mandate holds great potential for the Caribbean Regional Program, but it is too early to evaluate the effects of his presence. The U.S. embassies in Belize, Jamaica, the Bahamas, Trinidad and Tobago, and Suriname are members of the Partnership Framework, but statements from respondents indicate that, except for Jamaica, the embassies are not consistently involved in the activities of the Caribbean Regional Program. Although it is not their assigned role, embassy respondents suggested they could take on a coordinating role for the country in which they are based if they had the information needed to do so. Coordination appears to be strongest in Jamaica, where the Ambassador has mandated regular meetings encompassing USG agencies, USG technical assistance partners, and the government. In addition, the Jamaican Government has taken a strong stand in conveying its need to be aware of all activities and have a say in all funding decisions. Strong action by the Embassy, although outside of its mandate, has resulted in players in Jamaica reporting awareness of others’ activities and feeling a sense of uniform focus. Such country-level coordination could serve as a model for other Caribbean Regional Program countries with U.S. Embassies. Technical Working Groups: The evaluation team did not ask questions specifically about this structure, but several Caribbean Regional Team members and a few technical assistance partner respondents mentioned the technical working groups as existing mechanisms that could be used more effectively. These groups (referred to as Interagency Working Groups in the Operations Manual (PEPFAR CRP 2011) are tasked with developing the Regional Operations Plan and coordinating country work plans across agencies and with the National AIDS Programs and other partners to share information across agencies both within and across Technical Working Groups. Efforts have been made through the Technical Working Groups and biweekly calls and ad hoc calls and through personal attempts to engage other colleagues; there are efforts made to ensure collaboration. Don’t know if we’re being held accountable enough internally to make sure the collaboration works. — PEPFAR Respondent #12 Steering Committee and Technical Advisory Group At the start of the Partnership Framework, two structures were created to provide political and technical support and oversight to the Caribbean Regional Program activities and to foster ownership of the Partnership Framework. These include: (1) a 22-member Steering Committee comprised of Ambassadors, Ministers of Health, and two regional organizations (PANCAP and the Caribbean Regional Network of People Living with HIV and AIDS); and (2) a TAG comprised of 24 technical experts of key constituencies. It has been difficult for these two structures to hold effective meetings due to travel expenses and inadequate communication mechanisms. In the first year, the Coordinator’s Office made efforts to engage, involve, and maintain a vibrant group. In general, participation in conference calls has been minimal, and the effectiveness of phone calls for communication was questioned by several members. These groups are important, but only as strong as individual participation. Some respondents were unaware that they are listed as members of these groups. 24 I am a member [of the TAG] but I never participate. The conference calls are not convenient. You have to call in. They don’t call you. — Government Respondent #11 I am a member of the TAG but it is not a functional entity. I can’t say that it is functioning as a real technical advisory group. I have served on similar bodies and we met at a prescribed time to review implementation, look at technical issues and challenges, and come up with strategic responses. That is not happening here. — Regional Entity Respondent #2 The Steering Committee has been particularly difficult to organize. The description of its role in the Operations Manual (PEPFAR CRP 2011) is cursory, and some Caribbean Regional Team members noted its role is more for political and diplomatic purposes than for oversight. It has potential, though, for generating greater engagement at the political level, which is necessary for sustainability of country programs. 25 PART 3: CONCLUSIONS AND RECOMMENDATIONS FOR THE WAY FORWARD CONCLUSIONS The four dimensions of country ownership outlined in the FY 2012 Country Operational Plan Guidance were not available to provide specific guidance to the Caribbean Regional Program at the start of the Partnership Framework. However, the Program’s various technical assistance mechanisms can be viewed as contributions to these dimensions and to how they serve the overall goal of sustainable and epidemic-relevant country and regional HIV responses. The evaluation team proposes a conceptual theory of action (see Figure 4 below as a lens for assessing efforts to date, and as a frame for the Caribbean Regional Program moving forward). This theory of action proposes how the PEPFAR mechanisms (or actions, represented by the arrows, with regional partners in green) could work with specific Caribbean stakeholders (blue squares) to build capacity in each dimension of country ownership (green squares) to achieve the Partnership Framework goal (top rectangle). Some projects/mechanisms may work across dimensions, and the entities listed in the arrows are intended to be illustrative. The Caribbean Regional Program works to achieve technical improvements in the region’s HIV response; it also works to move countries along the continuum toward ownership and sustainability. The four main evaluation questions, around which the findings and conclusions are presented, correlate to these four dimensions of country ownership. Evaluation Question 1: How relevant and responsive is the content of the Partnership Framework (as it is being implemented) to the needs of country and regional partners in achieving progress in combating the HIV epidemic? The Caribbean Regional Program’s focus on technical assistance and partnership for country ownership differs from previous donor approaches in the region. Understanding and appreciation of this approach has increased since 2010, but it has been a learning curve for the Caribbean Regional Program as well as for country and regional partners. While the conceptualizing of country ownership has been evolving globally and within PEPFAR II, there has been insufficient conceptual dialogue between the PEPFAR Caribbean Regional Program and its partners related to country ownership, and its practical implications for countries and achievement of the Partnership Framework goals. The collaborative approach used in developing the Partnership Framework provided a strong start in fostering all four dimensions of country ownership by engaging key stakeholders at all levels of the response (mutual accountability, institutional and community ownership), aligning the Framework with the CRSF and national strategic plans (political ownership and stewardship), and identifying gaps not filled by other donors (capabilities). Both countries and the Caribbean Regional Team remain committed to the Partnership Framework, and the Framework includes many of the areas that countries feel are priorities for greater sustainability and ownership of their HIV response, although levels of commitment differ in intensity among the countries. 26 The evaluation findings indicate some disconnect between country and Caribbean Regional Team perspectives about how their partnership can play out. PEPFAR as a U.S. funding mechanism has operational constraints in certain areas that limit how funds can be spent, and considerable flexibility in other areas. There is limited understanding at the country level about the extent to which country partners can discuss, amend, or change in their annual work plans to meet their priority needs. Greater communication is needed to address the incongruity between countries’ impression that PEPFAR has predetermined activities and management in the absence of adequate country input, and, on the other hand, the Caribbean Regional Team’s desire for countries to take the lead and to use PEPFAR resources to best support country-determined efforts. Figure 4. PEPFAR Caribbean Regional Program—Theory of Action Evaluation Question 2: How is the Caribbean Regional Program’s approach to building capacity and fostering ownership needed to move the region closer to sustainability post￾PEPFAR? The Caribbean Regional Program has brought concrete technical assistance and capacity building to the region, and enhanced countries’ abilities to manage their HIV programs and resources through training and technical assistance in the areas of prevention, strategic information, laboratories, and health systems strengthening. This capacity enhancement crosses multiple sectors: government (ministries and defense forces), civil society, NGOs, regional entities, and to a lesser extent, the private sector (through private sector assessments and follow-up). There is recognition that training alone is insufficient to ensure sustainable capacity. The dimensions of institutional and country ownership and mutual accountability call for further organizational capacity building of civil society and mentorship of government counterparts to allow them to play their roles more effectively. 27 The Caribbean Regional Program currently lacks a shared articulation of country ownership among all partners and a consistent method for monitoring and measuring progress toward ownership and sustainability. Building capacity for greater country ownership and sustainability is not a quick process, and requires clear goals, strategies, and metrics to stay focused and measure progress in tangible terms. The PFIP and country work plans can play a critical role in this process. They are intended to be flexible, responsive documents, and there is space and time to modify them to meet country and regional partners’ changing needs for capacity building in service of country ownership and sustainability. All countries are concerned about sustainability. While some countries are taking concrete steps and have greater capacity to move in that direction, the situation varies widely across the region. The success of these measures and systems is important for the development of political ownership and sustainability. Many respondents, across stakeholder groups, suggested that the Caribbean Regional Program should conduct focused discussions with governments and regional entities to develop transition plans in a way that builds political will and fosters realistic steps within the current economic climate and existing staffing context. The PEPFAR Strategic Plan states the importance of strategic coordination that furthers the reach of bilateral assistance, leverages the work of multilateral organizations, promotes country ownership, and increases the sustainability of national health programs (PEPFAR 2009b). Other international entities, such as the Pan American Health Organization (PAHO) and the Global Fund, play important roles in influencing policy and practice. More strategic links are needed between the Caribbean Regional Program and other HIV donors. Evaluation Question 3: Is the Caribbean Regional Program’s regional model appropriate and functional to create synergies and economies of scale in the region? The Caribbean Regional Program’s “regional” approach, which combines country-specific and regional activities, is consistent with the history of regional cooperation in the Caribbean. Direct technical assistance to countries and regional entities, and access to regional goods and services (e.g., regional training initiatives, laboratory strengthening) supported by PEPFAR funding represent success in moving toward technical and managerial capabilities, and institutional and country ownership. Yet the regional nature of the Partnership Framework and the country specificity it allows are not always implemented as intended, nor are these always clear to all stakeholders. There have been some challenges in implementing the regional approach, such as the number of USG agencies and technical assistance partners involved, and ineffective communication and coordination. The Caribbean Regional Team needs to articulate adequately the regional approach, explaining to all actors (country, regional entities, and technical assistance partners) where and how they fit into the overall approach, and more importantly, how they contribute to building country ownership and sustainability. Each country wants to get the maximum it can and to have the approach tailored to its situation. The Caribbean Regional Program allows some access to a wide range of smaller countries that would not necessarily have been offered an individual PEPFAR program, and has generally avoided an approach that is uniform across countries and regional entities. 28 Evaluation Question 4: How are the Caribbean Regional Program structures and processes for coordination and collaboration facilitating or hindering achievement of country ownership? Effective coordination, both internally and externally, is critical to achieving mutual accountability and effective functioning. The Caribbean Regional Program’s structure and operation have both facilitated and hindered its ability to work effectively to build capacity and sustainability for country ownership at the country and regional levels. It is an extremely complex program, requiring coordination and collaboration across six agencies, 22 technical assistance partners, 12 countries, and two regional entities. Much has been accomplished in terms of technical assistance, and relationships with country and regional partners are generally good. However, the lack of cohesion among USG agencies and technical assistance partners has translated into a fragmented approach, resulting in some confusion and feelings of disempowerment among many country and regional partners. Countries, technical assistance partners, and the Caribbean Regional Team all expressed the need and desire for “One PEPFAR”—that is, a clear articulation of what the Caribbean Regional Program (as opposed to an individual USG agency or project) brings and does, how all USG agencies will work in concert to increase political and institutional ownership of countries, mutual accountability between the Caribbean Regional Program and countries, and collaboration across USG agencies in addressing countries’ key priorities. There is a need to strengthen country-level coordination and for countries to take greater leadership in coordinating in-country efforts. This includes providing input into how Caribbean Regional Program resources are used, so that countries can take advantage of flexibility where it exists, and at the same time working with USG agencies to determine how best to work within the Program’s limitations. The “lack of absorptive capacity” or “low burn rate” needs to be anticipated and tackled systematically. Underestimation or miscalculation of required inputs, together with countries’ lack of experience in monitoring and evaluation (M&E), may hinder the achievement of programmatic outputs. A particular challenge in the region is protracted procurement processes, including slow recruitment of project staff, which delay implementation of well-laid plans and concomitant under￾spending. There appears to be a desire within the Caribbean Regional Team to improve working relationships, engage with greater respect, and improve dynamics; all of these are key to moving forward. The activation of the Senior Technical Team and the arrival of the U.S. Ambassador to Barbados and the Eastern Caribbean augur well for improvement. Although these developments are too nascent for the evaluation to assess, they are steps in the right direction. The external stakeholder structures (the Steering Committee and TAG) have not been effective, and there appears to be little ownership among members. These structures have the potential for greater value to the Caribbean Regional Program, if they can be reinvigorated, particularly around developing a strategy and monitoring plan for greater country ownership. In addition to these two structures, more needs to be done to identify ways in which regional mechanisms can be used more effectively to engage governments and civil society. Before proceeding to the recommendations, Box 6 summarizes the broad conclusions of the evaluation. 29 Box 6. Summary of Key Evaluation Messages • The Caribbean Regional Program has contributed to strengthening country capacity in all countries in understanding and responding to the HIV epidemic. • The Caribbean Regional Team is clear on its partnership approach with countries and putting countries in the lead, but stakeholders have not always adequately understood this. The lack of clear definitions of country ownership and metrics contributed to this disconnect. • The lack of a “One PEPFAR” approach within the Caribbean Regional Team encumbers its efforts toward country ownership and sustainability, which require a coordinated effort easily understood by the countries. There is evidence of willingness to bridge the gaps within the Caribbean Regional Team. • There is a desire among the Caribbean Regional Team and countries to develop and implement plans that increase country capacity for greater ownership and sustainability. RECOMMENDATIONS FOR STRENGTHENING COUNTRY OWNERSHIP AND SUSTAINABILITY BY 2014 The Caribbean Regional Program has made important contributions to the region’s HIV response, laid the groundwork for mutual accountability through the Partnership Framework, and developed good working relationships with countries. The key question going forward is how best to shape PEPFAR assistance in the remaining two years in ways that further the four technical goal areas, while explicitly building the capacity of countries and regional entities in the four dimensions of country ownership. There is still time to articulate clearly a joint vision (with countries) of desired results for 2014 related to country ownership and sustainability, develop a theory of action linking Caribbean Regional Program activities to the four dimensions of country ownership, develop a “One PEPFAR” approach and system, and create the monitoring tools needed to track progress toward country ownership. This section presents recommendations in the form of specific guidance to the Caribbean Regional Team, other PEPFAR actors, participating national and regional partners, and technical assistance partners to accelerate efforts toward sustainable, country-owned, epidemic-relevant, country and regional HIV responses in the Caribbean. The translation of this guidance into actions plans remains the responsibility of the Caribbean Regional Team together with its partners. OVERARCHING RECOMMENDATIONS FOR COUNTRY OWNERSHIP The Caribbean Regional Team should: • Clearly articulate and communicate the Caribbean Regional Program’s “regional” approach and theory of action for country ownership and sustainability. The Caribbean Regional Team should create a framing document (with visuals) to convey clearly what the regional approach means for the Program, how all the Program parts fit together, and roles and responsibilities of countries, regional entities, technical assistance partners within the regional 30 approach. This articulation must also include a theory of action that delineates the pathways through which program actions lead toward country ownership and sustainability. The Caribbean Regional Team and its country and regional partners should: • Develop a common definition of country ownership and its dimensions. Using country planning sessions or other opportunities, the Caribbean Regional Team and country and regional partners should specifically discuss and agree on a definition of country ownership and its key dimensions. Country ownership should not be confused with “graduation,” but rather be understood as a strategic investment and partnership toward a common goal. The four dimensions outlined by OGAC can serve as a starting point for these discussions. • Define country ownership goals by 2014. During country planning meetings, it will be critical to identify both short- and long-term goals for each country and regional entity in terms of country ownership and sustainability, as well as benchmarks for assessing progress. These goals must be achievable and take into account non-PEPFAR funded efforts in-country. Even if clear metrics do not yet exist, a joint, collaborative assessment and a discussion of findings of where the country currently stands are important. • Develop country-specific and regional entity-specific ownership/sustainability plans. Countries need clear strategies and plans to help them meet their country ownership and sustainability goals. These plans should include short-term, medium-term, and long-term systematic benchmarks and investments. (See Box 7 below for possible components of such an ownership/sustainability plan). Country ownership can be examined within a range of domains that should be monitored and evaluated throughout the remaining period of the PEPFAR Caribbean Regional Program, with the countries and regional entities in the lead.12 • Build aspects of this ownership/sustainability plan into FY 2013 and FY 2014 operational planning. These plans must be jointly monitored and reviewed annually for progress toward the country ownership goals and benchmarks, with any needed adjustments to technical assistance identified to advance progress. The Caribbean Regional Team should: • Examine country-specific ownership goals across countries for synergies and how to address them. Country ownership plans may include areas for economies of scale, as well as cross-learning opportunities. This examination should also assess the need to update the Partnership Framework and PFIP, and a plan for doing so. 12 The AIDSTAR-One Capacity Assessment Tool for Country Ownership of HIV Care and Treatment (Gutmann, Franco and Messner 2013) may be useful for these purposes. 31 Box 7. Potential Components of a Plan for Country Ownership and Sustainability 1. Country ownership goals. This section outlines where the country wants and needs to be with its HIV programs within specified timeframes (e.g., before 2014, by 2014, beyond 2014). The goals can include institutional, political, social, and financial targets related to the four goal areas of the PFIP, or the four dimensions of country ownership. It may be useful to conduct a collaborative assessment of where the country is now, which can be used as a benchmark, determining what needs to be done to move the country from baseline to end goal. 2. Country strategies for achieving goals. This section articulates how the country will organize efforts to achieve goals and benchmarks that will be used to assess progress toward country ownership. 3. PEPFAR support to achieve country ownership goals. This section delineates PEPFAR￾supported technical assistance that will help the country achieve its benchmarks. This section should clearly illustrate the pathway from technical assistance to results that contribute to country ownership goals. 4. Other (non-PEPFAR) support for achieving country ownership goals. The plan should acknowledge other efforts in place to support sustainability and country ownership of the HIV response. 5. Monitoring progress. This section includes the benchmarks and indicators of progress that are reviewed and adjusted quarterly and annually. RECOMMENDATIONS FOR PEPFAR’S DIMENSIONS OF COUNTRY OWNERSHIP Political Ownership and Stewardship Government has a vision for HIV support, works with civil society, the private sector, and other funding partners to clearly articulate its priorities and plans for program development, and has oversight of specific activities conducted by all stakeholders. Governments across the region are not at the same level of political ownership and stewardship capacity, nor is there uniform engagement of civil society and other sectors. The Caribbean Regional Team has the instruments to support political ownership and stewardship, but needs to assess how and where these instruments can best contribute to political ownership in each country. The Caribbean Regional Team should: • Examine how it can better support political ownership through its current portfolio of technical assistance partners and projects, or by giving a clearly defined role for government, or by shifting focus to currently supported activities. Countries and regional entities should: • Articulate clearly to the Caribbean Regional Team their own vision for their country’s HIV response and the support needed to strengthen their capacity. They should recognize that while the Caribbean Regional Program may not be able to address all needs, it has some flexibility, especially within certain mechanisms. • Be more proactive in demanding better coordination among Caribbean Regional Program agencies and partners in-country, and establish systems for tracking what USG agencies and partners are doing. This includes clearly articulating communication procedures, such as whom to notify when technical assistance partners are arriving in-country for short-term assignments and what specific in- or out-briefings are required. 32 The Caribbean Regional Team and countries should: • Take shared responsibility for ensuring that both government and civil society are actively involved in country-level planning discussions. Institutional and Community Ownership Local institutions (e.g., governmental and civil society) own final decisions for each program stage, and have responsibility for programs and fund management. The Caribbean Regional Team should: • Present its program as “One PEPFAR” and communicate explicitly about the entirety of its country-level efforts. Work plans should be consolidated in ways that facilitate country stakeholders’ ability to track what is happening, and technical assistance partners’ capacity to coordinate with each other. • Intensify opportunities for international technical assistance partners to work in collaboration with regional and country-level technical assistance partners in a way that builds capacity of host￾country nationals so that a greater share of technical assistance can shift to local providers. • Lend greater support to countries’ decentralization and integration efforts and help identify alternate efficient health care models. • Shape the current portfolio to provide stronger support to CSOs so they can play a greater role in national and regional program decision making and implementation within the HIV response. Countries should: • Take leadership on work planning and scheduling of Caribbean Regional Program activities to ensure these activities are country-led and planned. Technical and Managerial Capabilities Country leadership has the technical and management capabilities to oversee programs and make adjustments and shifts over time. Planning must be deliberate to provide ample opportunities for local capacity to perform activities. The Caribbean Regional Program supports training, fosters human capacity building, and aims to provide technical assistance through partnership. These capacity building efforts need to be sharpened to target competencies required for the various dimensions of country ownership, as desired and needed by the countries. The Caribbean Regional Team and countries should: • Based on the country and regional ownership goals and country ownership plans, determine the competency building needs to achieve the goals and build into the next two fiscal year operational plans the mechanisms for providing support for this capacity building. Any training should have clearly articulated post-training goals that address how new behaviors enabled by the training serve the goals of countries. Formal evaluations of PEPFAR-funded training and technical assistance for capacity building should be built into M&E systems. • Require all external technical assistance partners, in collaboration with country teams and the relevant PEPFAR partner agency, to develop an explicit plan for building capacity to ensure skill and capacity transfer to host-country nationals. 33 Mutual Accountability, Including Finance A strong sense of defined accountability exists between governing bodies and the citizens it serves, as well as between country leadership and donors for measurable results over time. Information and processes are transparent, and civil society has mechanisms for providing input/ feedback. Roles and responsibilities are explicit. The Partnership Framework sets the stage for mutual accountability with countries. Monitoring of progress toward country ownership, mentioned under “Overarching Recommendations for Country Ownership” (above) can foster greater accountability. The Caribbean Regional Team and Countries and Regional entities should: • Regularly assess government and Caribbean Regional Program progress toward commitments under the Partnership Framework and PFIP, update and refine both the Partnership Framework and PFIP as needed based on new information, and determine actions required in cases of noncompliance. • Jointly take responsibility for developing annual operational plans, with countries in the lead to further multisector planning efforts. Quarterly country work plan review meetings should maximize in-person or virtual platforms (e.g., Adobe Connect) with the Caribbean Regional Team, country and regional entities, and TA partners. The entities should ensure that these meetings are focused, have a substantive and realistic agenda, and provide a safe space for open discussion about successes, challenges, and action steps to address the challenges. Engaging technical assistance partners in this process will allow them to modify/alter their work to align with changing needs and priorities. The Caribbean Regional Team should: • Review the role of the Steering Committee and TAG for opportunities to use them to generate greater engagement and progress toward country ownership. This includes working with members to articulate roles and responsibilities and establish rules and systems for engagement and mutual accountability. Participation can be facilitated using communication technology such as video or virtual platforms. • Examine whether, in its arsenal of mechanisms and partners, the Program can provide more capacity building to CSOs for organizational strengthening and policy advocacy so that members of key populations are present at key decision-making fora and their needs included in the national policy agenda. RECOMMENDATIONS FOR STRENGTHENING THE CARIBBEAN REGIONAL PROGRAM’S COLLABORATION AND COORDINATION The Caribbean Regional Program brings significant assets to the region to strengthen countries’ HIV responses, but the effectiveness of these assets will improve if the various USG agencies involved work in concert toward their technical and country ownership and sustainability goals. The Caribbean Regional Team should: • Create a “One PEPFAR” voice and branding, and conduct joint agency and technical assistance partner visits whenever possible. Discussions with country and regional partners should clearly articulate what is PEPFAR-funded and what is not, and these discussions should examine the possibility of synergies that are not yet operationalized. 34 • Identify and agree on someone, preferably in-country, who can represent “One PEPFAR” and coordinate with and across USG agencies in-country. For OECS countries it may be possible to have one representative for multiple countries. Existing Embassy Points of Contact for the Program can potentially play this role if they are adequately informed and empowered, and are fully utilized by the PEPFAR Regional Team. This could greatly improve communication and coordination, and overall Program success. The U.S. Ambassador to Jamaica’s monitoring efforts can serve as an example, and the systems and processes developed at that embassy can be used as a model for other countries. Once the “One PEPFAR” representatives have been identified, their roles and responsibilities in the Program should be clearly communicated to national, regional, and technical assistance partners so they know who needs to be informed, by whom, when, and for what. • Add staff to the Coordinator’s Office to ensure effective coordination across agencies, provide necessary support for both policy development and in-person discussions, and follow up to develop effective country ownership plans and monitor progress toward country ownership and sustainability. OGAC and U.S. Ambassadors should: • Take active steps to hold the Caribbean Regional Team accountable for coordination and collaboration. COMMENTS ON GENDER INTEGRATION Gender is a crosscutting theme in the Partnership Framework and PFIP, but when the evaluation team reviewed several Caribbean Regional Program documents they found few references to integrating gender equality into other goal areas. The Caribbean Program Team should: • Examine how gender equality can be better integrated into the Partnership Framework goals of health systems strengthening, strategic information, and laboratory strengthening. Going beyond sex-disaggregated data, a simple gender analysis of work plans and program activities under each of these areas could inform ways of addressing gender inequality. 35 REFERENCES Gutmann, Mary, Lynne Miller Franco, and Lyn Messner. 2013. Capacity Assessment Tool for Country Ownership of HIV Care and Treatment. Arlington, VA: USAID’s AIDS Support and Technical Assistance Resources, AIDSTAR-One, Task Order 1. Organisation for Economic Co-Operation and Development, Development Assistance Committee (OECD DAC). 2010. Quality Standards for Development Evaluation. DAC Guidelines and Reference Series. Paris: OECD. PEPFAR (The United States President’s Emergency Plan for AIDS Relief). 2009a. Guidance for PEPFAR Partnership Frameworks and Partnership Framework Implementation Plans. Version 2.0. Washington, DC: PEPFAR. Available at http://www.pepfar.gov/documents/organization/120510.pdf (accessed September 2012). PEPFAR. 2009b. The U.S. President's Emergency Plan for AIDS Relief: Five-Year Strategy. Washington, DC: PEPFAR. Available at http://www.pepfar.gov/documents/organization/133035.pdf (accessed September 2012). PEPFAR. 2011a. PEPFAR: Addressing Gender and HIV/AIDS. Washington, DC: PEPFAR. Available at http://www.pepfar.gov/documents/organization/185947.pdf (accessed September 2012). PEPFAR. August 2011b. PEPFAR FY 2012 Country Operational Plan (COP) Guidance. Washington, DC: PEPFAR. Available at http://www.pepfar.gov/documents/organization/169694.pdf (accessed September 2012). PEPFAR. n.d. PEPFAR Supporting Country Ownership, Key to a Sustainable Response. Washington, DC: PEPFAR. Available at http://www.pepfar.gov/documents/organization/195465.pdf (accessed September 2012). PEPFAR Caribbean Regional Program (CRP). 2009. Caribbean Regional HIV and AIDS Partnership Framework 2010–2014: Five-Year Strategic Framework to Support Implementation of Caribbean Regional and National Efforts to Combat HIV and AIDS. Washington, DC: PEPFAR. PEPFAR CRP. 2010. Caribbean Regional HIV and AIDS Partnership Implementation Plan 2010–2014: Five-Year Strategic Framework to Support Implementation of the Caribbean Regional and National Efforts to Combat HIV/AIDS. PEPFAR CRP. 2011. Partnership Framework Management Operations Manual. Version 4. PEPFAR CRP. 2012. Midterm Evaluation of the PEPFAR Caribbean Regional Program: Terms of Reference. 36 37 APPENDIX A PEPFAR CARIBBEAN REGIONAL PROGRAM GOAL AREAS Goal Area Specific Strategies Prevention: Reduce estimated number of new HIV infections in the Caribbean. • Develop comprehensive approach • Expand services • Reduce stigma and discrimination • Establish referral systems Strategic Information: Improve capacity of national governments and regional organizations to increase availability and use of quality, timely HIV/AIDS data to better characterize the epidemic and support evidence-based decision making for improved programs, policies, and health services. • Collect information to characterize the epidemic • Implement M&E strategies • Build strategic information capacity • Harmonize data collection Laboratory Systems: Increase capacity of national governments and regional organizations to improve the quality and availability of diagnostic and monitoring services and systems for HIV and related sexually transmitted and opportunistic infections, including tuberculosis, under a regional network of tiered laboratory services. • Develop a functional regional reference laboratory network • Improve laboratory services and systems Health Systems Strengthening: Improve capacity of national governments and regional organizations to effectively lead, finance, manage, and sustain the delivery of quality HIV prevention, care, treatment and support services at regional, national and community levels over the long term. • Improve human resource management systems • Improve financial management capacity of national HIV programs • Improve integration and efficiency of national HIV partners • Strengthening national leadership and governance in HIV programming Source: PEPFAR CRP 2010. 38 39 APPENDIX B COMMENTS AND FEEDBACK FROM STAKEHOLDERS Of the 23 representative stakeholders who were targeted to review the final draft report, 14 provided feedback, including staff of Ministries of Health, regional technical assistance partners, international technical assistance partners, regional entities, and international organizations. Much of the feedback reiterated important points already made in the report. Other feedback included corrections and areas that needed more clarification, and these were incorporated directly into the body of the report. This Appendix summarizes some comments that represent important additional thoughts that can contribute to forward planning and use of this midterm evaluation report. These comments fell primarily into three categories: usefulness of the evaluation and report itself, ownership/sustainability planning, and coordination and collaboration. USEFULNESS AND VALUE OF THE EVALUATION Stakeholder comments confirmed the usefulness of an evaluation process to help sharpen the focus of the Caribbean Regional Program. Both technical assistance partners and Ministry of Health staff responded positively to the report content, and spoke of its usefulness in terms of conveying the “big picture” of the Caribbean Regional Program, serving as a valuable tool for joint accountability and cross learning, and outlining the need for countries to transition to government sustainability. The recommendations were viewed as very useful, and a Ministry of Health representative wrote: “I am eager for PEPFAR to implement the recommendations in the report.” The report was seen, not only as a midterm review of the Caribbean Regional Program, but also as an assessment of the Caribbean situation as it pertains to countries and regional entities. From this perspective, some believe that the report could serve as a valid reference document for Caribbean governments and a range of other stakeholders. COUNTRY OWNERSHIP AND SUSTAINABILITY PLANNING Reviewers from all stakeholder groups remarked positively about the framing of country ownership in the report, although they felt strongly that country ownership must be defined by consensus. Reviewers felt that the four dimensions of country ownership and the Theory of Action presented in the report provide a framework for moving forward and developing a shared definition of country ownership. Several reviewers pointed out that the concept of country ownership “gets muddled on the ground” between political and financial ownership. There is recognition that both are needed, but many felt that the Caribbean Regional Program needs to engage more on the political dimension of country ownership and create the linkages that could leverage opportunity across all countries. Reviewers remarked that the “transition” language used by the Caribbean Regional Program has 40 often been interpreted as “withdrawal,” and this confusion is compounded by a lack of clear language on sustainability by the Program, or one that fits into broader global approaches.13 One Ministry of Health reviewer noted a lack of alignment between the Caribbean Regional Program’s financial support for the four goal areas (HIV prevention, strategic information, laboratory strengthening, and health systems strengthening) and the “four dimensions of country ownership” (political ownership and stewardship, institutional/community ownership, technical and managerial capability, mutual accountability including finance). With respect to recommendations related to capacity building of CSOs, stakeholders noted that there is a wide variation in the level of organization and influence of CSOs within countries and regionally that must be recognized, and that it would be useful to engender dialogue about their purpose and value in order to render them most effective. COORDINATION AND COLLABORATION Technical assistance partners who reviewed the final draft spoke to the need for greater detail and further discussion with the Caribbean Regional Team about how to improve coordination of technical assistance to countries. Several specific ideas for addressing these issues emerged from the feedback, including (1) considering restricting the number of technical assistance partners operating in a given country at any one time, or prioritizing assistance offered to countries in order to recognize the absorptive capacity of countries already limited by internal structures; (2) harmonizing partner activities among themselves and in countries as key to maximizing success and minimizing country frustration; and (3) briefing technical assistance partners in the annual operational planning process so that they know how to modify their work in light of Ministry of Health-PEPFAR discussions. AREAS FOR ADDITIONAL EXPLORATION Stakeholder comments highlighted the importance of gender integration, and that gender analysis should have received more attention in the evaluation. While gender analysis was not explicitly included in the evaluators’ scope of work, it represents an important point for the Caribbean Regional Program to consider in its strategic planning. One Ministry of Health reviewer recommended that future evaluations make gender integration explicit and conduct more interviews with appropriate entities that work on gender affairs in countries to see how they work with the HIV/AIDS programs to integrate gender equality. A regional technical assistance partner reviewer remarked that the evaluation would have benefited from greater exploration and inquiry into rights of and gender-based programming for key populations, including transgender persons, a subcategory that is poorly understood in the region. 13 The evaluators, in response to feedback, modified the language in the report from “transition plans” to “ownership/sustainability plans.” 41 APPENDIX C TERMS OF REFERENCE MIDTERM EVALUATION OF THE PEPFAR CARIBBEAN REGIONAL PROGRAM, JANUARY 2012 The PEPFAR Caribbean Regional Program is issuing this competitive request for a midterm evaluation of its initial implementation and progress toward fulfillment of its Partnership Framework. The overall goal of the evaluation is to contribute to the long-term success of the CRP by identifying areas in which the initiative shows a strong likelihood of building a sustainable HIV￾related infrastructure in the region, areas in need of improvement, and factors that are contributing to or impeding progress. Identification and description of factors promoting country ownership and sustainability are of particular interest. The knowledge gained by this evaluation will be used by the U.S. Government Interagency Team responsible for PEPFAR and its regional partners to make midcourse corrections and program improvements that are responsive to both local country needs and PEPFAR’s global objectives. 1. EVALUATION TOPIC Midterm evaluation of the PEPFAR Caribbean Regional Program for period 2010-12. 2. BACKGROUND AND RATIONALE The U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) Caribbean Regional Program is in its third year of implementation. The technical assistance program is part of PEPFAR II, reauthorized by the U.S. Congress in July 2008. The new legislation requires U.S. Government (USG) agencies supporting HIV/AIDS activities abroad to work closely with national, regional, and international partners to develop a non-binding, five-year joint strategic plan or Partnership Framework based upon shared goals and objectives, mutual contributions, and measurable outcomes. The U.S.–Caribbean Regional HIV/AIDS Partnership Framework was developed and signed by 12 participating Caribbean governments and 2 regional partners. The aim of the Framework is to reduce HIV/AIDS incidence and prevalence in the Caribbean region; build the capacity of national governments to develop and maintain sustainable, comprehensive and effective national HIV/AIDS programs; and strengthen the effectiveness of regional coordinating agencies and NGOs to provide quality, cost-effective goods and services to bolster national HIV/AIDS programs. By coordinating HIV/AIDS initiatives and combining USG and Caribbean regional and national resources, this Framework seeks to further strengthen Caribbean national government and regional 42 capacity for a more effective response to mitigate the epidemic’s impact across the region. This Framework represents an agreement between the U.S. Government and Caribbean partners to establish strategic alignment in the fight against HIV/AIDS through joint decision making in setting programmatic priorities and partner contributions. It was developed through extensive consultations with national and regional Caribbean stakeholders, and utilizes and builds upon the respective strengths and technical expertise of all partners.14 The Framework is designed in alignment with the national HIV/AIDS strategic plans of each partner country and with the Caribbean Regional Strategic Framework on HIV and AIDS, 2008-12 (CRSF). The CRSF articulates the vision and collective priorities of Caribbean governments through their membership in The Caribbean Community (CARICOM) and their support for CARICOM’s Pan-Caribbean Partnership against HIV/AIDS (PANCAP). Through mutual ownership and accountability, shared principles and joint oversight, this Framework supports the achievement of Caribbean national and regional goals and objectives as well as PEPFAR’s global goals.15 Further USG investment in the Caribbean, in addition to substantial earlier and ongoing PEPFAR engagement in Haiti, Guyana, and the Dominican Republic, will help ensure that participating Caribbean national governments and regional partners better leverage existing and new resources to improve or scale up essential HIV and AIDS prevention, care, and treatment services. Ultimately, the Framework aims to increase partner countries’ capacity to develop, lead, finance, and implement sustainable, comprehensive and effective national HIV and AIDS programs with the understanding that national governments will increasingly assume primary strategic and financial responsibility over the long-term. Partners, Roles and Contributions Twelve Caribbean countries are signatory partners to this Partnership Framework: Antigua and Barbuda; the Bahamas; Barbados; Belize; Dominica; Grenada; Jamaica; St. Kitts and Nevis; St. Lucia; St. Vincent and the Grenadines; Suriname; and Trinidad and Tobago.16 Ministries of Health, Ministries of Finance, Ministries of Education, National Defense Forces, National AIDS Commissions, and National HIV/AIDS Programs are all important public sector counterparts under this Framework. Two regional organizations mandated to coordinate the Caribbean HIV and AIDS response at the regional level are also signatory partners: the CARICOM Secretariat, on behalf of PANCAP and the Secretariat of the OECS, on behalf of its HIV/AIDS Project Unit (HAPU). In addition, a host of national, regional, and international nongovernmental, civil society, and private sector organizations, many of whom already play an active role in the Caribbean’s HIV and AIDS response, are non-signatory partners and implementers under this Framework. While official signatories are the primary partners under this Framework, other non-signatory organizations may, and in some cases do receive funding or technical assistance as implementing partners. The USG directly funds or provides technical assistance to national and regional NGOs and civil society 14 The U.S. Government, in partnership with PANCAP, hosted a consultative meeting April 29–30, 2009, in St. Lucia to solicit stakeholder input regarding strategy development for this Partnership Framework. Initial input from the St. Lucia meeting was used to develop a first draft of the Framework, which was then circulated in June and July for review by more than 75 Caribbean stakeholders from Ministries of Health and National AIDS Programs in all 12 partner countries as well as CARICOM, the OECS, and numerous nongovernmental and international organizations involved with HIV/AIDS in the Caribbean region. The USG received many substantive responses in writing and by phone and underwent internal deliberations to produce this final draft of the U.S.–Caribbean Regional HIV and AIDS Partnership Framework. 15 PEPFAR’s global goals include treatment for at least 3 million people, prevention of 12 million infections, care for 12 million people, including 5 million orphans and vulnerable children, and training of at least 140,000 new health care workers in HIV/AIDS prevention, treatment, and care. 16 The U.S. Government is developing separate bilateral Partnership Frameworks with the Dominican Republic, Guyana, and Haiti. This U.S.–Caribbean Regional Partnership Framework and the subsequent Implementation Plan seek to incorporate the sharing of best practices and technical expertise from HIV and AIDS programs in these three countries with the wider Caribbean region. 43 organizations where such assistance is consistent with the achievement of Partnership Framework goals and with the national strategic plans of Framework partner governments. In addition, the USG seeks to collaborate with other international partners such as the Joint United Nations Programme on HIV/AIDS (UNAIDS), the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM), the World Bank, the William J. Clinton Foundation (Clinton Foundation), and the PAHO HIV Caribbean Office (PAHO/PHCO) to ensure complementarity of efforts and avoid duplication of activities. Under the leadership of the U.S. Department of State, the following six USG agencies support implementation of this Framework: 1. U.S. Department of State with Missions to: Bridgetown, Barbados, and the OECS; Kingston, Jamaica; Paramaribo, Suriname; Port-of-Spain, Trinidad and Tobago; Nassau, the Bahamas; and Belmopan, Belize. 2. U.S. Agency for International Development with Missions to: Bridgetown, Barbados, and the OECS (Eastern Caribbean Program); and Kingston, Jamaica. 3. U.S. Department. of Health and Human Services: a. Centers for Disease Control and Prevention, Caribbean Regional Program based in Bridgetown, Barbados. b. Health Resources and Services Administration based in Rockville, Maryland. 4. U.S. Peace Corps Eastern Caribbean Regional Program and Suriname, Belize, and Jamaica Country Programs. 5. U.S. Department of Defense. Under this Framework, the United States’ senior representative representing all six USG agencies working on HIV/AIDS in the Caribbean region is the U.S. Ambassador to Barbados and the OECS. This Ambassador, along with the U.S. Ambassadors to Jamaica, Suriname, Trinidad and Tobago, the Bahamas, and Belize, as well as representatives from the USG agencies listed above, partner with the national and regional Caribbean Framework signatory and non-signatory partners to jointly guide and implement this Framework. Caribbean national governments and regional organizations partnering under this Framework have identified and agreed to specific contributions and responsibilities to achieve the Frameworks goals and objectives. All contributions have been approved by the signatory partners. USG contributions were determined by identifying how best to utilize the USG’s comparative advantage and expertise to achieve Framework goals, primarily through a model of technical assistance emphasizing focused technical support, mentoring, and capacity building of health care systems and personnel. To ensure that this Framework accurately reflects Caribbean regional and country-level strategic priorities and represents the best roadmap for the allocation of USG funds and technical assistance, consultations with partners have been ongoing from the PFIP development and onward. Extensive consultations will ensure ongoing dialogue to clarify and update partner contributions and to strike the appropriate balance between bilateral and regional as well as public sector and nongovernmental investments of USG resources. 44 3. EVALUATION OBJECTIVE Purpose: 6. To ascertain if the PEPFAR Caribbean Regional Program is helping countries to reach a sustainable national AIDS program model. 7. To identify the key factors contributing to or impeding project results. 8. To make recommendations for program adjustments. The findings of the evaluation will be used to inform the way forward for USG agencies and partner governments involved in the PEPFAR program. Scope The time period being evaluated is FY 2010 through FY 2012. Evaluation timeline: April through September 2012, final report due October 2012 4. USERS OF THE EVALUATION The users of the evaluation data will be the USG PEPFAR Caribbean interagency team (CDC, DOD, HRSA, USAID, Peace Corps, State) responsible for managing the regional program. The findings will also be shared with OGAC, and signatory country and regional partners. The evaluation will focus on the overall project implementation (big picture, regional program) in partnership framework countries as well as with the two regional agencies of the OECS/HAPU and CARICOM/PANCAP. Evaluation Questions The midterm evaluation will focus on the initial period of implementation of PEPFAR Caribbean Regional Program to determine how well the program is functioning and how likely it is to meet its longer-term objectives. The evaluation will focus on the overall management of the initiative and its potential to facilitate sustainability in the region. Of particular interest to the USG interagency team is the developing potential for: 9. Stewardship—Ownership by the governments of the vision for sector support. 10. Institutional ownership—Local institutions owning the final decisions for each stage of program development; local institutions manage funds and have responsibility for programs. 11. Capabilities—Country leadership having technical and management capabilities to oversee programs and make adjustments and shifts over time. The primary questions the evaluation must address are: • To what extent is the PEPFAR program achieving the Partnership Framework goals (in particular, increased country ownership of AIDS programs and the development of a sustainable Caribbean response to HIV)? • In what ways, if any, are country partners benefiting from the PEPFAR program? • Is the regional approach the most appropriate model? • What aspects of the Program are well implemented and which require improvement? 6. STAKEHOLDERS • Steering Committee members (Ministers and Ambassadors) • Technical Advisory Group (TAG) members • Implementing partners • Civil Society • Governments: − Ministers of Health − Permanent Secretaries of Health − Head of National AIDS Programs − Appropriate staff members within the Ministries of Health − Staff from the Ministries of Finance, Planning 7. EVALUATION DESIGN: APPROACH AND METHODOLOGY Evaluation Methods: Evaluators who take a participatory or utilization-focused approach to developing and carrying out the evaluation will be given priority. Evaluators are expected to refine their proposed approach in consultation with the USG team. Minimally, the evaluation must include, but need not be limited to: • Document review of pertinent records to be provided by the PEPFAR Caribbean Regional Coordinator’s office; • Interviews with the Interagency team members staff and implementing partners; • Interviews with in-country counterparts, including governmental and nongovernmental partners; • Interviews with international donor agencies. Evaluators are expected to observe the OECD DAC 2010 standards in carrying out the evaluation. Evaluation Approach: The evaluator is expected to develop the specifics of the evaluation in collaboration with the PEPFAR Caribbean Regional Program to ensure that PEPFAR needs are met by the outcomes of this process. The evaluation team is expected to propose a detailed work plan for collecting the necessary information and data. The work plan should include proposed plans for the evaluation methods, including sampling, measurement, and data collection, as well as a timeline for completion of the work. The plan should specify when and how the USG interagency team will be engaged in decision making. The plan should address how the proposed plan is culturally and linguistically competent, as well as how DAC guidelines will be observed. 8. DATA SOURCES AND PROCEDURES The PEPFAR Caribbean Regional Coordinator’s office will provide the following list of materials. Additional documents can be requested as necessary: • The Caribbean Regional HIV/AIDS Partnership Framework 2010–2014 • The Caribbean Regional HIV/AIDS Partnership Framework Implementation Plan 45 46 • Annual work plans • Regional Operational Plans • Annual and semi-annual reports • OGAC guidance documents • Project reports Introductory contact information for stakeholders/interviewees will also be provided. Data Review: The evaluation team will review the various project documents and reports included in the list of key documents and any other relevant materials. (A list of key documents to be included in Annex A.) Interviews and Consultation Meetings: The team will also conduct interviews and consultation meetings with stakeholders and key informants, including but not limited to USG country teams (including activity managers), relevant host government authorities, and implementing partners. Field Visits: The team will undertake a multi-country sample review of PEPFAR Caribbean Region Partners, sub-partners, beneficiaries, and relevant USG mission staff. To accomplish these visits, it is possible that the team may split into smaller delegations for the field visits. It is hoped that a minimum of four countries will be visited during the evaluation period. Conference calls should be conducted for countries and partners not visited. 9. QUALITY ASSURANCE A meta-evaluator will be hired to evaluate the work performed by the evaluation team relative to the standards. All information and reports related to the PEPFAR Caribbean Regional midterm evaluation will remain the property of the USG PEPFAR program and for internal use only. Written permission from PEPFAR/OGAC will be necessary for the publication or presentation of any information from the midterm evaluation. 10. EVALUATION ACTIVITIES AND SCHEDULE Location The evaluation will require approximately four to six weeks of field work in five Caribbean countries, in addition to four to six weeks for report compilation. • Countries to be visited are the Bahamas, Barbados, Jamaica, Trinidad and Tobago, and an OECS country. • Agencies to be visited are CDC, USAID, DOD, HRSA, Peace Corps and State in at least two (Barbados and Jamaica) countries. • Interviews with partners and USG representatives will occur in Barbados and in other countries, and by teleconference. Deliverables Deliverables will include: • One draft evaluation report due by October 2012 with a preliminary report in September. The PEPFAR Caribbean Regional team will provide feedback/suggestions, which will be expected to be addressed and reflected in the final report. • One final evaluation report, including graphics, images, and stories to illustrate points, as well as recommendations for future interventions. • One PowerPoint presentation summarizing the findings of the evaluation due in October 2012. • One workshop with the PEPFAR team to help them understand the findings of the evaluation and begin to build lessons for the future. • Various meetings with other stakeholders where the evaluators will present the findings and recommendations. The workshops/meetings will take place in Barbados with conference call-in to PEPFAR staff in other locations. Evaluation Manager The Evaluation Manager is the PEPFAR Coordinator. He will serve as the primary point of contact for the evaluation team and liaison to the USG interagency team. Team Composition We anticipate a two-member team, one lead evaluator with another area expert with evaluation experience. We look for experience in the Caribbean Region, in evaluation of HIV/AIDS programs, and familiarity with USG implementing partners and the PEPFAR platform. Work in evaluation of sustainable programs and country ownership is desirable. 11. ADMINISTRATIVE AND LOGISTICAL SUPPORT Logistical Support The Evaluation team will be responsible for arranging its own: • International travel such as flights to and from evaluation locations • Local transportation to and from in country meetings • Hotel arrangements • Printing and photocopying • Report production • Equipment (computers, storage drives) • Managing of appointments • Data collection and analyses costs The PEPFAR Regional Program Coordinator’s office will provide: • Introductory contact with persons to be interviewed • Report reproduction • Report dissemination • Activity guidance 47 48 ANNEX A. BACKGROUND DOCUMENTS Caribbean Regional Partnership Framework 2010–2014 Caribbean Regional Partnership Framework Implementation Plan Regional Operational Plan (ROP) 10 ROP 11 ROP 12 APR 10 APR 11 Annual Meeting Report 2011 Annual work plans Project reports from agencies OGAC guidance documents addressing country ownership Caribbean Regional Strategic Framework 2008–2012 National Strategic Plans—all partner countries completed 49 APPENDIX D DETAILED EVALUATION METHODOLOGY: DATA COLLECTION, ANALYSIS, AND LIMITATIONS EVALUATION TEAM The evaluation team members have complementary and relevant strengths. Dr. Lynne Franco, the evaluation team leader, has conducted numerous evaluations, and is a quality improvement specialist; Carl Browne is an expert in HIV programming across Latin America and the Caribbean; and Lyn Messner and Dr. Gillian Mason have strong experience in HIV, gender equality, and qualitative data collection and analysis. EVALUATION PROCESS The evaluation team engaged the Caribbean Regional Team and other key stakeholders throughout the evaluation process to ensure that the evaluation findings and report would be as useful as possible. These activities, participants and purpose are described in Table 2. Table 2. Participatory Approach to Evaluation Design Activity Description Participants Purpose One-day evaluation design meeting in Barbados May 9, 2012 10 Caribbean Regional Team members17 To clarify and focus the evaluation, refine the evaluation questions, and finalize the evaluation approach and methodology, including identification of key informants, countries to visit, and expectations for dissemination and use of evaluation findings. Phone or email interviews between May 17 and 23, 2012. 9 Caribbean Regional Team members To get input from agencies that were not represented at the May evaluation design meeting to ensure the evaluation would be useful for all agencies. Phone interviews between May 25 and 28, 2012 5 Technical Advisory Group members To get input from representative stakeholders to ensure the evaluation would be useful for them. 17 USAID, CDC, Peace Corps, and the Department of State were represented. 50 Activity Description Participants Purpose Conference call on June 8, 2012 Caribbean Regional Team To touch base on the evaluation design and interview guides prior to in-country visits In-person briefing on June 22, 2012 Caribbean Regional Team PowerPoint presentation on interim findings from the first three country visits Conference call on July 13, 2012 Caribbean Regional Team For briefing and discussion after the field visits to discuss plans for getting stakeholder input into draft report Conference call on August 7, 2012 Caribbean Regional Team To finalize plans for getting stakeholder input into draft report Conference call on August 24, 2012 Caribbean Regional Team To finalize timeframe for feedback into the draft report SAMPLING AND DATA COLLECTION This performance evaluation collected primary qualitative data using a variety of methods, which are listed below. Document Review: The Caribbean Regional Team provided multiple documents to the evaluation team as background information. The evaluation team reviewed these and several other documents listed in Appendix F of the Midterm Evaluation Report. Semi-structured Interviews: The evaluation team conducted interviews with 105 individuals during field visits to five of the 12 Partnership Framework countries. The team conducted phone interviews with 48 individuals representing a range of stakeholders, including representatives from seven Partnership Framework countries not visited, PANCAP and OECS, Caribbean Regional Team members, international organizations, and regional and international technical assistance partners. Most interviews were with single individuals, but 24 interviews involved two to five people. All phone and in-person interviews were conducted using an interview guide (see Appendix E of this Midterm Evaluation Report) and began with a statement of confidentiality and the option to terminate the interview at any time. The interview guides were reviewed by the Caribbean Regional Team, tested with one Ministry of Health official, and subsequently modified based on those inputs. Additional modifications were made by the team after the first three country visits to improve wording and flow for more focus and clarity. The interview sample was developed with input from the Caribbean Regional Team, who provided the following lists: Caribbean Regional Team, technical assistance partners, Steering Committee, TAG, and national and regional points of contact. The evaluation team solicited input from selected TAG members, national points of contact, and from suggestions during interviews. The full potential list included 326 names. The evaluation team made the final selection based on achieving a representative cross section of key stakeholders related to activities in each country. Table 3 provides an overview of stakeholders interviewed. It should be noted that 12 interviewees, in addition to their categorization in Table 3, were also listed as members of either the Steering Committee or TAG. 51 In-country Visits: The evaluation team made field visits to five countries, Antigua and Barbuda, Barbados, Jamaica, the Bahamas, and Trinidad and Tobago, in June. These countries were selected by the Caribbean Regional Team, based on the following considerations: the mix of USG agencies and implementation mechanisms in the country, size of country and projects, HIV prevalence, and presence of other donors. The evaluation team divided into two smaller teams comprised of one international and one Caribbean evaluator. During each three-day country visit, the team interviewed a wide range of stakeholders (15–23) to provide a 360-degree view of the Caribbean Regional Program in that country. Interviewees were from government (e.g., Ministries [Health, Gender, and Finance], defense forces, National AIDS Programs), U.S Embassies, the Caribbean Regional Team, civil society organizations, technical assistance partners, private sector, academics, and relevant international organizations. Table 3. Key Stakeholders Interviewed by Category Stakeholder Group In Person Interviewees (During Field Visits) Phone Interviewees Total Number and Percentage of Interviewees Ministries of Health 31 12 43 (28.1%) Civil society organizations 26 4 30 (19.6%) PEPFAR country and regional technical assistance partners 13 0 13 (8.5%) Caribbean Regional Team 16 16 32 (20.9%) Other government ministries 8 3 11 (7.2%) PEPFAR international technical assistance partners 5 6 11 (7.2%) Caribbean regional entities (PANCAP, HAPU) 1 5 6 (3.9%) International organizations (PAHO and UNAIDS) 3 1 4 (2.6%) Private sector 2 1 3 (2%) Total 105 48 153 (100%) Phone Interviews: Interviews were conducted by telephone or Skype from late June to early August 2012. For the seven Partnership Framework countries not visited, the evaluation team attempted to interview at least three individuals, generally including the National AIDS Program Manager, a key PEPFAR technical assistance partner (international or regional), and other key stakeholders. Interviewees were determined by the type of assistance and technical areas addressed in that country. For one country, only two people were interviewed due to unavailability of the originally identified person and lack of a suitable substitute in the allotted time. At the regional level, a sample of Steering Committee and TAG members, Caribbean Regional Team members, and 52 individuals at PANCAP, OECS/HAPU were interviewed. The team also interviewed persons from international organizations. Data Collection during the June 2012 Portfolio Review Meeting The evaluation team used the occasion of the June 2012 PEPFAR Regional Program Portfolio Review in Barbados to facilitate a group data collection session with Ministry of Health partners and international and regional technical assistance partners, on the afternoon of June 20, 2012. This session produced short individual appreciative evaluation interviews with 25 participants, and group data from a World Café session on key evaluation themes. In addition, at least one evaluation team member sat in on the Portfolio Review sessions each day (June 19–20) and the one-day USG-only meeting (June 21) and took notes. These sessions gave the team access to the thoughts of those who it did not get a chance to interview such as PEPFAR Deputy Principals. ANALYSIS All interview data were transcribed during the interview with necessary cleaning up and elaboration performed afterward. Information from the document review was extracted and likewise coded, as appropriate. Data were housed on a password-protected space to which only the evaluation team had access. All data regardless of source were coded and analyzed using Dedoose, version 4.3 (2012).18 The evaluation team developed an initial coding book specific to each evaluation question and based on shared experiences during data collection. The code book was refined by using a sample of interviews to ensure that ideas from all stakeholder groups were represented. The evaluation team coded and discussed three interviews together to ensure consistency in coding approach. Coding was done by those directly involved in the interviews, note taking, or reading of the document with additional codes being created as needed. Under the rubric of the four evaluation themes, content and thematic analytic techniques were used by the team to analyze the coded data. In order to facilitate theme development, interviews were categorized based on factors such as role with the Caribbean Regional Program (e.g., Caribbean Regional Team member, government, civil society organization, technical assistance partner, regional entity); associated country (e.g., Barbados, multiple countries); and where possible time with the Caribbean Regional Program (i.e., before PFIP, after PFIP). To establish cross-validation across stakeholders and triangulation, special note was taken during analysis of the specific source of each piece of data. To ensure confidentiality of information, all identifiable information was removed from quotes where needed. STAKEHOLDER FEEDBACK The Caribbean Regional Team and the evaluation team developed a list of 23 representative stakeholders to review and provide feedback on the final draft of the evaluation report. Table 4 provides a summary of stakeholders receiving the report and providing feedback either in writing and/or during a virtual meeting (December 7, 2012) using Adobe Connect. Efforts were made to ensure that stakeholders had received the report and deadlines were extended to facilitate feedback. The virtual meeting, facilitated by an expert curriculum designer who specializes in virtual learning environments, provided a participatory forum for stakeholders to interact with each other and the evaluation team in a dynamic way to discuss evaluation findings and recommendations. In order to maintain confidentiality as much as possible, no PEPFAR team was present on this call. 18 Dedoose, Version 4.3, cross-platform application for managing, analyzing, and presenting quantitative, qualitative and mixed-method data (SocioCultural Research Consultants, LLC, Los Angeles, CA). 53 Table 4. Stakeholders Providing Feedback by Category Stakeholder Group Received Report Provided Feedback Percentage Ministry of Health 10 6 60% Regional technical assistance partner 2 2 80% International technical assistance partner 5 4 80% Regional entity 2 1 50% Civil society 1 0 0% International organization 2 1 50% Total 22 14 64% LIMITATIONS This evaluation was of the Caribbean Regional Program’s efforts to build a sustainable HIV response in the region. It was not an evaluation of any specific agency’s projects, or a thorough organizational assessment of the Program. Even within its boundaries, the evaluation has several limitations: • No baseline information: The evaluation team was tasked with ascertaining whether the Caribbean Regional Program is helping countries reach a sustainable and epidemic-relevant national AIDS program model, but no data are available about where countries were at the start of the Caribbean Regional Program. • Sampling: The evaluation team made every effort to speak to a representative group of stakeholders, and gathered data on all 12 countries in the region, but our sample of CSOs was biased toward recipients of the Ambassador’s Small Grants Program. The evaluation team interviewed at least one CSO in seven of the 12 countries, and at least one non-Small Grant recipient CSO during all five in-country visits. Thus, the evaluation team cannot be certain that the report presents a full understanding of how civil society plays its role or its constraints to participation in country ownership. • Difficulty for some respondents to differentiate between activities supported by PEPFAR and other donors: The Caribbean Regional Program has not always branded its activities as such so some respondents may not have spoken of actions taken by the Program or omitted actions the Program is supporting. Some activities currently supported by the Caribbean Regional Program started pre-PEPFAR, which often contributed to confusion about Caribbean Regional Program and non–Caribbean Regional Program activities. Interviewees also confused Caribbean Regional Program activities with those of others international bodies such as UNAIDS. While this is a result in and of itself, it is possible that respondents have included results related to other donors’ efforts. • Reticence of respondents to speak freely for fear of reprisal: Each respondent was assured of confidentiality in the interview process. While the evaluation team cannot clearly identify an instance, it is possible than some respondents feared reprisal and so did not give a complete 54 picture of their experience with the Caribbean Regional Program. If this happened, there may be a possibility that the evaluation report is missing relevant information. • Data from interviews were not audio-recorded and in some cases, only one interviewer was present at an interview: While most in-person interviews in-country were conducted with a team of two evaluators, not all could be scheduled this way. Phone interviews were done by only one interviewer. While evaluators made the best efforts to take verbatim notes when interviewing alone, it is possible that data capture was not as optimal as with two interviewers. • Challenges in getting broader stakeholder feedback on evaluation report findings, conclusions, and recommendations: Due to the cost of bringing people together, face-to￾face, in such a large and spread-out region, the evaluation team had to rely on email and conference calls as the major means of communication with the Caribbean Regional Team and stakeholder community to validate findings and gather their input and feedback. These mechanisms limit the depth of conversation, understanding of results and recommendations, and possibly engagement with the action plan moving forward to address the recommendations. 55 APPENDIX E DATA COLLECTION TOOLS Midterm Evaluation PEPFAR Caribbean Regional Program IN-DEPTH INTERVIEW GUIDE For PEPFAR Interagency Team Members Country: Date: Name/Title/Affiliation of Respondent: Contact information of Respondent: Interviewer: I. INTRODUCTION: Thank you for making the time to speak with me today. Before we begin this interview, I want to let you know that any information or examples we gather during this interview process will not be attributed to any specific person or institution, unless you tell us that you would be willing to have your responses either be quoted in the evaluation report or otherwise attributed to you. You are also free to not respond to any of our questions or to stop the interview at any time. As you know, we are conducting a midterm evaluation of the PEPFAR Caribbean Regional Program to ascertain how well this program is assisting countries in reaching sustainable and epidemic￾relevant national HIV/AIDS program models, and what could be improved. As part of the PEPFAR team, we greatly appreciate your perspective and views on PEPFAR’s contributions, gaps, and the next steps needed to continue to advance country and regional capacity for sustained efforts in HIV/AIDS. Our interview will take about one hour. Before we begin, do you have any questions about this interview? II. Building Country and Regional Capacity and Ownership A. Most Significant Changes 1. In your estimation, in the past two years what have been the most significant changes in the countries you have worked with or at a regional level to reduce the effects of HIV prevalence or new infections? Why are these changes significant? 2. What is your role in the PEPFAR Caribbean Regional Program? 56 B. Future Vision and Needs for HIV/AIDS in the Caribbean 3. Take a moment, and imagine we are in 2015, and coming together to celebrate the PEPFAR Caribbean Regional Program’s accomplishments. Reflect on what people are saying: a. What changes have been made? b. How were these successes achieved? c. Who was involved? Explain to me what you are seeing. C. Understanding and Commitment to the PEPFAR Partnership Framework 4. How would you rate the level of country commitment to the PF and the PFIP in the countries your agency is working in? What aspects of the PF and PFIP facilitate or hinder this level of commitment? 5. How do you think the PF and PFIP could be made more responsive to the needs of countries in the region? D. Level of Capacity and Ownership 6. What mechanisms or systems are in place in the countries your agency works in that will allow countries to address the AIDS epidemic after PEPFAR ends? What actions or support are needed to move countries further down the trajectory toward sustainability? 7. How does your agency or the PEPFAR Interagency team define country ownership? How does your agency or the PEPFAR Interagency team monitor the level of country ownership and management? E. PEPFAR Contributions 8. In what ways has PEPFAR contributed to countries’ abilities to take on local ownership and better prioritize HIV programs and plans, and implement transparent systems for accountability, decision making, and management of funds? PROBE OR EXPLORE: Stewardship (clear articulation of vision, priorities, and plans; transparent oversight); institutional ownership (local ownership of decisions, management of funds); accountability (transparent systems for accountability between government and civil society and between country leadership and donors; explicit roles and responsibilities with consequences for failure in performance)? 9. In what ways have PEPFAR programs strengthened countries’ abilities to reach those most vulnerable to HIV infection? 10. How can PEPFAR better help countries maintain and strengthen their responses to HIV and AIDS? III. Meeting the Needs of Country and Regional Partners A. Responsiveness of Partnership Framework and Implementation Plan 11. In what ways has PEPFAR support strengthened the ability of countries your agency works with to: a. Prevent HIV, including reduction of stigma and discrimination b. Use information to better manage the epidemic c. Strengthen laboratory capacity for diagnosis and monitoring d. Improve technical and leadership capacity within your health systems 12. How do you feel that PEPFAR support is addressing key priorities of the countries you work in? What other priorities are not being adequately addressed? IV. Appropriateness of the Regional Model A. Regional HIV/AIDS Efforts 13. This program was designed as a regional program, with specific country supports. In what ways is PEPFAR support at a regional level working well to support countries HIV/AIDS programs? What could be improved? PROBE FOR SYNERGIES AND ECONOMIES OF SCALE. 14. In what ways do you feel that the regional program is well understood by your key stakeholders? In what ways do you feel it is misunderstood? V. PEPFAR Program Functioning A. Efficiencies and Effectiveness of Interagency Efforts 15. PEPFAR involves several different U.S. government agencies, each providing a specific type of expertise. In what areas do you feel that collaboration and coordination are most effective across agencies? In which areas are they less effective? 16. How do you feel the current level of collaboration and coordination across agencies is affecting the countries you are working in? 17. What opportunities exist to improve collaboration as an interagency team? How could your agency contribute to improved collaboration and coordinated efforts? 18. What do you feel are the purposes of the Steering Committee and TAG? How have they functioned to serve the program purposes? How can they better support the program? VI. Next Steps A. Suggestions for Improvements 19. Reflecting on your vision for HIV/AIDS program you shared at the beginning of the interview and thinking about all the comments and information you have shared with me so far, please tell 57 58 me three wishes you have for improving PEPFAR’s support to countries and regional partners. This could include removing obstacles or enabling other achievements to follow on the same path. 20. Is there anything else that you want to tell me but didn’t because I did not ask the right question? Any other comments/insights/questions you would like to share? THANK YOU VERY MUCH FOR YOUR TIME AND INSIGHTS. Do you have any questions for me? 59 Midterm Evaluation PEPFAR Caribbean Regional Program IN-DEPTH INTERVIEW GUIDE For Stakeholders Working with PEPFAR Country: Date: Name/Title/Affiliation of Respondent: Contact information of Respondent: Interviewer: I. Introduction Thank you for making the time to speak with me today. Before we begin this interview, I want to let you know that any information or examples we gather during this interview process will not be attributed to any specific person or institution, unless you tell us that you would be willing to have your responses quoted in the evaluation report or otherwise attributed to you. You are also free to not respond to any of our questions or to stop the interview at any time. As you know, we are conducting a midterm evaluation of the PEPFAR Caribbean Regional Program, which serves the 12 countries, PANCAP, and OECS, which are signatories to the PEPFAR Partnership Framework. The purpose of the evaluation is to ascertain how well this program is assisting countries in reaching sustainable and epidemic-relevant national HIV/AIDS program models, and what could be improved. You were suggested as a key person to inform this evaluation and we greatly appreciate your perspective and views on PEPFAR’s contributions, gaps, and next steps needed to continue to advance country and regional capacity for sustained efforts in HIV/AIDS. We understand that PEPFAR is not the only partner providing funding and/or technical assistance in the region, and that we need to understand their contributions in this broader context. Your input will help PEPFAR consider how to adjust the program. Our interview will take about one hour. Before we begin, do you have any questions about this interview? II. Building Country and Regional Capacity and Ownership A. Most Significant Changes 1. In your estimation, in the past two years what have been the most significant changes in your country’s capacity to reduce the effects of HIV prevalence or new infections? Why are these changes significant? 2. What is your role in or relationship with the PEPFAR Caribbean Regional Program? AND/OR What is your role in or relationship with the HIV/AIDS response? 3. FOR SMALL GRANTS RECIPIENTS: Can you tell us about the grant you received through the U.S. Embassy? What was the focus of your program? How was the process for you? 60 B. Future Vision and Needs for HIV/AIDS in the Caribbean 4. Take a moment, and imagine we are in 2015, and coming together to celebrate the accomplishments your country has made in responding to HIV and AIDS. Reflect on what people are saying: a. What changes have been made? b. How were these successes achieved? c. Who was involved? Explain to me what you are seeing. C. Understanding and Commitment to the PEPFAR Partnership Framework 5. What is your understanding and knowledge of the PEPFAR Partnership Framework and its Implementation Plan? 6. In what ways do you feel the Partnership Framework and Implementation Plan are responsive to the needs of your country? How could it be more responsive to your country’s needs? D. Level of Capacity and Ownership 7. A major issue with support to the HIV response is building country's capacity and increasing country's ownership of the various programs. What mechanisms or systems are in place in your country to continue activities after PEPFAR ends? What concerns you? What actions or support are needed to achieve sustainability of national HIV programs? E. PEPFAR Contributions 8. One of the major foci for this evaluation is sustainability and country ownership. Thinking about PEPFAR’s role in this, identify ways PEPFAR has contributed to your country’s ability to take on local ownership and better prioritize HIV programs and plans, and implement transparent systems for accountability, decision making, and management of funds? PROBE OR EXPLORE: Stewardship (clear articulation of vision, priorities and plans; transparent oversight); institutional ownership (local ownership of decisions, management of funds); accountability (transparent systems for accountability between government and civil society and between country leadership and donors; explicit roles and responsibilities with consequences for failure in performance)? 9. In what ways have PEPFAR programs strengthened your country’s ability to reach those most vulnerable to HIV infection? 10. How can PEPFAR better help you maintain and strengthen your country’s ability to respond to HIV and AIDS? III. Meeting the Needs of Country and Regional Partners A. Responsiveness of Partnership Framework and Implementation Plan 11. In what ways has PEPFAR supported your country’s work in: a. Preventing HIV, including reduction of stigma and discrimination b. Using information to better manage the epidemic c. Strengthening laboratory capacity for diagnosis and monitoring d. Improving technical and leadership capacity within your health systems 12. Are there other priorities that PEPFAR could address? IV. Appropriateness of the Regional Model A. Regional HIV/AIDS Efforts 13. This program was designed as a regional program, with specific country supports. In what ways is PEPFAR support at a regional level working well to support your country’s HIV/AIDS programs? What could be improved? PROBE FOR SYNERGIES AND ECONOMIES OF SCALE. V. PEPFAR Program Functioning A. Efficiencies and Effectiveness of Interagency Efforts 14. PEPFAR is comprised of several U.S. Government agencies, each providing a specific type of expertise. Which agencies are supporting HIV efforts in your country? In what ways are these multiple sources of support working well for your country? Are there ways in which having multiple sources of support is not working well for your country? 15. [THIS QUESTION IS TO BE ASKED OF THOSE WHO SIT ON EITHER THE SC OR THE TAG.] What do you feel are the purposes of the Steering Committee OR Technical Advisory Group? In what ways has the Steering Committee OR TAG been successful toward this purpose? What can be improved? VI. Next Steps B. Suggestions for Improvements 16. Reflecting on the vision for HIV/AIDS program you shared at the beginning of the interview and thinking about all the comments and information you have shared with me so far, please tell me three wishes you have for improving PEPFAR’s support your country’s efforts in controlling HIV and AIDS. This could include removing obstacles, or enabling other achievements to follow on the same path. PROBE: In what specific ways can PEPFAR support the achievement of these wishes? 17. Is there anything else that you want to tell me but didn’t because I did not ask the right question? Any other comments/insights/questions you would like to share? THANK YOU VERY MUCH FOR YOUR TIME AND INSIGHTS. Do you have any questions for me? 61 62 63 APPENDIX F DOCUMENTS REVIEWED Ali, Z., A. Nartket, S. Bluemer, A. Lum Lock, G. Deane, and D. Picou. 2009. “Distance Learning and eLearning Resources for Training Healthcare Providers in Trinidad and Tobago.” [Abstract.] Paper presented at 55th Annual Caribbean Health Research Council (CHRC) Conference, 2010, Port-of-Spain, Trinidad and Tobago. April 21–24. Caribbean Community (CARICOM) Pan Caribbean Partnership Against HIV/AIDS (PANCAP). 2008. Caribbean Regional Strategic Framework on HIV and AIDS 2008–2012. Georgetown, Guyana: CARICOM/PANCAP. Caribbean Health Research Council (CHRC). 2012. Project Narrative-Proposal to Establish an Expanded Caribbean HIV/AIDS Monitoring and Evaluation Technical Assistance Unit within the Caribbean Health Research Council (CHRC) Under the President’s Emergency Plan for Relief (PEPFAR). St. Augustine, Trinidad and Tobago: CHRC. Conn, W. 2010. “The Role of PEPFAR in the Caribbean Region.” Slides presented at Healthy Caribbean Coalition and PAHO/WHO Caribbean Civil Society Capacity Building Workshop on Chronic Diseases, Barbados, October 27–28, 2010. Gairy, K., and P. Joseph. 2012. Report of Antigua and Barbuda Training of Trainers in the Teachback Methodology. Gomez, P. 2010. U.S. Ambassador’s HIV Prevention Program Final Report: Prevention Awareness and Stigma Reduction Videos for the Bahamas. Government of Grenada. National HIV and AIDS Strategic Plan 2012–2016. St. George’s: Government of Grenada. Government of Jamaica. 2008. HIV and AIDS in Jamaica National Strategic Plan 2007–2012. Final Draft. Kingston: Government of Jamaica. Government of St. Vincent and the Grenadines. HIV and AIDS National Strategic Plan 2010–2014. Kingstown: Government of St. Vincent and the Grenadines. Government of Suriname. 2009. National Strategic Plan Draft Matrix 2009–2013. Paramaribo: Government of Suriname. Government of Suriname. 2009. Suriname National Strategic Plan 2009–2013. Paramaribo: Government of Suriname. Government of Trinidad and Tobago. 2003. Trinidad and Tobago Five-Year National HIV/AIDS Strategic Plan 2004–2008. Version 2. Port-of-Spain: Government of Trinidad and Tobago. Government of Trinidad and Tobago. 2010. Strengthening the Capacity of the Government of Trinidad and Tobago to Expand the HIV and AIDS Strategic Information, Laboratory Testing and Prevention Programmes. PEPFAR project proposal. Port-of-Spain: Government of Trinidad and Tobago. 64 Health Systems 20/20 and SHOPS. 2012. Antigua and Barbuda Health Systems and Private Sector Assessment 2011. Bethesda, MD: Health Systems 20/20 and SHOPS projects, Abt Associates Inc. Kemp, K. 2011. U.S. Ambassador’s HIV Prevention Program Final Report: High School Peer Education Workshops. Ministry of Health of Antigua and Barbuda. 2007. National Business Plan for Health 2008–2010. St. John’s: Government of Antigua and Barbuda. Ministry of Health, Human Services and Family Affairs. 2003. The St. Lucia National HIV/AIDS Strategic Plan 2005–2009. Castries: Government of St. Lucia. Ministry of Health, Social Transformation and Consumer Affairs. 2011. National Strategic Plan (NSP) for HIV/AIDS in Antigua and Barbuda (2012–2016). St. John’s: Government of Antigua and Barbuda. Ministry of Health of Suriname. 2012. Expanding the Coordinated HIV Prevention, Strategic Information, and Laboratory Programs in Suriname under the President’s Emergency Plan for AIDS Relief (PEPFAR). Prepared for Centers for Disease Control and Prevention (CDC), U.S. Department of Health and Human Services (DHHS). Paramaribo: Government of Suriname. National Advisory Council on HIV/AIDS. n.d. National HIV/AIDS Strategic Plan, Saint Kitts and Nevis, 2009–2013. Draft. Basseterre: Government of Saint Kitts and Nevis. National AIDS Commission. 2006. National Policy on HIV/AIDS. Belize. Belmopan: Government of Belize. National HIV/AIDS Commission. n.d. National Strategic Plan for Prevention and Control of HIV 2008– 2013, Bridgetown: Government of Barbados. United States President’s Emergency Plan for AIDS Relief (PEPFAR). 2009. Guidance for PEPFAR Partnership Frameworks and Partnership Framework Implement Plans. Version 2.0. September 14, 2009. Washington, DC: PEPFAR. PEPFAR. 2009. U.S. President's Emergency Plan for AIDS Relief: Five-Year Strategy. Washington, DC: PEPFAR. PEPFAR. 2011. Fiscal Year 2010: PEPFAR Operational Plan. Washington, DC: PEPFAR. PEPFAR. 2011. PEPFAR: Addressing Gender and HIV/AIDS. Washington, DC: PEPFAR. PEPFAR. 2011. PEPFAR Caribbean Regional Program in Jamaica. November 2011 draft. PEPFAR: Washington, DC. PEPFAR. 2011. PEPFAR FY 2012 COP Guidance. Washington, DC: PEPFAR. PEPFAR. 2011. U.S Caribbean Regional HIV/AIDS Partnership Framework First Annual Meeting and Caribbean Region Chiefs of Mission HIV/AIDS Meeting, Barbados, 12–13 January 2011: Meeting Report. PEPFAR: Washington, DC. PEPFAR Caribbean Regional Program (CRP). 2010. Caribbean Regional HIV and AIDS Partnership Framework 2010–2014: Five-Year Strategic Framework to Support Implementation of Caribbean Regional and National Efforts to Combat HIV and AIDS. Washington, DC: PEPFAR. 65 PEPFAR CRP. 2010. Caribbean Regional HIV and AIDS Partnership Framework Implementation Plan 2010–2014: Five-Year Strategic Framework to Support Implementation of Caribbean Regional and National Efforts to Combat HIV and AIDS. Washington, DC: PEPFAR. PEPFAR CRP. 2010. Fiscal Year 2011 Program Agency Narratives. PEPFAR: Washington, DC. PEPFAR CRP. 2011. Consolidated Country Work Plans. Excel spreadsheets. PEPFAR: Washington, DC. PEPFAR CRP. 2011. FY 2012 Regional Operational Plan (ROP). Governance and Systems Technical Area Narrative. Washington, DC: PEPFAR. PEPFAR CRP. 2011. Partnership Framework Management Operations Manual. Version 4. PEPFAR: Washington, DC. PEPFAR CRP. Caribbean Region FY 2011 Regional Operational Plan (ROP). Executive Summary. Washington, DC: PEPFAR. PEPFAR CRP. Caribbean Region FY 2012 Regional Operational Plan (ROP). Executive Summary. Washington, DC: PEPFAR. PEPFAR CRP. FY 2012 Country Operational Plan (COP) Technical Area Narratives Template (Excel spreadsheet). Washington, DC: PEPFAR. PEPFAR CRP. FY 2012 Regional Operational Plan (ROP). Prevention Technical Area Narrative. Washington, DC: PEPFAR. PEPFAR CRP. “Fiscal Year 2012 Health Systems Strengthening for Improved HIV/AIDS Outcomes, System Responsiveness, Risk Protection and Improved Efficiency.” PowerPoint presentation. Washington, DC: PEPFAR. PEPFAR CRP. 2012. General Talking Points, Antigua, January 2012. PEPFAR: Washington, DC. PEPFAR CRP. 2012. General Talking Points, The Bahamas, January 2012. PEPFAR: Washington, DC. PEPFAR CRP. 2012. General Talking Points, Barbados, January 2012. PEPFAR: Washington, DC. PEPFAR CRP. 2012. General Talking Points, Belize, January 2012. PEPFAR: Washington, DC. PEPFAR CRP. 2012. General Talking Points, Jamaica, January 2012. PEPFAR: Washington, DC. PEPFAR CRP. 2012. General Talking Points, Trinidad & Tobago, January 2012. PEPFAR: Washington, DC. PEPFAR CRP. 2012. PEPFAR Country Work Plans templates. Excel spreadsheets. PEPFAR: Washington, DC. PEPFAR CRP. “Support in Antigua and Barbuda.” PowerPoint presentation. PEPFAR: Washington, DC. PEPFAR CRP. “Support in Bahamas.” PowerPoint presentation. PEPFAR: Washington, DC. PEPFAR CRP. “Support in Belize.” PowerPoint presentation. PEPFAR: Washington, DC. PEPFAR CRP. “Support in Jamaica.” PowerPoint presentation. PEPFAR: Washington, DC. PEPFAR CRP. “Support in Trinidad and Tobago.” PowerPoint presentation. PEPFAR: Washington, DC. 66 PEPFAR. n.d. PEPFAR Supporting Country Ownership, Key to a Sustainable Response. Washington, DC: PEPFAR. Available at http://www.pepfar.gov/documents/organization/195465.pdf (accessed January 2013). Towbin, D. 2011. USAID/Barbados and Eastern Caribbean Partners’ Meeting June 7–9, 2011, Washington DC. Washington, DC: Global Health-Technical Assistance Project. Trinidad and Tobago Health Training Centre (TTTC). n.d. Training Evaluation Report on HIV and Gender-Based Violence Workshop July 5–6, 2011. St. Augustine: TTTC. TTTC. n.d. “Pilot Testing, Trinidad and Tobago.” [Abstract.] St. Augustine: TTTC. TTTC. n.d. Report on the Clinical Preceptorship of Dr. Jose Davy. St. Augustine: TTTC. TTTC. n.d. Workshop Report: HIV, Violence And Psychosocial Issues, Trinidad and Tobago March 23–24, 2011. St. Augustine: TTTC. U.S. Department of State (DS), Embassy of the U.S., Nassau. n.d. Bahamas Small Grants Program Funding Recommendations. Nassau: DS. DS, Embassy of the U.S., Paramaribo. n.d. Ambassadors’ HIV Prevention Program Year End Report FY 2009. Paramaribo, Suriname: DS. DS, Embassy of the U.S., Paramaribo. n.d. PEPFAR Small Grants Program in Suriname: Proposals Selected by Embassy Paramaribo for Funding through the 2012 PEPFAR Small Grants Program. Paramaribo, Suriname: DS. 67 APPENDIX G LIST OF AGENCIES AND TECHNICAL ASSISTANCE PARTNERS ACTIVE AND PRESENT IN THE 12 COUNTRIES 68 P Staff in country A Activities without permanent staff V Vacancy C Closing COUNTRY Antigua and Barbuda Bahamas Barbados Belize Dominica Grenada Jamaica St. Kitts and Nevis St. Lucia St. Vincent and the Grenadines Suriname Trinidad and Tobago PEPFAR Agency Presence State Department P P P P P P P CDC V P P P USAID P P HRSA DOD P P Peace Corps C P P P P C P P C Technical Assistance Partners Presence CDC Technical Assistance Partners Local and Regional Partners MOH (CoAg) P P P P P P PANCAP A A A A A A A A A A A A CHRC A A A A A A A A A A A P CHLI A A A A A A P A A A A A International Partners AFENET A A P A A A P A A A A P NASTAD A P A A A A P A A A A P UCSF P A A A A A A A A A A A DOD Technical Assistance Partners International Partners PSI A A P A P P Charles Drew University P P HRSA Technical Assistance Partners Local and Regional Partners UWI - CHART A A A A A A P A A A A A OECS-HAPU A A A A P A International Partners I-TECH A A A A A A A A A A A A USAID Technical Assistance Partners Local and Regional Partners Caribbean HIV AIDS Alliance P P P P P P P P MOH /Jamaica P International Partners HS 20/20 A A A A A A A A A A SHOPS A A A A A A A A A A MEASURE Evaluation A A A A A Health Policy Project A A P A A P A A A A A World Learning P P C-Change A P State Department Small Grants A A A A A A A A A A A TOTAL Technical Assistance Partners 15 14 14 10 16 14 18 16 15 15 14 14 For more information, please visit aidstar-one.com. AIDSTAR-One John Snow, Inc. 1616 Fort Myer Drive, 16th Floor Arlington, VA 22209 USA Phone: 703-528-7474 Fax: 703-528-7480 Email: info@aidstar-one.com Internet: aidstar-one.com