JULY 2013 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Ruth Kornfield, Ann von Briesen Lewis, Hubert Ibi, and Fidèle Mbadu through the Global Health Technical Assistance Bridge III Project. USAID/DEMOCRATIC REPUBLIC OF THE CONGO: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION Cover Photo by Ruth Kornfield USAID/DEMOCRATIC REPUBLIC OF THE CONGO: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION USAID Contract #AID-OAA-C-13-00032 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 13-B3-013) is available in printed or online versions. Online documents can be located in the GH Tech website at www.ghtechproject.com. Documents are also made available through the Development Experience Clearinghouse (http://dec.usaid.gov). Additional information can be obtained from: GH Tech Bridge 3 Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com This document was submitted by Development and Training Services, Inc., with CAMRIS International and Engility-IRG to the United States Agency for International Development under USAID Contract No. AID-OAA-C-13-00032. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION i ACKNOWLEDGMENTS The evaluation team would like to thank USAID/DRC for their cooperation and support for this evaluation. Our thanks also extend to DRC authorities from the National Program for the Fight Against AIDS (PNLS) and the National Multisectoral Program for the Fight against AIDS (PNMLS) for their cooperation. We would especially like to thank Trad Hatton, ProVIC Project Director, for his time and willingness to explain this complex project and respond to our myriad questions. In addition, we fully appreciate the cooperation of the ProVIC national and provincial technical staff for their availability for interviews and group discussions. We also thank the management staff of the Kinshasa, Matadi, and Katanga ProVIC offices for their logistical support. This evaluation could not have taken place without the willingness of all the NGO sub￾grantees, who not only provided valuable information but who arranged our meetings with project participants. Finally, we extend many thanks to members of the PLWHA self-help groups, the Champion Community Steering Committees, and the child-to-child groups who participated in the group discussions. ii USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION iii CONTENTS ACRONYMS................................................................................................................V EXECUTIVE SUMMARY ....................................................................................... VIII I. INTRODUCTION ................................................................................................1 DEFINITION OF THE PROBLEM................................................................................................................. 1 SUMMARY OF THE PROJECT ..................................................................................................................... 3 PURPOSE OF THE EVALUATION (KEY QUESTIONS) ............................................................................... 7 II. EVALUATION METHODS & LIMITATIONS ..................................................9 EVALUATION DESIGN............................................................................................................................... 9 LIMITATIONS ...........................................................................................................................................11 III. FINDINGS...........................................................................................................13 RELEVANCE .............................................................................................................................................13 EFFECTIVENESS: RESULTS ACHIEVED COMPARED TO TARGETS.........................................................13 INTEGRATION .........................................................................................................................................26 CONTINUUM OF CARE ..........................................................................................................................28 EFFICIENCY..............................................................................................................................................29 GENDER CONSIDERATIONS...................................................................................................................31 IV. CONSIDERATIONS FOR THE INTEGRATION APPROACH....................32 CONSIDERATIONS CONCERNING INTEGRATED PROJECT MANAGEMENT........................................32 CONSIDERATIONS OF INTEGRATION CONCERNING PREVENTION, CARE, AND SUPPORT .............32 V. CONCLUSIONS AND RECOMMENDATIONS ............................................33 CONCLUSIONS .......................................................................................................................................33 RECOMMENDATIONS .............................................................................................................................35 ANNEXES ANNEX I. SCOPE OF WORK.............................................................................................................39 ANNEX II. DOCUMENTS CONSULTED.........................................................................................57 ANNEX III. LIST OF PERSONS CONTACTED ..............................................................................61 ANNEX IV. METHODOLOGY ............................................................................................................64 ANNEX V. DATA COLLECTION TOOLS .....................................................................................73 ANNEX VI. PROVIC TIMELINE ..........................................................................................................83 ANNEX VII. EVOLUTION OF PROVIC RESULTS FRAMEWORK............................................87 ANNEX VIII. SUMMARY OF PROVIC ACTIVITIES BY PEPFAR GOALS ................................. 91 ANNEX IX. DETAIL OF PROVIC PROGRAM ELEMENTS BY REGION, AS OF JUNE 2013 ......................................................................................................................................................................93 ANNEX X. DISCLOSURES OF REAL OR POTENTIAL CONFLICTS OF INTEREST ..........93 iv USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION TABLES Table 1. Results Achieved Compared to Targets ......................................................................14 Table 2: Attrition Rate of Community Outreach Workers ....................................................16 Table 3: Number of Pregnant Women Tested in Kiamvu Hospital Before and After the Start of ProVIC ..................................................................................................................................19 Table 4: Change in Percent of HIV-Positive Test Results in Kiamvu Hospital from 2007 to 2013 ................................................................................................................................................20 Table 5: Key Populations Reached by Second Quarter Year 4 Compared to Target .....21 Table 6. Major factors contributing to or hindering project achievements ........................34 FIGURES Figure 1. Repartition of Funding Year 4 (Total Year 4 Funding USD 3,054,000) .................5 Figure 2. ProVIC Expenditures by Year .........................................................................................5 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION v ACRONYMS AIDS Acquired Immunodeficiency Syndrome ANC Antenatal clinic ART Antiretroviral therapy ARV Antiretroviral drugs BCC Behavior change communication CBO Community-based organization CC Champion Community CCSC Champion Community Steering Committee CDC Centers for Disease Control CD 4 Cluster of differentiation 4 CI Chemonics International CODESA Zonal-level Development Committee COP Chief of party COSA Zonal-level Health Committee CSW Commercial sex worker DHS Demographic Health Survey DRC Democratic Republic of the Congo EGPAF Elizabeth Glaser Pediatrics Foundation FGD Focus group discussion GBV Gender-based violence GDRC Government of the Democratic Republic of the Congo GFATM Global Fund for AIDS, Tuberculosis and Malaria HBC Home-based care HCT HIV counseling and testing HIV Human Immunodeficiency Virus HSS Health systems strengthening IGA Income-generating activities IHAA International HIV/AIDS Alliance INGO International non-governmental organization M&E Monitoring and evaluation MAP Multi AIDS Program (World Bank) MINAS Ministry of Social Affairs, Humanitarian Action and National Solidarity vi USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION MOH GDRC Ministry of Health MOU Memorandum of understanding MSH Management Sciences for Health MSM Men who have sex with men NGO Non-governmental organization OGAC Office of the General AIDS Coordinator OI Opportunistic infection OVC Orphans and vulnerable children PATH Program for Appropriate Technology in Health PEPFAR President’s Emergency Plan for AIDS Relief PITC Provider-initiated testing and counseling PLWHA Persons living with HIV/AIDS PMTCT Prevention of mother-to-child transmission PNLS National Program for the Fight Against AIDS and Sexually Transmitted Diseases (DRC Programme National de la lutte contre le SIDA) PNMLS National Multisectoral Program for the Fight Against AIDS (Programme National du SIDA) PRONANUT National Program for Nutrition of the Ministry of Public Health ProVIC Integrated Project for HIV/AIDS in Congo (Projet Intégré de VIH/SIDA au Congo) PSI Population Services International RIG U.S. Government Regional Inspector General RNOAC/GS Réseau National des Organisations d’Assises Communautaires des Groupes de Soutien des PVV (Network of National Community Organizations of Groups of Support of People Living with HIV/AIDS) SCMS Supply chain management system T&C Testing and counseling UNC University of North Carolina VCT Voluntary counseling and testing USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION vii viii USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION EXECUTIVE SUMMARY BACKGROUND AND CONTEXT Poverty, weak health infrastructure, a young population, ongoing conflict, and pockets of high HIV/AIDS prevalence combine to make the Democratic Republic of Congo (DRC) a priority country in the global fight against HIV/AIDS. The 2007 Demographic Health Survey (DHS) indicated a generalized epidemic with geographic and population differences. The overall HIV prevalence is estimated to be between 1.2% and 1.6%.1 The majority of cases are under the age of 24. Rates for youth are more than twice as high in urban (19%) than in rural areas (8%). Rates are higher among women (1.9%) than men (0.9%). The 2011 Antenatal Clinic (ANC) data showed prevalence rates from 1.35% to 6.86% and pregnant women with a prevalence rate of roughly twice that of other women. Key populations including commercial sex workers, truckers, miners, fishermen, and members of uniformed services have more than triple the rates of the rest of the population. PROJECT DESCRIPTION ProVIC is funded through a $49 million USAID AIDSTAR Task Order for 2009–2014, implemented by the Program for Appropriate Technology in Health (PATH), with consortium members the Elizabeth Glaser Pediatric AIDS Foundation (EGPAF), the International HIV/AIDS Alliance (IHAA), and Chemonics International (CI). The project objective is “to empower local organizations to plan, manage and deliver quality HIV/AIDS services, contribute to the reduction of HIV prevalence and mitigate its impact on people living with AIDS (PLWHA) and their families.” The design integrates an HIV package of: (a) prevention, counseling, and testing; (b) care, support, and treatment; and (c) health systems strengthening. The model is based on a community development strategy consisting of creating “Champion Communities” (CCs) which are to become self-sustaining. CCs are to mobilize the community to identify problems and to develop and implement a work plan to resolve them. The ProVIC idea is that the steering committee would identify HIV as the problem around which they would mobilize the “community.” Fourteen NGO sub-grantees formed 44 CCs in Kinshasa, Katanga, and Bas￾Congo; South Kivu and Kisangani were added without NGO sub-grantees. CC Steering Committees (CCSCs), representing local structures, mobilize communities through volunteer outreach workers who raise HIV awareness and promote HIV testing, and link these communities to prevention, care, and support services. Health system strengthening is supported through 44 cash and in-kind grants to public- and private-sector health facilities and health zone offices in CC catchments. Key populations are also targeted. PURPOSE OF THE EVALUATION (KEY QUESTIONS) The purpose of the evaluation is “to determine the effectiveness of the ProVIC program and, in particular, the CC implementation strategy and also make recommendations for future community mobilization programming, with an emphasis on Orphans and Vulnerable Children (OVC).”2 1 Centers for Disease Control and Prevention, DRC Operational Plan 2010. 2 Scope of Work, GH Tech Bridge III Evaluation Team (see Annex I). USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION ix Before he left his post in May 2013, the Acting Director of the Health Office noted that there is little time remaining for the ProVIC program, and little opportunity for course correction at this stage. He requested, therefore, that the evaluation be “forward-focused,” highlighting both the integrated model of services delivery, and the challenges of building strong community linkages. Following these suggestions, the team has focused on evaluating the Champion Community Strategy for its effectiveness in HIV community mobilization and as a model for linking community and clinic-based services to assure PLWHA and OVC holistic care. The evaluation also looked at the functioning of integration in management, service provision and continuum of care. The key evaluation questions, reordered to reflect their presentation in the present document, are: 1. How has community involvement increased as a result of Champion Communities, and what difference has this made in the health of the target populations? 2. What are the factors that contributed to achieving and not achieving the program’s results? 3. What is the efficacy of ProVIC’s community mobilization approaches for OVC? 4. How has the quality of HIV prevention, care, and treatment services improved? 5. To what extent has ProVIC improved access to services for key populations in the DRC? 6. What are the strengths and weaknesses of the integrated approach? 7. Have costs been saved by using the Champion Community service delivery model? 8. What key aspects of the ProVIC model should be continued or not in future HIV/AIDS programs? METHODOLOGY Method and Design Since this evaluation was to be “forward-focused” and employ a mixed methods approach, the evaluation team followed the direction of USAID/DRC to carry out a “case study” type of design with fewer selected sites which were studied in depth. At each site the team researched each project component, studying how it functioned and was linked to others to make up the large integrated program. Sampling Criteria for site selection were purposive and based on practical feasibility. They included rural and urban representation, geographical and cultural dimensions, proximity to NGO offices, and presence of key populations. Eight sites of different NGOs—two of the five in Kinshasa, all three in Katanga, and all three in Bas-Congo—were evaluated. Data Collection methods included review of documents, open-ended in-depth interviews, focus group discussions (FGDs), direct observation, and photography. Data collection tools were developed for each method and category of people. In each of the eight sites, FGDs were held with Champion Community Steering Committees, OVC, community outreach workers, pregnant women, sex workers, and (MSMs). Interviews were held with national and provincial￾level ProVIC staff, NGO staff, government authorities, health care providers, and key informants including staff of USAID and INGOs. Observation locations included PMTCT consultations, MSM “hot spots,” and truck stops. Data analysis consisted of content analysis using matrixes to summarize data, then arriving at results using triangulation. x USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION MAJOR FINDINGS The findings were analyzed in terms of project relevance, effectiveness, and sustainability, with a focus on the contribution of the Champion Community Strategy and the integrated approaches used. Relevance The project objective remains valid, although the program focus has changed to the prevention of mother to child transmission. The effects of the program are mainly consistent with the intended purpose. Effectiveness The Champion Community (CC) Approach: In answer to the question about community involvement, the “Champion Community” did not function as a “community” engaging the population in their activities and decision-making. It basically consisted of the steering committee and volunteer outreach workers who were recruited to carry out community HIV awareness￾raising, with an emphasis on contacting a large number of people so as to reach ProVIC project target indicators. While the CCSCs were supposed to be independent, they relied heavily on the NGOs. The functions of the CCSCs partly duplicated those of the existing Zonal-level Health Committees (COSAs). Orphans and Vulnerable Children (OVC): ProVIC reached its target OVC beneficiaries who were receiving at least one service depending on need, of which school fees and health care were primary. The psychosocial support component was weak, with inconsistent meetings of child-to-child groups. Program planning for home visits was inadequate, with only two NGO social worker staff for a few hundred children. The potential for Champion Community Steering Committees to contribute support to OVC was demonstrated by three of the nine, who used funds from successful micro-enterprises initially supported by ProVIC to pay school fees for beneficiaries not covered by the NGO. Prevention, Care, and Treatment (PMTCT): Since ProVIC there has been an increase in the demand for HIV screening at the PMTCT centers, possibly partly motivated by the increased availability of ARV drugs in addition to the ProVIC awareness-raising activities. Trainings contributed to improvement of the technical platform and availability of medical supplies, but limited access to CD4 counts, treatment for opportunistic infections (OIs), and quality of consultations remains a problem. The referral system linking clinic patients to NGOs for services and vice versa did not function consistently. PLWHA Self-help groups were organized by the NGOs independently from the CCSC, and although the members were unaware of the CCs, they reported that the groups were helpful. Key Populations: ProVIC has been most effective in its work with MSM. NGOs mainly implement interventions with key populations independently from the Champion Communities. There is no coherent behavior change communication strategy used to target the key populations, and information given is sometimes erroneous or inappropriate. Integration: Integrated project management—although a single management structure facilitates coordination of a single project offering the full HIV package, problems of harmonization of components persisted. HIV health service integration—delivering all HIV services integrated into those of one health center avoids stigmatization of the HIV-positive client and facilitates follow- USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION xi up. Continuum of care—the links between community-based and clinic-based services were not consistently strong enough to assure continuum of care for PLWHA. Cost saving: Within the limitations of this evaluation, the team was unable to reliably establish cost of comparable service delivery in the DRC. The design and goal of the Champion Community model is not to maximize cost efficiency, but to fight against HIV/AIDS. Reliance on volunteer workers appears to be low cost, but the support costs of the supervisory NGO overhead, training, material, transport and staff must be considered in the balance. Overall, with only ProVIC information, the team concludes that the approach does not (nor is it designed to) maximize cost saving in service delivery. Sustainability: Low capacity, ineffective income-generating activities (IGAs), and CCs’ strong dependence on NGOs make it unlikely that HIV activities would persist beyond ProVIC. CONCLUSIONS The Champion Community approach was intended to mobilize communities to address their HIV/AIDS problems, but in practice it was a program run by NGOs, and a steering committee with volunteer outreach workers that were highly dependent on the NGOs for their activities. The key approach of involving significant numbers of the population in the community to address their HIV/AIDS problem did not appear to work. However, the steering committee through volunteer outreach workers did contribute to increasing community HIV/AIDS awareness and testing. Community mobilization has an important role to play in HIV programs in the DRC. However, the Champion Community Model as such is not the most effective strategy to use where HIV is not a priority problem. Large integrated projects may compromise quality, and while a single management structure reduces coordination problems, program harmonization and community￾clinic linkages remain a problem. RECOMMENDATIONS Overall: Place zone at the center of programming to include clinical services, systems strengthening, and a robust community component, with special programs for key populations, OVCs, care, and support. Realize the benefits of the capacity-building investment to date and continue support to stronger NGOs. Keep the community involved and strengthen Ministry of Health (MOH) health committees. Champion Community: For sustainability, integrate the CCSCs into the Zonal-level Health Committees; add activities for HIV and promotion of PMTCT; increase the number of voluntary outreach workers for the extra activities; and strengthen the zone supervisory and oversight roles for the health committee and outreach workers. Orphans and Vulnerable children: Education—to promote sustainability, USAID should encourage the state to provide free education as required by the constitution; promote the use of block grants; and promote community support and school IGAs for school fee waivers for OVC. Health care—to ensure access to free health care, health care funding should be pooled and paid directly to health facilities to be used to support all OVC in the health zone. The Ministry of Health and Ministry of Social Affairs should award certificates of indigence to OVC for free care. Psychosocial support and child protection should be strengthened. xii USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION PMTCT: Strengthen the promotion of male participation in PMTCT and monitoring for quality of services. PEPFAR and USAID should include indicators for quality of services. Care and Support to PLWHA: Strengthen services to include: (a) treatment for opportunistic infections as well as ARV; (b) provision for cluster of differentiation 4 (CD4) counts for all PWLHA; (c) clinic-based and community-based support for adherence to ARVs; and (d) promotion and strengthening of self-support groups. Key Populations: A technically sound, coherent gender-based behavior change communications program differentiated for each type of key population should be developed. MSM interventions should continue to be strengthened and advocacy activities increased to reduce stigmatization. Strengthening Integration: Include a strong mechanism for cooperation between the contracting companies and coordination of components, no matter whether the program providing the complete HIV package is managed through one very large project covering all components or through individual projects for each component. Promote integration of clinic￾based services and use of mobile HIV testing units for key populations. Improve linkages between community- and clinic-based services through innovative mechanisms that strengthen referral systems. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 1 I. INTRODUCTION DEFINITION OF THE PROBLEM Poverty, weak health infrastructure, a young population, ongoing conflict, and pockets of high HIV/AIDS prevalence combine to make the DRC a priority country in the global fight against HIV/AIDS. The 2007 Demographic Health Survey (DHS) in the DRC indicated a generalized HIV/AIDS epidemic with stark geographic and population differences. The majority of new HIV/AIDS cases are among people under 24 years of age. The overall HIV prevalence in the DRC is estimated to be between 1.2% and 1.6%%.3 Rates are twice as high in urban (19%) than in rural areas (8%) and higher among women (1.9%) than men (.9%) The 2011 ANC data showed urban prevalence rates ranging from 1.35% to 6.86%. Pregnant women are particularly at risk; antenatal clinic (ANC) surveillance data indicate that pregnant women had a prevalence rate of roughly twice that of other women. High-risk and high-prevalence populations congregate in geographic “hot spots,” such as border crossings, transport corridors, ports, and regions with a large military presence. The already elevated rates of most key populations including commercial sex workers (CSWs), truckers, miners, and members of the uniformed services are often more than triple or quadruple the rates of the rest of the country. Truckers have a national prevalence rate of 3.3%, but in Katanga, long-haul truckers from southern African countries have an HIV prevalence rate of 7.8%. A 2008 survey of the military in Kinshasa indicated 7.5% prevalence among women and 3.6% among men. A 2006 bio-sero survey found a prevalence rate of 16.9% among CSWs, and rates in the provincial capitals of Katanga is 23.3%. Fifty-five percent of miners; 32.9% of the military, 75.1% of street boys, and 81.1% of street girls report multiple sex partners within the past 12 months, therefore increasing their risk for transmission. The HIV epidemic has left more than 51,000 additional orphans and vulnerable children.4 The 2007 DHS found that 13.1% of children did not live with either biological parent. Government of DRC Health Strategies and Structure Budgetary support from the Government of the DRC for health is low. In 2012, health accounted for only 5.4% of the annual budget, which is far below the 15% recommended standard. Bilateral and multilateral donors largely support health care through projects contracted to international NGOs which provide a minimum package of health services offered in the health zones. The fight against HIV/AIDS in the DRC is coordinated by the National Multisectoral Program for the Fight Against AIDS (PNMLS), which includes all government ministries and representatives of civil society. Chaired by the head of state, PNMLS directs multisectoral responses to HIV/AIDS. Within the Ministry of Health, the National Program for the Fight Against AIDS and Sexually Transmitted Diseases (PNLS) directs the response to the HIV/AIDS epidemic and is coordinated by the Ministry of Health. PNLS and PNMLS are represented at the national and provincial levels. The Minister of Social Affairs, Humanitarian Action and National Solidarity (MINAS) is responsible for overseeing orphans and vulnerable children (OVC). In the absence of a national government-supported social security plan, provision of basic social services is grossly 3 Centers for Disease Control and Prevention, DRC Operational Plan 2010. 4Id. 2 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION inadequate and of poor quality. Moreover, because there is no national policy regarding support for OVC, or a national action plan, current interventions for OVC largely depend on donor support and donor priorities, and are ad hoc, poorly coordinated, piecemeal, and insufficient. The health system in the DRC has three tiers: (1) the central level which includes the MOH, the Secretary General of the MOH, and directorates of national disease-specific programs; (2) the intermediate level composed of 11 provincial health departments and 48 administrative health districts; and (3) the peripheral level with 515 Health Zones containing over 8,000 health centers. On the zonal level there are staff responsible for community health who supervise two types of volunteer outreach workers—the community health workers who conduct health promotion and community mobilization activities, and the community treatment workers who deliver a limited set of medical interventions such as treatment of diarrhea, fever, referral of malnourished children to health facilities, and distribution of contraceptives. Zonal-level Health Committees (COSAs) and/or Community Development Committees (CODESAs) supervised by the Health Zone mobilize voluntary outreach workers for health awareness campaigns. This system is effective for community health education when it functions properly, but many of the committees are dysfunctional. U.S. Government Response President’s Emergency Plan for AIDS Relief (PEPFAR) The U.S. Government and the GDRC have endorsed a five-year strategy, “to jointly implement national goals to reduce the transmission of HIV and to minimize the negative impacts on the Congolese people.” The components are: (1) prevention; (2) treatment, care, and support; (3) care of orphans and vulnerable children; and (4) health systems strengthening.5 PEPFAR DRC partners include USAID, the Department of Health and Human Services through the Centers for Disease Control and Prevention (CDC), the Department of State, and the Department of Defense (DOD). PEPFAR DRC activities began in 2007, and the overall budget has grown from $64 million to a total of $150 million. Current budgets are based on $56.2 million per year. CDC has led in clinical treatment, DOD in prevention in the high-risk uniformed services population, and USAID in community mobilization and involvement. The two major PEPFAR USAID programs are the USAID Integrated Program for HIV/AIDS in Congo (known by its French acronym ProVIC), which is the largest PEPFAR program, and the Integrated Health Program (IHP, also known as ProSANI), which mainly focuses on other health issues. Other PEPFAR partners, some of which work with ProVIC, include Population Services International (PSI) (condom social marketing); AXXes (PMTCT, capacity-building); C-Change (Behavior Change Communication (CC), social mobilization, training); Management Sciences for Health (mechanisms for managing drugs and lab products, pharmaceutical policies); University of North Carolina/Kinshasa School of Public Health (PMTCT); and the World Food Program. Multilateral and Other Donors Development donor partners, including private-sector partners, faith-based, community-based, and non-governmental organizations, work with the GDRC to improve prevention, treatment, and care activities. Other donors during the life of ProVIC have included the Global Fund to 5 The Partnership Framework Document to Support Implementation of the DRC National HIV and AIDS Response, April 2010. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 3 Fight AIDS, Tuberculosis, and Malaria (GFATM), the World Bank Multi-country AIDS Program (MAP), and United Nations agencies, especially UNAIDS. SUMMARY OF THE PROJECT Objective, Implementation Structure The DRC Integrated HIV/AIDS Project, Projet Integré de VIH/SIDA au Congo (ProVIC), is funded through a USAID AIDSTAR Task Order. It is a five-year (2009–2014), $49 million project implemented by the Program for Appropriate Technology in Health (PATH), with consortium members the Elizabeth Glaser Pediatric AIDS Foundation (EGPAF), the International HIV/AIDS Alliance (IHAA), and Chemonics International (CI). Key GDRC partners are the MOH, the Ministry of Social Affairs (MINAS), PNLS, and PLMLS. ProVIC was designed to implement the prevention, care and support, and service delivery program through community participation and mobilization, with a primary focus on most at-risk populations (commercial sex workers, truckers, fishermen, miners, men who have sex with men) and a secondary focus on the general population. In the past year, ProVIC added treatment to its continuum of care, and the project is undergoing a change in focus following the PEPFAR emphasis on PMTCT (see Annex VI for Program Timetable). According to PATH, the objective of the project is “to empower local organizations to plan, manage and deliver quality HIV/AIDS services,” contribute to the reduction of incidence and prevalence of HIV, and mitigate its impact on people living with AIDS (PLWHA) and their families. The original Results Framework of ProVIC identified the U.S. Government goal: basic health conditions of Congolese people improved, and the project objective: incidence and prevalence of HIV/AIDS reduced and its impact on PLWHA and their families mitigated. The project design integrates an HIV package of (1) HCT and prevention; (2) care, support, and treatment; and (3) health systems strengthening. Three intermediate results are as follows (see Annex VIII for Summary of Activities):  IR1: HCT and Prevention Services expanded and improved in target areas o Sub IR 1.1 Community’s abilities to develop and implement prevention strategies strengthened o Sub IR 1.2 Community and facility-based HCT services increased and enhanced o Sub IR 1.3. PMTCT services improved o Sub IR 1.4 Community and facility-based GBV prevention and response services strengthened (added in 2013 -see Annex VII for evolution of results framework)  IR2: Care, support and treatment for PLWHA and OVC improved in target areas o Sub IR 2.1 Palliative care strengthened o Sub IR 2.2 Care and support for OVC strengthened o Sub IR 2.3 HIV treatment improved in target areas (added March 2013)  IR 3: Strengthening of health systems supported o Sub IR 3.1 Capacity of provincial government health systems supported o Sub IR 3.2 Capacity of NGO providers improved o Sub IR 3.3 Strategic information systems at community and facility strengthened 4 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION Champion Community The Champion Community model is at the core of the ProVIC approach. It is designed to reinvigorate social activism and involve community leaders to participate in identifying needs, planning solutions, and implementing projects, using the fight against HIV/AIDS as a common issue. The model is based on general community development strategy adapted from a water and sanitation project in Madagascar. It consists of creating “communities” within designated sectors of health zones, each of which covers a population of 40,000 to 60,000. Within each of these areas, the project implementers (sub-grantee NGOs) form community steering committees composed of representatives of local associations and institutions. Each steering committee is to hold meetings with the population within their catchment area at which they identify key problems, develop a workplan to address them through mobilization of the community by using links to their associations and institutions to do so. According to the community development model, the community chooses whatever problem it considers most pertinent, and the project supports the activity one way or another. If the community fulfills the objectives of its workplan, the community receives a monetary reward to be used for the betterment of the community and is named a “Champion Community.” In the application of the model by ProVIC, the Champion Communities are supposed to address problems that concern HIV/AIDS, whether or not the population perceives AIDS as a priority problem. Each steering committee is supposed to be actively involved in mobilizing the community for HIV prevention through HIV awareness-raising and promotion of HIV testing and counseling, and for care and support for PLWHA and OVC by linking them to clinic- and community-based services. The steering committees are to do this through volunteer outreach workers selected from their respective associations and institutions. The committees’ workplans also allow them to address other problems they identify along with the HIV community activities. ProVIC has formed 44 Champion Communities which are supported by 14 NGO sub￾grantees in five geographic areas, linking communities to prevention, care, and support. Additionally, ProVIC supports health system strengthening and the continuum of care through cash and in-kind grants to public- and private-sector health facilities and health zone offices in areas with Champion Communities. Annual Fixed Obligation Grants support salaries, supplies, and equipment at 24 private-sector hospitals, and annual “Accords” for locally procured supplies and equipment assist 20 government health facilities. Since 2012, with a strategic shift toward support of Health Zones, ProVIC began making in-kind grants valued at $5,000 to 35 Health Zone offices. (Annex IX: Details of ProVIC Program Elements by Region) The project structure consists of a ProVIC national office in Kinshasa with provincial offices in Katanga (Lubumbashi), Bas-Congo (Matadi), Province Oriental (Kisangani), and South Kivu (Bukavu) (which has been closed due to lack of security). In addition to overseeing the provincial offices, the national office manages accords with 23 health facilities (in-kind support) and 32 Health Zones (small annual grants in cash), and Annual Fixed Obligations with 22 private hospitals. (Annex IX) The regional offices include staff from each consortium member responsible for its project component. Each regional office sub-grants to NGO partners, which coordinate from 4–13 Champion Communities and provide support services to OVC and PLWHA. The Champion Communities oversee the community outreach work of the volunteer outreach workers. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 5 Costs The original Task Order signed by USAID and PATH on September 30, 2009, provided $44,873,203 over the 5-year life of the project. Key personnel positions were negotiated among consortium members, and in an unusual move, subcontractor Chemonics International was chosen to fill the Chief of Party position. An additional $5 million was made available in April 2011 to accelerate the change to the PMTCT platform, support the move to open a new regional office in Kisangani, and move toward direct support to Health Zones. Figure 1 shows the relative proportion of ProVIC funds for each major program component as of June 2013. Figure 1. Repartition of Funding Year 4 (Total Year 4 Funding USD 3,054,000) According to our interview with the DRC PEPFAR Coordinator, the PEPFAR budget figures are approximately $56.2m per year. These are planning figures and may fluctuate as “plus ups” and new initiatives may become available. Figure 1 shows that the largest percentage of the budget is for NGOs, followed by support for private facilities, government facilities, and Health Zones. This chart does not include an estimated $500,0006 in training costs for the year. Training programs are provided at virtually all levels of ProVIC programming, including the estimated 1,800 outreach workers, MOH officials and technicians, NGO technical staff and Health Zone counterparts, Champion Communities, etc. Figure 2 shows that program expenditures by (program) year reflect the initial weak performance and very late start-up in the initial months of implementation and subsequent pipeline and non-performance issues. The RIG Audit Report of November 2011 that specified deficits, problems, and issues, the replacement of the Chief of Party in January 2012, and the arrival of the new COP in April 2012 began a new chapter of ProVIC performance. 6 Reported by ProVIC chief of party. $5,845 $8,756 $12,427 $14,069 $8,475 $0 $2,000 $4,000 $6,000 $8,000 $10,000 $12,000 $14,000 $16,000 Yr 1 Yr 2 Yr 3 Yr 4 Yr 5 (6 mos) ProVIC Program Costs (000) by year Figure 2. ProVIC Expenditures by Year 6 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION Programmatic Milestones and Changes Since ProVIC began in September 2009, the program has been challenged to adapt to a number of changes, due to both internal problems and external requirements of USAID and PEPFAR. Changes include key personnel, locations, target sites, provincial offices, changes in the consortium, changes in MOH partners, and changes in donor emphasis, new Intermediate Results, changes in funding, and changed project duration as well as dealing with the problem of the theft of testing supplies, and other events. (See complete timetable in Annex VI) Underlying Theory of Change The underlying theory of change in program design was based on a series of implicit assumptions and built on a foundation of USAID-supported community health work in the DRC, implemented by Family Health International (FHI), Catholic Relief Services (CRS), and other partners. The theory of change is a representation of how an intervention is expected to lead to desired results. A simple version of the ProVIC theory of change is as follows: Overall impact is intended to be the reduced incidence and prevalence of HIV/AIDS, which will eventually lead to an AIDS-free generation in the DRC. Assumptions implicit in the design of ProVIC include:  Alignment with GDRC PNLS and PNMLS policies and strategies.  Adaptation of the Chemonics Champion Community model to the fight against HIV/AIDS in a low-prevalence country.  Champion Communities can effectively identify and reach key populations.  Champion Communities Steering Committees will be motivated by technical support and prizes to identify and undertake small-scale community development projects directly or indirectly related to HIV/AIDS. There will be continued PEPFAR and GDRC support for community-based mobilization.  There is a need for a new representational community structure in the absence of functioning community-based health systems.  There needs to be efficient leveraging of skills and resources among donors (GFATM) and partners. Inputs: 1. USAID PEPFAR financial support to PATH-led consortium 2. Accords and FOGss [define?] to clinical services Activities: 1. Sub contracts to 14 Congolese NGO partners that manage support to OVC, PLWHA, MARPS 2. NGOs develop Champion Communitites; train outreach workers 3. Clinics get training, test kits, supplies, equipment Outputs: 1. CCSC design and implement development projects and win prizes 2. Outreach workers from CCSC motivate MARPS and general population to get tested 3. Care, support, and treatment services are available and improved Outcomes: 1. CCSC serves as catalyst for other community development projects 2. Key populations monitor their serostatus, practice safe sex; pregnant women and their partners attend ANC and PMCTC 3. Clinics are stocked, staffed with trained personnel USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 7 PURPOSE OF THE EVALUATION (KEY QUESTIONS) “The purpose of this evaluation is to determine the effectiveness of the ProVIC program and, in particular, the Champion Community implementation strategy and also make recommendations for future community mobilization programming with an emphasis on Orphans and Vulnerable Children (OVC).”7 This performance evaluation was intended to be a mid-term evaluation, but implementation slipped to halfway into the fourth year of the five-year program. Before he left his post in May 2013, the Acting Director of the USAID/DRC Health Office noted that there is little time remaining for the ProVIC program, and little opportunity for course correction at this stage. He requested, therefore, that the evaluation be “forward-focused,” highlighting both the integrated model of services delivery, and the challenges of building strong community linkages. Following this guidance, the evaluation has focused on evaluating the Champion Community Strategy for its effectiveness in HIV community mobilization and as a model for linking community- and clinic-based services to assure PLWHA and OVC holistic care. The evaluation also centered on looking at the functioning of integration on various levels—management, service provision, and continuum of care. The key evaluation questions are: 1. How has community involvement increased as a result of Champion Communities, and what difference has this made in the health of the target populations? 2. What are the strengths and weaknesses of the integrated approach? 3. How has the quality of HIV prevention, care, and treatment services improved? 4. To what extent has ProVIC improved access to services for key populations in the DRC? 5. What key aspects of the ProVIC model should be continued or not in future HIV/AIDS programs? 6. What are the factors that contributed to achieving and not achieving the program’s results? 7. What is the efficacy of ProVIC’s community mobilization approaches for OVC? 8. Have costs been saved by using the Champion Community service delivery model? 7Scope of Work, GH Tech Bridge III Evaluation Team (see Annex 1). 8 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 9 II. EVALUATION METHODS & LIMITATIONS EVALUATION DESIGN Given that this evaluation was to be “forward-focused” and employ a mixed methods approach, the evaluation team followed the direction of USAID/DRC suggestion to carry out a “case study” type of design in which the team would select fewer sights and study them in depth. In this way, in each site the team could research each project component in relation to the Champion Community Strategy, studying how the components functioned and were linked to make up the large integrated program. The evaluation team, consisting of four consultants—two American women and two Congolese men (Annex IV p. 6)—used a variety of data collection methods and developed specific data collection tools appropriate for each method. Data points were verified and triangulated in all cases, or identified by source whenever possible. All findings were evidence-based, and recommendations were based on analysis of the findings. In preparation for the evaluation, the team reviewed the key questions and identified sources and areas of emphasis (Annex IV matrix A), and prepared a more detailed matrix identifying approach, method, source, and analysis (Annex IV matrix B). Data Collection Methods Data collection methods included:  Review of documents (see Annex II)  Review of project documentation at the NGO  Formal structured open-ended interviews  Semi-structured individual interviews  Structured focus group discussions (FGDs)  Semi-structured informal group discussions  SWOT (strengths, weaknesses, opportunities, and threats) analysis group discussions  Direct observation  Photos, charts, and videos The team developed six data collection tools: three focus group discussion (FGD) guides—one for the Champion Community Steering Committees, one for project beneficiaries, and one for community outreach workers and peer educators; and three interview guides—one for DRC Government authorities in the Ministry of Health and Ministry of Social Affairs, one for INGOs and donors, and one for ProVIC staff and NGO sub-grantees. (See Annex V for data collection tools.) Data were collected in French, Swahili, Lingala, Kicongo, and English. For the local languages, the Congolese consultants led the interviews or discussions, and when a Congolese and American evaluator conducted interviews or discussions together with non-French speakers, the Congolese consultant translated for the American. Sometimes a FGD participant was able to translate the local language into French for the facilitators. 10 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION Sampling Sampling was purposeful and based on practical feasibility. The Scope of Work designated that data would be collected from ProVIC sites in Kinshasa, Katanga, and Bas-Congo because, of the five ProVIC target areas, these had the most activities going on at the time of the evaluation. South Kivu was closing down its project activities because of insecurity, and the ProVIC activities in Kisangani had not been functioning long enough to allow for a viable evaluation. The team obtained primary data at the community, district, provincial, and national levels in the areas visited. Secondary data were gathered on the ProVIC program in Bukavu and Kisangani. The team visited a total of eight sites, each one implemented by a different NGO sub-grantee, two in Kinshasa, three in Katanga, and three in Bas-Congo. (Annex IV: Table 2) Site selection took into account rural and urban differences, geographical and cultural dimensions, proximity bias, and the presence or absence of key populations. Kinshasa, Katanga, and Bas-Congo each represent different cultural areas of Congo with different economic bases, ethnicities, and languages. Since the project is meant to be mainly urban-centered, the majority of sites were selected in urban areas—six in larger cities, one in a smaller city (Kasumbalesa), and one rural site for comparison (Seke Banza in Bas-Congo). Since the influence of the provincial ProVIC office could be greater in sites that are in closer proximity to the project office because of facility of supervision, sites were selected at both close and distant proximity from the provincial office. For those sites at a distance, site selection was influenced by practical considerations of time limitation and quality of roads. The team visited hot spots of key populations in the urban areas and a large truck stop at the Zambian border in Kasumbalesa. (See Annex IV for sites visited.) All the NGOS in Bas-Congo (two NGOs) and Katanga (four NGOs) were evaluated. In Kinshasa the team selected two out of the five NGO sub-grantees. To permit a range of types, one NGO selected was a well-established PLWHA umbrella organization, while the other was responsible for the MSM activities and mobile HIV testing unit. Criteria for selection of the Champion Communities were based on availability of steering committee members for focus group discussions and on how well they functioned so that data was collected from both stronger and weaker Champion Communities. Data Collection Process At each site data were collected concerning every component from all involved stakeholders, including project beneficiaries. Data were collected in steps: first from the ProVIC office staff (national and provincial) for an explanation of the entire project, the Champion Community Strategy, and issues concerning integration, followed by individual staff interviews on each component. Second, similar data were collected from the NGO staff. The NGOs assisted in arranging meetings for focus group discussions with the Champion Community Steering Committees, the PLWHA self-help groups, the child-to-child groups, and the volunteer community outreach workers. A total of 22 focus group discussions were conducted (Annex IV: Tables 3, 4, 7). The medical expert of the team focused on the PMTCT and health systems strengthening components. He made structured observations of antenatal/PMTCT consultations, and conducted two focus group discussions with pregnant women clients and 20 interviews with health care providers, including the supervisors of the Health Zone volunteer community outreach workers. (Annex IV: Table 6). Interviews were also held with Ministry of Public Health USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 11 officials at the national, provincial and Health Zone level as well as government authorities in the Ministry of Social Affairs (Annex IV: Table 4 and Annex III list of contacts for details). The evaluation team selected MSM sites to visit wherever they existed and was able to evaluate MSM activities in Kinshasa, Lubumbashi, and Matadi. Group interviews were held with peer educator members of MSM networks and observations were made of a MSM hot spot in Kinshasa where there was a nighttime mobile HIV testing and counseling unit, and of a hair salon in Matadi set up by a MSM network with ProVIC IGA seed money. The team had focus group discussions with two groups of sex workers in Kasumbalesa. They also visited a large truck stop in Kasumbalesa on the border with Zambia, where observations were made of community outreach workers raising awareness of and distributing condoms to both truckers and sex workers. They held informal interviews with the truckers, sex workers, and the community outreach workers. A total of 13 structured observations were conducted of ongoing project activities, including of child-to-child group meetings, PMTCT consultations, outreach workers, and MSM peer educators conducting community awareness and observing the ProVIC-supported Health Zone training in Boma. (Annex IV: Table 5) Additional interviews were conducted with staff from USAID and INGOs which have either collaborated with ProVIC, been part of ProVIC, or are involved with community mobilization for HIV prevention, care, and support such as Catholic Relief Services(CRS), Population Services International (PSI), the University of North Carolina(UNC), and FHI 360. (For details, see Annex III: List of contacts.) Data Analysis At the end of each day, the evaluation team met together to debrief on their findings, discuss linkages of the various project components as they became evident or not, and developed working hypotheses to guide further data collection the following day. Interview guides were modified accordingly. Content analysis was conducted of qualitative data using matrixes to summarize data. Gender Considerations The team conducted the evaluation with a view through the gender lens. Gender implications of ProVIC effectiveness, efficiency, and relevance are integrated throughout the report. Data are gender-disaggregated whenever possible, and analysis includes the impact of gender, particularly regarding the significance of gender roles in PMTCT service delivery. LIMITATIONS The evaluation took place from May 28 to July 6, 2013, in the DRC. By the latter half of program activities, significant changes had been made to the original ProVIC Champion Community Strategy design, which prevented the evaluation team from seeing the strategy implementation at the time that all the original components were fully funded. There was no baseline study or mid-term evaluation for reference and comparison. To mitigate these limitations, the team followed the direction USAID/DRC and adopted a forward-looking approach to the evaluation, analyzing the project from a programmatic perspective. The team also tried to discern which current project weaknesses were due to the pivot shift, and took that into account in the evaluation. Access to field sites was limited due to uncertainties of plane schedules, road conditions, and security situations, which unfortunately limited time for data collection. 12 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 13 III. FINDINGS Findings were analyzed in terms of project relevance and the effectiveness and sustainability of each project component, with a focus on the contribution of the Champion Community Strategy. RELEVANCE The overall goal as stated in the PATH proposal (2009) is “to reduce the incidence and prevalence of HIV/AIDS and mitigate its impact on people living with HIV/AIDS (PLWHA) and their families. This entails reducing transmission among key populations as well as people living with HIV/AIDS.” The goal of the project remains relevant, and the activities of the project were basically consistent with the original objectives and intermediate results. Prevalence rates among most at-risk populations remain disproportionately high, and ProVIC consistently targets these key populations for prevention, care, and treatment support. By the close of the second program year, ProVIC was well-established and attaining or exceeding targets for raising awareness, HIV counseling, testing, and prevention services.8 However, changing donor priorities and the PEPFAR shift to the PMTCT platform reduced the emphasis and financial support for community mobilization for HIV counseling and testing. Under the PMTCT “pivot” prevention is focused on increased support and services to women of reproductive age and their partners and other key populations and away from general awareness campaigns and promotion of counseling and testing. ProVIC adapted to programmatic shifts while continuing ongoing health systems strengthening and technical support and training to PNLS and NGO sub-grantees. However, the evaluation found little evidence that the geographically defined Champion Community was the most efficient and effective mechanism to reach key populations such as MSM, truckers and commercial sex workers. The transient, occupationally defined, and stigmatized at-risk populations are not participants in the Champion Community Steering Committees, nor are the Committees structured to improve and increase PMTCT services. EFFECTIVENESS: RESULTS ACHIEVED COMPARED TO TARGETS Based on ProVIC reporting documents,9 the table below shows the evolution from Year 1 to Year 4 of six principal indicators selected from 25 of the PEPFAR indicators relevant to the project. ProVIC has surpassed the expected targets for half the indicators and are below in the other half. The project is over target for three indicators: OVC education and vocational training (143%); individual or small-group prevention interventions (105.6%); and eligible adults or children with one-care medical service (103%). But the project is below target for the other three indicators: in-service health care worker training (92.7%); pregnant women informed of HIV status (84.5%); and individuals receiving HIV testing and counseling (T&C) services and receiving their test results (76%). 8 ProVIC Annual Report Year 2. 9 ProVIC Annual Reports Year 1,2,3,4. 14 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION Table 1. Results Achieved Compared to Targets Indicator Year 1 Year 2 Year 3 Year 4 (Semi￾annual) Total Target Result Target Result Target Result Target Result Target Result % Number of the targeted population reached with individual and/or small-group level preventive interventions that are based on evidence and/or meet the minimum standards required 345,000 316,302 360,000 543,940 480,000 443,692 528,000 226,897 1,449,000 153,0831 105.6% Number of individuals who received testing and counseling (T&C) services for HIV and received their test results 144,700 77,936 173,088 162,710 220,000 143,075 220,000 108,678 647,788 492,399 76.0% Number of pregnant women with known HIV status (includes women who were tested for HIV and received their results) 11,500 6,881 23,000 28,336 50,000 32,801 50,000 24,496 109,500 92,514 84.5% Number of eligible adults and children provided with a minimum of one-care service 18,500 19,524 18,930 20,310 20,680 15,560 21,691 1,6200 689,555 71,594 103.8% Number of eligible children provided with educational and/or vocational training 3,700 3,296 1,000 2,072 1,000 1,642 1,300 2,092 6,350 9,102 143.3% Number of health care workers who successfully completed an in-service training program 400 450 400 342 250 160 40 40 1,070 992 92.7% USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 15 Champion Communities Steering Committees: According to the Champion Community approach, the steering committees are supposed to meet with the community, identify problems, and then develop and implement action plans addressing those problems. The steering committee prepares progress reports to be submitted to the NGO, which in turn will compile and transmit them to the ProVIC provincial office. Their activities are supposed to involve the participation of the entire community. However, discussions with members of all eight steering committees evaluated indicated that they did not engage in meetings with the population for their activities. In fact their main activity was limited to ensuring that their volunteer outreach workers achieve the objective of providing HIV information to a large number of people in their catchment area, as required by ProVIC target indicators. The NGO sub-grantees stipulate the exact target number as agreed upon with ProVIC. Interviews with steering committee members and NGO staff indicated that all eight of the Champion Communities evaluated depended on the NGO for the development of workplans and their implementation. When asked how the plans were developed, typical answers were: “The NGO called a meeting and made the workplans; we couldn’t do it without them.” Or “We do not have a report of activities; it is the NGO who develops and keeps it.” The NGO staff also reported working very closely with the steering committees to develop their workplans and had difficulty orienting them toward HIV/AIDS. Review of the workplans also revealed great similarities, indicating the strong influence of ProVIC through the NGOs. An evaluation commissioned by ProVIC in 2012 and conducted by the National PNLS concluded: “The NGO bypassed the steering committee in the planning process and did everything in its place.” An assumption of the Champion Community design is that steering committees will be motivated to achieve the targets numbers of people reached by HIV awareness-raising by a $3,000 reward given at the end of each year if they reach the target. Discussions with the steering committees demonstrated that these prizes did motivate the steering committees to ensure that outreach workers returned from each awareness-raising activity with a list of at least 25 names of people contacted. However, the fact that all the steering committees received this prize raised questions as to the validity of the submitted reports and of the assessment criteria. Close review of the lists of names and signatures returned by the outreach workers showed that for many the names and signatures were all in one hand writing. When the evaluators brought this up with the outreach workers and steering committees, they replied that since many of the people couldn’t write, it was necessary for the outreach worker to make the list. ProVIC staff and other key informants doubted this, saying that most people can write their own names. The steering committees were supposed to use this money to help their communities in some way, either as seed money for an income-generating activity, the profits of which could be used to support OVC or PLWHA, or to solve a particular problem in their communities. According to the project record of prizes awarded and what was done with them by the thirty steering committees in the three provinces evaluated, six CCSCs bought school benches of which three specified an arrangement with each school to waive fees for five OVC for two years; five bought plastic chairs, tables, tents, and generators to rent for community functions; four built public latrines; three started butcher shops; three bought small flour mills; one bought a hearse; one purchased a chain saw to be used for a lumber business; one bought materials for the outreach 16 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION workers; and one started a pharmacy. The records did not have specific information for the use of the money for the remaining five CCSCs. Focus group discussions with the steering committee members revealed that the butcher shops did not succeed for lack of refrigeration; one of the flour mills is not functioning; the hearse broke down; and the lumber business failed because of the high cost of transport for the lumber. The project record indicated that the money would be used for support of PLWHA or OVC, but did not specify exactly how, except for the school waivers. When steering committee members were asked how they would assist the OVC, those who gave specific answers reported they would either provide food for OVC or pay school fees. Others were not able to give a specific answer even after probing. Community outreach workers: Data revealed an attrition rate of community outreach workers, as illustrated in the table below. Table 2: Attrition Rate of Community Outreach Workers Sites/Town NGO Champion Com. Workers trained Workers active % Attrition MATADI JADISIDA MVUZI KITOMESA 40 40 28 25 30 .0 37.5 CEMAKI KINZAU MVUETE 40 17 57.5 LUBUMBASHI OLASEC TABACONGO 30 20 34.0 KASUMBALESA BAK CONGO BAFWANO 40 25 37.5 KINSHASA RNOAC MAFUTA KIZOLA 45 30 34.0 The attrition rate ranged from 30% to 57%, with an average of 38.4%. During four different FGDs with outreach workers, 78% (28/38) of the participants said the reason was lack of money for transport. The other 26% (10/38) reported that it was lack of remuneration. The participants in the discussions with the five Champion Community Steering Committees corroborated the answers of the outreach workers, with 62% (15/24) giving as a reason a lack of transport money and 25% (6/24) as lack of remuneration. Another reason was added by members of two of the steering committees, who stated that some outreach workers were motivated by the mistaken expectation that they would receive part of the cash prize of the Champion Community and were disillusioned when the money was given to the steering committee instead. In the absence of baseline data and given that people are reached by several sources of HIV/AIDS information such as radio and television, it is not possible to know to what degree ProVIC interventions have increased knowledge or changed behavior. However, according to verbal reports from staff of NGOs, members of the steering committees, and some of the outreach workers, there is a sentiment that there is beginning to be a reduction in stigmatization of PLWHA, and records from the HIV testing sites all showed an increase of clients since the beginning of ProVIC. The Community (Population): Although the Champion Community approach specifies that communities identify their own problems and establish a workplan to address them, a top-down approach was used instead. On the national level, ProVIC is bound by its contractual agreements with USAID/PEPFAR to implement an HIV/AIDS project, and strongly orients the NGOs in their respective agreements. In turn, the NGOs strongly orient the Champion Community Steering USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 17 Committees because of the NGOs’ obligations to ProVIC. When the evaluation team walked around the neighborhoods of NGO offices informally asking people if they knew about the Champion Community or the steering committees, respondents replied that they were not aware of them. Of the 30 pregnant women interviewed in the PMTCT clinics, none had ever heard of the Champion Communities either. The catchment areas are very large, with 40,000 to 60,000 inhabitants, and in urban areas people do not necessarily associate with each other in their neighborhoods. There is no mechanism for accountability to the community of the activities that are carried out, nor for the population to influence the decisions of the steering committees and NGOs. Thus one can say that the committee exists, but the “community” does not. The effectiveness of activities was dependent on the dynamism and leadership of the NGO and the steering committee rather than on the broader “Champion Community.” PLWHA-Self-Help Groups ProVIC absorbed existing PLWHA self-help groups from previous projects, and NGO sub￾grantees added new ones. At present, there are about 40 self-help groups of typically 25 members each, which according to ProVIC staff meet at least monthly with ProVIC support for travel costs and facilitation. Through these groups, members are supposed to receive psychosocial support, access to support for their children in OVC care, referrals to the World Food Program at their Health Zone for nutritional assistance, and pertinent information to help them cope with their seropositive status. The focus group discussions indicated a range of quality and degree of dependence on the NGO for their activities. The FGD participants from the three self-help groups organized by RNOAC were the least dependent and showed a strong sense of belonging. The members participated in the monthly meetings organized by RNOAC and also described the ways they supported each other a part from those monthly meetings by visiting one another, accompanying each other to the hospital when sick, and helping each other solve personal problems related to their HIV￾positive status. They described a functioning system for savings and loans, in which each member contributed a small sum of money monthly that was used for loans when a member was in need. The discussions with the other self-help groups revealed a strong dependence on the NGO. They typically saw each other only at the monthly meeting whenever the NGO provided transport, and reported that attendance was not consistent. For example, in one of the focus group discussions only two of the nine participants had attended the previous meeting. Seven said they were busy running their small businesses selling things, and two were new and had not known about the meeting. Ninety-eight percent of participants of all the groups reported that the self-help groups were effective in providing psychosocial support. A PLWHA testified that through the activities of the self-help group, they were “united and live with great cohesion.” Another said, “Today we live as brothers and sisters, we visit each other and with the benefits of income-generating activities, we bring food to our colleagues who are most needy.” NGO and ProVIC staff and members of Champion Community Steering Committees reported the same impression. An NGO social worker reported, ‘‘The PLWHA came out of their shells and now lead a positive life.” Some of the groups have undertaken income-generating activities with help from ProVIC. To qualify for IGA support, groups develop workplans and budgets, which are submitted to the NGO and ProVIC for review and approval. According to a review undertaken by USAID LIFT 18 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION (Livelihood & Food Security Technical Assistance Project), some agriculture-based IGAs have had some success, but overall failure rates are high due to lack of entrepreneurial skills, access to capital, management skills, and commitment. ProVIC has realized its weakness in this area and has requested technical assistance from the USAID LIFT project, and they have explored ways to improve this component. The discussions revealed that the members of the self-help groups were unfamiliar with the overall Champion Community approach, but named awareness-raising and PLWHA home visits as activities organized by the NGO. The members of the groups were not engaged directly with the Champion Community Steering Committee, and in the different groups, anywhere from two or three to half of the discussion participants were aware of the Champion Community Steering Committee. Prevention, Care, and Treatment (Includes PMTCT) The 20 service providers interviewed in the health facilities all reported a perception that since the Champion Community outreach workers’ HIV awareness-raising activities began, there have been positive changes in the acceptance of HIV/AIDS screening and willingness of PLWHA to reveal their HIV status. The NGO staff interviewed was of the same opinion, and the reports given by the PLWHA during the focus group discussions supported this finding as well. They explained that people were more accepting of them and that the support given one another in the self-help groups encouraged them to speak openly about their status and to convince others to go for testing. During the four years of ProVIC interventions, project records show that the numbers of people tested in ProVIC-supported testing sites were highest in Year 2 and declined in Year 3. However, the evaluation team does not know to what degree there has been an increase in numbers of people tested since before ProVIC interventions began, as compared to after, because the project has no baseline data. The team visited one health facility supported by ProVIC that had kept data on HIV testing rates for the years before and after the beginning of ProVIC. The evaluators used these data as an example of the possible effect of ProVIC on increased testing. The integrated HIV testing center of Kiamvu in Matadi, Bas-Congo receives both patients counseled by hospital providers as referred by Champion Community outreach workers and patients from the fee-standing HIV testing center which has just been closed. Opened in 2006 with the support of the Belgian Technical Cooperation, ProVIC began support in 2009, and the Champion Community activities began implementation in 2010. At this clinic there has been no increase in the number of women attending the antenatal clinic since ProVIC began, but there has been an increase in the demand for testing. The chart below shows an increase in the number of people tested from 2010—which was the second year of intervention of ProVIC in this Health Zone and the first year of operation of the Champion Community in this health area in Kiamvu Health Center—as compared to numbers tested from 2007–2009 before ProVIC interventions began. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 19 Table 3: Number of Pregnant Women Tested in Kiamvu Hospital Before and After the Start of ProVIC It is difficult to ascertain the degree to which this increase was caused by ProVIC awareness￾raising activities. During this period of ProVIC interventions, drugs became available for HIV￾positive pregnant women, which could have had a profound effect on demand for PMTCT services. While hospitals did have referral slips from ProVIC-referred clients, the method for storing and tracking these slips was so unsystematic that it was not possible to distinguish between clients referred by ProVIC and others who came on their own. In the health facilities in Boma and Matadi where focus group discussions were held with pregnant women clients, none of them had been provided information on HIV testing or PMTCT by any outreach worker. Radio was their main source of information on the subject. An analysis of the rate of positive tests over time showed that as more pregnant women were tested, less and less were HIV-positive. In 2007, 24% of pregnant women tested positive, while in the first half of 2013, only 7% tested positive. This could indicate that the objective of testing all pregnant women in an area of low HIV prevalence may not be the most effective way of identifying the majority of people who are HIV-positive. The chart below shows the change in proportions between those tested and those with a positive HIV result in the integrated HIV testing centers of the reference hospital of Kiamvu. 0 2000 4000 6000 Number of tested people 20 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION Table 4: Change in Percent of HIV-Positive Test Results in Kiamvu Hospital from 2007 to 201310 Interviews with health care providers and the chief medical officers at the provincial and zonal levels revealed that they all agreed that the support of ProVIC for numerous trainings has contributed to the improvement of the quality of services. However, since there was no baseline and it was not practically possible for the evaluators to observe PMTCT services in other health facilities not supported by ProVIC for a comparison, the degree to which this was due to ProVIC-supported training could not be determined. The evaluator did observe that the nurses used the correct system for counseling and testing, giving group information to waiting clients and individual counseling after testing. However, this led to long waiting periods and terse counseling sessions due to the very large numbers of patients that were being served by only a couple of providers. In all the medical facilities observed, ProVIC-provided medical supplies were available and being properly used and maintained. In the medical facility at Boma and Kiamvu, the system of treating biomedical waste has been rehabilitated or reconstructed. However, the walls and doors of the new incinerators have been damaged by heat. One problem cited by health providers in all facilities visited, except for Sendwe Hospital in Lubumbashi, was the lack of testing materials. This was caused by a large theft that ProVIC experienced in the beginning of the project, as well as the freezing of the Global Fund’s support for ARVs. Providers also complained of their heavy workload caused by the increase of clients. Key Populations The key populations targeted by ProVIC are appropriate for the profile of the HIV epidemic in the DRC. They are commercial sex workers (CSWs), truckers, fishermen, miners, and men who have sex with men (MSM). According to the Year Four Semi-Annual Report (see Annex B), ProVIC has reached 130% (48,480) of the semi-annual target. The largest numbers of the key populations reached are truckers (19,683) and CSWs (16,470). The number of MSM reached was 2,065. 10 Kiamvu Hospital data. 0% 5% 10% 15% 20% 25% % of positive result, from 2007 to 2013, at the integrated HCT of Kiamvu hospital, Matadi. % of positive result USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 21 Table 5: Key Populations Reached by Second Quarter Year 4 Compared to Target #Key Pop reached Year 4 Target Bas￾Congo Katanga Kinshasa Province Oriental South Kivu Y 4 achievement to Q2 Y4 semi annual target 75,000 7,160 8,856 17,728 7,122 7,714 48,580 130% CSWs N/A 2,347 2,217 7,638 2,324 1,944 16,470 N/A Truckers N/A 3,507 3,330 6,845 3,392 2,609 19,683 N/A Fishermen N/A 761 127 1,619 573 1,667 4,747 N/A Miners N/A 0 1,310 0 754 936 3,000 N/A MSM N/A 523 100 1,414 17 11 2,065 N/A Other Vul. Pops N/A 22 1,772 212 62 547 2,615 N/A Male N/A 4,810 6,231 10,395 4,707 5,558 31,701 N/A Female N/A 2,350 2,625 7,333 2,415 2,156 16,879 N/A ProVIC activities concerning key populations given in the Year 4 Semi-Annual Report include: (1) revision of the peer educator training module for MSM and other key populations; (2) peer educator workshops focused on at-risk populations in the five provinces, reaching 175 members of key populations of CSWs and MSM; (3) sensitization of female sexual partners of MSM and other lesbian, gay, bisexual, and transgender individuals; (4) promotion of proper use of condoms and water-based lubricants; (5) promotion of safer sex and healthy behaviors; (6) voluntary HIV counseling and testing (VCT); (7) referral and proper treatment of sexually transmitted infections; (8) proper care of anal infections; (9) orientation to self-help groups for psychosocial support of seropositive MSM and CSWs; and (10) risk awareness and vulnerability￾mapping. Each of the key populations’ components of the project was evaluated from three perspectives:  The contribution of the Champion Community Strategy  The linkages or integration of the community-based and clinic-based interventions  The quality of the information given by peer educators or outreach workers. Contribution of Champion Community Strategy: Interviews with NGO staff in charge of the key populations project component, as well as with ProVIC staff in Kinshasa, Katanga, and Bas-Congo, all indicated that the interventions targeting highly stigmatized groups such as MSM and CSWs were largely implemented by the NGOs separately and parallel to those activities of the Champion Community Steering Committees (CCSC). The NGOs used three main ways to reach key populations for peer education. They established networks of MSM, some of which became associations. They established CSW associations and/or targeted existing ones which had been supported by previous projects. And they identified geographic areas of predominately high-risk populations such as truck stops, mining and fishermen communities, or neighborhoods where commercial sex workers live and conduct business. NGOs set up mobile HIV testing 22 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION units in hot spots that provided testing services to both MSM and CSWs, and provided miners with referrals to nearby ProVIC-supported health facilities. During the focus group discussions with the nine CCSCs, when asked directly if they participated in any activities with MSM, CSWs, truckers or fishermen, participants agreed that the outreach workers only indirectly reached these people when they were among the general population within the Champion Community catchments, but that the outreach workers did not provide any special information for them apart from what was given to everyone in general. However, two CCSCs had participated in targeting truckers and transporters. This was in Kinsau Mvuete, where one member of the steering committee is a representative of the local truckers and transporters’ association. He reported that he mobilized association members as peer educators to provide information to the drivers and transporters at the parking stands in the town. This steering committee also assisted the NGO sub-grantee in identifying hot spots of sex workers in their town based on PNLS mapping. The second example was in Kasumbalesa, where evaluators observed Champion Community outreach workers providing information and distributing condoms to truckers and sex workers at the very large truck stop. Discussions with authorities from PNLS and PNMLS, as well as with the NGO staff who implement the key population components, indicated that they did not see a relation between the Champion Community Strategy and the interventions targeting key populations. According to interviews with the ProVIC chief of party, the Champion Community Strategy is applicable to the key populations’ component of the project in that they are “transversal” communities. He explained that while the MSM or CSWs are not unified geographically, they form a type of community in that they share common identities and problems. An interview with the president of the Board of PSI, who himself has been involved with research and programs for HIV prevention among CSWs since the first HIV interventions in Kinshasa in the 1980s, and a review of documents indicated that CSWs have been organized into associations for peer education since the early 1990s in Kinshasa. NGO staff in Matadi and ProVIC staff in Kinshasa reported that ProVIC has “inherited” such associations in Kinshasa and Matadi from other projects who had been working with them before ProVIC began. MSM networks are new, and few gather as associations because of their continued strong stigmatization. The truck parks at Kasumbalesa on the Zambian border are made up of transient truckers who probably never see each other again since they travel long distances and come from varying countries. Linkages Between Community- and Clinic-Based Services: Discussions with ProVIC staff, NGO staff, MSM, and commercial sex workers all revealed a persistent problem of follow￾up for services of key populations for whom mobile testing units have been set up in hot spots. A summary of the reasons given indicate that the problem is mostly due to practical issues rather than ProVIC’s strategy. They include:  High mobility of high-risk groups such as truckers, many sex workers, fishermen, and miners. The truckers in Kasumbalesa mainly come from southern or east Africa and do not spend much time in Congo. Participants in two focus group discussions with commercial sex workers in Kasumbalesa reported that they did not stay in Kasumbalesa for long, but travelled from town to town “following the money.” They made circuits to Lubumbashi and other towns connected by good roads to Lubumbashi.  The sex workers and MSM who frequent the hot spots do not necessarily live in the Health Zone of the hospital supported by ProVIC and prefer to go for HIV consultations in clinics USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 23 in other parts of the city, either closer to where they live or where they usually seek other health services. The ProVIC chief of party had suggested this hypothesis, which was supported by informal conversations with several MSM and sex workers by the evaluators when observing one such hot spot in Kinshasa.  The MSM interviewed did not consider the Catholic hospital supported by ProVIC in Kinshasa near one of the MSM hot spots as an appropriate referral health center. MSM at the “hot spot” told the evaluators that they perceive the service providers as likely to discriminate against them and did not want to go there. World Production staff reported that the Catholic sisters were not comfortable with providing services to the MSM either.  MSM interviewed reported that many MSM in their networks preferred to be integrated into the health service just as any other patient, and for reasons of stigmatization and confidentiality, did not want to be traced or followed up by a project.  MSM who have just tested positive are already dealing with a very sensitive issue with regard to their sexual preference and are not ready emotionally to deal with being followed up by others or becoming members of self-help groups. (This was reported to the evaluator by the MSM social worker from JADISIDA.) Quality: The strategy of targeting MSM by developing or using existing networks, along with mobile testing units in hot spots, has been quite effective, especially in Kinshasa where there are many such places openly frequented by MSM. In Lubumbashi and Matadi, where few MSM have revealed their identity in public, the focus group discussions and individual interviews with MSM indicated that ProVIC interventions have not only increased access to HIV information, but have also contributed to slowly reducing homophobia. JADISIDA’s work in Matadi is an example of an effective program which started with one MSM who was hired as a social worker specifically to work on the MSM project component. In the interview with this social worker, he explained how he had established a network of over 400 men to whom HIV information is provided through word of mouth, cell phone messaging, and social media. The evaluation team visited a hair salon which was started by the core group of this MSM network with IGA seed money from ProVIC. The MSM working at the salon told us that it has been successful financially as they were able to pay bills and produce a small income. The salon not only attracts other MSM, but and brings in both men and women walk-by customers. The MSM working there told the evaluators that the salon has been contributing to their social acceptance. There is not a coherent behavior change communication strategy used to target the key populations. The information is not based on an assessment of the gaps in knowledge and the barriers to and factors which promote behavior change for each of the high-risk groups. Although the training module covers issues pertaining to key populations, the information given to sex workers and truckers in Kasumbalesa revealed no differentiation and was very limited. For example, sex workers were counseled about abstinence even though this is not the most appropriate prevention approach for this key population. Truckers were briefly educated about condom use, with little information being provided about HIV testing. There was no information provided as to where along their route truckers may receive other HIV services. Information was also sometimes erroneous. For example, one outreach worker demonstrated to members of the team the use of female condoms by putting a male condom on the dildo used for demonstration purposes. A meeting with a group of MSM in Lubumbashi, who were supported by a ProVIC-supported NGO, World Production, revealed that half the group was actually women who have sex with women. The discussion revealed that 24 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION no distinction was being made between the needs of men and women who have same-gender sex in terms of HIV prevention. Since there was no baseline evaluation, there is no way to determine if there has been any change in knowledge or behaviors. While this is difficult to do for highly mobile targets such as long-haul truckers and sex workers, it could be done with the more stable sex worker associations in Kinshasa, fishermen villages, and truckers who live in town and make short-term trips. Orphans and Vulnerable Children (OVC) The ProVIC OVC program includes the following interventions: 1. Psychosocial support 2. Provision of waivers for health care costs 3. Educational support 4. Economic empowerment, including technical skills training for older children 5. Nutritional support 6. Legal support. According to the Year 4 Semi-Annual Report,11 ProVIC surpassed overall OVC targets for provision of health care (198%), school and vocational training (322%), psychosocial support (125%), and nutritional support (316%). The areas where ProVIC did not achieve its targets were in income-generating activities (91%) and legal services (20%). All of the NGO sub￾grantees visited offered the first four interventions. Only JADISIDA in Matadi described a couple of instances when legal support was provided, and this was because the Monitoring and Evaluation Officer of the ProVIC provincial office also happens to be a lawyer. Psychosocial Support: Two types of psychosocial support were provided: child-to-child groups and home-visits. The staff of all the NGOs visited reported that they had organized child￾to-child groups of OVCs. According to discussions with the NGO and ProVIC technical staff on the national and provincial levels, in the first years of the project each NGO designed its own program with varying degrees of quality. ProVIC responded to this in July 2012 by developing a manual on the child-to-child approach with a standardized set of activities, and conducted trainings based on the manual. However, the data indicated that not all the NGOs visited had a clear understanding of this intervention. According to the descriptions provided by the staff of two of the six NGOs visited, the OVC beneficiaries are divided into groups which are supposed to meet monthly. A facilitator leads them in activities including lessons in life skills such as health, hygiene, good nutrition, group life, and study skills. The children also play games and are given a meal. In the other four sites visited, when asked to describe what happened in the child-to-child groups, the NGO social workers responsible gave vague responses and did not appear to know, although they all did report that the children were given meals at the meetings. The team was able to observe child-to-child group meetings at three sites—one in Bas-Congo and two in Katanga. Although the team intended to observe a meeting only if it was on the regular schedule, the NGOs set up meetings especially for the evaluators. The meetings varied 11ProVIC PATH Consortium. Annex B: ProVIC Semi-Annual M&E Data. May 15, 2013. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 25 in quality. Two were quite lively sessions, with very good facilitators. A third was a dry lecture on cholera, and the children appeared quite bored. One was carried out in a school after school hours, and another took place outside a school where all the neighboring (“non-OVC”) children crowded around to observe the event. Discussions with the children meeting in the school indicated that they appreciated the group meetings. The lessons helped them with their school work. The games were fun, and the children enjoyed being with each other. When asked what else they would like to do during the meetings, they interpreted the question as, “What else do you need or want?” and their answers were: more food at home, clothes, and other very basic material objects that their households lacked. Interviews with the NGO staff of all three sites indicated that in actuality the meetings were not being held as regularly as they were supposed to be. Discussions with the children in the three groups confirmed that the meetings did not take place regularly. For example, a child in Kasumbalesa said, “We do not often meet; the last meeting was held in February (three months earlier).” There are two social workers in each NGO who are supposed to visit anywhere from 100 to 500 OVC depending on the number of beneficiaries of the NGO. The social worker staff in all the NGOs evaluated indicated that they either did not make home visits at all, or when they did, it was on a very sporadic basis. They all gave the same reasons: the number of children to be visited was too large to cover, and the distances of their households from the NGO office were too great to make by foot as they had no other means of transport. When they did make home visits, the social workers visited a few OVC who lived close to the NGO office. There was no calendar of visits, no plan on what was supposed to take place during the visits, and no criteria for selection of which OVC needed home visits. When asked what they did when they visited, none of the social workers was able to give a precise answer, and they showed some confusion between assessment visits to new OVC beneficiaries to ascertain their needs, and visits to provide psychosocial support. Medical and Health Care Support: In all the sites visited, provision of health care was organized for OVC. Two modes of payment by the NGO were being used. In five of the six NGOs visited, each time a child is ill she/he has to go to the NGO office to obtain a referral slip permitting free medical service at the ProVIC-supported health center. Interviews revealed that NGO staff did not seem to be aware of the difficulties this system may cause the OVC in obtaining the medical care, should the office be closed or far from the medical facility and their home. Only one NGO has a contract with the Health Zone whereby reimbursements for services are made for OVC who are put on a hospital beneficiary list and treated directly. Nutritional Support: Nutritional support is very limited, and two types are provided, only one depending on ProVIC. The NGO refers malnourished OVC to the health centers for nutritional supplements supported by the National Program for Nutrition of the Ministry of Public Health (PRONANUT). None of the NGOs had a system to follow-up with these children. ProVIC supports a meal at the monthly child-to-child group meetings, but with the reduction of funding for such activities, staff of all the NGOs reported that they could not provide these meals for all the OVC on their beneficiary lists. For example, in Lubumbashi, one NGO could only provide meals to 53 of their 300 OVC beneficiaries. There was one instance which demonstrated that a strong Champion Community Steering Committee can contribute to 26 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION OVC nutritional support. This particular steering committee contributed money from its IGA to the NGO so that 150 OVC could receive meals (the project was expected to support 130). Educational Support: All NGOs receive ProVIC funds to support OVC education, which covers payment of school fees and a school kit consisting of a uniform, bag, books, and pens. Interviews with NGO staff revealed two specific concerns. The first was that even though ProVIC had exceeded its targets according to the Year 4 Semi-Annual Report, all six NGOs visited had more OVCs on their beneficiary lists than were covered. For example, in Lubumbashi, the NGO OLASEC identified 300 OVC, of whom only 90 were enrolled. Reports from the NGO CEMAKI in Bas-Congo (Kinzau Mvuete) indicated that in Year 1, 27 of 351 OVC identified were enrolled; in Year 2, only 59 of 400 were enrolled; and in Year 3, 90 of 518 OVC were enrolled. The second concern was that NGO staff did not always receive the funds to pay the children’s tuition in time; initially, the children were turned away from class because of this. In response, NGO staff made special arrangements with the directors to allow the children to continue to attend school until funding was received to pay the tuition. A third concern was raised by the children themselves during the OVC focus group discussions. When OVC were able to attend school while other children in their guardian household could not, the OVC experienced resentment from the other children in the household. Interviews with authorities in PNMLS raised this concern also. Authorities at PNMLS in Bas-Congo were in favor of a strategy used by UNICEF that increased the number of children placed in schools with which agreements were signed to waive fees in return for seed money for income-generation activities, the profits of which would be used to support the education of the OVC. Relation of Champion Community to OVC Educational Support: While the provision of educational services to OVC is provided by ProVIC directly through each NGO, some examples of Champion Community Steering Committees have used their IGA seed funds and/or prize money to support OVC education. For example, one Champion Community Steering Committee paid school fees for a total of 64 children during a four-year period. In Matadi one of the steering committee’s presidents is the director of a vocational school where OVC are enrolled. With the prize money received, the committee bought benches for the school and also started a business renting out chairs, the profits of which are used to enroll OVC in the vocational training center. Socioeconomic Strengthening: ProVIC provides seed money to NGOs for financial support for income-generating activities (IGAs). The vast majority have not succeeded. However, business establishment for OVC completing vocational training appears to be successful. In Matadi and Kinzau Mvuete, JADISIDA and CEMAKI have set up training workshops for dressmaking where girls are given sewing machines upon completion of a 6-month training course. They are then able to run their own dressmaking businesses and train other girls, as well. Legal protection: Activities addressing legal protection of OVC are very limited. Only 20% of the target number of children received these services. These activities had not yet really begun in the sites visited. INTEGRATION Previous to ProVIC, each component of the HIV/AIDS continuum of care package was designed, funded, and implemented as a separate project through contracts with several companies. Each USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 27 company was specialized in and responsible for a different component. Collaborating partners filled gaps of the package not covered by PEPFAR, the major one being ARVs, which were provided by Global Fund. The participation of several companies working independently resulted in problems of coordination of a coherent prevention program and made it difficult to ensure that each PLWHA and OVC beneficiary received the full package. ProVIC was designed as an overarching, integrated project to solve these problems. However, the evaluation found that many of these problems have persisted. Three aspects of integration were evaluated: 1. Programmatic integration—holistic HIV program managed within one large project. 2. HIV health service integration—all HIV health services are provided within one health facility and are part of the other medical services. 3. Continuum of care—PLWHA have access to all components of care and support, including medical care, psychological support, economic support, and nutritional support. Programmatic Integration—Holistic HIV Program Managed Within One Large Project According to interviews with staff contracted by the different companies of the consortium, the four different consortium members, each with links to its own U.S. headquarters, have hindered consistent coordination of project activities. This has been a problem both on the national and provincial level. In the beginning of the project each technical group, contracted by a different company, made its own workplans separately with guidance from its respective company headquarters and functioned almost as separate projects. When PATH took over the directorship, the chief of party set up an improved system in which, after developing separate workplans for each component, the technical staff came together to harmonize them into a single plan with a single time line. According to both ProVIC staff members and the chief of party, this improved project coordination. Interviews with the technical staff showed that they lacked a common vision of the project as a whole, even though they had one comprehensive workplan. The technical staff of each component tended to work separately, focusing on their own areas of expertise. Interviews with staff members on the national and provincial levels showed a lack of awareness of the current activities of the other components, which also tended to function as independent units. HIV Health Service Integration—All HIV Health Services Are Provided Within One Health Center and Are Part of the Other Medical Services During the focus group discussions, the PLWHA from all eight participating self-help groups were united in their preference for HIV services to be provided within one health facility as part of the other medical services. They agreed that this type of service was best because it avoided stigmatization, facilitated access to all services, and reduced the cost of transport. They complained about CD4 count machines being in separate facilities at a distance from where the PLWHA usually went for services. There was a range of opinions on the preferred place for HIV testing. On the one hand, the FGD participants preferred testing centers that were integrated into the regular health facility services. But, on the other hand, they explained that when there is a freestanding testing unit in the community close to many people’s homes, or a mobile unit placed in a highly frequented place such as market, it is easier to access than a health clinic. The discussions in the different groups revolved around the degree to which HIV was stigmatized in their communities. Where it is less stigmatized, community-based testing centers, in addition to 28 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION those in health facilities, were favored. In areas where HIV is still highly stigmatized, the participants preferred HIV testing to be done in the health facilities. Interviews with social worker staff of ProVIC and NGOs in both Lubumbashi and Matadi corroborated this. They reported that in their work with PLWHA they found that patients preferred going to the health facility for HIV services since they were already used to accessing the facility for other ailments. AMO-Congo in Lubumbashi, which has a freestanding testing center with a large HIV sign on its gate, reported losing clients to a nearby health center when it began offering HIV testing. CONTINUUM OF CARE Link between Community-based and Clinic-based Services for OVC and PLWHA The links between all the actors in the community and the medical services are not consistently strong enough to ensure the continuum of care for the PLWHA, as demonstrated by the following: 1. A careful examination revealed that a good system for reference and counter-reference had been set up between the freestanding HIV testing center, the zonal health center, and ProVIC NGOs. Interviews with the NGO staff consistently demonstrated that the reference and counter-reference system is poorly understood and implemented, resulting in decreased care and difficulties in follow-up. The system includes referral slips from the community and mobile testing centers to the closest ProVIC-supported health center, with a counter-referral slip that the patient can take to the NGO for other services. While the health centers do collect the referral slips from patients who present them, interviews with health care providers and observations of the filled-out referral slips remaining in the health facilities indicated that they do not always give the patient the counter-referral slip. 2. While the NGOs give the PLWHA or OVC a referral slip to take to the health center for services, none of the NGOs had specifically designated someone to follow up at the health center to make sure that the person sought the services. ProVIC monitoring and evaluation staff reported that they keep a count of the number of the ProVIC referral slips the hospital collects, and this number is used for reporting purposes. 3. Discussions with the Champion Community Steering Committee members and the outreach workers indicated that they do not consistently refer PLWHA and OVC to the health centers or to the ProVIC-supported NGOs. When outreach workers described what they say during awareness-raising activities in the community, the focus was on HIV prevention, distributing condoms, and promotion of HIV testing, and rarely anything about the OVC and PLWHA services provided by the ProVIC-supported NGO. 4. One of the methods used by NGOs to ensure OVC free medical care is to require the children to go to the NGO office each time they are sick to obtain a referral slip, and from there go to the health center. As the NGO offices are not necessarily very near the health center, this is an inconvenience for the OVC and may reduce their access to ProVIC￾supported services. Champion Community Relation to Health Zones ProVIC independently set up a system of community outreach workers parallel to that of the Health Zone. The steering committees are similar to the Zonal-level Health Committees and/or the Zonal￾level Development Committees, both of which have a system of outreach workers. Interviews with Zonal Health Officials revealed that they would prefer ProVIC to collaborate directly with the Zone so USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 29 that their work would complement each other and reinforce that of the Zone. ProVIC responded to this by including the Health Zone in ProVIC activities, with shared monthly meetings, integrating some of the zonal outreach workers with those of the Champion Community, and sharing reports on community education activities. Interviews with officials at the national, provincial, and zonal levels indicated a strong opinion on the part of PNLS and PNMLS, as well as chief medical officers and other health officials, that the Zonal-level Health Committees and Champion Community Steering Committees should be integrated and function as a single structure, with one set of outreach workers responsible for the Health Zone. EFFICIENCY Within the limited scope of this evaluation, the evaluation team with USAID and the ProVIC office explored programmatic and managerial changes and alternatives to realize future cost savings while maintaining quality programs in the remaining months. The evaluation team identified five specific changes already foreseen and/or underway that may reduce costs in the future. Geographic Focus: ProVIC maintains five provincial offices, with professional and clerical staff, rental vehicles, etc. In response to the PEPFAR consolidation and integration of services in three areas, ProVIC will close the South Kivu office in August 2013 and the Bas-Congo office during the coming year. With only three regional offices (Katanga, Kisangani, and Kinshasa) and the headquarters in Kinshasa, administrative costs will be reduced. “Kisangani Model:” ProVIC has decided to work in Kisangani without the support services of an NGO sub-grantee. The four newly established Champion Communities have identified over 100 volunteer outreach workers who will provide data directly to the ProVIC provincial office, rather than through an NGO partner. The average amount of the annual standard grant to the 14 sub-grantees is $191,000. The advantage is that ProVIC staff will take on additional reporting responsibilities, with more quality control, direct supervisory oversight, and greater closeness to the community, which could save costs in the long run. The disadvantage is the loss of potential to build the capacity of local NGO partners for long-term sustainability. Narrow Focus on Activities on the PMTCT Platform: In order to comply with USAID’s suggested narrowed focus related to the “PEPFAR Pivot,” ProVIC decided to no longer undertake activities outside the PMTCT focus. For example, prevention activities aimed at the general population have ended; ProVIC no longer supports income-generating activities; and the program also no longer supports the Champion Communities in Bas-Congo and South Kivu, or the OVC and PLWA in those areas, staff training activities, and other program expenses. Reduced Travel Costs: ProVIC staff suggested that with USAID assistance to overcome a UN bureaucratic hurdle, ProVIC staff could more readily and easily board MONUSCO flights when space in available, thus extending the travel budget and increasing site visit time. According to ProVIC, USAID has recently established a new vehicle rental policy and procedure that reduces vehicle costs. Supply Chain Management System: Use of the supply chain management system to streamline importation and distribution of supplies has reduced program costs and time. 30 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION Sustainability Champion Communities: The team’s findings show that it is very unlikely that the Champion Communities or Steering Committees will persist after ProVIC ends. The evaluators asked all nine of the Champion Community Steering Committee representatives if the CCSC would continue beyond the life of ProVIC. None made such a claim. A range of reasons was given:  CCSCs are structured around the fight against HIV/AIDS. Because the DRC is a low￾prevalence country, HIV/AIDS is not necessarily considered a high priority.  CCSC depends on the NGO. Meetings are organized and supported by the partner NGOs, which have the funds to pay for transportation and meeting costs, and the prizes. These subsidies will not continue after the project ends.  CCSC members reported that they lack sufficient knowledge to run IGAs or disseminate information on HIV, and would need further training from the NGOs if they were to continue to function. Interviews with government authorities corroborated this information. In addition, they indicated that since the Champion Communities were created and developed solely by the project—and are not linked to existing Zonal-level Health Committees—they would not be sustained after the end of the project. Sustainability for Prevention of Mother to Child Transmission (PMTCT): Integrating PMTCT into maternal and child health services is key to sustainability because the testimonies of HIV-positive pregnant women attending clinics encourage other clients to be tested and find out their serological status. Potential threats to PMTCT sustainability are the mobility of trained personnel and the reduction of funds to pay for transportation during outreach activities for client follow-up. Staff motivation will decrease once transport costs are no longer reimbursed for home visits, for searching for clients lost to follow-up, and for HIV T&C conducted in maternity clinics at neighboring PMTCT sites. At this point, PMTCT services will be unable to meet the increased demand and will have to depend on outside support. OVC and PLWHA Community Support: While steering committees were trying to use some of their money from IGAs and their rewards to support OVC and PLWHA, it was extremely limited. Since the prize money will cease, the IGAs have not been successful, and the steering committees are heavily dependent on the NGOs, it is unlikely that this support will continue after ProVIC ends. Key Populations: The programs for key populations depend upon outside funding and technical expertise. By adding an MSM to their social worker staff, JADISIDA has been quite effective in building an MSM network, which has the potential to develop into an association. With continued support, this network could be strengthened to leverage its own funding, but this would require time and a committed group of MSM, which may or may not be possible. MSM networks in Kinshasa are more fully developed with systems for peer education, but are not yet strong enough to leverage their own funding and lack HIV expertise among their members. However, ProVIC peer education training has laid the groundwork to strengthen continued network support. ProVIC’s interventions targeting key populations where they are geographically concentrated, like the truckers in Kasumbalesa, the miners in Luisha, or the fishermen in their villages in Bas-Congo and elsewhere, are introduced externally, so when those interventions cease, there will be no structure left to support them. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 31 GENDER CONSIDERATIONS Gender-Based Violence ProVIC has responded actively to USAID’s direction to focus more on gender based violence (GBV. According to the Year 4 Semi-Annual Report, ProVIC has implemented several activities in Kisangani and Kinshasa. They include:  Integrating GBV activities in 14 health facilities offering PMTCT  Two training sessions for health care providers on the management of sexual and other gender-based violence (SGBV) for doctors and nurses  Community-level advocacy activities through the Champion Communities  Seven-day training of eighty outreach workers and peer educators (including 38 women and 42 men) from four Champion Communities, training around HIV/AIDS, GBV, and family planning  Support of health care workers in PMTCT sites and self-help groups to use a screening form developed by the University of North Carolina and the PNLS. Increasing Gender Equity in HIV/AIDS Programs and Services, Including Access to Reproductive Health Services The new focus of ProVIC on PMTCT has important gender equity implications regarding access to services for men and non-pregnant women. Project activities promote male participation in PMTCT through community education and by providing free HIV screening, consultations, and drugs to husbands of the women frequenting antenatal services. During Years 2, 3, and 4, as the number of women participating in PMTCT increased, the number of men increased along with them, with a consistent 5% participation. During discussions with members of PLWHA self-help groups in Matadi and Lubumbashi, men expressed the concern that neither they nor other men were comfortable going to the antenatal clinic for HIV services. Low male participation in PMTCT is not unusual as indicated by studies conducted in the DRC12 and elsewhere.13 Only 30% of Champion Community outreach workers are women. The outreach workers do not particularly focus on men or women when raising awareness in the general population, and the messages are not gender-specific. Workers distribute both male and female condoms. The Champion Community Steering Committees are about 50:50 female to male, but males tend to dominate decision-making. Key populations targeted are gendered: male groups are MSM, fishermen, and truckers; female groups are commercial sex workers. In Lubumbashi, the MSM group visited by the team comprised as many women who have sex with women as MSM. No distinction was made between the differences in HIV transmission and other HIV risk factors for the two groups, nor was group-specific information being provided. 12 Ditekemena, J. et al., “Male Partner Voluntary Counseling and Testing Associated with the Antenatal Service in Kinshasa DRC: A Randomized Controlled Trial. “International Journal of STD & AIDS, 24(3). 13 Morfaw, Frederic et al. “Male Involvement in Prevention Programs of Mother to Child Transmission of HIV: A Systematic Review to Identify Barriers and Facilitators.” Systematic Reviews, 2. 32 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION IV. CONSIDERATIONS FOR THE INTEGRATION APPROACH CONSIDERATIONS CONCERNING INTEGRATED PROJECT MANAGEMENT 1. While divisiveness can occur when the home office of each company takes too great a technical lead in developing workplans, without this input a large integrated project will lose the benefit of the specialized expertise of each company. 2. It appeared that in this integrated project, aspects of each component were somewhat skimped upon in a way that would not have occurred in separately financed projects. But the problems of coordination and harmonization in separately financed projects are much greater than in a single integrated one. 3. The advantage of having separately contracted projects for each component of the HIV/AIDS package is that each company can develop a more comprehensive, focused program, and concentrate its expertise on running it effectively. But without effective mechanisms to ensure coordination of flow of financing and timing of implementation, the functioning of the project as a whole is hindered. CONSIDERATIONS OF INTEGRATION CONCERNING PREVENTION, CARE, AND SUPPORT 1. The integration of prevention, care, and support provides a greater level of comfort for the beneficiary when there is effective coordination between the providers of the different services. There is a risk of overlap and/or gaps in the package of services if there is insufficient coordination between structures such as PNLS, the ProVIC provincial office, the Zonal-level Health Committee, and the NGO, and between the stand-alone HIV testing centers and those within health centers. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 33 V. CONCLUSIONS AND RECOMMENDATIONS CONCLUSIONS The conclusions will be discussed in terms of the evaluation questions.14 How has community involvement increased as a result of Champion Communities and what difference has this made in the health of the target populations? The Champion Community approach was intended to mobilize communities to address their HIV/AIDS problems but in practice, it was a program run by NGOs, and a steering committee with volunteer outreach workers that were highly dependent on the NGOs for their activities. The key approach of involving significant numbers of the population in the community to address their HIV/AIDS problem did not appear to work. However, the steering committee through volunteer outreach workers did contribute to increasing community HIV/AIDS awareness and testing. The NGO supported PLWHA self-help groups and the child-to-child groups contributed to the psychological health of the PLWHAs and OVCs respectively. The Champion Community approach provides a means for the population to identify a problem in their community, and work together to solve it. However, before people will take action on their own, they need to perceive the issue as a high priority. Therefore, the Champion Community approach is more appropriate in areas where HIV/AIDS is a central problem. The use of a committee such as the Champion Community Steering Committee as a means of identifying outreach workers and overseeing their work has had tangible results in terms of raising awareness of the surrounding population, increasing HIV testing, and contributing to PLWHA stigma reduction. However, this is not an innovative approach and is basically the same strategy set up by the MOH on the Health Zone levels with Zonal-level Health Committees and volunteer outreach workers. While the strategy may be viable under certain circumstances, it is based on volunteers, which remains problematic in a difficult economic environment like the DRC. The Champion Community approach depends too much on support from the ProVIC sub￾grantee NGOs, and on the strength and leadership of the Champion Community Steering Committees, in order for it to be sustainable. What are the strengths and weaknesses of the integrated approach? Quality is the most important issue concerning advantages and disadvantages of a large single integrated project that provides a complete HIV service package as compared to separate projects providing services for each component. In the case of ProVIC the quality of implementation was compromised in several areas which appeared to be due to the complex process of managing such a large project requiring so many different types of technical expertise and multiple consortium members. However, contracting each component of the HIV package as a separate project adds more difficulties to the effective coordination and harmonization of 14 The answer to the question concerning the aspects of model that should be kept or not is in the recommendation section. 34 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION all the components of the HIV package, but may gain advantages in quality, as each project focuses solely on one component with one contractor highly specialized in that area. The advantage to providing HIV services integrated in other health services is that it reduces stigmatization, time and cost of transport. However, the disadvantage of providing HIV testing only within clinic sites is that it reduces access because of distance from potential clients, whereas mobile and/or free-standing testing units set up where large numbers of general or key populations congregate increase access. How has the quality of HIV prevention, care and treatment services improved (PMTCT)? Numerous trainings contributed positively to the quality of PMTCT. The availability of supplies and materials in the supported clinic sites have contributed to raising the technical platform. However, since there was no baseline, nor the possibility to make comparisons with other non￾ProVIC supported services it was not possible to ascertain the degree to which the improvements were due to ProVIC interventions. To what extent has ProVIC improved access to services for key populations in DRC? ProVIC was notably effective in the improvement of access of HIV/AIDS services for MSM through the establishment of new MSM communication networks, some of which have become fledgling associations. However this was done by the work of the NGOs as a separate activity parallel to the Champion Community Steering Committee activities. Targeting areas of high geographical concentration of key populations such as truck stops and mining camps demonstrates viable strategies for increasing access to HIV/AIDS services but issues of quality of these interventions limited their effectiveness. What are the factors that contributed to achieving and not achieving the program’s results? The following chart summarizes the major factors contributing to or hindering project achievements. Table 6. Major factors contributing to or hindering project achievements Factors contributing to project achievements Factors hindering project achievements Strong technical expertise within Consortium partners Apparent weakness is cooperation, information sharing, and cohesion. Offices and officers are fragmented-hired by and reporting to HQ. According to PNLS, PNMLS, “good” working relationship after new COP. Alignment with GDRC priorities, common training curricula and “accords” assistance at the zone levels. According to ProVIC, USAID has had a secession of 5 different contracting offices (4 in Nairobi) with whom to work, with different interpretations and requirements. Replacement of COP with new COP in 18th month followed by rapid and effective rebuilding to speed implementation. “Disastrous” first COP caused delays, withdrawal of CRS from consortium, loss of project credibility, HR problems, pipeline problems Since SCMS took over procurement and distribution, few stock-outs or difficulties Misunderstandings regarding the role of PATH in procurement, failure of Global Fund, delay in USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 35 reported at clinical sites. Overall positive impact in quality and continuum of care, authorization from PEPFAR to include ARVs￾breaking the continuum of care. Theft of tests. NGO partners: those with dynamic leadership, strong M and E, previous experience were able to function well and support steering committees and other programming-according to ProVIC reports and site visits. Underperforming NGOs fell short in program coherence and understanding. Team noted poor quality in technical service delivery (esp. OVC,) and inactive CCs. Champion Communities: Team visited 9 CCs and found some to be dynamic, engaged, and productive. Factors include quality of NGO leadership and support, committed CC leadership, common vision of problems and solutions vis-à-vis HIV/AIDS. Some CCs exist only on paper and meet only when called and supported by ProVIC. “Winning” the “Champion” designation lost it’s meaning with routine annual “prize” of $3000. (Or $4000? Or $1200?) . Steering committee members interviewed were unclear about the process, and evaluators found little or no awareness of the CC concept outside the steering committee Extraordinary effort and commitment by some outreach workers to mobilize community, obtain data, reach targets for the CC, NGO. Outreach worker attrition rates of 50% or higher were found in the 9 CCs visited by the evaluation team. Lack regular rewards, support. Unmotivated. What is the efficacy of ProVIC’s community mobilization approaches for OVC? ProVIC’s community mobilization approach did not contribute much to OVC because the main interventions concerning OVC were implemented by the NGO directly, with practically no participation of the CCSC or outreach workers. However, potential for assistance of OVC by such a steering committee’s leadership was demonstrated to a limited degree. Is there a cost-saving using the Champion Community service delivery model? Within the limitations of this evaluation, the team was unable to reliably establish comparable cost of comparable service delivery models in the DRC. The design and goal of the Champion Community model is not to maximize cost efficiency, but to build community competence, self￾efficacy and sustainability around the fight against HIV/AIDS. Reliance on a large cadre of volunteer community outreach workers appears to be a low cost element of the model, but the support costs of the supervisory NGO overhead, training, material, transport and staff, must be considered in the balance. Overall, with only ProVIC cost information, the team concludes that the Champion Community approach does not (nor is it designed to) maximize cost saving in service delivery. RECOMMENDATIONS 1. Aspects of model that should be continued or not: After careful consideration of the actual functioning of the Champion Community Strategy, the evaluators suggest the following: a. Health Zone: Place the Health Zone at the center of programming to include clinical services, systems strengthening, and a robust community component, with special programs for key populations, OVCs, care, and support. b. NGO Partners: Realize the benefits of the capacity-building investment to date. For example, NGOs such as JADISIDA, CEMAKI, Bak Congo, and World Production have shown improvements in performance and capacity. An evaluation should be conducted 36 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION with NGOs using set criteria, and work should be continued with those proven and promising partners. It is this type of capacity-building that will contribute to sustainability in the long run. c. Community Involvement: Keep the community involved and strengthen Zonal-level Health Committees (COSAs) or Development Committees (CODESAs). The Champion Community Steering Committees should be absorbed into the COSAs or CODESAs, which should then be reinforced to become multisectoral. Include youth and both female and male PLWHA, and strengthen the commitment and participation of religious leaders and municipal authorities. d. Outreach Workers: The Champion Community outreach workers should be consolidated into those of the COSAs or CODESAs. They should include youth, especially unmarried men and women who may have fewer responsibilities than adults and be more dynamic and willing to volunteer. Ways should be found to motivate and compensate outreach workers (by offering mobile phones, free medical consultations, bikes, prizes). e. Self-Support Groups for PLWHA: Maintain the PLWHA self-help groups because they provide important psychological support, especially when a person first learns of his/her HIV status or has been rejected by his/her family and friends. f. Orphans and Vulnerable Children: Rather than continuing with the child-to-child program, which includes only OVC, develop recreational and supportive activities for all the children in a neighborhood so that the OVC are not separated from other children and further stigmatized. g. Collaboration with Government of DRC Structures: Continue to strengthen collaboration with PNLS, PNMLS, and MINAS in the national HIV program. h. Reduce targets: The evaluation team suggests that the very high targets set by PEPFAR be adjusted so as not to encourage false reporting and compromise the quality of interventions. 2. In reference to the question concerning Orphans and Vulnerable Children, since the Champion Community Strategy was not very effective the following is recommended. a. Education: Since ProVIC or donors in general do not and cannot support education for all OVC, advocate for the state to apply free education as provided in the constitution. When providing external assistance for OVC, promote the use of block grants and school- and community-based income generation activities to pay for school fees and materials. Ensure that all the children within an OVC household have the means to attend school so that the child taken into the family for care is not the only one going to school, while others of the same age are not because the parents cannot afford to pay the fees. b. Health care: To ensure access to free health care, pool the health care funding and pay it directly to health facilities to be used to support all OVC in the Health Zone. Advocate to the Ministry of Health and Ministry of Social Affairs to award certificates of indigence to OVC for free care. c. Child protection: Add child protection community awareness-raising to community awareness projects. Popularize laws on the protection of OVC and laws relating to the rights of children in general. Provide legal assistance involving the community when OVC rights are violated (children accused of witchcraft or theft). To help solve problems of inheritance, conduct awareness campaigns in the community so that parents write wills. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 37 d. Nutrition: Since provision of a monthly meal to OVC does not contribute much nutritional value to their diet, use a family-centered approach to provide nutritional support, including promotion of kitchen gardens and raising of livestock. e. Psychosocial support: Provide social workers with the skills to address psychological issues facing OVC. f. Criteria for selection: Establish concrete criteria for OVC selection, and if funds are insufficient to support OVC resources, priority should be given to children who have lost both parents. 3. In reference to the question concerning prevention, care and treatment the following recommendations are made to strengthen the program: PMTCT a. Patient-centered care: Promote a monitoring system which focuses on patient￾centered services, with on-the-job constructive supervision. b. Male participation in PMTCT: Promote interventions to increase male participation in PMTCT, which includes making PMTCT services more male-friendly and a community peer education program with husbands who accepted participation in PMTCT. c. Indicators for monitoring quality: In all proposals and agreements ensure that the indicators include those that allow for monitoring of the quality of the services offered. PLWHA To improve services for PLWHA, the following is recommended: d. Increase access to treatment for opportunistic infections: Provide drugs for opportunistic infections in a larger number of PLWHA treatment sites and at reduced cost. e. Increase access to laboratory tests for PLWHA: Support funding to provide more CD4 testing machines in more sites and at reduced cost. f. Care and support programs should emphasize adherence to ARVs: Add issues concerning ARV adherence to care and support programs. g. Strengthen self-help groups to address pertinent PLWHA issues: Conduct research to explore why PLWHA do not consistently participate in self-help group meetings and what needs they would recommend the self-help groups could meet and include PLWHA in designing self-help group activities. 4. In reference to the question concerning key populations, USAID should support strengthening the NGOs who have demonstrated effective programs in the following ways: a. Development and implementation of a technically sound program: Develop and implement a coherent gender-based behavior change communication strategy that clearly distinguishes each key population using state-of-the-art behavior change theory. b. MSM programs: Continue and strengthen existing and new MSM networks and promote MSM associations, as they are effective strategies for HIV peer education and stigma reduction. 5. Integration Recommendations for integration are the following: a. Strong mechanisms for coordination: Whether the program providing the complete HIV package is managed through one very large project covering all components, or through individual projects for each component, USAID should include 38 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION a strong mechanism for cooperation between the contracting companies and coordination of components. b. Strong director: For a large integrated project to function effectively, a strong director with good management skills and technical knowledge is necessary. c. Collaborating partner coordination: If the full package funded by PEPFAR also depends on outside collaborators like UNAIDS or Global Fund, then it is necessary to agree upon coordination mechanisms; otherwise the PEPFAR-funded project may experience gaps in service provision. d. Mobile testing units should be maintained for key populations especially in “hot spots” accompanied by interventions to strengthen the links to clinic based services. e. Linkages between community and clinic based services should be strengthened for both PLWHA and OVC including promoting a better understanding of the referral system by NGOs and health care providers. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 39 ANNEX I. SCOPE OF WORK Global Health Technical Assistance Bridge 3 Project GH Tech Contract No. AID-OAA-C-13-00032 SCOPE OF WORK May 6, 2013 I. Title USAID/DRC: Integrated HIV/AIDS Program in the Democratic Republic of the Congo (ProVIC) Contract: Global Health Technical Assistance Bridge 3 Project (GH Tech) II. Performance Period On/around May 20, 2013–on/around August 16, 2013 III. Funding Source Mission-funded IV. Purpose of Assignment The purpose of this evaluation is to determine the effectiveness of the ProVIC program and in particular, the Champion Community implementation strategy and also make recommendations for future community mobilization programming with an emphasis on OVC. The evaluation is expected to provide results not only on the overall effectiveness of the program, whether it met its intended objectives or not, but also will provide more detailed input into which elements of the program worked and which did not. The results will provide an overall assessment of the model and will lead to many decision points including whether the model should be scaled up and/or which elements of the model should be eliminated and/or strengthened. V. Background A. Country Context The Democratic Republic of Congo (DRC) is a strategic priority for U.S. foreign assistance due to its size, location, and geopolitical role in the region. Despite its tremendous economic potential and its enormous natural resource wealth, the DRC is among the world's poorest and least developed countries. In 2011 the United Nations Development Program (UNDP) ranked the DRC as the least developed country in the world (ranked 168 out of 168). Pervasive corruption, historical political instability, and a lack of infrastructure severely limit both domestic and foreign investment. The Democratic Republic of the Congo is home to nearly 70 million people. For almost 20 years, DRC has had some of the worst ratings on health indicators and performance on human development and governance measures. As mentioned above, the DRC ranked last in the world in the UNDP Human Development Index and the Human Poverty Index for developing countries (HPI-1). The leading causes of death and disability include malaria, diarrheal diseases, respiratory infections, violence, and road collisions. At 1.3%, DRC’s adult HIV prevalence is lower than that of many of its neighbors which have some of the highest prevalence rates in the world. Regardless of the low reported prevalence, HIV prevention and testing activities remain 40 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION essential for the general population as well as women and urban populations in particular. In addition, it is critical to ensure services to the populations residing in Eastern DRC to track potential changes due to movement across borders by mobile and other at-risk populations, as well as the influx of people into the DRC for reconstruction economic opportunities. DRC’s maternal mortality rate is 549 per 100,000 live births. DRC’s total fertility rate is 6.3 children per women, contraceptive prevalence rate is only 6%, and unmet need for family planning is estimated at over 24%. Fewer than 45% of children with a fever receive any care from a trained professional, yet 72% of women giving birth reported receiving assistance from a trained professional. Due to long periods without adequate data collection, the country lacks reliable markers by which to accurately gauge trends. The 2007 Demographic Health Survey (DHS) estimates suggest that infant, child, and maternal mortality increased between 2002 and 2007. Only 48% of the population has access to an improved drinking water source, and less than 18% have access to adequate sanitation; over 16% of children under five experienced an episode of diarrhea in the two weeks preceding the survey. According to the 2007 DHS, nearly 45% of DRC’s population lives in urban areas. The rate of urbanization in the DRC is increasing each year due to insecurity in many rural areas and the promise of better economic opportunities in urban areas. The Government of the DRC (GDRC) provides very little budgetary support to the health sector. While the total budget allocated to health has increased each year since 2000, the proportion of the budget allocated to health fell to just 2.5% of the GDRC’s overall budget in 2008. Although the Ministry of Health (MOH) is asking all donors to support a “minimum” package of basic health services available in those facilities where they work, at present, this does not occur in all health zones (HZs), and some donors do not support the complete primary health care (PHC) health package in all the health facilities in the zones where they work. When MOH services were disrupted in the 1990s due to civil war, faith-based organizations (FBOs) and non-governmental organizations (NGOs) became the primary source of medical care for large segments of the Congolese population. Many of the FBOs had already been active in DRC for decades. This dependence on FBOs and NGOs has since been formalized through the establishment of service delivery contracts with the large religious denominations and selected NGOs. There are also some NGOs engaged in service delivery that do not have direct contracts with the MOH but derive authority from donor projects. Finally, there are some NGOs that hold contracts with provincial medical health inspectors (MIPs) that have been “grandfathered” from before the Health System Strengthening Strategy (HSSS) and new decentralized plans were formulated. Within the past five years, many initiatives have been undertaken at the national and district level. The World Bank-supported Multi-country AIDS Program (MAP) ended in May 2011. The MAP project worked in 11 provinces. Global Fund implementation was frozen due to mismanagement issues in 2011; National AIDS Control Program (PNLS) revised the five-year strategy in 2011 to align it with the national health development plan (PNDS). B. Context of the HIV/AIDS Epidemic in the DRC The 2007 Demographic Health Survey (DHS) in DRC indicated that the country is facing a generalized HIV/AIDS epidemic with stark geographic and population differences. The majority of new HIV/AIDS cases are diagnosed among people less than 24 years of age; and the epidemic has distinct geographic patterns. Though the overall HIV prevalence in DRC is 1.3%, rates are twice as high in urban versus rural areas (1.9% to 0.8%) and higher among women and men (1.9% vs. 0.9%). While HIV prevalence remains higher in urban areas, it has increased in certain USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 41 rural areas, particularly those near geographic hotspots, which may be a result of sexual networks that bridge low-prevalence groups engaging in risk behaviors with members of high￾prevalence groups. High-risk and high-prevalence populations often congregate in geographic “hotspots,” such as border crossings, transport corridors, ports, and regions with a large military presence. The already elevated rates of most-at-risk populations (MARPs), which includes commercial sex workers (CSWs), truckers, miners, and uniformed services are often more than triple or quadruple the rates in the rest of the country. Truckers have a national prevalence rate of 3.3%, but in Katanga, long-haul truckers from southern African countries have an HIV prevalence rate of 7.8%. A seroprevalence survey conducted in Kinshasa in 2008 indicated that prevalence in the military was 7.5% among women and 3.6% among men. A 2006 bio-sero survey found a prevalence rate of 16.9% among CSWs, and rates in the provincial capitals of Katanga and Kasai Oriental were elevated to 23.3% and 24.5%. Fifty-five percent of miners; 32.9% of the military, 75.1% of street boys, and 81.1% of street girls report multiple sex partners within the past 12 months, therefore increasing their risk for transmission. Pregnant women are particularly at risk; antenatal care (ANC) surveillance data from 2010 indicate that pregnant women had a prevalence rate of roughly twice that of other women at 2.0%. The 2011 ANC data showed urban prevalence rates ranging from 1.35% in Bukavu to 6.86% in Tshikapa. Furthermore, gender inequalities, war, and political and economic instability resulted in widespread sexual violence, intimate partner violence, physical abuse, and an increase in commercial sex work. Large numbers of orphans and vulnerable children represent a notable consequence of the HIV epidemic in the DRC. The 2007 DHS found that 13.1% of children did not live with either biological parent and that globally an estimated 4.5 million children had lost one or both parents to AIDS and other causes. UNICEF estimated in 2006 that there were 40,000 street children in the country, 14,000 of whom were living in Kinshasa. Challenging social, political, and economic factors contribute to a weakening and breakdown of families, resulting in the separation of children who may seek alternative survival strategies on the street or through dangerous labor or armed groups. The DRC suffers a high level of poverty. The 2006 poverty assessment showed that 71% of Congolese households lived below the poverty line. Macroeconomic data suggest that the situation has not improved much since then. Poverty, a major underlying cause of children becoming separated from family support, also contributes to significant levels of marriage dissolution, teenage pregnancy, sexual exploitation, and abuse. In addition, a 2006 study by Javier Aguilar Molina revealed accusations of witchcraft to be a particularly problematic cause of family separation. The geographic size of the DRC and its logistical hurdles create a unique set of challenges for delivering services. Currently, the health system in the DRC has three tiers: 1) a central level which includes the MOH, the Secretary General of the MOH, and Directorates of national disease-specific programs; 2) an intermediate level composed of 11 provincial health departments and 48 administrative health districts; and 3) the peripheral level with 515 HZs containing over 8,000 health centers (HCs). Approximately an equal number of health sites are publically and privately supported. In addition, the health system relies on two types of volunteer community health workers: 1) community health providers whose activities are limited to health promotion and community mobilization activities; and 2) community treatment workers who deliver a limited set of interventions (i.e. treatment of diarrhea, fever, and referral of malnourished children to health facilities, plus distribution of a limited number of family 42 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION planning commodities). Most provinces use a centralized pharmaceutical procurement system through the Federation of Essential Medicine Procurement Agencies (FEDECAME), combined with a decentralized distribution system supported by existing distribution hubs (CDRs). The U.S. Government is providing significant technical assistance and commodities in supply chain management at various levels of the system to build capacity and avoid stock-outs of essential medication. C. Project Identification The DRC Integrated HIV/AIDS Project (ProVIC) is funded through a Task Order contract with the United States Agency for International Development (USAID) for the period October 2009 through June 2014. It is implemented by Program for Appropriate Technology in Health (PATH), and includes Chemonics International, the Elizabeth Glaser Pediatric AIDS Foundation (EGPAF), and the International HIV/AIDS Alliance (IHAA) as consortium members. Key Government of DRC (GDRC) partner institutions include the MOH, Ministry of Social Welfare, Ministry of Gender, and the DRC AIDS Commission (PNMLS). Civil society partners at the national and district level are also critical partners. ProVIC was designed by USAID in July 2009 to implement the USAID/DRC Integrated HIV/AIDS prevention, care and treatment service delivery program with a primary focus on the MARPs and a secondary focus on the general public. This program was expected to address the gaps in the quality and quantity of services provided in selected areas by leveraging resources and activities provided by the U.S. Government PEPFAR implementing partners, the Global Fund, World Bank, United Nations, and other stakeholders in accordance with the Ministry of Health's recommended approach and in alignment with the DRC strategic plan (both the MOH and the multi-sector strategic plan). Throughout the past three years, ProVIC has operated within a dynamic environment of evolving donor priorities, funding mechanisms, and system constraints. The RFTOP for the program that became the DRC Integrated HIV/AIDS program was issued in July 2009 by USAID. Due to an increase of U.S. Government HIV/AIDS resources for DRC, the scope of the contract was expanded from $44 million to $49 million in year three to include an additional location. D. Project Approach The objective of the DRC Integrated HIV/AIDS Project, Projet Integré de VIH/SIDA au Congo (ProVIC) is to contribute to the reduction of the incidence and prevalence of HIV and mitigate its impact on people living with HIV/AIDS (PLWHA) and their families in the DRC. It will achieve this objective by improving HIV/AIDS prevention, care, and support services in and around 49 Champion Communities (CCs); increasing community involvement in health issues and services beyond facility-level services through sustainable community-based approaches; and increasing the capacity of government and local civil-society partners, and thereby empowering new local organizations to plan, manage, and deliver high-quality HIV/AIDS services. ProVIC uses these objectives as a strategic guideline for linking project activities to results. ProVIC’s approach is based on the following strategies:  Integrated and innovative community-based approach. The approach empowers community members to identify their needs and develop strategies to address them, both through simple actions that can be taken by the communities themselves and through linkages to services such as HIV counseling and testing (HCT), palliative care, prevention of mother-to-child transmission of HIV (PMTCT), and support groups for PLWHA and orphans and vulnerable children (OVC) that are supported by ProVIC or other partners USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 43 in the area. The project is also supporting communities to integrate the needs of the MARPs into its strategies by incorporating them into the community goal-setting process and developing strategies for outreach to those communities.  Sustainability through capacity-building at all levels. The project works toward sustainability of interventions through a strategy of helping local institutions to take ownership of activities while also increasing the capacity of government to implement activities in public health facilities, and also to coordinate and supervise activities at the provincial and health zone level. The components of our strategy include encouraging communities to take responsibility for their health outcomes, strengthening the capacity of government partners in supervision and oversight, and building the capacity of non￾governmental organizations (NGOs) and civil society partners to plan and manage HIV/AIDS interventions. These communities are directly linked with health zone leadership structures to ensure these activities are coordinated with government planning.  Leveraging resources through partnerships. ProVIC continues to reach out to other partners that work in HIV/AIDS and other sectors, such as the Global Fund to Fight AIDS, Tuberculosis and Malaria (linkages for long-term anti-retroviral care), United Nations Children’s Fund (provision of HCT services linked to the United Nations Children’s Fund ), Joint United Nations Program on HIV/AIDS (mapping and services to MARPs, particularly men who have sex with men (MSM) and CSW), United Nations Population Fund (gender-based violence (GBV) services, including provision of post￾exposure prophylaxis), Population Services International (linkages of PSI WASH initiatives to CCs), Management Sciences for Health/Strengthening Pharmaceutical Systems (strengthening of hospital capacity to manage commodities), University of North Carolina (referrals in Kisangani), and Communication for Change (development of messages for care and support) to leverage their resources and create more integrated services at the community level. E. Project Organizational Structure and Management The project is structured to implement activities and achieve results in the five targeted provinces, Katanga, Orientale, Bas-Congo, South Kivu, and Kinshasa, with the office in Kinshasa providing overall supervision and management. The Chief of Party and Deputy Chief of Party oversee the project strategy and the design and integration of activities, and ensure results are being met. Technical specialists in Kinshasa develop project strategies and activities in their areas and supervise them across the five provinces. The four regional offices in Matadi, Lubumbashi, Kisangani, and Bukavu are headed by regional coordinators who supervise technical specialists in prevention, care, support, and monitoring and evaluation (M&E). The specialists receive guidance and input from the technical specialists in Kinshasa and from international experts in their home offices. Each office also has a grants manager and administrative staff who manage the funding of partners in that region. Kinshasa’s regional activities are overseen by a Kinshasa regional coordinator with support from a deputy regional coordinator and input from the technical specialists based in Kinshasa. The project team is organized to be a streamlined single operating unit that will benefit from the expertise of all the consortium members while functioning as one integrated project. The four international partners provide managerial and technical backstopping to the field, in addition to providing quality control of deliverables and guidance on compliance with U.S. Government regulations. VI. Objectives and Planned Results A. Key Goals and Objectives of the Project: 44 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION Program Goal: to reduce the incidence and prevalence of HIV and mitigate its impact on PLWHA and their families. Program Objectives: 1. Improve the accessibility and quality of HIV/AIDS prevention, care, and treatment services in the selected areas; 2. Increase community involvement in health issues and services beyond facility￾level services through sustainable community-based approaches; 3. Increase the capacity of government and local civil society partners (seeking to empower new local organizations) to plan, manage, and deliver quality HIV/AIDS services; 4. Assist the government to develop, disseminate, and implement evidence￾based policies such as counseling and testing, adult care and treatment, and OVC policies that result in improved service quality. B. Life of Activity Expected Results: 1. HIV counseling and testing and prevention services expanded and improved in target areas a. Communities’ ability to develop and implement prevention strategies strengthened; b. Community- and facility-based HIV counseling and testing services increased and enhanced; c. Prevention of mother-to-child transmission of HIV services strengthened. 2. Care, support, and treatment for PLWHA and OVC improved in target areas a. Palliative care strengthened; b. Care and support for OVC strengthened. 3. Strengthening of health systems supported a. Capacity of provincial government health systems supported; b. Capacity of non-governmental providers improved; c. Strategic information systems at community and facility levels strengthened. VII. Approach and Implementation A. USAID’s Integrated Model Before FY 2009, USAID’s HIV/AIDS program was primarily focused on filling gaps at the service delivery level, providing HIV services with a focus on a few high-prevalence urban hubs. USAID/DRC’s HIV/AIDS programs have traditionally been focused primarily on community-level efforts to address the epidemic. With an increasing budget, there have been additional opportunities to strengthen USAID’s involvement in facility-level care, the critical links between community and clinic level services, and target-high prevalence hotspots which should include rural and peri-urban sites. Given the extremely limited resources of the U.S. Government HIV program overall, anti-retrovirals (ARTs) have not been procured. However, several U.S. Government programs are leveraging other donors’ investments in ART drugs and services to complement U.S. Government services. The HIV/AIDS specific gaps identified by a 2013 OGAC assessment visit:  There is a need to reinforce and expand current programs in prevention, HCT, TB/HIV, palliative care, and OVC in the USAID focus areas. A more rational and USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 45 comprehensive package of services delivered to PLWHA and OVC is needed for palliative care, home base care, and OVC programs which address nutrition.  There are significant gaps in the linkages between community- and facility-level activities that USAID-funded activities should help bridge.  Developing and strengthening capacity of local NGOs (seeking new indigenous organizations and strengthening existing ones) to plan, manage, and implement HIV activities will be critical and needs to be addressed.  There are numerous gaps in ART services that could be addressed by future USAID programming and quality of services. U.S. Government partners are well positioned to provide quality ART ancillary services as well as technical assistance (TA) to other organizations.  There is a gap in prevention activities including sexually transmitted infections (STI) treatment for PLWHA and other high-risk populations, that should be addressed in future USAID programming as a way of providing more comprehensive services to highly vulnerable populations.  There is a gap in realizing the possible synergy from opportunities to integrate HIV programming with other health programming such as family planning, malaria, safe water, maternal and child health, and food and nutrition.  There is a need to address relevant and key policy gaps in HIV prevention care and treatment to improve service delivery through developing, updating, and disseminating identified policies. In addition, during a country visit from PEPFAR Deputy Principals in May 2009 a list of key principles was identified to guide program development. These principles include: 1. Prioritize areas and programs with the potential for greatest impact. 2. Prioritize complementary programs (among U.S. Government and with other major partners). 3. Develop programs based on strategic information and programmatic data obtained through evaluations, studies, and annual reporting. 4. Facilitate ownership of activities by key stakeholders. 5. Recognize that the need is huge and focus on key strategic positioned programs. 6. Quality programs in each area require a complement of services, equipment, staff, and support. 7. Focus on what is feasible. 8. Prioritize key geographical focus zones. 9. Identify key local partners with whom to work. 10.Align with the GDRC national HIV/AIDS strategic plan (using national and direct indicators). 11.Recognize the challenge of government interest that all health zones have equal access to services; however, given limited resources there is a need to focus on highest￾impact areas based on epidemiology, existing programs/systems, travel challenges, and ensuring quality programs. B. Strategy of Intervention The Technical Approach of the ProVIC Project stated that they would be able to address the following: 1. HCT and Prevention Improved 2. Strengthened Communities to Develop and Implement Prevention Strategies 3. Improve Access to and Quality of Community- and Facility-based HCT 46 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 4. Increase Access to Comprehensive PMTCT Services 5. Coordinate with Existing Prevention and Social Marketing Activities 6. Care, Support, and Treatment Improved 7. Strengthened Access to and Quality of Palliative Care 8. Strengthen Access and Quality of OVC Care and Support 9. Health System Strengthening and Human Capacity Development Through its central strategy of building champion communities, ProVIC seeks to bring about change and indeed to transform communities by empowering them with the tools to define and meet community objectives. This strategy includes fueling change in addressing and tackling the principal determinants and underlying conditions that make people and whole communities vulnerable to contracting HIV and developing AIDS. It also includes change from classic assistance that has characterized international HIV/AIDS responses within the past few decades, to a more holistic, dignified approach guided by three principles: innovation, integration, and sustainability. C. Champion Community Approach The Champion Community approach engages communities to develop and implement cost￾effective and successful prevention strategies. The model is designed to bring communities together to achieve their social and economic development objectives and engage them in promoting and adopting new behaviors linked to those objectives. It also fosters renewed social cohesion. Champion Communities are unique in their use of a clear incentive system—most important, visible returns on investment—to quickly initiate and then sustain new individual behaviors and societal norms. Under the model, communities receive tools and resources to meet community-identified objectives within an 18-month time frame, after which they will be publicly awarded Champion Community status and receive community-identified incentives. Progress toward objectives is assessed in an inclusive manner, reinforcing community engagement and mobilization. The model ensures women’s needs are addressed by involving representative organizations, using gender-related tools, targeting women for prevention messages, and fostering discussions between men and women to jointly address health challenges. Because of their strong relationships with USAID targeted communities, selected NGOs or umbrella organizations execute the model’s roll-out. Community counselors are actively involved as facilitators and key enablers for institutionalizing the approach and preparing the exit strategy. ProVIC’s interventions are supporting service delivery while expanding the capacity of communities and institutions to mobilize and sustain community action against HIV/AIDS. At the core of ProVIC’s strategy are the 43 Champion Communities, distributed across 34 health zones, in urban areas in the provinces of Kinshasa, Katanga, South Kivu, Orientale, and Bas￾Congo. The principles that guide ProVIC’s vision, strategies, and actions contain the essential ingredients for bringing about a self-reliant approach to the fight against HIV/AIDS. They encourage and empower national and regional governments and local communities to work in synergy and in close collaboration with each other. In this context, the community becomes the key point of departure from which all HIV/AIDS-related interventions emanate. The program supports prevention activities designed to reduce the incidence and prevalence of HIV/AIDS and mitigate its impact on PLWHAs and their families. This entails reducing transmission among MARPs (including commercial sex workers and their clients, truckers, miners, the military and police, youth, street children, and other categories as identified through existing or future behavior surveys), as well as PLWHA. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 47 Other main program activities are HIV/AIDS Counseling and Testing (HCT), treatment of tuberculosis-HIV co-infection, and care and support for PLWHAs and OVCs. Human capacity development and health system strengthening activities are also undertaken as part of the program. Priority activities reinforce current USAID-funded activities in addition to the prevention of mother-to-child transmission (PMTCT), when appropriate, in South Kivu (Bukavu￾Uvira), Bas-Congo (Matadi-Boma), Orientale province (Kisangani), and Katanga (Lubumbashi, Kasumbalesa, Likasi, Kipushi, Kolwezi) transport corridors. D. USAID/PEPFAR ProVIC OVC Activities in the DRC PEPFAR/USAID supported efforts to carry out a Rapid Appraisal, Analysis, and Action Planning (RAAAP) process in 2005 that culminated in the National Plan of Action for Orphans and Vulnerable Children 2010–2014. USAID/PEPFAR continues to support efforts to coordinate and monitor at the national level child protection activities as well as programming for children affected by HIV/AIDS and build the capacity of the Ministry of Social Affairs, Humanitarian Action and National Solidarity (MINAS) and key divisions of social affairs (DIVAS) to lead such efforts. In FY 2010, the U.S. Government provided supplemental funding in the amount of $15 million to help meet the emergency protection needs of vulnerable populations. E. Governmental Roles and Structures for OVC Several government ministries have responsibilities relevant to the protection of children separated from their families, the principal one being the MINAS. It is responsible for initiating, coordinating, and implementing policies for the social protection of vulnerable groups, including OVCs. At the provincial level, MINAS provides services through the DIVAS or the urban division of social affairs (DUAS). The Appendix includes an overview of relevant responsibilities of other ministries as well as information on coordination groups and legal information relevant to children at risk of separation or outside family care. There is strong protective legislation for children in place in the DRC. In ratifying the United Nations Convention on the Rights of the Child, the Government of the DRC obligated itself to address the full range of children’s rights. The national constitution in two articles mandates action to protect children. These are reinforced by the 2009 national Child Protection Law, which provides for wide-ranging protection. While these instruments are laudable and significant, governmental capacity and budgetary allocations for their effective implementation are very limited, and the situation of the majority of children is precarious. In addition, the country’s legal framework for the protection of children was substantially strengthened with the adoption of the Child Protection Law in 2009. To better apply this law, MINAS has issued a decree on social care (arrêté sur le placement social) and (in conjunction with the Ministry of Gender) another on support to vulnerable families. F. Donor and NGO Roles in OVC Programming UNICEF implements several initiatives to protect vulnerable children, including a large, nationwide effort to support children orphaned and made vulnerable by HIV/AIDS, training for “social assistants” employed by government and NGO to provide direct services for vulnerable children and families, and several efforts intended to strengthen national protection policies, as well as national and local child protection coordination and monitoring mechanisms. As noted previously, the World Bank is in the process of initiating a five-year, $10 million project for street children primarily in Kinshasa. Some of the project’s public information and capacity-building activities for MINAS personnel will extend to other parts of the country as well. 48 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION A range of local and international NGOs implement activities intended to support the protection and well-being of children and families and prevent and respond to family separation. Many of the local NGOs operate with very limited resources and personnel. In Kinshasa most are members of the REEJER (Réseau des Educateurs des Enfants et des Jeune de la Rue) network. There is no comparable network in Bukavu or Mbuji Mayi. Most international NGOs are members of the COPERF (Collectif des Organisations Internationales pour la Protection des Enfants en Rupture Familiale sur la ville province de Kinshasa) network, which seeks to coordinate activities and harmonize approaches. VIII. Existing Information Sources The following information document and sources are available and relevant to the study:  GDRC: o 2010–2014 HIV/AIDS National Strategic Plan o 2010–2014 National Plan of Action for Orphans and Vulnerable Children o National Health Development Plan o National HIV Surveillance Data o  USAID: o PEPFAR DRC Partnership Framework o Original Request for Task Orders proposals (RFTOP) OAA-GH-OHA-09-0012 IQC under AIDSTAR SECTOR 1–Technical Services; o ProVIC Audit Findings and Recommendation o Community Champion Evaluation Findings o PMTCT Acceleration Plan o PEPFAR Scale-up Document o HIV Assessment Report  ProVIC: o Task Orders contract and amendments o Annual and quarterly reports o Annual workplans, results framework, and performance monitoring plan o Strategy papers for core services o Tools, training materials, guidelines, etc. o Grantee stories, lessons learned, case studies o Internal assessments and reviews  Donor: o World Bank Proposal for street kids: http://web.worldbank.org/external/projects/main?pagePK=64283627&piPK=73230&theSi tePK=40941&menuPK=228424&Projectid=P115318 IX. Evaluation Rationale and Key Questions The Automated Directive System (ADS) 203.3.6.1 requires that an evaluation is conducted when there is a distinct and clear management need to address an issue. The evaluation will be USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 49 able to provide answers both at programmatic and strategic level by addressing the question of whether ProVIC achieved the intended goals and the cost value added for the Champion Community. The evaluation will also reveal lessons learned about program implementation that will have a bearing on scaling up HIV/AIDS interventions and replication of similar programmatic interventions nationwide. The evaluation will focus on answering following illustrative questions based on the objectives of the project. Final evaluation questions will be developed by the evaluation team in collaboration with USAID/DRC at the Team Planning Meeting. A. How has community involvement increased as a result of the intervention (primarily the Champion Community), and what difference has this made in the health of the target populations? The champion community model was a key innovation that the ProVIC consortium brought to the table and was one of the main reasons that they were selected for implementation of this Task Order. This model has gained popularity and has recently been adopted by USAID’s Integrated Health Program to help manage malaria activities. We want to know if this intervention should be standardized and included in all future activities. Additionally, one of the selling points of the Champion Community model was that it would get the community involved in preventing new HIV infections. While we will not be able to measure changes in prevalence on this small scale, it would be useful to know if health-seeking behaviors and attitudes have shifted. B. What are the strengths and weaknesses of the integrated approach? As USAID decides on the next model of community engagement, what lessons can we learn from the ProVIC experience and how can we avoid many of the pitfalls that befell them? C. How has the quality of HIV prevention, care, and treatment services improved as a result of the intervention? ProVIC had both community- and facility-level interventions. While this question tries to discover if community engagement resulted in communities demanding better services at the facility level, it is more focused on the facilities that ProVIC supported itself. Are these facilities better off now than they were at the beginning of the ProVIC intervention? Did the fixed-obligation grants (FOGs) and memorandum of understanding (MOU) allow for adequate oversight, and was funding sufficient to engender change? D. To what extent has ProVIC improved access to services for MARPS in DRC? One of the common criticisms we hear about ProVIC is that they are able to target MARPs with strong messages of prevention, but not necessarily refer them to the appropriate facilities for treatment. We want to know if this is true. Are there systems in place that work to ensure that MARPs are not lost to follow up when they are referred to services? Are there recommendations that can be made to avoid this in future programming? E. What key aspects of the ProVIC model should be continued and/or discontinued in future HIV/AIDS programming? The ProVIC project was the first time USAID/DRC had pursued an integrated approach to providing HIV services. As it was originally envisioned, community programs would link closely to facility programs. In reality, this did not always happen. Champion Communities were put in places where the community already had a certain sense of mobilization and ProVIC chose to work in high-volume facilities. While on occasion, the community services did link up with the community components, this was not always the case. We are trying to decide whether to break the next award into two projects (facilities-based and community care and support/prevention). Answering this question will be critical in our decision-making process. 50 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION F. What are the factors that contributed to achieving or not achieving the program’s results? What aspects of the ProVIC project were critical to achieving planned results and what factors served as a constraint? These factors will be instrumental to the design of new HIV/AIDS programming. G. What is the efficacy of ProVIC’s community mobilization approaches which emphasize services for Orphans and Vulnerable Children (OVC) currently being implemented? The ProVIC project has used a variety of community mobilization models as have the various donor and NGO partners mentioned. Conducting a limited desktop assessment of these models will provide valuable insights into the best strategic approaches for future OVC planning in the DRC. H. Is there a cost savings using the Champion Community service delivery model? The ProVIC project spent a significant portion of its budget targeting these communities for prevention and sensitization activities. There is a push within PEPFAR to limit outreach in the general population and only focus on key populations. Are there any economies of scale that occurred in bringing down the overall cost of community interventions? Are there additional recommendations that could further increase cost savings? X. Evaluation Design and Methodology A. Evaluation Purpose The purpose of this evaluation is to determine the effectiveness of the ProVIC program and in particular, the Champion Community implementation strategy and also make recommendations for future community mobilization programming with an emphasis on OVC. The evaluation is expected to provide results not only on the overall effectiveness of the program, whether it met its intended objectives or not, but also will provide more detailed input into which elements of the program worked and which did not. The results will provide an overall assessment of the model and will lead to many decision points including whether the model should be scaled up and/or which elements of the model should be eliminated and/or strengthened. B. Evaluation Methodology This is a summative performance evaluation which will use a mixed-method approach by collecting and analyzing both qualitative and quantitative data. It will be useful to compare overall performance with the three scenarios where ProVIC operated: 1) Stand-alone Champion Communities (without clinical linkages); 2) Stand-alone clinical programs (without community linkages); and 3) Champion Communities with linkages to ProVIC-supported facilities. It may use some or all of the following data collection methods: 1) Review of project documents; 2) Key informant interviews; 3) Focus group discussions; 4) Comparative assessments of community mobilization models emphasizing OVC; and 5) Data collection at the facility and/or community level. Data will be collected by a team at multiple levels including national, provincial, district, and community levels. Site visits will be conducted to a sample of sites selected to provide a picture of the overall program as part of a negotiated agreement between the evaluation team and the Mission. The sample frame of sites will be affected by program needs, by the level of security in each of the four targeted provinces, and the availability of safe transport between the field site and Kinshasa. ProVIC will provide a complete list of program sites and work with USAID to USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 51 randomly select sites that will be visited for the key informant and focus group interviews. Once the sites are chosen, key informants at those sites will be selected through consultations with ProVIC staff and the USAID Team. The sites and informants sampled will take into consideration rural and urban differences as well as geographic and cultural dimensions of the areas where the program is implemented. The evaluation team will employ a variety of complementary methodologies for data collection. The team will share the methodologies selected as well as the data collection tools with USAID and selected GDRC counterparts for approval prior to commencing field work. Proposed points of contact/interviews:  Key central government officials from Ministry of Health/AIDS Control Program, DRC AIDS Commission, Ministry of Social Affairs  Local government officials including representatives from the Provincial HIV/AIDS Committee  National, district, and community-level partners including non-governmental, faith-based, and people living with HIV/AIDS  USAID representatives and representatives of the U.S. Government Emergency Plan Team members  Others as determined appropriate XI. Evaluation Products A. Deliverables Expected outputs from the mid-term evaluation are: 1. Draft workplan and data collection tools 2. Debriefing of preliminary findings at the USAID/DRC office 3. Draft of the final evaluation report with following elements:  Detailed review of actual results of the project compared with planned results;  Well-documented review of the effectiveness of the strategy and approach implemented by the project;  List of clearly defined recommendations that may affect the scope of the project during its last year and implications for the design of new HIV/AIDS programs in health;  A separate report describing the various community mobilization models, results achieved, strengths and weaknesses of each model, best practices, and recommendations for future community mobilization approaches for use in USAID/DRC PEPFAR programs, with an emphasis on OVC. [This deliverable was annulled in the meeting held with USAID DRC in discussions to arrive at a common understanding of the evaluation questions and expectations of the evaluation. It was determined that this was not really within the scope of the evaluation, and the time was not at all sufficient to allow for such a report.] 4. Final evaluation report: the final evaluation report will be completed within 10 working days of receiving USAID’s and ProVIC’s comments. B. Evaluation Report Outline The evaluation report shall include the following sections, at a minimum:  Executive Summary: overview of the evaluation and key findings and recommendations  Introduction and Background: definition of the problem, summary of the project, purpose of the evaluation 52 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION  Findings: relevance, effectiveness, impact, and sustainability  Considerations for integration approach  Considerations for Champion Community approach  Conclusions  Recommendations GH Tech will provide an evaluation report template to the evaluation team. In order for there to be adequate time for the final report to be professionally edited and formatted, USAID/DRC must provide final approval on the report by no later than July 26, 2013. Once the report has been edited and formatted, GH Tech will provide USAID/DRC with one electronic copy of the report. In the event that the approval process is delayed, GH Tech will work with USAID/DRC to identify other mechanisms that would be able to edit and format the final report. XII. Evaluation Team The evaluation team will be composed of two international consultants, one of whom will act as team leader, two national experts, and a minimum of one USAID/DRC staff member. An additional USAID/Washington HIV specialist may also join the team. The experiences and knowledge expected from experts are:  Team leader: Master’s degree (a Ph.D. holder preferred) in subject matter expertise with at least ten years of experience in the health sector and evaluations in Africa. The candidate should have strong background in HIV/AIDS-related subject matters such as: tuberculosis and OVC. French language proficiency is a must, and previous experience with the DRC settings is strongly preferred.  Second international consultant: Master’s degree in subject matter expertise with at least ten years of experience in the health sector and evaluations in Africa. The candidate should have strong background in HIV/AIDS-related subject matters such as: tuberculosis and OVC. French language proficiency is a must, and previous experience with the DRC settings is strongly preferred.  National consultants: Master’s degree in Public Health with at least five years of experience in Africa. The candidate should have strong background in HIV/AIDS-related subject matters such as: tuberculosis, OVC, public-private partnerships, and prior experience with local governance and civil society. One of the national consultants will need to have specialist expertise in OVC and community-based interventions.  USAID/Washington HIV/AIDS M&E technical assistance: The candidate should be able to understand the PEPFAR mechanisms and speak to the complexity of PEPFAR programming requirements, particularly the Next Generation Indicators.  One USAID/DRC staff member: The staff member will assist the team with any information needed from USAID, as well as participate in the data collection for the evaluation. The Team Leader will determine the overall appropriate level of participation as to limit evaluation bias. XIII. Evaluation Management A. Roles and Responsibilities Government of DRC (PNMLS and Ministry of Health,) USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 53  Serve as key points of reference and information, including key documents, for final review  Participate in oral debriefing  Review and comment on final report USAID’s roles and responsibilities:  Select and contract the evaluators  Have a full-time USAID/DRC staff member or representative to participate in the evaluation  Manage the evaluation process  Review draft workplan and data collection tools  Review draft report and provide feedback  Sign off on the final report  Submit evaluation report to USAID/PPC/CDIE ProVIC’s roles and responsibilities are to:  Participate in final review scope of work development  Provide relevant documents as needed  Provide logistical support for the evaluation team including office space, assistance with setting up meetings and interviews, and providing transport where no other means are possible. Evaluation Team Leader’s roles and responsibilities:  Guide and manage evaluation exercise  Participate in Evaluation  Responsible for all deliverables to USAID  Provide briefing to team  Provide initial draft of final report to USAID  Revise draft based on feedback and provide final draft to USAID Second International Consultant’s role and responsibilities:  Assist team leader in the management of the evaluation exercise  Participate in the evaluation  Contribute to evaluation design and strategy  Supervise selected elements of evaluation as indicated by team leader (e.g. OVC focus)  Assist in preparation and finalization of all drafts and reports  Support other management, administrative, and technical needs identified by team leader National Consultants’ roles and responsibilities:  Provide any pertinent information that may affect the implementation of the evaluation strategy  Contribute to evaluation strategy  Support data quality  Assist in the data collection processes  Provide translation support to team leader when needed 54 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION B. Logistics Evaluation logistics will be carried out bight Tech Bridge III and the evaluation team. When possible, ProVIC will provide necessary informational and other support to the evaluation team during field visits. The proposed sites of field visits will be determined ahead of time through a randomized selection process. Due to security and logistical constraints in the DRC, these sites will then be finalized closer to the start of the field work in order to ensure that the locations are secure and feasible to be reached within the budget allocated for the evaluation. Only two sites outside of Kinshasa will be visited, Lubumbashi in the southeast of the country and Matadi in the west. The overall sample frame of sites will be affected by project needs, by the level of security in each of the six targeted provinces, and the availability of safe transport between the field site and Kinshasa. The team could do two field visits per week during weeks two and three of the evaluation, with a visit of approximately two or three days per province. The team may split up for one or more field visits. The USAID staff will have prepared for the team’s visit by arranging meetings with the relevant project staff and stakeholders. GH Tech Bridge III and the evaluation team will organize groups for focus group interviews. The specific questions and foci of these discussions will depend on the emphases that the evaluation team desires the team to investigate. There will also be site visits to clinics, local community group sites—and others, if appropriate—to determine the quality of the services being provided. XIV. Schedule A. Timing Team members will be expected to work approximately five weeks in DRC for conducting field activities. The evaluation team international consultants will spend one week in literature review and gathering evaluation materials. They will, likewise, spend two weeks for wrapping up the evaluation report starting from data collection finalization. The in-country activity is expected to commence on/aroundMay27, 2013, and a draft report submitted by on/around June 28, 2013. USAID will review and provide comments on the draft evaluation report within a period of about two weeks of its receipt, and the evaluation team leader will be required to submit to USAID/DRC a final version of the evaluation report no later than July 19, 2013. This final report will be submitted by electronic file. USAID/DRC will review the final report within a period of one week and submit to GH Tech Bridge III for edits and formatting by July 26, 2013. GH Tech Bridge III will require up to 30 days to complete editing and formatting. B. Level of Effort (LOE) The following is an illustrative table of the LOE. Dates may be modified based on availability of consultants and key stakeholders, and amount of time needed for field work. Activity Team Leader Internationa l Consultant National Consultants Review documents and begin drafting evaluation protocol and survey instruments 5 days 5 days 5 days Travel to country 2 days 2 days — In-country briefing with USAID, team planning meetings 3 days 3 days 3 days Fieldwork (including travel days) 15 days 15 days 15 days USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 55 Preliminary data analysis and synthesis; drafting report and presentation materials with additional follow-up meetings 6 days 6 days 6 days Debriefing of Mission staff 0.5 days 0.5 days 0.5 days Stakeholders’ presentation on preliminary findings 0.5 days 0.5 days 0.5 days Report revisions and continued preparation 4 days 4 days 3 days Team departs country 2 days 2 days — Mission sends technical feedback/comments on draft to Team Leader — — — Draft revised by Team Leader and team 5 days 3 days 1 day Total LOE 43 days 41 days 34 days A six-day work week is approved while in-country. XV. Cost Estimate GH Tech will provide a cost estimate for this activity. 56 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 57 ANNEX II. DOCUMENTS CONSULTED 1. Centers for Disease Control and Prevention. DRC Operational Plan 2010. 2. Ditekemena, J. et al., “Male Partner Voluntary Counseling and Testing Associated with the Antenatal Service in Kinshasa DRC: A Randomized Controlled Trial.” International Journal of STD & AIDS, 24(3). 3. Documentation formation sur l’approche Enfant pour Enfant. May 2011. 4. Draft du guide pratique de prise en charge nutritionnelle des PVVIH.June2012. 5. Draft du guide simplifie des soins palliatifs. 6. DRC Ministry of Health. Atlas 2010: Prévention de la transmission mère-enfant du VIH Cartographie des intervenants et interventions. Kinshasa: DRC Ministry of Health, 2011. 7. Echo du PEPFAR—Bulletin d’Information PEPFAR. RDC: May 2013. 8. Echo du PEPFAR (PEPFAR DRC newsletter, Congolese and Americans in partnership) 9. Guide d’intégration des activités des programmes dans les Zones de Sante, Direction Nationale de lutte contre la maladie. 10. Guide de formation sur la vie positive, prévention positive et soins palliatifs du VIH/SIDA. November 2010. 11. Guide de prise en charge de l’infection à VIH chez l’enfant en RDC. 2010. 12. Guide de prise en charge de l’infection à VIH chez l’enfant en RDC (rev.). August 2010. 13. Guide du formateur des formateurs en conseil et dépistage volontaire du VIH/SIDA. September 2004. 14. Guide du formateur en conseil du VIH/SIDA. September 2004. 15. Mampuya, Charly. PEPFAR Guidance for Orphans and Vulnerable Children (OVC) Programming: A Brief Overview. January 2013. 16. Manuel de formation des intervenants auprès des OEV. 17. Manuel de suivi et évaluation des interventions en faveur des Orphelins et autres enfants vulnérables. October 2009. 18. Manuel des participants: atelier de formation des formateurs. January 2011. 19. Manuel des participants sur l’Approche Communauté Championne. January 2011. 20. Manuel pour améliorer les services de conseil et dépistage du VIH, version pour expérimentation sur le terrain. OMS-PEPFAR. 21. Menu des services PEPFAR. 22. Module de formation sur l’approche Enfant pour Enfant. July 2012. 23. Module intégré des prestataires en VIH/SIDA. January 2011. 24. Module sur l`Approche AVEC. 25. Morfaw, Frederic et al. “Male Involvement in Prevention Programs of Mother to Child Transmission of HIV: A Systematic Review to Identify Barriers and Facilitators.” Systematic Reviews, 2. 26. National Program for the Fight Against AIDS (PNLS).Guide pratique du prestataire pour la prise en charge psychosociale des personnes vivant avec le VIH/SIDA et les personnes affectées pat le SIDA. Kinshasa: PNLS, June 2010. 27. National Program for the Fight Against AIDS (PNLS).Normes et directives en conseil et dépistage du VIH. Kinshasa: PNLS, November 2009. 28. National Program for Nutrition of the Ministry of Public Health (PRONANUT). Protocole National de Prise en Charge Nutritionnelle. Kinshasa: PRONANUT, April 2012. 58 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 29. Normes de qualité dans l`organisation d`un CDV (intégré, communautaire, mobile du jour ou de nuit). 30. Partnership Framework Document to Support Implementation of the DRC National HIV and AIDS Response. April 2010. 31. PATH-FINAL-Executed Award. 32. Plan national de développement sanitaire 2011–2015. Kinshasa: DRC Ministry of Health. 33. The President’s Emergency Plan for AIDS Relief (PEPFAR). Comprehensive HIV Prevention for People Who Inject Drugs, Revised Guidance. Washington, DC: PEPFAR, July 2010, at http://www.pepfar.gov/documents/organization/144970.pdf. 34. The President’s Emergency Plan for AIDS Relief (PEPFAR). Country Operational Plan (COP) Guidance. Washington, DC: PEPFAR, August 2, 2011, at http://www.pepfar.gov/documents/organization/169694.pdf. 35. The President’s Emergency Plan for AIDS Relief (PEPFAR). Country Operational Plan Guidance Appendices. Washington, DC: PEPFAR, December 2011, at http://www.pepfar.gov/documents/organization/169695.pdf. 36. The President’s Emergency Plan for AIDS Relief (PEPFAR). Guidance for Orphans and Vulnerable Children Programming. Washington, DC: PEPFAR, July 2012, at http://www.pepfar.gov/reports/guidance/c53568.htm. 37. The President’s Emergency Plan for AIDS Relief (PEPFAR). Guidance for the Prevention of Sexually Transmitted HIV Infections. Washington, DC: PEPFAR, August 2011, at http://www.pepfar.gov/documents/organization/171303.pdf. 38. The President’s Emergency Plan for AIDS Relief (PEPFAR). Next Generation Indicators Reference Guide: Version 1.1. Washington, DC: PEPFAR, August 2009, at http://www.pepfar.gov/documents/organization/81097.pdf. 39. The President’s Emergency Plan for AIDS Relief (PEPFAR). PEPFAR DRC OVC Strategy. Washington, DC: PEPFAR, October 2012. 40. The President’s Emergency Plan for AIDS Relief (PEPFAR). Technical Guidance on Combination HIV Prevention. Washington, DC: PEPFAR, May 2011, at http://www.pepfar.gov/documents/organization/164010.pdf. 41. Protocole National de la prévention de la transmission du VIH de la mère a l`enfant. July 2010. 42. ProVIC Annual Report Year 1. 43. ProVIC Annual Report Year 2. 44. ProVIC Annual Report Year 3. 45. ProVIC Annual Report Year 4. 46. ProVIC. Year 1 Workplan. March 2010. 47. ProVIC PATH Consortium. Aligning with the PEPFAR Strategic Pivot. April 2013. 48. ProVIC PATH Consortium, Annex B - Y4 semi-annual M & E data, May 15, 2013. 49. ProVIC PATH Consortium. Beneficiaries Focus Groups Guide (draft). 50. ProVIC PATH Consortium. FINAL Published RIG Audit. 51. ProVIC PATH Consortium. FY 2010 Annual Report. October 2010. 52. ProVIC PATH Consortium. Original Statement of Work. 53. ProVIC PATH Consortium. Proposal Projet RNOAC Reviewed. August 2012. 54. ProVIC PATH Consortium. Revised Annex F GANNT Chart - for submission. 55. ProVIC PATH Consortium. Revised PMEP. Kinshasa, March 2010. 56. ProVIC PATH Consortium. Revised Year 3 Annual Report. January 2013. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 59 57. ProVIC PATH Consortium. Revised Year 4 Workplan Narrative for submission. 58. ProVIC PATH Consortium. Statement of Work: Submission to COR. March 2013. 59. ProVIC PATH Consortium. Updated PMEP - ProVIC Year 4 & 5 - for submission. 60. ProVIC PATH Consortium.Y2 Annual Report, Annex A: Results Table for USAID. 61. ProVIC PATH Consortium. Y2 Annual Report for USAID, submission. November. 62. ProVIC PATH Consortium. Year 2 Workplan FINAL. October 2010. 63. ProVIC PATH Consortium.Y4 Semi Annual Report. May 2013. 64. Rapport synthèse RAAAPRDC final submission. October 2011. 65. Target Setting, PEPFAR 2013. 66. Tech_Vol_Integrated_HIV-AIDS_2009. 67. To fight HIV/AIDS, English May 2013. 68. U.S. Agency for International Development (USAID). Evaluation: Learning from Experience: USAID Evaluation Policy. Washington, DC: USAID, January 2011, at http://toolkits.urbanreproductivehealth.org/toolkits/measuring-success/evaluation-learning￾experience-usaid-evaluation-policy. 69. U.S. Agency for International Development (USAID). Global Health Technical Assistance Project: GH Tech Bridge III, Contract No. AID-OAA-C-13-00032. Washington, DC: USAID. 70. U.S. Agency for International Development (USAID).How to Prepare Evaluation Reports. 71. U.S. Agency for International Development (USAID). How-to note, preparing evaluation report, Number 1, Version 1.0. November 2012. 72. USAID/DRC. Katanga Trip Supervision. August2012. 73. USAID-evaluation-report-template. 74. World Health Organization (WHO). Guidance on Couples HIV Testing and Counselling, Including Antiretroviral Therapy for Treatment and Prevention in Serodiscordant Couples. Geneva: WHO, April 2012, at http://whqlibdoc.who.int/publications/2012/9789241501972_eng.pdf. 60 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 61 ANNEX III. LIST OF PERSONS CONTACTED N° NAME FUNCTION ORGANIZATION LOCATION 1 Joshua Karnes Deputy & Acting HIV and Health Officer USAID Kinshasa 2 Lillian Benjamin Health Team USAID Kinshasa 3 Kai Beard Program Officer USAID Kinshasa 4 Olivier Mumbere Monitoring and Evaluation USAID Kinshasa 5 Charly Mampuya HIV Program Management Specialist USAID Kinshasa 6 François Xavier N’susi HIV Program Management USAID Kinshasa 7 Emmanuella Delva Gender Fellow USAID Kinshasa 8 Rachel Boketa Gender Specialist USAID Kinshasa 9 Jyoti Schlesinger Acting Team Leader, Office of Health USAID Kinshasa 10 Jason Wolfe Senior Economic Strengthening Advisor USAID Kinshasa 11 Lai Beac Program Officer USAID Kinshasa 12 Monica Carlson Program Officer USAID Kinshasa 13 Thibaut Mukaba Family Planning/Health Officer USAD Kinshasa 14 John Bernow Health Officer USAID Kinshasa 15 Lolem Ngong Coordinator PEPFAR Kinshasa 16 John Ditekemena Country Director EGPAF Kinshasa 17 Dr. Marcel Yotebieng Scientific Advisor UNC Kinshasa 18 Dr. Mukolizimba Jean Luc Provincial Representative ASF/PSI Kinshasa 19 Dr. Papy Anau Swala Director, HIV/TB ASF/PSI Kinshasa 20 Dr. Didier Adjoua Deputy Country Representative ASF/PSI Kinshasa 21 Trad Hatton Director, ProVIC ProVIC Kinshasa 22 Rianney Gay SGVB Specialist ProVIC Kinshasa 23 Salva Mulongo Community Mobilization Specialist ProVIC Kinshasa 24 Mitterand Katabuka Senior Pediatric Specialist ProVIC Kinshasa 25 Emmanuel Mpiana OVC Specialist ProVIC Kinshasa 26 Alioune Baedara SOW Community-Based Specialist ProVIC Kinshasa 27 Georges Ntumba Deputy Director, ProVIC ProVIC Kinshasa 28 Herbie Senior Grants Manager ProVIC Kinshasa 29 Denise Monitoring and Evaluation ProVIC Kinshasa 30 Ghislaine Babungu Community Mobilization Assistant ProVIC Kinshasa 31 Gisèle Semde Abla Project Director, FHI 360 FHI360 Kinshasa 32 Dr. Astrid Mulenda Senior Technical Officer FHI360 Kinshasa 62 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 33 Eddy Kuvituanga Senior Program Officer FHI360 Kinshasa 34 Kelly Yotebieng Head of Programming CRS Kinshasa 35 Nicole Shabani Mala Health Coordinator CRS Kinshasa 36 Dr. Bossiky Deputy Coordinator PNMLS Kinshasa 37 Dr. J.P. Kabuawu Former Deputy Coordinator PNLS Kinshasa 38 Jean Lukela Coordinator CC RNOAC Kinshasa 39 Dr. Angèle Assily Medical Coordinator PNLS Lubumbashi 40 Dr. François Mpanga Monitoring and Evaluation Officer PNLS Lubumbashi 41 Faustin Bemba Behavior Change Communication Officer PNLS Lubumbashi 42 Kapongo Fréderic Program Assistant World Production Lubumbashi 43 Adelard Mutombo Coordinator World Production Lubumbashi 44 Sylvain Tshama Program Manager World Production Lubumbashi 45 Raphael Tshimata Deputy Coordinator World Production Lubumbashi 46 Daddy Ndal-yav Technical Staff World Production Lubumbashi 47 Solange Kamong Technical Staff BAK-CONGO Kasumbalesa 48 Asha Elendra Technical Staff BAK-CONGO Kasumbalesa 49 Patrik Kasongo Coordinator BAK-CONGO Kasumbalesa 50 Dr. Franck Mulumbwa Monitoring and Evaluation BAK-CONGO Kasumbalesa 51 Dr. Bokar Medical Director HGR Kasumbalesa 52 Tecla Mwana Mulenda Program Manager OLASEC Lubumbashi 53 Kat-Francine Social Worker OLASEC Lubumbashi 54 Daudet Muyumba Monitoring and Evaluation Officer OLASEC Lubumbashi 55 Ernest Mwamba Coordinator OLASEC Lubumbashi 56 Nathalie OVC Program Officer OLASEC Lubumbashi 57 Dr. J.C. Kiluba Coordinator ProVIC Lubumbashi 58 Teddy Kalema Deputy Senior Grants Manager ProVIC Lubumbashi 59 Babeth Katumbo Care and Support Specialist ProVIC Lubumbashi 60 Lidya Kabamba Prevention Specialist ProVIC Lubumbashi 61 Henri Alolia Monitoring and Evaluation ProVIC Lubumbashi 62 Triomphe Saidi Grants Assistant ProVIC Lubumbashi 63 Dr. Angèle Assily Provincial Medical Coordinator PNLS Lubumbashi 64 Faustin Bemba Behavior Change Communication PNLS Lubumbashi 65 Dr. François Mpanga Monitoring and Evaluation PNLS Lubumbashi 66 Carole Mwela Kabajula Head Office (Director of Hospital) Kenya Hospital Lubumbashi 67 Agnès Muzama PMTCT Manager Kenya Hospital Lubumbashi 68 Jean Marie Ilunga IO and ARV Nurse Lubumbashi 69 Bokar Medical Director Doctor Lubumbashi 70 Kalunga Mwange TB Nurse Lubumbashi 71 Kabuiz Beatrice PMTCT Nurse Lubumbashi 72 Kasongo Mukabi Other Nurse Lubumbashi 73 Maman Joce Data Manager Nurse Lubumbashi 74 Kongolo Silvain PMTCT Doctor Lubumbashi USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 63 75 Emmanuel Mpanzu C&L ProVIC Matadi 76 Dr. Didier Kamere Regional Program Coordinator ProVIC Matadi 77 Enoch Nzau Regional Monitoring and Evaluation ProVIC Matadi 78 Bobwa Rodrigue Prevention Officer ProVIC Matadi 79 Isidore Mabiala Executive Secretary PNMLS Matadi 80 Neville Lumfuakiadi Family Planning & Laboratory PNLS Matadi 81 Vicky Mabiala Provincial Medical Director PNLS Matadi 82 Aimé Dinzau Provincial Medical Director PNSR Matadi 83 Eddy Sasi Panzu Coordinator JADISIDA Matadi 84 Hugor Mavembo Technical Staff JADISIDA Matadi 85 Jean Claude Phanzu Technical Staff JADISIDA Matadi 86 Claude Luyindula Technical Staff JADISIDA Matadi 87 Nsibi Ndosimao Medical Director of the Health Zone Doctor Matadi 88 Remy Ndungu Lanzosth Community Animator Nurse Matadi 89 Luzolo Ndongala Health Zone Administrator Administrative Manager Matadi 90 Mabulu Vila HIV Counseling and Testing Advisor Nurse Matadi 91 Nlandu Masanga PMTCT Advisor (Counseling) Nurse Matadi 92 Nzomono Kingimbi Chief of Health Zone Doctor Matadi 93 Bakangana Director of Kinzau Health Center Doctor Matadi 94 Jean Masumu Head of Protection Office Social Affairs Matadi 95 Emandjala Albert Program Manager CEMAKI Kinzau Mvuete 96 Manaka Joachim Technical Staff CEMAKI Kinzau Mvuete 97 Yamba Muaka Social Assistant CEMAKI Kinzau Mvuete 98 Muanda Albert Social Assistant CEMAKI Kinzau Mvuete 99 Françoise Masota PLWHA Clinic Nurse HGR Boma 100 J.C. Mvuololo Principal FOSIBAC Boma 101 Malou Nkalulu Laboratory Technician HGR/BOMA Boma 102 Ntoto Michel PID Health District Boma 103 Flavien Makiadi Managing Administrator Health Zone Boma 64 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION ANNEX IV. METHODOLOGY In a review of the key questions, the team identified key sources and areas of emphasis as seen in the chart below: MATRIX A ProVIC Performance Evaluation: Interview Focus Areas and Data Source Emphases Focus, Key Questions USAID￾Activity Manager Josh, Olivier, and others ProVIC COP Trad ProVIC Technical Advisors GDRC National Level (PNLS, PNMLS) GDRC Health Offices, Clinics Provincial Level ProVIC NGO Partners Champion Communities (CPCC) Other Donors and INGOs (CRS, FHI, GFATM) Ultimate Beneficiaries (OVC, PLWHA, CSWs, MSM, PMTCT, Truckers, Fishermen) 1. How has community involvement increased as a result of the intervention (primarily the Champion Community), and what difference has this made to the health of the community? X X X X X X X X X 2. What are the strengths and X X X X X X X USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 65 weaknesses of the integrated approach? 3. How has the quality of HIV prevention, care, and treatment services improved as a result of the intervention? X X X X X X 4. To what extent has ProVIC improved access to services for MARPs in the DRC? X X X X X 5. What key aspects of the ProVIC model should be continued and/or discontinued in future HIV/AIDS programming? X X X X X X X 6. What are the factors that contributed to achieving or not achieving program results? X X X X X X X X 66 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 7. What is the efficacy of ProVIC community mobilization approaches which emphasize services for OVC currently being implemented? X X X X X X X X 8. Is there a cost￾saving using the Champion Community? X X USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 67 MATRIX B In addition the team prepared a more detailed matrix identifying approach, method, source, analysis. This matrix below acted as a guideline in both data collection and analysis. Getting to Answers: Program or Activity: USAID/DRC ProVIC Evaluation Team Members: Ruth Kornfield, Methods for Data Collection Evaluation Questions Type of Answer/Evidence Needed (description; comparison; cause and effect) and notes on special requirements or sources of data Method Data Source Sampling or Selection (if applicable) Data Analysis Methods (e.g. frequency distributions, trend analysis, content analysis) 1. How has community involvement increased as a result of the intervention (primarily the Champion Community), and what difference has this made to the health of the community? Description: Secondary data, including project contract, workplans, semi-annual and annual reports, relevant technical reports, assessments or studies; CC workplans, primary data—people's input Collect documents; conduct meetings; interviews, discussions, and FGDs ProVIC COP and Team and technical advisors; USAID activity manager; PNLS, PMNLS, NGO partners, CPCC leaders Purposive, for documents and persons with the richest source of relevant information, random selection of site visits within target areas Content analysis 68 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 2. What are the strengths and weaknesses of the integrated approach? Description: Secondary data, including USAID guidance documents, reports, relevant technical reports, assessments or studies Collect documents; conduct meetings; interviews, discussions and FGDs PNLS, PMNLS, other donors, USAID and PEPFAR Guidance and technical advisors Purposive, for documents and persons with the richest source of relevant information Content analysis 3. How has the quality of HIV prevention, care, and treatment services improved as a result of the intervention? Description, before and after (in absence of baseline), NGO partner reports, semi￾annual and annual reports, CC annual plans, follow-up Collect documents; conduct meetings; interviews, direct observation, discussions and FGDs MOH and PNLS, PMNLS, ProVIC COP and technical staff HQ and regional, NGO partner contracts and reports, SAP and AP, Purposive, for documents and persons with the richest source of relevant information Content analysis 4. To what extent has ProVIC improved access to services for MARPs in the DRC? Description: opinion surveys, secondary data, CPCC workplans, NGO reports; primary data—people's input Collect documents; conduct meetings; interviews and discussions and FGDs USAID activity manager, ProVIC partner NGOs, CPCC, other MARP programs, PNLS, PMNLS Purposive, for documents and persons with the richest source of relevant information Content analysis USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 69 5. What key aspects of the ProVIC model should be continued and/or discontinued in future HIV/AIDS programming? Description: Secondary data, including semi-annual and annual reports, workplans, relevant technical reports, assessments or studies; primary data—people's input KII, FGDs, document study, discussions, opinion surveys, questionnaires USAID, ProVIC, PEPFAR comparables, DEC and other evaluation sources, other stakeholders Purposive, for documents and persons with the richest source of relevant information Content analysis 6. What are the factors that contributed to achieving or not achieving program results? Description: Secondary data, including workplans, semi-annual and annual reports, relevant technical reports, assessments or studies; primary data— people's input Documents, reports, oral history, time line, KII, discussions COP and technical leads, NGO partners, CPCC, other stakeholders Purposive, for documents and persons with the richest source of relevant information Content analysis 7. What is the efficacy of ProVIC community mobilization approaches which emphasize services for OVC currently being implemented? Description: Secondary data, including semi-annual and annual reports, focused on OVC programs, relevant technical reports, assessments or studies; primary data—people's input Collect documents; conduct meetings; interviews, discussions and FGDs NGO partners with OVC activities, other programs (comparable), USAID, ProVIC, provincial authorities Purposive, for documents and persons with the richest source of relevant information Content analysis 8. Is there a cost-saving using the Champion Community approach? Description: secondary data: contracts and workplans, budgets Collect documents USAID Program and Health Offices, ProVIC, PNLS, PMNLS Purposive, for documents and persons with the richest source of relevant information Content analysis 70 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION The table below is a summary of the sites visited. PROVINCE SITE NGO PARTNER Kinshasa Champion Community meeting place RNOAC MSM hot spot/mobile testing center PSSP Bas-Congo Matadi JADISIDA SekeBanza CEMAKI Boma JADISIDA BDOM Katanga Lubumbashi AMO-Congo  Commune Kenya OLASEC  Commune Kamalundo World Production Kasumbalesa Bak-Congo Type of person or group from whom data were collected Number ProVIC offices, HQ/Kinshasa, Provincial: Katanga, Bas-Congo 3 of 6 NGO partners 8 of 14 Health Zone Offices: Kinshasa, Katanga, Bas-Congo 5 General Reference Hospitals (two ProVIC-supported) 3 Provincial health centers 2 Provincial Medical Directors 2 PNLS, National and Provincial Offices 3 PNLMS, National and Provincial 3 PLWHA self-support groups in six locations 9 OVC Child-to-Child groups in Kinshasa, Katanga, and Bas-Congo 3 Volunteer outreach workers group meetings 4 Champion Community Steering Committees 9 MSM networks-group meetings 2 MSM and sex worker “hot spot” with mobile HIV testing unit 1 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 71 Sex worker groups 2 Large trucker park at border 1 Training for MOH health systems strengthening Bas-Congo 1 day long The following tables provide a summary of data sources. Table 1: Focus Group Discussions by Category of Group and Geographic Area Focus Group Discussions Kinshasa Katanga Bas￾Congo Total Community Outreach Workers 1 1 1 3 Champion Community Steering Committees 1* 2 2 5 PLWHA Self-Help Groups 1** 3 2 6 OVC 1*** 1*** 1 3 MSM 1 1 1 3 Sex Workers 2 2 Pregnant Women 1 1 2 Total 5 11 8 24 *Included members from three Steering Committees. **Included members from three Self-Help Groups. ***Included OVC from two Child-to-Child Groups. Table 2: Observations by Category of Group and Geographic Area Kinshasa Katanga Bas-Congo Total MSM Hair Salon 1 1 MSM & Sex Worker “Hot Spot” Mobile Clinic 1 1 Health Facilities 3 3 6 Outreach Workers at Truck Stop 1 1 Child-to-Child Group Activities 2 1 3 Health Zone Training 1* 1 1 6 6 13 72 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION *Observations were made during one day of a week-long training for Health Zone personnel supported by ProVIC in collaboration with PNLS as part of the ProVIC health systems strengthening activities. Table 3: Interviews with Health Care Personnel by Province Health Care Personnel Interviewed by Province Katang a Bas￾Congo Tot al Medical Director of Health Zone 1 2 3 Medical Director of General Reference Hospital 2 1 3 Data Manager 1 2 3 Nurse Supervisors 1 1 2 Nurses in Charge of HIV Voluntary Counseling and Testing 1 1 2 Nurses in Charge of Prevention of Mother-to-Child Transmission 2 2 4 Staff in Charge of Community Education (Animateur communautaire) 1 2 2 Total 9 11 19 Table 4: Interviews with Key Informants and ProVIC Personnel* Kinshasa Katanga Bas￾Congo ProVIC Staff National and Provincial level x x x ProVIC NGO Sub-grantees x x x PNLS x x x PNMLS x x x Ministry of Social Affairs x x x USAID Health & M&E staff x Family Health International 360 x Catholic Relief Services x Population Services International (PSI) x University of North Carolina x EGPAF Regional Representative x *See Annex III for list of names and titles of persons interviewed. Evaluation Team Members GH Tech Bridge III, through dTS, Inc. provided an expert team of U.S. and Congolese evaluators with extensive experience in the DRC and other African and PEPFAR countries and who have carried out research and evaluations of community-based service provision and HIV/AIDS prevention, care, and treatment. The team was composed of the Team Leader, Ruth Kornfield, Ph.D., social anthropologist and HIV program evaluation specialist; Mbadu Fidèle, demographer; Hubert Ibi, MD, MPH; and Ann von Briesen Lewis, a development evaluation specialist. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 73 ANNEX V. DATA COLLECTION TOOLS Group Discussion Guide Champion Community Steering Committee Introduction: We are part of an independent evaluation team contracted by USAID to conduct a performance evaluation of ProVIC with an emphasis on the Champion Community model of community mobilization to improve quality and access to integrated HIV/AIDS services. The results of this evaluation will be used to guide future programming. Thank you for coming. Your open and candid responses and answers are very important to us. (The interviewer will NOT ask the broad questions, but they will be a guide to make sure that the probing questions are obtaining data and informing the broad questions.) A. How has community involvement increased as a result of the intervention (primarily the Champion Community), and what difference has this made in the health of the target populations? With community involvement have any health seeking behaviors and attitudes shifted? 1. Introduction of each member focusing on their position in the community (e.g. Chef de Zone, priest or pastor, teacher, etc. (look at gender composition also) 2. Explain what a Champion Community is. How do they work? Are you a “Champion Community?” What does that mean? If so, how did you get to be one? 3. How was this CPCC formed? a. Who initiated the establishment of the CPCC? b. How were each of you chosen? c. What are each of your interests in being a member of CPCC? 4. What is the purpose of the CPCC? 5. Describe the structure of the CPCC. 6. Before the CPCC, was there another committee that did the same things as you do? (Explain. If so, what is your relation to the other committee now? Does it still exist?) 7. How do you function? Meetings? Action plan? What is the division of tasks? Who decided what activities would be on your action plan (if they have one)? 8. What do you do? (Role in the community) Give examples. How do you do it? What difficulties have you had? What has worked well for you? 9. What is your relation with the Health Zone, with the ProVIC office, with the NGO (linked to this CPCC)? 10. What is your relation with health animators, with relais communautaire, peer educators? What do they do? Are you satisfied with what they do? Why? 11. Have your activities influenced the community in any way? If so, how? If not, why not? (Probe focusing on health-seeking behavior, HIV behavior, OVC care and support, PMTCT, pre-natal care, knowledge and behavior change.) B. How has the quality of HIV prevention, care, and treatment services improved as a result of the intervention? (This could be done by having them map the services first, then having them tell us who goes to each service, and finally asking the questions about each of the services.) 1. Before the CPCC was established, were there HIV prevention activities in your community? 74 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION a. If so, what were they? Were they useful? How? Did they have any effect on members of the community? If so, how? If not, why not? b. Since this CPCC has been established have there been HIV prevention activities in your community? If so, what are these activities? Are they different from the activities before the CPCC was formed? How? c. Did or does the CPCC play a role in any of these prevention activities? If so, what? If not, why not? d. Have these activities benefited members of your community in any way? If yes, how? If no, why not? Which community members have benefited (children, adults, professional sex workers, etc.)? How have they benefited? Describe this for each category of persons. e. Do you think people have changed their behavior in any way as a result of these prevention activities? If yes, how? If no, why not? f. Do you have any suggestions as to ways to improve the prevention activities in your community? 2. Is there any type of care and support service for PLWHA or OVC in your community? If so, what are they? If not, why not? (Separate questions for PLWHA and for OVC, probing a lot for OVC.) a. If so, did these services exist before the establishment of the CPCC? Which ones? b. Have there been any new ones added since the establishment of the CPCC? If so, which ones? c. Does the CPCC have anything to do with these services? If so, explain. d. Explain how the PLWHA and/or the OVC have access to these services. e. Are these services useful? If not, why not? If yes, how? f. Are these services better, worse, or the same as what existed before? How? g. Has CPCC contributed in any way to these services? How? If not, is there a way that the CPCC could contribute? Explain. 3. What about the health services at the clinic? Services in general, services for OVC, services for PLWHA, services for pregnant women, services for HIV counseling and testing? (Separate questions for each category of service) a. What services exist now? b. What services existed before the CPCC? c. Are you satisfied with the health services? If yes, explain. If no, why not? d. Has the CPCC done anything about these services (e.g. encouraging people to go for PMTCT, HIV counseling and testing, etc.)? C. To what extent has ProVIC improved access to services for key populations in the DRC? 1. In your community, do you know who is at highest risk of contracting HIV? If no, then do not continue questioning on this subject. If yes: 2. Who are they? 3. Are there any special activities for them concerning HIV prevention, care, and treatment? If so, what are they? 4. Did these activities exist before the CPCC? Which ones existed and which ones did not? 5. Did the CPCC contribute in any way to these activities? If so, how? 6. Do you think these groups of people (name the key population) benefit from these activities? If not, why not? If so, how? USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 75 7. Do you know if the (name group) have problems getting to these services? If so, explain. 8. What could be done to ensure that they are able to access these services? 9. Is there anything the CPCC could do to help these people access the HIV service? Explain. D. What key aspects of the ProVIC model should be continued and/or discontinued in future HIV/AIDS programming? 1. What do you think is the best way to get the community involved in HIV prevention; care and support of OVC and PLWHA; and ensuring that pregnant women go for pre￾natal services, receive HIV counseling and testing, and get treatment if necessary? 2. Do you think the establishment of the CPCC is a good way to do this? 3. What are the advantages of having the CPCC and Champion Communities? 4. What are the disadvantages of having the CPCC and Champion Communities? 5. What are the difficulties that the CPCC has in trying to carry out its activities? E. What are the factors that contributed to achieving or not achieving the program’s results? 1. What parts of your workplan/planned activities have you been able to achieve? What has made this possible? 2. What parts of your workplan/planned activities have you not been able to do? Why? 3. What do you need in order to be able to do those activities that you have not done? 4. What problems have you encountered in trying to carry out your workplan? How have you resolved them? 5. For those problems not resolved, what type of assistance do you need to resolve them? F. What is the efficacy of ProVIC’s community mobilization approaches which emphasize services for Orphans and Vulnerable Children (OVC) currently being implemented? 1. Has the CPCC organized or promoted any activities to help OVC? If so, what have you done? Explain how you have carried out these activities. 2. What difficulties have you had in carrying out these activities? How were the difficulties addressed? 3. Did other members of the community participate in helping OVCs? If so, how? 4. What ideas do you have of ways that the community members could better assist OVC? 5. How could you realize these ideas? 6. What resources would you need? How could community members provide these resources? Would they be willing to do so? If not, why not? 7. What types of assistance did OVC receive before the CPCC was established? 8. How were they implemented? If it was different from how OVC are assisted now, which way is better—what was happening before or what has been happening since the CPCC has been established? Why? G. Is there a cost-savings using the Champion Community service delivery model? 1. Did the CPCC receive any money for their activities? 2. If so, how much? Why did the CPCC receive the money? What did it use the money for? 3. Were there any problems about this money? Explain. How were the problems resolved? Thank you very much. Your participation has been very helpful. 76 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION I. Discussion Guide for Community Outreach Workers and Peer Educators (Les groupes de discussion seront séparés : un pour les animateurs communautaires, un pour les relais communautaires et un pour les pairs éducateurs.) 1. Qu’est-ce que vous faites comme (animateur communautaire, relais communautaire, pair éducateur) ? a. Quelles sont vos activités ? Les décrivez. b. Quand a été la dernière fois que vous avez fait une de ces activités ? Décrivez-le. 2. Quelle est la différence entre un relais communautaire, un animateur de santé et un pair éducateur ? Leurs activités sont-elles différentes ? Si oui, quelles sont les différences ? 3. Quand avez-vous commencé ce travail ? 4. Qui vous a recruté pour ce travail ? Comment? 5. Comment s’est organisé votre travail de (animateur communautaire, relais communautaire, pair éducateur) ? a. Avez-vous un plan d’action que vous suivez ? Si oui, décrivez-le. b. Qui vous supervise? Comment? 6. Selon vous, quels sont les objectifs de vos activités ? 7. En rapport avec vos activités, quels sont les résultats qui sont attendus de vous ? 8. Pensez-vous que vous avez atteint ces résultats ? Expliquez. 9. Quelles sont les difficultés que vous rencontrez dans l’atteinte des résultats ? 10. Qu’est-ce qui facilite votre travail de (relais communautaire, animateur communautaire, pair éducateur) ? 11. Vous travaillez dans la communauté depuis un certain temps dans le domaine du VIH. Est-ce que vous pensez que le niveau des connaissances, des attitudes et des pratiques de la population vis-à-vis du VIH/SIDA est mieux maintenant qu’avant ? Si oui, qu’est-ce qui vous permet de l’affirmer, et si non, comment l’expliquez-vous ? 12. En tant que (relais communautaires, animateurs communautaires, pairs éducateurs), vous devez mener des activités de sensibilisation, de mobilisation communautaire. Y-a-t￾ils, d’après vous, des innovations qu’a apportées PROVIC dans le travail que vous faites maintenant, étant donné que certains d’entre vous le faisaient avant l’arrivée de PROVIC ? 13. Dans l’exécution du projet PROVIC, qu’est-ce que vous pouvez considérer comme des points forts à garder et comme des points faibles à améliorer ? 14. En matière de VIH, qu’est-ce que la population souhaite le plus ? et pensez-vous que PROVIC apporte la solution à ce souhait de la population ? Expliquez davantage. 15. Pensez-vous que la population a réellement accès aux services de VIH depuis l’arrivée du projet PROVIC ? Si oui ou non, expliquez. 16. Faites-vous des activités spécifiques pour les OEV ? Si oui, quelles sont ces activités ? 17. Savez-vous ce qui est une « communauté championne ? » a. Si oui, expliquez. b. Faites-vous partie d’une « communauté championne ? » c. Que font les membres d’une « communauté championne ? » d. Comme participant, qu’est-ce que vous faites ? e. Croyez-vous que votre « communauté championne » est utile ? Si non, pourquoi pas ? Si oui, comment ? USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 77 18. Depuis qu’il y a une « communauté championne », est-ce qu’il y a les changements dans votre communauté ou chez vous ? Si oui, expliquez. 19. Si on vous demande d’aller implanter cette approche « communauté championne » ailleurs, dans une autre province, une autre région, qu’est-ce que vous allez prendre de bon à amener et qu’est-ce que vous allez laisser de mauvais (les forces et les faiblesses) ? 20. Avec votre expérience de (relais communautaire, animateur communautaire, pair éducateur), quelle est la stratégie la meilleure qui permettrait une bonne utilisation des services de VIH dans la population (prévention et prise en charge) ? 21. Quelles sont les contraintes /difficultés que vous trouvez dans l’exécution des activités de PROVIC ? 22. Quelles recommandations pouvez-vous faire à PROVIC pour que la population puisse bénéficier des services de VIH de qualité ? Merci beaucoup pour votre collaboration II. Interview Guide for Government Authorities : PNLS, PNMLS, MOH, MINAS Bonjour, Je suis Monsieur………………, Madame……………………….. Nous faisons une évaluation de PROVIC, Projet intégré de VIH qui est financé par l’USAID, et nous sommes en train de rencontrer les différents partenaires, les bénéficiaires des services la communauté, afin de récolter leurs avis sur le projet. Le PNLS, le PNMLS étant des structures officielles du Gouvernement dans le domaine du VIH, nous avons estimé important de les rencontrer et d’avoir leurs avis sur ce Projet. C’est la raison pour laquelle nous avons sollicité cet entretien avec vous, et merci d’avance pour avoir accepté de nous recevoir. (i) Le Gouvernement de la RDC a de nombreux défis dans le domaine du VIH. De quel type de partenaire/partenariat avez-vous besoin pour atteindre les objectifs du pays en cette matière ? (ii) Connaissant PROVIC – ses activités et son approche – pensez-vous qu’il participe à l’atteinte des objectifs de santé que poursuit le pays ? Expliquez-nous comment. (iii) Dans le cadre de l’aide au développement (Déclaration de Paris), il est souhaité que les activités des partenaires tiennent compte des priorités du pays, que leurs activités s’intègrent dans le plan national de développement du pays. Pensez-vous que c’est le cas pour PROVIC? (iv) Comme vous le savez, PROVIC utilise l’approche communautaire pour offrir les différents services de VIH à la population. Cette approche consiste à faire prendre conscience à la population des problèmes du VIH, à les mobiliser afin qu’ils puissent se prendre en charge. Ces actions s’organisent dans un rayon de 40.000 à 65.000 habitants et il y a des relais communautaires qui sensibilisent, qui réfèrent la population dans les FOSA. Dans ce rayon, il organise aussi beaucoup d’autres activités de lutte contre le VIH : la PTME, le dépistage du VIH, etc. Nous voulons que vous puissiez nous donner votre avis sur cette approche : ses avantages, ses désavantages. 78 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION (v) Il a souvent été entendu que cette approche du PROVIC, dont on vient de parler toute à l’heure, est une innovation, une spécificité dans la prestation des services dans la communauté. De par votre expérience dans ce domaine du SIDA, est-ce que c’est aussi votre avis ? Si réellement cette approche est spécifique, quelle est sa plus￾value comparée à d’autres approches utilisées dans la lutte contre le VIH dans un passé récent ? (vi) Connaissant cette approche de « communauté championne », pouvez-vous la recommander à travers d’autres régions, d’autres pays comme approche de prestation des services dans la communauté ? (Si oui ou non, demander des explications) (vii) Pensez-vous que dans l’offre des services de VIH, PROVIC suit les normes et directives édictées par le Ministère de la santé/ Gouvernement en cette matière ? (viii) Nous voulons savoir si PROVIC appuie le système de santé (renforcement du système de santé) et comment cela se traduit-il ? (ix) Comment dans l’ensemble appréciez-vous la collaboration de votre organisation (PNLS, PNMLS) avec PROVIC ? (x) Quelles recommandations pouvez-vous faire à PROVIC afin que ces activités contribuent mieux à l’amélioration de la santé des populations Congolaises ? (xi) Si vous avez un autre commentaire à faire vis-à-vis de tout ce dont on vient d’échanger, nous sommes intéressés à l’écouter parce que nous savons qu’il va nous apprendre encore davantage. Merci beaucoup pour tout ce temps nous accordé et surtout pour les informations reçues. III. Draft Interview Guide Other Donors/INGOS (The interviewer will NOT ask the broad questions, but they will be a guide to make sure that the probing questions are obtaining data informing the broad questions.) A. How has community involvement increased as a result of the intervention (primarily the Champion Community), and what difference has this made in the health of the target populations? (For CRS, UNC, FHI, PSI, etc., with community programs? Appropriate modifications can be made for the donors as they will have less specific information than the INGOs.) 1. Describe your approach to community involvement with HIV prevention, HIV counseling and testing, PLWHA and OVC care and support, and HIV treatment. 2. Are you familiar with the Champion Community approach of ProVIC? If, so how does your approach differ from the Champion Community approach? 3. What are the advantages and disadvantages of each approach? 4. Which strategy (perhaps even suggest another one) do you think would be most effective in involving the community in OVC and PLWHA care and support, treatment, and increased access to PMTCT? Why? USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 79 5. To what degree do you think the Champion Community approach to community involvement may have resulted in changing HIV health-seeking behaviors and attitudes? Why? 6. Would it be worthwhile continuing the Champion Community approach? Or would you suggest a different community mobilization model or ways of improving the effectiveness of the Champion Community model? B. What are the strengths and weaknesses of the integrated approach? 1. What are the strengths and weaknesses of an approach that channels funding for the entire PLWHA and OVC continuum of care, linking community to clinic-based care to one contractor in the form of one project, rather than separating the different components of continuum of care as separate projects? 2. What would be the best way to link clinical services with community prevention activities? 3. What importance do you think community involvement has in the effectiveness of OVC and PLWHA care, support, and treatment interventions and PMTCT? Why? C. How has the quality of HIV prevention, care, and treatment services improved as a result of the intervention? 1.What knowledge do you have concerning the ProVIC project? 2.Do you have any idea if communities are demanding better services at the facility level as a result of the project? Explain. 3. Are these facilities better off now than they were at the beginning of the ProVIC intervention? If no, why not? If yes, what are the improvements? Do you have ideas as to why? 4. What do you think about the use of fixed-obligation grants and collaborative awards with private and government hospitals respectively to improve clinic-based services? D. To what extent has ProVIC improved access to services for key populations in the DRC? 1. From your knowledge of ProVIC, would you say that the project has improved access to services for key populations in the DRC? If not, why not? If so, why? 2. Has ProVIC tackled the problem of referrals to services and follow-up of key populations? 3. Do you have any recommendations that can be made to ensure that key populations are not only targeted for prevention but also have access to services and are not lost to follow-up upon referral? Explain. 4. What would you say are the strengths and weaknesses of ProVIC in addressing both prevention and referrals for services of key populations in their target communities? 5. What strategies to you think would be most effective both to improve services and increase access with referrals and follow-up for key populations? E. What key aspects of the ProVIC model should be continued and/or discontinued in future HIV/AIDS programming? 1. The ProVIC model has one project including both community-based and clinic-based interventions. The major question is whether funding should be continued to integrate as one project both types of interventions, or should community-based interventions be 80 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION funded as separate project from clinic-based projects, as had been being done previous to ProVIC? 2. What do you think are the advantages and disadvantages to both strategies: (a) integration of community-based and clinic-based support as one funded project; (b) separation of community-based and clinic-based support? 3. What would you suggest as the most effective strategy to link community-based support to clinic-based support? 4. What would you suggest as the most effective strategy to link prevention interventions to effective health-seeking behavior, increased access to clinical services, and improvement of services? F. What is the efficacy of ProVIC’s community mobilization approaches which emphasize services for Orphans and Vulnerable Children (OVC) currently being implemented? 1. What is your approach to community mobilization for increasing access to services for OVC? 2. How does it compare to the ProVIC Champion Community Approach? 3. What are the advantages and disadvantages of your approach and that of ProVIC? 4. What improved approach or strategy would you suggest? I. Beneficiaries Focus Groups Guide EXAMPLE/SHELL ONLY- to be adapted and supplemented with other techniques May be used with CSW, PLWHA, Self-help Groups, and OVC guardians, MSM (truckers/fishermen, etc.) Introduction: We are part an independent evaluation team contracted by USAID to conduct a performance evaluation of ProVIC with an emphasis on the Champion Community model of community mobilization to improve quality and access to integrated HIV/AIDS services. The results of this evaluation will be used to guide future programming. Thank you for coming. Your open and candid responses and answers are very important to us. Rules: 1. Please participate as much as you feel comfortable, and please keep the identities and opinions of others to yourself. We are confidential and anonymous in this room—we will not attach any names to any information. 2. Courtesy: Please allow each person who wishes to speak, to speak once before you speak a second time. 3. You may ask for clarification of a question at any time. 4. Please let us know if there are questions we should have asked or other important information we should know. In general, we want to learn from the beneficiaries about access to services, quality of services, unmet needs, coverage, role of ProVIC, costs/benefits of integrated services, if possible before and after ProVIC, other models. 1. How did you learn of this meeting? (Who asked you to come?) 2. Is this a regular group of people who know each other? 3. When you need a service or help, where do you go? 4. How do you know where to go to get those services? 5. What services are provided? Please be very specific. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 81 6. Are they regularly available, or do you have to wait? 7. Are they the services you need, or are there other needs? 8. Do you feel confident that you are getting what you need? 9. What would happen if ProVIC were not there (i.e. before 2009)? 10. Can you name any ways in which ProVIC has helped improve the health of your community? 11. Do you have to go to many places to get what you need, or are they all together? 12. What would you tell a friend who needs help about your experience? 13. What have we forgotten to ask? IV. Interview Guide for ProVIC Sub-Grantee NGO Introduction: We are part an independent evaluation team contracted by USAID to conduct a performance evaluation of ProVIC with an emphasis on the Champion Community model of community mobilization to improve quality and access to integrated HIV/AIDS services. The results of this evaluation will be used to guide future programming. 1. Could you please give us a quick overview of your (your organization’s) involvement with ProVIC? How many CCs? 2. How long have you (your organization) been involved and in what ways? Which services and populations do you work with? 3. How did you learn about ProVIC, and what was the process of applying for participation? 4. Do you work with other donors and other projects? 5. Can you tell us about your relationship with the ProVIC office? What services do they provide your organization, and what are your reporting requirements? How would you characterize your relationship? 6. Champion Communities a. What is your role with the CCs? b. How are they formed? c. Who makes up the CC? (positions, titles) d. How many CCs do you work with? How was that number established? e. What are the criteria for selection? f. How long did it take to establish the CCs? g. Were some rejected? If so, why? h. Have some changed membership, or has membership remained stable? i. How often do they meet? j. May we see an agenda and/or minutes? k. Does a community member—a CSW, for example, or an OVC—know about ProVIC and/or the CPCC? l. Does the CC facilitate access to services? In what ways? 7. Model: We’ve been asked to advise on the efficacy of Champion Community model of community mobilization. a. How would you explain the Champion Community model of community mobilization to an outsider? b. Have you seen it in action? c. Do you think that the CC improves community health? If so, how? Please provide examples. If not, why not? d. What do you think is the best part of the CC approach? e. What doesn’t work as you expect? Can you give us examples? 82 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION f. Do you think this model should be continued? Why? g. Would it be useful to make changes that could make the model work better? If not, why not? If yes, describe the changes. h. Are you familiar with other models of community mobilization either here in the DRC or elsewhere that you suggest we study for comparison? i. Can you provide an example of unintended benefits or costs/positive or negative consequences of this approach? 8. Efficacy a. From your point of view, has ProVIC improved the health of participating communities? How do you know? What signs do you see of improved health? b. If you could choose, would you change the CC model in the future? How would you change it? c. ProVIC has been through a number of changes since 2009. Explain the changes. How well has it adapted to the changes? How have those changes affected your work? d. In your opinion, what are the factors that contributed to achieving or not achieving program results? e. As you know, ProVIC contracts with eight NGO partner organizations that work with 49 Champion Communities, building local civil society capacity and providing fixed obligation grants to support small-scale community projects related to HIV/AIDS. - Are they the right partners? - Do you think they use the money well? - Can you think of less expensive or better ways to provide communities HIV/AIDS services? - Do you think this is cost effective? - Why or why not? f. Would you advise USAID to support integrated services in the future, or are specific programs specializing in P, C or T better? What would you change? g. What is your overall view of ProVIC? h. With whom do you suggest we meet to further assist in our data collection? USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 83 ANNEX VI. PROVIC TIMELINE Dates Program Milestones External Action 2009 09/30/2009 Contract signed via AIDSTAR for consortium 09/09 COP arrives 12/09 Initial PMEP developed 2009 CO is in Nairobi; Dr. Laurent Kapesa is USAID/COTR 2010 Q2-Q4 Opening of field offices and recruitment of field staff 02/10 Needs assessments completed among NGO partners inherited from RESA+ 07/10 CRS leaves consortium (was to do OVC) replaced by IHAA Q2-Q3 ProVIC inherits 16 PMTCT sites and provides technical assistance to ProSANI for 8 others 04/10 NGO partners: RFA process launched. 456 proposals received. 72 shortlisted and then trained Q3-Q4 ProVIC leadership in revising national PMTCT care and treatment guidelines Q3-Q4 FY 2010 4 Pilot CCs are started 09/10 14 NGO Partners contracted Q4 FY 2010 COTR Kapesa leaves. Josh Karnes becomes COR. New CO in Nairobi GF activities frozen in DRC First ProVIC COP departs All consortium budgets re-aligned 2011 04/11 New COP Trad Hatton arrived 04/11 RIG Audit Report 04/11 PMTCT Acceleration funds ($10m?) to expand sites and increase PMTCT 04/11 At request of USAID, Kisangani Proposal submitted to USAID by PATH for $9.1 million 84 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 05/11 ProVIC shifts from “bridge funding” to 14 new partners 05/11 PEPFAR Amb. Goosby visit 06/11 Training of NGO partners in Child-to-Child approach 06/11 U.S. Ambassador and partner “couples-tested” at CC ceremony 6/11 Revised Kisangani Proposal submitted First Kisangani proposal resubmission 07/11 ProVIC first to implement PMTCT triple therapy in all sites 08/11 Jean Ntumba takes position as Senior Admin Finance Specialist for ProVIC 07/11 USAID DRC conducts financial review 10/11 Revised Kisangani Proposal submitted USAID rule changes and PATH asked to resubmit 2011 USAID push to go to Kisangani First CC Prize awarded Reduced program time by six months 10-12, 2011 DRC elections preparation and elections violence Most ProVIC outreach activities stopped due to insecurity, particularly prevention and mobile HCT. 06/12 Theft of 50,000 HIV tests from airport warehouse Q1-Q2 ProVIC introduces individual beneficiary files for all OVC/PLWA Q1-Q2 ProVIC introduces salesforce database as center of M&E system Change to zone strategy New CO from Nairobi. 06/12 Dr. Loulou Razaka provides technical support to CCs 07/12 New GBV indicators included with Kisangani amendment for Kisangani and Kinshasa only Mid-2012 ProVIC introduces PIMA CD4 machines 01/12Q3 SCMS takes over supply chain/procurement 04/12 Added Kisangani office (Note from ProVIC: Kisangani amendment signed 12 months after original proposal) 07/12 ProVIC presents nighttime mobile HCT for MSM model to International AIDS Conference 07/12 Opened Kisangani office 10/12 Increased focus on health zone strategy. Health zone accords introduced. 11/12 ProVIC introduces Mentor Mother approach for PMTCT USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 85 11/12 Senior PEPFAR DP visit to Kinshasa and Katanga 12/12 ProVIC initiates “Improvement Collaborates” QA/QI approach in four Kinshasa hospitals Charly Mampuya become COR, replacing Josh Karnes ProVIC and ProNaNUT (government nutrition agency) finalize National Protocol on nutrition and HIV 2013 02/13 PEPFAR Strategic Pivot: ends all general pop activities. PMTCT platform replaces CC as organizing principle of ProVIC. 03/13 ARV included and new ARV indicators 03/13 New GBV indicators Q2-Q3 Expanded PMTCT sites from X to 75. 04/13 USAID instructs ProVIC to stop all testing in Sud-Kivu 06/13 – 07/13 Integrated Training on PMTCT for strategic pivot sites. Modules developed jointly with PNLS. 06/13 New CO, based in Kinshasa. 06/13 ProVIC evaluation 10/13 Sud-Kivu office closes, and Bas-Congo community work stops. Only BC clinical services continue 2014 03/14 Closeout begins. Field activities stop. 04/14 Regional Field Offices close. 05/14 Kinshasa office closes. End of program. 86 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 87 ANNEX VII. EVOLUTION OF PROVIC 88 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 89 90 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 91 ANNEX VIII. SUMMARY OF PROVIC ACTIVITIES BY PEPFAR GOALS Prevention Primarily ProVIC Champion Communities • CCC awareness campaigns and community projects • CCC outreach workers selected, trained, dispatched •Outreach workers encourage HCT (especially pregnant women), identify orphans and vulnerable children, distribute sefe sex messages and condoms •Outreach workers refer clients to to testing centers • PLWHA prevention with positives •GBV prevention awareness and training Treatment, Care and Support Primarily NGOs • Testing centers refer positive clients to NGO to be enrolled in support services: Self-help groups, possible nutritional support, medical referrals, medication, possible income-generating activities • Antiretroviral drugs treatment available through clinics • TB/STI screening and treatment integrated into HIV treatment at clinics •NGO social workers provide follow-up and house visits •NGOs report activities and data to ProVIC regional offices and government health zone offices Orphans and Vulnerble Children Primarily NGOs •Orphans and vulnerable children are enrolled in NGO support services: vouchers for heath care, school tuition waivers, possible nutritional support, possible job training, some legal support services •NGOs develop and sign accords with schools for tuition waivers for OVC •NGOs conduct monthly Child-to-Child psychosocial support program Health Systems Strendgthening • ProVIC grants and accords to to private- and public-sector service provide in-kind supplies and equipment • ProVIC supports health center and Health Zone offices with materials and supplies • ProVIC grants to Health Zones provide office support equipment and materials • Policy guidance and assistance to PNLS, PNMLS • Training at all levels 92 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 93 ANNEX IX. DETAIL OF PROVIC PROGRAM ELEMENTS BY REGION, AS OF JUNE 2013 ProVIC Regions Number of Champion Communities Number of Health Zones Supported Number of FOGs (Private Facilities) Number of Accords (Public Facilities) NGO Partners Bas-Congo (closing) 11* 6 8 JADISIDA, CEMAKI Kinshasa 6* 11 4 4 PSSP, RNOAC, SWAA, FP, TIFLE South Kivu (closing) 10 5 2 FFP, ACOSYF, ALUDROFE Katanga 13 * 10 3 5 OLASEC, BAK CONGO, WP, BDOM Orientale (Kisangani) 4 3 6 1 0 TOTALS 44 35 13 20 14 94 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION ANNEX X. DISCLOSURES OF REAL OR POTENTIAL CONFLICTS OF INTEREST USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 95 96 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION Name Ruth Kornfield Title Team Leader for ProVIC/DRC evaluation Organization GH Tech Bridge 3 Consultancy Position Award Number (contract or other instrument) Contract Number: AID-OAA-C-13-00032 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) ProVIC/DRC I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. 1. I have a niece that works for PATH HQ in Washington D.C. (mainly works in East and Southern Africa) 2. I used to work for CRS as HIV/AID advisor for central African region and as such was involved in the AMITIE project which may be a precursor to the ProVIC project. 3. I have worked for several NGOs which may be seen as a competitor with the implementing organization including CRS, FHI , URC, CARE, CEDPA, MSH. 4. Short consultancy with PATH in 1992 conducting family planning research in Algeria. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 97 Signature Date May 20, 2013 98 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION 99 100 USAID/DRC: INTEGRATED HIV/AIDS PROGRAM (ProVIC) FINAL EVALUATION For more information, please visit http://www.ghtechproject.com/resources GH Tech Bridge 3 Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com