Final Evaluation of the Project for Expanding the Role of Networks of People Living With HIV/AIDS Final Evaluation of the Project for Expanding the Role of Networks of People Living With HIV/AIDS Young-Mi Kim, Jhpiego John Lukwago, Jhpiego Consultant Stella Neema, Jhpiego Consultant ii Acknowledgments The evaluation of the Project for Expanding the Role of Networks of PHA in Uganda was made possible with support from the United States Agency for International Development (USAID)/Uganda via a subcontract to Jhpiego from the Population Council. This report was written by Dr. Young Mi-Kim, Dr. John Lukwago, and Dr. Stella Neema. The authors are grateful to Dr. Esther Namuli, Maxima Tibwita, David Mugisa, and Victor Guma from Makerere University for their outstanding contributions and input to the completion of this evaluation. Special thanks to staff at the Population Council (Dr. Sam Kalibala and Deborah Weiss) and Jhpiego (Sharon Kibwana and Amanda Ajulu) for their guidance and review of the report. We would especially like to thank the participants in the study—particularly the staff of the Alliance in Uganda— the Field Support Officers who accompanied us to the field, the districts, and health facilities; and the communities that participated in the interviews and the group discussions. This report was made possible through support provided by the United States Agency for International Development (USAID)/Uganda under the terms of Contract No. GHH-I-02-07-00034-00. The contents are the responsibility of the authors and do not necessarily reflect the views of USAID or the United States Government. Jhpiego is an international, non-profit health organization affiliated with The Johns Hopkins University. For nearly 40 years, Jhpiego has empowered front-line health workers by designing and implementing effective, low-cost, hands-on solutions to strengthen the delivery of health care services for women and their families. By putting evidence-based health innovations into everyday practice, Jhpiego works to break down barriers to high-quality health care for the world’s most vulnerable populations. www.jhpiego.org The Population Council conducts research worldwide to improve policies, programs, and products in three areas: HIV and AIDS; poverty, gender, and youth; and reproductive health. www.popcouncil.org Published in September 2009. Copyright © 2009. United States Agency for International Development. Suggested citation: Kim, Young-Mi, John Lukwago, and Stella Neema. 2009. “Final evaluation of the project for expanding the role of networks of people living with HIV/AIDS,” Final Report. New York: Population Council. This document may be reproduced in whole or in part without permission of the Population Council provided full source citation is given and the reproduction is not for commercial purposes. iii Table of Contents Abbreviations and Acronyms v Executive Summary 1 Section One: Introduction 4 1.1 Structure of the Evaluation Report 4 1.2 HIV/AIDS in Uganda 4 1.3 USAID Support for PHA Networks 4 1.4 Rationale for the Evaluation 7 1.5 Key Evaluation Questions 7 Section Two: Methodology 9 2.1 Evaluation Design 9 2.2 Study Districts 9 2.3 Data Collection Methods 9 2.4 Quality Assurance, Analysis, and Presentation 10 2.5 Ethical Considerations 10 Section Three: Evaluation Findings 12 3.1 Project Design 12 3.2 Project Performance 15 3.3 Project Management, Coordination, and Staffing 21 Section Four: Discussion 23 Section Five: Conclusions And Recommendations 26 References 29 iv Section Six: Annexes 31 Annex A : Key Informant Guide for Senior Staff and Project Cognizant Technical Officer (CTO) 31 Annex B : Key Informant Guide: Service Provider IDI Guide 34 Annex C : Key Informant Guide: Field Support Officers 35 Annex D : Key Informant Guide: Network Support Agents (NSA) 36 Annex E : Key Informant Guide: PHA Group Leaders 37 Annex F : Focus Group Discussions: PHA Group Members 39 Annex G : Focus Group Discussions: Non-PHA Community Members 41 Annex H : PHA Networks Project Staffing Structure 42 Annex I : Typology of PHA Involvement 43 v Abbreviations and Acronyms AIDS Acquired Immune Deficiency Syndrome ACP AIDS Control Programme ART Antiretroviral Therapy ARV Antiretroviral CBO Community-Based Organization CME Continuing Medical Education COP Chief of Party CSO Civil Society Organization CTO Cognizant Technical Officer DHO District Health Officer EGPAF Elizabeth Glaser Pediatric AIDS Foundation FBO Faith-Based Organization FGD Focus Group Discussion FSO Field Support Officer GIPA Greater Involvement of People Living With HIV/AIDS GoU Government of Uganda HBC Home-Based Care H/C Health Center HCT HIV Counseling and Testing HIV Human Immunodeficiency Virus JCRC Joint Clinical Research Centre M&E Monitoring and Evaluation MEEPP Monitoring and Evaluation of Emergency Plan Progress MIPA Meaningful Involvement of People Living with HIV/AIDS MOH Ministry of Health NACWOLA National Community of Women Living with HIV/AIDS in Uganda NAFOPHANU National Forum for PHA Networks in Uganda NGO Non-Governmental Organization NSA Network Support Agent NULIFE Food and Nutrition Interventions for People Living with HIV/Nutrition is Life NUMAT Northern Uganda Malaria, AIDS/HIV and TB Program OVC Orphans and Vulnerable Children PEPFAR President’s Emergency Plan for AIDS Relief PHA People Living with HIV/AIDS vi PMTCT Prevention of Mother-to-Child Transmission POMU Positive Men’s Union PSI Population Services International PwP Prevention with Positives TASO The AIDS Support Organization TB Tuberculosis TSW Treatment Support Worker UAC Uganda AIDS Commission UNAIDS Joint United Nations Programme on HIV/AIDS USAID United States Agency for International Development USG United States Government VCT Voluntary Counseling and Testing Final Evaluation of the Project for Expanding the Role of Networks of PHA 1 Executive Summary In July 2006, the International HIV/AIDS Alliance launched a three-year project on ―Expanding the Role of Networks of People Living with HIV/AIDS in Uganda.‖ The project, which is funded by the United States Agency for International Development (USAID)/Uganda, sought to increase the involvement of people living with HIV/AIDS (PHA) in the HIV/AIDS response and to improve access to and utilization of prevention, treatment, care, and support services for PHA households. PHA groups at the community level form the backbone of the Networks Project. With training and support from project staff, partner PHA groups build their capacity to offer a wide array of home- and community-based services, including palliative care, adherence counseling, HIV prevention activities, and material support for orphans and vulnerable children (OVC). Selected members of the PHA groups receive special training to serve as Network Support Agents (NSAs). The NSAs refer PHA for services, work alongside health care workers at health facilities, and follow up clients in the community. The Alliance’s Field Support Officers (FSOs) are responsible for mentoring the NSAs and PHA groups, working with monitoring and evaluation (M&E) staff to collate and analyze service data, and working with grants managers to oversee the award of small grants to PHA groups. USAID/Uganda commissioned an evaluation of the Networks Project’s design, strategies, and performance in order to improve its implementation and learn from its successes and challenges. The study was a cross-sectional descriptive and analytical process evaluation employing both rapid participatory assessment and appraisal techniques. It utilized mixed methodologies in data collection, including a desk review, 113 key informant interviews, and 16 focus group discussions. The evaluation was conducted in 11 project districts. The evaluation’s main findings and recommendations are as follows: Access to and utilization of services: Monitoring data and key informants agree that the PHA Networks Project has increased access to and utilization of home- and community-based services (including palliative care, OVC support, and wraparound services), increased referrals to health facilities, and increased the involvement of PHA in service delivery. Given these accomplishments, USAID should consider not just continuing, but increasing, its support for the project so that it can consolidate its efforts in current project districts and eventually expand coverage further. Network Support Agents (NSAs): The NSAs are one of the most successful components of the project. They have been able to mobilize PHA to utilize existing services, including health facility-based services and wraparound services provided by community-based organizations (CBOs) and non-governmental organizations (NGOs). They compensate for a shortage of staff at health facilities, where they direct and counsel clients. They also play a critical role in tracking and following up clients. For example, they identify antiretroviral (ARV) drug defaulters in the community and either refer them back to the facility or inform the facility if they have moved away, transferred to another facility, or died. Because NSAs live openly and positively with HIV, they have also helped reduce the stigma that previously deterred PHA from seeking care, and have encouraged those who test positive to disclose their status to spouses and family members. The Alliance should provide further support to 2 NSAs by offering them ongoing and in-depth training on relevant issues, such as child counseling and tuberculosis (TB) care, and by addressing issues raised during interviews, such as increasing the allowances they are given to cover expenses. Monitoring and evaluation (M&E): The Alliance recently introduced new tools, including referral registries and NSA diaries, to strengthen and streamline an onerous data collection system and begin gathering qualitative data to better understand and explain the project’s achievements. However, further capacity building is needed among PHA groups and FSOs so that they are better able to analyze and apply data to project activities, offer feedback, and document and share lessons learned and best practices. Small grants: The grants process has empowered PHA groups to develop innovative ideas and services, manage funds, and ―own‖ the activities they develop. The large number of PHA groups seeking grants, however, has overwhelmed the funds available, the grants department (which trains PHA groups on proposal writing and grant management), and the FSOs (who follow up grant recipients). The Alliance needs to hire additional staff to oversee the grants process and should also offer PHA groups that are managing grants additional training. Staff shortages: When the project expanded from 14 to 40 districts, staffing levels became inadequate, workloads became unmanageable, and the quality of management and operational delivery suffered. The Alliance needs to hire more staff at the district and regional levels—including additional grant managers, M&E officers, and most especially FSOs—to divide up the workload and offer adequate support to the PHA groups and NSAs. Community engagement: The Alliance has successfully facilitated active community engagement and empowerment. The PHA groups operate at the community level and have developed strong partnerships through community engagement activities such as home visits, sensitization activities, referrals to health facilities, and OVC support. As a result, community members feel a sense of ownership and believe that they are addressing real problems or needs in their own communities. The Alliance should capitalize on this achievement by seeking to involve local authorities, NGOs, and CBOs operating in each district in community activities sponsored by PHA groups. District partnerships: At the inception of the project, heavy buy-in of district officials was attained. However, these partnerships and linkages have not been maintained over time, even though District Health Officers (DHOs) play an important role in placing NSAs at health facilities. The Alliance needs to reestablish district-level support through advocacy, by having FSOs and PHA groups attend District Health Team meetings, and by routinely sending the Alliance’s service delivery reports to the district health office (provide quarterly feedback reports to districts). Consolidate and strengthen project efforts in current districts: The project’s expansion from 12 to 40 districts, as well as its replication in a further 12 districts, demonstrates that the network approach can be rapidly and effectively brought to scale. However, a lack of human and financial resources undermined the smooth operation of some activities, such as grant management and evaluation, during this expansion. Therefore, the Alliance should temporarily stop its expansion efforts and focus instead on consolidating and strengthening project efforts in current districts before rolling out its interventions to a larger geographic Final Evaluation of the Project for Expanding the Role of Networks of PHA 3 area. This effort will also ensure improvement in the quality of mentoring and support supervision provided to the NSAs and the groups of people living with HIV and AIDS. Greater involvement of people living with HIV/AIDS (GIPA) and meaningful involvement of people living with HIV/AIDS (MIPA): As a direct result of the project, PHA have moved from playing a strictly passive role as patients to an active role as service providers in the community and in designated health facilities. In fact, the project is a rare example of putting the widely endorsed GIPA and MIPA principles into practice in a systematic manner and on a countrywide basis. When weighed against typologies of involvement, however, it is clear that the PHA Networks Project could do still more. The project has not yet pushed PHA involvement to the highest levels, in which PHA play an active role in management, policymaking, and strategic planning. The approach, the lessons learned, and the limitations of the Networks Project should be documented and shared widely as an inspiration for others. 4 Section One: Introduction 1.1 Structure of the Evaluation Report This evaluation report is divided into the following five sections:  Section One reviews the need for the Networks Project, the project itself, and the evaluation.  Section Two describes the evaluation methodology, including evaluation design, study districts and participants, data collection methods, and ethical considerations.  Section Three presents the findings of the evaluation regarding project design, project performance, and project management, coordination, and staffing.  Section Four discusses the major issues emerging from the findings and how they affect project functionality.  Section Five presents conclusions and recommendations. 1.2 HIV/AIDS in Uganda By the end of 2006, an estimated 2.3 million Ugandans had been infected with HIV; of these, approximately 1.4 million were living with HIV, and almost one million had died due to AIDS-related illnesses since 1982 (Ministry of Health Uganda 2006). A report by the Uganda AIDS Commission estimated that more than 70 percent of people infected with HIV did not know their HIV status, and only 23 percent of the population had access to HIV counseling and testing (HCT) services (Uganda AIDS Commission 2006). Although 42 percent of the population in need received antiretroviral therapy (ART) in 2005, the number of people in need of ART continues to grow each year and is projected to reach 238,000 in 2012—far outstripping the capacity of the national health care system and district government to respond. Despite efforts by the Government of Uganda (GoU) and its development partners, many people living with HIV/AIDS (PHA) do not access services because of factors such as lack of awareness, ignorance of their HIV status, long distances to ART sites, lack of financial resources, and stigma. Various approaches have been used to increase access to care and to address barriers to utilization of HIV/AIDS services; one such approach employs social and community-based support mechanisms. While community-based groups and other organizations are able to provide vital support to PHA, their organizational and technical capacities are limited, and they frequently operate in isolation from PHA groups. 1.3 USAID Support for PHA Networks Through the President’s Emergency Plan for AIDS Relief (PEPFAR), the United States Agency for International Development (USAID)/Uganda has supported Ugandan and Final Evaluation of the Project for Expanding the Role of Networks of PHA 5 international organizations to develop and strengthen networks of service delivery to enhance access to comprehensive HIV/AIDS services for PHA and their families. These networks aim to create linkages that facilitate access to a full continuum of HIV/AIDS services delivered through non-governmental organizations (NGOs), faith-based organizations (FBOs), and public sector institutions (USAID Uganda and Marie Stopes International 2008). One such organization is the International HIV/AIDS Alliance and its project: ―Expanding the Role of Networks of People Living with HIV/AIDS in Uganda‖ (PHA Networks Project), a three￾year, US$7.7 million project awarded in July 2006. 1.3.1 Project Goals, Objectives, and Principles The overall goals of the project are to strengthen the capacity of PHA networks at the national and district levels and PHA groups at the sub-district and community levels to become involved in the HIV/AIDS response, and to improve access to HIV/AIDS prevention, treatment, care, and support services for PHA households. The project has two strategic objectives: (1) to increase utilization of HIV/AIDS prevention, treatment, care, and support services by PHA households, and (2) to increase the organizational capacity of PHA networks at the national, district, sub-district, and community levels. The project’s expected results are:  Increased access to HIV/AIDS prevention, treatment, care, and support services by PHA households;  Strengthened and supported organizational capacity of the PHA networks and groups at the national, district, sub-district, and community levels;  Increased involvement of PHA in HIV/AIDS service delivery at health facilities and in communities; and  The establishment of partnerships between PHA groups and organizations providing support to orphans and vulnerable children (OVC) and other HIV￾related programs. As suggested above, one of the essential principles underlying the project is the greater involvement of people living with HIV/AIDS (GIPA). The GIPA principle was first formalized at the 1994 Paris AIDS Summit and has since been endorsed by more than 190 countries at United Nations meetings. GIPA aims to realize the rights and responsibilities of people living with HIV, including their right to self-determination and participation in decision-making processes that affect their lives (UNAIDS 2007). GIPA holds benefits for the individual, by improving self-esteem and decreasing isolation; for the organization, by changing perceptions and providing valuable insights; and for the community, by breaking down fear and prejudice. Ultimately the engagement of people living with HIV in policymaking and program development and implementation enhances the quality, acceptability, and effectiveness of the AIDS response. It is important, however, that such involvement be meaningful, rather than token—giving rise to the companion principle of meaningful involvement of people living with HIV/AIDS (MIPA) (SAfAIDS n.d.). The 6 Alliance proposed to promote GIPA and MIPA by increasing the access of PHA to services, empowering and training PHA to deliver HIV/AIDS services themselves, and ensuring PHA involvement in management and policymaking at a national level. 1.3.2 Project Design The project operates at three levels. At the community level the project aims to strengthen PHA groups and community-based service providers so they can better participate in the network model of service delivery. At the district level, the project works with the District Health Officers (DHOs) and other health personnel to build a continuum of care for PHA. At the national level, the project develops strategic partnerships with the Uganda AIDS Commission (UAC) and the Ministry of Health (MOH) to place volunteers from PHA groups at government health centers. The project also collaborates with strategic organizations such as The AIDS Support Organization (TASO), Joint Clinical Research Centre (JCRC), Northern Uganda Malaria, AIDS/HIV and TB Program (NUMAT), Food and Nutrition Interventions for People Living with HIV/Nutrition is Life (NULIFE), Population Services International (PSI), and the Elizabeth Glaser Pediatric AIDS Foundation (EGPAF) to promote scale-up of project interventions and ensure sustainability. PHA groups form the backbone of the Networks Project. The Alliance has grouped them into PHA network clusters; on average, eight PHA groups belong to each cluster, although the number ranges from as few as six to as many as 23. Alliance staff members work directly with one lead, or liaison, PHA group in each cluster to build its capacity to offer services. Each lead group, in turn, is responsible for building the capacity of the other PHA groups in its cluster. PHA groups act as community service delivery points for a range of HIV services, including home-based palliative care, adherence counseling, HIV prevention activities, and patient tracking. They also conduct community-based activities including the provision of material support to PHA and OVC, income-generation activities, male involvement activities, and sensitization in schools. The Networks Project offers grants to PHA groups to fund these activities. For example, some PHA groups have applied for and received grants for income￾generating activities that they can manage as a group and use to support OVC; thus grant money has gone to support keeping of poultry and pigs, tending of kitchen gardens, and purchasing of ploughs. Other PHA groups have received grants to carry out home-based care (HBC) activities among their members or to conduct prevention sensitization in schools and among uniformed communities. Certain members of the PHA groups play a special role in the project. These Network Support Agents (NSAs) are selected from members of PHA groups who live openly with HIV. They receive basic training in order to facilitate and manage linkages between home￾based, community-based, and health facility-based care. NSAs also work closely with PHA group members to coordinate referrals for services to health facilities, NGOs, and other community organizations. They volunteer at health facilities, where they work alongside health care workers and support the delivery of services by receiving clients and directing them where to go for what services, conducting client education and counseling, and assisting with patient registration and records. Back in the community, they counsel clients on drug adherence and positive living. Final Evaluation of the Project for Expanding the Role of Networks of PHA 7 The Networks Project is led by the Chief of Party (COP), who works closely with the project staff responsible for day-to-day project operations, including planning, project implementation, technical support, financial management, and reporting. (See Annex H for a staffing matrix.) Specific roles of key project staff are as follows:  Deputy Chief of Party: Supports the COP in routine administrative and operational activities.  Field Support Officers (FSOs): Five regional FSOs serve as a link between the Alliance’s office and project partners at the district and community levels. The FSOs are responsible for mentoring NSAs and PHA groups. They also monitor field activities by collating program reports from the NSAs and liaison groups and submitting them to the head office.  Grant Officer: Leads the process of soliciting concept notes from PHA groups, supports the efforts of lead PHA groups to consolidate the work plans of smaller PHA groups, prepares disbursement schedules, and tracks grants provided to PHA groups.  Technical Advisor: Helps assure the quality of services offered by ensuring that targets are met, that PHA groups and NSAs receive high-quality training, and that NSAs have access to adequate and ongoing technical support, supervision, mentoring, and training.  Monitoring and Evaluation Advisor: Manages the reporting and monitoring of project data and indicators. The project also works with an Advisory Committee, whose members include representatives from the MOH, PHA groups, HIV/AIDS advocates, and stakeholders from the districts. The Advisory Committee provides governance, oversight, and technical guidance to the project’s senior management team. The International HIV/AIDS Alliance Headquarters office based in Brighton, United Kingdom also provides overall project oversight. Implementation began in July 2006. The original intention was to implement the project in a total of 14 districts, beginning in seven districts and then extending to seven more. However, the promising results in these initial target districts led USAID to increase funding by US$4.7 million, for a total obligated funding amount of US$7.7 million in September 2007, in order to expand project coverage to 40 districts. In addition to the 40 districts where the Alliance carries out direct implementation of the PHA Networks Project, there are other districts where similar projects are implemented by partners of the Alliance. These include the 12 districts of Northern Uganda and the West Nile region. 1.4 Rationale for the Evaluation The purpose of this evaluation is to assess the PHA Networks Project’s start-up activities and determine whether there is early evidence pointing to project objectives being reached and sustained. The evaluation examines the effectiveness of the project design, strategies, and performance in order to improve the project implementation and learn from its successes and challenges. Specifically, the evaluation focuses on whether the design of the project supports its ability to achieve its goals and objectives; whether the project has achieved expected results; how well the project management is functioning; and how it has been affected by the 8 rapid scaling-up. Although the evaluation was originally intended as a midterm review, it was ultimately carried out at the end of the project. 1.5 Key Evaluation Questions The evaluation addressed the following key questions: Design 1. How is the project contributing to the achievement of GoU and United States Government (USG) goals regarding increased access to HIV prevention, comprehensive continuum of care, treatment, and support, and is the project able to demonstrate this? 2. Is the USG support through the Networks Project a recommended approach to continue supporting PHA in order to achieve GoU and USG goals regarding increased access to and utilization of HIV prevention, care, and treatment and support services? 3. Does the project design and structure adequately support and facilitate achievement of the desired results? Performance 4. Is the project on track to achieving its overall objectives and results? 5. Are the systems and mechanisms for documenting lessons learned and good practices in terms of the network model effective? 6. Is the process of providing grants to PHA groups well designed, administered, and evaluated? 7. Is the emphasis placed at the community, sub-county, and the district levels appropriate? 8. Has the placement of NSAs in health facilities made a difference in increasing PHA access and utilization of HIV/AIDS services? 9. Are NSAs adequately supported with regard to ongoing training, support, and supervision? Management, Coordination, and Staffing 10. How has the requirement for significant scale-up affected the management and operational delivery of the project? Is the project staffing structure appropriate to support project development and scale-up? 11. How are partnerships developed and maintained at the different levels (national, district, and community)? Final Evaluation of the Project for Expanding the Role of Networks of PHA 9 Section Two: Methodology 2.1 Evaluation Design This was a cross-sectional descriptive and analytical process evaluation employing rapid participatory assessment and appraisal techniques. Community HIV/AIDS interventions such as the PHA Networks Project are based on a results chain that includes the project’s inputs, processes leading to its immediate outputs, and outcomes and impacts. This evaluation focused on the process and outputs part of this results chain. 2.2 Study Districts Evaluation activities focused on 11 districts (Bududa, Hoima, Iganga, Kabale, Kanungu, Kayunga, Kibaale, Luwero, Mayuge, Mbale, and Namutumba), which were purposely selected to cover at least one district from each of the five regions in which the project operates, and to capture start-up and expansion districts, thus maximizing the chances of finding lessons learned and best practices. The selected districts also represented the highest (Iganga, Luwero, Mbale) and the lowest (Kibale, Kisoro, Namutumba) performers, based on data from annual reports including: understanding and implementation of the network model of referrals and linkages by PHA groups and NSAs, the ability to reach more individuals and recruit more PHA as group members, and the inclusion of wraparound activities such as the Basic Care and Prevention Package provided by PSI and distributed to PHA. This package includes antibiotics, insecticide-treated nets, safe water system, condoms, information on HIV/AIDS care and treatment services, and nutrition interventions facilitated by the NULIFE project. 2.3 Data Collection Methods 2.3.1 Desk review The evaluation team undertook a review of the literature on the PHA Networks Project. The review focused primarily, though not exclusively, on documents about the project. Documents such as the USAID Cooperative Agreement 617-A-00-06-00008-00, Special Study of HIV/AIDS Service Networks in Uganda, and Project Annual Reports 2006/07 and 2007/08 were reviewed. The goal of the review was to obtain information to answer the key evaluation questions and to determine which evaluation questions would need detailed primary data collection and verification from the field. Key findings relevant to the study objectives were summarized. Unclear sections were noted, and clarification was provided by the PHA Networks Project staff. 2.3.2 Key informant interviews A total of 113 key informant interviews were conducted. Key informants were purposely selected for their knowledge of the project at the MOH, UAC, Monitoring and Evaluation of Emergency Plan Progress (MEEPP), district governments, health facilities, and service 10 delivery partners. They included Alliance senior and mid-level management, FSOs, NSAs at sample health facilities, PHA group leaders, representatives of district NGOs, DHOs, HIV/AIDS focal persons, the in-charges of selected health facilities, and counselors directly engaged with the NSAs. The interviews were designed to assess how the project was contributing to the achievement of GoU and USG goals and whether management of the project was functioning well. For each interview, at least two consultants participated, with one asking relevant questions and the other taking notes; the aim was to minimize the duration of the interview since many of the key informants had busy schedules. Interviews were audio-taped after permission was obtained from the respondent. 2.3.3 Focus group discussions (FGDs) FGDs were conducted with members of PHA groups to assess the reach and effectiveness of the project activities and to collect recommendations for improving their activities and the overall project design. In total, 11 FGDs were conducted with PHA members and five with non-PHA community members. Participants were randomly selected. One FGD was also conducted with the Alliance Advisory Committee. 2.4 Quality Assurance, Analysis, and Presentation After each day of data collection, the moderator and note taker discussed key findings in a debriefing session, and edited, cleaned, and summarized the data. The note taker also expanded the notes into more detailed versions. Missing information or inconsistencies were identified and where possible the consultants appropriately followed up with key informants within one working day or at a time convenient to the key informant via a telephone conversation. For data analysis, audio-taped data were transcribed verbatim. Textual data were then explored using content analysis (Riley 1990). Data were read and re-read by the analysts in order to identify emerging themes from the transcripts (Glaser and Strauss 1967). All data relevant to each theme were identified and examined using the process of constant comparison, in which each item is checked or compared with the rest of the data in order to establish analytical categories. Typical quotes were also selected and included in this report in order to emphasize responses without losing the original context of the meaning. 2.5 Ethical Considerations The study team obtained informed consent from all persons who were interviewed after explaining the goals and objectives of the study, confidentiality safeguards, and the potential risks and benefits. Furthermore, the team provided assurances that it would utilize the information only for the purposes of the study. No names of individual informants have been used in this report without their consent. Final Evaluation of the Project for Expanding the Role of Networks of PHA 11 Consultants collected the recorded information and notes directly following the discussion and sealed them in an envelope. The expanded notes were added to the other information and resealed in an envelope, which was stored in a locked cabinet, to which only the Team Leader, Co-team Leader, and Secretary had access. Transcriptions were stored electronically under password protection. 12 Section Three: Evaluation Findings 3.1 Project Design Q1. How is the project contributing to the achievement of GoU and USG goals regarding increased access to HIV prevention, comprehensive continuum of care, and treatment and support, and is the project able to demonstrate this? The Networks Project contributes to the achievement of GoU and USG goals of increased access to HIV services and support primarily by facilitating linkages and referrals between the community and its PHA groups, health facilities, and district HIV/AIDS stakeholders. Initial steps toward building a referral system were taken during the community engagement process, when a resource map exercise was conducted to identify existing health facilities and community services. Stakeholders were also asked to name which services they were in position to provide to PHA as part of the network model. This information formed the basis for service directories, which NSAs use when making referrals to complement their own knowledge base of the services available in their home communities. NSAs refer PHA for a wide variety of services, including HIV counseling and testing, the treatment of opportunistic infections and prophylaxis, prevention of mother-to-child transmission (PMTCT), ART, prevention and palliative care, and OVC care and material support. NSAs hold referral meetings in the health sub-districts where they work to create a coordinated approach to providing services to people living with and affected by HIV and AIDS. The NSAs follow up clients to determine whether those referred have actually received the services and whether they require other support. NSAs also follow up clients they meet in the ART clinic, checking whether they are taking their medication and responding well to treatment. According to key informants, the large number of clients referred by NSAs who come to health facilities is evidence that the referral system works. The referral component of the model is weakened, however, by the limited coverage of HIV/AIDS services in rural sub-districts and stock-outs of HIV test kits and drugs to treat opportunistic infections at health facilities. NSAs attend networking meetings with facility-based service providers. Before the project began, there were no point persons responsible for facilitating these kinds of linkages. Currently 1,302 NSAs serve in 643 health facilities and nearby communities. (See Question 8 for further description of the NSAs’ role and activities.) They [NSAs] have managed to convince the communities that HIV/AIDS is real and the people believe them, in a way they never believed the health workers. They [NSAs] have helped to bridge the gap between the health care system and the community; people are more comfortable coming to the clinics since they know they will be finding their peers at the facility who will show them around. (Health Unit In-Charge—Key Informant) The monitoring data presented in Table 1 demonstrate how the work of the NSAs and PHA groups has increased access to services. NSAs and PHA group members referred 115,819 individuals to health care facilities and HBC during the 4 th quarter of 2008. This is an enormous increase from the 4th quarter of the previous year, when only 4,649 referrals were made. The same trend can be observed for several other services. From the 3rd quarter of Final Evaluation of the Project for Expanding the Role of Networks of PHA 13 2007 to the 3rd quarter of 2008, the number of individuals provided with ART adherence counseling and support rose from 17,808 to 176,784, while the number receiving HIV/AIDS education and awareness prevention rose from 81,762 in the 4th quarter of 2007 to 633,233 in the 4th quarter of 2008. Table 1 Number of services and referrals provided by PHA group members and NSAs in PHA networks Service provided July–Sept. 2007 Oct.–Dec. 2007 Jan.–March 2008 April–June 2008 July–Sept. 2008 Oct.–Dec. 2008 ART literacy and education 30,922 35,124 50,756 326,009 425,233 * ART adherence counseling and support 17,808 19,251 25,474 120,731 176,784 * HIV/AIDS follow-up counseling 8,365 8,332 10,271 77,499 129,936 * Referrals for health facilities and community￾based services 4,956 4,649 6,023 60,148 78,831 115,819 Community follow-up 7,529 10,394 10,125 61,075 93,266 * HIV/AIDS education and awareness prevention 55,836 81,762 74,456 500,532 573,295 633,233 * Data are not available from the Alliance because of a change in the monitoring form. Q2. Is USG support through the Networks Project a recommended approach to continue supporting PHA in order to achieve GoU and USG goals regarding increased access to and utilization of HIV prevention, care, and treatment and support services? Given the project’s achievements on two fronts—service delivery and MIPA—the USG should continue to support the Networks Project and its approach to increasing access to and utilization of HIV services. Key informant interviews confirm the data in Table 1: health workers reported that the NSAs increased referrals of PHA to health facilities. The impact was greatest in voluntary counseling and testing (VCT) and accessing ARVs and septrin. According to both interviews and FGDs, the project’s reliance on NSAs to link health facilities and community services has led to a high level of acceptance and participation of PHA in care and treatment. Health workers view the NSAs as complementing their own work and therefore extending coverage and utilization of services by PHA. The community has also responded positively to the project’s approach: Before the Alliance came people used to spend the whole day at the hospital, but these days they don’t. The NSAs try to help us go through very fast. Also people have increased in number coming to the health unit for various services. Also those who have tested are many unlike before. (Non-PHA—FGD) By its very nature, the project’s design has promoted GIPA and MIPA along with service utilization. The sheer number of NSAs and the number of health facilities at which they serve 14 indicate greater involvement by PHA. Interviews with NSAs suggest that their involvement is also more meaningful, because their work draws on their own personal experiences in dealing with HIV/AIDS as patients, family members, and neighbors. The NSAs also report that the project’s approach has helped transform PHA’s role and self-image more generally, as PHA have moved from playing a strictly passive role as patients or recipients of services to an active role as service providers in designated facilities and in the community. As the following quote illustrates, this transformation has been deeply meaningful to NSAs: I have achieved by giving services to my fellow PHA. I feel honored. I feel I have done a lot and I feel recognized… (NSA—Key Informant) At the same time, the NSAs’ work, combined with their public disclosure of their own HIV status, has encouraged greater involvement by all PHA in the community: They have also brought us together as PHA. If I see someone with a rash I tell him to do something that will help him. (PHA Group—FGD) Q3. Does the project design and structure adequately support and facilitate achievement of the desired results? To achieve the results described above, the project designed a network model that strengthens the ability of PHA groups to effectively respond to the HIV pandemic. The Alliance organized 430 PHA groups into 55 network clusters and identified the strongest PHA group in each cluster to serve as a liaison with the project. Interviews with the Alliance’s project staff underscored that this structure enabled them to reach a far greater number of PHA groups with training and capacity-building activities. Another key element in the project design—the small grants process—also helped build the capacity of the PHA groups, both by providing management and financial skills and by encouraging innovative proposals to better serve PHA. (Small grants are further described under Q6.) In this way, the project design and structure enabled the PHA groups to provide a wide variety of services to PHA in the community, including educational support, memory book writing, finance management, home-based palliative care, psychosocial support, follow-up care, and prevention with positives (PwP) (i.e., ART education and counseling and positive prevention education). Once energized and empowered, this network of PHA groups was able to forge new links between the community and the health care system, which are essential to providing a continuum of home-based, community-based, and health facility-based care—a continuum that offers PHA a complete package of care throughout the stages of HIV disease progression. NSAs, who are drawn from the ranks of PHA groups, serve as especially active links in the network: they mobilize PHA in the community and refer them to services from PHA groups, community-based organizations (CBOs), NGOs, and health facilities; work alongside health care workers in facilities; and follow up clients in the community. This network model has facilitated the achievement of project results. It supports strained health facilities and community-based delivery systems, while also putting affected populations at the core of the response to HIV and AIDS. The model has also produced results rapidly and at scale. Final Evaluation of the Project for Expanding the Role of Networks of PHA 15 3.2 Project Performance Q4. Is the Networks Project on track to achieving its overall objectives and results? Service data show that the PHA Networks Project is indeed meeting many of its objectives. Some results have exceeded expectations, as shown by the comparison of planned and achieved outcomes for the first three years of the project in Table 2. For example, in project Year 2, twice as many individuals were provided with palliative care as planned (71,679 versus 39,000). During the first six months of Year 3, the project achieved or exceeded annual targets for palliative care and referrals. Table 2 PHA Networks Project targets and achievement Service outcomes Year 1 (2006/07) Year 2 (2007/08) Year 3 (2008/09)* Number of individuals provided with palliative care Planned 0 39,000 52,250 Achieved 0 71,679 70,394 Number of new service outlets providing HIV palliative care Planned 40 140 183 Achieved 42 418 183 Number of referrals made by NSAs and PHA groups Planned 2,500 8,645 234,739 Achieved 3,432 9,605 260,821 Number of PHA groups provided with small grants Planned 24 29 54 Achieved 0 29 0 Number of partnerships established with organizations Planned 21 57 36 Achieved 32 52 22 Number of PHA groups that have conducted an analysis of organizational capacity Planned 40 50 34 Achieved 29 54 18 * First six months only. The PHA Networks Project has had a particularly strong impact on increasing access to services by filling gaps caused by a dearth of health care workers. All of the health workers interviewed said that the referral system operated by the NSAs helped increase the number of PHA served and also made reporting more efficient and effective. As one key informant explained: They [NSAs] are helping a lot because we were few and they have helped us do many things we couldn’t handle alone. For example we had failed in the following up of our clients on ART because of [lack of] manpower and now they are doing great here. They also do a lot in following up these patients. We now can tell who died, shifted, and who are alive especially with the defaulters which we were not doing before. (Health Unit In-Charge—Key Informant) However, some of the planned service outcomes were not achieved. For instance, the Alliance only began awarding small grants in the second year of the project and did not award any grants in the first six months of Year 3. The grants were delayed because it took 16 longer than anticipated to build the organizational capacity of PHA groups at the sub-district level. In addition, the number of PHA groups far outstripped the project’s estimates, requiring PHA groups to form consortiums at the sub-district level. Q5. Are the systems and mechanisms for documenting lessons and good practices in terms of the network model effective? The evaluation suggests that systems and mechanisms for documenting lessons learned and good practices need improvement. A performance monitoring and evaluation (M&E) work plan was in place at the start of the Networks Project, and an M&E Advisor was hired in November 2006. The advisor left the position in April 2008, and subsequently a replacement advisor was recruited. NSAs act as the primary data collectors. Throughout the first two and a half years of the project, they collected only service-level data using forms developed by the Alliance M&E department. These forms had two problems: they collected a minimal amount of qualitative data, and they collected large amounts of quantitative data that were not directly related to the core evaluation indicators, which decreased the efficiency of data collection, analysis, and reporting. The Alliance recognized these weaknesses and introduced a set of revised M&E tools between October and December 2008, which are designed to capture both qualitative and quantitative data on project performance. Referral registers monitor the number of people referred each month, the services for which they are referred, the services that they do not receive at referral sites, the services for which they are followed up, and the NSAs making the most and least referrals. The qualitative information collected includes case studies, lessons and experiences, quotes, and notes on the quality of services. The NSAs use this information to share experiences and challenges when they conduct their monthly meetings. NSAs have also been given diaries to document and share experiences with the project. Both the referral registries and diaries are new additions to the project, so it cannot yet be determined whether they have facilitated the documentation of lessons learned and good practices. Interviews at the project office revealed that some PHA groups lack sufficient capacity to carry out M&E. Specifically, they lack the ability to analyze the data collected and apply the findings to strengthen the project. FSOs also find it challenging to analyze data and provide feedback to PHA groups. While the project has provided capacity building in M&E, this aspect needs to be strengthened, especially in the areas of documenting better practices and the experiences of NSAs, collating data, and offering feedback to PHA groups. The Alliance shares project information with interested parties inside and outside Uganda by:  Sharing quarterly and annual reports with district partners, the UAC, and the AIDS Control Programme (ACP) of the MOH;  Holding briefing meetings with officials at the UAC, MOH, and USAID;  Feeding information into MEEPP which, in turn, shares the information with the ACP and UAC (although this mechanism needs to be enhanced); Final Evaluation of the Project for Expanding the Role of Networks of PHA 17  Making presentations at PEPFAR meetings for implementing partners, national workshops and conferences (such as the 2008 National AIDS Conference in Kampala organized by the UAC), and international conferences (such as the HIV Implementers Conference, Kampala July 2008 and the XV International Conference on AIDS and STIs in Africa in Dakar in December 2008); and  Publishing and distributing a booklet detailing the Networks Project and Alliance Uganda. Q6. Is the process of providing grants to PHA groups well designed, administered, and evaluated? The application process and awarding of grants to PHA groups is well designed. However, delays in approvals, lack of involvement by district officers, and the volume of grants and geographical coverage per grant manager hampers the effective administration and evaluation of grants. The process of providing grants begins by distribution of a request for concept notes to community development officers, to the HIV focal persons and at public places, such as health facilities, and during community engagements and NSA trainings. Project staff use a structured format to review concept notes and assess the objectives of proposed projects and the extent to which they meet eligibility criteria and follow administrative details of the submission. During the solicitation process, a few groups are sampled and surveyed to confirm their existence and to observe their activities. Grants are given only to groups that are registered at the sub-county level and are known to local officials, who provide them with a reference letter for the application process. PHA groups that submit promising concept notes are selected for capacity building in proposal writing and grant management. A three-day capacity-building meeting is convened to: (1) provide training on work plans, reporting, budgets, and other relevant topics, and (2) further assess the capabilities of the grant applicants. The contracting process is done in conjunction with USAID, following the submission of work plans, M&E plans, and negotiation memos. The grants department—particularly the Finance and Grants Manager—has an extremely heavy workload and is overstretched by the numerous groups that apply for grants. To alleviate this problem, a Grants Officer was recruited in 2008 to help PHA groups prepare work plans and assist in the overall tracking of grants. However, follow-up must still be done by the appropriate FSO. Since FSOs also are overstretched by their many responsibilities, some aspects of grant management may be suffering, for example, tracking progress and documentation. The strength of the granting process is that PHA groups are empowered to manage funds and ―own‖ the projects they develop. As a result, PHA groups have gained substantive skills in grant management, evidenced by the fact that the majority have been able to follow all of the required grant procedures. The evaluation also identified some important weaknesses in the granting process. First, the approval process is lengthy, taking as much as 18 weeks: six weeks for the call for concept papers; four weeks for alignment with the national strategic plan, PEPFAR indicators, and the project plan; four weeks for the development of work plans and M&E plans; two weeks for 18 negotiation; and two weeks for submission and approval. Second, district officials are not adequately involved in providing input on the grant applications. This lack of involvement is a disservice to the project because district officials are knowledgeable about the groups and can provide recommendations on which ones are most suited to receive grants. Third, the short duration of the project in the newest districts, where implementation began only in September 2008, will not leave enough time for the PHA groups to successfully go through the entire granting process. Fourth, there are insufficient resources to meet the overwhelming demand for grants by PHA groups. In Year 2, for example, grants were provided to just 29 groups—but 419 concept notes were received, and 55 short-listed groups developed and submitted implementation plans and budgets. Finally, the three days devoted to capacity￾building workshops are not sufficient to adequately cover Alliance finance and management guidelines, data collection, record management, and reporting. Q7. Is the emphasis placed at the community, sub-county, and district levels appropriate? Qualitative evaluation data suggest that engagement is greatest at the community level, where PHA reside. At this level, the project is structured so that the NSAs and PHA group members interact with one another and the community for referrals and follow-ups, and they engage in many network activities, such as home visits, sensitization, referrals to health facilities, palliative care, and OVC support. This emphasis on the community level is appropriate because NSAs, PHA group members, and other community members look to each other for advice before going to health facilities, as was noted during most FGDs with PHA and non￾PHA. PHA group leaders reported that access to services for OVC and PHA has been improved by linkages created by the NSAs at the community level. They further noted an increase in the disclosure of HIV-positive status to family members, group members, service providers, and community members, as well as a decrease in stigmatization and discrimination at the community level, largely because NSAs developed trusting relationships with the community. Key informants and FGDs at all levels underscored that these improvements would not have been possible without the NSAs in the community providing testimonies, counseling, and referral services. The NSAs act as respected role models in the community and are looked to for guidance, support, and expert knowledge on HIV/AIDS. They [NSAs] have opened up and disclosed their status. So whatever they tell us, we know that they have been through this because we have seen them before as they are part of us. Secondly, they have a good approach when talking to people. They are like a parent talking to his/her own child. (Non-PHA—FGD) The project’s emphasis at the sub-county level is also important and appropriate because this is the point where the lowest level of health facilities, including those providing ART, is located. A major strength of engaging NSAs as the main facilitators at the sub-county level is that they provide appropriate linkages between community members and the health facilities. Lead PHA groups also support these linkages. They include representatives from all of the smaller PHA groups in their network cluster as well as sub-county community leaders and NSAs. FSO key informants reported that the emphasis placed at the sub-county level was essential to building the relationship between various services and thus facilitating referrals for PHA; it also supports liaisons between PHA groups so that they can learn from one another. Final Evaluation of the Project for Expanding the Role of Networks of PHA 19 Key informants reported that the least engagement occurs at the district level. The project’s intention was to focus on empowering groups at the sub-district level, where PHA networks were weak or non-existent, as opposed to the district level, where networks of PHA already existed. At its inception, however, the project sought political support and buy-in at the district level via partnership visits. Such partnership visits have continued, but much less frequently. DHOs and District PHA network coordinators work together to identify health centers where NSAs can be placed, and they attend opening and closing ceremonies at training workshops. FSOs provide updates on project activities to DHOs and HIV focal persons. However, the advisory committee has expressed concern that the current level of engagement at the district level is insufficient, pointing out that there is no ownership of the program by the districts, and data are not regularly shared with the districts. Q8. Has the placement of NSAs at health centers made a difference in increasing PHA access and utilization of HIV/AIDS services? The evaluation found that the placement of NSAs in the community and at the health facilities is one of the most successful components of the Networks Project. The Alliance’s monitoring data demonstrate that the NSAs make a significant contribution to increasing access to and utilization of HIV/AIDS services. For example, the uptake of PMTCT services in the mid-eastern districts increased from 1,264 at the start of the project activities in 2008 to 15,892 in 2009. The NSAs also play a critical role in providing HBC to their fellow members: NSAs and PHA group members provided approximately 70,000 PHA with palliative care in Project Year 2 (2007/08) and in Project Year 3 (2008/09) (International HIV/AIDS Alliance in Uganda 2008). Interviews with health workers at the HIV/AIDS service unit indicated that they noticed an increase in patient volume because of NSAs: We were lacking in community awareness but ever since these NSAs came on board, the numbers of people that now come for HIV services has really increased. In ART, the numbers have increased. (In-charge, HIV unit of a Health Center—Key Informant) NSAs compensate for a shortage of staff at the health centers and help process patients more quickly by: informing patients where to go for specific services within the health center; receiving patients who report to the ART clinic and directing them to the appropriate consultation room; conducting client education and counseling on HIV testing, nutritional issues, preparation for the initiation of ARVs, adherence to septrin ARVs and TB medication, and the transmission, prevention, and treatment of sexually transmitted infections; assisting with patient registration and weighing; counting and packing septrin pills; and locating client ART records and placing them in the appropriate consultation room for doctors and nurses. In addition, NSAs play a critical role in identifying ARV drug defaulters in the community and either referring them back to the facility or informing the health center if they have moved away, transferred to another facility, or died. Because NSAs live openly and positively with HIV, they have helped reduce the stigma that previously deterred PHA from seeking care. Their inherent empathy and credibility have also facilitated access to services. For example: 20 NSAs, unlike the health workers, are able to give the clients some practical experiences in their testimonies—not theory—which they find a lot more believable, getting very encouraged to take on their advice as a consequence. The public/community members find the NSAs much more approachable and are able to take them into their confidence. This has fostered confidence in the health system as a result. (District Health Officer— Key Informant) Interviews also revealed that NSAs emphasize prevention with positives (PwP), through which PHAs are encouraged to avoid re-infection and disclose their status. NSAs’ own public disclosure has encouraged those who test positive to disclose their status to spouses and family members. Q9. Are NSAs adequately supported with regard to ongoing training, support, and supervision? Before NSAs begin working at a health facility, they receive five days of training on counseling, nutrition, TB/HIV, home-based care, ART, and adherence counseling and education. Twice yearly refresher courses cover counseling skills, issues related to gender, HBC, disclosure, TB/HIV, OVC care, basic care package, caring for care givers, sexual and reproductive health services, and integration of sanitation with HIV/AIDS management. Key informant interviews with health workers, who are a primary source of support and supervision for NSAs on a day-to-day basis, revealed that NSAs are given adequate training initially. However, informants cited the need for more in-depth training on subjects such as counseling for children, couples, and discordant couples; bereavement and stress management counseling; TB care; and dispensing of septrin. Supervision and on-the-job training by health workers is another, less formal source of training for NSAs. Health workers also make efforts to fill perceived gaps by inviting NSAs to attend relevant training activities conducted for health facility staff. Yes, we train them on job. Any issue that they are looking in, we train them as they do it. It is not formal training. We also have [Continuing Medical Education] CMEs on Fridays. These NSAs are usually invited in relevant areas which could include nutrition, hygiene and they are given chance to present. (Health Worker—Key Informant) During their interviews, NSAs raised some additional issues that go beyond training and supervision. They complained that their allowances, which are supposed to cover transportation and other expenses, were insufficient, and also that they lacked raincoats, gumboots, umbrellas, and other gear to protect them against the weather. …especially in the rainy season we find it hard to travel. We don’t have things like gumboots [and] raincoats which sometimes make us miss work. In addition, even though we are given allowances, they are not enough, especially with the rising prices. (NSA—Key Informant) Final Evaluation of the Project for Expanding the Role of Networks of PHA 21 Some NSAs would like to receive greater financial benefits, whether in the form of salaries, compensation, or incentives, for their work. They point to the fact that they are working overtime as demand from health centers has increased and there are more PHA to cover at the community level. 3.3 Project Management, Coordination, and Staffing Q10. How has the requirement for significant scale-up affected the management and operational delivery of the project? Is the project staffing structure appropriate to support project development and scale-up? The staffing structure was well designed originally, but staffing levels have become inadequate as the project has scaled up. When the project expanded from 14 to 40 districts, the workload of field staff increased to unmanageable levels, and consequently the quality of management and operational delivery suffered. Delays in awarding small grants (see Q6) are a typical example of the negative consequences of the project’s rapid expansion. In response, the Alliance hired additional staff, including a Deputy Chief of Party, Grants Officer, Finance Assistant, Project Support Officer, and more FSOs. Currently, there are five FSOs, one Finance and Grants Manager, one Grants Officer, and one M&E Advisor, all of whom are overextended. Problems are exacerbated by the complex interrelationships between them. For example, when the Grants Manager became overwhelmed by the growing volume of grant applications and grantees, much of the work related to monitoring and managing the grants was reassigned to FSOs—adding to their already heavy responsibilities for collecting and collating M&E data and submitting it to the project office. FSOs reported feeling especially overworked and in need of more help. Since the expansion, each FSO is in charge of about eight districts and responsible for managing project activities, including small grants, for approximately 200 NSAs and 20 PHA groups. FSOs say there is simply not enough time for them to offer meaningful supervision and technical assistance over such a large area. One FSO described the problem in detail: My region is big. I take care of eight districts. The support supervision I would really want to offer to the NSAs and PHA groups, I am not able to offer it now. You may be needed in two districts at the same time. There are two staff. Me and the driver… So you find that I can only attend one of the two meetings and they will think that Alliance is not interested in such an activity. This is a big challenge because hopeful if USAID grants the year 2 districts, I don’t know because a group that has started getting grants needs more support than the others. Because I have a group in the district so if all are supported by groups I will have 27 groups. So if I am not able to offer support supervision to all the eight districts now before the granting process of the groups, I don’t know what it will be because they are supposed to comply to the grants specifications. (FSO—Key Informant) 22 Q11. How are partnerships developed and maintained at the different levels (national, district, and community)? At its inception, the Networks Project initiated partnerships at the national, district, and community levels to build relationships and gain support. However, these partnerships have primarily been maintained and intensified at the community level, because that is where the project places most emphasis. At the national level, the project has developed strategic partnerships with UAC, MOH, and key national level networks and forums of PHAs such as the National Forum for PHA Networks in Uganda (NAFOPHANU), National Community of Women Living with HIV/AIDS in Uganda (NACWOLA), Positive Men’s Union (POMU), and many other PEPFAR and non-PEPFAR programs. The Alliance entered into a memorandum of understanding with the MOH and UAC, in which the MOH agreed to provide a favorable policy environment for the project at its facilities. Some of the national partnerships with NGOs strengthened district and community partnerships. After initial partnership development efforts, however, little activity has taken place at the national level. Nonetheless, the positive impact of the NSAs on service delivery at health facilities has led to increased recognition of the added value of PHA in HIV/AIDS service delivery and contributed to a broader debate on task-shifting to lay cadres of health care workers, including PHA. At the district level, the Networks Project initiated the partnership-building process with technical, political, and NGO and community-based organization leadership. Interviews with FSOs and DHOs indicated that the project introduced the community engagement and network models to district stakeholders during meetings held at the beginning of the project. They also reported that partnerships at the district level are maintained through the several activities: FSOs and other Alliance staff make visits to the district to meet with the district medical officer and HIV/AIDS focal person and review the district HIV/AIDS strategic plans; DHOs help identify health centers where the project can place NSAs; district officials attend opening and closing ceremonies at training workshops; and in some cases, FSOs share offices with partners, such as the Church of Uganda Kabale Diocese and local NGOs. Although ongoing partnerships are being maintained with district health offices, their intensity has declined over time. At the community level, respondents report that partnerships have been developed through a community engagement approach. As described earlier, NSAs and PHA group members are respected within the community. They invite representatives from local organizations, the local council, and other community leaders to meetings, where they discuss their vision and report on their activities. Community leaders provide input into project activity planning as part of the community engagement. In addition, health facility in-charges verify and sign the NSAs’ reports of routine service data after checking that the NSAs actually delivered the services. In some cases, churches and mosques provide places where PHA groups can meet. However, in other places the level of support for the PHA groups from community leaders is still minimal. Final Evaluation of the Project for Expanding the Role of Networks of PHA 23 Section Four: Discussion The goal of the PHA Networks Project in Uganda is to enhance the involvement of PHA in service delivery and policy formulation, in order to increase access to and utilization of HIV/AIDS prevention, treatment, and care and support services (International HIV/AIDS Alliance in Uganda 2007). This is a clear expression of the GIPA and MIPA principles discussed in Section 1.3.1. GIPA recognizes the right of people living with HIV/AIDS to active, free, and meaningful participation in decision-making processes that affect their lives, including the right to self-determination. It also aims to enhance the quality of the effectiveness of the AIDS response (UNAIDS 2007; Morolake 2009) MIPA invokes PHA as active agents of change rather than passive recipients of services. The PHA Networks Project in Uganda has clearly contributed to greater and meaningful involvement by PHA. Of the seven possible types of involvement described by UNAIDS (UNAIDS 2007), the Networks Project emphasized and was most successful in the following four categories:  Personal well-being. PHA take an active role in decisions about treatment, self￾education about therapies, opportunistic infections and adherence, and positive prevention.  Treatment roll-out and preparedness. PHA educate others on treatment options, side effects and medication adherence, and serve in the capacity of home-based and community health-care workers.  Leadership and support group networking and sharing. PHA seek external resources, encourage participation of new members of support groups, or simply participate by sharing their experiences with others.  Advocacy. PHA advocate for resource mobilization for PHA networks and policy change. The project did less to advance the remaining three categories of PHA involvement. In the future—if resources allow—the project may want to explore promoting PHA involvement in:  Campaigning and public speaking. PHA are spokespersons in communication campaigns or speakers at other public events.  Policymaking. PHA participate in development and monitoring of HIV-related policies at all levels. The final category proposed by UNAIDS is most appropriate for Alliance staff or top PHA leaders:  Program development and implementation. PHA participate in the governance of international HIV/AIDS organizations, and in the design, implementation, monitoring and evaluation of programs. Another way to analyze the project’s achievements with regard to GIPA and MIPA is based on a typology of PHA involvement in the delivery of prevention, care, and support services 24 developed by the Population Council/Horizons Program and the International HIV/AIDS Alliance (2003). This typology describes five potential areas of involvement in service delivery (see Annex I). PHA can: (1) use services, (2) support activities and services, (3) deliver HIV/AIDS services, (4) plan and design services, and (5) manage or lead organizations, or influence policymaking and strategic planning processes. The intensity of their involvement ranges from access (if PHA are simply using services), to inclusion (if PHA help with an organization’s non-HIV related tasks and deliver services informally or occasionally), to participation (if PHA help plan and deliver services on a regular basis), to greater involvement (if PHA have influence over day-to-day management, policymaking, and strategic planning). A study of PHA involvement in four countries (Burkina Faso, Ecuador, India, and Zambia) found that PHA usually achieve only the lowest two levels of involvement—access and inclusion (Horizons Program and International HIV/AIDS Alliance 2003). In contrast to the situation in these countries, the PHA Networks Project in Uganda has moved toward higher levels of involvement. The project has achieved participation, by:  Supporting almost full-time NSAs to increase referral linkages and provide counseling services in health facilities; and  Building the capacity of PHA network groups and offering them small grants to design, manage, and evaluate their own projects. Greater involvement in management, policymaking, and strategic planning remains on the horizon. To reach this next level, the Networks Project may want to explore interventions that engage PHA in professional and leadership positions, for example, as doctors and managers. Religious leaders who are living with HIV/AIDS could also play an important role here. The PHA Networks Project’s effort to build the capacity of PHA groups to plan and manage their own activities through networks has been a major component of the effort to increase GIPA and MIPA. The project learned from and built on an approach pioneered by the Alliance in Zambia in 2004–2006, which showed success (Samuels 2008). In Uganda, creating clusters of PHA groups—and building the capacity of the coordinating committees that led each cluster—enabled the groups to share experiences and expertise and solicit technical assistance. While the mechanisms created to support PHA groups are working relatively well, more and longer-term support is needed to make the approach sustainable. In addition, more effort is needed to document and share these best practices, so that others may learn from the experience. The small grants awarded to PHA groups in Uganda provide another way to increase MIPA by enabling them to plan and implement new activities. There has been little experience with small grants to support PHA groups in other countries, and the most efficient and effective ways to administer the grants are not well documented. In Thailand, PHA groups have tried various income-generation schemes including, for example, contracting with a Japanese company to produce bags, scarves, and clothes for export (International HIV/AIDS Alliance in Uganda 2006). Results were mixed. The grants program in Uganda has proven much more successful, probably because of the Alliance’s capacity-building efforts in proposal writing and grant management. Participating PHA groups have learned how to plan activities within a given budget and timeframe and implement and monitor activities effectively—skills that they can continue to apply in the future as they become increasingly involved in the Final Evaluation of the Project for Expanding the Role of Networks of PHA 25 HIV/AIDS response. To overcome current bottlenecks in the grants process and achieve its full potential, however, the Alliance needs to explore solutions to ensure that there are adequate staffing levels for the grant management process. NSAs have played a key role in Uganda as referral links between the community and health facilities and as counselors. Much of their success stems from their openness about their own HIV status and direct experience with the illness, which has given them the ability to break down barriers with PHA. Their position is modeled on and combines the roles of two different types of PHA workers engaged by the Alliance’s earlier Zambia project: Treatment Support Workers, who were employed in ART clinics full time, and Treatment Mobilizers, who acted as full-time links between community partners and the Treatment Support Workers in the ART clinics (Samuels 2008). Combining the two roles strengthened the linkage between the community and the clinic, and also provided for the kind of follow-up essential to a true continuum of care. Experiences in other African countries with enhancing MIPA and GIPA have found that success is more likely when communities are proactively involved in ensuring their own well-being and in responding to the HIV/AIDS epidemic (Horizons Program and International HIV/AIDS Alliance 2003). The involvement of PHA in project development and implementation, as well as policy and advocacy, can improve a project’s relevance, acceptability, and effectiveness. This involvement allows PHA and affected communities to make a powerful contribution by enabling individuals and communities to draw on their life experiences in responding to HIV. In turn, their contribution reduces stigma and discrimination by making the services more acceptable to the PHA community. It also increases the effectiveness and appropriateness of the response to HIV. The PHA Networks Project in Uganda is contributing to the effort to find the most appropriate and effective ways to increase MIPA and GIPA and achieve these goals. 26 Section Five: Conclusions and Recommendations  Increase resources to support the PHA Networks Project: Evidence from the evaluation suggests that the PHA Networks Project has increased access to and utilization of home- and community-based services (including palliative care, OVC support, and wraparound services), increased referrals to health facilities, and increased the involvement of PHA in service delivery. These accomplishments have been achieved within the context of the national strategic plan for HIV/AIDS and the USG Country Operational Plan. However, a lack of financial and human resources has hindered the effective implementation and geographical expansion of the project’s activities. USAID should increase its support for the PHA Networks Project in order to expand its coverage and deepen its impact.  Assess the revised M&E tools: The Alliance recently introduced new M&E tools to streamline data collection and reporting and also to capture the qualitative information needed to understand and explain the achievements documented by the quantitative evaluation. The Alliance needs to determine whether the new tools have increased the efficiency of the M&E system and improved the documentation of lessons learned and best practices.  Build capacity for monitoring and evaluation: The Alliance should increase both the number and capacity of M&E staff and FSOs, who are largely responsible for data collection and analysis. They, along with the NSAs and PHA groups, need additional training and mentoring in data collection and analysis. The Alliance should also encourage M&E staff members to work more closely with FSOs to facilitate the process of data collection, analysis, and reporting.  Document and share lessons learned and best practices: The Alliance should encourage information sharing and feedback between NSAs, PHA groups, district and national level partners, and other stakeholders. Ideally, the NSAs and PHA groups should meet monthly or bi-monthly to share experiences and learn from one another as a social network. Because FSOs have additional skills and oversee multiple PHA groups, they are in a good position to examine and assemble the information on lessons learned and best practices, provide feedback to the NSAs and PHA groups, and submit this information to the M&E department.  Build capacity for grant management: The Alliance should increase the number of Grant Officers, ideally one per region, to foster efficient and effective monitoring and management of grants and to ease the workload of FSOs. PHA groups that are managing grants should be offered additional training in areas such as proposal writing, income generation, M&E, management of bank accounts, and generation of financial documents. This kind of training could strengthen the ability of these groups to compete for funds from other agencies, which would help foster sustainability.  Address the concerns of NSAs: NSAs represent the Alliance’s most successful achievement in involving PHA in HIV/AIDS service delivery and access to and utilization of services. To ensure that the NSAs remain engaged in project activities, the Alliance should search for the additional resources needed to respond to the Final Evaluation of the Project for Expanding the Role of Networks of PHA 27 requests made by NSAs during interviews, notably for an increase in their allowances and for protective gear.  Continue to build the capacity of NSAs: NSAs’ knowledge and skills need to be continually updated so that they keep abreast of new developments in HIV/AIDS. This gap could be addressed through peer training of trainers—training a few NSAs to function as master trainers or resource persons who, once trained, could in turn train their fellow NSAs. This training cycle could be repeated regularly, for example, bi￾annually.  Provide more support to the Alliance’s field offices at the district/regional level: Staff shortages at the district and regional levels are undermining the effectiveness of the project. Notably, FSOs’ workloads have increased—both in terms of the volume of work and the diversity of activities for which they are responsible—to an unsustainable level. To properly support the extended role of FSOs, the Alliance needs to hire a second FSO at each field office, or an assistant to help the existing FSO, so that one person can remain in the office while the other is in the field. Adding another Grant Officer and M&E Officer would also lessen the FSOs’ workload.  Ensure the sustainability of community engagement: The Alliance has successfully facilitated active community engagement and empowerment. The PHA groups operate at the community level and have developed strong partnerships through community engagement activities, such as home visits, sensitization activities, referrals to health facilities, and OVC support. As a result, community members feel a sense of ownership and believe that they are addressing real problems or needs in their own communities. The Alliance should capitalize on this achievement by seeking to involve local authorities, NGOs, and CBOs operating in each district in community activities sponsored by PHA groups.  Strengthen advocacy and partnerships with district officers: The active engagement of DHOs in the PHA Networks Project is important to ensure its proper functioning and its sustainability because the NSAs need the support of the DHO to work at public health facilities. At the inception of the Networks Project, heavy buy-in of district officials was attained. However, these partnerships and linkages have not been maintained over time. The Alliance needs to reestablish district-level support through advocacy and more frequent contact with district managers. For example, the FSO could attend some of the quarterly District Health Team meetings and regularly send the Alliance’s service delivery reports to the DHO, so that district offices could keep abreast of project activities and offer support where possible. PHA liaison groups should also participate in district and sub-county meetings.  Consolidate project efforts in current districts before expanding: The Alliance’s human resources have not grown quickly enough to match the project’s expansion to 40 districts and replication in another 12 districts. This has undermined the smooth operation of some activities, such as grant management, evaluation, documentation of best practices, and partnering with district offices. The Alliance should temporarily stop its expansion efforts and focus instead on consolidating and strengthening project efforts in current districts by hiring additional staff and building the skills of current staff, strengthening the M&E and grant management systems, and improving the training and supervision offered to NSAs and PHA groups. When and if increased 28 financial and human resources become available, the Alliance may be able to roll out its successful interventions to a larger geographic area. The Alliance has succeeded in empowering PHA to provide services effectively at the facility and community levels. It is a rare example of putting the principle of MIPA into practice in a systematic manner and on a countrywide basis. The approach and the lessons learned should be documented and shared widely in Uganda and elsewhere. Final Evaluation of the Project for Expanding the Role of Networks of PHA 29 References Glaser, B.G. and Strauss, A. L. 1967. The Discovery of Grounded Theory. Chicago, IL: Aldine. Horizons Program. 2002. ―Greater involvement of PLHA in NGO service delivery: Findings from a four-country study,‖ Horizons Research Summary. Washington, D.C.: Population Council and International HIV/AIDS Alliance. Horizons Program and International HIV/AIDS Alliance. 2003. The Involvement of People Living with HIV/AIDS in Community-based Prevention, Care and Support Programs in Developing Countries. A Multi-country Diagnostic Study. Brighton, UK: Population Council and International HIV/AIDS Alliance. International HIV/AIDS Alliance in Uganda. 2008. Expanding the Role of Networks of People Living with HIV/AIDS in Uganda. Annual Report: July 2007–June 2008. Kampala: International HIV/AIDS Alliance in Uganda. ———. 2007. Expanding the Role of Networks of People Living with HIV/AIDS in Uganda: First Year Project Report (July 2006–June 2007). Kampala: International HIV/AIDS Alliance in Uganda. ———. 2006. Supporting Community Actions on AIDS in Developing Countries: Positive People’s Groups: the Story of People Living with HIV Mobilizing to Improve Their Lives in Upper Northern Thailand. Kampala: International HIV/AIDS Alliance in Uganda. ———. no date. Expanding the Role of PHA Networks Program: Detailed Results Framework. Kampala: International HIV/AIDS Alliance in Uganda. Ministry of Finance, Planning and Economic Development [Uganda]. 2007. Poverty Eradication Action Plan Report. Kampala: MoFPED. Ministry of Health [Uganda]. 2006. Uganda HIV/AIDS Sero-behavioural Survey 2004–05. Kampala: MOH. (March) Morolake, O., Stephens, D., and Welbourn, A. 2009. ―Greater involvement of people living with HIV in health care,‖ Journal of International AIDS Society 12: 4. Riley, J. 1990. Getting the Most from Your Data. London: King’s Fund. SAfAIDS. no date. MIPA: Mainstreaming the Principle of Meaningful Involvement of People Living with or Affected by HIV/AIDS (MIPA) into HIV/AIDS Programming. Available at: http://www.safaids.net/files/MIPA_Brochure.pdf. Samuels, Fiona et al. 2008. ―Engaging communities in supporting HIV prevention and adherence to antiretroviral therapy in Zambia,‖ Horizons Research Summary. Washington, DC: Population Council. 30 Uganda AIDS Commission. 2006. The National HIV & AIDS Strategic Plan 2007/08– 2011/12. UNAIDS. 2007. ―The greater involvement of people living with HIV (GIPA),‖ Policy Brief. Geneva: UNAIDS. ———. 1999. ―From principle to practice: Greater involvement of people living with or affected by HIV/AIDS (GIPA),” Best Practice Key Material. Geneva: UNAIDS. United States Agency for International Development (USAID)/Uganda and Marie Stopes International (MSI). 2008. Special Study of HIV/AIDS Service Networks in Uganda. United States Agency for International Development (USAID). Cooperative Agreement 617- A-00-06-00008-00. Washington, D.C.: USAID. Final Evaluation of the Project for Expanding the Role of Networks of PHA 31 Section Six: Annexes Annex A : Key Informant Guide for Senior Staff and Project Cognizant Technical Officer (CTO) A] Project Design 1a. How is the project contributing to the achievement of GoU and USG goals regarding: a) Increased access to HIV prevention b) Comprehensive continuum of care c) Treatment and support 2. Is the USG support through the Networks Project a recommended approach to continue supporting PHAs in order to achieve GoU and USG goals regarding: a) Increased access to HIV prevention b) Comprehensive continuum of care c) Treatment and support N.B. Probe to show evidence that the project is or is not the best way to achieve GoU and USG goals (i.e., compared with other approaches). 3. Does the project design and structure adequately support and facilitate achievement of the desired results? Probe: How a) community engagement, b) stigma and discrimination, c) positive prevention, and d) capacity building are blended. Describe the project structure and show evidence for whether staffing, support systems, and budgets are adequate. B] Project Performance 4a. Is the Networks Project on track to achieving its overall objectives and results? Probe: To what extent has the project achieved the project results? 4b. What are the strengths affecting achievement of results? 4c. What are the weaknesses affecting achievement of results? 4d. What are the opportunities affecting achievement of results? 4e. What are the threats to the project? 5. Are the systems and mechanisms for documenting lessons and good practices in terms of the network model effective? Probe: Describe examples of use of learning to improve project. How does the design of the M&E system support documentation of lessons learned? 32 6a. How is the process of providing grants to PHA groups organized (please describe the process: how it is administered and evaluated, how grants fit within the model)? 6b. What are the a) Strengths of the process? b) Weakness of the process? 6c. What suggestions/recommendations do you propose for improving the grant-making process? 6d. i. Describe the level of engagement placed at the community, sub-county, and district levels. ii. Is the emphasis placed at the community, sub-county, and the district levels appropriate? (Why/why not?) iii. What are the gaps at each of these levels? 7. Has the placement of NSAs at Health Centers (H/Cs) made a difference in increasing PHA access and utilization of HIV/AIDS services (if so, how)? Probe: Effectiveness of NSAs as an alternative source of human resources for health? Adequacy of the support given to NSAs (training, support supervision, etc.)? C] Project Management, Coordination, and Staffing 8a. How has the requirement for significant scale-up affected the management and operational delivery of the project? Probe: Extent to which management and operational delivery of the project differs between start-up and expansion districts, and between the start-up phase and the expansion phase? 8b. Describe the strengths and weaknesses of the staffing structure and how it relates to project scale-up. 8c. Is the project staffing structure appropriate to support project development and scale-up? Probe: Risks associated with rapid scale-up of this approach and how these can be mitigated against? 9. How are partnerships developed and maintained at the different levels (national, district and community)? Probe: How partnerships are managed with US Government, GoU, CSOs, and other stakeholders? How the project ensures access to wraparound services such as malaria, nutrition, reproductive health, and family planning? Final Evaluation of the Project for Expanding the Role of Networks of PHA 33 10. Any overall recommendations to improve the performance of the project lessons learned, best practices)? 34 Annex B : Key Informant Guide: Service Provider IDI Guide District Health Officers/service providers:  Doctors  Nurses  Counselors 1. Activities at health facility a) What does the NSA do at your health facility? (Probe) Support VCT, PMTCT, ARV service delivery? b) How do you feel about the role and activities of NSAs? c) What challenges do you think they have in carrying out their activities at this facility? d) How are these challenges usually overcome? 2. Training a) Do you provide any training to the NSA? If yes, what sort of training have you provided to them? b) Do you think the training they received from the ―Alliance‖ is adequate to help them do the work they are expected to do at this facility? c) What else do you think they should be trained in? 3. Supervision/on-going support a) Who is the immediate supervisor of the NSA at this facility? Whom do they report to? How often do they meet with their supervisor? b) Is there any support you give to NSA? If so, what support? c) What other support do you think they need? 4. Achievements a) How do you feel the NSA has helped you to achieve provision of health services at your facility? b) How do you think they have benefited from working with you at this facility? 5. Suggestions a) What suggestions do you have to improve the work of NSA at your facility? b) What suggestions do you have for the ―Alliance‖? Final Evaluation of the Project for Expanding the Role of Networks of PHA 35 Annex C : Key Informant Guide: Field Support Officers 1. Activities at community a) What do the NSA do in the community? b) How do you feel about the role and activities of NSAs in the community? c) How are partnerships developed and maintained at the different levels (district and community)? d) Is the emphasis placed at the community, sub-county, and the district level appropriate? e) What challenges do you think they have carrying out their activities in the community? f) How these challenges are usually overcome? 2. Activities at health facility a) What do the NSA do at your health facility? (Probe) Support VCT, PMTCT, ARV service delivery? b) How do you feel about the role and activities of NSAs? c) What challenges do you think they have carrying out their activities at this facility? d) How these challenges are usually overcome? 3. Granting to PHA groups a) How do you carry out the granting process? b) What are the strengths and weaknesses of the process? c) What suggestions do you have to improve the granting process 4. Suggestions a) What suggestions do you have to improve the work of NSA (Probe—in the community and at the health facility)? b) Describe the strengths in your region of operation. c) What are the weaknesses affecting achievement of results in your region of operation? d) In your view, what are the opportunities and threats to the project? e) What suggestions do you have for the ―Alliance‖? 36 Annex D : Key Informant Guide: Network Support Agents (NSA) Introduction 1. Activities at health facility a) What do you do at health facility? (Probe) Support VCT, PMTCT, ARV and delivery? b) What are the feelings of the other health facility staff toward your role and activities? c) What challenges do you have carrying out your activities at facility? d) How do you overcome these challenges? 2. Activities in community including PHA groups a) What do you do at community level? (Probe) Support individuals and families living with HIV/AIDS with basic care needs, appropriate referrals? b) How do you work with different PHA groups in the community? c) What challenges do you have carrying out your activities in community? d) How do you overcome these challenges? 3. Training a) What training have you received from the ―Alliance‖? b) Is it adequate to help you to do the work? c) What else do you want to be trained in? 4. Supervision/on-going support a) Who is your immediate supervisor? Whom do you report to? How often do you meet with your supervisor? (in the community and at the health facility) b) What support do you get from ―Alliance‖? Is it adequate for you to do the ―Alliance‖ work? c) What other supports would you like to have? 5. Achievements (personal and work) a) What do you feel you have achieved working as a NSA? b) (Probe) As an individual and at a member of community? 6. Suggestions a) What suggestions do you have to improve your work as a NSA? b) What suggestions do you have for PHA groups? c) What suggestions do you have for the―Alliance‖? Final Evaluation of the Project for Expanding the Role of Networks of PHA 37 Annex E : Key Informant Guide: PHA Group Leaders 1. Introduction a) As a group leader for the PHA group, what are your main roles and responsibilities? b) How long have you been a leader of this group? c) How were you elected? 2. Training and technical support a) What types of training have you got from the ―Alliance‖ or NSA? b) How useful was this training? How are you using what you learned from the training? c) Have you as a group leader been visited by the ―Alliance‖ staff? d) What was it about? 3. Activities and meaningful participation a) What service/activities is your group providing that are supported by ―Alliance‖? (Probe) Community- and family-based HIV/AIDS counseling and testing (HCT)? Couple counseling and testing? Community outreach? Putting in place the post-test clubs? b) What OVC service do you provide as a group? c) How do you as a leader of the group participate in the group’s activities? d) How do you benefit being a leader of this group? e) How are you a as leader of the group involved in the decision-making process of the group? 4. Linking with NSAs and service referral a) Do you have links with NSAs? How often? What do you discuss with the NSAs? b) How do you work with NSAs for referral? 5. Networking with other PHA groups a) Have you met with other PHA groups in your sub-district? Which ones? What do you do when you meet other groups? b) How is the coordination managed? How is the flow of information in their network? 38 6. Suggestions a) What are the main challenges you face as a group leader? b) What challenges are faced by the group you head in carrying out the ―Alliance‖ activities? (Probe) Community- and family-based HIV/AIDS counseling and testing (HCT)? Couple counseling and testing? Community outreach? Putting in place the post-test clubs? c) How do you overcome these challenges? d) What have been the major achievements you think you have made as a leader of the PHA group? What has worked well? Why? e) How can those activities you are carrying out be improved? What recommendations/ suggestion do you have to improve your activities? f) What suggestions do you have for the ―Alliance‖? Any other information that you want us to know about your group’s activities? Final Evaluation of the Project for Expanding the Role of Networks of PHA 39 Annex F : Focus Group Discussions: PHA Group Members Introduction 1. Group composition and organization a) What is the composition of this group (total membership by gender, composition of the leadership by gender)? b) How many years have you met as a group? c) What are the main activities of this group? d) How is the leadership of this group elected and how frequently does the leadership change? e) Does the group have rules and regulations? If so, what are theses rules and regulations? 2. Training and technical support a) What types of training have you got from the ―Alliance‖? When? Describe the content of training. (Probe) Community- and family-based HIV/AIDS counseling and testing (HCT)? Couple counseling and testing? Community outreach? Putting in place the post-test clubs? b) How useful was this training? How are you using what you learned from the training? c) Have you as a group been visited by the ―Alliance‖ staff? If so, what was it about? 3. Activities and meaningful participation a) What service are you providing as a group that are supported by the ―Alliance‖? (Probe) Community- and family-based HIV/AIDS counseling and testing (HCT)? Couple counseling and testing? Community outreach? Putting in place the post-test clubs? b) What OVC service do you provide as a group? c) How do you as ordinary members of the group participate in the group’s activities? d) How do you benefit by being a member of the group? e) How are you as ordinary members of the group involved in the decision-making process of the group? 4. Linking with NSAs and service referral a) Are any of your members NSAs? If so, how many? How were they selected? Are you satisfied with the selection criteria and selection procedures? b) How do you work with NSAs for referral? 5. Networking with other PHA groups a) Have you met with other PHA groups in your sub-district? Which ones? What do you do when you meet other groups? b) How is the coordination managed? How is the flow of information in their network? 40 6. Suggestions a) What are the main challenges this group is facing to carry out the ―Alliance‖ activities? (Probe) Community- and family-based HIV/AIDS counseling and testing (HCT)? Couple counseling and testing? Community outreach? Putting in place the post-test clubs? b) How do you overcome these challenges? c) What have been the major achievements of this group? What has worked well? Why? d) How can those activities you are carrying out be improved? What recommendations/ suggestion do you have to improve your activities? e) What suggestions do you have for the ―Alliance‖? Any other information that you want us to know about your group’s activities? Final Evaluation of the Project for Expanding the Role of Networks of PHA 41 Annex G : Focus Group Discussions: Non-PHA Community Members Introduction 1. Which community groups target people with HIV and AIDS, including orphans and vulnerable children? 2. If PHA groups supported by Alliance are not mentioned, probe for it. 3. What are the main activities of this/these groups? 4. How does the community participate in the group activities (probe any benefit from the group)? 5. Have you been approached by NSA in your community? What do they usually do/tell you? Probe: Sensitizations (VCT, Seek for treatment, stigma, and discriminations) 6. What do you like about the groups/NSA? 7. What don’t you like about the groups/NSA? 8. How can the activities they are carrying out be improved? What recommendations/suggestions do you have to improve those activities? 9. Is there any other information that you want us to know concerning the groups and HIV and AIDS activities in your community? 42 Annex H : PHA Networks Project Staffing Structure Chief of Party Finance and Grants Manager Drivers Program Support Officer Deputy Chief of Party (Programs) Accounts Officer and Assistant International HIV/AIDS Alliance Secretariat Advisory Committee Technical M&E Advisor Grants Officer Advisor Field Support Officers Office Helper Logistician Final Evaluation of the Project for Expanding the Role of Networks of PHA 43 Annex I : Typology of PHA Involvement Types of PHA Involvement Areas of Involvement for PHA PHA use the services of the NGO PHA support activities and services of the organization PHA deliver HIV/AIDS services PHA plan and design services PHA manage the organization, run or influence the policymaking and strategic planning process Access Yes No No No No Inclusion Yes Yes, for non￾HIV-related tasks Yes, informally; e.g., as occasional volunteers No No Participation Yes Yes Yes, formally; e.g., as part￾time or full-time service providers (staff or regular volunteers) Yes, they usually plan the services they deliver, and may also be consulted on other services No Greater Involvement Yes Yes Yes, formally; e.g., as part￾time or full-time service providers (staff or regular volunteers) Yes, they plan the services they deliver. They may also contribute directly to planning of other services Yes; e.g., as program or project coordinators, directors, trustees