MID-TERM REVIEW OF THE NORTHERN UGANDA MALARIA, AIDS AND TUBERCULOSIS (NUMAT) PROGRAM   FINAL EVALUATION REPORT August 2009 This document was produced for review by the United States Agency for International Development (USAID/Uganda). It was prepared by Gary Leinen, Daniel Kibuuka Musoke, Fred Bagenda and Edgar Mulogo from The Mitchell Group, Inc. for the Uganda Monitoring and Evaluation Management Services, herein referred to as UMEMS, number 617-C-00-08-00012-00 1 ACKNOWLEDGEMENTS The Mid-Term Review (MTR) of the Northern Uganda Malaria, AIDS and Tuberculosis (NUMAT) program was made possible with the support from the United States Agency for International Development (USAID) through the Uganda Monitoring and Evaluation Management Services (UMEMS), which is managed by The Mitchell Group, Inc. (TMG), located in Washington, D.C. This report was written by Gary Leinen, Dr. Daniel Kibuuka Musoke, Dr. Edgar Mulogo, and Dr. Fred Bagenda. The authors are very grateful to USAID staff (Rhona Walusimbi, Andrew Kyambadde and Okello James for their support and active participation in the review), UMEMS staff (Patricia Rainey, Nestore Jalobo and Caroline Kasabiti who provided excellent logistical support in Uganda and work space for the team in Kampala), Obol James and Dr. Onyai Patrick for their outstanding work in planning, conducting and timely translating all the focus group discussions. Finally, the team wishes to thank The Mitchell Group, Inc., home office (TMG/HQ), especially Scott Bleggi, Lans Kumalah, and Alex McKenzie for the excellent logistical support, relevant and useful background documentation, and swift contracting actions that located the team leader and got him to Kampala in record time. The support from TMG/HQ assured that the mid-term review could be mobilized and completed on time. We would especially like to thank the participants in the review—the staff of the NUMAT program, the districts, health facilities, Civil Society Organizations (CSOs) and the communities that participated in the interviews and the Focus Group Discussions (FGDs). 2 TABLE OF CONTENTS ACKNOWLEDGEMENTS ......................................................................................................... 1 TABLE OF CONTENTS .............................................................................................................. 2 LIST OF ACRONYMS AND ABBREVIATIONS ..................................................................... 3 EXECUTIVE SUMMARY ........................................................................................................... 6 1. INTRODUCTION ................................................................................................................... 9 1.1 THE SOCIAL AND POLITICAL CONTENT IN NORTHERN UGANDA .............. 9 1.2 PROJECT BACKGROUND ........................................................................................ 9 1.3 EVALUATION PURPOSE ........................................................................................ 10 1.4 EVALUATION METHODOLOGY .......................................................................... 11 1.5 STUDY LIMITATIONS ............................................................................................ 11 2 EVALUATION FINDINGS ................................................................................................... 12 2.1 ACHIEVEMENT OF TARGETS ............................................................................... 12 2.2 COORDINATION STRUCTURES AND PARTNERSHIPS .................................... 13 2.3 CAPACITY BUILDING ............................................................................................ 17 2.4 PROGRAM RESPONSE TO A CHANGING ENVIROMNENT ............................. 27 2.5 SUSTAINABILITY .................................................................................................... 30 CONCLUSIONS ......................................................................................................................... 34 RECOMMENDATIONS ............................................................................................................ 35 ANNEX A: BIBLIOGRAPHY ................................................................................................... 37 ANNEX B: SCOPE OF WORK FOR THE MID-TERM REVIEW ....................................... 38 ANNEX C: FRAMEWORK FOR THE EVALUATION OF KEY QUESTIONS ................ 42 ANNEX D: LIST OF PEOPLE INTERVIEWED .................................................................... 44 ANNEX E: KEY COLLABORATING PARTNERS ............................................................... 47 ANNEX F: DATA COLLECTION TOOLS ............................................................................. 49 3 LIST OF ACRONYMS AND ABBREVIATIONS AIC AIDS Information Center AIDS Acquired Immunodeficiency Syndrome ANC Ante-Natal Care ART Anti-Retroviral Therapy ARV Anti - Retroviral AVSI Association of Volunteers International Service BCA Behaviour Change Agent BCC Behaviour Change Communication CAO Chief Administrative Officer CB-DOTS Community Based - Directly Observed Treatment, Short Course CBO Community Based Organisation CMD Community Medicine Distributor COH Channels of Hope CORP Community Owned Resource Persons CSO Civil Society Organisation CSW Commercial Sex Worker DAC District AIDS Committee DAT District AIDS Task Force DDMC District Disaster Management Committees DHT District Health Team DLFP District Laboratory Focal Person DPTC District Planning Technical Committee DQA Data Quality Assessment DR-SCE Decentralized Response Self Coordinating Entity EID Early Infant Diagnosis FAO Food and Agricultural Organization FBO Faith Based Organization FGD Focus Group Discussion FSG Family Support Groups GoU Government of Uganda HAART Highly Active Anti-Retroviral Therapy HBMF Home Based Management of Fever HC Health Centre HCT HIV Counselling and Testing HIV Human Immunodeficiency Virus HMIS Health Management Information System 4 HRH Human Resources for Health HRM Human Resource Management HUMC Health Unit Management Committees HW Health Workers IDP Internally Displaced People IEC Information Education and Communication IGA Income Generating Activities IP Implementing Partner JCRC Joint Clinical Research Centre JSI John Snow, Inc. LG Local Governments LQAS Lot Quality Assurance Surveys M&E Monitoring and Evaluation MARP Most at Risk Populations MAT Malaria AIDS and Tuberculosis MOH Ministry of Health MoLG Ministry of Local Government MoPS Ministry of Public Service MTR Mid-Term Review NGO Non Governmental Organisation NSA Network Support Agents NSP National HIV/AIDS strategic plan NUMAT Northern Uganda Malaria, AIDS, and TB Project OI Opportunistic Infection OPD Out Patient Department OVC Orphans and Vulnerable Children PHA People Living with HIV/AIDS PLHIV People Living with HIV/AIDS PMTCT Prevention of Mother to Child Transmission PRDP Peace, Recovery and Development Plan PTC Post Test Club PY Project Year RH Reproductive Health SAC Sub-County AIDS Committee SAT Sub-County AIDS Task Force SGBV Sexual and Gender Based Violence SPEAR Supporting Public Sector Workplaces to Expand Action and Responses against HIV/AIDS TASO The Aids Support Organisation TB Tuberculosis TOT Trainer of Trainers 5 UAC Uganda AIDS Commission UMEMS Uganda Monitoring and Evaluation Management Services UN United Nations UNAIDS United Nations Join Programme on HIV/AIDS UNICEF United Nations Children's Fund USAID United States Agency for International Development USG United States Government VHT Village Health Teams WHO World Health Organisation WV World Vision 6 EXECUTIVE SUMMARY USAID Uganda’s primary purpose for undertaking a mid-term review of the NUMAT program was to determine whether or not the program is on track to achieve its objectives within the existing time frame and funding parameters, and to ascertain if any changes in program or management strategies are needed for the remaining two years of the project that would increase its impact in Northern Uganda in light of the changing political and social context. The review was broad-ranging, covering questions of the NUMAT program’s overall programmatic effectiveness, whether it is addressing capacity-building and sustainability issues in the dynamic environment of Northern Uganda, and if it is effectively improving coordination and partnership. A team of six evaluators composed of an international team leader, three local consultants (public health physicians) and two research assistants conducted the Mid-Term Review (MTR) between June 29th and July 24th 2009. The review was conducted in four districts (Gulu, Pader, Lira and Oyam), which were specifically selected from the list of nine NUMAT districts based on sub-regional representativeness. The study population included USAID staff, NUMAT staff, District Local Government officials, key civil society partners and beneficiaries for both health facility based and community based services. The review team used largely qualitative methods of data collection to obtain information on which this report is based. As a key limitation, the time allocated to this evaluation was relatively short; therefore, the review team could not cover every aspect of the program, including visiting all the nine districts. The findings in this report are organized along the evaluation questions rather than by methodology in order to bring multiple approaches to bear on each question without allowing one source to dominate the process. The key findings and recommendations from the review are presented below: 1) This review concludes that the program is largely on track to achieve its targets with the exception of PMTCT indicators and TB tests performed. The low achievement on PMTCT indicators is attributed to the low proportion of institutional deliveries and inadequate follow￾up for babies born to HIV positive mothers. In order to increase uptake of PMTCT services, NUMAT should scale up support for PMTCT integrated outreach services to the former IDPs now returning to their communities, support continuous health education so that mothers deliver in health facilities, and introduce packaging of PMTCT ARVs for the babies not born at the health facility in order to ensure there are no missed opportunities. There are existing routine MCH services now being provided at the lower level facilities. Services at the ANC clinics, young child clinics and out patient clinics should be targeted with support in form of supplies such as test kits, laboratory reagents, visual aids, integration manuals, and MOH registers. 2) NUMAT support for strengthening coordination structures has largely been effective at the district level, although the process is still weak at the sub-county level. This is probably attributed to the recent resettlement process of internally displaced populations and weak government structures at the sub-county level. NUMAT supported districts should place more emphasis on supporting the lower level structures such as the SATs, SACs, HUMCs and VHTs, since over one-half of the displaced population has resettled in their home areas. This 7 will facilitate the coordination of service delivery closer to the resettling populations. Quantification of the type and level of support was beyond the scope of this evaluation; however, support provided at the district level should be extended to at least the sub-county level. 3) The districts have not been very effective in taking a lead in coordinating the different agencies working in the region. NUMAT should continue providing both technical and financial support to the districts to enable them take on a more prominent role in the coordination of different partners delivering MAT interventions. The program stakeholders during the MTR dissemination meeting in Gulu resolved that NUMAT should have more dialogue and openness in the planning and implementation. At the same meeting the district attendees expressed readiness to take the lead in the coordination process. 4) NUMAT has made major strides in addressing the capacity building and service delivery needs which are consistent with the district priorities regarding the delivery of MAT interventions. However, there are still service delivery gaps which are largely attributed to inherent weaknesses in the national health system. The NUMAT program should develop their workplans around the realities of the health system and continually liaise with the MOH and local governments to facilitate systemic change of service innovations. At risk are malaria interventions as the MOH has not been able to assure adequate treatment supplies. 5) Most of the health facilities supported by NUMAT deliver integrated MAT interventions in a limited manner. This is mainly because of the low level of awareness about the concept of integration among the health workers, lack of job aids and IEC materials about service integration in the health facilities. NUMAT should support districts to enhance delivery of MAT interventions in an integrated manner by ensuring that facilities have the most up to date IEC materials, guidelines and job aids about integrated service delivery, and also receive training, mentoring and support supervision. Specifically NUMAT can support the training and mentoring in service integration and provision of MOH manuals for integration. 6) The NUMAT program has successfully responded to the changing environment in Northern Uganda by transitioning from emergency care to development assistance, as evidenced by the increasing provision of services closer to the resettling populations and revitalization of lower level coordination structures such as VHTs and HUMCs. To further increase access to services for the resettling populations, NUMAT should seriously consider supporting districts to extend services to lower level service delivery points and bring on board more partners who are able to provide services such as HCT at the household level. This can be achieved by providing more grants to CBOs, FBOs, and NGOs on a competitive basis. NUMAT should facilitate staff recruitment where necessary and continue with the logistical support which is currently being provided. 7) The capacity building efforts by the NUMAT program will have a sustained effect on service delivery of MAT interventions because of the competencies developed among the health 8 service providers at both the health facility and community levels. However, some aspects of NUMAT support will not be sustained by the districts after the project ends. Examples include major infrastructure improvement because of limited resources allocated by GoU towards capital development and formal training of health service providers. NUMAT should now begin dialogue with the district local governments as the project winds down in order to build consensus on an appropriate exit strategy that will ensure sustainability of currently supported MAT interventions. Our overall assessment is that NUMAT has successfully evolved in the first half of its 5-year project cycle and is successful in making useful interventions. However, with a more strategic direction and focus in PY4 and PY5, the program is likely to surpass most of its original performance targets. 9 1. INTRODUCTION 1.1 THE SOCIAL AND POLITICAL CONTENT IN NORTHERN UGANDA North-central Uganda was faced with a 20-year armed insurgency which resulted in thousands of deaths among children, women and men and led to a situation in which entire communities were displaced and lived in camps. Many children existed as “night commuters”, walking miles every evening to sleep in safety and then walking back to their village to attend school the next morning. The security situation in Northern Uganda has been considerably and steadily improving during the last three years. Peace and stability have greatly helped to transform the humanitarian assistance operating environment, and to restore people’s hope of returning to their respective homes. All Internally Displaced People (IDP) camps in the five districts of the Lango sub-region have been officially phased out as of early 2008. Additionally, there is a gradual increase in the IDP population of the Acholi sub-region moving back to their original villages. The population coming out of the camps has been particularly weakened by their long stay there, both socially and economically. It is a highly vulnerable group with an urgent need for basic social services in the areas of resettlement where infrastructure is severely lacking. The Government of Uganda (GoU) is leading the implementation of the Peace, Recovery and Development Plan (PRDP) for Northern Uganda, which is aimed at enhancing the people’s capacity to recover from the adversities of the long conflict. The 2005 National HIV/AIDS sero-behavioural survey showed that the HIV prevalence in Northern Uganda is higher (8.2%), compared to the national average (estimated at 6.4%). This is attributed to various factors, including the long-standing conflict with its displacement of populations, food insecurity leading to transactional sex and rape, all of which were compounded by lack of access to health care in the conflict-stricken areas. There is also some anecdotal evidence suggesting that new HIV infections are disproportionately affecting the Northern Region where there is a shift towards more risk-taking behaviours, particularly an increase in multiple sexual partners, non-spousal sex and a decrease in condom use. 1.2 PROJECT BACKGROUND The Northern Uganda Malaria, AIDS TB Project (NUMAT) was designed to support expansion of access to and utilization of HIV & AIDS, TB and malaria activities in the Northern Uganda districts of the Acholi sub-region (Gulu, Pader, Amuru, and Kitgum) and the Lango sub-region (Lira, Amolatar, Dokolo, Apac, and Oyam). It is funded through a Cooperative Agreement with the United States Agency for International Development (USAID) for the period August 2006 through August 2011. It is implemented by JSI Research and Training Institute, Inc., and includes World Vision (WV) and the AIDS Information Center (AIC) as consortium members. The objectives of the project are: • Improved coordination of HIV/AIDS and TB responses • Increased access to and utilization of quality HIV/AIDS, tuberculosis and malaria prevention, care and treatment services 10 • Decreased vulnerabilities for specific groups to HIV/AIDS and other infectious diseases • Increased access of PHAs and their families to wrap-around services (care and support) • Improved use of Strategic Information for planning health services The major strategies and inputs for the program are: Strategies y Strengthening existing service provision sites to increase coverage and quality of services offered y Supporting new sites in peripheral areas to offer comprehensive services y Strengthening integrated outreach and service delivery y Strengthening existing local structures to coordinate and integrate services y Addressing critical human resource needs y Involving beneficiaries in planning and delivering of services (including PHAs) Inputs y Training: in-service and pre-service TOTs and direct service providers (mainly by the districts) y Direct Technical assistance through support supervision and mentoring (central, district or NUMAT); Districts are always involved y Infrastructure development – mainly in lab sector (targeting 27 labs) y Equipment support (labs, PTCs, ANC, FSGs) y Supplies and reagents – test kits and other lab consumables y IEC/BCC materials (print, audio & video) y Medicines – ARVs y Direct funding to districts/sites and community groups – through grants & other mechanisms 1.3 EVALUATION PURPOSE USAID Uganda’s primary purpose for undertaking a mid-term review of the NUMAT program was to determine whether or not the program is on track to achieve its objectives within the existing time frame and funding parameters, and to ascertain if any changes in program or management strategies are needed for the remaining two years of the project that would increase its impact in Northern Uganda in light of the changing political and social context. This review has covered the following questions: a) What factors have contributed to the success or failure across all components of NUMAT thus far – what is working or not working? b) Is NUMAT effectively addressing capacity building and service delivery needs in a changing environment? c) To what extent has NUMAT effectively addressed the changing Northern Ugandan environment from emergency relief to development assistance? d) How effective has NUMAT been in strengthening existing district-based coordination 11 • Document review • Key informant interviews • Focus group discussions • Self assessment • Health facility assessment checklist structures and partnerships between NUMAT with other key players (USG Implementing Partners, GOU partners and civil society)? e) As far as sustainability, to what extent have NUMAT-supported districts benefited from NUMAT inputs, including: developing strategic plans; gap analyses; monitoring and evaluation; and use of program data for decision making? 1.4 EVALUATION METHODOLOGY A team of six evaluators composed of an international team leader, three local consultants (public health physicians) and two research assistants conducted the Mid-Term Review (MTR). The team was constituted by Uganda Monitoring and Evaluation Management Services (UMEMS), which is funded by USAID/Kampala and managed by The Mitchell Group, Inc. The review took place between June 29th and July 24th 2009 and was conducted in four districts (Gulu, Pader, Lira and Oyam), which were specifically selected from the list of nine NUMAT districts based on sub-regional representativeness. Within each study district, two health facilities were visited at each level of care (regional referral/district hospital, HCIV, HCIII) to make up five health facilities in each district. Additionally, two civil society partners providing wrap around HIV/AIDS services were selected and visited for field interviews in each district. The study population included USAID staff, NUMAT staff, District Local Government officials, key civil society partners and beneficiaries for both health facility based and community based services (Annex D). The review team used a combination of methods to collect information on which this report is based. The different data collection tools which were used are found in Annex E. Most of the information collected was qualitative in nature and, therefore, it was audio-taped and then transcribed verbatim. It was analysed using content analysis. Typical quotes were also selected and included in the reports in order to emphasize the response given without losing the original context of the meaning. The quantitative data was tabulated and analysed using proportions and graphs. 1.5 STUDY LIMITATIONS NUMAT is reportedly the largest USAID funded health program in Northern Uganda and is characterised by a wide range of interventions and geographical coverage. The time allocated to this review was relatively small and, therefore, the review team could not cover every aspect of the program, including visiting all of the nine districts. Nevertheless, the review team feels they were able to give a balanced assessment of the program based on the information obtained through interviews, document reviews, and field visits. 12 2 EVALUATION FINDINGS 2.1 ACHIEVEMENT OF TARGETS The team reviewed project monitoring data to assess whether NUMAT has effectively and rapidly scaled up services through training, technical assistance, and logistical support. A review of selected performance monitoring indicators shows that the program is largely on track to achieve its targets. The percentage achievement for selected indicators is shown in figure 1 below. Figure 1: Percentage achievement for selected performance monitoring indicators based on the project’s mid-term targets The quantification of the indicator for abstinence was based on numbers individuals reached with abstinence messages. With the exception of PMTCT indicators and TB tests performed, the program has surpassed the 50% achievement, which is to be expected midway in the project cycle. The low achievement on PMTCT indicators is attributed to the low proportion of institutional deliveries and inadequate follow-up of babies born to HIV positive mothers. The lack of qualified laboratory personnel has affected the functioning of laboratories in conducting 13 diagnostic tests such as sputum analysis for TB diagnosis, hence the low performance on TB tests. 2.2 COORDINATION STRUCTURES AND PARTNERSHIPS How effective has NUMAT been in strengthening existing district-based coordination structures and partnerships between NUMAT with other key players (USG implementing partners, GOU partners and civil society)? Coordination is defined as a process of harmonizing the activities of Malaria, AIDS, Tuberculosis (MAT) stakeholders, both government and non-government, through organized planning, implementation, monitoring and evaluation of the disease responses, which are consistent with the Government of Uganda’s (GOU) coordination guidelines and policies. Coordination structures for HIV/AIDS, TB and Malaria exist from the district level to the village level. District focal persons for each of the above disease conditions are in place in all of the districts. With the insurgency, the coordination structures became non-functional1 , and the District Disaster Management Committees (DDMCs) and the UN Cluster system were adopted for coordinating health activities at the district level (Health and Nutrition Sector Working Groups). Following the transition from emergency to stability, there has been revitalization of the District AIDS Committee (DAC), District AIDS Task Force (DAT), Sub-County AIDS Task Force (SAT) and Sub-County AIDS Committee (SAC) in some districts. The above coordination structures have not been replicated at the parish level in all of the districts. However, the Health Unit Management Committees (HUMCs) and the Village Health Teams (VHTs) do exist in some districts. The NUMAT program works with other partners involved in TB/HIV and malaria activities, including the Ministry of Health (MoH), Uganda AIDS Commission (UAC), international agencies like the UN, and other implementing partners (Annex E). In order to determine how effective NUMAT has been in strengthening existing district-based coordination and partnerships, the mid-term review team assessed the support provided to the coordination structures and the extent to which they are harmonizing implementation of MAT responses by different partners. NUMAT’s efforts to strengthen coordination reflects national policies and guidelines, and also takes into account the uniqueness of the operational context that is still making a transition from an emergency situation to stability in Acholi and Lango sub –regions. NUMAT has so far engaged with national and local partners to facilitate building systems for delivery of MAT responses. The program is placing emphasis on strengthening decentralized coordination at District, Sub-county, Parish, and Village levels. 1 A coordination structure is regarded as functional if, in addition to being constituted by its defined membership, it carries out all its roles as stipulated in the district coordination guidelines 14 At the national level, NUMAT is working with the Uganda AIDS Commission and members of the Decentralized Response Self Coordinating Entity (DR-SCE), including UNAIDS and other NGOs, UN agencies, and donors that are supporting the MAT response in the nine NUMAT supported districts. In the Acholi-sub-region where the impact of the conflict was the most vicious, NUMAT has been working within the UN-supported cluster systems and strengthening District Disaster Management Committees’ (DDMC) coordination mechanisms. While in the Lango sub-region, NUMAT has supported the work of District Health Teams (DHT), District AIDS Committees (DACs), and the District HIV/AIDS teams (DATs) under the auspices of AMICAALL and other key stakeholders to strengthen effective coordination of Malaria, HIV and TB activities. NUMAT conducted capacity assessments of HIV&AIDS coordination structures in all nine districts prior to providing support. These were used to inform ‘need-based support’ to districts. Key cross-cutting weaknesses that were identified included: inability of the structures to effectively function due to the existence of other parallel coordination structures; lack of meetings; lack of district HIV&AIDS strategic workplans; non-ratification of HIV&AIDS coordination guidelines; poor composition of membership and poor funding. NUMAT addressed these issues by providing both technical and financial support which has facilitated the reactivation of all the district HIV&AIDS committees (DACs). NUMAT has strengthened district coordination structures and partnerships through provision of funds, extensive district planning, conducting TOTs in the districts to establish and strengthen structures such as the DATs, DACs, SATs, SACs, HUMCs and VHTs. The functioning of the above structures varies from district to district. For example, in Oyam District, the DAT and DAC are fully functional; however, the SATs and SACs are non￾functional in all the seven sub-counties. In Pader, all the SACs and SATs have been formed; however, very few are fully operational. In districts where the coordination structures are functional, the members now understand their roles and responsibilities; they meet more regularly and conduct field monitoring visits for HIV&AIDS activities to harmonize implementation of services at the different levels. Box 1 highlights the key achievements in strengthening coordination structures, as well as harmonization of implementation. 15 Joint planning and supervision of MAT services is now taking place because of the improved coordination structures. The typical quote below illustrates the support provided by NUMAT towards strengthening coordination structures; “We have received financial and technical support from NUMAT for five sub-counties to revitalize coordination structures.” (Key informant, Lira District). The review shows that NUMAT is actively engaging USG partners both in Northern Uganda and in Kampala to complement their roles and responsibilities, and to share information about activities and experiences. However, coordination with and among partners (including USG IPs) has not been very effective. There are areas of overlaps in the activities supported and gaps still exist in service delivery. The districts have not been very effective in taking a lead in coordinating with the different agencies working in the region. Box 1: Key Achievements in Strengthening Coordination Structures • Joint assessment of HIV/AIDS coordination structures at district, sub-county and urban council level was conducted. The exercise was aimed at obtaining information on the existing situation of HIV/AIDS coordination in these local governments with the view to guide plans for strengthening coordination in these local governments • Joint training of trainers for HIV/AIDS coordination with Uganda AIDS Commission and other partners in all of the 9 program districts • Development of joint district/NUMAT plans. Two cycles of funding have so far been provided to all the 9 districts • Four districts (Kitgum, Pader, Dokolo and Amuru) have developed HIV/AIDS strategic plans in accordance with the National HIV/AIDS Strategic Plan (NSP) with the support from NUMAT • Quarterly meetings and field monitoring of HIV/AIDS activities by district and sub-county HIV/AIDS structures (DACs, DATs, SACs and SATs) are taking place in some areas • Reactivation of health unit management committees (HUMC) in return sites and training of HUMCs - a total of 45 HUMC were supported • At a lower level NUMAT has strengthened functioning of coordination structures like VHTs, NSAs, etc. • Two regional PMTCT support supervision and review meetings were organized and drew in participation of MOH, UNICEF, WHO, and other international and local based health development organisations • One malaria conference to share experiences and best practice for the region was organized • Organisational self assessment and strategic plan development was supported for Gulu, Kitgum, Lira, Dokolo, Apac and Oyam PHA forums and other CSOs • Participation in national, regional and districts coordination meetings has increased • Zonal and TB supervisors have been trained and financially supported to have quarterly meetings, and at least one quarterly on-spot site visit to the health facilities 16 Enabling factors • Working within existing government structures and closely with local leaders has built trust within the community • Working within existing systems, e.g., network support agents and home visitors have been instrumental in mobilising the communities • NUMAT’s responsiveness to district specific needs identified jointly (by both NUMAT and the districts) • Presence within the region and districts (headquarters in Gulu and 6 other district offices) • Involving such service beneficiaries as people living with HIV in the planning process Challenges • Meeting the expectations and demands of the communities and the local government in the region, given the enormous rehabilitation and development needs caused by the 20-year conflict • Limited district engagement at the commencement of the whole NUMAT project, hence initial misunderstandings between NUMAT and some districts • Delay in implementation and accountability of NUMAT-district funded activities has limited the opportunities for further funding of new activities • Weak sub-county level governments limit effective coordination of HIV & AIDS responses • Ensuring that Health Units Management Committees are functional and accountable to the communities they represent • High expectations by some district officials, for example, expecting higher safari day allowances during supervision and monitoring visits, have adversely affected their participation in planned activities • Civil society participation in HIV&AIDS activities in the region is still very poor due to the absence of strong district networks, internal organizational weaknesses, and meager funding 17 2.3 CAPACITY BUILDING b) Is NUMAT effectively addressing capacity building and service delivery needs in a changing environment? How has NUMAT addressed any observed implementation issues noted during USAID’s supervisory field visits for which feedback has been provided? The biggest change that has occurred in the North between the time NUMAT began services and the present has been the shift by a significant proportion of the population from their IDP camps to their homes or to satellite camps. In order to determine whether the program has effectively addressed the capacity building and service delivery needs in a changing environment, the review team assessed the extent to which NUMAT has provided technical and financial support to the districts (health facilities and civil society partners) to address gaps in capacity and service delivery in relation to the changing environment. Prior to the provision of technical and financial support to the districts, NUMAT conducted participatory needs assessments in the areas of laboratory infrastructure; counselling rooms; wrap around HIV/AIDS services; Human Resources for Health (HRH); monitoring and evaluation capacity of personnel; coordination of HIV interventions; and, ART site assessments and malaria services. The gaps that were identified formed the basis for NUMAT program support in the following areas: strategic planning; sub-granting; refurbishment of laboratories; training; logistics and supplies; human resource for health; community service networks; supervision and mentoring; management of strategic information. Strategic planning To-date, four districts (Kitgum, Dokolo, Pader and Amuru) have developed HIV/AIDS strategic plans in accordance with the National HIV/AIDS strategic plan (NSP). The other districts already had developed their strategic plans. These will be used by the districts, by NUMAT and all other partners as a guide to comprehensive, integrated and coordinated HIV&AIDS responses. Sub-granting NUMAT has also supported institutional and organizational development of Civil Society Organizations and Faith Based Organizations (FBOs) involved in MAT responses through sub￾granting and undertaking of organizational self-assessment and strategic plan development, especially for PHA networks for efficient and effective delivery of services to their constituencies. They provided sub-grants to four organisations, namely AVSI (PMTCT), Gulu Youth Centre (HCT), Kitgum Youth Centre (HCT) and Children Fund International (SGBV). 18 Refurbishment of laboratories NUMAT has so far completed the refurbishing of laboratories and counselling rooms in twelve health units, three in each of the districts of Amuru, Gulu, Dokolo and Amolatar. These are now model laboratories in their districts. Procurement of refurbishing services is in progress for the sites in the districts of Lira, Oyam, Apac, Pader and Kitgum. Sixty-eight laboratories received selected equipment based on the gaps that were identified in the laboratory needs assessments. The equipment included, but is not limited to, microscopes, water filters, and externally heated autoclaves, neubeur counting chambers, tally counters, refrigerators, centrifuges, a blood bank, microliter pipettes, timers, and pipette fillers. The laboratories that were refurbished were found to be fully functional and carrying out the diagnostic tests for HIV, TB and malaria. In conjunction with the Ministry of Health and Making Medical Injections Safer project, thirteen health workers (8 from districts and 5 NUMAT staff) were trained as trainers in Health Care Waste Management (HCWM), to initiate training for other health workers and to sensitize the community on HCWM. In addition, HCWM practices assessment was conducted in 41 health units, results disseminated at the national level and in three of the project districts. Posters on waste segregation were distributed to sixty-four health units. These activities have resulted in some improvement in waste segregation and final waste disposal at health facilities. NUMAT Supported training activities HIV/AIDS TB LABORATORY MALARIA COMMUNITY PREVENTION LOGISTICS MANAGEMENT HMIS ƒ TOT for HIV/AIDS coordination ƒ 150 and 490 health workers trained in VCT & RCT, respectively ƒ 30 health workers oriented in the new PMTCT policy ƒ 87 health providers trained in PMTCT ƒ 46 trained in Integrated Infant and Young Child Feeding Counselling ƒ 203 trained in Early Infant Diagnosis for HIV (EID) ƒ 86 service providers trained in the implementation of FSGs ƒ 220 health workers from 28 NUMAT supported HIV clinics trained in basic chronic care ƒ 35 districts home based care trainers trained ƒ 1096 home visitors trained ƒ 600 health workers trained in TB/HIV activities ƒ 262 health workers trained in CB-DOTS ƒ 5 officers, 1 from NUMAT and 4 from the districts attended a training of trainers (TOT) on TB/HIV ƒ 207 VHT members trained on TB/HIV interaction ƒ 133/150 laboratory personnel received refresher training ƒ 120 clinicians reoriented on rational utilization of laboratory services ƒ 26 currently sponsored for 2 year certificate course ƒ Training of district trainers in Home Based Management of Fever (HBMF) in 6 districts ƒ Village Health Teams (VHTs) as Community Medicine Distributors (CMDs) in HBMF ƒ 71 Behaviour Change Agents (BCA) master trainers trained ƒ 300 BCAs trained ƒ 80 CSW and women’s groups trained in Business Skills Development ƒ 28 CSW developed business plan proposals ƒ 87 Community resource persons trained in value based program including promotion of abstinence and being faithful among the youth ƒ 650 leaders trained in “Channels of Hope" (COH) methodology ƒ 140 PHA volunteers deployed in 60 health facilities as NSAs ƒ training 69 health workers from supported sites ƒ HMIS focal persons and record clerks trained in all districts on relevant aspects of M&E ƒ Trained 140 people in the use of strategic informatio n for planning and budgetting 19 Logistics and supplies • Provision of PMTCT ARVs, septrin for OI’s, adult ARV drug formulations • Provision of furniture, e.g., chairs, tables, and cabinets • Provision of stationery, e.g., patient registers • Provision of IEC/BCC materials and equipment such as booklets, posters, televisions and videos • Provision of bicycles for transport, specifically to the TB sub-county supervisors, and also transport to the districts to deliver various inputs to health facilities • NUMAT has also financed the procurement of equipment to facilitate collection, analysis, storage and timely reporting of HMIS data at district, regional and national levels. • Trained laboratory staff in the collection of samples (CD4 and DBS), testing, and return of results Human resource for health • Assisted and provided funding to the districts of Gulu, Kitgum, Pader and Oyam for advertising, and facilitated the District Service Commissions (DSCs) in conducting interviews to recruit new staff • Induction of newly trained health workers was done for Gulu, Kitgum and Pader. A total of 242 were inducted • Joint performance improvement of reproductive health and PMTCT activities were conducted in all health units offering RH and PMTCT services in the 9 districts • Partnered with Makerere University and Gulu University to attract young undergraduates for field attachment (nurses, medical students, pharmacists) in peripheral health units in the region. To date a total of 157 students have participated in this partnership • Laboratory training curriculum has been developed jointly with partners and Ministry of Health • Pre-service sponsorship of 26 students for laboratory technician training • Assessment of the Lira Distance Education Centre • NUMAT has partnered with the Capacity Project, MoH, MoLG and MoPS in order to improve staffing levels in the region through regular monthly consultative meetings, joint organisations of regional conferences on HRM, and training (performance improvement and payroll management) Community service networks • Provision of transport in the form of a motorcycle and bicycles • Provision of IEC/BCC materials and condoms • Provision of space for some of the community based network groups Supervision and mentoring and improved strategic information management 20 • District/NUMAT supervision has been on going in the areas of coordination, strategic planning, sub-granting, laboratory services, training, logistics management, malaria, HIV/AIDS and TB services, HMIS, and community service networks. • NUMAT has so far contributed towards improved district management of strategic information in the following ways: o NUMAT provided training to district personnel and local organizations to design M&E plans in line with the national HIV& AIDS monitoring framework. As a result, districts can now compare their HIV/AIDS data against the national information base and adjust district activities as necessary to improve upon the outcomes of their activities. NUMAT is also training HMIS focal personnel and record clerks in all its districts on relevant aspects of M&E aiming at building their skills in data management and fostering among them a culture of using data for decision making. o NUMAT is continuing to train record clerks in correct methods of data collection. District HMIS staff are now able to better manage data collection from the lower level health units and conduct analysis, report results to the regional headquarters, disseminate, and utilize the data. Record assistants at health sub-districts are also being trained with skills to extract data from service registers and to enter this information into the HMIS summary forms, emphasizing HIV/AIDS, tuberculosis, and malaria indicators. • The program has also been procurring equipment to facilitate collection, analysis, storage and timely reporting of HMIS data at district, regional and national levels. In the communities, people living with HIV and AIDS (PHA) networks are being trained and provided with registers to track data of chronic care patients. • Data Quality Assessments (DQA) on reported data is being conducted to ensure that the quality of the collected data meets required standards. • NUMAT is supporting district teams, in both the Health and Community Development Departments, to conduct support supervision and collect service data on a monthly basis for submission to the central level and to meet project reporting needs. • NUMAT has employed both the Lot Quality Assurance Survey and Health Facility Assessment methodologies as rapid, cost effective sampling methods used to measure coverage of key indicators while identifying gaps in performance. Overall, more than 140 participants from the 9 districts have been trained on both methodologies. • The program has also been sponsoring regular district and HSD level meetings for technical teams on themes of data use and management. Exchange meetings allow sharing of promising practices, challenges, lessons and review of service data. • NUMAT has been working with the Ministry of Health, Uganda AIDS Commission and USAID Implementing Partners to improve the use of strategic information. The program also ensures linkages between program level results and national level disease trends. Community HIV prevention activities • The NUMAT HIV prevention activities primarily target the following groups for HIV prevention: girls and women at risk of survival sex; commercial sex workers; discordant couples; uniformed service members/mobile male populations; PHA (for positive prevention); young married women; orphans and other vulnerable children; youth 10-24 21 years; and men, including fishing communities, long distance truck drivers and farmers’ groups. • NUMAT uses the ABC strategy and is working closely with other partners to roll out key elements of a comprehensive strategy for balanced and targeted prevention, namely: to strengthen district capacity and linkages to coordinate prevention activities; to plan and implement message dissemination; to improve prevention component of clinical services; and to establish and strengthen community-based groups to reinforce messages and provide ongoing support to the most-at-risk populations. • The NUMAT most-at-risk prevention strategy seeks to phase in targeted HIV prevention activities for adults and assures that all prevention and protection efforts are targeted to key groups that drive the HIV epidemic in the north, an area which is not currently receiving attention from other implementers. The MARPs intervention has so far been used to stimulate dialogue in communities to bring about positive change in behaviours and practices through the efforts of Behaviour Change Agents who encourage positive behavioural change and the use of specific health products and services among their peers. This is contributing to decreased vulnerabilities for specific groups susceptible to HIV and AIDS and other infectious diseases. • Most At Risk Populations interventions build on existing networks of communication between individuals, families, and community members to develop a network of Behavior Change Agents (BCAs). Community Based Workers trained as TOT who organize trainings for BCAs and employ the peer to peer approach to pass on the health messages. • The BCAs are trained and empowered with skills to mobilize and educate their peers in their communities about health issues. The BCAs not only act as MARPS Intervention campaign advocates at the community level, but also support NUMAT’s other community mobilization activities, and some act as community based product distributors. NUMAT’s concept allows use of the network as an interpersonal and community communication vehicle, which can be mobilized for various health topics, including HIV/AIDS, Malaria, and Tuberculosis. • NUMAT is also mandated to ensure decreased vulnerabilities for specific groups, including youth exposure to HIV/AIDS and other infectious diseases. Therefore, NUMAT seeks to reduce the rate of new HIV infection among young people (aged 10-24 years) through avoiding risk, with emphasis on abstinence and risk reduction, being mutually faithful to one partner, and using condoms correctly and consistently. • To ensure that the voices of youth and their parents are part of the design and planning process for HIV prevention activities, the project works with existing community groups to establish Youth Advisory Groups and Parent Advisory Groups (YAGs and PAGs) at the district level that provide advice on youth-related issues, messages, themes, activities and processes. To ensure coordination with sub-county stakeholders, the project facilitates the establishment of Sub-County Advisory Committees (SACs). In collaboration with the District Community Development Department, and the District Education Officers, NUMAT utilizes and builds upon existing community and district resources by actively engaging local NGOs and CBOs, Community Resource Persons (CORPS), community, youth leaders, and other stakeholders to deliver the project. NUMAT aims at surrounding young people with a supportive environment at every level, 22 from the inner circle of family outwards to peers, schools, places of worship, and the wider community. • NUMAT is working with Faith Based Organizations as one of its strategies to combat the stigma and discrimination against persons who are HIV positive in Northern Uganda. Channels of Hope targets religious leaders of all denominations and challenges their discriminatory attitudes towards care and support for people living with and affected by HIV and AIDS. Following the sensitization, the faith leaders select 3-4 people in each congregation to form a congregation HIV/AIDS Task Team who are provided with a five day workshop. The congregational HIV/AIDS task teams work with different congregations to implement, monitor, and evaluate the action plans developed to respond to HIV/AIDS. The CHATTs work with congregational leaders and trained facilitators to promote utilization of HCT services, life skills for youth, and working with congregation members to change social norms. • The review shows that NUMAT faces the following challenges in implementing community prevention activities: o Social cultural practices that make women vulnerable to all forms of exploitation o Drop out of BCAs due to high expectations in the form of monetary rewards o Moving populations have made follow up of the trained persons difficult as result of the resettlement program o Too few youth peer educators as compared to the general youth populations in the districts o Dropout rate of community resource persons is high because they relocate to other places and get employment elsewhere, and because this is voluntary work o Long distances for services as people are resettling to their original communities o Most CHATT members have low educational backgrounds and, therefore, have difficulty in data collection and reporting o There are so many fragmented congregations which places a heavy work load on the TOTs In terms of future prospects, the program strongly considers the following ways for logically implementing community HIV prevention: o Strengthening HIV prevention interventions among married couples o Enhancing interventions that increase comprehensive knowledge among youth on HIV/ AIDS and its risk factors o Faith Based leaders are the custodians of culture and morality and are the most sustainable and reliable people who can confront HIV/AIDS issues, including prejudice; therefore, working with these FBO leaders would ensure continuity and sustainability o Linking faith institutions to cultural interventions continues their custodial roles in upholding morality in the face of an increasing HIV /AIDS epidemic 23 o Increasing the proportion of young people who have a comprehensive knowledge on HIV/AIDS o Strengthening peer to peer approach for out of school youth o Improving the quality of service provision for SGBV survivors, both by the legal protection structures and health facilities o Improving the quality of SGBV service assessments o Increasing geographic coverage of community awareness on SGBV and advocacy o Promotion of positive behaviour change/life skills o Linking women’s groups to micro credit and IGA for survival alternatives o Reaching school age CSWs with educational support o Supporting remarriage counselling dialogues in FBOs o Upgrading the accumulated saving and credit association and the village saving and loan association groups to SPM (Selecting, Planning and Management of Business) o Increasing access to HCT to MARPs through worksites/camps Extent to which NUMAT addressed issues flagged by USAID during supervisory visits USAID flagged the following issues for NUMAT to address: coordination, working in collaboration with other USG partners, establishing linkages, resolving duplication, and increasing efforts towards sustainability. NUMAT is improving coordination of MAT responses in Northern Uganda, as presented under review question one (section 2.2). NUMAT is actively working with USG partners both in Northern Uganda and in Kampala to complement their roles and responsibilities, and to share information about their activities and experiences. In Northern Uganda it is part of the implementers meeting in Gulu for IEC, ART, Coordination, SGBV working groups. The Key collaborating USG agencies are presented in Annex E. NUMAT is spearheading linkages though its integration on a number of key areas for more effective and efficient delivery of quality HIV/AIDS services to local populations such as TB/HIV collaboration aimed at improving the quality of life of patients living with either HIV or TB alone, or those who are co-infected. Other linkages are NUMAT/AFFORD partnerships. Linkages between Most at Risk Populations (MARPs) and HCT are being established and continuously evaluated. From the planning stage, the two managers of HCT and MARPS work closely together to plan and provide the necessary services to MARPS, including HCT. Together with the responsible district authorities and collaborating CBOs, the MARPs manager supports mobilization activities and other related community services. HCT outreaches to MARPS are then conducted to these various groups. Because sustainability is a key focus in PY 4, NUMAT is addressing this issue through capacity building, expanding services to lower level units, including scaling up of ART at HC IIIs and strengthening coordination structures. NUMAT has also held strategy planning meetings with SPEAR, Baylor, AIDS Relief, Visions in Action, JCRC, TASO, Unity, and other key partners to avoid duplication of services offered in the same catchment areas. 24 Enabling factors • Working within existing government employment structures • Working within existing systems, e.g., network support agents and home visitors, communities have been successful in creating greater care and support for persons living with HIV/AIDS. • NUMAT’s responsiveness to district specific needs have been identified jointly by both NUMAT and the districts • Availability of trainable staff at the health facilities in each district Challenges • The transition from camps to satellite camps, and finally to homes, is characterized by high dropout rates of trained Community Owned Resource Persons (CORPs) as they relocate, and some of the trained CORPs opt to work elsewhere after resettling • There is limited coordination among partners that train health workers in the region and this has contributed to duplication • Recruitment process for community-based health workers is a lengthy and bureaucratic process resulting in long delays in staffing within the districts • Even after recruiting health workers, the drop-out rates have been alarmingly high due to delayed access to payroll, lack of housing and social amenities, and poor supplies or equipment in most health units. Other factors include poor remuneration, cross-over of health staff to NGOs, and inequitable and erratic transfers of health workers to areas where their skills might not be required • The lengthy procurement procedure for refurbishing laboratories caused delays. Districts must provide documents such as: o application for Prequalification for Building Works for the financial year, o recommendations by district contracts committee against which the CAO circulates the list of pre-qualified service providers o lists of pre-qualified service providers for the financial year • Monthly support supervision by District Laboratory Focal Persons (DLFPs) is not conducted in a timely manner because funds go through the district systems which tends to delay disbursement for activities • Record assistants are few, and many facilities do not have a dedicated person for data collection. Additionally, there is a generally weak culture of data utilization, particularly at lower level health units. The data infrastructure is also weak and there are multiple parallel systems and project reporting requirements that generate competing interests and inefficiencies within the district HMIS office. Perceptions of Service Beneficiaries on availability, accessibility and quality of MAT services The review team assessed the perceptions of the service beneficiaries about the availability, accessibility and quality of services offered, both at the health facilities and at community-based organizations supported by NUMAT. The service beneficiaries compared the services during the time when they were still in the camps to when they returned to their homes. 25 Overall, the service beneficiaries reported an improved range and quality of services being offered at the NUMAT supported sites when compared to those provided during the emergency period. The improvement in the quality and accessibility to services is illustrated by the typical quotes below: “When the war ended, people of different specialities like you started coming to assist us thus the quality of service now is good.” FGD participant, Pader District “In the past, you could not see the eye of someone because people had lost weight completely but now we have access to septrin, ARVs which have led to improvement in the health of people. There are no bed ridden patients and people are not dying as in the past. If someone tells you that I am HIV positive, you may not believe because in the past people lost weight that you could notice that there is a problem”, FGD participant, Oyam District. Service availability and accessibility In all the FGDs that were conducted, beneficiaries reported an increased accessibility and availability to the following MAT services, both at the health facilities and in their respective communities; health education and sensitization, laboratory testing; drugs, Insecticide Treated Nets (ITNs) and free distribution of condoms, referral and wrap around services. The following challenges were reported to affect accessibility to available services: • Too few staff at the health facilities results in patients not being seen • Long distances to travel to some health facilities • Difficulty in accessing CD4 count tests • Frequent drug stock outages are consistently reported by VHTs and CMDs • Difficulty in getting TB treatment supporters in the community The following positive developments in availability and accessibility were noted: The vulnerable groups in the communities reported increasing utilization of the following services: • ITNs for pregnant women and children below five years as illustrated by the typical quote below: “NUMAT first brought mosquito nets for prevention of malaria which was given to PHA then later water guard and jerrycans to have clean drinking water.” FGD participant, Pajule HC IV. • HCT and PMTCT services • HBMF for children in the villages • Early Infant Diagnosis for HIV, which is offered free of charge • Availability of HAART for both children and adults • Health education messages targeting all age groups, including the youth against HIV, TB and malaria • Post exposure prophylaxis in the event of rape or defilement 26 The beneficiaries in all the FGDs that were conducted also reported increased access to wrap around services like food support, school fees, IGAs, family planning services, etc., which are provided by different organizations such as Uganda Red Cross, Health Alert Uganda, Action International, Samaritan Purse, Marie Stopes, and TASO. Beneficiaries reported directly benefitting from the capacity building activities that are implemented by NUMAT. This is illustrated by the typical quote below: “In building capacity, the power given to us by Red Cross and NUMAT through training of Home Visitors, Behaviour Change Agent and TOT (training of trainers) in home- based care and other issues has made us to talk on HIV/ AIDs issues without fear.”Pajule HV IV Perceived quality of health facility interventions Beneficiaries agree that, although a number of services are currently in place at the health facilities, the client load at the facilities is quite high. The high client load is attributed to the resettling population and limited numbers of health workers who have taken up positions at rural facilities. Compromised quality of services, as reported by the beneficiaries, includes long waiting times and drug stock outages. Perceived quality of community interventions Most of the beneficiaries reported good quality of services offered by the NUMAT supported organizations. Services were reported to be provided by the organizations up to the village level. The trained volunteers complement the efforts of health workers at the health facilities, and this has reduced the client waiting time. The formation of support groups has led to reduction in the stigma among clients receiving HIV/AIDS interventions. Most beneficiaries, however, noted that the volume of services is still inadequate. The beneficiaries report that this tends to cause conflicts between those who receive and those who do not receive the services. Most of the service beneficiaries reported higher expectations for services at the community level than were always available. This is illustrated by the typical quote below: “The disparity in the services like government giving medication only and other organisations giving medication plus other services like food distribution must be looked into because patients live together. They share information and ideas so if they are divided by services it is very painful. If your colleague has got medicine from this facility and yours is from another place then that is not good” FGD Layibi, Gulu 27 In all the FGDs conducted, the study respondents suggested several ways to improve implementation of MAT activities (Box 2). Box 2: Suggestions by Service Beneficiaries for Improving Interventions by NUMAT Supported Institutions • Building more health facilities nearer the communities • Increasing staffing at the health facilities • Constant supply of anti-TB drugs • Motivation (monetary) of the volunteers • Emphasis on community awareness on TB prevention, signs and symptoms • Increased access to IGAs for PLHIV • Health education on drug adherence and prevention starting at the village level • Training of health visitors/volunteers • Active follow – up of patients in their homes • Strengthening adherence and counselling • Providing and supplying food to the clients • Increasing the number of qualified staffing in ART clinic • Intense campaigns against stigma, discrimination • Performing HIV tests and giving ARVs daily rather than on specific days • Good coordination/ harmonising the services offered by the different players on the issues of HIV/ AIDS • Transport facilitation for VHT/ Volunteers • Increasing number of health facilities offering ART services for beneficiaries’ easy access • Training more PHAs and attaching them to health facilities • Strengthening of HBMF with inclusion of adults as beneficiaries • Free distribution of ITN to all house hold inhabitants, not just pregnant women • More VHTs recruited to move around and sensitise the community 2.4 PROGRAM RESPONSE TO A CHANGING ENVIROMNENT a) To what extent has NUMAT effectively addressed the changing environment from emergency to development? In light of the changing social and political context in Northern Uganda, the services and structural approaches NUMAT began implementing have changed and are highlighted below: • A shift from the reliance on integrated service outreach to IDP camps to strengthening service delivery at the lower level fixed facilities where the populations with returning refugees. As people have gradually moved away from camps, there was a slight downward change in coverage indicators as the population spread out and away from 28 camps. This necessitated a restructuring of services in the lower level facilities which were much closer to the returning populations. The timeliness of this shift was dictated by contextual factors such as the security and political environment. Although there was relative peace at the time when project started, most of the people were still in the camps so the services had to be provided where the majority of the population was located. • The program also simultaneously deployed technical staff at the district level so planning and supervision could begin to better serve the community. • Shift from reliance on Camp Management Committees for coordination and support of community based activities to working with regular community structures at the parish and sub county level in the selection and support of Community Resource Persons. • The project now more directly supports regular district management and coordination structures (DACs, DATs, DHT, and DPTCs) than was the case during emergency relief efforts in the camps. • Additionally, the regular district structures are now in charge of planning, for example, District Planning Technical Committee (DPTC), DHT, DACs and DATs as opposed to DDMCs. Planning now focuses on services in the villages as opposed to services in the camps, and it is led by local leaders as opposed to Camp Management Committees. The NUMAT support to the districts is now part of a regional development plan, the PRDP • The role of UN agencies and emergency relief work efforts is being greatly reduced. Now district local governments are slowly coming back and performing their roles as the true managers of the district health (and other) affairs. • Some changes in the program’s strategy have not been directly influenced by the population’s shift from camps to homes, but by other broader developments in the region and in the country. For instance, with peace becoming more consolidated, the level of relief aid to the region has gone down substantially. Movement within and between districts has become much easier, including travel to and from neighbouring Sudan. This has resulted in an increase in transactional sex. As a result, NUMAT has deliberately increased HIV prevention interventions focussing on the Most at Risk Populations (MARPs) which include uniformed forces, commercial sex workers, people living with and affected by HIV&AIDS, boda associations, market vendors, truck drivers and drinking club members. MARPS are being supported with economic empowerment for alternative livelihoods, business skills training, linking to vocational skills training and formal educational opportunities, and access to farming inputs through the Food and Agricultural Organizations (FAO). • The program changed the prevention strategy in consultation with partners and assessments. There has been a refocus of prevention activities to specific target groups like married couples, recognizing that new infections are high in this group. • The project has also reviewed its approach of disseminating HIV prevention messages, changing from approaches that relied on mass mobilisation campaigns in camps to those that are based on small group interactions and one-to-one contacts which are more effective in the present circumstances. 29 Enabling factors: • Partnership with Local Governments (LGs) has worked relatively well. The LGs are involved in the activities and they feel their priorities are being addressed. The support provided in areas like staff recruitment and support supervision is well appreciated. • The scaling up of specific services like ART at lower levels and periphery sites can be termed a success. • There is an increased awareness and greater involvement on the part of communities in HIV/AIDS issues. PHAs and community volunteers have contributed to this awareness at the grassroots. There is more demand for services, more disclosure and openness about HIV sero-status, and an increased level of general awareness about HIV services at the community level. • The investment in infrastructure, particularly laboratories, has helped improve services generally in the supported facilities. Other services, not directly supported by the project, have also had a positive impact on service delivery. Challenges • Because the high demand for services, created partly by the NUMAT’s interventions, has not been accompanied by concomitant support by the MoH in complementary areas such as immunizations, diarrheal programs, etc. Consequently, MoH funds may be drawn away from the geographical areas in which NUMAT operates. • Generally not much progress has been made in health financing. Health workers are still poorly motivated and, therefore, their attrition rate is still high. Attracting them to peripheral facilities has remained a challenge. • Coordination among partners (including USG IPs) has not been very effective. There are areas of overlap in the activities supported and gaps still exist in service delivery. Some districts have not been very effective in taking the lead in coordinating different agencies working within their region. • The very specific and limited mandate of NUMAT restricts the extent to which the project can respond to the priorities and needs of a district, even where such support would enhance service delivery, e.g., health workers housing or direct support for food and nutrition supplements for PHAs. • The project relies on the performance of public service workers to reach its overall goal. Because of the working culture in the public service, project efforts have often been frustrated. In several instances, public officers show a lack of commitment and a disregard of work ethics when carrying out their responsibilities. Their actions are dictated more by short term individual benefits than by professional commitment to deliver quality services. Projects like NUMAT have to travel a thin line between adapting to the environment in which they operate and conforming to the requirements of their grant with USAID. 30 2.5 SUSTAINABILITY b) As far as sustainability is concerned, to what extent have NUMAT districts benefited from NUMAT inputs, including strategic plans, gap analyses, monitoring and evaluation, and use of program data for decision making (Lot Quality Assurances Surveys – LQAS)? The review team assessed the sustainability of NUMAT implemented interventions by determining the extent to which: • Districts develop and implement comprehensive and integrated work plans for HIV/AIDS, TB, and Malaria • Services are provided in an integrated2 manner in NUMAT supported health facilities • LQAS surveys have been institutionalised by the districts and used to collect information for local decision making and planning • NUMAT reporting systems are mainstreamed with the government’s HMIS • Readiness/preparedness of the district coordination structures to plan for prevention activities against HIV, TB and Malaria when the NUMAT program ends • NUMAT has instituted health facility-based and community-based referral/service networks that benefit the vulnerable groups (PLHIV, OVCs, women, children, etc.), but were formerly non-operational. NUMAT supported nine districts to develop plans and budgets for MAT activities. Four out of the nine districts completed drafts of a five year HIV/AIDS strategic workplan that will be used as a guide for comprehensive, integrated, and coordinated HIV/AIDS interventions. All of the key informants at the district level reported that districts have the capacity to develop the HIV/AIDS plans and use them as a resource mobilization tool. Delivery of HIV/AIDS, TB, and Malaria services in an integrated manner is considered to be a key component of program sustainability. The review team assessed the extent to which the NUMAT supported health facilities deliver HIV/AIDS, TB and Malaria services in an integrated manner. The assessment was done at hospital, HC IV and HC III levels, and focused on following key components of integrated service delivery: HCT, reproductive health, TB/HIV collaborative activities, logistics management, malaria services, laboratory and ART services. Figure 2 shows the overall rating of health facilities and their level of integrated service delivery. 2 The review team defined service integration as a health care approach whereby a client is catered for holistically as a result of an individual provider and the health facility. Integration can be categorized as physical integration (where space is shared for services on the same day and in the same room) and as functional integration (where the same health worker provides more than one service and the records are able to capture relevant information from more than one service). 31 Figure 2: Overall Rating of Health Facilities on Level of Integrated Service Delivery The figure above illustrates that the majority of health facilities were rated fair (13/19) with regard to integrated delivery of services. Figure 3 shows the overall rating on integrated service delivery by level of care. Overall, the delivery of integrated services for HIV/AIDS, TB and malaria vary with the level of care; however, the hospitals and HC IVs perform better than the HC IIIs. Figure 3: Overall rating on integrated service delivery by level of care The review team identified the following challenges that affect the delivery of integrated services for HIV/AIDS, TB and malaria at the health facilities that were visited: a) Lack of service delivery policies and guidelines, as well as knowledge in areas of service integration. 32 b) Limited job aids and IEC materials at the facilities, and those available often do not mention integrated services. c) Stock out of essential drugs makes it difficult to conduct integrated services on a daily basis. For example, if malaria drugs are not available, it becomes difficult for health workers to screen patients who have fever for malaria. There are no job aids for requisitioning drugs and supplies in some facilities. d) Some of the health workers were not aware of the concept of integrated management for HIV/TB/Malarias and, therefore, need to be trained in a formal setting. This is illustrated by the typical quote below: “Integration is going on to a limited extent in some health facilities. Most staff have not been sensitized yet on integration, but some are providing integrated service.” Key Informant, Pader District Local Government. “The integration challenge at facility level referral takes place but may not be properly implemented due to long distances and cost of transport.” Key Informant, Gulu district e) Poor referral systems, e.g., no policy/guidelines, no referral documentation including a feedback loop, and no formal provider coordination. The NUMAT program has so far completed two rounds of LQAS surveys to provide periodic evaluation of progress towards addressing HIV & AIDS, Malaria, and TB to guide program implementation and to enhance evidence-based planning and decision making. The review shows that the district officials fully appreciate the usefulness of the LQAS survey data and use it in planning and decision making. To date, a total of 70 participants from the 9 districts have been trained in the LQAS methodology. Interviews with some of the district officials that participated in the survey rounds show that their role was limited to data collection, and they still lack competence in the design, planning, implementation, and analysis of LQAS data. This is illustrated by the typical quote below: “The LQAS was done in this district. The HMIS focal person was called to Gulu and trained on collection of information. Analysis was done by NUMAT; however, it was used to develop the strategic plan.” Key Informant, Pader District The review shows that the NUMAT reporting system is mainstreamed in the GoU HMIS. Most of the indicators prescribed in the PMP are using data obtained from the district HMIS. Within the districts that were visited by the review team, the data infrastructure is still weak, with multiple parallel systems and project reporting requirements that generate competing interests and inefficiencies within the district HMIS office. NUMAT provided technical and financial support which led to the reactivation of all district HIV/AIDS committees (DACs). All the district officials who were interviewed reported that members of the DACs know their roles and responsibilities, conduct field monitoring visits, and would continue to support coordination activities when the program ends. 33 In the attempt to strengthen community and health facility based networks, NUMAT has worked with partners to build collaboration between USAID and non-USAID implementing partners, local government, FBO/CSOs, other NGOs, PHA networks, and the private sector to create effective linkages for essential wrap around services. NUMAT has trained PHA networks/groups to increase access to wrap around services through advocacy. PHA volunteers are selected to serve as service navigators, connecting fellow PHAs and their families to wrap around service support. The program has conducted service mapping exercises and developed a service directory of available wrap around services by conducting joint meetings of service providers to streamline the referral system and to develop referral and tracking tools. Even with NUMAT support, there remain significant challenges: • Lack of a clear handbook for health workers about referral/networking for comprehensive HIV/AIDS care • Some of the health workers who were interviewed expressed lack of knowledge about services offered by other providers within the same catchment area • Many health workers who were interviewed reported lack of mutual communication/feedback about the clients referred • It has been difficult to find ways to motivate and support CBOs, FBOs, and NGO participation in referral networks without providing direct financial support • Huge wrap around service needs for PHA households emerging out of conflict areas and returning to their villages/communities • Limited male involvement in PHA networking 34 CONCLUSIONS a) NUMAT has been exceptionally effective in teaching coordination to district level health staff. This review has shown that NUMAT support has been crucial in the revitalization of the district level coordination structures in all the districts. This process has, however, been much slower at the sub-county level. This is attributed to the population’s recent resettlement process and the weak government structures at that level. Where coordination structures have been set up, they are fulfilling their mandate to a considerable extent. Activities undertaken by the coordination structures, such as periodic meetings and supervision visits, are, however, limited by lack of funds and transport. b) Some districts have not been effective in taking a lead in coordinating the different agencies working in their region. c) NUMAT has made major strides in addressing the capacity building and service delivery needs, which are consistent with district priorities regarding the delivery of MAT interventions. Although the service beneficiaries report an improvement in the availability and quality of services offered at the NUMAT supported sites, there are still service delivery gaps which are largely attributed to inherent weaknesses in the national health system. At the community level, the volume of services offered by the community based prevention networks does not meet the communities’ expectations. d) Most of the health facilities supported by NUMAT deliver MAT interventions in an integrated manner to a limited extent, mainly because of low level of awareness about the concept of integration among the health workers and lack of job aids and IEC materials. e) The NUMAT program has successfully responded to the changing environment in Northern Uganda from emergency to development as evidenced by the increasing provision of services closer to the resettling populations and revitalization of lower level coordination structures such as VHTs and HUMCs. f) The capacity building efforts by the NUMAT program will have a sustained effect on service delivery of MAT interventions because of the skills developed among the health service providers at both the health facility and community level. Some aspects of NUMAT support, however, will not be sustainable by the districts after the project ends. 35 RECOMMENDATIONS NUMAT a) The program should place more emphasis on supporting the lower level health structures such as the SATs, SACs, HUMCs and VHTs because most of the populations have resettled in their home areas. This will facilitate the coordination of service delivery closer to the resettling population. b) NUMAT should continue providing both technical and financial support at the district level enabling them to take on a more prominent role in the coordination of different partners delivering MAT interventions. c) NUMAT should provide financial support at the district level enabling these districts to absorb the NSAs into the VHTs. d) NUMAT should provide financial support to the districts to conduct coordination meetings for all implementing partners of MAT and also conduct follow up site supervision visits. e) To increase access to services by the resettling populations, NUMAT should consider extending services to lower level service delivery points in the form of outreach, bringing on board more partners who can provide services such as HCT. More grants should be provided to CBOs, FBOs, and NGOs on a competitive basis. f) In order to increase uptake of PMTCT services, NUMAT should: • Scale up support for PMTCT integrated outreaches to the returning communities • Extend PMTCT services to HCIIs, especially those in geographical areas with a service gap and with at least minimum infrastructure • Strengthen the role of family support groups in PMTCT activities • Support continuous pre-natal health education so that more women will choose to deliver in health facilities. • Combine PMTCT ARVs for newborns to ensure no missed opportunities in cases where the mother does not deliver in a health facility. g) Given the high burden of malaria that resettling communities are already experiencing, scale up the distribution of LLINs targeting pregnant women and children below five years of age. h) NUMAT should enhance the capacity of all district health facilities to deliver MAT interventions in an integrated manner, ensuring that facilities have adequate IEC materials, guidelines, and job aids relating to integrated service delivery, training, mentoring, and support supervision. 36 i) NUMAT should assist district local governments to improve the fiscal management of grants with the intent to motivate and retain health workers. j) NUMAT workplans for PY4 and PY5 should remain fairly flexible because there are still large numbers of the population in the Acholi sub-region who will be moving back to their villages. k) NUMAT should plan with the MoH ACP in PY4 and PY 5 to develop an exit plan/strategy for maintaining all ART clients on treatment after NUMAT ends. l) NUMAT should continue to strengthen collaboration and partnerships with various implementing partners to expand access to wrap around services by PHAs and their families. The program should continue strengthening the roles of PHA networks in expanding access to quality HIV/AIDS services under district leadership. m) NUMAT should continue strengthening the capacity of district health officials in the use of LQAS to generate information for local decision making by actively involving the district HIMIS focal persons in the design, planning, implementation, analysis, and dissemination of LQAS data. USAID n) USAID should amend NUMAT’s current cooperative agreement and increase its overall Level of Program (LOP) funding for PY4 and PY5 to allow the program to procure ITNs and also scale up the malaria prevention activities. MoH o) Develop and distribute IEC materials and job aids on integrated management of MAT interventions at the health facilities. p) Working in close collaboration with the NUMAT program and partners, the MoH should establish a functional system for laboratory equipment maintenance and repair. q) Assure that there are no stock outages of essential drugs, especially for Malaria and TB. 37 ANNEX A: BIBLIOGRAPHY Northern Uganda Malaria AIDS and Tuberculosis programme Annual Report October 2007- September 2008. Northern Uganda Malaria AIDS and Tuberculosis programme Briefer for Mid-term review July 2009. Northern Uganda Malaria AIDS and Tuberculosis programme PEPFAR Semi Annual report 2009. Northern Uganda Malaria AIDS and Tuberculosis programme PEPFAR Annual report 2008. LQAS Survey Report 2008 Uganda National HIV/AIDS Sero-behavioural prevalence Survey, 2005 Northern Uganda Malaria Control baseline assessment Report 2007 Assessment of M&E capacity of personnel in NUMAT districts, 2007 Assessment Report of the situation for coordination of HIV interventions in Acholi and Lango sub-regions, 2008. PEPFAR Reports 2008 and Semi 2009 Comprehensive Report on ART assessment 38 ANNEX B: SCOPE OF WORK FOR THE MID-TERM REVIEW BACKGROUND The Northern Uganda Malaria, AIDS TB Project (NUMAT) is funded through a Cooperative Agreement with the United States Agency for International Development (USAID) for the period August 2006 through August 2011. It is implemented by JSI Research and Training Institute, Inc., and includes World Vision (WV) and AIDS Information Center (AIC) as consortium members. NUMAT is funded by USAID with funding from the President’s Emergency Plan for AIDS Relief (PEPFAR), the President’s Malaria Initiative (PMI) and Child Survival Infectious Disease resources. The total five-year budget for the project is $29,999,441. In support of the National Strategic Framework for HIV/AIDS in Uganda, the NUMAT program was designed to support expansion of access to and utilization of HIV & AIDS, TB and malaria activities in the Northern Uganda districts of the Acholi sub-region (Gulu, Pader, Amuru, and Kitgum) and the Lango sub-region (Lira, Amolatar, Dokolo, Apac, and Oyam). The aim of the project is to expand access to services by actively building upon existing networks and expanding the geographic coverage through strengthening of local government responses and expanding the role of communities in planning, implementing and monitoring activities. NUMAT’s efforts contribute to the Government of Uganda’s (GOU) / Poverty Eradication Action Plan (PEAP) plans as outlined in the Peace Recovery and Development Plan (PRDP) for Northern Uganda and largely contributes to PRDP’s Objective 2 of rebuilding and empowering the community. Focusing on service delivery at the district and lower levels, NUMAT’s objectives include: • Improving coordination of HIV/AIDS, TB and malaria services for internally displaced persons (IDPs) and other populations • Increasing access to and utilization of quality HIV/AIDS, Tuberculosis and Malaria prevention and care & treatment services, • Decreasing vulnerabilities for specific groups to HIV/AIDS, TB and malaria • Increasing access of people with HIV/AIDS (PHAs) and their families to wrap-around services through effective partnerships with other programmes Throughout the past two and half years, NUMAT has operated within a dynamic environment of evolving security situations. Initially, most of the NUMAT programming focused on supporting activities within the IDP camps. With the recent population shift of about 80% of the IDPs moving to either satellite camps or back to their homes, programming has increasingly focused on providing services closer to the resettling population. Consequently, the overall district plans being supported by NUMAT and other development partners are shifting from emergency to development mode. PURPOSE OF EVALUATION The Automated Directive System (ADS) 203.3.6.1 requires that an evaluation is conducted when there is a distinct and clear management need to address an issue. The primary purpose of this evaluation is to determine whether NUMAT is on track to achieve its objectives within existing time frame and funding parameters and recommend any changes in program or management strategies for the remaining years of the project that would increase its impact in Northern Uganda in light of the changing political and social context. The evaluation will cover all the three project areas: HIV/AIDS, TB and malaria. KEY EVALUATION QUESTIONS This evaluation entails examining a number of questions: 1. What factors have contributed to success or failure – what is working or not working? 2. Is NUMAT effectively addressing capacity building and service delivery needs in a changing environment? 3. To what extent has NUMAT effectively addressed the changing environment from emergency to development? 4. How effective has NUMAT been in strengthening existing district-based coordination structures and partnerships between NUMAT with other key players (USG implementing partners, GOU partners and civil society)? 5. As far as sustainability is concerned, to what extent have NUMAT districts benefited from NUMAT inputs, including strategic plans, gap analyses, monitoring and evaluation, and use of program data for decision making (Lot Quality Assurances Surveys – LQAS)? PERFORMANCE PERIOD The evaluation will begin on or about June 29, 2009 and will require approximately 30 working days of effort: 7 days for preparation and document reviews; 11 days field work; and 17 days for data analysis, debriefs with USAID and other stakeholders and report writing. The level of effort for the Team Leader will be slightly higher than for other team members as s/he will be responsible for producing the draft and final evaluation reports. The Team Leader will provide a final report to USAID no later than September 15, 2009. 39 EXISTING INFORMATION SOURCES The following information documents and sources are available and relevant to the study: • GOU: National frameworks, policies and implementation guidelines from Uganda AIDS Commission, Ministries of Health, Local Government and Gender, Labor and Social Development, Local Government Development Plans and reports • USAID: Original Request for Application, Emergency Plan related documentation, and The Role of Evaluation in USAID http://www.dec.org/pdf docs/PNABY239.pdf • NUMAT: o Cooperative Agreement and amendments o Annual and quarterly reports o Annual work plans, results framework and performance monitoring plan o Strategy papers for core services o Tools, training materials, guidelines, etc. o Documents illustrating beneficiary, lessons learned, case studies o Internal assessments and reviews o Other METHODOLOGY The evaluation team will be required to propose a clear methodology to answer all the evaluation questions. With regard to data quality, the evaluation team is expected to be familiar with USAID data quality standards for objectivity, validity, reliability, precision, utility and integrity and be able to apply them in the final report, by identifying such data limitations as may exist with respect to these standards (ADS 78.3.4.2 - http://www.usaid.gov/policy/ads//500/578.pdf) and ADS 203.3.5.1- http://www.usaid.gov/policy/ads/200/203.pdf). 7. EVALUATION TEAM COMPOSITION The evaluation team will be comprised of one international and 4 - 5 national experts. The national experts will be senior technical staff from key Government agencies and civil society, one staff member from NUMAT and one staff member from USAID/Uganda. The team should possess the skills and experience below: Team Leader • Demonstrated experience (10 years) in HIV/AIDS, TB and malaria program evaluation in Africa. Uganda experience is highly preferred • Solid experience (? in implementing health capacity building and service delivery programs in developing countries. • Solid understanding of HIV/AIDS, tuberculosis and malaria service delivery in a multisectoral, decentralized environment • USAID programming and post conflict experience is desirable. National Experts • Solid understanding of the Uganda HIV/AIDS, tuberculosis and malaria service delivery in a multisectoral, decentralized environment • Public/Private partnerships, with a focus on local government and civil society • Capacity building of key leadership and service delivery organizations HIV/AIDS and Health program evaluation • Familiar with post conflict environment and resettlements • Sustainable development DELIVERABLES The evaluation team is expected to deliver the following outputs to USAID/Uganda: Deliverable Week Due 1. An inception report to be reviewed by USAID and selected GOU partners. The report will include i) A detailed work plan showing a timeline for each evaluation activity to be undertaken, including field work ii) Methodology detailing sampling/selection procedures for districts, CBOs/NGOs, facilities and any other beneficiaries to be visited. iii) “Ready for pre-test” instrument for data collection First 2. Oral debriefing to USAID, to present key findings, conclusions and recommendations prior to submission of draft report. Third 3. Oral debriefing to USAID, selected GOU and other partners to present key findings , conclusions and recommendations prior to submission of draft report Third 4. Draft evaluation report in both hard copies (5) and one electronic copy for review by USAID and selected GOU counterparts. Fourth 40 Deliverable Week Due 5. Final evaluation report in both hard copies (5) and one electronic copy incorporating feedback from USAID and selected GOU counterparts. The final report should not exceed 30 pages excluding the executive summary and annexes. Sixth ROLES AND RESPONSIBILITIES Government of Uganda (Uganda AIDS Commission, Ministry of Health, Ministry of Local Government) • Provide feedback on inception report • Serve as key points of reference and information, including key documents, for final review • Participate in oral debriefing • Review and comment on draft report UMEMS’s roles and responsibilities are to: • Manage the evaluation process including facilitating a one-day, Team Planning Meeting (TPM) for this evaluation study at NUMAT/Gulu • UMEMS shall submit a technical and cots proposal to USAID for approval. The technical proposal shall include an illustrative evaluation design and work plan that shall be fine tuned by the evaluation team, and proposed evaluators. • Submit evaluation report to USAID/PPC/CDIE USAID’s roles and responsibilities are to: • Write the evaluation Scope of Work • Approve UMEMS technical and cost proposal for the evaluation • Review draft report and provide feedback • Sign off the final report NUMAT’s roles and responsibilities are to: • Provide feedback on inception report • Provide relevant documents as needed • Provide logistical support for the evaluation team including office space at NUMAT Gulu, assistance with setting up meetings, interviews and providing transport (2 vehicles) • Have a staff member on the evaluation team on a full time basis ILLUSTRATIVE REPORT OUTLINE Cover page (Title of the study, the date of the study, recipient’s name, name(s) of the evaluation team. Preface or Acknowledgements (Optional) Table of Contents List of Acronyms Lists of Charts, Tables or Figures [Only required in long reports that use these extensively] Executive Summary [Stand-Alone, 1-3 pages, summary of report. This section may not contain any material not found in the main part of the report] Main Part of the Report 1. Introduction/Background and Purpose [Overview of the MTR evaluation. Covers the purpose and intended audiences for the MTR and the key questions as identified in the SOW) 2. Study Approach and Methods [Brief summary. Additional information, including instruments should be presented in an Annex]. 3. Findings [This section, organized in whatever way the team wishes, must present the basic answers to the key evaluation questions, i.e., the empirical facts and other types of evidence the study team collected including the assumptions] 4. Conclusions [This section should present the team’s interpretations or judgments about its findings] 5. Recommendations [This section should make it clear what actions should be taken as a result of the study] 41 6. Lessons Learned [In this section, the team should present any information that would be useful to people who are designing/manning similar or related new or on-going programs in Uganda or elsewhere. Other lessons the team derives from the study should also be presented here.] Annexes [These may include supplementary information on the evaluation itself; further description of the data collection/analysis methods used; data collection instruments; summaries of interviews; statistical tables, and other relevant documents.] 42 ANNEX C: FRAMEWORK FOR THE EVALUATION OF KEY QUESTIONS Evaluation Questions Types of answers/ Evidence Needed (Distribution, comparison ( to what) cause and effect (and notes on any special requirements or data source) Methods for data collection e.g, records, structured observation, Key informant interviews, mini-survey Sampling or selection Approach (If one is needed) Data Analysis methods e.g Frequency distribution, trends, analysis, cross tabulation Method Data sources 1. How effective has NUMAT been in strengthening existing district￾based coordination structures and partnerships between NUMAT with other key players (USG implementing partners, GOU partners and civil society)? • Analytic descriptions of all the district coordination structures. (up to lowest service delivery level) • Analytic descriptions of all the district coordination structures strengthened by NUMAT • Analytic description of partnerships between NUMAT and other key players (USG, GOU and civil society). • Establish the extent to which the district coordination structures are harmonising the implementation of HIV,TB and malaria implementation by partners • Desk review of program documents and reports • Key informant interviews • National policy and implementation guidelines for HIV, TB and Malaria • NUMAT program documents (annual workplans, quarterly and annual reports) • District annual performance reports • Key informants (National level partners, NUMAT Staff, District NUMAT staff, CAO, members of DHAC, Deputy CAO in charge of Health, DHO, District focal persons, Sub-county chief , implementing partners and CSOs) Purposive sampling Content analysis and interpretation of qualitative findings 2. Is NUMAT effectively addressing capacity building and service delivery needs in a changing environment? How has NUMAT addressed any observed implementation issues captured during USAID’s supervisory field visits for which feedback has been provided • Analytic description of the changing political and social environment within the NUMAT districts of operation • Establish the extent to which NUMAT has provided technical and financial support to districts to address the gaps in capacity and service delivery in relation to the changing environment • Establish the extent to which NUMAT has addressed issues flagged by USAID during supervisory visits • Client perspective of the services offered by NUMAT supported health facilities and organizations • Desk review of program documents and reports • Key informant interviews • Focus Group Discussions with service beneficiaries • NUMAT program documents (annual workplans, quarterly and annual reports) • District annual performance reports • Key informants (National level partners, NUMAT Staff, District NUMAT staff, CAO, members of DHAC, Deputy CAO in charge of Health, DHO, District focal persons, Sub-county chief , implementing partners and CSOs) • Service beneficiaries who regularly use the health facilities Purposive sampling Content analysis and interpretation of qualitative findings 3. To what extent has NUMAT effectively addressed the changing environment from emergency to development? • Establish the management strategies undertaken by NUMAT to ensure that the programming focuses on providing services closer to the settling populations. • Establish the extent to which the annual district plans supported by NUMAT have changed from emergency to development mode • Desk review of program documents and reports • Key informant interviews • NUMAT program documents (annual workplans, quarterly and annual reports) • District annual performance reports Key informants (National level partners, NUMAT Staff, District NUMAT staff, CAO, members of DHAC, Deputy CAO in charge of Health, DHO, District focal persons, Purposive sampling Content analysis and interpretation of qualitative findings 43 Sub-county chief , implementing partners and CSOs) 4. As far as sustainability is concerned, to what extent have NUMAT districts benefited from NUMAT inputs, including strategic plans, gap analyses, monitoring and evaluation, and use of program data for decision making (Lot Quality Assurances Surveys – LQAS)? • The extent to which districts can develop and implement comprehensive and integrated wok plans for HIV/AIDS, TB and malaria. • The extent to which services are provided in an integrated manner in NUMAT supported districts. • The extent to which LQAS surveys have been institutionalised by the implementing facilities ( used to collect information for local decision making and planning) • Establish linkages and/ or main streaming of NUMAT reporting systems with the GoU HMIS • Establish the readiness/preparedness of the district coordination structures to plan for prevention activities against HIV, TB and Malaria when the NUMAT program ends • Establish the extent to which NUMAT has strengthened health facility based and community based referral/service networks that benefit the vulnerable groups (PLHIV, OVCs, women, children etc) • Desk review of program documents and reports • Key informant interviews • NUMAT program documents (annual workplans, quarterly and annual reports) • District annual performance reports Key informants (National level partners, NUMAT Staff, District NUMAT staff, CAO, members of DHAC, Deputy CAO in charge of Health, DHO, District focal persons, Sub-county chief , implementing partners and CSOs) Purposive sampling Content analysis and interpretation of qualitative findings 5. What factors have contributed to success or failure across all components of NUMAT thus far – what is working or not working? • Compare the actual program results against the targets prescribed in the PMP (annually) • Compare access and utilization of services before and after resettlement • Analytic description of the factors that hindered or facilitated the achievement of program results • Describe lessons learnt and best practices • Suggest recommendations for the year IV and V workplans • Review the baseline and periodic performance information collected annually against the project PMP • Desk review of program documents and reports • Key informant interviews • NUMAT program documents (annual workplans, quarterly and annual reports) • District annual performance reports • Key informants (NUMAT DCoP. District, NUMAT staff, Deputy CAO in charge of Health, Secretary for Social Services, DHO, District focal persons, Sub-county chiefs. Health facility staffs) Purposive sampling as per number 1 above. Content analysis and interpretation of qualitative findings 44 ANNEX D: LIST OF PEOPLE INTERVIEWED Names Organization Designation Akaki Thomas Bell Lira District Health Team Lira District Local Government Dr. P.Kusolo Lira District Health Team District Health Officer Hon Ayena Patrick Lira District Health Team Lira District Local Government /Secretary for Health Dr. Mulwani Erisa NUMAT-Lira District Technical Officer Okao Ben Abor Lira District Health Team District TB & Leprosy Supervisor & HIV Focal Person Omoo Henry Lira District Health Team Biostatistician Awio Joel Lira District Health Team Senior Nursing Officer Otim Bernard Lira District Health Team Vector Control Officer Opio Patrick Lira District Health Team OP Acai Christine Lira District Health Team Supplies Officer Okol Etwop Leo Lira District Health Team Medical Records Assistant Awongo Robert Lira District Health Team Medical Records Assistant Ms. Okello Mary Francis Ogur HC IV Ophthalmic Clinical Officer Omach Lusiano Ogur HC IV Nursing Officer - ART Clinic Amongi Suzan Ogur HC IV Enrolled Mid-wife Moro Vincent Amach HC IV Medical Clinical Officer Epilla Lilly Amach HC IV Nursing Officer Mwesige Rashid Amach HC IV Vector Control Officer Apita Jimmy Amach HC IV Health Assistant Okello Fred Barr HC III Nursing assistant Onono Jenifer Awach HC IV Registered Midwife Omwony Samuel Cwero HCIII Comprehensive Nurse Otiti Sepriano Minakulu HC III Health Unit In-Charge Sister Rosemary Aber Hospital Hospital administrator Dr. Kitara David Aber Hospital Medical Superintendent Apio Grace Loro HC II Health Unit In-Charge Odur Constatine Loro HC II Nursing assistant Odongo Moses Loro HC II Support staff Ojok Walter Loro HC II Support staff Kadde Stephen NUMAT Medicines Logistics Officer Tumukurate Espilidon NUMAT Malaria Services Manager Musana Basil NUMAT HCT/PMTCT Officer Dr. Kansiime Edgar NUMAT PMTCT Services Manager Ochora Michael NUMAT IEC/BCC Manager Rwekikomo Frank NUMAT PHA Networks Manager Sera Diana NUMAT Monitoring and Evaluation Manager Tumusherure Edson NUMAT TB Services Manager Kanwagi Robert NUMAT Prevention Services Manager Oloya William NUMAT Capacity Building Manager Opwonya John Odong Gulu District Health Office Senior Medical Clinical Officer /DTLS, HIV/AIDS FP Okot Lukach Gabriel Gulu District Health Office Principal Clinical Officer/Senior Health Educator Idiba Yoweri Gulu District Health Office Biostatistician (DHIS) 45 Achiro Lucy Grace Gulu District Health Office Supplies Officer Muloya Felix Gulu District Health Office District Laboratory Focal Person Ojok Richard Naptali Gulu District Health Office Records Assistant /HMIS Focal Persons Dr. Upenytho George Gulu Regional Referral Hospital HIV/AIDS Focal Point Person Ogwal Tom Gulu Regional Referral Hospital Senior Medical Laboratory Technician Opio George Pius Gulu Regional Referral Hospital Medical Records Officer Alioru A. M Olal Gulu Regional Referral Hospital Senior Nursing Officer Okwera Milly Gulu Regional Referral Hospital Senior Nursing Officer Akello Grace Lalogo HC IV Midwife Acede Ambrose Lalogo HC IV TB Clinic Lawoko Morris Onen Lalogo HC IV Medical Clinical Officer ( HIV/TB) Okidi Alfred Lalogo HC IV Lab Department Nyeko Richard Lalogo HC IV Medical Clinical Officer Lakony David Maxwell Lalogo HC IV Records Assistant Acaye Robert Layibi Tech HCII Medical Clinical Officer Aceng Suzan C. Lira District Health Team SCC Obonyo Alex Ambrosoli Mission Hospital Administrator Olanya Baptist Ambrosoli Mission Hospital Enrolled Comprehensive Nurse Omoro Luciano Ambrosoli Mission Hospital Laboratory Assistant Otto Josephine Ambrosoli Mission Hospital Medical Officer A' Clinic Kidega Betty Ambrosoli Mission Hospital Registered Midwife Myango Patient Ambrosoli Mission Hospital Medical Officer Acan Mary Theresa Ambrosoli Mission Hospital Registered Nurse/ PMTCT Counsellor Sr. Amito Liberata Ambrosoli Mission Hospital Senior Nursing Officer Akullo Hellen Pader Health Center III Pader HC III Acayo Ventorine Pader Health Center III Pader HC III Akello Beatrice Pader Health Center III Pader HC III Oryema John Bosco Pader Health Center III Medical Laboratory Technician Dr Akena Simon Peter NUMAT - Pader District Technical Officer Lakony Lino Pader District Biostatistician Okidi Dominic Pader District District Health Inspector/District Malaria Focal Person Bimeny P'Adonga Pader District Assistant Health Educator/PMTCT Dr. Oola Janet Pader District District Health Officer Oloya Sylvesto Pajule HC IV Nursing Officer Billi Charles Pajule HC IV Laboratory Technician Kinyera Geoffrey Pajule HC IV Porter Ayot Mirriam Pajule HC IV Nursing Assistant Onono Jenifer Pajule HC IV Nursing assistant Ayaa Hellen Olal Pajule HC IV Nursing Officer Dr. Olal Ayella Richard Pajule HC IV Senior Medical Officer Atepo Richard Oyam District Health Team District Health Educator/Malaria FP Abic Tom Oyam District Health Team District TB & Leprosy Focal Person Alinga John Bosco Oyam District Health Team District Laboratory Focal Person Akello Sarah Oyam District Health Team Stenographer Secretary Ocen Gregory Oyam District Health Team Principal Health Inspector Dr. Okello Patrick Oyam District Health Team Medical Officer Dr.Okello Anam Silvanus Oyam District Health Team Senior Medical Officer Rapa Robert Oyam District Health Team Cold Chain Technician 46 Alinga John Bosco Anyeke HC IV Oyam District Laboratory Technician Akello Ojok Ernesta Anyeke HC IV Oyam District Nurse Counsellor (ART Clinic) Akullu Zilder Rose Anyeke HC IV Oyam District Nursing Officer Abic Tom Anyeke HC IV Oyam District TB/Leprosy Focal Person Oola A. Stella Anyeke HC IV Oyam District Data Clerk Dr. Okello Anam Anyeke HC IV Oyam District Medical Officer Ojul Joyce Lira District Health Team Medical Records Assistant Ongany Milton Pader HC III Senior Clinical Officer 47 ANNEX E: KEY COLLABORATING PARTNERS Name of the Organization Type of Organization Baylor College Center for Disease Control Cynapsis Grantee Joint Medical Stores National JCRC National Supply Chain Management Services USAID IP PHA Forums AIDS Control Programme National World Health Organisation Palliative Care association of Uganda NGO All district Health Officers Govt All district HIV Focal Persons Govt Christian Children Fund Grantee District Gender Officers UNFPA regional office & District offices Straight Talk Foundation CBO Family Impact CBO District Education officers World Vision/ SPEAR USAID IP Acholi private sector Acholi Economic Network CBO Kitgum PHA Forum CBO Gulu PHA Forum CBO Oyam PHA Forum CBO Apac PHA Forum CBO Amolatar PHA Forum CBO Lira PHA Forum CBO Amuru PHA Forum CBO Lira Young Positives Forum CBO Dokolo PHA forum CBO International HIV/AIDS Alliance ,Kampala USAID IP NuLife Project, Kampala USAID IP National Guidance and Empowerment of PHA (NGEN +) National The Uganda Network on Law ,Ethics & HIV/AIDS ( UGANET) National World Vision, Gulu District Based 48 National Forum of PHA networks in Uganda (NAFOPHANU),Kampala National JCRC , Gulu National Uganda National Young Positive Association (UNYPA), Kampala National The Positive Men union (POMU), Gulu CBO TASO,Gulu National Good Samaritans, Gulu. CBO ACDI VOCA, Lira USAID IP World Food Programme (WFP) International PACE (Formerly PSI), USAID IP Meeting Point, Kitgum CBO AVSI NGO AIDS Information Center (AIC) National Christian Children Fund District Based Kitgum Youth Center NGO Gulu Youth Center NGO Visions in Action USAID IP Malaria Consortium USAID IP Health Communcations Partnership USAID IP WHO Lira USAID IP USAID/ACE Project USAID IP MEEPP USAID IP UMEMS USAID IP American Refugee Council USAID IP Ministry of Health Resource Center National National Malaria Control program USAID IP World Health Organisation International UNICEF USAID IP AFFORD USAID IP AMICAAL National Uganda AIDS Commission National NACEAS National National TB & Leprosy Control (NTLP) National TB Cap USAID IP Zonal TB Supervisors Regional Uganda Malaria Survillance Project National Central Public Health Labs National USAID/MMIS (on waste management) USAID IP 49 ANNEX F: DATA COLLECTION TOOLS Key Informant Interview Guide This tool will be used to interview NUMAT staff, NUMAT grantees, and Local Government officials including health facility staff and Civil Society Partners, about the following: 1. Coordination and partnerships for HIV/AIDS, TB and Malaria services 2. Capacity building for HIV/AIDS, TB and Malaria services 3. Delivery for HIV/AIDS, TB and Malaria services and perceived quality by clients 4. NUMAT program Management 5. Sustainability for HIV/AIDS, TB and Malaria services Coordination and partnerships a) Existing district based coordination structures and partnerships Probe for • District level coordination structures and partnerships • Lower level (sub-county, parish and community level) coordination structures and partnerships • Other coordination structures and partnerships • Compare the management/operations of the partnerships/coordination structures; How has NUMAT supported you to manage the transition? b) Extent to which coordination structures and partnerships have been strengthened by NUMAT Probe for • District level coordination structures and partnerships • Lower level (sub-county, parish and community level) coordination structures and partnerships • Other coordination structures and partnerships c) Extent to which strengthened coordination structures and partnerships are harmonizing implementation of TB, HIV/AIDS and Malaria services Probe for joint planning among the different partners d) For the coordination structures and partnerships in delivery of HIV/AIDS, TB and malaria activities: Probe for the following • What worked well and why? • What did not work well and why? • If you had to replicate the process which aspect would you retain? Capacity building for HIV/AIDS, TB and Malaria services (including cross cutting issues of IEC/BCC, laboratory services, M&E, etc) a) Have the districts developed their own strategic plans? Have they used the LQAS 2008 data in order to make management decisions? b) In what areas has the NUMAT program assisted you to better coordinate, plan and allocate budget based on available data? c) Probe for any issues flagged by USAID during the supervisory visits, to what extent have you addressed those issues? d) How much of your current time and effort is devoted emergency services, do you have a plan of transition from emergency to development. How has NUMAT assisted you in this transition? e) For capacity building in delivery of HIV/AIDS, TB and malaria activities: Probe for the following • What worked well and why? • What did not work well and why? • If you had to replicate the process which aspect would you retain? Delivery for HIV/AIDS, TB and Malaria services and perceived quality by clients (including cross cutting issues of IEC/BCC, laboratory services, M&E, etc) a) To what extent has NUMAT supported you to strengthen the delivery of HIV/AIDS, TB and malaria services (including for the vulnerable groups and wrap around services) b) Do the districts undertake joint planning for TB, HIV/AIDS and Malaria, are these services integrated? 50 c) In what areas has the NUMAT program assisted you to better coordinate, plan and allocate budget for HIV/AIDS, TB and Malaria service delivery based on available data? d) Probe for any issues flagged by USAID during the supervisory visits, to what extent have you addressed those issues? e) How have you changed your service delivery strategies to match the changing social and political environment in Northern Uganda? f) To what extent are the service providers supported by the NUMAT program providing services for HIV/AIDS, TB and Malaria in an integrated manner? g) For delivery of HIV/AIDS, TB and malaria activities: Probe for the following • What is working well and why? • What did not work well and why? • If you had to replicate the process which aspect would you retain? • Perceived quality of services Program Management (NUMAT transition from emergency to development) [For NUMAT interviews this will be the first question] a) Describe the initial mandate of the NUMAT project. b) Probe for changes to strategic approach given the shift from emergency to development mode. c) Probe for how changes in strategic approach have impacted on annual district planning and implementation given the changing environment. d) For program management changes in strategic approach in PY2 and PY3 Probe for the following • What is working well and why? • What did not work well and why? • If you had to replicate the process which aspect would you retain? • Perceived quality of services Sustainability of HIV/AIDS, TB and malaria services a) Do as a district develop you own plans and budgets for HIV/AIDS, TB and malaria activities? b) Are these three integrated? c) What data do you use to inform your planning and budgeting? d) Which Health Management Information System (HMIS) are you using? e) If the NUMAT project were to suddenly end are you capable to continue monitoring data collection and planning HIV/AIDS, TB and malaria services? f) For sustainability of HIV/AIDS, TB and malaria services in PY2 and PY3 Probe for the following • What is working well and why? • What did not work well and why? • If you had to replicate the process which aspect would you retain? • Perceived quality of services Health facility Integration Assessment Tool Performance standards for the different components of integrated service delivery Yes No HIV Counselling and Testing 1. Presence and utilization of the current national guidelines on counselling 2. Staff orientation on HCT services (the benefits, purpose, and the roles of the staff). 3. Training of service providers in HCT and provision of HCT services according to national guidelines ( within the last 6 months) 4. Presence of functional HCT coordination committees 5. Utilization of MoH recommended tools to collect HCT data by the service providers. Additionally, the data should be complete, accurate and submitted by the end of each month to the appropriate level 6. Analysis and utilization of HCT reports for continuous quality improvement 7. Privacy and confidentiality; The counsellors should conduct all counselling sessions in settings that provide protection from others seeing or hearing what is going on; and also the counsellors should inform clients of confidentiality procedures of HIV testing during pre-test counselling 8. Follow up and referral: A referral directory should be available that shows the scope and nature of services provided at each of the sites. Reproductive Health 9. Presence and utilization of the most recent PMTCT implementation guidelines 51 10. Training of health workers providing health education in ANC clinics or MCH clinics about PMTCT services 11. Providing PMTCT education to all women attending the ANC clinic 12. Pre-test counselling sessions should cover the following areas: HCT, ART, FP, MTCT, infant feeding, risk reduction, referral and follow up 13. Health facilities providing PMTCT services should provide the recommended comprehensive package of services to all pregnant women in accordance to national guidelines 14. Counsellors who conducts pre-test counselling should also carry out post- test counselling 15. Accessibility; all facilities offering PMTCT services should do so on all working days (Monday to Friday) 16. Follow up and referral; all HIV positive women should on discharge following delivery, be referred to the post natal clinic (using the MoH referral note) 17. PMTCT counsellors should have adequate counselling materials in the counselling room (like job aids, demonstration materials) before each counselling session starts 18. Early infant diagnosis and linkage to ART services Malaria services 19. Presence and utilization of most recent malaria implementation guidelines 20. Training of health workers providing health education in malaria clinics about Malaria 21. Providing health education about malaria to all women attending the malaria clinics 22. Counsellors who conducts pre-test counselling should also carry out post- test counselling 23. Provision of treatment for malaria on all working days (Monday to Friday) 24. Health facility in-charges should order and stock adequate supplies of anti malarias Collaborative TB/HIV activities 25. Presence and utilization of guidelines for TB/HIV collaborative activities at health facilities 26. Training all health workers on TB/HIV collaboration 27. Service providers should conduct Joint TB/HIV planning meetings 28. Supervision should be carried out by external supervisors 29. Presence of community based Directly Observed Therapy (CB-DOTs) 30. Reporting of collaborative CB-DOTs/HIV activities 31. TB/HIV activities should be planned and discussed in meetings regularly 32. Referral system between HCT and Diagnostic treatment Unit 33. Intensified TB case finding –probe for recently introduced WHO guidelines to screen HIV positive patients for TB Logistics Management 34. Presence of Appropriate guidelines for requisition of anti malarials, anti-TB, ARVs and OIs, FP and lab reagents 35. Training of all health workers on management of logistics and supplies 36. Health facilities should conduct Joint logistics planning meetings 37. Supervision should be carried out by external supervisors on management of supplies and logistics 38. Management of supplies and logistics should be planned and discussed in meetings regularly 39. Health facility should have the necessary drugs (anti malarials, anti-TB, ARVs co-trimoxazole, and contraceptives and supplies at all times 40. Health facility should have the necessary stock cards for drugs and supplies at all times Laboratory services 41. The laboratory should carry out HCT/TB and Malaria diagnostic tests 42. Laboratory personnel providing have undergone an MOH approved laboratory malaria/TB/HCT training (emphasis on any form of refresher training and probe for duration of training) 43. A laboratory should have registers which are integrated 44. There should be sufficient stocks of HIV testing kits, malaria and TB diagnostic reagents (NUMAT is only responsible for stop gap measures) 45. Laboratory should have SOPs for HCT, TB, Malaria 46. Laboratory should have a binocular microscope that is functional 47. Conduct internal and external support supervision 48. The laboratory should carry out HCT/TB and Malaria diagnostic tests Antiretroviral Therapy (ART) Services 49. Training in integrated management of adult illnesses (IMAI) and integrated management of childhood illnesses (IMCI) 50. Availability of laboratory monitoring 51. Creation of community linkages and community based follow up 52. Internal and external support supervision mechanisms 52 Focus Group Discussion Guide This tool will be used to conduct group interviews with program beneficiaries. The purpose of the FGDs will be to study the group perspectives of the quality of services offered at the health facilities and the capacity which has been build with in communities to provide HIV/AIDS, TB and Malaria services. Ethnic Region District Sub county Village Name of the Moderator Name of the note taker Number of discussants FGD start time FGD end time Introduction Good morning/afternoon/evening to you all. My name is ………………………………………………….. and my colleague is ……..……………………….. (Mention his/her names). We are here today to discuss about the provision of health services in your community and how your community has been supported by the NUMAT programme before and after the armed conflict in Northern Uganda. All your responses will be treated with utmost respect, and remember there are no “right” or “wrong” answers as we want information based on your experiences, observations and feelings. Please feel free to ask for clarifications where needed. You do not have to reveal any personal information if you do not want to. All your answers shall be completely confidential and your name shall not be directly mentioned in the report. Before we begin I request that we all introduce ourselves and mention how long you have lived in this area. I request that you speak one at a time as well as loudly and clearly when answering a question so that all your views are understood and written down. When making a point during the discussion, you may choose either to use your name or not. To help us capture the whole discussion and ensure that we do not miss anything that you say, I kindly request you to allow me use the tape recorder here. May I use the tape recorder? May I continue with the interview? Thank you for accepting to take part in this discussion. 1) What do you think are the key health problems in this community? 2) When people have a health problem, where do they normally go for assistance? Probe for all types of assistance - prevention, care and support. 3) How would you compare health services now with the time when there was conflict in this community? Probe for availability, accessibility and quality of services. 4) You mentioned that Malaria is a serious problem in your community, tell us about the accessibility, availability and quality of services (please probe each of these issues one at a time) provided in your community. Probe for services provided at the health facility and in the community. 5) You mentioned that tuberculosis is a serious problem in your community, tell us about the accessibility, availability and quality of services (please probe each of these issues one at a time) provided in your community. Probe for services provided at the health facility and in the community. 6) You mentioned that HIV/AIDS is a serious problem in your community, tell us about the accessibility, availability and quality of services (please probe each of these issues one at a time) provided in your community. Probe for services provided at the health facility and in the community. 53 7) How have service providers in the areas of malaria, HIV/AIDS and tuberculosis addressed the health needs of vulnerable groups (probe for women, adolescents, children, girls and OVCs, youth, people affected by conflict and PLHIV). 8) Apart from the health facilities, which other organizations provide HIV, TB and malaria services in your community? Which services do they offer? Probe for any wrap around services in case they are not mentioned. 9) For the organizations mentioned in (8) above, what is your view about accessibility, availability and quality of services. Probe for HIV, TB and malaria services separately. 10) Is the community involved in the planning, implementation and monitoring health activities in your areas? If no, probe why they are not involved. If yes, probe how they are involved. 11) Has there been any capacity building with regard to planning, implementation and monitoring of health services in this community. Probe for training of different community groups (women groups, PLHIV, youth groups etc) and specific training done. 12) In your view, what do think should be done in order to improve accessibility, availability and quality of TB, HIV and malaria services in your community?