GOAL Uganda Scaling up Innovative HIV/AIDS Interventions Through Strong Partnerships Amongst Conflict-Affected Populations of Pader (Agago) District, Northern Uganda, 2008 -2011 END PROJECT EVALUATION REPORT NOVEMBER 2011 Submitted by: FARST AFRICA Submitted to: GOAL/Uganda GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 2 Table of Contents ACRONYMS .................................................................................................................................................... 3 EXECUTIVE SUMMARY ............................................................................................................................... 4 1.0 INTRODUCTION ..................................................................................................................................... 8 1.1 DISTRICT PROFILE SUMMARY.......................................................................................................................8 1.2 LOCATION AND ADMINISTRATIVE PROFILE ..................................................................................................8 1.3 BACKGROUND...............................................................................................................................................9 1.4 OBJECTIVES OF THE PROJECT AND INTERVENTIONS UTILIZED .......................................................................9 1.5 PURPOSE AND OBJECTIVES OF EVALUATION ...............................................................................................10 2.0 EVALUATION METHODOLOGY...............................................................................................................12 2.1 QUALITATIVE DATA ....................................................................................................................................12 2.2 QUANTITATIVE DATA ANALYSIS AND REVIEW .........................................................................................................13 2.3 SAMPLING AND DATA COLLECTION .....................................................................................................................14 2.4 DATA ANALYSIS AND REPORT WRITING ...............................................................................................................14 2.5 QUALITY ASSURANCE AND QUALITY CONTROL MEASURES .......................................................................................14 2.6 LIMITATIONS IN THE ASSESSMENT PROCESS...........................................................................................................15 3.0 EVALUATION ASSESSMENT FINDINGS ...................................................................................................16 3.1 IMPROVING KNOWLEDGE, ATTITUDE, BEHAVIOR AND PRACTICE LEVELS OF THE TARGET POPULATION................................16 3.2 IMPROVING CARE AND SOCIO‐ECONOMIC SUPPORT SERVICES TO PLHIV AND AFFECTED GROUP IN THE TARGET AREAS.........21 3.3 STRENGTHENING THE ORGANIZATIONAL CAPACITIES OF LOCAL CBOS AND PLWHA ASSOCIATIONS, INCLUDING MAINSTREAMING OF HIV, GENDER AND CHILD PROTECTION................................................................................................................29 3.4 OTHER GENERAL FINDINGS................................................................................................................................35 3.5 PROJECT DESIGN AND APPROPRIATENESS OF APPROACH USED (SUSTAINABILITY, COMMUNITY NEEDS AND       ADDRESS OF POST CONFLICT CONTEXT)......................................................................................................................................................37 3.5.1 Sustainability .........................................................................................................................................37 3.5.2 Project address of community needs.....................................................................................................39 3.5.3 Post conflict programming ....................................................................................................................40 3.6 MISSED OPPORTUNITIES ............................................................................................................................40 3.7 EXTERNAL FACTORS THAT AFFECTED PROJECT RESULTS............................................................................41 3.8 PROSPECTIVE NEED FOR PROJECT REPLICATION AND STRATEGIES FOR MAXIMIZING EFFECTIVENESS .........42 4.0 LESSONS LEARNT ..................................................................................................................................47 5.0 CHALLENGES.........................................................................................................................................50 6.0 CONCLUSION........................................................................................................................................51 REFERENCES: ...........................................................................................................................................................52 APPENDIX ................................................................................................................................................................53 Appendix 1 Terms of Reference ...................................................................................................................53 Appendix 2 Schedule for Field work............................................................................................................58 Appendix 3 Case Studies..............................................................................................................................71 Appendix 4 Sub County Plenary Fora on Way Forward .............................................................................76 Appendix 5 Wol Kico Primary School, Children’s Focus group participants.............................................78 Appendix 6 Evaluation tools.......................................................................................................................79 GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 3 ACRONYMS ART Antiretroviral Therapy ARVs Anti- Retroviral drugs CBOs Community Based Organizations CDO Community Development Officer CHFPs Community HIV/AIDS Focal Persons CRS Catholic relief services DDP District Development Plan HBC Home Based Care HCT HIV Counselling and Testing HIV/AIDS Human Immune Virus/Acquired Immune Deficiency Syndrome HLGs Higher Local Governments HMIS Health Management Information System HSD Health Sub District HUMC Health Unit Management Committee LG Local Government LLGs Lower Local Governments NAADS National Agriculture Advisory Services NMS National Medical Stores OPD Out Patients Department OVC Orphans and Vulnerable Children PEPFAR President’s Emergency Plan for AIDS Relief PDC Parish Development Committee PMTCT Prevention Of mother to Child Transmission of HIV PTA Parent Teacher Association T&C Testing and Counselling VHT Village Health Team VSLA Village Savings and Loan Association GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 4 EXECUTIVE SUMMARY GOAL secured USAID/PEPFAR funding of 1,322,548 USD to implement a 3 year New Partners Initiative HIV Project in Pader /Agago Districts from 2008 to 2011. The project activities were implemented through capacity building of Local CBO partner organizations to deliver comprehensive and coordinated responses to HIV. Activities focused on prevention of transmission of HIV, care and support to people living with HIV and mitigation of its effect on infected and affected individuals, families and communities. A 30 % cost share was contributed to from Irish Aid funding. The project was implemented in the sub counties of Lapono, Paimol, Parabongo, Omiya Pacwa, Lukole, Wol, Agago Town Council and Kalongo Town council. The project activities scaled-down in September 2011 and an evaluation was commissioned by GOAL in October – November 2011 to establish project impact, lessons and recommendations for future similar programming. The evaluation sought to assess the progress made in achieving the project results, review the project design and implementation strategies with respect to observable results (or lack thereof); suitability of GOAL and partner technical competency to undertake the planned activities; the technical support and capacity building for partner organizations and other community structures; relationship with project stakeholders and other service providers and lastly to identify and document lessons learnt and best practices observed during the project. The evaluation methodology included a documents review, key informant interviews with project stakeholders and beneficiaries; focus group interviews with beneficiaries; plenary group dialogue meetings and case study interviews of identified best practices. Amongst respondents interviewed were District leadership of Agago and Pader districts; Sub county leaders of Paimol, Wol and Kalongo Town council; Kalongo hospital, Paimol and Wol Health center staff representatives; Home Based Care, HCT, Stepping stones, Youth peers, OVCs beneficiaries plus guardians and Child Protection committee members. Findings were that the project successfully achieved its objectives. Two CBOs as planned have been supported to implement innovative IEC/BCC HIV prevention approaches plus HIV care activities. The geographical and population coverage of the CBOs increased following GOAL support. Geographical coverage of the CBOs services increased from 2 sub counties to 8 sub counties. Stepping stones methodology has been successfully implemented, as evidenced by increased knowledge of communities about HIV, risky behavior and ways to reduce risk of getting HIV infection; Reduced stigma and discrimination of youth living with HIV and enhanced positive living of youth living with HIV; Reduced gender based violence amongst participating community members’ families (ref case studies, appendix 3) in addition to extending coverage of activities to reach Men and women, young and old The total number of people reached through Stepping Stones over the life of the project is 11,572 individuals (5,275M & 6,297F) and this represents 144% of the planned project target. “The stepping stone approach enabled community members to be more assertive and take a more personal, social and community wide responsibilities for their actions. The stepping stone sessions helped in GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 5 clearing the myths and misconceptions about HIV/AIDS and created a more acceptable environment for people living with HIV (PLHIV)”1 . The Stepping stones evaluation also showed that there was increased acceptability of condoms as a means of preventing HIV infection as well as for family planning in marital settings, contrary to previous perceptions that linked condoms to promiscuity. There was also reported increase in condom use evidenced by the fact that people were buying and getting free condoms from health service providers such as GOAL. People also acquired skills of how to properly use condoms. From the quantitative data, of those who reported recent sexual activity (N=244), those who had used a condom at last sex were 21.7% (n=53). Condom use at last sex was reported more among males compared to females in Bugiri, but more among females in Pader; and higher among younger males and females compared to their older counterparts. Out of those who had had sex with a non-regular sexual partner, only 35.9% (n=23) had used a condom.2 Participants in all FGDs reported reduced rates of alcohol consumption, also attributed to Stepping Stones which had transformed most men and women. The quantitative results show that only 37.8% of the respondents reported that they consumed alcohol as at the time of this study. Alcohol consumption was more prevalent among males (60.4%), compared to females (24.3%), more prevalent among older age groups for both sexes; and more prevalent in Pader compared to Bugiri.3 Furthermore, it was noted in the same report that “men from different locations visited reported that they reduced their sexual encounters with multiple partners, with some reporting that they were now faithful to their spouses. The men also reported that this had enabled them to reduce their movements and to spend more time with their families. As a result there was increased trust and love among spouses and thus more harmony in families. There was also a reported decrease in transactional sex in both districts but the practice was reported more persistent in urban areas.” 4 Other benefits of the Stepping Stones strategy noted include better communication between spouses/partners and between parents and children. “Participants reported improved communication between spouses on subjects that were previously unlikely to be talked about such as condoms, sexuality, family planning, and money matters. Women also reported more freedom to talk about family planning and the need for education of their children. It was reported by several women especially the young ones that they were as a result able to space their children unlike before.”5 During the final evaluation exercise, focus group discussions, it was reported that there was reduction in unwanted pregnancies; reduced stigma and discrimination against PLHIV; reduction in risky cultural practices for example, the practice of widow inheritance was reported to have tremendously decreased though it still existed. Most participants in Pader reported that people now test for HIV before going into widow inheritance.6 Almost all the stages and components of the Stepping Stones were judged by the different stakeholders as having been successful. What appears to have been particularly striking was the relevance of the content to the issues in the local contexts. It is evident that participants were able to use the training sessions to identify and discuss the most critical problems that are part of their daily lives. SS peer group members served as role models, and had a positive influence on their children, their 1 GOAL Uganda Annual PEPFAR report Oct 2008- Sept 2009 2 Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011 3 Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011 4 Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011 5 Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011 6 Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011 GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 6 spouses, and the community at large. The SS members have also been actively teaching others about what they learn, thus having a multiplier effect. The drama used by the Stepping Stones groups appears to have been an effective behavior change communication channel. In some instances, flexibility and innovativeness were applied to overcome emerging challenges such as the provision of mobile HCT in Kalongo to address the lack of easy access to VCT at health centres. This was a good practice that contributed to programme success.7 The project has also achieved increased access to HCT services and improved Care services and livelihoods for targeted HIV/AIDS affected groups. Key Informants noted that provision of HCT services closer to communities through door to door HCT campaigns and outreaches greatly increased community access to HCT services and knowledge of their HIV status. The project was able to reach 24,446 (10,812 (44%) M and 13,634 (56%) F) and exceeded the 3 year project target of 18,000 people for testing and counselling services, by 136%. The project also provided Home Based Care support to 906 (301 M and 605 F) PLHIV through trained Community Counseling Aides. The project achieved its project target of providing Home based care to 900 PLHIV. Project target for community based support to OVC and child headed households has been achieved with 1001 (508M & 493 F) OVC reached with comprehensive support including food and nutrition, education support, psychosocial support, health care services, and child protection activities. Analysis of Child status data showed improvement in some of the scores over project support period. Net improvement in scores was noted in 60% of cases. Improvement in child status indices was most marked in Parabongo and Wol sub-counties. It was noted that the CBOs staff that were implementing the project activities were trained and provided with the capacity to undertake the activities. In addition, all volunteers that were involved in project activities implementation have been trained. Amongst the missed opportunities noted were difficulty in retaining trained HCT volunteers; and inadequate linkage and involvement of the VHTs. Interviews with the local government leaders (sub counties) revealed that the project design and implementation strategies were appropriate to the community served. The approach of implementation through local CBOs who in turn worked through local community volunteers brought services closer to the community and enhanced ownership of the project. The approach of implementation through the CBO partners and focused capacity building plus technical support during implementation, was designed to foster sustainability beyond project life. The use of community volunteers and mobilizing target populations into groups to undertake activities was also supportive of sustainability of activities beyond project life. Furthermore a deliberate process of handover was undertaken towards project closure whereby dialogue meetings were held with project stakeholders including project beneficiaries, community members and district and sub counties leaders. Efforts to link beneficiaries to development projects with similar objectives operating within the district are ongoing. Furthermore, referral linkages have been supported at community level through the community volunteers and with health facilities like Kalongo hospital and projects like CRS and AVSI that provide treatment and care services for PLHIV. Some external factors were noted to have had an impact on project activities for example split of Pader district in the second year of project roll out which necessitated re-mobilizing of support to 7 Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011 GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 7 the newly established district of Agago. A yellow fever outbreak also affected project activities in the third year and caused a slowdown of activities. In addition, project staff and other stakeholder transfers happened in the course of the project. The project design focused on implementation through existing community structures and this was appreciated and found to be effective as a capacity building strategy, by the stakeholders. In addition, measures to ensure sustainability like involvement of Local Government officials and establishment of community management commitments have been fostered to a great extent. During project implementation, a number of lessons were learnt that can facilitate project replication and can be shared with other partners implementing similar activities for greater impact. Lessons include finding that door to door HCT campaigns led to increased access to HCT services for the communities; usefulness of community volunteers in providing home based care services for PLHIV plus HCT mobilization; preference for secondary school support of OVC and relevance of implementation through local CBOs for sustainability. Amongst challenges noted were the negative gender cultural norms; dependency syndrome amongst communities as a result of years living in camps and receiving handouts; overwhelming community needs preventing universal coverage of project services due to limited resources e.g. not all needy children in a home were enrolled in the project. In summary, the project was successfully implemented. Targets were achieved and in some cases exceeded. A few missed opportunities were noted and these together with lessons learnt are documented in further detail within the report, for address in future programming. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 8 1.0 INTRODUCTION 1.1 District Profile Summary Agago District is bordered by Kitgum District to the north, Kotido District to the northeast, Abim District to the east, Otuke District, to the south, and Pader District to the west. Agago district headquarters is located approximately 80 kilometers (50 miles), by road, southeast of Kitgum, the nearest large town. This location lies approximately 370 kilometers (230 miles), by road, north of Kampala, the capital of Uganda, and the largest city in that country. Agago District is one of the newest districts in Uganda. It was established by Act of Parliament and began functioning on 01 July 2010. Prior to that date, it was part of Pader District. The district is part of the Acholi sub region. In 2002, the population of Agago District was recorded at approximately 184,000. 1.2 Location and Administrative Profile Figure 1: Map of Uganda Showing Location of Agago District Kl T Kil k Li P l Pd T Ot Pt Ll L ti A h lib Pjl Adil Lkl L Wl Pb Pi l P A At Ugand a GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 9 1.3 Background GOAL is addressing multi-dimensional aspects of the HIV pandemic in five districts of Uganda, namely Bugiri, Namayingo, Abim, Pader and Agago Districts. GOAL works with partner organizations to deliver comprehensive and coordinated responses to HIV specifically working on prevention of transmission of HIV, care and support to people living with HIV and mitigation of its effect on infected and affected individuals, families and communities. GOAL was awarded a PEPFAR New Partners Initiative (NPI) grant, allowing GOAL to increase its coverage of HIV and AIDS interventions, as a response to the accelerated resettlement process, which evolved over the life of the project. The goal of the project was to contribute towards the reduction of transmission of HIV and to mitigate the impact of HIV amongst the displaced, resettled and returned populations in Pader and Agago Districts (formed out of the previous Pader District). Over the life time of the GOAL PEPFAR project targeted eight sub￾counties in Agago district were targeted, including Lapono, Paimol, Parabongo, Omiya Pacwa, Lukole, Wol, Agago Town Council and Kalongo Town Council. The project was implemented in partnership with the two community-based organizations (CBOs) in the operational area: Wagwoke Wunu and the Prayer HIV/AIDS Group (an association of people living with HIV and currently with a membership of more than 250 people). Wagoke Wunu was responsible for the delivery of IEC/BCC activities under prevention, and Prayer Group delivered activities under care, support and mitigation. As part of these activities, both CBOs were responsible for community mobilization, volunteer management and the mobilization of target groups for project activities under prevention, care and support. 1.4 Objectives of the project and interventions utilized The principle objective of this project was to contribute towards the reduction of transmission of HIV and mitigation of the impacts of HIV amongst the displaced, resettled and returned populations in Pader (now Pader and Agago) District. Specific objectives (and strategies and interventions employed to achieve these objectives, presented in the sub-bullets) of this programme aimed to: 1. Bring about positive improvements in the knowledge, attitude, behavior and practice levels of the target population through: a. Improved access to targeted and innovative IEC/BCC services amongst the general public and youth through the Stepping Stones methodology8 (a behavior change communication prevention methodology9 ), targeted routine sessions through peer education and Community HIV Focal persons (CHFPs), as well as general community HIV awareness campaigns targeting youth as well as the general community.. The target beneficiary groups are: youth and adult groups for the Stepping Stones meetings, youth and general public in the community awareness meetings and the general public as well for the HCT component. 8 HIV prevention methodology; knowledge of this methodology is an added bonus. 9 Change of risky sexual behaviours and increase in testing as result of IEC: therefore please consider both aspects. Dimensions of risky sexual behaviour which GOAL has assessed previously can be obtained from the Stepping Stones outcome evaluation; you may want to suggest other aspects/dimensions for evaluation however GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 10 b. IEC through print and mass media. Target beneficiary group is the general public. 2. Ensure improved care and socio-economic support services to PLHIV, OVC and affected groups in the target areas through: a. Increased provision and coverage of home based care services. Target beneficiary group: people living with HIV. b. OVC Support care, through the provision of 3 or more OVC core program areas. Target beneficiary group: OVCs, OVC caregivers, Head/OVC Focal Teachers c. Increased coverage of livelihood and social support service provision (including pilot project with VSLA methodology). Target beneficiary group: PLHIV and OVC households 3. Strengthen the organizational financial and technical capacities of local CBOs and PLHIV Associations, including mainstreaming of HIV, gender and child protection. 1.5 Purpose and Objectives of evaluation The evaluation was to provide quantitative and qualitative assessments of the approach that GOAL utilized and determine whether GOAL achieved what it set out to do in this project. GOAL is also interested in continuing support to HIV service delivery so the evaluation results will be used as a lesson learning process to identify strong points and reasons for success and identify gaps that need to be addressed in future programming (and recommendations and possible existing resources that can be tapped into to respond to these gaps). The findings from this evaluation will also help GOAL Uganda decide whether approaches used can be scaled-up or replicated to other regions where there is a programming need. The main objective of the evaluation was to assess the relevance, effectiveness, efficiency, performance and the sustainability of the project interventions, draw lessons and make recommendations for similar future projects. Specific objectives of the Evaluation were to: 1. Assess the progress made in achieving the project results (intended, unintended, positive and negative outcomes in individuals and communities) in relation to the: a. project design and implementation strategies b. technical competency to undertake the planned activities, c. technical support and capacity building for partner organizations and other community structures d. relationship with project stakeholders and other service providers e. external factors f. The consultant will be asked to analyze baseline (2009) and follow up (July 2011) child status index information (already captured) to comment on any outcome changes in the OVC we support. g. An additional element should look at whether inclusion or exclusion in the programme or whether programme activities do not further stigmatize and/or discriminate beneficiaries and non-beneficiaries h. Assess the project design and appropriateness of the approach used including its sustainability in the context of service delivery and required need in communities GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 11 where GOAL works Determine the prospective need for the replication of the project and identify ways to maximize effectiveness in case of an extension or replication in similar or other geographical areas within Uganda 2. Assess the project design and appropriateness of the approach used including its sustainability in the context of service delivery and required need in communities where we work (please consider all of the above bullets as well in this regard) 3. Determine the prospective need for the replication of the project and identify ways to maximize effectiveness in case of an extension or replication in similar or other geographical areas within Uganda 4. Identify and document lessons learnt and best practices of the project. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 12 2.0 EVALUATION METHODOLOGY Combinations of qualitative and quantitative methods were used for this evaluation. Data collection was undertaken in the following main sites:  Offices of GOAL, partner CBOs, local governments in Pader and Agago districts  3 Health facilities that have participated in HCT services under the project; and have been referral centers for clients identified and sent through the project  3 Youth Centers supported by the project  Beneficiary/target communities in 2 sub-counties (one where project implementation is considered to have been relatively successful; and one where implementation has not progressed so well); and 1 Town Council targeted by the project. 2.1 Qualitative data Qualitative data was collected through focus group discussions; plenary group dialogue meetings, and key informant interviews. Table 1 below presents details about the respondent audiences and approaches to qualitative data collection used. Table 1: Proposed respondents and approach for qualitative data collection Respondents Data collection method Scope/Number Men, women and youth (both males and females) in communities that were targeted and reached by HIV prevention interventions supported through the project Focus groups of age and sex disaggregated respondents (as structured in Stepping Stones dialogue) Focus group sessions for stakeholders in the Stepping Stones process for each community Case study examples of individual and family level HIV risk reduction Key informant interviews with service volunteers in HIV prevention 4 FGDs in each in each of the 3 Lower Local Governments (LLGs) sampled 1 Plenary session in each LLG Stepping stones facilitators; 2 case examples of impact/benefit from stepping stones and HCT (individual or family-level benefit) 6 KII (2 SS Facilitators; 2 Peer Educators; 2 HCT providers) Community Volunteers; facilitators; Adults and children from households/ families that were targeted and reached by HIV care and Focus Group Discussion of adults Focus group interview with OVC 1 FGD in each LLG 3 KII in each LLG HCT volunteers; HBC, CCAs; OVC care givers and CPC members and OVC focal teachers OVCs GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 13 OVC support interventions of the project Case study examples of beneficial HBC service and SS 1 HBC and 1 SS examples (individual or family-level) Leaders in organizations involved in implementation of the project and related service institutions Key Informant Interviews 4 KII with GOAL staff 6 KII with partner CBOs 3 KII with local government staff 2 KII with ‘referral health facilities’ All project stakeholder categories (representatives) Open Evaluation Forum 1 half-day session for 3 LLGs A total of 12 focus group discussions, 13 key informant interviews, 3 case study documentations and 3 plenary dialogue meetings plus discussions with Agago and Pader Local government leadership were held. In addition, a total of 17 KIs were done with key project stakeholders. 2.2 Quantitative data analysis and review The quantitative approach focused on secondary analysis of data and reports relevant to project implementation and the monitoring, evaluation and learning processes therein. Key data in this regard included:  Child Status Index data from baseline (2009) and follow up (July 2010) surveys;  Stepping Stones outcome evaluation data from 2010 and 2011; and  GOAL annual Home Based Care and Stepping Stones outcome reports from 2009 and 2010. This was complemented with analysis of service records at 3 health facilities and 3 Youth Centers in selected project target sub-counties. Records extraction and analysis in these facilities focused on: 1. Education and Counseling provided for HIV prevention 2. Participation in HCT service promotion and provision 3. Provision of referral services for HIV prevention and AIDS care GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 14 2.3 Sampling and Data Collection Purposive sampling of the local government partners at district and sub counties was used. Three local government units viz Kalongo town council project area; A sub county where the activities of the project had been undertaken successfully and another where activities were slower, were selected purposively to give the all round picture of the project performance. The two organizations that were sub-grantees and implementing partners, Wagwoke Wunu and Prayers Group were purposively selected for interview. Beneficiaries that were in the sub counties selected were also purposively selected to give information on how the project had benefitted them, what sustainability measures were planned to keep activities going and what could have been done better for greater impact. After the individual interviews and focus group discussions, representatives of a cross section of the respondents were invited to a plenary session at the sub county to give further input and verify the findings from the evaluation. Data was collected during meetings organized by GOAL and the two partners. The GOAL team and Partner organization teams were not part of the interview process at sub county level. A copy of the tools used during the evaluation, are appended (Appendix 4). Tools used include key informants interview guide, focus group discussion guide with beneficiaries and in-depth discussion guides with facilitators. A tool for discussion with OVC beneficiaries was designed to elicit information from the children through drawing. 2.4 Data Analysis and Report Writing Data collected was used in compiling summaries to address the evaluation objectives. The proceedings of the plenary sessions for each sub-county and the town council were also summarized and included in the analysis. The main themes emerging from the analysis were then used in generating the project evaluation report. A documents review process also helped to get further project performance information and this too was used in compiling the report. GOAL project team were also consulted over key emerging themes and consulted over areas where clarity was needed for the consulting team. 2.5 Quality Assurance and Quality Control measures The consulting team designed standardized data collection tools for the data collection exercise. A process of orientation to the tools and tools review was undertaken by the two interviewing teams. A session of review and refining of the tools was held with the GOAL team before commencing the data collection Selection and mobilization of the required respondents was undertaken by the GOAL and partner teams allowing the evaluation team to complete all required interviews with key project beneficiaries and key informants at all levels. A review of the collected data and compiling of the thematic summaries from the raw data forms was undertaken by the team leaders by collating responses from similar respondents across sub counties. These were compiled into the report. A meeting to review findings was undertaken with GOAL team for clarification purposes. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 15 Triangulation of information provided was done through meetings with GOAL staff, further interviews with CBO staff, 3 final Sub county plenary sessions and a project documents review. 2.6 Limitations in the assessment process    The exercise had to be undertaken in 8 days. The assessment was dependent on reported feedback from the beneficiaries. It was not possible to undertake observations of the reported activities. Some of the effects of the project may be seen well after a longer period beyond the project closure particularly with regards to sustainability. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 16 3.0 EVALUATION ASSESSMENT FINDINGS This section outlines the progress made under each of the project objective. 3.1 Improving knowledge, attitude, behavior and practice levels of the target population The project set out to improve knowledge, attitude, behavior and practice levels of the target population through improved access to targeted and innovative IEC/BCC services amongst the general public and youth through the Stepping Stones methodology; and IEC dissemination to the general public through print and mass media. Findings on the progress with this are summarized below. Stepping stones methodology as an innovative IEC/BCC strategy has been successfully implemented by the project with increased knowledge of communities about HIV risk and risk reduction behavior. It is noted from documents review that through SS, there was increased knowledge about HIV and AIDS; Increased Uptake of HCT; reduction in Risky Behaviors- “I used to be a drunkard and my wife was about to leave me before the training begun so I was saved by this training because when I look at myself now and before, there is a lot to be appreciated out of the training, I think I would even be dead by now if this intervention had not come in handy “(FGD, Older Males, Wol Sub-county; Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011) Furthermore, it was noted in the same report that “men from different locations visited reported that they reduced their sexual encounter with multiple partners, with some reporting that they were now faithful to their spouses. The men also reported that this had enabled them to reduce their movements and to spend more time with their families. As a result there was increased trust and love among spouses and thus more harmony in families. There was also a reported decrease in transactional sex in both districts but the practice was reported more persistent in urban areas.” Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011). Other benefits noted include better communication between spouses/partners and between parents and children. “Participants reported improved communication between spouses on subjects that were previously unlikely to be talked about such as condoms, sexuality, family planning, and money matters. Women also reported more freedom to talk about family planning and the need for education of their children. It was reported by several women especially the young ones that they were as a result able to space their children unlike before” Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011). Other outcomes noted were improved Gender Equality at home and Better relations between Family Members; Reduced Conflicts and Gender Based Violence at Home- “To me the most important bit of the training was on alcohol because before I used to drink and even fail to plan for my family and in most cases when I drink, that is the time I go for women and also bring violence in my house, so the way I see changes in my life, alcohol was bringing me other problems so when I left drinking, all my other weaknesses disappeared” (FGD, Young Males, GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 17 Lapono Sub-county; (Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011). The Stepping stones evaluation also showed that there was increased acceptability of condoms as a means of preventing HIV infection as well as for family planning in marital settings, contrary to previous perceptions that linked condoms to promiscuity. There was also reported increase in condom use evidenced by the fact that people were buying and getting free condoms from health service providers such as GOAL. People also acquired skills of how to properly use condoms. From the quantitative data, of those who reported recent sexual activity (N=244), those who had used a condom at last sex were 21.7% (n=53). Condom use at last sex was reported more among males compared to females in Bugiri, but more among females in Pader; and higher among younger males and females compared to their older counterparts. Out of those who had had sex with a non-regular sexual partner, only 35.9% (n=23) had used a condom (Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011) Participants in all FGDs reported reduced rates of alcohol consumption, also attributed to Stepping Stones which had transformed most men and women. The quantitative results show that only 37.8 of the respondents reported that they consumed alcohol as at the time of this study. Alcohol consumption was more prevalent among males (60.4%), compared to females (24.3%), more prevalent among older age groups for both sexes (Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011) During the final evaluation exercise, focus group discussions, it was reported that there was reduction in Unwanted Pregnancies; reduced stigma and discrimination against PLHIV; reduction in risky cultural practices for example, the practice of widow inheritance was reported to have tremendously decreased though it still existed. Most participants in Pader reported that people now test for HIV before going into widow inheritance (Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011. Condom distribution to reduce on the rate of HIV infection of HIV and early pregnancy were also promoted using SS approach10. These observations were also made in Wol and Paimol during the evaluation exercise. It was noted from the Stepping stones discussion that domestic violence had reduced (ref case studies, appendix 3). Issues with gender violence in the community were however reported during focus group discussions in Kalongo TC. It was reported that there was a lack of care for families and pregnant mothers by males that led to fights, in addition to reluctance to seek services. Other outcomes shared by the beneficiaries of stepping stones included resumption of fireplace stories to children; increased condom use promotion; reduction of stigma and discrimination as evidenced by increased openness about HIV/AIDS status; increased uptake of HCT; reduction in wife inheritance practice.(FGD Wol Stepping stones) From the GOAL Annual PEPFAR report 2008/2009 it was noted that the stepping stone approach had enabled community members to be more assertive and take a more personal, social and community wide responsibilities for their actions. Furthermore, it was observed that the 10 Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011 GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 18 stepping stone sessions helped in clearing the myths and misconceptions about HIV/AIDS and created a more acceptable environment for people living with HIV (PLHIV). Men from different locations visited reported that they reduced their sexual encounter with multiple partners, with some reporting that they were now faithful to their spouses. The men also reported that this had enabled them to reduce their movements and to spend more time with their families. During the final evaluation, the communities interviewed showed willingness to continue with the SS model to address other community actions on issues such as road construction (SS was seen as a strategy that can assist in addressing challenges in HIV risk behavior as posed by ongoing road construction works within a community), environment hygiene, and sanitation improvement. Almost all the stages and components of the Stepping Stones were judged by the different stakeholders as having been successful. What appears to have been particularly striking was the relevance of the content to the issues in the local contexts. It is evident that participants were able to use the training sessions to identify and discuss the most critical problems that are part of their daily lives. SS peer group members served as role models, who cause a positive influence on their children, their spouses, and the community at large. The SS members have also been actively teaching others about what they learn, thus having a multiplier effect. The drama used by the Stepping Stones groups appears to have been an effective behavior change communication channel. In some instances, flexibility and innovativeness were applied to overcome emerging challenges such as the provision of mobile HCT in Kalongo to address the lack of easy access to VCT at health centres. This was a good practice that contributed to programme success (Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011). Community HIV Focal Persons: A total of 25-30 CHFPs were trained per Sub County. Community HIV Focal persons (CHFPs) facilitated targeted and general community IEC/BCC sessions. CHFPs also implemented targeted community IEC/BCC sessions with groups of not more than 25 individuals, for a three month period. The focus of these sessions included topics such as faithfulness, awareness on sexual and gender based violence and the promotion of T&C and PMTCT services. Training and empowering of PLHIV and community through the targeted sessions led to increased community mobilisation for HIV prevention, referral and follow up. (Focus group discussions, Wol, Kalongo, Paimol) Increased access to HCT services It was noted in the interviews that HCT services access to the communities has scaled up and targets were reached. The HCT program was able to reach 24,446 (10,812 M and 13,634 F) over the life of the project. These individuals were reached with HCT services offered through seven static sites and HCT outreaches carried out in 32 Parishes. The project implemented a Home Based HCT and scaled up support to static sites from 3 to 7 over the life of the project to reach and exceed its target for the number of people accessing HCT services. Results from the efforts include: GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 19 o Increased access to HCT services due to the introduction of HCT volunteer cadre and provision of HCT services at outreach sites o Roll out of Home based HCT services (door to door) o Procurement of and provision of HCT testing kits to targeted HCIIIs leading to a reduction in HCT supplies and logistics stock out as a result of GOAL support e.g. testing kits, drugs for care plus buffer stocks were provided at HCIII,. IEC targeting youth and the community As a result of the bill boards’ messages set up in trading centers, (with photographs of the people known to the community) engagement with IEC messages was noted. The messages were appreciated and raised meaningful dialogue about issues communicated. In addition, the youth centers created a youth friendly space that brings youth together for peer education through routine targeted sessions and interactions that fostered increased awareness about HIV and how to prevent it e.g. all youth at the interview in Paimol Youth center (20) had undergone HCT and knew their HIV status as a result of peer mobilization during interactions at the youth center. It was also noted that youth were drawn to youth centers for games and the peers used this opportunity to share HIV prevention messages and mobilize them for HCT. There was also reported reduction of stigma and discrimination among youth living with HIV and enhanced positive living of youth living with HIV (peer educators during the focus group discussions in Wol and Paimol). Radio Messages& Talkshows: Project design enabled FM radio messages coverage to reach men and women, young and old. Two radio stations were used. During the life of the project, 52 radio programmes and messages were aired, reaching approximately 30,000 people in each year. This estimate was based on GOAL’s 2008 KABP survey findings, that 50% of the target population (127,000) attributed their knowledge on HIV to radio broadcasts. It was noted that radio talk shows listenership extended beyond the project area from interviews with district and CBO partners. Knowledge, Attitude, Practice and Behavior Annual surveys analysis of progress in HIV prevention GOAL has undertaken annual KAPB surveys as a way of understanding the situation within the areas where the programme operates. Findings reflect improvement in indicators related to HIV prevention, as illustrated in the table below, which in part can be correlated to GOAL’s work within the districts (although it cannot be strictly attributed to it alone as GOAL’s catchment area is smaller than the surface area/ spread of the two districts) GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 20 Table 2: Trends in HIV prevention indicators status in Agago and Pader districts, 2009- 2011 HIV May￾09 Feb-10 Apr-11 % of respondents with knowledge of at least two ways of prevention of HIV 33.80% 57.40% 50.20% % of respondents who indicate that they would support/accept community member who was living with HIV 75.60% 90.00% 83.20% % of respondents who know at least two ways of HIV transmission 46.10% 46.87% 68.90% % of respondents who have ever heard of VCT 83.90% 93.80% 99.60% % of respondents who have heard of and know how to access VCT 96.10% 83.30% 93.40% % of respondents who have ever tested for HIV Total 79.60% 62.20% 84.20% Males 74.30% 60.00% 81.80% Females 81.10% 64.30% 84.40% % of respondents who would be willing to disclose their HIV status to spouse 52.40% 42.30% 53.80% % of respondents who would tell no-one of their HIV status 7.50% 20.10% 9.50% % of respondents able to identify a condom who know that it can be used to reduce the risk of STIs and HIV infection 85.60% 75.60% 98.70% % of respondents able to identify a condom who know where to access them in the community 83.10% 91.90% 91.20% % of respondents who used a condom during their last sexual encounter 9.40% 10.50% 11.80% % of respondents who would go to a clinic for treatment of an STI 85.70% 89.30% 90.50% % of respondents who would notify their sexual partner if they had an STI 6.50% 1.20% 4.40% Source: GOAL KABP Report 2011 for Agago and Pader Districts The PEPFAR-project was the major support to HIV prevention interventions in the Agago area over the years covered in the table (2009 to 2011). The positive changes reflected in the indicators are correlated to the efforts of the programme in the same years. Improvement is noted in general for all indicators except for disclosure and notifying partner if one had an STI. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 21 3.2 Improving care and socio‐economic support services to PLHIV and affected group in the target areas Improvement in care and socio-economic support services of PLHIV and affected groups in the target area was addressed through home based care and livelihoods support to target beneficiaries including OVC and PLHIV. Findings on project progress under this objective are summarized below The project provided Home Based Care support to 906 (301 M and 605 F) PLHIV over the life of the project through trained Community Counseling Aides. The outcomes noted as a result include, reduced stigma and discrimination and increased disclosure of PLHIV status due to improved care and support for PLHIV. From the outcome evaluation report 2010 it was also noted that there was reduction of incidence of illness most notably malaria and diarrheal diseases amongst PLHIV, and improved sense of well-being amongst beneficiaries. The beneficiaries reported that reduction of malaria was due to sleeping under mosquito nets (75%). Beneficiaries were also supplied with chlorine and water vessels which they use for water purification. These were verified to be in use for 74% of a sample of beneficiaries supported by prayer group during the evaluation11. The project also provided Home based care kits to PLHIV that contained items for prevention of water borne infections (Waterguard and water containers), prevention in positive people (condoms), leaflets on positive living, oral re-hydration salts, and long-lasting insecticide treated bed nets (LLITNs). Community Counseling Aides: The project targeted to train 130 CCAs. 121 CCAs have been trained in the delivery of home based care during the project of which 119 are active. PLHIV have accessed home based care services and been supported by their communities. For example, it was reported in Wol that the community members support gardening and growing of foodstuff for PLHIV when they are bed ridden. Furthermore, through education to community members, the level of stigma to HIV/AIDS is gradually going down. Training and support supervision of community volunteers that implemented the activities at community level was provided by the CBO facilitators with oversight from GOAL team. Livelihoods programming: The project provided 750 PLHIV and 1000 OVC households with livelihood support which included training in vegetable gardening and poultry rearing, and the provision of gardening inputs such as tools, seeds and poultry. The project through Prayer Group has also mobilized 750 beneficiaries and trained them under the VSLA intervention and fostered a savings culture. Furthermore, the project also fostered a culture of stocking of seeds for the following season. Previously, all produce was sold off leaving no seeds stock for the following season. 11 Home based care and support group programming. Outcome evaluation report, November 2010. GOAL Uganda. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 22 Other positive results were noted under this objective as indicated below: Revitalizing decentralized HIV & AIDS structures: Under the decentralized HIV&AIDS response, coordination of HIV&AIDS activities is undertaken by District HIV&AIDS committees and a similar structure at the sub county level. As a result in part of issues arising in light of the project implementation, and participation of GOAL and its partners in District Coordination platforms, plans are underway to revive the Sub￾county AIDS Committees, to ensure improved coordination and mobilization of support towards HIV programs at sub county level. Revitalization of this sub-county level coordination body will mean strengthened linkages and channels to the district and national HIV & AIDS response efforts. The realization of the need for revitalizing the decentralized HIV & AIDS structures at sub￾county level is a positive result of project efforts. Village Savings and Lending Associations: VSLA activities are to be promoted to sustain continued support of children in school e.g. setting up a group farm for group members to grow produce for sell. Group members have been mobilized to continue activities as a group beyond project life “We will not leave our group and we will show our aim and commitment to our community” (Paimol sub-county) As per original project design, the Livelihoods program has been supported under GOAL cost sharing fund. OVC support: The project provided support to a total of 1,001 (508M & 493 F) OVCs with comprehensive support including food and nutrition, education support, psychosocial support, health care services, and child protection activities. Supported OVCs were selected with the involvement of the local sub-county councils and through schools, and churches. The selection criteria focused on the neediest OVC to be able to bring beneficiaries numbers down to manageable numbers within available resources. It was noted that the OVCs were many in the community and that the project could only enroll those that were identified as being most vulnerable based on the selection criteria and process. The project also supported Child Protection Committees in providing referral linkages for OVC caregivers on issues related to child protection12. CPC members shared that due to the interaction between CPCs and the parents/guardians and the OVC, there was better relationships between parents/guardians, teachers and OVC. CPC members further observed that there was a reduction in emotional trauma amongst supported OVC and that the OVC settled and were able to concentrate on their studies. A teacher was assigned to support OVC activities at each school. This in most instances was the head teacher. Through this approach focused attention to OVC was provided. (Wol Parent/Guardians/ CPC members and Teachers’ Focus group discussion) 12 GOAL Scaling up Innovative HIV/AIDS Interventions through Strong Partnerships amongst Conflict-Affected Populations of Pader (Agago) District, Northern Uganda, 2008 -2011. Annual Report 2010/11 GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 23 Amongst the positive outcomes noted from the OVC support, there was improvement in relationships between the parents, guardians and children due to improved parent guardian child communication. This was noted with some OVC as communicated in Wol and Kalongo during focus group discussions. Contrary to this overall finding, in other sub counties strained relations between OVC and guardians were noted to be rampant e.g. in Paimol, and this was re- echoed at the Paimol sub county plenary session. It was noted that there is a misunderstanding over roles and responsibilities of children versus child’s rights. It was noted during the plenary session in Paimol that this issue should be followed up and addressed in future programming for OVC. Supported OVC reported that they had hope for the future and shared what they want to become in life. Amongst future aspirations mentioned by OVC were a President, teachers, doctors, lawyers and nurses. This showed that OVC were no longer hopeless as they had a vision for the future. Enrollment of school going children increased due to the project’s support and children without birth certificates were issued with one. In addition, OVC shared that before GOAL support, they were involved in child labour to get money for scholastic materials. With the advent of GOAL support, they shared that there was reduction in child labour Community support to OVC and child headed households was also noted to have improved. Support mentioned included provision of food and support in making shelter (Wol Parent/Guardians, CPC members and Teachers’ Focus group discussion). OVC beneficiaries were linked to the sub-county e.g. list of OVC taken to sub county and registered at local government. Analysis of Child Status Index Data GOAL has undertaken Child Status Index M&E assessment exercises to track progress under OVC support. It was possible to link 388 records (by name, age and sub county of residence) in the first and second round Child Status Index measurements, for March 2010 and June 2011 CSI data. A total of 367 children among these (94.6%) had records for 10 or more of the 12 domains, and these were included in the analysis. Overall, the basic analysis conducted revealed limited change in the overall and domain-specific scores between the two points of measurement. The average CSI scores by domain are generally average, in the range of 2.3 to 3.4; and are largely similar at the two points of measurement, as shown in the table below. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 24 Table 3: Comparison of Child Status Index scores 2010 and 2011 Domain Average score – Mar 2010 Average Score – Jul 2011 1A: Food security 2.67 2.97 1B: Nutrition & Growth 2.87 3.01 2A: Shelter 2.29 2.60 2B: Care 2.71 2.82 3A: Abuse & Exploitation 3.25 3.23 3B: Legal Protection 2.86 3.12 4A: Wellness 3.08 3.12 4B: Health Care Services 3.05 3.10 5A: Emotional Health 3.07 3.24 5B: Social Behavior 3.23 3.22 6A: Performance 3.38 3.40 6B: Education & Work 3.27 3.28 Source: GOAL Child Status Index data 2010/11 The graphs below illustrate the change in scores, across different parameters of dis-aggregation. Fig 2: Overall progress in Child Status Index scores, 2010- 2011 Source: GOAL Child Status Index data 2010/11 The left hand axis (y axis) indicated the number of children with an overall improved score across all parameters. The x axis represents the grades of improvement, no improvement or decline in performance. The total scores show that 60% of cases reported a net improvement; there was a decline in 36% of children and no net change in 4%. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 25 Fig 3: Child Status Index changes by sub county 2010-2011 Source: GOAL Child Status Index data 2010/11 Dis-aggregation of scores by sub-county shows a similar general trend. Improvement is more notable in Parabongo (the green line) and Wol sub-counties (the orange line) as evidenced by the larger number of children from these sub-counties appearing in the ‘improved’ section (the 4 sections to the right of the ‘static’ column. The trend in Omiya-Pacwa (the black line) reflects more decline than improvement (i.e. larger proportion of children showing a general decline on parameters used in the survey than those showing improvement). GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 26 Fig 4: Changes in Food security, Nutrition, Shelter and Care scores Source: GOAL Child Status Index data 2010/11 Overall, about one-third of the children remained static at the two points of CSI measurement, in March 2010 and June 2011; at 32% with respect to 1A (food security, blue line), 37% on 1B (nutrition and growth, maroon line), ; 30% on 2A (shelter-green line); and 34% on 2B (care, black line). Improvement was slightly higher for food security (at 43%) compared to nutrition and growth (37%); and for shelter (at 44%) compared to care (38%). GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 27 Fig 5: Changes in Child Protection and Health scores Source: GOAL Child Status Index data 2010/11 The proportion of children that remained static over the period assessed was similar on all four parameters of child protection and health; between 34% and 38%. Improvement was higher for the legal protection domain (3B, at 41%, maroon line,), compared to the other domains which ranged from 31% for 3A (Abuse and Exploitation, blue line); through 34% for 4A (Wellness, green line); and 36% for 4B (Health Care Services, purple line). GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 28 Fig 6: Changes in Psychosocial and Education scores Source: GOAL Child Status Index data 2010/11 The largest proportions of children were static for both the Psychosocial and education domains; at 41% for both emotional health (5A, blue line) and social behavior (5B, maroon line); 48% for educational performance (6A, green line); and 40% for education and work (6B, purple line). Improvement was lower for social behavior (at 29%) and educational performance (at 28%); and better for education and work (at 33%) and for emotional health (at 37%). These results may be a reflection of the limited time of intervention; in view of the fact that the OVC National Strategic Program and Plan for Implementation (NSPPI13) advocates for a minimum intervention period of three years; in order to attain meaningful and sustainable change in OVC status. Another factor that may explain the scores may be the timing of measurement, which may reflect seasonal differences (e.g., in food availability, family focus on school grades, etc.). 13 The OVC National Strategic Program and Plan for Implementation 2005/06 -2009/10 is costed based on this projection (page 55)., MoGLSD GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 29 3.3 Strengthening the organizational capacities of local CBOs and PLWHA Associations, including mainstreaming of HIV, gender and child protection Two CBOs as planned were supported to implement innovative IEC/BCC HIV prevention approaches plus Care, support and mitigation activities. The geographical and population coverage of the CBOs increased following GOAL support. Geographical coverage of the CBOs area of operation increased over the life of the project from two sub counties to eight sub counties by Wagwoke Wunu, and from five to eight sub-counties by Prayer Group Prayer group started as a CBO in 2003 with 23 members. Wagwoke Wunu started operations in 2005 and has been funded by World Food Progamme, German Agro, Action AID, NUMAT and more recently GOAL. Both CBOs have a board of directors and operational staff. i. Strengthened organisational capacities of targeted CBOs and PLWHA associations The design of the project provided for strengthening the CBOs’ management. It was noted during the interviews that internal control systems of both organizations were improved. Financial management policies, internal and external audits were instituted as a part of the process of organizational systems strengthening. Technical support and capacity building for partner organizations and other community structures Through GOAL’s Partnership Approach, the two CBO partners received capacity building in the form of training and on-site technical support and mentoring, delivered by the GOAL Technical Officers and Partnership Officer. The CBO partners on the other hand were also provided training: The trainings are part of GOAL’s Partnership Approach. This capacity building was delivered to CBOs directly by GOAL through the Capacity Building Manager and the GOAL project officers including the Partnership Officer and various Technical Officers. Trainings included: a. Gender mainstreaming b. Child protection c. IEC/BCC d. Stores management e. Financial training f. Proposal writing g. Governance and Management h. Project cycle management i. Advocacy j. Human resources k. Strategic Management GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 30 It was noted that the CBOs staff that were implementing the project activities were trained and provided with the capacity to undertake the activities. Wagwoke Wunu staff that was undertaking prevention activities had been trained in IEC, BCC, peer education and stepping stones methodology implementation. Wagwoke Wunu has a total of 5 facilitators who provide technical support to volunteers in implementation of community level activities. The Project Coordinators (both partners) also make visits to the field to verify field visit reports and ongoing activities, to ensure quality. Prayer group staff members were provided with skills as Trainer of trainers for Home based care and Community counseling Aides training and support supervision. Prayer group activities are implemented through community volunteers supported by the facilitators. All volunteers that were involved in project activities implementation had been trained. For example, counseling training was conducted for the community counseling aides. Furthermore, health centers’ counterparts that were involved in the project were also trained. Particularly for HCT services delivery. In addition, there was regular support supervision and technical support from GOAL technical officers to CBO partners to ensure that implementation was being undertaken within guidelines provided and that quality services were delivered to communities served, by the CBO staff. Partner Organizational Capacity Assessment (OCA): An Organizational Capacity Assessment exercise is facilitated on an annual basis, by the Capacity Building Manager - GOAL Uganda. Partners assess their capacity in 7 areas namely Governance, Management Practices, Human Resource Management, Financial Resource Management, Service Delivery, Partnering and Sustainability using the GOAL Organizational Capacity Assessment Tool (OCAT). Based on the assessment, partners create a work plan to address capacity building in each capacity area. In subsequent years, the partners are also assessed against the progress made against their work plans. The purpose of GOAL Uganda’s OCAT includes: - To guide the partner NGO assessment team through a number of questions referring to the organizational capacity area - To score the level of capacity from personal views and experience - To provide information that is analyzed to identify critical issues that need to be worked on as a process to improve the NGO services to the beneficiaries. This process has assisted the CBO partners to proactively work on gaps and build areas of weakness It was noted that the CBO OCA process by GOAL has resulted in improvement as illustrated in the results presented in the Table below. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 31 Table 4: Trends in Organizational Capacity Assessment for GOAL CBO partners (Prayer group and Wagwoke Wunu) Capacity Area OCA -2009 TOCA -2010 CLOCA -2011 Organizational Development 3.1 3.7 4 Program Management 3.2 3.9 3.7 Financial Management 4 4 4 Administration 3.8 3.8 4 Human Resource Management 3.8 3.8 3.9 Project Performance Management 2.7 3.8 3.8 Governance 3.8 3.5 4 Source: GHH-A-00-09-00001 Annual Report Oct 10 – Sep 11 It is worth noting that the OCA process for 2010 combined both organizational and technical elements specific to three areas of project focus (OVC support, HIV Counseling and Testing; and AIDS Care and Support). However, the process did not specifically assess other elements of HIV prevention (beyond HCT). OCA exercises were done for both CBO partners on an annual basis, and these resulted in clear improvements as reflected in the scores presented in Table below. Table 5: Trends in Organizational Capacity scores for the GOAL CBO partners Timing of OCA Prayer HIV/AIDS Group Kalongo Scores Wagwoke Wunu Scores Apr 2009 24% 40% Apr 2010 84% 85% Apr 2011 81% 85% Progress in OCA work plan implementation Apr 2010 88% 67% Apr 2011 76% 86% Source: GOAL Partners OCA Report Sept 2011 GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 32 As a grantee under the New Partners Initiative (NPI), GOAL Uganda staff participated in a number of trainings to improve technical capacity as well as address organizational capacity building, delivered by NPI’s Technical Assistance partner, NUPITA. Technical assistance and capacity building was delivered in relation to a number of technical and organizational areas including; OVC Management, Quality Assurance and Improvement, NPI Close-Out, HIV Prevention, Referral, Financial Management, Documentation and Communications, and Resource Mobilization14. ii. Organisational Responsiveness to HIV through Mainstreaming At CBO level, both organizations have developed workplace HIV/AIDS policies. Scale up of outreach activities were aimed at reaching communities as they resettled after conflict with the realization that HIV prevalence and risky behaviour was high amongst the post conflict communities. 14 GOAL Scaling up Innovative HIV/AIDS Interventions through Strong Partnerships amongst Conflict-Affected Populations of Pader (Agago) District, Northern Uganda, 2008 -2011. Annual Report 2010/11 GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 33 Table 6: Summary of Evaluation Findings of GOAL Uganda Scaling up Innovative HIV/AIDS Interventions Through Strong Partnerships Amongst Conflict Affected Populations of Pader (Agago) District, Northern Uganda, 2008 -2011, for objective 1 and 2 Area Outputs Outcomes Improved access to targeted and innovative IEC/BCC services amongst different sections of the target community Stepping stones: 1,923 (712 M and 1,211 F) reached with Stepping Stone sessions 2008/09 2010 3300 participants and 132 groups trained 2011 6500 participants reached in 65 groups CHFPs targeted community HIV education sessions coverage Peer Educators – targeting Youth: 2011 Youth-focused messages and activities, including drama and puppetry shows. A total of 6,963 youth (3,768M and 3,195F) reached through targeted sessions and 26,892 (14,050 M and 12,842 F) reached through general campaigns IEC 2011: 1,000 posters, 2,396 t-shirts, 1,000 leaflets, 200 brochures and 8 billboards Increased Community Knowledge about HIV and AIDS in the 8 sub counties Increased Uptake of VCT services in the 8 sub counties Reported Reduction in Risky Behaviors in the 8 sub counties Better Communication between Spouses/Partners and between Parents and Children Improved Gender Equality at home and Better relations between Family Members Reported Reduced Conflicts and Gender Based Violence at Home Reported Reduction in Unwanted Pregnancies in the 8 sub counties Reported Reduced Stigma and discrimination against PLHIV; - Stepping stones fostered disclosure amongst PLHIV Reported Reduction in Risky Cultural Practices in the 8 sub counties o e.g. wife inheritance ; sex before marriage, fighting, drinking and may others (Reports from Key Informants & Beneficiaries during Evaluation exercise) Reported Reduction in Risky Cultural Practices in the 8 sub counties GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 34 Area Outputs Outcomes Radio shows: 2011 25 radio programmes Increased access to HCT services Home Based HCT services: Outreach HCT services: Static HCT services: 7 health facilities supported to provide HCT services 25,519 (11,279M & 14,240F) individuals with T&C services through support to 7 static sites and outreaches services Referrals: 9,154 referrals made by community volunteers and 6,403 Increased HCT coverage in the 8 sub counties Increased coverage of care in the 8 sub counties Improved care and socio economic support services to the PLHIV, OVCs and affected groups in the target areas 2011 906 (301 M & 605 F) individuals with HBC services 121 CCAs (65 M & 56 F) provided services 1,001 OVC (508 M & 493 F) supported with poultry and fruit seedlings. Also given farming tools Increased coverage of care to 906 beneficiaries in the 8 sub counties Increased coverage of Home based Care to 906 beneficiaries in the 8 sub counties Reported Reduced Stigma and discrimination in the 8 sub counties Improved Livelihoods of PLHIV in the 8 sub counties. Education of 1001 OVC Improved livelihoods for 1001 OVC Child protection sensitization to guardians and community and, HIV awareness, health care support and referrals for OVC GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 35 3.4 Other General findings Project stakeholders and other service providers’ interactions Goal team interaction with stakeholders included the stakeholders meetings and feedback sessions. Implementing CBOs also reported that they actively interacted with Sub County and district authorities as a result of the project technical capacity building activities extended to them. The CBOs contributed to the district planning processes. Other interactions included monthly meeting with volunteers e.g. CCAs for client feedback; Quarterly meetings with the OVC care givers stepping stones facilitators and leaders and teachers; meetings with peer groups; HCT volunteers’ coordination meetings at health facilities. Sub granting to Local CBOs for direct Implementation Interviews with the local government leaders (Sub counties) revealed that the project design and implementation strategies were appropriate to the community served. The approach of implementation through local CBOs who in turn worked through local community volunteers brought services closer to the community and enhanced ownership of the project. Community members too, reported that they had found the approach responsive to their needs. (Plenary sessions Wol, Paimol and Kalongo TC) CBO partners supported by the project also noted that the approach of capacity building and sub granting to them had strengthened their organizations. As a result, they are now in final stages of registering as NGOs after fulfilling required criteria, upgrading from CBO level. It was noted that all aspects of the project were implemented through the local CBOs except for HCT which was implemented directly by GOAL. Use of community volunteers Under project design, CBO partners had facilitators that undertook training and sensitization sessions with the community volunteers and supervised the community level activities. Activities were implemented through lower level community structures, like the Peer educators, CCAs CHFPs Home based care volunteers OVC care givers .This approach ensured that the project activities reached the communities at village level Referral and partnership building for comprehensive care services At community level, the network of volunteers provided information to the community members and beneficiaries for relevant health services and hence was the first level of referral to health facilities at parish and sub county level. Onward referral for care services was instituted at health facilities for purposes of achieving comprehensive care and treatment services for beneficiaries. PLHIV were referred to the CRS program at Kalongo Hospital for ART. Through project support, referral boxes and registers were introduced for referral follow up in consultation with health service providers. This system helped to track how many referred clients actually reported to the health centre to which they were referred, GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 36 Gender Gender mainstreaming is one of the key approaches used by GOAL in all its programming. At a program level there are gender focal people who do internal mainstreaming and ensure that at the programs/interventions level the gender considerations are addressed. M&E data is sex disaggregated and the data is routinely reviewed to ensure that there are no barriers created through the different interventions. Recruitment of staff was responsive to Gender and the CBOs facilitators recruited include both men and women; with Wagwoke Wunu- having 5 staff (2 men and 3 women); and Prayer group having 8 staff (2 women and 6 men). Also the beneficiaries and community volunteer selection took gender responsiveness into account. As regards activities with the communities, gender dynamics were noted as follows:  It was noted that where couples are targeted in stepping stones then the outcomes to gender based violence improve  Some cases of stigma and domestic violence were reported by women after returning home with project inputs or supplies which their male partners feared would expose their HIV status to neighbours and the wider community. (Focus Group discussions, Kalongo, Paimol)  The outpatient attendance at Paimol health centre III was 98% women; this was also noted in Wol. Stepping Stones’ design, which consisted of gendered peer groups, was gender sensitive and in the context of decisions made during the group meetings e.g. the time for the meeting was decided after consulting women and men on the most appropriate time, giving the group members room to do their chores that would otherwise prevent them from participating if meetings were inappropriately timed. Gender dynamics were also noted in Youth activities programming. It was observed that more female youth go for HCT services than the male youth. Furthermore, games have been used in mobilizing youth for HIV prevention sensitization. It was noted however that the games available to date mostly target male youth and this creates a missed opportunity for female youth. Despite this, however, project HCT targets for males, were achieved. One of the girls in the focus group mentioned “Girls would come to the youth center more if there are activities like learning how to weave, crafts, pottery, cooking lessons etc but now I come only if there are trainings and this too depends on the timing of the training, if I am free”. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 37 3.5   Project design and appropriateness of approach used (Sustainability, community needs and       address of post conflict context)   The evaluation process also focused on the project design and approach used to assess their appropriateness to sustainability, community needs and post conflict programming. The following subsections are a summary description of findings under this evaluation objective. 3.5.1 Sustainability It was noted that sustainability of activities had been planned for right from design. The approaches to foster sustainability as catered for in project design are summarized below: a. Capacity Building and CBO Systems strengthening: The approach of direct implementation through the CBO partners and focused capacity building plus technical support during implementation, was designed to foster sustainability beyond project life. The efforts by GOAL in Management and Governance capacity building for the CBOs empowered and prepared CBOs organizations to carry on activities beyond project support within a strategic planning framework. b. Use of local community volunteers and structures The use of community volunteers and mobilizing target populations into groups to undertake activities was also supportive of sustainability of activities beyond project life. e.g. HIV- CHFPs (141) in 8 sub counties The pool of community volunteers that have emerged out of the project can also be used during local initiatives or campaigns that may be implemented by the sub-county or district levels. Direct support to already existing structures such as schools (head teachers and OVC focal teachers) and child protection committees are another available community resource. c. Sustainability of Youth Peer activities Youth centers committees were set up and oriented to their roles in managing the youth centers. . This has fostered local ownership, management of youth centres and continuity of youth centre activities that will continue beyond project life. Youth peer educators have already contacted the sub counties for prospective partnership and integration of their activities into the sub county plans. In line with sustainability planning, Kalongo Town Council youth have already submitted their details to the district for registration as a CBO to continue with the peer education work as a group d. Partnerships and collaborations: Efforts have been made to establish partnerships to support project activities. The Livelihoods project has been linked to district NAADS program. Efforts to link beneficiaries to development projects with similar objectives operating within the district are ongoing. Efforts to transfer GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 38 clients and beneficiaries to other projects or programmes where they can access similar support has been made where possible e.g. transfer of OVC to World Vision support. CBO partners have also been trained in project proposal writing to assist them to secure other funding and establish other partnerships and collaborations. e. Sustainability of results and services It was noted that the communities had been mobilized to be innovative and the sub counties are taking lead in sustaining the project activities within available resources. In the meetings with sub county leaders in Wol, Paimol and Kalongo, leaders reported that they had noted the impact the project had made and were looking for ways to incorporate some of the project activities in their programming within the sub county plans. In addition, during a number of meetings and particularly at the Kalongo TC plenary session, it was emphasized that sustainability needed to start with the targeted community members themselves, and taking on community development as their personal responsibility. An elderly OVC guardian said. “Let us revive our earlier culture of hard work and self reliance” f. Project Exit coordination Handover sessions with respective local government offices: A deliberate process of activities handover was undertaken a full year ahead of the closure of the programme whereby dialogue meetings were held with project stakeholders viz District, sub counties and communities. Stakeholder close-out meetings were held in each Sub-county of operation. Participants included local government leaders and representation of beneficiaries and community volunteers from across the project interventions. The meetings were held in June￾July 2011 prior to project close-scale down. The dialogue meetings generated options from the stakeholders on how activities would be maintained beyond project lifespan. This was noted to have been a strategic approach that fostered stakeholder involvement in identifying strategies for post project implementation support and ownership. Focus on sustainability is demonstrated by the fact that some sub counties have now taken up sponsorship programs for OVC; youth groups are taking on IGA activities (e.g. Youth have taken on farming and grown groundnuts beans and maize for sell- Paimol and Wol Peer groups); community support to PLHIV (communal digging to provide food for PLHIV). The following efforts towards sustainability were also noted.  On-going advocacy by CBOs for volunteers integration into the health system  Names of volunteers are have been taken to the district for consideration for any future project recruitments  District hired most trained volunteers for the ongoing indoor spraying program  Some volunteers have been integrated into the District NAADS program because of the trainings and capacity they already have GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 39 Interviews with the key informants showed that there was a sustainability plan beyond project closure for example the sub-county officials in Wol provided the following plan for continuity.  Peer educators will continue doing voluntary work and a committee is already selected in preparation of the project end  HCT mobilization is ongoing and referral is done to Wol H/C III or Kalongo Hospital for continued care or HIV testing  Sustainable livelihood was incorporated in the sub county program (Paimol sub county)  Sustainability planning for OVC services: During the key informant interviews, the following recommendations were raised towards sustaining OVC programs. Parents are encouraged to take more responsibility for their children, love them, and train them to work hard and try to endeavor to find school fees e.g. through brick making. Further suggestions towards sustainability included the need to ensure that livelihoods inputs e.g. chicken which were supplied to OVC assist the nutrition, generate money for scholastic materials for OVC; Innovativeness of parents and guardians and parents should be stepped up. Care givers are being empowered to take over some OVC support activities and use available resources. Head teachers should consider use of PTA funds to continue supporting OVCs and lastly linking OVC beneficiaries with the sub-county local governments for further support. Despite all the approaches to sustainability as mentioned above, it was not clear to what extent the project activities could be sustained beyond the project life. Challenges like limited funding of the District community development department, which oversees community based activities, are a barrier to sustainability of some activities. The local governments would need to identify additional resources to support these activities. 3.5.2 Project address of community needs The project’s target community has been emerging from a conflict and emergency state. HIV/AIDS prevalence had been noted to be high in the region and there was need to address risky behaviours through effective IEC and BCC approaches. Key Informants noted that provision of HCT services closer to communities through door to door HCT campaigns and outreaches greatly increased community access to HCT services and knowledge of their HIV status. Referrals were provided for care for those that were found to be positive. Beneficiaries also noted that the project activities had been responsive to their needs. As regards Peer Education activities, provision of youth centres and board games, which provided the youth with their own space related to needs of youth (e.g. recognition of youth as a unique target group; mobilization of youth for youth friendly services; youth focused IEC; use of sports alongside HIV testing amongst others) in the communities. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 40 3.5.3 Post conflict programming The design was post conflict sensitive in ensuring recruitment of the volunteers from the communities they were returning. As the communities were resettled back to the villages from the camps, the project approach had to consider scale up for example, the model for HCT deliver through outreaches and door to door campaigns was instituted to increase coverage as people resettled back into their villages. 3.6 MISSED OPPORTUNITIES There are a few missed opportunities that were noted as relates to the project design.  It was noted, that some cadres of volunteers could not be retained beyond the project life. The selection of HCT volunteers in particular was also noted to have had gaps. Volunteers selected were not residents of the communities in which they were volunteering, during the life of the project, in some cases. It was feared that after the project closure, this capacity was not likely to remain in the communities. o While this volunteer model was employed strategically to maintain confidentiality of individuals receiving home to home testing, the long-term retention of these volunteers in their intervention areas may have been a missed opportunity for sustainability.  It was also noted that local government leaders at the sub-county level who participated in the evaluation (Wol, Paimol and Agago Town Council) thought that there was not enough consultation and involvement of the sub county local government leaders at project inception and during implementation. Reflection of the project activities within the sub county development plans therefore was noted to be lacking. Efforts were made however by GOAL and project partners to have the project aligned to District plans, which would likely form the basis by which Sub-Counties develop their plans. It should also be noted that some of the sub county plans were not in place at the time of project design, and that not all the Sub-County leaders who participated in the evaluation were in their current roles at the time of project design and the earlier years of implementation. There was more involvement with sub counties towards the end of the project. The perceived gap by local governments was also noted to be a result of the project design whereby the CBOs supported and project volunteers interfaced more directly with sub counties than GOAL.  Inadequate involvement and linkage with the VHT at community level. It was explained that the project activities required more volunteers than the traditional four VHT members who are normally busy as they are tapped by all partners so the approach aimed at recruiting more community volunteers. It is to be noted that at community level, the VHT are a structure that is linked to the national health system and their involvement ensures alignment of project activities at community level, within national health system.  While GOAL disseminated project closure communication to all health unit in-charges beneficiaries and local governments, this information did not necessarily trickle down and reach all health centre staff engaged in the project activities (namely HCT). GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 41 This was further reviewed and found to be a reflection of possible health systems management gaps whereby the information sharing amongst all staff at health facilities was not well disseminated.  Some key capacity building activities were conducted at the end of the project e.g. proposal writing which was conducted August 2011. This activity could have been planned earlier for smooth transition between GOAL exit and development of other partnerships using the skill.  Limited provision of capacity building for Health Center staff was a missed opportunity  Increased mobilization for HCT led to increase in numbers of beneficiaries requiring ARVs within context of drug stock outs or inability of the Health Centers to treat all PLHIV who need treatment, leading to delayed initiation of treatment of identified beneficiaries in some instances. 3.7 External Factors that affected project results Local Government structure changes: The project was initially implemented in the original Pader district. The district was split halfway through the project implementation into Pader and Agago districts. After the split, the project area of operation fell under Agago district. Engagement with the district officials then had to change to cater for this. Furthermore, there have been new political leaders elected in the newly formed Agago within the life of the project which necessitated new engagement processes with the local government structures. In addition the recent political campaigns interrupted some activities as beneficiaries were attending to their political leaders. Infrastructure Network: In some area of the district operation, the roads are impassable especially during the rainy seasons. This limits access of services to affected beneficiaries. The Local government with ZOA is working on the roads and the situation is gradually improving. Stakeholder transfers and projects closure: Project staff and other stakeholder transfers happened in the course of the project. This in some ways affected the project implementation due to the transitions that happened in adjusting to these changes. Projects by nature have a defined timeline and when there have been synergies built between projects for provision of comprehensive services, when one closes, it affects the impact of services of other Yellow Fever outbreak: During the final year of the project, there was an outbreak of Yellow fever and this greatly impacted on project field activities, specifically with HCT outreaches as mobilization of communities was not possible for fear of disease transmission. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 42 3.8 Prospective need for Project replication and strategies for maximizing effectiveness This section outlines the strategies that were implemented during the project plus additional recommendations of those activities that would further maximize effectiveness of current approach; that would be critical to replicate for successful roll out and implementation of a similar project in case of an extension or replication in similar or other geographical areas within Uganda. The recommendations are summarized under the three project objectives and others that are beyond the project objectives are outlined under general recommendations. 3.8.1 SO1 Improvements in the knowledge, attitude, behavior and practice levels as relates to HIV for the target population Innovative use of IEC/BCC is effective and the project needs to replicate these and further scale them up in subsequent programming. The stepping stones methodology, targeted sessions, music dance and drama, and IEC need to be further utilized for behavioral change. There is also need to document the impact of these IEC/BCC methodologies and disseminate them further at national level to address the challenge already noted in national programming of lack of documentation of what has worked or not worked as regards IEC/BCC in Uganda (Health sector HIV/AIDS Strategic plan 2010/11)15 There is need for a focused dissemination plan of reports generated to date e.g. Stepping stones evaluation. HCT services were effective in reaching wider coverage especially through the outreaches and door to door strategies. For future programming in HCT services scale up, there is need to further build on such innovations. Documentation of the GOAL experience with HCT scale up and dissemination is also recommended. HIV prevention for special groups like discordant couples and other vulnerable population sub groups and Most at risk populations (MARPs) needs further emphasis. In line with the national prevention guidelines, identification of the most at risk populations in the project area would lead to further impact in HIV prevention programming. This needs to be explored more in future. The Youth centers are an innovative approach to targeting and reaching youth which should be further scaled up in future programming Overall for HIV prevention, given the approach of combination prevention, for future programming, it is important to identify and establish partnerships that complement project HIV prevention services so that beneficiaries are able to access a package of HIV prevention services beyond project input. 15 Health sector HIV/AIDS strategic plan 2010/11 -2014/15, June 2011, page 14 GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 43 3.8.2 SO 2 Improving care and socio-economic support services to PLHIV and affected group in the target areas Use of community volunteers in first level care and treatment services and referral of PLHIV at community level needs to be continued. The VHT cadre too needs to be targeted as a strategic community level resource within the national health system for continuity of interventions beyond project life. In addition, training and capacity building of the health facility staff for quality care and treatment services should be continued as part of GOAL’s contribution to a health systems strengthening approach. As demonstrated under this project, the practice of partnerships and linkages for comprehensive HIV care services mapping should continue. Gaps in project support should be established for effective referral especially for ART as a key service for PLHIV. Integration of sexual reproductive health services is a key approach being promoted by MoH in HIV programming. Future programming needs to take SRH services integration into consideration either as a direct project intervention or as part of partnerships and collaborations. VSLA needs to be scaled up and promoted as a model for PLHIV economic empowerment. For OVC programming, the family approach in support of OVC should be continued as opposed to targeting of only one OVC in a family, given that in most cases all other children in the household are likely to be vulnerable too. Furthermore, the project needs to explore secondary education support as opposed to primary education support as suggested during OVC care committees’ focus group interviews Given the overwhelming demand for OVC support activities, there is need to foster partnerships with other organizations undertaking OVC programming to maximize effectiveness of interventions and for sustainability, as displayed by linking with World Vision in this project. Future OVC programming should ensure partnerships building for greater coverage and impact. 3.8.3 SO 3 Strengthening the organizational capacities of local CBOs and PLWHA Associations, including mainstreaming of HIV, gender and child protection The approach of implementation through CBOs needs to be continued for sustainability and further capacity building. The GOAL OCA process is a model that can be further shared with other partners involved in CBO capacity building. Building the capacity of supported CBOs for accessing other sources of funding beyond GOAL needs to be continued as part of the GOAL CBOs’ exit strategy. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 44 3.8.4 General recommendations: Sustainability approach in design Approaches to foster sustainability in the project design as displayed by the project should be scaled up in future programming with involvement of local governments at all levels and existing community structures. This also calls for engagement with a coordinated district response in HIV prevention, care and mitigation support for PLHIV and OVC, where this exists. M&E systems Beyond the project, Child status index as an innovation in OVC M&E and needs to be documented and shared. In addition, there is need to harmonize overall OVC M&E with the existing district/ national OVC M&E system for ease of comparison across district / national OVC programming efforts. As regards to the annual KABP surveys, comparable respondent categories need to be instituted for comparisons across different surveys, internal and external to GOAL. Gender programming Gender consideration in programming is noted to be important for successful outcomes. Future programming should therefore continue to undertake gender mainstreaming as an integral part of overall programming for effectiveness. Involvement of the local government partners and their conceptualization of the project design especially clarity about GOAL role and the CBO partner roles needs to be well communicated from project conception and on a consistent basis throughout project life, to manage expectations based on other models. Implementation with existing community, institutional and local government development plans  The project design focused on implementation through existing community structures. This is key in fostering sustainability right from project inception and roll out. However, the missed opportunities noted like inadequate involvement and linkage with the VHT at community level; inadequate coordination with sub county local governments at the planning stage needs to be addressed for maximizing impact for future projects.  Health Unit and Village Health committees are existing community support structures for improved delivery of services at health facilities and within communities. For future projects, it would be strategic to ensure participation and capacity building of these structures for maximizing effectiveness and sustainability of project services as a way of strengthening Health facility capacity within a health systems strengthening approach. There is an opportunity to involve and train VHT in the community intervention e.g. AMREF experience working in partnership with health facilities. TBAs too are an existing link that can be involved to enhance mobilisation and referral of pregnant mothers for PMTCT GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 45  Integration of project activities into local government development plans at district and sub county levels for coordination and ownership, to maximize effectiveness is a strategy that needs to be replicated and strengthened for future project activities replication. In addition, it is necessary to integrate project activities e.g. prevention activities into ongoing institutional activities like schools music, drama, football and netball competitions for the sustainability of the prevention activities. It is important to have project focus aligned to local government approach to ensure smooth integration at project close e.g. Goal targeted individual beneficiaries (e.g. PLHIV, OVC etc) but many government programs target groups (e.g. women’s groups, youth groups, PLHIV groups etc) posing a challenge to integration. Coordination of CBO/NGO activities: It was noted that there was a difference in the facilitation/remuneration of community level volunteers across CBOs operating in the project area. This created competition amongst community members as regard to partners to work with. In future, for maximizing projects impact, it is advisable that a coordinated system be developed under the District/sub county local governments to standardize/ harmonize operations of CBOs/NGOs. Youth friendly services: Provision for youth friendly services increased access for services to Youth and this requires replication. Youth had opportunity to have satellite HIV testing sites at their youth centers to address their reluctance to seek services at the Health centers. Furthermore youth communicated need for the following in order to maximize effectiveness of youth centers: o continued use of puppetry and drama activities as attractive to youth o Need for visual materials like TV and Exchange visit to other areas would maximize the effectiveness of peer education o Youth involvement in the construction of the youth centre would enhance ownership o Continued peer mobilization and sensitization o Continued involvement and commitment of Local Government officials in youth center management commitment to ensure sustainability o Formation of peer educators group to continue lobby for support from other partners and Government o Formation of youth clubs in schools to continue sensitization of other youths o Establish VSLA and farming as IGA to support the center and continue with peer education o There is interest in the establishment of a library with books and computers at the youth centers GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 46 Gender responsiveness in programming for Youth center activities e.g. current approach supports male youth more than the females. There is need to incorporate activities like the pottery, cookery and talks for girl youth at the youth centers Participatory skills: It was noted that for success of the project activities, it is key that the staff and volunteers are equipped with community participatory communication skills. Approaches that foster community participation would also be important to replicate as they foster community engagement and ownership of project activities. The participatory approach that was used in identification of OVCs is one example that was highlighted, of good community participatory practices used, during the project. Taking services closer to the people: The innovation of establishing service delivery outreaches and door to door services led to increased uptake and coverage of services. This approach can be replicated through supporting existing service providers at static facilities to conduct outreaches and needs to be adopted in future similar project designs. Under IEC/BCC -Bill boards should be erected at the parish center levels to continue talking to the youth and the community. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 47 4.0 LESSONS LEARNT    During project implementation a number of lessons were learnt that can facilitate project replication and can be shared with other partners implementing similar activities for greater impact. These lessons are outlined below under each project objective: SO1 Improvements in the knowledge, attitude, behavior and practice levels as relates to HIV for the target population The stepping stones methodology is an innovative approach that has been noted to have had impact in behavioural change for HIV risky practices within the project area. Furthermore, the methodology has been accepted, owned and implemented by the community once capacity was built. Grouping community members by age group and gender during SS dialogue sessions was noted to enhance free communication and engagement that facilitates behaviour change. This approach therefore is appropriate for replication given its potential for sustainability beyond project life In addition to SS, door to door sensitisation by CHFPs and using drama strategy was very effective in community mobilisation, for HIV prevention services uptake. Locally adapted IEC material that reflects the target population context is effective in engaging communities for HIV prevention messaging. Use of pictures of youth for local IEC material e.g., billboards, IEC wall posters was attractive to youth and led to their engagement with message through discussions and its uptake Other aspects of effective IEC included T-shirts with messages for community mobilization in HIV/AIDS prevention. The door to door HCT campaigns and outreaches increased access of HCT services for the target communities. This approach was helpful in rapid scale up of HCT services and is worth considering during scale up of HIV prevention activities. HCT volunteers and multiple community HCT outreach sites are an effective innovation in rapid scale up of HCT services The project supported health facilities through training of HCT volunteers that were attached to health facilities and this assisted in reducing workload of health workers. More people accessed HCT services through door to door services and outreaches due to availability of services closer to them. It also ensured quality HCT services delivery. Another resource that should be tapped in future is the VHTs in community level health programming, to support existing health structures. Services integration was a missed opportunity e.g. Sexual reproductive health focus was not strong and this would have been integrated into the HIV prevention activities. Non availability of female condoms was also highlighted during interviews. There was limited integration of Sexual reproductive services like Family planning and PMTCT in the project activities. Family Planning messages and PMTCT were not a primary project focus as another NGO actor, CRS, was engaged in PMTCT programming in the target area at the time of project design. Effort was made however, to include PMTCT messages in IEC sessions and campaigns, radio shows, and stepping stones sessions. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 48 SO 2 Improving care and socio-economic support services to PLHIV and affected group in the target areas Systems strengthening approach in improving care and social economic services for PLHIV is important. Trained community volunteers supported beneficiaries at community level and also provided for a referral system at community level and this is important in future programming in improving access to services for PLHIV Problem sharing among PLHIV was good because it helped the beneficiaries to cope with their challenges through peer support As regards to OVC programming, scholastic materials (books, pens and bags) are the most appreciated inputs by the OVC. The girls particularly appreciated provision of sanitary towels and this needs to be a replicated in subsequent programming as it has been noted that monthly periods contribute to the reasons for the girl child absenteeism and school dropout. Parents and guardians also highlighted that they would prefer to have support for secondary education than primary education. SO 3 Strengthening the organizational capacities of local CBOs and PLWHA Associations, including mainstreaming of HIV, gender and child protection Sub granting to local CBOs coupled with elaborate plans for their capacity building fosters sustainability of activities and contributes to development of local community led initiatives that can continue to implement activities beyond project life. The example of work undertaken in capacity building of Wagwoke Wunu and Prayer group as outlined above is a best practice model that needs to be replicated during scale up. Alongside this, strengthening capacity of the grantee organization needs to be planned for as strong technical support backstopping resource for the sub grantees. Other General Lessons: Sustainability: Cooperation and working in partnership with the local governments is critical for sustainability. The existing local government structures offer an opportunity for continuity of some key project activities beyond project life. Examples like integration of project activities within district and sub county plans as were undertaken are important to replicate. Reflection of these activities within local government plans ensures that they become owned and further developed by the local authority as local government and not project priorities. Commitment to community voluntary work can be fostered as part of project activities and can lead to community ownership of activities. The following comment was made by the participants GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 49 of a stepping stones focus group discussion in Wol “We will continue teaching our people even after the project winds up” Coordination: Partners working with the same communities need to coordinate their activities - Lack of coordination among partners affects the implementation of project activities e.g. when World food program carries out commodities distribution to community at the same time as sensitization or training activities the community is torn between the two Role of faith groups: The belief in faith that forms the foundation of Prayer Group impacted on the lives of PLHIV and later their intervention in the project. Tapping into faith based groups is one way of using existing organized community structures and in future replication of activities this can be proactively followed up as FBOs are normally a ready and organized self motivated structure that can be supported/ provided with additional capacity to replicate project activities. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 50 5.0 CHALLENGES The following challenges were summarized from the interviews and focus group discussions with respondents.  In general, it was noted that a dependency syndrome had been developed by the community following the protracted civil conflict and this was affecting community productivity and innovation  The project design provided for direct interaction between the supported CBOs and lower local governments, this however was not well understood by local government stakeholders and created the perception that GOAL activities were not well coordinated with Local governments  Overwhelming community needs prevented the universal coverage of project services due to limited resources e.g. not all needy children in a home were enrolled in the project. Many needy children were considered but resources limited the number that could be selected. Fees for secondary education and access to scholastic materials beyond the life of the project are another challenge highlighted by parents and guardians of OVC.  Sustainability of outreach and door to door testing may be a challenge if there is no support for transporting the health workers/volunteers to carry out the outreach service GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 51 6.0 CONCLUSION The project achieved its objectives and planned targets within the project lifespan as outlined in details above. Under objective one, the stepping stones methodology outcomes show that the intervention was able to address the challenge of a lag in behavioral change had been noted at the project planning stage. Furthermore the IEC through print and mass media and education sessions for general community facilitated by CHFPs also achieved the desired outcomes of improving knowledge of communities on HIV prevention as noted in the KABP survey. The youth focused interventions are also noted to have had an impact on youth knowledge of HIV prevention and control of risky behavior from the documents reviews and also reports during the evaluation exercise. Innovations adopted during the planned HCT scale up, enabled achievement of set targets and have provided lessons for future programming. Under objective two, it is noted that as per plan, there was improved coverage and access to HBC care services for PLHIV at community level. OVC too were provided with comprehensive services and planned coverage targets were attained. Livelihoods support was extended to both PLHIV and OVC as planned. The planned capacity building of two CBOs was also successfully achieved, through capacity building trainings, technical support and mentoring. Organizational capacity building was measured and tracked through the GOAL OCA process. HIV/AIDS mainstreaming was also implemented with the two CBOs. Throughout the project implementation and towards project activities scale down, a sustainability focus was achieved as planned for in the project plan16. The project design was also noted to have been appropriate with some missed opportunities and areas whereby project activities could have been further maximized for effectiveness explained above. The recommended areas of emphasis in case of an extension or replication in similar or other geographical areas within Uganda are outlined under the recommendations section for the Project teams’ attention. 16 Detailed implementation plan . Scaling up Innovative HIV / AIDS Interventions through Strong Partnerships amongst Conflict – Affected Populations of Pader District, Northern Uganda, 2008 – 2010. Detailed implementation plan GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 52 References: 1. Detailed implementation plan. Scaling up Innovative HIV / AIDS Interventions through Strong Partnerships amongst Conflict – Affected Populations of Pader District, Northern Uganda, 2008 – 2010. Detailed implementation plan 2. GOAL Child Status Index data 2010/11 3. GOAL KABP Report 2011 for Agago and Pader Districts 4. GOAL Monitoring and Evaluation plan, Attachment C 5. GOAL Partners OCA Report Sept 2011 6. GOAL Scaling up Innovative HIV/AIDS Interventions through Strong Partnerships amongst Conflict-Affected Populations of Pader (Agago) District, Northern Uganda, 2008 -2011. Annual Report 2010/11 7. GOAL Uganda PEPFAR concept note 8. GOAL Uganda Annual PEPFAR report Oct 2008- Sept 2009 9. GOAL Uganda Annual PEPFAR report 2009/10 10. Health sector HIV/AIDS strategic plan 2010/11 -2014/15, June 2011, page 14 11. Home based care and support group programming. Outcome evaluation report, November 2010. GOAL Uganda. 12. Report on Evaluation of the Stepping Stones HIV Behavioral Change Methodology in GOAL Uganda, April 2011 GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 53 Appendix Appendix 1 Terms of Reference Final Evaluation of GOAL’s HIV PEPFAR Funded Project in Pader & Agago Districts Release date: 14 September. Deadline for submission: 5pm, 23 September. Name of the project Scaling up of innovative HIV interventions through strong partnerships amongst conflict￾affected populations of Pader District, Northern Uganda, 2008-2011 Background GOAL is addressing multi-dimensional aspects of the HIV pandemic in five districts of Uganda, namely Busgirl, Namayingo, Abim, Pader and Agago Districts. GOAL works with partner organizations to deliver comprehensive and coordinated responses to HIV specifically working on prevention of transmission of HIV and mitigation of its effect on infected and affected individuals, families and communities. GOAL was awarded a PEPFAR New Partners Initiative (NPI) grant to work on reducing the transmission of HIV and to mitigating the impact of HIV amongst the population in Pader and Agago Districts (formed out of the previous Pader District). The initial work was with displaced communities; however, with the restoration of peace GOAL adjusted its strategies and is reaching out to the general community as they resettle. GOAL works in two counties of Pader District: Agago and Arum; although only Agago county is covered under the PEPFAR project. Out of the nine sub-counties in Agago County, the main target areas of operation for the project are the northern most sub-counties of Lapono, Paimol, Parabongo and Wol and Kalongo Town Council. Project activities have also been extended to Lukole Sub-County to respond to identified gaps in the HIV services in this location. The project is implemented in partnership with the two community-based organizations (CBOs) in the operational area: Wagwoke Wunu and the Prayer HIV/AIDS Group (an association of people living with HIV and currently with a membership of more than 250 people). These CBOs are responsible for activities related to community organization and the mobilization of target groups for stigma reduction and care and support. Objectives of the project and interventions utilized17 The principle objective of this project is to contribute towards the reduction of transmission of HIV and mitigation of the impacts of HIV amongst the displaced, resettled and returned populations in Pader (now Pader and Agago) District. Specific objectives (and strategies and interventions employed to achieve these objectives, presented in the sub-bullets) of this programme aim to18: 17 Please see the attached appendix for further information on the project and it’s achievements thus far. 18 MEL Co will work through the log frame and the strategy of the project with the consultant to familiarize them on the questions which need to be asked and answered through this evaluation. GOAL is looking to the consultant to inform and enrich these questions with experience in PEPFAR programming and the HIV thematic field in general. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 54 a. Bring about positive improvements in the knowledge, attitude, behavior and practice levels of the target population through: a. Improved access to targeted and innovative IEC/BCC services amongst the general public and youth through the Stepping Stones methodology19 (a behavior change communication prevention methodology20) and targeted community awareness meetings. The target beneficiary groups are: youth and adult groups for the Stepping Stones meetings, general public in the community awareness meetings and the general public as well for the HCT component. b. IEC through print and mass media. Target beneficiary group is the general public. b. Ensure improved care and socio-economic support services to PLWHA and affected group in the target areas through: a. Increased provision and coverage of home based care services. Target beneficiary group: people living with HIV. b. Increased coverage of livelihood and social support service provision (including pilot project with VSLA methodology). Target beneficiary group: OVC households c. Strengthen the organizational capacities of local CBOs and PLWHA Associations, including mainstreaming of HIV, gender and child protection. Purpose of this evaluation The project is coming to its conclusion and therefore needs to be evaluated as stated in the project document. The evaluation will provide quantitative and qualitative assessments of the approach that GOAL utilized and determine whether GOAL achieved what it set out to do in this project. GOAL is also interested in continuing support to HIV service delivery so the evaluation will be used as a lesson learning process to identify strong points and reasons for success and identify gaps that need to be addressed in future programming (and recommendations and possible existing resources that can be tapped into to respond to these gaps). The findings from this evaluation will also help GOAL Uganda decide whether approaches used can be scaled-up or replicated to other regions where there is a programming need. Objectives of the evaluation The main objective of the evaluation is to assess the relevance, effectiveness, efficiency, performance and the sustainability of the project interventions, draw lessons and make recommendations for similar future projects. Specific objectives are to: 19 HIV prevention methodology; knowledge of this methodology is an added bonus. 20 Change of risky sexual behaviours and increase in testing as result of IEC: therefore please consider both aspects. Dimensions of risky sexual behaviour which GOAL has assessed previously can be obtained from the Stepping Stones outcome evaluation; you may want to suggest other aspects/dimensions for evaluation however GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 55 5. Assess the progress made in achieving the project results (intended, unintended, positive and negative outcomes in individuals and communities) in relation to the: a. project design and implementation strategies b. technical competency to undertake the planned activities, c. technical support and capacity building for partner organizations and other community structures d. relationship with project stakeholders and other service providers e. external factors f. The consultant will be asked to analyze baseline (2009) and follow up (July 2011) child status index information (already captured) to comment on any outcome changes in the OVC we support. g. An additional element should look at whether inclusion or exclusion in the programme or whether programme activities do not further stigmatize and/or discriminate beneficiaries and non-beneficiaries 6. Assess the project design and appropriateness of the approach used including its sustainability in the context of service delivery and required need in communities where we work (please consider all of the above bullets as well in this regard) 7. Determine the prospective need for the replication of the project and identify ways to maximize effectiveness in case of an extension or replication in similar or other geographical areas within Uganda 8. Identify and document lessons learnt and best practices of the project. Bidders for this consultancy will have to outline key research questions based on information provided about the project and the assignment in this document, including proposed methodology and analysis plan. The consultant will be provided with GOAL annual HBC & SG outcome reports from 2011, 2010 & 2009 and then Stepping Stones outcome evaluation data from 2011 and 2010. All monitoring and project data and documentation will be provided to the consultants. Scope of work and estimated timeline The total number of days of this consultancy is estimated at 24 working days and the work is required to start at the beginning of October and the final report submitted and signed off by GOAL no later than the end of October. This consultancy is divided into the following components (estimated duration indicated for each):  Preparation of the evaluation process. This will include a briefing meeting with the MEL Coordinator, reading of relevant documentation to get an overview of expectations and the project, respectively. This phase will also include the finalization of the inception report which will outline the details regarding the methodology, the tools and techniques to be utilized to achieve the objectives of the consultancy and a timeframe to complete the assignment successfully. Duration: 4/5 days  Conducting field activities as per the agreed inception report. The consultant/s will be requested to present preliminary findings to the HIV project manager and other key staff in Kalongo at the conclusion of the field visit. Duration: 7/8 days, excluding 2 days of travel to Kalongo.  Analysis and report writing. This will include a draft report to be submitted to the GOAL team for comments within 14 days upon returning from the field. GOAL will GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 56 endeavor to provide comments within 3 days of receipt of the draft report. The consultant/s will be asked to present the findings described in the draft report in Kampala to key GOAL staff, on the day of submission of the draft report. A final report will be produced thereafter. Duration: 10 days. Expected outputs The consultant/s is expected to produce:  Inception Report  Draft Report & presentation (and any related, generated documentation)  Final Report & amended presentation which will accompany the final report and be used for onward dissemination of evaluation findings  Success story of a beneficiary, per intervention (HBC, SS and OVC)  Any other process generated documentation Management of the evaluation GOAL’s MEL Coordinator will take the overall responsibility of managing the evaluation from GOAL’s perspective; the consultant/s will work closely with designated Kalongo staff to ensure successful fieldwork. The evaluation consultant or team shall ensure and maintain continuous involvement of major stakeholders at the district level (including USAID Northern Uganda Resident Representative) throughout the entire evaluation process and ensure the required quality of work is produced and the assignment successfully completed. Qualifications of the consultant/s The assignment will be contracted to a consultant or team of consultants with significant experience in the area of HIV prevention, care and support (experience with USAID/PEPFAR is desirable) as well as with senior level skill and experience in conducting evaluations. The consultant/s should have:  Post-graduate level degree in Public Health, Social sciences or other relevant field of study  Experience in evaluating PEPFAR funded programs or experience and expertise in conducting large scale evaluations as the lead consultant  Experience in implementing HIV prevention, care and support or PEPFAR funded programs; and understanding of USAID/PEPFAR reporting requirements  Competence in data management associated with evaluations  Excellent analytical and report writing skills; strong communication skills  Ability to work independently and achieve expected results.  Fluency in English and ability to work in multi-cultural setting. Knowledge of Luo will be an added advantage. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 57 Submission of Proposals Interested bidders are required to submit a technical and financial proposal for this consultancy (as separate documents). The proposal should include only the following information:  Technical proposal outlining the proposed technical response to the objectives and other elements, as they have been described above. This needs to cover the type of research questions you would ask in relation to the objectives, the proposed methodology and respondents and analysis plan. Please provide innovative research methods for each of respondent group and confirm that the proposed evaluation components are adequate to cover the assignment objectives; if not suggest a modified timeline with justification please (no minimum page restriction but not to exceed 10 pages).  CV of the proposed consultant/s (please specify the language proficiency of each suggested person)  Brief description of evaluations completed, and your role within each one, which speak about your competence and capacity to complete this assignment proficiently (no more than 1 page in total).  Please provide a writing sample, preferably of recent evaluations complete. A selection of report is adequate; we do not require any sensitive information so please submit questions that you deem will demonstrate your writing and analytic skills only (unrestricted page limit).  Proposed cost for 24 days of work and other related costs in a separate document to the above mentioned items. GOAL will provide transport and a standard per-diem of 20,000 per day when travelling to or in the field locations. Please include the per-diem calculation in your proposal (unrestricted pages limit). NOTE: All costs associated with proposal preparation, submission and/or negotiation cannot be reimbursed as a direct cost of the assignment. All technical questions regarding this TOR should be directed to: astiglic@ug.goal.ie and CC: lmacchione@ug.goal.ie, jnmaher@ug.goal.ie and gmasendi@ug.goal.ie . The proposals are due 5pm on 23 September. No late submission will be accepted. Proposals are to be submitted electronically to astiglic@ug.goal.ie with CC to lmacchione@ug.goal.ie, jnmaher@ug.goal.ie and gmasendi@ug.goal.ie. The preferred consultant will be notified within two weeks of the deadline. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 58 Appendix 2 Schedule for Field work FINAL EVALUATION OF GOAL’S HIV PEPFAR FUNDED PROJECT IN PADER & AGAGO DISTRICTS EVALUATION SCHEDULE MONDAY, OCTOBER 31, 2011 – TRAVEL DAY (KAMPALA –KALONGO) TUESDAY, NOVEMBER 1, 2011 TIME SUB￾COUNTY & PARISH ACTIVITY RESPONDANTS EVALUATION TEAM LOCATION 9:30AM – 12:30PM KALONGO T/C, TOWN WARD FOCUS GROUP DISCUSSION GOAL STAFF:  Laura Macchione – Project Manager  Drasi Joseph – OVC Technical Officer  Charles Lwanga – Finance & Administration Officer  Dominic Ocen – Resource Room Attendant  Kevin Heraniah – Pader/Agago Programme Manager ALL Laura GOAL OFFICE TUKAL 12:30PM – 1:30PM LUNCH 1:30PM – 3:30PM KALONGO T/C, TOWN WARD FOCUS GROUP DISCUSSION WAGWOKE WUNU STAFF ALL Paulino (Monday 31st Oct.) WAGWOKE WUNU OFFICE 3:30PM – 5:30PM KALONGO T/C, TOWN WARD FOCUS GROUP DISCUSSION PRAYER GROUP STAFF ALL Bosco (Monday 31st Oct.) PRAYER GROUP OFFICE GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 59 WEDNESDAY, NOVEMBER 2, 2011 – EVALUATION TEAM 1 TIME SUB￾COUNTY & PARISH ACTIVITY RESPONDANTS EVALUATION TEAM LOCATION 8:30AM￾9:00AM TRAVEL FROM KALONGO T/C - WOL 9:00AM – 10:30AM WOL KEY INFORMANT INTERVIEW 1 HCT HEALTH CENTRE STAFF Drasi WOL HCIII 10:30AM – 12:00PM WOL KEY INFORMAT INTERVIEW 1 HCT VOLUNTEER Drasi WOL HCIII 12:00PM￾12:30PM TRAVEL FROM WOL-KALONGO T/C 12:00PM – 1:00PM LUNCH (MAMA MARIA) 1:00PM￾1:30PM KALONGO T/C, TOWN WARD KEY INFORMANT INTERVIEW 1 HCT HEALTH CENTRE STAFF Drasi DR. AMBROSOLI MEMORIAL HOSPITAL 1:30PM￾3:30PM KALONGO T/C, TOWN WARD KEY INFORMANT INTERVIEW 1 HCT VOLUNTEER Drasi DR. AMBROSOLI MEMORIAL HOSPITAL 3:30PM￾5:30PM KALONGO T/C, TOWN WARD FOCUS GROUP DISCUSSION KALONGO T/C GOVERNMENT  LCIII  SUB-COUNTY CDO  HEALTH ASSISTANT(S)  YOUTH COUNCILLOR Bosco (Monday) WEDNESDAY, NOVEMBER 2, 2011 - EVALUATION TEAM 2 TIME SUB￾COUNTY & PARISH ACTIVITY RESPONDANTS EVALUATION TEAM LOCATION GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 60 8:30AM￾9:00AM TRAVEL FROM KALONGO T/C - PAIMOL 9:00AM￾10:30AM PAIMOL KEY INFORMANT INTERVIEW 1 HCT HEALTH CENTRE STAFF Drasi PAIMOL HCIII 10:30AM￾12:00PM PAIMOL KEY INFORMAT INTERVIEW 1 HCT VOLUNTEER Drasi PAIMOL HCIII 12:00PM￾12:30PM TRAVEL FROM PAIMOL – KALONGO T/C 12:30PM - 1:30PM LUNCH (MAMA MARIA) 1:30PM￾2:00PM TRAVEL FROM KALONGO T/C - PAIMOL 2:00PM￾4:00PM PAIMOL FOCUS GROUP DISCUSSION PAIMOL S/C GOVERNMENT  LCIII  SUB-COUNTY CDO  HEALTH ASSISTANT(S)  YOUTH COUNCILLOR Drasi 4:00PM – 4:30PM TRAVEL FROM PAIMOL – KALONGO T/C THURSDAY, NOVEMBER 3, 2011 – EVALUATION TEAM 1 TIME SUB￾COUNTY & PARISH ACTIVITY RESPONDANTS EVALUATION TEAM LOCATION 8:30AM￾9:00AM TRAVEL FROM KALONGO T/C - WOL 9:00AM – 10:30AM WOL FOCUS GROUP DISCUSSION PAIMO Wol L S/C GOVERNMENT  LCIII  SUB-COUNTY CDO  HEALTH ASSISTANT(S)  YOUTH COUNCILLOR Ojara (Monday Morning) 10:30AM WOL FOCUS GROUP STEPPING STONES: Ojara ( Monday GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 61 – 12:30PM DISCUSSION  SST FACILITATOR  2 OLDER MEN  2 OLDER WOMEN  2 YOUNGER MEN  2 YOUNGER WOMEN  1 HCT STAFF FROM NEAREST HEALTH CENTRE morning) 12:30PM￾1:00PM TRAVEL FROM WOL-KALONGO T/C 1:00PM￾2:00PM LUNCH (MAMA MARIA) 2:00PM￾4:00PM KALONGO T/C FOCUS GROUP DISCUSSION STEPPING STONES:  SST FACILITATOR  2 OLDER MEN  2 OLDER WOMEN  2 YOUNGER MEN  2 YOUNGER WOMEN  1 HCT STAFF FROM NEAREST HEALTH CENTRE Ojara (Monday morning) THURSDAY, NOVEMBER 3, 2011 - EVALUATION TEAM 2: TIME SUB￾COUNTY & PARISH ACTIVITY RESPONDANTS EVALUATION TEAM LOCATION 8:30AM￾9:00AM TRAVEL FROM KALONGO T/C - PAIMOL 9:00AM￾11:00AM PAIMOL FOCUS GROUP DISCUSSION STEPPING STONES:  SST FACILITATOR  2 OLDER MEN  2 OLDER WOMEN  2 YOUNGER MEN  2 YOUNGER WOMEN  1 HCT STAFF FROM NEAREST HEALTH CENTRE Ojara (Monday ) 11:00AM￾1:00PM PAIMOL FOCUS GROUP DISCUSSION PEER EDUCATION:  PEER EDUCATOR  10 PE TARGETED BENEFICIARIES  1 HCT STAFF FROM Ojara (Monday) GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 62 NEAREST HEALTH CENTRE 1:00PM￾1:30PM TRAVEL FROM PAIMOL – KALONGO T/C 1:30PM￾2:30PM LUNCH (MAMA MARIA) 2:30PM￾4:30PM KALONGO T/C FOCUS GROUP DISCUSSION PEER EDUCATION:  PEER EDUCATOR  10 PE TARGETED BENEFICIARIES 1 HCT STAFF FROM NEAREST HEALTH CENTRE Stella (Monday) FRIDAY, NOVEMBER 4, 2011 – EVALUATION TEAM 1 TIME SUB￾COUNTY & PARISH ACTIVITY RESPONDANTS EVALUATION TEAM LOCATION 8:30AM￾9:00AM TRAVEL FROM KALONGO T/C - WOL 9:00AM – 10:30AM WOL FOCUS GROUP DISCUSSION PEER EDUCATION:  PEER EDUCATOR  10 PE TARGETED BENEFICIARIES 1 HCT STAFF FROM NEAREST HEALTH CENTRE Stella (Wednesday 2nd Nov.) 10:30AM￾12:30PM WOL FOCUS GROUP DISCUSSION HBC:  4 CCAS  8 HBC CLIENTS Ricky (Monday 31st Oct.) 12:30PM￾1:00PM TRAVEL FROM WOL – KALONGO T/C 1:00PM￾2:00PM LUNCH 2:00PM￾4:00PM KALONGO T/C FOCUS GROUP DISCUSSION HBC:  4 CCAS  8 HBC CLIENTS Odong Okello (31st Oct) GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 63 FRIDAY, NOVEMBER 4, 2011 - EVALUATION TEAM 2: TIME SUB￾COUNTY & PARISH ACTIVITY RESPONDANTS EVALUATION TEAM LOCATION 8:30AM￾9:00AM TRAVEL FROM KALONGO T/C - PAIMOL 9:00AM￾11:00AM PAIMOL FOCUS GROUP DISCUSSION HBC:  4 CCAS 8 HBC CLIENTS Grace /innocent (Monday 21st Oct) 12:30PM￾1:00PM TRAVEL FROM PAIMOL TO KALONGO T/C 1:00PM￾2:00PM LUNCH 2:00PM￾3:30PM KALONGO T/C FOCUS GROUP DISCUSSION OVC SUPPORT STRUCTURE:  4 OVC CAREGIVERS  4 CHILD PROTECTION COMMITTEE MEMBERS  4 HEAD/OVC FOCAL TEACHERS Beatrice (Monday 31st Oct.) 3:30PM￾5:00PM KALONGO T/C FOCUS GROUP DISCUSSION OVC:  6 FEMALE OVC  6 MALE OVC Beatrice (Monday 31st Oct.) MONDAY, NOVEMBER 7, 2011 – EVALUATION TEAM 1 TIME SUB￾COUNTY & PARISH ACTIVITY RESPONDANTS EVALUATION TEAM LOCATION 8:30AM￾9:00AM TRAVEL FROM KALONGO T/C - WOL 9:00AM – 11:00AM WOL FOCUS GROUP DISCUSSION OVC SUPPORT STRUCTURE:  4 OVC CAREGIVE RS  4 CHILD PROTECTI Beatrice (Friday 4th Nov.) GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 64 ON COMMITT EE MEMBERS  4 HEAD/OV C FOCAL TEACHER S 11:00AM￾1:00PM WOL FOCUS GROUP DISCUSSION OVC:  6 FEMALE OVC  6 MALE OVC Beatrice (Friday 4th Nov.) 1:00PM￾1:30PM TRAVEL FROM WOL – KALONGO T/C 1:30PM￾2:15PM LUNCH 2:15PM￾3:15PM TRAVEL FROM KALONGO T/C – AGAGO DISTRICT HEADQUARTERS 3:15PM￾5:00PM FOCUS GROUP DISCUSSION/KEY INFORMANT INTERVIEWS DISTRICT GOVERNMENT:  RDC  CAO  DHO  DEO  DCDO Bosco (Wednesday 2nd Nov.) AGAGO DISTRICT HEADQUARTERS 5:00PM￾5:30PM TRAVEL FROM AGAGO DISTRCT HQ – KALONGO T/C MONDAY, NOVEMBER 7, 2011 - EVALUATION TEAM 2: TIME SUB￾COUNTY & PARISH ACTIVITY RESPONDANTS EVALUATION TEAM LOCATION 8:30AM￾9:00AM TRAVEL FROM KALONGO T/C - PAIMOL 9:00AM￾10:30AM PAIMOL FOCUS GROUP DISCUSSION OVC SUPPORT STRUCTURE:  4 OVC CAREGIVERS  4 CHILD PROTECTION COMMITTEE MEMBERS Beatrice (Thursday 3rd Nov.) GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 65  4 HEAD/OVC FOCAL TEACHERS 10:30AM￾12:00PM PAIMOL FOCUS GROUP DISCUSSION OVC:  6 FEMALE OVC  6 MALE OVC Beatrice (Thursday 3rd Nov.) 12:00PM￾1:00PM PAIMOL, PACABOL, OPIRO VILLAGE CASE STUDY AKELLO CHRISTINE 1:00PM￾1:30PM TRAVEL FROM PAIMOL TO KALONGO T/C 1:30PM￾2:30PM LUNCH 2:30PM￾2:45PM TRAVEL TO KALONGO T/C, NIMARO PARISH, AKADO VILLAGE 2:45PM – 3:45pm KALONGO T/C, NIMARO PARISH, AKADO VILLAGE CASE STUDY VINCENT OKOT (STEPPING STONE BENEFICIARY) 3:45PM￾4:00PM TRAVEL BACK TO KALONGO/TC, TOWN WARD TUESDAY, NOVEMBER 8, 2011 – EVALUATION TEAM 1 & 2 TIME SUB￾COUNTY & PARISH ACTIVITY RESPONDANTS EVALUATION TEAM LOCATION 8:30AM￾9:00AM TRAVEL FROM KALONGO T/C – PADER DISTRICT HEADQUARTERS 9:00AM – 11:00AM FOCUS GROUP DISCUSSION/KEY INFORMANT INTERVIEWS DISTRICT GOVERNMENT:  RDC  CAO  DHO  DEO  DCDO Paulino (Wednesday 2nd Nov.) PADER DISTRICT HEADQUARTERS 11:00AM￾12:00PM TRAVEL BACK TO KALONGO T/C GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 66 12:00PM￾1:00PM LUNCH 1:00PM￾1:30PM TRAVEL FROM KALONGO T/C TO WOL SUB-COUNTY, KALAGUM PARISH, TOMORA WEST VILLAGE 1:30PM￾2:30PM WOL, KALAGUM PARISH, TOROMA WEST VILLAGE CASE STUDY ANA ACAYO (STEPPING STONES BENEFICIARY) 2:30PM￾3:00PM TRAVEL FROM WOL SUB-COUNTY TO KALONGO T/C 3:00PM￾5:00PM KALONGO T/C FOCUS GROUP DISCUSSION GOAL PROJECT TEAM GOAL KALONGO OFFICE WEDNESDAY, NOVEMBER 9, 2011 – EVALUATION TEAM 1 & 2 TIME SUB￾COUNTY & PARISH ACTIVITY RESPONDANTS EVALUATION TEAM LOCATION 8:30AM￾9:00AM TRAVEL FROM KALONGO T/C – PAIMOL 9:00AM – 12:00PM PAIMOL FINAL GROUP FORUM (MAX 50)  GOAL STAFF (2)------------------ -------  PARTNER STAFF (10)----------- --------  PAIMOL S/C GOVERNMENT (4)------  STEPPING STONES (10)-------- --------- o 1 LEADER o 1 FACILITATOR o 2 YOUNG WOMEN o 2 YOUNG MEN o 2 OLDER WOMEN o 2 OLDER MEN  PEER EDUCATION (8)---------- --------- o 2 PEER EDUCATORS o 6 PE BENEFICIARIES  2 CHFPS----------------------------- ------  2 HCT HEALTH CENTRE STAFF----  OVC (14)---------------------------- Drasi Alice (Monday 7th Nov.) Ojara (Thursday 3rd Nov.) Ojara (Friday 4th Nov.) Stella (Monday 7th Nov.) GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 67 ------ o 2 OVC CAREGIVERS o 2 CHILD PROTECTION COMMITTEE MEMBERS o 2 HEAD/OVC FOCAL TEACHERS o 8 OVCS  HBC (8)------------------------------ -------- o 2 CCAS o 6 HBC CLIENTS Ojara (Friday 4th Nov.) Drasi (Friday 4th Nov.) Beatrice (Tuesday 8th Nov.) Grace/Innocent (Monday 7th ) 12:00PM￾12:30PM TRAVEL FROM PAIMOL – KALONGO T/C 12:30PM￾1:30PM LUNCH 2:00PM￾5:00PM KALONGO T/C FINAL GROUP FORUM  GOAL STAFF (2)------------------ --------  PARTNER STAFF (10)----------- ----  KALONGO T/C GOVERNMENT (4)--  STEPPING STONES (10)-------- ----- o 1 LEADER o 1 FACILITATOR o 2 YOUNG WOMEN o 2 YOUNG MEN o 2 OLDER WOMEN o 2 OLDER MEN  PEER EDUCATION (8)---------- -- o 2 PEER EDUCATORS o 6 PE BENEFICIARIES  2 CHFPS----------------------------- --------  2 HCT HEALTH CENTRE STAFF---  OVC (14)---------------------------- -------- o 2 OVC CAREGIVERS o 2 CHILD PROTECTION COMMITTEE MEMBERS o 2 HEAD/OVC FOCAL TEACHERS o 8 OVCS  HBC (8)------------------------------ -------- o 2 CCAS Drasi Alice (Monday 7th Nov.) Ojara (Monday 7th Nov.) Ojara (Monday 7th Nov.) Stella (Monday 7th Nov.) Ojara (Monday 7th Nov.) Drasi (Friday 4th Nov.) Beatrice (Wednesday 9th Nov.) Odong John GOAL NEW SITE GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 68 6 HBC CLIENTS (Monday 7th THURSDAY, NOVEMBER 10, 2011 – EVALUATION TEAM 1 & 2 TIME SUB￾COUNTY & PARISH ACTIVITY RESPONDANTS EVALUATION TEAM LOCATION 8:30AM￾9:00AM TRAVEL FROM KALONGO T/C – WOL 9:00AM – 12:00AM WOL FINAL GROUP FORUM  GOAL STAFF (2)------------- -------  PARTNER STAFF (10)------ ------  WOL S/C GOVERNMENT (4)----  STEPPING STONES (10)--- - o 1 LEADER o 1 FACILITATOR o 2 YOUNG WOMEN o 2 YOUNG MEN o 2 OLDER WOMEN o 2 OLDER MEN  PEER EDUCATION (8)----- -------- o 2 PEER EDUCATORS o 6 PE BENEFICIARIES  2 CHFPS------------------------ --------  2 HCT HEALTH CENTRE STAFF  OVC (14)----------------------- ------- o 2 OVC CAREGIVERS o 2 CHILD PROTECTION COMMITTEE MEMBERS o 2 HEAD/OVC FOCAL TEACHERS o 8 OVCS  HBC (8) o 2 CCAS Drasi Alice (Wednesday 9th Nov.) Ojara (Tuesday 8th Nov.) Ojara (Tuesday 8th Nov.) Stella (Tuesday 8th Nov.) Ojara (Tuesday 8th Nov.) Drasi (Friday 4th Nov.) Beatrice (Wednesday 9th Nov.) Ricky (Tuesday GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 69  6 HBC CLIENTS 8th Nov.) 12:00AM￾12:30AM TRAVEL FROM WOL – KALONGO T/C 12:30AM￾1:30PM LUNCH 1:30PM￾2:30PM KALONGO T/C DEBREIF  GOAL PROJECT TEAM GOAL KALONGO OFFICE FRIDAY, NOVEMBER 11, 2011 EVALUATION TEAM TRAVELS TO KAMPALA GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 70 GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 71 Appendix 3 Case Studies Home Based Care Case Study My name is Akello Christine, aged 35 years. I live in Opiro Central Ward, Pacabol parish, Paimol Sub County, Agago District. I became very sick in 2008; I was admitted in Kalongo hospital and was seriously ill. I was tested for HIV and found to be HIV positive. I had no one to take care of me in hospital as all my family members had rejected me fearing to contract HIV if they touched me. After being discharged from hospital I didn’t want to go back home where I had been rejected. I came across Prayers HIV group in 2008. I was immediately enrolled as a home based care beneficiary. The home based care services have been very useful to me. Community counseling Aides (CCAs) who are also PLHIV conduct home visiting to us fellow PLHIV. The home based care services I received included Psychosocial support, Health Education, Referrals for treatment of Opportunistic Infections (OIs) and HBC kit which contains condoms, ORS, Water Guard, a Water vessel, and Mosquito Nets. To benefit from HBC services I disclosed my HIV status, and have lived openly with HIV. Before I was enrolled for HBC services, I was totally weak and I could not help myself and hence was hopeless over my future. I was frequently admitted at the Hospital because by then I was not getting any support such as psychosocial support and treatment. After being enrolled under HBC program, I started getting support from Prayer Group HIV. I started having hope in life and soon realized that I could still live a productive for very many years. I learnt that the HBC holistic approach including Psychosocial support, Health Education talks, Spiritual support, GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 72 Referrals for the treatment of Opportunistic Infections (OIs) and provision of Home Based Care kits improved my health, and gave me hope which has enabled me live a productive life. |Given my experience, I know that the intervention can prolong the lives of other PLHIV like it has done for me. I recommend that HBC services should not be confined to Agago district but should be extended to other districts to benefit other PLHIV with such life saving interventions like HBC. Furthermore, stock out of supplies should be minimized. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 73 Stepping Stones Case Study My name is Okot Vincent. I am 65 years old and I live in Kalongo Town Council, Agago District. I joined Stepping Stones in 2008 together with my wife when Wagwoke Wunu came to our village and sensitized us on how stepping stones could assist in changing risky behaviors, improve community engagement and develop community actions to address community problems. I and my wife are members of Akado circle older men and older women stepping stones groups. As a member of stepping stones older men group, I realized that Stepping Stones intervention is an appropriate way of passing on HIV prevention messages in an interesting manner that keeps the participants engaged and focused on changing the negative behaviors of the community that would otherwise put them at risk of contracting the HIV virus. Such risky behaviors include alcoholism, wife inheritance, domestic violence, and reluctance to go for VCT among others. Before I was enrolled as a stepping participant (beneficiary), I didn’t know much about hygiene to the extent that I didn’t have a pit latrine at my home. It was through attending stepping stones sessions that I realized having a pit latrine and a clean hygienic environment is important. I constructed a decent latrine and I now keep my compound clean. We also learnt many other things. For example conserving the environment; counseling skills to encourage HIV disclosure and support to those who are already HIV positive. We also learnt how stigma and discrimination can be damaging to the lives of those infected and how one can change from risky behavior. Grouping of peers within the same age brackets promotes openness during the discussion of sensitive issues on HIV prevention and spread. Additionally, stepping stones has proven in community as the most effective way of preventing the spread of HIV since it empowers both women and men to negotiate for safer sex other than telling you the dangers only. We however experienced some challenges especially during rainy seasons where rain sometimes interrupts sessions since meetings are conducted under a tree. There were no hard copy materials to keep GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 74 for future reference and the time allocated for the sessions was too short for members to fully understand the issues. I hope that this intervention that empowers community members of different ages to address community issues can be extended to other sub-counties within Agago and other districts. This would help change behaviors of people who have returned home from IDPs with risky behavior. Some of the participants were drunkards before joining stepping stones groups but have since stopped drinking. Other practices like wife beating and wife inheritance, have also gone down amongst participants of stepping stones. We have also have gone for HIV counseling and testing and are now very responsible men and women. Stepping Stones facilitators were selected from our community and trained to conduct stepping stones sessions which were held twice a week. The stepping stones process starts with general mobilization in all the parishes to highlight the beneficiaries in question about how the intervention would be implemented and willing bodies register prior to commencement of sessions. Beneficiaries are then grouped into four different peer groups according to their age groups to allow free discussions of sensitive issues including correct and consistent use of condoms whilst demonstrating the use. Peer sessions are followed by general discussions for all the four peer groups i.e. (older men, older women, younger men and younger women) at the end of every session. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 75 Stepping Stones Case study My name is Anna Acayo 37 years old from Wol sub-county, Agago District. I am a beneficiary of Stepping Stones. My behavior drastically changed from an alcoholic to a stable mother as a result of the discussions we had through stepping stones. The sessions I attended focused on behavior change as regards heavy alcohol consumption, building stable marriages, prevention of HIV, testing for HIV and reforming relationships between parents and their children. Before participating in stepping stones discussions, I was a drunkard and would abuse people including my own children whenever I was drunk (with tears as she narrated story). My life was in turmoil and I reached a stage where I contemplated committing suicide. Following discussions during stepping stones sessions however, I started on a journey of reform. I divorced the husband who had inherited me (my late husband’s brother) and started to care and show love to my children. We even started planning together as a family. I now live with my children and feel so happy. I can now do proper house work including preparing food for my family members. I planted cotton and sold it. With the proceeds, I bought two female goats. I had three goats but one died. I have planted mango trees and other trees around my home and my home now looks nice. I am very happy. I have changed my life style a great deal. I went for HIV testing and I was found HIV negative, I celebrated having escaped the disease. I did not expect to be negative, following the kind of life I was leading. I am thankful for the support I was given that enabled me to turn around my life. I want GOAL project to continue and cover the entire region to reach those who are hopeless and misled. Sensitization of the community using the stepping stones method should continue, more trainers are needed to train others who are not yet trained. I would also like another project to come on board to support us in paying school fees for our children. Thank you to the GOAL project. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 76 Appendix 4 Sub County Plenary Fora on Way Forward Wol Sub county plenary discussion o Community pledged to continue community awareness to reduce stigma and discrimination o Youth to form groups to enhance self reliance o Need for solidarity among beneficiaries o VSLA group start up for sustainability o Sub-county authorities should integrate different groups/volunteers into government programmes such as VHTs o Involvement of the youth by the sub-county in various sub-county activities like contract works o Need for continued support for OVCs who have completed P.7 to join secondary education o Both beneficiaries and volunteers to collectively work in collaboration with sub￾county authorities o Need to continue lobbying from other sources to continue working on HIV o Need to own the project outputs(joint ownership of the projects) o Reduced dependency on external support(handouts) o Need for community support for OVCs o Need for increased uptake of HCT services by the community even past the project period o Need for parents involvement in support of OVCs education o Need to support OVCs secondary education o Head teachers should send names of OVCs to the sub-county so that those who pass well can be supported o Disclosure should be promoted o Parents should start educating youth at the fire place o Need to regularly involve all the stakeholders especially beneficiaries in review meetings o Prioritization of HIV in the local government budgets particularly mobile HCT o HCT should override condom usage o Avail sub-county with list of volunteers, data of those infected to help in planning processes o Avoid redundancy and laziness GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 77 o Youth to register as a CBO at the sub-county level o Condoms should be availed only to youth who are above 18 years of age o Youth to distance themselves from risky behaviors e.g. going to disco halls o Lobby for medical equipment from development partners o Ensure active participation of parents and teachers in sub-county planning meetings 5.2 PAIMOL Sub county plenary discussion o SS facilities and beneficiaries will continue sensitizing people about HIV/AIDs o Strengthening of peer educators activities by the sub-county o OVCs should engage in agricultural activities mainly growing cash crops like cotton to raise school fees o CCAs will continue encouraging community members to disclose their HIV status o Need to collaborate with the sub-county authorities o There is need to promote linkages between groups as a way of promoting farming activities o Sub-county authorities to look for alternative support for the OVCs o Sub-county should provide OVCs particularly child headed families with land for cultivation of cash crops o OVCs should concentrate on education rather than going for early marriage whereas the sub- county should support them with scholastic materials o Sub-county authorities will integrate the activities into the various government programmes o Need for communal support/ownership of children o NAADS and NUSAF should target OVCs with IGAs o Strengthening of CPCs by the sub-county o Children should differentiate between rights and responsibilities o Need for sensitization of both children and parents on child protection issues o GOAL should look for alternative funding sources for partner CBOs to continue with the project activities o There l’s need to contract spacious youth center and equip it with computers o Family planning methods should be done with consent of men o Sub-county will target OVCs caregivers with various support under government programmes o Support will be given to best performing OVCs by the sub-county GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 78 Appendix 5 Wol Kico Primary School, Children’s Focus group participants Names Future Dream (What they want to become) Komakech Simon Peter Doctor Chanamita Micheal President Ojok Robert Olanya Teacher Anena Scovia Nurse Anek Sunday Nurse Achan Collins Nurse Oryema Bosco Driver (encouraged to think Bigger) GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 79 Appendix 6 Evaluation tools Tools 1-3 were used in preparatory planning and documents review. The following tools were used during the data collection field work. Tool 4: Key Informant Interviews – Professionals (DHT members, Subcounty officials, Health workers, CBOs) GOAL through PEPFAR support has worked with partner organizations to deliver comprehensive and coordinated responses to HIV specifically working on prevention of transmission of HIV and mitigation of its effect on infected and affected individuals, families and communities in Agago district. You have been identified as a respondent for the ongoing evaluation exercise of this project. I kindly request you to provide information towards the questions below exhaustively, to guide the District and GOAL in planning further for these services in Agago and other districts. 1. Progress made in achieving the project results Project aspect Related Project Results Intended: Unintended: Positive Negative Positive Negative Project design and implementation strategies To individuals: To communities: To individuals: To communities To individuals: To communities To individuals: To communities Technical competency to undertake the planned activities To individuals: To communities To individuals: To communities To individuals: To communities To individuals: To communities Technical support and capacity building for partner organizations and other To individuals: To communities To individuals: To communities To individuals: To communities To individuals: To communities GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 80 community structures Relating with project stakeholders and other service providers To individuals: To communities To individuals: To communities To individuals: To communities To individuals: To communities External factors To individuals: To communities To individuals: To communities To individuals: To communities To individuals: To communities If there is Stigmatization and/or discrimination of beneficiaries and non-beneficiaries effect of project criteria for inclusion / exclusion in the programme To individuals: To communities To individuals: To communities To individuals: To communities To individuals: To communities 2. Service delivery and required need in communities a) How appropriate has the project design been in addressing service delivery and community need for services? (Appropriateness of Project design) GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 81 b) How has the project addressed technical competency of implementers for undertaking the planned activities. Has this been appropriate for project effectiveness? c) How has the project dealt with technical support and capacity building for partner organizations and other community structures? Has this been appropriate for project effectiveness? d) How has the project worked with project stakeholders and other service providers? How would you describe the relationship between project and other stakeholders? e) What external factors (if any) may have affected planned project services delivery and how? f) Gender: In your opinion has the project been gender responsive in its design and implementation? g) Conflict affected population: In your opinion did the project pay attention to the uniqueness of the post conflict transition. If yes how? 3. How has sustainability of services delivery been addressed/planned for during project implementation (list interventions that have been undertaken to ensure sustainability, by District, Sub county, GOAL. 4. Is there need to replicate the project activities in other sub counties and districts – Why and how can this be done? 5. How can the project activities be further maximized for effectiveness in case of extension or scale up beyond project area of operation? 6. CBOs Organizational Capacity: Describe what the project has done in the following areas of CBOs / your organization, giving examples that show outcomes and impact: - Administrative capacity - Organizational set up - Financial systems - Good governance (Leadership skills development, Constitution and other operating instruments e.g. organizational policies) Thank You Very Much GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 82 Tool 5: Focus groups discussion guide21 GOAL through PEPFAR support has worked with partner organizations to deliver comprehensive and coordinated responses to HIV specifically working on prevention of transmission of HIV and mitigation of its effect on infected and affected individuals, families and communities in Agago district. You have been identified as a respondent for the ongoing evaluation exercise of this project. I kindly request you to provide information towards the questions below exhaustively, to guide the District and GOAL in planning further for these services in Agago and other districts. General section: How have the project activities helped you? What are the results that you attribute to the project? {HIV Prevention (Stepping stones); HBC (PLHIV); HCT (General Public)} What positive results have you noted as a result of the project? What negative results if any, have you noted as a result of the project? Sustainability: ‐ What has the (Community, Government, NGOs/CBOs) contributed towards project activities to date ‐ What does the Community…… plan to continue doing in support of project activities even after the project closes? ‐ What measures have been put in place that can help project activities continuity beyond GOAL support? ‐ What additional measures should be put in place (and by whom) to ensure continued activities after project closure? Services delivery: ‐ What changes in services delivery have you noticed as a result of the project? ‐ What factors influenced service results? Positive/negative 21 This Focus group interview guide is to be used for HIV Prevention-Volunteers; HBC (PLHIV) and the Plenary sessions with Sub-county and District stakeholders GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 83 ‐ How have PLHIV and OVC been supported by the project? Project Approach- Alignment of project activities to Socio-cultural context: ‐ Share your thoughts about the way the project fitted within the socio cultural context of the communities served ‐ Describe what approaches worked out well and what did not work out so well /could have been done differently for further project success. ‐ Stigmatization: In your opinion, has there been any stigmatization of beneficiaries /non beneficiaries as a result of the project? {HIV Prevention (Stepping stones); HBC (PLHIV); HCT ( General Public)} (Explain) ‐ Gender: In your opinion has the project been gender responsive in its design and implementation? ‐ Conflict affected population: In your opinion did the project pay attention to the uniqueness of the post conflict transition. If yes how? ‐ In your opinion, what more could have been done by the project for effective HIV/AIDS prevention (HCT, HBC, IEC, BCC programming), HIV/AIDS care and mitigation (OVC and livelihoods programming) services delivery? Scale up: What aspects of the project should be considered if possible, for continuity plus scale up within and beyond Agago district? What other needs exist in the community and should be addressed in future interventions {HIV Prevention (Stepping stones); HBC (PLHIV); HCT (General Public)} Best Practices: In your opinion, what are some of the unique and very effective innovations that the project has implemented which need to be documented and shared beyond the district? {HIV Prevention (Stepping stones); HBC (PLHIV); HCT (General Public)} Thank You Very Much GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 84 Interview guide for Children – OVC, CLWHIV, General Children (Tool 6) Starter: With a song they like Interviewer and the OVC introduce themselves with a song Interactive discussion and interview: 1. How were you indentified by Goal as in need of support (outreach, referral, church, school, community volunteers)? 2. Draw how you were before goal and after? 3. Draw the type of support you received from Goal. (Family, social, peer networks, education support)? 4. Number the drawings according to what you needed most? (ranking) 5. What did you like about the project? 6. What didn’t you like about the project? 7. Tell me about how your community, school, friends and relatives have helped you? 8. How do you help them too? 9. How do you think other children can be helped? 10. What message do you want to give to GOAL and other organizations giving child support? 11. What message do you want to give to your community? (Closure with an interactive exercise – Blow bubbles and make a wish) Thank You Very Much Materials for session: Balloons, crayons/colored pencils, colored paper and scissors, glue, stars stickers, pencil, rubber, sharpener plus a ball, bubbles and drum. GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 85 Tool 7: In-depth Interviews – Volunteers (OVC, Prevention/Steppingstones, HBC, HCT) GOAL through PEPFAR support has worked with partner organizations to deliver comprehensive and coordinated responses to HIV specifically working on prevention of transmission of HIV and mitigation of its effect on infected and affected individuals, families and communities in Agago district. You have been identified as a respondent for the ongoing evaluation exercise of this project. I kindly request you to provide information towards the questions below exhaustively, to guide the District and GOAL in planning further for these services in Agago and other districts. 7. Progress made in achieving the project results Project aspect Related Project Results Intended: Unintended: Positive Negative Positive Negative Project design and Implementation strategies To individuals: To communities: To individuals: To communities To individuals: To communities To individuals: To communities Technical competency to undertake the planned activities To individuals: To communities To individuals: To communities To individuals: To communities To individuals: To communities Technical support and capacity building for partner organizations and To individuals: To communities To individuals: To communities To individuals: To communities To individuals: To communities GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 86 other community structures Relating with project stakeholders and other service providers To individuals: To communities To individuals: To communities To individuals: To communities To individuals: To communities External factors To individuals: To communities To individuals: To communities To individuals: To communities To individuals: To communities If there is Stigmatization and/or discrimination of beneficiaries and non-beneficiaries effect of project criteria for inclusion / exclusion in the programme To individuals: To communities To individuals: To communities To individuals: To communities To individuals: To communities GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 87 HBC Volunteers/Facilitators interview guide 1. Tell me about yourself? 2. What is your role in this project? 3. How are PLHIV in need of support identified (outreach, referral, church, school, community volunteers)? 4. In what ways do you support PLHIV (Practical, material, emotional, peer support)? 5. How do you support PLHIV and families? 6. What are the problems that PLHIV face? 7. How does you work help them overcome these problems? 8. What differences/changes have you seen as a result of HBC? 9. What were the factors which made you to achieve the results of your work with PLHIV? (Goal facilitation, linkages, referrals etc)? 10. What challenges did you face in the cause of your work? 11. Any lessons you have learnt while working providing HBC? 12. How do you think HBC has impacted on the lives of PLHIV and family? 13. What are the issues you think HBC projects like this should address in future? 14. What other needs exist in the community are present that can be address though similar way Goal has been working in the in the community? 15. Whose responsibility is it to offer HBC? (Goal, Government, family) 16. What should be the roles of Government, hospitals, schools, NGOs, community in meeting the needs PLHIV? 17. What example of a case study on HBC do you advice would be good for documentation Thank You Very Much GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 88 HCT Volunteers interview guide 1. Tell me about yourself? 2. What do you in the Goal Project? 3. How did you come be involved in the Goal project? 4. How do you identify people in need of HCT (outreach, referral, church, school, community volunteers)? 5. How do you support the HCT? 6. What were the factors which made you to achieve the results of your work with HCT? (Project design, Goal facilitation, linkages, Coordination, referrals etc)? 7. What challenges did you face in cause of your work? 8. Any lessons you have learnt while working in this project? 9. How do you think this project has impacted on their lives on the members of this community? 10. How would have the project performed better in the context of HCT? 11. What are the issues you think Goal should address in future work on HCT? 12. What other needs exist in the community are present that can be addressed though similar way Goal has been working in this community? 13. Whose responsibility is it to offer this HCT? (Goal, Government, family) 14. Any other thing you would want to share about this project? Thank You Very Much GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 89 OVC Volunteers interview guide 1. Tell me about yourself? 2. What is the focus of your work? 3. How would you define OVC? 4. How are OVC in need of support identified (outreach, referral, church, school, community volunteers)? 5. How do you support OVC and families? 6. In what ways do you support OVCs (Practical, material, emotional, peer support)? 7. What are the problems that OVC face? 8. How does you work help them overcome these problems? 9. What were the factors which made you to achieve the results of your work with OVC? (Goal facilitation, linkages, referrals etc)? 10. What challenges did you face in the cause of your work? 11. Any lessons you have learnt while working with OVC? 12. How do you think Goal work with OVC impacted on their lives and family? 13. What are the issues you think Goal should address in future work with OVC? 14. What other needs exist in the community at present that can be addressed though similar ways Goal has been working in the in the community? 15. Whose responsibility is it to offer this support? (Goal, Government, family) 16. What should be the roles of Government, schools, NGOs, community in meeting the needs of OVC? Thank You Very Much GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 90 Prevention Volunteers interview guide 1. What is the focus of your work? What does it involve? 2. What are the most effective prevention approaches in your view (stepping stones, youth centre, IEC? Why? 3. What were the factors which made you to achieve the results of your work? (Goal facilitation, linkages, referrals etc)? 4. What challenges did you face in cause of your work? 5. Any lessons you have learnt leant while working with Goal? 6. How do you think Goal work on HIV prevention has impacted on this community? 7. What are the issues you think Goal should address in future HIV prevention work? 8. What other needs exist in the community are present that can be address though similar way Goal has been working in the in the community? Thank You Very Much GOAL PEPFAR HIV PROJECT EVALUATION REPORT 2011 - PADER , AGAGO DISTRICTS 91 Tool 9: Best Practices Documentation Title: Best practice: Introduction: Background: (When it was implemented, target beneficiaries, why it was adopted, why it is identified as a best practice) Approach in implementation: (How best practice was implemented) Outcome/Impact of the best practice: (Results, data if available) Lessons Learnt: Challenges: Recommendations: (in scale up within Agago and other districts)