i ADVOCACY FOR BETTER HEALTH PROJECT Cooperative Agreement No: AID-617-A-14-0000 Submitted to U.S. Agency for International Development (USAID), Uganda Plot 1577 Ggaba Road, Nsambya. Kampala Submitted by Moses Dombo, Chief of Party USAID/PATH Advocacy for Better Health Project Tel: +256 312 393200 30th October 2017 The author’s views expressed in this publication do not necessarily reflect the views of The United States Agency for International Development or the United States Government MIDTERM EVALUATION REPORT ii TABLE OF CONTENTS TABLE OF CONTENTS.............................................................................................................................................. ii ABBREVIATIONS ................................................................................................................................................ iv LIST OF TABLES ................................................................................................................................................. v EXECUTIVE SUMMARY.......................................................................................................................................... vi 1.0 INTRODUCTION ................................................................................................................................................ 1 1.1 Project overview............................................................................................................................................... 1 1.2 The Development Hypothesis .......................................................................................................................... 2 1.3 Implementation structure ................................................................................................................................ 3 1.4 The Results Framework .................................................................................................................................... 3 1.5 Rationale for the Midterm Evaluation.............................................................................................................. 2 1.6 The purpose and objectives of the midterm evaluation .................................................................................. 2 1.7 Midterm Evaluation Questions......................................................................................................................... 2 2.0 EVALUATION METHODOLOGY.......................................................................................................................... 4 2.1 Introduction...................................................................................................................................................... 4 2.2 Study design, participants and sampling procedure ........................................................................................ 4 2.3 Data analysis and management ....................................................................................................................... 4 2.4 Evaluation Limitations...................................................................................................................................... 4 3.0 FINDINGS: AMELP PERFORMANCE................................................................................................................... 6 3.1 Introduction...................................................................................................................................................... 6 3.2 Overall performance towards the goal ............................................................................................................ 7 3.3 Result Area 1: Citizens demand for improved quality services........................................................................ 8 3.3.1 IR 1.1: Increased citizens’ awareness of their rights and responsibilities related to health and social services 9 3.3.2 IR 1.2: Community groups advance priorities for improved health and social services............................10 3.3.3 IR 1.2: Improved engagement between citizens and duty bearers ...........................................................12 3.4 Result Area 2: CSOs effectively advocate for and represent communities on policies / issues of citizens concern in the health and social sectors. ....................................................................................................................13 3.4.1 IR2.1: Increased utilization of evidence by CSOs to inform advocacy. ......................................................15 3.4.2 IR2.2: Effective participation of CSOs in local government planning, monitoring and accountability of health and social services............................................................................................................................................16 3.4.3 IR2.3: Enhanced co-ordination and collaboration among CSOs ................................................................17 3.5 Result Area 3: Institutional Capacity of CSOs strengthened...........................................................................18 4.0 FINDINGS: THEMATIC AREA PERFORMANCE..................................................................................................22 4.1 Introduction....................................................................................................................................................22 4.2 Maternal and Child Health (MCH) ..................................................................................................................22 4.3 HIV/AIDS .........................................................................................................................................................23 4.4 Malaria............................................................................................................................................................25 4.5 Orphans and Vulnerable Children (OVC)........................................................................................................26 4.6 Tuberculosis (TB) ............................................................................................................................................26 4.7 Family Planning (FP) .......................................................................................................................................27 4.8 Nutrition .........................................................................................................................................................28 4.9 Cross-cutting aspects......................................................................................................................................29 5.0 COLLABORATION, LEARNING AND ADAPTATION (CLA) APPROACH...............................................................32 5.1 Introduction....................................................................................................................................................32 5.2 Collaboration ..................................................................................................................................................32 iii 5.3 Learning ..........................................................................................................................................................33 5.4 Adaptation......................................................................................................................................................34 6.0 CRITICAL EVALUATION ASPECTS.....................................................................................................................35 6.1 Efficiency ........................................................................................................................................................35 6.1.1 Cost reduction approaches........................................................................................................................35 6.1.2 Time saving approaches.............................................................................................................................36 6.2 Effectiveness...................................................................................................................................................36 6.3 Project sustainability ......................................................................................................................................37 6.4 Relevance of the project................................................................................................................................. 40 6.4.1 Alignment with USAID Uganda’s CDCS 2016-2021 ....................................................................................40 6.4.2 Alignment with national and international policies and frameworks........................................................40 7.0 ACTIVITY IMPLEMENTATION ..........................................................................................................................41 8.0 APPROPRIATENESS OF THE PROJECT STRATEGY ............................................................................................48 9.0 LESSONS LEARNED, GOOD PRACTICES AND INNOVATIONS ...........................................................................49 10.0 IMPLEMENTATION GAPS AND CHALLENGES..............................................................................................51 11.0 CONCLUSIONS AND RECOMMENDATIONS ....................................................................................................54 11.1 Conclusions.....................................................................................................................................................54 11.2 Recommendation.......................................................................................................................................55 ANNEX 1: DETAILED EVALUATION METHODOLOGY....................................................................................................57 ANNEX 2: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – HIV/AIDS ................................................ 3 ANNEX 3: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – TB........................................................... 5 ANNEX 4: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – MALARIA ................................................ 5 ANNEX 5: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – MATERNAL, NEWBORN AND CHILD HEALTH (RMNCAH)........................................................................................................................................................ 6 ANNEX 6: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – FAMILY PLANNING................................. 6 ANNEX 7: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – Nutrition ................................................ 7 ANNEX 8: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – OVC & Other Cross Cutting Issues......... 8 iv ABBREVIATIONS ABH Advocacy for Better Health AIDS Acquired Immunodeficiency Syndrome CDCS Country Development Cooperation Strategy CDFU Communication for Development Foundation Uganda CIDI Community Instigated Development Initiative CSOs Civil Society Organisations CPR Contraceptive Prevalence Rate FAA Fixed Amount Award FGDs Focus Group Discussions FP Family Planning GoU Government of Uganda HIV Human Immunodeficiency Virus KIIs Key Informant Interviews MCH Maternal and Child Health MMR Maternal Mortality Rate M&E Monitoring and Evaluation MoH Ministry of Health NMS National Medical Stores MDR Multidrug Resistant NUPAS Non-US Organization Pre-Award Survey SABH Strengthening Advocacy for Better Health SRH Sexual and Reproductive Health TB Tuberculosis UBOS Uganda Bureau of Statistics UNAP Uganda Nutrition Action Plan USAID United States Agency for International Development WHO World Health Organisation v LIST OF TABLES Figure 1: Districts covered by the advocacy for better health .....................................................................2 Figure 2: Indicator number 1 progress performance at midterm ................................................................7 Figure 3: indicator number 2 performance progress at midterm ................................................................7 Figure 4: INDICATOR NUMBER 3 PERFORMANCE PROGRESS AT MIDTERM ................................................8 Figure 5: ONE OF ACTIVITIES INVOLVING CITIZENS DEMANDING FOR BETTER HEALTH SERVICES..............8 Figure 6: INDICATOR NUMBER 4 PERFORMANCE PROGRESS AT MIDTERM ................................................9 Figure 7: THE PATIENTS CHARTER DETAILING ROLES AND RESPONSIBILITIES OF CITIZENS......................10 Figure 8: ONE OF THE ACTION PLANS IMPLEMENTED BY AN ABH COMMUNITY GROUP IN BUGIRI DISTRICT......................................................................................................................................................11 Figure 9: INDICATOR NUMBER 5 PERFORMANCE PROGRESS AT MIDTERM ..............................................11 Figure 10: INDICATOR NUMBER 6 PERFORMANCE PROGRESS AT MIDTERM............................................12 Figure 11: Citizens attending an advocacy forum in Baitambogwe sub county, Mayuge district..............12 Figure 12: INDICATOR NUMBER 7 PERFORMANCE PROGRESS AT MIDTERM............................................13 Figure 13: INDICATOR NUMBER 8 PERFORMANCE PROGRESS AT MIDTERM............................................14 Figure 14: INDICATOR NUMBER 9 PERFORMANCE PROGRESS AT MIDTERM............................................15 Figure 15: INDICATOR NUMBER 10 PERFORMANCE PROGRESS AT MIDTERM..........................................16 Figure 16: A NEWSPAPER CAPTION ON DEMANDS BY ABH SUBGRANTEES DEMANDING FOR INCREASED HEALTH SECTOR FINANCING.......................................................................................................................16 Figure 17: INDICATOR NUMBER 11 PERFORMANCE PROGRESS AT MIDTERM..........................................17 Figure 18: INDICATOR NUMBER 12 PERFORMANCE PROGRESS AT MIDTERM..........................................19 Figure 19: INDICATOR NUMBER 13 PERFORMANCE PROGRESS AT MIDTERM..........................................19 Figure 20: INDICATOR NUMBER 14 PERFORMANCE PROGRESS AT MIDTERM..........................................20 Figure 21: INDICATOR NUMBER 15 PERFORMANCE PROGRESS AT MIDTERM..........................................20 Figure 22: THE PRESIDENT OF UGANDA VISITS THE STALL OF ABH SUBGRANTEE, NAFOPHANU DURING THE WORLD AIDS DAY IN KALANGALA DISTRICT ........................................................................................23 Figure 23: Talk show on HIV/AIDS Financing on NTV .................................................................................24 Figure 24: EVIDENCE FACT SHEET ON MALARIA POLICY ASKS FOR THE ABH PROJECT..............................25 Figure 25: THE SPEAKER OF THE PARLIAMENT OF UGANDA AND THE PRIME MINISTER OF UGANDA LAUNCH AND SIGN OFF THE PARLIAMENTARY NUTRITION PLATFORM ....................................................28 Figure 26: WHO Representative, Dr. Hafisa Kasule (holding mic) launching the manifesto .....................29 vi EXECUTIVE SUMMARY Project background: The USAID Advocacy for Better Health (ABH) is a five-year project (June 1, 2014 - May 31, 2019) implemented by PATH in collaboration with Initiatives Inc. with a goal to improve the quality, accessibility, and availability of health and social services in 35 districts of Uganda. The purpose of the project is to increase the citizens’ voice for quality service delivery. The project seeks to have; Citizens demand improved quality of services; CSOs effectively advocate for citizen’s concern in the health and social sectors; and CSOs’ institutional capacity is strengthened. A midterm evaluation was conducted to gather comprehensive data on the progress of the project indicators in selected project districts. Study design, participants and sampling procedure: A cross-sectional mixed methods design involving quantitative and qualitative methods of data collection was applied. The key MTE components included the; Household Survey; Health Facility Assessment (HFA); Organizational and Advocacy Capacity Assessment (OACA); Policy Tracking; Focus Group Discussions (FGDs); Key Informant Interviews (KIIs) and Data validation. These respondents were selected from 10 districts. These consisted of; males aged 15 – 54; females aged 15 – 49; youth aged 15 – 24; and OVCs aged 10 – 14. Thirty-three (33) Key Informants (KIs) and ninety-eight (98) FGD participants were involved including men, women and mixed gender youth. The response rate was 99.2% of the 1286 respondents targeted. This included 47.7% males and 52.3% females. Quantitative data were entered in Epidata V3.1 and analysed using Stata V13 software at univariate and bivariate levels. Transcriptions of qualitative data were made, typed notes coded and analyzed in ATLAS.ti version 6. Key findings of the midterm evaluation General progress on indicators: Collectively, the ABH project met targets for 10 out of 15 indicators. This fairly good performance was mainly attributed to objectives two (2) and three (3) of promoting CSOs to effectively advocate for issues of concern to citizens in health and social sectors, and building their institutional and technical capacity. Overall, three indicator targets vs the actual performed below the midterm set targets are; Indicator 8: % of districts with annual work plans that include citizens’ concerns for improved health and social services; and Indicator 11: Number of CSOs that are involved in joint advocacy initiatives. While the performance on these three indicators were below the set midterm targets, qualitative data suggested that project activities had improved citizens’ engagement with duty bearers, and the prospect of attaining end-term targets remains high. Result Area 1: Citizens demand for improved quality services: Overall, interventions under result area one (1) were below the midterm target. Only one indicator on understanding rights was achieved at 37.1% compared to the midterm target of 32.0%. The attainment of this target was partly a result of support and follow up from the sub grantees. On the other hand poor performance on other indicators was largely due to poor mobilization of citizens since the community groups focused on a few parishes. Result Area 2: CSOs effectively advocate for and represent communities on policies / issues of citizens concern in the health and social sectors: Support was provided and enabled CSOs to effectively advocate for and represent communities on policies / issues of citizens concern in the health and social sectors. CSOs’ advocacy agenda included engagement of government authorities in various ways leading to some actions on health and social service issues and policy gaps and focus was put on supporting and encouraging CSOs to collect especially through health facility assessments and its use in advocacy initiatives. CSOs’ attendance of government planning meetings at national and district levels was high, vii which was a major avenue for conveying citizens’ concerns. However, CSOs focused on achieving milestones to enable them recover funds used and missed out on other actions including joint advocacy. Result Area 3: Institutional Capacity of CSOs strengthened: There was high performance on all the four indicators. This good performance was largely attributed to the design of the support offered, focusing on the whole organization. The constant follow-up by staff and willingness of the sub grantees to learn were critical in having the improvements. Efficiency: The Fixed Amount Award (FAA) provided flexibility regarding meeting of milestones. With proper planning, some CSOs managed to meet their milestones at low costs and in a relatively short time since they controlled the whole process. There was deliberate effort by CSOs and the ABH team to use their facilities for some meetings and trainings. CSOs and some community groups integrated some of the ABH interventions into their routine activities. For instance, some CSOs trained champions and leaders of community group outside the ABH arrangement, leading to cost savings for the project. The formation and use of coalitions was a major approach and key to advocacy initiatives, that were in most cases, associated with co-financing interventions. Similarly, working through network organizations such NAFOPHANU facilitated accessibility to their structures and an avenue for cheaper and timely implementation of some activities such as evidence collection. Capacity building by project staff saved funds that would have been paid to consultants while the Integrated Support Supervision approach saved time and enhanced quality. The capacity building interventions improved the capacity of CSOs to deliver their advocacy interventions. Key informants revealed that the project provided support that was requested by CSOs, which led to improvements at organizational level. Collaboration, Learning and Adaptation approach: The project had collaborations with other USAID Implementing Partners through joint initiatives, performance review meetings and participation in other relevant coordination platforms. The project provided a platform to enhance linkages and complement efforts of different programs to pursue common objectives. For instance, a network of 31 CSOs was formed in Southwestern region. The project promoted learning ranging from OACA through training and participation in international events. Four editions of the Advocacy Alerts! Newsletter were produced and distributed, which facilitated documentation and sharing of information among staff, partners and donors. Staff participation in international conferences promoted learning through presentations Key challenges and gaps in project implementation Community groups focus on a few parishes within their sub counties: While community groups are mandated to raise community awareness in all villages within their respective sub counties, they concentrated on a few parishes. The frequency of large scale engagement of community members within the reached parishes, was limited to about one per year. Poor perception of advocacy by some duty bearers and high demand against low supply: Some of the duty bearers and leaders still have negative attitudes towards advocacy. They tended to view who ever advocates for improved services negatively. On the other hand, the demand for services in several communities exceeds the resources available at health facilities. Evidence collection and use: While mechanisms for linking district and national level CSOs for data collection and use were in place including use of social media, there CSOs still expressed a gap in having a more formal and well-coordinated mechanism. viii Slow adaptation and understanding of the FAA mechanism: The FAA mechanism and associated retirement of milestones minimizes on the number of documents required to liquidate transaction expenses, the number of documents used as MoVs was still big. This contributed to delays in the retirement of milestones and subsequently negated implementation. Though sub grantees performed very well in meeting their milestones, they delayed to retire them. This led to a very low burn rate as money was held up in milestones and low burn rate below the 75% minimum. Subsequently, the project under spent on the year’s obligated budget. Key recommendations i. Community groups covered one or two parishes largely due to limited funding. The ABH senior management should work with CSOs to refocus this community component to cover a small number of parishes within a sub county based on available funds. ii. Given the limited funding, the CSOs should collaborate with religious institutions and use their gatherings and structures to pass over the messages. The messages can be delivered at a convenient time, for instance during or at the end of the service or mass. iii. The media messages are not clearly understood by citizens at community level. The ABH project team should conduct an assessment of the communications component that will inform its further development including the design, packaging and delivery. iv. Most government officials had not fully appreciated the value of advocacy. The ABH project senior management should offer more support to CSOs in engaging the senior leadership of districts and the Uganda Local Governments Association to support on this. v. The ABH senior management should work with sub grantees to strengthen or establish more accountability platforms that will enable tracking of performance commitments. Linkage with very high ranking decision makers such as Permanent Secretaries will be more critical for the success of these accountability mechanisms. vi. To strengthen coordination for data evidence collection and use among district and national level CSOs, the project management should strengthen on current modalities especially ensuring linkage of other CSO staff. vii. The media messages are not clearly understood by citizens at community level. Even when some people hear some radio messages, it takes long for them to associate with it. The ABH project team should conduct an assessment of the communications component that will inform its further development including the design, packaging and delivery. viii. CSOs should incorporate VHTs as resource persons to help raise aware in communities. The VHT structure was wide spread and VHTs commanded good respect, in their communities and good at communicating on health issues. However, since they work closely with health facility staff, their role should largely focus on raising awareness. 1.0 INTRODUCTION 1.1 Project overview The Advocacy for Better Health Project is a five-year initiative (1st June 2014 – 31st May 2019) funded by the United States Agency for International Development (USAID) with an estimated budget of $20million. The project’s goal is to improve the quality, accessibility, and availability of health and other social services. The project aims to enhance the capacity of citizens and Civil Society Organizations (CSOs) to carry out effective advocacy for increased investment and accountability by decision-makers so as to improve the quality and availability of essential health and social services in 35 target districts in Uganda. The project’s expected results include: 1. Citizens demand improved quality services 2. CSOs effectively advocate for issues of citizens’ concern in the health and social sectors 3. Institutional capacity of CSOs strengthened The project’s strategies include; mobilizing, motivating, and empowering citizens and CSOs with information, skills, tools, and systems to more effectively advocate for accessible and high-quality health and social services. The project further sought to; strengthen platforms and mechanisms for meaningful and productive engagement between communities and decision-makers to influence policies and decisions that affect their citizens’ daily lives. PATH, in partnership with Initiatives, seek to apply a catalytic approach to empower citizens through PLHIV groups / networks, MARPS, munomukabi, youth, women, VHT and Health Unit Management Committees with skills, tools, and systems to effectively advocate for accessible, high-quality health and social services. Specifically, the project seeks to: • Utilize multiple integrated community interventions and communication strategies that enable community members to tell their own stories, generate dialogue and debate, and foster critical thinking related to health and social service delivery needs and issues. • Engage with existing community networks and structures to mobilize and empower citizens to become effective health and social services advocates. • Strengthen CSOs organizational systems and processes to be more effective and sustainable in order to effectively represent community interests and collaborate and interact with local decision-makers and service providers. • Expand, revive, or (if necessary) establish mechanisms for citizen engagement at the health facility, sub-county, and district levels to improve citizens’ and CSOs’ ability to participate, act, and influence policy change in public and private spheres. The projec’s implementation covered was at the national level and in the following 35 target districts; Budaka; Bududa; Bugiri; Bukwo; Bushenyi; Busia; Butaleja; Ibanda; Iganga; Isingiro; Kabale; Kalangala; Kaliro; Kamuli; Kamwenge; Kanungu; Kapchorwa; Kasese; Kayunga; Kiruhura; Kisoro; Kumi; Kyenjonjo; Luwero; Mayuge; Mbale; Mityana; Mpigi; Nakasongola; Namutumba; Ntungamo; Pallisa; Rukungiri; Sembabule; and Sironko. 2 FIGURE 1: DISTRICTS COVERED BY THE ADVOCACY FOR BETTER HEALTH 1.2 The Development Hypothesis Figure 1: The change process Citizens’ awareness of their rights & responsibilities related to health and CSOs’ advocacy and organizational capacity improved Citizens believe they can hold their leaders accountable & influence policies to improve CSOs better facilitate policy analysis, civic engagement and represent citizens’ interests with their leaders Citizens demand solutions & changes in health & social policies Decision makers change policies and become accountable Improved health and social services 3 The project’s theory of change was based on the beliefs that IF citizens’ knowledge and awareness of their rights and responsibilities were increased (to stimulate collective consciousness); and IF the capacity of CSOs was built to effectively empower and represent communities, THEN, citizens would believe and have confidence that they can hold their leaders accountable and influence them to change health and social policies in their favour. This empowerment and confidence would motivate citizens and CSOs to demand for better health and social services from their leaders’/decision makers. The persistent collective voice and actions from citizens and CSOs would compel decision makers to respond by changing the necessary policies and take other actions that lead to improvements in the accessibility, availability and quality of health and social services. The conceptualization of the theory is illustrated in figure 1 above. The project sought to ensure that these‘IF” statements are met, thereby contributing to meeting the project’s overall objective. 1.3 Implementation structure PATH was the prime recipient responsible for overall implementation and success of the project. PATH’s main implementing partner was Initiatives Inc, which was responsible for organizational capacity building of CSOs. PATH also provided sub grants to 20 Ugandan CSOs to implement the project at national level and district levels. The project was supported by an Advocacy Advisory Group (AAG) comprised of eminent Ugandans who provide technical assistance and advice to the implementation team to bolster project outcomes. The AAG played a catalytic role in improving the health and wellbeing of Ugandan citizens by championing citizen advocacy and decision-maker accountability throughout the health and governance system and supporting, facilitating and promoting the project approaches and activities. The project team comprised of technical and management professionals as well as support staff based in the central office in Kampala and three regional offices in Kampala, Mbale and Mbarara. These provided oversight to teams employed by partner CSOs in ensuring that planned activities were timely implemented, adequately monitored and results thereof accurately documented and reported. 1.4 The Results Framework The Project’s results framework was used for planning, monitoring, management and communication of the project’s inputs and results. The framework explains the cause-effect linkages between the intermediate results and higher level results; as well as the critical assumptions for successful achievement of projects results. The project modified and adopted the Results Framework (RF) developed by USAID/Uganda as presented in the Request for Applications (RFA) documents that guided the design of this project. The project was designed to contribute towards improved quality, accessibility and availability of health and social service delivery by working towards three strategic objectives. To achieve this goal, the project pursued three key results areas (KRs) as illustrated below. 1 ADVOCACY FOR BETTER HEALTH PROJECT: RESULTS FRAMEWORK IR3.1 CSO instituti onal capacity strength ened. Increased capacity of local governments and facilities to deliver quality services Goal: Improved quality, availability and accessibility of health and social services R1: Citizens demand improved quality of services Sub R3: Institutional capacity of CSOs strengthene d 1. Target community groups are PLHIV groups/networks, MARPS, muno mukabi, youth, women, VHT and Health Facility Management Committees 2. The project operates in 35 project districts of Uganda 3. Key focus services are: malaria prevention, HIV prevention, care and support, PMTCT, TB, maternal and child health, sexual reproductive health OVC and nutrition R2: CSOs effectively advocate for issues of citizen’s concern in health & social sectors. IR1.1: Increase d citizens’ awaren ess of their rights & responsi bilities. IR1.2: Commu nity groups advance prioritie s for improve d health & social services. IR2.1: Increase d utilizatio n of evidence by CSOs to inform advocac y IR1.3: Strength ened forums for citizens to engage with duty bearers. IR2.3: Enhance d co￾ordinati on & collabor ation among CSOs IR 2.2: Effective particip ation of CSOs in LG planning , monitori ng & account ability of health & social services 2 1.5 Rationale for the Midterm Evaluation In accordance with the cooperative agreement between PATH and USAID, the project required to undertake a midterm evaluation to measure the degree to which the project is meeting its key objectives, relevance, efficiency, effectiveness and sustainability. This was sought to document lessons learnt, strengths and weakness/gaps to be improved upon so that the project achieves its objectives. In line with USAID’s policy, the mid-term evaluation was overseen by an external consultant. The results of the midterm evaluation were envisaged to be used to update performance indicators with progress to guide the project interventions. There was also envisaged to provide performance aspects of the project including; progress on implementation, expected results, and effect on partners based on the findings of subsequent evaluations. 1.6 The purpose and objectives of the midterm evaluation The purpose of the midterm evaluation sought to gather comprehensive data on the progress of the project indicators in selected project districts. The objectives of the mid-term evaluation were to: 1. Determine the extent to which project deliverables (inputs, training, equipment and services) have been delivered and results attained; 2. Establish the extent to which citizens have been empowered to demand for improved health and social services in their communities. 3. Establish the extent to which service delivery has improved at the selected health facilities in terms of reduced absenteeism, compliance with closing and opening hours and consistent stocks of essential drugs as per Ministry of Health guidelines. 4. Determine the extent to which citizens are involved in making decisions concerning delivery of quality health and social services in their communities. 5. Establish the extent to which sub grantee CSOs have effectively advocated for citizens’ concerns in the health and social sectors. 6. Establish the improvements in the organizational and advocacy capacity of sub grantee CSOs. 7. Identify key success areas and lessons, and make the appropriate recommendations to the project team, the donor and other stakeholders to guide decision-making and planning. 1.7 Midterm Evaluation Questions Based on the above indicators, the midterm evaluation sought to answer the following broad questions; • To what extent are citizens empowered to demand for improved health and social services in their communities. • Are there improvements in service delivery at health facilities in terms of change in absenteeism, compliance with closing and opening hours and consistent stocks of essential drugs as per Ministry of Health guidelines? • To what extent are citizens involved in making decisions concerning delivery of quality health and social services in their communities? • To what extent the sub grantee CSOs have effectively advocated for citizens’ concerns in the health and social sectors? • What is the current organizational and advocacy capacity of sub grantee CSOs? • What is the extent to which districts have utilized or included CSO and citizen issues in their planning processes? 3 The midterm evaluation sought to answer the following specific areas of project strategy, design and implementation: i) Relevance – Assess design and focus of the project • To what extent did the Project achieve its overall objectives? • What and how much progress has been made towards achieving the overall outputs and outcomes of the project? • To what extent were the results (impacts, outcomes and outputs) achieved? • Were the inputs and strategies identified, and were they realistic, appropriate and adequate to achieve the results? • Was the project relevant to the identified needs? ii) Effectiveness- Describe the management processes and their appropriateness in supporting delivery • Was the project effective in delivering desired/planned results? • How effective were the strategies and tools used in the implementation of the project? • How effective has the project been in responding to the needs of the beneficiaries, and what results were achieved? • What are the future intervention strategies and issues? iii) Efficiency - Of Project Implementation • Was the process of achieving results efficient? Specifically did the actual or expected results (outputs and outcomes) justify the costs incurred? • Were the resources effectively utilized? • Did project activities overlap and duplicate other similar interventions (funded nationally and/or by other donors? • Are there more efficient ways and means of delivering more and better results (outputs and outcomes) with the available inputs? • Could a different approach have produced better results? iv) Sustainability • To what extent are the benefits of the projects likely to be sustained after the completion of this project? • What is the likelihood of continuation and sustainability of project outcomes and benefits after completion of the project? • Describe key factors that will require attention in order to improve prospects of sustainability of Project outcomes and the potential for replication of the approach? 4 2.0 EVALUATION METHODOLOGY 2.1 Introduction This section presents the study design, data collection methods, sampling and sample size estimation, data analysis, validity and reliability of findings, challenges and limitations faced. 2.2 Study design, participants and sampling procedure A cross-sectional mixed methods design involving quantitative and qualitative methods of data collection was applied. This was generating both statistical and explanatory information relevant for improving the project in the next phase. Primary data collection was divided into seven (7) components including the; Household Survey; Health Facility Assessment (HFA); Organizational and Advocacy Capacity Assessment (OACA); Policy Tracking; and Data validation as well as Key Informant Interviews (KIIs) and Focus Group Discussions (FGDs). Several documents were reviewed as part of the MTE. A total of 1,276 citizens including 47.7% males and 52.3% females participated in the survey. This represented 99.2% of the target respondents, which is adequate for generalization of the study population. The sample size was selected using the formula proposed by Cochran, W. G. 1963.1 This sample consisted of males aged 15 – 54; females aged 15 – 49; youth aged 15 – 24; and OVCs aged 10 – 14; selected using probability proportional to sample from the ten (10) districts that were covered by the baseline survey. Thirty-three (33) Key Informants (KIs) were involved including USAID Implementing partners; sub grantee staff; ABH project staff; Health Facility (HF) In-charges; and District Health staff. Ninety-eight (98) FGD participants were involved including men, women and mixed gender youth. Details of the methods section are in Annex 1. 2.3 Data analysis and management Quantitative data were entered in Epidata V3.1 and analysed using Stata V13 software. The analysis output requested in the TOR can be categorized into univariate and bivariate. In univariate analysis, charts and frequency distribution tables for single variables were used to show the values of indicators. In bivariate analysis cross tabulation between key variables was used to identify relationships. Qualitative data capture was through use of both digital voice recorders and note-taking. Transcriptions were made, typed notes read thoroughly and codebook developed and data analyzed in ATLAS.ti version 6. 2.4 Evaluation Limitations The evaluation process was faced with limitations though they were insignificant to deter progress and the results. The following are among possible limitations of the data collected: • In Key Informant Interviews, Focus Group Discussions, and Household Surveys there is the possibility of recall bias, as respondents may selectively remember and report elements of the project to study staff. 1 Cochran, W. G. 1963. Sampling Techniques, 2nd Ed., New York: John Wiley and Sons, Inc. 5 • In Key Informant Interviews, Focus Group Discussions, and Household Surveys there is also the possibility of social desirability bias, where respondents provide answers that they believe study staff are looking for. • The data collection faced logistical challenges, such as difficulty reaching some places within Bukwo district due to heavy rains on the roads, and the busy schedules of Key Informants. These logistical challenges delayed data collection. In order to minimize the possible impacts of these limitations, probes were used to help respondents recall activities, and data collection questions were presented neutrally and in private in order to minimize social desirability bias. 6 3.0 FINDINGS: AMELP PERFORMANCE 3.1 Introduction Presented in this section are the findings of the ABH project mid-term evaluation. The assessment involved comparison of the mid-term values and set targets. The indicators are numbered following the format in the Activity Monitoring, Evaluation and Learning Plan (AMELP). The score represents the project’s high performance across several indicators. Details are reflected in figure below. Table 1: Summary table on AMELP Indicator Targets PERFORMANCE INDICATOR Baseline MTE Target MTE Actuals INDICATOR 1: % of men and women who say health service delivery in public health facilities has improved in target districts the last one year. 54.7% 62% 65% INDICATOR 2: % health facilities that report improvements in service delivery2 0.0% 25% 49% INDICATOR 3: % of citizens who report having participated in an activity to demand for improved health and social services in the last one year 29.6% 45% 48% INDICATOR 4: % of citizens who demonstrate understanding of rights and responsibilities related to health and social services. 14.0% 32% 51% INDICATOR 5: % of community groups whose action plans advance into implementation phase. 0.0% 48% 128% INDICATOR 6: Number of functional3 advocacy forums at sub county level. 0 418 416 INDICATOR 7: % of sub grantee CSOs that demonstrate influence on health and social services agenda 0.0% 75% 100% INDICATOR 8: % of districts with annual work plans that include citizens’ concerns for improved health and social services. 0.0% 50% 40% INDICATOR 9: % of CSO advocacy initiatives which are supported by evidence 0.0% 60% 100% INDICATOR 10: % CSOs actively involved in public sector planning processes. 0.0% 60% 150% INDICATOR 11: Number of CSOs that are involved in joint advocacy initiatives 4 16 11 INDICATOR 12: % of CSO with overall improvements in organisational capacity. 0.0% 70% 90% INDICATOR 13: % of sub-grantee CSOs that demonstrate improvements on the Advocacy components of the OACA4 0.0% 80% 85% INDICATOR 14: Number of sub-grantee CSOs that attain adequate performance as defined in Organizational and Advocacy Capacity Assessment. 0 6 16 2 Service delivery improvement will entail reduced absenteeism, personnel completing full working hours, reduction in essential drug stock outs as per MOH guidelines 3 Functionality of advocacy forums will be measured on regular meetings, structured agenda, structure, minutes of meetings, follow up and attendance 4 Skills and capability strengthened in key areas including policy analysis, M&E, strategic advocacy and communications. 7 0% 25% 49% 0% 10% 20% 30% 40% 50% 60% Baseline Year 3 Target Y3 Achievement 55% 62% 65% 45% 50% 55% 60% 65% 70% Baseline Year 3 Target Y3 Achievement FIGURE 2: INDICATOR NUMBER 1 PROGRESS PERFORMANCE AT MIDTERM FIGURE 3: INDICATOR NUMBER 2 PERFORMANCE PROGRESS AT MIDTERM PERFORMANCE INDICATOR Baseline MTE Target MTE Actuals INDICATOR 15: Number of sub grantee CSOs with management systems that qualify them to receive direct donor funding in accordance with USAID’s NUPAS 0 1 2 3.2 Overall performance towards the goal INDICATOR 1: % of men and women who say health service delivery in public health facilities has improved in target districts the last one year. This indicator is at goal level and was designed to measure the proportion of men and women who report that health service delivery in public health facilities has improved in target districts during the last one year. The services measured are related to HIV/AIDS, MCH, Malaria, Reproductive Health, TB, Nutrition and services for OVC. The performance to date is reflected in the chart below; Compared to the baseline of 55%, the proportion of the communities that are satisfied with the quality of services has increased to 65% of men and women that acknowledged improvement in health service delivery in public health facilities. Further analysis showed that malaria (40%), immunization (16%) and HIV/AIDS (15%) were cited as the major health service delivery improvement areas among men and women. The least health service delivery area that registered improvement was MCH (2%). Uganda governments’ efforts to reduce the disease of malaria and immunization through distribution of mosquitos nets and mass immunization of children under five against 6 killers respectively could be a contributing factor towards citizens’ perception about improvements in malaria and immunization services. INDICATOR 2: % health facilities that report improvements in service delivery5 This indicator is at goal and was designed to measure the health facilities that report reduced absenteeism, personnel working full day working hours, reduction in essential drug stock outs as per MOH guidelines, health facility full opening hours. Improvements denote reduction in essential drug stock outs, fulfilment of MOH guidelines on the health unit opening hours and reduced absenteeism of the health unit personnel. The project’s target of improving health facility service delivery has been met at 49 % of the surveyed health facilities at the level of health center III’s and health center IV’s. This is in 5 Service delivery improvement will entail reduced absenteeism, personnel completing full working hours, reduction in essential drug stock outs as per MOH guidelines 8 30% 45% 48% 0% 10% 20% 30% 40% 50% 60% Baseline Year 3 Target Y3 Achievement FIGURE 4: INDICATOR NUMBER 3 PERFORMANCE PROGRESS AT MIDTERM comparison to the midterm evaluation targets of 25%. Improvements have been primarily realized in terms of health workers’ improved adherence to working time, leading to reduction in any health facility closures from 47% to 3%. However, there are still disruptive and rampant stock outs of drugs, commodities and supplies at the facilities which were prevalent in 72% of the surveyed health facilities. 3.3 Result Area 1: Citizens demand for improved quality services This section highlights the extent to which planned activities and outputs, geared at enabling citizens to demand for improved quality services, were delivered. Varying action to meet set outputs resulted in progress on mid-term indicators. In spite of underperformance on some indicators, qualitative findings did suggest there has been progress in Results Area 1, as described by CSO staff: The public was kind of sleeping. They did not know that demanding for what, for their rights, that you demand for it. […] But now they know that it is in their power to have these services and to push for them even when their services are there, to push for the better service delivery. That was not there before. And to me, that is one of the biggest things that has been done. - Executive Director, District-level CSO, July 2017 INDICATOR 3: % of citizens who report having participated in an activity to demand for improved health and social services in the last one year. This indicator measures ccitizens who report having participated in an activity to demand for improved health & social services in the last one year. Citizens refer to Ugandans who are entitled to healthcare rights. Citizens’ participation includes contacting a duty bearer and/or attending community meetings. Activities include citizens’ participation in forums, meetings. Demand is an action taken by citizens to improve availability, accessibility and utilization for health & social services. Services considered are maternal and child health, TB, SRPH, Nutrition, malaria prevention & OVC. The performance to date is reflected in the chart below; At baseline, 30% of the surveyed citizens reported to have participated in an activity to demand for improved health and social services in the last one year was mainly through attending community meetings and getting together with peers to discuss issues of concern in their community. At midterm, this percentage has increased to 48% of the total respondents who reported having participated in an activity to demand for improved health and social services in the last one year. For instance, a number of focus group participants spoke of having contacted their duty bearers on health services issues: one of the FGD participants said that FIGURE 5: ONE OF ACTIVITIES INVOLVING CITIZENS DEMANDING FOR BETTER HEALTH SERVICES 9 FIGURE 6: INDICATOR NUMBER 4 PERFORMANCE PROGRESS AT MIDTERM “You write a letter and deposit it in the suggestion, and later its opened and the letter is discussed among local leaders with health workers and forge a way forward.” - Women’s FGD Participant, August 2017 Engagement activities include attending an advocacy forum, members attending community group meetings, monitoring availability of health workers and drugs at the facility and getting together with peers to discuss issues of concern in their community. 3.3.1 IR 1.1: Increased citizens’ awareness of their rights and responsibilities related to health and social services Traditional platforms such as barazas (citizen engagement meetings), Munnomukabi groups (Self Help groups), PLHIV support group network meetings; and saving and credit cooperative (SACCO) meetings were applied to raise citizens’ awareness. For instance, 105 community advocacy forums including community dialogues, interface meetings and awareness sessions were conducted across the 35 districts in the first two (2) years. Drama was embraced as an edutainment approach to create awareness on rights and responsibilities. This aided modeling desired behaviors. Two sub-grantees; RHU and ARUWE, conducted 12 drama shows focusing on the Patients’ Charter. These initiatives contributed to the 51% of citizens who knew their health rights and 62% who knew their health responsibilities. The project engaged media houses in various ways including; broadcasts and placements of advocacy messages on radio, TV and in newspapers. For instance, in the first two (2) years, sub grantees conducted 284 live call-in radio talk shows on local FM radio stations, raising citizens’ awareness on health rights and responsibilities. Further, 24 episodes of a radio serial drama were broadcast under the popular award winning Rock-Point 256, aired on 22 local FM stations across the 35 districts. In order to focus the radio talk shows, the Patient’s Charter was used as source of information. The project considered the different languages in the project districts. Accordingly, a total of 8,280 radio spots were broadcast in English and five (5) local languages including Runyankole - Rukiga, Luganda, Lukhonzo, Ateso and Kupsabiny on three (3) national and eight (8) district-based radio stations. INDICATOR 4: % of citizens who demonstrate understanding of rights and responsibilities related to health and social services. This measures the proportion of citizens who mention at least 3 health rights & responsibilities as defined in the Government of Uganda Patients’ Charter. Rights are inalienable entitlements as enshrined the legal and policy frameworks that are agreed upon through consensus that they can be claimed by anyone based on their needs & aspirations. The performance to date on this indicator is presented in the chart below; 10 14% 32% 51% 0% 10% 20% 30% 40% 50% 60% Baseline Year 3 Target Y3 Achievement FIGURE 7: THE PATIENTS CHARTER DETAILING ROLES AND RESPONSIBILITIES OF CITIZENS Citizen knowledge on their rights and responsibilities is measured by proportion of citizens who mention at least three health rights as defined in the Government of Uganda Patients’ Charter. At baseline, 28.7% of the sample stated at least three health rights thus demonstrating understanding of rights related to health and social services. The project supported all CSO partners with IEC materials such as the patient’s charter, the 90-90-90 treatment targets poster and other awareness materials to support their activities aimed at raising awareness on citizen rights and responsibilities specifically those related to health. At midterm, 50.9% of citizens who knew their health rights and 62.4% who knew their health responsibilities. Disaggregation of the data shows no significant difference between male (18.8%) and female (181.3%) reporting to have understood their health rights. Among those who know their health rights, the main source of information was radio (27%), followed by health facilities (22%) and Advocacy forums being the least source of information. Key Informant Interview’s with CSO staff also found that the project was increasing citizens’ knowledge of their rights: To me I think the project has opened people’s minds and are informed. They know what they are supposed to get. People are getting knowledge all the time through meetings. - DHE, July 2017 Among those who know their health rights, 34.9% reporting having experienced abuse on their health rights. Among those who experienced abuse on their health rights, 85.7% reported having taken action including reporting to the duty bearers, whereas 14.3% did not report cases abuse on their health rights to any duty bearer. 3.3.2 IR 1.2: Community groups advance priorities for improved health and social services A total of 479 community groups were oriented and facilitated to identify advocacy issues and conduct community empowerment activities in the 35 districts. This enabled community groups to appreciate and learn how to mainstream advocacy in their activities. Up to 270 media champions, leaders of Community Based Organizations (CBOs), and Community Awareness Facilitators (CAFs) were also trained to appreciate and mainstream advocacy. Media and Advocacy Champions are key in mobilizing the community groups and following up issues. The project, through sub-grantees, supported 479 community groups to identify advocacy issues and develop action plans. These were used to raise issues and seek action by decision makers in the 35 districts. INDICATOR 5: % of community groups whose action plans advance into implementation phase. Community groups whose action plans advance into implementation phase. This indicator looks at the steps being taken by citizens to own and implement the action plan or present it to the duty bearers. 11 0% 48% 128% 0% 20% 40% 60% 80% 100% 120% 140% Baseline Year 3 Target Y3 Achievement FIGURE 9: INDICATOR NUMBER 5 PERFORMANCE PROGRESS AT MIDTERM FIGURE 8: ONE OF THE ACTION PLANS IMPLEMENTED BY AN ABH COMMUNITY GROUP IN BUGIRI DISTRICT For action plans to qualify to be in “implementation phase”, the community group must receive response from a duty bearer6 with tangible steps taken in response to the demands of the citizens. Community groups include MARPS associations, PLHIV networks/groups, Health Unit management committee, VHT, women groups, youth groups, muno mukabi groups. The indicator measurement considers action plans as recommendations made by community groups and the progress as below; Community groups have been active in their grass root empowerment and advocacy activities including conducting health facility service delivery tracking, conducting monthly meetings to discuss evidence collected and then develop advocacy action plans. For instance, in Pallisa district, the health facility monitoring exercise revealed that Puti-Puti Health Centre III had a maternity ward but did not have delivery beds, among other gaps identified. A community group mobilized citizens about the issues and agreed to call a stakeholders’ meeting that was attended by health workers, sub county staff, group members, champions and other citizens. At the meeting, participants agreed to write a letter to the District Health Officer highlighting citizen’s dissatisfaction with having a maternity ward without beds. In response, 6 delivery beds were allocated to Puti-Puti Health Centre III. This is anticipated to attract more mothers to come and access maternal health services at the health center. Tracking the action plans that are developed regularly, the community groups including people living with HIV. Key action plans involve contacting duty bearers, mobilization, health education/promotion, collecting evidence and holding meetings. For instance, the regular tracking done by Bugiri Municipal Advocacy Group in Bugiri district has helped to prevent persistent stock-out of HIV drugs, a community group in Bududa district caused a meeting at Kachonga Health Centre III where stakeholders decided to approach the district and urge it to find money to clear the outstanding electricity bill of over one million shillings; and all the 479 community groups (in 35 districts) are engaged in Monitoring and tracking availability of health workers, drugs and commodities at the HC III in their communities. 6 Duty Bearer is a person who has the mandate to provide a service. 12 0% 418 416 0 50 100 150 200 250 300 350 400 450 Baseline Year 3 Target Y3 Achievement FIGURE 11: INDICATOR NUMBER 6 PERFORMANCE PROGRESS AT MIDTERM 3.3.3 IR 1.2: Improved engagement between citizens and duty bearers INDICATOR 6: Number of functional7 advocacy forums at sub county level. A forum will be a platform to share information, receive feedback, interact and build relationship among the citizens and between them and the service providers. Advocacy forum is said to be functional if it meets regularly, with a structured agenda, clear governance structures (leadership), minutes of meetings, follow up on key actions, full participation of diverse community members, and attendance by duty bearers to listen to citizen concerns and provide feedback. An advocacy forum refers to public hearings/meetings for advocacy purposes only. Diverse community members may include and not limited to considerations on gender, age, race, religion, marginalised and MARPS. Advocacy forums as a community empowerment platform continue to provide avenues for citizens to interface with their duty bearers on issues of major concern. Advocacy forums continue to yield commendable results from the duty bearers. For instance, at a forum in Kamuli district, citizens tasked the DHO to explain why drug stock-outs remained widespread in their facilities. The in-charge for Budadiri HCIV, in Sironko district, commended the use of advocacy forums to search for solutions to challenges crippling service delivery. The forums were attended by the community members and have been able to attract key decision makers like CAOs, DHOs, Political wing, community development officers, health in charges, parish chiefs, LCIII chairpersons and health assistants. The issues presented by the community advocacy champions in the forums included but not limited staffing levels of health workers, inadequate essential drugs in the health facilities, lack of water in maternity wards, late coming and early departure of mid wives and other health workers, low ANC attendance, lack of gloves in health facilities, limited accommodation for health workers, lack of viral load machines and lack of ambulances to help expectant mothers during emergency situations, health workers are poorly motivated while others have no uniforms and equipment. The forums are a great opportunity for citizens to raise issues in the presence of their duty bearers so that they make critical decisions that will improve health service delivery. 7 Functionality of advocacy forums will be measured on regular meetings, structured agenda, structure, minutes of meetings, follow up and attendance FIGURE 10: CITIZENS ATTENDING AN ADVOCACY FORUM IN BAITAMBOGWE SUB COUNTY, MAYUGE DISTRICT. 13 0% 75% 100% 0% 20% 40% 60% 80% 100% 120% Baseline Year 3 Target Y3 Achievement FIGURE 12: INDICATOR NUMBER 7 PERFORMANCE PROGRESS AT MIDTERM 3.4 Result Area 2: CSOs effectively advocate for and represent communities on policies / issues of citizens concern in the health and social sectors. Result Area two (2) was designed to empower sub-grantees to capture citizens’ concerns related to health and social services, gather related evidence and use it to engage duty-bearers to take action. The actions expected from duty-bearers included: policy formulation and implementation, allocation of budgets and establishing programs to address citizens’ concerns for policy change, where necessary. The evaluation reveals largely high effectiveness of interventions and good progress. Under this result area, the strategy used involved PATH’s ten-part process which involves technical assistance through an e-learning course, a three-day workshop using PATH’s Policy Advocacy Strategy Development Curriculum, and accompanying tools. There was also peer-to-peer mentoring, modeling opportunities, and study visits regarding implementation of specific aspects of an advocacy strategy. Training of trainers was carried out for CSO representatives, with particular emphasis on networks and umbrella CSOs. This built a cadre of skilled facilitators who were resourceful during strategy development workshops and provided technical support within their organizations INDICATOR 7: % of sub grantee CSOs that demonstrate influence on health and social services agenda. This indicator denotes the proportion of sub grantee CSOs that demonstrate influence on health and social services agenda. Sub grantee CSO refers to any organization that has received grants from the USAID Advocacy for Better Health Project. Influence refers to making government take action on the health and social service issues or policy gaps in response to citizens’ voice and CSO advocacy agenda. The sub grantees are expected to influence public sector entities to take action address improvements in Maternal and Child Health, Malaria, TB, HIV/AIDS, Nutrition, OVC, Sexual and Reproductive Health. The progress on this indicator is as below; Sub grantee CSOs have exploited various windows of opportunities to engage duty bearers and present their packaged evidence and registered successes. In Kamuli, JIACOFE was out to ensure that district Council develops an ordinance for reduced absenteeism and school dropout of children. The ordinance is currently in a draft form pending presentation to council. In Ibanda district, HEPS presented evidence to district officials on Health worker absenteeism and non-functionality of HUMCS and their effect on Maternal and Child Health. As a result, the CAO recently issued circular to all civil servants to adhere to the code of conduct (reporting at duty stations on time 8:00 am and leaving at 5:00 pm). In Budaka district, MUCOBADI engaged Chief Administrative Officer and DHO on health workers’ absenteeism and as a result, CAO and DHO have taken action and culprit health workers have since been issued with warning letters and urged to refer to the terms of their contracts. 14 0% 50% 40% 0% 10% 20% 30% 40% 50% 60% Baseline Year 3 Target Y3 Achievement FIGURE 13: INDICATOR NUMBER 8 PERFORMANCE PROGRESS AT MIDTERM In Kalangala, the District Health Officer was engaged through the public advocacy forums and Joint stakeholders’ planning meetings to introduce a biometric machine (which has been installed) to monitor attendance of health workers on duty was adversely affecting public health service delivery at health facilities. Again, Kalangala has appointed and trained all the 7 HUMCs in the district after intervention from the District Health Officer who frequently wrote to sub county chiefs and health in-charges asking them to renew these committees. In Kasese, DHO is fast-tracking accreditation of HCII to HCIIIs and disciplinary action has been taken and still in Kasese, absenteeism has been tackled by cautioning affected duty bearers with written memos. NAFOPHANU engaged the media to popularize the 90-90-90 strategy and create awareness on the impact of HIV/AIDS drug stock out to the attainment of 90-90-90 targets and consequently build a critical mass of advocates for ARV/ TB drugs. NAFOPHANU further compiled evidence on HIV/TB drugs which was used to engage National Medical Stores to ensure sufficient and sustained stocks and NMS clarified that in case of any assistance or complaints, districts can send messages to 6090 or call the toll free lines 0800200015 (MTN), 0800300333 (AIRTEL). INDICATOR 8: % of districts with annual work plans that include citizens’ concerns for improved health and social services. The 35 targeted districts will incorporate citizens’ issues of concerns into their annual work plans for improved health and social services. Citizens’ concerns are referred to as their health and social needs and priorities as presented by sub grantee CSOs. The progress on this indicator is presented here below; ABH staff have over time built their partnership with the district departments and in so doing, the departments have appreciated the advocacy gaps that are backed by evidence as real issues that can be prioritized and budgeted for. When appreciated, district local governments have prioritized and ensured budget allocations for the year FY 2017/2018. Among the issues that have been budgeted for include promoting family planning and youth friendly services in Mayuge and Kamuli; recruitment of midwives, anesthetist, laboratory technologists in Nakasongola, Luwero, Kayunga; upgrading of Minani HC II to HCIII in Iganga with a tune of UgShs 100,000,000 million; the district of Kumi has allocated UgShs 7,000,000 for the installation of water, fixing doors and terracing of the maternity wards at Ongino Health Center III. Advocacy for Better Health engagements are expected to influence work planning process of local governments. During the course of Year 2, 3 districts have included priority concerns in work plan for FY 2016/2017. In Kamuli district, costs and schedules to pass an ordinance for reduced pupil absenteeism and school dropout to ensure compliance with the UPE policy have been provided for. In Bugiri district, a dialogue meeting in Buwunga Sub County documented inadequacy mosqito nets, lack of delivery kits for expectant mothers and lack of HIV testing kits at Buwunga H/C III. During the forum, the LCIII chaiperson committed to increase the budget for mosquito nets to ensure all mothers coming for ANC receive a mosquito nets and was included in the subcounty plan. During the follow up meeting, the project 15 0% 60% 100% 0% 20% 40% 60% 80% 100% 120% Baseline Year 3 Target Y3 Achievement FIGURE 14: INDICATOR NUMBER 9 PERFORMANCE PROGRESS AT MIDTERM established that duty rosters and arrival books have been introduced both at Buwunga H/C III and Busoga H/C II for purposes of time management. While there were many districts that incorporated citizen concerns, some district officials expressed that there were supply-side limitations that prevented them from addressing citizen concerns, which may have contributed to the underperformance of this indicator. One district official explained it as such: Even if you advocate, people are going to overwhelm us. […] Demand has been created, fine, but what is it that we can do about that demand? […] We are government, yet you know our government. […] We can’t deliver everything even if we know. - District Health Officer, July 2017 3.4.1 IR2.1: Increased utilization of evidence by CSOs to inform advocacy. The project supported the use of data in enhancing advocacy initiatives at both district and national levels. Sub-grantees were supported to enrich their advocacy initiatives with field data, which became vital in influencing action by decision makers and duty bearers. INDICATOR 9: % of CSO advocacy initiatives which are supported by evidence. CSO will develop initiatives using available or own generated data to advance citizens’ issues of concern. Advocacy initiatives are citizens’ issues which need to be addressed. Evidence means sub grantee CSOs should collect, analyse, and use information or facts to inform advocacy work. Achievement of this indicator implies that the data is analysed and used to advance advocacy issues to the duty bearers and have yielded results - in this context results shall be at least one action taken as a result of the CSO during the reporting time towards improvement of health and social services. In order to understand citizens’ priorities, the project staff at different levels collected data and analyzed it. For quality data, community assessments and action plans were applied. Health Facility Assessments and policy mapping were conducted to identify policies, guidance and financial documents governing district and national service delivery systems and their shortcomings. Stakeholder mapping was conducted, leading to the development of matrices for identifying relevant decision-makers, influencers, interest groups, coalitions or networks, and other CSOs at various levels. The matrices included: specific interests; ability to affect the political environment and relationships with one another. This facilitated the advocacy process. Generating evidence through regular tracking and monitoring availability of health commodities and health workers at public health facilities remains a critical part of the community empowerment process. The project supports the capacity of the community groups in using health facility assessment tools, how to write reports, how to collect and package evidence and documentation for meaningful engagement with the duty bearers. Sub grantee CSOs have generated community responsive evidence to inform advocacy efforts. All 16 0% 60% 150% 0% 20% 40% 60% 80% 100% 120% 140% 160% Baseline Year 3 Target Y3 Achievement FIGURE 15: INDICATOR NUMBER 10 PERFORMANCE PROGRESS AT MIDTERM FIGURE 16: A NEWSPAPER CAPTION ON DEMANDS BY ABH SUBGRANTEES DEMANDING FOR INCREASED HEALTH SECTOR FINANCING the implementers have continued to gather and update evidence through health facility assessments, desk reviews, fact finding missions and using score cards. The evidence has been packaged in form of fact sheets, presentations, posters and dossiers. For example, MAFOC was able to package project achievements into power point presentations, fact sheets and briefs that were shared with the duty bearers, RHU developed fact sheets on Adolescent health policy, HEPS developed evidence dozier on HUMCs’ non functionality, STF developed policy briefs on teenage pregnancies and issue papers on malaria, teenage pregnancy, HIV/AIDS, Orphans and Vulnerable Children (OVC), malnutrition. The findings were disseminated through decision makers contact meetings, district level coordination meetings and engagement with peer organization in coalitions. The evidence generated is being utilized to pursue a number of initiatives including convening a stakeholders’ dialogues to discuss strategies of maintaining sufficient stocks of the 13 Life Saving Commodities, taking action against absenteeism of health workers, district HIV/TB task forces meeting in districts presenting advocacy issues related to HIV/TB testing and treatment, banning importation of non-quality assured Quality Malaria Rapid Diagnostic Tests (m-RDTs) in Uganda, call for action and highlight Health sector gaps in the 2017 FY budget. CSO sub grantees continued to update their evidence to remain relevant and up to date to be able to engage and influence duty bearers to take action. A number of data collection tactics were employed to collect evidence, gathering more evidence on the issues in areas of HIV/AIDS/ TB/ Malaria/ MCH/FP/Nutrition, developing and updating evidence dossiers and making use of data collected by community groups. Evidence was collected, packaged and using different modes including letters, fact sheets or power point presentations and shared during district meeting, CSO coalition meetings, one on one meetings, community meeting or in national newspapers. 3.4.2 IR2.2: Effective participation of CSOs in local government planning, monitoring and accountability of health and social services INDICATOR 10: % CSOs actively involved in public sector planning processes. This indicator measures the proportion of Sub-grantee CSOs actively involved in public sector planning processes. Active involvement of CSOs means sub grantee CSOs must attend, present at a public planning meeting and be involved in monitoring or follow up of public sector programs. Public sector refers to government entities including the districts, sub counties, parishes and villages. The progress on this indicator is summarised below; 17 4 16 11 0 2 4 6 8 10 12 14 16 18 Baseline Year 3 Target Y3 Achievement FIGURE 17: INDICATOR NUMBER 11 PERFORMANCE PROGRESS AT MIDTERM The CSOs have been utilising the advocacy strategies developed using the PATH’s 10-part advocacy strategy development to pursue goals for improvement in services related to HIV/AIDS, TB, MCH, OVC and Nutrition. Sub grantees utilise meetings at the districts to engage the duty bearers by presenting fact sheets, dossier and making presentation on the status of health services in the districts. Tis triggers discussion and call for action. Key meetings and platforms being utilized are Sub county council meetings, district council meetings, district HIV coordination meetings, extended District Health Team meetings. At national level, CSO sub grantees participated in the relevant technical working groups (like the HIHV/AIDS, MCH, and Nutrition) in the ministry of health where a number of issues were pushed. ABH was represented at MCH TWG and has been championing availability and access to LSCs including oxytocin. Civil Society Organizations continued to present evidence about advocacy issues and amplify citizens’ concerns and voices in the district meetings for integration in district priorities. A number of planning and budgeting meetings were attended by our sub grantee organizations. For example, Kyenjojo District, stock out of essential supplies in health facilities, absenteeism of health workers and the need to prioritize health sector were presented. In Mayuge and Iganga where stock out of essential medicines and limited accredited ART sites were reported through the DMC. At the national level, CEHURD participated in Maternal and Child Health Technical Working Group meetings, Civil Society Budget Advocacy Group and has ensured that maternal new born are integrated. Advocacy for Better Health has fully participated in the review of the budget framework paper of the Ministry of Health with emphasis to allocate resources to recruitment of midwives. In Kasese, an advocacy champion in Karambi sub-county participated in the technical planning committee and his presentation sparked the resolution by the district to leverage funds for maternity beds and fencing of the health facility in Karambi. In Mbale, the issue of lack of sanitary facilities in schools was presented which attracted a lot of discussion and resulted in Chief Administrative Officer tasking the District Education Officer to make a follow up and present status report to the District Executive Committee. 3.4.3 IR2.3: Enhanced co-ordination and collaboration among CSOs INDICATOR 11: Number of CSOs that are involved in joint advocacy initiatives. CSOs involve or partner with peers for collective advocacy in response to identified service/policy gaps. Joint advocacy initiatives are advocacy issues sub grantee CSOs would like to address collectively in relation to the project thematic areas. CSOs to be counted are only sub grantee CSOs funded by PATH through USAID Advocacy for Better Health Project.The project identified and pursued opportunities to bring CSOs together to promote alignment of advocacy strategies, plans and resources. This included use of forums, networks and communities of practice. The forums enhanced the collaboration of CSOs with other stakeholders to strengthen District NGO Forums to support coordination among NGOs. Within the forums, CSOs emphasized advocacy for operationalization of national policies and plans especially at district and national levels. The project supported networks or umbrella organizations, bringing members together for advocacy strategy development workshops to align or develop joint advocacy strategies. Some sub-grantee CSOs led joint advocacy initiatives in their respective districts. The progress on this indicator is as below; Working in networks and coalitions is a priority for CSOs to push a common advocacy 18 agenda. A number of functional coalitions and networks have been formed to address particular issues under the advocacy for better health. NAFOPHANU is working with peers CSOs to ensure that patient empowerment to demand for Viral Load testing services is fulfilled. The coalition is also following up on the commitment by Ministry of Health that by end of October 2016, each ART clinic will be able to draw blood samples from patients for onward transmission to the National Laboratory in Butabika for viral testing. HEPS is coordinating with other CSOs on the Western Uganda Coalition for Health Advocacy (WUCHA) where joint initiatives on advocacy to review national HUMCs guidelines are being followed. Similarly, ACODEV engaged the Kasese Civil Society Network (KADDE-NET) and other CSOs and engaged in mass campaign to end maternal deaths in the district. CEHURD has been pursuing the immunization fund as established by the Immunization Act; JIACOFE continued bringing to the attention of coalition members the issue of HIV/TB drug stock outs at facilities in Mayuge and Kamuli; ARUWE held coalition network meetings to evaluate progress on previous advocacy issues fronted by coalition members as well as prioritizing new issues affecting service delivery under MCH and HIV/AIDS. HEPS has coordinated regional ABH sub grantees and CSO member’s advocacy learning and strategizing meeting for joint planning and engagements. The advocacy for better health has attracted involvement of different stakeholders including through the 1st National Presidential Dialogue on Quality of Health Services in the country, peoples’ parliament on NTV, Parliamentary forum on Quality of health services and Parliamentary forum on Nutrition. The project continued to promote coalition and network building both at national and in the districts. For example, in Kasese, ACODEV mobilized partner CSOs to follow up shortage of ARVs in the health facilities within the district. JIACOFE organized two coalition meetings to discuss HIV/TB issues in Mayuge and Kamuli districts. In Kalangala, KADINGO organized a coalition meeting for partners to champion push system of drugs, health emergence transport and unreliability of power supply for health facilities. This was part of the pre-event activities towards commemoration of World AIDS Day in Kalangala district. Advocacy for Better Health in partnership with FANTA have championed Uganda Multi-Stakeholder Nutrition Advocacy Network with World Vision Uganda and UCCOSUN, UNICEF, Cotton On, SPRING, USAID/RECO-PIN, HIVOS East Africa, CUAMM, Concern Worldwide and UGAN Society. A joint nutrition advocacy agenda was developed on strategies for advancing nutrition advocacy in the country. 3.5 Result Area 3: Institutional Capacity of CSOs strengthened Result Area 3 aimed at strengthening the capacity of partner CSOs, focusing on both organizational and technical to effectively manage and implement their advocacy interventions. The organizational capacity component included Governance, Administration, Financial Management, Human Resources Management, Organizational management and Project Performance Management aspects; while the technical capacity focused on skills for carrying out effective advocacy. Under this result area, significant achievements have been recorded. INDICATOR 12: % of CSO with overall improvements in organizational capacity. This indicator measures proportion of sub grantee CSOs that demonstrate improved management systems as defined in the OCA. Overall scores in six management areas: administration, human resource management, financial management, organizational management, and project performance management will be reviewed annually. Improvement refers to an overall increase of 6% in annual OCA scores in the 6 management components. This indicator counts the number of CSOs with improved institutional management scores of 0.06 units annually, up to 3.5 and the progress is below; 19 0% 70% 90% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Year 3 Target Y3 Achievement 0% 80% 85% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Baseline Year 3 Target Y3 Achievement FIGURE 18: INDICATOR NUMBER 12 PERFORMANCE PROGRESS AT MIDTERM FIGURE 19: INDICATOR NUMBER 13 PERFORMANCE PROGRESS AT MIDTERM The results show remarkable progress in the performance of all CSOs in all the six sub sections. The OACA results show improvements in all the 6 sections. The CSOs worked very hard on their actions to attain the improvements. Leadership commitment to the OACA action plan was critical to achieving the improvements. Significant growth has been exhibited in areas of administration, performance and financial management although they scored lower than the rest of the sections. Governance remains the strongest capacity area. Each of the CSOs was supported to develop an action plan that will guide capacity building in priority areas. Key sub grantees with greater improvements (above 0.18 improvements in scores) include ACODEV, Uganda Red Cross Society, LADA, KACSOA, FLEP, CIDI and IDO. INDICATOR 13: % of sub-grantee CSOs that demonstrate improvements on the Advocacy components of the OACA. 8 This denotes percentage of sub grantee CSOs that demonstrate improvements on the advocacy components of the OACA. The advocacy component of the OACA has been adopted and developed from the USAID advocacy index. Improvement refers to increase in annually score for each advocacy area. This indicator counts the number of CSOs. The performance towards the 5 year targets is a s below; This measures improvement on the advocacy index components which are Advocacy Strategy Development and Implementation, Policy and Budget Analysis and Development, Gathering and Use of Evidence, Engagement with Decision-Makers, Advocacy Communications and Outreach, Networking and Collaboration, Community Mobilization and Empowerment, Women, Youth and MARPs. The results show improvements on the key components above. Key sub grantees with greater improvements (above 0.18 improvements in scores) include STF, NAFOPHANU, ACODEV, Uganda Red Cross Society, LADA, KACSOA, FLEP, CIDI, JIACOFE and IDO. 8 Skills and capability strengthened in key areas including policy analysis, M&E, strategic advocacy and communications. 20 0 6 16 0 2 4 6 8 10 12 14 16 18 Baseline Year 3 Target Y3 Achievement 0 1 2 0 1 1 2 2 3 Baseline Year 3 Target Y3 Achievement FIGURE 20: INDICATOR NUMBER 14 PERFORMANCE PROGRESS AT MIDTERM FIGURE 21: INDICATOR NUMBER 15 PERFORMANCE PROGRESS AT MIDTERM INDICATOR 14: Number of sub-grantee CSOs that attain adequate performance as defined in Organizational and Advocacy Capacity Assessment. This denotes count of CSOs that attain adequate performance as defined in Organizational and Advocacy Capacity Assessment. USAID Advocacy for Better Project capacity building models will be assessed annually on the 7 capacity elements: g governance; administration, human resource management, financial management, organizational management, program management and project performance management. A sub grantee is regarded to have attained aadequate performance when it scores 3.0 to 4.0 as defined in the OACA. Progress on this indicator at Year 3 is summarized below. The project has provided support to CSOs by offering targeted technical support in key areas including human resources management, supportive supervision, strategic planning and governance. Other key trainings focused on advocacy strategy development, resource mobilization, fixed amount awards and understanding USG compliance requirements. These efforts have yielded capacity improvement to achieve adequate performance for CSOs. CSOs that have achieved this capacity include STF, NAFOPHANU, ACODEV, Uganda Red Cross Society, LADA, KACSOA, FLEP, HEPS, CIDI, JIACOFE and IDO. INDICATOR 15: Number of sub grantee CSOs with management systems that qualify them to receive direct donor funding in accordance with USAID’s NUPAS. This denotes count of CSOs that attain performance of at least 3.51 in the overall OACA which reflects on the qualification of direct USAID funding. A sub grantee is regarded to have attained strong performance when it qualifies for direct donor funding as defined in USAID’s NUPAS. This indicator will be measured from year three onward. Ccapacity building assessment for NUPAS will be assessed on all 7 capacity elements: governance; administration, human resource management, financial management, organizational management, program management and project performance management. Progress on this indicator at Year 3 is summarized below. The project conducted a mock NUPAS on four selected CSOs to assess their readiness for the NUPAS as part of the expectation by USAID to have at least four CSOs attain the NUPAS score. The NUPAS assessment focused on key elements of the OACA which include Legal structure, Financial management and internal control systems, Procurement systems, Human Resources systems, Project performance management and Organizational sustainability. The two CSOs that have qualified this year are STF which scored 3.6 and JIACOFE which has scored 3.7. The two CSOs will be presented as ready for direct funding from USAID. The project plans to work with an additional six 21 organizations in Year four towards achieving the final project target of four CSOs. 22 4.0 FINDINGS: THEMATIC AREA PERFORMANCE 4.1 Introduction Presented below are the highlights of the coverage of the thematic areas such as Malaria, HIV/AIDS, TB, OVCs, MCH, Family planning and nutrition. The thematic areas were designed to address the focus areas including health financing and HRH, covered under all thematic areas. Commodity security was planned to be addressed under all thematic areas except nutrition while OVC issues would be covered in all thematic areas except TB and family planning. However, given the limited funding for OVC, more attention has been on the first three (3). While the project targeted “everything on health”, coverage of the thematic areas was largely driven by need and the funding available. Accordingly, HIV/AIDS was most covered and OVC least covered respectively. Details of the policy engagements per thematic areas are presented in annex 2. 4.2 Maternal and Child Health (MCH) The project did not plan for specific MCH interventions in the first two years although it partly focused on women under two activities in the first year. These were; developing sub-county action plans with at least 30%focus on interest groups including women, and establishing special forums to reach interest groups including women. In the third year, the project sought to address domestic health financing, HRH, health commodities security relating to MCNH, as well as responding to emerging health advocacy issues. While some advocacy plans were developed and forums established in the first year, they largely focused on engaging communities on general health and social service delivery issues affecting them, without specifying MCH issues. Such generalization may have led to omission of key MCH aspects. Conversely, whereas there were no specific MCH interventions planned in year two, related issues were presented in meetings at sub county and national levels. These focused on eliciting support for increasing the availability and security of medicines and other health commodities. To this effect, the project facilitated tracking of the availability of drugs and commodities, including those for MCH such as Oxtoccine, by CSOs and community groups. For instance, in year three (3), a total of 472 community groups tracked stock availability at health facilities in the 35 districts. The project also raised citizens’ awareness on MCH issues through four (4) radio spot messages which focused on stock-out of drugs and commodities, among other concerns. At national level, CEHURD collaborated with other CSOs and met the management of NMS. Focus was on strategies for efficient stocking of the essential life-saving commodities, including MCH medicines. Realizing the limited funding for health, the project supported initiatives for mobilizing financial resources at sub-county, district and national levels. For instance, in Year Two, a planning meeting of Karambi sub-county in Kasese District led to a resolution to mobilize funds for delivery beds. Similarly, the project supported more related interventions in year three, including raising MCH issues at district level meetings, including at District Health Management Team (DHMT) meetings and budget conferences. For instance, lack of a maternity ward at Butunduzi HCIII in Kyenjojo District was pointed out at the district budget conference. The project facilitated CSOs and community groups to monitor HRH including midwives who are key in MCH. ABH presented evidence to the MOH MCH working group on the use and storage of oxytocin, conducted sustained media advocacy (largely through TV and radio), and held multiple face to face meetings with the PS 23 of health. These activities resulted in a directive by the Permanent Secretary MoH to District Health Officers to ensure the use and storage of oxytocin in the UNEPI cold chain fridges. This will result in availability of oxytocin in many remote facilities. Working with a team of legal experts, ABH developed draft guidelines for the implementation of the Immunization Fund. In response to a request from the Permanent Secretary MoH, PATH supported the gazetting of the Immunization ACT, which clearly spells out the establishment of the Immunization Fund. The project will continue working with stakeholders to ensure its implementation. 4.3 HIV/AIDS The project seeks to support the attainment of the 90-90-90 targets, implementation of the Test and Treat Policy, and respond to emerging health issues. The project further seeks to build the capacity of selected national umbrella and network sub-awardees in advocacy. The project’s interventions under the HIV/AIDS thematic area largely focused on gaps relating to medicines and commodity security, followed by HRH issues and health financing. Participation in major events like commemoration of the World AIDS Day was targeted. Focus was largely on stock￾out of HIV/AIDS and TB drugs and other essential health commodities. In Year Three (3), the project seeks to address the limited domestic health financing, low levels of HRH and security for health commodities, including those for people living with HIV/AIDS. To this effect, the project seeks to influence specific policy formulation, modification or implementation. The project, through partner CSOs, participated in public planning processes from Sub-county to national levels and advanced HIV/AIDS drug issues. This was through various avenues including: DMC, DHC and TWG meetings. For instance, in Kalangala, Kamuli, Kabale, Mbale, Busia and Ibanda districts, stock outs of ARVs were reported by citizens in DMC meetings. The project supported gathering of related evidence by developing and updating evidence dossiers focusing on HIV drug issues. For instance, in year two, media tours were conducted to document HIV drug stock-out, while linkage FIGURE 22: THE PRESIDENT OF UGANDA VISITS THE STALL OF ABH SUBGRANTEE, NAFOPHANU DURING THE WORLD AIDS DAY IN KALANGALA DISTRICT 24 facilitators in Kaliro District tracked and revealed stock outs of ARVs in Bumanya HC IV in year three. The project strengthened and enhanced coalition and network building at district and national levels to address HIV/AIDS issues. For instance, ACODEV mobilized other CSOs in Kasese District to follow up on citizens’ concern of shortage of ARVs and agreed to petition the Speaker of Parliament. NAFOPHANU also engaged MoFEPD, MoH, the Speaker of Parliament and NMS to address ARV stock outs while AAG participated in reviewing the status of ARVs. Similar engagements were held at sub-county level through community advocacy forums. The project’s support to address HIV/AIDS issues also focused on HRH. This was intended to reduce the effects of HRH on HIV/AIDS service delivery. To this effect, the project supported generation of evidence to advance the advocacy agenda. For instance, media tours were conducted to document HRH issues relating to HIV/AIDS. The project contributed to planning for HIV/AIDS services through participation in public planning processes at Sub-county, district and national levels, as well as through community advocacy forums advancing HIV/AIDS issues. The key planning avenues included: DMC, DHC, and District HIV Committee (DHIVC), meetings. For instance, sub-grantees attended the DMC meetings in Bugiri, Ibanda and Isingiro districts and demanded that health workers to handle HIV/AIDS. The project further strengthened community groups’ planning for HIV/AIDS. The project held a one-hour talk show on NTV that focused on health financing specifically for HIV. The panelists on the talk show came from Advocacy for Better Health and Uganda AIDS Commission. The show was popularized through all social media platforms for NTV including. The project implemented advocacy initiatives geared at increasing financing for HIV/AIDS services. This was largely through the attendance of DMC, DHT, TWG meetings and budget conferences. This ensured that priority advocacy issues of citizens’ concern are incorporated into the district plans and the budget framework papers of FY 2017/18. For instance, in Year Three, the project mobilized CSOs to advocate against the 30% cut in the health sector budget, which had been stipulated in FY 2017/18 Budget Framework Paper. The Uganda AIDS Commission (UAC) convened a number of meetings to develop a road map for writing the Anti HIV Stigma and Discriminatory Policy. Through its partner, NAFOPHANU, ABH advocated for the writing of this policy, in which, UNAIDS has now picked interest, and is donating resources for a consultant to lead the policy development process. The MOH launched the consolidated Guidelines for HIV Prevention and Treatment in December 2016. The project has continued to work on socialization of the policy, guidelines and the provisions thereof, especially among PLHIV and key populations such as sex workers. FIGURE 23: TALK SHOW ON HIV/AIDS FINANCING ON NTV 25 ABH regularly shares stock status updates on HIV and TB commodities with the Ministry of Health through the project’s sub awardee, National Forum of People Living with HIV Networks in Uganda (NAFOPHANU). Additionally, 479 community groups in ABH’s 35 focus districts conduct monthly health facility assessments (HFAs) to generate advocacy evidence on health commodity status especially at health centers II and III. ABH, through NAFOPHANU, convenes quarterly meetings between CSOs and National Medical Stores to discuss health commodity/ drugs stock-out concerns and ways to ensure increased collaborative accountability. Through media engagement and advocacy efforts—including meetings with the Parliamentary HIV/AIDS committee, the Health Committee, the Rt. Hon. Speaker of Parliament, and CSO coalitions—the project was able to ensure retention of the health budget of FY 17/18 at Uganda shillings 1.8 Trillion, averting a 30% cut originally proposed by government. In preparation for, and anticipation of the NHIS, the project supported members of the Parliamentary Committee on Health to undertake a benchmarking trip to Rwanda. Upon their return, the committee members worked with the project and other CSOs to secure the long awaited Certificate of Financial Implication from MoFPED, which adds to the momentum towards establishment of the NHIS. 4.4 Malaria In Year Two, the project involved participation in commemorating the World Malaria Day to enhance visibility and share the advocacy agenda through audio-visual print materials or exhibition stalls. The key planned products included newspaper articles and flyers. In Year Three, the project planned to address domestic health financing, HRH, health commodities security relating to malaria; as well as responding to emerging health advocacy issues. To this, the project aimed to influence policy formulation, modification or implementation, including discussion of key impediments to the prevention and control of malaria, especially related to ACTs, RDTs, community uptake of IRS where relevant, and ITN use. The project further planned to enhance sub-grantees’ capacity to capture, analyze, translate and disseminate evidence. It also involved raising citizens’ awareness by using active two-way SMS platform, and enhancing community tracking and monitoring of commodity stock status for malaria drugs. Implementation of malaria-specific interventions was effected in Year Three, largely focusing on medicines and commodities security. This involved raising citizens’ awareness, especially working with CDFU which developed four radio spot messages on different aspects. For instance, the project developed posts about malaria drug stock-outs at NMS, generating interest and discussion from the followers. The project contributed to planning for malaria services through sub-grantees’ participation in various meetings. For instance, sub-grantees attended the budget conferences and raised concern about FIGURE 24: EVIDENCE FACT SHEET ON MALARIA POLICY ASKS FOR THE ABH PROJECT 26 stock-out of malaria RDTs in Sironko, Kyenjojo, Ibanda, Kiruhura, Kamwenge, Rukungiri, Kanungu, Kisoro, Sembabule, Kayunga, Luwero, Nakasongola, Iganga, Kaliro, Mayuge and Kamuli districts. The project also supported other emergent aspects that affected malaria including; on enabling gathering of additional evidence on various issues including malaria by developing and updating evidence dossiers. For instance, HEPS, a sub grantee, in collaboration with MoH generated evidence that informed advocacy for the implementation of the 2012 WHO malaria Test, Track and Treat policy and the Uganda Malaria Strategic plan (2014-2020). While advocacy to increase the budget for health also related to malaria, HRH and OVC aspects were not addressed. This contravenes the strategic direction of the project. 4.5 Orphans and Vulnerable Children (OVC) The project planned to elevate the plight of OVCs by advocating for necessary changes or implementation of related policies. It sought to enhance the quality of Universal Primary Education, reduce school dropout rates and teenage pregnancy, and address child labor and other forms of child abuse. To this effect, the project sought to support sub-grantees to participate in government planning and budget meetings and conferences to ensure that health and social services issues of citizens. This way, OVCs’ concerns would be voiced, prioritized and incorporated in district and national budget priorities. The general project direction was to incorporate health financing, commodity security and HRH in interventions targeting OVCs. In Year three (3), planned interventions included participation in District OVC Committees (DOVCCs) and Sub County OVC Committees (SOVCCs). The project mapped and identified critical knowledge gaps within the population on a number of health issues, including insufficient mechanisms to address OVC issues. The project supported CSOs to raise citizens’ awareness of their rights and responsibilities. This was done through three (3) radio messages focusing on the plight of OVCs and other channels. In Year Two (2), the project ensured the integration of OVCs’ concerns into district plans and budgets through sub-grantees’ participation in related meetings such as the District OVC Coordination Committees (DOVCC). Further support addressed high school dropout and absenteeism by children, especially OVCs in Universal Primary Education schools. For instance, in Mityana District, RACOBAO raised a need for an ordinance on the elimination of absenteeism and school dropout of children in UPE schools. The district council under minute 241/EDU/29/09/15 passed this ASK. In year three (3), the project also supported sub-grantees to conduct health facility assessments in districts to identify health service delivery gaps relating to OVCs. While the OVC program area was no longer funded in year three (3), related interventions were integrated in other focus areas. However, special forums to reach specific groups, including OVCs planned in year one (1) were not conducted. The meeting with stakeholders and parliamentarians to kick-start the development of guidelines for establishing the Immunization Fund was not held. This would have been the only intervention directly focusing on raising domestic funding for immunization as per the 2016 Immunization Act. 4.6 Tuberculosis (TB) The planned TB interventions largely focused on addressing stock-out of TB drugs, with a few directed to HRH gaps. However, no specific interventions targeted TB in Year One (1) and only awareness raising, focusing on four (4) priority issues including TB drug stock-out, was planned in Year Two (2). The project 27 involved the development and broadcast of radio talk shows, radio spots and drama series, and the production and dissemination of advocacy materials. In Year Three (3), the project seeks to address the limited domestic health financing, low levels of human resources for health and health commodities security relating to TB. The project also seeks to respond to emerging health issues, as well as generate and use evidence in advocacy. The project further seeks to raise community awareness and connect partners and key stakeholders using an interactive two-way SMS platform to address major TB issues. The implementation of TB interventions largely focused on averting drug stock-outs. In year two (2), the project facilitated engagements for policy advocacy regarding the TB drug stock outs. For instance, NAFOPHANU led engagements with MoFPED, MoH, the Speaker of Parliament and NMS, aimed at halting the stock-out of TB drugs. Conversely, the AAG, jointly with other USG IPs, MoH and NMS, reviewed the status of health commodities, including TB drugs. In year three (3), the project raised citizen’s awareness on various issues including TB drug stock using media platforms and interpersonal channels. For instance, 15 live call-in radio talk shows were hosted by 15 FM radio stations in commemoration of World TB Day. Similarly, four radio spot messages were developed and aired on 15 radio stations addressing TB drugs, among other concerns. The project supported gathering of evidence on TB for use in developing and updating relevant dossiers. For instance, Linkage Facilitators in Kaliro District tracked and revealed stock-out of anti TB drugs in Bumanya HC IV. The project supported building coalitions with other CSOs to handle TB issues. For instance, JIACOFE organized two coalition meetings in Mayuge and Kamuli districts focusing on integration of HIV/TB services across departments. National level support also included coalition building. For instance, CEHURD, in collaboration with other CSOs, held a meeting with the management of NMS focused on identifying challenges and discussing strategies for efficient stocking of the TB drugs and other essential life￾saving commodities. The project worked with other key TB players and oriented members of the Parliamentary Caucus on TB in related policy matters, with emphasis on gaps in the prevention and control of TB. However, focusing only on medicines for TB largely contravened the project’s direction of addressing two other aspects of health financing and HRH. ABH worked with Champions from Parliament to establish the Parliament TB Caucus, which will steer any future legislation on TB issues, and also to create a platform through which the profile of issues affecting TB services can be raised. The need for ensuring increased funding for TB was highlighted to government through different forums including at the national TB day (March 24th in Tororo) and during the wider debate on preventing the 30% budget cuts to the FY 17/18 health sector. An appropriation analysis of how much was ring fenced for TB commodities is yet to be done. The project through its sub awardee—NAFOPHANU, partnered with the Ministry of Health National TB, Leprosy Program to develop the Desk Guide for prevention and control of TB. The guide is at the stage of editing and finalization 4.7 Family Planning (FP) The project’s interventions on FP largely focused on increasing financial resources and to a lesser extent, the HRH issues. The project supported national level initiatives through CEHURD, as a member of the Civil Society Budget Advocacy Group (CSBAG), to advocate for integration of family planning issues in the national budget framework paper 2017 / 2018. The need to finance FP services was further affirmed by the participation of project staff and CSOs in analyzing the budget framework paper of the MoH. Emphasis of the project was on a need to allocate resources for recruitment, motivation and retention of critical health workers, specifically midwives. Evidence generation also involved refining through meetings with duty-bearers. For instance, RHU met with the DHO, the CAO and the LCV Chairperson in 28 Kabale to refine evidence on gaps affecting the delivery of FP and MCH services. Key among the issues fronted was a need to increase domestic health financing for FP services. The project supported sub￾grantees to generate evidence on health service delivery gaps relating to FP, through health facility assessments. Focus was on the availability of relevant health workers at the facilities, including the functionality of HUMCs. Evidence generated was packaged into reports, evidence dossier, position papers and factsheets, which were used in advocacy. Contrary to the project’s strategic direction, there was no focus on FP medicines and consumables. In an attempt to elicit more contribution to address FP issues, sub-grantee CSOs participated in planning and budgeting processes, especially at district level through DHMT and DMC meetings, and during budget conferences. For instance, sub-grantees asked for the prioritization of FP in the district plans and having related budgets during DMC meetings in Kanungu and Kamwenge districts. Sub-grantees also attended budget conferences. In Kumi, Mbale, Namutumba, Budaka and Mityana districts, they presented a need to introduce long lasting FP methods in some health facilities and asked for increase in budgets for FP. Sub-grantees also attended DHMT meetings in Bukwo, Kapchorwa and Kabale districts and pushed for skilling some health workers in new and long term FP methods. In partnership with other civil society activists under the Family Planning Budget Advocacy Group, ABH advocated for the prioritization of family planning in the FY 217/18 national health budget to the Ministry of Health and Parliamentary Committee on Health. In partnership with the Uganda Youth and Adolescent Health Forum, the project funded and participated in the consultations of young people on the current Adolescent Health Policy that is under review by MoH. In collaboration with the Ministry of Gender, Labor and Social Development, the project has conducted a sustained national campaign to end teenage pregnancy and child marriages. Despite ongoing advocacy efforts for the finalization of the School Health Policy, cabinet approval was deferred due to the ongoing review of the sexuality education framework by Ministry of Educations, Sports, Science and Technology. 4.8 Nutrition The project raised awareness on the use of media platforms and interpersonal channels, especially community group meetings and advocacy forums focusing stock-out of commodities. For instance, the project worked with CDFU to develop four radio spot messages on nutrition commodities. The project supported gathering of additional evidence on issues regarding nutrition and developing and updating evidence dossiers, especially on data use by the community groups and ABH staff. FIGURE 25: THE SPEAKER OF THE PARLIAMENT OF UGANDA AND THE PRIME MINISTER OF UGANDA LAUNCH AND SIGN OFF THE PARLIAMENTARY NUTRITION PLATFORM 29 FIGURE 26: WHO REPRESENTATIVE, DR. HAFISA KASULE (HOLDING MIC) LAUNCHING THE MANIFESTO The project contributed to planning for nutrition services through the participation of sub-grantees in various meetings focusing on HRH. For instance, sub grantees attended the DMC meetings in Bugiri, Ibanda and Isingiro districts and raised the issue of inadequate human resources to provide nutrition services at the health facilities. The project involved coalitions to promote delivery of nutrition services. For instance, it partnered with FANTA and Office of the Prime Minister to launch the Parliamentary Forum on Nutrition. It later facilitated orientation of the members of the Parliamentary Forum on Nutrition, district LCV Chairpersons, CAOs and District Planners in western Uganda on nutrition issues. the project also organized and coordinated a regional meeting of USAID implementing partners’ meeting in Mbale. This led to development of strategies for regular updates on health and social service issues and identification of areas of collaboration, including nutrition. ABH was at the fore of the establishment of the Nutrition Advocacy Platform, comprised of partners such as World Vision, FHI 360 (USAID FANTA, and CHC), JSI SPRING Project, UNICEF and indigenous CSOs. This platform meets quarterly to review progress on nutrition issues that require advocacy, so as to lay out plausible courses of action. Working with other partners implementing nutrition interventions, ABH held two high level policy dialogue meetings with members of Parliament and other stakeholders, as a way of mobilizing their support. ABH supported the establishment of the Parliamentary Forum on Nutrition (PFN), which was officially launched by the Rt. Hon Speaker of Parliament (Hon. Rebecca Kadaaga) and the Rt. Hon Prime Minister (Hon Dr. Ruhakana Rugunda). This forum will be leveraged to expedite any legislation related to nutrition. As way of ensuring that nutrition is prioritized in district plans and budgets, the project and collaborating partners (OPM, USAID-FANTA, World Vision) funded regional experience sharing meetings with district leaders in Western, Eastern and Central Uganda. Nearly 400 district leaders including Members of Parliament, CAOs, RDCs, District Planners, DHO and LC V Chairpersons attended these meetings. A number of commitments (including the establishment and functionalization of District Nutrition Coordinating Committees) were made during these regional meetings. The project will pursue the implementation of these commitments. Working with OPM, the project also ensured that a menu of all nutrition specific and sensitive indicators—disaggregated by sector—is generated and monitored during the Result Based Monitoring implemented by government. 4.9 Cross-cutting aspects These include: HRH absenteeism and late coming, lack of accommodation for health workers, non￾functionality of health units as raised in Bududa and Kisoro districts in Year Three (3) and non￾functionality of HUMCs as pointed out in Bugiri and Kalangala districts. Support was also provided to sub-grantees to focus their OACA action plans and reviewing progress covering various components and thematic areas. For instance, LADA addressed the HIV/AIDS workplace policy. Conversely, the project raised citizens’ awareness through dissemination of the Patients’ Charter during the World AIDS Day, radio and print media, including radio talk shows; and press conferences and press releases. The project in partnership with other like-minded CSOs developed and launched a citizens’ manifesto that was inter alia calling upon political aspirants to consider increasing financing for health sector as a 30 priority when elected into office. The launch held on 15th September 2015, was well attended by CSO representatives, the media, and UN agencies. There were cross-cutting aspects addressed that indirectly affected the seven (7) thematic areas. In a bid to generate national concern and conversation on the quality of health services in the country, Advocacy for Better Health worked with some members of Parliament led by Hon. Herbert Kinobere and Hon. Rosette Mutambi to establish the Uganda Parliamentary Forum on Quality Health Services Delivery. This forum, which was launched by the Rt. Hon Speaker of Parliament, who is also its patron, boasts a total of 215 members. The project will continue engaging with this forum and others established to ensure legislative movement on priority advocacy issues related to health. Advocacy for Better Health (ABH) also organized a national level dialogue to discuss the need for government of Uganda to absorb PEPFAR funded staff into the health system. From this dialogue, a multi-sectoral Task Force (comprised of the MoH, MoFPED, Ministry of Public Service, Ministry of Gender, Labor and Social Development, Health Service Commission, Public Service Commission, PEPFAR and USAID representatives, as well as representatives from the Private Not for Profit Medical Bureaus) was established. The Under Secretary of MoH chairs this task force, while PATH-ABH and IntraHealth- Strengthening Human Resources for Health were nominated to coordinate Secretariat responsibilities of this Task Force. The project also sponsored two sessions of the popular People’s Parliament TV show on NTV with a major focus on MCH and HIV issues. The show on MCH was held in Mityana district that has had episodes of unexplainable maternal deaths in the past. It attracted the participation of districts leaders (District Health Officer, District Chairperson, and Resident District Commissioner) who discussed the critical health sector challenges in the district. These ranged from poor state of maternity wards, to break down of referral system, limited community sensitization on nutrition, lack of ambulance, water shortage in health facilities, stock outs of essential drugs and other lifesaving commodities. ABH together with the Parliamentary Forum on Quality Health Services organised the 1st National Presidential Dialogue on Quality of Health Services in the country at Colline Hotel, Mukono. The dialogue was attended by a number of people close to 400 from civil society organisations, Government, Ministries, Parliament, Donor agencies, USAID and implementing partners. The President of Uganda who was expected to be the Chief Guest was represented by The Rt. Hon. 1st Deputy Prime Minister Gen. Moses Ali who read the President’s speech. This was a welcome effort to be able to discuss the challenges in the health sector. Gen. Moses Ali officially launched the dialogue to be a yearly event to take stock of what is happening in the health sector and what needs to be improved upon. 31 32 5.0 COLLABORATION, LEARNING AND ADAPTATION (CLA) APPROACH 5.1 Introduction The Collaboration, Learning and Adaptation (CLA) model is a powerful tool in fostering development effectiveness. Collaborating helps to identify areas of shared interest and enables working together. It helps to reduce duplication of efforts, and share knowledge across sectors and institutions. The project developed a CLA Agenda and planned collaboration with other USAID Implementing Partners through joint initiatives, performance review meetings and participation in other relevant coordination platforms. The project planned to provide a platform to enhance linkages and complement the efforts of different programs to pursue common objectives. The review revealed more availability of initiatives on collaboration and learning than adaptation. Related achievements are presented below: 5.2 Collaboration The project formed collaborations through sub-grantee CSOs and series of breakfast meetings with the Parliamentary Committee on Health were held to discuss government’s funding for health for the financial year (FY) 2015/16. As a result, CSOs working on health submitted issues for prioritization to the Parliamentary Committee on Health regarding wage bill enhancement for health workers, enhancement of Primary Health Care at district level and basic non-monetary needs for health workers such as education, capacity building and housing. The project also carried out national level policy advocacy engagements and formed the Advocacy Advisory Group (AAG) comprising of volunteer national and district-level decision-makers, CSOs, the media, private-sector and representatives of leading health and social service-delivery projects in Uganda. The platform has linked CSOs, ensured complementary efforts, and strengthened joint advocacy planning among related projects in Uganda. One key success of the project has been revamping coalition-building by CSOs, through various avenues such as meetings on MNCH and life-saving commodities. This resulted in key advocacy outcomes. Notable among them include: advocacy for increasing stocks for Oxytocin at health facilities. This prompted the MoH to recommend storage of Oxytocin in UNEP refrigerators. Some CSOs like CEHURD presented such issues in the media at no cost. The CSOs’ involvement in the government’s budgeting process, especially through technical working committees at national level, had improved. Joint advocacy activities with other CSOs both at district and national levels were initiated through collaborations. For instance, LADA, a CSO in the south western region led the formation of the SWANET, comprises of 31 like-minded CSOs in the region. This network is currently considered as a platform to champion health and social service advocacy issues affecting citizens, before the responsible duty bearers. Collaboration with other stakeholders was deep-rooted in the project’s design and implementation. In a bid to understand the operating landscape and identify opportunities for collaboration and synergies, the project team held meetings with several USAID-funded Implementing Partners. These include; Communication for Health Communities (CHC), Strengthening Decentralization for Sustainability (SDS), STRIDES for Family Health, STAR –South West, and Governance, Accountability and Performance Program (GAPP). These engagements provided opportunities for introducing the project and learning from ongoing projects’ experiences including sub-granting mechanisms, baseline assessments, developing monitoring and evaluation subsystems, communication strategies and stakeholder management. The project worked closely with stakeholders such as the USAID Monitoring, Evaluation and Learning Contract; SDS and Accountability Can Transform (ACT) Health project. This aided the 33 finalization of the project’s Results Framework. The project team worked with CHC and CDFU to develop and broadcast communication (mass media) messages about the development of the project’s communication strategy. During the implementation period, collaborations were made with different stakeholders to address specific needs. For instance, linkage with the Strengthening Human Resources for Health project provided evidence on the status of human resources for health and its impact on health service delivery. To help identify challenges that youth and OVCs face and how they would be tackled, the project collaborated with the Ministry of Gender, Labor and Social Development and OVC IPs including Strengthening the Ugandan National Response for Implementation of Services for Orphans and other Vulnerable Children (SUNRISE OVC) and Sustainable Comprehensive Responses (SCORE) for Vulnerable Children and their families. A partnership with the Uganda Stop TB campaign aided documentation of critical advocacy issues affecting TB prevention and control. The project also participated in the PEPFAR Implementing Partners’ Meeting, leading to sharing of experiences and lessons in working with key populations. 5.3 Learning The project was involved in various learning platforms and events including; regional reflection meetings through which sub-grantees’ progress and performance are reviewed; Grants Management Collaboratives (GMCs) held at regional level to enhance capacity development; and e-newsletters. The project further promoted learning through; documentation of lessons learnt by video new learnings of successes and best advocacy practices; annual performance review meetings; attendance of relevant national and international subject matter conferences; participation in commemoration of the international and national days; relevant national and international subject matter conferences; and documentation and sharing for project staff through papers, abstracts and presentations at national and international conferences. These platforms and events enabled to the project to enhance; CSOs’ continuous reflection and improvements; sharing successes across board; informed work planning annually; and improved the project’s outlook. The project conducted Year One and Two Organizational and Advocacy Capacity Assessments (OACA) to measure progress on the capacity development plans. The reports reveal that out of the 19 sub￾awardees, 17 demonstrated improvements in their overall scores, while one maintained the same score, and another had a drop in the overall score. Even for sub-awardees that improved in the overall scores, there was a decline in some of the sub-section scores. The improvements are attributed to the support received from the project, and commitment of the sub awardees to implement their action plans. The project included CSOs’ quarterly self-review and update of OACA action plans though not fully adhered to. This was strengthened through the integrated support supervision that enabled CSOs to receive support from the project team regarding the review and update OACA plans. The review process enabled ascertainment of the status for each CSO’s implementation of learning actions and required support to address remaining actions. Project staff further, participated in Implementing Partners’ meetings to share experience and lessons in working with key populations. The project produced and disseminated four editions of the Advocacy Alerts! Newsletter9 . 9 The Advocacy Alerts! Newsletter can be accessed on the following link https://path.box.com/s/4fkrz7flwf30fyoom4qrzqj7wxyzynns 34 This facilitated documentation and sharing of information among staff, partners and donors. These presented simplified information about major events, meetings and other advocacy milestones that the project had been accomplished. These were helpful as they kept stakeholders abreast up to date. In order to share with and learn from other stakeholders at international level, the project supported some staff to attend and presented at international conferences. Some of the conferences attended include; the American Evaluation Association Conference (AEA), 2016 held in Atlanta GA, USA; and the Pan African Social Accountability Learning Lab held in Manzini Swaziland; and the National Health Insurance Scheme benchmarking in Rwanda; and the Annual Health and Humanitarian Logistics (HHL) Conference in Copenhagen, Denmark 2016; these enabled the staff to get acquainted with new areas including; information Systems for supply chain management; cutting-edge technology in warehousing, logistics and supply chain management. Other key learning events where the ABH team shared and learnt from include the American Evaluation Association Conference (AEA), 2016 in Atlanta GA, USA; and the Health Systems Research conference in Vancouver. This information is helpful in shaping advocacy. 5.4 Adaptation The project devised means of adapting good practices learnt from various platforms and events. The GMCs involved the Plan Do Study Act” (PDSA) process that included the adaptation of good practices. For instance, during the GMC held in December 2016, sub grantees expressed gaps in ensuring effective use of evidence in advocacy while others shared their related successes. Subsequent GMCs also focused on improving evidence use in advocacy. For instance, data collection through HFAs and its use in community and district advocacy forums has been adopted by all sub grantees and community groups. Nonetheless, the adaptation from learning platforms and events was low. 35 6.0 CRITICAL EVALUATION ASPECTS 6.1 Efficiency The MTE applied measures of efficiency addressing waste in the process; at the level of inputs - cost reduction approaches; and at process level, focusing on; timeliness of activities and processes leading to quality enhancement. 6.1.1 Cost reduction approaches i. There was poor understanding of the FAA mechanism at the beginning, leading to omission of some activities from CSO budgets, and challenges in implementation as described by CSO’s: The challenge with ABH there, is the mode of funding. […] Because at times, as an organization we don’t have funds. And uh, ABH wants you to keep on running the program but you find you are constrained and don’t have money and they are saying, you work, and we shall reimburse after you have worked. So for many organizations, ours inclusive, you’ll find it is challenging. - Executive Director, District-level CSO, July 2017 However, over time some CSO’s found the FAA mechanism provided cost savings due to its flexibility with a focus on milestones and not the details of expenses involved in the process, which enabled meeting of outputs at low cost. Staff recruited by CSOs were an additional resource for other projects. This ultimately led to cost savings as noted: “… you save on some costs because they will not ask you for instance how you reached the ministry. What they want is the milestone. There is a lot of flexibility, I am able use these Project Officers on other activities link” (National level CSO stakeholder interview, August 2017). ii. Use of milestones and their retirement using means of verification (MoV), minimizes the time that staff would take to review accountability reports and supporting documents. Only MoVs documents were required without the usual documents required for liquidating partner advances. iii. The findings revealed that some project meetings and trainings were conducted within the PATH or sub grantee premises. This saved funds that would otherwise have been spent in other venues though such may compromise quality if not well planned. To elucidate this, one participant noted: “We said restored our coalition meetings. Civil society come together to discuss maternal and child health. They sit here in this board room”. (National level CSO stakeholder interview, August 2017). iv. The design of the project incorporated cost saving mechanisms especially its dependency largely on CSOs and community groups that integrate advocacy initiatives into their routine activities. Some trainings of community group members and Champions on some thematic areas were conducted by CSOs using their own resources under other projects as noted: 36 “We trained community groups and Champions on some areas not yet covered by ABH funds. We could not wait since there was a gap yet they needed the knowledge, for instance on reproductive health matters” (CSO staff). v. The use of coalitions was a major approach and key to advocacy initiatives. The project’s linkage with other organizations facilitated joint contribution towards the costs of key outputs. For instance, through joint funding under the Nutrition Advocacy Platform, the project championed improvement of nutrition in Uganda and co-led the launch of the PFN, graced by the Rt. Hon Speaker of Parliament and the Rt. Hon Prime Minister. vi. The utilization of network structures facilitated some activities to be carried out without use of ABH project funds. This largely related to acquisition of data for evidence based advocacy. While there was no formal arrangement to enable national level CSOs to get evidence from district level CSOs, NAFOPHANU used her members in the districts to get the required evidence. One CSO staff emphasized this as stated: “NAFOPHANU has been using her members to get evidence from across districts because it is easier for it. Other national level implementers like us finding it costly to reach there and get evidence”. (National level CSO stakeholder interview). vii. Building the capacity of CSOs largely relying on project staff was a good cost saving approach. While some consultants were contracted to support some interventions, they were significantly few. 6.1.2 Time saving approaches The project design involved different elements that facilitated time saving during implementation. Notable among the time saving elements and deterrents are as follows: i. The establishment and use of regional offices helped provide timely support to CSOs without technical teams from the national office. One CSO staff noted about support from regional offices: “We work together with them. They come and support us conduct some activities especially at district level and also when we have problems to sort out” - National level CSO stakeholder interview, August 2017. ii. The Integrated Support Supervision approach created an opportunity for comprehensive support from more than one technical area as well as strengthening of their advocacy activities during the same period. 6.2 Effectiveness Community Empowerment The project worked with community groups to empower citizens on their rights and responsibilities related to health and social services. The communities through the groups demanded for policy based changes from decision-makers especially at sub county and district levels, and held them accountable for the quality of health services. The avenues leveraged for this interface include; community dialogues, the use of print and electronic media, and through edutainment. The Patients’ Charter formed the 37 foundation for the messages on rights and responsibilities. This attainment was partly a result of support and follow up from the sub grantees. Advocacy The project handled advocacy efforts on four cross-cutting priorities: health commodity security, human resources for health (HRH), domestic financing, and orphans and vulnerable children. CSOs advocated from the facility to the highest levels of decision-making. On the other hand, community members were engaged in advocacy at community level though in a few communities. CSOs’ advocacy agenda included engagement of government authorities in various ways leading to some actions on health and social service issues and policy gaps and focus was put on supporting and encouraging CSOs to collect especially through health facility assessments and its use in advocacy initiatives. CSOs’ attendance of government planning meetings at national and district levels was high, which was a major avenue for conveying citizens’ concerns. However, some of the CSOs focused on achieving milestones to enable them recover funds used and missed out on other actions including joint advocacy. While there was good representation of citizen’s concerns by CSOs at government meetings, most government authorities were still less supportive of advocacy to service delivery initiatives; leading to low incorporation of citizen’s concerns in district plans. Nonetheless, other local governments had limited resource to cater for the citizens’ issues raised. Capacity building The capacity of sub awardees to conduct advocacy greatly improved, as exhibited by various aspects including; their ability to develop advocacy strategies, collection of evidence and use it to engage decision makers, which has resulted in duty bearers addressing some key issues affecting health service delivery. Sub grantees were trained in organizational areas including; strategic planning and development of systems and processes to enhance documentation, human resource management, board oversight, and resource mobilization. Some sub grantees noted that the skills gained from the resource mobilization training and follow up support enabled them to bid on, and win new grants. The training of sub grantees on advocacy enhanced their ability to continue improving on various advocacy fronts as reflected from results of Capacity assessments carried out including the assessment carried out as part of the MTE that showed meeting of all targets. 6.3 Project sustainability The project design has a key result area on institutional capacity development. This was envisaged to be the pivot for sustainability. The capacity development structure was designed to move selected partners through assessment, improvement and review relative to identified standards, with eventual independence from the program as the ultimate goal. The project team was also mandated to create a prospective exit strategy in collaboration with each partner organization within its first year of support. Over the course of the program, besides repeated assessments, technical support approaches targeted to each specific CSO were catered for to build technical and organizational capacity for dynamic development and transformation. These ought to include: e-learning curricula; workshops; training of trainers; short-term consultancies; mentorships; and peer-to-peer learning. The final phase of program support was envisaged to focus assessment and assistance on ensuring that CSOs demonstrate that the various components of sustainability-related strength are in place. The project was also designed to support CSOs to attain knowledge, access and eligibility to an array of financial support sources for policy advocacy. 38 The mid-term evaluation incorporated key questions to assess achievement of the milestones demonstrating full program sustainability, including: 1) CSOs are capable of achieving “acceptable” NUPAS scores; 2) CSOs demonstrating ability to source additional funding; 3) CSOs demonstrate ability to plan and implement advocacy work; and 4) CSOs routinely initiate, execute, and evaluate advocacy work. Tentative process assessment shows that key steps have been taken along the sustainability framework especially considered strengthening of existing community and government institutions. The project has worked with community groups in target districts and mandated them to steer advocacy in their communities. The project has trained community groups in various components of advocacy, enabling them to get skills that will be utilized beyond the project lifetime. The involvement of community groups in government planning processes has been key in the project. The participation of CSOs in public sector planning processes has improved with 80% of them now actively involved, ahead of the 60% mid-term target. The momentum built may continue beyond the project. The project expects to “graduate” at least four CSO partners with the systems and skills to qualify as prime recipients for future USAID funding. It is also expected that there will be additional networks or umbrella organizations and CSOs that possess the organizational and advocacy capacity, experience, and expertise to develop and implement programs to effectively represent community interests and advocate for health and social policy change. The project envisaged that the CSOs and umbrella organizations would be able to strengthen the capacity of their members, partners, and associated community groups to actively engage in joint advocacy and accountability activities. The assessment reveals good progress in capacity development towards these outcomes. Training of sub grantees on resource mobilization equipped them with key skills that they use to access more funding. Following the resource mobilization training conducted in year one (1), sub-awardees started developing and or refining their resource mobilization strategies. By the end of the year over 50% of the CSOs had completed their strategies. These include; ARUWE, STF, ACODEV, NAFOPHANU, HEPS, MAFOC, LADA, CEHURD and Family Life Education Program (FLEP). Sub awardees like Literacy Action Development Agency (LADA), ACODEV and FLEP won new projects, an achievement they partly attribute to skills gained in fundraising and capacity during the first two (2) years of the project. The support and encouragement to network and establish joint advocacy initiatives under the project had already lead to formation of network organizations. For instance, SWANET was established by LADA and comprises 31 CSOs in south-western Uganda. It has already carried out district level joint advocacy initiatives. The network is currently viewed as a platform to champion health and social service advocacy issues affecting citizens. While details of SWANET’s operations are not yet available, it is a good step in sustaining advocacy initiatives. However, some key ingredients of the sustainability approach have not been adequately handled. The project has worked with community groups in target districts and mandated them to steer advocacy in their communities. The project was envisaged to enhance the sustainability of community and government engagement functions and mechanisms by pursuing policies and institutional frameworks that require community participation in policy and planning processes. Such policies not been seen and institutional frameworks still require more clarity and formalization. 39 The project planned to use key players such as the AAG, champions’ networks, and advocacy fellowships to bring multiple stakeholders together to build system-wide commitment for advocacy and accountability, leverage resources and foster lasting working relationships. The sustainability of advocacy programs also needs to be enhanced by the formation of advocacy communities of practice and the Grants Management Collaborative. These forums would provide a platform for shared learning about challenges, successful practices and results arising from implementation of advocacy and organizational development strategies and activities. While AAG and champions are already part of the project, fellowships have not been recruited. Furthermore, the sharing of information between AAG and champions cannot easily be traced. This reduces the usefulness of the otherwise good approach. 40 6.4 Relevance of the project 6.4.1 Alignment with USAID Uganda’s CDCS 2016-2021 USAID/Uganda’s 2017/2021 Country Development Cooperation Strategy (CDCS 2.0) is vital in aligning UASID’s supported interventions to achieve the Uganda’s Development Cooperation Strategy. Uganda’s National Development goal is to become a prosperous, stable and democratic society and the Government of Uganda’s (GOU) “Vision 2040”. USAID Uganda’s 2017/2021 CDCS 2.0 focuses on: Health systems strengthening and epidemic control, increased literacy, agricultural sector development, enhanced food security and reduced household and community vulnerability to shocks. It also seeks to promote biodiversity and natural resource management, climate change adaptation and inclusive and accountable democratic governance. The project’s goal is to improve quality, availability and accessibility of health and social services. This is aligned to USAID Uganda CDCS’ Development Objectives One (1) and Two (2). The project’s Result Area one (1) promotes citizens’ demand for improved quality of services, focusing on increased citizens’ awareness of their rights and responsibilities. It also focuses on ensuring that community groups advance priorities for improved health and social services, and improved engagement between citizens and duty bearers. These are largely in line with CDCS’ focus on inclusive and accountable democratic governance. The project’s Result Area Two (2) focuses on enabling CSOs to effectively advocate for issues of citizens’ concern in health and social sectors through increased utilization of evidence to inform advocacy. It also involved their effective participation in local government planning, monitoring and accountability of health and social services. This was done through enhancing co-ordination and collaboration, which are also key pillars of CDCS’ CLA approach. Result Area Three (3) promotes strengthening of institutional capacity of CSOs, which are clearly stipulated in USAID Uganda’s CDCS provisions on Health systems strengthening. 6.4.2 Alignment with national and international policies and frameworks The ABH project is well-aligned to various Government of Uganda (GoU) policies and frameworks on FP, SRH, MCH; malaria, nutrition, TB, HIV/AIDS and OVCs. Interventions under each of these components have been designed with due regard to the relevant policies. Some of the key initiatives of the ABH project that particularly conform to the GoU’s policies and strategies are: increasing the involvement of local communities; improving the governance and accountability of districts, improving accountability by duty bearers through service provision, and improving accessibility and use of services. The ABH project design and implementation strongly support the GoU’s initiatives, in especially health. 41 7.0 ACTIVITY IMPLEMENTATION IR 1.1: Increased citizens’ awareness of rights and responsibilities related to health and social services In order to empower citizens, the project planned to employ a two-pronged approach to community empowerment and engagement. First, the project work supported CSOs to identify and strengthen their own community mobilization efforts, focusing on more inclusive engagement of marginalized groups and effective action planning that can be leveraged for advocacy efforts. Second, it further supported CSOs to work directly with community groups to ensure that they (i) are interested in advocating for their communities’ health, (ii) have the necessary information on their rights and responsibilities and key health and social issues and policies in order to advocate, (iii) develop effective action plans for influencing relevant planning and budgeting processes, and (iv) to participate in key platforms to ensure their voices are heard when decisions affecting their lives are being made. Mapping of assets and resources: The project started with community mapping of existing assets for community empowerment and advocacy at national level and within the 35 project districts. Mapping aided the identification of existing assets and resources for community empowerment, such as Village Health Teams (VHTs) and communication channels that national- level CSOs use to mobilize communities for health and social services-related advocacy. Mapping further revealed vital information, including: the key role of VHTs in the dissemination of health information, limited feedback mechanisms and the publication of IEC materials in English, yet many Ugandans do not understand it. These assets and resources supported planning for project interventions. Nonetheless, VHTs have not been adequately utilized yet they are vital in raising community awareness. Development / customizing of the communication strategy: Realizing the vital value of evidence, a formative study was conducted in three (3) districts of Sembabule, Kiruhura and Mityana to inform the development of a project’s communication strategy. Developed in 2017, the communication strategy guided all communication interventions and was later customized by all the 20 sub-grantee CSOs to suit their contexts and communication needs. This led to standardization and enhanced the quality of advocacy. However, the formative study was conducted only in districts from central and south western regions and could have missed salient issues specific to some communities in eastern Uganda. Raising citizens’ awareness on health rights and responsibilities: Traditional platforms such as barazas (citizen engagement meetings), Munnomukabi groups (Self Help groups), PLHIV support group network meetings; and saving and credit cooperative (SACCO) meetings were applied to raise citizens’ awareness. For instance, 105 community advocacy forums including community dialogues, interface meetings and awareness sessions were conducted across the 35 districts in the first two (2) years. Drama was embraced as an edutainment approach to create awareness on rights and responsibilities. This aided modeling desired behaviors. Two sub￾grantees; RHU and ARUWE, conducted 12 drama shows focusing on the Patients’ Charter. These initiatives contributed to the 50.9% of citizens who knew their health rights and 62.4% who knew their health responsibilities. However, 205 community advocacy forums and 1,188 drama shows 42 were not implemented in the same period. These would have provided more avenues for community members to share their concerns to duty bearers and decision makers. Partnership and engagement with the media: The project engaged media houses in various ways including; broadcasts and placements of advocacy messages on radio, TV and in newspapers. For instance, in the first two (2) years, sub grantees conducted 284 live call-in radio talk shows on local FM radio stations, raising citizens’ awareness on health rights and responsibilities. Further, 24 episodes of a radio serial drama were broadcast under the popular award winning Rock-Point 256, aired on 22 local FM stations across the 35 districts. In order to focus the radio talk shows, the Patient’s Charter was used as source of information. The project considered the different languages in the project districts. Accordingly, a total of 8,280 radio spots were broadcast in English and five (5) local languages including Runyankole - Rukiga, Luganda, Lukhonzo, Ateso and Kupsabiny on three (3) national and eight (8) district-based radio stations. Overall, a total of 8,844 radio advocacy engagements including talk shows, serial dramas and spot messages were aired compared to only 550 that were planned. The focus was on the four priority advocacy issues including; Orphans and vulnerable children, health commodities security, human resources for health and domestic health financing. These engagements were on all the seven thematic areas of the project. These media engagements increased citizens’ awareness on their health rights and responsibilities as well as need for action from duty bearers as noted under progress of result area one. The project further reached citizens through a series of 24 advocacy strip messages in Uganda’s two leading dailies (New Vision and Daily Monitor). These enabled citizens to get information on drug stock￾outs, health worker absenteeism, school drop-outs and citizens’ rights and responsibilities. Some sub￾grantees published a number of articles and advertorials in the same newspapers. This did not only increase citizens’ awareness but also the project’s visibility and the priority advocacy issues. For instance, RHU, a sub grantee, was behind the captivating article that hit headlines in the media talking about the high rates of HIV infection among commercial sex workers in Kabale and Kisoro. The article was published in the New Vision following the media tour organized by RHU in Kabale, targeting MARPs. The project harnessed the power of the social media and other communication channels to target a certain population. To this end, Communication for Development Foundation Uganda (CDFU), a sub grantee, developed a social media and other communications strategy. This enabled ccitizens to write letters and send SMS to; and call their representatives expressing their concerns on issues that affect health and social service delivery. The exercise was key in bridging the gaps between citizens and their leaders. Conducting forum theatre (interactive drama) performances: The project raised citizens’ awareness of rights and responsibilities related to health and social services through 347 functional advocacy forums, conducted in different sub-counties. Specifically, Drama (forum theatre) was embraced by some of the sub-grantees as an educate-entertainment approach that created awareness on health rights and responsibilities and other advocacy issues through modeling desired behaviors using characters in the drama. These provided communities an avenue to ask and get response from duty bearers. The engagement of children in forums to promote their understanding of rights and responsibilities and use them as advocacy agents in their families and communities was not measured and effective not ascertained in the evaluation. 43 Development and dissemination of tools and guidelines: The project developed and disseminated tools and guidelines including the Patient’s Charter, which was translated into different local languages and comic books. The project translated, printed and disseminated 88,650 copies of the Patients’ Charter in eight local languages, namely; Luganda, Runyakitara, Lukhonzo, Lumasaba, Rufumbira, Kupsabiny, Lusoga and Ateso. These materials were the centerpiece for educating citizens about their rights and responsibilities and were distributed largely at HFs. However, while children’s forums and child parliaments play a critical role in promoting children’s participation and empowering them on issues of health rights, these creative approaches were not applied. Orientation of community groups and advocacy champions: A total of 426 community groups were oriented and facilitated to identify advocacy issues and conduct community empowerment activities in the 35 districts. This enabled community groups to appreciate and learn how to mainstream advocacy in their activities. Up to 270 media champions, leaders of Community Based Organizations (CBOs), and Community Awareness Facilitators (CAFs), among others, were also trained to appreciate and mainstream advocacy. Media and Advocacy Champions were key in mobilizing the community groups and following up issues. However, most community groups still found it hard to mainstream advocacy in their routine activities, expressing their inability to perform well with limited funding. Support to community groups to advance advocacy agendas: The project, through sub-grantees, supported 426 community groups to identify advocacy issues and develop action plans. These were used to raise issues and seek action by decision makers in the 35 districts. The project exceeded the mid-term target of 310 community groups. The project worked with Community Process Facilitators (CPFs) who were earlier on trained to support community groups to improve and implement their action plans. The training covered several aspects including guidance on how to contact duty bearers and basic evidence gathering to inform their next steps. For instance, HEPS supported 17 community-based organizations to develop action plans on the advocacy issues that they prioritized as a pertinent to the communities. These initiatives kept the mid-term target on course, with 47.0% of community groups whose action plans advanced to the implementation phase, closer to the 48.0% midterm target. Local leaders are confident that the activities have helped to increase citizens’ engagement with duty bearers, as noted by one stakeholder: I think to me, the biggest success which I am seeing is citizens getting confidence in interfacing with duty bearers and telling them exactly what is affecting them and coming out with action of this is what we want. - Executive Director, District-level CSO, July 2017 Use of the community scorecard for social accountability: The project developed a “user-friendly version” of the community scorecard, which was piloted by KADINGO in seven (7) fishing communities in Kalangala District. The scorecard was not widely used; it was replaced by the Health Facility Tool that was viewed as to be more appropriate for community empowerment process and for collecting evidence for use in advocacy. Whereas change of tools can be good in project management, the short time taken to abandon the scorecard points to a gap in planning. While the project had achievements relating to increased citizens’ awareness of their health rights and responsibilities, there was no evidence of implementation of some planned interventions, especially in year two (2). These include: bi-annual meetings with media advocacy champions at regional level to discuss emerging advocacy issues and required media advocacy techniques, and meetings with editors and media champions to re-orient them on priority advocacy issues and ask them to devote more 44 coverage to them. Other planned but unimplemented activities include; partnerships with 200 schools to engage children in advocacy forums, and organizing forum theatre (interactive drama) performances and none was conducted in year three (3). Inability to implement the above interventions may have slowed the project’s progress. Result Area 2: CSOs effectively advocate for and represent communities on policies / issues of citizens concern in the health and social sectors. Result Area two (2) was designed to empower sub-grantees to capture citizens’ concerns related to health and social services, gather related evidence and use it to engage duty-bearers to take action. The actions expected from duty-bearers included: policy formulation and implementation, allocation of budgets and establishing programs to address citizens’ concerns for policy change, where necessary. The evaluation reveals largely high effectiveness of interventions and good progress as reflected below. Intermediate Result 2.1: Increased utilization of evidence by CSOs to inform advocacy The project supported the use of data in enhancing advocacy initiatives at both district and national levels. Sub-grantees were supported to enrich their advocacy initiatives with field data, which became vital in influencing action by decision makers and duty bearers. In order to understand citizens’ priorities, the project staff at different levels collected data and analyzed it. For quality data, community assessments and action plans were applied. Health Facility Assessments and policy mapping were conducted to identify policies, guidance and financial documents governing district and national service delivery systems and their shortcomings. Stakeholder mapping was conducted, leading to the development of matrices for identifying relevant decision-makers, influencers, interest groups, coalitions or networks, and other CSOs at various levels. The matrices included: specific interests; ability to affect the political environment and relationships with one another. This facilitated the advocacy process. IR2.2: Development and implementation of comprehensive advocacy strategies that address health and social service delivery issues identified by communities. Under this result area, the strategy used involved PATH’s ten-part process which involves technical assistance through an e-learning course, a three-day workshop using PATH’s Policy Advocacy Strategy Development Curriculum, and accompanying tools. There was also peer-to￾peer mentoring, modeling opportunities, and study visits regarding implementation of specific aspects of an advocacy strategy. Training of trainers was carried out for CSO representatives, with particular emphasis on networks and umbrella CSOs. This built a cadre of skilled facilitators who were resourceful during strategy development workshops and provided technical support within their organizations. 45 IR2.3: Improved advocacy networking, coordination and collaboration among like-minded CSOs The project identified and pursued opportunities to bring CSOs together to promote alignment of advocacy strategies, plans and resources. This included use of forums, networks and communities of practice. The forums enhanced the collaboration of CSOs with other stakeholders to strengthen District NGO Forums to support coordination among NGOs. Within the forums, CSOs emphasized advocacy for operationalization of national policies and plans especially at district and national levels. The project supported networks or umbrella organizations, bringing members together for advocacy strategy development workshops to align or develop joint advocacy strategies. Some sub-grantee CSOs led joint advocacy initiatives in their respective districts. However, apart from participation in commemorative days, the planned activism days or advocacy weeks were not organized. This limited options for advocating for specific aspects where joint action would have been applied. IR2.4: Improved enabling environment for CSO advocacy. In order to provide CSOs with opportunities to access decision-makers and participate in district-level planning, the project sought to improve the enabling environment for advocacy. This was through meetings with Desk Officers – who present CSOs’ issues to their wider sector teams and then up to the DHMT, CAO, Resident District Commissioner, and the Local Council V Chairperson. The project enhanced collaboration with professional associations and regulatory bodies such as the National Medical Council and the Nursing Council, and Nurses Union. However, even when the project planned to use the strategy of engaging national champions including MPs, ministry officials, leaders of professional associations, the media, private-sector and faith-based organization leaders, this has yet to be fully effected. While the above attempts were initiated, CSOs still faced challenges in conveying the nature of advocacy work to district officials: Our biggest challenges in improving service delivery are the duty bearers. For instance, we make noise about the health facility, then we are looked to be inciting people. District level CSO stakeholder interview, July 2017 3.2.3 Result Area 3: Institutional Capacity of CSOs strengthened Result Area 3 aimed at strengthening the capacity of partner CSOs, focusing on both organizational and technical to effectively manage and implement their advocacy interventions. The organizational capacity component included Governance, Administration, Financial Management, Human Resources Management, Organizational management and Project Performance Management aspects; while the technical capacity focused on skills for carrying out effective advocacy. The criteria for selecting sub grantees was set up and used to the 20 sub-grantees CSO, conducting of Organizational and Advocacy Capacity Assessments (OACA) for each partner, and preparation of improvement plans for each CSO. These achievements were instrumental in project implementation. The project provided support to CSOs by offering targeted technical support to address specific needs. Such support largely addressed key areas including human resources management, supportive supervision, strategic planning and governance. Other key trainings focused on advocacy strategy development, resource mobilization, fixed amount awards and understanding USG compliance requirements. The specific achievements are highlighted below: 46 Further, the twenty (20) CSO sub-grantees were facilitated in OACA and supported to develop capacity building plans with guided technical support. Nine (9) out of the twenty (20) organizations including KACSOA, LADA, FLEP, KADINGO, ARUWE, CIDI, RACOBAO, IDO were supported to develop strategic plans (one completed and three are in the process). The ones for JIACOFE, HEPS and MUCOBADI were reviewed and guidance provided. This facilitated the review of organizational vision, mission and structures. This also helped them to understand their key stakeholders and how to engage them for better advocacy and overall organizational growth. Furthermore, nine (9) of the organizations trained in support supervision including; STF, MUCOBADI, JIACOFE, MAFOC, NAFOPHANU, KACSOA, HEPS, LADA and IDO were also supported to develop supervision plans, checklists, and Standard Operation Procedures (SOP)for the different activities. Seven (7) Advocacy Officers from sub-grantee CSOs and 10 ABH technical team members were trained in Policy Communication. Table 2: Specific support to CSO sub grantees No. Activity Number of sub grantees 1 Selected 20 2 Supported to develop strategic plans 9 3 Trained in support supervision 9 4 Trained in Policy Communication 7 Given the vital role of Boards of Directors, nine (9) organizations including; KADINGO, KACSOA, MAFOC, JIACOFE, LADA, IDO, ACODEV, CIDI and ARUWE, were supported to strengthen their governance structure and review the terms of reference for their boards. This enabled the Boards to have better appreciation of their roles, advocacy and how they can champion it within their organizations. Five sub￾grantees were supported to strengthen their governance systems through orientation of their board and management teams. Organizations have developed and (or) refined their board terms of reference and the board performance evaluation process. The training equipped board members to appreciate their roles and support organizations to achieve their project targets, a key ingredient in the success of the project. “Our board was not meeting as frequently as it ought to have been doing it. But now, with their help and insight we have a full board which was expanded from 5people to 9 people and it is meeting frequently and looking at governance issues”. - District level CSO stakeholder interview, July 2017 The same 20 organisations were guided to review and refine their human resources policies to ensure better human resource management and compliance with both national and USG requirements. The project provided technical support to twenty (20) organizations to review and refine their policy documents and guidelines including Human Resources (HR) manuals, Finance and Administrative manuals, and procurement procedures. This has streamlined processes for the sub-awardees and promoted compliance with both national and USG requirements. Whereas this activity was not planned, it was helpful in the implementation of the project and general organization development. Support was also extended to eight (8) CSOs to finalize their resource mobilization strategies in the first two years. Some sub-awardees raised additional funding using skills gained from training. This intervention was also not planned for but may have contributed to the CSOs’ success in raising additional funding. Three (3) Grants Management Collaborative (GMC) were conducted, providing an 47 opportunity for 20 CSOs to share their experiences and approaches and learn from one other. Nonetheless, only one (1) of the two (2) GMCs planned in the first two (2) years was conducted. Guidance was also provided on developing policies, motivation, retention, performance management and supervision. While there was no clarity on the coverage of supportive supervision as a component, the training availed partner CSOs with skills and knowledge of better management of their staff. The training in resource mobilization was a good one. We came up with a resource mobilization strategy and we are seeing changes in terms of resources. We have mobilized resources moving from 700m to 1.7billion annually. - District level CSO stakeholder interview, July 2017. 48 8.0 APPROPRIATENESS OF THE PROJECT STRATEGY CSOs’ partnerships: CSOs’ partnership with other USAID-funded programs to support their communities in advocacy has been weak. CSOs’ engagement with existing health activists, such as the 800+ Popular Opinion Leaders trained under the USAID AFFORD health marketing project, was not evident. Important to note is that grassroots collectives, particularly women, young people, PWD, and MARPs, have not been a key focus in implementation, yet it was planned to address existing gaps. This limited the participation of such key groups and missed out on their experience in furthering community interventions. The use of one community group to cover the whole sub-county was found to be ineffective. Most community groups focused largely on parishes where their offices were located. As a result, many people in the target communities were not reached. Dedicated activism: Apart from participation in commemorative days, planned dedicated activism days and (or) advocacy weeks were not organized. This limited CSOs’ options for joint advocacy action on specific matters. Although the project planned to engage national champions including ministerial officials, leaders of professional associations, media, the private-sector and religious leaders, a few of these had been involved in this role. Adaptive initiatives: The project incorporated initiatives for operations research to enable the identification and sharing of evidence-based innovative practices and tools to be adapted within and across projects. However, this was not applied yet it could have spurred internal improvements and enable other organizations to learn from the project’s successes. The project design incorporated learning collaboratives focusing on Grants Management Collaboratives. Three (3) sets, held at regional level were conducted and provided opportunity for new learning, use of data and strengthening CSO and staff practices. These collaboratives are key to promoting continuous learning among partners. Capacity building: The project design also catered for the application of 18-month Fellowships focused on advocacy and organizational development. Fellows were meant to be recruited from within the CSOs. The recruitment of Fellows was overly delayed due to legal issues concerning their facilitation. Ten (10) Fellows and ten (10) Interns were due to begin at the start of year four (4). Instead of the initial 18- month fellowship period, it was reduced to 12 months. This reduction in the fellowship period limits the magnitude of skills and knowledge that Fellows will acquire. The long delay to start the program denied the Fellows opportunity to gain knowledge and skills in advocacy and organizational development. This, subsequently, denied opportunity to CSOs to build extra human resources for future project interventions. The project did not implement two (2) other key planned activities including; a refresher training on advocacy strategy development and training for 20 CSOs on Government planning processes and budget tracking and monitoring was also planned but not implemented. 49 9.0 LESSONS LEARNED, GOOD PRACTICES AND INNOVATIONS Use of evidence to inform development of organizational and operational tools and advocacy initiatives was helpful. Notable among the good practices are: 1. Use of results of the formative study to inform the development of CSOs’ communication and advocacy plans, though its coverage was limited to only three (3) districts; two (2) in central and one (1) in western Uganda). 2. Use of evidence to inform the advocacy agenda was helpful in eliciting action of duty-bearers and decision-makers. For instance, in Ibanda district, Coalition for Health Promotion and Social Development (HEPS) presented evidence to district officials on Health worker absenteeism and non-functionality of HUMCS, as well as their effect on Maternal and Child Health. As a result, the Chief Administrative Officer (CAO) issued a circular to all civil servants to adhere to the code of conduct, including reporting to their duty stations on time (8:00 am) and leaving at 5:00 pm. Audio recordings of community dialogue proceedings were used during radio talk shows to generate further debate through live call-in from the listeners. This was very helpful in getting feedback based on citizens’ concerns. 3. Media analysis and use of related results to inform capacity building initiatives. The findings from a media analysis by CEHURD were used to orient 24 journalists in responsible media advocacy, which includes mainstreaming human rights in their reports on health issues. 4. Coordinated and streamlined media messages promote inclusiveness. For instance, the radio spot messages, radio talk shows and radio serial drama were guided by talking points and scripts approved by PATH and USAID. The media champions in Mbale profiled stories that were aired on radio and published in print media while some recordings of community dialogues were broadcast, generating debate from listeners on issues discussed. For instance, on Elgon Radio 101.4 FM in Mbale district, the media champion focused on drug stock-outs and health worker absenteeism at Jewa HC III and Kigezi HC II in Mbale District. 5. The use of GMCs including the PDSA process provided an opportunity for CSOs to share and learn from each other; identify problems or barriers to achieving their program targets. Conducting a root cause analysis and designing change strategies for improving performance were key components that promoted continuous learning. Through the change strategies, CSOs were able to systematically learn from their own programs and demonstrated approaches that are effective to improving their performance. 6. Policy advocacy is a new concept to most CSOs, and thus requires considerable time and effort for them to understand and implement it. 7. Engagements with relevant government Ministries, Departments and agencies not only yields project buy-in but also ensures the project’s invitation for participation and involvement in related working groups. 8. Holding consensus-building meetings with partners is an efficient way of developing common positions on strategic issues. This was manifested in the design and strategy consensus-building meeting that yielded the finalization of the Project Results Framework, 9. Sustained participation in district planning processes such as DMC and Extended District Technical Planning meetings not only enables the project to gain visibility and network, but also paves way for CSOs’ advancement of citizen’s health and social services priorities. 10. Online technical assistance needs to be supplemented with technical assistance visits to the sub￾grantees in order to ensure shared understanding on the use of tools and provide on-site support for some of the actions; and 11. Coordinating joint advocacy initiatives especially at the national level requires a lot of commitment, astuteness and mobilization 50 12. The provision of ample time and environment for citizens to raise their concerns and duty bearers provide information on any anomalies or opportunities, is critical for successful and harmonious relations between citizens and duty bearers. 51 10.0 IMPLEMENTATION GAPS AND CHALLENGES Community groups focus on a few parishes within their sub counties: While community groups are mandated to raise community awareness in all villages within their respective sub counties. One CSO described the challenge of covering the entire sub-county with one community group in this way: The only issue there is maybe the coverage. Each of the groups I am talking about, it is one group in each sub county. And a sub county is such a huge establishment, […] how does it share the good and better pieces of their information with other groups, of other communities within the sub county? We don’t know. - Executive Director, July 2017 Further, the frequency of large scale engagement of community members through forums and music dance and drama, was limited to about one per year. These have been linked to the limited funds though slightly more has been was allocated for this component in the fourth year. About half (49.3%) of respondents saw or heard an advocacy message from the project, about the same proportions (50%) listen to radio and or watch or listen to television. Reaching them through interpersonal communication would therefore, be helpful in raising awareness in communities, but requires expanding coverage to truly reach the entire sub-county Lack of understanding of advocacy by some duty bearers: Some of the duty bearers and leaders still have do not understand advocacy or have negative perceptions. They tend to view who ever advocates for improved services as an enemy, and want CSO’s to provide the services directly instead. To this elucidate this one participant noted: Of course, ABH project, as a software project, at the initial stage was a bit of a challenge because [CSO] was solely known for hardware. […] But now when we came with ABH, we are saying hi, government, it is your role to make sure that these [services are delivered]. People were saying, but eh man, yesterday you were supporting us, now you are coming saying eh, do your work. So it was a bit of a challenge, it took long for people to understand us. - Program Officer, District-level CSO, July 2017 Advocacy in some communities faces the same challenge. Citizens reported fear of the likely negative outcomes if one dared to expose wrong doing by a public official as stated: “You see reporting these health workers wouldn’t be a problem but the challenge is that most of us fear to be victimized, so no one is willing to say so and so did this, or if the health workers come to know that the person who reported them is from Kibale, or any other village, will hate the whole village”. - FGD Participant, August 2017 Supply side constraints: The demand for services in several communities exceeds the resources available at health facilities and within districts. Key informants revealed that health workers operate under much pressure from citizens and other stakeholders, and districts do not always have adequate resources to provide further health services, even when they are aware of community needs. This may partly account for limited changes to district plans and budgets. . One stakeholder stated: 52 So when you go there, the first meeting they will tell you, yes we have heard these things but we don’t have the money. Now, why should you go there in another meeting and you are talking about the same thing and they’ve told you they don’t have the money about? - Executive Director, District-level CSO, July 2017 Evidence collection and use: While mechanisms for linking district and national level CSOs for data collection and use were in place including use of social media, there CSOs still expressed a gap in having a more formal and well-coordinated mechanism. Network organizations like NAFOPANU were noted to have performed well in terms of data collection from districts due to its extensive reach. While mapping was conducted involving such groups, they were not trained on their roles and responsibilities relating to representing community interests and ensuring community members are aware of decision-makers’ actions. The application of the community scorecard for social accountability was only piloted and replaced by HFAs. Relatedly, while public hearings where target audiences were meant to be brought together to share experiences and discuss potential solutions were largely not applied yet they would have been good avenues for discussions. The use of annual surveys of networks, umbrellas, and IPs as well as operations research were not conducted yet they would have facilitated generation of additional data. Efficiency i. While the FAA mechanism and associated retirement of milestones minimizes on the number of documents required to liquidate transaction expenses, the number of documents used as MoVs was still big. This contributed to delays in the retirement of milestones and subsequently negated implementation. Whereas the ABH project management had supported CSOs to quicken the process, especially by submitting milestones early without waiting to accumulate them, the retirement process still took relatively long as noted: “The documents from one CSO are usually very many; it takes a lot of time reviewing them” (ABH staff, September 2017). ii. Though sub grantees performed very well in meeting their milestones, they delayed to retire them. This led to a very low burn rate as money was held up in milestones, Consequently, the threshold burn rate of 75% was not met in time to enable the project management access the remaining obligated funds given that this situation was noted towards the end of the contractual period. Subsequently, the project under spent on the year’s obligated budget. iii. The project was designed to raise citizens’ awareness on their health rights and responsibilities using community groups within sub counties among others. The key strategy was that community groups would integrate awareness raising into their none ABH supported activities. However, most community groups focused on very few parishes, largely due to limited funding. For instance, in Bushenyi District, Kyeizoba Sub county HIV Network effectively worked in two (2) out of eight (8) parishes while Mastered Seed effectively worked in one (1) out of the six (6) parishes of Bitooma Sub county. This big gap limited the timely flow of information to citizens. One CSO describes it succinctly: Now if the champion comes from here, and the group is here, then that means there is also no regular mandate to support this champion to move from this parish of his and go and support, and support the group. So that means you realize that now, with the advocacy, it is almost concentrated where the group is, and where the champion is located. - Program Officer, District-level CSO, July 2017 53 iv. The project was hit by reduction of the year two (2) budget from the initial submission of approximately US dollars 4.4 million to approximately US dollars 3.5 million. This affected the accomplishment of major project activities. While the project management endeavored to address this challenge by re-prioritizing and rescheduling activities in the work-plan, several CSOs halted their work. This left a big back log that partly contributed to overall low performance especially at community level. v. While the use of project staff to build capacity of CSOs saved costs, it was associated with delays in delivery especially for tailored trainings. Each CSO have some unique needs that necessitated a separate training. Given the big number of CSOs with such needs, the project staff were unable to attend to most of them on time as noted: “There are times when staffs are busy doing other project work and unable to provide training to some specific partner needs. Sometimes this delays our capacity building support” (ABH staff interview, September 2017). . 54 11.0 CONCLUSIONS AND RECOMMENDATIONS 11.1 Conclusions Project progress: Overall, the ABH project met or surpassed targets for 10 out of 15 (66.7%) indicators. This fairly good project performance was mainly attributed to objectives two (2) and three (3) of promoting CSOs to effectively advocate for issues of concern to citizens in health and social sectors, and building their institutional and technical capacity. The prospect of attaining end-term targets remains high. Performance on most (75%) of the indicators under result one (1) geared towards enabling citizens to demand improved quality of services met targets. Improvements were noted regarding service delivery at public health facilities. The midterm target of public health facilities recording improvements in service delivery was exceeded by 24%. This was largely attributed to health workers’ improved adherence to working time, leading to reduction in any health facility closures from 47% to 3%. Effectiveness: Attainment of targets was partly a result of support and follow up on the sub grantees. On the other hand, poor performance on other indicators was largely due to; poor mobilization of citizens since the community groups focused on a few parishes. More attention should be paid to reaching manageable communities, redesigning communication messages and approaches, and increasing community groups’ consistency in meeting and following up with duty bearers on issues raised. Interventions for facilitating CSOs to effectively advocate for and represent communities on policies / issues of citizens concern in the health and social sectors were effective with overall performance of 60%. CSOs’ advocacy agenda included engagement of government authorities in various ways leading to some actions on health and social service issues and policy gaps and focus was put on supporting and encouraging CSOs to collect especially through health facility assessments and its use in advocacy initiatives. CSOs’ attendance of government planning meetings at national and district levels was high, which was a major avenue for conveying citizens’ concerns. However, some of the CSOs focused on achieving milestones to enable them recover funds used and missed out on other actions including joint advocacy. While there was good representation of citizen’s concerns by CSOs at government meetings, most government authorities were still less supportive of advocacy to service delivery initiatives; leading to low incorporation of citizen’s concerns in district plans. Nonetheless, other local governments had limited resource to cater for the citizens’ issues raised. Efficiency: Most of the project’s strategies were executed to a good degree, leading to efficiency. The Fixed Amount Award (FAA) provided flexibility regarding meeting of milestones, use of sub grantee and PATH resources and good framework for capacity building focusing on the whole institution facilitated efficiency. Through implementing the capacity building framework including OACAs, action planning, structured and tailored training, as well as review of the action plans, the project has improved the quality of service delivery by CSOs. Whereas training by project staff saved costs, it delayed some tailored trainings due to small number of ABH project staff and big number of CSOs. While the FAA mechanism reduces costs, the number of documents for MoVs was relatively big and delayed the retirement process. One of the key strategies of reaching citizens was through the integration of awareness raising interventions into community groups’ routine work. However, most community groups worked in only few parishes denying and did not reach several citizens. 55 Relevance of the project: The project was aligned to; USAID Uganda’s CDCS 2016-2021. For instance, it promotes health systems strengthening and seeks to promote biodiversity and natural resource management, climate change adaptation and inclusive and accountable democratic governance. It is also aligned with national and international policies and frameworks including those on; FP, SRH, MCH; malaria, nutrition, TB, HIV/AIDS and OVCs. Interventions under each of these components were designed with due regard to the relevant policies. Thematic areas: District-based sub-grantees were assigned to focus on a primary and secondary thematic area while national sub-grantees focused on only one thematic area. There was more concentration on HIV/AIDS/TB (53.3% at district level, 42.9% at national level and 48.1% combined) and less on family planning (6.7% at district level, none at national level and 3.3% combined). Considering the secondary focus areas, HIV/AIDS/TB was still covered, by almost half (46.7%) of the district CSOs and the least covered was nutrition at 13.3%. Nutrition issues were not assigned to any CSO at district level. Support to OVCs was integrated into other focus areas since it was no longer receiving specific funding. 11.2 Recommendation ix. Community groups were supposed to integrate awareness raising on citizens’ awareness on their health rights and responsibilities, into their other activities in respective sub counties. However, most community groups have focused on one or two parishes largely due to limited funding. This largely accounts for the low performance on understanding of health rights and responsibilities. Whereas some additional resources have been allocated in year four (4) for this component, the gap is still big. The ABH senior management should work with CSOs to refocus this community component to cover a small number of parishes with a sub county. This should be guided by the available funds. Nevertheless, the media campaign will continue reaching all parishes. x. Given the limited funding, the CSOs should collaborate with religious institutions and use their gatherings and structures to pass over the messages. The messages can be delivered at a convenient time, for instance during or at the end of the service or mass. xi. The media messages are not clearly understood by citizens at community level. Even when some people here some radio messages, it takes long for them to associate with it. The ABH project team should lead a process of redesigning the media messages to make them more appealing to citizens at community level. xii. CSOs should incorporate VHTs as resource persons to help raise aware in communities. The VHT structure was wide spread and VHTs commended good respect, in their communities and good at communicating on health issues. However, since they work closely with health facility staff, their role should largely focus on raising awareness. xiii. Most government officials had not fully appreciated the value of advocacy. CSOs’ presentation of citizens’ concerns in government planning meetings improved but very few districts incorporated such issues in their plans. The ABH project senior management should offer more support to CSOs in engaging the senior leadership of districts, with a view of getting support for citizens’ concerns. The senior management of ABH project should also take lead in collaborating with the senior management of the Uganda Local Governments Association to support on this. xiv. The visibility of the project is not good among some stakeholders even when it is supporting a lot of work. The ABH senior management should work with sub grantees and agree on a mechanism that will strengthen branding of the project. Partners need to acknowledge the support from the project at every supported event. The ABH communications team should not take long to post achievements on the social media platforms. 56 xv. The ABH senior management should work with sub grantees to strengthen or establish more accountability platforms that will enable tracking of performance commitments. Linkage with very high ranking decision makers such as Permanent Secretaries will be more critical for the success of these accountability mechanisms. xvi. To strengthen coordination for data evidence collection and use among district and national level CSOs, the project management should strengthen on current modalities especially ensuring linkage of other CSO staff apart from Executive Directors. One of the options is to establish an online portal for use by the project stakeholders especially sub grantees. 57 ANNEX 1: DETAILED EVALUATION METHODOLOGY 1.0 Methodology This section presents study design, process of the review, methods of data collection used, sampling and sample size estimation, data analysis, validity and reliability of findings, challenges and limitations faced. 1.1 Study design, participants and sampling procedure A cross-sectional mixed methods design involving quantitative and qualitative methods of data collection was applied. This was generating both statistical and explanatory information relevant for improving the project in the next phase. Primary data collection was divided into seven (7) components including the; Household Survey; Health Facility Assessment (HFA); Organizational and Advocacy Capacity Assessment (OACA); Policy Tracking; and Data validation as well as Key Informant Interviews (KIIs) and Focus Group Discussions (FGDs). Several documents were reviewed as part of the MTE. A total of 1,237 citizens including 47.2% males and 52.8% females participated in the survey. This was selected using the formula proposed by Cochran, W. G. 196310 represents 96.2% of the target sample, which is good enough for generalization of the study population. This sample consisted of males aged 15 – 54; females aged 15 – 49; youth aged 15 – 24; and OVCs aged 10 – 14; selected using probability proportional to sample from the ten (10) districts that were covered by the baseline survey. Thirty-three (33) Key Informants (KIs) were involved including USAID Implementing partners; sub grantee staff; ABH project staff; Health Facility (HF) In-charges; and District Health staff. Ninety-eight (98) FGD participants were involved including men, women and mixed gender youth. 10Cochran, W. G. 1963. Sampling Techniques, 2nd Ed., New York: John Wiley and Sons, Inc. 58 Table 1: List of KIs and FGD Participants Type of Respondent Number of interviews National-level CSO’s 2 National-level duty bearers 1 District-level CSO’s 7 District-level duty bearers 9 Implementing partners 3 Health facility staff 7 Chief of Party 1 Other ABH staff 3 Total 33 Participant Group Number of Respondents Men 32 Women 31 Youth 35 Total 98 1.2 Sampling methods and procedure Table 2: List of KIs and FGD Participants Selection level / criteria Sampling Method Sampling procedure Districts Purposive In order to enhance comparability with baseline data, all the ten districts covered during the baseline survey will be purposively selected for this midterm evaluation. Sub counties Simple random sampling Selected from a list obtained from UBOS. A total of 22 sub counties have been selected. Parishes Simple random sampling Selected from a list obtained from UBOS. Two target parishes for each sub county have been selected except one where only one was due to a small number of respondents required. Villages Simple random sampling Lists of villages in each target parish will be obtained from local leaders. Two target villages will then be selected by simple random method. Households Simple random sampling • A list of households in each village will be obtained from local leaders and used in the selection process. In cases where the lists are not available, the Field Supervisor will work with the team and local leaders to make it. • To obtain the sampling interval k , the total number of households will be divided by the 15 (total of households required per village). • From the first interval we selected a random start g using 59 1.3 Sample size determination The sample size has been determined using formula for proportions proposed by Cochran (1963) stated below: 2 2 0 d pqz n = Where: z = the z-statistic is the value at α% level of significance in a two-tailed probability distribution; p = probability of selection following binomial distribution (percentage of the men and women who have listened to a radio at least once a week); q = probability of non-selection following binomial distribution; and d = permissible error during study Given that the response rate at baseline was 100%, the above sample size has been adjusted upwards by only 5% to cater for likely non responses. When the above are substituted in the formula, considering a confidence level of 95% and plus or minus 2.8% margin of sampling error, the final sample size of 1,286 has been obtained as shown below: Z z² p q D d² z²pq z²pq/d² Target sample (z²pq/d²)*1.05 1.96 3.8416 0.5 0.5 0.028 0.000784 0.9604 1,225 1,286 1.4 Operational Definitions of age groups (categories of respondents) simple random sampling. Simple random sampling will be used to select the random start from the first to the fifth one, using papers with index numbers of each. • From the random start g , the sampling interval k will be added to the random start to get the second household. This pattern will continue until they get all respondents. • In case the chosen household does not have a qualifying respondent, it will be replaced with the next one on the list. Respondents Simple random sampling • In the situation where two or more eligible respondents are in one household, one person will randomly be selected after tossing folded pieces of paper with index numbers for each eligible person. • Only one respondent will be selected from each house. No respondents in the same category will be selected from consecutive households • The respondents per village will include:  Three (3) OVCs of mixed gender  Four Youth - 2 girls and 2 boys  Three (3) males aged 15 – 54  Four (4) females aged 15 – 49 60 The operational definitions of respondents’ categories used at baseline will be applied and are listed below: 1. Men (15-54): Any male person aged 15 to 54 years and has a living in partner 2. Women (15-49): Any female person aged 15 and above and has a living in partner. 3. Youth (15-24) who do not have a living in partner; and 4. OVC (10-14 years): A child in this age group who are in a state of being or likely to be in a risky situation, where he / she is likely to suffer significant physical, emotional or mental harm that may result in his / her human rights not being fulfilled 1. Orphans 2. Children infected and affected by HIV&AIDS 3. Children with disabilities 4. Children in worst forms of child labour 5. Children from child headed households 6. Children experiencing various forms of abuse and violence e.g. survivors of sexual violence; 7. Abandoned children/neglected children 8. Children in contact with the law 9. Out of school children 10. Children living in chronicle poverty(in poverty stricken /impoverished) households 11. Children living with the elderly persons 61 1.5 Sample size per district Probability proportional to population size11 has been applied to determine the target sample per district. Details are presented in Table 1.3. Table 1.3: Sample distribution District No. of sub counties % contribution Male Female Total Population 10 - 54 years (60.75% NPHC 2014) % contribution Sample size After 5% over sampling No of Researchers Assistants (RAs) No. of tools per RA No of days No of sub counties in study No. of parishes Average no. of respondents per parish No. of villages Average no. of respondents per village Busia 10 0.10 157,415 166,247 323,662 196,625 0.10 119 125 4 6 5 2 4 30 8 15 Bukwo 4 0.04 45,258 44,098 89,356 54,284 0.03 33 34 2 5 3 1 1 30 2 15 Kumi 7 0.07 117,007 122,261 239,268 145,355 0.07 88 92 3 6 5 2 3 30 6 15 Kaliro 5 0.05 116,787 119,412 236,199 143,491 0.07 87 91 3 6 5 2 3 30 6 15 Bushenyi 7 0.07 116,410 118,030 234,440 142,422 0.07 86 90 3 6 5 2 3 30 6 15 Isingiro 10 0.10 236,619 249,741 486,360 295,464 0.15 178 187 6 6 5 3 6 30 12 15 Kasese 29 0.28 339,455 355,537 694,992 422,208 0.21 255 267 9 6 5 4 9 30 18 15 Kabale 19 0.18 254,414 273,817 528,231 320,900 0.16 194 203 7 6 5 3 7 30 14 15 Luwero 13 0.13 230,451 226,507 456,958 277,602 0.14 167 176 6 6 5 3 6 30 12 15 Kalangala 7 0.07 31,047 23,246 54,293 32,983 0.02 20 21 2 5 2 1 1 30 1 15 Totals 104 3,343,759 2,031,334 1.00 1225 1286 45 22 86 11Uganda Bureau of Statistics 2016, The National Population and Housing Census 2014 – Main Report, Kampala,Uganda 62 Note: I. Sample size distribution across districts is based on their proportional population sizes. II. Given that in most surveys that involve enumeration area (EA) / cluster selection, which are villages in this case, the EA sizes tend to range from 5 to 20, to minimize the design effect, the EA selected is 15. Fifteen is normally a manageable number and lies nearly just above the mid￾way and the highest EA sizes, hence the statistical rational of the researcher’s choice. A maximum of 15 respondents will therefore, be considered per village. III. A total of 86 EAs (villages) will be covered. This includes a maximum of two villages per parish. IV. A maximum of two parishes will be considered per sub county. V. The exact numbers of respondents per category will be determined after getting a common understanding with the team members about the definitions. 1.6 Sampling methods and procedure Given the variations of target districts in terrain, geographical location, proximity, size and language spoken by the inhabitants, a multi stage sampling technique will be used as showed below. Table 1.4 Sampling methods and procedure Selection level / criteria Sampling Method Sampling procedure / reason Districts Purposive In order to enhance comparability with baseline data, all the ten districts covered during the baseline survey will be purposively selected for this midterm evaluation. Sub counties Simple random sampling Using the list of obtained from UBOS, target sub counties in each of the ten districts have been selected by applying simple random sampling. Sub counties have been proportionately randomly selected for the study using random tables. A total of 22 sub counties have been selected. Parishes Simple random sampling Applying lists of parishes for each district, the two target parishes for each sub county have been selected using simple random sampling. A total of 43 parishes have been selected. Villages Systematic and Simple random sampling Within a parish, systematic sampling will be used to select the first village in which the health centre II or III is located. In case the parish does not have a health centre II or III, the biggest communal facility like a church, school or market will be considered. The second village will be selected using simple random sampling by rotating a pen. In this case, the village in the direction of the pen’s head will be selected so long as it is located within the same parish. A total of 86 villages will be covered. Households Simple random sampling While in each EA, the Research Assistants will first locate its centre such as a church, health centre or market and then rotate a pen to get a random direction. In the direction of the pen’s head, the Research Assistants will skip the first household and visit the second one with a respondent. This pattern will continue until they get all respondents. In case the chosen household does not have a qualifying respondent, it will be replaced with the one immediately neighboring it in the same direction. If two households are at the 63 same distance in the desired direction, simple random sampling will be applied to select one of them after tossing folded pieces of paper with index numbers for each household. Respondents Simple random sampling In the situation where two or more eligible respondents are in one household, one person is randomly selected after tossing folded pieces of paper with index numbers for each eligible person. In order to get all categories of target respondents, a female aged 15 – 49 years is selected from the first household; males aged 15 – 54 years from the second household; youth aged 15 – 24 years from the third household and an OVC from the fourth household. A respondent who discontinues with the interview is replaced by another one from within the same or neighbouring household depending on availability. Persons in the same household will be prioritized for replacement. Focus Group Discussion Participants Purposive sampling Groups of females; males and youth of both gender, of the same age group were selected purposively from one communitiesin the sampled sub counties, with help of the guides. Each FGD constituted of between six (6) and twelve (12) participants. Mobilisation of the groups was carried out beginning on the first day of data collection and appropriate time agreed on with the CSO. 1 1.7 Selected study areas and related personnel allocation Table 1.5 Selected study areas and required Region District Sub county Work load No. of 'RAs & Supervisors No. of day s Parish Questionnaire s (Qnnaires) FGD s For Qnnaires For FGDs Supervisor s Central Kalangal a Kalangala Town Council Kalangala B 21 2 2 Luwero Bamunanika Kyampisi 176 3 4 2 1 7 Sekamuli Zirobwe Bukimu Nakigoza Katikamu Kyalugondo Migadde Wester n Bushenyi Bitooma Nyanga 90 3 4 2 1 4 Kyeizooba Kitagata Rutooma Kabale Kamuganguzi Buranga 203 6 1 7 Kicumbi Mayengo Katuna Town Council Kiniogo Mukarangye Kyanamira Kanjobe Nyabushabi Kasese Karambi Kamasasa 267 6 1 7 Kisolholho Munkunyu Kitsutsu Nyakatonzi Bugoye Bugoye Ibanda Katooke Nyamwamba Nyakasanga II Rukoki Isingiro Kakamba Kashumba 187 6 1 5 Murema Ngarama Kagaaga Ngarama 2 Birere Kasaana Kyera Eastern Busia Busitema Chawo 125 3 6 1 4 Syanyonja Lumino Hasyule Lumino Kumi Atutur Akalaba 92 5 1 3 Kapokin Nyero Ariet Bukwo Bukwo Town Council Kapkorosoi 34 2 3 Torasis Kaliro Bukamba Nangala 91 4 2 1 4 Bumanya Kasuleta Kyani Total 1,286 45 6 8 . 3 ANNEX 2: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – HIV/AIDS Policy targeted Key Ask or policy area / issue addressed Progress made/ achieved Key Sub grantee involved The HIV/AIDS Control Act 2014. The Act has provisions that may increase the stigma and discrimination around HIV and discourage more people from being tested and treated. Government to develop and pass a separate HIV Anti-Stigma and Discriminatory Policy Various meetings held at UAC, concept note developed. UNAIDS willing to fund consultant to write policy. ABH also met the Ag. DG who pledges to involve the team in planning meetings NAFOPHANU The New Consolidated Guidelines for HIV Prevention and Treatment 2016 https://ug.usembassy.g ov/u-s-government￾supports-ministry￾health-launching-test￾treat-guidelines￾uganda/ Government to fully implement the provisions of the test and start requiring ART initiation to all persons testing positive for HIV at any CD4 cell count Guidelines launched in December 2016. Project now working on socialization of the policy and provisions especially among PLHIV and Key and priority populations to demand but also educate others of this development has taken place NAFOPHANU, MAFOC, STF, IDO, KADINGO, RHU (KATUNA MARPS), ACODEV, JIACOFE, ICWEA, UNASO National Medicines Policy 2015 National Medicines Policy 2015 whose overall goal is to ensure the availability, accessibility, affordability and appropriate use of essential medicines of appropriate quality, safety and efficacy at all times Government to ensure adequate supply of HIV/AIDS drugs and related supplies -Stock out campaign held in 2015 that resulted in GFATM frontloading funds. -Regular stock status updates being shared by NAFOPHANU -Monthly HFAs generating advocacy evidence. - Quarterly meetings with NMS NAFOPHANU, UNASO, MAFOC, STF, IDO, KADINGO, RHU (KATUNA MARPS), ICWEA, ACODEV, JIACOFE 4 National budget for the FY 2017/18 UAC (2015) National HIV and AIDS Strategic Plan 2015/2016- 2019/2020 Government should increase funding for HIV to 150bn from the current 95bn especially in view of the T/S - Gov’t to implement innovative ways like the NATF and HIS that would increase Domestic funding -Meetings held with the Parliamentary HIV/AIDS committee of and the Health Committee respectively. -Joint press conference held and position paper on BFP FY17/18 shared with policy makers. Budget retained at 1.8 Trillion UGX -Part of NATF advocacy (HE President’s Fast Track Initiative on Ending AIDS by 2030-key) NAFOPHANU, UDN, UNASO, MAFOC, STF, IDO, KADINGO, RHU (KATUNA MARPS), ICWEA, ACODEV, JIACOFE Comprehensive HIV/AIDS services in sub counties with HCIIs -Districts to establish outreach services, and upgrade facilities to HC III -Reducing stock out of HIV/TB commodities • District level engagement held with leaders in Busia. • Directive came from MoH asking the CAO to provide information on sub counties that require upgrade to HCIII • Recruitment of critical cadres (from 52%-57%) in Busia • 4 HFs in Mbale received drugs for the 1st time in 2017 (Jeewa, Bukasakya, Bumasikye, Bukiende) MAFOC, NAFOPHANU, UNASO, MAFOC, STF, IDO, KADINGO, RHU (KATUNA MARPS), ICWEA, ACODEV, JIACOFE Comprehensive HIV/AIDS services in sub counties with HCIIs Districts to establish outreach services, and upgrade facilities to HC III Wrote to MoH to upgrade health facilities within the fishing communities -Positive feedback from MoH MAFOC, STF, IDO, KADINGO, RHU (KATUNA MARPS), ICWEA, ACODEV, JIACOFE 5 ANNEX 3: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – TB Policy targeted Key Ask or policy area / issue addressed Progress made/ achieved Key Sub grantee involved Domestic Financing Increase domestic funding for TB activities in the country from current 4% funding to 25% to less donor dependency -Supported the launch of the Parliament TB Caucus in anticipation of any legislation on TB to raise profile of the issues. NAFOPHANU, UNASO, MAFOC, STF, IDO, KADINGO, RHU (KATUNA MARPS), ICWEA, ACODEV, JIACOFE TB Desk Guide Develop TB Desk Guide for the prevention and control of TB in health facilities - Draft developed. Planned retreat to finalize guide from 27-29 June 2017 NAFOPHANU ANNEX 4: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – MALARIA Policy targeted Key Ask or policy area / issue addressed Progress made/ achieved Key Sub grantee involved Test, Treat and Track policy 2012 Tax exemption of RDTs • Evidence shows this as not an issue & done illegally • URA invoking the East African Customs Union law to tax MRDTs • High level national level dialogue held HEPs Uganda Malaria Strategic plan 2015- 2020 DHO to prioritize and allocate a budget to revitalize VHT structure to ensure increased community education on malaria prevention and management programs by September, 2018 • Kamuli- DHO allocated budget for VHT facilitation • Budget for training, and IEC material to ease VHT- put under unfunded priorities in the 2016/2017 district budget • Mayuge : 2 HCIII (Buwaiswa and Malong) and 1 HCIV (Kityerera) allocated part of PHC funds to VHT in the quarterly workplans JIACOFE Malaria Test , Treat, and Track policy 2012 The mandate for testing quality and approval of MRDT in the country to be put under NDA - Included in the in revised Malaria Control Strategy HEPs 6 ANNEX 5: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – MATERNAL, NEWBORN AND CHILD HEALTH (RMNCAH) Policy targeted Key Ask or policy area / issue addressed Progress made/ achieved Key Sub grantee involved Roadmap for Accelerating the Reduction of Maternal and Neonatal Mortality and Morbidity in Uganda, 2007–2015 Ensuring availability and accessibility of maternal and child UN lifesaving commodities (oxytocin, magnesium sulfate, misoprostol, implants, female condoms, chlorhexidine, ORS, zinc and Amoxicillin) Presented status evidence to MCH cluster, who presented it to Commissioner RH. This is being pursued. CEHURD, WRA-U Roadmap. Strategy 3: strengthening human resources for health for maternal and newborn. Government to recruit 3000 midwives -Prioritized in FY 17/18 budget advocacy -Ministry of Health presented this at the inter-ministerial meeting held at MoFPED. -Included in the GFF -HR campaign planned CEHURD, WRA-U ANNEX 6: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – FAMILY PLANNING Policy targeted Key Ask or policy area / issue addressed Progress made/ achieved Key Sub grantee involved Family Planning 2020 Presidential commitments (2012) Reproductive Health Commodity Security Strategic Plan, 2009/10–2013/14 (To increase public sector/government budget allocation and expenditure on RH commodities, including contraceptives, to 80 percent by 2015). Honoring Presidential commitment to allocate $5m for FP commodities every year for five years. -Engaged Ministry of Health and Parliamentary Committee on Health to ensure that 2017/18 national budget include the pledged funds -(Appropriation analysis of FY 17/18 budget yet to be done) Reproductive Health Uganda Districts prioritizing FP. Integrating FP in the district plan and budget. District Costed Implementation Plan for FP developed for Mityana RACOBAO 7 Adolescent Health Policy 2004 The national policy guidelines and service standards for SRHR, 2011(guides the implementation of Adolescent health policy and emphasizes Youth Friendly Services) MOH to implement Adolescent Health Policy -Engaged MoH and commitment to scale up implementation was obtained. - Participation in the current review of the Policy being conducted by MoH -Some districts started implementation by establishing youth corners to provide Youth Friendly Services. For instance inBugiri, Namutumba and Busia RHU, STF School Health policy 2014 Government to finalize the school health policy Policy was developed. Cabinet approval deferred due to the review of the sexuality education framework RHU Immunization ACT 2016 (Established an immunization fund) Finalize the immunization guidelines to operationalize immunization fund -Draft guidelines developed. -Presentation to MoH Top Management , and engagements with MPs planned CEHURD ANNEX 7: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – Nutrition Policy targeted Key Ask or policy area / issue addressed Progress made/ achieved Key Sub grantee involved The Uganda Food and Nutrition Policy 2003 Domestic financing for nutrition at national and district levels, given the absence of a specific budget vote to finance nutrition activities on an annual basis in Uganda - Draft policy under review - Established a Uganda National Nutrition Advocacy platform PATH leading Uganda Nutrition Action Plan (UNAP) 2011-2016 -Held two high level policy dialogue on nutrition with MP and other stakeholders -Supported the establishment, orientation & recent launch of PFN as a platform to speed up any legislation PATH leading 8 OBT Adopt performance monitoring indicators for nutrition track and monitor performance • Presented this to the Nutrition Secretariat in OPM. • Further engagements to be held with them PATH leading DNAP -District Local Governments to empower district nutrition coordination committees to execute their mandate -District Nutrition Coordination Committees to implement the district nutrition action plans in the districts • Held regional experience sharing meetings with district leadership in Western Uganda. • Commitments made and will be followed • Experience sharing for East Central planned end July PATH leading ANNEX 8: THEMATIC POLICY ADVOCACY PROGRESS AND OUTCOMES – OVC & Other Cross Cutting Issues Policy targeted Key Ask or policy area / issue addressed Progress made/ achieved Key Sub grantee involved UPE POLICY Drop-out rate, absenteeism and poor performance in UPE JIACOFE- The ordinance to end child labor passed by District Council. ACODEV MAFOC KACSOA LADA JIACOFE RACOBAO Anti- Female Genital Mutilation (FGM) Act 2010 Limited compliance with Anti- Female Genital Mutilation (FGM) Act 2010 • Community mobilization to engage in OVC related dialogue. • Issue was challenging and dropped KACSOA HRH (availability, motivation, and retention) -GoU should recruit more critical cadre in the health sector -Busia recruited 3 additional midwives STF -Regular attendance to duty by HWs -Busia district is tracking attendance to duty on a monthly basis, and discussed at EDHMT STF National Budget FY 2017/18 Government to increase health sector budget to 10%. Dialogue was held with MoH, MoFPED and Parliament. Budget now retained at 1.8trillion UGX UDN Source: Adapted with adjustments from the Policy Tracking Tool