September 2016 The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development, the United States Government, the United Kingdom’s Department for International Development, or the United Kingdom Government. MID-TERM EVALUATION USAID/UGANDA AND DFID GOVERNANCE,ACCOUNTABILITY, PARTICIPATION AND PERFORMANCE (GAPP) ACTIVITY i ABSTRACT The Governance, Accountability, Participation and Performance (GAPP) Activity was funded by United States Agency for International Development and UK Department for International Development to improve governance and accountability, enhance fiscal management, and increase citizen participation at local level. GAPP implementation focused on three broad areas: 1) support and strengthen national-level accountability institutions in their oversight role; 2) strengthening local government systems in revenue, procurement, finance, and planning; and 3) strengthen civil society organizations to improve voice and accountability. The purpose of the mid-term evaluation was twofold: 1) help USAID/Uganda learn about which interventions should be deepened or re-directed in future; and 2) understand how GAPP complements efforts of other sectors. Data was collected by document review, key informant interviews, focus group discussions and a validation workshop for findings, conclusions and to publicize recommendations. Content analysis was used to extract meaningful inferences to answer evaluation questions. The theory of change was found valid. There was minimal coordination between USAID sector teams. GAPP influenced: 1) national accountability institutions for local government oversight, 2) local government functionality and 3) citizen participation, voice and accountability to improve local service delivery. Effective approaches for programming included: 1) engagement with national accountability institutions, 2) fulltime embedded technical specialists, 3) focus on core local government functions and 4) working with appointed and elected officials. GAPP contributed to improvements in: 1) information flow and practices for local government procurement and accountability, 2) improvement in performance of supported local governments and 3) effective mechanism for unblocking citizen voice and participation in local governance and accountability processes. The Collaborating, Learning and Adapting model has not facilitated cross-sectoral collaboration and learning due to other sector teams’ little idea about GAPP’s activities and results. ii MID-TERM EVALUATION OF USAID/UGANDA and DFID GOVERNANCE, ACCOUNTABILITY, PARTICIPATION AND PERFORMANCE (GAPP) ACTIVITY The 7-year USAID/Uganda- and DFID-funded Governance, Accountability, Participation and Performance Activity (2012-2019) was designed with the goal to increase participation, accountability, and local governance for effective service delivery. To achieve this goal, Research Triangle Institute (primary implementing partner), together with its consortium partners, Development Alternatives Incorporated Inc. and Development and Training Services, have implemented a set of interventions providing technical and other assistance to national accountability institutions, key partners in the policy advocacy agenda, local government units and civil society/private sector organizations in Uganda to achieve the objectives of the Activity, which fall under USAID/Uganda’s Development Objective (DO) 1: Increase economic growth from agriculture and the natural resource base in selected areas and population groups; DO 2: Democracy and governance systems strengthened and made more accountable; and DO 3: Improved health and nutrition status in focus areas and population groups. DISCLAIMER The author’s’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development, the United States Government, the United Kingdom’s Department for International Development, or the United Kingdom Government. iii ACKNOWLEDGEMENTS This work was commissioned by USAID/Uganda and UK/DFID and was prepared independently by Gary Woller, Zie Gariyo, Daniel Lukwago and John Kiyaga-Nsubuga under contract with the QED Group, LLC. Monitoring, Evaluation and Learning Program (MELP). The authors appreciate the technical advice and support from the USAID Mission and MELP staff in designing and conducting the mid-term evaluation and preparing this mid-term evaluation report. Special thanks are extended to the USAID technical team for its input and assistance throughout the mid-term evaluation process, particularly Harriet Muwanga, GAPP Contracting Officer’s Representative (COR) and Morris Nsamba, Democracy & Governance M&E Specialist. The evaluation team also appreciates the input and cooperation of the management and staff of the GAPP Activity, in addition to their support during data collection. The input from all respondents, including those at national levels, local and lower local governments and national and district-level Civil Society Organizations, in addition to other Activity beneficiaries, is also much appreciated. Finally, the evaluation team acknowledges the technical and administrative assistance of Gilbert Matabi, Augustine Wandera and Charlotta Sandin from the USAID/Uganda Monitoring, Evaluation and Learning Program. iv TABLE OF CONTENTS ABSTRACT ................................................................................................................................ i ACKNOWLEDGEMENTS ...................................................................................................... iii ACRONYMS ............................................................................................................................. vi EXECUTIVE SUMMARY......................................................................................................... ix Evaluation Background and Purpose ...................................................................................... ix Evaluation Methodology .......................................................................................................... ix Findings/Conclusions ................................................................................................................ ix Recommendations .................................................................................................................... x EQ1: Managing Theory of Change and assumptions.................................................................. x EQ2: Cross-sectoral coordination and collaboration.................................................................. x EQ3a: Influence on national accountability institutions ........................................................... xi EQ3b: Influence on systemic functioning of local governments ................................................ xi EQ3c, EQ3d & EQ5: Effectiveness of GAPP’s interventions ...................................................... xi EQ4: Future programming ......................................................................................................... xi 1. INTRODUCTION ............................................................................................................. 1 2. CONTEXT-GAPP OVERVIEW ....................................................................................... 1 3. EVALUATION PURPOSE ............................................................................................... 3 4. EVALUATION METHODOLOGY .................................................................................. 3 Evaluation Questions ................................................................................................................ 3 Mid-term Evaluation Team ...................................................................................................... 4 Geographic Coverage ............................................................................................................... 4 Data Collection Methods .......................................................................................................... 4 Data Analysis Methods ............................................................................................................. 4 Methodological Limitations ...................................................................................................... 5 5. FINDINGS.......................................................................................................................... 5 5.1 Evaluation Question 1: To what extent has GAPP’s theory of change and underlying assumptions held true? How did GAPP manage any changes in the ToC or assumptions? ........ 5 5.2 Evaluation Question 2: What are we learning about coordination and management of cross-sectoral efforts from GAPP implementation within USAID and DFID? ........................... 10 5.3 Evaluation Question 3a: How well are GAPP interventions influencing the performance of national accountability institutions (NAIs) in their oversight function at both national and local governments? .................................................................................................................... 14 5.4 Evaluation Question 3b: How well are GAPP interventions influencing the performance of systemic functioning of local government? ........................................................................... 21 5.5 Evaluation Question 3c, 3d & 5: How well are GAPP interventions influencing the performance of citizen participation in local governance processes and service delivery? How v well are GAPP interventions influencing the performance of unblocking voice and accountability channels in local government? How well is CSO advocacy and monitoring of service delivery contributing to improving the quality and effectiveness of service provision at the local governments (LGs) and lower level governments (LLGs)? .......................................... 27 5.6 Evaluation Question 4: Which GAPP technical approaches are proving more effective and relevant for future programming? ...................................................................................... 31 6 RISKS POTENTIALLY AFFECTING GAPP RESULTS AND SUSTAINABILITY .. 33 6.1 Risks Potentially Affecting GAPP Component 1 Results and Sustainability ................. 35 6.2 Risks Potentially Affecting GAPP Component 2 Results and Sustainability ................. 37 6.3 Risks Potentially Affecting GAPP Component 3 Results and Sustainability ................. 40 7 CONCLUSIONS ............................................................................................................. 45 8. RECOMMENDATIONS .................................................................................................. 47 EQ1: Managing Theory of Change and assumptions................................................................ 47 EQ2: Cross-sectoral coordination and collaboration................................................................ 48 EQ3a: Influence on national institutions .................................................................................. 48 EQ3b: Influence on systemic functioning of local governments ............................................... 49 EQ3c, EQ3d & EQ5: Effectiveness of GAPP’s interventions ..................................................... 50 EQ4: Future programming ........................................................................................................ 52 ANNEX 1: GAPP MID-TERM EVALUATION SCOPE OF WORK ................................... 53 ANNEX 2: EVALUATION METHODOLOGY .................................................................... 61 ANNEX 3: BIBLIOGRAPHY .................................................................................................. 69 ANNEX 4: KII AND FGD DISCUSSION GUIDES .............................................................. 72 ANNEX 5: PERSONS INTERVIEWED ................................................................................. 90 ANNEX 6: EVALUATION MATRIX..................................................................................... 98 ANNEX 7: STRUCTURAL WEAKNESSES IN LOCAL GOVERNANCE SYSTEMS .... 102 ANNEX 8: CASE STUDY 1-HURAMO .............................................................................. 104 ANNEX 9: CASE STUDY 2-GLOFOD ............................................................................... 107 ANNEX 10: CASE STUDY 3-MACCO ............................................................................... 110 ANNEX 11: CASE STUDY 4-LOCAL REVENUE MOBILIZATION IN LIRA AND HOIMA MUNICIPALITIES .................................................................................................. 114 vi ACRONYMS A4D Agency for Desired Development ACODE Advocates Coalition for Development and Environment ACFODE Action for Development ATWG Accountability and Transparency Working Group AYDL African Youth Development Link CAO Chief Administrative Officer CBO Community Based Organization CDO Community Development Officer CEEP Community Empowerment for Education Improvement Project CEDO Community Empowerment Development Organization CEGED Centre of Governance and Economic Development CEPA Centre for Policy Analysis CFO Chief Financial Officer CIDI Community Integrated Development Initiatives CIID Criminal Investigations and Intelligence Department CLA Collaborating, Learning, and Adapting CODECA Community Development and Conservation Agency COSASE Committee on Statutory Authorities and State Enterprises CPS Corporate Planning and Strategy CSBAG Civil Society Budget Advocacy Group CSO Civil Society Organization DAI Development Alternatives Inc. DBTA District-Based Technical Assistance DDP District Development Plan DEC District Executive Council DEO District Education Officer DHO District Health Officer DISO District Internal Security Officer DLG District Local Government DOP District Operational Plan DPC District Police Commander DPP Directorate of Public Prosecutions DRP District Resource Pool DRG Democracy, Human Rights and Governance dTS Development and Training Services FGD Focus Group Discussion FINMAP Financial Management and Accountability Programme EQ Evaluation Question ETS Embedded Technical Specialist FGD Focus Group Discussion FINMAP Financial Management and Accountability Programme FOWODE Forum for Women in Democracy FtF Feed the Future FY Fiscal Year vii HEPS Coalition for Health Promotion and Social Development HLG Higher Local Government HURAMO Human Rights Activist Monitoring Organization GAPP Governance, Accountability, Participation and Performance Activity GLOFOD Global Forum for Development GOU Government of Uganda IAD Internal Audit Department IGG Inspector General of Government IP Implementing Partner IPPS Integrated Personnel Performance System IPS Institute of Parliamentary Studies KDFA Kayunga District Farmers Association KII Key Informant Interview LDNGOF Lira District NGO Forum LG Local Government LGAC Local Government Accounts Committee LGDF Local Government Development Framework LGFAR Local Government Financial and Accounting Regulations LGFC Local Government Finance Commission LGPAC Local Government Public Accounts Committee LLG Lower Local Government LRM Local Revenue Mobilization MACCO Mayank Anti-Corruption Coalition MC Municipal Council M&E Monitoring and Evaluation MEC Municipal Executive Council MEL Monitoring, Evaluation and Learning MELP Monitoring, Evaluation and Learning Plan MEP Monitoring & Evaluation Plan MLG Municipal Local Government MOF Ministry of Finance MoFPED Ministry of Finance, Planning and Economic Development MOH Ministry of Health MoLG Ministry of Local Government MP Member of Parliament MSME Micro, Small and Medium Enterprise MTE Mid-term Evaluation NACWOLA National Community of Women Living with HIV/AIDS NAI National Accountability Institution NDP National Development Plan NGO Non-governmental organization NPA National Planning Authority NSA Non-state actor OAG Office of the Auditor General OBT Output Budgeting Tool OVC Orphans and Other Vulnerable Children viii PAC Public Accounts Committee PDU Procurement and Disposal Unit PEMCOM Public Expenditure Management Committee PEPFAR President’s Emergency Plan for AIDS Relief PFM Public Finance Management PLOCA Partner Local Organization Capacity Assessment PPDA Public Procurement and Disposal of Public Assets Authority PS&LG Public Service & Local Government PSO Private Sector Organization RDC Resident District Commissioner RFD Reconciliation for Development RTI Research Triangle Institute SEATINI Southern and Eastern African Trade Information and Negotiations Institute SUGAR Strengthening Uganda’s Anti-Corruption and Accountability Regime SDG Sustainable Development Goal SDS Strengthening Decentralization for Sustainability Program SMC School Management Committee SMS Short Message Service SOW Scope of Work TA Technical Assistance TIU Transparency International in Uganda ToC Theory of Change TOR Terms of Reference ToT Training of Trainers UPE Universal Public Education WVBC Women’s Village Budget Club UDN Uganda Debt Network UGX Uganda Shilling ULGA Uganda Local Government Association UWONET Uganda Women's Network VfM Value for Money WASH Water, Sanitation and Hygiene YDI Youth Development Initiatives in West Nile YLP Youth Livelihood Programme YoGAL Youth in Governance and Leadership ix EXECUTIVE SUMMARY Evaluation Background and Purpose The United States Agency for International Development (USAID) and (since 2015) UK Department for International Development (DFID) funded Governance, Accountability, Participation and Performance (GAPP) Activity is a seven-year (2012-2019) activity with the goal to increase participation, accountability, and local governance for effective service delivery by accomplishing the following results: (1) improve the legal, policy, regulatory, and institutional environment to meet demands for more democratic governance, (2) improve fiscal management and accountability processes in the delivery of services in targeted districts, and (3) increase the capacity of citizens and communities to participate in local governance—voice and accountability. To achieve this goal and its attendant results, GAPP implements a strategy with three primary Activity components: (1) support and strengthen national-level government accountability institutions in their oversight and interaction with local governments (LGs); (2) strengthen LG systems in revenue, procurement, finance and planning; and (3) strengthen civil society organizations (CSOs) and other non￾state actors (NSAs) to improve voice and accountability. GAPP’s theory of change (ToC) is as follows: If Government of Uganda (GOU) and non-government-led efforts are supported to improve democratic governance and accountability, and citizens’ efforts are mobilized to engage and partner with local governments (LGs), then service delivery will be more equitable and efficient. The purpose of the GAPP Mid-Term Evaluation (MTE) is twofold: (1) helping USAID/Uganda learn about which GAPP interventions should be deepened or re-directed in the work planning for subsequent years; and (2) aiding understanding how GAPP supplements efforts of other sectors. Evaluation Methodology The MTE used four primary data collection methods: secondary document review, key informant interviews (KIIs), focus group discussions (FGDs) and a validation workshop. Overall, the MTE team conducted 105 KIIs with 241 persons drawn from the followings stakeholder groups: national accountability institutions (NAIs), national-level and district-level CSOs, LG and lower local government (LLG) technical and elected officials, GAPP staff, DFID staff, and staff at other development programs working in democracy and governance. In addition, the MTE team conducted 16 FGDs involving 132 persons, including 61 males and 71 females who participated in GAPP’s grantee CSOs under GAPP Component 3. In total, the MTE fieldwork covered five regions, 12 districts, five municipal councils, and one municipal division. Findings/Conclusions The GAPP theory of change appears to be empirically valid, albeit on a case-by-case basis. That is, by supporting GOU institutions and officials to improve governance and accountability, and by working simultaneously with civil society to mobilize citizens to engage and partner with LGs, then LG service delivery can be made more equitable and efficient. GAPP’s strategy to target national accountability institutions, core governance functions at the local government and sub-county levels, and civil society constitute critical leverage points within the local governance system, such that ignoring either of them is unlikely to produce the type of systemic change sought by USAID/Uganda. GAPP’s strategy of targeting districts and municipalities and then from there working downward to sub-counties, divisions and town councils is also appropriate. These are the levels of local government with budgets and where x services are delivered and thus where improvements in LG capacity and accountability are likely to produce the biggest improvements in LG service delivery. GAPP’s work with national accountability institutions seems to have influenced the performance of their oversight function in several important ways. However, these outcomes are for the most part tentative in nature and will require a significant amount of follow-up to embed them within the relevant institutions and the LG policy system. GAPP support has also contributed to several improvements in the performance of supported LG units. Local revenue collection has increased, internal conflicts within District Executive Councils and Municipal Executive Councils have reduced and the internal processes in efficiency, procurement, compliance, planning and reporting practices have improved. It is too early to determine whether the above improvements represent sustainable, systemic improvements to the functioning of supported LGs. GAPP’s work with civil society has proven effective in giving voice to citizens, including especially women and youth, to engage in advocacy efforts on their behalf, on behalf of their association members (for private sector organizations), or on behalf of other community members. Notably, many of these positive outcomes occurred in USAID/Uganda’s priority areas of health, education and agricultural production and/or in crosscutting priorities of women and youth. Several GAPP technical approaches are proving effective and relevant for future programming. Engagement with key national accountability institutions is a critical component of any LG strengthening program. Specifically, the tripartite outreach meetings have demonstrated promise for generating meaningful engagement among stakeholders, increasing voice for local citizens, and increasing LG accountability. Although supporting youth parliaments poses significant operational challenges, the demographic trends in Uganda make them, or some other youth engagement/mobilization strategy, a critical component of any citizen engagement strategy within a local governance program. Finally, the Expert Panel has potential to contribute significant value-added to GAPP and other DRG programs, although GAPP has yet to draw on this potentially valuable resource. To date, cross-sectoral collaboration and learning between the USAID/Uganda Democracy, Human Rights and Governance (DRG) team and the health, agriculture, and education teams has been minimal to the point where other technical teams have little idea what GAPP is doing or what its results have been. Recommendations EQ1: Managing Theory of Change and assumptions  In future DRG Activities, target LG units for support using a ‘geo-focusing’ targeting strategy potentially using a variety of targeting strategies.  Consider pairing a system building approach with a district-based technical assistance (DBTA) approach, like that used by Strengthening Decentralization for Sustainability (SDS), which directly targets assistance to strengthen specific health, agricultural or education outcomes. EQ2: Cross-sectoral coordination and collaboration  Undertake a comprehensive diagnosis of the Mission CLA system to identify what is missing, or what has broken down, in the current system and, based on this diagnosis, adjust the system to improve its functioning.  As part of the Mission CLA system, technical teams need to reach agreement on indicators, methods of measurement, and cost allocation for reporting the results of DRG Activities to the other technical teams that go beyond the current standard reporting frameworks. xi  Institutionalize CLA systems within supported communities and consider integrating CLA activities and obligations as part of CSO grant agreements and organizing grantee CSOs into something akin to learning communities. EQ3a: Influence on national accountability institutions  Further engage with the leadership of NAIs and other institutions to strengthen ownership of GAPP interventions and to clear audit backlogs at LGs.  Increase national policy engagement, especially on issues that affect service delivery at LG levels, including LG financing through strategic engagement with centers of power and individuals that make change happen.  Discuss with Parliament to ensure that the tripartite outreach meetings are institutionalized. In addition, combine the tripartite meetings with advocacy efforts of GAPP grantees to ensure that the tripartite outreach meetings are sustained.  GAPP needs to continue strengthening the capacity of Institute of Parliament Studies to ensure that there is continuous orientation and capacity building of members of the oversight committees of Parliament.  Engage in strategic collaborations with the executive arm of government on LG policy issues and public accountability.  Increase funding flexibility to NAIs to enhance innovation, ownership and sustainability of GAPP interventions.  Increase coordination among donors to enhance alignment and learning on best approaches to support / facilitate government institutions. EQ3b: Influence on systemic functioning of local governments  Engage the MoLG as an entry point where possible, such as by lobbying MoLG to develop appropriate enforcement mechanisms or to participate in council orientation.  Consider benchmarking performance across GAPP-supported districts, providing short performance enhancement training to staff in key LG units, supporting the LG statistics function, and incorporating GAPP activities into LG development plans.  Move quickly down to sub-counties to address the numerous challenges with planning, financial management, procurement, and accounting at that level.  Consider strategies to increase resources to enable GAPP to provide the level of service needed in targeted sub-counties and other LLGs. EQ3c, EQ3d & EQ5: Effectiveness of GAPP’s interventions  Revise GAPP’s grant making strategy, consider consolidating the grantees into a smaller number of CSOs with recognized potential for achieving breadth and/or depth of scale along with appropriate grant terms.  Consolidate the grantees into a smaller number of CSOs with recognized potential for achieving systemic change, which in turn implies CSOs with the capability to expand both horizontally across districts and vertically within districts.  Reassess the process for delivering capacity building to CSO grantees, including their capacity and bandwidth to provide required assistance. Consider further including participation in cross-learning CLA activities as a requirement of the grant terms. EQ4: Future programming  Combine the tripartite meetings with advocacy efforts of GAPP grantees and using regional accountability committees to enhance compliance and sustainability. xii  Develop a more ambitious plan using the Expert Panel, including using panel members to advocate on policy issues tapping into their networks for multiple programmatic purposes.  Support the implementation of learning platforms at the regional or district level targeted to LG political and technical officials, private sector stakeholders, and service providers in priority sector. 1 1. INTRODUCTION While decentralization in Uganda is arguably a cornerstone of democratic governance, for many Uganda citizens it has yet to deliver on its promise of effective democratic governance, improved service delivery, increased accountability, and citizen engagement at the local level. In this light, the United States Agency for International Development Mission in Uganda (USAID/Uganda) and the UK Department for International Development (DFID) are supporting the seven-year (2012-2019), $35.5 million Governance, Accountability, Participation and Performance (GAPP) Activity,1 implemented by RTI International with its consortium partners Development Alternatives Inc. (DAI) and Development and Training Services (dTS). The goal of GAPP is to increase participation, accountability and local governance for effective service delivery. The objectives of the Activity are (1) to enhance accountability within local government institutions, (2) to strengthen the Government of Uganda (GOU) accountability institutions, (3) to improve local government governance, and (4) to support civil society organizations (CSOs) and private sector organizations (PSOs). 2. CONTEXT-GAPP OVERVIEW In November 2012 USAID/Uganda (hereafter USAID) awarded a USD 17.7 million contract to RTI International, with its consortium partners DAI and TS, to implement the GAPP Activity in 25 districts.2 In January 2015 the scope of work (SOW) expanded through a partnership with DFID, which doubled funding to USD 35.5 million; increased the length of the Activity from five to seven years; expanded technical breadth to a range of complementary work areas; deepened geographic coverage in existing districts with greater coverage of sub-counties, town councils and municipalities; expanded to ten additional districts3 in northern Uganda and the Bunyoro sub-region; and created and staffed an additional four regional offices bringing the total number of regional offices to five. In addition, the expansion was aimed at enhancing the enabling environment to improve delivery of health services, education and economic growth in target districts. 1 GAPP is technically an Activity, which is defined by USAID (ADS 201.3.4) as follows, “An activity carries out an intervention or set of interventions, typically through an implementing mechanism such as a contract, assistance program, or partnership with another U.S. Government Agency, the partner country government, other donors and development assistance agencies, NGOs and the private sector. It may also be an intervention undertaken directly by USAID staff that contributes to achieving a Project Purpose such as policy dialogue, capacity building services, or coordination with stakeholders.” So as to avoid confusion in this report, the word ‘Activity’ with an uppercase ‘A’ refers to GAPP, while the word ‘activity’ with a lowercase ‘a’ refers to specific activities, or interventions, implemented by GAPP either on its own or by its implementing partners. 2 Agago, Amolatar, Amuru, Apac, Kitgum, Gulu, Lamwo, Lira, Nebbi, Nwoya, Oyam, Pader, Koboko, Arua, Buliisa, Hoima, Masindi, Kibaale, Kalungu, Kayunga, Kiboga, Masaka, Mityana, Mubende, Mukono 3 Adjumani, Moyo, Yumbe, Maracha, Zombo, Kole, Otuke, Alebtong, Dokolo, Kiryandongo 2 Text box 1. GAPP Theory of Change and Underlying Assumptions GAPP seeks to increase participation, accountability and local governance for effective service delivery by accomplishing the following results: 1. Improve legal, policy, regulatory, and institutional environment to meet demands for more democratic governance. 2. Improve fiscal management and accountability processes in the delivery of services in targeted districts. 3. Increase the capacity of citizens and communities to participate in local governance—voice and accountability. GAPP’s theory of change (ToC) is as follows: If Government of Uganda (GOU) and non-government-led efforts are supported to improve democratic governance and accountability, and citizens’ efforts are mobilized to engage and partner with local governments (LGs), then service delivery will be more equitable and efficient. Further underlying GAPP’s ToC are three key assumptions: 1. When accountability is strengthened, and improved, the opportunity for corruption to take place is diminished and governance outcomes in health, education and agricultural systems such as responsiveness, equity and efficient use of resources are affected positively. 2. Closer government, increased citizen monitoring and transparency are expected to characterize decentralization and should thus be associated with a move toward increased accountability and improved service delivery. 3. Supporting the local governments will strengthen government accountability to citizens by involving citizens in monitoring government performance and demanding corrective actions. GAPP is implemented through three major interlinked intervention components corresponding to three broad results areas, as seen in Table 1. Table 1. GAPP Results Areas and Corresponding GAPP Activity Components GAPP Result Area GAPP Component 1) Improved legal, policy, regulatory and institutional environment to meet demands for more democratic governance. 1) Supporting and strengthening national-level government accountability institutions in their oversight and interaction with LGs. 2) Improved fiscal management and accountability processes in the delivery of services in targeted districts. 2) Strengthening LG revenue, procurement, financial, and planning systems. 3) Improved capacity of citizens and communities to participate in local governance. 3) Strengthening CSOs and other NSAs to improve voice and accountability. 3 3. EVALUATION PURPOSE This report presents the findings, conclusions and recommendations emanating out of the GAPP mid￾term evaluation (MTE). The purpose of the GAPP MTE is twofold: (1) to help USAID and DFID learn about which GAPP interventions should be deepened or re-directed in the work planning for subsequent years; and (2) to aid understanding how GAPP supplements efforts of other sectors. The primary users of the MTE findings are USAID Mission staff across the different Mission offices, DFID, RTI and its partners and other USAID and DFID-funded implementing partners (IPs), the Uganda Ministry of Local Government (MOLG), and other national stakeholders with interest in strengthening decentralized governance and accountability systems. (See Annex 1 for a copy of the MTE SOW.) 4. EVALUATION METHODOLOGY This section presents a brief discussion of the methodology used to implement the GAPP MTE. A more detailed presentation of the evaluation methodology can be found in Annex 2. Evaluation Questions This evaluation reviewed GAPP performance in each of GAPP’s three Activity components between December 2012 and July 2016 to answer the five evaluation questions (EQs) listed below. Text box 2. Evaluation Questions Annex 6 contains an Evaluation Matrix that links each of the above five evaluation questions to a set of more detailed sub-questions (which are used to guide the data collection and analysis), the type of evidence that will be used to answer the questions, data sources, data collection instruments, data analysis methods and the types of evidence used to answer each question. EQ1. To what extent has GAPP’s theory of change and underlying assumptions held true? How did GAPP manage any changes in the ToC or assumptions? EQ2. What are we learning about coordination and management of cross-sectoral efforts from GAPP implementation within USAID? EQ3. How well are GAPP interventions influencing the performance of: a) National accountability institutions in their oversight function at both national and with local governments? b) Systemic functioning of local government? c) Citizen participation in local governance processes and service delivery? d) Unblocking voice and accountability channels in local government? EQ4. Which GAPP technical approaches are proving more effective and relevant for future programming? EQ5. How well is CSO advocacy and monitoring of service delivery contributing to improving the quality and effectiveness of service provision at the local governments and lower local governments? 4 Mid-term Evaluation Team The GAPP MTE team consisted of four members, including one international consultant and three local Ugandan consultants. Local evaluation team members were selected for their backgrounds and expertise in areas corresponding to GAPP’s three Activity components: national-level accountability institutions (Component 1), LGs and lower-local governments (LLGs) (Component 2) and district and national CSOs (Component 3). Geographic Coverage Overall, the MTE team conducted fieldwork in five regions, 12 districts, five municipal councils, and one municipal division. The districts visited and stakeholders interviewed during the fieldwork were selected in close consultation with USAID and GAPP management team. Data Collection Methods The GAPP MTE used four primary data collection methods: secondary document review, key informant interviews (KIIs), focus group discussions (FGDs) and a validation workshop. Overall, the MTE team undertook the following data collection activities:  Reviewed 43 documents, both internal and external to the GAPP Activity (see Annex 3 for a bibliography of documents reviewed).  Conducted 105 KIIs with 241 persons, broken down as follows (See Annex 4 for copies of the KII and FGD discussion guides and Annex 5 for a list of persons interviewed): o Component 1: 13 KIIs with 19 persons drawn from parliamentary oversight committees, other national-level accountability institutions (NAIs), and national-level CSOs. o Component 2: 48 KIIs with 123 persons, including 56 LG and LLG technical officers, 55 LG and LLG elected officials, and 12 GAPP embedded technical specialists (ETSs) all together covering 6 districts (Apac, Gulu, Arua, Koboko, Kiryandongo, Masaka), 5 municipalities (Lira, Gulu, Koboko, Hoima, and Masaka) and 1 division (Kimanya/Kyabakuza in Masaka municipality). o Component 3: 28 KIIs with 67 people, including 11 KIIs with 30 key informants at district￾level CSOs, 11 KIIs with 30 key informants at national-level CSOS, and 7 KIIs with LG officials. o Other: 15 KIIs with 31 staff members at GAPP, USAID, DFID and other donors (GIZ), development programs (SUGAR, SDS, MELP), and members of the GAPP Expert Panel.  Conducted 16 FGDs with 132 persons, including 61 males and 71 females in six districts (Arua, Gulu, Lira, Oyam, Masindi, Kayuga).  Conducted a half-day validation workshop on October 18, 2016 with selected Activity stakeholders to disseminate the evaluation findings in a participatory format to increase understanding of the MTE findings and improve their utility with respect to stakeholders’ values and views. Data Analysis Methods The MTE team used content analysis to analyze the qualitative data collected during the fieldwork. Content analysis is a well-established systematic technique for analyzing qualitative data that compresses text into fewer content categories based on explicit rules of coding and then identifies patterns among the codes. Content analysis allows researchers to draw inferences from qualitative data by objectively and systematically identifying specific themes and sub-themes within the data and assessing their relative importance in answering the evaluation questions. 5 Methodological Limitations The evaluation methodology used for the GAPP MTE involves the following methodological limitations. Sampling: Owing to time and resource constraints specified in the MTE SOW, the evaluation covered only 10 of 35 GAPP-supported districts. These districts may or may not provide a fully representative picture of GAPP activities and thus potentially limit the MTE’s generalizability across the Activity and other local governance strengthening programs. Subjective Measurements: The qualitative methodology used in the GAPP MTE means that the evaluation findings are based largely on subjective perceptions of KII and FGD participants. The MTE team, moreover, was not able to verify/validate the numerous anecdotes of successful or unsuccessful GAPP activities related by key informants. Sustainability: This mid-term performance evaluation was done while most interventions were ongoing, which made it difficult for the MTE team to determine whether reported/observed outcomes have endured or are likely to endure once GAPP support ends. Attribution: The qualitative nature of the MTE—which in turn involved small sample sizes and the absence of comparison measurements—means that the evaluation methodology is not able to generate sufficient evidence to attribute reported/observed results to GAPP activities. 5. FINDINGS 5.1 Evaluation Question 1: To what extent has GAPP’s theory of change and underlying assumptions held true? How did GAPP manage any changes in the ToC or assumptions? General Validity of the GAPP Theory of Change GAPP stakeholders uniformly agree that the GAPP ToC and its underlying assumptions are valid, specifically that by supporting GOU institutions and officials to improve governance (including technical and political skills) and accountability; and by working simultaneously with civil society to mobilize citizens to engage and partner with LGs, then LG service delivery will (or can) be more equitable and efficient. The MTE team, moreover, found no evidence indicating that the ToC was invalid either at Activity inception or due to changing environmental or programmatic factors. GAPP itself made no changes to its strategy or approach suggesting that it found the ToC to be valid; tactical adjustments it did make were generally aimed at improving the operationalization of the ToC. GAPP’s three-pronged strategy targeting NAIs, LGs, and civil society is well conceived in that it targets three critical levers influencing the structure and performance of local government, and it covers a reasonably comprehensive set of key actors shaping local government outcomes at the national, local and civil society levels. Furthermore, the numerous examples of improved citizen voice, LG accountability and LG service provision found during the fieldwork (see Sections 4c-4d and Annexes 8-11), which can be plausibly linked to GAPP activities, suggest that TOC and underlying assumptions are empirically valid as well, at least on a case-by-case basis. 6 Validity of the GAPP ToC in the Context of a ‘Systems Approach’ The MTE team sees GAPP as an extension of the Mission’s emphasis on a systems approach to development programming, as found in the following language from USAID’s draft 2017-2021 Country Development Cooperation Strategy (CDCS): “. . . USAID will work to contribute towards the establishment of well-functioning and Ugandan-owned and –led economic, governance and social systems . . . rooted in an understanding of systems thinking, which seeks to look at the ‘whole’ of an issue . . . While systems thinking will govern the way in which interventions are holistically planned and conceptualized, activities and projects will continue to be targeted and focused on concrete results that are intrinsically linked to the bigger picture” (p. 12). GAPP’s approach to achieving systemic change is to work with key institutions and actors within the local governance system to improve the way the system functions. This includes working directly on the supply side with LG technical and political officials, working on the demand side with civil society to increase citizens’ voice and engagement with local government and working at the national level to improve the enabling environment for local governance. The MTE team finds this type of systems approach to be appropriate as evidence gathered during the MTE confirms that problems of local governance are systemic in nature and thus require a systemic approach to address them. With regards to the GAPP’s work with the supply side of LG service provision, its focus on strengthening the core functions of local government is consistent with a systems approach. GAPP has defined these core functions to include: local revenue mobilization (LRM); LG leadership and council development; procurement, internal audit and financial accountability systems; and planning and budgeting processes. These core functions constitute the minimum of technical and political functions at the LG level that must all be in place and functioning well for systemic change to occur. Intervening on the demand side is likewise a critical part of any systems approach, owing to several systemic failures that supply-side interventions alone cannot address. These include pervasive corruption and persistent resource and capacity shortfalls at the LG level. GAPP’s demand-side strategy is thus fully consistent with the CDCS, which notes that there is an “urgent need to support the demand-side of the political system, which includes non-state actors and leaders… These include opening up political space and encouraging deeper political engagement of the citizenry” (p. 48). Finally, the MTE team finds that GAPP’s work with NAIs is also consistent with a systems approach. Again, quoting the CDCS: “Strong systems also have a role in upholding the rule of law… Leadership and good governance ensure that strategic policies and regulatory frameworks exist, are implemented appropriately at all levels and that effective oversight, coalition building and accountability structures are in place to promote positive practices” (p. 55). “Addressing inadequacies at all levels is important, because the entire complex system’s functionality is a pre￾requisite to an effective decentralized government. Failure to take this into consideration was a primary weakness of the STARS DBTA program; despite successfully building health service delivery capacity, limited attention to management left facilities ill-equipped for long-term knowledge management.” (Learning from Evaluations: District Level Service Delivery and Local Government Partnership, p. 3) “Again, focus should be on the core functions of local government. If you focus in one area you may strengthen one leg, but if the rest of the body is rotten, eventually the leg may fall off, even if it has been strengthened.” –Member of Expert Panel 7 In summary, the MTE finds that vigorous engagement at each of the above three levels is necessary to achieve a systemic impact on LG performance. This finding stands in contrast to language in the CDCS, which asserts that “. . . for the longer term, a renewed emphasis on the demand-side of political systems is vital, with only a very limited and selective engagement on the supply-side.” (p. 109). An important caveat of a system approach for the purposes of GAPP and this MTE is that taking a systems approach imposes heavy burdens on the implementing agency in that the systemic failures related to local governance are many and are deeply rooted in the country’s political culture and structure. Addressing these systemic failures requires much time, significant resources and a great deal of patience and perseverance. The infusion of DFID money, along with the requirement that GAPP reaches all sub-counties in each of the 35 districts where it works, suggests that the current strategy to achieve systemic change is to expand Activity coverage as broadly and as deeply as possible, given existing funding and resource constraints. However, the MTE team questions whether the money and resources currently allocated to GAPP are sufficient to achieve systemic change. While the evidence indicates that the ETSs do good work, their ability to cover all 35 districts and related sub-counties (not to mention extending coverage to municipalities, divisions and town councils) and all the key LG officials (both technical and political) within all of those LG units is questionable. Even more questionable is the ETSs’ ability to provide the type of extensive and on-going mentoring and technical assistance required to reinforce and embed messages and skills among LG officials, particularly given high rates of turnover among those officials. In this light, the MTE team agrees with one key informant who observed, “What GAPP is doing right now is working on building blocks . . . To get to this level takes time and a lot of effort.” This is not to say that GAPP cannot generate positive outcomes in terms of LG accountability and service provision in the short- to medium-term (in fact a wealth of anecdotal evidence suggests that it has) but rather that achieving the type of systemic change sought by USAID and DFID will likely only begin to emerge over the longer-term. Thus, on further examination, the strategy of extensive coverage reflected in GAPP’s current SOW appears to be at odds with the aims of the CDCS, particularly given the lessons learned during CDCS 2011-2016 that a “uniform geographic approach will not meet the needs of an integrated strategy.” As explained further, “USAID Uganda will therefore apply a systematic ‘geo-focusing’ approach to its planning, to identify those corridors, districts and specific sites of intervention where USAID’s investments are likely to yield the greatest, most cost-effective impact” (p. 16). Consistency of the GAPP Targeting Strategy and its ToC Another aspect of LG targeting that is relevant to this discussion of the GAPP ToC is at what level of local government GAPP activities should be targeted. One of the lessons learned from the Strengthening Decentralization for Sustainability Program (SDS, GAPP’s predecessor program) was that the benefits “For example if you look at schools, the problem is that the entire school system or education system is broken. If you improve the LGPAC and make it more accountable and transparent, will this make the schools work better? It may result in the improvement of some schools, but the systematic problems of the education system still remain. This is the same sort of thing in other areas; the problems are embedded, institutionalized and systematic . . . The strategy should involve systems thinking, that is to think of it as a system and how to intervene to fix the system. You cannot fix one aspect of the system and expect the system to change.” –Member of Expert Panel 8 derived from working at one level of local government do not create spillover effects, either across that level or at other levels of local government. Thus, to generate improved performance and service outcomes at any specific LG level, it requires direct interventions at that level. A similar conclusion was reached in the assessment of the USAID-funded Strengthening TB and HIV/ AIDS Responses (STAR) District-Based Technical Assistance (DBTA) Program. The GAPP MTE similarly finds that focusing solely, or predominately, at the higher LG (HLG) level (districts and municipalities) or lower LG (LLG) levels (sub-counties, divisions and town councils) is unlikely to produce the type of systemic change USAID and DFID seek. As stated by one key informant, “The only way to make interventions work within a district is to do so vertically. You cannot work at the district level without working at the sub-county level. If you address only one level, then the issues will remain in the other level, and they all have to work to be able to even begin to address the systemic problems.” GAPP’s design has internalized this lesson by initially targeting its interventions on HLG units and functions to enhance accountability and service provision at that level. It is anticipated that GAPP’s focus will increasingly turn to LLGs—including municipal divisions and town councils but particularly sub-counties—to complete the loop. Key informants interviewed for the MTE generally agreed that GAPP’s decision to focus its interventions at the district and sub-county levels was appropriate given that these are the levels of local government with budgets and where public services are delivered. At the same time, however, LLGs are chronically understaffed and LLG staff lack the technical abilities relative to their HLG counterparts, a problem that is greatly exacerbated by the proliferation of districts across the country. (HLGs also suffer from the same types of limitations as the LLGs, albeit generally to a lesser degree.) In this case, it makes sense to target assistance initially to the HLGs, where the absorptive capacity for technical assistance (TA) is greater and then work downward from there using some combination of Activity staff (e.g., ETSs), local resource persons (e.g., district resource pools) and consultants. In this context, it is important to note that starting in FY 2016/2017, development grants from the central government will be sent directly to sub-counties, while the main role of districts will be to monitor, supervise and provide technical guidance and support.4 This development further supports GAPP’s approach of starting with the HLGs and working downward, but it also highlights the need for GAPP to move down quickly to the LLGs to help them address the capacity and governance challenges they face so that they can manage the larger volume of resources that will soon be flowing their way. Changes in GAPP Design or Strategy There have been few changes of note in the GAPP design or strategy since Activity inception. Two innovations that merit discussion is the creation of the ETSs and the DRP, both of which were suggested by DFID in association with its joint funding of GAPP. 4 Previously the central government transferred money to the local governments with 52 conditional and unconditional transfer and grants. These are to be consolidated into 12 grants. This will leave the local governments with more flexibility related to discretionary spending. This is the first stage of a three-stage process, so the local governments that perform well during the earlier stages and meet the standards set by the national government will receive more money and enjoy more discretionary spending. 9 Embedded Technical Specialists: Prior to the creation of the ETSs, GAPP utilized consultants to deliver much of its TA to LG officials under Component 2. The MTE team sees the ETS delivery model as a significant improvement over the consultant delivery model. Basing three full-time, dedicated ETSs, each with a different technical specialty, in strategic locations in each of GAPP’s five regions of operation allows GAPP to improve the breadth and depth, continuity, coordination, and (assumed) cost￾effectiveness of LG technical assistance relative to the consultant delivery model. To the MTE team’s knowledge, GAPP is the only program of this nature operating in Uganda that utilizes something akin to the ETS model. District Resource Pool: The DRP is a cohort of district level experts in different fields (e.g. financial management, accounting, procurement, planning, etc.) who are trained by the GAPP ETSs to provide periodic technical guidance, backstopping and support to their LLG counterparts. The creation of the DRP internalizes one of the lessons learned from the SDS and STARs evaluation, specifically that “further mechanisms can be set in place for ensuring knowledge is transferred. In the SDS activity, a lack of orientation for new staff kept capacity building to the individual, meaning knowledge was lost when trained-staff left. One solution to this may be seen in STARs, where facility-based mentors in a sort of ‘train-the-trainer’ model helped transfer skills to new recruits” (MELP, p. 4). Consistent with this objective, the DRP is aimed at creating a cascading effect by which technical expertise at the district level is imparted to LLG levels to help address significant implementation issues there. Under GAPP’s implementation strategy, the ETSs initially are to focus on the HLGs then subsequently cascade down to the LLGs using the DRPs under the guidance of the ETSs. Presumably, the ETSs’ coverage of the 35 districts would not be a problem if a strong DRP cohort could be raised, trained and effectively deployed. Additionally, GAPP maintains a resource pool of national consultants, which it can deploy to complement the work of the ETSs and DRPs, as necessary. Thus, the DRP is a potentially valuable innovation, but there exist certain concerns about its viability. The way it works now is that, although the DRP members are recruited to work in any district where they may be needed, and were indeed used widely during local council orientation, in practice few districts are willing to release their staff to go to another district due to their own acute staff shortages. Another concern has to do with the incentives for district experts to participate actively in the DRP, particularly within a culture of monetized public service. Per GAPP, the DRP members are motivated by the payment of a per diem, which can be a non-trivial amount of income for them given the low salaries paid to LG officials. Another motivation is that they can use their work on the DRP to pad their resumes to find other employment outside of local government. A final concern, which is closely linked to the first, is the sustainability of the DRP once GAPP ends. If a primary motivation for participating in the DRP is the payment of a per diem, then once the per diem ends, so will the motivation. Added to this, the departure of DRP members from local government service, aided by their membership in the DRP, builds within the DRP structure the seeds for its own obsolescence. GAPP is upfront in acknowledging these concerns, which it concedes threaten the sustainability of the DRP model. Thus, rather than viewing the DRP as a sustainable strategy for building capacity at the LLG level, GAPP sees the DRP as a kind of “reserve army, which kicks in when it is needed” to be leveraged by other DRG programs operating within those districts later. 10 5.2 Evaluation Question 2: What are we learning about coordination and management of cross-sectoral efforts from GAPP implementation within USAID and DFID? CDCS 2017-2021 maintains the commitment in CDCS 2011-2016 to focus on, among other things, health, education, agricultural development and accountable, effective democratic governance. CDCS 2017-2021, however, goes beyond CDCS 2011-2016 in its commitment to integrate programming across the above priority areas. This commitment “reflects the recognition that the development challenges addressed through CDCS 2017-2021are complex, in the sense that they operate within an environment where multiple actors and systems . . . have a significant role in creating, influencing and/or potentially resolving those development challenges. The challenges may therefore only be effectively and sustainably addressed through engagement with these systems” (p. 7). In more practical terms, this means that future Mission programming will take “a cross-cutting and interconnected approach to all projects and activities, including those operating within set funding streams . . .” (p. 7). GAPP was designed with such integration in mind. As described in its project appraisal document, GAPP was intended to be a holistic Activity utilizing different funding mechanisms from other technical areas within the Mission. Learning was to be embedded in all its activities that informed the Mission and other technical teams on the issues involved in integrating DRG programming in other Mission and DFID priority areas, particularly health, education and (for USAID) agricultural production. GAPP’s current strategy is to promote integration via its grant making activities under Component 3 through which it has supported several national and district-level CSOs working in the above priority areas. GAPP’s work with these CSOs would presumably have created a good deal of learning about how civil society can be mobilized, within the context of DRG interventions, to generate outcomes of interest for these other priority sectors. Unfortunately, this has not happened. Rather, evidence gathered during the MTE indicates that interaction between GAPP and the health, education and agricultural technical teams in the Mission has been minimal. GAPP has recently begun to engage more proactively with the agricultural team, including inviting team members to advise its grant applicants, and reaching agreement on three indicators that GAPP will report to the agriculture team over the remaining life of the Activity. However, GAPP has yet to engage productively with the health or education teams, and vice versa. The lack of engagement between GAPP and the other Mission technical teams has produced two results. The first result is that none of the other three technical teams has much idea what GAPP is doing or what kind of results it is producing. The second result (which may also motivate the lack of engagement) is a skepticism among the health and agriculture teams that a ‘systems building’ approach, which focuses on strengthening governance systems as opposed to focusing on directly strengthening specific public services, will generate results of interest to them. One source of this skepticism is the indirect causal pathway that systems strengthening activities take to produce health and agricultural outcomes. This indirect causal pathway stands in contrast with the more direct causal pathways that are involved in providing direct support to service providers (e.g. health center staff or agriculture extension providers) or to beneficiaries (e.g., pregnant and lactating women or smallholder farmers), such as what occurred in SDS. Health team members understand and see value in the SDS approach, because it targets grants directly at developing the capacity of districts to provide health related services, which in turn facilitates the measurement of results. They are skeptical that GAPP’s systems building approach can achieve similar results. Take LRM, for example, with so much of the central grant money earmarked, and with such a 11 small local revenue base, they are skeptical that increasing local revenue sources will be sufficient to improve health services, particularly when the revenue is spread across all other service areas. In the case of agricultural production, the causal pathway from DRG support to improve on-field agricultural production is more indirect than with health and is influenced by numerous factors over which DRG programs have little control (e.g., input usage, production methods, post-harvest methods, weather, market access, etc.). Yet, per a member of the Mission agricultural team, this is the bottom line for agricultural production support: “Feed the Future funding assumes that strengthening local government in turn strengthens the organizations and entities of local government leading to an improvement in smallholder farmer on-farm production, sales, income and so forth. Therefore, Feed the Future is funding GAPP.” Neither the health nor agricultural teams believe that DRG/GAPP has made the case, or “connected the dots,’ demonstrating how governance systems strengthening contributes to improved outcomes in health and agricultural production. Abetting this are the different reporting frameworks under which GAPP/DRG and the other technical teams operate and a breakdown in the Mission’s Collaborating, Learning and Adapting (CLA) system,5 each of each is discussed below. The PEPFAR Next Generation and Feed the Future indicators are the primary reporting frameworks for projects receiving PEPFAR and FtF funding. Scrolling down the list of indicators in each framework, one finds many indicators along the lines of “Prevalence of children 6-23 months who consume targeted nutrient-rich value chain commodities” or “Farmer's gross margin per hectare, per animal, per cage obtained with USG assistance.” One can immediately see that GAPP would have difficulty reporting on such indicators, and even if it could, the causal linkages between GAPP activities and the reported results would be tenuous at best. On the other hand, GAPP’s Monitoring, Evaluation and Learning (MEL) Plan consists of indicators such as “Percentage of LGs with a functional LG Public Accounts Committee” or “Percentage of local governments receiving GAPP assistance that improve performance.” Such results, as useful as they might be for GAPP, are unlikely to be useful at PEPFAR or FtF. 5 In this context, a CLA system refers to a Mission-wide system for generating, disseminating and acting on learning within the Mission as a whole, within sectoral teams and across sectoral teams. “In DC the folks ask: ‘there are so many different funding mechanisms and types of project interventions, so why do you give family planning money to GAPP? How is GAPP improving family planning?’ With so many other intervention strategies, which can demonstrate direct linkage to health outcomes, GAPP can only point to indirect outcomes, and so far, it has not been able to demonstrate them.” –Member of Health Team “There are very stringent guidelines in Washington, DC for budgets, and there is greater scrutiny. The result is that it has become less flexible in where it allocates its money. It is not a strong case to say that a project strengthens local government in general and down the road we will see benefits in improved service provision. In terms of PEPFAR money, anything that is not direct service delivery will be hard to get money. GAPP initially got $400,000 from Maternal and Child Health, which was then increased to $800,000. We are still wondering what the $400,000 was used for and what it produced.” –Member of Health Team 12 Indicators in the PEPFAR and FtF reporting frameworks that GAPP might conceivably report on—such as “Percentage of health facilities with no stock outs of new sterile syringes in the prior 6 months” or “Number of for-profit private enterprises, producers organizations . . . receiving USG food security related organizational development assistance”—pose two additional issues. First, the scale of GAPP activities in these areas, and thus their outcomes, are likely to be small thereby contributing little to achieved results. Second, indicators counting outputs, such as the number of producer associations assisted, say nothing about how GAPP is contributing to outcomes in other priority areas. This latter issue can be seen in the three FtF indicators that GAPP agreed to report to the agriculture team: “Number of food security private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and community-based organizations receiving assistance,” “Number of micro, small and medium enterprises, including farmers, receiving business development services from assisted sources,” and “Number of policies/recommendations/administrative procedures in each of the 4 stages of development as a result of GAPP assistance.” One might reasonably ask which of these indicators is helping GAPP to “connect the dots.” The different technical teams in the Mission are, understandably, tethered to the reporting frameworks associated with their funding streams; however, this complicates cross-sectoral learning in the Mission and makes it difficult for GAPP (or any other systems building DRG Activity) to comply with the reporting frameworks in a way that both makes sense for GAPP and satisfies the other teams’ standards of evidence. This in turn raises another issue related to the public goods aspect of cross-sectoral learning. If learning from GAPP’s performance (and by extension other DRG programs) serves a public goods function by informing programming decisions across the Mission (and at DFID as well), how should the burden for generating this information be shared among the benefitting parties? Public goods theory says that the cost of generating public goods should be shared across those benefitting from the good. In this case, the costs include both monetary and/or non-monetary costs, including, in the latter case, time and effort to collaborate proactively for joint learning and action, for example via engagement by members of the respective technical teams and/or Activity contracting officer representatives. While contributing cross-sectoral funding for program activities is an important element of this cost sharing, one lesson learned from GAPP’s experience is that absent a pro-active commitment by all parties to engage with each other, together with a learning system that facilitates this engagement, providing funding alone is unlikely to generate the level of cross-sectoral learning and action sought. The Mission CLA system was developed by USAID with precisely this goal in mind to “ensure that the CDCS works as a ‘living strategy,’ providing guidance and reference points not only for implementation but also for learning and course correction as needed . . . [by] facilitating collaboration internally and with external stakeholders . . . to inform funding allocations, program design and project management . . .” The evidence gathered during the MTE, however, indicates that the CLA system at the Mission is not generating the type of cross-sectoral learning or collaboration necessary to inform adaptations for the specific purpose of integrating health, agricultural production or education into DRG programming, and vice versa. (The MTE did not generate findings to determine how well the Mission CLA system is working in other areas.) Not only did the MTE team note a lack of internal collaboration, learning and adapting within the Mission but it also noted a limited degree of collaboration among the USAID, DFID or other donor￾funded DRG programs operating in the country, including GAPP, SDS, and Strengthening Uganda’s Anti￾Corruption and Accountability Regime program (SUGAR). In the eight districts where GAPP and SDS overlapped (Amuru, Dokolo, Gulu, Lira, Kayunga, Mityana, Pader and Oyam), there was not a significant 13 attempt to generate cross-program learning, even though the two programs were aiming at largely the same results. When asked to describe the ways in which they collaborated with GAPP, several key informants at other development programs could cite different procedural or operational forms of collaboration with GAPP, but they were unable, by and large, to cite specific instances in which they collaborated with GAPP to generate or act on joint learning. This type of finding is not unique to GAPP. The SDS evaluation also looked at CLA and found that “although the CLA concept was often mentioned in the program documents . . . there is limited evidence that this formed an integral part of the SDS program approach” (p. 36). Where collaboration efforts did take place between GAPP and other programs, they were (again citing the SDS evaluation), “helpful, but they were more of dissemination rather than reflection” (p. 36). GAPP also has its own internal CLA system, which includes several processes to capture and disseminate learning. GAPP refers to these as ‘learning moments. Learning moments in turn include events dedicated primarily toward external and internal dissemination and learning. The former consist of experience sharing events and ‘learning markets,’ in which external stakeholders are invited to join in participatory discussions and joint learning on topics of relevance. The latter consist of such things as post-activity reviews, semi-annual and annual reviews, evaluations, staff activity updates, quarterly review meetings and portfolio reviews. These internal learning moments have led to several internal course changes and adaptations (see text box below). Overall, GAPP has demonstrated a good degree of flexibility in adapting its intervention strategy, within the limits allowed by its SOW. Its internal CLA system has been an important contributor to this process. Text box 3. Lessons Learned Actions taken by GAPP in response to learning moments:  GAPP identified an opportunity to conclude all parliamentary business with a particular audit report if the tripartite outreach meeting for that report was turned into a parliamentary hearing. It changed the tripartite meetings to achieve this purpose.  GAPP learned that doing mentoring without first establishing the relevant legal framework and standards made mentoring more challenging than necessary. It institutionalized the practice of involving all key stakeholders in a training workshop prior to commencing mentoring activities.  GAPP learned that new political leaders needed more concerted mentoring during their initial days in office. It instituted a mentoring program for all DECs and MECs during their first 100 days in office.  GAPP used learnings from semi-annual and annual reviews to (1) scale-down interventions planned for a given FY to reduce the number of carry-over activities, (2) redesign its NSA capacity building strategy, and (3) change from a training approach to mentorship for supported LGs. Given that GAPP is generating its own learning, it would also seem to have a role to play in the Mission’s CLA system. Ideally, learning generated by the GAPP internal CLA team—including performance information related to its MEL Plan indictors and GAPP ‘learning moments’—should be ‘uploaded’ into the Mission CLA system and disseminated within the Mission (and presumably also DFID), along with the other learning being generated by the CLA system. However, this does not appear to be happening. 14 5.3 Evaluation Question 3a: How well are GAPP interventions influencing the performance of national accountability institutions (NAIs) in their oversight function at both national and local governments? The MTE team addressed this question from the perspective of GAPP’s Component 1 objective of “improved legal, policy, regulatory, and institutional environment to meet demands for more democratic governance.” Component 1 programming is structured around two themes: (1) strengthening NAIs, processes, and policy in support of LG accountability and (2) carrying out strategic national policy advocacy for improved LG financing, accountability, participation, and service delivery. Theme 1 Activities and Results Under Theme 1, GAPP worked primarily to strengthen the capacity of three parliamentary oversight committees—Public Accounts Committee (PAC), Local Government Accounts Committee (LGAC), Committee on Statutory Authorities and State Enterprises (COSASE)—the Office of the Auditor General (OAG), and the Public Procurement and Disposal of Assets Authority (PPDA). Text box 4. Theme 1 Activities Activities implemented under Theme 1 include the following:  Supported the Centre for Policy Analysis (CEPA) to assess parliamentary oversight committees focusing on obstacles to timely consideration of OAG reports by those committees.  Supported the Institute for Parliamentary Studies (IPS) to orient oversight committee members and staff on their roles and responsibilities.  Supported parliamentary oversight committees OAG, and PPDA to conduct tripartite outreach meetings in 23 district local governments.  Supported PPDA to carry out 56 procurement audits in 34 local government units6 covering 880 contracts Fiscal Year (FY) 2013 valued at UGX 20 billion, 1,033 contracts in FY 2014 valued at UGX 96 billion, and 800 contracts in FY 2015 valued at UGX 102 billion.  Supported the National Audit of UK to strengthen the OAG’s general and Value for Money (VfM) auditing capacity, particularly in maternal health and universal primary education. Further supported PAC to validate the findings of the OAG general and VfM audits. 6 Agago, Amolatar, Amuru, Apac, Arua, Kitgum, Gulu, Lamwo, Lira, Nebbi, Nwoya, Oyam, Pader, Koboko, Yumbe, Maracha, Kole, Otuke, Alebtong, Dokolo, Buliisa, Hoima, Kibaale, Masindi, Kiryandongo, Kayunga, Kiboga, Masaka, Mityana, Mubende, Mukono DLGs; and Mukono & Masindi Municipalities. 15 Assessment of Bottlenecks to Expeditious Consideration of Audit Reports Because of the sensitivity of CEPA assessment findings, many were not well received by key offices in Parliament, thereby reducing the possibility that the recommendations would be adopted. To ensure that there was buy-in on the part of Parliament, Parliament Directorate of Legal and Legislative Affairs scrutinized the language of the report before its final publication. From the study, CEPA produced a paper on “Policy and Practices Issues for Reform: Strengthening the Oversight Functions of Parliament,” which recommended the production of a handbook providing guidance to Parliament on its oversight committees on their procedures function. GAPP subsequently supported the production of the “Hand￾Book on Procedures of Accountability Committees.” The handbook is currently awaiting validation/adoption by the Parliament oversight/newly formed accountability committees prior to its dissemination. Orientation of Parliamentary Oversight Committee Members and Staff GAPP’s work with the parliamentary oversight committees has helped contribute to several outcomes suggestive of improving capacity within those committees to undertake their statutory roles. Due in part to the structured orientation and capacity building provided by GAPP, some improvements in the performance of parliamentary oversight committees have been noted. Text box 5. Outcomes of PAC Activities Sample of outcomes of PAC activities:  The COSASE and PAC are focusing on key outstanding issues rather than considering every query.  All the committees are enforcing rules that discourage diversion of committee business to other issues outside the OAG report.  The LGAC is splitting the committee into sub-committees to consider separate business, to increase the volume of work considered.  Committees are limiting the number of witnesses to be heard to a small sample of only key witnesses.  Committees are categorizing audit reports into audit opinions, and prioritizing consideration accordingly, starting with disclaimers and qualified opinions. This approach ensures that more critical issues receive priority consideration and action as a best practice.  The LGAC is working with up-to-date reports now resulting in a reduction in the backlog of OAG reports. Tripartite Outreach Meetings The major objective of the tripartite outreach meetings is to enable the LGs, communities, private sector organizations (PSOs) and CSOs in the districts to access and better understand the information 16 and data generated by OAG and PPDA audit reports,7 thereby empowering them to monitor and ensure accountability and service delivery. Text box 6. Actions Generated by Tripartite Outreach The tripartite outreach meetings generated several district-specific actions:  The Kiboga District Contracts Committee wrote to PPDA forwarding a list of contractors to be blacklisted, including a company that benefitted from five contract awards worth UGX 1.045 billion but that delivered poor quality work.  Oyam District established, by council resolution, an LGPAC on May 28, 2013.  Pader District also established an LGPAC by council resolution on June 21, 2013, and at the same council meeting, established a District Service Commission.  District interest in strengthening the LGPA also increased. For example, a letter was received from Pader District requesting training support for the newly appointed LGPAC members.  In Apac, the district council convened an extraordinary meeting to create a LGPAC. Further, the Resident District Commissioner (RDC), working together with the area members of Parliament, arrested one contractor and Sub-County Chief for abandoning his work, despite having received full payment in anticipation of its completion. Overall, the tripartite outreach meetings greatly facilitated and complemented the work of Parliament, OAG and PPDA to disseminate their reports and obtain community participation and feedback on LG financial accountability and service delivery. The interaction between LGAC and LGPACs was, by all accounts, very helpful in terms of enhancing learning and sharing and providing support by LGAC to LGPACs. The tripartite meetings, moreover, increased interaction between Parliament and other accountability institutions, especially OAG, and increased citizen participation on accountability issues. The LGAC was pleased with the tripartite meetings to the extent that it has stated a desire to continue with it once GAPP ends. 7 The reports discussed included: Auditor General’s Reports for FY 2012/13 and 2013/14 and PPDA reports of 2008/09, 2011/12, 2012/13 and 2013/14. 17 Support to the PPDA The procurement audits generated three important results within participating LGs. First, they highlighted to LGs the procurement gaps and weaknesses, which the LGs can now address. Second, the capacity building/mentoring approach adopted by the PPDA provided an opportunity for the LGs to improve their systems and practices as the audit itself was being conducted. Third, PPDA engaged the LGs on corrective action when it presented the audit report. The audit results indicated an overall improvement in procurement practices among the supported districts. The number of LGs receiving a ‘satisfactory’ or ‘highly satisfactory’ rating increased from four in 2012/13 to 23 in 2014/15 (see Figure 1). Per PPDA,8 several factors have plausibly contributed to the improvement in procurement outcomes. These include: the involvement of the contracts committees in approval of solicitation documents, procurement methods, shortlisting bidders and evaluation committees and evaluation reports; improved record keeping; adherence to procurement plans; following the procurement and disposal procedures in accordance with PPDA law; and independence of functions and powers of key procurement stakeholders. The support provided by GAPP enabled PPDA to conduct annual audits of these entities and provide advice on how best they could improve their procurement processes. Consequently, the submission of quarterly reports to the PPDA by GAPP-supported LGs has also improved, both in terms of regularity and quality. 8 PPDA report on Procurement and Disposal Audit Reports for FY 2012/13, 2013/14 and 2014/15 in USAID/GAPP-supported areas “The tripartite meetings were very good since they involved the local population. The meetings are used by MPs to build the capacity and confidence of the LGPACs. This is very critical for ensuring effective accountability at local levels.” –CPS “Through the tripartite outreach, citizens were able to discuss and comment on audit reports and provide information on financial accountability. This approach used by GAPP, where their committee went to the ground (rather than waiting to receive reports in Parliament) was very good; it enhanced participation of the citizens in discussing the accountability of local governments.” –LGAC “After the first outreaches to local governments, we asked Parliament to allow us to use this approach as a model of operations for the LGAC. Currently, the LGAC has embraced this approach and the committee has taken this approach as a method for its work. When I was handing over to the current committee, I encouraged my successor to embrace this outreach. Even without GAPP support, this will continue.” –LGAC 18 Figure 1. Risk Rating of Local Governments Source: PPDA report on Procurement and Disposal Audit Reports for FY 2012/13, 2013/14 and 2014/15 in USAID/GAPP-supported areas The procurement audits initially involved out-sourcing auditors (consultants) to supplement the PPDA staff to undertake audits. However, after consultations with PPDA, the annual reflection exercise for FY2015 and an expert panel meeting held in September 2015, GAPP changed its strategy to support the PPDA staff to undertake procurement audits in the GAPP-supported districts themselves. This change in approach produced a cost savings of around UGX 200 million from UGX 300 million to about UGX 100 million. Support to the OAG OAG credited GAPP support for helping it improve its auditing function, especially with regards to conducting public debt and value for money (VfM) audits and reducing audit backlogs. PAC similarly credited GAPP with helping build the capacity of its members to review and validate VfM findings prior to the finalization of the audit report. Inter-Agency Collaboration The inter-agency meetings supported by GAPP through PAC were designed to (1) eliminate possible duplications by enabling members to mutually support each other through proper coordination, (2) encourage members’ action on audit findings through criminal or administrative actions by government, and (3) generate policy recommendations for improved accountability. Primary recommendations emerging from this meeting include the following:  Allow the issuance of a treasury memorandum based on a vote or issue, rather than on the entire financial report.  Truncate the accountability process by having IGG and CID take on some matters that are before Parliament. 19  Fast track a law to guide the recovery of public assets and funds lost from persons found guilty in cases of corruption. However, during the GAPP annual review and reflection exercise in 2014, GAPP realized that this activity was not achieving the planned objectives, and thus it decided to stop supporting it. GAPP further agreed to support the SUGAR technical advisory facility to take on this activity considering its extensive interaction with the relevant institutions. Theme 2 Activities and Results Theme 2 under component 1 aimed at carrying out strategic national policy advocacy for improved financing, accountability, participation and service delivery. Under theme 2, GAPP supported the Local Governance Finance Committees and the Local Government association, provided grants to national level CSOs and provided policy reform proposals to Public Service and Local Government (PS&LG). Text box 7. Theme 2 Activities Activities implemented under Theme 2 include the following:  Supported LGFC to produce a report establishing the relevant local governance policy issues affecting LG financing, participation, accountability and service delivery.  Supported the Uganda Local Government Association (ULGA) to champion LG policy dialogue and reform.  Provided grants to five national-level CSOs (CSBAG, FOWODE, ACFODE, CEPA, and ACODE) to engage in advocacy on issues that affect local-level service provision.  Presented the following four policy reform proposals to the PS&LG during consideration of the 2014 Local Government Bill: (1) Elevate the head of the district Internal Audit Department (IAD) to the level of Chief Financial Officer (CFO), (2) create a dedicated unit within the CFO’s department to manage local revenue in line with the functions specified in Section 80 of the Local Government Act, (3) establish minimum academic qualification for district and municipal councilors, and (4) improve the functionality of LGPACs by making them part of the council committees. Policy Study Report Six hundred copies of the Policy Study Report were disseminated to national and local government stakeholders, while the report findings were validated in a high-level meeting in Kampala presided over by the Minister of State for Local Governments and attended by LG officials from Arua, Nwoya, Hoima, and Kiboga districts. Through this interaction, GAPP established a partnership with PS&LG to provide leadership in future discussions around policy implementation. LG Policy Dialogue and Reform From the Policy Study Report, ULGA identified eight advocacy strategy issues and three points of specific interest for targeted action policy dialogue and reform. These 11 policies and their status are shown in Table 2. The negotiation, redesign, and approval of GAPP’s grant to ULGA took longer than 20 anticipated, being finally completed in January 2016. This delay has put ULGA’s policy advocacy efforts behind schedule, although GAPP and ULGA anticipate that the grant framework now in place will enable ULGA to build its capacity over time to advocate for LGs at the national level. Table 2. Issues Targeted by ULGA for Higher-Level Policy Dialogue and Reform # Owner Issue Description Issue Source Issue Type Theme/ Objective Approach Policy Development Stage 1 ULGA Target 1A: 2013 JARD resolution implemented to ensure increment of percentage of total transfers to Local Governments from the National Budget from 18.8% to 38% Policy Study +ULGA Policy LG Financing Advocacy and Petitions from LGs Debate 2 ULGA Target 1B: 1% VAT transferred to Local Governments ULGA Policy LG Financing Advocacy to affect the 1% transfer Debate 3 ULGA Target 1C: Local Governments allowed to engage in the collecting and retaining levies from fisheries and forestry products within their legislative boundaries (Source: PS) Policy Study Policy LG Financing Advocacy for a retention policy Debate 4 ULGA Target 1D: Need to create a dedicated LRM Unit in the office of the CFO Comp 2 Activity Policy LG Financing Advocacy Presen ted 5 ULGA Target 1E: Increased financial independence of the District Department of Internal Audit and the Local Government Public Accounts Committee through the institution of a separate Accountability Vote Comp 2 Activity Policy LG Financing Advocacy for policy change Presen ted 6 ULGA Target 1F: Revised mechanism of conducting the Sector Negotiations, as well as implementation and enforcement of the signed Agreements Policy Study Policy LG Financing Advocacy to develop formal guidelines for conditional grants sector negotiations Debate 7 ULGA Target 1G: National LED policy decentralized as a strategy to promote local development and investment initiated by local governments themselves Policy Study+ ULGA Policy LG Financing Advocacy to implement LED policy Debate 8 ULGA Target 2A: The scale of District Internal Auditors lifted from scale U2 to U1E Comp 2 Activity Policy LG Account Advocacy to revise policy Present ed 9 ULGA Target 2B: The minimum required capacity needs and skills for local government councilors to effectively perform their mandated roles and responsibilities identified and agreed upon Policy Study Policy LG Acc’bility Advocacy and Petitions from LGs Present ed 10 ULGA Target 2C: ULGA Charter of Ethics and Accountability translated into a Statutory Instrument as a mechanism of strengthening Ethical and Accountable code of Conduct within all districts and lower local governments ULGA Policy LG Acc’bility Advocacy to create a Statutory Instrument Under Prepara tion 11 ULGA Target 2D: Establishment of a Proactive and Preventive Conflict Management Strategy to promote improved local service delivery and accountability at Local Government level ULGA Policy LG Acc’bility Advocacy Under Prepara tion 21 CSO Advocacy Grantee CSOs successfully engaged with the Executive and Parliament to enhance budget proposals and influence policy outcomes in health, education and agriculture budgets. Twenty-nine pro-poor and gender sensitive CSO alternative budget proposals were adopted, including 14 during FY 2015 and FY 2016. In the policy area, key results include the rejection of the proposed taxes on agricultural inputs in FY 2015; the restructuring of the National Agricultural Advisory Services to institute a single spine extension system; and an increase in the universal primary education (UPE) capitation grant per child from UGX 7000 to UGX 10,000. CSO advocacy has further led to enhanced government recognition of civil society in the budgeting processes. For instance, CSOs have been co-opted on various government committees, such as Ministry of Finance (MOF) steering committees on the Budget Transparency Initiative and Budget Monitoring Unit, the Public Expenditure Management Committee and the Works, Energy and Accountability Sector Working Group. Grantee CSOs were also invited to the MOF quarterly budget release press conference as well as to the regional budget consultative workshops for all LGs and municipalities. These spaces were previously unchartered territories for CSOs. Policy Reform Proposals Ultimately, PS&LG did not include any of the above four policy reforms in its final bill; however, GAPP made inroads in presenting itself as a resource that the committee can rely on for technical input concerning LGs. For example, the initial presentation included only three policy reforms, but the committee made a specific request to GAPP to provide technical advice on the fourth policy reform and extended the time for GAPP to submit a technical response to its questions. 5.4 Evaluation Question 3b: How well are GAPP interventions influencing the performance of systemic functioning of local government? Under Component 2, GAPP’s technical assistance to LGs thus far has overwhelmingly been targeted at district and municipal levels and has been focused on the following four core functions of local government: (1) LRM; (2) LG leadership and council development; (3) procurement, internal audit and financial accountability systems; and (4) planning and budgeting processes. Local Revenue Mobilization Text box 8. Local Revenue Mobilization Activities LRM support activities include the following:  Organized an LRM workshop in Hoima in January 2016 to discuss key issues in LRM.  Supported the LGFC to provide municipalities with local revenue management software.  ETS worked with municipal councils to establish potential revenue and taxpayer databases, update tax registers, perform tax assessments, privatize collection of certain taxes and set realistic reserve prices. 22 GAPP LRM support has led in several cases to enhanced revenue generation, enabling the municipalities to allocate more resources to areas that have no central grants, such as administration, operation and maintenance, LGPAC, internal audit and procurement. For example, with GAPP support, Koboko Municipal Council has adopted several strategies to enhance local revenue mobilization, including revenue source identification, taxpayer enumeration and assessment, community dialogues involving political leaders and annual taxpayer education. The latter provides opportunities for taxpayers to explain their challenges as a way of reducing enforcement costs. Key informants in Koboko attribute these strategies for helping to expand the municipal council’s tax revenue from UGX 652 million in FY 2013 to UGX 699 million in FY 2015. Figure 2 shows the changes in local revenue mobilization across 30 LLGs spanning 12 districts in which GAPP is working. As seen there, total revenues collected increased by 37.5% from UGX 8.3 billion in 2011/2012 to UGX 11.4 billion in 2015/2016 (provisional). This is equal to an average increase of UGX 103.9 million per LLG.9 Figure 2. Changes in Local Revenue Mobilization in GAPP-Supported LLGs during FY 2012 – 2015 Source: GAPP monitoring data 9 These and other results achieved under Component 2 lack comparison results in non-GAPP districts, so it is not possible to determine what the net change was in GAPP districts compared to non-GAPP districts. 23 Local Government Leadership and Council Development GAPP embedded technical specialist (ETS) oriented nearly all supported district and municipal councils on their roles and functions, formation of council committees, council rules of procedure and the relationship between political officials and technical staff. This was done through regional workshops, attended by political officials and technical staff, as well as individual mentoring. Key informants reported several positive outcomes from GAPP’s council development activities, including improvements in (1) understanding of council roles and responsibilities, (2) management of council meetings, (3) budgeting processes, (4) public accountability and (5) planning and community dialogues. Examples of each are shown in the accompanying text box. Text box 9. Outcomes of Council Development Activities Local council development led to several positive outcomes:  GAPP identified an opportunity to conclude all parliamentary business with a particular audit report if the tripartite outreach meeting for that report was turned into a parliamentary hearing. It changed the tripartite meetings to achieve this purpose.  GAPP learned that doing mentoring without first establishing the relevant legal framework and standards made mentoring more challenging than necessary. It institutionalized the practice of involving all key stakeholders in a training workshop prior to commencing mentoring activities.  GAPP learned that new political leaders needed more concerted mentoring during their initial days in office. It instituted a mentoring program for all DECs and MECs during their first 100 days in office.  GAPP used learnings from semi-annual and annual reviews to (1) scale-down interventions planned for a given FY to reduce the number of carry-over activities, (2) redesign its NSA capacity building strategy, and (3) change from a training approach to mentorship for supported LGs. Procurement, Internal Audit and Financial Accountability Systems With GAPP support, PPDA has provided TA to LG Procurement and Disposal Units, particularly on internalization of amendments to the PPDA Act. The assisted PDUs have since registered significant improvement in terms of compliance, planning and reporting. Key informants report that the management, timeliness and quality of the procurement process has improved. Several assisted PDUs are moving away from traditional, manual procurement to system-based eProcurement and are demonstrating more vigilance in contracting, which has helped to undermine fraud. Some PDUs have, on their own, also sensitized various stakeholders (e.g., suppliers, vendors, business people, etc.) on procurement processes. 24 Internal Audit Text box 10. Internal Audit Support Activities Internal audit support activities implemented include the following:  Organized regional trainings for internal audit staff, accounting officers, secretaries for finance, heads of finance, clerks to council, LGPACs and procurement staff on enhancing accountability in local government.  Provided training to internal audit staff and customized audit manuals to promote understanding of the internal audit function in enhancing accountability through investigation and reporting.  Provided direct capacity building to LGAPCs on relevant laws and regulations and their roles, functions and mandates.  Facilitated PAC to deliver capacity building for LGPACs on handling internal audit reports and applying relevant laws.  Facilitated PPDA to deliver an orientation for LGPACs in Acholi region covering issues related to the roles and functions of LGPAC, councils and political wing in promoting accountability. Internal audit staff training: Key informants credit the above training on audit processes with improving the audit function within their LG units. Specific improvements cited include improved reporting due to the introduction of uniform and standardized reporting criteria, instigation of a systemic and legal approach to audit investigations and more timely completion of quarterly audit to LGPAC, DEC, MoLG and the Internal Auditor General. GAPP’s internal monitoring results, however, suggest that improvements in LG auditing processes remain anecdotal to this point and that systemic improvement across the GAPP-supported districts is yet to be obtained. Per Figure 3, in FY 2015 31 of 35 LGPAC’s were rated as ‘functional,’ as measured by the number of meetings held, number of reports submitted to the DEC and number of committee members. This number fell to 28 of 35 LGPACs in FY 2016. Two of the districts, however, formed new LGPACs in FY 2015 (Kitgum and Agago), so if these are excluded, then there was a net drop of one LGPAC in FY 2015. Interestingly, of those non-functioning LGPACs in FY 2016, only Kitgum, Agago and Pader were carry-overs from FY 2015. In contrast, Lira, Kiryandongo, Adjumani and Zombo went from functional to non-functional. 25 Figure 3. Number of Functional LGPACs during FY 2015 – 2016 Source: GAPP internal monitoring data Capacity building for LGPACs: According to key informants, capacity building activities have led to improvement in how LGPACs conduct their work. For example, pressure from the Koboko District LGPAC has led to accountabilities for advances to be made within the mandatory three days specified by law. The Koboko LGPAC now focuses mainly on queries raised in internal audit reports, as opposed to going through vouchers as used to happen previously. Furthermore, the LGPAC Secretary now checks with the internal auditor to see if issues raised earlier have been addressed thus enabling the LGPAC to handle more issues at a sitting than before. Some LGPACs are flexing their muscles to enforce accountability. For example, a head teacher of a primary school in Arua district, who had taken advantage of absence of a school management committee to embezzle UGX 13 million, was made to refund it. Text box 11. Example of Internal Audit Activity in Arua During FY 2015/16, GAPP conducted a workshop and provided technical guidance and backstopping to internal audit and finance staff in Apac District on their functions, including local revenue generation. The Public Financial Management Act 2015 was discussed in detail to enable participants to internalize its provisions. District councilors were also trained on financial oversight. According to district officials, GAPP’s support has led to several improvements, including (1) early submission of the 2016/2017 internal audit annual work plan and budget to the CAO, LGPAC, Regional Audit Committee and the Internal Auditor General; (2) improvement in the generation of sub-county and district revenue plans; (3) enumeration and registration of tax payers; and (4) faster development of district procurement plans due to early submission of relevant information by user departments. 26 Financial Accountability Systems GAPP provided technical support on financial management to Chief Financial Officers and their staff on budgeting, financial management and accounting. For example, the ETSs organized a two-day workshop in Masindi in June 2016 where they provided technical backstopping on different aspects of financial management, including preparation of the local revenue register, chart of accounts and fixed assets register. The CFOs were also guided on answering queries raised by the Auditor General. The most dramatic result coming from this assistance is that in FY 2015, 38 of 42 GAPP-supported district local governments (DLGs) and municipal local governments (MLGs) received unqualified audit opinions from the OAG compared to only 11 in FY 2011 (see Figure 4). In the Lango region, all LG accounts used to receive qualified opinions, but in FY 2015, only Apac District received a qualified opinion (due to a salary scam). Although this achievement cannot necessarily be attributed to GAPP support alone, key informants insist that GAPP’s technical assistance contributed significantly to this result. Figure 4: Number of Unqualified Audit Opinions Received by GAPP-Supported DLGs and MLGs 2010/2011 - 2014/2015 Source: GAPP monitoring data “The training received in risk assessment has been eye-opening. I learned that many I thought were not risky are in fact risky, for example roads and technical services. In health I previously focused mostly on finance and not on outputs, such as the number of patients attended or the utilization of drugs relative to the drugs that were supplied. As another example, in the past I only considered the number of kilometers of road built, but now I know I need to consider the thickness and width of the road. …Another area GAPP has helped is in report writing. In the past, every department has a different format, but today there is a standard reporting format, which is more understandable. This is important because it helps create a more uniform understanding of the issues so they can be addressed. …Payroll auditing is another area where GAPP has helped. Before the approach to payroll auditing was cumbersome, but today we can easily go through the payroll and find information.” –Internal Auditor 27 Planning and Budgeting Processes Planning and budgeting support activities implemented include the following:  Provided financial and logistical support to NPA staff to undertake a mid-term review of district development plans (DDPs) piloted in eight GAPP-supported districts and two GAPP-supported municipal councils (Nwoya, Gulu, Kibaale, Mityana, Kalungu, Amolatar, Oyam, Nebbi, Hoima MC and Gulu MC).  Conducted a two-day ToT to orient the GAPP national resource pool, LG staff and district planners on the new planning guidelines.  Supported LGs in developing five-year DDPs using guidelines disseminated by the NPA.  Because of the above activities, key informants noted a marked improvement in the quality of district workplans, budgets and quarterly performance reports, district planners disseminated the new LGDP guidelines within all 35 GAPP-supported districts and all DDPs were completed and submitted to the NPA for review and comment. 5.5 Evaluation Question 3c, 3d & 5: How well are GAPP interventions influencing the performance of citizen participation in local governance processes and service delivery? How well are GAPP interventions influencing the performance of unblocking voice and accountability channels in local government? How well is CSO advocacy and monitoring of service delivery contributing to improving the quality and effectiveness of service provision at the local governments (LGs) and lower level governments (LLGs)? Findings related to EQs 3c, 3d and 5 overlap to the extent that they are addressed jointly below. To date under Component 3, GAPP has provided grants to 48 CSOs, of which 17 are national-level CSOs and 31 are district-level CSOs. GAPP-supported CSOs have implemented a variety of approaches aimed at raising the awareness and building the civic competencies of local community members together with community-based accountability mechanisms. The typical CSO intervention incorporated a variety of these approaches through some combination of citizen engagement with public officials, often at service delivery points, and advocacy or dissemination activities with members of the broader community. During the KIIs with CSO representatives and FGDs with community participants/beneficiaries, the MTE team heard of many positive outcomes demonstrating how CSO and citizen collaborations had produced positive outcomes in a variety of areas, including improvements in public accountability and service provision. Due to space limitations, the discussion below focused on a sub-set of GAPP Component 3 activities, which is illustrative of the types of activities implemented by grantee CSOs: community monitoring, youth parliaments, support to PSOs, and support to LG local revenue mobilization efforts. The outcomes reported below are also illustrative of the outcomes uncovered during the MTE fieldwork. Further, in recognition of the USAID and DFID’s prioritization of health, education and agricultural production, together with the crosscutting priorities of women and youth, the findings reported below place an emphasis on outcomes demonstrating a positive contribution to these priority areas. (A more in-depth discussion of Component 3 CSO citizen engagement outcomes can be found in the case studies presented in Annexes 8-11.) 28 Community Monitoring To involve the broader public in overseeing LG budget execution and service delivery, CSOs can use various community monitoring methods, which are participatory methods used to engage communities in analyzing government expenditures and service delivery at the local level and hold LG officials accountable for them. These methods not only can be used to improve public expenditures and outcomes but also can be a means of empowering communities and giving voice to citizens’ concerns. Variations on the ‘standard’ community monitoring method used by GAPP-supported CSOs include public expenditure tracking and women’s village budget clubs (VBCs). Community monitors have contributed to several improved outcomes in the districts visited by the MTE team, including those found in the accompanying text box. Text box 12. Outcomes of Community Monitoring Activities Outcomes generated by GAPP-supported community monitoring activities:  In Lira District, HEPS community monitors were instrumental in relocating short-lived HIV/AIDS testing kits to health centers where they could be used prior to expiration.  In Arua District, NACWOLA community monitors helped secure solar power generators, together with solar powered fridges, for a health center so that the center could receive and store regular supplies of medicine, including antibiotics and anti-malarial drugs.  At the same health center, community monitors prodded the local government to dig a borehole and set up a system for harvesting rainwater, which has enabled the maternity ward to access water.  Three GAPP-supported CSOs in the Masindi District (FOWODE, CEDO and Masindi District NGO Forum) used public expenditure tracking to facilitate a number of service improvements at local schools and health centers, including (1) hiring 67 new primary school teachers and 17 health center staff in Masindi and 25 health center staff in Buliisa, (2) constructing two houses for health workers and five houses for teachers in Masindi and Buliisa, (3) procuring 1,140 school desks by the local government in Masindi, (4) constructing nine classroom blocks in Masindi and three in Buliisa, and (5) constructing five pit latrines at local hospitals and health centers. Youth Parliaments In Lira (GLOFOD and Lira District NGO Forum) and Oyam (AYDL) Districts, GAPP worked with youth groups to set up youth parliaments consisting of young men and women between the ages of 25 and 30 years to engage LG officials and other community stakeholders on governance issues within their communities. In Lira district for instance, GLOFOD has worked with over 150 youth in two rural sub￾counties and three divisions in Lira Municipality, while the Lira NGO Forum supported 174 youth to form youth parliaments in the Lira District and Amolatar District. “In Palaro Sub-County, there was a case of mismanagement of the Youth Livelihood Programme funds by one of the group members. The chairman of the group had used the money to trade in products without the involvement of the group members and ended up diverting the money. He bought 30 bags of groundnuts for the funds worth UGX 1,540,000. The matter was reported to the Sub-County officials and the money was recovered from the chairman.“ – A4D 29 Youth parliaments can be an effective mechanism for mobilizing youth around issues that affect their lives and those of other community members, as the examples in the accompanying text box demonstrate. Text box 13. Outcomes of Youth Parliament Activities Outcomes generated by GAPP-supported youth parliament activities:  A youth parliament supported by AYDL discovered that, contrary to law, police were detaining suspects over 48 hours without charging them in court. Through the Youth Parliament’s intervention, police have since suspended the practice. The youth parliament now positions itself as an advocate for those detained by police by intervening to ensure that police follow the correct procedures.  The youth parliament in Lira Municipality successfully lobbied the division planning and budget authorities to increase the youth budget from UGX 200,000 to UGX 1,500,000.  In agricultural production, the youth parliament supported by Lira District NGO Forum participated in debates on taxation for increased agricultural inputs and rejection of the agricultural tax.  In health service delivery, the youth parliament in Oyam successfully engaged with health service providers to install six refrigerators and a solar power lighting system at different health centers, replace a drunkard ambulance driver with a (presumably) sober one, and get nurses and a nursing assistant posted at yet another health center.  In the area of education, the Oyam youth parliament successfully advocated for the reinstitution of school inspections, the construction of latrines at primary schools and the hiring of additional female teachers in a school with 600 female students but only two female teachers. Strengthening Private Sector Organizations GAPP-supported CSOs also work with a variety of PSOs so as to strengthen their capacity to advocate for the interests of their members. These CSO-Collaborations have produced a number of successes, a sampling of which are presented in the accompanying text box. “The youth Parliament empowered us. We were mobilized and counseled. Some of the children in the community had dropped out of school. Now we work to monitor and so far we have sent back 300 primary pupils back to school. We look at it as a responsibility. In railway division, we took back 3 children whose parents could not afford scholastic materials we negotiated with the school authorities to allow them to continue with studies. We mobilize the children with disabilities back to school. We have data on all the children with disabilities.” –Members of Youth Parliament, Lira Municipality 30 Text box 14. Outcomes of PSO Activities Outcomes generated by GAPP-supported PSO strengthening activities:  The GAPP-supported CSO CEDO helped five PSOs—Buliisa Lake Shell Collectors’ Association, Buliisa Boda Boda Riders’ Association, Masindi Market Vendor’s Association, Masindi Motorcycle Operators’ Association and VSLA Federation—grow their membership by over 220, 500, 1000, 1800 and 6500 members, respectively.  CEDO further helped empower the women market vendors in Masindi to assume control over the management of the central market. With association backing, the women were also able to demand better garbage collection and sanitation, which led to provision of more public sinks and public latrines in all urban markets.  GAPP-supported CSO CEGED’s support to the Market Vendors Association in Arua (85% of whom are women) has empowered it to engage with LG officials to create additional market stalls, improve the security, cleanliness, hygiene, and availability of water in the local market, and set the criteria to register vendors (unregistered vendors pose significant competitive challenges to registered vendors operating within the commercial laws). Local Revenue Mobilization CSOs can also play a useful role of improving local revenue mobilization within targeted districts. For example, GAPP national-level CSO grantee SEATINI implemented an LRM program in Kitgum, Pader and Lamwo Districts working with the local CSO Kitgum Women Peace Initiative. Major activities implemented included mutual dialogues between private sector and LG leaders, community platforms through local CBOs and neighborhood assemblies, radio talk shows, organized community dialogues, capacity building of LGs, CBOs, and private sector and advocacy on taxation and LR generation at local and national levels. The trainings of civil society and private sector in LRM led to the formation of a Hotel Owners and Managers Association to improve on the coordination, mobilization, communication and advocacy around LRM issues. Each of the districts recorded an increase in local tax collections from FY 2014 to FY 2015. In Kitgum local tax collections increased from UGX 493 million to UGX 543 million. In Pader revenue from operation permits increased from UGX 0 to UGX 7.1 million, while collections from the local service tax increased from UGX 18.7 million to UGX 37.2. Finally, in Lamwo overall tax revenues increased from UGX 6.6 million to UGX 85.3 million.10 10 SEATINI-Uganda (2016), End of Project Evaluation: Enhancing Opportunities for Local Revenue Mobilization and Accountability for Improved Service Delivery in Kitgum, Pader and Lamwo Districts 31 5.6 Evaluation Question 4: Which GAPP technical approaches are proving more effective and relevant for future programming? This section identifies both general and specific technical approaches taken by GAPP to date, which have either demonstrated their soundness or which appear to offer sufficient potential that they are strong candidates for replicability in future LG strengthening programs. A summary of these approaches is in Table 3 at the bottom of this section. Component 1 In terms of its general technical approach, key informants interviewed during the MTE universally agree that engagement with key NAIs and government ministries, such as under GAPP Component 1, is a critical component of any intervention aimed at improving LG accountability and service provision, as much of what happens at the local level is directly or indirectly influenced by the performance of these NAIs, including how the governance policies enacted at the national level affect the governance enabling environment at the LG level. As discussed above, however, program engagement at this level should be targeted to cover NAIs operating across the entire accountability cycle. More specifically, the tripartite meetings (including Parliament, OAG & PPDA) show potential for generating meaningful engagement among stakeholders and increasing voice for local citizens. Their main value is enhanced accountability by subjecting local government operations to public scrutiny, similar to oversight committee sittings in Parliament, which was not previously the case. In fact, one of the LG officials in Kiryandongo complained about this approach saying that it exposed some “confidential” matters to the public. This is precisely its value. LG officials should operate knowing that what they do may be open to public scrutiny. Component 2 The embedded technical specialists have been assigned on a regional basis to provide local governments with guidance, mentoring, support and technical backstopping in core LG functions. This has proved to be an effective approach to help LGs enhance their capacity and operate with minimal interruption, while adhering to central government policies, regulations and guidelines. LG technical and political officials uniformly see the ETSs as extremely useful resources to provide them with much needed technical support in their day-to-day work. The ETSs’ work method is very much the same as on-job￾training and is widely appreciated by the LG officials interviewed during the MTE. GAPP occasionally organizes leadership fora, induction meetings, workshops and retreats for elected council members and technical staff to discuss or review specific issues and agree on the way forward. These are also intended to enable the participants to share experiences, learn from each other and internalize a broad range of issues within the shortest possible time. If properly handled, these events have proven to be an effective mechanism for learning and sharing experiences. Both councilors and technical staff voice strong satisfaction with these types of events while emphasizing that any single event is not sufficient and that there is a need for on-going assistance using a variety of methods. GAPP’s strategy to focus on the four ‘core’ functions of local governance—LRM; LG leadership and council development; procurement, internal audit and financial accountability systems; and planning and budgeting processes—is also a particular strength of GAPP’s programming strategy worthy of replication in future local governance programming. In light of resource limitations, there is a need to focus Activity efforts where they can produce the biggest impact on LG capacity on performance for the resources expended. The four core areas selected for focus by GAPP appear by all accounts to be well chosen. 32 Up to this point, GAPP ETSs have concentrated on providing TA at the HLG level (districts and municipalities) so that they can later provide effective monitoring, supervision, guidance, support and technical backstopping to LLGs using a cascading approach leveraging resources developed during their work with HLG officials (e.g., DRPs). In concept, this is a good strategy, particularly to the extent the GAPP can successfully leverage resources it had worked to develop at the HLG level, although it remains to be seen whether the DRPs can play this role effectively. While this cascading approach is widely understood and appreciated by GAPP stakeholders, there is a general feeling among stakeholders, even at the district level, that GAPP support should have gone down to LLGs, especially sub-county level, much earlier, as capacity and resource deficiencies at the LLG level are particularly acute. Thus while the cascading approach is a well-founded strategy, its timing may not have been ideal. Component 3 GAPP-supported CSOs used a variety of approaches designed to increase citizens’ voice by mobilizing them to advocate for improved accountability and service provision. A common characteristic of these approaches was the combination of different engagement strategies in which a group of citizens were organized for the purpose of engaging with LG officials (both elected and technical), service providers (e.g., health center workers) and the community at large via community participation activities (e.g., radio, roundtables, community theater, monthly cleaning exercises). Each of the various methods implemented reviewed here—including community monitoring, youth parliaments, and public expenditure tracking—appears to have contributed in certain cases to improved LG accountability and service provision, including in recognized priority areas for USAID and DFID, specifically health, education, agricultural production, women and youth. Each, moreover, shows sufficient potential that it merits consideration for future USAID and DFID DRG programming, again provided that it is part of a package of interventions that also includes community-wide engagement/participation activities. Of the different CSO approaches examined during the MTE, the youth parliament is perhaps one of the most difficult to organize and sustain. Given, however, the large youth bulge in the country, the fact that youth interests are frequently overlooked in setting LG spending priorities and in point-of-service delivery, and given the emphasis in CDCS 2017-2021 on demand-side advocacy, some form of strategy for giving youth voice and engaging in LG advocacy efforts would seem to be an essential component of future DRG programming in the country. GAPP-supported CSOs also worked with PSOs to advocate for their interests with LG officials. These efforts also appear to have produced a number of successes demonstrating that, if effectively organized and empowered, PSOs can exert effective voice in advocating for their commercial interests. A number of the PSOs supported under Component 3 were women-centric, further suggesting that working with PSOs can serve the dual purpose of improving conditions for micro and small business operators and economically and politically empowering women. General One potentially useful feature of GAPP Activity design is the so-called ‘Expert Panel.’ According to the Terms of Reference (TOR) for the Expert Panel, its purpose is to serve “as a think tank around the program that will also distil major lessons and best practices arising out of program implementation in addition to reflecting upon any changes in the program’s operating environment” (pp. 2-3). As such, the Expert Panel compromises an integral component of GAPP’s collaborating, learning and adapting (CLA) strategy. 33 Unfortunately, the Expert Panel has yet to function, despite its inclusion in the original Activity design, making it impossible at the mid-term to assess its value added to GAPP activities and results. To date, GAPP has completed the formation of the Expert Panel, but it has yet to engage the panel members for any of the purposes spelled out in its TOR, notwithstanding the stated desire of existing panel members (or at least those interviewed for the MTE) to work with GAPP. GAPP readily concedes that it has yet to engage the Expert Panel owing to having “so many balls in the air” trying to get the Activity up and running that the Expert Panel “kind of got lost in the shuffle.” Nonetheless, GAPP asserts that it has successfully “set up the foundation [for the Expert Panel] and is now preparing to turn the key.” In fact, GAPP has invested a good deal of work in setting up the Expert Panel and developing its strategy. As such, it expects the Expert Panel to begin functioning and serving its role presently. Table 3. Technical Approaches with Particular Relevance for Future DRG Programming Component 1 Engagement with NAIs on the LG governance enabling environment across the accountability cycle Tripartite outreach meetings Component 2 Embedded technical specialists Leadership and council development Focus on limited set of ‘core’ LG functions Cascading TA beginning at HLGs and working down to LLGs leveraging resources developed by the program Component 3 Mobilizing citizens using a toolbox of approaches, including public engagement/participation strategies, to increase voice and engage with LG officials to improve accountability and service Youth parliaments or other methods to mobilize youth and give them voice Strengthening PSOs as a method to increase LG accountability and service provision and economically and politically empower women General Expert panel as both advisory board and advocacy partner 6 RISKS POTENTIALLY AFFECTING GAPP RESULTS AND SUSTAINABILITY The findings presented above indicate that GAPP’s work under Activity Components 1-3 have produced a number of positive outcomes in terms of local government accountability and service provision, citizen 34 participation and voice and the local governance enabling environment. At the same time, however, the MTE also uncovered a number of risks threatening either the effectiveness of GAPP interventions or the sustainability of GAPP results down the road. These risks are summarized in Table 4 and described at length below. Table 4. Summary of Risks Potentially Affecting GAPP Results and Sustainability Component 1 Limited ownership of GAPP interventions among key NAIs Limited GAPP engagement on policy issues Limited capacity and motivation at parliamentary oversight committees Persistent backlog of audit reports Incomplete coverage across the accountability cycle Component 2 Structural weaknesses within the LG enabling environment  Centralized taxation system  Few local revenue sources  Little local revenue flexibility  Staffing and resource shortages  Corruption and monetization of public service  Political interference from the central government  Proliferation of districts Mismatched Activity resources relative to Activity objectives Short-term nature of GAPP interventions Issues with local government leadership and council development  Lack of exposure to key policy frameworks and development perspectives  Difficulties of political officials to get data and information from technical staff  Lack of communications strategy  Lack of understanding surrounding conflict of interest  High turnover of councilors Issues with procurement  Lack of funding due to reliance on locally generated revenues  Limited understanding of or intentional undermining of procurement systems  Lack of monitoring at the local level and at the national level by MoLG and LGAC Issues with internal auditing  Lack of funding for internal audit department and LGPAC due to reliance on locally generated revenues  Lack of feedback or action taken on internal audit recommendations  Lack of specialized training for internal audit staff  Failure to table LGPAC recommendations  High turnover of district accounting officers 35  Large backlogs of audits (due in part to above) Issues with planning and budgeting processes  Lack of citizen participation and domination of budgeting process by vested interests  Demands from the center that tax planning department resources  High turnover of planning and budgeting staff  Understaffing in planning departments Component 3 Lack of linkages between national and local advocacy efforts Limited scale/diffused impact of GAPP CSO support Limited timeframe of GAPP CSO support Short length of grant funding Lack of capacity development for supported CSOs 6.1 Risks Potentially Affecting GAPP Component 1 Results and Sustainability Limited Ownership of GAPP Interventions among Key NAIs According to key informants, ownership of GAPP interventions at number of GAPP-supported NAIs is relatively weak. A few factors appear to be contributing to this result, although to different degrees among the relevant institutions. First, some see themselves as having had limited engagement with GAPP and thus have yet to perceive the value-added of working with it. Second, some see GAPP as more focused on its priorities rather than on their priorities. Statements such as “they (GAPP) came to us and we helped them [emphasis added] to implement their activities” were common. Third, some find GAPP’s support inflexible and overly prescriptive in nature, thereby limiting their ability to innovate and producing perceptions of disempowerment. Key informants claim that the grant disbursement schedules and performance milestones inhibit flexibility and innovation and are not always in synch with each other. In addition, the two-year funding timeframe is too short, which limits both the kind and sustainability of improvements that may be achieved. Each FY GAPP convenes an annual work planning meeting with supported NAIs to discuss and agree on priorities for the year ahead. This discussion is to be informed by the institutions’ strategic focus and priorities, as well as GAPP’s scope of work and resource envelope. However, there are a number of requests made by the NAIs, which GAPP sometimes does not support. GAPP attributes this to a shift in the CDCS on the kind of support GAPP can provide at the national level, which it says has been difficult for national institutions to understand. “We were contracted to deliver certain milestones, which gives little room for flexibility. Any changes would have required GAPP approval and changes in the contract.” – GAPP-Supported NSA “There is rigidity in the disbursement of funds and sometimes disbursements are not aligned with the planned activities.” –GAPP-Supported NSA 36 The challenge of ownership is further worsened by the fact that GAPP cannot provide grants to government institutions and pay professional fees to GOU staff for work that is within their mandate and undertaken during office hours. This is demotivating for these entities. GAPP has had experience where staff of one national entity withdrew from an activity when it became apparent that they would not be paying professional fees. While the MTE team acknowledges the reasons why GAPP does not pay GOU staff professional fees, it is, nonetheless, a factor influencing the motivation of officials within GAPP-supported NAIs. Limited GAPP Engagement on Policy Issues According to key informants at PS&LG, LGFC, NPA and MoLG, GAPP has had limited engagement with them over the course of the Activity. Key informants at the LGFC, for example, insist that, despite its legal mandate to advise the government on LG funding, GAPP has not officially engaged with it in a meaningful way. However, according to GAPP, LGFC was engaged in rolling out the Local Revenue Database, LGFC staff facilitating a Training of Trainers for GAPP consultants and staff and also piloting of the database in Masaka Municipal Council. The contradictions point to the fact that there is limited institutional memory or lack of institutionalized engagement. Most government agencies prefer to use ‘focal persons’ as a point of engagement; however, these focal persons do not effectively communicate to all staff of the institution to enhance collaboration and learning. Limited Capacity and Motivation at Parliamentary Oversight Committees The performance of parliamentary oversight committees is hampered by the short 2½-year tenure of the parliamentary standing committees. When the Committee Chair changes, the operations and level of engagement change and sometime requires orientation of new members, and this takes time. In addition, the capacity of some Members of Parliament (MPs) is low, and this limits the level of debate and articulation of issues brought forward for their consideration. Furthermore, the proliferation of districts across Uganda makes it hard for the committee to deal with all of the LGs. The extent to which MPs on the oversight committees engage on accountability issues depends to a large extent on their motivation, and their motivation frequently depends on what monetary compensation they receive. The monetization of public service is a pervasive problem that afflicts all levels of government in Uganda, and USAID policies regarding facilitation of public officials constrain GAPP’s ability to address this problem directly. For instance, the fixed rates provided by GAPP are not commensurate with those of Parliament. Persistent Backlog of Audit Reports With GAPP support, the relevant parliamentary committees have cleared most of the recent audit reports, but there remain significant backlogs. Audit backlogs have a historical perspective in that before 1996, there was only PAC, which was tasked to handle both central government and LG reports. PAC, however, has tended to concentrate on the central government reports and to disregard the LG reports. The situation improved in 1997 when the LGAC was established to handle the LG reports, “On the policy side, we did not do well. Issues that were raised by the committee, such as increasing local government financing or improving the quality of service delivery, especially education, have not been addressed by the government.” –PS&LG 37 although this only partially ameliorated the audit backlog. Some KIIs noted the key factor contributing to the audit backlog is the shortage of clerks at the committees who are responsible for writing reports. Other respondents, however, disputed this explanation asserting that parliamentary committees have a sufficient number of clerks but the issue is that some MPs do not take their roles and responsibilities seriously. To a large extent the barriers effecting parliamentary oversight committees have more to do with political economy type of challenges and lack of commitments by MPs rather than lack of technical knowhow. Incomplete Coverage across the Accountability Cycle Resolutions, proposals, bills, etc. emanating out of parliamentary oversight committees must often then be acted on by other agencies within the executive arm of government; however, there can be minimal appetite within the executive arm to act on these recommendations. GAPP has not effectively engaged the executive arm of government to ensure that the accountability cycle is complete. 6.2 Risks Potentially Affecting GAPP Component 2 Results and Sustainability Structural Weaknesses in the LG Enabling Environment There exist a number of persistent structural weaknesses in the LG enabling environment, which potentially nullify any attempt to improve the functioning of local governance and improve LG accountability and service outcomes. These structural weaknesses are already well known to many Activity stakeholders and are thus (in the interest of time and space) summarized in Annex 7. Mismatched Activity Resources Relative to Activity Objectives The infusion of additional funding from DFID carried with it the corresponding obligation for GAPP to extend its activities to all sub-counties in each of the 35 Activity districts, not to mention the municipalities, districts and town councils located in GAPP’s five regions of operation, numbering overall well into the hundreds of LG units. The creation of the ETS position and DRPs were intended to help GAPP achieve this objective. Notwithstanding, the MTE team is skeptical that GAPP can successfully achieve this objective and provide the type of intensive and on-going technical and other support required by elected and technical officials at both HLG and LLG levels to embed capacity and service improvements in how the HLGs and LLGs function, pursuant to achieving sustainable systemic change. The ETSs are already over-tasked, and while the DRPs are a potentially useful tool to supplement the ETSs’ efforts, particularly at the LLG levels, they remain an unproven tool and one, moreover, that has built in a number of structural weaknesses that potentially compromise their effectiveness and sustainability. GAPP also has at its disposal a cadre of consultants into which it can tap to supplement the work of the ETSs, but it is again questionable whether they can fill the gap between resource demands and resource supply. They might alternatively be used along with, or in lieu of, the ETSs to train, monitor and supervise the DRPs. Finally, there remains much greater scope for GAPP to integrate its Component 2 and 3 activities to use the latter (particularly CSOs) to supplement the efforts of the former (particularly ETSs). Short-Term Nature of GAPP Interventions Sustainable change requires time for improvements in LG processes and functions to become institutionalized, or in other words, embedded into the normal routine of how the LGs operate. The short-term nature of GAPP support—whether due to resource constraints that limit how much follow- 38 up assistance GAPP can provide over time or the limited timeframe of GAPP’s grant support—may not be sufficient to ensure that change is institutionalized within the supported LGs. The SDS evaluation reached a similar finding, “SDS support has led to some improvements in LG governance and leaderships. However, without additional support, these improvements are not likely to be sustained . . . SDS needs to continue providing support (both technical and financial) to these districts for a few more years to consolidate the gains so far in strengthening LG systems” (p. XIII). Similarly, in its synthesis of past USAID district support programs, MELP concluded, “Any intervention, but particularly ones putting districts in the driver seat, is unlikely to be successful without additional capacity building. No blueprint exists for what works or does not, but longer-term mentorship and systems approaches may be the most effective, and sustainability is critical” (p. 3). At the mid-point in GAPP, it is not possible to say with certainty how its interventions will play out over time, but SDS experience offers some sobering lessons learned. By the time of the SDS final evaluation, a number of its LG interventions were already showing signs of missing or diminished sustainability. Again, it is too early to determine whether this is the fate of GAPP’s LG interventions, but the same issues affecting the sustainability of the SDS interventions—or what the SDS evaluation referred to as “factors both programmatic and LG based that limited the program from being fully effective in strengthening district systems” (p. 38)—are no less present here. Issues with Local Government Leadership and Council Development A major concern about GAPP’s council orientation is that the councils were not exposed to key government policy frameworks and development perspectives (e.g. the National Development Plan, Sustainable Development Goals and Vision 2040) including authoritative interpretation of key legislation like the Local Government Act (CAP 243) and the Local Government Financial and Accounting Regulations (LGFARs). This is a significant omission making a repeat of orientation necessary. LG leaders need to understand how events at local levels link to the country’s policy and development frameworks, as well as its broad national vision. At the very least, this should be addressed during the induction that GAPP is planning to undertake with MoLG in 2017. Several challenges still have to be overcome in order for the political and technical sides to work in partnership. Among other things: (1) difficulties by political officials in getting data and information from technical staff due to poor record keeping and unwillingness of technical staff to release data/information that may reflect badly on their performance; (2) lack of a communication strategy, resulting in mutual suspicion between political leaders and technocrats, as each side does not understand what the other is doing; (3) general lack of understanding on the political side of what constitutes conflict of interest; and (4) high (up to 70%)11 turnover of councilors at each election cycle. Issues with Procurement The procurement function in all LGs is poorly funded due to its total reliance on locally generated revenue. Procurement requires a large number of physical reports, and therefore much photocopying, but funds are not enough to buy the required supplies. There are no funds to issue advertisements either, so procurement of goods and services is sometimes done through selective bidding. Due to lack 11 Cardno Emerging Markets. (2012), “Local Government Induction of Councilors: Final Training Report.” 39 of logistics, procurement staff often cannot go to sub-counties to monitor the procurement done at that level, and this poses significant attendant risks. The lack of adequate logistics and vehicles can also have significant impact on monitoring and supervision at sub-county and parish levels. In addition, some people do not appreciate the importance of procurement systems and try all means to frustrate them. In particular, procurement staff often complain about conflict of interest requests from political leaders within the system. Thus, a great deal of sensitization is required for people to understand that things would run better if procurement systems were followed. Finally, PDU staff at times lack the necessary skills to detect fraud. The accountability system at district level is not being monitored closely, otherwise DECs in Koboko and Gulu could not have refused to table LGPAC reports for four years without MoLG’s awareness of such a gross anomaly. Accountability in LGs cannot be effectively promoted unless LGPAC, procurement, and internal audit functions are effectively supported and LGPAC recommendations are discussed and acted upon by DECs/MECs in a timely manner. Thus LGPACs need more support and oversight from MoLG and the Local Government Accounts Committee (LGAC) of Parliament. Issues with Internal Auditing Similar to the procurement function, internal audit departments have limited funding because they are wholly reliant on local revenues. As a result, most of their logistical and operational requirements are continuously frustrated. This is a serious matter, which can be demonstrated by the fact that all primary and secondary schools in Koboko district have not been audited since 2008. According to one internal auditor it would appear that “The meager allocations to audit may be intended to limit its scope.” Internal audit departments, moreover, do not receive feedback on actions taken, if at all, on their recommendations. This is because either LGPAC members do not promptly work on internal audit reports due to funding challenges, or because LGPAC reports and recommendations are deliberately not tabled for discussion by the DEC and council, as in the cases of Gulu and Koboko districts. Inaction on their reports and recommendation greatly demoralizes internal audit staff, which has to keep reporting the same issues in subsequent reports. Finally, internal audit staff do not receive specialized training to handle increasingly sophisticated audit challenges. For example, the internal audit staff in Gulu received only two days of training after the Ministry of Finance, Planning and Economic Development (MoFPED) introduced the Integrated Financial Management Information System (IFMIS), the Single Treasury Account, the new Finance Act, decentralized payment to districts, and the new format for financial accounts. Even worse, internal audit staff received no training on the Integrated Personnel Performance System after its introduction. Notwithstanding, internal audit staff require continuous training to keep abreast of developments in their field so that they can effectively deal with increasingly sophisticated fraud. LGPACs also face several challenges in their operations. First, there is no mechanism for compelling the DEC chairperson to table LGPAC recommendation to council. On top of this, they meet only once a quarter for only two days (due to limited finances to pay their sitting allowances) to discuss the internal audit reports. LGPACs are wholly dependent on locally generated revenue, which is miniscule and is shared with other functions that do not have grants, while district accounting officers turn over frequently. As a result, many LGPACs are handling huge backlogs. For example, at the time of the MTE, the Gulu LGPAC was still working on the 2012 internal audit report, while all subsequent reports up to 2015 were still pending. 40 Finally, DEC Chairs often deliberately refuse to table LGPAC reports for discussion by the DEC and council if they contain adverse findings or recommendations. For example, Gulu and Koboko DECs and district councils have not discussed any LGPAC report or recommendation for the last four years. Clearly there is a loophole, which unscrupulous DEC Chairs can exploit to defeat the accountability framework at the LG level. Issues with Planning and Budgeting Processes Improving the LG planning and budgeting process also involves a number of challenges. Although planning is supposed to be participatory and bottom-up, many stakeholders do not turn up due to participation fatigue or lack of facilitation. Sometimes those that attend have challenges articulating their priorities. Furthermore, due to lack of facilitation, Community Development Officers (CDOs) frequently do not go down to help local citizens articulate their interests. As a result, there exists heavy reliance on the opinions of local leaders and other opinion makers when generating input for the planning and budgeting processes. Staff turnover, moreover, is high among planning and budgeting staff, especially among the CDO cadre. CDOs are frustrated due to continuous neglect and lack of facilitation; their training is also irregular due to financial constraints. Planning departments also face complications. Too many reforms initiated at the center are over￾exerting their already limited manpower. For example, the output budgeting tool (OBT) places so much demand on planning staff time that they do not have enough time for other important components of their work, such as going to sub-counties or even addressing the gaps identified in the NPA development plans. The District Planner is required to fill in information on all health centers as well as secondary and primary schools in the OBT within a specified period. Yet, planning departments are grossly understaffed. In Bunyoro region, for example, the Kibaale District planning department has one staff, Buliisa and Kiryandongo Districts have two staff, and Hoima District has three staff. None of these districts has a Senior Planner, the one who should be in charge of planning. Yet, like other departments, they cannot fill vacant posts due to the freeze on recruitment imposed by the central government that has been in place for over five years. 6.3 Risks Potentially Affecting GAPP Component 3 Results and Sustainability Lack of Linkages between National and Local Level Advocacy The evaluation team noted few linkages between local and national-level advocacy efforts. National-level CSOs engaging in national-level advocacy efforts are limited by the inability of their district-level partners to forge or sustain linkages for national-level advocacy, particularly given that the national CSOs do not provide their district-level partners the financial or other support to engage at the national level. GAPP supports a number of national-level CSOs that have established a district-level field office (e.g., HEPS, FOWODE, Transparency International, CEDO and Build Africa) however, even these CSOs have a limited presence in districts and sub-counties and have yet to create robust linkages connecting their district-level partners to national policymakers for advocacy purposes. This finding echoes that in the STAR DBTA evaluation, summarized by MELP as follows, “Furthermore, failing to recognize the programmatic linkages between districts and health facilities (STAR 37) or the benefits of including district and national advocacy campaigns for increased financial support limited the project’s effectiveness and sustainability” (p. 4). 41 Notwithstanding, there exists an opportunity for GAPP to support and build advocacy linkages between the local and national levels by supporting national-level CSOs with capacity to work across multiple districts and linking district-level CSOs to national advocacy networks. While GAPP’s grant support to national CSOs obliged them to work with their district partners and build their capacity, this support did not include advocacy support above the LG level nor money transfers that would enable their district partners to expand their own outreach to additional districts or sub-counties. According to the District Operational Plan (DOP) evaluation central government ministries have in place some form of regulation—law, circular, policy, directive, etc.—stipulating that donor organizations and their IPs should inform the relevant district and ministry officials about their activities. (These regulations, however, appear to exist largely in name only.) The DOP process implemented by the SDS Activity offers a mechanism for GAPP-funded CSOs to inform the relevant central and district government ministries about their activities and to engage them in advocacy efforts. Nonetheless, in the eight districts where GAPP and SDS activities overlap, this does not appear to be happening. Limited Scale of GAPP Support GAPP’s strategy of supporting large numbers of district-based CSOs with small grants makes sense given the language found in the GAPP SOW, which quotes the February 2011 Advocacy Assessment Report, “CSO advocacy engagement in Uganda is generally weak, particularly at the sub-national level where capacities are weak and CSOs less organized . . .. CSOs have a weak base, lack conceptual clarity on issues for advocacy, are too donor dependent and end up pursuing ‘donor issues’ rather than community generated issues . . .. CSOs “We have participated in national-level advocacy when we worked with a national-level civil society organization to collect signatures for a petition to Parliament to campaign against taxation of agricultural inputs. However, the issue was handed to them by national-level CSO, and they only collected the signatures. The petition was later presented to Parliament in 2014. It did not originate from us. Nor did the citizens debate the merits and demerits of it.” –District-Level CSO “The project was undertaken in only three sub-counties in the district due to limited time and limited funds for the project. Even in the three sub-counties, due to limited funding, only 10 monitors per sub-county could be facilitated, although the amount of work they had to do was a lot. It would be useful if the project is implemented in all the sub-counties covering all the facilities in each sub-county rather than a few sub￾counties and few facilities.” GAPP-Supported CSO “The community monitoring system should be expanded to include all sub-counties in the district rather than just a few of them.” –GAPP-Supported CSO “To call it citizen participation when only a few people are monitoring in only a few sub-counties is not the way to do serious business.” –GAPP-Supported CSO “Because of limited resources, we recruited only 16 volunteers who were doing monitoring in the sub-counties. These are too few to monitor all the health, education and agriculture activities and organize engagements for the community to actively influence the service delivery process.” –GAPP-Supported CSO 42 also lack coordination and have a weak institutional base . . . very few CSOs possess the capacity to effectively engage the state in technical processes” (p. 5). In this context, achieving scale might appropriately be defined as working with as many CSOs as possible with the objective of leaving behind a sufficient cadre of CSOs capable of carrying on with the work once the Activity ends. This objective, referred to by key informants as “letting 1,000 flowers bloom,” appears to motivate GAPP’s approach to date. Yet at the same time, it was the most often voiced criticism of GAPP’s approach under Component 3 across key informant categories, including CSOs. The practical implication of this approach is the spreading of limited resources across too many CSOs thus limiting what the CSOs can do, where they can do it, and the timeframe over which they do it. A number of the district-level CSOs have operations that scan multiple districts, yet the grant support they receive typically only allows them to implement the GAPP activities in a sub-set of districts. In other cases, the grant support is not sufficient for the supported CSO to achieve full coverage in the districts where it is working. An alternative grant making strategy suggested by several key informants would be to target a smaller number of better resourced and more capable CSOs operating in multiple districts and provide them with sufficient funding and technical assistance to allow them to scale-up activities both within and across those districts, with the aim in certain cases to expand operations over time to other districts. Assuming that this alternative strategy is the opportunity cost for GAPP’s current grant strategy, then it becomes clear that the current strategy involves an important trade-off. By spreading grants across many CSOs, GAPP achieves breadth of scale across CSOs and districts, but it does so by sacrificing depth of scale. On the other hand, targeting grants to a smaller number of more capable CSOs with operations in multiple districts, and helping them to scale up both within and across those districts, potentially achieves both greater breadth and depth of scale. Grantee CSOs also noted that GAPP’s funding requirements did not differentiate between the CSOs with capacity to work across districts, membership based CSOs, or national CSOs with field offices but “The grants are only enough to cover a few sub-counties and a few facilities in each sub-county. Yet for a large district such as Arua – with 28 sub-counties, 235 primary schools and 56 health units – this is a drop in the ocean.” –GAPP-Supported CSO “Public dialogues are important platforms for citizens to engage with duty bearers and air out their issues. However, during the design processes, we under budgeted for public dialogues. We had only two dialogue meetings in each year in each sub-county and one dialogue at district level in the 18 months of the project. Instead we should have quarterly dialogues if they are to have impact.” –GAPP-Supported CSO “Currently, CSOs are thinly spread and have small interventions that cannot allow them to produce the desired results. It also undermines sustainability, learning and continued good practice.” –GAPP-Supported CSO “GAPP interventions are more like piloting. They are too small, cover small areas and are too scattered in focus areas to create impact.” –GAPP-Supported CSO 43 lumped all of them together with corresponding mixed results. In response, GAPP argued that many of the CSOs lacked tested and reliable systems to handle large grants. However, this cannot be said of all grantees. Some district-level CSOs (e.g., district networks) and national level CSOs have the capacity to handle large grants and also multiple grants from various donors. Others have established field offices that enable them to operate effectively across multiple locations. For example, MACCO works with all five (original) West Nile districts. However, it received a grant to work in only three sub-counties in one district of Arua. The Lira NGO Forum has a presence in five districts and 10 sub-counties, and it wanted to scale up in those locations, but with GAPP’s support it worked in only two districts and five sub-counties. A4D works in five districts, but GAPP supported it only to work in 3 sub-counties within a single district. With appropriate technical assistance like that extended to NAIs and LGs, these CSOs should be able to access, utilize, and account for larger grants, while at the same time producing the outcomes and the systemic changes GAPP, USAID or DFID seeks. Spreading grant resources so thinly across multiple CSOs also affects the impact that GAPP is able to achieve. As seen above, working through CSOs can indeed be an effective means for increasing accountability and service delivery at the LG level. However, given the shallow depth of GAPP￾supported CSO interventions, this impact is likely to be smaller and less systemic in scale, a point acknowledged by members of the GAPP management team, “Working with CSOs is a bit different as you can see results in specific technical areas where you have community involvement. In some cases, you do not need too much money or resources to create change, but these changes tend to be limited in scale.” This has further implications for USAID strategy, as stated in CDCS 2017-2021, and the MEL function. In the first case, the limited scale of potential GAPP impacts appear less likely to create the scale of change necessary to satisfy the CDCS 2017-2021’ objective for systemic change. The small scale and disbursed nature of change, moreover, complicates measurement of that change. Assuming that at some point the Mission desires more scientifically rigorous evidence of GAPP’s impact (or its DRG programming more generally), smaller and more disbursed changes will be harder and more costly to measure using standard research methods (e.g., experimental or quasi-experimental evaluation designs) than larger and more concentrated changes. Finally, for the purpose of demonstrating the value added by DRG interventions to other technical areas—specifically health, education and agricultural production—the diffused impact of the current grant strategy on indicators of importance to these other areas makes it that much harder to state the case. However, in redesigning the consolidation approaches, care will have to be taken to ensure diversity of approaches and interventions, both in scale and orientation. For instance, membership based district level organizations should be required to demonstrate clearly in their proposals how their members are incorporated in the implementation of the proposed interventions at each relevant LG level with clearly demonstrated intervention strategies and approaches, which include capacity building of the members, a signed Memorandum of Understanding and adherence to GAPP’s implementation guidelines. Short Length of Grant Funding. A common issue raised by GAPP grantees was that the short 18-24 month duration of grant funding adversely affected their ability to implement and sustain their interventions on the ground and achieve a meaningful impact. The adverse effects of the (perceived) short grant funding period is in turn exacerbated by the low capacity and resource base from which many of the district-level CSOs are starting, not to mention the LG officials and community members, many of whom in the latter case have no or limited experience in civic engagement. Once a grant is finalized, it takes months or longer to build relationships with LG officials and citizens, build their capacity, and then get activities up and going to the point where they are beginning to generate positive outcomes, leaving relatively little time to achieve anything approaching systemic change prior to the grant’s conclusion. As noted by a number of 44 CSOs, moreover, the short grant funding period disrupts the CSOs’ continuity of operations thus stymieing their ability to create the momentum and scale of operations that would contribute to their long-term sustainability. Lack of Capacity Development for Supported CSOs GAPP technical monitors are supposed to provide grantee CSOs capacity building assistance, largely in the form of one-on-one mentoring, in areas related to governance, planning and financing. Indeed, building the capacity of Ugandan civil society is a primary objective of GAPP’s Component 3 activities in addition to being a primary part of GAPP’s long-term sustainability strategy. Interviewed CSOs do appreciate the support GAPP provided them during the proposal writing stages and GAPP capacity building in proposal writing, financial reporting, activity reporting and monitoring and evaluation (M&E) in fulfillment of GAPP grant guidelines. However, the CSOs expected that capacity building to include strategies related to long-term organizational development, such as mentoring, in addition to coaching on gaps identified during the Partner Local Organization Capacity Assessment (PLOCA). The CSOs noted that the findings of the PLOCA were never addressed, either in terms of providing technical assistance or in terms of a grant to solicit for technical assistance from external service providers. They further noted that GAPP did not provide support to CSOs to organize forums for CSOs working in the same districts or regions to share experiences and learn from each other and where they could interact regularly and periodically with LG institutions. “The implementation period is too short to implement such an ambitious project. Community engagement and social accountability initiatives require investments over a longer period of time if changes are to be sustainable.” –GAPP-Supported CSO “Developing and sustaining relations with high-level decision makers take time. Despite their willingness to support the project activities, most LG officials are overwhelmed by their work and are constrained by lack of adequate resources to follow up with demands of the communities.” –GAPP-Supported CSO “Engaging youth takes a long time, so two years is not enough to build the capacity of youth to continually engage. When the project ends, they stop.” –GAPP-Supported CSO “GAPP funding has focused mainly on the project implementation . . . but little on the institutions implementing the grant. GAPP does not offer institutional support. For CSOs to be effective, focus should also be put on building institutions. The grant helped us to build relationships with the community; however, as a membership organization, we also have our core mandate of serving the members. GAPP did not provide support for constituency servicing.” –GAPP-Supported CSO “Our capacity has not been built to enable us to extend our work to other sub-counties. The interface meetings, the media activities, community meetings, quarterly meetings are likely to stop.” –GAPP￾Supported CSO 45 7 CONCLUSIONS Drawing on the findings presented above, the MTE offers the following conclusions in response to each of the evaluation questions. 7.1 Evaluation Question 1: To what extent has GAPP’s theory of change and underlying assumptions held true? How did GAPP manage any changes in the ToC or assumptions? The GAPP ToC and it underlying assumption continue to hold true at the GAPP midpoint. Judging by the numerous examples of improved LG accountability and service provision found during the MTE, the GAPP ToC also appears to be empirically valid, albeit on a case-by-case basis. That is, by supporting GOU institutions and officials to improve governance and accountability, and by working simultaneously with civil society to mobilize citizens to engage and partner with LGs, then LG service delivery can be made more equitable and efficient. To date, there have been no significant changes to the ToC or its assumptions; what changes have occurred have been aimed at improving operationalization of the ToC. The GAPP Activity design is broadly consistent with USAID/Uganda’s emphasis on a ‘systems approach,’ as described in CDCS 2017-2021. In particular, GAPP’s strategy targeting NAIs, core governance functions at the LG and LLG levels and civil society constitute critical leverage points within the local governance system, such that ignoring either of them is unlikely to produce the type of systemic change sought by USAID/Uganda. GAPP’s strategy of targeting districts and municipalities and then working downward to sub-counties, divisions and town councils is also appropriate. These are the levels of local government with budgets and where services are delivered and thus where improvements in LG capacity and accountability are likely to produce the biggest improvements in LG service delivery. 7.2 Evaluation Question 2: What are we learning about coordination and management of cross-sectoral efforts from GAPP implementation within USAID? We have learned little about coordination and management of cross-sector issues from GAPP implementation. To date, cross-sectoral collaboration and learning between the USAID/Uganda DRG team and the health, agriculture and education teams has been minimal to the point where these other technical teams have little idea what GAPP is doing or what its results have been. Contributing factors to this conclusion include: (1) skepticism among other USAID technical teams that governance systems strengthening activities generate outcomes of interest to them, (2) reporting frameworks for health and agriculture that include indicators that are inappropriate for DRG activities, (3) reporting frameworks for DRG activities that generate results of insufficient scale or include indicators that fail to convey useful information about priority outcomes and (4) the failure of the Mission CLA system to facilitate joint learning and action across Mission technical/sector teams. 7.3 Evaluation Question 3a: How well are GAPP interventions influencing the performance of national accountability institutions (NAIs) in their oversight function at both national and local governments? GAPP’s work with NAIs have influenced the performance of their oversight function in a number of important ways, including improvements in the following areas: (1) information about LG accountability issues, (2) collaboration between national and local accountability institutions, (3) accountability processes at the national and local levels, (4) procurement practices across multiple LG units, (5) capacity and operational processes within parliamentary oversight committees, (6) clearance of OAG reports, (7) audit validation processes and (8) strategy to engage national and local actors’ policy reform efforts. These outcomes are for the most part tentative in nature and will require a significant amount of follow-up to embed them within the relevant institutions and the LG policy system. Nonetheless, they 46 represent a good start and are a sampling of the types of outcomes that can be achieved working with NAIs at this level. 7.4 Evaluation Question 3b: How well are GAPP interventions influencing the systemic functioning of local governments? GAPP support has contributed to a number of improvements in the performance of supported LG units in terms of improved (1) local revenue mobilization; (2) LG leadership and council development; (3) procurement, internal audit and financial accountability systems; and (4) planning and budgeting processes. Notable improvements reported include the following.  LRM: Increased local revenue collections, which in some cases has allowed LGs to allocate more money to areas with no earmarked central government grants.  LG leadership and council development: Reduced internal conflicts within DECs and MECs together with improvements in the efficiency and productivity of the councils’ internal processes, councils’ supervision of technical staff, and the councils’ monitoring of LG services.  Procurement, internal audit and financial accountability systems: Improved procurement, compliance, planning and reporting practices among PDUs; internal operational processes among PDUs and LGPACs, and OAG audit opinions among supported districts and municipalities.  Planning and budgeting processes: GAPP-supported districts successfully drafted and submitted to the NPA for review and feedback 5-year development plans using guidelines disseminated by the NPA. It is too early to determine whether the above improvements represent sustainable, systemic improvements to the functioning of supported LGs. 7.5 Evaluation Question 3c: How well are GAPP interventions influencing citizen participation in local governance processes and service delivery? Evaluation Question 3d: How well are GAPP interventions unblocking voice and accountability channels in local government? Evaluation Question 5: How well is CSO advocacy and monitoring of service delivery contributing to improving the quality and effectiveness of service provision at the local governments (LGs) and lower local governments (LLGs)? GAPP’s work with civil society under Component 3 has proven effective in giving voice to citizens, including especially women and youth, to engage in advocacy efforts on their behalf, on behalf of their association members (for PSOs), or on behalf of other community members. Notably, many of these positive outcomes occurred in USAID/Uganda’s priority areas of health, education and agricultural production and/or in crosscutting priorities of women and youth. Thus the evidence gathered during the MTE supports the conclusion that CSO advocacy and the monitoring of service delivery can be an effective mechanism for (1) involving citizens in local governance processes and service delivery, (2) unblocking voice and accountability channels in local government and (3) contributing to improving the quality and effectiveness at the LG and LLG levels. It can also be an effective mechanism for pursuing improvements in health, education and agricultural production outcomes as part of an integrated intervention strategy. Similarly, it can be an effective mechanism for empowering women and youth and advocating for the interests of private sector organizations. 47 7.6 Evaluation Question 4: Which GAPP technical approaches are proving more effective and relevant for future programming? A number of GAPP technical approaches are proving effective and relevant for future programming. Under Component 1, and more generally, engagement with key NAIs is a critical component of any LG strengthening program. Specifically, the tripartite outreach meetings have demonstrated particular promise for generating meaningful engagement among stakeholders, increasing voice for local citizens, and increasing LG accountability. Under Component 2, technical approaches meriting note include the use of full-time ETS and council orientation and training activities. Other sound technical approaches include: (1) focusing technical assistance activities on ‘core’ LG functions and (2) working with both elected and technical LG officials. Under Component 3, the general approach of working through CSOs using a package of intervention strategies is sound technical approach, provided that the package includes community-wide engagement/participation activities. Working with PSOs is an effective approach both to increase citizen voice and LG accountability and to promote the economic and social empowerment of women. Although supporting youth parliaments poses significant operational challenges, the demographic trends in Uganda make them, or some other youth engagement/mobilization strategy, a critical component of any citizen engagement strategy with any local governance program. Finally, the Expert Panel has potential to contribute significant value-added to GAPP and other DRG programs, although GAPP has yet to draw on this potentially valuable resource. 7.7 Risks to GAPP Results and Sustainability Notwithstanding the numerous positive results under each of GAPPs three Activity components uncovered during the MTE, there exist a number of risks threatening such results moving forward and their sustainability. Risks related to Component 1 include: (1) limited ownership of GAPP interventions among key NAIs, (2) limited GAPP engagement on policy issues, (3) limited capacity and motivation among parliamentary oversight committees, (4) persistent audit backlogs and (5) incomplete coverage by GAPP across the accountability cycle. Risks related to Component 2 include: (1) persistent structural weaknesses within the LG enabling environment, (2) mismatched Activity resources relative to Activity objectives, (3) the short-term nature of GAPP interventions and (4) persistent funding, capacity and operational issues related to the core LG functions. Risks related to Component 3 include (1) lack of linkages between national and local advocacy efforts, (2) limited scale/diffused impact of GAPP CSO support, (3) short length of grant funding and (4) the lack of capacity development for GAPP-supported CSOs. 8. RECOMMENDATIONS Drawing on the findings and conclusions presented above, the MTE team recommends the following changes to improve GAPP and to inform future USAID and DFID DRG programming. The recommendations are arranged by MTE evaluation questions. EQ1: Managing Theory of Change and assumptions 1. In future DRG Activities, target LG units for support using a ‘geo-focusing’ targeting strategy. Criteria for geo-focusing might include, for example, LGs that are more or less advanced on certain benchmarking criteria, LGs in strategic locations (e.g., oil producing areas, agricultural greenbelts), LGs with relatively high concentrations of certain target populations (e.g., IDPs, youth, other 48 ethnic/marginal groups), LGs with pre-existing citizens’ groups or PSOs that can be leveraged, or LGs overlapping with GOU policy clusters.12 2. Use a systems building approach (which implies coverage of core LG functions at all LG levels) for any future local governance strengthening programs. There is no reason, however, that a systems building approach cannot be paired with a DBTA approach, similar to that used by SDS, which directly targets assistance to strengthen specific health, agricultural or education outcomes. EQ2: Cross-sectoral coordination and collaboration 3. Undertake a comprehensive diagnosis of the Mission CLA system to identify what is missing, or what has broken down, in the current system such that no cross-sectoral learning has occurred with regards to integrating DRG into health, agriculture and education programming, and vice versa. Based on this diagnosis, next adjust the system to strengthen the following: (1) processes for cross￾sectoral learning within the Mission, (2) processes for facilitating collaboration across USAID and DFID-funded DRG programs and other DRG-related programs/initiatives and (3) processes for ‘uploading’ information from funded DRG programs (e.g., GAPP) and integrating them into the Mission-wide CLA system. 4. As part of the Mission CLA system, reach agreement on indicators, methods of measurement and cost allocation for reporting the results of DRG Activities to the other technical teams that go beyond the current standard reporting frameworks. This is likely to require ‘outside-the-box’ thinking, including some combination of quantitative and qualitative data. 5. Institutionalize CLA systems within supported communities and consider integrating CLA activities and obligations as part of CSO grant agreements and organizing grantee CSOs into something akin to learning communities, which might be done regularly (e.g., quarterly) at the district or regional level so that CSOs grantees can share their experiences, practices, approaches and implementation models with DLGs and lower Local Governments (LLG). The current GAPP approach requires CSO grantees to incorporate quarterly review meetings in their proposals. However, these are part of the MEL process for each CSO and do not reincorporate learning between CSOs within and across districts or regions. EQ3a: Influence on national institutions 6. Further engagement with the leadership of NAIs and other institutions to strengthen commitment and ownership of supported interventions. This will require agreeing on the interventions that meet the interests of these institutions. In addition, GAPP engagement should embrace institutional wide interactions beyond the so called “focal persons.” 7. Increase national policy engagement, especially on issues that affect service delivery at LG levels, including LG financing. For example, work with the MoLG and Parliamentary committee on PS&LG to provide leadership on increasing national policy engagement on issues that affect service delivery 12 A policy cluster is a term used here to describe an aggregation of LG units over a defined geographic area used by GOU ministries for policy administration purposes. 49 at the LG level. GAPP should directly or indirectly (through ULGA and other NSAs) strategically engage strategic centers of power and individuals that make change happen. Some of these include the President, Secretary of the Treasury, and MoFPED. 8. Institutionalize and sustain the tripartite outreach meetings. Discuss with Parliament to ensure that the tripartite outreach meetings are institutionalized. Enhance advocacy in ensuring that during the outreach meetings the committees make binding recommendations so as to minimize double work of scrutinizing the reports again in Parliament. In addition, it may be beneficial to combine the tripartite meetings with advocacy efforts of GAPP grantees so as to ensure that the tripartite outreach meetings are sustained. 9. Given the high turnover of MPs, GAPP needs to continue strengthening the capacity of Institute of Parliament Studies to ensure that there is continuous orientation and capacity building of members of the oversight committees of Parliament. 10. Strategically engage with the executive arm of government on LG policy issues and public accountability. 11. Beyond insisting that national-level institutions lead activities themselves, allow some flexibility during implementation, especially the management of funds. This will enhance innovation, ownership and sustainability of the interventions. 12. Increase coordination efforts with donors and other governance programs in order to enhance alignment and learning. Furthermore, GAPP needs to engage with other governance programs on the best approaches to support/facilitate government institutions. EQ3b: Influence on systemic functioning of local governments 13. Engage the MoLG as an entry point for GAPP Component 2 activities where possible. Possible engagement strategies include:  Lobby the MoLG to develop appropriate enforcement mechanisms for ensuring that LGPAC reports and recommendations are discussed by DECs/MECs and councils as provided for in the LGPAC guidelines.  Orient LG councils to ensure that local governments’ councilors gain full understanding of key government policies, development and legislative frameworks and the country’s Vision 2040.  GAPP should support the NPA to undertake regular review of DDPs and training/orientation of the District Planners on the planning guidelines to enhance learning and sustainability 14. Consider adopting the following activities as part of GAPP’s Component 2 activities:  Benchmark performance across GAPP-supported districts. Benchmarking would help LGs internalize how other LGs have solved problems and then apply that knowledge to their own circumstances. It would also help establish clearer standards toward which LG units can work.  Provide short performance enhancement trainings to staff in key LG units (e.g. financial management, internal audit and procurement), which will allow them to stay abreast of changes 50 in their fields. Such training should go beyond the training being provided by ETSs to include tailored courses provided by recognized training institutions. Examples include VfM audit and advanced computerized audit.  Support the statistics function among LGs so as to ensure that planning is based on comprehensive, accurate, current and relevant statistics.  Incorporate GAPP’s activities into LG development plans. GAPP’s support activities should be incorporated into DDPs so as to comply with government requirements and to eliminate counterpart suspicion about GAPP’s intentions.  Move quickly down to sub-counties to help address the numerous challenges with planning, financial management, procurement, and accounting at that level to enable GAPP to provide the level of service needed at the LLG level (including also divisions and town councils), while maintaining its level of service at the HLG level. USAID/DIFD and GAPP should consider strategies to increase resources commensurately. Potential strategies include, but are not limited to the following: o Increase funding levels to hire additional staff, whether from USAID or DFID or by taking on new partners. o Employ outside contractors (e.g., consultants) to take on the role of training and supervising the DRPs thus freeing up the ETSs to focus on their other work. Or alternatively, task the ETSs with the training and supervising of DRPs and employ outside contractors to pick up the slack in other areas. o Integrate GAPP Components 2 and 3 more closely such that GAPP grant support to CSOs is targeted specifically to help build that capacity of LG technical and/or political officials as a supplement to work being done by the ETSs, DRPs or outside contractors. o Relax the requirement that GAPP must work in each sub-county in all 35 districts to allow it to focus on a smaller sub-set of sub-counties. Alternatively, adopt a rolling strategy in which expansion to additional sub-counties is dependent on achieving a certain level of success in existing sub-counties. In the latter case, the goal of complete coverage is maintained but given more flexibility depending on progress achieved. If a decision is made to focus on a smaller number of sub-counties, or to adopt a rolling strategy, this in turn implies that perhaps GAPP should focus efforts at the relevant districts as well, as part of a systems building strategy. EQ3c, EQ3d & EQ5: Effectiveness of GAPP’s interventions 15. Revise GAPP’s grant making strategy. The “let 1,000 flowers bloom” approach might have made sense at Activity inception, but in light of the emphasis in CDCS 2017-2021 on a “systematic geo￾focusing approach,” it is no longer appropriate. There is general agreement among stakeholders that it makes sense in the second half of the Activity to consolidate the grantees into a smaller number of CSOs with recognized potential for achieving systemic change, which in turn implies CSOs with the capability to expand both horizontally across districts and vertically within districts. The grant 51 terms—including amount and length—should be sufficient to allow grantees to achieve scale and set in place mechanisms to secure the sustainability of their interventions. A number of options exist for achieving this objective, which may be used in combination.13 These options include, but are not limited to, the following:14  Support national-level CSOs with existing networks or operations across and/or within districts.  Support district-level CSOs with existing networks or operations across and/or within districts.  Support consortiums of CSOs that collectively have existing networks or operations across and/or within districts. Consortiums may be organized around, for example, geographic overlap, methodologies, thematic areas, target groups or any combination of the preceding.  Support CSOs that create advocacy platforms, such as regional development forums and those that want develop linkages between local and national level advocacy processes, so as to increase citizens’ voice in the national level advocacy processes.  Offer long-term grants that extend over the life of the Activity with options for renewal on a periodic basis (e.g., every two years) based on meeting performance milestones. 16. Reassess the process for delivering capacity building assistance to CSO grantees, including the technical specialists’ capacity and bandwidth to provide the extent and depth of assistance required. Consolidating the number of grantee CSOs should alleviate much of the burden on the technical specialists thus allowing them to provide more focused assistance. As part of this capacity development, consider including participation in cross-learning CLA activities as a requirement of the grant terms. (Note that GAPP informed the MTE team that it is developing a new CSO capacity building strategy.) 13 A risk of such a consolidation strategy is that in trying to pick ‘winners’ (e.g., CSOs with the greatest potential for achieving scale) GAPP may end up picking the wrong CSOs. For example, selected CSOs may lack the absorptive capacity to undertake a larger advocacy role, may fail to leverage the grant funds to scale-up or improve their operations, or simply may fail to deliver on expectations or promises. Another risk is that grant money will flow to CSOs more skilled at capturing rents from donor agencies than in engaging in public advocacy (the ‘usual suspects’ syndrome). Yet another risk is that in concentrating support among relatively few, prominent CSOs that GAPP will accomplish little in terms of developing the capacity of civil society within Uganda to engage with LGs on accountability and service deliver issues, which is a secondary objective of GAPP’s CSO support. The MTE team recognizes these, and other, risks but believes that the potential upside of extending the breadth and depth of CSO and citizen activism within Activity HLGs and LLGs makes the risk acceptable. 14 It should be pointed out, that allowable grant amounts were initially determined by GAPP’s contract with USAID. The MTE has since been informed that USAID has agreed to changes in GAPP’s grant making approach. For one, GAPP has instituted changes to CSO contractual obligations that will enable it to increase the grant awards for selected CSOs in the next rounds of grants. CSOs such as CSBAG, CEPA, ACODE, CEDO and ULGA have already benefited. In the new strategy, GAPP will support national level organizations to consolidate local generated issues and increase the reach of the district level voices. Thematic leaders will be selected and indicators harmonized across districts so that results can be consolidated. At this point GAPP will build on existing grants and thereby address issues with both the duration as well as the grant amounts. 52 EQ4: Future programming 17. Combine the tripartite meetings with advocacy efforts of GAPP grantees, and use regional accountability committees to enhance compliance with tripartite meeting findings and sustainability of the tripartite process. 18. Develop a more ambitious plan of using the Expert Panel and consider increasing funding to GAPP for this purpose. For example, the Expert Panel could be used to advocate on policy issues and/or to advise GAPP on how to build and then implement an advocacy campaign. Members of the panel also have wide business and social networks and exert influence in different ways and with different national-level or district-level institutions. GAPP can tap into members’ professional and social networks for multiple programmatic purposes, for example, to advocate with the GOU to take on the GAPP interventions in other non-GAPP areas. Quoting one member of the Expert Panel, “If you say you’re doing a workshop for the LGPAC, this does not inspire me. The panel needs to be more focused on an engagement strategy.” 19. Support the implementation of regular (e.g., quarterly) learning platforms at the district or regional level targeted to political leaders, HLG and LLG technical officials, private sector stakeholders and service-level facilities/providers in priority sectors (e.g., health, education and agriculture, private sector, youth, women organizations, people with disabilities). This will ensure regular sharing of key findings in service delivery requiring urgent action and policy issues that need to be addressed through legislation (ordinances/bylaws), planning and budgeting processes, in addition to allowing CSOs to showcase their approaches and interventions. 53 ANNEX 1: GAPP MID-TERM EVALUATION SCOPE OF WORK Mid-term Performance Evaluation of USAID’s Governance Accountability, Participation and Performance (GAPP) Statement of Work 1.0 Background Decentralization in Uganda has been described by some as a cornerstone of democratic governance, yet for others it failed to deliver on the promise of effective democratic governance, improved service delivery, increased accountability and citizen engagement at the local level. USAID and DFID/Uganda support a seven-year (2012-2019), USD 35.5 Million Governance, Accountability, Participation and Performance (GAPP) activity, implemented by RTI International with its consortium partners Development Alternatives Inc. (DAI) and Development and Training Services (dTS). The goal of GAPP is to increase participation, accountability and local governance for effective service delivery. The objectives of the activity include, a) to enhance accountability within local government institutions; b) to strengthen the Government of Uganda (GOU) accountability institutions; c) to improve local government governance; d) to support civil society organizations (CSOs) and other non-state actors. The Activity consists of three broad result areas, namely; 1. Improved legal, policy, regulatory and institutional environment to meet demands for more democratic governance. 2. Improved fiscal management and accountability processes in the delivery of services in targeted districts. 3. Improved capacity of citizens and communities to participate in local governance. Introduction In November 2012 USAID, awarded a USD 17.7 million contract to RTI International with its consortium partners Development Alternatives Inc. (DAI) and Development and Training Services (dTS), to implement the GAPP activity in 25 districts15. The scope of work was expanded through the partnership with DFID which doubled funding, expanded technical breadth to a range of complementary work areas; deepened geographic coverage in existing districts with greater coverage of sub-counties, town councils and municipalities; and expansion to ten additional districts16 in northern Uganda and the Bunyoro sub-region. In addition, the expansion was aimed at enhancing the enabling environment to 15 Agago, Amolatar, Amuru, Apac, Kitgum, Gulu, Lamwo, Lira, Nebbi, Nwoya, Oyam, Pader, Koboko, Arua, Buliisa, Hoima, Masindi, Kibaale, Kalungu, Kayunga, Kiboga, Masaka, Mityana, Mubende, Mukono 16 Adjumani, Moyo, Yumbe, Maracha, Zombo, Kole, Otuke, Alebtong, Dokolo, Kiryandongo 54 improve delivery of health, education and economic growth in target districts from the current 25 to 35 districts. The GAPP’s theory of change is “if Government of Uganda (GOU) and non-government-led efforts are supported to improve democratic governance and accountability, and citizens’ efforts are mobilized to engage and partner with LGs, then service delivery will be more equitable and efficient”. The GAPP activity assumes that closer government, increased citizen monitoring and transparency are expected to characterize decentralization and should thus be associated with a move toward increased accountability and improved service delivery. Supporting the local governments should strengthen government accountability to citizens by involving citizens in monitoring government performance and demanding corrective actions. The premise is that when accountability is strengthened and improved, the opportunity for corruption to take place is diminished and governance outcomes in health, education and agricultural systems such as responsiveness, equity and efficient use of resources are affected positively. The GAPP is implemented through three major interlinked intervention components:  Supporting and strengthening national-level government accountability institutions in their oversight and interaction with LGs;  Strengthening LG revenue, procurement, financial, and planning systems;  Strengthening civil society organizations (CSOs) and other NSAs to improve voice and accountability. 2.0 Purpose of the Evaluation The purpose of the proposed mid-term evaluation of GAPP is twofold; a) to help USAID learn about which GAPP interventions should be deepened or re-directed in the work planning for subsequent years; and b) to aid understanding how GAPP supplements efforts of other sectors. The primary users of the evaluation findings are USAID /Uganda Mission staff across the different offices, DFID, RTI and its partners and other USAID funded IPs, MOLG, and other national stakeholders with interest in strengthening decentralized governance and accountability systems. This evaluation will review GAPP performance between December 2012 and July 2016. The Evaluation will examine the three components of the GAPP activity as defined in the GAPP SOW to answer the following questions; 3.0 Key Evaluation Questions (i) To what extent has the GAPP’s theory of change (ToC) and underlying assumptions held true? How did GAPP manage any changes in the ToC or assumptions? 55 (ii) What are we learning about coordination and management of cross-sectoral efforts from GAPP implementation within USAID? (iii) How well are GAPP interventions influencing performance of; a) National accountability institutions17 in their oversight function at both national and with LGs b) Systemic functioning of LGs18 c) Citizen participation in local governance processes and service delivery d) Unblocking voice and accountability channels in local government (iv) Which GAPP technical approaches19 are proving more effective and relevant for future programming? (v) How well is CSO advocacy and monitoring of service delivery contributing to improving the quality and effectiveness of service provision at the LGs & LLGs? The Evaluation team will be invited to provide feedback on these questions and make recommendations on how they can be improved to remain responsive to the purpose and utility of the evaluation. 4.0 Evaluation Methodology The Evaluation Team will develop and share a detailed methodology with USAID and select partners for approval before commencing fieldwork. The proposed methodology should demonstrate the consultants’ understanding of the level of complexity of such an evaluation and how it will be addressed. It is important that views and perspectives of different stakeholders on implementation processes, effectiveness of interventions and resultant outcomes are captured. While this SOW does not seek to be overly prescriptive of what methodology is adequate, it is highly encouraged that the Evaluation Team considers alternative methods especially ‘Outcome Harvesting’ amongst others20. The Evaluation Team will justify the choice of the methodology in line with the evaluation questions proposed. The proposed methodology should explicitly state the sampling procedures for identifying key informants, Focus Group participants, and others. When selecting KIs the Evaluation Team will pay attention to the following categories CSO, district political and technical leaders and others, farmers and market associations, USAID Mission staff and other donors, participating and/or did not participate in GAPP activities. For political leaders’ attention should be paid to both the outgoing and incoming. The group of individuals who have not had a relationship with GAPP will be interviewed as a counterfactual. The methodology should outline the sampling frame for activity sites including none activity sites to be visited by the Evaluation Team. GAPP has a geographical strategy and has implemented different activities at varying levels of intensity over various local governments. In designing the sampling framework for LGs to be visited, the Evaluation Team should therefore take into consideration the 17 OAG, UPPDA & PAC of Parliament. 18 in terms of planning, budgeting, local revenue mobilization, fiscal management, procurement and audits 19 Leadership development, CSO advocacy, citizen participation, partnership, integration. 20 http://usaidlearninglab.org/complexity-aware-monitoring/basics 56 different treatment packages received by the different LGs to ensure coverage of the breadth of LG treatment offered by GAPP to LGs. The evaluation should also attempt to explain differences in performance between districts. This will help the mission and the GAPP to know which treatment package is more effective, why and where as much as possible. Note that this is not designed as an impact evaluation by USAID standards, but expects the evaluation Contractor to propose different ways that will provide the required rigor necessary to strengthen the quality of findings and conclusions. The evaluation should establish gender differential effects as well as different age groups along the different treatment regimes of the project. Given that GAPP engagement in some of the districts has not been continuous the Evaluation Team should specify how they will address recall/response/reporting bias in the evaluation methodology. The consultants should also put forward a detailed plan for how the Evaluation Team skills set structure will handle sensitive and complex questions and approaches in activity sites to be visited. Relevant documents to be reviewed are included in Annex 1. 5.0 Deliverables Deliverables Tentative Due date In-Briefing: Introduction of the Evaluation Team and discussion of the scope of work, initial presentation of the proposed evaluation methodology and work plan, and other emerging issues that may affect the evaluation. June 27, 2016 Inception report: Draft and present an Inception Report to be reviewed by USAID. The report will include: ● The evaluation team’s interpretation of the key evaluation questions and their approach to how each question will be addressed. The Consultant shall attach a completed evaluation design matrix using the template attached as Annex 2. ● Methodology including sampling/selection procedures for selecting respondents/informants to be interviewed, and project sites to be visited. ● A detailed work plan showing a timeline for each evaluation activity to be undertaken, including the field work and allocation of expertise efforts as well as the methods to be used to answer each evaluation question. ● Detailed analysis plan/map for each of the evaluation questions, including the proposal data extraction methods for each question as well as a justification of the proposed method. ● Draft instruments for data collection. July 13, 2016 Draft and present to USAID and selected partners a detailed Desk Review of secondary data Within 5 working days after finalization of inception report Field work and interviews 10 days Presentation of preliminary findings from fieldwork: Power Point presentation at a minimum covering major findings, conclusions Within 5 working days after completion of 57 Deliverables Tentative Due date and recommendations for improvement. Presentations will be made to both the Mission and external multi-stakeholder group targeting the implementing partners and other project stakeholders. The evaluation team will liaise with the Mission to agree on the dates, audience, venue and other logistical arrangements for this briefing. fieldwork Draft Evaluation Report: The content should cover all the main elements of the report including major findings, conclusions, lessons learned, and relevant annexes thus complying with the USAID Evaluation Report standards set out in Annex 3. The draft report is expected within 6 working days after the oral presentation of preliminary findings. Within 6 working days after oral presentation of preliminary findings Final Draft Evaluation Report: Final Draft Evaluation Report incorporating comments from USAID and other stakeholders submitted within 5 days of receipt of the comments. The Final Draft Report should not be more than 25 pages of text in body of the report (excluding the executive summary, table of contents, glossaries, and annexes), provided in 4 hard copies and 1 electronic copy. This final draft report will also include a 2-4 page briefer that highlights the evaluation findings, conclusions and recommendations presented in an attractive and easy to understand format for the wider public use. Within 5 working days after receipt of comments from USAID and select partners Final Report: A final report (not exceeding 25 pages, excluding the annexes) incorporating final edits within 5 working days of receiving final comments from USAID and other stakeholders. The Mid-Term Evaluation Report should be cleared by the USAID. Within 5 working days Synthesis product: A 2-4-page briefer highlighting key findings, conclusions and recommendations from the evaluation for public use. With the final report After Action Review Within a week after submission of the Final Report 10. Cleaned, labeled and ready to use electronic copies of data sets: A CD of all data records from the evaluation i.e. ready-to-use full data sets from the quantitative data (if any), interview transcripts, and other documented survey and qualitative responses. All reports should be provided in four (4) hard copies and one (1) electronic copy. With the presentation of the final report 6.0 Team Composition and Qualifications The evaluation will be conducted by external evaluators. In entirety, the Evaluation Team will comprise, at a minimum, persons with expertise in the following areas, (a) public financial management; (b) local governance and decentralization; and (c) civil society and citizen engagement. The following are Key Personnel: 58 (i) Team Leader: The team leader should be an evaluation expert with expertise in one of the desired areas of focus. Responsibilities:  The team leader will be responsible for coordinating the activities, assign tasks to team members and supervise performance. S/he will be the main point of contact between the evaluation Activity Manager at USAID, QED and the Evaluation Team.  The Team Leader will review all outputs and be responsible for submitting quality products to USAID on a timely basis.  Developing, managing and, as needed, communicating updates to USAID on evaluation progress  Ensuring that the tasks within the evaluation work plan are those best suited (and most efficient) to achieving the objective of the evaluation. Education and Relevant experience:  Advanced degree in Social Research Methods, Social Sciences, Public Administration and Management or related fields.  Significant expertise in conducting evaluations with a minimum of 10 years’ experience in evaluation management with some evaluations conducted on governance issues at the local government level.  Played significant role designing, managing or executing a minimum of seven evaluations, at least five of them preferable in local government governance issues, public sector activities.  Must have been a team leader for a minimum of five evaluations, with at least one evaluating activities of similar scale.  Skills in quantitative and qualitative evaluation design and methods.  Must have demonstrated experience producing high-quality reports for evaluations of complex, multi-sector programs. Other team members will perform tasks in a professional manner and assist the Team Leader to develop and review deliverables. (ii) Local Governance Expert Responsibilities:  S/he will contribute to the design of relevant tools, collect data, prepare and participate in the writing the report. Education and Relevant experience:  A minimum of a Master’s degree in governance of local governments or Public Administration and Management or Development Studies, Applied Research or related fields.  Minimum of seven years’ experience implementing and evaluating local government development programs.  Technical expertise in strengthening local government systems.  Must demonstrate knowledge and experience with the functioning of decentralized local governments in Uganda. (iii)Public Finance and Accountability Expert Responsibilities: 59  S/he will contribute to the design of relevant tools, collect data, prepare and participate in the writing the report. Education and Relevant experience:  Postgraduate degree in economics, finance and accounting or in any other relevant discipline.  Demonstrable expertise and working knowledge in the following areas: public procurement and public finance management.  Proven experience assessing fiscal policy including public expenditure management, budget planning and public procurement,  Good understanding of the public sector in Uganda  Excellent analytical and reporting skills 7.0 Institutional Relationship and Reporting  Whereas the consultants will be supervised by the QED Group LLC in their daily work, they will be answerable to USAID for all deliverables  USAID will review and approve all deliverables produced by the consultants for this evaluation 8.0 Level of Effort (LOE) required The GAPP mid-term evaluation is estimated to begin on June 27th, 2016 and be completed no later than September 20th, 2016. Task LOE  Literature review and Preparation of inception report 10  Fieldwork and presentation of preliminary findings 15  Preparation of draft evaluation report 12  Preparation of final evaluation report 5 Summary of LOE by Evaluation Team members Team leader 42 Local governance expert 25 Public finance management & accountability expert 25 CSO and Citizen participation expert 25 9.0 Roles and Responsibilities USAID Monitoring, Evaluation and Learning program USAID Monitoring, Evaluation and Learning Activity, implemented by the QED Group LLC will be responsible for the evaluation by ensuring the recruitment and management of a competent team of consultants to execute the assignment in strict compliance with USAID standards and contracting 60 requirements. QED Group LLC will be responsible for the day-to-day management of the consultants and quality assurance of all products and deliverables before they are submitted to USAID. This shall involve: ▪ Review of USAID SOW and provision of comments especially on the clarity of the tasks and allocation of effort ▪ Procure and supervise consultants ▪ Review consultants’ work, especially key deliverables, to ensure they respond to the SOW and meet USAID quality standards ▪ Facilitate and supervise fieldwork, including advise the consultants on identifying translators where needed ▪ Provide office space, assistance with logistics, and requirements as required by the consultants while conducting the evaluation ▪ Submit a copy of the duly approved Mid-Term Evaluation Report to the Development Experience Clearing House USAID’s roles and responsibilities include: ▪ Review and approval of the SOW and all deliverables from the consultants ▪ Provide recommendations to and clearance on selected evaluators for this assignment GAPP roles and responsibilities are to: ▪ Provide guidance to the consultants with setting up meetings and interviews, and identification of the sites ▪ Review and comment on draft deliverables. 10.0 Schedule and Logistics 11. Internal Timelines a) Finalization of the SOW, mid-April b) Sourcing and procurement of consultants, April to end of May 2016 c) Finalization of Fieldwork, August 12, 2016 d) Report Finalization, September 16, 2016 61 ANNEX 2: EVALUATION METHODOLOGY Annex 2 presents in greater depth the evaluation methodology used during the GAPP MTE. Evaluation Team The evaluation team consisted of an international Team Leader and three local subject matter experts: Public Finance Management & Local Accountability Expert, Local Government Expert, and Civil Society & Civic Engagement Expert. Team Leader: Dr. Gary Woller was responsible for overseeing all aspects of the evaluation and for ensuring the successful and timely completion of all evaluation deliverables. Dr. Woller focused primarily on interviewing Kampala-based stakeholders, including members of the GAPP management team, ETS based in GAPP’s Kampala office, and key informants at USAID, DFID and other development programs working in the same or similar space as GAPP. He also participated in interviews with district and municipal government officials in Masaka District and Masaka Municipality. Public Finance Management & Local Accountability Expert: Daniel Lukwago focused on interviewing key informants at national-level accountability institutions participating in Component 1 activities—including parliamentary oversight committees, autonomous government agencies and GoU ministries and agencies in addition to national-level CSOs participating in GAPP Component 3 activities. He also participated in interviews with district and municipal government officials in Masaka District and Masaka participating in GAPP Component 2 activities. Local Government Expert: Dr. John Kiyaga-Nsubuga focused on interviewing local government officials participating in Component 2 activities, including those interacting with and/or benefitting from GAPP’s Component 1 activities. Civil Society & Civic Engagement Expert: Zie Gariyo focused on interviewing regional GAPP staff (including the Regional Coordinators and ETS’ based in three of GAPP’s regional offices), key informants from district-level and national-level CSOs participating in GAPP Component 3 activities, based both in the districts and in Kampala, and local community members benefitting from the activities of national and district-level CSOs receiving GAPP grant support. Geographic Coverage As seen in Table A2.1, the MTE fieldwork covered five regions, 12 districts, five municipal councils and one municipal division. All fieldwork for Component 1, in addition to fieldwork on crosscutting topics, took place in Kampala. 62 Table A2.1. Districts Visited during the MTE Fieldwork Region/District Component 2 Component 3 Lango Apac District X Lira District X Lira Municipal Council X Oyam District X Acholi Gulu District X X Gulu Municipal Council X West Nile Arua District X X Koboko District X Koboko Municipal Council X Bunyoro Kiryandongo District X Hoima Municipal Council X Masindi District X Central Masaka District X Masaka Municipal Council X Kayunga District X To select the MTE study districts, the MTE team drew on the criteria described below:  SOW Constraints: The MTE SOW allocated only 10 days for fieldwork, so it would not be possible to achieve a comprehensive coverage of the 35 GAPP districts during the fieldwork.  Regional representation: Fieldwork took place in all five regions in which GAPP currently works: Acholi, Bunyoro, Central, Lango, and West Nile.  Coverage of GAPP regional offices: Selection of districts prioritized those districts in which GAPP has its regional offices.  Geographic dispersion and travel time: The MTE team sought to select districts that were close enough to each other, and connected by good road networks, that would allow them to both achieve a reasonable representation of the 35 districts in which GAPP works and to spend a sufficient amount of time in each district to meet with the targeted stakeholders.  Urban vs. rural representation: The MTE team sought to achieve a reasonable representation of urban and rural districts. Of the nine districts selected for the fieldwork, three—Apac, Kayunga 63 and Kiryandongo—are considered rural districts, while the remaining are considered urban districts.  New vs. old districts: The MTE team sought to include at least one of the new districts added in conjunction with the DFID funding. Of the 12 districts covered by the fieldwork, Kiryandongo is a new district, while the remaining eight are ‘old’ districts in which GAPP has been working since Activity inception.  Access to municipal councils: The MTE team sought to include districts in which municipal councils existed nearby so as to ensure that the fieldwork for Component 2 also included a reasonable representation of municipal councils in addition to district governments. Overall, five municipal councils were covered by the fieldwork located in Gulu, Hoima, Koboko, Lira and Masaka.  Implementation progress: The evaluation team sought to select districts in which a number of activities had already been implemented and were far enough along in their progress that results might be observed and/or lessons learned. Data Collection Methods The GAPP MTE used four primary data collection methods: secondary document review, KIIs, FGDs and a validation workshop. A short description of each is found below. Secondary document review: The MTE team reviewed a total of 43 documents, including documents internal to the GAPP Activity and documents external to the GAPP Activity, including in the latter case background documents produced by USAID/Uganda and independent evaluations of other USAID￾funded DRG-related programs in Uganda. Key informant interviews: KIIs are in-depth, semi-structured and facilitated discussions conducted with small groups of Activity stakeholders. KIIs are semi-structured in that they utilized a discussion guide listing the topics to be covered during the interview, while also following an open-ended format allowing interviewees the space to describe their experiences in their own words and allow the interviewer to probe for information to uncover new insights into key aspects of stakeholder perceptions, GAPP’s performance and the Activity operational environment and to pursue other lines of inquiry as they emerged during the interview. KII discussion guides were created for different categories of GAPP stakeholders relevant to their role in relation to the Activity. KIIs took on average about 60 minutes to complete depending on the stakeholder interviewed. Interviewers took and then transcribed detailed notes for each KII. Tables A2.2-A2.4 show the number key informants interviewed during the MTE fieldwork related to GAPP Components 1-3 distributed by the type of stakeholder, while Table A2.5 shows the number of KIIs conducted for ‘other’ stakeholders not falling specifically under Components 1-3. 64 Table A2.2. Number of Key Informants Interviewed for GAPP Component 1 Organization Number Parliament Committee of Public Services & Local Government (PS&LG) 1 Local Government Accounts Committee (LGAC) 1 Public Accounts Committee (PAC) 1 Institute of Parliamentary Studies (IPS) 1 Corporate Planning and Strategy (CPS) 2 Total Parliament 6 National Accountability Institutions Office of the Auditor General (OAG) 3 Public Procurement and Disposal of Assets Authority (PPDA) 2 Total NAIs 5 Other Institutions National Planning Authority (NPA) 1 Local Government Finance Commission (LGFC) 1 Ministry of Local Government (MoLG) 1 Financial Management and Accountability Programme (FINMAP) 1 Total Other Institutions 4 National-Level CSOs Uganda Local Government Association (ULGA) 1 Centre for Policy Analysis (CEPA) 3 Civil Society Budget Advocacy Group (CSBAG) 1 Total National-Level CSOs 4 Total 19 Table A2.3. Number of Key Informants Interviewed for GAPP Component 2 Location Technical Staff Elected Officials GAPP ETS Total Lango Region - - 2 2 Acholi Region - - 3 3 West Nile Region - - 2 2 Bunyoro Region - - 2 2 Central Region - - 3 3 Apac District 4 - - 4 Lira Municipal Council 8 5 - 13 Gulu District 6 7 - 13 Gulu Municipal Council 1 - - 1 Arua District 5 11 - 16 Koboko District 2 7 - 9 Koboko Municipal Council 2 2 - 4 Kiryandongo District 6 7 - 13 Hoima Municipal Council 4 8 - 12 Masaka District 13 4 - 17 Masaka Municipal Council 4 4 - 8 Kimanya/Kyabakuza (Masaka Division) 1 0 - 1 Total 56 55 12 123 65 Table A2.4. Number of Key Informants Interviewed for GAPP Component 3 Organization Number District CSOs Centre of Governance and Economic Development (CEGED) 1 Mayank Anti-Corruption Coalition (MACCO) 3 National Community of Women Living with HIV/AIDS (NACWOLA) 5 Youth Development Initiatives in West Nile (YDI) 1 Human Rights Activist Monitoring Organization (HURAMO) 5 Lira District NGO Forum (LDNGOF) 1 Global Forum for Development (GLOFOD) 1 Community Development and Conservation Agency (CODECA) 1 Reconciliation for Development (RFD) 1 Agency for Desired Development (A4D) 9 Build Africa 2 Total District CSOs 30 National CSOs Civil Society Budget Advocacy Group (CSBAG) 1 Forum for Women in Democracy (FOWODE) 2 Advocates for Coalition for Environment and Development (ACODE) 4 Coalition for Health Promotion and Social Development (HEPS) 3 Southern and Eastern African Trade Information and Negotiations Institute (SEATINI) 3 African Youth Development Link (AYDL) 3 Uganda Debt Network (UDN) 2 Transparency International in Uganda (TIU) 3 Uganda Women's Network (UWONET) 2 Action for Development (ACFODE) 2 Community Integrated Development Initiatives (CIDI) 6 Total National CSOs 31 Government Officials Arua 6 Gulu 1 Total Government Officials 7 Total 66 66 Table A2.5. Number of Other Key Informants Interviewed Organization Number GAPP 10 USAID Uganda Mission 12 GAPP Expert Panel 2 DFID 2 GIZ 1 Strengthening Uganda’s Anti-Corruption and Accountability Regime (SUGAR) 1 Strengthening Decentralization for Sustainability (SDS) 2 Monitoring, Evaluation and Learning Program (MELP) 1 Total 31 Focus group discussions: FGDs are in-depth moderated discussions with small groups of GAPP stakeholders participating in or benefitting from Component 3 activities. A major advantage of FGDs, as opposed to other forms of qualitative research, is their flexibility for in-depth exploration of the attitudes and experiences of FGD participants towards GAPP, processes used during GAPP Activity implementation and the results of the activities in which they participated. FGDs are semi-structured in that they use a discussion guide but also involve in-depth probing by the FGD facilitator, while also allowing the facilitator the flexibility to pursue other avenues of inquiry as they emerge. To help keep participants focused throughout the session, FGDs were organized at the venues familiar to the participants. FGD discussion guides were created for different categories of GAPP stakeholders relevant to their role in relation to the Activity. FGDs included on average nine participants, and took on average about 60 minutes to complete. The FGD moderator took and then transcribed detailed notes for each FGD. Table A2.6 shows the focus groups conducted, their distribution across GAPP districts and across GAPP stakeholders, and the number of FGD participants (both male and female) in each case. Table A2.6. Number of FGD Participants District FGD Male Female Total Arua Potential CSO Networks 1 2 3 Arua Community Volunteer Groups (CAGs) 5 5 10 Arua Arua Livestock Dealers Association 4 0 4 Arua Arua Market Vendors/Women Association 5 2 7 Arua Arua Boda Boda Association 7 0 7 Arua CSO consultative meeting (New) Arua District 1 2 3 Gulu Palaro Market Vendors Association 4 12 16 Lira TIU Lira Officer/Local Partners Meeting 5 1 6 Lira HEPS Lira Officer/Local Partners 3 3 6 Lira Youth Assemblies Lira Municipality 6 7 13 Oyam Oyam Youth Advocates Foundation 3 6 9 Masindi Masindi Motor Cycle (Boda Boda) Operators Association 6 0 6 Masindi Nyantonzi Primary School (PTA/SMC/Staff) 8 4 12 Masindi Masindi Market Women Vendors Development Group 0 20 20 Kayunga Kayunga District NGO Forum 3 4 7 Kayunga Kayunga District Farmers Association 0 3 3 Total 61 71 132 67 Validation workshop: On October 18, 2016, the USAID Monitoring, Evaluation and Learning Program (MELP) facilitated a half-day validation workshop in Kampala for selected GAPP stakeholders. At the evaluation workshop, members of the MTE team presented their preliminary findings, conclusions and recommendations and facilitate a discussion around them with the participating stakeholders. Participants in the validation workshop were drawn from diverse stakeholder groups located in or near Kampala and were purposively selected to participate based on their identified interested in and commitment to GAPP. General objectives of the validation workshop were to disseminate the evaluation findings to key GAPP stakeholders in a participatory format so as to increase understanding of the MTE findings and improve their utility with respect to stakeholders’ values and views. As such, the validation workshop served the function of the After Action Review proposed in the evaluation SOW. More specific objectives of the validation workshop are as follows:  Identify and correct factual errors and ambiguities in the MTE findings.  Review the key issues that emerged from the analysis of the MTE findings for validation by key GAPP stakeholders.  Make sure that key issues related to GAPP operations and outcomes are not missed and, if so, to ensure their inclusion in the final MTE report.  Identify areas for action by GAPP and other activity stakeholders. Data Analysis Figure A2.1 below traces through the general approach to content analysis used by the MTE team to analyze the data collected during the MTE fieldwork. The specific approach to applying content analysis to the MTE data involved the following six steps: Step 1—Extract and Summarize the Data: This step involved reading through all of the data (text) collected and summarized the information that is relevant for answering the evaluation questions, while temporarily setting aside information that appears less relevant for answering these questions. Step 2—Identify Patterns and Connections: This step involved coding the data extracted in Step 1. Using a standardized iterative process, recurrent themes and sub-themes were identified along with the individuals or organizations expressing those concepts. Step 3—Analyze the Data: This step involved analyzing the data so as to better understand whether and how Activity interventions have affected GAPP stakeholders, the significance of these effects, how they are distributed among stakeholders and stakeholder subgroups, and the process by which they occurred. During this step, specific quotes or examples to be used in the GAPP MTE Report were extracted from the data. Step 4—Review, Reflect, and Discuss: This step involved a thorough review of the analysis and findings, including going back to the information set aside in Step 1 and giving it a second look to reassess its relevance to the analysis and findings and, where appropriate, adding it back into the analysis. Step 5—Answer the Evaluation Questions: This step involved taking the results of the analysis done in Steps 1-4 and applying them to provide specific and detailed answers to each of the evaluation questions posed in the evaluation SOW. 68 Step 6—Construct Case Studies: Once the data was analyzed using the above methodology, the MTE team selected four studies to highlight an illustrative set of GAPP activities and their results. Figure A2.1. Basic Approach to Content Analysis 69 ANNEX 3: BIBLIOGRAPHY Adjumani, Alebtong. (2015). “National Accountability Agencies Outreach to Nine Local Governments,” RTI International & Development Alternatives, Inc. Bosco, John Asiimwe, Julian Bagyendera, Andrew Balyeku, William Emmet, Namudda Jolly, Albert Kalangwa, Joy Mukaire, Edgar Mulogo, Caroline Nabacwa, Evelyn Namubiru, Xavier Nsabagasani, Nathan Nshakira, Vincent Owarwo, Eric Sewagudde, Barbara Tamala, Joel Tamale, and Charlotte Zikusooka. (2015). “Evaluation of USAID’s District-Based Technical Assistance (DBTA) Model as Applied under Strengthening Tuberculosis and HIV/AIDS Responses (STAR) Projects in East, East-Central, and South￾West Uganda,” QED Group LLC. Cardno Emerging Markets. (2012), “Induction of Local Government Councilors: Final Training Report.” Centre for Policy Analysis. (2104). “Governance, Accountability, Participation, and Performance (GAPP) Program Strengthening the Oversight Function of Parliament of Uganda: An Assessment of Accountability Committees.” Center for Policy Analysis. “Local Government Accountability: Key Extracts from the Auditor General’s Report on Local Government, Health, Education and Agriculture.” Center for Policy Analysis. (2014). “Strengthening the Oversight Function of Parliament of Uganda: An Assessment of Accountability Committees.” Center for Policy Analysis. (2015). “Policy and Practices Issues for Reform: Strengthening the Oversight Functions of Parliament.” Development Alternatives, Inc. (2013). “Governance, Accountability, Participation and Performance (GAPP) Program: Assessment of Capacity of National Level Institutions.” Development & Training Services. (2013). “Uganda GAPP Project: Gender and Social Inclusion Assessment.” Kibombo, Vincent Owarwo, Alfred Nuamanya, Nasreen Jessani, Kenneth Otikal, Patrick Mubangizi, and Carol Awiimwe. (2016). “Evaluation of USAID Strengthening Decentralization for Sustainability (SDS) Project.” Local Government Finance Commission. (2014). “Annual Report.” Local Government Commission. (2012). “Review of Local Government Financing: Financing, Management and Accountability for Decentralized Service Delivery,” p. 152. Monitoring, Evaluation and Learning Program. (2016). “Learning from Evaluations: District Level Service Delivery and Local Government Partnership.” National Planning Authority. (2015). “Second National Development Plan: 2015/16-2019/20.” Onya, Okebe and Patrick Kandole. (2014). “Report on the Study of National and Local Government Audit Institutions,” RTI International. 70 Public Procurement Disposal of Public Assets Authority. “PPDA Report on Procurement and Disposal Audit Reports for FY 2012/13, 2013/14 and 2014/14 in USAID/GAPP Funded Areas.” RTI International. (2013). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: January 2013–March 2013.” RTI International. (2013). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: April 2013–June 2013.” RTI International. (2013). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: July 2013–September 2013.” RTI International. (2013). “Terms of Reference for the Governance, Accountability, Participation And Performance (GAPP) Program Expert Panel.” RTI International. (2014). “Governance, Accountability, Participation, and Performance (GAPP) Program Partner Local Organization Capacity Assessment: Uganda Local Government Association.” RTI International. (2014). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: October 2013–December 2013.” RTI International. (2014). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: January 2014–March 2014.” RTI International. (2014). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: April 2014–June 2014.” RTI International. (2014). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: July 2014–September 2014.” RTI International. (2015). “Creating an Enabling Environment for Improved Service Delivery: Stories from the Field,” Governance, Accountability, Participation and Performance Program.” RTI International. (2015.) “Governance, Accountability, Participation and Performance (GAPP) Program Citizen Participation in Local Governance: Baseline Survey.” RTI International. (2015). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: October 2014–December 2014.” RTI International. (2015). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: January 2015–March 2015.” RTI International. (2015). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: April 2015–June 2015.” RTI International. (2015). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: July 2015–September 2015.” RTI International. (2015). “Terms of Reference for the Governance, Accountability, Participation and Performance (GAPP) Program Expert Panel.” 71 RTI International. (2016). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: October 2015–December 2015.” RTI International. (2015). “Creating an Enabling Environment for Improved Service Delivery: Stories from the Field.” RTI International. (2016). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: January 2016–March 2016.” RTI International. (2016). “Governance, Accountability, Participation and Performance (GAPP) Program Quarterly Report: April 2016–June 2016." RTI International. (2016). “Governance, Accountability, Participation and Performance (GAPP) Program Revised Activity Monitoring and Evaluation Plan.” SEATINI-Uganda. (2016), “End of Project Evaluation: Enhancing Opportunities for Local Revenue Mobilization and Accountability for Improved Service Delivery in Kitgum, Pader and Lamwo Districts.” SoCha LLC. (2016). “District Operational Plan Final Evaluation.” Tetra Tech ARD. (2016). “Democracy, Human Rights, and Governance Assessment of Uganda.” USAID. (2011). “Evaluation Learning from Experience: USAID Evaluation Policy.” USAID. (2015). “Strengthening Governance and Accountability at the Local Level: Governance, Accountability, Participation and Performance (GAPP): Statement of Work,” Global Health Performance Cycle Improvement Project. USAID Uganda. (2016). “USAID Country Development Cooperation Strategy: Final Draft-August 2016.” 72 ANNEX 4: KII AND FGD DISCUSSION GUIDES NATIONAL LEVEL INSTITUTIONS General Questions  (EQ1) At the outset of the programme what did you understand to be the rationale and aims behind Governance, Accountability, Participation and Performance (GAPP)? What led to this understanding? (e.g. induction / theory of change workshop)  (EQ1) To what extent do you think the GAPP’s hypothesis “if local government systems are strengthened and non-government led efforts are supported to improve accountability and democratic governance; service delivery will be more equitable, effective and efficient” is true? Has this changed over time and if so, in what ways?  (EQ3) How well do you think the GAPP interventions are contributing to, or will contribute to improvement in the performance of:  National accountability institutions in their oversight function at both national and within LGs?  Systemic functioning of LGs?  Citizen participation in local governance processes and service delivery?  Unblocking voice and accountability channels in local government?  (EQ4) Is GAPP focusing on the right organisations/ institutions and processes to improve local governance and accountability?  (EQ3) What do you see as the primary challenges facing local governance and service delivery in Uganda today? To what extent do you think GAPP is helping in addressing these challenges?  (EQ1) Given the proliferation of local governments, which level of LG do you think is the best place for GAPP to target assistance (i.e. district, municipality, sub county)? Why? Public Accounts Committees (PAC) & Local Government Accounts Committee (LGAC)  (EQ1) What activities are you implementing with support of GAPP? Why these activities? What are you seeking to accomplish and how?  (EQ4) What type of technical and other capacity building assistance have you received from GAPP? What are the major results?  (EQ3) To what extent has GAPP enhanced the functioning of your committee towards fulfilling its mandate (e.g., oversight of central and LG finances)?  (EQ3) During the last two years, what changes has your committee made in the way it handles audit queries and recommendations by the OAG? How are these changes impacting on public finance management and accountability in Uganda?  (EQ3) To what extent has the program enhanced your interactions and working relations with other accountability and oversight bodies/ entities (i.e. OAG, DPP, PPDA) (i.e. prosecutions, investigations, action plans and resolutions)? How effective are these collaborations? What are some of the key results? Challenges? 73  (EQ3) To what extent has the program helped to improve your committee’s collaboration and engagement with LGPACs? How effective are these collaborations? What are some of the key results?  (EQ3) To what extent has the GAPP support enhanced outreach activities to LGs and communities? How are these outreaches changing local accountability?  (EQ3) What challenges does your committee have in fulfilling its mandate [i.e. fully process the audits, handling the backlog of audit reports-annual and 44 VFM audits, delays in (or lack of) Parliamentary Approvals of PAC Reports]?  (EQ3) What changes/improvements should be made by GAPP in order to help your committee address some of these challenges? Committees on Public Services & Local Governments (PS & LGs)  (EQ1) What activities are you implementing with support of GAPP? Why these activities? What are you seeking to accomplish and how?  (EQ4) What type of technical and other capacity building assistance have you received from GAPP? What are the major results?  (EQ3) To what extent has GAPP affected the functioning of your committee towards fulfilling its mandate (e.g., oversight on matters of LG policy, including LG financing issues)?  (EQ3) To what extent has GAPP helped to strengthen national stakeholders’ collaboration and dialogue on LG financing, accountability and service delivery? Key results? Challenges?  (EQ3) What challenges does your committee have in fulfilling its mandate?  (EQ3) What changes/improvements should be made by GAPP in order to help your committee address some of these challenges? Institute for Parliamentary Studies (IPS)  (EQ1) What activities are you implementing with support of GAPP? Why these activities? What are you seeking to accomplish and how?  (EQ4) What type of technical and other capacity building assistance have you received from GAPP? What are the major results?  (EQ3) To what extent has GAPP supported you in building the capacity of Members of Parliament and parliamentary staffers?  (EQ3) To what extent has GAPP helped you to build the capacity of accountability committees of parliament, such as PAC, LGAC, and PS & LG to fulfill their mandate? What the major results of your capacity building efforts?  (EQ3) What challenges does your institution have in fulfilling its mandate?  (EQ3) What changes/improvements should be made by GAPP in order to help your institution address some of these challenges Office of the Auditor General (OAG)  (EQ1) What activities are you implementing with support of GAPP? Why these activities? What are you seeking to accomplish and how? 74  (EQ4) What type of technical and other capacity building assistance have you received from GAPP? What are the major results?  (EQ3) To what extent has GAPP supported your institution to fulfill its mandate (i.e. to carry out value-for-money audits)? How has this support tailored to the needs of your institution? What the key results?  (EQ3) During the last two years, what changes have you noticed in regards to how Parliamentary (PAC, LGAC) handle your audit queries and recommendations? How are these changes impacting on performance of your institution? Challenges?  (EQ3) To what extent has the GAPP support enhanced your outreach activities to LGs and communities? How are these outreaches changing local accountability?  (EQ3) To what extent has the GAPP support enhanced your interactions and working relations with other accountability and oversight bodies/ entities (i.e. PAC, LGAC, DPP and PPDA)? How effective are these collaborations? What are some of the key results?  (EQ3) During the last two years, what changes has your institution made in the way it undertakes audits? How are these changes impacting on public finance management and accountability in Uganda?  (EQ3) What challenges does your institution have in fulfilling its mandate?  (EQ3) What changes/improvements should be made by GAPP in order to help your institution address some of these challenges Public Procurement and Disposal of Asserts Authority (PPDA)  (EQ1) What activities are you implementing with support of GAPP? Why these activities? What are you seeking to accomplish and how?  (EQ4) What type of technical and other capacity building assistance have you received from GAPP? What are the major results?  (EQ3) To what extent has GAPP supported your institution to fulfill its mandate (i.e. carry procurement audits especially at LG levels)?  (EQ3) To what extent has GAPP helped your institution to support and engagement with LG PDUs? How effective is this support and engagement? Any key results? Challenges?  (EQ3) To what extent has the GAPP support enhanced your outreach activities to LGs and communities? How are these outreaches improving local accountability?  (EQ3) To what extent has the GAPP support enhanced your interactions and working relations with other accountability and oversight bodies/ entities (i.e. PAC, LGAC, DPP and OAG)? How effective are these collaborations? What are some of the key results?  (EQ3) During the last two years, what changes has your institution made in the way it handles procurement audits (i.e. strengthen sanctions for flawed procurement and failure to implement PPDA recommendations)? How are these changes impacting on public procurement especially at LG levels?  (EQ3) What challenges does your institution have in fulfilling its mandate?  (EQ3) What changes/improvements should be made by GAPP in order to help your institution address some of these challenges 75 Financial Management and Accountability Programme (FINMAP)  What does your program do? What is it seeking to accomplish and how? Why are you using such strategies / approaches?  (EQ1) What do you know about the GAPP program? How have your programme interacted with GAPP?  (EQ3) To what extent do you think the GAPP is contributing to the strengthening of Public Finance Management and Accountability in Uganda i.e. strengthening interactions and working relations among accountability institutions?  (EQ2) How has GAPP interacted with other Public Finance Management programmes in Uganda? What lessons can be learned from these interactions?  (EQ3) What PFM and accountability reforms have been implemented as a result of or support from GAPP? How effective have these reforms been? Any key results? Challenges?  (EQ1) What do you see as the primary challenges of Public Finance Management in Uganda today? Why? What do you think GAPP con do to contribute towards addressing these challenges? Ministry of Local Government (MoLG) & Local Government Finance Commission (LGFC)  (EQ1) What activities are you implementing with support of GAPP? Why these activities? What are you seeking to accomplish and how?  (EQ3) To what extent has GAPP helped your institution to support the improvement in the functioning of LGs (esp. planning, budgeting, local revenue generation, procurement, audit)?  (EQ3) To what extent has GAPP supported your institution to cause policy dialogue on mechanisms for enhancing LG financing? Any key results? Challenges?  (EQ3) To what extent has GAPP supported your institution to fulfill its mandate (i.e. advice on LG financing)?  (EQ3) To what extent has GAPP helped to strengthen national stakeholders’ collaboration and dialogue on LG financing, accountability and service delivery? Key results? Challenges?  (EQ3) What challenges does your institution have in fulfilling its mandate?  (EQ3) What changes/improvements should be made by GAPP in order to help your institution address some of these challenges Uganda Local Government Association (ULGA)  (EQ1) What activities are you implementing with support of GAPP? Why these activities? What are you seeking to accomplish and how?  (EQ4) What type of technical and other capacity building assistance have you received from GAPP? What are the major results?  (EQ2) How do work with GAPP staff to review programmes, activities and resolve issues that emerge during the course of the activity implementation?  (EQ2) What are the strengths and weaknesses in the GAPP grant making processes? Are they clearly articulated to a sub-grantee like yourself? Are they easily understood? Is there flexibility in their implementation? 76  (EQ3) How has GAPP helped you to enhance your advocacy on local government issues at the national level, especially on LG financing?  (EQ3) To what extent has your engagement with key stakeholders like the MoFPD, MoLG, LGFC, and Parliament led to necessary reforms in fiscal decentralization (e. g, improving of fiscal transfers to LG)? Any key successes? Challenges?  (EQ3) What challenges does your institution have in fulfilling its mandate?  (EQ3) What changes/improvements should be made by GAPP in order to help your institution address some of these challenges? National Planning Authority (NPA)  (EQ1) What do you know about the GAPP program? How have your institutions interacted with GAPP? What activities? Why these activities? What are you seeking to accomplish and how?  (EQ3) What type of technical and other capacity building assistance have you provided to LGs with support from GAPP? What are the major results? Challenges? 77 LOCAL GOVERNMENT OFFICIALS  What GAPP interventions, including technical and other capacity building assistance, have you received to enhance performance and effectiveness in your specific area (i.e. planning and budgeting, financial management and accounting, internal audit, procurement, local revenue generation and management, enhancing accountability)? What are the major results?  What do you see as the primary challenges facing local governance and service delivery in your district today? To what extent do you think GAPP is helping in addressing these challenges?  Given the proliferation of LGs, which level of LG do you think is the best place for GAPP to target assistance (i.e. district, municipality, sub county)? Why?  Specifically: Planning and Finance  What strategies have you developed towards improving local revenue generation and management? Any key results? Challenges in generating local revenues? How can these challenges be addressed?  To what extent have you allocated locally generated revenues to service delivery sectors (education, health, agriculture)?  What strategies have you put in place to improve financial management and accountability? Any key results? Challenges?  How do you involve citizens in planning and budgeting? Any key results? Any challenges? How do propose to overcome these challenges?  To what extent has GAPP helped to improve your department collaboration and engagement with national institutions (i.e. NPA, LGFC, AOG)? How effective are these collaborations? What are some of the key results?  To what extent has GAPP helped to improve your accountability to local citizens? What are some of the key results? Political (District Executive Council, Councilors)  To what extent has GAPP enhanced the functioning of your committee towards fulfilling its mandate?  During the last two years, what policies/ regulations/administrative procedures have you developed and operationalized to improve service delivery at the local level as a result of GAPP support?  During the last two years, what changes has your committee made in the way it handles audit queries and recommendations by the LGPAC? How are these changes impacting on public finance management and accountability in the district?  To what extent has the program enhanced your interactions and working relations with national level institutions (i.e. MoLG, LGFC, OAG, PPDA)? How effective are these collaborations? What are some of the key results? Challenges?  To what extent have you engaged civil society organisations in decision-making (i.e. resource allocation, revenue mobilization) and monitoring service delivery? Any results? Challenges and how they can be overcome? 78  To what extent has GAPP helped to improve your accountability to local citizens? What are some of the key results? Local Government Public Accounts Committee  To what extent has GAPP enhanced the functioning of your committee towards fulfilling its mandate (e.g., oversight of LG finances; number of meetings held year, reviews audit reports, submission of reports to DEC, etc.)?  During the last two years, what changes has your committee made in the way it handles audit queries and recommendations by the DIA? How are these changes impacting on public finance management and accountability in the district?  To what extent has the program strengthened your interactions and working relations with national level accountability and oversight bodies/ entities (i.e. OAG, PPDA)? How effective is this collaboration? Any major key results? Challenges?  To what extent has GAPP helped to improve your committee collaboration and engagement with LGAC of Parliament? How effective are these collaborations? What are some of the key results?  What challenges does your committee have in fulfilling its mandate?  What changes/improvements should be made by GAPP in order to help your committee address some of these challenges?  To what extent has GAPP helped to improve your accountability to local citizens? What are some of the key results? Procurement and Audit  To what extent has GAPP has the GAPP supported your department to fulfill its mandate (e.g., to carry out audits)? How effective has this support been? What are some of the key results? Challenges?  During the last two years, what changes has your institution made in the way it handles procurement / financial audits? How are these changes impacting on public procurement especially at LG levels?  During the last two years, what changes have you noticed in regards to how the District (DEC, LGPAC) handle your audit queries and recommendations? How are these changes impacting on performance of your department? Challenges?  To what extent has the GAPP support enhanced your interactions and working relations with national level institutions (i.e. OAG, PPDA)? How effective are these collaborations? What are some of the key results?  What challenges does your department have in fulfilling its mandate?  What changes/improvements should be made by GAPP in order to help your department address some of these challenges?  To what extent has GAPP helped to improve your accountability to local citizens? What are some of the key results?  (EQ2) How is GAPP’s support to governance helping to improve service delivery in health, education and agriculture?  What challenges are being encountered in this process, and how are they being addressed? 79  How is GAPP coordinating with other actors doing similar work in those areas?  What lessons have been learned in the process?  (EQ1) What changes have been registered due to GAPP interventions with respect to: o Overall local government performance and effectiveness? o Local government accountability to citizens? o Citizen ability to monitor local government programmes? o Responsiveness, equity and efficient use of resources?  What lessons have been learned in these processes?  What improvements are required to generate better results in the above areas?  What GAPP interventions have been particularly successful? Why?  How is the local government coordinating with other actors in enhancing service delivery?  (EQ4) GAPP is currently providing its support to several districts, municipalities and lower local governments. Is this approach likely to produce the most optimal results, or should the support be concentrated in fewer local governments?  If yes, why is the current approach the most optimal one?  If not, to which local government category (districts, municipalities, lower local governments) should GAPP support be directed? Why these local governments?  What challenges have been encountered and how have they been addressed?  What lessons have been learned in the process?  From your observation, what improvements in the interventions may be required to help generate better results?  At what level of government do you think programs like GAPP should focus? For example, district, municipal, town council, etc. Why? Is there any other comment you would like to make? 80 CIVIL SOCIETY ORGANISATIONS (CSOs) and PRIVATE SECTOR ACTORS (PSAs)  EQ1: GAPP Theory of Change  What is your understanding of GAPP’s work, what it intends to achieve and where it intends to achieve it?  What are the strengths and weaknesses of GAPP’s approaches/interventions?  In your view, do you think GAPP approaches are appropriate to achieve GAPP’s Goal?  If yes, in what ways? If not, what are the reasons you give?  EQ2: Coordination  How often do you have meetings with GAPP staff to review programmes, activities and resolve issues that emerge during the course of the activity implementation?  How often do you have meetings with district/sub-counties in the sector (education, health, agricultures) in which you work?  EQ2: Management  What are the strengths and weaknesses in the GAPP grant making processes? Are they clearly articulated to a sub-grantee like yourself? Are they easily understood? Is there flexibility in their implementation?  What are the strengths and weaknesses in the grants management process (including application processes, monitoring and reporting)?  What works well? What does not work well? Give examples.  EQ 3: Citizen participation / GAPP Interventions and Performance  In regard to citizen participation in local governance (decision making) and service delivery, what is your organization’s view of the performance of GAPP’s interventions? What works well? What does not work well?  What has changed/not change? What reasons do you give for the change/no change?  What are the major challenges that impede citizen participation in governance processes?  What is being done to change the situation?  EQ4: Recommendations for the Future  What in your organization’s view/recommendation should GAPP change in its interventions/delivery mechanisms/ strategies if it has to more effectiveness? Efficiency? Impact? Learning?  What reasons do you give for your proposals?  At what level of government do you think programs like GAPP should focus? For example, district, municipal, town council, etc. Why?  EQ5: How is CSO advocacy and monitoring of service delivery contributing to improving the quality and effectiveness of service provision in the LGs /LLGs?  In what ways has GAPP built your capacity in advocacy/engaging decision makers/monitoring service delivery in the sector in which you work? 81  What are the successes /achievements /challenges by your organization in the areas in which you work /the grant was given? What facilitated? What hindered?  What has been your organization’s contribution in facilitating the citizen to monitor service delivery? Participate in planning and formulation of plans and budget at LLG/district level?  What local governance or accountability activities has your organization mobilized citizens to participate in with support from GAPP funding?  What has been the contribution of GAPP in facilitating the citizen to monitor service delivery? Participate in planning and formulation of plans and budget at LLG/district level?  What local governance or accountability activities has your organization mobilized citizens to participate in with support from GAPP funding? [PROBE FOR ACTIVITIES LIKE CONFERENCES, BUDGET TRACKING EVENTS, PARTICIPATION IN DRUGS OR SUPPLIES MONITORING AS PART OF HEALTH MANAGEMENT COMMITTEES, MONITORING STUDENT OR TEACHER ABSENTEEISM AS PART OF SCHOOL MANAGEMENT COMMITTEES ETC.]  Are there platforms /spaces for citizens to engage with decision makers to express their demands for improved service delivery? How well are citizens able to use these spaces to present their issues?  Are citizen issues incorporated in the decisions at LLG /district? Provide evidence. If yes, give examples? If not, give reasons?  Do the decision makers /service providers respond to citizen concerns on the quality and delivery of services? 82 LOCAL GOVERNMENT OFFICIALS ENGAGEMENT WITH CSOS (DHO, DEO, PO)  EQ1: GAPP Interventions  What is your understanding of the work of GAPPP generally in the district?  What in your view has worked well/not worked well?  What the opportunities/weaknesses of the GAPP interventions?  In what ways, have the GAPP contributed to the performance of the district sectors in which they are support?  What the challenges/impediments that affect the performance of GAPP interventions?  EQ2: Coordination & Management  What is your understanding of the work of GAPPP in the education/health/agriculture sectors in the district?  What in your view has worked well/not worked well?  Do you have regular programme review/coordination meetings with GAPP staff/CSO sub-grantees to review performance in the sectors? How often are the meetings?  If not, what is the reason you give?  EQ3: Citizen Participation  Are there spaces for citizen participation in decision-making and service delivery in the district/lower local government?  Are there policies/guidelines for citizen participation in local Government decision-making process?  EQ4: Recommendations for the Future  What factors should GAPP/CSO sub-grantee incorporate in their intervention straggles to achieve higher effectiveness/efficiency/impact/learning from GAPP interventions in the sector?  What reason do you give for your suggestions?  At what level of government do you think programs like GAPP should focus? For example, district, municipal, town council, etc. Why?  EQ5: CSO Advocacy  Are you aware of the work of GAPP-supported CSOs in the health, education and agriculture sectors?  What in your view has been the contribution of CSOs/citizen groups in monitoring service delivery?  Are there any changes in service delivery that you think have been caused by CSOs with GAPP￾supported interventions? If yes, give some examples (sector). If not, what do you think are the challenges CSOs face in implementing the interventions?  What external factors (beyond the control of LG/CSO) impede achievement of interventions? 83 CSO Beneficiaries  EQ1: CSO Interventions  What is your knowledge and understanding of [CSO’s] work/interventions and what they are intended to achieve?  Do you have regular meetings with [CSO] staff to discuss GAPP interventions?  How often do you meet with [CSO’s] staff?  What are the lessons citizens have learnt from [CSO’s] interventions? Give examples.  Describe CSO’s activities you participated in for purposes of influencing local governance or accountability issues.  What enabled your participation? What constrained you fully /effective participation?  EQ3: Citizen Participation in Monitoring and Service Delivery  With [CSO’s] support, what training/orientation have you been given to acquire skills and knowledge of monitoring service delivery? What has worked well? What has not worked well?  What are your experiences in planning, budgeting or monitoring service delivery in education, health, and agriculture sectors at the community level?  What do you like most/east about the work you do? Please give reasons.  What are the achievements/successes/changes resulting from your participation in the planning/budgeting processes, monitoring services delivery in health, education, agriculture? Please give examples.  What are the challenges/impediments that hinder you from full participation?  EQ3: Citizen Voice and Accountability  How has [CSO’s] interventions facilitated you (citizens) to hold regular meetings with Government officials (LLG)/local leaders to present you issues for action?  How often do you consult with government officials and local leaders to present your issues during formulation of plans/budgets?  In what ways do officials/leader communicate to the citizens about the government policies/programmes/and service delivery issues?  What are the challenges/impediments that hinder you (citizen) from effectively engaging with decision makers/service providers?  EQ4: Recommendations for the Future  In what ways should [CSO’s] interventions be improved/changed to achieve better performance in future?  What are the reasons for your suggestions?  EQ5: Citizen Advocacy  In what ways has [CSO’s] intervention enabled citizens to participate in decision making at local government level?  In what ways has [CSO’s] intervention facilitated the citizens to monitoring service delivery in the health, education and agriculture sector? 84  What positive changes have in improving service delivery in the sectors? Give examples.  What challenges/impediments do citizens face in influencing decision making/monitoring service delivery? 85 USAID/DFID STAFF  (EQ1) What is the basis for GAPP’s approach? Why was this approach selected as compared to other approaches?  (EQ1) To what extent have the activities been implemented as you intended /hoped? Where deviations have occurred, why do you think they occurred? How do they affect how you view the program?  (EQ1) What challenges or problems, both internal and external to the program, are you aware of that have arisen? What implications do they have?  (EQ4) Which of GAPP’s activities do you believe to be more effective? Least effective? Why?  (EQ3 & 5) What results have you observed at both the national and local levels in terms of things such as improved capacity of national government accountability institutions, local governments and CSOs; local government accountability and service provision; systemic functioning of local governments; and citizen participation and voice? What factors are contributing to these results and how?  (EQ2) What do you see as the overlap with regards to GAPP in terms of cross-sectoral cooperation and results, particularly across health, education and agriculture?  (EQ2) To what extent are you are collaborating with different actors across sectors – such as health, education and agriculture – among other USAID departments, other donors, government, non-state actors and other stakeholders on issues related to GAPP? What is driving this cooperation or lack of cooperation?  (EQ1) Given the proliferation of local governments, which level of local government do you think is the best place to target assistance? Why?  (EQ1 & 4) What are the key lessons you have learned at this point in the program’s implementation?  (EQ1) Are you familiar with the GAPP Theory of Change? [Review Theory of Change.] Do you believe the GAPP Theory of Change is still valid? Why or why not?  (EQ1) Are you familiar with the assumptions underlying GAPP Theory of Change? [Review assumptions.] Do you believe the assumptions are still valid? Why or why not?  (EQ4) Knowing what you now know, what recommendations do you have for improving GAPP and for future programming in this area? 86 GAPP Staff  (EQ1) What activities are you implementing and with what partners?  (EQ1) To what extent does the program address an identified need? How well does the program align with government priorities?  (EQ1) Why these activities and these partners? What are you seeking to accomplish and how?  What results have you achieved to date?  (EQ1) To what extent have the activities been implemented as planned? Where deviations have occurred, why did they occur?  (EQ1) What challenges or problems, both internal and external to the program, have you encountered and how have you dealt with them or plan to deal with them?  (EQ4) How well is GAPP being implemented? What has been done in an innovative way? What should be done differently so as to increase GAPP’s effectiveness?  (EQ1) What strategic adaptations have you made to the program strategy, design or implementation? Why did you make them? What has been the result?  (EQ1) What further adaptations are you considering? Why?  (EQ3 & 5) What results have you observed at both the national and local levels in terms of things such as improved capacity of national government accountability institutions, local governments and CSOs; local government accountability and service provision; systemic functioning of local governments; and citizen participation and voice? What factors are contributing to these results and how?  (EQ4) Which of your activities have proven to be more effective? Least effective? Why?  (EQ2) To what extent are you observing collaboration among different actors across sectors – such as health, education and agriculture – among government and non-state actors, USAID and other donors, GAPP and other stakeholders? What is driving this cooperation or lack of cooperation?  (EQ2) To what extent are you observing synergies in local government accountability and service delivery across different sectors, particularly health, education and agriculture?  (EQ1) Given the proliferation of local governments, which level of local government do you think is the best place to target assistance? Why?  (EQ4) What are the key lessons you have learned at this point in the program’s implementation?  (EQ1) Are you familiar with the GAPP Theory of Change? [Review it with interviewee.] Do you believe the GAPP Theory of Change is still valid? Why? 87  (EQ1) Are you familiar with the assumptions underlying the GAPP Theory of Change? [Review the assumptions.] Do you think the assumptions are valid? Why?  (EQ4) Knowing what you now know, what recommendations do you have for improving GAPP and for future programming in this area? 88 GAPP REGIONAL COORDINATORS AND EMBEDDED TECHNICAL SPECIALISTS  (EQ2) How is GAPP’s support to governance helping to improve service delivery in health, education and agriculture? What challenges are being encountered in this process, and how are they being addressed?  How is GAPP coordinating with other actors doing similar work in those areas?  What lessons have been learned in the process?  (EQ1) What changes have been registered due to GAPP interventions with respect to: o Overall local government performance and effectiveness? o Local government accountability to citizens? o Citizen ability to monitor local government programmes? o Responsiveness, equity and efficient use of resources?  What lessons have been learned in these processes?  What improvements are required to generate better results in the above areas?  What GAPP interventions have been particularly successful? Why?  How is the local government coordinating with other actors in enhancing service delivery?  (EQ4) GAPP is currently providing its support to several districts, municipalities and lower local governments. Is this approach likely to produce the most optimal results, or should the support be concentrated in fewer local governments?  If yes, why is the current approach the most optimal one?  If not, to which local government category (districts, municipalities, lower local governments) should GAPP support be directed? Why these local governments?  What challenges have been encountered and how have they been addressed?  What lessons have been learned in the process?  From your observation, what improvements in the interventions may be required to help generate better results?  Is there any other comment you would like to make? 89 OTHER PROGRAMS (DGF, SDS, USMID, SUGAR, GIZ)  What does your program do? What is it seeking to accomplish and how? Why are you using this approach?  What results have you achieved to date?  (EQ1) What do you see as the primary challenges facing local governance in Uganda today? Why? How should these challenges be addressed? How is your program helping in addressing these challenges?  What do you know about the GAPP program? Its approach to strengthening local governance? What are your perceptions of GAPP?  (EQ1) [Explain GAPP’s Theory of Change.] What do you think about GAPP’s Theory of Change? Do you think it is valid? Why or why not?  To what extent are different programs working on local governance collaborating or sharing learnings? In case of any collaboration, what have you learned from these other programs? From GAPP specifically?  (EQ1) Given the proliferation of local governments, which level of local government do you think is the best place to target assistance? Why?  (EQ4) What are the key lessons you have learned at this point in your program’s implementation? Knowing what you now know, what recommendations do you have for improving governance programs in Uganda and for future programming in this area? 90 ANNEX 5: PERSONS INTERVIEWED Name Organization Position Phone Email GAPP Eva Matsiko GAPP Chief of Party 0774-568887 ematsiko@uganda-gapp.rti.org Tom Kyakwise GAPP Deputy Chief of Party 0757-578580 tkyakwise@uganda-gapp.rti.org Sarah Okwarre GAPP CSO Private Sector Advisor 0783-876540 sokwaare@uganda-gapp.rti.org Albert Oduman GAPP Nat’l Accountability & Policy Advisor 0751-510007 aoduman@uganda-gapp.rti.org Chris Gumisiriza GAPP Grants Manger 0776-280016 cgumisiriza@uganda-gapp.rti.org Jonathan Gamusi GAPP MEL Manager 0776-280082 jgamusi@uganda-gapp.rti.org Deborah Kyamagwa GAPP Senior MEL Specialist 0776-203121 dkyamagwa@gmail.com Charity Komujjurizii GAPP MEL Officer 0776-280022 ckomujjurizi@uganda-gapp.rit.org Godfrey Egulwa GAPP Central Region ETS 0783-875385 gegulwa@rti.org Freddie Kasirye GAPP Central Region ETS 0785-612720 fkasirye@rti.org Bernard Mfitmukiza GAPP Central Region ETS 0780-246295 bmfitmukiza@rti.org Joshua Okebe GAPP Lango Region ETS 0776-203057 okebeonya@gmail.org Wycliff Magala GAPP Lango Region ETS 0785-613012 wmagala@rti.org Andrew Kuteesa GAPP Bunyoro Region ETS 0785-612752 akuteesa@uganda-gapp.rti.org Damalie K. Namuyiga GAPP Bunyoro Region ETS 0772-824704 dnamuyiga@rti.org Daniel Eriku Ibaga GAPP West Nile Region ETS 0705-616868 deriku@rti.org Harold Ronnie Miiro GAPP West Nile Region ETS 0772-180916 hmiiro@uganda-gapp.rti.org Michael Tebere GAPP Acholi Region ETS 0783-875524 mtebere@rti.org Robert Opira GAPP Acholi Region ETS 0772-877655 ropira@tri.org Henry Semanda GAPP Acholi Region ETS 0776-203048 rsemanda@rti.org Mr. Moses Banduga GAPP IT Assistant/UBRIDGE 0776-251719 mbamduga@uganda-gapprti.org Godber Tumushabe GAPP Expert Panel 0752-841312 godber.tumushabe@gmail.com Edgar Agaba GAPP Expert Panel 0772-506445 edagaba@hotmail.co.uk USAID Harriett Muwanga USAID GAPP COR 0772-138465 hmuwanga@usaid.gov Nils Mueller USAID Director DRG Office 0772-138378 nmueller@usaid.gov Morris Nsamba USAID DRG Project Management Specialist 0772-138483 mnsamba@usaid.gov Peter Birigenda USAID M&E Specialist-Economic Growth 0772-138472 pbirigenda@usaid.gov Kyambadde Paul USAID Health Systems Strengthening 0772-138524 akyambadde@usaid.gov Rhobbinah Ssempebuna USAID Health Systems Strengthening rssempebwa@usaid.gov 91 Diana Harper USAID Global Health dharper@usaid.gov Gabby Boutemy Social Solutions MECAP Fellow gboutemy@social solutions.biz Norbert Nubiru USAID Program Management Specialist Civil Society 0772-138546 nmubiru@usaid.gov Sylva Tagabo USAID OFM 0772-138462 stagabo@usaid.gov Daniel Mutumba USAID OFM dmutumba@usaid.gov Leah Denise Wyatt USAID Learning Fellow 0772-138388 lwyatt@usaid.gov DFID Alex Stevens DFID Senior Governance Advisor 0772-700082 a-stevens@dfid.gov.uk Joyce Ngaiza DFID Governance Advisor 0772-700024 j-ngaiza@dfid.gov.uk Development Programs Sandy Richter GIZ Head of Project 077-1004450 sandy.richter@giz.de Stephen Emasu SUGAR Senior Special Advisor 0772-710810 stephen.emasu@sugarfacility.org Denis Okwar SDS Program Chief of Party 0772-666866 denis.okwar@uganda-sds.org Robert Kalemba SDS Program Senior Director for Sustainability 0772-700928 robert.kalemba@uganda-sds.org Charlotta Sandin MELP Deputy Chief of Party 0784-977723 csandin@qedgroupllc.com Component 1 Parliament Hon. Grace Puchiny PS&LG Former Chain & Member LGAC 0774-348001 kwiyucwiny@gmail.com Hon. Winfred Kiiza LGPAC Former Chair 0782-383339 wkiiza@parliament.go.ug Hon. Paul Mwiru PAC Former Vice Chair 0772-448936 mwiruzet@yahoo.comcomcom Charles Binwe IPS Acting Director 0772-642461 cbinwe@parliament.go.ug Dison B. Okumu CPS Director 0712-402347 dbokumu@parliament.go.ug Ms. Josephine CPS 0776 123523 Component 1 National Accountability Institutions Fredrick Akyaire OAG Assistant Director 0772-976310 fredrick.akyaire@oag.go.ug Samuel Iga Kalanda OAG Senior Principal Auditor 0772-344609 samuel.kalanda@oag.go.ug Liz Nambuya OAG Assistant Director 0772-93344614 liz.nambuya@oag.go.ug Benson Turanye PPDA Director Performance Monitoring 0772-414328 bturamye@ppda.go.ug Hilda Mwesigwa PPDA 0772-448936 hmwesigwa@ppda.go.ug Component 1 Other National-Level Institutions David Katungi NPA Senior Planner LG Planning 0772-447475 dkatungi@npa.com Adam Babale LGFC 0772-519889 adam.babale@lgfc.go.ug John Genda Walala MoLG Director LG Inspection 0772-935884 jgwalala008@yahoo.com Mr. Johnson Mutesigensi FINMAP Programme Coordinator 0772 374422 johnson.mutesigensi@finance.go.ug 92 Component 1 National-Level CSOs Emmanuel Kitamirike CEPA Associate Director 0754-140029 kitamirikeemmanuel@gmail.com Regan Wamajji CEPA Content Developer John Asiimwe CEPA Research Coordinator Rose Gamwera ULGA Secretary General rose.gamwera@ulga.org Component 2 Local Government Tom Richard Atim Apac District District Planner 0772-974438 richatim@yahoo.com Gapson Yeko Ogwal Apac District Chief Finance Officer 0774-787444 gapson.yekko@gmail.com Nelson Odwe Apac District Internal Auditor 0772-584068 nelson.odwe@yahoo.com David Opeto Apac District Senior Procurement Officer 0750-671090 david.opeto@yahoo.com Rebecca Acen Lira MC Clerk to Council 0782-321487 becky601@gmail.com Mike Ogwang Olwa Lira MC Mayor 0772-522909 mveveolwa@yahoo.com Walter Oputa Lira MC Secretary for Finance 0782-352523 wopota13@gmail.com Lawrence Aliga Lira MC Deputy Mayor 0775-312837 Harriet Achiro Lira MC Sec. Community Based Services 0774-426391 harrietachiro@gmail.com Richard Okello Lira MC Secretary for Works 0774189350 rokientreprisesltd@yahoo.com Jimmy Okello Lira MC Senior Internal Auditor 0772-669082 jimmy.okello@yahoo.com Nelson Okello Lira MC Internal Auditor 0392-840346 okelnelson@gmail.com Jimmy Okao Lira MC Senior Procurement Officer 0782-312804 okaojimmy@gmail.com Awio Patrick Lira MC Principal Treasurer 0776-54455 awiopatrick@yahoo.com Denis Calvin Ogwal Lira MC Senior Accountant 0773-119911 denisogwalcalvin@gmail.com Denis Okello Ayini Lira MC Statistician 0772-692198 demisronah@gmail.com Denis Nixon Opio Lira MC Senior Planner 0785-291383 opiodenisnixon@gmail.com John Okwanga Gulu District Council Speaker 0777-755014 Lillian Stella Lalam Gulu District Woman LCV Palaro 0774-593832 stellalalam@gmail.com Nancy Akello Gulu District Youth LCV Palaro 0773-393968 Simpleman Oboi Gulu District Laroo Division LCV 0783-933099 justinesimpleman@yahoo.com Jackson Oyugi Gulu District LGPAC Chairperson 0772-880034 jacksonoyugi@yahoo.com Labeja Canon Gulu District LGPAC member 0712-241650 oyelaokot@gmail.com Eugene Oola Gulu District Planner 0772-355690 eugeneoola@yahoo.co.uk Geoffrey Oyoo Gulu District Accounts Assistant 0773-282266 geoffreyoyoo@yahoo.com William Itim Gulu District Accounts Assistant 0772-976881 williamitim@gmail.com Jane Lamony Akello Gulu District Head of Finance 0772-620592 janelamony@yahoo.com Moses Okumu Gulu District Procurement Officer 0782-509926 mosokumu@gmail.com 93 Paska Nyero Gulu District Principal Internal Auditor 0775-084280 paskanyero@yahoo.com Kidega Nicholas Gulu MC Head of Finance 0772-517618 kidegagn@gmail.com Shaphana Andeku Arua District Planner 0772-394420 shaphan.glead@gmail.com Sam Aitaa Arua District Agriculture Head of Finance 0772-455100 aitaasamk@yahoo.com Alex Afayoa Arua District Revenue Officer 0751-818113 afayoaalex@yahoo.com Roy Angumaniyo Arua District Principal Internal Auditor 0772-515719 royangumario@yahoo.co.uk Sam Draku Arua District Procurement Officer 0773-907143 drakusam@yahoo.com Godfrey Opima Arua District Chair, Production 0782-110208 Kamilo Sabo Arua District Chair, Finance, Planning & Admin 0772-515472 sabokamilo@yahoo.com Manzuh A. Hamza Arua District Chairman, Works 0772-666392 mh411908@gmail.com Onduma Sulaiman Arua District Council Speaker 0772-061837 ondumasulaiman@gmail.com Dria Genesis Acema Arua District Vice Chair & Sec. Social Services 0772-892970 acemagenesis@gmail.com Afsa Abas Chandiru Arua District Deputy Speaker 0782-781088 afsaabas@gmail.com Natalie Drateru Arua District Secretary Production 0782997898 draterunatalie@gmail.com Stella Aletiru Arua District Chair, Social Services 0773-885001 stellaaletiru@gmail.com Alayo Buga Arua District DLCV Councilor 0782-980094 buga08@yahoo.com Luke Asua Arua District DLCV Councilor 0772-515532 sualuke@gmail.com Josephine Andiru Arua District Chairperson LGPAC 0777-230764 Sunday Arua Lematia Koboko District Senior Procurement Officer 0772-515418 sundaylematia@gmail.com Ibrahim Abbas Koboko District Internal Auditor 0772-330450 abasibra@hotmail.com Brani Aluma Koboko District Member LGPAC 0772-483903 branialuma@yahoo.com Grace Goro Koboko District Secretary, Social Services 0772-967257 gorograce@gmail.com Hassan Said Nginya Koboko District Chairman LCV 0772-985695 nginyah@yahoo.com Isaac Todoko Koboko District Secretary Production 0782-666912 Godfrey Adrico Koboko District LCIII Chair Dranye Sub-County 0775-858315 Stephen Sanya Mutto Koboko District Vice Chairperson 0782-884303 snyamstephen@gmail.com Jamila Akandru Koboko District Councilor 0777-622324 Francis Bobohi Koboko MC LCIII Chairman, West Division 0782-782025 Peter Atibini Aloro Koboko MC LCIII Chairman, North Division 0773-842080 aloropeteratibini@gmail.com Patrick Anguzu Koboko MC Revenue Officer 0776-375565 panguzu@gmail.com Yasin Amule Koboko MC Head of Finance 0772-861733 amuleyasin@gmail.com Zephaniah Kwizera Kiryandongo Dist Ag. Internal Auditor 0772-636949 kwizerazephaniah@gmail.com Milton Ndiroraho Kiryandongo Dist Procurement Officer 0782-619752 tonicsnyakes@gmail.com 94 Tommy Obong Cook Kiryandongo Dist Chairperson LGPAC 0774-262066 Elizabeth Biingi Kiryandongo Dist Senior Finance Officer 0782-032525 R. James Onyai Kiryandongo Dist Population Officer 0772-830266 mugujula@gmail.com Moses Atuha Ndaaga Kiryandongo Dist Planner 0772-686504 mosesatuha@yahoo.com Edith Adyeri Aliguma Kiryandongo Dist Vice Chairperson 0786-764863 Martin Andiga Kiryandongo Dist Secretary, Finance 0772-344586 andigamartin@gmail.com Fredrick Kyosimire Kiryandongo Dist Speaker 0782-678484 kyosimirefredrick@gmail.com Joan Kunihira Kiryandongo Dist Deputy Speaker 0783-662563 Aronet Katusiime Kiryandongo Dist Councilor PWDs 0784-844044 Rashid Ogetta Kiryandongo Dist Secretary Health 0782-378648 Daniel Dacan Kiryandongo Dist Clerk to Council 0782-339114 dacandaniel@gmail.com Ismail Isingoma Hoima MC Chairperson, Finance Committee 0772-966825 isingomaismail@ail.com Cissy Rwabugoma Hoima MC Member, Finance Committee 0772-325511 rwabugom@gmail.com B. Jane Magezi Hoima MC Member, Finance Committee 0772-488092 janemagezi@gmail.com M. Godfrey Baraza Hoima MC Secretary, Finance 077-613989 godfreympanuka@gmail.com Sula Kasanja Hoima MC Secretary, Production 0772-308641 Daniel Kaseregenyi Hoima MC Clerk to Council 0782-493032 dankaseregennyi@gmail.com Grace Mary Mugasa Hoima MC Mayor 0772-363192 mayorhoima@gmail.com Nelson Businge Hoima MC Deputy Mayor 0774-770602 nesonbusinge@gmail.com Ronald Basiima Hoima MC Sec Community Based Services 0787-803587 basiimaronald@gmail.com Robert Katongole Hoima MC Senior Finance Officer (Revenue) 0772-640074 robert.katongole@gmail.com M. Annet Kambubi Hoima MC Principal Treasurer 0772-644987 annetkambubi@yahoo.com Sunday Lukwamuzi Masaka District District Planner lukyavicent@yahoo.com Hadijah Nankya Masaka District Senior Procurement Officer 0772-035918 95 Mose Mutebi Masaka District District Internal Auditor babumbamutebi@yahoo.com Joseph Kasozi Masaka District Chief Financial Officer 0772-847918 kasozijoseph@gmail.com Nancy Katami Masaka District Senior Finance Officer Nancy.katami@gmail.com Janat Kisekka Masaka District Vice Chairperson LCV Charles Zziwa Masaka District Secretary Finance Jamil Mawanda Masaka District Secretary Production Gordon Mayanja Masaka District Secretary Social Services Shazon Nambuusi Masaka District Secretary LGPAC-Clerk to Council Sharontwijukye1@gmail.com J.B.M Mayanja Masaka District Chairperson LGPAC 0758-687210 Betty N. Lubega Masaka District Member LGPAC 0772-445499 Vincent Kyambadde Masaka District Member LGPAC 0772-674856 Edward Kiwanuka Masaka Municipality Deputy Town Clerk 0755-773377 msmuncov@gmail.com Pontiano Mayiito Masaka Municipality Senior Planner 0701-697429 pmayiito@yahoo.com Sauda Namuleme Masaka Municipality Principal Treasurer 0751-000910 saudasaava@yahoo.com Masaka Municipality Masaka Municipality Chief Finance Office 0706-624616 nantamasandra@gmail.com Ibraham Mutebi Masaka MC Clerk to Council mutebi.ibraham12@gmail.com Denis Nsereko Masaka MC Deputy Speaker 0755-434955 Irene Nantongo Masaka MC Speaker 0755-311411 Siraje Luyiima Masaka MC Chair Finance 0704-045221 Johns Baptist Mukasa Masaka MC Councilor-People with Disabilities 0758-216635 Component 3 Local Government Acia Marion Arua District District Education Officer 0782-565903 aciamarino@yahoo.com Angupale George Arua District /C Rhino Camp Health Centre IV 0772-660647 anupalegeorge@yahoo.com Andima Gabriel Arua District LCIII Chairperson Rhino Camp Sub￾County 0779-3348886 andimagabriel40@yahoo.com Okulega Lawrence Arua District Rhino Sub-County Internal Security Officer 0775-277132 ouklladu@gmail.com Ayikobua Seraphine Arua District Rhino Camp HCIV/Aura Chairman HUMIC 0782-072745 Jumaywa Arua District Rhino Camp HC IV/Arua Staff 0782-582841 Oola Eugine Gulu District District Planner 0772-358696 96 Component 3 National-Level CSOs Julius Mukunda CSBAG Coordinator 0752-202154 jmukunda@csbag.org Jane Nalunga SEATINI Executive Director 0772-581849 jnalunga09@gmail.com Nelly Busingye SEATINI Programme Officer Regina Navuga SEATINI Programme Officer Ahmed Hadji AYDL Team Leader 0702-799015 jahi2007@yahoo.co.uk Teopista Kiiza AYDL Program Coordinator Hassan Ndugya AYDL Project Officer Tumwebaze Patrick UDN Executive Director 0752-822000 Julius Kapwepwe UDN Director of Programmes Kymuhendo Moureen FOWODE Field Officer/Masindi Field Office 0768-670042 moureen.kyomuhendo@fowode.org Emma Kashaija FOWODE Programme Officer 0704-008956 emma.kashaija@fowode.org Arthur Bainomugisha ACODE Executive Director 0772-421963 code@acode-u.org Jonas Mbabazi ACODE Research Officer 0771-665575 Ankunda Phoebe ACODE Research Officer 0706-836870 Peter Wandera TIU Executive Director 0752-504631 pwandera@tiuganda Francis Ekadu TIU Program Coordinator Betty Etimu TIU Programme Officer Prima Kakooza HEPS Director Capacity Building &Training Susan Ether Kilnade HEPS Program Officer 0782-728354 jkilande@heps.or.ug Noerine Mugwangya HEPS Program Assistant/Citizen Engagement 0782-738584 Rita Aciro UWONET Executive Director 0772-468041 raciro@uwonet.or.ug Sarah Agwang UWONET Programme Officer Regina Bafaki ACFODE Executive Director 0782-516006 rbafaki@acfode.org Blenda Kyomuhendo ACFODE Programme Assistant Jjuuko Fulgensio CIDI Executive Director 0782-570444 jfulgnsio@cidiuganda.org Gladys Nantume CIDI Project Manager 0772-628228 gnantume@cidi.org Evelyn Miyingo CIDI Project Coordinator 0775-175868 ezalwango@cidiuganda.org Margaret Nakigudde CIDI Assistant RM 0779-159851 mnakigudd@cidiuganda.org Kevin Bunnya CIDI M&E Officer 0752-944744 kkizito@ cidiuganda.org Rose Mwambazi CIDI SPO WASH 0772-311571 cidi@cidiuganda.org Component 3 District-Level CSOs Abdu A. Moses CEGED Programme Office 0758-320185 abdutiyou@gmail.com Nyimaleng Consolate MACCO Project Officer 0392-945886 conso1@hotmail.co.uk 97 Wadio Neima MACCO Project Officer 0773-33011 Munduni Micheal MACCO Accounts Assistant 0782565328 Aseru Dona Abiniku NACWOLA Chair 0772-535719 aserudona@yahoo.com Atibuni Rose NACWOLA Regional Coordinator, 0772-440602 atibunirose@gamil.com Candiru Beatrice NACWOLA Project Officer 0782-415712 bituracan@yahoo.com Mattima Winfred NACWOLA Finance and Admin. Officer 0980-120665 bituracan@yahoo.com LMs. Lillian Diriwaru YDI Coordinator 0772-448876 lbdiwa70@gmail.com Geoffrey Odong HURAMO Executive Director 0782-557673 huramouganda@yahoo.com Nyeko Paulinus HURAMO Board Chair Onyuta Albert HURAMO Board Secretary Okethi Jaffer HURAMO Accounts Assistant Topaco Stephen HURAMO Field Officer Moses Odong LDNGOF Executive Director 0785-051051 Morris Chris Ongom GLOFOD Chief Executive Officer 0774-016223 gloford.uganda2009@gmail.com Asiku Micah CODECA 0772-905861 Happy Rogers Build Africa Programme Officer/CEEP/Masindi Branch 0774-85112 Specioza Kiwanuka Build Africa Country Director 0772-971766 kiwanuka@build-africa.org David Mwesigwa RFD Programme Officer 0782-165799 albomwe@yahoo.com 98 ANNEX 6: EVALUATION MATRIX Evaluation Question Sub-Questions Performance Measure Data Source (Primary and or Secondary) Data Collection Instrument Data Analysis Plan To what extent has the GAPP’s theory of change (ToC) and underlying assumptions held true? How did GAPP manage any changes in the ToC or assumptions? To what extent has GAPP implementation adhered to that assumed in the ToC? What deviations in GAPP implementation occurred, why did occur, and what effect did they have? What changes has GAPP made in activity design or strategy and why did it make those changes? How valid do GAPP stakeholders believe the ToC and underlying assumptions to be and why? Evidence that GAPP implementation has adhered to ToC and results of deviations Effect of external or unanticipated events on GAPP implementation and results Evidence that GAPP recognized and responded to deviations from ToC Evidence that outcomes are emerging along the ToC causal chain as predicted (e.g., monitoring, transparency, accountability, improved services) Consensus among stakeholders that GAPP ToC and assumptions remain valid Secondary data review KIIs with all GAPP stakeholders FGDs with beneficiaries of grantee CSOs Validation workshop KII discussion guides FGD discussion guides Identification of themes and sub￾themes via content analysis Triangulation of findings from multiple data sources What are we learning about coordination and management of cross￾sectoral efforts from GAPP implementation within USAID? During GAPP implementation, how did USAID internally (DO2, DO3, DO1, FtF, FANTA etc.) coordinate and manage cross-sectoral efforts (investments of Evidence that USAID departments are coordinating with each other Evidence that USAID & GAPP CLA activities are active and producing Secondary data review KIIs with LGOs, CSOs, GAPP, USAID & OPs FGDs with KII discussion guides FGD discussion guides Identification of themes and sub￾themes via content analysis Triangulation of findings from 99 Evaluation Question Sub-Questions Performance Measure Data Source (Primary and or Secondary) Data Collection Instrument Data Analysis Plan other sectors) and other donors? How did GAPP implement, respond and /or inform other sectors, donors and GoU about emerging issues, learnings and adaptions? Are there ‘synergistic’ effects at the LG level in which interventions in one sector spillover into other sectors? What barriers exist to cross-sectoral efforts and learning and what are options for addressing them? results Evidence that activities at the LG and LLG levels are producing cross-sectoral ‘spillover’ effects beneficiaries of grantee CSOs UAID/Uganda & GAPP CLA activities and results Validation workshop multiple data sources In-depth case studies How well are GAPP interventions influencing performance of e)National accountability institutions in their oversight function at both national and within LGs? f) Systemic functioning of LGs? g)Citizen participation At what level of government is GAPP technical assistance proving most effective? At what level should assistance be targeted in the future? To what extent are expected outcomes emerging or expected to emerge, including increased citizen engagement and voice, increased citizen LG monitoring, increased LG Evidence that national accountability institutions are functioning with improved capacity and producing results Evidence that LG governance institutions are functioning with improved capacity and producing results Evidence that citizens are monitoring LG processes and service delivery Secondary data review KIIs with all GAPP stakeholders FGDs with beneficiaries of grantee CSOs Validation workshop KII discussion guides FGD discussion guides Identification of themes and sub￾themes via content analysis Triangulation of findings from multiple data sources In-depth case studies 100 Evaluation Question Sub-Questions Performance Measure Data Source (Primary and or Secondary) Data Collection Instrument Data Analysis Plan in local governance processes and service delivery? h) Unblocking voice and accountability channels in local government? accountability and improved service provision? What factors explain their emergence or non-emergence? What barriers continue to exist to improving national accountability oversight, functioning of LGs, citizen advocacy and participation and LGO service delivery and what are options for addressing them? Evidence that citizens are empowered and have channels to voice concerns Evidence that LGs are listening to citizens Evidence that LGs are improving service delivery Which GAPP technical approaches are proving more effective and relevant for future programming? What are the different technical approaches GAPP has used? What is rationale behind them? How satisfied are GAPP stakeholders with the different technical approaches? What do they see as the strengths and weaknesses of these approaches? To what extent are different technical approaches linked to observed or emerging outcomes? Are there other approaches that GAPP has Evidence of relative effectiveness of different GAPP technical approaches Secondary data review KIIs with all GAPP stakeholders FGDs with beneficiaries of grantee CSOs Validation workshop KII discussion guides FGD discussion guides Identification of themes and sub￾themes via content analysis Triangulation of findings from multiple data sources In-depth case studies 101 Evaluation Question Sub-Questions Performance Measure Data Source (Primary and or Secondary) Data Collection Instrument Data Analysis Plan not tried but which have proven effective elsewhere? How well is CSO advocacy and monitoring of service delivery contributing to improving the quality and effectiveness of service provision at the LGs & LLGs? To what extent are CSOs engaged in advocacy and LG/LLG service monitoring? How have GAPP interventions influenced citizen participation in LG/LLG governance and LG/LLG service monitoring? Where citizen participation and monitoring have improved, is there evidence of existing or emerging improvements in LG and LLG service delivery? Evidence that CSO’s are undertaking advocacy activities Evidence that citizens are monitoring LG and LLG service delivery Evidence that LGs and LLGs are paying attention to advocacy activities and listening to citizens Evidence that LGs and LLGs are improving service delivery Secondary data review KIIs with all GAPP stakeholders FGDs with beneficiaries of grantee CSOs Validation workshop KII discussion guides FGD discussion guides Identification of themes and sub￾themes via content analysis Triangulation of findings from multiple data sources In-depth case studies 102 ANNEX 7: STRUCTURAL WEAKNESSES IN LOCAL GOVERNANCE SYSTEMS There exist at the LG level a number of persistent, structural weaknesses that present significant challenges of a general nature over which GAPP (and other DRG programs) have varying degrees of influence. These are summarized below. The taxation system in Uganda is highly centralized with almost all taxes collected by the national level Uganda Revenue Authority. The central government, moreover, at times interferes with LG’s abilities to raise tax revenue. For example, the GoU abolished the graduated tax, which was the main source of LR for the LGs, while boda boda operators in Gulu Municipality refused to pay parking fees and market vendors in Lira and Hoima Municipalities to refuse to pay market dues in FY 2015 both claiming that the President had exempted them from doing so. These factors combine to leave LGs little room to generate their own sources of revenue. What revenue LGs do receive from the centre, moreover, are heavily earmarked thus leaving little left over for discretionary spending purposes. LGs thus lack the discretion, autonomy and resources to respond to local needs and priorities. The LGFC estimates that locally generated revenue currently accounts for less than 3% of the LG budget.21 Furthermore, central government transfers have also been declining as a proportion of the national budget falling from 25% in 2005/06 to 15% in 2013/14.22 LGs generally have limited capacity to plan and effectively manage such limited resources for improved service delivery. LGs are perpetually understaffed—owing in part to the GOU-mandated requirement freeze that has been in place for several years—and LGs face perpetual challenges in terms of hiring and retaining qualified technical staff. Low salaries in particular make it hard to attract or retain qualified staff leading to loss of morale and motivation among existing technical staff. Some staff members have been in acting capacity for up to six years, even though public service regulations clearly state that public officials should not be in acting capacity for more than six months. This, in addition to inadequate resources and logistics, is preventing effective monitoring and supervision of service delivery, especially at LLG levels. A number of technical positions, moreover, are staffed by ‘acting’ technical staff, (it can take up to years to receive approval from the center to make formal hires) or by absent technical staff (e.g., staff member is attending school elsewhere). Added to this is a general lack of productivity-enhancing resources (e.g., computers, software, vehicles) to do the job. These challenges are particularly acute at the LLG levels. Corruption remains endemic at all levels of government in Uganda. A common way it manifests itself at the LG (and national) level is the widespread monetization of public service, in which elected officials and technical staff members seek extra financial or other compensation to do what is otherwise part of their job. Political interference from central government can also be a challenge, particularly (as seen above) with regards to local revenue mobilization. The GOU’s push for bringing services nearer to the people (and for extending government patronage) has resulted in the creation of more districts—and therefore more LLGs—that are not economically 21 Local Government Commission. (2012). “Review of Local Government Financing: Financing, Management and Accountability for Decentralized Service Delivery,” p. 152. 22 Local Government Finance Commission. (2014). “Annual Report.” 103 viable due to significant increases in administrative cost overheads of service delivery23 and which struggle to fill technical staff positions. Despite this, in September 2015 Parliament approved an additional 23 districts that will be operational over the next four years bringing the total to 135 districts.24 23 National Planning Authority. (2015). “Second National Development Plan: 2015/16-2019/20,” page 277. 24 New Vision of 4th September, 2015. 104 ANNEX 8: CASE STUDY 1-HURAMO Background HURAMO Uganda is a community based organization that was founded in 2006 with the core objective of providing a platform for lobbying and policy dialogue with local government and other institutions to promote democratic governance and accountability and advocate for improved healthcare and human rights through monitoring, research, documentation and reporting. HURAMO received a grant from GAPP to implement a two-year social accountability project titled Citizens’ Knowledge Building and Policy Dissemination in 5 sub-counties of Gulu District: Bungatira, Unyama, Paicho, Patiko and Palaro in Aswa County. The project objectives are as follows: 1. Strengthen the capacity of private sector organizations to demand for improved service delivery in Gulu district by 2017. 2. Increase citizens’ awareness in local governance in targeted sub-counties in Gulu District by 2017. 3. Increase citizens’ knowledge on the key highlights of LPAC, OAG and PPDA reports by 2017. Activities Supported by the GAPP grant financing, HURAMO implements a suite of activities within the targeted sub-counties. These activities include service monitoring, community engagements, radio programs, accountability meetings and roundtable meetings. Service Monitoring: To build citizen capacity to participate in local governance and demand accountability, HURAMO identified 82 persons as service monitors to monitor public service delivery in their communities. The service monitors consist of senior citizens, retired public servants, retired accountants and retired teachers who are vetted by the community members. The service monitors visit health and education facilities and compile reports of service gaps at each facility after which they submit a report to HURAMO about the availability and quality of the services. The monitors also attend Parent Teacher Association meetings where they provide information to the parents about the problems of the schools. They also attend the planning meetings at health facilities to share the findings from their monitoring visits. Service monitoring is time and labor intensive. Monitors must cover a large number of sites and a large geographic area, but without reliable transport. Most of the monitoring activities are carried out using boda bodas, but this is not a good option when it rains. At times, HURAMO has to hire a vehicle to transport the monitors, but this is not a sustainable alternative. Given all of this, it is not clear how much citizen engagement can be achieved during the remaining grant period. Once the grant ends, moreover, so will most of the monitoring activity. Community Engagements: HURAMO trained community members in the project areas on land rights for development and in service and delivery monitoring. A total of 59 participants (19 males and 40 females) attended the training sessions, including members of the land area committees. Participants were selected according to their age group, which enabled the participants to generate issues of their concern relative to their peers. 105 Radio Programmes: HURAMO produces and airs radio talk shows so as to extend its outreach within the target communities and engage them on topical issues of good governance and accountability. Accountability Meetings: HURAMO holds quarterly accountability meetings with local councils 1-3, Sub￾County Chiefs and Community Development Officers (CDOs). During the meetings, participants review the sub-county plans and report on issues awaiting resolution. Leaders provide feedback on issues raised during the last meeting and describe what actions have been taken. Roundtable Meetings: HURAMO organizes roundtable meetings for each sector (agriculture, education, health and private sector) in each sub-county. The meetings are attended by LG officials—including CDOs, Sub-County Chief, LC3/LC5, District /Sub-County Speaker, and leaders from the Boda Boda associations, market vendors associations, village savings and loan associations (VSLAs) and farmer groups—so as to discuss issues pertaining to each sector. Facilitators moderate the discussion during the roundtable and work with the Chairs of sectoral committees to organize issues for discussion. Notable Results At a roundtable meeting, the community monitors in Unyama Sub-County raised the issue of Hepatitis B in Okinyero Village. The monitors reported that an infected person was deliberately contaminating the water. The LC3 Chair confirmed that there were a rising number of Hepatitis B cases in the village. When the village leaders talked to the mother of the patient, she denied any role in the outbreak. Monitors mobilized the community to draft a petition that was submitted to the Speaker through the District Health Officer (DHO). The council mandated the DHO to follow-up on the matter. The DHO subsequently reported that 12,000 people had been infected. The Speaker next petitioned the Ministry of Health (MOH), which, as a result, selected Gulu as one of 10 districts to benefit from national Hepatitis B program to get free Hepatitis B testing and immunization. The MOH delivered the vaccines to health centers II, III and V. In March 2016, the Uganda National Students Associations requested that testing and immunization be extended to schools, which was approved. In May 2016, the testing was carried out. In Amich Teacher Training College 193 students were tested, of which 27 were found positive. In Sir Samuel Baker SS, 183 students were tested and seven tested positive. In Trinity Secondary School, 180 students were tested and nine were found positive. All persons testing positive for Hepatitis B were referred to Gulu Hospital for treatment. HURAMO recognizes that many people in the Gulu District are affected by land conflicts. Some of the sub-counties where it works, such as Palaro Sub-County, were affected by civil war that led to break down of the clan system. In the process, land rights were violated as clan leaders grabbed land and sold it to business people. HURAMO worked with the District Lands Officer to sensitize community members about their land rights, land tenure systems in Acholi (in Acholi, there is a communal land ownership tenure system) and land access and ownership. HURAMO also sensitized community members about equal land rights between men and women under the communal tenure system. Men had started selling off land, which was creating conflicts, even at the household level. Because of the war, there was a shortage of land and women were being disposed of land access rights because they had lost husbands during the war. Relatives were evicting the women because, according to them, women have no land ownership rights. Now the attitudes are changing, and women are able to claim their right to access land because they know the law is on their side. 106 In Their Own Words: Palaro Market Vendors Association We started the Palaro Market Vendors Association in a camp for internally displaced persons (IDPs) in 2010. Today the association has 40 members. Each member pays UGX 5,000 as membership fees and contributes another UGX 4,000 each week to her/his VSLA account. While in the IDP camp, we needed to make some money to look after our families. We started by purchasing produce from the nearby villages and sold it to make profit. Later we started a VSLA through which we have managed to save UGX 1.8 million to date. The savings act as a revolving fund. Members can borrow up to UGX 50,000 and pay back a profit of UGX 5,000 at end of the month. All the money must be returned if a member has to borrow more money from the VSLA. We hold VSLA meetings every Thursday. Members pay UGX 1,000 if they arrive late, which we put in a pool and use for emergencies. We also pay 1,000 per month for the welfare fund, which we use in case a member gets a problem. Through the group, we have become more united as members. Most members do not have an education but now we have received training from HURAMO and we have learned to interact with sub￾county officials. The sub-county officials are now open with us and when we have meetings we invite them. HURAMO taught us about land rights. As women, we did not know we have rights of accessing land. For instance, one of us had boundary problem in her household. Her in-laws claimed ownership of the tract of land her father-in-law had given her to cultivate. We reported the matter to the LC1 Chairperson who convened a meeting. The father-in-law confirmed that he had given her the land to use, but the in-laws insisted it belonged to them. Although we lost the case, it revealed that women can question decisions of the clan and get a fair judgment. HURAMO has taught us how to do advocacy. As a group, we demanded that the sub-county provide a latrine and shelter for the market. We wrote a petition and submitted it to the sub-county. The sub￾county selected a company to come and construct the latrine. Our group now coordinates with the sub-county on how the market is organized and coordinated. We are negotiating to buy some plots in the trading center where we can construct some houses to rent out or where we can do our business. We petitioned the sub-county to build market stalls for us. We got feedback that the construction is in plan and will start by November. When a borehole broke down, we reported the matter to the water source committee, which in turn presented the matter to the sub-county. Repairs are being undertaken. We worked with the sub-county to set license fees for vendors in the market. Now, the vendors are paying a flat rate of UGX 5,000 per year. Formerly, they paid UGX 15,000 per year. This was too high for many of the vendors. We are happy with the sub-county, because now they listen to us. When we have issues, we discuss. We have also learned the power of influence as a group. If we worked as individuals, we would not have achieved so much. There is still a lot we need. We do not have a bank account for our group. We are forced to lend out all the money in the VSLA because we don’t have where to keep it. We want to participate in budget planning, but we lack knowledge of the budgets. Yet we contribute to the revenue of the sub-county. We lack negotiation skills to enable us engage with government officials meaningfully. We lack knowledge of tendering processes to enable us to compete for tenders to run our own market. Lastly, we lack mobilization skills. We want the group to be inclusive of both men and women to boost its membership. Now there are more women than men. Men are still reluctant to join our group. 107 ANNEX 9: CASE STUDY 2-GLOFOD Background GLOFOD is district-based CSO in Lira founded in 2010 (as Global Partnership for Development) by 11 professionals with its main focus on advocacy, research and development. Nine of GLOFOD’s founders are still active and constitute its founding board. It currently has five staff (two females and three males). GAPP supported GLOFOD to fund a project titled Youth in Governance and Leadership (YoGAL) project, which started in 2015 and ends in January 2017. The project is being implemented in six sub– counties (2 rural Lira and Barr and 4 divisions in Lira Municipality). The project’s overall objective is to contribute to youth empowerment for increased youth participation in LG planning and development processes, leadership and accountability in the sub-counties of Lira, Barr, Lira Municipality (Adyel, Ojwina, Railways and Central Divisions) and Lira district by January 2017. Specific objectives include: 1. Enhance the capacity of 150 youth parliament members and youth leaders in youth participation, governance, leadership and development in Uganda 2. Develop and nurture youth parliaments to act as mentorship hubs for youth in governance, leadership, accountability and citizenship 3. Build the institutional capacity of GLOFOD to deliver the project’s objectives and increase project visibility Activities Supported by the GAPP grant financing, the project operates through a youth parliament model, which is composed of leaders from between 25-30 active youth groups based in the different parishes/villages, within each of the six sub-counties where it is implemented. The youth parliament seeks to empower local government staff and political leaders, as well as the youth, to create an atmosphere of respect, cooperation and joint decisions on issues that affect young people. Since its formation in 2015, YoGAL aims to contribute to the development of capacities of 150 youth in all six sub-counties and, through youth outreach programmes, benefit an additional 8,000 youths (3,825 females, 4,175 male and 650 persons with disabilities.) The project engages youth in drama, music, dance which can help them remain busy and even to earn an income. They are hired during the mobilization of the community for things like immunization campaigns. Notable Results Working with youth parliaments and youth leaders at sub-county and division levels caused a reawakening in LLG budget allocation to the youth sub-sector. There has been substantial growth in these allocations since FY 2014/15 when GLOFOD started working on youth issues in the targeted project areas. For instance, GLOFOD has empowered youth to participate in different activities, including water source protection, sanitation issues and sensitizing youth on government development YoGAL has “empowered the poor and vulnerable youth in attempting to address the root causes of poverty, injustice, low level of youth civic competences, limited participation in local government development processes and under-development in Lira district of Northern Uganda after at least two decades of war.” 108 programs. As a result, youth are actively engaging the local leader for information on funding sources for their project. The youth parliament engaged the division council to advocate for an increase in the allocation for the youth budget. The youth argued that since they constitute 70% of the community, they should be given a larger budget than the UGX 200,000 they were getting so as to enable them make their own development plans. In the areas of health, youth have many health demands, which are not fully met. One demand, particularly for female youth, is access to reproductive health services. Health center staff frequently hold negative attitudes toward youth and fail to deliver the reproductive health services they need. Working with health service providers in Lira Municipality, the youth parliament was able to get a youth advocate appointed at each health facility to whom the youth can go when they need to seek improved health services or to receive health counseling. Youth parliament members attended a district stakeholder engagement meeting in May 2016. Also attending were youth representatives to the district, LG leaders and district wealth coordinators. The youth in Barr sub-county complained that they do not have a coordination office and are always being tossed out any time new visitors come to the district. Now the sub-county has provided them an office space to coordinate their activities. In Their Own Words: Youth Parliament The youth Parliament empowered us. We were mobilized and counseled. Some of the children in the community had dropped out of school. Now we work to monitor, and so far, we have sent 300 primary pupils back to school. We look at it as a responsibility. In railway division, we took back three children whose parents could not afford scholastic materials, and we negotiated with the school authorities to allow them to continue with studies. We mobilize the children with disabilities to return to school. We have data on all the children with disabilities. We do community sensitization on proper hygiene, garbage collection, clean water and water sources. We rate the most and worst performing divisions in hygiene. Railway is the worst performing. We have mobilized the community to protect water sources by discussing their roles and responsibilities. The community has put in place bylaws, which will guide them in proper hygiene. When you are caught you pay a fine of UGX 15,000. One person has already been caught and fined so far. Because of our good work, we have received visitors from Mt. Meru in Tanzania who came to teach us about waste disposal, after which they taught us to recycle the waste. Now the strong smells have stopped. In Central Division, the youth have many problems. We mobilized them and sensitized them on different issues. They did not know about being in groups, so we organized them in groups and invited the CDO to talk to them. They did not know about the Youth Livelihood Programme funds; 16 youth have formed a group to apply to YLP. They also did not know how to read and write. We started a PAL class for them. So far, 22 have been recruited although only seven attend class. Most men do not want to attend the PAL class, but they send their wives. They think adult classes are for women. Adel has a bigger challenge; there is no health center in the entire division. The youth parliament has been working to sensitize the community to go and check out their HIV status. We have talked to the DHO to incorporate us in the government programmes and to let us participate in the community sensitization meetings to teach youth about HIV, Hepatitis B and other health issues. 109 During rainy seasons in Tegola Parish, the water tends to submerge the houses. As the youth parliament, we mobilized the community to dig a water channel to drain the water. We went and talked to the Red Cross, and it agreed to give us UGX 6 million. Now the water channel is there. We also got the first aid skills and registered with Lira Referral hospital. Now six of us (3 girls and 3 boys) are Red Cross first aiders. We also demanded the election of leaders at LC1 l as some had been elected to the division council. We petitioned the Principle Assistant Council Town Clerk. The by-election was held on 1 July 2016 to fill the gaps. Five people (three male and two female) were elected to different positions. “The youth parliament has done a great job, especially in providing leadership in the community. Before GLOFOD came, I was very green about budget issues. But now, I can participate in discussions. Before the youth parliament, I was a youth representing women at parish level. When elections came in February, I got elected to represent women in the division council. Now, I have the skills to debate. During debates, I make sure that women issues are not left out. As a result, five women have benefitted from the Youth Livelihood Programme fund and eight women benefited from the Community Development Driven Funds (CDD), and another 11 benefitted from the Northern Uganda Social Action Fund.” –Member Youth Parliament 110 ANNEX 10: CASE STUDY 3-MACCO Background The Mayank Anti-Corruption Coalition (MACCO) is a coalition of civil society organizations that represent all the West Nile districts (Moyo, Arua, Nebbi, Yumbe, Adjuman and Koboko) in the anti￾corruption effort. MACCO’s mandate is in the areas of anti-corruption, policy advocacy, monitoring service delivery and capacity building for partner CSOs to deliver on their mandates. With grant support from GAPP, MACCO is implementing the 18-month Social Accountability Actions for Community Empowerment (SAACE) project. The project is being implemented in two rural sub￾counties (Dadamu and Ayivuni) in one urban division (River Oli) in Arua District. The main goal of the project is to strengthen the capacity of local CSOs and PSOs to influence education and healthcare policy programs. To do so, it mobilizes citizens to engage government officials to account for and respond to the needs of citizens in Arua District. The specific objectives of the project are as follows: 1. Strengthen the capacity of PSOs in Arua district to demand for information so as to participate in local revenue generation and management by June 2017. 2. Strengthen the capacity of CSOs in Arua district to mobilize citizens to engage with local government Institutions by June 2017. 3. Empower communities in Arua district to monitor and evaluate the performance of universal primary education and the access and quality of healthcare services by June 2017. The primary target beneficiaries of the project include:  20 Private sector groups and coalitions  4 CSOs in Arua District (Arua District NGO Network, West Nile Development Agency, United Humanitarian Association and Aivu Rural Participatory Project for Development)  30 community monitors (10 per sub-county) from Dadamu, Ayivuni and River Oli  Primary schools and health facilities, such as the PTAs, school management committees (SMCs) and health unit management committees (HUMCs) Activities Community Monitoring In each sub-county, community monitors were selected with participation and recommendation of the local communities and trained to monitor service delivery based on agreed indicators. So far, MACCO has trained and equipped 166 community service monitors with skills and knowledge to monitor primary education and health center performance and engage duty bearers—including technical staff, councilors and DEC—to improve education and health standards in Arua District. MACCO trained community service monitors have visited 20 facilities, including 15 primary schools and 5 health centers (both health center 3 and health center 2). A total of number 1,715 participants were reached in 27 of the monitoring visits, including 839 males and 876 females (participants for the other three monitoring visits haven’t been analyzed yet). In education some of the indicators included absenteeism of the pupils and teachers and availability of adequate teachers and facilities (classrooms and latrines). In health facilities, some of the indicators included supply and availability of drugs; attendance, absenteeism and tardiness of health workers; late opening of the health facilities; and cleanliness and hygiene. Monitoring findings discussed with Head Teachers and I/C of the health center to find 111 immediate solutions. Others, such as construction of classrooms and latrines, which are beyond the mandate of the school or health facility, are discussed at a public dialogue in the sub-county. Local Revenue Mobilization During the course of the project, MACCO organized eight dialogue meetings on LRM, which were attended by 107 persons, including sub-county officials and leaders, district leaders and technical officials. The dialogue meetings were organized to share information with duty bearers, community and PSOs. CSO Capacity Development MACCO conducted a capacity assessment of four CSOs to establish their needs for capacity building trainings in mobilizing citizens to engage with local government institutions. Consequently, MACCO trained the four CSOs in internal governance, so as to be credible as an agent to mobilize citizens to hold leaders accountable, and in policy and budget analysis. It then organized two fora for promoting linkages between CSOs and government bodies to build district coalitions on thematic policy areas of health and education. Ninety-one persons (55 female and 36 male) attended the fora where policy thematic groups were formed to have a stronger voice to influence policy processes at district levels. The thematic groups were able to identify advocacy issues in healthcare and education and ranked them according to the level of seriousness for engagement with stakeholders. MACCO also organized a policy impact analysis training for the staff and board members of the four selected CSOs to enrich information for relevant policy formulation and analysis. An assessment of the impact of specific policies in health and education were identified and drafted in the form of policy briefs for engagement. Advocacy MACCO organized several engagement meetings with local authorities, which included the following:  Organized eight sub-county dialogue meetings in the three project sub-counties to present service delivery issues to the sub-county and district leadership for action. Overall, 255 persons (161 males and 94 females) attended the dialogues.  Organized district dialogue meeting to present service delivery issues for attention of district officials and other key stakeholders at the district level.  Organized district/municipal lobby event to advocate for pro-poor policy formulation and change in health and education sectors. As a result of this lobbying event, municipal/district officials committed to address service delivery issue/gaps.  Conducted four community radio programme that included community voice recording on service delivery issues in all the 20 facilities selected across the project sub-counties.  Organized four radio talk shows meant to seek responses from the DEO and DHO on the issues that were raised during community voice recording. Through the radio talk show, citizens from within and outside the district were able to interact with the DHO and DEO and received instant responses on issues that were affecting service delivery from their various facilities. MACCO estimates that around 5 million people were sensitized through the four radio talk shows. 112  Relayed four radio spot messages on Voice of Life FM to sensitize community members on their roles and responsibilities in improving service delivery in health and education. Again, MACCO estimates that around 5 million people were sensitized through the four radio spots. Notable Results Community Monitoring At IVUNI Health center, problems with tardiness and absenteeism of the health workers was reported by the monitors to the Sub-County Chief who promised to talk to the I/C of the Health Center. Soon thereafter, health workers started reporting on time and were available to attend to patients/clients. At Orivu Primary School, the school was sharing the pit latrine with the surrounding community. The matter was brought to the attention of the sub-county officials who engaged the community to construct their own pit latrines. At Kubo primary school in Aivuni Sub-County, monitors reported that there was no latrine for girls and that both teachers and pupils were sharing one latrine. The SMC did not have money to contract latrine. The matter was reported to the sub-county authorities who promised to take action. A five-stance latrine for girls has since been constructed at the school. In Aivuni Sub-County, the monitors reported that there was high incidence of pupil absenteeism, especially on market days, as pupils would be found at markets instead of being at school. The sub-county authorities passed a resolution that truant any child should found in markets would be arrested and their parents would be made to pay a fine to get them released. The sub-county authorities also started visiting markets and arresting any pupils found there. The problem is now less prevalent than before. Pupils are not allowed to go to markets and their parents have vowed to comply. Community monitors alerted Odruva Sub-Country authorities that the primary school lacked housing for teachers. They and parents resolved to raise UGX 2 million out of 6 million to build houses for the teachers, while the sub-county pledged the balance of UGX 4 million. In Aivuni Sub-County, the health center did not have transport, and had a shortage of staff. At the sub￾county dialogue organized by MACCO, monitors raised this issue to the sub-country officials. Some months later, the health center received a motorcycle and five new staff were posted and 10 new beds delivered to the center to boost service delivery. In another case, community monitors reported two health center workers, who were notorious for asking for bribes from patients. The Center Director reprimanded the workers who promised not to do it again. A mid-wife who was fond of asking for bribes from pregnant women was also reported and reprimanded. She promised to stop the habit. Subsequently, deliveries have increased from the previous 15 per month to 40 deliveries per month at the health facility. 113 In Dabo market, sub-county authorities tendered private tenderer to construct a pit latrine to serve the market. In the two years he had been at Luvu primary school in Arua District, the Head had registered drastic deterioration in performance of both the teachers and pupils. During the monitoring visits, the child education monitors discovered that one-half of the pupils from P4-P7 were missing classes daily and no follow-up was being made. Parents did not attend meetings, while teacher absenteeism was regular. During the monitoring visit, parents and some of the teachers opened up about the unethical behavior of the Head Teacher that was leading to the poor performance. They reported that he was drinking alcohol, even during class hours, dressing unprofessionally and using abusing language against the parents, hence demoralizing them from participating in school activities. At the sub-county dialogue meeting at Dadamu sub-county headquarters, the Head Teacher was drunk and could not respond properly to the allegations. The District Inspector of Schools and other sub-county officials attended the meeting and grew irritated by the Head Teacher’s obvious drunken behavior. The Sub-County Chief ordered the arrest of the Head Teacher who spent the night at the sub-county detention facility. As punishment, he was made to do community service by slashing the whole compound of the sub-county. The head teacher being quizzed by the officer in charge of the detention facility Local Revenue Mobilization In Aivuni sub-county, issues of double taxation were raised. Market vendors pointed out that the landlord and the sub-county agents were demanding rent and dues from the vendors, yet the sub-county was paying rent to the landlord where the market was located. Since the dialogue, only the sub-county now collects rent from the markets vendors. 114 ANNEX 11: CASE STUDY 4-LOCAL REVENUE MOBILIZATION IN LIRA AND HOIMA MUNICIPALITIES With regards to local revenue mobilization, experiences have varied across the GAPP-supported LGs, including how the extra revenues have been put to use. For example, when Anaka Town Council in Nwoya district increased its local revenue collection in FY 2015/2016 to UGX 30 million (from UGX 10 million in FY 2014/2015), it applied the windfall to sanitation and garbage collection thus leading to marked improvement in those two areas. On the other hand, Koboko town council (recently promoted to municipality) used the extra local revenue generated to pay casual workers, enhance sanitation and promote youth enterprises. Lira and Hoima municipalities, however, stand out as two examples where GAPP interventions on LRM appear to have produced particularly notable results. With GAPP assistance, Lira municipality has developed realistic tax registers and plans for outsourcing collection of property tax, trading licenses and urban authority permits. Some control measures have also been put in place, such as requiring successful bidders to pay the first three months upfront, and also providing the municipal council with post-dated checks for the remaining three quarters of the financial year. Reserve prices set by the municipality are now more realistic, as evidenced by a reduction in public complaints on that score. Improvements have also been made in tax collection. Taxpayers now pay directly into the bank where they are issued payment slips; they then present the receipts to the municipal cash office where the payment is entered into the cashbook. This process is still manual, but bank reconciliation is done monthly. A computerized system has been installed and is expected to go into operation in August/September 2016. The municipality has also mounted a vigorous radio tax education programme. By the time of the MTE, the Municipal Mayor and the Secretary for Finance had held two radio talk shows during which they engaged the public on several development issues, including local revenue generation and service delivery. All this notwithstanding, less than 50% of potential revenues are being collected due to capacity and staff shortages and numerous leakages. Some taxpayers are not willing to go to the banks (citing long bank lines) and often ask others to do the banking for them, with all its attendant risks. Political interference has also contributed to tax generation challenges. For example, boda boda riders refused to pay parking fees in FY 2015/2016 claiming that the President had exempted them from doing so. Furthermore, while some taxpayers are simply ignorant of taxation issues and need sustained education on the benefits from, and modalities for, tax payment, others are reluctant to pay because they do not think they are getting sufficient services for their taxes. And then there is the case of the LRM software that was supplied to Lira municipality by the LGFC in February 2016. It developed issues, but the consultants that installed it never returned to address them, with the result that the finance department is using an Excel database instead. (This problem is not limited to this case. For example, in Masaka Municipality, Kimanya/Kyabakuza Division also stopped using the LRM software for the same reason and is using Excel and Access databases instead.) Hoima municipality has leveraged GAPP support to undertake the following LRM activities: (1) taxpayer enumeration and assessment; (2) tendering out revenue sources (e.g., markets, bus/taxi park, trading licenses, property rates/tax, advertisement, loading and off-loading, liquor licenses); (3) computerizing billing and collecting revenue through banks; (4) direct collection by the municipality (e.g., fees for approval of building plans); (5) sensitizing and educating citizens on tax issues through radio and TV with the assistance of local leaders; (6) developing a new property valuation roll to replace the old one that had expired; (7) starting the tendering process early so that by the end of July most of the revenue 115 tenders for FY 2016/2017 had been awarded; and (8) accounting to taxpayers through radio and barazas on the revenues collected and services provided with them. These measures have led to significant improvement in revenue generation; local revenue in Hoima increased from UGX 1,473.2m in FY 2011/12 to UGX 1,989.0m in FY 2014/15. This has enabled the municipality to allocate more money to collecting garbage, maintaining roads, clearing the drainage system, repairing boreholes, protecting springs, repairing desks in pilot schools and constructing a fence for St. Aloysius Secondary School in Bujumbura Division. Notwithstanding, Hoima municipality experienced a precipitous drop of UGX 768.7 million in local revenue generation during FY 2015/2016 due to political interference (total local revenue collections for FY 2015/2016 were UGX 1,220.2 million). Vendors in the new Hoima municipality market refused to pay the rates that had been set by the municipality (i.e. UGX 40,000 per stall) claiming that the President had allowed them to set the rates themselves. It was only after protracted negotiations involving municipal staff, RDC, District Internal Security Officer (DISO) and vendor representatives that agreement was reached on the two compromise figures of UGX 15,000 and UGX 10,000. The municipality lost UGX 768.7 million in potential revenues as a result.