USAID TANZANIA: THE WAJIBIKA PROJECT MID￾TERM EVALUATION FEBRUARY 7, 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by Dr. Anna Nswilla, Dr. Emanuel Malangalila, Gene Peuse, Danny Dedeyan, and Mary Wieczynski Furnivall through USAID and Deloitte Inc. USAID Tanzania: THE WAJIBIKA PROJECT MID-TERM EVALUATION Copies of this document can be requested from USAID/Tanzania. Additional information can be obtained from: USAID/Tanzania P.O Box 9130 686 Old Bagamoyo Road, Msasani Dar es Salaam, Tanzania Tel: (255) 22 229 4490 Fax: (255) 22 266 8421 arebold@usaid.gov A Mid-Term Evaluation of the Wajibika Project, USAID/Contract # GHH-I-00-07-00064-00 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 3 TABLE OF CONTENTS Acknowledgments .................................................................................................................... 4 Acronyms .................................................................................................................................. 5 Executive Summary .................................................................................................................. 6 1. INTRODUCTION 10 2. BACKGROUND ...................................................................................................11 2.1 An Overview of Local Government Authorities in Tanzania .........................11 2.2 USAID’s System Strengthening Program Strategies and Priorities ..............14 2.3 The Wajibika Project ...................................................................................14 2.4 The Wajibika Project in Context of Public Sector Reform, Capacity Building, and Decentralization .....................................................15 3. EVALUATION PURPOSE AND METHODOLOGY ...................................................19 3.1 Purpose and Objectives ..............................................................................19 3.2 Evaluation Methodology ..............................................................................19 4. EVALUATION FINDINGS AND CONCLUSIONS ....................................................21 4.1 Findings ...................................................................................................21 4.2 Conclusions ................................................................................................30 5. RECOMMENDATIONS ............................................................................................34 5.1 Project Recommendations and Next Steps Before End of Project ................34 5.2 Other Issues and Technical Considerations .................................................38 ANNEXES .....................................................................................................................41 Annex A: Scope of Work ..............................................................................................42 Annex B: Evaluation Tools ...........................................................................................52 Annex C: Evaluation Calendar and Logistics ................................................................58 Annex D: Persons Contacted .......................................................................................61 Annex E: References ..................................................................................................65 Annex F: GOT's Planning and Budgeting Cycle ..........................................................66 Annex G: The Wajibika Project's FY 2011 Annual Program .........................................67 Annex H: Forthcoming Support to Local Government Authorities in Tanzania .............70 USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 4 ACKNOWLEDGMENTS The evaluation team wishes to thank the United States Agency for International Development’s Health and HIV/AIDS team in Tanzania for its generous support and for their time and assistance throughout the assessment. The team also wishes to thank Prime Minister’s Office, Regional Administration and Local Government and the Ministry of Health and Social Welfare for making the time to meet and discuss critical issues in their regions. We also appreciate the assistance of the USAID implementing partners, Abt Associates and, particularly, the Wajibika team in Tanzania for their hospitality and support. Special thanks go to Peter Kilima, Mary Kasonka, and Conrad Mbuya, especially for their assistance in organizing the team’s in￾country logistics and for facilitating stakeholder meetings. Their excellent support ensured a smooth and fruitful visit. We also thank Linda ole Moi Yoi for her skillful editing. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 5 ACRONYMS CAG Controller and Auditor General CBG Capacity Building Grants CCHPs Comprehensive Council Health Plans CDG Council Development Grants CHMTs Council Health Management Teams CSO Civil Society Organization D by D Decentralization by Devolution DHMT District Health Management Team FBO Faith-Based Organization GOT Government of Tanzania HIV/AIDS Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome ICT Information and Communication Technology IPSAS International Public Service Accounting Standards LGA Local Government Authority LGCDG Local Government Capital Development Grant LGRP II Local Government Reform Program II M&E Monitoring and Evaluation MOF Ministry of Finance MOHSW Ministry of Health and Social Welfare MTEF Medium-Term Expenditure Framework NAO National Audit Office NGO Non-Governmental Organization OVC Orphans and Vulnerable Children P4P Pay for Performance PCCB Prevention and Control of Corruption Bureau PEPFAR President’s Emergency Plan for AIDS Response PMP Performance Monitoring Plan PMU Procurement Management Unit PMO-RALG Prime Minister’s Office, Regional Administration and Local Government PPRA Public Procurement Regulatory Authority RAS Regional Administrative Secretary SWAp Sector-wide Approaches TB Tuberculosis TBHSP Tanzania Basic Health Service Project USAID United States Agency for International Development USG United States Government USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 6 EXECUTIVE SUMMARY OVERVIEW Building on over four decades of partnership and collaboration between the Government of Tanzania (GOT) and the United States Agency for International Development (USAID), the Wajibika project was developed to help strengthen Tanzania’s local government authorities (LGAs), especially in the areas of governance and accountability. The Wajibika project performance dates are December 2009 – December 2012, and it is funded through a three-year contract implemented by Abt Associates with a $10,000,000 funding ceiling. The project has three overall objectives: • to improve governance through strengthened programmatic and fiscal accountability in the target districts • To strengthen support for decentralized management by the Prime Minister’s Office, Regional Administration and Local Government (PMO-RALG), and the Ministry of Health and Social Welfare (MOHSW) • To support the PMO-RALG in developing an effective expansion plan for the Government of Tanzania The Wajibika project started field operations in February 2010, working with eight councils in the Iringa region. In February and March 2010, the Wajibika project conducted a rapid assessment of the eight councils to inform project design and measurement. In October 2011 during project year two, the project expanded activities to an additional 19 councils in Pwani, Dodoma, and Morogoro regions, bringing the total number of supported regions to 27. The USAID mission in Tanzania commissioned the mid-term evaluation of the Wajibika project to provide USAID and other Wajibika stakeholders a review of the project’s progress by documenting planned and unanticipated achievements and challenges and by reassessing its objectives and likely outputs. A team of two USAID technical experts, one colleague from the MOHSW, and two external consultants conducted the evaluation in December 2011 and January 2012. After meeting with USAID and project representatives in Dar in Salaam, the team spent two weeks in country, meeting with a wide range of government stakeholders in Iringa and Coast regions. SUMMARY OF FINDINGS Through a review of background information, project documents, and interviews with stakeholders and project implementers, the evaluation team identified the following summarized strengths, gaps, opportunities and threats: Key Strengths • Project capacity building in support functions (procurement, financial management, internal auditing, ICT, and planning) impacts the performance of the entire council. • LGA professionals who received training and mentoring in support functions report now meeting international standards in their work. • The project has demonstrated applicability beyond technical support to only the health sector, with some mentors having provided strategic planning and technical assistance to the education, water, and community development sectors. • The project is beginning to assist with the integration of health into other sector budgets (e.g., through data organization, OVC is included in department technical and planning budgets such as agriculture, water, and health). • Regions noticed performance improvement within councils and requested support to strengthen their own LGA oversight function and technical capacity. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 7 • The project is beginning to operationalize lower level participation in strategic planning, thus helping to meet national planning guidelines. • Stakeholders identified the use of mentoring as a value-adding element unique to the project because of the daily interaction to meet the needs of individuals and entire departments. • The project has demonstrated that peer learning between LGAs can have a significant capacity building effect. • The project contributes to the piloting of pay for performances (P4P) in Pwani region through assistance on strengthening support functions and strategic planning. Key Gaps • Project scale up to additional regions occurred before a formal evaluation of the pilot initiative. • Monitoring and evaluation (M&E) is focused at the process level and has limited measurement of improved performance and increased sustainability. • Although a standardized approach is evolving, the project has not clearly articulated its technical approach, which is critical for future expansion. • The project will start advocacy to the GOT for the inclusion of the Wajibika technical approach as a standard practice; this is an ambitious achievement for a three-year project. • Some regional secretariats expressed a lack of understanding of the project, although their overall impressions were positive. • Not all CSOs and NGOs participate in council strategic planning and this undermines the quality of the development and monitoring of strategic plans. • There is a need for the project to strengthen linkages with PMO-RALG and MOHSW at the national level, especially as the project advocates integrating the Wajibika approach as a LGA strengthening standard practice. Opportunities: There has been a sea change within the GOT related to governance and accountability, and this enhances the potential of the Wajibika project to achieve sustained results. The Regional Secretariat is a natural entry point to strengthen LGA performance; the linkage with the project is the Local Government Section that is designed to support the development of LGAs. In support of GOT accountability, CSOs that work in social accountability, monitoring across all sectors, have the potential to inform council strategic plans. Threats: Although there is a Government policy towards decentralization, not all sectors are devolving at the same pace. Furthermore, in some cases, there is a lack of technical cooperation between councils within a region, and regions do not network with the councils. This creates project challenges, which potentially can be turned into opportunities as Wajibika strengthens its capacity building focus on regions. In addition, many councils inconsistently coordinate with USG- and non-USG projects to include activities and resources in strategic planning, and this often results in plans that greatly underestimate the scale and scope of activities and budget. SUMMARY OF CONCLUSIONS: The Wajibika project already has demonstrated a contribution to Tanzania’s national efforts in public sector reform by accelerating decentralization by devolution (D by D) and by supporting the move towards e-Government. Within only two years of project implementation, the GOT has expressed a high degree of ownership of the technical approach, which follows many international best practices. Mentoring has played a critical role in the project, and stakeholders say this is the positive difference between this project and other LGA-strengthening initiatives. The government has expressed a keen interest in having the technical approach become a standard LGA capacity strengthening practice. However, despite the project’s successes to date, the three-year timeframe is too short a period to expect full transition into Government systems. Project elements that would facilitate such a transition include: USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 8 • A phased-in budgeting of associated costs into council and regional budgets • The continued expansion of the Wajibika approach to other sectors outside of health • Expanded collaboration with other GOT institutions with related governance and accountability oversight functions, e.g., the Department of Internal Auditing, the Public Procurement Regulatory Authority (PPRA), the National Audit Office (NAO), and the Ethics Secretariat. • The incorporation of the shadow budget into district-level strategic planning to ensure fully informed governance and oversight • Evaluation and dissemination of the project’s pilot phase is a key first step towards transitioning the technical approach to a national standard LGA strengthening practice. Although stakeholders are overwhelmingly enthusiastic about the project and reported several positive results, including improved performance, the project must immediately start to quantify these results through expanded monitoring and measurement. In addition, it is important to note that the leadership and technical expertise of the Wajibika team has been key to the success of the project to date. However, the team must have continuing guidance and support to assist with a complex project transition that would help inculcate the technical approach as a standard LGA strengthening practice. In support of governance, accountability, and LGA strengthening, USAID and other donors supporting multi-sector programs at district levels should ensure transparency and complete participation by supported partners in annual planning. This will help councils conduct accurate planning and oversight, with increased knowledge of shadow budgets. The following table presents what the evaluation team observed as Wajibika’s technical approach (although the project has not formally codified defined models or approaches, per se). The Wajibika Approach Goal: Strengthen LGA Governance and Accountability Wajibika Core Services Council level: Strengthen supply service capacity:  Financial management, procurement, ICT, internal auditing Strengthen sector technical capacity:  Strategic planning  Sector-specific technical assistance Increase LGA revenue generation Regional level: Strengthen advisory & oversight capacity Regional & council levels: Advocacy for Wajibika-elements in LGA budgets Professional subscriptions WajibikaPlus Regional level: Strengthen supply service capacity:  Financial management, procurement, ICT, internal auditing Regional & council levels: Inclusion of “users” into layman’s technical and oversight capacity building  Elected councilors  District Executive Directors  Department heads  Managers Key Stakeholders PMO-RALG Ministry of Finance Department of Internal Audits MOHSW Other Sectoral Ministries How? Advocacy Mentoring Technical Assistance Peer Learning Training Documentation Mainstreaming the Wajibika project into donor and technical dialogue structures USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 9 RECOMMENDATIONS The following are recommendations to guide USAID’s support to the GOT in LGA strengthening, with specific “by end of the project” recommendations to the Wajibika project implementing team: Recommendation #1: Define, evaluate, and disseminate the Wajibika technical models Overall • The project must assess, define, package, and disseminate the project’s technical models. • The project should not roll out to other regions during the remaining life of the project, but focus on solidifying and measuring achievements. Before the end of the Wajibika project: • Finalize the project’s first phase evaluation, the definition of Wajibika technical models, and initiate dissemination before the end of project year three. • Source substantial support from Abt Associates Recommendation #2: Measure project results and increased sustainability Overall • Quantify project results at output and outcome levels with a focus on measuring improved performance and increased sustainability. • Where possible, use common indicators (e.g., MTEF, World Bank Basic Health Services project). Before the end of the Wajibika project: • Update the Performance Monitoring Plan to include indicators; as soon as possible, collect and document baseline information • Develop and implement council- and region-level tools that measure performance. Recommendation #3: Integrate Wajibika’s approach as a GOT standard LGA strengthening practice Overall • Work closely with the GOT to develop the Wajibika Transition Strategy. Elements include: • Inclusion of Wajibika in dialogue structures • Defined packages of services • A phased-in budgeting of associated costs • Increased generation of local revenues • The sequenced expansion to other sectors • Expanded collaboration with GOT Before the end of the Wajibika project: • With substantial technical assistance from Abt Associates, work with PMO-RALG, the MOHSW, and other line ministries to develop the Wajibika Transition Strategy, operational plan, and indicators • As feasible, start implementing the strategy Recommendation #4: Strengthen the Wajibika project to act as a transition mechanism Overall: • The Wajibika project must evolve from an implementation project to a state-of-the-art resource to support the GOT. Issues include: • Avoid premature project roll out until existing programs are robust • On-the-ground mentoring for project staff • Formation of a project advisory committee Before the end of the Wajibika project: • Abt Associates to develop a technical assistance plan to support the Wajibika team, including mentoring support for the project as they implement the transition strategy. Recommendation #5: Strengthen the technical quality of existing council and regional programs Overall: • Before additional project roll out, ensure the thorough implementation of the service packages. Elements include: • A mapping of current interventions • Improved measurement • Link to other technical assistance • Decentralized mentor coordinators Before the end of the Wajibika project:  Mapped implementation and measurement activities to ensure full coverage; decentralized mentor coordinators. Recommendation #6: Coordinated donor support for LGA strengthening Overall: • USAID and other donors should actively coordinate LGA strengthening support: • Support participation of donor-funded partners in strategic planning • Harmonize partner planning with GOT • Work with the World Bank-supported LGA project to scale-up Wajibika Before the end of the Wajibika project: • USAID to enter into dialogue with the GOT and the World Bank to establish areas of collaboration under the Tanzania Basic Health Service Project. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 10 1. INTRODUCTION Recent reforms in local governance, notably the launch of the second local government reform program, the Local Government Reform Programme II: Decentralization by Devolution (July 2008–June 2013), offer a historically unprecedented opportunity to support and strengthen the capacity of local government authorities (LGAs) to operate independently of central and regional authorities in managing Council health service delivery. In support of these reforms, in December of 2009, USAID awarded funding to Abt Associates under the AIDSTAR II mechanism for a project called “Wajibika,” which means “be accountable” in Swahili. Wajibika supports the Government of Tanzania’s (GOT) initiatives to strengthen Local Government Authorities’ capacity for financial and programmatic accountability under the decentralization by devolution (D by D) program. Abt Associates is working on this initiative with the Prime Minister’s Office-Regional Administration and Local Government (PMO-RALG), the Ministry of Health and Social Welfare (MOHSW), as well as with Family Health International. In support of the GOT’s initiatives to strengthen Local Government Authorities’ capacity for financial and programmatic accountability, the Wajibika project employs a variety of interventions, including skills in prioritization, planning, budgeting, and better usage of data and more effective feedback mechanisms related to program execution. Other interventions include improved internal controls and financial management/reporting, improved performance incentives; and mentoring and on-the-job training for local government authorities. This comprehensive approach is intended to assist the GOT in fostering greater country ownership, improved governance, and increased potential for integrated, sustainable and high quality programs. In addition, the Wajibika project aims to promote strategic advocacy with stakeholders at all levels of government, spurring collaboration among LGAs to encourage better performance and motivating workers through appropriate incentives. The Wajibika project is underway in all eight districts of Iringa, where focus has been placed on strengthening districts to prioritize, implement, and monitor essential health programs, catalyzing synergy between and among health programs, ensuring fiscal accountability, and coordinating health resources such as faith-based, private, Global Fund, basket funds, and USG funding. The project has since expanded to an additional 19 districts in the Dodoma, Morogoro, and Pwani regions. The aims of this mid-term evaluation of the Wajibika project, which took place during the first two weeks of December 2011, were: 1) To determine the extent to which the expected results of strengthened programmatic fiscal accountability have and will be achieved; and 2) to provide information either for mid-course correction or planning of future funding activities. In addition, this evaluation is expected to help stakeholders understand successful approaches that should be emphasized in future programming. The five-person evaluation team was composed of two individuals from USAID, one from the Ministry of Health and Social Welfare (MOHSW) and two consultants. Wajibika project staff supported the team by organizing logistics and meetings in several Wajibika sites including: district and regional Councils in Iringa; district councils in Mufindi, Kilolo and Bagamoyo; and town councils in Njombe and Kibaha. For the evaluation team’s detailed scope of work, process, approaches and tools employed, see Section 2 below and Annexes A and B. For the complete list of persons contacted during the evaluation and for the full schedule of team activities, see Annexes C and D. Annexes E, F, G, and H provide references and additional background information on the Wajibika project. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 11 2. BACKGROUND 2.1 An Overview of Local Government Authorities in Tanzania 2.1.1 LGA organization and structure Local Government Authorities (LGAs) in Tanzania form part of an administrative division of the territory into administrative units as stipulated under article 2(2) of the Constitution of the United Republic and guided by The Regions and Districts (Establishment Procedure) Act No. 12 of 1994. Presently, there are 25 regions, 132 districts, and 516 divisions in Mainland Tanzania. A region is headed politically by a Regional Commissioner while the head of the Civil Service at that level is the Regional Administrative Secretary (RAS). At the district level, the District Commissioner (also political) is the principal assistant to the Regional Commissioner at that level and the District Administrative Secretary is the head of the Civil Service. The mission of the regional tier of the Central Government is to perform the two major roles of Development and Administration. Table 1 presents the relationship between regional and district authorities in Tanzania. The RAS’ development role centers on building the capacity of and backstopping the LGAs in the performance of their mandated roles in an effective and efficient manner. The Regional tier, using its Regional Secretariat, performs this role through four clusters, each headed by an Assistant Administrative Secretary, all of who report to the RAS. The clusters at RAS are Management Support Services, Economic Development Support Services, Physical Planning and Engineering Support Services and Social Sector Support Services. A fifth cluster of Staff Support Services serves the Secretariat itself internally. The administrative role of the Regional Secretariat facilitates and assists LGAs to undertake and discharge their responsibilities by providing an enabling environment and representing Central Government within the region. In actuality, the Regional Secretariat’s relationship with LGAs is principally one of policy interpretation, advice, coordination, monitoring, enforcement and the creation of an enabling environment for the LGAs to discharge their duties Acknowledging the need for sustained attention and involvement of LGAs, the Government of the United Republic of Tanzania (GOT), with support from development partners, launched the Local Government Reform Program I (LGRP) in 1999 that provided a framework for the Decentralization by Devolution (D by D) vision promulgated by the GOT three years earlier. This effort is going a step further under the current Local Government Reform Program II (LGRP II; 2009–2014), with four components: 1) to create an enabling environment for D by D through the institutionalization of D by D in government operations, decentralization of human resources, improved fiscal decentralization, and enabling a conducive legislative framework; 2) improving capacity of LGAs through leadership and management development in LGAs, organizational and human resource development, strengthening financial management as well as planning, monitoring and evaluation systems while improving the use of information technology in LGAs; 3) facilitating citizen participation and accountability; and 4) program management and accountability. D by D is an evolving process. Although significant progress has been made, more effort is required to achieve the objectives of this key reform area. For example in 2008/09, 84% of the councils met all minimum conditions (MCs) to access CDG as compared to 52% in 2007/08 and also 95% of the councils met all MCs to access CBG in 2008/09 as compared to 88% of the councils in 2007/08. While the strategy for D by D is in place and supported at the highest levels of government, it is not adequately embedded across ministries, departments and agencies, and LGAs. The legislative framework is not yet fully compliant with LGRP II, with: 1) service-delivery functions transferred to local governments; 2) fiscal transfers account for a very large proportion of total local government revenue; USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 12 3) a system through which funds are transferred from the center to local governments has evolved from an ad hoc, discretionary system to one in which horizontal allocations to local governments are made on the basis of formulae determined by equity and efficiency principles; and 4) the chief source of discretionary local government development funding is the Local Government Development Grant. Also there are interruptions of devolution of human resource management, such as through the centralized appointments of LGA staff and regular turn over. Financial management and procurement at the LGA level pose continuing challenges. Table 1: The Relationship between Regional and District Authorities in Tanzania 2.1.2 The evolution of LGAs: funding and performance measures The GOT issues funds to LGAs on an annual basis following a comprehensive planning and budgeting process (see Annex F: GOT’s Planning and Budgeting Cycle). The flow of financing to Tanzanian LGAs has traditionally come under one of four flows of funding, each with a separate functional role and its own management challenges for local councils. One system is the Recurrent Block Grant, with the Personal Emoluments and Other Charges Components. Challenges include local councils’ abilities to determine the balance between those two components against a budget with a single formula-based transfer ceiling for each sector. Regional Secretariat Planning & Coordination Section Health & Social Welfare Section LGA Management Services Section Education Section HR Legal Internal Audit IT Finance Elected & Appointed Councilors and Standing Committees District Executive Director District Legal Unit (Other Sections) District Procurement Unit Region is charged with the role of policy interpretation, advice, coordination, and monitoring of LGAs District Internal Audit Unit Education Department (Other Departments) (Other Departments) Planning & Coordination Department Health & Social Welfare Department USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 13 Another funding source is Recurrent Subventions and basket funds. These funds are provided through sectors with earmarked Other Charges resources and represent around 8% of an LGA’s annual budget. Subventions and basket funds often require special financial management procedures. Own Source Revenues represents approximately 8% of an LGA’s annual budget. This source of funding includes property tax, local government taxes on goods and services, business and professional licenses, motor vehicles, turnover taxes, entrepreneurial and property income, and administrative fees and charges. Development grants and development funds are another important source of LGA funding. These funds include intergovernmental transfers for capital infrastructure and development activities. Sectoral development transfers operate with separate planning, accounting, and reporting requirements, and have separate financial management manuals for different development funding streams. They often require separate bank accounts. As a result of the unwieldy development grants and development funds systems and processes, PMO￾RALG has advocated the harmonization of development grants through the implementation of the Local Government Capital Development Grant (LGCDG) system. This would provide discretionary development funds to Local Authorities and be fully integrated into the government’s budget process. The system will, overtime, become the mechanism through which all development funds will be transferred to LGAs. According to the Principal Secretary of the PMO-RALG: as the LGDGS takes root and more sector specific grants are incorporated, more effort is needed on the part of all stakeholders to support the mechanism through which all development funds will be channeled to LGAs. Integrated into the LGCDG system, development funds will flow to local authorities through Council Development Grants (CDG), Capacity Building Grants (CBG), and Sector Specific Grants. The GOT has established a link between the financing of local governments and their performance to promote compliance with national policies and with legal and regulatory frameworks. They include the following criteria: financial management, participatory planning, pro-poor budgeting, budget execution and the broader areas of local governance such as transparency and accountability, council functional processes, and the involvement of Lower Local Governments and communities. This performance-based approach introduces an incentive system that allows for adjustment of the annual grant allocations to each LGA, upwards or downwards, based on a set of performance indicators. Currently, both the GOT and development partner basket-funding arrangements finance the LGCDG system. All of those funds flow to LGAs through the Consolidated Fund in the Ministry of Finance. Over time, however, the system will be completely absorbed into the Government Budget. An LGA’s access to the LGCDG system funds, which represent approximately 70% of the funds available to LGAs for development, is dependent on its meeting a set of Minimum Conditions (MCs) which ensure that the funds transferred to LGAs are properly utilized and in compliance with the statutory and administrative requirements laid out by PMO-RALG. The MCs are derived from laws, regulations, and national guidelines and exist to ensure that funds remitted to LGAs are utilized effectively, efficiently, and with integrity and that they are sustainable. The most common reasons for LGAs failing to meet MCs relate to the submission of financial reports, lack of internal audit capacity, and adverse audit reports by the Controller and Auditor General’s (CAG) annual audit. In addition to Minimum Conditions, PMO-RALG also rates LGAs against a set of Performance Measures as a means of introducing incentives for performance improvement by providing for adjustments in the yearly size of the grant received. Performance Measures can reward well-performing LGAs with a 20% increase in LGCDG allocation or reduce it by 20%. Specifically, Performance Measures indicators aim to:  Evaluate the performance of the LGA in key functional areas such as financial management, procurement, and others. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 14  Evaluate LGA compliance with broad policy guidelines on governance issues such as transparency and accountability.  Provide incentives for improved Local Government performance and stimulating the demand for capacity building support. 2.2 USAID’s System Strengthening Program Strategies and Priorities Following the U.S. President’s speech in Accra, Ghana in July 2009, the USAID FY2011 budget request for Africa made a strong case for increased support to Tanzania’s ability to govern justly and democratically. While Tanzania is widely viewed as being ahead of many of its African neighbors in terms of holding elections and creating levels of government at municipal/rural district levels, the request identifies these levels of government as key links to overcoming poor citizen satisfaction in service delivery and governance. Improved transparency and accountability in Tanzania would require assistance to local government officials in budgeting, planning, and accounting for the funds they manage.1 As such, it is important to consider the Wajibika project in this context, and as a key activity set forth by USAID and the Department of State for comprehensive support to LGAs. The Wajibika project also contributes to many USAID development priorities in Tanzania. The project contributes directly to the second intermediate result under Tanzania’s Global Health Strategy, “Improved Health Systems to Strengthen the Delivery of Healthcare Services”. In addition, the project supports two of the main six strategic goals under Tanzania’s Partnership Framework: leadership, management, accountability, governance and evidence-driven strategic decision making. The Wajibika project also supports nascent initiatives under USAID Forward in Tanzania. The Agency intends to intensify efforts and resources devoted to building host country systems, and strengthen the capacity of Tanzanian institutions to design, implement, and monitor development projects. 2.3 The Wajibika Project On December 16, 2009, USAID/Tanzania contracted Abt Associates to implement a three-year activity titled Wajibika, or “be accountable” in Swahili, with the long term goal of strengthening LGA capacity to independently plan, implement, monitor, and adequately report on programmatic and fiscal progress of health related programs with emphasis on HIV/AIDS, orphans and vulnerable children (OVC), integrated MNCH/FP, malaria prevention, care and treatment. The project’s funding ceiling is $10,000,000. Table 2 presents the Wajibika project’s three key objectives and intended results. The Wajibika project started full operations in the field in February 2010, starting activities with eight councils in the Iringa region. In February and March 2010, the Wajibika project conducted a rapid assessment of the eight councils to inform project design and measurement. In October 2011 during project year two, the project expanded activities to an additional 19 councils in Pwani, Dodoma, and Morogoro regions, bringing the total number of supported regions to 27. Criteria for scale-up included, but were not limited to, regions that were accessible to the project (in terms of time, cost and other resources), and with other potential partners with resources that could be mobilized and coordinated to help meet Wajibika’s objectives (e.g., Africare, FHI Tunajali, PSI, Water Aid, World Vision). In addition, the project selected contiguous councils within the region to facilitate scale up. Similar to the work conducted in Iringa region, the Wajibika team conducted a rapid assessment of these councils, using the findings to guide and tailor project design. 1 State-USAID FY2011 budget request for Africa, p.6. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 15 Table 2: The Wajibika Project’s Key Objectives and Intended Results Objective 1: To improve governance through strengthened programmatic and fiscal accountability in the target districts Result 1.1 A menu of innovative and scalable interventions for promoting local government authorities’ capacity and practices to effectively manage and account for their district’s health, HIV/AIDS, and OCV services; and ensure programmatic and fiscal accountability developed. Result 1.2 Pilot testing of Council strengthening interventions in eight councils is completed and the pilot test results are disseminated to all stakeholders to get buy in for scale up. Result 1.3 A minimum of 27 councils implement council strengthening practices. Result 1.4 Councils involved in the intervention align incentives with desired results related to management of programs. Result 1.5 Strengthened councils receive clean audits in the LGA Basket Funds audit. Objective 2: To strengthen support for decentralized management by the Prime Minister’s Office, Regional Administration and Local Government (PMO-RALG), and Ministry of Health and Social Welfare (MOHSW) Result 2.1 Strengthened councils demonstrate ability to prioritize and plan based on CCHP guidelines and PlanRep2 so that the CCHPs reflect appropriate costs for HIV/AIDS prevention, care, treatment, OVC, reproductive health and family planning, and maternal and child health. Result 2.2 An inventory of all sources of all funds is developed and included in planning and budgets, including USG, private and faith-based facilities, and other donors and organizations. Result 2.3 GOT is assisted in providing clear guidance to councils on roles and responsibilities under decentralization by devolution (D by D) Result 2.4 Pilot testing of performance-based approaches by the target councils is completed, the results of the pilot test are documented and an increased number of target councils replicate the performance-based approaches. Objective 3: To support the PMO-RALG in developing an effective expansion plan for the Government of Tanzania Result 3.1 Effective interventions identified, evaluated and adopted by the GOT with support from Wajibika Result 3.2 Policies and regulations are implemented to accelerate national scale-up of programs that facilitate performance improvement in terms of programmatic management and fiscal accountability. Result 3.3 An inventory of local financial management and strategic planning consultants, companies and organizations developed, and used to provide technical assistance at the district level and to support the broader expansion of effective interventions used under the Task Order. 2.4 The Wajibika Project in Context of Public Sector Reform, Capacity Building, and Decentralization 2.4.1 State building and capacity development The process of strengthening public sector reform in developing countries is complex, often poorly understood, and is entrenched within a myriad of external factors that may help or hinder well-meant efforts. In Capacity Development and State Building: Issues, Evidence, and Implications for DfID, published in 2005, the author Graham Teskey explores whether there is a general theory for state building or capacity building in developing countries that can act as a basis for policy guidance. Although one of the conclusions is that donors can make a positive contribution to state building, donors must clearly distinguish between areas where change can or cannot be affected, and be realistic about the amount of time required to influence change. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 16 Any contribution made toward public sector organization or institutional capacity building must be carefully targeted and designed. Approaches and challenges should be clearly defined, activities sequenced, with stakeholder organizations carefully chosen. Global lessons learned include a focus on accountability for a greater chance of sustainability, employing well-targeted and specific incremental approaches, strengthening specific organizations and shaping values and attitudes that influence wider collective behavior and norms, and prioritizing capacity development. The building of capacity generally is conceptualized on three different levels: (a) building the capacity of individuals, (b) building the capacity an organization, and (c) building the capacity of the system of institutions within which that organization operates. In Capacity Building in Africa: An OED Evaluation of World Bank Support published in 2005 by The World Bank Operations Evaluation Department, the three levels of capacity building are presented. The World Bank report illustrates the complexity of building capacity at the institutional level: Public sectors are often weak, not just because of their lack of capacity, but also because their weakness benefits powerful interests that seek to avoid taxation, regulation, or other reforms. External assistance can help on both the supply and demand sides of the process by providing inputs to enhance the functioning of the public sector and by strengthening structures of demand and accountability. But it cannot directly influence the cultural norms and political economy underpinning the demand for public sector performance. Therefore, capacity- building efforts will succeed only where they take adequate account of the prevailing local politics and institutions…. (p. 8) 2.4.2 Alignment of the Wajibika project with public sector reform and decentralization in Tanzania The project name Wajibika—Be Accountable—and its first objective in particular, indicate the intent for the program to achieve progress toward government accountability in Tanzania. An internationally acceptable standard definition for government accountability might be a proactive process by which public officials inform about and justify their plans of action, their behavior, and results and are sanctioned accordingly. The Wajibika project’s technical approach has targeted more specifically the first part of this definition, the justifying plans of action and results, and therefore the program aligns well, in its hypothesis, with the theory of accountability. The Wajibika project’s objectives and activities harmonize with some general findings on the benefits of decentralization in Africa, inasmuch it is a means to strengthen D by D. Devolution has shown some possible impact on better service delivery, but its clear advantage over other forms of democratization, particularly de-concentration, is its inherent democratic local governance by extending the electoral aspect of democracy down to local levels and facilitating downward accountability.2 The Wajibika project objectives, and its implemented activities, support both of those tested paradigms. Research in Africa is not conclusive regarding otherwise accepted theories that decentralization in itself can provide incentives for economic growth by creating competition between local governments. Although the effect of the Wajibika project on this correlation would be difficult to determine in a mid￾term evaluation, some of the program’s activities may in fact lend themselves to drawing some lessons about motivations and incentives for LGAs to perform more efficiently. Although results are yet to be seen in regards to successfully implementing public sector performance at the local levels, the Wajibika project’s design for adapting targeted interventions for enabling LGA 2 Dickovick, J. Tyler and Rachel Beatty Riedl, Comparative Assessment of Decentralization in Africa: Final Report and Summary of Findings, USAID, September 2010, p. 68. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 17 personnel to implement budget and planning reforms are well supported by best practices in public sector reform in Africa. The Wajibika project’s scope of work is not focused on that demand side of public sector performance, yet its long term success will have to be evaluated in the Tanzanian context of citizen demand for transparency, accountability, and public service delivery. The Wajibika project is also relevant and timely with respect to the GOT’s governance reform programs, which revolve around the following six core governance reforms:  National Anti-Corruption Strategy and Action Plan (NACSAP)  Public Financial Management Reform Programme (PFMRP) Legal Sector Reform Programme (LSRP)  Local Government Reform Programme (LGRP)  Public Service Reform Programme (PSRP)  Business Environment Strengthening for Tanzania Programme (BEST) Wajibika’s approach for sustainable strengthening of LGAs is meant as a tool to ensure better delivery of healthcare services, yet it intersects several of the Government Reform Programs, most notably, the PSRP, LGRP II, and PFMRP, which involve a number of stakeholders, including 13 basket fund development partners (see Table 3). The implementation progress of these reforms, overall, has been significantly slower than anticipated.3 This gap, particularly under the LGRP II, has left a natural space in which an activity like the Wajibika project can intervene, presumably creating positive precedents to be adopted by the GOT and the host for most LGRP II efforts, the PMO-RALG. The Reform Coordination Unit’s assessment in 2011 provides an indication of the Wajibika project’s potential for bridging some gaps in reform implementation. It states, for instance, that, while development of written procedures is a part of any solution to the operational challenges facing the reforms, government officers are often unclear about existing procedures but not in the habit of consulting documents to gain clarity. As is observed later in this evaluation report, the Wajibika project has had some early success with its partner councils in creating some clarity on such procedures, especially with regard to procurement, auditing, and financial management. Table 3: Summary of Reform Phases and Development Partners Most Relevant to Wajibika’s Objectives Reform Lead Agency Phase I Phase II Phase III Current Basket DPs PSRP PO-PSM 1999-2007 2008-2012 IDA, UK, Canada LGRP II PMO￾RALG 1998-2008 2009-2014 Belgium, Finland, Germany, Ireland, Japan, Netherlands, Sweden PFMRP MoFEA 1998-2004 2004-2008 2008-to present IDA, Denmark, Japan, Norway, Sweden, Switzerland, UK, Germany 2.4.3 Forthcoming LGA strengthening projects in Tanzania The $100 million Basic Health Services Project, approved on December 20, 2011 by the World Bank Board, will assist the GOT over a period of four years in improving the equity of geographic access and use of basic health services across districts and in enhancing the quality of health services being delivered. Its technical approach rests on three mechanisms: 1. Financing annual per capita grants to LGAs to support service delivery at the district level and funding medical supplies, drugs, vaccines and contraceptive commodities. 3 Reform Coordination Unit, Assessment of Operational Procedures underpinning the Core Governance Reform Programmes, KPMG Tanzania, May 2011. P.7. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 18 2. Providing funds through a capacity building grant mechanism to improve the capacity of LGAs to manage their health services, and would finance technical assistance, training and systems strengthening interventions, with a focus on improved public financial management, monitoring and evaluation (M&E), facility management, human resources management, procurement and governance and accountability mechanisms (including social accountability). 3. Funding central level interventions including training, technical assistance, management guidance through PMO-RALG and the Regional Health Management Teams, and initiatives to support the performance management process at the central and regional levels, and strengthening of central level oversight structures. The expected outcomes include indicators to measure improvements in accessibility, use and performance, including the proportion of births taking place in a health facility, outpatient attendances per clinical health worker and per capita, and stock outs of tracer drugs and vaccines. All indicators will be monitored at the national and LGA levels, and reductions in the variations between LGAs will be monitored. This project will use a pooled funding mechanism along with other development partners to support the overall health systems development efforts in Tanzania. The project will use further support to the health sector to help create new financing mechanisms to get funding to front-line service providers and to strengthen service delivery systems, thereby improving service delivery at the local level. The project will transform the existing Health Basket Fund in two ways, including changing from unconditional grants to performance-based and equity-focused grants to LGAs, and will develop an approach which can be used across multiple sectors once the appropriate financing mechanisms, monitoring and evaluation systems, and governance and accountability structures are in place. In addition to the Basic Health Services Project, there are three other World Bank financed projects that support capacity building of LGAs, namely the Agricultural Sector Development Project (ASDP), the Local Government Support Project (LGSP) and the Water Sector Support Project (WSSP). Annex H presents a summary of these other three projects. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 19 3. EVALUATION PURPOSE AND METHODOLOGY 3.1 Purpose and Objectives The USAID mission in Tanzania commissioned the mid-term evaluation of the Wajibika project to provide USAID and other Wajibika stakeholders a review of the project’s progress by documenting planned and unanticipated achievements and challenges and by reassessing its objectives and likely outputs. The evaluation proposes mid-course corrections and considers possible longer-term scale-up undertakings based on the Wajibika experience to date. Annex A contains the mid-term evaluation’s Scope of Work. The objectives established for the mid-term evaluation team are to:  Determine the overall progress and achievements in relationship to the key Wajibika project objectives and tasks since the beginning of the program;  Review progress made toward improved governance through strengthened programmatic and fiscal accountability in councils;  Review the progress made toward strengthening support for decentralized management by PMO￾RALG and the MOHSW to improve programmatic and fiscal accountability at the council level, and improved Comprehensive Council Health Plans;  Determine whether the project is on track to develop an effective expansion plan with PMO-RALG for the GOT to undertake the interventions and improved district performance and accountability to all districts based on the lessons from the learning districts;  Identify whether the opportunity to contribute toward the pilot for Performance-based Financing was useful, and whether there are additional sustainable opportunities that could be explored without the U.S. Government financing the bonus payments; and  Identify successes and recommend potential mid-course corrections to ensure optimal results. 3.2 Evaluation Methodology General Approach Rapid Appraisal is an approach that draws on multiple evaluation methods and techniques to quickly and systematically collect data when time in the field is limited. Thus the evaluation team opted to utilize this methodology for its evaluation exercise. Rapid Appraisal teams can gather and analyze relevant information within days or weeks. Rapid Appraisal is particularly useful for formative evaluations during the mid-course of an activity. These quick assessments by external Rapid Appraisal teams often yield insights that may escape activity implementers engaged in the routine collection of quantitative and qualitative data because these teams are not connected to the day-to-day implementation that sometimes constrain the time available for implementers to pursue objective reflection from disengaged perspectives. Data Gathering Through an iterative participatory process, the evaluation team developed a list of evaluation questions with potential follow up probing questions, based on the objectives and initial questions set forth in the Scope of Work (see Annex B: Evaluation Tools). The evaluation team triangulated data (the application of at least three different data collection methods or the application of one method among different groups of informants) to identify evaluation findings. This generated quantitative and qualitative data as a check on the validity of findings from any single method or source. The evaluation team used group interviews, individual interviews, and document review as tools used to gather information: USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 20 Group Interviews: Group interviews were the primary sources of information. The Tanzanian members of the evaluation team co-led the questioning with the U.S. members mainly taking notes and asking questions intermittently. The evaluation team asked questions in English first and then repeated them in Swahili, allowing respondents to choose their preferred language. If Swahili was used, the co-leaders would translate and summarize the information for their other U.S. colleagues. Because most of the informants understood English but may have preferred to speak in Swahili, this process worked well because it allowed them an opportunity to hear how the interviewers received their input. These group interviews were semi-structured with the team following a list of key questions but conducting in-depth probing when pertinent, interesting information was offered. For example, whenever information that was at odds with information collected previously, it would stimulate a line of extemporaneous questioning that focused on this difference. On the other hand, if information conformed to information collected previously, the interviewer would paraphrase and summarize what was said, seek confirmation, and then pursue the next interview questions. The ideal group size for group interviews is 8-12 individuals, but in most instances the LGAs convened between 19-22 staff members. It would have been impolite and logistically awkward to divide into two groups, so the interviewers solicited input from as many as possible. In each meeting approximately 8- 11 individuals were the primary informants. Individual Interviews: At certain times the evaluation team conducted interviews with individuals. For example, mentors did not participate in the group interviews and were interviewed afterwards. Other key informants were selected for their knowledge and key roles in Wajibika. A few key informants did not have intimate knowledge of Wajibika, but they provided information and useful insights from their general knowledge of local government strengthening. Document Review: The evaluation team reviewed documents and reports available from USAID and the Wajibika team prior to the fieldwork and helped inform the questions prepared in advance of the field trip. The evaluation team gathered additional documents during the field trip from local government authorities, key informants, and the Wajibika implementation team. Data Processing and Analysis During field visits, at the end of each day the evaluation team gathered around a laptop and projector to record the notes they had taken during the interviews. For example, at the end of the first day the team spent four hours recollecting and discussing the statements made by informants. The subsequent daily recording of information took approximately two-three hours. By sharing each other’s notes and discussing the implications at the end of each day, evaluation team members helped stimulate recall and clarification of what was said or what was observed (and what was not said or observed). The evaluation team took care to make a full account of the gathered information, even recording statements that appeared to represent an outlier perspective and that seemed to be at odds with the preponderance of information gathered. At the end of the field trip while all team members were still in country, for each of the evaluation tool’s questions the evaluation team leader combined answers from various sources to each question. The evaluation team then set aside a day to review the recorded information to refresh the team’s recall and to brainstorm on findings, conclusions, and recommendations. The evaluation team then worked virtually to construct the report. The evaluation team presented preliminary findings and recommendations to the Wajibika team and to USAID/Tanzania during separate meetings. The comments and further information generated from debriefing discussions have been incorporated into the writing of this report. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 21 4. EVALUATION FINDINGS AND CONCLUSIONS The following are assessment findings for the Wajibika project. The descriptions and program implications for specific project elements, organized under the project’s three objectives and related expected results, outline in detail programmatic strengths and gaps, which are influenced by partners and implementing agencies. Following this are opportunities and threats. These are outside direct programmatic control, and can either enhance or undermine the overall impact of the project. The conclusions discuss the issues and approaches that are relevant for the remainder of the Wajibika project and beyond. These have implications for the GOT and USAID programming. 4.1 Findings 4.1.1 Objective 1: To improve governance through strengthened programmatic and fiscal accountability in councils Expected results under Objective 1 include: a menu of innovative and scalable interventions for promoting local government authorities’ capacity and practices to effectively manage and account for their district’s services and ensure that programmatic and fiscal accountability developed; the pilot testing and dissemination of council strengthening interventions in eight councils; and the roll out of interventions to a minimum of 27 councils. From the work in the field, the evaluation team assessed program activities pertaining to Objective 1 that the Wajibika project is implementing within LGAs. These include: Work at the District Level: Strategic and Annual Planning Strengths:  The Wajibika project improved the Council Health Management Teams’ (CHMTs) capacity to prepare Comprehensive Council Health Plans (CCHPs) as per national guidelines, and with participation from lower level facilities. Mentors assisted with the interpretation of national CCHP guidelines and facilitated the process for lower level and community participation. The project provided a template for incorporating data from lower level facilities into CCHPs, including HIV/AIDS and OVC costs. As a result, LGAs reported that they are now using bottom-up, participatory methodologies to develop CCHPs, with council and lower level staff and community members now understanding the use and implementation of CCHPs. Councils say the CCHPs are complete and compliant with national guidelines.  In some councils, some community and civil society representatives participate in the CCHP preparation. The project helped to facilitate stakeholders’ workshops in which the council presents the draft CCHP and partners also present their plans and activities for incorporation. Some partners provided budget information. Participating stakeholders include Tunajali, EngenderHealth, Restless Development, PSI, IntraHealth, and BMAF.  Some council planning departments reported that the Wajibika project assisted them with the development and implementation of annual council strategic plans. As a result, some councils now have a multi-sectoral planning team with participation from many departments and this reflects comprehensive ownership. Tools include a multi-sectoral council plan, action plan, and supporting Gantt chart, developed through participation with all council sectors and departments, and not just by the planning department. This has resulted in better communication between departments within the councils, cross-fertilization, and ownership of strategic plans by the entire team and not just one department. Gaps:  The Wajibika project has rolled out support for strategic and annual planning inconsistently throughout all supported regions with some councils focusing on CCHP development only and USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 22 others including council strategic planning. It is unclear if these variations are purposeful, based on the project’s pre-implementation assessments and other considerations.  The project did not adapt and implement an optimization tool for strategic planning.  The quality of epidemiological data from health facilities used to develop CCHPs is unclear.  Although some CSOs and NGOs participate in CCHP planning, there is not universal participation within and across councils. CCHPs do not include all non-GOT funded interventions and activities.  Not all CHMTs in Wajibika-supported councils were included in CCHP training and thus were unable to provide comprehensive supervision in the implementation of health programming. Work at the District Level: Financial Management, Procurement, and Auditing Strengths:  District-level internal auditors report that, through Wajibika￾supported capacity building, they have changed the way they conduct auditing. The project assisted in addressing identified gaps in audit planning, conducting audits, and reporting as per international standards. Auditors received training by the Tanzania Institute of Internal Auditors in risk-based planning, auditing, and reporting. Now councils prepare workplans with follow up actions based on audit findings with risk indicators. As a result, these councils reported reduced audit queries.  The project paid the one-year subscription for 90 internal auditors in Wajibika-supported councils to the Institute of International Auditors through the Institute of Internal Auditors of Tanzania. For these auditors, this was the first time they were linked to this resource. They received certificates in Internal Auditing to validate their continuing education. The website gives them tools such as audit programs and workplan templates. Some councils will include forthcoming subscription fees in future council budgets. In addition, the project created a forum for internal auditors to discuss issues and problems and to act as a liaison with upper-level supervisors to address issues.  Council-level accountants and financial managers reported that capacity building trained them to meet international financial management standards (IPSAS). As a result, they report reduced audit queries and better financial management planning, reporting, financial statements, and account closure. A number of councils reported that better financial reports with good data resulted in considerably improved council meetings shortened from 8 hours to 3 hours. Procurement management units (PMUs) reported that Wajibika support has increased their capacity to meet international standards in procurement and tendering. Many PMUs attended training conducted by PPRA in Dodoma in procurement and tendering. In some councils, councilors, tender board members, and accountants received training in procurement and tendering together to better understand procurement processes and conflicts of interest. As a result, these councils reported reduced conflict around procurement and tendering. Quotes from District/Regional Stakeholders: “We recommend Wajibika more than 100%” “Wajibika is implemented by the Council” “We are proud of what we have achieved with the support of Wajibika… we now meet international standards (in financial management, auditing, and procurement)” “Wajibika has helped us open our minds” “Wajibika training has been an eye opener” “Wajibika is a special NGO compared to all other NGOs” “I have been here for eight years and before Wajibika the departments worked in silos. Now there is communication and a better understanding of who does what” “We (council strategic planners) shifted from being planners to coordinators of planning” “People are now results-oriented and solution-focused” “The mentor is highly committed, cares, cooperates, and we improve day after day” “Planning used to be proscribed and now there are other areas of planning” (referring to councils’ work in increasing local revenue generation) USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 23 Gaps:  The Wajibika project did not consistently build the capacity of “user departments” at the council and regional levels to enhance their oversight function and ability to use tools and resulting data. For example, councilors are responsible for approving council budgets, but many do not have financial management and budgeting skills for this role. Some councils have started this capacity building (e.g., some Wajibika councils have been trained in procurement and tendering regulations and procedures).  The Wajibika project was not designed to support similar financial management, procurement, and audit capacity building activities at the regional level. Regions have noted how much their councils have gained through Wajibika support and request similar support. This is considered to be important for the sustainability of the Wajibika approach. Regions conduct their own procurement, review audit reports, and play and advisory role to councils. They must have up-to-date capacity in these areas to be most effective in their roles. However, the project has identified support to regions as a need and is beginning to design capacity-building programs. Work at the District Level: Other Forms of Support for Health and Other Sector/Departments within Councils Strengths:  In responding to requests for assistance in several councils (albeit outside of the original project design), the Wajibika project provided the following support to health, other sectors, and other departments within councils:  The Wajibika project helped the pharmacy departments interpret the MOHSW and planning guidelines and they received training in procurement. Pharmacy staff reported now having a procurement plan with defined procurement activities.  Councils reported having received good support from their mentor who gave them assistance in dealing with various confidential and complicated HR issues.  In Education, one mentor helped organize school committees to address the funding gap for schools.  In Health, the project helped one council organize data and OVC is now integrated into each department’s technical and planning budgets (e.g., agriculture, water, health).  In Community Development, staff received training in project design and implementation to help them identify and oversee projects funded through the Tanzania Social Action Fund (TASAF). In addition, in one council the mentor visited villages and worked with the communities to develop a plan to bridge gaps in government budget in HIV/AIDS. This council is now expecting funding from TACAIDS. Gaps:  The Wajibika project has not consistently rolled out its technical approach towards supporting other sectors and departments. It is unclear if these variations are purposeful, based on the project’s pre-implementation assessments and other considerations. On the whole it appears that the abilities and interests of the mentor drove the forms of support given. Work at the Regional Level: Strengthen Advisory & Oversight Function Strengths:  The Wajibika project has started to work with the Local Government Section within the regional structure to strengthen financial management, internal audit at the regional level to strengthen their financial management and audit functions and ICT.  On the whole, the project is seen to be complementary of the Local Government Section’s role in supporting councils. Gaps: USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 24  In some cases, the regions do not understand the goals and objectives of the Wajibika project. Although the Wajibika project has identified the need to work more with regions to strengthen their technical and oversight capacities, to date there are few linkages between the project and the regions. Capacity Building: Training, Technical Assistance, and Sensitization Strengths:  Stakeholders reported that training has been instrumental in assisting them to meet international standards in their work, such as in internal auditing, procurement, and financial management.  Training has contributed to strengthened team work and planning. The project has linked councils to additional resources (e.g., the Tanzania Institute of Internal Auditing) that serve to provide ongoing technical assistance and support.  Stakeholders reported that technical assistance provided by the Wajibika project assisted with improving multi-sectoral/multi-disciplinary participatory strategic planning at the council level. Wajibika also assisted with the set up and use of systems and tools (e.g., setting up and improving Epicor, troubleshooting provided by ICT teams).  Sensitization helped different members and committees within the council, such as PMU, internal auditing, and their respective users, to understand processes, standards, and the roles and responsibilities of different sections within councils. This resulted in stronger and more efficient performance, meetings, and teamwork, with reduced conflict within the councils.  Stakeholders reported that training has resulted in reported improved performance (e.g., in implementing tendering processes, in conducting internal audits). Gaps:  The Wajibika project has not consistently measured the result of training, technical assistance, and sensitization on performance and although stakeholders state there are several improvements in their work, processes, and systems due to the project, these are self-reported.  The Wajibika project has rolled out training, technical assistance, and sensitization support inconsistently throughout all supported regions; it is unclear if these variations are purposeful based on the project’s pre￾implementation assessments and other considerations. Capacity Building: Mentoring Strengths:  Stakeholders identified the use of mentoring as providing unique support because of the daily interaction with mentors who address the needs of individuals and entire departments. Every interviewed council unreservedly recommended the mentoring approach for capacity building, noting that while regional support is appreciated, the daily interaction on the part of mentors gave Wajibika Best Practice: Additional Local Revenue Generation The Wajibika project has assisted councils to utilize best practices from other councils in increasing district revenues. Before Wajibika, Njombe council started a Revenue Enforcement Taskforce and the council paid two people for their training at the Institute of Tax Management to learn more about revenue collection. The council then focused on collecting revenue from buildings and land. The council established a task force and started community-based revenue collection in villages with incentives. The taskforce continues to identify other areas for revenue generation. This became part of Wajibika when, with the help of the mentor, Mufindi council came on a study tour to Njombe to learn about their experience. After that, Mufindi council discussed which sources of revenue to increase, and decided to focus on property tax on buildings and land. As per their assessment, current local revenue is Tsh 63million. They project that, once they implement their own community-based revenue collection program, this figure will increase to Tsh. 183milion. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 25 them a better opportunity to provide comprehensive support than regional officials, who often can spend only a few days of support.  Most LGAs had glowing recommendations for their mentors, citing their excellent work. Their approach has been to listen and observe carefully before providing advice. Most of the mentors were very experienced and displayed helpful coaching skills. Gaps:  Although all councils and regions professed great enthusiasm for mentoring, all agreed that there is a need for adjustments to the approach. For example, there is no standardized approach for involving councils or regions in the recruitment of mentors. One council provided names and recommendations, although others did not know how the mentor was recruited or relieved of duties. There is no apparent mechanism within the LGAs or regions to monitor mentors or assess their performance although the project conducts these activities. Although the mentor reports to the Council Director (who is part of the CMT) and is monitored through reports, many thought this role would be best situated within the region. The project has no mechanisms in place to measure the results of mentoring on individual or organizational performance.  Some regional secretariats expressed a lack of a full understanding of mentoring and in some cases, individuals within councils did not fully understand mentors’ roles (e.g., as an assistant to perform work duties).  The quality of mentoring is based inherently on the qualities and abilities of individuals. Not all mentors have the skills and ability to support the Wajibika approach, and this hinders council performance. For example, one mentor in one council was able to assist with facilitating lower level participation in CCHP development, but did not have the skills to support the development of the actual CCHP. As a result, the council received poor CCHP review scores that year. The Process of the Piloting and Roll Out of the Wajibika Approach (the Menu of Interventions) Strengths:  Stakeholders unanimously agree that the Wajibika approach, with some modifications, should be taken to scale throughout the country. This includes the expansion of the Wajibika approach to other sectors, such as agriculture and education, within councils and the rollout of the project to new geographic areas.  Some councils have successfully initiated the expansion of technical support through the Wajibika project to sectors in addition to health, such as with council-level strategic planning.  Wajibika conducted an initial assessment within regions to inform the development and start up of the project. Gaps:  Wajibika rolled it its program to additional regions and councils before thoroughly clarifying and testing what the core and expanded package of services and interventions and the regional and council levels ought to be. This has led to inconsistencies in project implementation between councils and regions, and a lack of clarity among stakeholders as to what the “Wajibika approach” means.  The project did not conduct an assessment of the pilot phase before the roll out to additional regions and councils.  The roll out Wajibika to other regions before solidifying the Wajibika approach and ensuring robust programs has overstretched existing staff and resources. 4.1.2 Objective 2: Strengthen support for decentralized management by PMO-RALG and the MOHSW Expected results under Objective 2 include strengthened CCHPs that reflect appropriate costs; an inventory of external resources; the provision of clear guidance to councils on roles and responsibilities USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 26 under decentralization by devolution; the inventory and documentation of existing incentive approaches; technical support for Pay for Performance programs, and support for the expansion of integrated financial management systems within councils. Strengths:  The Wajibika project assisted with PlanRep 2 and Epicor training and the provision of equipment within supported councils, in partnership with PMO-RALG. When additional IT support was needed, Wajibika ICT staff members were available for technical assistance. The project provided servers and computer equipment.  The Wajibika project supports the piloting of the Pay for Performance (P4P) program in Pwani region. Support consists of working with councils to support strategic planning, financial management, procurement, and internal auditing functions, similar to other work being done in other councils, as a method to strengthen underlying council capacity to implement P4P. Gaps:  The Wajibika project is still in the initial steps of conducting an inventory and documentation of existing incentive approaches and conducting an inventory of external resources for use by councils.  As previously noted in Objective 1, although the project has supported the process of strengthening CCHPs that reflect appropriate costs, there are major gaps in including the full resource envelope available within specific councils into CCHPs. 4.1.3 Objective 3: Develop an effective expansion plan with PMO-RALG to undertake the interventions and improved district performance and accountability to all districts Expected results under Objective 3 include: effective interventions identified, evaluated and adopted by the GOT; policies and regulations are implemented to accelerate national scale-up; inventories of external resources are established for use at the council level; and strengthened potential for a transition of USG programs to a long-term-sustainability model for the Wajibika approach. Strengths:  The project is in the process of keeping an inventory of different mentors and distributing that inventory to councils so they can utilize that support (e.g., for CCHP development). The project intends to post this inventory on SHAREPOINT for easy access. Eventually, this inventory will contain trainers at regional level, trainers from the Institute of Internal Auditors and PPRA, and other capacity building resources.  The Wajibika project has influenced policies and regulations to promote strengthened governance and accountability. For example, the project identified a number gaps and non-compliance by councils on the regulatory structure of the PMU. The project conducted advocacy initiatives with the Deputy Permanent Secretary and the PPRA on addressing the observed gaps. As a result of the project’s interventions, a circular was issued to specify the position of the PMU whereby the Head of PMU were asked to report directly to the Accounting officer. Multiple memberships were eliminated in the project areas. The responsibilities of various parties in the procurement process were imparted through sensitization meetings with the said parties and tailor made trainings given by PPRA and Wajibika officials. Gaps:  Although the project has documented some specific activities, it has not yet started a formal process of evaluating Wajibika interventions and using the results to advocate for a formal adoption by the GOT as standard practices. The three-year project is well into project year three, which means that there is little time to conduct this comprehensive activity. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 27 4.1.4 Issues that cross-cut all three project objectives Monitoring and Evaluation Strengths:  In some councils, the project implements evidence-based M&E whereby staff go to the field and collect evidence of project activities, including photographs. Gaps:  Although the project’s performance monitoring plan (PMP) includes some indicators that measure outcomes (e.g., per cent of Wajibika-supported councils that receive a clean overall financial audit report, percent of Wajibika-supported councils that received a clean Health Financial Basket audit report), most indicators measure progress at the input and output level. In addition, the reporting of outcome-level indicators does not include a baseline or measure performance within context (e.g., how Wajibika-supported councils performed against the national average). The project does not report on all PMP indicators within its semi- and annual reports. Annex G presents the indicators and results that the project reported on in its FY 2012 Annual Program Report.  Generally, Wajibika project reporting does not capture activities and achievements in a comprehensive manner. For example, the project implements many innovative activities but these are not included in the PMP or in reports. As such, the project misses an excellent opportunity to communicate the breadth and depth of the program and achievements.  It is unclear if the data utilized by councils for strategic planning and oversight is comprehensive, of high quality, or applicable to the task at hand. Coordination and Linkages to Other Projects Strengths:  In some councils, USG-funded projects participated in strategic planning activities at the council level as a result of Wajibika facilitation of lower level planning. For example, during the development of CCHP many USG government-funded HIV/AIDS projects (e.g., Tunajali, Capacity, EngenderHealth, Champion) participated in the CCHP, and some presented budget data. Councils reported that this participation reduced activity duplication and helped prioritize activities. In other councils, local sources of resources (e.g., CHF, Mawaki, New Forest, cost sharing from district hospitals) were included in lower health facility plans, and therefore in the CCHP, and captured as a result of Wajibika support. Gaps:  Between councils, there are inconsistencies in how councils link to USG and non-USG projects in terms of coordination and resource planning. 4.1.5 Opportunities and Threats Opportunities Accountability  There is a sea change within the GOT related to accountability. For example, the GOT is restructuring its auditing oversight functions as an independent function to increase auditing accountability. The Department of Treasury is creating a department of Internal Audit within the Auditor General to oversee internal audit functions within ministerial, regional, and LGA levels. Procurement and Tendering  In procurement and tendering, a recent PPRA report indicated some poor procurement practices in many councils and that report was submitted to the Prevention and Control of Corruption Bureau (PCCB). This demonstrated a critical need to train both user departments, including heads of departments, and the procurement unit to improve procurement procedures and the Public Procurement Act. This may facilitate additional interest at the council and regional level for USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 28 enhanced Wajibika support, particularly for related capacity within management with supervisory and oversight responsibilities. Audits  The uses of funds under the council budget are based on clean audit results; money is not released otherwise. Strategic Planning  The GOT requires a number of LGA strategic planning deliverables (e.g., a five year strategic council plans, annual CCHPs) and there are existing GOT policies and tools to facilitate strategic planning (e.g., CCHP guidelines and template). As LGAs strengthen their planning processes, there will be more demand for Wajibika support.  The government’s policy includes planning through community participation (e.g., CCHP guidelines, Opportunities and Obstacles to Development (O & OD) Guidelines that are used with communities for planning). Increasing implementation of community participation might create demand for Wajibika support in more LGAs.  It is a government requirement to map all resources at the council level for strategic planning; the Wajibika project can support this activity through coordination with district councils, and facilitate the incorporation of this data into strategic planning.  Donor reviews, Public Expenditure Reviews, and national health accounts can help capture the shadow budget for inclusion into council strategic plans and CCHPs.  CSOs that work in social accountability monitoring across all sectors have the potential to inform council strategic plans. There is also potential for these CSOs to support councils in the participatory development of strategic and annual plans. Support/Capacity Building/Mentoring  The project can use Control and Auditor General reports to assist with the measurement of performance support (e.g., CAG report, council audit reports, PPRA and CCHP reviews).  There is a natural entry and linkage point to strengthen LGA performance at the regional level to which the Wajibika project can align. The Regional Secretariat is comprised of sector-specific technical department support the technical components of the councils. Within the Regional Secretariat, the Local Government Sector is designed to support the development of LGAs.  There is a great need for Wajibika support across sectors and departments to respond to governance and accountability needs (e.g., prioritization & planning, sector-specific plans)  Most councils have acknowledged the need for strengthening LGAs, including mentoring.  There are existing capacity building and technical assistance resources in Tanzania into which training and support related to the Wajibika project can be permanently based. These include: zonal training centers for CCHP development and monitoring, data for decision making based on simple epidemiological analysis, and data management; the Tanzania Institute for Internal Auditing for planning, auditing, reporting; PPRA for procurement regulations and procedures; the National Audit Office for sharing resources on government field audits; and the Institute of Financial Management for training in and compliance with IPSAS.  The Capacity Building Grant under the Local Government Development Grant System can be a long￾term source of funding for training and technical assistance.  Development partners and the GOT are scrutinizing the overall approach towards workshops & seminars paid for in government budgets, and this has the potential to on-the-job/in-situ capacity building and support.  The forthcoming World Bank LGA strengthening project can serve as an additional platform and complementary source of resources for USAID-funded LGA strengthening projects.  The capacity building grants under the Local Government Development Grants system can support expansion to other sectors through capacity building (e.g., Urban and Environment grant, District Agriculture grant, Capacity Building grant). USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 29  It is a policy of the government to contract supportive professional services. The project’s posted inventory, coupled with reminders that this resource exists, will help LGAs to access and source contracted services.  There are organizations in Tanzania that provide technical assistance to strengthen data quality and use; these resources might be used to strengthen council-level strategic planning and oversight.  There are some efforts on behalf of the GOT and donors to strengthen decentralized coordination that the project can utilize (e.g., the placement of a Coordinator in Iringa to coordinate USG-funded projects).  Although LGS staff members within regions themselves do not receive mentoring, many expressed interest in strengthening their own capacity to advise and oversee councils through mentoring. Mentoring is seen as a discreet, trustworthy approach during which individuals can be honest about professional self-improvement.  Both regions and councils expressed willingness to use their own sources to pay for mentors. The mentoring process has raised important questions within regions, such as what role should the regions have with mentors and whether the regions should play the role of mentoring themselves. Among other things, these questions indicate a key level of acceptance for the mentoring approach.  The Local Government Section within each region provides a potential platform as a “home” for mentors. The Assistant Administrative Secretary, Local Government coordinates all issues pertaining to local government issues and it is possible that the mentors would join this team.  The acceptance of mentoring is creating an opportunity for establishing a cadre or association of mentoring within the government. It shifts our continued reliance on training to in-company coaching and capacity building. This shifts the need from outsourcing external support to internal reliance. Threats Support/Capacity Building/Mentoring  Although there is a government policy towards decentralization, not all sectors are devolving at the same pace, which is to be expected. Strategic planning within different sectors may not be comprehensive or aligned to the same bottom participatory method.  Participatory planning, including non-GOT resources and budget figures, is not always comprehensive (e.g., full inclusion of donor funding, NGO/CSO participation). As such, councils do not have a full understanding of the entire resource envelope, including all non-GOT resources and budgets, and council strategic plans are based upon an incomplete analysis of funding and resources. For example, there are 57 NGOs receiving USAID funding in Njombe council. When the District Executive Director attended a USAID review meeting in Iringa, he found out that the council receives Tsh 800m in health funding, mostly HIV/AIDS. The CCHP had a figure of Tsh 53million only.  The demands for Wajibika services might outstrip the ability to deliver, and quality might suffer.  In some cases, there is a lack of technical cooperation between councils within a region and regions do not network with the councils.  Some councils reported that technical assistance from the Local Government Sector within regions is not always available to them.  Although there is a government policy towards decentralization, not all sectors are devolving at the same pace, and within LGAs there might be different needs or timetables for technical assistance in strengthening governance and accountability.  Workshops & seminars from NGOs have supported the development of CCHPs in some councils; any future reductions in these forms of support may affect the implementation of CCHPs.  Any future reductions in public-sector trainings and incentives may discourage participation and affect performance.  There are implications on incentives and motivations, which are expensive (e.g., allowances and per diems). This might impact organizational culture. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 30  Key mentor attributes include maintaining complete confidentiality and discretion. Individual breeches in these areas might taint the overall LGA trust, and thus success of this capacity building approach.  Local Government Development Grants system is also within a pilot phase.  Not all councils are at the same level of development and the future success of Wajibika roll out to additional geographic areas may vary within regions.  The potential demand for support from the Wajibika project by regions and councils may outstrip capacity.  P4P is dependent upon donor funding and political will; the Wajibika project has little influence on the overall success of the initiative. 4.2 Conclusions Contribution to the GOT’s D by D The Wajibika project demonstrates a contribution to Tanzania’s national efforts in public sector reform by accelerating D by D and supporting the move towards e-Government. The technical approach follows many international best practices, including highly targeted activities, incremental implementation, engagement of carefully selected stakeholders, a focus on influencing individual and collective behaviors and values, and an initial embedding of the technical approach and resources into existing infrastructure and institutions for increased sustainability. All interviewed GOT stakeholders considered the Wajibika project to be pertinent, successful, and well accepted within the government. Stakeholders report a substantial contribution to the operationalization of devolution by decentralization. Some stated that the project is assisting with accelerating the process. The growing focus on public sector accountability within the GOT might create additional receptivity and demand for the Wajibika project to support sector-specific, internal auditing, procurement, and financial management capacity building. With the increased emphasis on accountability, there is an opportunity for the Wajibika project to strengthen all LGA sectors and departments and regions as per their own advisory and oversight roles and data use. This comes with expanded collaboration and advocacy for the Wajibika approach with other GOT institutions with related oversight and accountability functions (e.g., Department of Internal Auditing, Ministry of Finance, PPRA, NAO, and the Ethics Secretariat). Transitioning to an standard LGA capacity building practice Within only two years of project implementation, the GOT has expressed high degree of ownership of the technical approach and keen interest in this becoming a standard practice for LGA development. Transition elements include:  Establish and communicate expectations as to what the Wajibika project can and cannot do as the Wajibika approach becomes standard practice.  The integration of the Wajibika approach (capacity building, supervision, mentoring) into national, regional, and council structures and systems  A phased-in budgeting of associated costs into council and regional budgets  The continued expansion of the Wajibika approach to other sectors outside of health  Strengthening the advisory and oversight role of regions  Expanded collaboration with other government institutions with related governance and accountability oversight functions (e.g., Department of Internal Auditing, PPRA, NAO, Ethics Secretariat) USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 31  Expanded collaboration with Tanzanian institutions to provide long-term training and technical assistance (e.g., academic institutions, government training institutions, NGOs with capacity building expertise, private consulting firms, and alliances of independent consultants)  The incorporation of the shadow budget into council-level strategic planning to ensure fully informed governance and oversight The evaluation and dissemination of the project’s first phase is a key first step towards transitioning the technical approach to a national LGA-strengthening standard practice. The advantages of the project roll out to three additional regions before finalizing and evaluating this phase include:  The not heavily prescriptive methodology for implementing the first phase of the Wajibika technical approach became an opportunistic iterative learning process to investigate a range of interventions (e.g., support for multi-sector council strategic planning, technical support to sectors outside of health at the council level).  The innovation of individual mentors contributed greatly to testing a range of interventions.  The project’s first phase evaluation can include an assessment of the process of geographic roll out and applicability of the Wajibika technical approach to other parts of the country. Unfortunately, the project roll out occurred before the project defined what the core and expanded package of services for council and regional levels were to be (or a defined “Wajibika approach”). Implications include the complication of the project’s first phase evaluation by a technical program implemented in an inconsistent manner across councils and regions. The lack of a defined technical approach will dilute the focus and quality of future expansion (working with additional sectors and departments within currently-supported councils) and further geographic roll out. Although stakeholders reported several positive results emanating from the Wajibika project, including improved performance, the project must immediately start to quantify these results through expanded monitoring and measurement. Although the Wajibika project focused on health as the entry point to strengthening governance and accountability, the project provided immediate benefit to the other sectors by strengthening core LGA support functions (procurement, strategic planning, internal auditing). In some instances, Wajibika provided specialized support to other sectors (e.g., community development). As such, the Wajibika project has demonstrated that its approach might be applicable to all LGA sectors and departments, and the project can be used to support USAID/Tanzania’s capacity building portfolio beyond the health sector. The three-year project timeframe is too short a period for the transition into government systems. If support for the Wajibika approach continues, there must be a careful balance between roll out and ensuring that existing programs are robust enough to withstand diluted focus and support. USAID and other donors supporting multi-sector programs at council levels should ensure transparency and complete participation by supported partners in annual planning. This will, among other results, help councils conduct accurate planning and oversight, with increased knowledge of shadow budgets. The leadership and technical expertise of the Wajibika project team in Tanzania has been key to the success of the project to date. As the project solidifies and expands, it is critical for the team to be adequately staffed and with staff placed in an appropriately decentralized manner. The Wajibika team must have continuing guidance and support to assist with a complex project transition. Other Technical Issues As the Wajibika project continues to solidify and roll out, key technical issues and challenges include:  Defining the core and expanded packages of services at the council and regional levels with supporting advocacy for all stakeholders to comprehend fully the Wajibika approach USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 32  The inclusion of regions, which are critical for the sustainability of the overall approach, and in particular the Local Government Sections  Building the capacity of councilors, users, and supply departments for a cohesive approach to accountability and oversight  Quality assurance and mentoring  Placing training in Tanzanian institutions  The participation of CSO/NGOs/communities in strategic planning and monitoring  Increasing participation of CSOs that work in social accountability monitoring across all sectors in LGA strategic planning and oversight  Increasing technical cooperation and shared best practices/lessons learned between councils, regions, and mentors  Data quality at the lower level for strategic planning, monitoring, and oversight Mentoring has played a critical role in the project and stakeholders say this element is the difference between the success of the Wajibika project and other LGA-strengthening initiatives. Mentoring has taken capacity building from a “training only” approach to a performance improvement approach that combined different approaches anchored in mentoring. The success of Wajibika is highly dependent on the quality of the mentor. As the GOT’s approach towards capacity building moves towards in-situ support, capacity building approaches might shift towards more mentoring and coaching in the workplace. This overall movement might create more demand for the Wajibika project within councils and regions. Standardizing sustained mentoring approaches is feasible and realistic, considering the nature of the approach; however ensuring the quality of results-focus mentoring will become increasingly important and complex. The Wajibika project can continue to play an indirect support role to councils implementing P4P through the continued roll out of the core and expanded packages of services at the council and regional levels to strengthen overall governance and accountability. There are also possibilities to extend support to data management and data quality auditing to support council accountability for data. Although the Wajibika project has not formally codified defined models or approaches, (what in the future might be called “the Wajibika approach”), Table 4 presents what the evaluation team observed as the Wajibika packages of services. Nonetheless, the final codified approach(es) will require substantial review and evaluation. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 33 Table 4: Potential Wajibika Packages of Services The Wajibika Approach Goal: Strengthen LGA Governance and Accountability Wajibika Core Services Council level: Strengthen supply service capacity:  Financial management, procurement, ICT, internal auditing Strengthen sector technical capacity:  Strategic planning  Sector-specific technical assistance Increase LGA revenue generation Regional level: Strengthen advisory & oversight capacity Regional & council levels: Advocacy for Wajibika-elements in LGA budgets Professional subscriptions WajibikaPlus Regional level: Strengthen supply service capacity:  Financial management, procurement, ICT, internal auditing Regional & council levels: Inclusion of “users” into layman’s technical and oversight capacity building  Elected councilors  District Executive Directors  Department heads  Managers Key Stakeholders PMO-RALG MOF Department of Internal Audits MOHSW Other Sectoral Ministries How? Advocacy Mentoring Technical Assistance Peer Learning Training Documentation Mainstreaming the Wajibika project into donor and technical dialogue structures USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 34 5. RECOMMENDATIONS The following recommendations and technical considerations are intended to guide the Wajibika project in becoming a state-of-the-art resource to the GOT as it goes through its processes of public sector reform and decentralization. The aim is to assist the project in continuing to align to international best practices through a capacity development approach. The recommendations also intend to align USAID investments more closely to GOT and other donor initiatives in LGA strengthening, most notably the World Bank-funded Basic Health Services Project. Finally, the recommendations intend to promote USAID Forward objectives, especially strengthening partner capacity to improve aid effectiveness and sustainability and strengthen collaboration and partnership with donors and international organizations. 5.1 Project Recommendations and Next Steps Before End of Project 1. Define, evaluate, and disseminate the Wajibika technical models: Before the end of the project, the Wajibika project must assess, define, package, and disseminate the project’s technical models, or the “Wajibika approach”. The project should not roll out to other regions during the remaining life of the project, but focus on solidifying and measuring achievements. Steps include:  Finalize the project’s first phase with a robust evaluation and compilation of best practices and lessons learned during project expansion and roll out as an operationalization of the GOT’s Decentralization by Devolution policy  Based on evaluation findings, define the core and expanded packages of services at the council and regional levels with supporting advocacy plans for all stakeholders to comprehend fully the Wajibika approach  In close collaboration with the MOHSW & PMO-RALG, start the dissemination of the Wajibika approach and technical models to national, regional, council, and lower level stakeholders. Before the end of the Wajibika project: The Wajibika project will finalize the evaluation of the project’s first phase, the definition of Wajibika technical models, and initiate dissemination before the end of project year three. The Wajibika project will include this activity in the project’s third year workplan, and conduct it with substantial support from Abt Associates. 2. Measure project results and increased sustainability: The Wajibika project will revise its PMP and quantify project results at output and outcome levels with a focus on measuring improvements in performance and increased sustainability. Table 5 includes sample indicators but as and where possible, the project should use common indicators (e.g., MTEF, World Bank Basic Health Services project). In addition, performance against indicators may be direct or indirect, and as a contribution to a program that receives complementary funding from other donors. Where possible, measurement will indicate performance against a baseline (e.g., per cent of Wajibika￾supported councils that receive 90% or higher CCHP score, pre- and post-intervention). In addition, the Wajibika project will develop performance indicators and tools that measure the technical robustness of programs within specific councils and regions, the comprehensive coverage of core and expanded packages of services, and specific steps taken by each council and region towards increased sustainability. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 35 Table 5: Sample Indicators for the Wajibika Project Percent reduction in audit queries in Wajibika-supported councils and regions Percent of CCHPs that include budget figures and activities from all USG-funded activities within respective councils Percent of councils/regions that implement the core package of Wajibika services Percent of councils/regions that fund the Wajibika core package of services, by entire core package of services/specific elements within the package Percent of councils implementing local revenue generation activities Percent increase in generated local revenues, projected/actual Percent of mentors who are paid by the GOT Percent of Wajibika-supported councils that receive 90% or higher CCHP score The ratio of CCHP scores between Wajibika- and non-Wajibika- supported councils Percent of Wajibika-supported councils that pass the PPRA, CAG Number of LGA best practices disseminated by mentors to other councils, by type of best practice, number of councils per best practice Number of councils/regions that implement disseminated LGA best practices, by type of best practice, number of councils per best practice Before the end of the Wajibika project: During the last project year, the Wajibika project will update its Performance Monitoring Plan to include indicators that measure different facets of sustainability and project achievements at the outcome and output levels. As soon as possible, the project will start to collect data on these indicators, and where possible, include a baseline. The project should also develop and implement indicators and tools that measure performance at the council and region levels. 3. Integrate the Wajibika technical approach as a GOT standard LGA-strengthening practice: The Wajibika project needs to demonstrate vision and ability to transfer the technical approach to the GOT. The project should work closely with the GOT to develop the Wajibika Transition Strategy. This strategy should articulate and measure a phased transition and advocacy plan to integrate the Wajibika approach as a GOT standard practice for LGA strengthening. The strategy should include an operational plan and indicators. Elements should include:  Inclusion of Wajibika in existing technical dialogue structures  Defined core and expanded packages of services at the council and regional levels  The integration of the core and expanded packages of services into national, regional, and council structures and systems  A phased-in budgeting of associated costs into council and regional budgets  Increased generation of local revenues  The inclusion of donor support for project activities into existing government systems (e.g., capacity building paid through the Local Capacity Building Grant under the Local Government Development Grant System)  The sequenced expansion of the Wajibika approach to other sectors outside of health (e.g., agriculture, education) through the deliberate implementation of the Wajibika packages of services.  The use of capacity building grants under the Local Government Development Grants system to support expansion to other sectors through capacity building (e.g., Urban and Environment grant, District Agriculture grant) with support to councils to utilize these resources for their sectoral objectives  Strengthening the advisory and oversight role of regions USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 36  Expanded collaboration with other government institutions with related governance and accountability oversight functions (e.g., Department of Internal Auditing, PPRA, Ethics Secretariat)  The incorporation of the shadow budget into council-level strategic planning to ensure fully informed governance and oversight Before the end of the Wajibika project: The Wajibika project, with substantial technical assistance from Abt Associates, will work closely with PMO-RALG, the MOHSW, and other line ministries to develop the Wajibika Transition Strategy, operational plan, and indicators. As feasible, the project will start strategy implementation. 4. Strengthening the Wajibika project to act as a transition mechanism: As the GOT adopts the Wajibika approach as a standard practice for strengthening LGAs, the Wajibika project must evolve from an implementation project to a resource that supports the GOT with state-of-the-art technical support and advice. Issues include:  The GOT and donors should avoid project roll out to additional regions until the project’s first phase is completed and existing council and region programs are robust  The provision of targeted technical assistance by Abt Associates to the Wajibika project, particularly during the development of the Wajibika Transition Strategy, including the expansion of measurement and monitoring indicators and practices, and the provision of international best practices/approaches in public sector reform and devolution.  Evolving from an implementation project to a state-of-the-art resource organization is a complex for even the most seasoned professional; Abt Associates should provide on-the￾ground mentoring to Wajibika project staff to assist with this delicate and complicated process.  PMO-RALG should work with line ministries and the Wajibika project to create an advisory committee to advise on the sustainability/transition of Wajibika. Members should include key government stakeholders, including institutions with governance and accountability oversight (e.g., Department of Internal Auditing, PPRA, Ethics Secretariat). Before the end of the Wajibika project: Abt Associates will develop a technical assistance plan to start the implementation of the Wajibika Transition Strategy, to be included in the project’s third year workplan, and source mentoring support for the Wajibika project itself as they implement the strategy. The project should ensure that it is present and contributes substantially in national-level dialogue structures. 5. Strengthen the technical quality of existing council and regional programs: Before the roll out of project activities into additional councils is undertaken, the Wajibika project must ensure that existing programs implement the core packages of services at the council and regional levels. Elements include:  A mapping of interventions and activities currently underway in councils and regions  The development of performance indicators and tools that measure the technical robustness of programs (see Recommendation #2)  Link to other resources to support the technical quality of programming (e.g., capacity building in data management (collection, interpretation, and application for planning) to strengthen the quality of CCHPs and council strategic plans  Decentralize mentor coordinators to the regional level for enhanced support, quality assurance, and oversight to mentors supporting councils (e.g., one mentor coordinator in Dar es Salaam to support Pwani and Dodoma, one mentor coordinator in Iringa to support Morogoro, Iringa, and Njombe) Before the end of the Wajibika project: The third year project workplan will include mapping, implementation, and measurement activities to ensure the full coverage of core and expanded USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 37 packages of services within councils and regions. The project will decentralize mentor coordinators. As the Wajibika project continues with the implementation of the third year workplan, the project will take into account additional technical considerations presented in Section 4.2. 6. Coordinated donor support for LGA strengthening: USAID and other donors should actively coordinate their support for LGA strengthening. For example, donors should support the construction and monitoring of robust council-level strategic plans. This includes encouraging the participation donor-funded partners in council-level annual planning and monitoring activities with budgets. If possible, USAID should harmonize its partner planning with the GOT planning cycle (e.g., align partner workplan development with the GOT fiscal year and planning calendar). Annex F includes the GOT’s Planning and Budgeting cycle. Table 6 indicates the most appropriate time USAID-funded partner work plans should be finalized for these partners to contribute to the LGA planning process in their respective regions. Table 6: Harmonization of LGA and USAID-funded partner planning cycles Month Task November – December PMO-RALG issues budget guidelines to LGAs after consultation with sector ministries January LGAs communicate Indicative Planning Figures (IPFs) to ward development committees and village councils. 50% of LGCDG is allocated to villages and wards for planning; remaining 50% is allocated by the LGAs themselves. February Village plans prepared using O & OD, supported by district & ward facilitators, reviewed by village councils and approved by village assemblies. March WDC reviews and consolidates village plans into ward plans, and forwards to LGAs. March – April Ward plans are reviewed by LGA planning departments, and combined with LGA￾level development and recurrent budgets. The LGAs can send wards’ plans back for revision. March seems to be the most appropriate time for USAID-partner workplans to be finalized for partners to contribute to LGA planning. April LGA plans are reviewed by council planning committees and submitted to the full council for debate and approval. April LGA plans forwarded to regional secretariats for review, compilation, and forwarded to PMO-RALG. LGAs may be asked to revise plans before they are sent to PMO-RALG. May LGA plans are reviewed and consolidated into a single PMO-RALG plan and budget for submission to Parliament. June – July PMO-RALG plan and budget debated and approved by Parliament. August Final approved budget communicated to LGAs and disbursement begins. September Wards and villages are informed of the final plans, and project implementation begins. In particular, the World Bank supported Tanzania Basic Health Service Project (TBHSP) appears to be an ideal match for scaling-up the Wajibika capacity building model. The mechanisms that the TBHSP has established for the transfer of funds down to the LGA level might be used for rolling-out and sustaining the Wajibika approach. Before the end of the Wajibika project: USAID should enter into dialogue with the GOT and the World Bank to establish areas of collaboration under the Tanzania Basic Health Service Project, and champion the project in existing donor coordination fora. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 38 5.2 Other Issues and Technical Considerations Project Expansion and Roll Out  As PMO-RALG and other donors consider the roll out of the Wajibika approach to other councils, they should consider packaging best practices and lessons learned for PMO-RALG’s use. The MOHSW might agree to play a role in advocacy to promote what it gained under the project to other line ministries.  Wajibika should work with PMO-RALG, Directorate for Local Government, at the national level to update policy that inculcates the Wajibika approach as standard practice. This should include the provision of mentors, either as a Government established post or as a contracted service.  The Wajibika approach must be mainstreamed into the dialogue structure of the SWAp and within the appropriate technical working groups. The Role of Regions in Supporting the Wajibika Approach  PMO-RALG, with line ministries, need to clearly define the role of regions supporting the Wajibika approach. Interventions must be phased in, with regions working with Wajibika to prioritize areas of support. Potential areas of support, to be included within core and expanded packages of services for the regions, include:  Support to regions to simplify policies and guidelines, disseminate them, and include them in strategic and other planning  Strengthening capacity to interpret project data and monitoring oversight  Assistance with timeliness of reporting (e.g., consolidation of LGA reports)  Strengthening LGA advisory capacity (e.g., M&E, financial management, procurement rules and regulations)  Strengthening the Regional Secretariat’s capacity in support functions (e.g., strategic planning, financial management, procurement, and internal auditing)  Supporting the use of tools and systems at the regional level (e.g., ICT and computer usage, PlanRep, Epicor)  Positioning regions to assume some aspects of mentoring  Strengthening the capacity of Regional Health Management Teams (focused on health but can expand to other sectors in later phases) Capacity Building: Training, Technical Assistance, and Mentoring One key area of sustainability is the permanent placing of Wajibika-related training and technical assistance into Tanzanian institutions from which regions and councils can source support. Options might include:  Zonal training centers for CCHP development and monitoring, data for decision making based on simple epidemiological analysis, data management, and mentor training  The Tanzania Institute for Internal Auditing for planning, auditing, reporting  PPRA for procurement regulations and procedures  The National Audit Office for sharing resources on government field audits  The Institute of Financial Management for training in and compliance with IPSAS. Over time, regions and councils should plan for training and technical assistance within annual budgets, and consider the use the existing capacity building grants under the Local Government Development Grants system to support expansion to other sectors through capacity building (e.g., Urban and Environment grant, District Agriculture grant, Capacity Building Grant). The sustainability of mentoring is of paramount importance to ensure quality roll out and expansion of the Wajibika approach over time. The Wajibika project needs to work closely with PMO-RALG and other stakeholders to address many critical issues around mentoring, including: USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 39  The placement of mentors within the GOT: Many stakeholders thought the mentor should be placed within the Local Government Sector within the region and report directly to Assistant Administrative Secretary, with indirect lines of communication to the DED. Many mentors stated a marked preference to avoid direct supervision and salary payment from the council since it was seen as a conflict, limiting freedom to comment and advise. Therefore a mechanism for overseeing mentors’ performance with clear supervisory and reporting lines is needed. Councils and regions should be directly involved in the recruitment and supervision of mentors, and work with mentors to determine priority areas of support.  Mentoring and mentoring support at the regional level: There are questions as to the role of the region and mentoring. Many stakeholders thought the regional structure should receive mentoring themselves in order to properly advise and oversee the councils. Some thought regions should be capacitated to sustain the Wajibika approach and provide mentoring post￾project. There is a need for a regional mentor coordinator, or possibly one coordinator managing a realistic multi-region caseload, to provide support to mentors working with councils.  The nature and composition of mentors: Some stakeholders questioned whether one mentor is enough. Although councils stated that they must maintain transferred knowledge to sustain the Wajibika approach, even without a mentor, this is a long-term process. Some thought that during years 1-2, councils should have more than one mentor with the full range of skills and experience to assist with the Wajibika approach. Over time the mentor could be phased out, or have a roving mentor.  Ensuring the quality of mentoring: Although the quality of mentoring is based inherently on the qualities and abilities of individuals, the Wajibika project must develop processes and tools to standardize mentoring approaches, as feasible and realistic. Ensuring the quality of results￾focus mentoring will become increasingly complex as the project rolls out to additional regions and councils. Approaches include:  The establishment of basic mentor qualifications and areas of expertise (e.g., substantial LGA experience, willingness and ability to learn quickly, excellent listening skills, and technical competence in 3-4 areas (such as strategic planning, financial management, procurement, human resources).  The development of a comprehensive mentoring curriculum be developed that contains practical skills assessment exercises and technical/mentoring skills training modules, and ensure that mentors will have a minimum set of mentoring skills before being assigned to LGAs.  Other approaches include performance-monitoring tools, 360-degree reviews, and the collection and dissemination of mentoring best practices and lessons learned to other mentors.  Compensation for mentors: PMO-RALG and the project should work closely together to determine the best way forward into assimilating mentoring into the LGA program. Options include making mentors full time members of the government or contracting them through existing contracting mechanisms. The Wajibika project should investigate how BMAF transitioned their staff to the GOT. Over time, the GOT should consider financing mentor positions and possibly including a cadre for mentoring.  Disseminated mentoring best practices, approaches, and lessons learned: This helps ensure a continual learning process and assists with appropriate levels of standardization as the mentoring program expands. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 40  Non-financial incentives for mentors: The project should consider the use of non-financial incentives to motivate mentor performance (e.g., recognition incentives “mentor of the year”, “highest CCHP score”; learning incentives such as inter-council peer assessments, study tours, mentor/council technical cooperation and support fora). Support for Council-Level Activities In general, there is a critical need to train user departments and support function units. This ensures a more thorough understanding of roles and responsibilities and data and tool use. It also reduces misunderstandings and conflicts. Examples include:  Capacity building with the procurement unit, councilors, DED, and department heads in:  Procurement rules and regulations, with a focus on tendering.  The transition towards E-Government and computer skills  Fiduciary management  The uses and benefits of internal auditing  The implementation of existing revenue laws and opportunities for increasing local revenue generation  Training planners and auditors in Epicor The Wajibika project should consider the incorporation of social and community accountability structures in the councils into the Wajibika approach as another approach to increasing accountability. As the GOT consolidates accounts and plans across sectors at the council level, the Wajibika project should support this process. Possibilities include support for establishing one consolidated data center at the region with links between the region and all councils. As the project expands to support other sectors within councils, the project should consider prioritizing support to sectors that have the equivalent of CCHPs (e.g., Agriculture, Water) as an entry point via strategic planning. There are future GOT plans for consolidated multi-sector annual planning, and the Wajibika project should be poised to support this process. This is important for USAID in regard to Feed the Future. The Wajibika project should investigate how community development officers and Council HIV/AIDS Coordinators (CHACs) might participate in the project in support of HIV/AIDS and OVC. This cadre could assist other sectors in crosscutting sectoral plans that involve community participatory approaches and non-state actors. Pay for Performance As previously noted, P4P is dependent upon donor funding and political will and the Wajibika project has little influence on the overall success of the initiative. However, there is potential for continued Wajibika support:  An indirect method of support is the continued roll out of the core and expanded packages of services at the council and regional levels to strengthen overall governance and accountability  A direct method of support is to expand support to strengthen data management capacity and conduct data quality audits. Since P4P is paid performance based on achievement against targets, data becomes a de-facto form of currency. Councils will need to strengthen their accountability for collecting, monitoring, and auditing these data. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 41 ANNEXES A. Scope of Work B. Evaluation Tools C. Evaluation Calendar and Logistics D. Persons Contacted E. References F GOT’s Planning and Budgeting Cycle G The Wajibika Project’s FY 2011 Annual Program Report Results H Forthcoming Support to Local Government Authorities in Tanzania USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 42 Annex A: Scope of Work Statement of Work Wajibika Mid-Term Performance Evaluation I. PURPOSE USAID/Tanzania requires a team of two consultants to conduct a mid-term evaluation of the Wajibika Project to (1) determine the extent to which the expected results of strengthened programmatic fiscal accountability have and will be achieved; and (2) provide information either for mid-course correction or planning of future funding activities. In addition, this evaluation is expected to help stakeholders understand successful approaches that should be emphasized in future programming. II. BACKGROUND As the United States Government (USG) transitions more programs to local organizations/governments, and as USAID proceeds with the USAID forward procurement reform, it is essential to mitigate risk and increase aid effectiveness by strengthening accountability. Strengthening accountability is particularly important at the district level, where funding flows for health service delivery. In December of 2009, USAID awarded Abt Associates, under the AIDSTAR II mechanism, funding for a project called “Wajibika,” which means “be accountable” in Swahili. Wajibika supports the Government of Tanzania’s (GOT) initiatives to strengthen Local Government Authorities’ capacity for financial and programmatic accountability under the decentralization by devolution (D by D) program. Recent reforms in local governance, notably the launch of the second local government reform program, the Local Government Reform Programme II: Decentralization by Devolution (July 2008–June 2013), offer a historically unprecedented opportunity to support and strengthen the capacity of local government authorities (LGAs) to operate independently of central and regional authorities in managing Council health service delivery. Abt Associates is working on this initiative with the Prime Minister’s Office-Regional Administration and Local Government (PMO-RALG), the Ministry of Health and Social Welfare (MOHSW), as well as Family Health International, on this initiative. The Wajibika project employs a variety of interventions to increase accountability through improved prioritization, planning, and budgeting skills; better data use; more effective feedback mechanisms related to program execution; improved internal controls and financial management/reporting, improved performance incentives; and coaching and on-the-job training for local government authorities. This comprehensive approach will assist the GOT in fostering greater country ownership, improved governance, and increased potential for integrated, sustainable and high quality programs. Wajibika will promote strategic advocacy with stakeholders at all levels of government, spurring competition among LGAs to encourage better performance and motivating health workers through appropriate incentives. Wajibika has been underway in all eight districts of Iringa, where focus is being placed on strengthening districts to prioritize, implement, and monitor essential health programs, catalyzing synergy between and among health programs, demonstrating results, ensuring fiscal USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 43 accountability, and coordinating health resources such as faith-based, private, Global Fund, basket funds, and USG funding. Now the program has been expanded to an additional 19 districts in the Dodoma, Morogoro, and Pwani regions. Both the MOHSW and PMO-RALG have agreed to 1) implement promising practices or systems remedies identified under Wajibika to the remainder or the country, and 2) reduce ambiguity that precludes effective health system decentralization. During this past year, Wajibika also participated in a performance-based financing pilot in the Coast Region, designed to better align incentives with desired performance and outcomes. The pilot was planned and implemented by the Clinton Foundation, while Wajibika’s role was to ensure the monitoring and accountability systems were in place. The Wajibika team was to have applied the following key approaches: Promote strategic advocacy and country ownership with stakeholders. Wajibika has conducted advocacy with stakeholders at all levels, including those responsible for the decentralization process under the Office of the PMO-RALG; The Regional Administrative Secretary; Regional Commissioners; the MOHSW; and Council and community stakeholders. Strengthen partnerships among all sectors. Wajibika fostered the coordination of public, private, and commercial sector partnerships to give Non-Government Organizations (NGOs), Faith Based Organization (FBOs), civil society organizations, and commercial accounting and auditing firms a voice at national, regional, and Council health sector meetings. Motivate health workers through a pay-for-performance system. To the extend possible given programs developed by the MOHSW, Wajibika supported the infrastructure for options related to incentive based performance improvement to Council teams based on their increased resource accountability and resource coordination. This was somewhat limited due to other constraints around Tanzanian’s pay-for-performance efforts. Spur competition among LGAs to encourage better performance. Wajibika applied market￾based principles to foster competition among the first eight “startup” Councils and the scale￾up Councils, providing LGAs with an incentive to improve the planning, budgeting, execution of integrated programs, and monitoring. Mentoring and coaching on-the-job was employed as the primary mode of skills transfer to GOT staff, especially at the LGA level using Mentors provided with enhanced skills by the Wajibika team and seconded to the councils. III. OBJECTIVES This mid-term evaluation shall:  Determine the overall progress and achievements in relationship to the key Wajibika project objectives and tasks since the beginning of the program;  Review progress made toward improved governance through strengthened programmatic and fiscal accountability in councils;  Review the progress made toward strengthening support for decentralized management by PMO-RALG and MOHSW to improve programmatic and fiscal accountability at the council level, and improved Comprehensive Council Health Plans; USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 44  Determine whether the project is on track to develop an effective expansion plan with PMO-RALG for the GOT to undertake the interventions and improved district performance and accountability to all districts based on the lessons from the learning districts;  Identify whether the opportunity contributes toward the pilot for Performance-based Financing was useful, and whether there are additional sustainable opportunities that could be explored without the U.S. Government financing the bonus payments.  Identify successes and recommend potential mid-course corrections to ensure optimal results. IV. KEY SUCCESS FACTORS The following outcomes are expected to determine successful implementation within the districts:  Improved skills and effectiveness  Improved methods of prioritization, particularly use of data  Improved financial planning, management and accountability  Improved program planning and management  Improved monitoring of results  Increased effectiveness in program implementation  Improved resource mobilization capacity based on potential funds available V. EVALUATION METHODOLOGY The Evaluation team shall use facilitative methods and activities that will enhance collaboration and dialogue among counterparts particularly the MOHSW and the MOF. The Evaluation team shall work under the supervision and guidance of the AOTR and USAID Program Office, in collaboration with the Local Government Liaison in the Health Resource Secretariat at the MOHSW. The Program Office will organize all internal USAID meetings including linking the evaluation team with the Health Office Team Leader, Systems Strengthening team, and other technical staff. The Evaluation team shall propose and organize the evaluation process. The evaluation design and work plan shall be presented to the COTR and relevant Health and HIV/AIDS Team members for comments during an initial meeting prior to arrival of the evaluation team in Tanzania. The Wajibika COTR will arrange for an initial introductory meeting with appropriate MOHSW and PMO-RALG staff at the outset of the process. Where necessary, the COTR may participate in meetings with the GOT representatives and partners. A general list of relevant stakeholders and key partners will be provided to the Evaluation team by the COTR at the time of arrival but the Evaluation Team will be responsible for expanding this list as appropriate and arranging the meetings and appointments so as to develop a comprehensive understanding of the program and services offered under the guidance of the COTR. Internal USAID/Tanzania meetings will include, at a minimum:  Initial organizational/introductory meeting at which the Evaluation Team will present an outline and explanation of the design of the Evaluation; USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 45  Mid-term evaluation review with the Program Office Team to outline progress and implementation problems; and  Final Evaluation debrief/summary of the data, draft recommendations and report. Prior to conducting fieldwork, the evaluation team will review existing literature and data, including program strategies, quarterly reports, cooperative agreements and modifications and other reports and documents reflecting Wajibika’s work in Tanzania. Over time, Wajibika has documented the evolution of the project and implementation of various health system strengthening activities. Annex 1 is a detailed list of reports, studies, publications and other documents for review. Field visits: the project is being implemented in all districts in Iringa, Coast, Morogoro and a few within select regions. The Evaluation Team shall arrange to visit selected sites supported through Wajibika in consultation with the COP and Wajibika in country leadership. The selected sites should include a representative sample of activity sites and should represent variety along the following dimensions: geographical location, technical focus of activities and sub-partner. The Evaluation Team will be accompanied by a member of staff from USAID/Tanzania Program. The site visits will involve interviews with District Executive, Internal Audit and Health Management Teams, health facilities sites and interpersonal communication and capacity building activities coordinated by the Wajibika project. The purpose of these site visits is to gain a better understanding of the technical competence of Wajibika staff and the attitudes and knowledge of key beneficiaries, the constraints encountered in the various categories of activity implementation, and key target audiences’ perception of their needs in order to provide quality services. The Evaluation Team should outline key meetings to coordinate post field visits in order to share findings and get final inputs before preparing the report. VI. TEAM COMPOSITION Team Leader/ Senior Evaluation Specialist: should have a post graduate degree in health or an applicable social sciences field. S/he should have at least 5 years senior level experience working in Health System Strengthening and Reproductive Health/Population Programs in a developing country. S/he should have extensive experience in conducting qualitative evaluations/assessments. Excellent oral and written skills are required. The Team Leader should also have experience in leading evaluation teams and preparing high quality documents. A writing sample is required. The Team Leader will take specific responsibility for assessing and analyzing the project’s progress towards targets, factors for such performance, benefits/impact of the strategies, and compare with other possible options. S/he will provide leadership for the team, finalize the evaluation design, coordinate activities, arrange periodic meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations into a high quality document. S/he will write the final report. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to the USAID/Tanzania team and other major partners. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 46 Health Systems Strengthening Technical Advisor: should have a post graduate degree in public health or related subject. S/he should have at least 10 years’ experience in decentralization local government support, as well as some knowledge of innovative incentive based approaches and implementation in developing countries and demonstrated knowledge of state-of–the-art strategies. S/he should be knowledgeable in program assessment and evaluation methodologies in human resource and business plan development and training in information monitoring systems. S/he should have extensive experience in conducting program evaluations/assessments and a thorough knowledge of evaluation methodologies and organizational and institutional capacity building. In addition, the evaluation team should comprise of individuals with the following expertise: 1. Extensive development evaluation experience with substantial work experience in the African context. Experience in Tanzania is preferred. 2. Experience with PEPFAR funding 3. Experience in Monitoring and Evaluation of system strengthening programs. 4. Ability to take initiative to deliver required outputs and planned results. 5. Excellent information technology skills, including word processing, database applications, presentation software and Internet. 6. Excellent analytical skills and report writing ability. 7. Excellent interpersonal communication skills including ability to facilitate and work in a multi-disciplinary team. 8. Extensive experience working with the US government and particularly USAID funded programs 9. Extensive experience with USAID rules and regulations for grants and financial management 10. Knowledge of Tanzania government systems 11. Wide-ranging capacity development expertise 12. Experience using participatory approaches and methodologies 13. Vast qualitative research expertise 14. Vast quantitative research experience including statistical expertise 15. Fluency in Kiswahili is imperative due to communication with targeted beneficiaries. (at least one team member) 16. At least 80% of the time should be spent in Tanzania VII. TIMELINE USAID/Tanzania anticipates that the period of performance of this assessment will be 37 days. This would include preparation days, in-country work in Dar es Salaam and the regions, and report writing and finalization. The assessment will take place September 15- October 31, 2011. The following is a sample timeline. Task/Deliverable Duration 1. Review background documents & offshore preparation work 3 days 2. Travel to Tanzania 2 days 3. Meet with USAID/Tanzania Health team 1 day 4. Information and data collection 14 days USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 47 5. Mid-evaluation meeting 1 day 6. Data analysis 4 days 7. Draft assessment report in-country 3 days 8. Debrief meetings with USAID, Wajibika team, GOT Counterparts and key Stakeholders 2 days 9. Prepare final evaluation report 3 days 10. Depart Tanzania 2 days 11. Offshore Finalization of Contract Deliverables 2 days TOTAL # days 37 days Note: Final Report to be delivered before departure from Tanzania VIII. LOGISTICS A six-day workweek is authorized; local holidays are not authorized. The Evaluation Team will be responsible for all offshore and in-country logistical support. This includes arranging and scheduling meetings (with exception to previously mentioned meetings with GOT and initial introductory meetings), international and in-country travel (including vehicle rentals), hotel bookings, working/office space, computers, printing and photocopying. A local administrative assistant/secretary may be hired to arrange field visits, local travel, hotel and appointments with stakeholders. In addition, the Evaluation Team Leader is responsible for draft and final report development, as well as other eligible expenses associated with the completion of the assignment. IX. EXPECTED DELIVERABLES The contractor deliverables shall include: 1. A proposed timetable for evaluation period. 2. A written methodology plan during the pre-evaluation meeting to include: a. stakeholder analysis b. final evaluation design c. data collection instruments d. data collection plan e. operational work plan 3. A proposed outline of the report on the findings and recommendations of evaluation with possible issues for discussion during the mid-evaluation meeting (within 2 weeks of arrival in-country). 4. A completed draft of evaluation report to USAID/Tanzania for presentation during the debrief meeting that will be held 3 days before departure (see report format provided in “IX. Reporting Requirements” below). 5. List of all reviewed/cited sources, including interviewees in final report. 6. A completed and revised final document delivered to USAID/Tanzania (after incorporating feedback from USAID/Tanzania) a month after the debrief meeting (2 hard copies and a USB Flash drive containing the report, final instruments, qualitative transcripts and all raw data). USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 48 X. REPORTING REQUIREMENTS The findings from the evaluation will be presented in a draft report at a full briefing with USAID/Tanzania, GOT and Wajibika, and possibly at a follow-up meeting with key stakeholders. The format for the evaluation report is as follows:  Executive Summary: concisely state the most salient findings and recommendations (2 pp);  Table of Contents (1 pp);  Introduction: purpose, audience, and synopsis of task (1 pp);  Background: brief overview of Local Government Authorities in Tanzania, USAID’s System strengthening program strategies and priorities, brief description of the Wajibika project, purpose of the evaluation, relationship of the intervention to improved management of district health programs (2-3 pp);  Methodology: describe evaluation methods, including constraints and gaps (1-2 pp);  Findings/Conclusions/Recommendations: for each objective area (17-20 pp);  Issues: provide a list of key technical and/or administrative issues, if any (1-2 pp);  Future Directions: to include mid-project course corrections/modification for improved impact (2-3 pp);  References (including bibliographical documentation, meetings, interviews and focus group discussions);  Annexes: evaluation methods, schedules, interview lists and tables – should be succinct, pertinent and readable. The final version of the evaluation report will be submitted to USAID/Tanzania in hard copy as well as electronically. The report format should be restricted to Microsoft products and 12- point type should be used throughout the body of the report, with page margins 1” top/bottom and left/right. The report shall not exceed 30 pages, excluding references and annexes. After the debrief meeting, the evaluation team shall incorporate oral comments received from USAID and stakeholders. Within 2 weeks of receiving oral comments, the evaluation team leader will send 2nd draft to the Mission for final, written, comments. Within 2 weeks of receiving the final comments from the Mission, the Evaluation Team, through the Evaluation Team Leader, shall send the final report: 2 hard copies and a USB flash drive containing the report and all raw data. XI. ESTIMATED LEVEL OF EFFORT (LOE) The bidder will submit a proposed budget indicating team composition, CVs, salaries, international and regional travel, M&IE, in-country expenses and transportation, report printing and binding and other office supplies and logistical support services (i.e. laptop, battery pack, paper, communication costs) miscellaneous direct costs. The budget will be based on the following estimated level of effort: Offshore prep work International travel In-country working days Report finalization Team Leader/ Senior Evaluation Specialist 3 days 4 days 28 days 5 days USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 49 System Strengthening Technical Advisor 2 days 0 days 28 days 5 days TOTAL LOE 5 days 4 days 56 days 10 days ANNEX 1: Selected List of Background Materials  Wajibika Semi-Annual/Quarterly Reports  Project description  Rapid Assessment of Local Government Authorities in Iringa Region  Annual Workplans  Semi-annual Performance Report  Monitoring data  Baseline data – methods, instruments, results  USAID Evaluation Policy http://www.usaid.gov/evaluation/ ANNEX 2 – List of Regions Iringa – all districts (using former designation that included Njombe) Morogoro – all districts Dodoma – all districts Coast – all districts ANNEX 3 – Illustrative Questions Has Wajibika developed a model of support that provides sustainable strengthening of the councils? Is the mentor role a viable mechanism to help strengthen capacity in a direct, but unobtrusive way? Has the program made a contribution to better governance? Is the support from Wajibika to improve prioritization, budgeting, planning, appropriately pitched to the districts and PMORALG, and is it being effectively integrated into standard procedure at the council level? Are the supported councils ready to ensure that Comprehensive Council Health Plans are shaped according to the specific needs of the council, and then can one follow those plans through execution, monitoring, and accounting for results and accounting for financial inputs? Has the support from Wajibika influenced the achievement of the performance measures/indicators? Does the council have an improved approach to using Epicor and Planrep since the initiation of Wajibika in the region? Are these tools being used, and if not, why not? Are there other tools that have been introduced, e.g., the web-based tool to share experiences or the Wajibika help desk to help with troubleshoot inputs. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 50 In the planning process, have the districts begun to think about all resources (Global Fund, USG, Basket, etc.) as their available pool of resources to plan and budget a full picture of programs? Has the support provided to the councils by Wajibika helped to promote more effective prioritization of interventions and expenditures? Has the support provided to the councils by Wajibika resulted in a more participatory approach to developing the Comprehensive Council Health Plans? How? Who is involved now who may not have been involved before? Lower level facilities? Community-based groups? Local citizens? Has the Wajibika assistance from the district mentor helped to streamline the budgeting and reporting processes? Are there specific performance indicators that can capture the strengthened approaches that the Wajibika program has supported? Do the internal auditors have adequate knowledge/skills to increase accountability? To what extent are the auditors involved in the planning, execution, and monitoring of activities. Are they able to independently report audit issues to anyone outside of the council? Is the working relationship between Wajibika and PRO-RALG apparent, and has PMO-RALG and/or the regional administration become more involved in the districts performance as a results of the work with Wajibika? A question for PMORALG and the MOHSW: Have lessons learned from Wajibika’s involvement in councils of the four target regions (Iringa, Morogoro, Dodoma, and Coast) been integrated into ongoing practices for all districts (use example of internal auditors)? How well have the Wajibika interventions been integrated/adopted within the context of the LGRP? What are the lessons learned, challenges, and successes in getting the activity integrated at the local level? To what extent has Wajibika improved the approval rating of the Comprehensive Council Health Plans as submitted? How many were accepted without changes? Is there a participatory response set up when there are reductions to funds, etc? Have the districts accepted the role of the Wajibika mentor? Has this function helped them see the benefit of more and more highly skilled staff? What is Wajibika’s exit strategy? At what point does Wajibika declare “success?” Has this been discussed/agreed with the GoT? Does the plan for USAID/Forward, which could result in direct funding to the districts, change that thinking? USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 51 Has there been some competition that has developed between targeted LGAs that is deemed to be beneficial? Examples? Do Wajibika interact with ALAT (Association of Local Authorities of Tanzania)? If so, around what issues? [I vaguely recall that HPI were collaborating with ALAT. Do Wajibika interact with ALAT (Association of Local Authorities of Tanzania)? If so, around what issues? Is there a solid role for Wajibika to plan in any rollout of performance-based financing (i.e., in Coast)? USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 52 Annex B: Evaluation Tools Group Interviews and Individual Information Question Guide Overarching Issues & Questions Governance and ownership of Wajibika  Council Chair/RAS/DED (district executive director): to what extent do you consider Wajibika to be a useful project?  What changes have occurred in your district/region as a result of Wajibika’s presence? (probe: program planning and management, financial management, audit queries, CCHP)  Regional Secretariat:  How does the Regional Secretariat benefit from the Wajibika project?  Has the region become more involved in monitoring district performance as a result of Wajibika? How?  PMORALG/MOHSW: to what extent do you consider Wajibika to be a useful project?  What changes have occurred as a result of Wajibika’s presence? (probe: program planning and management, financial management, audit queries, CCHP) Rollout and expansion of Wajibika  Rollout to additional districts  Council Chair/RAS/DED: would you as RAS/DED recommend rolling out the Wajibika approach to other districts? Why?  What would be needed to rollout the Wajibika approach to other districts?  What modifications would you like to make to the project as/if the project is expanded to other sectors within districts?  PMO-RALG: would you recommend using the Wajibika approach to other districts? Why?  What would be needed to rollout the Wajibika approach to other districts?  What modifications would you like to make to the project as/if the project is expanded to other sectors within districts?  Expansion to other sectors within districts  Council Chair/RAS/DED: how do you see other sectors accepting and implementing the Wajibika approach?  Council Chair/RAS/DED: what modifications would you like to make to the project if the project is expanded to other sectors within districts?  PMO-RALG: how do you see other sectors accepting and implementing Wajibika and what is needed to engage other ministries?  PMO-RALG: what modifications would you like to make to the project as/if the project is expanded to other sectors within districts? Sustainability of Wajibika  Council Chair/RAS/DED: What are the issues around the sustainability of Wajibika?  For each of the issues, probe for solutions/approaches USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 53  PMO-RALG: What are the issues around the sustainability of Wajibika?  For each of the issues, probe for solutions/approaches  PMO-RALG/RAS: To what extent has Wajibika helped strengthen accountability within the LGAs (e.g., financial management, performance objectives, mentor performance)?  Are there strengths/gaps/ opportunities to bolster these (e.g., incorporation into employee performance objectives)?  Council Chair/RAS/DED/PMO-RALG: at the end of three years, what would you like to see as the major accomplishments?  Wajibika: How does Wajibika see sustainability and how do they propose to address that? what is Wajibika’s exit strategy? At what point does Wajibika declare “success?”  Has this been discussed/agreed with the GOT?  USAID: how does the plan for USAID/Forward, which could result in direct funding to the districts, influence the current and future Wajibika approach? Coordination & linkages to other projects  RAS/DED: what other annual planning, financial management, and procurement projects are operating in your council? Other programs strengthening the local governments?  What are the pros and cons of each?  How do these projects complement each other?  Probe: Tanzania social action fund (TSAF); local government development grant system (LGDGS); JICA  RAD/DED/ Wajibika: are there any other complementary US-funded projects?  If yes, how does Wajibika work with them?  Probe: SCMS, Human Resource for Health  Wajibika: how does the project support the CHMT to coordinate activities with other projects?  USAID: what can the USG do to ensure harmonization/linkages between complementary US-funded projects? Project Measurement  Wajibika/USAID: are there performance indicators that capture the strengthened approaches under Wajibika?  What is the project intending to achieve at the output/outcome level?  Examples: districts with functioning mentors, responses to audit queries, measuring improved accountability, evidence-based planning, inclusion of other resources in district-level planning through CCHPs, appropriate resource usage & disbursement, the level to which CCHPs are monitored, the extent to which the CCHPs include lower level planning, etc. Way forward for USAID  Evaluation team to compile: what are the best practices, lessons learned, and recommendations to USAID for the second half of the implementation of the Wajibika project and beyond? USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 54 Project Objective 1: To improve governance through strengthened programmatic and fiscal accountability in councils Expected Result 1.1: A menu of innovative and scalable interventions for promoting local government authorities’ capacity and practices to effectively manage and account for their district’s health, HIV/AIDS, and OVC services; and ensure programmatic and fiscal accountability developed. Expected Result 1.2: Pilot testing of council strengthening interventions in eight councils is completed and pilot test results are disseminated Expected Result 1.3: A minimum of 27 councils implement council strengthening practices (the interventions identified from the overall menu relevant to each council). Expected Result 1.4: Strengthened councils receive clean audits in the LGA Basket Funds audit Questions: Mentors  PMO-RALG/RAS/DED/Council Chair: Is the Wajibika use of mentors helpful? In what way?  Have the districts accepted the role of the Wajibika mentor? How?  Was the LGA involved in getting the mentors in place? How?  How have the mentors helped improve planning, budgeting, financial management, and reporting?  What modifications would you recommend for the mentoring system?  How should mentors be hired and used (within and between sectors)?  What needs to be done to ensure that mentoring is sustained within the Government system? CCHPs  MOHSW/DED/RAS/PMO-RALG:  To what extent has Wajibika helped the councils to more carefully follow the CCHP development guidelines? (e.g., based on burden of disease, etc).  To what extent has Wajibika resulted in better inclusion of HIV/AIDS & OVC data?  To what extent has Wajibika helped councils monitor these plans, accounting for results & financial inputs?  To what extent has the support from Wajibika influenced the achievement of the performance measures/indicators?  To what extent has the support from Wajibika resulted in a more participatory approach to developing and monitoring CCHPs and resource allocation?  Who is involved now who may not have been involved before (e.g., lower level facilities, community-based groups, private sector, village committees, local citizens, governance at the facility level/ boards)? USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 55  MOHSW/PMO-RALG: how does the quality of the Wajibika-supported CCHPs compare to other districts? Audits  Wajibika: to date, how many supported LGAs have received clean LGA Health Basket Funds audits as compared to previous years before Wajibika support?  RAS/DED: do the internal auditors have adequate knowledge/skills to perform their work in supporting the council as compared to previous years before Wajibika support?  To what extent are the auditors involved in the planning, execution, and monitoring of activities?  To what extent are the councils utilizing the internal audit?  To what extent do the RAS oversee the use of the internal audit? Project Objective 2: To strengthen support for decentralized management by PMORALG and MOHSW Expected Result 2.1: Strengthened councils demonstrate ability to prioritize and plan based on CCHP guidelines and Plan Rep2 so that the CCHPs reflect appropriate costs for HIV/AIDS prevention, care, treatment, OVC, reproductive health and family planning, and maternal and child health. Result 2.2 An inventory of external sources of resources is developed and included in planning and budgets, including private and faith-based facilities, and other donors and organizations. This will reflect ability to include all potential resources into plans for an effectively managed district Result 2.3: GOT is assisted in providing clear guidance to councils on roles and responsibilities under decentralization by devolution (D by D) Result 2.4: Existing incentive approaches in use by councils are inventoried and documented, information about the programs is disseminated, and those programs found to be feasible are advocated for implementation by other councils. Result 2.5: Technical Support is provided to partners implementing pay for performance programs in at least one region in the Wajibika Area Result 2.6: Assist PMORALG in IFMS training to ensure expansion of the use of these tools to Wajibika councils and beyond CCHP preparation  Wajibika: Preparation of CCHPs using CCHP guidelines and Plan-Rep2 & EIPCOR tools  To what extent has Wajibika influenced an improved approach to using Epicor and Planrep for CCHP design, monitoring, and evaluation?  If not, why not? USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 56  Are there other tools that have been introduced, e.g., the web-based tool to share experiences or the Wajibika help desk to help with troubleshoot inputs.  RAS/DED: To what extent has Wajibika influenced councils’ ability to account for and plan around all available resources (Global Fund, USG, Basket, private sector etc.)?  Have councils mapped all key partners to identify all sources of resources, and do these partners participate in the CCHP development and monitoring process?  Has Wajibika support helped promote more effective prioritization of interventions and expenditures? Support to the GOT in decentralization by devolution (D by D)  Wajibika: Has Wajibika documented incentive packages that are practiced in LGAs?  Has the project conducted advocacy to share these different packages for implementation by other councils?  PMO-RALG/Coast RAS/DED: What role has Wajibika played in supporting the design, implementation, monitoring and evaluation of pay for performance (P4P) in one district?  Is there a solid role for Wajibika to plan in any rollout of performance-based financing in other districts? Assistance to PMO-RALG to expand the use of IFMS tools to Wajibika councils and beyond  Wajibika: what sort of support is Wajibika providing to PMO-RALG in expanding the use of IFRMs tools (e.g., EPICOR), including the needed equipment?  Is there a solid role for Wajibika to plan in any rollout of IFRM tools in other districts? Project Objective 3: To develop an effective expansion plan with PMO-RALG to undertake the interventions and improved district performance and accountability to all districts Expected Result 3.1: Effective interventions identified, evaluated and adopted by the GOT with support from Wajibika Expected Result 3.2: Policies and regulations are implemented to accelerate national scale-up Expected Result 3.3: An inventory of local financial management and strategic planning consultants, companies and organizations developed, and used to provide technical assistance at the district level and to support the broader expansion of effective interventions used under this Task Order. Expected Result 3.4: Strengthened potential for a transition of USG programs to a long-term ―sustainability model,‖ whereby key planning, operating, monitoring, and financial reporting functions for health, HIV/AIDS prevention, care, and treatment, priority health programs, as well as OVC services, could transfer to the GOT. Acceleration of national scale-up through policy & regulations  PMO-RALG/RAS/Regional Secretariat/Wajibika: what policies and regulations has Wajibika identified that would need to be changed in order to facilitate implementation of interventions? USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 57  What steps has the project taken to lobby/advocate and negotiate to political and policy stakeholders to implement performance based approaches? Inventory of consultants and resources  Wajibika: what steps has the project taken to identify local financial management and strategic planning resources that could provide district-level assistance?  Has the project develop a database of qualified local firms and individuals who can provide technical support to councils upon request? GOT adoption of Wajibika approach  Wajibika: what steps has Wajibika taken halfway through the project to document lessons learned, success stories, and effective interventions in 27 councils?  Everyone: what steps has the project taken to strengthen a transition of the Wajibika approach to a long-term sustainability model?  What is needed for districts, regional health management teams, and PMO-RALG to buy-in to the Wajibika approach?  Has the project undertaken advocacy for Councils to integrate HIV/AIDS prevention, care, and treatment, priority health programs, as well as OVC interventions into MTEF?  Is there additional support Wajibika could provide to develop an effective expansion plan with PMO-RALG to undertake the interventions and improved district performance and accountability to all districts? USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 58 Annex C: Evaluation Calendar and Logistics Evaluation Team: Dr. Anna Nswilla, MOHSW Dr. Emanuel Malangalila, World Bank Mr. Gene Peuse, USAID Mr. Danny Dedeyan, USAID Ms. Mary Wieczynski Furnivall (coordinator), consultant The evaluation team will receive support from Ms. Mary Kasonka from Wajibika. Ms. Kasonka will support the team with the set up of meetings and the consulting calendar, the introduction of the consulting team to stakeholders and the start of meetings, and with tracking down any additional resources. Schedule (DRAFT) Sat. Dec. 3  Mary Furnivall flies to Dar es Salaam from Nairobi, spends night at Alexander’s Hotel Sun. Dec. 4  The evaluation team drives to Iringa departing o/a 7:00 am:  US Embassy vehicle: takes Gene, Emmanuel, and Mary F (Gene is point of contact)  Wajibika vehicle: takes Mary K and Anna (Mary K is point of contact)  The two vehicles will meet at Kibaha Njuweni petrol station (8:00am) Mon. Dec. 5 – Fri. Dec. 9  The evaluation team conducts interviews with identified councils in Iringa  The final selection of districts is TBD and should comprise of a mix of well-performing and teams who have had challenges to order to best assess best practices and lessons learned. Districts might include:  Monday: Iringa Regional Secretariat (courtesy visit); spend night in Iringa  Monday: Iringa Municipal Council; spend night in Iringa  Tuesday: Kilolo District Council (issues with mentor) ; spend night in Iringa  Wed: Mufindi District Council (strong); spend night in Njombe  Thursday: Njombe Town Council (strong); spend night in Iringa  Friday: Drive back to Dar The evaluation team (2 vehicles) returns to Dar es Salaam  7 – 8 pm each evening is spent in team debriefings Sat. Dec. 10  Rest Sun. Dec. 11 USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 59  Rest Mon. Dec. 12  Kibaha Town Council  Bagamoyo District Council  Anna conducts interviews with DMOs in Dodoma Tues. Dec. 13  9:00 – noon: The evaluation team meets the Wajibika team  2:00 – 5:00: Evaluation team meeting at World Bank  Anna conducts interviews with DMOs in Dodoma Wed. Dec. 14  9am – 5pm: The evaluation team reviews findings, conclusions, and recommendations, meeting at the World Bank (Emmanuel to arrange)  5pm: Mary F returns to Kenya Thur. Dec. 15  Free Fri. Dec. 16  9:00 – 11:00: Emmanuel, Anna, Gene & Danny review DMO interviews; email finding, conclusions, and recommendations to Mary F for compilation TBD  Timing in January for report construction and reviews  Presentation and briefing schedule Logistics Wajibika will manage the majority of logistics for the evaluation team while conducting fieldwork. This includes:  Organizing meetings with stakeholders  The provision of briefing materials and project information  The provision and use of one vehicle with a driver and one LCD projector  Hotel bookings and any other travel requirements  Introducing the evaluation team during meetings  The provision of any additional information and resources as required As field meetings are set up, Wajibika will make sure that, as much as possible, the following time is available to the team for daily feedback sessions (while in Iringa: 7-8pm). While in Iringa, ensuring the availability of a small hotel conference room or similar space for these debriefs would be ideal, together with a project-provided LCD projector. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 60 Evaluation Team Contact Information Dr. Anna Nswilla, MOHSW Coordinator, District Health Services Ministry of Health and Social Welfare Cell phone: +255 0754-293617 +255 0784-339525 answilla@yahoo.co.uk Dr. Emanuel Malangalila Consultant Cell phone: +255 0784 411 146 Mr. Gene Peuse HIV/AIDS, Private-Public Partnerships, and Senior Marketing Advisor USAID Cell phone: +255 0789 788103 gpeuse@usaid.gov Mr. Danny Dedeyan Democracy & Governance Officer, USAID Cell phone: +255 689151430 ddedeyan@usaid.gov Ms. Mary Wieczynski Furnivall Consultant Cell phone: +254 715 500 037 Mfurnivall@gmail.com Wajibika Team Contact Information Ms. Mary Kasonka Wajibika Cell phone: +255 757 576264 mkasonka@wajibika.org Conrad Mbuya Wajibika Cell phone: +255 715 334575 cmbuya@wajibika.org Peter Kilima Wajibika Cell phone: +255 784 620620 pkilima@wajibika.org USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 61 Annex D: Persons Contacted Monday, 5 December 2011 Interviews with Iringa Regional Administrative Team 1 Vincent M James Acting Regional Administrative Secretary 2 Nuhu Mwasumilwe Asst Admin Sec for Planning Coordination 3 Alloyce G Mawere RFO 4 Jonas C Kilime CSA 5 Sifael Kivamba Local Government Officer 6 Paulo Lehamed Legal Officer 7 Gillian Bukori Regional Internal Auditor 8 Tubone A.Kasabe Acting Chief Accountant Interviews with Iringa Municipal Council 1 Malala Paulo Acting Municipal Director 2 David Mpagama Municipal Health Officer 3 Jane Kiwone Acting District Nursing Officer 4 Martih Salila District AIDS Coordinator Anthony Kilale mentor 5 Paul Nanzi Acting Municpal Treasury 6 Juliitha Majenge Municipal Dental Officer (MDO) 7 Samson Manjale Supplies Officer 8 Nyenza Mathias AG: Municipal Agriculture Land Development Officer (MALDO) 9 Martin Ndaro Acting Municipal Human Resource 10 Gervas Ndaki Deputy Mayor 11 Robert Seme MSEO 12 Zaina Savga Municipal laboratory Technician 13 Lucy Mfafya VFC 14 Lucy Kimoi MUPO 15 Dr Nfabaye Acting Municipal Medical Officer of Health 16 Alfred Antony Pharmacist 17 Gasper Nsanye Municipal Social Welfare Officer 18 Luhamba MM Me 19 Mussa Nyamba MCCO 20 Prosper Msivala Acting MS 21 Sezaria Andrelo Acting MRCYCO 22 Vincent Kalinga Acting Municipal Education Officer 23 Antony Kilale Mentor, Iringa Municipal Council USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 62 Tuesday, 6 December 2011 Interviews with Kilolo District Council 1 Herman Msuha Acting District Executive Director 2 Eriko Kawanga Acting District Community Development Officer 3 Dr. Wilfred Rwehangure District Medical Officer 4 Irikaeli Nimbise DCT 5 Bruno Ndunguru DMHCO 6 Pelesi Majani District Internal Auditor 7 Joyce Thomas District Planning Officer 8 Bernard Makkuppe DTTS 9 Mickmess Mlwe District Medical Officer 10 Rhoba Kunzuenea District Reproductive and Child Health Coordinator 11 Michael Simwanza District Pharmacist Wednesday, 7 December 2011 Interviews with Mufindi Council 1 Peter Mawere Acting District Executive Director 2 Masalu Maaya District Treasury 3 Egid Nkwera Economist 4 Dinday Njile District Internal Auditor 5 Lisso Mkama Iternal Auditor 6 Joshua Ngombe Accountant 7 Steven Bavu Human Resource Officer 8 Dr Bazil Tware Acting District Medical Officer 9 Carisidaher Msore Deputy Pharmacist 10 Elisha Nyamara Social Welfare Officer 11 Emmanual Mnzava DEDO 12 Clement Sangale Supplies Officer 13 Peter Luhanda Supplies Officer 14 Moses Tawete DR CHCO 15 Bernard Mbumbumba District Health Officer 16 Samson Kihongou District LaboratoryTechnician ( CHMT member 17 Atupakisye Mwakifawamba Acting District Health Secretary 18 Andambike Kyomo Economist 19 Stephen Haule Acting District Nursing Officer (CHMT Member) 20 Erick Anhone Internal Auditor 21 Rehema Moshi Iternal Auditor Yessaya Mwakyalanda Mentor, Mufindi Council USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 63 Thursday 8th December 2011 Njombe Town Council 1 George L. Mkindo Council Director 2 Salehe A. Msanda Economist 3 Lameck D. Kinyunyu Land, Natural Resources and Environment 4 Mathias Mbafu Head of PMU 5 Yohanna P.Kalinga Community Development Officer 6 Msena N. Bina Human Resource Officer 7 Ruth A. Chilongola Town Nursing Officer 8 Daniel B. Mwasongwe Council HIV/AIDS Coordinator (CHAC) 9 Mwl. Stanford P.Kihombo Town Education Officer ( Primary Dept) 10 Alex Waziri Internal Auditor 11 Dr. Seraphina Sanyika CHMT Member 12 Dr. David Mtahindwa Town Health Secretary (THS) 13 Yusufu Kiduge Health Secretary 14 Maximillian Tabonwa Town Council Treasury (TCT) 15 Dr. James Likowa Town MedicalOfficer 16 Leo C.Chaula CHMT Member ( Town TB/Lep Coordinator 17 Mwl. Remigius Mkali Adult Education Officer 18 Lubongo Lukwange Ag Town Education Officer- Secondary Regional Secretariat COAST Monday 12th Decemberr 2011 1 Bernard M.N.Nzungu Regional Administrative Secretary 2 Dr Romilus Kahwili AG. RMO 3 Obadia Malima Internal Auditor KIBAHA TOWN COUNCIL Monday 12th Decemberr 2011 1 Jenifa C. Omulo Town Director 2 Hope S. Gwimile Town Education Officer 3 Suyaan Ngai Afia Kilimo/ Mifugo 4 Alice T.Msemwa Town Education Officer 5 Ndobusa J.K Council Committee Clerk 6 Marco J. Mangu Internal Auditor 7 Tertia W. Simba Human Resource Officer 8 Clement E. Mrango Supplies Officer 9 George L. Mkombo Town Planning and Environment Officer USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 64 BAGAMOYO DISTRICT COUNCIL Monday 12th Decemberr 2011 1 ShukuruHalfan Council Chairperson 2 Luca M. Muani DPLO 3 Angelo Albert AG DTR 4 Jackson Ndayisula Supplies Officer 5 AbdulKarim Kanyamala AG.District Internal Auditor 6 Hamisi Milassy Supplies Officer 7 Rumelius Bakirane Sen. Supplies Officer 8 Tumainiel John Kamnde Supplies Officer KIBAHA TOWN & BAGAMOYO DISTRICT COUNCILS Monday 12th Decemberr 2011 1 Dr. Kaniki DMO- Kibaha Town Council 2 Peter Mallya Accountant Kibaha Town Council 3 Hope Rutatina Environmental Health Kabaha Town Council 4 Gerge Hanta Pharmacist Kibaha Town Council 5 Emannuel Maiseli AG. Tuberculosis Lab. Technician 6 Faustina Kayombo Social Welfare Officer 7 Lumumba Amry Town Health Officer Kibaha TC 8 Bernadetha Lyimo District Economist 9 Mwinyi Kambi Salum District Tuberculosis and Leprosy Coordinator 10 Alex B.Mliga District Home Based Care Coordinator KTC 11 Sikujua Mturo AG. Reproductive and Child health Coordinator 12 Shedrack Max HMIS - coordinator - Bagamoyo 13 Bonaventure Sagamilwa District Health Secretary Bagamoyo 14 Bwijo A. Bwijo DMO- Bagamoyo 15 Mgalula LH Environmental Health Bagamoyo District Council 16 Dr A.P. Mwenda District dental officer 17 Joyce S.Nganzo AG istrict Health Officer 18 Abdi Mohammed District Cold Chain Operator 19 Laurent Kalindima District Laboratory Technologist 20 Dinah Atinda District Nursing Officer The Wajibika Team Peter Kilima Chief of Party Mary Kasonka Associate Director - Technical Strategic Planning Conrad Mbuya Technical Advisor, Transition Planning Abdul Kitulo CTA Finance Alfred Moshi F&A Manager Matiko Machonchonya IT Specialist James Kabagile IT Specialist Gideon Muganda M&E Officer USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 65 Annex E: References  Wajibika Project Description  Wajibika First Year Annual Workplan: October 2009 – September 2010  Wajibika Second Year Annual Workplan: October 2010 – September 2011  Wajibika Third Year Annual Workplan: October 2011 – September 2012  Wajibika Performance Monitoring Plan (October 2010)  Wajibika Fiscal Year 2011 Semi-Annual Report  Wajibika Fiscal Year 2011 Annual Report  Wajibika project. September 2011. Report of Wajibika Stakeholders’ Meetings Held in Iringa, Morogoro, Dodoma and Pwani. Bethesda, MD. Wajibika project, Abt Associates Inc.  Wajibika project. September 2011. Report of Wajibika Introductory Visits to Dodoma, Morogoro and Pwani Region Bethesda, MD. Wajibika project, Abt Associates Inc.  Wajibika project. October 2010. Short Report on Training of Council Internal Auditors Bethesda, MD. Wajibika project, Abt Associates Inc.  Mbuya, Conrad and Mary P. Kasonka. December 2010. Report of Follow-up Supervision After Training on Comprehensive Council Health Plans, December 13-17, 2010, Iringa Region. Bethesda, MD. Wajibika project, Abt Associates Inc.  Kilima, Peter, Conrad Mbuya, Mary P. Kasonka, Abdul Kitula, Victima and Gideon Muganda. July 2011. Report of Advocacy to Senior Council Staff and Councillors in Dodoma, Iringa, Morogoro and Pwani Regions July 6-29, 2011. Bethesda, MD. Wajibika project, Abt Associates Inc.  USAID Evaluation Policy http://www.usaid.gov/evaluation/ USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 66 Annex F: GOT’s Planning and Budgeting Cycle Key steps in Government’s Planning and Budgeting Cycle4 Step Activity Purpose Responsibility Timing 1 Preliminary preparation of Budgets and Work￾plans (as part of MTEF) prior to issuing of PBGs  Review of past year performance and the current mid-year budget  Identify activities; establish costs; prioritize activities; prepare MTEF document Department Heads lead managers in activity planning, working with budget officers October to December/ January confirm? 2 Plan and Budget Guidelines (PBG) prepared and issued  Forecast revenue, recurrent and development expenditure  Sets budget ceilings for institutions  Instructions for Institutions to prepare MTEF Budget Guidelines Committee (representatives from central ministries) Starts October and Guidelines issued in Dec/January 3 Institutions continue and finalise budget estimates and MTEFs  As in Step 1 above  Nb. Data inputted into key budgeting, planning and reporting software Institutions January to March 4 Institutional scrutiny of MTEF and submission to MoFEA  Budgets of departments reviewed  Consolidate Institutional budget (under ceilings)  Prepare budget MTEF memorandum Institutional Budget Committee January to March Submission by mid-March 5 Central Ministries scrutiny of MTEF and consolidation of National Plan and Budget  Consolidate Institutional MTEFs into National Plan and Budget  Ensure budgets conform to ceiling, strategic priorities etc MoFEA; PO-PSM (for personnel emoluments); PMO-RALG (for regions/ councils March to April 6 DP & Government Consultations (before submission to IMTC)  Confirm donor financial commitments in the coming National Budget. DPs; MoFEA, Sector Ministries March to April 7 Cabinet Secretariat and IMTC scrutiny of National Plan/ Budget and advice to Cabinet  Institutions may be asked to amend estimates and budget estimates Cabinet Secretariat and IMTC March to May 8 Cabinet approval/ printing  Review and discussion Cabinet March to May 9 National Budget brought down  National budget presented to Parliament, discussed, & passed Government and Parliament June 4 Source: Medium Term Strategic Planning and Budgeting Manual (updated November 2008). The Public Expenditure Review process has not been included in this table but is an important of the planning process. USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 67 Annex G: The Wajibika Project’s FY 2011 Annual Program Report Results The following are the indicators reported on in the Wajibika project’s FY 2011 Annual Program Report: Indicators Level of completion 1 Number of councils with signed MOUs with Wajibika (19/19) 100%. 2. Percent of Wajibika councils with Wajibika Mentors (25/27) 93% councils have mentors;(two mentors resigned) 3. Number of trainings designed/developed by Wajibika Four types of trainings were designed during this reporting period. The training of internal auditors, CCHP Preplanning Training for health facility in charges and CHMT Members, IPSAS training for accountants and Internal Auditors and EPICOR training for accountants. 4. Number of trainings Implemented All four types of trainings were implemented. 5. Number of meetings held with national-level stakeholders to discuss Wajibika interventions 12 Meetings were held with CAG, PMORALG, PMORALG-DPS, Epicor team, MOHSW and MOF The meetings are: 1. Meeting with PMO-RALG on Rolling out the Integrated Financial Management System (IFMS) 2. Meeting with PMO-RALG Deputy Permanent Secretary 3. Meeting with MOHSW Legal Advisor 4. Meeting with MOF Personal Assistant to the Permanent Secretary 5. Meeting with Director of Local Government Mr. Kahitwa and Director of Finance Ms Mchome 6. Meeting with Assistant Commissioner of Budgeting Mr. Mwilima of Ministry of Finance 7. Meeting with Permanent Secretary Deputy Permanent Secretary PMORALG – Mr. Kattanga and Mr. Maswi respectively 8. Meeting with Technical Director University Computing Centre Mr. G. Wilson USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 68 9. Meeting with CEO of Public Procurement Regulatory Authority (PPRA) Dr. R. Mlinga 10. Meeting with Controller and Auditor General – Mr. L. Utouh 11. Meeting with PMORALG Deputy Permanent Secretary 12. Meeting with EPICOR team 6. Wajibika councils that received a clean overall financial audit report (14/27) 51.8% of the Wajibika Councils received clean overall financial reports 7 Percent of Wajibika councils that received a clean Health Financial Basket audit report 12 out of 27 8 Percent of Wajibika councils that are using Epicor 9 out of 27 The following are the reported achieved benchmarks that were included in the Wajibika project’s FY 2011 Annual Program Report: Benchmark Status Disseminate rapid assessment results study findings. Completed in 19 scale-up councils. Conduct debriefing meeting with USAID Completed Conduct project related meetings with key stakeholders at the national level (semi-annually and as needed) Completed with PMORALG, MOF and MOHSW Conduct meetings with key stakeholders at regional and council levels to provide feedback on the menu of interventions and agree on the way forward. Completed Introduce web-based tool and helpdesk service in scale – up councils to share best practices and resources, highlight model councils and provide feedback. In progress, the Web-tool launching started in September Procure servers and desktops for scale up councils After change of plan of using a Central Server in PMORALG head office the need for procuring servers for each council has been obviated. The saving out of this will be used to procure relevant hardware for the councils. There are plans to procure N computers (where one computer acts as a USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 69 server and others as clients) Conduct supportive supervision and technical assistance from Wajibika head office to mentors and councils. In progress Mentors providing technical, management and administrative support In progress Identify or assist in preparation of relevant written resources and disseminate to 21 councils. In progress Document incentive packages that are practiced in LGAs In progress Conduct advocacy to share different packages that exist in LGAs In progress Provide technical support to partners in at least one region to design, implement , and monitor and evaluate P4P In progress in Pwani region. Support PMORALG in training of IFMS (Epicor) In progress. Support the development of council Strategic Plans. In progress Continue with Project Advocacy at National level to keep key stakeholders informed. In progress Support /advocate for the creation and functioning of a pre-budgeting resource coordination meeting among council level stakeholders or participate in meetings that are already planned In progress. Collect the various financial reports formats currently being used from councils. In progress Hire mentors for 19 scale up councils Done Registration of Internal Auditors to IIA Completed Training of Internal Auditors in all 19 scale-up councils. Completed Advocacy to councilors, council head of departments and audit committee members Completed Training of Accountants on IPSAS Done Recruitment of additional staff: M& E officer, Mentor coordinator, council mentors, EPICOR/IT Specialist and Driver Completed USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 70 Annex H: Other Donor Support to Local Government Authorities in Tanzania In addition to the $100 million Basic Health Services Project, approved on December 20, 2011 by the World Bank Board, there are three other World Bank financed projects that support capacity building of LGAs, namely the Agricultural Sector Development Project (ASDP), the Local Government Support Project (LGSP) and the Water Sector Support Project (WSSP). The following is a summary of these other three projects: World Bank-Financed LGA Support Projects Project Agricultural Sector Development Project (ASDP) Local Government Support Project (LGSP) Water Sector Support Project (WSSP) Closing Date 06/30/2013 06/30/2012 02/29/2012 Amount (incl. AF) $156.85 million $155.68 million $210.58 million Disbursed $84.01 million $123.77 million $99.82 million Components providing support to Local Government Authorities (LGA’s) Local Level Support to improve LGA capacity to plan, support and co￾ordinate agricultural services and improve the local regulatory environment for economic activity in agriculture, particularly local level taxation. Sub￾components include: (i) agricultural investment, (ii) agricultural services, and (iii) capacity building and reform. Local Government Capital Development Grant System supports (a) Council Development Grant (CDG), and (b) Capacity Building Grant (CBG). LGDG transfers will be non-sectoral, distributed on a formula basis to LGAs, who will invest in accordance with local needs. CBG provides resources to help build capacity to access/manage the CDG Scaling-up of Rural WSS: support LGAs in providing water and sanitation services by implementing District WSS Plans. Includes improvements in water supply (e.g., shallow wells, boreholes with hand pumps) and sanitation services (latrine promotion and hand washing) to rural communities, health centers, and schools. National Level Support to reform agricultural services, primarily research and extension; improve the sector policy framework; carry out preparatory work and investment in national level irrigation; simulate market development; and improve food security/ Management and Institutional Development supports PMO-RALG in the implementation, monitoring, evaluation and audits of the Project and the transfer program supported by Component 1 while building the capacity of Scaling-up of Urban WSS Services supports the execution of utility business plans and support improvements in water supply (mainly piped schemes) and sanitation services (sewerage systems, latrines promotion, hand washing USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 71 Project Agricultural Sector Development Project (ASDP) Local Government Support Project (LGSP) Water Sector Support Project (WSSP) sector co-ordination. Sub-components: (i) agricultural services; (ii) irrigation development (proposed Basket Fund; (iii) market and private sector development; (iv) food security; and (v) co-ordination, M&E. PMO-RALG to execute the functions as part of routine activities. program). In particular, services to the urban poor will be emphasized. Partners WB, EU, Japan, Irish Aid, IFAD, and DANIDA have in principle agreed to provide joint support to the program through the ASDP Basket Fund WB, Netherlands, Ireland, Denmark, Finland, Norway, Sweden, DFID, EU and UNDP/UNCDF WB, AfDB, USG, Germany, JICA, France, Netherlands, Switzerland, UN Habitat, GEF Basket Fund Yes Yes Yes LGDG linkage Yes Yes Yes USAID/Tanzania: The Wajibika Project Mid-Term Evaluation 72