Prepared under the Democracy and Governance Analytical Services Indefinite Quantity Contract No. AID-OAA-I-10-00004 Prepared for Task Order No. AID-522-TO-14-00004 under USAID Indefinite Quantity Contract for De￾mocracy and Governance Analytical Services No. AID-OAA-I-10-00004. MAY 2014 This publication was produced for review by the United States Agency for International Development. It was prepared by Democracy International, Inc. HONDURAS MID-TERM PERFORMANCE EVALUATION OF THE TRANSPARENT LOCAL GOVERNANCE AND IMPROVED SERVICE DELIVERY PROJECT (USAID/NEXOS) AND THE DECENTRALIZED ENABLING ENVIRONMENT PROJECT (USAID/DEE) FINAL REPORT DISCLAIMER This is an external evaluation. The views expressed in this document are the authors' and do not neces￾sarily reflect the views of the United States Agency for International Development or the United States Government. Submitted to: USAID/Honduras Prepared by: Yemile Mizrahi, Team Leader Kris Thorpe, Monitoring and Evaluations Expert David Carias, Local Expert Juan Carlos Valeriano, Research Assistant Contractor: Democracy International, Inc. 7600 Wisconsin Avenue, Suite 1010 Bethesda, MD 20814 Tel: 301-961-1660 Email: info@democracyinternational.com www.democracyinternational.com MID-TERM PERFORMANCE EVALUATION OF THE TRANSPARENT LOCAL GOVERNANCE AND IMPROVED SERVICE DELIVERY PROJECT (USAID/NEXOS) AND THE DECENTRALIZED ENABLING ENVIRONMENT PROJECT (USAID/DEE) FINAL REPORT MAY 2014 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS TABLE OF CONTENTS ACRONYMS .................................................................................................................................................................................................................................... I EXECUTIVE SUMMARY........................................................................................................................................................................III LESSONS LEARNED.............................................................................................................................................................................................................XIII INTRODUCTION....................................................................................................................................................................................................................... 1 METHODOLOGY...................................................................................................................................................................................................................... 2 LIMITATIONS AND APPROACHES FOR MINIMIZING BIAS ........................................................................................................................ 3 BACKGROUND & OVERVIEW ......................................................................................................................................................................................... 4 THE TRANSPARENT LOCAL GOVERNANCE AND IMPROVED SERVICE DELIVERY PROJECT (USAID/NEXOS) 7 FINDINGS AND CONCLUSIONS................................................................................................................................................................................... 9 RECOMMENDATIONS .......................................................................................................................................................................................................26 THE DECENTRALIZED ENABLING ENVIRONMENT PROJECT (USAID/DEE)..............................................................................29 FINDINGS AND CONCLUSIONS................................................................................................................................................................................30 RECOMMENDATIONS .......................................................................................................................................................................................................37 LESSONS LEARNED..............................................................................................................................................................................................................39 ANNEX A: SCOPE OF WORK AND EVALUATION QUESTIONS...................................................................................................... A-1 ANNEX B: KEY EVALUATION QUESTIONS (USAID/DEE PROJECT)................................................................................................ B-1 ANNEX C: KEY EVALUATION QUESTIONS (USAID/NEXOS PROJECT) ......................................................................................C-1 ANNEX D: LIST OF INTERVIEWEES........................................................................................................................................................................ D-1 ANNEX E: FOCUS GROUP QUESTIONS (USAID/NEXOS PROJECT)...............................................................................................E-1 ANNEX F: FOCUS GROUP QUESTIONS (USAID/DEE PROJECT) ....................................................................................................... F-1 ANNEX G: INDICATORS (USAID/NEXOS PROJECT)................................................................................................................................ G-1 ANNEX H: ANALYSIS OF USAID/NEXOS WATER AND HEALTH INDICES ............................................................................. H-1 ANNEX I: DATA VALIDATION EXERCISE (USAID/NEXOS PROJECT) ............................................................................................. I-1 ANNEX J: INDICATORS (USAID/DEE PROJECT) .............................................................................................................................................. J-1 ANNEX K: DATA VALIDATION EXERCISE (USAID/DEE PROJECT).................................................................................................. K-2 ANNEX L: DOCUMENTS REVIEWED...................................................................................................................................................................... L-1 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ACKNOWLEDGMENTS Democracy International (DI) would like to express its gratitude to USAID/Honduras for facilitating this evaluation project’s field work in Honduras. DI’s evaluation team also wants to express its gratitude to the USAID/NEXOS and USAID/DEE implementing partners in Tegucigalpa: Deloitte Consulting LLP and FOPRIDEH, a local NGO in Honduras. Both projects’ Chiefs of Party, Sigifredo Ramirez and Ronal Fiallos, opened their doors to the evaluation team, introduced their staff and shared information openly and unre￾strictedly. Without their valuable collaboration, this report could not have been written. i MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ACRONYMS AJAAM Asociaciones de Juntas de Agua y Alcantarillado Municipal AMHON Asociación de Municipios de Honduras ANAMMH Alianza Nacional de Mujeres Municipalistas de Honduras AO Assistance Objective CEDE Comisión Ejecutiva para la Descentralización COMAS Comisiones Municipales de Agua y Saneamiento DI Democracy International DRG Democracy, Human Rights and Governance DRL State Department Bureau of Democracy, Human Rights and Labor USAID/DEE Decentralization Enabling Environment Project ERSAPS Ente Regulador de los Servicios de Agua Potable y Saneamiento FOPRIDEH Federation of Nongovernmental Organizations for the Development of Honduras GOH Government of Honduras GPS Global Positioning Systems HIV Human Immuno-Deficiency Virus INE Instituto Nacional de Estadísticas IR Intermediate Result LAPOP Latin American Public Opinion Project MOH Ministry of Health NGO Non-governmental Organization OMM Oficina Muncipal de la Mujer PMP Performance Management Plan RFA Request for Acitvity SANAA Servicio Nacional de Acueductos y Alcantarillados SIEP Servicio Informativo Ecuménico y Popular TSC Tribunal Superior de Cuentas UCSL Unidades de Supervisión y Control Local UFIM Técnico Unidad Financiera Intermunicipal ULAT Proyecto Unidad Local de Apoyo Técnico para Salud UN United Nations ii MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS UNDP United Nations Development Program USAID United States Agency for International Development UTD Unidad Técnica de la Descentralización TSC Tribunal Superior de Cuentas iii MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS EXECUTIVE SUMMARY At the request of USAID/Honduras, Democracy International Inc., (DI) conducted a mid-term performance evaluation of two local governments and decentralization five-year projects: the USAID/NEXOS project implemented by Deloitte Consulting LLP and the Decentralization Enabling Environment (USAID/DEE) project implemented by the Federation of Nongovernmental Organizations for the Development of Hon￾duras (FOPRIDEH), a Honduran NGO. These two projects complement one another: the USAID/NEXOS project seeks to improve municipal governments’ capacity to provide public services and respond to citizen demands, while the USAID/DEE project seeks to promote broader public support for decentralization and a better enabling environment to promote further decentralization reforms. The goals of this mid-term evaluation were to: (1) learn if these two USAID local government projects are achieving their expected results so far; (2) assess the projects’ effectiveness, relevance, and sustainability; (3) identify the factors that contribute to or obstruct project results; (4) assess beneficiaries’ perceptions and satisfaction with project activities; and (5) provide specific recommendations to USAID on the design of future activities and directions in the implementation of these projects. The overall goal is to generate objec￾tive information to provide USAID with an opportunity to improve management, coordination, and imple￾mentation of its local government projects, thereby enhancing USAID’s ability to accomplish its goal of im￾proving accountability and transparency of local governments in Honduras. METHODOLOGY This mid-term performance evaluation was conducted by Yemile Mizrahi, Senior Governance Expert and Team Leader; Kris Thorpe, Monitoring and Evaluation Expert; David Carías, Local Expert; and Juan Carlos Valeriano, Research Assistant. Before the fieldwork in Honduras, the team conducted an extensive desk review of both projects’ documentation and scholarly articles relevant to decentralization, social service delivery, and local governments in Honduras. The team traveled to Honduras to conduct field work from April 27 to May 16. During this time, the team interviewed a total of 150 individuals through focus group discussion sessions and individual meetings. For the USAID/NEXOS project, the team traveled to Macuelizo and Nueva Frontera in the Mancomunidad (Municipal association) of MAVAQUI, and to Gracias, Belen and San Manuel Colohete in the Mancomuni￾dad of COLOSUCA. In each municipality, the team interviewed municipal and technical officials (both from the municipality and from the Mancomunidad). Additionally, the team conducted focus group discussions with USAID/NEXOS’ beneficiaries from different civil society organizations including water boards and asso￾ciation of water boards, municipal transparency commissions and community health councils—the so-called health promoters. For the USAID/DEE project, the team traveled to La Paz in the Mancomunidad of MAN￾SUCOPA, where it held three focus group discussion sessions with mayors, municipalities’ technical officials and CSOs from seven municipalities who have been beneficiaries of the USAID/DEE’s project. Additionally, the team interviewed former and current national government officials, former legislators with whom USAID/DEE had closely collaborated, leaders of national and regional non-governmental associations and representatives of other USAID funded projects. In addition to information collected through these meetings, the team was also tasked with validating data generated and/or provided by USAID/NEXOS and USAID/DEE to USAID. For the Deloitte project, the team replicated the survey conducted to health and water service providers, using the same survey instru￾ment used by USAID/NEXOS. Similarly, to assess the reliability of the financial data reported by USAID/NEXOS and USAID/DEE, the evaluation team contrasted data reported by USAID/NEXOS and USAID/DEE with official financial information reported by each municipality to the Tribunal Superior de Cuentas (TSC) as well as with information provided by the municipality’s financial staff to the evaluation team in every municipality visited in this evaluation. iv MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS THE TRANSPARENT LOCAL GOVERNANCE AND IMPROVED SER￾VICE DELIVERY PROJECT (USAID/NEXOS) The Transparent Local Government and Improved Service Delivery Project, “USAID/NEXOS” is a five-year, $18,999,985 project that seeks to increase citizen satisfaction with locally provided services. The public ser￾vices targeted in this project are primarily health and water, although other priority sectors like education and road maintenance were also considered for later years in the project’s life. The project works in 43 municipalities, primarily located in the Western region of Honduras. Most of the municipalities benefited by the project’s activities are in the C and D category. In addition to the activities undertaken by each one of the project’s components (sub-results), the USAID/NEXOS project has a small grants program and a fund for small infrastructure projects. This project is well articulated and the causal relationships between sub-results, results and overall objective are well developed. The results are realistic and within USAID’s manageable interests. The project is con￾sistent with local conditions, particularly the poor level of service delivery; low levels of citizens’ trust in their government and in democracy in general; the negative and highly uninformed citizens’ perception on decen￾tralization; and the relatively more favorable citizens’ opinion on local governments which offers an oppor￾tunity to reestablish trust in democracy and governance. USAID/NEXOS developed 16 indicators to monitor and evaluate its performance. Although USAID/NEXOS adequately distinguishes between impact, outcome and output indicators, most of the indi￾cators identified as outcome measures are outputs and not outcomes. Only one of these outcome indica￾tors (indicator n. 10) is adequately identified as an outcome indicator. This undermines the project’s ability to tell its story and explain its accomplishments, which are anecdotally apparent in the project’s narrative reports. FINDINGS AND CONCLUSIONS USAID/NEXOS OVERALL RESULT: LOCALLY-PROVIDED SERVICES IN RESPONSE TO CITIZEN NEEDS IMPROVED 1. Constraints Operating environment. The USAID/NEXOS project operates in a difficult, if not hostile, environment. De￾centralization is still poorly understood and much less supported by the public in Honduras. Leaders and unions at the ministries of health and education and the Servicio Autónomo Nacional de Acueductos y Alcantarillados, SANAA, the utility in charge of water management before it was decentralized and now responsible for providing technical assistance to decentralized water systems, actively oppose the decentrali￾zation of public services, claiming decentralization equates privatization and, therefore, an imminent increase in water tariffs. Negative incentives for tax collection. Municipal authorities have not faced sufficient incentives to rise own source revenue. Operating in a highly centralized political system, where municipalities had limited functions and where most of their income came from transfers from the central government, mayors avoided collect￾ing taxes and fees, fearing that would alienate their supporters during elections. This structure of incentives did not change significantly with the passage of the Law of Municipalities of 1990, which entitled municipal governments to collect property and sales taxes and other fees (such as livestock and exploitation of natural resources). Moreover, for electoral reasons, the central government approved certain provisions, such as v MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS debt forgiveness and tax and fee exemptions, that negatively affect municipal revenue. Tax amnesty weak￾ens tax collection levels and promotes perverse incentives for paying taxes. Clientelism and patronage. The culture of service delivery and accountability is weak in most Honduran municipalities. Clientelism and patronage continue to drive citizen participation; personal relationships with municipal authorities are critical in gaining access to goods and services. Services are still perceived as “fa￾vors” mayors distribute in exchange for loyalty. For the most part, citizens do not regard public services as a citizen right. Mayors still concentrate significant power in their hands even though municipal councils are supposed to represent diverse municipal interests and numerous citizen participation mechanisms are legally recognized to check and balance the power of the mayor. Weak civil society organizations. A project that seeks to increase citizen participation and especially to in￾crease citizens’ demand for better service provision requires a robust network of independent civil society organizations. In Honduras, however, civil society has remained extremely weak, poorly organized and highly dependent on local government authorities. The so-called “Comisiones Ciudadanas de Transparen￾cia,”which the Law of Municipalities recognizes as civil society entities responsible for oversight of local gov￾ernment affairs, are in the majority of cases, practically subordinated to the mayor. Outside these commis￾sions, there are no independent civil society organizations, and even fewer advocacy groups, particularly in poor and rural municipalities. Implementation delays. During the first year of the project’s operation, USAID/NEXOS had a slow start in setting up and staffing its office. In large part, this is due to the nature of doing business in Honduras, where permits and paperwork take a long time. But in part the delays were also due to cumbersome bureaucratic procedures the project had to complete—both for USAID and Deloitte headquarters—before it could begin operations. These delays negatively affected the implementation of some activities, particularly in mu￾nicipalities in the A and B category. USAID/NEXOS also experienced delays in the launching of its small grants and small infrastructure pro￾grams. The RFA to civil society organizations was recently released, and USAID/NEXOS is expected to begin reviewing applications by the end of May. Small infrastructure projects, on the other hand, have also experienced delays due to extensive bureaucratic procedures. All small infrastructure projects are designed by USAID/NEXOS and sent for approval to USAID. According to project’s staff, USAID takes between two to three months to approve a project. USAID however alleges that implementers do not follow USAID and GOH regulations in their proposals, a problem that causes delays in the approval process. Since the beginning of the project, USAID/NEXOS has prepared plans for 17 small infrastructure projects; 13 have been submitted to USAID, but only five have been approved. 2. Progress on Indicators According to its last semi-annual report, USAID/NEXOS shows good progress on most of its indicators. Out of USAID/NEXOS 16 indicators, five indicators show results beyond expectations, five are acceptable, three are below expectations, and one is at risk. Measurements for Indicators 1 and 2 are not yet available.1 1 Indicator 1 is based on an opinion survey that will be conducted at the end of the project. Data for Indicator 2 (percentage of own-source municipal income invested in locally provided services for municipalities A/B) was not collected in the semi-annual report. vi MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS In C/D municipalities, where the majority of project activities are being implemented, most indicator targets are being met. But in A/B municipalities, virtually all targets on the indicators are lagging behind. According to project’s staff, this is due to the delays in implementation referred to in the previous section. The two impact indicators related to the provision of water and health services, the average index of prima￾ry health decentralized services and the average water service integral index, show impressive results. How￾ever, the evaluation team found flaws in the methodology used to design the survey instruments and collect data for these indices. Problems related to recall bias, social desirability bias, possible selection bias, and lack of data verification question the validity and accuracy of the indices’ values. The USAID/NEXOS project was designed to focus primarily on health and water services. In fact, its impact indicators measure project’s results in relation to these two services. Yet the project has implemented activi￾ties in wider range of sectors, including roads, garbage collection, citizen security, youth, gender issues and the forums on the political transition before and after the electoral period. Although the number of activities implemented thus far is impressive, the thematic dispersion undermines the project’s ability to achieve bet￾ter and more sustainable results. Moreover, working in 43 different municipalities also disperses and frag￾ments project activities and resources. 3. Quality of Service Delivery According to recent available data, over the past 10 years social indicators have shown a modest improve￾ment in Honduras. Life expectancy at birth increased from 71 years in 2004 to 73 years in 2012; the per￾centage of the population living below the national poverty line fell from 64.7 in 2004 to 60 in 2010; and the percentage of rural population using an improved drinking water source increased from 74.6 in 2005 to 81.5 in 2012.2 Yet, despite progress in these aggregate numbers, many challenges remain. In the case of health, infant mortality rates are still high at 24 percent for urban populations and 33 percent for rural populations (data from 2006). The ratio of doctors per 10,000 inhabitants was 1:1 from 2005 to 2008.3 The maternal mortality ratio (per 100, 000 live births) is 120 compared to 68 at the regional level, and the incidences of malaria, HIV and tuberculosis are much higher than the regional average.4 The health care system is highly fragmented, with inequality in coverage across regions. Investments in health care infra￾structure have lagged behind; hospitals and health clinics lack sufficient medications, staff and medical equip￾ment to treat their patients. In the case of water, while water coverage has increased during the past10 years—95.7 percent of urban households and 48 percent of rural households are connected to water systems—the quality of water ser￾vice is still highly deficient due to obsolete and deteriorating infrastructure, lack of maintenance, inappropri￾ate chlorination of water and unsustainable tariff systems. Citizens interviewed in the five municipalities visit￾ed by the evaluation team complained about the poor quality of water and inconsistencies in service provi- 2 World Bank, World Development Indicators, Honduras. http://data.worldbank.org/country/honduras. The im￾proved drinking water source includes piped water on premises (piped household water connection located inside the user’s dwelling, plot or yard), and other improved drinking water sources (public taps or standpipes, tube wells or boreholes, protected dug wells, protected springs, and rainwater collection). 3 For an interesting analysis of inequality and poverty in Honduras, see UNDP, Informe Sobre Desarrollo Humano. Honduras 2011. Reducir la inequidad. Un desafío Impostergable. Honduras, March 2012. 4 2012 Data from the World Health Organization, http://www.who.int/gho/countries/hnd.pdf?ua=1 vii MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS sion, sometimes having access to water only a few hours during a week. Moreover, in some places, water sources are highly contaminated. 4. Coordination USAID/NEXOS works in a coordinated fashion with USAID’s health project ULAT (Unidad Local de Asis￾tencia Técnica). USAID/NEXOS and ULAT work together in improving health officials’ management skills at the local level. Currently they are developing a training curriculum for health officials. USAID/NEXOS does not work in a coordinated manner with the USAID/DEE project. Despite the pro￾jects’ shared objectives (strengthening the enabling environment for decentralization), the projects have not coordinated their activities and actually work in different regions of the country. Beyond holding a few in￾formative meetings, the two projects have not shared experiences or sought ways of supporting one anoth￾er. Finally, USAID/NEXOS has not coordinated with Honduran government agencies, donors and/or NGOs that work in areas relevant to the project.5 Most notably, USAID/NEXOS does not coordinate with ER￾SAPS (Ente Regulador de los Servicios de Agua Potable y Saneamiento), a regulatory agency that provides technical assistance to water service providers across the country. 5. Gender Consistent with USAID’s March 2012 Gender Equality and Female Empowerment Policy, gender equality and female empowerment are core development objectives, fundamental for the realization of human rights and key to effective and sustainable development outcomes. Gender equity and strengthened local govern￾ance are mutually reinforcing. The absence of women from local decision-making bodies, as well as custom￾ary restrictions that prevent women from participating in the public sphere, compounds and exacerbates gender inequality USAID/NEXOS has successfully mobilized women across its activities. This is no small achievement, particu￾larly considering the existing constraints for engaging women in civic activities in many municipalities. While USAID/NEXOS has spearheaded innovative initiatives to establish women’s offices and draft their agendas in several municipalities, the outcomes of this initiative remain to be seen. Many of these offices are only beginning to operate or are not yet in operation. Importantly, other than disaggregating indicators by males and female, USAID/NEXOS does not have gender-sensitive indicators to measure results of its efforts to mainstream gender issues into its activities. Discrimination, weak civil and political participation, and domestic abuse are common problems for women across the country. Yet, there are few women advocacy groups, particularly at the local level. While the municipal women offices (Oficina Municipal de la Mujer) attempt to deal with these and other problems, the absence of a well-articulated and organized demand for women rights may undermine the effectiveness and sustainability of these efforts. 5 According to official data from the Ministry of Finance, the government of Honduras has received USD $119.2 million for water and sanitation projects from the World Bank, the Inter-American Development Bank, the Banco Centroamericano de Intergración Económica and bilateral agreements with the government of Spain. Out of this total, $55.4 million remain undisbursed. For health services, the government received $275 million from the Inter￾American Development Bank, the Banco Centroamericano de Integración Económica and bilateral agreements with the governments of Korea, Austria and Italy. Out of this total, $156.8 million remain undisbursed. viii MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS RECOMMENDATIONS 1. Indicators and PMPs USAID/NEXOS should consider amending its Performance Management Plan to alter the articulation of its sub-result 2.3 and to include (or substitute) outcome indicators for Sub-results 2.1 and 2.3. As argued above, sub-result 2.3 is vaguely articulated and does not reflect the activities USAID/NEXOS is actually im￾plementing on the ground. Indicators for Sub-results 2.1 and 2.3 are all output indicators that do not ade￾quately measure USAID/NEXOS achievements. Based on the analysis and limited data validation exercise of the water service integral index and the primary health decentralized services index, the evaluation team recommends a fuller data audit to verify the accura￾cy of the data being collected. 2. Project’s Scope Consider concentrating the project’s geographic scope, limiting the number of municipalities benefitted by the program and focusing activities only in C/D municipalities. A/B municipalities are bigger, more complex and sophisticated and offer fewer opportunities to demonstrate significant results for a project like USAID/NEXOS, geared at improving decentralized service delivery at the local level. It would take a consid￾erable amount of resources (both human and financial) to bring about significant change in larger municipali￾ties. Avoiding too much geographic dispersion will allow USAID/NEXOS to invest more resources in fewer places, provide more continuous assistance to its beneficiaries and increase the likelihood of obtaining posi￾tive results. Avoid thematic dispersion by focusing on health and water service delivery. These are the most pressing needs of Honduras’ citizens. Although education, roads, citizen security are important sectors, USAID/NEXOS dilutes its potential impact if it spreads its resources too thinly across many different and unrelated activities. Small infrastructure projects should be more clearly linked to the overall project’s objective of improving decentralized health and water service delivery. Greater coordination between the small infrastructure pro￾jects and the projects’ three sub-results generates positive synergies to maximize project’s results. Consider reducing the total budget allocated for small grants and allocating the funds in small infrastructure projects. It is not clear USAID/NEXOS will be able to execute 1.3 million dollars in small grants in the time remaining in the project’s life. 3. Sustainability Increase follow-up after capacity-building and technical assistance activities. USAID/NEXOS should work with beneficiaries to draft follow-up plans with well-established goals, deliverables, benchmarks, and time￾lines. This would generate greater incentives for trained staff members to implement the skills they have learned and, eventually, change the way of doing things. Additionally, consider conditioning continued project support on the accomplishment of specific and meas￾urable results. Follow-up plans could be used to measure and reward (or not) beneficiaries’ performance. “Performance Based Incentives” can become a powerful mechanism to generate greater incentives for pub￾lic officials to improve their performance and become more accountable to citizens and to those who fund their capacity building efforts. Consider designing a sustainability plan for allowing Mancomunidades to charge a fee for services provided and therefore not be too dependent on donor support. Without independent sources of funding, it is un￾clear how many consultants providing technical assistance to municipalities (currently funded by USAID/NEXOS) will be able to survive once the project comes to an end. ix MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS Provide training and technical assistance to civil society organizations to build their capacity to write pro￾posals and apply for additional funding from government sources and/or other donors. Design more user friendly guides and manuals to institutionalize USAID/NEXOS’ capacity building efforts and leave a legacy after the project’s completion. This is particularly needed for financial technical staff in the municipalities who in most cases, only have thick and not user-friendly manuals left by the Spanish coopera￾tion and which typically are not consulted by public officials. 4. Coordination Conduct a mapping exercise of existing resources in Honduras for water and health services. Mapping will reveal gaps in financing and possible opportunities for leveraging resources in these sectors. Coordinate with existing government institutions, agencies, donors, and NGOs that are already investing resources in water and health and have existing technical expertise that could be leveraged to improve pro￾gram results. USAID could also match funds from different institutions towards this end. This would avoid duplication of efforts and create greater synergies among existing programs. 5. Targeting Beneficiaries Ensure beneficiaries have the capacity to use the equipment they receive. Tailor technical assistance to the specific needs and capacities of different stakeholders. For instance, illiterate people require different training methodologies and tools. New financial officials may require more basic training than more experienced officials. 6. Communications USAID/NEXOS’ consultants should be clear about the scope of the project and its limitations to avoid cre￾ating false expectations among beneficiaries. Promising to invest in “projects” that are unfeasible generates frustration. This can potentially undermine the project’s credibility. Identify successful experiences in the delivery of health care and water services. Systematize these experi￾ences, promote exchanges among stakeholders and disseminate the results. In coordination with USAID/DEE conduct a communications campaign to educate citizens and generate greater awareness on the benefits of decentralization, the importance of paying for services, and the health benefits of water chlorination. Project efforts will not be sustainable if citizens do not change traditional pat￾terns of behavior. THE DECENTRALIZED ENABLING ENVIRONMENT PROJECT (USAID/DEE) The Decentralization Enabling Environment program (USAID/DEE) started on February 22, 2011, and has a current expected completion date of February 22, 2016. With $2 million from USAID and $400,000 from FOPRIDEH for an overall investment of $2.4 million, the project works in [57] municipalities in seven Man￾comunidades: MANOFM, MANSUCOPA, MUNASBAR, MAVASEN-AMVAS, MANCURIS, MANLESIP, and MAMSA. USAID/ DEE seeks to promote the enabling environment necessary for the decentralization of government services to the local level in order to better respond to citizen needs. The program aims to strengthen the capacity of national institutions, local governments and civil society to promote, promulgate and enforce legislative reforms to decentralization. USAID/DEE’s results framework is not clearly articulated. The overall result is vaguely stated, and Sub-results 1 and 2 overlap and are not clearly defined. In fact, a comprehensive legal framework for municipal autono￾my includes municipal fiscal autonomy. Complicating matters further, the first two sub-results are not distinct x MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS from the overall objective, making the causal connection between results and sub-results difficult to estab￾lish. USAID/DEE has identified 10 indicators to measure project results. Although USAID/DEE does not classify indicators as outcome or output indicators, most of the indicators could be considered outcome indicators, but most of these indicators are poorly defined, not direct, and not adequate to measure project’s results. Indicators for Sub-results 1 and 2 overlap. This further reveals the overlap between the two sub-results. Finally, several indicators have unrealistic targets, which undermine program success. FINDINGS AND CONCLUSIONS USAID/DEE’S OVERALL RESULT: ENABLING ENVIRONMENT FOR DECENTRALIZA￾TION STRENGHTENED 1. Constraints Operative environment. Several external factors have hampered USAID/ DEE’s progress. While these con￾straints are outside of FOPRIDEH’s control, they nevertheless frustrated implementation and made it harder for the program to achieve success. As with any political process, decentralization engenders winners and losers—those that will benefit from increased power and authority at the local level and those that stand to lose ground during this process. Opponents of decentralization—proponents of the status quo—such as unions and central government ministries (e.g., health, education, finance, SANAA) remain strong in Hondu￾ras. In addition, there is widespread confusion surrounding decentralization and its benefits among civil society and the public at large. Many people are either completely unfamiliar with the concept or erroneously con￾flate it with privatization. In a focus group, for instance, several mayors told the evaluation team that their constituents do not have a high opinion of decentralization, largely owing to the fact that they don’t know what decentralization entails. Without a general conception of decentralization and its attendant benefits, let alone widespread support for decentralization, the program’s advocacy efforts will have limited success. Negative incentives for tax collection. Electoral processes also negatively affect the structure of incentives for decentralization efforts, particularly with regard to fiscal reforms. General elections were held in 2013; as a result, electoral consideration took precedence over fiduciary responsibilities and mayors standing for reelection were hesitant to raise taxes or collect back taxes. Moreover, the central government has ap￾proved certain provisions, such as debt forgiveness and tax and fee exemptions, that negatively affect munic￾ipal revenue. Tax amnesty weakens tax collection levels and promotes perverse incentives for paying taxes. Citizens form bad habits and wait for future amnesties. In a focus group in MANSUCOPA, mayors affirmed this sentiment and said that their citizens do not view paying taxes as a civic duty. There are delays in munic￾ipalities receiving transfers from the central government and delays in the update of municipal tax infor￾mation. Capacity gaps. Despite many capacity-building efforts, municipalities still face severe capacity gaps in keeping accurate, timely income tax records. Not only are there high levels of turnover among local government technical staff, but many are political appointees and not necessarily qualified to perform their job, particular￾ly in C and D municipalities. While organizational and human resource manuals with job positions with at￾tendant technical competences exist in the majority of municipalities, they are not frequently used. 2. Progress on Indicators As defined by the PMP, USAID/ DEE demonstrates poor indicator performance, with 70% of the project’s indicators falling below expectations as of December 2013. The project is “below expectations” on seven of xi MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 10 indicators, even if some are below the target by a small margin of less than 2 percent. (Indicators 1.1, 1.3, 2.1, 2.2, 2.3, 3.1, and 3.2). The project exhibits “excellent” performance in the number of individuals to have received USG-assisted training (F. Indicator 2). In 2013, 1,019 individuals received USG-assisted training, exceeding the established target of 250 by more than 400 percent. While this is an output indicator, it demonstrates that USAID/DEE is actively engaging beneficiaries and the sex-disaggregated data indicates equitable gender balance in pro￾gram activities. The program also demonstrates “acceptable” performance in the percentage of laws and reforms promoting decentralization drafted with USG assistance (F Indicator 1) and passed by the GOH (Indicator 1.2). However, as these laws have not been passed by the National Congress as of this writing, Indicator 1.2 should have been one (one of these regulations was an executive decree that did not have to be approved by Congress). 3. Coordination There are several USAID programs that are working on decentralization. USAID/NEXOS, as described above, overlaps with USAID/DEE. Both programs share the same AO: More Responsive Governance) and IR (Locally‐Provided Services in Response to Citizens Needs Improved). The programs also both seek to promote civil society advocacy and strengthen municipal finance. While FOPRIDEH has had high-level meet￾ings with Deloitte, the team found no evidence of substantive coordination.6 Another USAID program, the Local Technical Support Unit, or Unidad Local de Apoyo Técnico (USAID/ULAT), provides technical assistance and policy reform to the Ministry of Health (MOH) to decen￾tralize health services. While the programs could have worked together on the legal framework and policy advocacy, the evaluation team did not find evidence that USAID/DEE has coordinated with USAID/ULAT. FOPRIDEH is a grantee of the USAID program IMPACTOS, a civic advocacy and transparency promotion program. Yet, USAID/DEE did not coordinate or shared information and lessons learned with IMPACTOS in its work with civil society organizations. By not effectively coordinating with USAID/NEXOS, USAID/ULAT, USAID/IMPACTOS, FOPRIDEH missed opportunities to further broaden the constituency and political will for decentralization. Nevertheless, USAID/ DEE has worked in a highly coordinated fashion with several other stakeholders at the national and local levels, including the Municipal Association of Honduras, or Asociación de Municipios de Honduras (AMHON), the Decentralization Technical Unit, or Unidad Técnica de la Descentralización (UTD), the Secretary of the Interior and Population (SEIP), Secretary of Planning, Secretary of Finance, the Executive Commission for State Decentralization (CEDE), Superior Accounts Tribunal (TSC), UFIM, individ￾ual legislators, and congressional committees. 4. Gender Gender equity and strengthened local governance are mutually reinforcing. FOPRIDEH is to be commended for its gender-sensitive, inclusive approach that does not view women as merely victims or beneficiaries but as active participants in their own development. The program’s Gender Mainstreaming Strategy, approved by USAID in May 2013, evinces a holistic approach to commitment to gender equity that aims to leverage 6 The team did not see any matrixes referring to collaboration between these two projects. Furthermore, when the team asked both COP’s the question of collaboration, they acknowledged they did not really collaborate with one another, especially FOPRIDEH, whose COP admitted that meetings with USAID/NEXOS were “a waste of time.” See below. xii MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS the unique abilities of women and girls, anticipate potential obstacles to female involvement, and design program activities to promote equitable access. FOPRIDEH has also successfully incorporated women into their program, particularly given the barriers Honduran women face to participation in the public sphere. USAID/DEE has seen relatively equitable levels of participation by men and women in program activities. For instance, from July to December 2013, there were 426 participants under Results 2 and 3, of which 234 (55%) were women. Nevertheless, while these indicators that measure the existence of women’s participation, there is no indicator that measures the qual￾ity of this participation in program activities or its effect. Women’s participation in politics, however, is necessary but not sufficient for political empowerment. The presence of women in local government is important, but it does not reveal whether these women pro￾mote women’s issues. In La Paz, for example, all mayors—men and women—did not know of a single women’s group in their municipality. In addition, several respondents told the team that the head of the OMM, when it is filled, can be a political favor bestowed by the mayor, and in many cases are not true ad￾vocates for gender equity. In the civil society focus group, the head of a women’s group said that she feels “as if she is fighting for women’s rights alone.” RECOMMENDATIONS With approximately a year and a half remaining in FOPRIDEH’s present cooperative agreement, it would be counterproductive to drastically change the objectives of the program or reverse ongoing activities. Never￾theless, there are several adjustments that can be made to USAID/DEE’s present activities to address some of the concerns raised in this report. 1. Results Framework and PMP. To the extent feasible, FOPRIDEH should revise its PMP and redefine its results and indicators. USAID and FOPRIDEH should consider establishing more realistic targets for program indicators. Sub-results (particularly Sub-results 1 and 2) should also be renamed to re￾fer to advocacy, municipal financial autonomy, and civil society to improve clarity. While creating new outcome indicators might be impractical at this point, USAID/Honduras should ensure that fol￾low-on programs have a clearly defined results framework, with distinct outcome indicators for each result that are direct, measurable, timely, practical, and attributable. 2. Advocacy. USAID should encourage the GOH to convene the Foro Nacional de Descentralización to discuss the status of the different decentralization laws before the National Congress. The donor community should also encourage a dialogue between the new government authorities and sup￾porters of decentralization laws and come to a new agreement on a package of legislation that can be passed successfully by Congress. This dialogue could be moderated by neutral international ex￾perts who could help achieve consensus between opposing parties to arrive at a final version of the decentralization law that can be voted on. 3. Robust Public Awareness Campaign. USAID/DEE should greatly expand its public awareness efforts and a robust public education campaign should be established as part of follow-on programs. The Honduran public needs to become better informed about decentralization: what it is, its benefits, and citizens’ roles and responsibilities. Misconceptions and ignorance about decentralization not on￾ly stymie program efforts, but widespread popular support for decentralization laws in the legisla￾ture could foment political will and move the process forward. To reach out and engage a wider audience, FOPRIDEH should improving the definition of their key messages and their target audi￾ences and consider social media as a possible medium. xiii MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 4. Financial Autonomy. FOPRIDEH’s model of working through the Mancomunidades is good and should be continued in follow-on programming to promote greater sustainability of project efforts. The project, however, should consider working with Mancomunidades to design a financial sustain￾ability handover plan, such as charging a fee for services, to allow technical officials to continue providing training and technical assistance to municipal officials. In addition, USAID/Honduras should incorporate incentives to municipal authorities to improve their own-source revenue, such as condi￾tioning further assistance on the achievement of particular benchmarks, in future assistance. 5. Civil Society. Where possible, USAID/DEE should follow up on its trainings and technical assistance programs offered to civil society organizations to ensure that agreements reached and operational plans designed in these sessions are implemented. Moreover, subsequent programming should tailor capacity building efforts to beneficiaries’ particular needs and feature participatory follow-on meth￾ods, such as tailored mentoring and coaching approaches, which will be more conducive to sustain￾ability and partner ownership. Furthermore, civil society is weak and frequently subject to the [incli￾nation] of the presiding mayor, frustrating project efforts. FOPRIDEH should go beyond established local civil society organizations, like municipal transparency committees, to identify potential com￾munity leaders, such as patronatos, who could be more independent from government authorities. Since decentralization efforts are wholly dependent upon active citizen participation, future pro￾gramming in this area should strengthen the capacity of civil society organizations. As civil society in Honduras remains underdeveloped and as local resources remain constrained, public pressure on the central government to legislate change is marginal at best. Legislative and executive buy-in and cooperation will be necessary to pass a law on decentralization and civil society organizations must develop the capacity aggregate and vocalize demands. Public education (as described above) and civil society capacity-building initiatives are, therefore, key inputs for successful decentralization. 6. Strengthened Coordination. To avoid duplicating efforts, leverage limited resources, and promote institutional synergy, FOPRIDEH should work with other programs, including other USAID pro￾grams. USAID/DEE should collaborate closer with USAID/NEXOS to improve and standardize technical assistance packages provided to technical municipal officials and civil society organizations. Although these two projects have adopted different methodologies in their work with financial mu￾nicipal authorities, both programs work to strengthen municipal finance and should learn from one another and identify and exchange best practices. USAID/DEE should also coordinate with USAID/IMPACTOS on the design of social auditing strategies and tools. One particular area where additional coordination is necessary is advocacy. A broader constituency for decentralization is needed to effectively overcome lagging political will. By speaking with one voice, organizations would be better able to exert greater influence on the National Congress and advance decentralization. LESSONS LEARNED 1. Donor-funded projects can maximize their results when they avoid dispersion and fragmentation of activities and scope. In poor countries such as Honduras, where citizen needs are so high, donors face the temptation to address as many needs as possible in the largest number of places. Yet, “less is more” when it comes to achieving positive results and making a difference in a particular commu￾nity. Working more intensely and continuously in fewer places increase the potential of achieving xiv MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS positive results. These achievements yield to a greater sense of local ownership and can ensure more buy-in on the part of local authorities. 2. Years of donor-funded capacity-building initiatives—mainly training and technical assistance—have neither significantly transformed beneficiaries’ patterns of behavior nor reduced their dependency on international donors. In many places skills learned in training programs remain untapped, manuals unread and equipment unused. At a higher level, donor assistance has also not lead to significant reform; municipal governments remain extremely dependent on transfers from the central govern￾ment, efforts to increase own source revenue have not improved municipal financial autonomy and civil society organizations remain technically and organizationally too weak to demand greater ac￾countability and advocate better service provision. The quality and coverage of public services— education, health, water, housing, economic development, etc.—remain insufficient, and thus public services are unable to adequately satisfy citizen needs. For donors, this may be an opportunity to innovate and introduce more creative approaches to affect traditional patterns of behavior. That requires paying attention to the structure of incentives that drive behavior and designing interven￾tions that takes these incentives into account. For example, conditioning donor support on the ac￾complishment of certain goals and objective results, rewarding good performance and penalizing underperformance can increase beneficiaries’ motivation to change their ways of doing things and eventually develop a greater stake in reform. 3. Donor-funded projects should plan sustainability strategies during project implementation. Often, trainings and technical assistance initiatives are not adequately followed up; reader-friendly manuals and guides are not produced and beneficiaries lack the resources to implement many of the solu￾tions learned through donor support. In addition to providing more consistent follow-up of capacity building initiatives, donors should consider assisting beneficiaries to leverage additional resources and engaging local government authorities as options for ensuring greater sustainability. 4. Getting stakeholders with varied interests and disparate short- and long-term objectives to appreci￾ate their mutual interests and work together to support a common goal is an approach that is nei￾ther easy nor quick, but one that is essential in an increasingly fractious political environment. Coor￾dination can leverage limited resources and yield multiplier effects that will continue beyond the life of the program. Although institutional mechanisms exist to facilitate donor coordination—such as donor coordina￾tion tables—in practice donors tend not to coordinate activities and work in isolation from one an￾other. Coordination remains a challenge even within USAID-funded projects. USAID/NEXOS and USAID/DEE do not coordinate efforts even when coordination could benefit both projects. More important, USAID/DEE and USAID/IMPACTOS do not share information or lessons learned, even when FOPRIDEH manages both projects. Coordination can generate positive synergies and maxim￾ize project results, but it requires more than formal meetings where donors and implementers “talk” and “show” their projects. Coordination is not spontaneous; it requires effort and planning of practical and operative mechanisms to organize activities. 5. As women become empowered through the political process, they can use their voice to advocate across sectors, including health, education, agriculture, water and sanitation, security and violence prevention, and job creation. The continued and expanded attention to gender inequality and em￾powerment of women in the political process will lead to better outcomes not only for women but also for Honduras as a whole. Decentralization can provide women with an opportunity for political xv MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS empowerment, as barriers to entry—such as the ability to travel and spend time away from home, a large disposable income, a reasonable level of education, experience with political competition, and social and political connections—are lower at the local level. However, decentralization can al￾so reinforce structural inequalities and exacerbate discrimination against women. As such, it is im￾perative that USG-funded programs place appropriate emphasis on to ensure this does not occur. 6. Well-crafted outcome indicators are essential in measuring results and allowing implementers to “communicate their story.” Yet, often projects fail to design adequate outcome indicators and as a result, it becomes difficult to assess their achievements beyond anecdotal evidence. Crafting appro￾priate outcome indicators should be given a greater priority in project design. 1 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS INTRODUCTION 1. PURPOSE OF THE EVALUATION At the request of USAID/Honduras, Democracy International Inc., (DI) conducted a mid-term performance evaluation of two local government and decentralization five-year projects: the USAID/NEXOS project implemented by Deloitte Consulting LLP and the Decentralization Enabling Environment (USAID/DEE) project implemented by the Federation of Nongovernmental Organizations for the Development of Hon￾duras (FOPRIDEH), a Honduran NGO. These two projects complement one another; the USAID/NEXOS project seeks to improve municipal governments’ capacity to provide public services and respond to citizen demands, while the USAID/DEE project seeks to promote broader public support for decentralization and a better enabling environment to promote further decentralization reforms. The goals of this mid-term evaluation were to: (1) learn if these two USAID local government projects are achieving their expected results so far; (2) assess the projects’ effectiveness, relevance, and sustainability; (3) identify the factors that contribute to or obstruct project results; (4) assess beneficiaries’ perceptions and satisfaction with project activities; and (5) provide specific recommendations to USAID on the design of future activities and directions in the implementation of these projects. The overall goal is to generate objec￾tive information to provide USAID with an opportunity to improve management, coordination, and imple￾mentation of its local government projects, thereby enhancing USAID’s ability to accomplish its goal of im￾proving accountability and transparency of local governments in Honduras. Following USAID’s evaluation guidelines, the evaluation team analyzed the projects’ overall design, the causal linkages among objectives, results and sub-results, and the logic behind their theories of change. It then as￾sessed these two projects performance against project’s goals benchmarks, associated deliverables and out￾comes. For each project, the evaluation team developed findings, conclusions and recommendations. 2. KEY EVALUATION QUESTIONS The Honduras/USAID Mission provided a list of 13 questions in the Scope of Work for this evaluation. Please see Annex A. These are: 1. What are the constraints for achieving results of the project? How are these being addressed? How severe are these constraints in impacting future activities? 2. In what specific ways are the projects successfully building the capacity of the local governments to provide services? Where have these efforts failed? What has been the role of civil society organiza￾tions in this process? 3. In what specific ways are the projects achieving greater transparency and accountability of local governments? Where have these effort failed? 4. How are civil society organizations actively participating in local decision making processes? What approaches have been successful and why? 5. Describe the coordination with other USAID and /or Government of Honduras (GOH) projects. 6. In what specific ways are the projects creating an enabling and strengthened environment for de￾centralization? Where have these efforts failed? Is there overlap or complementarity between USAID/NEXOS and USAID/DEE or other USAID projects? 7. How is gender being addressed within these projects, especially in the area of gender equity? In what specific ways have these projects impacted gender equity issues? 8. Review USAID/NEXOS and USAID/DEE indicators and analyze which have the most significant da￾ta that adequately show project impact and results. 2 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 9. How might the projects be redirected to fill current gaps? What actions can be taken to improve the overall performance of the project? 10. How might these projects better coordinate with other projects? Identify potential opportunities to increase impact through USAID projects. 11. What do intended beneficiaries identify as the both positive and negative effects from the project activities? 12. Are communities improving service delivery? If so, how has this impacted their lives? 13. What do intended beneficiaries identify as potential sources of additional assistance within the scope of the projects? For presentation purposes, the evaluation team grouped these questions by result, sub-result and recom￾mendations. Please see Annex B and Annex C for the grouping of these questions for each project. METHODOLOGY This mid-term performance evaluation was conducted by Yemile Mizrahi, Senior Governance Expert and Team Leader, Kris Thorpe, Monitoring and Evaluation Expert; David Carías, Local Expert and Juan Carlos Valeriano, Research Assistant. Before the fieldwork in Honduras, the team conducted an extensive desk review of both projects’ documentation and scholarly articles relevant to decentralization, social service delivery, and local governments in Honduras. The team traveled to Honduras to conduct field work from April 27 to May 16. During this time, the team interviewed a total of 150 individuals through focus group discussion sessions and individual meetings. The team conducted qualitative, in-depth interviews with representatives from USAID/NEXOS, FOPRIDEH, USAID/Honduras, current and former central government officials, former legislators, mayors and other local political representatives, domestic and international nongovernmental organizations. To the greatest extent possible, the evaluation team interviewed a wide range of stakeholders who reflect diverse views and are representative with regard to organization, region, gender, socioeconomic status, and age, among other factors. Primarily conducted in Spanish, the interviews were generally guided, open-ended conversations, with topics and questions varying depending on the background and expertise of interviewees and demo￾graphic considerations, as appropriate. This flexible approach also allowed the team to explore unanticipat￾ed topics and adjust data collection accordingly. Annex D for a full list of people interviewed. For the USAID/NEXOS project, the team traveled to Macuelizo and Nueva Frontera in the Mancomunidad (commonwealth) of MAVAQUI, and to Gracias, Belen and San Manuel Colohete in the Mancomunidad of COLOSUCA. In each municipality, the team interviewed municipal and technical officials (both from the municipality and from the Mancomunidad). Additionally, the team conducted focus group discussions with USAID/NEXOS’ beneficiaries from different civil society organizations including water boards and associa￾tion of water boards, municipal transparency commissions and community health councils—the so-called health promoters. Focus group discussions were structured along four thematic areas: a) quality of health and water service delivery b) opportunities for citizen participation; c) responsiveness and transparency of municipal governments; and d) gender issues. Please see Annex E for the questionnaire used for the focus group discussion sessions. For the USAID/DEE project, the team traveled to La Paz in the Mancomunidad of MANSUCOPA, where it held three focus group discussion sessions with mayors, technical officials and CSOs from seven municipali￾ties who have been beneficiaries of the USAID/DEE’s project. Please see Annex F for the questionnaires used to structure these discussions. Additionally, the team interviewed former and current national gov- 3 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ernment officials, former legislators with whom USAID/DEE had closely collaborated, and leaders of national non-governmental associations. In addition to information collected through these meetings, the team was also tasked with validating data reported by USAID/NEXOS and USAID/DEE to USAID. For the Deloitte project, the team replicated the survey conducted to health and water service providers, using the same survey instrument used by USAID/NEXOS. This exercise was not designed to have statistical significance. Rather, it was a snapshot of the methodology used by USAID/NEXOS to collect data on their health and water service delivery indices. The overall goal of this exercise was to test the survey instrument and assess whether discrepancies existed between the data reported by USAID/NEXOS and the data collected by the team in the field. Before the field work, USAID and the evaluation team agreed that if the evaluation team detected major inconsisten￾cies, it could recommend a full data audit to verify the validity and reliability of these indices. Similarly, to assess the reliability of the financial data reported by USAID/NEXOS and USAID/DEE, the eval￾uation team contrasted data reported by USAID/NEXOS and USAID/DEE with official financial information reported by each municipality to the Tribunal Superior de Cuentas (TSC) as well as with information pro￾vided by the municipality’s financial staff to the evaluation team in every municipality visited in this evaluation (in the case of USAID/DEE, the evaluation team requested this information from the technical staff that came to the focus group session in La Paz). LIMITATIONS AND APPROACHES FOR MINIMIZING BIAS Evaluations are commonly subject to acquiescence bias, a type of cognitive bias where respondents answer questions in the way they think the interviewer wants them to rather than according to their true beliefs. Although it is impossible to determine the extent of this bias in the present evaluations, it is conceivable that respondents may have either been reluctant to criticize the programs during interviews and focus groups or had an incentive overstate the effectiveness of program activities, which would produce biased estimates of program impact. Focus group dynamics may have exacerbated this bias. For instance, during one focus group in Gracias, Maconumidad technical staff members joined the group halfway through. In several focus groups, the mayor’s presence may have affected how other participants responded, given the mayor’s con￾siderable power in the community. During one focus group with civil society leaders for example, when the team asked about corruption, the regidor (city council member) left the room and returned shortly with the vice-mayor. The team attempted to mitigate this bias by designing data collection instruments to elicit open, honest, and unbiased participation and avoiding leading questions. The team also repeatedly emphasized their neutrality, assured respondents of the anonymity of their answers, and confirmed findings by probing similar topics with various stakeholders to see whether findings converged and triangulating results. To the greatest extent possible, the team has utilized multiple data points to triangulate and substantiate partici￾pants’ responses. The timeframe of these evaluations also posed a significant limitation and required a narrow, focused ap￾proach and precluded more rigorous, quantitative evaluation techniques. Both programs work with more than 40 municipalities each throughout Honduras. Given the size of the team, the timeframe for the evalua￾tion, and budget considerations, the evaluation team was not able to visit all of these locations. Nonetheless, given the range of stakeholders consulted and the rigorous methods of qualitative and quantitative data 4 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS collection and analysis undertaken, the team is confident that this evaluation provides a well-researched, evidence-based evaluation of USAID/NEXOS and USAID/DEE’s performance and the appropriateness of the project’s activities. As the Rapid Appraisal techniques utilized in this evaluation rely on non-representative samples, the evalua￾tion team is not able to generalize results across the entire population. Due to the nonrandom nature of the sampling, the evaluation is subject to selection bias, with findings reflecting the feedback of stakeholders who may not are not fully representative. Nevertheless, by using a mixed-methods approach, the team can com￾pare data collected using one method to data collected using other methods. Triangulation minimizes bias and strengthens the validity of evaluation findings by acting as a check on the findings from any one method. Another frequent source of bias is co-intervention bias, which occurs when project beneficiaries are simul￾taneously receiving other (unaccounted for) interventions and it is difficult to determine additional marginal impact of program activities. The team was not in a position to fully isolate the impacts of USAID/NEXOS and USAID/DEE projects from local governance assistance provided by other international organizations and implementers, which were often involved in mutually reinforcing activities related to decentralization. BACKGROUND & OVERVIEW Honduran citizens continue to face entrenched governance problems that have negatively affected citizens’ support for and trust in democratic institutions. High rates of inequality and poverty, corruption, impunity, and weak public service delivery, coupled with an unparalleled increase in crime and violence, have left many citizens deeply disappointed with democracy. Honduras is still one of the poorest and most unequal countries in Latin America, with more than 60 per￾cent of the population living in poverty and 18 percent living in extreme poverty.7 Paradoxically, the coun￾try’s poor public service delivery is not directly related to inadequate or low levels of public funding. On the contrary, over the past few years the central government has greatly expanded its expenditures on general public services, but these have not shown an improved performance, particularly health and education. Ac￾cording to the World Bank, “Increased public expenditures have not delivered in terms of growth and im￾proved services…..limited improvements, especially in health and education outcomes despite high alloca￾tions to these sectors, suggest the need to improve the efficiency and quality of expenditures.”8 Weak management structures, inefficient internal controls and high levels of corruption explain much of this prob￾lem. Just as the evaluation team was conducting field work in Honduras, a major corruption scandal devel￾oped involving the former head of the Social Security Institute, who was charged with embezzlement and fraud.9 During his presidential campaign, President Juan Orlando Hernández promised to do “whatever he needed to do” to confront these challenges. Among the many pledges in his governance plan, the so-called “Vida Mejor” plan, he committed to continuing endorsing the decentralization process –which began with the passage of the Law of Municipalities in 1990 -- and to support the decentralization reforms that were still pending in Congress. In the still highly centralized Honduran government structure, where the main social 7 Human development report 8 World Bank, “Honduras Public Expenditure Review (2008-2012), p. 12. 9 El Herarldo, May 13, 2014. “Compraron 30 carros de lujo con dinero del Seguro Social de Honduras.” 5 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS services are still managed by the central government, devolving authority and financial resources to local governments was presented as a critical element in the government strategy to improve the quality and coverage of public services.10 Yet, upon taking office, the President seems to have changed his priorities, and as people interviewed for this evaluation reported, “placed the decentralization agenda on the backburner.” As many of his detractors fear, his promise to do whatever he needs to do to improve the security situation and alleviate poverty might include re-centralizing power in the hands of the presidency. Decentralization became, at least rhetorically, a core goal of former President Porfirio Lobo’s government (2010-2014). Coming to power after one of the most serious political crisis in the country, President Lobo believed that transferring resources to local governments was instrumental in reducing political polarization and ensuring local governments’ continued support for the new administration. In 2010, the government passed a legislative decree pledging to increase revenue transfers to the local level from 5 to 20 percent by 2017 and to 40 percent by 2038.11 Actual transfers to local governments, however, felt short of these tar￾gets. In 2010 and 2011, transfers to local government amounted to only 5 percent of total central revenue and in most cases, these transfers were delayed. Local governments continue to face problems with the transfer system which remains highly unpredictable. Continued shortfalls and delays of the budget transfers disrupt local governments’ capacity to plan and manage resources effectively. Not surprisingly, most of the debate on decentralization in Honduras has centered on the issue of fiscal transfers to local governments. Issues related to municipal government’s responsibility, accountability and institutional capacity have taken a secondary place. Although a few municipal governments have improved some services, particularly water and sanitation and in some cases primary health care, the overall quality of service remains low and the levels of investments in infrastructure inadequate to meet citizens’ demands. Indeed, most municipal governments spend their available resources on current expenditures and not on much needed infrastructure investments. Devolving resources and decision making authority to local governments who are better informed about local needs and demands is critical for improving service delivery. However, if municipal authorities lack technical and institutional capacities to manage complex services and if they lack sufficient incentives to im￾prove their performance in office, decentralization is unlikely to resolve the critical and enduring service delivery deficits in Honduras. These problems are even more daunting for poorer and smaller municipalities categorized as C and D by the Secretariat of the Interior.12 Despite more than 20 years of public debates and experience with decentralization reforms, decentraliza￾tion in Honduras still faces enormous opposition. Public opinion surveys demonstrate that an overwhelming 10 Municipal governments in Honduras have limited functions. They are responsible for potable water, solid waste management, regulations of public spaces, local infrastructure, including local roads and public lighting. Most mu￾nicipalities also spend resources on the maintenance on some schools and clinics, and often have to purchase medicines or equipment for their local clinics. World Bank Report, p. 35. 11 In 2009, the government passed a legislative decree 143-2009 which stipulated that central government should transfer 7 percent of total revenue to local governments by 2010, 8 percent in 2011, 9 percent in 2012, 10 percent in 2013 and 11 percent in 2014. Thereafter, the percentage was supposed to increase until it reached 40 percent by 2038. World Bank Report p.18. 12 Municipalities are categorized as A, B, C, and D depending on their human development index, levels of urbani￾zation, resource base, access to basic services and institutional capacity. Most municipalities in Honduras are in the C and D category. 6 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS majority of Honduran citizens still oppose decentralization and refuse to grant local governments more re￾sponsibilities and/or fiscal resources.13 In great part this is due to the absence of information on what de￾centralization entails and about its potential benefits. But this negative perception is also due to the strength and higher level of organization of decentralization opponents. Indeed leaders of public services ministries – health, education, infrastructure and public works—and public sector unions remain uncommitted to decen￾tralization because they fear losing power and authority if services are decentralized to the local level. Op￾ponents to decentralization have launched a negative campaign against the decentralization process, includ￾ing spreading fears that decentralization amounts to privatization of services and therefore, a way of reneg￾ing the government’s responsibilities on the poor to privilege the rich. Notwithstanding local governments’ many shortcomings and the negative opinion about decentralization in the country, Honduran citizens still regard their local governments as more representative and responsive to their needs than any central government institution. The last Latin American Public Opinion Project (LAPOP) demonstrates that citizens deeply distrust central government authorities. As a result, citizens are also losing support for democracy and democratic institutions. 14 A legacy of abuse of power, disregard for the law, corruption, impunity and a public security crisis all contribute to this negative perception. Local governments remain the most important repository of democratic legitimacy in Honduras. Improving local government performance therefore plays a critical role in shaping citizens’ attitudes towards govern￾ance, decentralization and democracy in the country. 13 According to the Latin American Public Opinion Project (LAPOP 2013), only 5.9% of Hondurans want to give a lot more responsibility to local governments in terms of public services. And 70% of Hondurans do not believe more taxes should be given to municipal governments. Latin American Public Opinion Project, “Political Culture in Hon￾duras and in the Americas.” Towards Equality of Opportunity. 2012.” USAID, 2013. See pp. 158-159 14 LAPOP, op.cit.2013. p.130. 7 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS THE TRANSPARENT LOCAL GOVERNANCE AND IMPROVED SERVICE DELIVERY PROJECT (USAID/NEXOS) RESULTS FRAMEWORK: OBJECTIVE, INTERMEDIATE RESULTS AND INDICATORS The Transparent Local Government and Improved Service Delivery Project, “USAID/NEXOS” is a five-year $18,999,985 project that seeks to increase citizen satisfaction with locally provided services. The public ser￾vices targeted in this project are primarily health and water, although other priority sectors like education and road maintenance were also considered for later years in the project’s life. The project works in 43 municipalities, primarily located in the Western region of Honduras. Most of the municipalities benefited by the project’s activities are in the C and D category. In addition to the activities 8 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS undertaken by each one of the project’s components (sub-results), the USAID/NEXOS project has a small grants program and a fund for small infrastructure projects. This project, which essentially reproduces Intermediate Result 2 of USAID/Honduras’ DG Strategic Frame￾work, is well articulated and the causal relationships between sub-results, results and overall objective are well developed. The results are realistic and within USAID’s manageable interests. The project is con￾sistent with local conditions, particularly the poor level of service delivery; low levels of citizens’ trust in their government and in democracy in general; the negative and highly uninformed citizens’ perception on decentralization; and the relatively more favora￾ble citizens’ opinion on local governments which offers an opportunity to reestablish trust in demo￾cratic governance. The project’s theory of change can be read as fol￾lows: If civil society advocacy for improved services in￾creases and If local institutions increase their capacity to provide decentralized public services, and If structures and systems to implement decentraliza￾tion reform are strengthened, then Local governments’ service delivery will improve and local governments will be more responsive to citi￾zens’ needs. This will contribute to increasing peo￾ple’s confidence in democracy. Sub-Intermediate Result 2.3 overlaps with USAID/DEE’s overall objective (see below), an area where the two projects are expected to collaborate. USAID/NEXOS developed 16 indicators to monitor, track and evaluate its performance. Although USAID/NEXOS adequately distinguishes between impact, outcome and output indicators, most of the indi￾cators identified as outcome measure outputs, not outcomes. Only one of these outcome indicators (indica￾tor n. 10) is adequately identified as an outcome indicator. Please see Annex G for a description and brief comments on USAID/NEXOS indicators. Aside from the highest levels impact indicators, which by definition cannot be directly attributable to the project, USAID/NEXOS has not developed more appropriate indicators to measure achievement of its results at a higher level (outcomes). This undermines the project’s ability to “tell its story” and explain its accomplishments, which although anecdotally, they are apparent in the project’s narrative reports. Indicators for Sub-result 2.3 are particularly inadequate to measure USAID/NEXOS results. In fact, USAID/NEXOS activities under this sub-result, which are mostly related to health service provision, do not correspond to the result as it is articulated, namely, strengthening systems and structures to implement re￾forms. USAID/NEXOS is not working on promoting new decentralization laws or regulatory frameworks. Rather, it is supporting local governments in the implementation of existing regulation, especially in relation to health care. As we explain below, it is paradoxical that USAID/NEXOS has the weakest indicators in a • Health. Although the health system is still central￾ized, the government has initiated a program to de-concentrate primary health centers in rural areas. In some cases, the Mancomunidad manag￾es the primary health centers in the different member municipalities; in other cases the Munic￾ipality directly manages the primary health cen￾ters. Funds for managing these centers are trans￾ferred by the central government, and are often delayed, as are most of the transferences to mu￾nicipalities. Funding is conditional on a quarterly evaluation conducted by the Ministry of Health. Health clinics are strictly evaluated against estab￾lished performance indicators. Clinics that obtain a lower than 85 percentage points in this evalua￾tion, are penalized with a withholding of a per￾centage of their transferences. • Water. The water system was decentralized to the municipalities in 2003 (Ley Marco del Sector Agua Potable y Saneamiento, Decreto Legislativo 119-2003). In the urban areas, most municipal governments are responsible for managing the water system, but in the rural areas, the service is managed by water boards, which are integrated by members of civil society on a voluntary base. 9 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS sub-result area where the project has introduced the most innovative and at least anecdotally, successful initiatives. FINDINGS AND CONCLUSIONS I. USAID/NEXOS OVERALL RESULT 2: LOCALLY PROVIDED SERVICES IN RE￾SPONSE TO CITIZEN NEEDS IMPROVED 1. Constraints Operating environment. The USAID/NEXOS project operates in a difficult, if not hostile, environment. De￾centralization is still poorly understood and much less supported by the public in Honduras. Leaders and unions at the Ministries of health, education and the Servicio Autónomo Nacional de Acueductos y Alcan￾tarillados, SANAA, the utility in charge of water management before it was decentralized and now respon￾sible for providing technical assistance to decentralized water systems, actively oppose the decentralization of public services.15 SANAA in particular has launched a campaign against decentralization of water services, spreading rumors that decentralization means privatization and rates will inevitably increase as a result. The culture of paying for services does not really exist in Honduras; citizens have been used to receiving services for free or paying a minimal flat fee for services they receive (water, trash collection, energy). Negative incentives for tax collection. The culture of non-payment extends to the municipal authorities as well. Operating in a highly centralized political system, where municipalities had limited functions and where most of their income came from transfers from the central government, mayors did not have sufficient in￾centives to collect and raise own sources of revenue. The structure of incentives did not change significantly with the passage of the Law of Municipalities of 1990, which entitled municipal governments to collect property and sales taxes and other fees (such as livestock and exploitation of natural resources). Moreover, the central government has approved certain provisions, such as debt forgiveness and tax and fee exemp￾tions, that negatively affect municipal revenue. Tax amnesty weakens tax collection levels and promotes perverse incentives for paying taxes. Undoubtedly, taxing people and charging fees for services is unpopular, especially for mayors seeking reelec￾tion. With few exceptions, and despite much technical assistance provided to municipalities by USAID and other donors, mayors have resisted efforts to expand their tax base and reduce their dependency on trans￾fers from the central government. Clientelism and patronage. The culture of service delivery and accountability is weak in most Honduran municipalities. Clientelism and patronage continue to drive citizen participation; personal relationships with municipal authorities are critical in gaining access to goods and services. Services are still perceived as “fa￾vors” mayors distribute in exchange for loyalty. For the most part, citizens do not regard public services as a citizen right. Mayors still concentrate significant power in their hands even though municipal councils are supposed to represent diverse municipal interests and numerous citizen participation mechanisms are legally recognized to check and balance the power of the mayor. Weak civil society organization. A project that seeks to increase citizen participation and especially to in￾crease citizens’ demand for better service provision requires a robust network of independent civil society organizations. In Honduras, however, civil society has remained extremely weak, poorly organized and highly 15 In some municipalities like Tegucigalpa, the water system is still centralized and managed by SANAA. 10 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS dependent on local government authorities. The so-called “Comisiones Ciudadanas de Transparen￾cia,”which the Law of Municipalities recognizes as civil society entities responsible for oversight of local gov￾ernment affairs, are, in the majority of cases, practically subordinated to the mayor.16 Outside these commis￾sions, there are no independent civil society organizations, and even fewer advocacy groups, particularly in poor and rural municipalities. As many other USAID projects, USAID/NEXOS has a mandate to mainstream gender issues into project activities. This becomes a challenging task when women are disempowered, remain subordinated to their husbands (or males in the household), and often are victims of domestic violence. In many rural and poor municipalities, women do not have decision making authority even in matters related to their own health. As the evaluation team heard in the field, in many of these municipalities, women do not see a doctor if their husband (or partner) does not authorize it. Despite the harsh situation faced by many women in these places, women rights groups or women advocacy organizations are weak or virtually inexistent. Implementation delays. During the first year of the project’s operation, USAID/NEXOS had a slow start in setting up and staffing its office. In large part, this is due to the nature of doing business in Honduras, where permits and paper work takes a long time. But in part the delays were also due to cumbersome bureaucrat￾ic procedures the project had to complete –both for USAID and Deloitte Consulting headquarters-- before it could begin operations. These delays negatively affected the implementation of some activities, particularly in municipalities in the A and B category. As we discuss in the next section, most targets on indicators rele￾vant to municipalities A and B were not achieved at the time of this evaluation. USAID/NEXOS also experienced delays in the launching of its small grants and small infrastructure pro￾grams. The total budgeted amount for small grants stands at $2.5 million dollars. Thus far, the project has only disbursed $800,000 on in-kind grants.17 The RFA to civil society organizations was recently released and USAID/NEXOS is expected to begin reviewing applications by the end of May. Disbursing $1.7 million dollars in small grants over the next two and a half years will be challenging, especially considering the bu￾reaucratic procedures civil society organizations need to complete before they can begin receiving funds. Small infrastructure projects, on the other hand, have also experienced delays due to extensive bureaucratic procedures. All small infrastructure projects are designed by USAID/NEXOS and sent for approval to USAID. According to project’s staff, USAID takes between two to three months to approve a project. USAID however alleges that implementers do not follow USAID and GOH regulations in their proposals, a problem that causes delays in the approval process. Since the beginning of the project, USAID/NEXOS has prepared plans for 17 small infrastructure projects; 13 have been submitted to USAID, but only 5 have been approved by USAID.18 2. Progress on Indicators According to its last semi-annual report, USAID/NEXOS shows good progress on most of its indicators. Out of USAID/NEXOS 16 indicators, five indicators show results beyond expectations, five are acceptable, 16 Ley de Municipalidades, 1990, Articulo 59-B. 17 Personal interview with Sigifredo Ramirez, NEXOS Chief of Party. May 15, 2014. 18 NEXOS semi-annual report, p. 65. 11 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS three are below expectations, and one is at risk. 19 Measurements for Indicators 1 and 2 are not yet availa￾ble.20 In C/D municipalities, where the majority of project activities are being implemented, most indicator targets are being accomplished. But in A/B municipalities, virtually all targets on the indicators are lagging behind. According to project’s staff, this is due to the delays in implementation referred to in the previous section. The two impact indicators related to the provision of water and health services, the average index of prima￾ry health decentralized services and the average water service integral index, show impressive results. How￾ever, the evaluation team found flaws in the methodology used to design the survey instruments and collect data for these indices. Problems related to recall bias, social desirability bias, possible selection bias and lack of data verification question the validity and accuracy of the indices’ values. Please see Annex H for a more thorough methodological discussion on these two indices. Although most mayors interviewed for this evaluation are aware of these indices and have appropriated their results, the story conveyed by these indices may be misleading. There seems to be a disconnect be￾tween the indices’ values and the reality of water and health service delivery found on the ground. For ex￾ample, in Belen, a small rural municipality in Lempira, the average value of the health index grew from 49.80 percent in 2012 to 66.80 percent in 2013. This is a big increase in just one year.21 Yet, doctors interviewed in Belen’s clinics complained they do not have adequate equipment and medications to treat their patients, some of whom actually die of tuberculosis.22 Similarly, in the case of the water index, the improvements in the total value of the index do not correspond with negative opinions about the quality of the water system in different municipalities, especially the frequent interruptions of water service, the absence or extremely erratic pattern of water chlorination, and the contamination of many water sources. In Nueva Frontera, for example, the water index shows an improvement from 52 percent in 2012 to 62.9 percent in 2003, but people interviewed by the team complained about the poor quality of water and deficiencies in service delivery. The USAID/NEXOS project was designed to focus primarily on health and water services. In fact, its impact indicators measure project’s results in relation to these two services. Yet, USAID/NEXOS has actually im￾plemented a wider range of activities in other sectors, including roads, garbage collection, citizen security, youth, gender issues and the organization of political transition forums before and after the electoral peri￾od.23 Although the number of activities implemented thus far is quite impressive, the thematic dispersion 19 NEXOS defines below expectations indicator values that are between 0-50% of their expected target; at risk, values between 50-75% of the expected target; acceptable, values between 75-100% and beyond expectations, values surpassing the target. See NEXOS semi-annual report, p. 9. 20 Indicator 1 is based on an opinion survey that will be conducted at the end of the project. Data for Indicator 2 (percentage of own source municipal income invested in locally provided services for municipalities A/B) was not collected in the semi-annual report. 21 The index’s technical and institutional variables obtained the highest scores -- 24.4 out of 30 for the institutional variable (81%) and 23.1 out of 30 for the technical variable (77%). The social variable obtained the lowest score of 19.3 out of 40 (48%). 22 When the evaluation team asked the mayor of Belen what he thought the value of the index meant, he con￾fessed he did not understand why the value of this index was so high and what it was actually indicating about the situation of health service delivery in his municipality. Interview with the mayor of Belén, May 7, 2014. 23 USAID/NEXOS semi-annual report, April 1-September 30, 2013. 12 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS undermines the project’s ability to achieve better and more sustainable results. Moreover, working in 43 different municipalities also disperses and fragments project activities and resources. It is difficult to maintain a more consistent presence and have a more sustained impact when resources are spread so thin in so many different municipalities. Indeed, in most municipalities visited by the evaluation team, beneficiaries complained of lack of more continuous presence of project’s staff and of inadequate follow-up. This was particularly the case for activities under sub-result 2.1 with civil society organizations. 3. Quality of Service Delivery According to recent available data, over the past ten years social indicators have shown a modest improve￾ment in Honduras. Life expectancy at birth increased from 71years in 2004 to 73 years in 2012; the per￾centage of the population living below the national poverty line fell from 64.7 in 2004 to 60 in 2010; and the percentage of rural population using an improved drinking water source increased from 74.6 in 2005 to 81.5 in 2012.24 Honduras’ UNDP Human Development Index, a composite index that assesses progress in three dimen￾sions of human development (health, education and poverty), also shows improvement from 1980 to 2012, from 0.45 to 0.63.25 Yet, despite progress in these aggregate numbers, many challenges remain. In the case of health, infant mortality rates are still high at 24 percent for urban populations and 33 percent for rural populations (data from 2006). The ratio of doctors per 10,000 inhabitants was 1:1 from 2005 to 2008.26 The maternal mortality ratio (per100, 000 live births) is 120 compared to 68 at the regional level, and the incidences of malaria, HIV and tuberculosis are much higher than the regional average.27 The health care system is highly fragmented, with inequality in coverage across regions. Investments in health care infra￾structure have lagged behind; hospitals and health clinics lack sufficient medications, staff and medical equip￾ment to treat their patients. In the case of water, while water coverage has increased during the past ten years –95.7 percent of urban households and 48 percent of rural households are connected to water systems-- the quality of water ser￾vice is still highly deficient due to obsolete and deteriorating infrastructure, lack of maintenance, inappropri￾ate chlorination of water and unsustainable tariff systems. Citizens interviewed in the five municipalities visit￾ed by the evaluation team complained about the poor quality of water and inconsistencies in service provi￾sion, sometimes having access to water only a few hours during a week. Moreover, in some places, water sources are highly contaminated. Water borne illnesses account for the second largest cause of infant mor￾tality in Honduras.28 24 World Bank, World Development Indicators, Honduras. http://data.worldbank.org/country/honduras. The im￾proved drinking water source includes piped water on premises (piped household water connection located inside the user’s dwelling, plot or yard), and other improved drinking water sources (public taps or standpipes, tube wells or boreholes, protected dug wells, protected springs, and rainwater collection). 25 http://hdr.undp.org/sites/default/files/Country-Profiles/HND.pdf 26 For an interesting analysis of inequality and poverty in Honduras, see UNDP, Informe Sobre Desarrollo Humano. Honduras 2011. Reducir la inequidad. Un desafío Impostergable. Honduras, March 2012. 27 2012 Data from the World Health Organization, http://www.who.int/gho/countries/hnd.pdf?ua=1 28 Data obtained from ERSAPS, http://www.ersaps.hn/documentos/interes/PAPSAC_Sintesis.pdf 13 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS Obviously, overcoming these structural problems is above and beyond USAID/NEXOS manageable inter￾ests. USAID/NEXOS’ activities can, however, have a positive effect on water and health care service provi￾sion at the local level.29 But the project has not developed appropriate indicators to measure results at this level. USAID/NEXOS indicators are either high level impact indicators (the water and health service indices and citizens’ satisfaction with health services) that are not directly attributable to the project’s activities, or low level output indicators that do not adequately measure the results (outcomes) of the project’s many different activities. 4. Coordination USAID/NEXOS works in a coordinated fashion with USAID’s health project ULAT (Unidad Local de Asis￾tencia Técnica). ULAT provides technical assistance and policy reform to the Ministry of Health (MOH) to decentralize health services and improve the national health system by working with the MOH to allocate funds for decentralization, assess the legal framework for the health sector, advocacy and negotiations on decentralization within the health sector, among other activities. USAID/NEXOS and ULAT work together in improving local health officials’ management skills. Currently both projects are developing a training cur￾riculum for health officials. According to ULAT’s Chief of Party, this is the first time that USAID’s health and governance projects work collaboratively and the results are highly satisfactory for both projects.30 Moreo￾ver, this collaboration has allowed the Ministry of health, ULAT’s main partner, to be exposed to a project that directly works with civil society organizations (community health councils) promoting preventive health measures at the community level. USAID/NEXOS does not work in a coordinated manner with the USAID/DEE project. Despite the pro￾jects’ shared objectives (strengthening the enabling environment for decentralization), the projects have not coordinated their activities and actually work in different regions of the country. Beyond holding a few in￾formative meetings, the two projects have not shared experiences or sought ways of supporting one anoth￾er. Finally, USAID/NEXOS has not coordinated with Honduran government agencies, donors and/or NGOs that work in areas relevant to the project.31 Most notably, USAID/NEXOS does not coordinate with ER￾SAPS (Ente Regulador de los Servicios de Agua Potable y Saneamiento), a regulatory agency that provides technical assistance to water service providers across the country.32 29 For example, NEXOS could measure water boards’ revenue collection, frequency of water chlorination, im￾provements in water infrastructure –such as water tanks, pumps or other small infrastructure projects. Similarly, it could measure number of patients attended at the clinics, number of people treated for common diseases, num￾ber of clinics that obtain 85 or more points on their quarterly evaluation by the Ministry of health, etc. These out￾come indicators could tell USAID/NEXOS story more appropriately and demonstrate how it is contributing to im￾proving decentralized service provision in response to citizens’ needs. 30 Interview with Dr. Juan de Dios Paredes, March 13, 2014. 31 According to official data from the Ministry of Finance, the government of Honduras has received USD $119.2 million for water and sanitation projects from the World Bank, the Inter-American Development Bank, the Banco Centroamericano de Intergración Económica and bilateral agreements with the government of Spain. Out of this total, $55.4 million remain undisbursed. For health services, the government received $275 million from the Inter￾American Development Bank, the Banco Centroamericano de Integración Económica and bilateral agreements with the governments of Korea, Austria and Italy. Out of this total, $156.8 million remain undisbursed. 32 In an interview with ERSAPS officials, they mentioned they had approached NEXOS, but they were told that “NEXOS could not establish any cooperative agreements with government agencies.” ERSAPS has developed man- 14 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 5. Gender Consistent with USAID’s March 2012 Gender Equality and Female Empowerment Policy, gender equality and female empowerment are core development objectives, fundamental for the realization of human rights and key to effective and sustainable development outcomes. Gender equity and strengthened local govern￾ance are mutually reinforcing. The absence of women from local decision-making bodies, as well as custom￾ary restrictions that prevent women from participating in the public sphere, compounds and exacerbates gender inequality USAID/NEXOS has successfully mobilized women across its activities. This is no small achievement, particu￾larly considering the existing constraints for engaging women in civic activities in many municipalities. While USAID/NEXOS has spearheaded innovative initiatives to establish women’s offices (Oficinas Munici￾pales de la Mujer, OMM) and draft their agendas in several municipalities, the outcomes of this initiative remain to be seen. Many of these offices are only beginning to operate or are not yet in operation. Im￾portantly, other than disaggregating indicators by males and female, USAID/NEXOS does not have gender￾sensitive indicators to measure results of its efforts to mainstream gender issues into its activities. Discrimination, weak civil and political participation, and domestic abuse are common problems for women across the country. Yet, there are few women advocacy groups, particularly at the local level. While the municipal women offices attempt to deal with these and other problems, the absence of a well-articulated and organized demand for women rights may undermine the effectiveness and sustainability of these efforts. II. SUB IR 2.1: CIVIL SOCIETY ADVOCACY FOR IMPROVED LOCAL SERVICES INCREASED 1. Transparency and Accountability USAID/NEXOS supports the Citizens’ Municipal Transparency Commissions (Comisiones Municipales de Transparencia), entities officially recognized by the Municipalities Law as civil society organizations responsi￾ble for overseeing and controlling local government authorities. USAID/NEXOS provides training and tech￾nical assistance to members of these commissions on advocacy and citizen participation strategies and methodologies for designing and implementing social auditing exercises. Most of these commissions, howev￾er, are subordinated to the mayor’s office and do not have the required independence to effectively over￾see and control local governments.33 In many poor and rural municipalities, members of these commissions uals and provides technical assistance to water service providers, but they do not have any funding to operate. ERSAPS also provides technical assistance for the establishment and operation of the citizen participation entities established by the law to oversee and control the provision of water services, the Comités Municipales de Agua y Saneamiento (COMAS) and the Unidades de Supervisión y Control Local (USCL). In the majority of municipalities these entities do not yet exist and/or operate effectively. ERSAPS entirely depends on donor support. Interviews with Ing. Luis Moncada, Ing. Hector Serra and Ing. Irma Escobar, April 28, 2014. 33 Municipal Transparency Commissions operate inside the municipality, where they are provided with office space. Members of this commission consider themselves as being part of the government structure; not part of civil society. 15 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS are often functionally illiterate. Despite the number of trainings received by USAID/NEXOS and other do￾nors, they still lack basic skills to conduct their oversight functions.34 In the water sector, USAID/NEXOS has supported individual water boards and associations of water boards (Asociaciones de Juntas Administradoras de Agua Municipales --AJAAM), assisting them in obtaining legal recognition (personería jurídica) and providing technical assistance and training to their members. Alt￾hough the Water and Sanitation Law (Ley Marco del Sector Agua Potable y Saneamiento) recognizes civil society entities responsible for the coordination of water service providers and oversight of water services, the Comités Municipales de Agua y Saneamiento (COMAS) and the Unidades de Supervisión y Control Local (USCL), in most municipalities these entities are either not yet established or they do not operate effectively. USAID/NEXOS has not worked directly with municipalities to assist them establish these entities. Water boards and association of water boards continue to operate independently from one another, often competing against each other for scarce resources. In the health sector, USAID/NEXOS has supported Community Health Councils in several municipalities (Consejos Comunitarios de Salud), civil society entities recognized by the Ministry of health as community organizations responsible for promoting and disseminating preventive health practices and encouraging citi￾zens to attend health clinics, particularly pregnant women. USAID/NEXOS has provided training to their members, and has particularly emphasized the importance of chlorinating water as a preventive health measure.35 USAID/NEXOS has also supported the establishment of women offices in the municipalities where it oper￾ates and has assisted in the design of their agendas. Although the evaluation team did not meet any inde￾pendent women networks in the municipalities it visited, USAID/NEXOS reports working with these net￾works in several places. Results of these efforts, however, are not yet evident. In many municipalities the women offices have just been established and they have not yet implemented the activities established in their agendas. Given the information provided by USAID/NEXOS in its Performance Management Plan (PMP), it is difficult to conclude that its efforts have resulted in improved transparency and accountability in local governments. USAID/NEXOS indicators for this sub-result do not adequately measure improvements in civil society ad￾vocacy or increased transparency and accountability of local governments.36 The only indicator identified by USAID/NEXOS as an outcome indicator (indicator n. 6) is essentially an output indicator that measures 34 In some cases, NEXOS has donated computer equipment to these commissions even when their members do not know how to use it. In Nueva Frontera, for example, computers and printers were still wrapped in their original plastic covers because members of the commission do not know how to use them. 35 According to several citizens interviewed for this evaluation, people do not like the taste of chlorinated water. They prefer boiling it even when the water is highly contaminated. Many people drink contaminated water from the tap alleging they have been drinking this water since they were children. 36 For example, some indicators of transparency and accountability could include: number of requests for infor￾mation attended by the government; frequency of publication of local government affairs; publication of budget information in formats and language that can be understood by citizens. 16 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS number of people attending meetings with local government authorities and not “participation in decision￾making” or “advocacy for improved service delivery.”37 Despite the many trainings and technical assistance efforts provided by USAID/NEXOS and other donors to these organizations, the results –increased transparency and accountability of local governments-- remain questionable. Anecdotal evidence suggests that transparency and accountability in local governments has not improved in municipalities where USAID/NEXOS operates. The virtual inexistence of independent civil society organizations, coupled with extremely low levels of education and lack of continuity and sustainability of capacity building efforts,38 particularly in poor and rural municipalities, has rendered many civil society organizations ineffective in exerting significant oversight of local government authorities. Local government’s authorities recognize the existence and legitimacy of civil society organizations and invite them to participate in open town hall meetings, but attendance to public events should not be confused with greater political participation and/or advocacy. There is no evidence that water boards, health councils, women groups or transparency commissions exert significant influence on government decision making or that local government authorities feel pressure from these and other civil society organizations to become more accountable for their actions. The single most important accountability mechanism at the local level still remains the municipal elections. In-between elections citizens do not have effective mechanisms to de￾mand greater transparency and accountability from their local governments officials. For example, social audits of public services have not resulted in any corrective action on the part of local government authori￾ties to improve public service delivery. Moreover, one of the social audits supported by USAID/NEXOS in the Mancomunidad of MAVAQUI, the social audit of health services, was invalidated by the Municipal Commissioner (Ombudsman) of Macuelizo because it was considered biased and methodologically flawed.39 Although USAID/NEXOS has installed software to track and monitor agreements reached between munici￾pal authorities and citizens in open town hall meetings (cabildos abiertos), consequences for lack of compli￾ance with these agreements remains unclear. Furthermore, the Municipal Secretary is the person responsible 37 Please see Annex G for a more elaborate discussion on indicators and some suggestions for improvement. 38 As discussed below, some beneficiaries complained that NEXOS did not adequately follow-up with them after they received training. In some cases, beneficiaries also said that NEXOS virtually disappeared from their munici￾pality after the first year. 39 The former head of the Municipal Transparency Commission in the municipality of Macuelizo worked for the Ministry of Health and as many of this Ministry’s unionized workers was an opponent of decentralization of health services. According to the current president of the Municipal Transparency Commission and the Municipal Com￾missioner of Macuelizo, this social audit exercise was biased and its findings could not be supported with evidence on the ground. As a result, the social audit was dismissed. In an interview with a consultant working for the Munic￾ipal Transparency Commission in the nearby city of Nueva Frontera, the problem with the social audit in Macuelizo was discussed and the alleged biases corroborated. According to him, this social audit was methodologically flawed; members of the transparency commission did not follow any of the established steps or procedures for conducting a social audit and its results were therefore invalid. In fact, the results of this audit were so poor and the report so badly written, that NEXOS’ staff had a hard time formatting the final version of this audit. Interviews with Benjamín Mondragón, President of the transparency commission in Macuelizo, and Nelson Silva, consultant for CASM in Nueva Frontera, May 1 and 2, 2014. 17 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS for maintaining this software and updating the information. Civil society organizations play no role in keeping track of these agreements or in accessing and verifying the information provided by the Municipal Secretary. It is not surprising that USAID/NEXOS reports that in municipalities where this software has been installed, 78 percent of agreements have been met completely and the remaining 22 percent are in the process of compliance.40 No cases were reported for non-compliance. 2. Citizen Participation Unlike other countries in the region, Honduras has established numerous mechanisms for civil society partic￾ipation, both at the local and central levels. Municipal Transparency Commissions, COMAS, USCL, and Community Health Councils are part of this thick institutional network of officially recognized civil society entities. Yet, for the most part, these entities “have been established in a highly corporatist fashion, giving the government an opportunity to control their agenda, and to undermine their effectiveness as independ￾ent observers and watchdogs of government activities.”41 In Honduras, most citizen organizations remain too weak to exert influence on public decision-making. Tra￾ditionally, citizens in small rural municipalities participate more in public events than citizens in larger urban municipalities. But participation remains characterized by clientelism and patronage. Citizens tend to partici￾pate to gain access to particular benefits distributed by public authorities in a highly discretionary manner. By far the most effective mechanism of citizen participation is the “community-based meeting,” the closer to the neighborhood level and most removed from the government’s corporatist controls. These meetings, held by the community leader (Patronato) in each community, are held to identify and prioritized communi￾ty-based needs. In municipalities where citizens “participate” in the preparation of the budget, municipal authorities aggregate the results of these community-based meetings to prepare the annual budget and identify its investment priorities in open town hall meetings. Obviously, not all the priorities identified by citizens at the community level are taken into account at the municipal level. The final budget and municipal investment plan are published and publicly disseminated. In some municipalities, USAID/NEXOS has assisted government authorities in organizing these participatory budget sessions. USAID/NEXOS has also assisted municipalities in organizing their open town hall meetings. In rural munici￾palities these meetings are well attended; in urban municipalities citizens participate much less. As mayors interviewed for this evaluation reported, although they have traditionally held open town hall meetings, the meetings were disorganized and quite chaotic.42 USAID/NEXOS has positively assisted municipalities in developing the agenda and structuring the discussion of these meetings. Yet, as noted before, it is unclear how much citizens actually participate in the decision making process during these sessions. USAID/NEXOS’ indicators (indicators 6, 7 and 9) do not measure citizen participation; they measure attendance to public meetings. But attendance is different from participation. 40 NEXOS semi-annual report, p. 37. 41 USAID DRG Assessment, p.27 42 Interview with Mayor Mario Cordón, Macuelizo. May 1, 2014. 18 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 3. Quality of Service Delivery As mentioned above, USAID/NEXOS’ impact indicators (the averages of the water and health indices) show improvements in health and water services in most municipalities. The improvements run across the indices three components: institutional strength, technical capacity and civil society participation. But in the case of the health index, the technical and institutional variables scored highest, while civil society participa￾tion lagged behind. In contrast, in the case of the water index, the civil society participation variable obtained the highest scores, while institutional strength lagged behind. Although USAID/NEXOS does not elaborate on these differences in its reports, the differences may be related to the fact that water boards –civil society organizations—are responsible for water service delivery in most rural places.43 In addition to the indices’ methodological problems discussed above, improvements or declines on the insti￾tutional, technical and/or civil society variables cannot be directly attributed to USAID/NEXOS, a fact duly recognized by USAID/NEXOS in its reports.44 Other than these high level impact indicators, USAID/NEXOS did not develop more appropriate indicators to measure improvements in service delivery as a result of its activities. As a governance project, USAID/NEXOS cannot be held responsible for resolving major challenges in the water and health service delivery systems. Most of USAID/NEXOS activities are related to strengthening institutional and managerial capacities of service providers. But if these activities are well designed, coordi￾nated and sustained, they could lead to some positive outcomes in service delivery. For example, in the water sector, water boards could chlorinate water effectively and more consistently, but that would require designing a sustainable system for purchasing and maintaining chlorine banks (so that the chlorination of water does not depend on whether water boards eventually receive chlorine from a donor or other gov￾ernment agencies) along with a strong communications campaign explaining the health benefits of chlorinat￾ing water. Small infrastructure projects or projects executed by the community (Proyectos Ejecutados por la Comunidad—PEC) such as repair and maintenance of water tanks and pipes, could be leveraged to im￾prove water quality in places where USAID/NEXOS operates. Similarly, in the health sector, USAID/NEXOS could have significant results in the improvement of decentralized health services. That would require consistent training and technical assistance to health managers, heads of primary health cen￾ters, and community health councils along with some basic infrastructure improvements. In some places, some of these outcomes are already apparent. In the case of Macuelizo, for example, in addition to enhanc￾ing the institutional and managerial capacities of professionals in the decentralized health system, USAID/NEXOS is also supporting the construction of a new health clinic from its pool of small infrastruc￾ture projects. This combination of efforts will likely have a positive impact in the quality of service delivered in this area. The mayor of Macuelizo was highly pleased with the USAID/NEXOS project and recognized that “people are motivated to attend capacity building events organized by the donor community and im￾plement their newly acquired skills because they see that a new clinic is being built. When people receive 43 In only a few cases, the service is still centralized and/or the municipality is directly responsible for water provi￾sion in rural areas. 44 Semi-Annual report, p.18. Coverage of water and health services, for example, may improve due to investments by other donors. 19 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS trainings and technical assistance but see no infrastructure projects, they get tired and eventually lose their interest in capacity building events.”45 In Honduras, bilateral, multilateral donors and international NGOs have invested significant resources in health, water and sanitation projects. USAID/NEXOS could coordinate with these organizations and lever￾age some of these resources to maximize project’s results. We develop these ideas further in the recom￾mendations section. 4. Perceptions of Beneficiaries Beneficiaries from different civil society organizations interviewed by the evaluation team –members of Mu￾nicipal Transparency Commissions, water boards, and community health councils-- expressed satisfaction with the training and technical assistance they received from the USAID/NEXOS project. Although some of the individuals interviewed acknowledged having received similar trainings from other donors in the past, they still valued their participation in USAID/NEXOS training events because “that gave them an opportuni￾ty to refresh their knowledge.”46 Yet, almost consistently, beneficiaries complained of lack of continuity of USAID/NEXOS activities in their municipality. As members of the water board in Macuelizo said “USAID/NEXOS trainings were excellent but the project abandoned us….USAID/NEXOS hired an engi￾neer who came to the municipality to work with us, but after one year, she left. No one from the USAID/NEXOS project has come ever since to follow-up with us or to provide additional technical assis￾tance.”47 In other municipalities, the team heard similar stories.48 When the team asked if USAID/NEXOS had left written guiding tools or manuals for water board members (or for members of the Municipal Transparency Commissions) the answer was negative. USAID/NEXOS assisted water boards and associations of water boards to obtain legal recognition (per￾sonería jurídica), but in the majority of cases the process is still pending and water boards members do not know the status of this process. It is unclear, however, what concrete benefits individual water boards would gain from obtaining legal recognition. A large number of water boards have been operating without formal legal recognition. Associations of water boards, much larger organizations, could significantly benefit from obtaining legal recognition because that would qualify them to apply for different sources funding. But indi￾vidual water boards are too small and for the most part, lack the technical skills to apply and manage inde￾pendent sources of funding. In virtually all the municipalities the evaluation team visited, members of the water boards expressed frustra￾tion with USAID/NEXOS for only focusing on technical assistance and trainings and not offering investments 45 Interview with Mr. Mario Cordón, mayor of Macuelizo, May 1, 2014. 46 Members of the Transparency Commission in Nueva Frontera. May 2, 2014. Members of transparency commis￾sions and water boards in different municipalities said they had received similar trainings as those offered by NEX￾OS from World Vision and from the Comisión de Acción Social Menonita (CASM). 47Members of the water boards in Macuelizo. May 1, 2014. 48 In Belen members of the water board said after NEXOS came to do a diagnostic of the water situation, no one has come back again. In San Manuel Colohete water board members also claimed they have not received technical assistance from NEXOS after the diagnostic study was conducted. 20 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS in infrastructure. They said USAID/NEXOS had promised them investments in “projects” but they only received trainings.49 This frustration underscores the high dependency of water boards and municipalities in general on donor support. As some members said in the interviews: “If we don’t get donor support, we cannot do anything here.”50 Yet, most water boards refuse to charge adequate tariffs for service delivery and remain financially unsustainable. Lack of municipal investment in infrastructure development has led to deteriorating water systems, health clinics, schools, roads, etc. Delayed, irregular and unpredictable transfers from the central government have undermined municipal governments’ capacity to invest in long-term and costly infrastructure projects, but municipal authorities have also lacked sufficient incentives to increase local sources of revenue. It is clear that years of donor-supported capacity building efforts have not lead to more sustainable solutions for local problems. Skills learned in different trainings are not well institutionalized or are simply not imple￾mented. Without sufficient incentives to alter traditional patterns of behavior, citizens and public officials end up dealing with the same problems over and over again and remain highly dependent on donor support. Although this is a long-term structural problem, USAID/NEXOS has not designed an innovative approach to deal with these issues. In large part the fragmentation and dispersion of project activities, undermines USAID/NEXOS ability to maintain a more sustained presence in the places it works, follow-up on its train￾ings and technical assistance more consistently, and supervise the implementation of new skills or solutions learned in training events. Furthermore, the volume of project activities has not allowed USAID/NEXOS to develop sustainability strategies including building capacities to search for new sources of funding, writing solid project proposals and leveraging additional sources of funding from government agencies or from oth￾er donors. III. SUB IR 2.2: LOCAL INSTITUTIONAL CAPACITY TO DELIVER DECENTRAL￾IZED SERVICES STRENGTHENED 1. Capacities of Local Governments Most of USAID/NEXOS activities under this sub-result are related to assisting municipal governments in￾crease their own-source of revenue by updating and expanding real estate cadaster, creating a taxpayers’ information system (Registro Unico de Contribuyentes, RUC); improving tax collection methods, and reduc￾ing tax arrears. Under this sub-result USAID/NEXOS also assisted municipalities in establishing women of￾fices and drafting their agendas; organizing and structuring open town hall meetings; and elaborating annual municipal investment plans (Planes de Inversión Municipal—PIM). With the exception of direct assistance to several municipalities in the provision of water services in urban areas and a solid waste disposal project in Nueva Frontera, USAID/NEXOS has not implemented activities to “strengthen local governments’ institutional capacity to deliver decentralized services.” Without a doubt, 49 When this subject was raised in an interview with NEXOS staff in Tegucigalpa, the COP said that many consult￾ants go to the field and promise things the project cannot provide. NEXOS is clear on what it can and cannot offer, but consultants are not always well informed. Furthermore, in many municipalities, beneficiaries do not always know how to distinguish between NEXOS and other USAID projects (such as ACCESS) that may have promised some infrastructure projects. 50 Interview with members of water boards and municipal officials in San Manuel de Colohete. May 5, 2014. 21 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS strengthening decentralized service delivery requires greater financial resources, but the linkage between increased municipal financial autonomy and improved service delivery is not automatic or self-evident. USAID/NEXOS identified two outcome and one output indicator for this sub-result, but only one indica￾tor–growth rate of own source revenue—can be appropriately classified as an outcome indicator.51 This indicator showed unsatisfactory results, falling 16 percent from October 2012 to September 2013.52 The other two indicators under this sub-result are output indicators measuring number of training and technical assistance sessions and/or number of beneficiaries. Although these two indicators had exceptional results, the outcome was still negative. The elections of 2013 adversely affected USAID/NEXOS’ results in the financial front. Introducing tax col￾lection initiatives during electoral cycles is politically risky for mayors, particularly those seeking reelection. This is a political constraint around the world, not only in Honduras. Given that Honduran mayors are elected for a four-year term, the last two years of the administration are typically not suitable for increasing tax collection efforts. One of the mayors expressed the situation candidly: “Electoral periods are not the most appropriate moment to increase taxes. I had to ask USAID/NEXOS to stop all they were doing with my financial team, especially updating the cadaster and launching tax audits to businesses that were either late on their tax payments or had filed lower tax reports. I have to admit that in this area, USAID/NEXOS did everything it could. We are the ones who fell short. Now, after the electoral cycle is over, we can begin afresh with these initiatives.”53 Although this is encouraging, local governments in Honduras have historically refrained from strengthening their financial autonomy by increasing tax collection efforts and charging appropriate fees for services. Local governments continue to rely on clientelism and patronage to win popular support; mayors avoid levying taxes for fear of alienating their supporters; citizens still believe public services should remain free of charge (water, trash collection, electricity). Despite many donor-funded programs supporting capacity building initia￾tives and equipment to increase local governments’ own source of revenue, local government authorities have not faced sufficient incentives to increase their financial autonomy. Their political will to reverse this legacy still remains in question. It is not clear this situation has changed significantly and therefore, that USAID/NEXOS results will be any different. In the five municipalities the evaluation team visited, the coverage and update of the real estate cadaster has not improved significantly since the municipal financial program funded by the Spanish cooperation conclud￾ed in 2010. In Macuelizo, for example, the GPS technology installed by the Spanish cooperation for cadastral surveying allowed the municipality to register and title 30 percent of its territory. The technology is no long￾er used and the cadaster coverage has not increased since 2010. USAID/NEXOS has assisted the cadaster 51 Importantly, NEXOS measures growth of own source revenue and not the proportion of own source revenue to total income, which is a much better indicator of financial autonomy. NEXOS has all the information to calculate this value if it chose to change this indicator. 52 NEXOS semi-annual report, April-September, 2013. 53 Mayor of Gracias, May 6, 2014. 22 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS office in updating real estate values, but these efforts have not been translated into increased revenue. In other municipalities, the evaluation team found similar results. USAID/NEXOS has introduced a good and innovative initiative to consolidate tax payers’ information. The Single Taxpayers Registry --Registro Unico de Contribuyentes, RUC— is a data base that allows municipal authorities update tax payers’ information and cross-reference their financial (fiscal) reports. The value of this instrument, however, is proportional to its use. In at least one case, the head of the office of tax collec￾tion said he did not see any value in the RUC; “in fact it only gives me additional work.”54 To reduce tax arrears, USAID/NEXOS has assisted municipal governments in preparing tax audits. Although USAID/NEXOS reports positive results from these initiatives (the recovery of more than 760,000 Lempiras in 9 municipalities), the overall result, growth of own source revenue, was not achieved. Moreover, at least in the municipalities visited by the evaluation team, the number of actual audits conducted was fewer than the number reported by USAID/NEXOS. 55 USAID/NEXOS has also provided technical assistance to municipalities in filing their accountability reports – the so-called declaración GL--to the Supreme Audit Institution (Tribunal Superior de Cuentas, TSC) and filing their financial reports to the Secretary of the Interior (Secretaría del Interior y Población--SEIP). Data on total municipal income reported to the central government should coincide with official municipal data and with the data reported by USAID/NEXOS, since they all come from the same data source. Yet, a small data validation exercise conducted by the evaluation team in five municipalities revealed discrepancies be￾tween data reported to the TSC and official data in municipal files. Please see Annex I for a graphic repre￾sentation of the results of the data validation exercise. It is clear that municipalities still face difficulties in keeping accurate, timely financial information records. The team did not hear a good explanation for this discrepancy. The evaluation team found no evidence that USAID/NEXOS has produced financial guides or manuals for municipal financial staff. In all five municipalities, when the evaluation team asked to see financial manuals or guides, the only available ones were those left by the Spanish cooperation. These guides are not user friend￾ly and the team saw no evidence they had been consulted. Although municipalities suffer from high staff turnover, it is clear that something more than capacity gaps or lack of adequate equipment is responsible for their financial weaknesses. USAID/NEXOS capacity building approach considers the Mancomunidad a strategic ally. This is an innova￾tive approach and potentially good for maximizing the results of the project. USAID/NEXOS provides train￾ing and technical assistance to the Mancomunidad financial officials (at the UFIM). These officials then go to the different municipalities to work directly with the municipality’s financial staff. Mancomunidad officials keep track of their training sessions and follow-up visits to each municipality in an official record (bitácora). But the record does not include specific benchmarks or goals municipal officials need to accomplish in- 54 Macuelizo, personal interview. May 1, 2014. 55 In the case of Gracias, for example, the Mayor acknowledged that after the first tax audit, he stopped this initia￾tive because he was seeking reelection and the tax audits were alienating many of his supporters. Yet in this mu￾nicipality NEXOS reported 7 audits. See semi-annual report, p.41. 23 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS between visits. Training and technical assistance are offered without sufficient attention to the results of these efforts. No indicators exist to measure results of capacity building initiatives. USAID/NEXOS only counts number of people trained, number of trainings offered, or number of municipalities assisted. Finally, although USAID/NEXOS approach to working with Mancomunidades is good for sustainability, the ability of Mancomunidad officials to keep providing training and technical assistance to municipalities after USAID/NEXOS concludes remains an open question. On the one hand, municipal officials not always wel￾come or appreciate the assistance the Mancomunidad has to offer. As one mayor said “mayors do not nec￾essarily like to have Mancomunidad’s technical staff on top of them.56” The same concern was voiced by one of the Mancomunidad’s technical staff: “municipal officials lack commitment; they have no incentives to apply what they learn or to improve their job performance.”57 On the other hand, the Mancomunidad does not charge any fees for the services it provides. Without donor support, it remains unclear how the Mancomunidad will find the resources to maintain its operations. 2. Perceptions of Beneficiaries For the most part, beneficiaries interviewed by the evaluation team expressed satisfaction with the quality of the training and technical assistance they received from the USAID/NEXOS project. Some individuals acknowledged they had received similar trainings from other donors in the past, particularly the Spanish cooperation. Paradoxically, these were also the individuals who showed less motivation to implement the skills learned in their different trainings. In municipalities where a new mayor is elected or where mayors changed parties when running for reelec￾tion, the entire technical staff is typically renewed. In the municipalities of Belen and Nueva Frontera where this situation was the case, the new staff did not have the adequate levels of skills to fulfill their responsibili￾ties. When the evaluation team asked a newly elected mayor why he did not keep some of the technical staff from the previous administration, he said he tried, but the staff did not want to stay. There is no ques￾tion that staff turnover negatively affects project results. In Gracias, a municipality ranked in the B category, municipal officials received training and technical assis￾tance together with officials from other municipalities in the Mancomunidad, which are poorer and more rural (C and D categories). They complained that the training they received was too basic for their skill level. Although the team did not hear a similar concern during its field work, the problem allegedly exists in other B municipalities—like Copán Ruinas and Carquín—which are grouped with C and D municipalities for train￾ing purposes. 56 Mayor of Belen, May 7, 2014. 57 Interview with Marcos Perdomo, Mancomunidad COLOSUCA, May 8, 2014. 24 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS IV. SUB IR 2.3: STRUCTURES AND SYSTEMS TO IMPLEMENT REFORM STRENGTHENED 1. Enabling Environment for Decentralization USAID/NEXOS activities under this sub-result are mostly related to increasing the managerial and institu￾tional capacities of decentralized health services. As written, however, this sub-result does not reflect what USAID/NEXOS is actually doing on the ground. “Strengthening structures and systems to implement re￾form” is a vaguely stated result. The outcome indicator for this sub-result, “number of decentralization con￾straints identified and addressed,” is even vaguer. The indicator is not direct, objective or adequate to meas￾ure project’s results, especially related to the delivery of decentralized health services. In addition to the index of health service, USAID/NEXOS does have one other impact indicator that measures citizens’ satisfaction with health services, but data for this indicator has only been collected once -- at the beginning of the project--when the baseline was established. Data will only be collected again at the end of the project. In interviews with project beneficiaries and citizens in the different municipalities visited by the evaluation team it became apparent that this is one of the areas where USAID/NEXOS has achieved its most impres￾sive results. Unfortunately, without more adequate outcome indicators to measure these achievements, USAID/NEXOS cannot adequately tell “its story,” including: the adoption of better financial and administra￾tive systems to manage health centers; improved information systems in health centers; use of new meth￾odologies to calculate infant mortality rates and analyze its causes; improved evaluations scores from the Ministry of Health’s quarterly evaluations to decentralized health systems; and at least anecdotally, improved citizen satisfaction with primary health care services. 58 Unlike any other public service in Honduras, the decentralized health system has generated good incentives for the improvement of service delivery. The Ministry of health has designed indicators to evaluate health clinics every quarter and withholds funding for underperforming centers. The quality of health clinics’ infor￾mation and administrative systems are included in this evaluation. USAID/NEXOS has used this evaluation system appropriately motivating health officials to improve their management and information systems. Notwithstanding the project’s many efforts, Honduras still faces many challenges in its decentralized health care system. Inadequate infrastructure, dearth of medications, insufficient personnel, lack of equipment, and scarce financial resources are just some of the most obvious problems. The needs are so extensive that it would be difficult for any donor program to demonstrate significant progress in overhauling these historic weaknesses. Yet in targeted locations, some improvements can be realistically expected if programs are well coordinated, sustained and effectively implemented. USAID/NEXOS has made significant efforts to coordinate its civil society activities with its health service delivery program (Sub-components 2.1 and 2.3). Community health promoters working in community- 58 Virtually everywhere where the health care system has been decentralized, citizens expressed being more satis￾fied with the quality of health care and said the system improved since it was decentralized. These opinions were heard frequently but they are anecdotal and do not necessarily represent a generalized opinion. 25 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS based health councils play an important and complimentary role in promoting use of health services in the community and disseminating preventive practices. USAID/NEXOS has also aligned most of its small infra￾structure projects and some of its small grants programs with overall sub-result’s objectives. Of the total 17 small infrastructure projects prepared by USAID/NEXOS, seven are related to the health sector—only four are related to the water sector, and the rest to projects that have no relationship with either of these two sectors, like the electrification of a soccer field and the remodeling of schools. In the Mancomunidad of COLOSUCA, USAID/NEXOS donated much needed laboratory equipment, its largest in-kind grant. The equipment, however, is still in storage waiting for the refurbishment of the health clinic in San Manuel Colo￾hete.59 USAID/NEXOS’ model of working with the Mancomunidad promotes greater sustainability of project ef￾forts. Officials responsible for decentralized health services at the Mancomunidad receive training and tech￾nical assistance from USAID/NEXOS; they then provide technical assistance to health clinic administrators in each municipality. USAID/NEXOS has also produced manuals and guides for trained staff. Using project data it is impossible to conclude if the environment for decentralization has improved as a result of USAID/NEXOS activities. Without a doubt, USAID/NEXOS has made significant progress in im￾proving the technical and managerial capacities of decentralized health administrators. But that in itself does not necessarily lead to a more enabling environment for decentralization, one where new laws and regula￾tory frameworks promoting decentralization are more likely to be passed by the government. In fact, as we argued in the introduction, since the new president took office earlier this year, decentralization seems to have been moved out of the government’s top agenda priorities. Anecdotal evidence heard in different interviews suggests that the enabling environment for decentralization in the country has actually deteriorat￾ed, as the president is attempting to concentrate power in the executive branch. Improving the quality and coverage of decentralized services could become a powerful and effective argument in favor of decentraliza￾tion, above and beyond the promotion of new decentralization laws or regulatory frameworks. 2. Beneficiaries’ Perceptions USAID/NEXOS technical staff has traveled to different project locations to provide technical assistance to health officials. Health officials interviewed by the evaluation team were highly satisfied with the quality of training and technical assistance they received from USAID/NEXOS. They recognized the direct involve￾ment of USAID/NEXOS staff and appreciated the project’s consistent engagement. USAID/NEXOS has produced manuals and guides for the operation of decentralized health systems. One of the most innovative manuals is the “operations guide” for managing decentralized health care agreements signed with the Ministry of health. The guide identifies the purpose and objectives of the agreement, ex￾plains the obligations and responsibilities of all parties involved, and describes the process for managing and 59 In an interview with the evaluation team, the Mancomunidad’s health care coordinator said that while the la￾boratory equipment is highly needed, the most urgent needs are in the maternity clinic, where the infrastructure needs substantial repairs and they lack necessary equipment like an ultrasound machine. Interview with Katherine Simon, May 5, 2014. 26 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS implementing the agreement. According to USAID/NEXOS, there is no precedent for a guide like this in Honduras. At the health clinic level, however, not everyone was aware of the existence of these manuals. Doctors managing health clinics interviewed for this evaluation said they knew some guides and manuals had been produced, but they had not seen them. The health coordinator at the Mancomunidad of COLOSUCA also said that some manuals do not yet exist and would be good to produce, for example, a manual on roles and responsibilities of decentralized health-care officials and a methodological guide on how to comply with the Ministry of Health’s indicators. Decentralized health officials also expressed the need for more training and technical capacity on human resource management and assistance in developing strategies for leveraging additional financial resources from government agencies and/or donors. RECOMMENDATIONS 1. Indicators and PMPs USAID/NEXOS should consider amending its Performance Management Plan to alter the articulation of its sub-result 2.3 and to include (or substitute) outcome indicators for Sub-results 2.1 and 2.3. As argued above, sub-result 2.3 is vaguely articulated and does not reflect the activities USAID/NEXOS is actually im￾plementing on the ground. Indicators for Sub-results 2.1 and 2.3 are all output indicators that do not ade￾quately measure USAID/NEXOS achievements. Please see Annex G for some suggestions on alternative outcome indicators. Based on the analysis and limited data validation exercise of the water service integral index and the primary health decentralized services index, the evaluation team recommends a fuller data audit to verify the accura￾cy of the data being collected. See Annex H for a full discussion on these findings. 2. Project’s Scope Consider concentrating the project’s geographic scope, limiting the number of municipalities benefitted by the program and focusing activities only in C/D municipalities. A/B municipalities are bigger, more complex and sophisticated and offer fewer opportunities to demonstrate significant results for a project like USAID/NEXOS, geared at improving decentralized service delivery at the local level. It would take a consid￾erable amount of resources (both human and financial) to bring about significant change in larger municipali￾ties. Avoiding too much geographic dispersion will allow USAID/NEXOS to invest more resources in fewer places, provide more continuous assistance to its beneficiaries and increase the likelihood of obtaining posi￾tive results. Avoid thematic dispersion by focusing on health and water service delivery. These are the most pressing needs of Honduras’ citizens. Although education, roads, citizen security are important sectors, USAID/NEXOS dilutes its potential impact if it spreads its resources too thinly across many different and unrelated activities. 27 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS Small infrastructure projects should be more clearly linked to the overall project’s objective of improving decentralized health and water service delivery. Greater coordination between the small infrastructure pro￾jects and the projects’ three sub-results generates positive synergies to maximize project’s results. Consider reducing the total budget allocated for small grants and allocating the funds in small infrastructure projects. It is not clear USAID/NEXOS will be able to execute 1.3 million dollars in small grants in the time remaining in the project’s life. 3. Sustainability Increase follow-up after capacity-building and technical assistance activities. USAID/NEXOS should work with beneficiaries to draft follow-up plans with well-established goals, deliverables, benchmarks, and time￾lines. This would generate greater incentives for trained staff to implement the skills they learned and even￾tually, change the way of doing things. Additionally, consider conditioning continued project support on the accomplishment of specific and meas￾urable results. Follow-up plans could be used to measure and reward (or not) beneficiaries’ performance. “Performance Based Incentives” can become a powerful mechanism to generate greater incentives for pub￾lic officials to improve their performance and become more accountable to citizens and to those who fund their capacity building efforts. Consider designing a sustainability plan for allowing Mancomunidades to charge a fee for services provided and therefore not be too dependent on donor support. Without independent sources of funding, it is un￾clear how many consultants providing technical assistance to municipalities (currently funded by USAID/NEXOS) will be able to survive once the project comes to an end. Provide training and technical assistance to civil society organizations to build their capacity to write pro￾posals and apply for additional funding from government sources and/or other donors. Design more user friendly guides and manuals to institutionalize USAID/NEXOS’ capacity building efforts and leave a legacy after the project’s completion. This is particularly needed for financial technical staff in the municipalities who in most cases, only have thick and not user-friendly manuals left by the Spanish coopera￾tion and which typically are not consulted by public officials. 4. Coordination Conduct a mapping exercise of existing resources in Honduras for water and health services. Mapping will reveal gaps in financing and possible opportunities for leveraging resources in these sectors. Coordinate with existing government institutions, agencies, donors, and NGOs that are already investing resources in water and health and have existing technical expertise that could be leveraged to improve pro￾gram results. USAID could also match funds from different institutions towards this end. This would avoid duplication of efforts and create greater synergies among existing programs. 5. Targeting Beneficiaries Ensure beneficiaries have the capacity to use the equipment they receive. 28 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS Tailor technical assistance to the specific needs and capacities of different stakeholders. For instance, illiterate people require different training methodologies and tools. New financial officials may require more basic training than more experienced officials. 6. Communications USAID/NEXOS’ consultants should be clear about the scope of the project and its limitations to avoid cre￾ating false expectations among beneficiaries. Promising to invest in “projects” that are unfeasible generates frustration. This can potentially undermine the project’s credibility. Identify successful experiences in the delivery of health care and water services. Systematize these experi￾ences, promote exchanges among stakeholders and disseminate the results. In coordination with USAID/DEE conduct a communications campaign to educate citizens and generate greater awareness on the benefits of decentralization, the importance of paying for services, and the health benefits of water chlorination. Project efforts will not be sustainable if citizens do not change traditional pat￾terns of behavior. 29 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS THE DECENTRALIZED ENABLING ENVIRONMENT PROJECT (USAID/DEE) RESULTS FRAMEWORK: OBJECTIVE, INTERMEDIATE RESULTS AND INDICATORS The Decentralization Enabling Environment program (USAID/DEE) started on February 22, 2011 and has a current expected completion date of February 22, 2016. With $2 million from USAID and $400,000 from FOPRIDEH for an overall investment of $2.4 million, the project works in [57] municipalities in seven Man￾comunidades: MANOFM, MANSUCOPA, MUNASBAR, MAVASEN-AMVAS, MANCURIS, MANLESIP, and MAMSA. USAID/DEE seeks to promote the enabling environment necessary for the decentralization of government services to the local level in order to better respond to citizen needs. The program aims to strengthen the capacity of national institutions, local governments and civil society to promote, promulgate and enforce legislative reforms to decentralization. As outline above, USAID/DEE’s results framework is not clearly articulated. The overall result is vaguely stated and Sub-results 1 and 2 overlap and are not clearly defined. In fact, a comprehensive legal framework for municipal autonomy includes municipal fiscal autonomy. Complicating matters further, the first two sub- 30 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS results are not distinct from the overall objective, making the causal connection between results and sub￾results difficult to establish. USAID/DEE has identified 10 indicators to measure project results. Although USAID/DEE does not classify indicators as outcome or output indicators, most of the indicators could be considered outcome indicators. Please see Annex J for analysis and brief comments on USAID/DEE’s indicators. However, most of the indi￾cators are poorly defined, not direct, and not adequate to measure project’s results. No indicators are iden￾tified for measuring the overall result, namely, “strengthened enabling environment for decentralized ser￾vices.” Indicators for Sub-results 1 and 2 overlap; a fact that only reveals the overlap between the two sub￾results. More importantly, the indicator, “percentage of laws and reforms passed by the GOH and drafted with USG assistance to promote decentralization compared to introduced initiatives” falls outside USAID and FOPRIDEH’s manageable interests. Formulating laws and presenting them to Congress is one thing; passing laws is another. As we discuss below, decentralization legislation is languishing in the National Con￾gress. Moreover, the indicator measures the number of laws passed without regard to quality or implemen￾tation. As such, these indicators incentivize the creation of new laws or amendments. More laws do not necessarily mean better laws and it is conceivable that a single comprehensive law would be preferable to piecemeal legislation. Indicator 2.3, “score of implementation of legislation to promote fiscal autonomy” is vague. In practice, FOPRIDEH measures the recovery of tax arrears by the municipality as a percentage of total municipal tax arrears, but the wording of the indicator does not convey this objective. Finally, several indicators have unrealistic targets, which undermine program success. For example, for Indica￾tor 1.2, targets for the percentage of laws and reforms passed by the GOH to promote decentralization is four in 2013, seven in 2014, and eight in 2015. Likewise, for Indicator 2.2, targets for the percentage of laws and reforms passed by the GOH to promote fiscal autonomy is two in 2013, four in 2014, and six in 2015. In light of the recalcitrant political environment and significant opposition to decentralization in the National Congress, these targets appear overly ambitious. In addition, for Indicator 2.1, USAID/DEE aims to increase fiscal municipal autonomy (i.e., own source revenue/total income) five percent each year. Given the low levels of own-source revenue in target municipalities (particularly in C and D municipalities), a five percent increase each year is quite substantial. Finally, for Indicator 2.3, the score of implementation of legislation to promote fiscal autonomy, implemented by the municipalities, is 10 percentage point increase each year, totaling a 40 percentage point increase over the life of the program. Given the challenges inherent in tax collection and enforcement, meeting this target might be challenging. FINDINGS AND CONCLUSIONS I. USAID/DEE’S OVERALL RESULT: ENABLING ENVIRONMENT FOR DECEN￾TRALIZATION STRENGHTENED 1. Constraints Operating environment. Several external factors have hampered USAID/DEE’s progress. While these con￾straints are outside of FOPRIDEH’s control, they nevertheless frustrated implementation and made it harder for the program to achieve success. First of all, as with any political process, decentralization engenders winners and losers—those that will benefit from increased power and authority at the local level and those that stand to lose ground during this process. Opponents of decentralization—proponents of the status quo—such as unions, central govern￾ment ministries (e.g., health, education, finance, SANAA), remain strong in Honduras. In addition, there is widespread confusion surrounding decentralization and its benefits among civil society and the public at large. Many people are either completely unfamiliar with the concept, or erroneously con- 31 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS flate it with privatization. In a focus group, for instance, several mayors told the evaluation team that their constituents do not have a high opinion of decentralization, largely owing to the fact that they don’t know what decentralization entails. Without a general conception of decentralization and its attendant benefits, let alone widespread support for decentralization, the program’s advocacy efforts will have limited success. Negative incentives for tax collection. Electoral processes also negatively affect the structure of incentives for decentralization efforts, particularly with regard to fiscal reforms. General elections were held in 2013; as a result, electoral consideration took precedence over fiduciary responsibilities and mayors standing for reelection were hesitant to raise taxes or collect back taxes. Moreover, the central government has ap￾proved certain provisions, such as debt forgiveness and tax and fee exemptions, that negatively affect munic￾ipal revenue. Tax amnesty weakens tax collection levels and promotes perverse incentives for paying taxes. Citizens form bad habits and wait for future amnesties. In a focus group in MANSUCOPA, mayors affirmed this sentiment and said that their citizens do not view paying taxes as a civic duty. There are delays in munic￾ipalities receiving transfers from the central government and delays in the update of municipal tax infor￾mation. Capacity gaps. Despite many capacity building efforts, municipalities still face severe capacity gaps in keeping accurate, timely income tax records. Not only are there high levels of turnover among local government technical staff, but many are political appointees and not necessarily qualified to perform their job, particular￾ly in C and D municipalities. While organizational and human resource manuals with job positions with at￾tendant technical competences exist in the majority of municipalities, they are not frequently used. 2. Progress on Indicators As defined by the PMP, USAID /DEE demonstrates poor indicator performance, with 70% of the project’s indicators falling below expectations as of December 2013. The project is “below expectations” on seven of ten indicators, even if some are below the target by a small margin of less than 2 percent. (Indicators 1.1, 1.3, 2.1, 2.2, 2.3, 3.1, and 3.2). The project exhibits “excellent” performance in the number of individuals to have received USG-assisted training (F. Indicator 2). In 2013, 1,019 individuals received USG-assisted training, exceeding the established target of 250 by more than 400 percent. While this is an output indicator, it demonstrates that USAID/ DEE is actively engaging beneficiaries and the sex-disaggregated data indicates equitable gender balance in pro￾gram activities. The program also demonstrates “acceptable” performance in the percentage of laws and reforms promoting decentralization drafted with USG assistance (F Indicator 1) and passed by the GOH (Indicator 1.2). However, as these laws have not been passed by the National Congress as of this writing, Indicator 1.2 should have been one (one of these regulations was an executive decree that did not have to be approved by Congress). 3. Coordination There are several USAID programs that are working on decentralization. USAID/NEXOS, as described above, overlaps with USAID /DEE. Both programs share the same AO: More Responsive Governance) and IR (Locally‐Provided Services in Response to Citizens Needs Improved). The programs also both seek to promote civil society advocacy and strengthen municipal finance. While FOPRIDEH has had high-level meet￾ings with Deloitte, the team found no evidence of substantive coordination. In fact, USAID/DEE’s Chief of Party admitted in an interview that “meetings with USAID/NEXOS were a waste of time.”60 60 Interview with Ronald Fiallos, May 15, 2014. 32 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS Another USAID program, the Local Technical Support Unit, or Unidad Local de Apoyo Técnico (USAID/ULAT), provides technical assistance and policy reform to the Ministry of Health (MOH) to decen￾tralize health services and improve the national health system by working with the MOH to allocate funds for decentralization, assess the legal framework for the health sector, advocacy and negotiations on decen￾tralization within the health sector, among other activities. While the programs could have worked together on the legal framework and policy advocacy, the evaluation team did not find evidence that USAID/DEE has coordinated with USAID/ULAT. FOPRIDEH is a grantee of the USAID program IMPACTOS, a civic advocacy and transparency promotion program. Yet, USAID/DEE did not coordinate or shared information and lessons learned with IMPACTOS in its work with civil society organizations to promote advocacy activities in favor of decentralization and accountability of local governments. By not effectively coordinating with USAID/NEXOS, USAID/ULAT, USAID/IMPACTOS, FOPRIDEH missed opportunities to further broaden the constituency and political will for decentralization. Nevertheless, USAID/DEE has worked in a highly coordinated fashion with several other stakeholders at the national and local levels, including the Municipal Association of Honduras, or Asociación de Municipios de Honduras (AMHON), the Decentralization Technical Unit, or Unidad Técnica de la Descentralización (UTD), the Secretary of the Interior and Population (SEIP), Secretary of Planning, Secretary of Finance, the Executive Commission for State Decentralization (CEDE), Superior Accounts Tribunal (TSC), UFIM, individ￾ual legislators, and congressional committees. At the central level, USAID/DEE works with its partners on higher-level legal and policy reforms. At the local level, FOPRIDEH has a network of affiliates, to whom they provide training on decentralization, fiscal autonomy, advocacy and other issues. With the TSC and AM￾HON, for example, USAID/DEE conducted workshops, experience exchanges, and trainings aimed at em￾ployees and municipal officials who work in financial, accounting areas, and UFIMs. USAID/DEE also engaged in discussions of the new Law of Municipalities with experts from SEIP/UTD and AMHON. USAID/ DEE also had an agreement with UNICEF to support a Youth Communications Network to conduct workshops to strengthen youth knowledge of local governance issues. Additional examples can be found in USAID/DEE’s semiannual reports. 4. Gender As mentioned above, gender equity and strengthened local governance are mutually reinforcing. FOPRIDEH is to be commended for its gender-sensitive, inclusive approach that does not view women as merely vic￾tims or beneficiaries but as active participants in their own development. The program’s Gender Main￾streaming Strategy, approved by USAID in May 2013, evinces a holistic approach to commitment to gender equity that aims to leverage the unique abilities of women and girls, anticipate potential obstacles to female involvement, and design program activities to promote equitable access. FOPRIDEH has also successfully incorporated women into their program, particularly given the barriers Honduran women face to participation in the public sphere. USAID/DEE has seen relatively equitable levels of participation by men and women in program activities. For instance, from July to December 2013, there were 426 participants under Results 2 and 3, of which 234 (55%) were women. Nevertheless, while these indicators that measure the existence of women’s participation, there is no indicator that measures the qual￾ity of this participation in program activities or its effect. The project has also sought to empower and strengthen the capacity of women in the decentralization process at the local level, including female mayors and vice-mayors, women in municipal commissions, mu￾nicipal women’s offices (OMM), the Municipal Women’s Alliance of Honduras, and the Women’s Network of Transparency Commission of Francisco Morazán. This participation has led to substantive progress on the Law of Municipalities and the Law of Decentralization. For example, in conjunction with the Women Munic￾ipalities of Honduras (ANAMMH), USAID/DEE conducted several workshops to discuss the Law of Munici- 33 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS palities. These workshops ultimately led to an amendment, drafted with ANAMH’s support, which specifies clear and precise duties and obligations for the OMM. Women’s participation in politics, however, is necessary but not sufficient for political empowerment. The presence of women in local government is important, but it does not reveal whether these women pro￾mote women’s issues. In La Paz, for example, all mayors—men and women—did not know of a single women’s group in their municipality. In addition, several respondents told the team that the head of the OMM, when it is filled, can be a political favor bestowed by the mayor, and in many cases are not true ad￾vocates for gender equity. In the civil society focus group, the head of a women’s group said that she feels “as if she is fighting for women’s rights alone.”61 II. SUB-RESULT 1.1. COMPREHENSIVE LEGAL FRAMEWORK FOR INCREASED MUNICIPAL AUTONOMY STRENGTHENED 1. Enabling Environment for Decentralization Under this sub-result, USAID/DEE engages in activities to strengthen the national decentralization frame￾work. To date, USAID/DEE has worked on four decentralization reforms: (1) the De-centralization Decree; (2) the Decentralization Law, (3) the Municipal Law, and (4) the Municipal Administrative Career Law (LCAM). FOPRIDEH worked hard to draft these laws in a highly participatory manner and achieve consen￾sus on the part of civil society, government officials, and legislators. Despite program efforts to advocate for their drafting and passage, however, these laws are stalled in Congress and have still not been passed. Unde￾terred, USAID/DEE has continued to publically discuss these laws and advocate for their passage. FOPRIDEH has sponsored discussions of the new Law of Municipalities with experts from SEIP, UTD, and AMHON, and separately with private sector stakeholders, which elicited an important suggestion from the business community on operating permits and registration. USAID/DEE subsequently presented a new draft to the technical committee in the National Congress. USAID/DEE has also used existing institutional mechanisms, such as the Executive Commission on Decen￾tralization, or Comisión Ejecutiva de Descentralización, and the National Forum on Decentralization, or Foro Nacional de Descentralización, to lobby in favor of the decentralization laws. To ensure decentralization reforms are successful, the program designed and implemented activities to promote public awareness of and support for decentralization. USAID/DEE created the Observatorio Social de la Descentralización, a website that provides Honduran citizens with in-formation about the decentraliza￾tion process in Honduras. As of this writing, the website has had more than 1.3 million visits since it first launched. The user-friendly website uses a variety of mediums, including videos, photo galleries, updated news articles, press releases, newsletters, interactive comments, and texts of law and policy drafts. With assistance from AMHON and UTD, the website also aggregates information on central government trans￾fers to the local level and municipal tax revenues and compiles this information in a downloadable report. In addition to the Observatorio website, USAID/DEE used traditional media to support its awareness raising efforts. In the past year, the program has made a concerted effort to increase its public profile, producing TV and radio programs that foster an understanding of decentralization, its benefits, and the role of citizens 61 Maritza N. Chavez, May 3, 2014. 34 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS and public officials. USAID/DEE has interviewed congressmen, presidential candidates, local officials, and subject matter experts to discuss these topics on the radio. Despite USAID/DEE’s advocacy efforts, however, the enabling environment for decentralization has not improved. The government’s political will for decentralization is questionable and enormous distrust and misconceptions of decentralization still exist in the country, as evidenced in the team’s interviews and focus group discussions. 2. Perception of Beneficiaries Overall, beneficiaries interviewed by the evaluation team were highly satisfied with USAID/DEE’s activities and approach to lobby for various decentralization reforms, including the Decentralization Law and the Administration Career Law. Beneficiaries particularly valued USAID/DEE’s highly participatory approach to discuss the different legal initiatives. However, current government officials interviewed at the Coordinación General de Gobierno and the Uni￾dad Técnica de Descentralización argued that the laws presented to the National Congress have serious weaknesses and need to be reviewed by the current administration. These government officials are new in the administration and did not take part in the discussions and debates on the laws that were presented to Congress. III. SUB-RESULT 1.2. LEGAL FRAMEWORK FOR MUNICIPAL FISCAL AUTONOMY STRENGTHENED 1. Local Capacity Under this sub-result, USAID/DEE implemented activities to strengthen municipalities’ fiscal autonomy. Fiscal autonomy confers a greater degree of accountability on locally elected officials, allowing them to more di￾rectly respond to local needs. Given the high level of dependence on the central government, especially among the poorer C and D municipalities, and pervasive corruption in the central government, this is an important step in the decentralization process. To this end, FOPRIDEH provided technical assistance to financial municipal authorities to strengthen their capacity to generate own-source revenue. The program conducted trainings on several topics, including budget administration, accounting, tax and internal control, accountability, and other topics related to munic￾ipal finances. For example, USAID/DEE worked with the Superior Accounts Tribunal (TSC) and AMHON to conduct workshops, experience exchanges, and trainings with municipal officials.) The project also worked with assisting the technical unit of the Mancomunidades Intermunicipal Technical Finance Unit (Téc￾nico Unidad Financiera Intermunicipal, or UFIM) so they could provide support to the municipalities. This approach, on the other hand, not only strengthens local capacity but also builds a cadre of local technical expertise that is essential in ensuring project sustainability. However, the evaluation team questions the sus￾tainability of these trainings. The team did not find evidence of existing manuals and/or operation guides provided after technical assistance sessions. In order to strengthen municipal financial management and promote fiscal transparency, USAID/DEE also worked with AMHON to expand the use of the Tributary and Financial Administration System (SAFT) 35 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS software in target municipalities. According to the program’s semiannual report, this “resulted in higher than expected collection of own-source revenue and proper implementation of fiscal laws.”62 In some municipalities, UASID/DEE assisted with reducing tax arrears through mediation with contributors and legal assistance. They provided a lawyer to the Mancomunidades to work on the recuperation of late taxes and worked directly with the office of tax control to help them raise revenue and understand the resources they could collect. In MANSUCOPA, for example, municipalities were able to recover nearly 1.5 million lempiras ($71,776), equivalent to 58 percent of the total 2.7 million lempiras ($131,178) that the municipality had presented to the court for recovery. Not only did this increase the municipalities’ own￾source revenue, but it also helps combat the pervasive viewpoint that there is no penalty for not paying taxes. However, despite the program’s efforts, FOPRIDEH did not achieve the sub-result’s overall objective – in￾creasing municipal fiscal autonomy. In fact, according to USAID/DEE’s semi-annual report, fiscal autonomy decreased 39 percent between 2012 and 2013. Moreover, there are widespread inconsistencies between municipalities’ official income records and income reports presented to the Supreme Audit Institution, the Tribunal Superior de Cuentas. In fact, a limited data validation exercise on income information in seven mu￾nicipalities revealed that there are broad inconsistencies between income data reported by USAID/DEE and official TSC records. For a graphic representation of this exercise please see Annex K. 2. Perception of Beneficiaries Activities under this sub-result had a mixed result with beneficiaries. In fact, mayors expressed great satisfac￾tion with USAID/DEE’s capacity building and technical assistance efforts. In fact, some mayors said that USAID/DEE’s technical assistance and quality of their technical experts is the best they have ever received (“lo mejor de lo mejor”)63. Mayors in particular appreciated the use of a lawyer to recover tax arrears. On the other hand, however, while technical municipal staff appreciated the assistance described above, they noted that they had received similar trainings in the past and complained that FOPRIDEH did not conduct a needs assessment before the technical assistance and trainings. Technical staff cited resource constraints one of the biggest obstacles they face, claiming they do not have sufficient personnel and computer equipment to do their job. IV. SUB-RESULT 1.3. CIVIL SOCIETY PARTICIPATION IN THE DECENTRALIZA￾TION PROCESS INCREASED 1. Civil Society Participation Under this sub-result, USAID/DEE provided training to several civil society organizations on decentralization laws, conflict resolution techniques, advocacy, public policy formulation, gender issues, and social auditing practices. For example, FOPRIDEH developed a workshop on public policy advocacy methods and negotia￾tion techniques in Santa Rose de Copán for 11 of its affiliates from the MANVASEN and AMVAS manco￾munidades, including World Vision and Habitat for Humanity. FOPRIDEH also worked with AMHON to 62 USAID/DEE Semi-annual report, January-June, 2013. 63 Jose Tejeda, Mayor of Cane, May 3, 2014. 36 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS organize a meeting between mayors from AMHON’s advocacy commission and 10 of FOPRIDEH’s local affiliate CSOs to identify a strategy to lobby for the four decentralization bills advanced by USAID/DEE. However, after these capacity-building workshops, in their interviews and focus groups with program bene￾ficiaries, the team did not find much evidence that FOPRIDEH followed up to ensure organizations imple￾mented skills learned. Greater civil society participation is needed to strengthen local autonomy and development and ensure these processes occur in an inclusive, transparent, responsive, and accountable manner. A focus on strengthening local government administrative structures and practices without a commensurate investment in building citizens’ participatory, oversight, and advocacy roles will jeopardize decentralization. However, civil society in Honduras is weak and often politically subordinate to the mayor. In fact, many CSO leaders trained by USAID/DEE were dismissed from their organizations with the change of government. For exam￾ple, the leader of a women’s network in La Paz that focused on HIV, domestic violence, and social auditing was forced out when a new mayor was elected. While USAID/DEE organized mesas temáticas, bringing together several types of CSOs to discuss a particular issue, a more sustained effort to create a robust, in￾dependent civil society in Honduras is needed, although this is outside the scope of this program. USAID/DEE’s indicators for this sub-result are output indicators and do not measure increased citizen par￾ticipation. One indicator measures attendance to meetings or events organized by the project, but as we discussed above, attendance is not the same as participation. The other indicator measures civil society per￾ception on decentralization, not participation. Furthermore, the data collection methodology for this indica￾tor is not statistically valid. USAID/DEE conducts a semiannual survey of CSO directors, municipal authori￾ties, government representatives, and other key stakeholders. The survey asks a series of five questions de￾signed to elicit respondents’ views on key decentralization issues. According to the program’s latest semian￾nual report, this indicator increased 17 percent in 2013, falling just shy of its 20 percent target, ostensibly demonstrating a sizeable increase in support for decentralization. However, by asking for agreement, these questions are leading, making it easier for the respondent to say “yes” than “no.” As such, this indicator is flawed and its results are biased. 2. Perception of Beneficiaries Overall, beneficiaries the team interviewed appreciated USAID/DEE’s innovative approach to adult learning, which combined presentation with practical exercises. Civil society representatives said the conflict resolu￾tion training was particularly useful. The sustainability of this assistance, however, is less clear. When the evaluation team asked what they learned from USAID/DEE’s trainings, one participant stated, “Civil society failed FOPRIDEH… We are at fault for not putting into practice what we learned.”64 The program also strove to increase support for decentralization and local autonomy among civil society and other key stakeholders. To this end, for example, FOPRIDEH organized an organizational culture work￾shop with municipal employees in MUNASBAR. Indeed, in the team’s interviews and focus groups, benefi￾ciaries expressed near unanimous support for decentralization. In the words of one civil society leader, “De￾centralization is important… The center is too centralized and nothing moves.” USAID/DEE also conducted activities to bolster beneficiaries’ awareness of the decentralization process. Program efforts in this area, however, appear to be less successful. Though anecdotal, very few civil society leaders and political repre￾sentatives the team interviewed knew what was happening with regard to the set of decentralization re￾forms before the National Congress. 64 Robelio Peñalba, May 3, 2014. 37 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS RECOMMENDATIONS With approximately a year and a half remaining in FOPRIDEH’s present cooperative agreement, it would be counterproductive to drastically change the objectives of the program or reverse ongoing activities. Never￾theless, there are several adjustments that can be made to USAID/DEE’s present activities to address some of the concerns raised in this report. The team presents a set of recommendations for the USAID/ /DEE program and future decentralization programming below. 1. Results Framework and PMP. To the extent feasible, FOPRIDEH should revise its PMP and redefine its results and indicators. USAID and FOPRIDEH should consider establishing more realistic targets for program indicators. Sub-results (particularly Sub-results 1 and 2) should also be renamed to re￾fer to advocacy, municipal financial autonomy, and civil society to improve clarity. While creating new outcome indicators might be impractical at this point, USAID/Honduras should ensure that fol￾low-on programs have a clearly defined results framework, with distinct outcome indicators for each result that are direct, measurable, timely, practical, and attributable. 2. Advocacy. USAID should encourage the GOH to convene the Foro Nacional de Descentralización to discuss the status of the different decentralization laws before the National Congress. The donor community should also encourage a dialogue between the new government authorities and sup￾porters of decentralization laws and come to a new agreement on a package of legislation that can be passed successfully by Congress. This dialogue could be moderated by neutral international ex￾perts who could help achieve consensus between opposing parties to arrive at a final version of the decentralization law that can be voted on. 3. Robust Public Awareness Campaign. USAID/DEE should greatly expand its public awareness efforts and a robust public education campaign should be established as part of follow-on programs. The Honduran public needs to become better informed about decentralization: what it is, its benefits, and citizens’ roles and responsibilities. Misconceptions and ignorance about decentralization not on￾ly stymie program efforts, but widespread popular support for decentralization laws in the legisla￾ture could foment political will and move the process forward. To reach out and engage a wider audience, FOPRIDEH should improving the definition of their key messages and their target audi￾ences and consider social media as a possible medium. 4. Financial Autonomy. FOPRIDEH’s model of working through the Mancomunidades is good and should be continued in follow-on programming to promote greater sustainability of project efforts. The project, however, should consider working with Mancomunidades to design a financial sustain￾ability handover plan, such as charging a fee for services, to allow technical officials to continue providing training and technical assistance to municipal officials. In addition, USAID/Honduras should incorporate incentives to municipal authorities to improve their own-source revenue, such as condi￾tioning further assistance on the achievement of particular benchmarks, in future assistance. 5. Civil Society. Where possible, USAID/DEE should follow up on its trainings and technical assistance programs offered to civil society organizations to ensure that agreements reached and operational plans designed in these sessions are implemented. Moreover, subsequent programming should tailor capacity building efforts to beneficiaries’ particular needs and feature participatory follow-on meth- 38 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ods, such as tailored mentoring and coaching approaches, which will be more conducive to sustain￾ability and partner ownership. Furthermore, civil society is weak and frequently subject to the [incli￾nation] of the presiding mayor, frustrating project efforts. FOPRIDEH should go beyond established local civil society organizations, like municipal transparency committees, to identify potential com￾munity leaders, such as patronatos, who could be more independent from government authorities. Future support could also create and strengthen the advocacy and capacity of a national-level mayors’ association to build support for decentralization. Since decentralization efforts are wholly dependent upon active citizen participation, future pro￾gramming in this area should strengthen the capacity of civil society organizations. As civil society in Honduras remains underdeveloped and as local resources remain constrained, public pressure on the central government to legislate change is marginal at best. Legislative and executive buy-in and cooperation will be necessary to pass a law on decentralization and civil society organizations must develop the capacity aggregate and vocalize demands. Public education (as described above) and civil society capacity-building initiatives are, therefore, key inputs for successful decentralization. 6. Strengthened Coordination. In order to avoid duplicating efforts, leverage limited resources, and promote institutional synergy, FOPRIDEH should with other programs, including other USAID pro￾grams. USAID/DEE should collaborate closer with USAID/NEXOS to improve and standardize technical assistance packages provided to technical municipal officials and civil society organizations. Although these two projects have adopted different methodologies in their work with financial mu￾nicipal authorities, both programs work to strengthen municipal finance and should learn from one another and identify and exchange best practices. USAID/DEE should also coordinate with USAID/IMPACTOS on the design of social auditing strategies and tools. One particular area where additional coordination is necessary is advocacy. A broader constituency for decentralization is needed to effectively overcome lagging political will. By speaking with one voice, organizations would be better able to exert greater influence on the National Congress and advance decentralization. Specifically, the program would have benefitted from a large-scale and sustained public relations campaign throughout the course of the program to promote the idea of local as a public good, to help mobilize broad-based public opinion and activism around media re￾form, and to further influence public policy agenda-setting on this issue. By working with other stakeholders, FOPRIDEH would have been able to undertake more strategic and effective advocacy initiatives. 39 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS LESSONS LEARNED 1. Donor projects can maximize their results when they avoid dispersion and fragmentation of activi￾ties and scope. In poor countries like Honduras, where citizen needs are so high, donors face the temptation to address as many needs as possible in the largest number of places. Yet, less is more when it comes to achieving positive results and making a difference in a particular community. Working more intensely and continuously in fewer places increase the potential of achieving posi￾tive results. These achievements yield to a greater sense of local ownership and can ensure more buy-in on the part of local authorities. 2. Years of donor funded capacity building initiatives –mainly training and technical assistance—have not significantly transformed beneficiaries’ patterns of behavior nor reduced their dependency on in￾ternational donors. In many places skills learned in trainings remain untapped; manuals unread and equipment unused. At a higher level, donor assistance has also not lead to significant reform: mu￾nicipal governments remain extremely dependent on transfers from the central government, efforts to increase own source revenue have not improved municipal financial autonomy and civil society organizations remain technically and organizationally too weak to demand greater accountability and advocate better service provision. Quality and coverage of public services –education, health, water, housing, economic development, etc.—remains insufficient and unable to adequately satisfy citizen needs. For donors, this may be an opportunity to innovate and introduce more creative approaches to affect traditional patterns of behavior. That requires paying attention to the structure of incen￾tives that drive behavior and designing interventions that takes these incentives into account. For example, conditioning donor support on the accomplishment of certain goals and objective results; rewarding good performance and penalizing underperformance can increase beneficiaries’ motiva￾tion to change their ways of doing things and eventually develop a greater stake in reform. 3. Donor funded projects should plan sustainability strategies during project implementation. Often, trainings and technical assistance initiatives are not adequately followed up; friendly manuals and guides are not produced and beneficiaries lack the resources to implement many of the solutions learned through donor support. In addition to providing more consistent follow-up of capacity building initiatives, donors should consider assisting beneficiaries to leverage additional resources and engaging local government authorities as options for ensuring greater sustainability. 4. Getting stakeholders with varied interests and disparate short- and long-term objectives to appreci￾ate their mutual interests and work together to support a common goal is an approach that is nei￾ther easy nor quick, but one that is essential in an increasingly fractious political environment. Coor￾dination can leverage limited resources and yield multiplier effects that will continue beyond the life of the program. 5. Although institutional mechanisms exist to facilitate donor coordination—like donor coordination tables--, in practice donors tend not to coordinate activities and work in isolation from one another. Coordination remains a challenge even within USAID-funded projects. USAID/NEXOS and USAID/DEE do not coordinate efforts even when coordination could benefit both projects. More importantly, USAID/DEE and IMPACTOS do not coordinate activities or share lessons learned, even when FOPRIDEH manages both projects. Coordination can generate positive synergies and 40 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS maximize project results, but it requires more than formal meetings where donors and implement￾ers “talk” and “show” their projects. Coordination is not spontaneous; it requires effort and plan￾ning of practical and operative mechanisms to organize activities. 6. As women become empowered through the political process, they can use their voice to advocate across sectors, including health, education, agriculture, water and sanitation, security and violence prevention, and job creation. The continued and expanded attention to gender inequality and em￾powerment of women in the political process will lead to better outcomes not only for women but also for Honduras as a whole. Decentralization can provide women with an opportunity for political empowerment, as barriers to entry—such as the ability to travel and spend time away from home, a large disposable income, a reasonable level of education, experience with political competition, and social and political connections—are lower at the local level. However, decentralization can al￾so reinforce structural inequalities and exacerbate discrimination against women. As such, it is im￾perative that USG-funded programs place appropriate emphasis on to ensure this does not occur. 7. Well-crafted outcome indicators are essential in measuring results and allowing implementers to “communicate their story.” Yet, often projects fail to design adequate outcome indicators and as a result, it becomes difficult to assess their achievements beyond anecdotal evidence. Crafting appro￾priate outcome indicators should be given a greater priority in project design. A-1 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ANNEX A: SCOPE OF WORK AND EVALUATION QUESTIONS A-2 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS A-3 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS A-4 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS A-5 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS A-6 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS A-7 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS A-8 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS A-9 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS B-1 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ANNEX B: KEY EVALUATION QUESTIONS (USAID/DEE PRO￾JECT) KEY EVALUATION QUESTIONS ORDERED BY RESULT AND SUB￾RESULTS Intermediate Result 1: Enabling environment for decentralized services strength￾ened 1. What are the constraints for achieving results the project? How are these being addressed? How severe are these constraints in impacting future activities? 2. Review USAID/DEE indicators and analyze which have the most significant data that adequately show project impact and results. 3. Describe the coordination with other USAID and /or Government of Honduras (GOH) projects. 4. How is gender being addressed within these projects, especially in the area of gender equity? In what specific ways have these projects impacted gender? Sub result 1.1: Comprehensive legal framework for increased municipal autonomy strengthened 1. In what specific ways are the projects creating an enabling and strengthened environment for de￾centralization? Where have these efforts failed? Is there overlap or complementarity between USAID/NEXOS and USAID/DEE or other USAID projects? 2. What do intended beneficiaries identify as the both positive and negative effects from the project activities? Sub Result 1.2: Legal framework for municipal fiscal autonomy strengthened 1. In what specific ways are the projects successfully building the capacity of the local governments to strengthen financial autonomy? Where have these efforts failed? What has been the role of civil so￾ciety organizations in this process? 2. What do intended beneficiaries identify as the both positive and negative effects from the project activities? Sub-Result 1.3: Civil society participation in the decentralization process increased 1. How are civil society organizations actively participating in local decision making processes? What approaches have been successful and why? B-2 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 2. What do intended beneficiaries identify as the both positive and negative effects from the project activities? 3. In what specific ways are the projects achieving greater transparency and accountability of local governments? Where have these effort failed? Recommendations 1. How might the projects be redirected to fill current gaps? What actions can be taken to improve the overall performance of the project? 2. How might these projects better coordinate with other projects? Identify potential opportunities to increase impact through USAID projects. 3. What do intended beneficiaries identify as potential sources of additional assistance within the scope of the projects? C-1 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ANNEX C: KEY EVALUATION QUESTIONS (USAID/NEXOS PRO￾JECT) KEY EVALUATION QUESTIONS ORDERED BY RESULT AND SUB￾RESULTS Result 2: Locally provided services in response to citizen needs improved 1. What are the constraints for achieving results the project? How are these being addressed? How severe are these constraints in impacting future activities? 2. Review USAID/NEXOS indicators and analyze which have the most significant data that adequately show project impact and results. 3. Are communities improving service delivery? If so, how has this impacted their lives? 4. Describe the coordination with other USAID and /or Government of Honduras (GOH) projects. 5. How is gender being addressed within these projects, especially in the area of gender equity? In what specific ways have these projects impacted gender? Sub-Result 2.1: Civil society advocacy for improved local services increased 1. In what specific ways are the projects achieving greater transparency and accountability of local governments? Where have these effort failed? 2. How are civil society organizations actively participating in local decision making processes? What approaches have been successful and why? 3. What do intended beneficiaries identify as the both positive and negative effects from the project activities? Sub-Result 2.2: Local institutional capacity to deliver decentralized services strengthened 1. In what specific ways are the projects successfully building the capacity of the local governments to provide services? Where have these efforts failed? What has been the role of civil society organiza￾tions in this process? 2. What do intended beneficiaries identify as the both positive and negative effects from the project activities? Sub-Result 2.3: Structures and systems to implement reform strengthened 1. In what specific ways are the projects creating an enabling and strengthened environment for de￾centralization? Where have these efforts failed? C-2 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 2. What do intended beneficiaries identify as the both positive and negative effects from the project activities? Recommendations 1. How might the projects be redirected to fill current gaps? What actions can be taken to improve the overall performance of the project? 2. How might these projects better coordinate with other projects? Identify potential opportunities to increase impact through USAID projects. 3. What do intended beneficiaries identify as potential sources of additional assistance within the scope of the projects? D-1 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ANNEX D: LIST OF INTERVIEW￾EES Name Organization Location 1 Ing. Irma Aracely Escobar Cárcamo Director Ente Regulador de los Servicios de Agua Potable y Saneamiento (ER￾SAPS) Tegucialpa 2 Ing. Héctor Serna Director ERSAPS Tegucigalpa 3 Ing. Luis Moncanda Consultant ERSAPS Tegucigalpa 4 César Gonzales Advisor to the Ministro General de Gobierno and Designado Presidencial de Gobernabilidad y Descentralización Tegucigalpa 5 Ing. Rolando Bú Director FOPRIDEH Tegucigalpa 6 Ronald Fiallos, FOPRIDEH General Coordinator Tegucigalpa 7 Abel Cantareo FOPRIDEH, Resultado 2 Autonomía Fiscal 8 Daniel Moreno FOPRIDEH, Resultado 3 Sociedad Civil Tegucigalpa 9 Wendy Aleman FOPRIDEH, Encargada del Observa￾torio de Centralización Tegucigalpa 10 Sigifredo Ramirez Director Deloitte, USAID/NEXOS Project Tegucigalpa 11 Lic. Mirta Gonzalez Deputy Director Deloitte, USAID/NEXOS Tegucigalpa 12 Alexander Rivera Deloitte, USAID/NEXOS Team Leader, Result I Tegucigalpa 13 Belinda Borjas Deloitte, USAID/NEXOS Team Leader, Result 2 Tegucigalpa 14 Maribel Suazo, Deloitte, USAID/NEXOS Team Leader, Result 3 Tegucigalpa 15 Yadira Recinos Deloitte, USAID/NEXOS Team leader, Small Infrastructure Pro￾jects Tegucigalpa D-2 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 16 Dayanira Laguna Deloitte, USAID/NEXOS Monitoring and Evaluation Tegucigalpa 17 Rina Flores Soto Deloitte, USAID/NEXOS Small Grants Tegucigalpa 18 Erika Rodriguez Deloitte, USAID/NEXOS Territorial Coordinator, MAVAQUI Tegucigalpa 19 Juan Carlos Morazan Deloitte, USAID/NEXOS Financial Specialist Tegucigalpa 20 Margot Contreras Deloitte, USAID/NEXOS Public Services Specialist Tegucigalpa 21 Luz Maria Aguilar Deloitte, USAID/NEXOS Health Specialist Tegucigalpa 22 Carlos Aguilar Deloitte, USAID/NEXOS Territorial Coordiator, Lempira Tegucigalpa 23 Jose Ramón Coto Deloitte, USAID/NEXOS Territorial Coordiator, Lempira Tegucigalpa 24 Sr. Jose Avilio Madrid Municipality of Macuelizo Jefe de Oficina del Catastro Macuelizo, Santa Barbara 25 Sr. Edilberto Pinto Municipality of Macuelizo Jefe de la Oficina de Control Tributa￾rio Macuelizo, Santa Barbara 26 Bessy Maricel Her￾nández Borjas Municipality of Macuelizo Tesorero Municipal Macuelizo, Santa Barbara 27 Mario Humberto Cordón Flores Mayor of Macuelizo Macuelizo, Santa Barbara 28 Benjamin Mondragon Representante Comisión Ciudadana de Transparencia Macuelizo, Santa Barbara 29 Nora Elizabeth Bonil￾la Responasable de la Unidad de Salud CESAMO O CESAR Macuelizo, Santa Barbara 30 Erika Yolanda Santos Technical staff, Municipality, Macuelizo Macuelizo, Santa Barbara 31 Liliana Carolina Ro￾driguez Technical staff, Municipality, Macuelizo Macuelizo, Santa Barbara 32 Samuel España Technical staff, Mancomunidad MAVAQUI Macuelizo, Santa Barbara 33 Merlin Roberto Vil￾lalvir Technical staff, Mancomunidad MAVAQUI Macuelizo, Santa Barbara D-3 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 34 Nathanael Sabillón Paz Manager Macomunidad MAVAQUI Macuelizo, Santa Barbara 35 Dulcia Yumila Gara￾bantes Technical staff, Mancomunidad MAVAQUI Macuelizo, Santa Barbara 36 Santiago Lavap Technical staff Municipality, Macuelizo Macuelizo, Santa Barbara 31 Victor Manuel Silva AJAAM Macuelizo, Santa Barbara 32 Uwaldo Pacheco AJAAM Macuelizo, Santa Barbara 33 Aura Estela Recinos AJAAM Macuelizo, Santa Barbara 34 Geovany Flores AJAAM Macuelizo, Santa Barbara 35 Ricardo de Jesús Ramírez Comité de Salud Macuelizo, Santa Barbara 36 Rigoberto Polanco AJAAM, Macuelizo Macuelizo, Santa Barbara 37 Armando Rios AJAAM, Macuelizo Macuelizo, Santa Barbara 38 Dimas Rivas Water Borad, Macuelizo Macuelizo, Santa Barbara 39 Reina Margarita Gar￾cía Water Board, Macuelizo Macuelizo, Santa Barbara 40 Delmi Licona Health Promoter Macuelizo, Santa Barbara 41 Rafael Heliodoro Valle Health Promoter Macuelizo, Santa Barbara 42 Marlon Edgardo Paz Reyes Health Promoter Macuelizo, Santa Barbara 43 Suyapa Ayala Citizen Macuelizo, Santa Barbara 44 Hayde Rivera Citizen Macuelizo, Santa Barbara 45 Waldin Paz Water Board Macuelizo, Santa Barbara D-4 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 46 Amelia Noriega Citizen Macuelizo, Santa Barbara 47 Delmi Reyes Corea Mayor, Nueva Frontera Nueva Frontera, Santa Barbara 48 Darwin Omar Ayala Municipality, Nueva Frontera Treasurer Nueva Frontera, Santa Barbara 49 Elsa Clementina Marroquin Municipality, Nueva Frontera Comisionado Municipal Nueva Frontera, Santa Barbara 50 Agner Eli Lara Municipality, Nueva Frontera Municipal Secretary Nueva Frontera, Santa Barbara 51 Porfirio Hernandez Representante Comisión Ciudadana de Transparencia Nueva Frontera, Santa Barbara 52 Otilia Murillo Representante Comisión Ciudadana de Transparencia Nueva Frontera, Santa Barbara 53 Nelson Silva Comisión de Acción Social Menonita Consultant Nueva Frontera, Santa Barbara 54 Lorenzo Garcia Moreno Nueva Frontera, Patronato Nueva Frontera, Santa Barbara 55 Luis Morel Nueva Frontera Council Member Nueva Frontera, Santa Barbara 56 Oscar Dubon Nueva Frontera Vice-Mayor Nueva Frontera, Santa Barbara 57 Antonio Ramos Ru￾ben Chinchilla Nueva Frontera AJAAM Nueva Frontera, Santa Barbara 58 Francisco Melchor Nueva Frontera AJAAM Nueva Frontera, Santa Barbara 59 Oscar Escobar Nueva Frontera AJAAM Nueva Frontera, Santa Barbara 60 Jose Efrain Juarez Nueva Frontera, Patronato Nueva Frontera, Santa Barbara 61 Maximiro Escobar Nueva Frontera Comite Municipal de Transparencia Nueva Frontera, Santa Barbara 62 Gilma Castillo Mayor, La Paz La Paz, MANSUCOPA 63 Fernando Mendez Mayor, Lejamani La Paz, MANSUCOPA D-5 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 64 José Tejeda Mayor, Cane La Paz, MANSUCOPA 65 Jesus Ivan Álvarez Mayor’s representative, Villa San Anto￾nio La Paz, MANSUCOPA 66 Denis Santos Mayor, San Sebastian La Paz, MANSUCOPA 67 Mario Palencia Mayor, Ajuterique La Paz, MANSUCOPA 68 Luz Maria Cruz Vice-Mayor, Lamani La Paz, MANSUCOPA 69 Eblin Nelin Mejia Girón Technical staff La Paz, MANSUCOPA 70 Denia Velásquez Technical Staff La Paz, MANSUCOPA 71 Reina Suyapa Iscoa Technical Staff La Paz, MANSUCOPA 72 Dunia Yamett Cca￾ceres Technical Staff La Paz, MANSUCOPA 73 Marvin Arias Technical Staff La Paz, MANSUCOPA 74 Mary Suyapa Castillo Technical Staff La Paz, MANSUCOPA 75 Olman Elio Barrera Technical Staff La Paz, MANSUCOPA 76 Robelio Peñalba Aprocafé, Gerente Departamental La Paz, MANSUCOPA 77 Edgardo Vásquez Productor Café La Paz, MANSUCOPA 78 José Angel Tinoco Presidente Comite PRR La Paz, MANSUCOPA 79 Enrique Castillo Director Ejecutivo Red de País La Paz, MANSUCOPA 80 Maritza N. Chavez FOPRIDEH, COMAS La Paz, MANSUCOPA 81 Eloisa Padilla Mansucopa. Tec Social La Paz, MANSUCOPA D-6 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 82 Crecencio Aguilar Tutule / Patronatos La Paz, MANSUCOPA 83 Jose Martín López Municipality Municipal Secretary San Manuel de Colohete, Lempira 84 José Asunción López Municipality Treasurer San Manuel de Colohete, Lempira 85 Candelario Martínez Fúnez Municipality Deputy Mayor San Manuel de Colohete, Lempira 86 Elvin Danilo Mejia Municipality Budget San Manuel de Colohete, Lempira 87 Jose Alonso Ma￾nueclos Municipality Environment Unit San Manuel de Colohete, Lempira 88 Roni Mejia Macomunidad COLOSUCA Manager San Manuel de Colohete, Lempira 89 Dilcia Marisol Rivera Municipality Oficina de Control Tributario San Manuel de Colohete, Lempira 90 Katherine Simón Hernández Mancomuidad COLOSUCA Funcionario responsable en Salud San Manuel de Colohete, Lempira 91 Dr. Julia Mondragon Responsable de la Unidad de Salud San Manuel de Colohete, Lempira 92 Luis Alonso Pérez Municipality Comisionado Municipal San Manuel de Colohete 93 Domingo Rodriguez Municipality Comisión Ciudadana de Transparencia San Manuel de Colohete, Lempira 94 Romilio Rafael Mar￾tínez Representante Comité Comunitario de Salud San Manuel de Colohete, Lempira 95 Leonilo Mateo San Manuel Colohete Water Board San Manuel de Colohete, Lempira 96 José Martín López San Manuel Colohete President of Comité de Desarrollo Comunitario San Manuel de Colohete, Lempira 97 Arnulfo Martinez San Manuel Colohete President Parent Association San Manuel de Colohete, Lempira 98 Jose Angel Vazquez San Manuel Colohete Water Board San Manuel de Colohete, Lempira 99 Fausto López San Manuel Colohete Water Board, treasurer San Manuel de Colohete, Lempira 100 Rosa Idalia Mejia San Manuel Colohete President, Parents Association San Manuel de Colohete, Lempira 101 Celestino Salcido San Manuel Colohete Water Board San Manuel de Colohete, Lempira 102 Roberto Sanchez San Manuel Colohete City Council San Manuel de Colohete, Lempira 103 Angel Mendez San Manuel Colohete Water Board San Manuel de Colohete, Lempira D-7 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 104 Juan Carlos Pascual Mancomunidad COLOSUCA San Manuel de Colohete, Lempira 105 Javier Antonio En￾amorado Rodriguez Mayor, Gracias Gracias, Lempira 106 Juan Ramón Cortés Municipality Manger Gracias, Lempira 107 Wilver Espinal Municipality Treasurer Gracias, Lempira 108 Enrique Flores Municipality Oficina de Catastro Gracias, Lempira 109 Maria Luisa Hernán￾dez Municipality Secretary of the Municipality Gracias, Lempira 110 Gracia Maria Mar￾tínez Mancomunidad COLOSUCA Funcionario responsable en Salud Gracias, Lempira 111 Marcos Perdomo Mancomunidad COLOSUCA Financial Administrator Gracias, Lempira 112 Sandra Melgar Municipality Manager, Finance and Management Gracias, Lempira 113 Leandro Espinoza Comisionado Municipal Gracias, Lempira 114 Javier Zelaya Municipality Municipal Auditor Gracias, Lempira 115 Alejandra María Díaz Gracias Comité Comunitario de Salud Gracias, Lempira 116 José Anael Argueta Gracias Comité Comunitario de Salud Gracias, Lempira Gracias, Lempira 117 Vilma Membreño Benitez Gracias Comisión Ciudadana de Transparencia Gracias, Lempira 118 Sonia Sarmiento Gracias Comité Comunitario de Salud Gracias, Lempira 119 Maria Luisa Quinta￾nilla Caceres Gracias Comité Comunitario de Salud Gracias, Lempira 120 Dr. Oneyda Diaz Responsable de la Unidad de Salud Gracias, Lempira 121 Maria Luisa Hernan￾dez Gracias Comisión Ciudadana de Transparencia Gracias, Lempira 122 Luis Alberto Reyes Gracias Water board Gracias, Lempira 123 Mario José Calix Gracias Comisión de Agua y Saneamiento (COMAS) Gracias, Lempira 124 Carmelo Orellana Gracias AJAAM Gracias, Lempira 125 Zeyda L. Urquia Coordinadora Acción Social Menonita Gracias, Lempira 126 Fernando Quintanilla, Gracias Water board Gracias, Lempira D-8 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 127 Rosa Enelina Orella￾na Gracias AJAAN Gracias, Lempira 128 Calet Perez Gracias Water Board Gracias, Lempira 129 Olvin Ruberman Reyes Mayor, Belen Belen, Lempira 130 Bessy Amanda Ma￾nueles Villanueva Municipality Treasurer Belen, Lempira 131 Johny Emiliano Mertoreno Municipality Oficina de Catastro Belen, Lempira 132 Noilan Nahun Ponce Benitez Municipality Oficina de Control Tributario Belen, Lempira 133 Dr. Oscar Rene Pi￾neda Ortíz Responsable de la Unidad de Salud CESAMO Belen, Lempira 133 Herminia Aguilar Belen Comisionado Municipal Belen, Lempira 134 Jose Rudyth Amaya Belen Comisión Ciudadana de Transparencia Belen, Lempira 135 Jose Adalid Anaya Belen AJAAM Belen, Lempira 136 Rodolfo Portillo R. Belen AJAAM Belen, Lempira 137 Santos Eladio Pinto Belen President, AJAAM Belen, Lempira 138 Mercedes Anaya Belen Deputy Mayor and AJAAM Belen, Lempira 139 Quenci Yaneth Per￾domo Belen Health Promoter Belen, Lempira 140 Nery Martinez Ar￾royo Belen Health Promoter Belen, Lempira 141 Karen A. Girón ASONOG Belen, Lempira 142 Mayra Elizabeth Bení￾tez ASONOG Belen, Lempira 143 Danilo Castillo, AMHON, Director Tegucigalpa 144 Wilson Geovanny Nuñez AMHON Gerente de Desarrollo Tegucigalpa 145 Cristian Vega AMHON Asistente de Municipios Tegucigalpa 146 Lourdes Durón AMHON Jefa de Descentralización Tegucigalpa 147 Tito Enriquez Secretaría del Interior. Director de la Unidad Técnica de Descentralización Tegucigalpa D-9 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS 148 Arnoldo Caraccioli Former Director of Unidad Técnica de la Descentralización. Tegucigalpa 149 Abog. Donaldo Re￾yes Avelar Former Member of Congress Comision de Descentralización Tegucigalpa 150 Dr. Juan de Dios Paredes ULAT-USAID (Health Program) Chief of Party Tegucigalpa E-1 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ANNEX E: FOCUS GROUP QUES￾TIONS (USAID/NEXOS PROJECT) Percepción Ciudadana sobre la Calidad de los Servicios Públicos 1. Cómo califican la gestión del actual gobierno municipal? 2. Qué tan satisfechos están ustedes con la calidad de los servicios municipales? Hacer referencia en particular a SALUD o AGUA POTABLE, según sea el caso 3. Ha cambiado la calidad del servicio en los últimos meses? Si si….Como y en qué específicamente. Como ha afectado este cambio en sus vidas diarias? 4. Que opinan del proceso de descentralización que ha transferido servicios y responsabilidades a los gobiernos munici￾pales? Es bueno, es malo o les da igual 5. Cuáles son las mayores dificultades que enfrenta la municipalidad para satisfacer las demandas de la población en ge￾neral y de sus organizaciones de sociedad civil en particular? 6. En su opinión cuales son las mayores fortalezas de la actual Corporación Municipal? 7. Que tanta información provee el municipio sobre los servicio públicos que ofrece? Oportunidades para la Participación Ciudadana 1. Qué canales o mecanismos existen para que los ciudadanos participen en los asuntos de su municipio? Cabildos abiertos, foros de rendición de cuentas, comités ciudadanos, auditoria social, etc. 2. Algún mecanismo es más efectivo que otro para participar? 3. ¿En esta municipalidad las organizaciones de sociedad civil son tomadas en cuenta para desarrollar acciones de forta￾lecimiento municipal? 4. ¿Cuál es el nivel de participación/entusiasmo de la población que asiste a los cabildos abiertos y otros mecanismos de participación? 5. ¿Qué mecanismos implementarían para mejorar la coordinación y comunicación con la municipalidad? 6. ¿Qué es lo que más les gusta de trabajar con la municipalidad? ¿Qué es lo que menos les ha gustado? 7. ¿Qué limitantes enfrentan ustedes para cumplir a cabalidad sus roles de liderazgo como sociedad civil? 8. Que capacidad tiene la sociedad civil para auditar a sus gobiernos municipales? Asegurar que no hay desvíos y que los servicios públicos se proveen de manera íntegra y justa 9. Han recibido alguna asistencia del Proyecto USAID/NEXOS o del Proyecto FOPRIDEH? 10. Que haría falta apoyar para mejorar la participación ciudadana? (PREGUNTA SIMILAR A 5—VER CUAL FUNCIONA MEJOR) E-2 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS Grado de Respuesta del Municipio a las demandas ciudadanas—Transparencia y rendición de cuentas 1. Que tan responsivos son los funcionarios municipales ante las demandas de la ciudadanía? 2. Existe algún ejemplo concreto en donde el gobierno haya tomado acciones como resultado de la participación ciudada￾na? O haya corregido el rumbo de alguna decisión de gobierno? 3. Qué tan transparente es el manejo de los recursos públicos? 4. Qué tanta corrupción piensan que existe en el municipio en general (del 1 al 10) y en la provisión de servicios públicos (acceso al servicio; ejercicio del gasto; contrataciones)? 5. Ha habido alguna consecuencia después de que alguna organización de la sociedad civil haya descubierto algún proble￾ma de corrupción a nivel municipal? Género y Desarrollo Municipal 1. En este municipio, las mujeres enfrentan problemas sociales diferentes a los de los hombres? 2. Los servicios públicos (agua, salud), son ofrecidos de manera indistinta a los hombres y a las mujeres? 3. Que acciones ha implementado el municipio para asegurar la equidad de género? F-1 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ANNEX F: FOCUS GROUP QUES￾TIONS (USAID/DEE PROJECT) Focus Group with Mayors Focus group with technical offi￾cials Focus group with representa￾tives from civil society organiza￾tions 1. Qué tipo de asistencia han recibido del proyec￾to USAID/DEE? Cuál es su percepción de esta asistencia? 1. Qué tipo de asistencia han recibido del proyec￾to USAID/DEE? Cuál es su percepción de esta asistencia? 1. Qué tipo de asisten￾cia han recibido del proyecto USAID/DEE? Cuál es su percepción de es￾ta asistencia? 1. Qué dificultades encuen￾tran para aumentar sus ingresos propios? 2. Qué dificultades encuen￾tran para incrementar sus ingresos propios? 2. Qué mecanismos de participación existen en los municipios? 2. Qué mecanismos de par￾ticipación tienen en sus municipios? 2. Qué tipo de asistencia les ha proporcionado FOPRIDEH para mejorar sus ingresos propios? 3. Que oportunidades existen para la parti￾cipación de las muje￾res? 3. Como se aseguran que las demandas ciudada￾nas son satisfechas? 3. Que tan afectiva ha sido la asistencia? 4. Que tanto han im￾plementado las he￾rramientas aprendi￾das en los talleres ofrecidos por el pro￾yecto USAID/DEE? 4. Qué tipo de organizacio￾nes civiles existen en el municipio? 4. Qué tipo de asistencia necesitarían adicional￾mente? 5. Que tan efectivo ha sido el proyecto USAID/DEE para promover la descen￾tralización en Hon￾duras? 5. Qué opinión tienen de las comisiones municipa￾les de transparencia? 5. Qué dificultades enfren￾tan en su planeación y ejecución del presupues￾to? 6. Que otro tipo de asistencia necesita￾rían recibir del pro￾yecto USAID/DEE. 6. Cuanta interacción exis￾te entre técnicos de dife￾rentes municipios? 6. Como se distribuye el gasto en sus municipios? 7. Qué tipo de regulaciones serían necesarias para que el municipio pueda cumplir con sus funcio￾nes y responsabilidades? Ley de descentralización, 7. Qué rol juega la manco￾munidad en la propor￾ción de asistencia técni￾ca en materia fiscal y fi￾nanciera? F-2 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ley de municipios? 8. Qué otro tipo de asisten￾cia necesitarían recibir del proyecto? 8. Qué otro tipo de asisten￾cia necesitarían recibir del proyecto? G-1 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ANNEX G: INDICATORS (USAID/NEXOS PROJECT) INDICATOR TYPE OF INDICATOR COMMENTS AND RECOMMENDATIONS Result 2. LOCALLY-PROVIDED SERVICES IN RESPONSE TO CITIZEN NEEDS IMPROVED 1. Percentage of re￾spondents who report high satisfaction with healthcare services de￾livery Impact Good indicator. USAID/NEXOS should consider including percentage of respondents who report high satisfaction with water service delivery. The survey includes this question; it could be included at no cost. 2. Percentage of own source municipal in￾come invested in local￾ly provided services (A/B) Impact Good indicator. The data, however, is not always accurate. The government of Honduras does not have consistent budget classifiers to identify ex￾penditures by sectors. 3. Percentage of munici￾pal income invested in locally provided ser￾vices (C/D) Impact Good indicator. The data, however, is not always accurate. The government of Honduras does not have budget classifiers to identify expenditures by sectors. 4. Average index of pri￾mary health decentral￾ized services. Impact Potentially good indicator, but the methodology to calculate this index is flawed. Please see Annex H for a discussion of this methodology USAID/NEXOS could measure number of patients attended at the clinics, number of people treated for common diseases, number of clinics that obtain 85 or more points on their quarterly evaluation by the Ministry of health. 5. Average water service integral index Impact Potentially good indicator, but the methodology to calculate this index is flawed. Please see Annex H for a discussion of this methodology USAID/NEXOS could measure: Water boards’ revenue collection; Frequency of water chlorination; Improvements in water infrastructure –such as water tanks, pumps or other small infrastructure projects. G-2 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS Sub-Result 2.1 CIVIL SOCIETY ADVOCACY FOR IMPROVED LOCAL SER-VICES INCREASED 6. Number of men and women of the civil so￾ciety participating in meetings with local au￾thorities for decision￾making with impact in the service improve￾ment performance Outcome This is not an outcome indicator. The indicator measures outputs: number of men and women at￾tending meetings with local authorities. The “im￾pact” of these citizens in service improvement is ill defined; data is collected from municipal records and civil society organizations. This is not an objec￾tive source for measuring whether or not citizens have impact in decision making. Not a clear way of measuring the outcome, namely, capacity to influ￾ence the decision making process. An alternative could be: Number of agreements reached by civil society and municipal authorities that are effectively imple￾mented. Source of this information should not come from municipalities, but from independent civil society organizations, even if these are national or regional NGOs such as ASONOG, CASM or other established NGOs that can have a more independ￾ent perspective on local government affairs. Number of corrective actions adopted by the mu￾nicipality in response to citizen advocacy interven￾tions (could be a social audit, a formal complaint, monitoring of the agreements formulated during open town hall meetings, etc). Frequency and quality of information provided by the municipal government. Budget information that is accessible to citizens and expenditure track￾ing experiences using this information Number of citizen requests for public information. 7. Number of civil society organizations receiving USG assistance en￾gaged in advocacy in￾terventions (cumula￾tive). Output As an output indicator, this is a good indicator. 8. Number of "muni￾cipios" in which the CSOs implements at least one mechanism for external oversight of public resources use supported by USG as￾sistance (cumulative). Output This is a good indicator, but the quality of these oversight exercises is questionable. First, social audits are not fully independent or “external” since they are carried out by the Municipal Transparency Commission. Second, the social audit of health ser￾vices conducted in Macuelizo was biased and its results were rejected by the Municipal commission￾er. The head of the Municipal Transparency Com￾mission worked for the Ministry of health and is an opponent of decentralization of health services. G-3 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS The audit did not follow established procedures. 9. Number of individuals who participate in local government and ser￾vice provider account￾ability events. Output As an output indicator, this is a good indicator. Sub-Result 2.2: LOCAL INSTITUTIONAL CAPACITY TO DELIVER DECEN-TRALIZED SERVICES STRENGTH￾ENED 10. Growth rate of own source revenue Outcome This is a good outcome indicator. But a better indi￾cator of municipal financial autonomy is the propor￾tion of own source revenue to total income. USAID/NEXOS has the data to calculate this indica￾tor. 11. Number of sub￾national government entities receiving USG assistance to improve their performance (cumulative) Outcome This is not an outcome but an output indicator. The indicator does not measure consequences of assis￾tance received by USG. Improved performance is also not defined. 12. Number of men and women who received USG assisted training including management skills and fiscal man￾agement to strengthen local government. Output As an output indicator, this is a good indicator. Sub-Result 2.3: STRUCTURES AND SYSTEMS TO IMPLEMENT REFORM STRENGTHENED 13. Number of decentrali￾zation constraints identified and ad￾dressed by service. Outcome This indicator is vague and does not measure results of USAID/NEXOS activities in the health sector, the focus of this sub-result’s activities. Moreover, the indicator is not direct, objective and/or practical. Not clear how “identified constraints” strengthens decentralization reform. Alternatives to measure outcomes of USAID/NEXOS activities in the health sector include: Adoption of new financial and administrative sys￾tems to manage health centers; Improved information systems in health centers; Use of new methodologies to calculate infant mor￾tality rates and analyze its causes; Improved evaluations scores from the Ministry of Health’s quarterly evaluations to decentralized health systems; G-4 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS Clean audits from the TSC 14. Number of men and women who received USG assisted training including management skills and fiscal man￾agement to strengthen decentralization. Output As an output indicator, this is a good indicator Activity 1 and Activity 2 SMALL GRANTS AND SMALL INFRASTRUCTURE PROJECTS 15. Number of funded ac￾tivities with civil socie￾ty organizations and mancomunidades. Output Good output indicator 16. Number of small infra￾structure projects exe￾cuted per year. Output Good output indicator H-1 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ANNEX H: ANALYSIS OF USAID/NEXOS WATER AND HEALTH INDICES USAID/NEXOS uses two indices, one for health and one for water, to measure program impact. According to USAID/NEXOS’s PMP, the health index seeks to “evaluate the average level of decentralized health ser￾vices in a municipality… [by] quantifying the degree of decentralization of health services, degree of advanc￾es in the service management, the degree of local ownership over primary healthcare services, and the im￾pact that these factors are having in the beneficiary municipalities.”65 The water index, on the other hand, aims to “comprehensively evaluate the level of water services provided… [by] determining the degree of advance in the service management, and the empowerment of local structures related to utilization and service delivery.” The indices present an innovative way to measure locally provided services, track this pro￾gress over time, and present this information in a cogent, easily digestible manner. An index, however, is only as sound as its construction and data collection practices. An index built on a flawed methodology or populated with biased data will yield an inaccurate depiction of program progress and impact. METHODOLOGY USAID/Honduras asked Democracy International to examine USAID/NEXOS’s health and water indices as part of the program evaluation. In order to assess the indices’ accuracy and appropriateness, the Monitoring and Evaluation Expert, Kris Thorpe, and the Research and Logistics Assistant, Juan Carlos Valeriano Álvarez, attempted to replicate data, to the greatest extent possible. The present exercise primarily relies on Rapid Appraisal, an approach that utilizes several evaluation methods to quickly, yet systematically, collect data, including: document review, key informant interviews, site visits, and focus groups. The team conducted interviews and focus groups with relevant stakeholders, depending on the index, in Macuelizo and Nueva Frontera in Santa Bárbara, and Belén, Gracias, and San Manuel Colohete in Lempira. The team also con￾ducted site visits to health centers with a doctor and dentist (Centros de Salud con Médico y Odontólogo, or CESAMOs), rural health centers (Centros de Salud Rural or CESARs), and maternity clinics. The replicat￾ed indices can be found below. LIMITATIONS Available resources constitute the primary limitation to this exercise. The timeframe and budget for this evaluation precluded more rigorous, quantitative evaluation techniques. The team was only able to visit five municipalities, out of the [42] municipalities in which the program operates. The five municipalities were chosen as a convenience sample, and are by definition nonrandom and nonrepresentative. As such, the evaluation team cannot generalize results across the entire population. Moreover, due to a compressed fieldwork schedule, the team spent one to two days in each municipality and replicated the indices concur￾rently or consecutively with other evaluation data collection activities. This potentially led to key informants not being available or, particularly when the evaluation team administered the questionnaire after a focus group, response fatigue. 65 Deloitte. “Transparent Local Governance and Improved Service Delivery Performance Monitoring Plan Septem￾ber 2011 – September 2016.” August 17, 2012. H-2 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS Another serious limitation to the exercise is the reliability of respondents. Unsurprisingly, given the rural demographic makeup, many respondents had extremely low levels of education and several were illiterate. Parts of both indices, however, are quite technical or require detailed, specific information. At times, the evaluation team was not confident that the respondents understood the question entirely, despite efforts made by the evaluation team. In these cases, the evaluation team has eliminated these responses. The evaluation team would like to emphasize that this is not a statistically significant or representative sam￾ple. This exercise is only intended as a “dipstick” to provide a preliminary, cursory, analysis of USAID/NEXOS’s indices. Since we replicated the indices using the same methods and interviewing the same categories of individuals as USAID/NEXOS, our replicated data is necessarily subject to the same limi￾tations we describe below. Moreover, without a representative sample, and given the limitations faced, we cannot make conclusive claims about accuracy of any particular index value. That is to say, we cannot say for certain what the “true” value of the index should be and make no claims that our replicated data is correct. In fact, given these challenges, it is probably not. Nevertheless, we are confident that our examination pro￾vides USAID with additional insight and understanding of the indices’ design and data collection. INDEX DESIGN66 The water index comprises three variables: (1) Technical, including service coverage and quality; (2) Civil Society, including the participation of civil society in service planning and management, and social control of providers; and (3) Institutional Strength, including the capacity of rural and urban service providers, the Mu￾nicipal Water and Sanitation Commissions (Comisiones Municipal de Agua y Saneamiento, or COMAS), the Monitoring and Local Control Units (Unidades de Supervisión y Control Local, or USCL), and the municipal government as the service provider. To populate the water index, USAID/NEXOS convenes focus group members from the following institutions: • Municipality, e.g., the mayor, vice-mayor, municipal secretary, technical representative from the Mu￾nicipal Environment Unit (Unidad Municipal Ambiental, or UMA), and the Community Develop￾ment [Promoter/Sponsor] (Promotor de Desarrollo Comunitario); • Commonwealth (mancomunidad), e.g., a coordinator from the Intermunicipal Technical Unit (Uni￾dad Técnica Intermunicipal, or UTI) or health and water technical representative; • COMAS, • USCL, e.g., civil society representatives and technical regulation and control representatives (e.g., Técnico de Regulación y Control or TRC); • Municipal Patronato Association (Asociación Municipal de Patronatos); • Service provider in the municipal capital, e.g., manager, administrator, technical officer; • Municipal Association of Water Boards (Asociación de Juntas de Agua y Alcantarillado Municipal, or AJAAM); and • Citizen Transparency Commission (Comisión Ciudadana de Transparencia), and the Municipal Commissioner (Comisionado Municipal).67 66 Information in this section is from USAID/NEXOS’s Performance Monitoring Plan and instructions for the applica￾tion of the health and water indices. It aims to provide the reader with an understanding of the construction of the indices and inform later sections. 67 USAID/NEXOS, however, did not use a focus group to populate the entire index. Rather, focus groups were used to answer questions on the following topics: service quality, participation of civil society, institutional strength of COMAS, USCL, and municipalities. H-3 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS As stated above, the health index aims to measures the level of decentralized health services in a municipali￾ty. The index only applies to C and D municipalities with decentralized health services, 32 municipalities in all. According to USAID/NEXOS’s PMP, the index “represents a measurable indicator of a municipality’s commitment to the improvement or expansion of local services (particularly on the part of the service ad￾ministration and civil society).”68 The health index serves as an impact indicator, overall Like the water index, the health index comprises three variables: (1) Technical, which assesses healthcare infrastructure, access, coverage, and quality, (2) Social, which measures the participation of civil society in service planning, decen￾tralized service management, and in social control of the centralized service, and (3) Institutional, which measures the capacity of the service manager, the service provider, and the municipal government as the service owner. The health index has five sub-areas of evaluation: (1) manager of health services, (2) municipality, (3) health service provider, (4) technical aspect, and (5) social aspect. To populate the index, USAID/NEXOS con￾venes the following individuals, according to each sub-area: • Manager of Health Services, including representatives of the mancomunidad, the municipality, or patronato, depending on who manages health care; • Municipality, including the mayor and municipal council members, • Health Service Provider, including the technical coordinator for health, the coordinator of health promoters, community committees or health support committees, the sectorial board, and local health-related NGOs linked to health topics; • Technical Aspect, including the technical coordinator for health, the coordinator of health promot￾ers, community committees or health support committees, the sectorial board, and local health￾related NGOs linked to health topics; and • Social Aspect, including the health manager for the municipality (i.e., the president of the manco￾munidad, mayor, or president of the council), coordinator of the UTI, technical coordinator for health, community committees for health, support committees, civil society representatives on the health sector board, municipal commissioner, citizen transparency commissions, and citizen trans￾parency commissions. For both indices, technical and institutional variables are weighted 30% each, and the social variable weighs 40%. Values range from 0% to 100%, with 100% representing optimal conditions for service provision. Both indices are also only applied to C and D municipalities, the poorest and least developed municipalities in Honduras. METHODOLOGICAL LIMITATIONS Quantitative data provides program managers with a useful tool to easily assess program progress and im￾pact, without having to examine all 53 components of the water index, for example. Understanding that the index increased five percentage points over the baseline paints an unequivocal picture of program impact. The quantitative nature of the index also allows program staff to track progress over time and compare results among municipalities. Quantitative data, however, often confers an aura of objectivity, which may or may not be accurate. It all depends on the quality and reliability of the data. Focus groups are an inherently subjective, qualitative data collection method. As such, using focus groups to populate an objective, quantita￾tive indicator presents several issues, many of which are discussed below. Another substantial problem with the indices is the lack of independent data verification. USAID/NEXOS does not use additional sources of information—such as official government statistics, public opinion data, or information collected by local water boards or health facilities—to triangulate index results. Data verification 68 NEXOS PMP. H-4 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS would bolster confidence in the indices and substantiate findings. The evaluation team, however, acknowl￾edges that there is a dearth of reliable data in Honduras on health and water indicators disaggregated by municipality. Moreover, while several questions in the water index are calculated based on USAID/NEXOS program data, the source of this data does not appear in the PMP or in the instructions for the application of the health index. Instrument Design Bias in instrument design leads to measurement error. As written, several elements of the indices are vague and open to interpretation.69 For example, in the water index, judging the extent to which civil society is incorporated in decisions relating to prioritization and implementation for investment in improvements and expansion of coverage is inherently subjective. One of the responses to this question, that civil society is “eventually” (eventualmente) incorporated into municipal decision-making, is vague and imprecise. Re￾spondents’ definitions and conceptions of involvement, beliefs about the role of civil society, and previous experience will influence this assessment. Other index elements are similarly confusing. For example, in the technical component of the health index questionnaire there is no clear logical distinction between (a) hav￾ing serious difficulties accessing at least one health facility in the winter, and (b) in the winter, access is inter￾rupted on the rainiest days in at least one health facility. It stands to reason that if access to a particular healthcare facility is interrupted in the winter on the rainiest days (b), then there would be serious difficulties accessing that facility in the winter (a). Similarly, respondents were often not sure what constituted a con￾nection to potable water and were not sure whether to include households that have had their service cut off or to only count active connections. Furthermore, in certain cases, index responses are not mutually exclusive and exhaustive. For instance, the water index does not allow for 100 percent of providers to chlo￾rinate the water; the highest response possible is 90 percent (hasta el 90% de los prestadores cloran el agua). In addition, it is impossible to know whether increases in the health index—better management, service provision, and health outcomes—are a result of USAID/NEXOS program activities or attendant benefits from decentralization in healthcare more generally. Without a credible counterfactual, we do not know if the index would have still increased in the absence of the program or not. Approximately 85 percent of the Honduran public is literate. 7071 Not only is the country’s literacy rate among the lowest in Latin America, but this statistic also obscures significant regional differences. According to the World Food Program, the illiteracy rate is 27% in rural areas, compared with 9.6% in urban zones.72 Moreover, in contrast to local government officials and health care providers, representatives from civil soci￾ety and other community members are more likely to be illiterate. In several instances, however, the indices request specific numbers or percentages or ask about the existence of various manuals, regulations, and legal documents. 69 We do not have the focus group questionnaires for the health index and thus cannot make judgments about its quality. 70 World Bank. Literacy rate, adult total (% ages 15 and above). World Bank Development Indicators. 2011 http://data.worldbank.org/indicator/SE.ADT.LITR.ZS. Note: Definitions of literacy also encompasses numeracy, the ability to make simple arithmetic calculations. 71 UNICEF. Honduras Statistics. 27 December 2013. http://www.unicef.org/infobycountry/honduras_statistics.html 72 World Food Program. Honduras. http://home.wfp.org/stellent/groups/public/documents/op_reports/wfp228650.pdf H-5 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS Data Collection Biases In this section, we highlight potential biases in data collection. With the information available, we cannot definitively say that these biases happened, only acknowledge their potential. All data collection, however, is subject to bias and the existence of bias does not necessarily mean it cannot or was not mitigated or that the whole index is necessarily flawed. One source of potential bias, known as “recall bias,” is respondents’ inability to accurately remember past details. The questionnaire used to populate the index asks respondents to recall specific numbers. For the water index, for example, respondents are asked in the spring73 to recall the number of weeks per month, days per week, and hours per day that water is available in the summer, on average. For the health index, respondents are asked to recall the number of preventable neonatal deaths for a given year, as well as the number of doctors, nurse practitioners, and nursing assistants for a given population. By asking respondents to remember specific, often quite detailed information, the indices potentially jeopardize data accuracy and completeness. The focus groups are also subject to response bias, a type of cognitive bias where respondents answer questions in the way they think the interviewer wants them to rather than according to their true beliefs. Response bias is more acute when the respondents know that the data being collected are for donor agen￾cies and thus will likely influence and inform future financial or in-kind assistance. For example, thinking that their responses were potentially linked to future assistance (whether or not this was true) might have led respondents to answer inaccurately or incompletely. Similarly, the focus groups might also be subject to social desirability bias, where respondents have a tendency to answer questions in a manner that will be viewed favorably by others, typically over-reporting “good” behavior (e.g., chlorinating water) and under￾reporting “bad,” or undesirable behavior (e.g., excluding civil society in municipal decisions). Due to the focus group’s length, the indices could also suffer from respondent fatigue. When a survey or, in this case, a focus group, takes a long time to complete, respondents become bored, tired, and/or disinter￾ested. With waning attention and motivation, respondents become less engaged and are more likely to answer “don’t know,” give perfunctory answers, give the same response to each question, or stop answering the questionnaire altogether.74 According to USAID/NEXOS’s instructions for application of the water in￾dex, the focus groups are estimated to have an average duration of three hours, though this time could be reduced or expanded, depending on the focus group members. Focus groups also have the potential to be unrepresentative and subject to selection bias. Though USAID/NEXOS has attempted to mitigate this bias by rescheduling focus groups unless 70 percent of the participants are available, the sample is vulnerable to undercoverage and nonresponse bias. This can poten￾tially bias results if the people who are not included in the focus group differ in a substantive way from par￾ticipants, e.g., people who live farther away from the town center, who are more likely to be poorer and more marginalized. Focus group dynamics also have the potential to bias index results. Power relationships, social standing, per￾sonalities, and interpersonal histories can influence participants’ responses. The team saw this first-hand in several locations. For example, in San Manuel Colohete, the evaluation team conducted a focus group with several water board presidents, among other individuals. With 16,000 inhabitants (1,500 inhabitants within the town center), San Manuel Colohete is a very small, extremely rural municipality. It was clear that the focus group participants knew each other very well, and often looked at each other before answering. In 73 Data collection occurs April – June, with results reported in September. Transparent Local Governance and Im￾proved Service Delivery Performance Monitoring Plan: September 2011 – September 2016. August 17, 2012. 74 Encyclopedia of Survey Research Methods. Lavrakas, Paul J. ed. Thousand Oaks, CA: SAGE Publications. 2008. H-6 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS addition, the mayor is often included in focus groups used to populate the indices. Given the mayor’s stand￾ing in the community and the power he or she wields over the day-to-day life of his or her constituents, the mayor’s presence in a focus group has the potential to bias results. Also known as dominant responder bias, or dominance bias, some participants may also dominate the conversation and try to convince, either con￾sciously or unconsciously, other respondents to answer in a certain way. Similarly, in a focus group, people with the strongest opinions are more likely to weigh in on a particular topic, thereby potentially skewing results. RECOMMENDATIONS A mid-term evaluation offers USAID and Deloitte a unique opportunity to assess implementation and per￾formance and make changes or course corrections where necessary to improve the overall effectiveness of the program. Based on our findings, we present a set of recommendations below. • Independent data verification. To the extent possible, USAID/NEXOS should use external data sources to verify data used to populate the indices. Triangulation would serve to minimize bias and strengthen the validity of the indices. Several independent data sources exist that Deloitte could leverage to act as a check on the indices. For example, maternal and neonatal mortality figures are available at each health clinic and, presumably, from the Ministry of Health. (Although the team was not able verify this or find GOH health data disaggregated by municipality, health clinics must report this data to the Ministry of Health, so this possibility exists at least in theory.) Health clinics also col￾lect information on several other health indicators, such as disease prevalence and contraception use, which could paint a broader picture of local decentralized healthcare services. USAID/NEXOS and the Mission could also construct a map of existing municipal health and water data, and identify ways the program could leverage this data to triangulate index results. USAID/NEXOS could also work with other local institutions to collect salient data. The Regulatory Body for Health and Sanitation Services (Ente Regulador de los Servicios de Agua Potable y Saneamiento, or ER￾SAPS) already collects data in several other municipalities on water indicators, such as water and sewer coverage, chlorination, average taxes, and the legal status of water boards, among other indicators.75 Other cost-efficient data collection alternatives include adding questions on an omnibus or other public opinion survey or a Computer-Assisted Telephone Interview (CATI). Honduras has a high mobile penetration, with the World Bank and others estimating mobile penetrations at 93 percent.76 77 Notable, however, these options are not without their limitations, particularly in the poorest, most remote areas. • Reformulate Performance Monitoring Plan. Due to the significant challenges described above, USAID and Deloitte should not rely on the health and water indices as indicator of program im￾pact. While the indices provide valuable granular information on several salient topics, for the rea￾sons stated above, they are not good indicators of program impact. 75 ERSAPS. Indicadores 2012: Sector Agua y Saneamiento. 2012. http://www.ersaps.hn/documentos/interes/Indicadores%202012.pdf 76 World Bank. Mobile cellular subscriptions, per 100 people. World Bank Development Indicators. 2012. http://data.worldbank.org/indicator/IT.CEL.SETS.P2 77 Lancaster, Henry. “Honduras – Telecoms Mobile and Broadband: Market Insights and Statistics. BuddeComm. May 2014. https://www.budde.com.au/Research/Honduras-Telecoms-Mobile-and-Broadband-Market-Insights￾and-Statistics.html?r=51 H-7 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS • Comprehensive Data Audit. Due to the shortcomings in methodology and design described above, the evaluation team recommends a comprehensive audit to assess the validity and accuracy of the data and methodology. While the present exercise cannot generalize findings or make definitive claims about the accuracy of a particular index value, however, enough red flags have been raised that the evaluation team harbors significant doubts about the veracity of the index and the sound￾ness of its design. I-1 MID-TERM PEFORMANCE EVALUATION OF THE NEXOS AND USAID/DEE PROJECTS ANNEX I: DATA VALIDATION EXERCISE (USAID/NEXOS PRO￾JECT) Results of Data Validation Exercise *In Nueva Frontera, the municipality was not able to provide the evaluation team information on municipal income. The technical staff was new and did not know how to access the information. J-1 MID-TERM PEFORMANCE EVALUATION OF THE USAID/ NEXOS AND USAID/DEE PROJECTS ANNEX J: INDICATORS (USAID/ DEE PROJECT) INDICATOR TYPE OF INDICATOR COMMENTS AND RECOMMENDATIONS Result 1: ENABLING ENVIRONMENT FOR DECENTRALIZED SERVICES STRENGTHENED NO INDICATORS IDENTIFIED AT THIS LEVEL Sub-Result 1.1: COMPREHENSIVE LEGAL FRAMEWORK FOR INCREASED MUNICIPAL AUTONOMY STRENGTHENED 17. Total resources managed by local government as percentage of total public resources Outcome Good indicator. Measures total amount of resources transferred to local gov￾ernments. But the indicator is not nec￾essarily attributable to FOPRIDEH’s activ￾ities. 18. Percentage of laws and reforms passed by the GOH and drafted with USG assistance to promote de￾centralization compared to estab￾lished initiatives. Outcome The result –passing laws and reforms—is outside USAID and FOPRIDEH’s manage￾able interests. More laws do not mean better laws. As currently formulated, indicators moti￾vate the drafting of more laws regarding of quality or implementation. This is a bad indicator to measure results of FOPRIDEH’s activities 19. Index of Implementation of the Municipal Administrative Career Law by Local Government to pro￾mote municipal autonomy Outcome The Municipal Administrative Career Law, like many other laws in the country, remains in legal limbo. It has not been implemented. The indicator has not been measured. Not clear how the index has been con￾structed Sub-Result 1.2: LEGAL FRAMEWORK FOR MUNICIPAL FISCAL AUTONOMY STRENGTHENED 1. Index of fiscal municipal autonomy of municipalities (own source reve￾nue/total income ) Outcome Good indicator measuring efforts of mu￾nicipalities to increase their own source revenue. Data validation exercise, however, showed inconsistencies between data J-2 MID-TERM PEFORMANCE EVALUATION OF THE USAID/ NEXOS AND USAID/DEE PROJECTS reported by FOPRIDEH and official data reported by municipalities to the Tribu￾nal Superior de Cuentas. See Annex H. 2. Percentage of laws and reforms passed by the GOH and drafted with USG assistance to promote fiscal au￾tonomy compared to established ini￾tiatives Outcome The result –passing laws and reforms—is outside USAID and FOPRIDEH’s manage￾able interests. More laws do not mean better laws. As currently formulated, indicators moti￾vate the drafting of more laws regarding of quality or implementation. This is a bad indicator to measure results of FOPRIDEH’s activities Same indicator used to measure sub￾result 1.1. Indicates these two sub￾results are not distinct from one another. 3. Score of implementation of legisla￾tion to promote fiscal autonomy, implemented by the municipalities Outcome This indicator is poorly articulated. It measures reduction of debt arrears (mo￾ra) or recovery of unpaid municipal tax￾es. The indicator is a good indicator of fiscal autonomy but needs to be re￾phrased to convey what it is measuring. Sub-Result 1.3: CIVIL SOCIETY PARTICIPATION IN THE DECENTRALIZATION PROCESS INCREASED 1. Number of representatives from civil society organizations who contribute to project activities Output It is not clear what “contribute to project activities” means. If it is measuring number of representatives from civil society organizations who participate in project activities, it should be rephrased. 2. Percentage of civil society percep￾tion on decentralization and munici￾pal autonomy Outcome Good indicator, but measurement is subjective and value of the indicator questionable. It is not based on a survey but rather on opinions collected infor￾mally in project group activities. F INDICATORS 1. Number of laws and amendments promoting decentralization drafted with USG assistance Output Good output indicator 2. Number of individuals who received USG assisted training, including management skills and fiscal man￾agement, to strengthen local gov￾Output Good output indicator J-3 MID-TERM PEFORMANCE EVALUATION OF THE USAID/ NEXOS AND USAID/DEE PROJECTS ernment and/or decentralization K-1 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ANNEX K: DATA VALIDATION EXERCISE (USAID/DEE PROJECT) As the table above shows, FOPRIDEH under reported total own source revenue in 2010 and over report￾ed own source revenue in 2013. The data above refers to only seven municipalities in the Mancomunidad of MANSUCOPA where FOPRIDEH conducts activities. L-1 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS ANNEX L: DOCUMENTS REVIE￾WED Banco Interamericano de Desarollo (BID). Estudio de Caso 04/2010: Perfil del Sector Municipal en la Repu￾blica de Honduras. Sector de Capacidad Institucional y Finanzas, Division de Gestion Fiscal y Muni￾cipal (ICF/FMM). Proyecto RG-K1100. 2010. El Congreso Nacional. Asociacion de Municipios de Honduras (AMHON). Ley de Municipalidades y Su Reglamento. Decreto Numero 134-90. 1993. El Instituto Nacional de Estadística (INE). Honduras: Encuesta Nacional de Demografía y Salud 2011-2012 (ENDESA 2011-2012). 2012. Ente Regulador de los Servicios de Agua Potable y Saneamiento (ERSAPS). Indicadores 2012: Sector Agua y Saneamiento en Honduras. 2012. Instituto Nacional de Estadísticas (INE). Encuesta Permanente de Hogares de Propósitos Múltiples (EPHPM) Mayo 2013. 2013. Partido Nacional de Honduras. El Plan de Todos: Para Una Vida Mejor. 2013. Pinel, Arturo Bendana et al. Avances en Salud: 2010-2011. Government of Honduras: Ministry of Health, 2011. United Nations Development Programme (UNDP). "Explanatory note on 2013 HDR composite indices: Honduras." In Human Development Report 2013. The Rise of the South: Human Progress in a Di￾verse World. 2012. United Nations Development Programme (UNDP). Informe sobre Desarrollo Humano Honduras 2011 (Reducir la inequidad: un desafio impotergable). 2012. United States Agency for International Development (USAID). Decentralization Enabling Environment (USAID/DEE): Semi-Annual Performance Report (January - June 2013). 2013. United States Agency for International Development (USAID). Democracy, Human Rights, and Governance Assessment of Honduras. Prepared by Tetra Tech ARD. March 2013. United States Agency for International Development (USAID). Transparent Local Governance and Im￾proved Service Delivery: Semi Annual Technical Performance Report. 2013. United States Agency for International Development (USAID). La Federación de Organizaciones No Gu￾bernamentales para el Desarrollo de Honduras (FOPRIDEH). Informe Final: Marco Legal de la Des￾centralizacion Fiscal en Honduras. 2012. United States Agency for International Development (USAID). Political Culture of Democracy in Honduras and in the Americas, 2012: Towards the Equality of Opportunity. April 2013. World Bank. Honduras: Public Expenditure Review. Report No: ACS6720 [82662-HN] Republic of Hondu￾ras. August 2013. World Bank. International Development Association and International Finance Corporation Country Part￾nership Strategy for the Republic of Honduras for the Period FY2012-2014. World Bank, 2011. L-2 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS World Health Organization. Country Cooperation Strategy: Honduras. WHO/CCO/06.04/Honduras. World Health Organization, 2006. L-3 MID-TERM PEFORMANCE EVALUATION OF THE USAID/NEXOS AND USAID/DEE PROJECTS U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov USAID | NEXOS Torre Alfa, Colonia Lomas del Guijarro Tegucigalpa, Honduras July 15th, 2014 Lic. Anais Henriquez Mission Economist USAID/Honduras Subject: Transparent Local Governance and Improved Service Delivery/ USAID|NEXOS. Contract # AID 522 – C – 11 – 00002. Dear Ms. Henriquez: We have received and reviewed the Mid-Term Evaluation prepared by Democracy International (DI ) of the Transparent Local Governance and Improved Service Delivery Project ( USAID|NEXOS). Even though the Evaluation Team recognizes that the time allowed to them for the evaluation of 2 projects was extremely short, and that they didn’t have time to visit most of the USAID|NEXOS Target municipalities (the Team only visited 5 out of more than 40 municipalities covered by the project), the evaluation provides insights and recommendations that may prove valuable for modifications to the Scope of the project and other improvements that USAID may wish to include in the USAID|NEXOS project. To assist with that determination of what changes to introduce into the Project, we would first like to clarify certain findings or conclusions in the Evaluation in order to better inform decisions concerning potential changes to the Scope. Those areas specifically are as follows: a. The Evaluators place significant emphasis on the PMP Indicators and their classification, especially the distinction between Outcome and Output Indicators. To clarify, Outcome Indicators are the result of the Output Indicators and should not be treated independently. We welcome a broader discussion with USAID to further refine the existing Output Indicators and Outcomes however we don’t recommend changes to the Outcome and Output Indicators at this time. b. The Evaluators state that Indicator 1 is based on an opinion survey that will be conducted at the end of the project. The survey is conducted every other year of the project and is presently taking place. The results of the first survey appear not to be factored into the Evaluators analysis. c. Concerning the Health and Water Indexes, we regret that the Evaluators weren’t able to thoroughly review the issues and circumstances that led to the creation of these indexes. We believe that had they understood the issues and background, they would have asked for additional information, and the comments would have been substantially different. However, we welcome the audit to review the accuracy and reliability of the indexes. ANNEX M : STATEMENT OF DIFFERENCE - 2 - d. USAID|NEXOS coordinates at two levels with other programs and institutions: at the national and executive level in Tegucigalpa (where the national institutions are based), and at the operational level in the different locations of the projects. The coordination issue is discussed in depth in annex 1.3. e. USAID|NEXOS welcomes the evaluator’s recommendation to reduce the number of Target Municipalities and the new programmatic emphasis. We look forward to discussing this recommendation with USAID. f. Finally, we would recommend that for future evaluations that USAID share the full scope of the evaluation with the Projects being evaluated. This would enable the project teams to make sure that all the necessary information is available to the Evaluation Team to help them perform their analysis during the short time frame provided. USAID|NEXOS appreciates the opportunity to review and comment on the Evaluator’s report. In the attached annex you will find additional information as a result of our review. Sincerely, Sigifredo Ramirez USAID|NEXOS COP PROYECTO FACILITANDO EL MARCO REGULATORIO PARA LA DESCENTRALIZACIÓN (DEE) COMENTARIOS A LA EVALUACIÓN DE MEDIO TÉRMINO DEL PROYECTO DEE/FOPRIDEH 17 de julio 2014 Sobre las Conclusiones de la firma evaluadora: 1. Conclusión firma evaluadora. As defined by the PMP, USAID/DEE demonstrates poor indicator performance, with 70% of the project’s indicators falling below expectations as of December 2013. The project is “below expectations” on seven of ten indicators (1.1, 1.3, 2.1, 2.2, 2.3, 3.1, and 3.2). The project exhibits “excellent” performance in the number of individuals to have received USG-assisted training (F. Indicator 2). In 2013, 1,019 individuals received USG-assisted training, exceeding the established target of 250 by more than 400 percent. While this is an output indicator, it demonstrates that USAID/DEE is actively engaging beneficiaries and the sex-disaggregated data indicates equitable gender balance in program activities. The program also demonstrates “acceptable” performance in the percentage of laws and reforms promoting decentralization drafted with USG assistance (F Indicator 1) and passed by the GOH (Indicator 1.2). However, as these laws have not been passed by the National Congress as of this writing, Indicator 1.2 should have been one (one of these regulations was an executive decree that did not have to be approved by Congress). Comentario del Proyecto: Respecto a este comentario aceptamos no haber puesto parámetros en el PMP para medir los target. Ya que no es tan cierto que todos los indicadores estén por debajo de las expectativas ejemplo el indicador 1.1 se logró en el periodo indicado un 8 de 10 que para nosotros es aceptable, el indicador 2. 3 se logró 8 de 10 aceptable. En otro de los indicadores se lograron 458 de 500 aceptable (3.1) 17 de 20 aceptable (3.2). Nuestra AOTR los calificó de no aceptable porque no se previó en el PMP, situación ésta que estamos proponiendo cambiar en los ajustes que ya hemos enviado a la agencia mediante el establecimientos de rangos. 2. Conclusión firma evaluadora. To this end, FOPRIDEH provided technical assistance to financial municipal authorities to strengthen their capacity to generate own-source revenue. The program conducted trainings on several topics, including budget administration, accounting, tax and internal control, accountability, and other topics related to municipal finances. For example, USAID/DEE worked with the Superior Accounts Tribunal (TSC) and AMHON to conduct workshops, experience exchanges, and trainings with municipal officials.) officials. The project also worked with assisting the technical unit of the Mancomunidades Intermunicipal Technical Finance Unit (Técnico Unidad Financiera Intermunicipal, or UFIM) so they could provide support to the municipalities. This approach, on the other hand, not only strengthens local capacity but also builds a cadre of local technical expertise that is essential in ensuring project sustainability. However, the evaluation team questions the sustainability of these 1 trainings. The team did not find evidence of existing manuals and/or operation guides provided after technical assistance sessions. Comentarios del Proyecto: a) No se menciona que las actividades de asistencia técnica desarrolladas por el resultado responden a una línea muy fina que en este sentido aprobó la oficial de enlace de la USAID para apoyar a las municipalidades de forma específica en el tema de administración tributaria, esto no es mencionado en el informe según fue expuesto ante la Comisión Evaluadora de Democracia International llevada a cabo el jueves 15 de mayo 2014 b) Si bien es cierto y de acuerdo al convenio USAID/FOPORDEH, no se ha integrado en una caja de herramientas todos los manuales, guías, formatos, instructivos, etc) utilizados por cada uno de los resultados en sus distintas actividades, también es cierto que particularmente por parte del resultado 2, se han facilitado las herramientas necesarias a cada participante beneficiados con las asistencias técnicas y demás capacitaciones impartidas como apoyo a sus actividades, dichas herramientas en su mayoría han sido entregadas en formato digital y provienen de fuentes como la SDHJGD, el TSC, la AMHON así como de otros proyectos que se ejecutaron en el pasado como el Programa USAID / G￾TAG. Se reitera que el proyecto no cuenta con la disponibilidad presupuestaria para reproducir en formato impreso herramientas de apoyo. Este comentario tampoco es incluido en el informe y fue debidamente expuesto en la reunión llevada a cabo el jueves 15 de mayo 2014. 3. Conclusión firma evaluadora However, despite the program’s efforts, FOPRIDEH did not achieve the sub-result’s overall objective – increasing municipal fiscal autonomy. In fact, according to DEE’s semi￾annual report, fiscal autonomy decreased 39 percent between 2012 and 2013. Moreover, there are widespread inconsistencies between municipalities’ official income records and income reports presented to the Supreme Audit Institution, the Tribunal Superior de Cuentas. In fact, a limited data validation exercise on income information in seven municipalities revealed that there are broad inconsistencies between income data reported by DEE and official TSC records. For a graphic representation of this exercise please see Annex K. Comentarios del proyecto: a) Con respecto a la disminución en el indicador 2.1 de autonomía financiera es importante señalar que la justificación a esta disminución se explica en el informe semianual de Julio a Diciembre 2013 en donde para este año en mención los incrementos en los ingresos de capital fueron mayores que en el 2012 producto del aumento en la transferencias sobre todo debido a los pagos o remanentes del 2012 pagados en el 2013 así como otros ingresos de capital percibidos por las municipalidades como ser subsidios y donaciones que se originan en algunos casos a los aportes adicionales, no así en los ingresos propios con un comportamiento contrario a los ingresos de capital es decir los ingresos propios disminuyeron en el 2012 y aumentaron en el 2013, sin embargo, lo más 2 importante aquí es destacar la marcada diferencia entre los ingresos propios en el numerador que forma el indicador y los ingresos totales del denominador que hacen que determina el crecimiento o disminución de este indicador de forma comparativa ente los años 2012 y 2013. Es importante mencionar que para el 2013 el incremento en los ingresos de capital particularmente en los renglones de subsidios del congreso nacional responde en algunos de los casos a compromisos de carácter político provocados por las elecciones generales llevadas a cabo en ese año. b) En relación a las inconsistencias encontradas, el DEE ha tomado siempre los datos proporcionados por las municipalidades que en todo caso deberían de ser los mismos del TSC o sea basados en las Rendiciones de Cuentas, sin embargo, y tal como se expuso en la reunión del 15 de mayo del 2014, los datos siempre varían debido a las siguientes y principales razones: el documento de Rendición de Cuentas del TSC es modificado en la municipalidad sin ser reportado al TSC, la municipalidad cuenta con varias versiones de la Rendición de Cuentas que han sido ajustadas por diferentes motivos, la diferencia en los tiempos de aprobación de la rendición de cuentas del TSC ha provocado atrasos al DEE para la recopilación de los datos en algunas municipalidades optando por recolectar dichos datos de forma directa de los sistemas de información que los generan, la información provista por el SAMI mostró ciertas debilidades con la opción de consolidación anual de las rendiciones trimestrales provocando atrasos y tomando la decisión de recolectar los datos de los sistemas de información existentes en cada municipalidad, siempre ha existido diferencia entre la información del TSC y la SDHGJD debido a los ajustes en los informes de proyectos. 4. Conclusión firma evaluadora Activities under this sub-result had a mixed result with beneficiaries. In fact, mayors expressed great satisfaction with DEE’s capacity building and technical assistance efforts. In fact, some mayors said that DEE’s technical assistance and quality of their technical experts is the best they have ever received (“lo mejor de lo mejor”) . Mayors in particular appreciated the use of a lawyer to recover tax arrears. On the other hand, however, while technical municipal staff appreciated the assistance described above, they noted that they had received similar trainings in the past and complained that FOPRIDEH did not conduct a needs assessment before the technical assistance and trainings. Comentarios del Proyecto: a) No es aceptable que se mencione en el informe que los técnicos municipales afirmen haber recibido los cursos de capacitación con anterioridad y entrar así en redundancia y en desperdicio de tiempo y recursos dejando un mal testimonio de la capacidad de coordinación del DEE en particular del Resultado 2, por consiguiente, se rechaza tal posición y tal como se expuso durante la reunión del 15 de mayo se reitera que todas las capacitaciones fueron realizadas en base a las demandas de necesidades en cada municipalidad que fueron debidamente comprobadas. Una de las formas de comprobar los requerimientos de capacitación es la aplicación de una evaluación de entrada que se acostumbra practicar a todos los participantes previo al inicio de la capacitación en donde en la mayoría de los casos se evidencia derivado de los malos resultados la 3 necesidad de la capacitación. Por otra parte, también se aplica una evaluación de salida al final de la capacitación en donde los resultados de forma contundente por un lado, confirman la necesidad que existía de realizar la capacitación y por otro lado, exponen la calidad de la enseñanza brindada y la capacidad de recepción de los capacitados derivado de los resultados favorables de dicha evaluación en contraste a los resultados de la evaluación de entrada. Por lo anterior deja muy en duda que los entrevistados en MANSUCOPA hallan afirmado tal posición y quizás ha sido un mal planteamiento de la pregunta hecha por los evaluadores o el aprovechamiento de los técnicos municipales para ampliar más los eventos de capacitación grupal que el proyecto en una línea muy fina ha venido desarrollando. b) Con relación a la afirmación de los técnicos de demandas de recursos como equipo de cómputo es importante mencionar que no obstante la naturaleza del proyecto en hacer incidencia política y contando con el visto bueno de la oficial enlace del Proyecto (AOTR) y en el marco de una estrategia de estímulo a las municipalidades y mancomunidades para un mayor acompañamiento a las acciones impulsadas por el proyecto, se realizaron donaciones de equipo con muchas esfuerzos debido a las restricciones presupuestarias. La dotación se realizó de forma congruente a un análisis de identificación priorizada de necesidades de forma coordinada con cada una de las Unidades Técnicas Intermunicipales de las mancomunidades. Aquí también es importante señalar que las respuestas de los técnicos municipales en el sentido de ver como un obstáculo no contar con equipo de cómputo son acertadas en algunos de los casos pero para otros se presenta como una oportunidad para hacer una solicitud indirecto de apoyo en este sentido, tal como se comentó en la reunión del 15 de mayo. Sobre las Recomendaciones de la firma evaluadora: 1. Results Framework and PMP. To the extent feasible, FOPRIDEH should revise its PMP and redefine its results and indicators. USAID and FOPRIDEH should consider establishing more realistic targets for program indicators. Sub-results (particularly Sub-results 1 and 2) should also be renamed to refer to advocacy, municipal financial autonomy, and civil society to improve clarity. While creating new outcome indicators might be impractical at this point, USAID/Honduras should ensure that follow￾on programs have a clearly defined results framework, with distinct outcome indicators for each result that are direct, measurable, timely, practical, and attributable. Comentarios del Proyecto: a) La evaluación intermedia del Proyecto ha recomendado redefinir sus resultados e indicadores, el proyecto considera que a estas alturas a más de tres años y medio de ejecución del proyecto no está de acuerdo ya que significaría realizar cambios sustanciales en el Convenio y PMP, lo cual no es conveniente ni para la agencia ni para FOPRIDEH/DEE. b) Ante la situación planteada se hace necesario reflexionar sobre un cambio de dirección o de ruta respecto al Proyecto, es decir, se justifica realizar un 4 cambio en el PMP en cuanto al indicador 1.2 del Resultado 1 y el indicador 2.2 del Resultado 2. Cambio no de redacción sino de interpretación, esto lleva a cambiar la tabla de target como se refleja en la propuesta de cambios en el PMP que hemos presentado a la agencia. c) En términos de ajustes destaca la inclusión en todos los indicadores de una tabla que facilita la medición final del indicador valorando el cumplimiento actual vrs los targets, por otra parte, el diseño ajustado del PMP propone modificaciones a la definición del indicador 1.2 donde el proceso de construcción y los targets del indicador incluye acciones de incidencia a nivel nacional y local que promueven la descentralización y la autonomía municipal, aquí destacan los siguientes temas: Ley de Municipios, Ley de Descentralización, Manuales inherentes a la Ley CAM, Reglamento de la Ley CAM y Ley de Contratación del Estado; en el caso del indicador 2.2 el proceso de elaboración y los targets se basa en la asistencia técnica y las acciones de incidencia que se realizarán en busca que las municipalidades cumplan en tiempo y forma con la elaboración y aprobación de los informes trimestrales de rendición de cuentas GL, la Elaboración y Aprobación de los Informes de Proyectos y la Aprobación de los procesos de Apremio Judicial. 2. Advocacy. USAID should convene the Foro Nacional de Descentralización to discuss the status of the different decentralization laws before the National Congress. The donor community should also encourage a dialogue between the new government authorities and supporters of decentralization laws and come to a new agreement on a package of legislation that can be passed successfully by Congress. This dialogue could be moderated by neutral international experts who could help achieve consensus between opposing parties to arrive at a final version of the decentralization law that can be voted on. Comentario: Foprideh no es la llamada a convocar al Foro Nacional de Descentralización, por reglamento aprobado entre la partes o sectores que conforman el Foro (AMHON, FOPRIDEH, GSC, GOBIERNO CENTRAL y otras) es la Secretaria del Interior hoy Secretaría de Derechos Humanos Justicia y Descentralización la encargada de convocar al Foro y en el momento que se hizo la evaluación no había ninguna razón o justificación de peso para realizar dicha convocatoria puesto que las nuevas autoridades del gobierno de turno todavía estaban en análisis de que hacer respecto al proceso de descentralización en el país. 3. Robust Public Awareness Campaign. DEE should greatly expand its public awareness efforts and a robust public education campaign should be established as part of follow-on programs. The Honduran public needs to become better informed about decentralization: what it is, its benefits, and citizens’ roles and responsibilities. Misconceptions and ignorance about decentralization not only stymie program efforts, but widespread popular support for decentralization laws in the legislature could foment political will and move the process forward. To reach out and engage a wider audience, FOPRIDEH should improving the definition of their key messages and their target audiences and consider social media as a possible medium. 5 Comentario: FOPRIDEH/DEE, durante los años 2012 y 2013, realizo una amplia campaña de difusión divulgación del proceso a través de medios nacionales y regionales de radio. Campaña que el último tercio del año 2013 fue extendido a medios televisivos estos últimos logrados en forma gratuita. FOPRIDEH/DEE en el tercer año de ejecución invirtió muchos recursos en talleres, jornadas etc, puesto que se tenía la idea que las leyes fueran aprobadas ese año con el apoyo de AMHON y las Secretarias de gobierno involucradas; sin embargo la aprobación de las mismas no dependió de FOPRIDEH a pesar de los enormes esfuerzos de cabildeo realizado entre los sectores. Por ahora el Proyecto no cuenta con recursos para realizar campañas de divulgación del proceso, pero puede gestionar a través de medios televisivos y radiales aunque no en forma sistemática. 4. Financial Autonomy. USAID/DEE’s model of working through the Mancomunidades is good and should be continued in follow-on programming to promote greater sustainability of project efforts. The project, however, should consider working with Mancomunidades to design a financial sustainability handover plan, such as charging a fee for services, to allow technical officials to continue providing training and technical assistance to municipal officials. In addition, USAID/Honduras should incorporate incentives to municipal authorities to improve their own-source revenue, such as conditioning further assistance on the achievement of particular benchmarks, in future assistance. Comentario: Esto también se comentó durante la reunión del 15 de mayo y tampoco se expresa en el informe de evaluación. La estrategia de sostenibilidad está basada en la apropiación de la UFIM de los procesos impulsados por el resultado 2, en tal sentido, en todas las asistencias técnicas y capacitaciones impartidas por el resultado 2 se ha provocado la participación de los técnicos UFIM para que estos pueda asumir el rol de asistentes técnicos locales, es importante recordar que el DEE logro que 6 de 8 municipalidades contrataran de forma permanente un técnico UFIM y con esto dar continuidad a la estrategia de sostenibilidad. Por otra parte, se ha promovido la contratación permanente de un profesional de derecho experto en apremio tributario judicial específicamente para trabajar en la MANSUCOPA lo que contradice cualquier posición en el sentido que no se cuenta con una estrategia de sostenibilidad para el tema relacionado con la autonomía financiera. 5. Civil Society. Where possible, USAID/DEE should follow up on its trainings and technical assistance programs offered to civil society organizations to ensure that agreements reached and operational plans designed in these sessions are implemented. Moreover, subsequent programming should tailor capacity building efforts to beneficiaries’ particular needs and feature participatory follow-on methods, such as tailored mentoring and coaching approaches, which will be more conducive to sustainability and partner ownership. Furthermore, civil society is weak and frequently 6 subject to the [inclination] of the presiding mayor, frustrating project efforts. FOPRIDEH should go beyond established local civil society organizations, like municipal transparency committees, to identify potential community leaders, such as patronatos, who could be more independent from government authorities. Future support could also create and strengthen the advocacy and capacity of a national-level mayors’ association to build support for decentralization. Since decentralization efforts are wholly dependent upon active citizen participation, future programming in this area should strengthen the capacity of civil society organizations. As civil society in Honduras remains underdeveloped and as local resources remain constrained, public pressure on the central government to legislate change is marginal at best. Legislative and executive buy-in and cooperation will be necessary to pass a law on decentralization and civil society organizations must develop the capacity aggregate and vocalize demands. Public education (as described above) and civil society capacity-building initiatives are, therefore, key inputs for successful decentralization. Comentarios: a) En lo referente al tema Participación de la Sociedad Civil el equipo evaluador enumera una serie de eventos ejecutados por el proyecto DEE, al respecto se refiere que no encontró muchas evidencias de que Foprideh brindaba un seguimiento para asegurarse de que las organizaciones implementaran las habilidades aprendidas. En nuestra opinión es muy general y no responde a las características de cada evento mencionado en el documento. Ejemplo: en el tema de Descentralización las acciones se enfatizaron en la socialización de los Proyectos Ley de Municipios y Ley de Descentralización constituyendo este el primer momento del proceso luego a manera de seguimiento nos preparamos para el segundo momento la capacitación sobre cada una de las leyes, lamentablemente el proceso por razones políticas tubo los retrasos por todos conocidos. b) Sobre los talleres de política pública cada uno está concebido para ejecutarse en dos o más módulos el primero para conceptualizar e informar sobre el proceso de formulación y el segundo para elaborar colectivamente con los mismos participantes del módulo anterior la política pública local y el establecimiento de un plan de acciones a implementar en el respectivo municipio donde se coordinen OSC y Gobiernos Municipales. c) Considerando la globalidad del proyecto se programó el 2014 y el 2015 para la ejecución de la segunda o tercera fase de los temas centrales. Ejemplo: Incidencia en Políticas Públicas, resolución de conflictos, procesos de negociación y las capacitaciones correspondientes al plan coordinado con el resultado dos, que resultan de los ajustes presupuestados considerados al inicio del 2014. 7 d) El evento de Incidencia realizado entre las comisiones especiales de AMHON y FOPRIDEH encaminado a impulsar la aprobación de la Ley de Descentralización, Ley de Municipios, Ley CAM y Reformas a las leyes ambientales respondieron a una perspectiva de proceso y se organizaron acciones debidamente planificadas cuyo seguimiento estuvo bajo la responsabilidad de la coordinación del resultado uno. e) Compartimos la idea que la sociedad civil es débil y con dificultadas para ser un actor de propuestas sólidas y contundentes. Además la percibimos muy politizada e ideologizada y consideramos que costará mucho construir un cuerpo cohesionado y coherente y con posiciones concertadas orientadas por un liderazgo con capacidad, compromiso y convicción. f) Es gratificante que los beneficiarios entrevistados hayan expresado satisfacción con el trabajo relacionado por la Federación. g) Aun con los pocos Recursos con que cuenta el proyecto FOPRIDEH/DEE seguirá contando e incorporando a sus afiliadas que trabajan el tema de desarrollo local, en las socializaciones de los instrumentos de ley que actualmente se está negociando y discutiendo con el gobierno central. Es preciso destacar que el proyecto se ha incorporado activamente en las nuevas discusiones promovidas por el gobierno central a través de la SDHGD para acordar un nuevo proyecto de descentralización 6. Strengthened Coordination. In order to avoid duplicating efforts, leverage limited resources, and promote institutional synergy, FOPRIDEH should with other programs, including other USAID programs. USAID/DEE should collaborate closer with USAID/NEXOS to improve and standardize technical assistance packages provided to technical municipal officials and civil society organizations. Both programs work municipal finance and should learn from one another and identify exchange and best practices. USAID/DEE should also coordinate with USAID/IMPACTOS on the design of social auditing strategies and tools. One particular area where additional coordination is necessary is advocacy. A broader constituency for decentralization is needed to effectively overcome lagging political will. By speaking with one voice, organizations would be better able to exert greater influence on the National Congress and advance decentralization. Specifically, the program would have benefitted from a large-scale and sustained public relations campaign throughout the course of the program to promote the idea of local as a public good, to help mobilize broad-based public opinion and activism around media reform, and to further influence public policy agenda-setting on this issue. By working with other stakeholders, FOPRIDEH would have been able to undertake more strategic and effective advocacy initiatives. Comentarios: a) Desde nuestro punto de vista en ningún caso hay duplicación de esfuerzos porque las áreas de cobertura entre los proyectos FOPRIDEH/DEE y 8 USAID/NEXOS son completamente diferentes no hay un tal solo municipio y mancomunidad que compartamos. Se entiende que hay duplicidad de esfuerzos y desperdicio de recursos cuando dos proyectos trabajan los mismos temas en los mismos municipios o mancomunidades. Si tenemos que reconocer que deberíamos estandarizar paquetes de asistencia en el tema de las finanzas municipales o al menos conocer que estamos haciendo cada quien y si es posible adaptar ciertas metodologías aunque esto es muy discutible. b) Por otra parte nuestro proyecto no trabaja el tema de auditoria social así que no entendemos por qué debemos coordinar con el Proyecto IMPACTOS. c) Abogacía. No estamos de acuerdo con lo que plantea la firma evaluadora porque desde nuestro punto de vista, hicimos lo que teníamos que hacer, incidencia legislativa, campañas por radio y televisión, talleres, jornadas de trabajo etc, que no se hayan aprobado las leyes, fuerzas extrañas y ocultas que desconocemos impidieron que se aprobaran dicho instrumentos de ley. 9 COMMENTS FROM USAID/DEE (ENGLISH TRANSLATION) 1. Executive Summary, Page xi Evaluator: As defined by the PMP, USAID/DEE demonstrates poor indicator performance, with 70% of the project’s indicators falling below expectations as of December 2013. The project is “below expectations” on seven of ten indicators (1.1, 1.3, 2.1, 2.2, 2.3, 3.1, and 3.2). The project exhibits “excellent” performance in the number of individuals to have received USG￾assisted training (F. Indicator 2). In 2013, 1,019 individuals received USG-assisted training, exceeding the established target of 250 by more than 400 percent. While this is an output indicator, it demonstrates that USAID/DEE is actively engaging beneficiaries and the sex-disaggregated data indicates equitable gender balance in program activities. The program also demonstrates “acceptable” performance in the percentage of laws and reforms promoting decentralization drafted with USG assistance (F Indicator 1) and passed by the GOH (Indicator 1.2). However, as these laws have not been passed by the National Congress as of this writing, Indicator 1.2 should have been one (one of these regulations was an executive decree that did not have to be approved by Congress). DEE’s Response: We recognize the absence of parameters in the PMP for the measurement of target achievement. However it is not entirely accurate that indicators have been accomplished below expectation; for example for Indicator 1.1, for the reported period, 8 of 10 was achieved which is acceptable according to us. Likewise, for Indicator 2.3, we achieved 8/10, which is also acceptable. For indicator 3.1, we accomplished 458 out of a 500 target, which is also acceptable; same case for 3.2 for which we reached a 17 out of 20 achievement. The AOR indicated this as non-acceptable because it was not considered in the PMP, which will be corrected in the framework of an amendment to the PMP, for the inclusion of ranges to measure success in the indicators. 2. Findings and Conclusions SUB-RESULT 1.2. LEGAL FRAMEWORK FOR MUNICIPAL FISCAL AUTONOMY STRENGTHENED, Page 34 Evaluator: To this end, FOPRIDEH provided technical assistance to financial municipal authorities to strengthen their capacity to generate own-source revenue. The program conducted trainings on several topics, including budget administration, accounting, tax and internal control, accountability, and other topics related to municipal finances. For example, USAID/DEE worked with the Superior Accounts Tribunal (TSC) and AMHON to conduct workshops, experience exchanges, and trainings with municipal officials. The project also worked with assisting the technical unit of the Mancomunidades (municipal associations) Intermunicipal Technical Finance Unit (Técnico Unidad Financiera Intermunicipal, or UFIM) so they could provide support to the municipalities. This approach, on the other hand, not only strengthens local capacity but also builds a cadre of local technical expertise that is essential in ensuring project sustainability. However, the evaluation team questions the sustainability of these trainings. The team did not find evidence of existing manuals and/or operation guides provided after technical assistance sessions. DEE’s Response a) The consultant did not mention that the referred technical assistance activities responded to specific circumstances during which municipalities received support on fiscal administration issues. This was approved by the AOR and was mentioned during an interview with the evaluators from Democracy International on Thursday, May 15, 2014. b) It is accurate to say that a toolbox has not been used for each of the different results and activities; however Result 2 in particular has provided beneficiaries with tools throughout trainings and technical assistance. Different tools have been delivered in digital format, including those from GOH institutions such as the Ministry of Human Rights, Justice, Decentralization and Governance, the Superior Court of Auditors (TSC), and others such as the Municipal Association of Honduras (AHMON) and other projects such as USAID / G-TAG. We wish to highlight that the project does not have an allocated budget for printed versions of these types of tools. This comment was not included in this report, even though it was informed during the May 15th meeting. 3. Findings and Conclusions SUB-RESULT 1.2. LEGAL FRAMEWORK FOR MUNICIPAL FISCAL AUTONOMY STRENGTHENED, Page 35 Evaluator: However, despite the program’s efforts, FOPRIDEH did not achieve the sub-result’s overall objective – increasing municipal fiscal autonomy. In fact, according to DEE’s semi-annual report, fiscal autonomy decreased 39 percent between 2012 and 2013. Moreover, there are widespread inconsistencies between municipalities’ official income records and income reports presented to the Supreme Audit Institution, the Tribunal Superior de Cuentas. In fact, a limited data validation exercise on income information in seven municipalities revealed that there are broad inconsistencies between income data reported by DEE and official TSC records. For a graphic representation of this exercise please see Annex K. DEE’s response a) The semiannual report for July –December 2013 provided an explanation of the reasons why indicator 2.1 decreased, primarily caused by the increase on capital revenue as a consequence of the transfers from the central to local governments, especially those owed since 2012 and paid until 2013. Other incomes included subsidies and donations. However, revenue collected from own sources showed a different behavior, decreasing in 2012 and increasing in 2013. It is important to highlight that the patent difference between “own source income” in the numerator and “total income” in the denominator, is responsible for determining the reduction or rise of the indicator when comparing 2012 and 2013. It is also important to mention that for 2013 some of the capital income came from subsidies from the National Congress, and responded to political commitments in the context of the General elections in November of that year. b) Regarding the reported inconsistencies, DEE has always used data provided by municipalities, which should be the same as those provided by the TSC based on the Accountability reports. Nevertheless, sometimes data varies when reports are modified by the municipalities without giving notice to the TSC. The time period for the approval of the accountability reports by the TSC makes it difficult for DEE to collect information from certain municipalities; therefore DEE has chosen to obtain data directly from the source information system. Nevertheless, the Integrated Municipal Administration System (SAMI in Spanish) displayed difficulties with the option of an annual consolidation of the quarterly accountability reports; which caused delays and led to the collection of data from existing information systems in each municipality. There have always been differences between the information from the TSC and the Ministry of Human Rights, Justice, Governance and Decentralization, due to adjustments made to projects reports. 4. Findings and Conclusions SUB-RESULT 1.2. LEGAL FRAMEWORK FOR MUNICIPAL FISCAL AUTONOMY STRENGTHENED, Page 35 Evaluator Activities under this sub-result had a mixed result with beneficiaries. In fact, mayors expressed great satisfaction with DEE’s capacity building and technical assistance efforts. In fact, some mayors said that DEE’s technical assistance and quality of their technical experts is the best they have ever received (“lo mejor de lo mejor”). Mayors in particular appreciated the use of a lawyer to recover tax arrears. On the other hand, however, while technical municipal staff appreciated the assistance described above, they noted that they had received similar trainings in the past and complained that FOPRIDEH did not conduct a needs assessment before the technical assistance and trainings. DEE’s Response a) DEE does not accept the report’s reference to technical municipal staff being previously trained in similar subjects and therefore a waste of time and resources; which is a negative reference to DEE’s coordination capacity, in particular for Result 2. During the May 15th meeting, we reiterated that all of the completed trainings were based on the demands from each municipality, which were dully verified. A way to corroborate training requirements is the application of an evaluation applied to all participants at the beginning of the process, which for the most part validates the need for a specific training. Additionally, an evaluation is applied at the end, which shows the teaching quality and the beneficiaries’ learning capacity, and compares results between the entry and exit evaluation. We believe there is reasonable doubt that those interviewed in MANSUCOPA made such affirmations, which might have been caused by a poorly stated question by the evaluators; or motivated by the desire from technical municipal staff for more training sessions. b) Regarding the demands from technical staff for computer equipment, it is important to mention that even though the nature of this project is to support advocacy efforts and that financial resources are limited, the AOR approved donations of such equipment to complement and motivate municipalities and Mancomunidades involvement in the projects’ activities. For this purpose an analysis of priority needs was done in coordination with Inter-Municipal Technical Offices in the Mancomunidades. It is necessary to highlight that in some cases responses from municipal technical staff about the need for this type of equipment, is perceived as an opportunity for an indirect request. This was also discussed at the aforementioned meeting in May. Recommendations from the Evaluators: 1. Results Framework and PMP. To the extent feasible, FOPRIDEH should revise its PMP and redefine its results and indicators. USAID and FOPRIDEH should consider establishing more realistic targets for program indicators. Sub-results (particularly Sub-results 1 and 2) should also be renamed to refer to advocacy, municipal financial autonomy, and civil society to improve clarity. While creating new outcome indicators might be impractical at this point, USAID/Honduras should ensure that follow-on programs have a clearly defined results framework, with distinct outcome indicators for each result that are direct, measurable, timely, practical, and attributable. DEE’s Response a) DEE does not agree with redefining results and indicators after more than three and a half years of implementation that might require making substantial changes to the Agreement and the PMP, which is not convenient for the Agency or FOPRIDEH/DEE. b) It is necessary to think about a change of direction for the Project, meaning that changes in the PMP are justified, especially for Indicator 1.2 from Result 1 and Indicator 2.2 from result 2. These will not be style changes but definition ones, which imply changing the target table in the PMP. c) In terms of adjustments, it is important to highlight the inclusion of a table for each indicator to assist the final calculation of achievement. On the other hand, a revised design of the PMP entails modifications to the definition of Indicator 1.2, built upon advocacy actions both at local and national levels to promote decentralization and municipal autonomy; which Includes topics such as the Municipalities Law, Decentralization Law, Manuals and Guidelines for the Administrative Municipal Career Law and the National Procurement Law. For Indicator 2.2, targets are set on the basis of technical assistance and advocacy actions for municipalities to comply with quarterly accountability reports, the preparation and approval of projects reports, and the approval of legal constraint processes. 2. Advocacy. USAID should convene the Foro Nacional de Descentralización to discuss the status of the different decentralization laws before the National Congress. The donor community should also encourage a dialogue between the new government authorities and supporters of decentralization laws and come to a new agreement on a package of legislation that can be passed successfully by Congress. This dialogue could be moderated by neutral international experts who could help achieve consensus between opposing parties to arrive at a final version of the decentralization law that can be voted on. DEE’s Response FOPRIDEH does not have the mandate to convene the National Decentralization Forum, as established in the operating guidelines agreed by its members (AMHON, FOPRIDEH, Civil Society Group, and GOH, among others). It is the Ministry of Human Rights, Justice, Decentralization and Governance the responsible party for this task. At the moment when the evaluation took place, newly appointed authorities were in place and were analyzing the decentralization process in Honduras. 3. Robust Public Awareness Campaign. DEE should greatly expand its public awareness efforts and a robust public education campaign should be established as part of follow-on programs. The Honduran public needs to become better informed about decentralization: what it is, its benefits, and citizens’ roles and responsibilities. Misconceptions and ignorance about decentralization not only stymie program efforts, but widespread popular support for decentralization laws in the legislature could foment political will and move the process forward. To reach out and engage a wider audience, FOPRIDEH should improving the definition of their key messages and their target audiences and consider social media as a possible medium. DEE’s Response During 2012 and 2013, FOPRIDEH/DEE, launched a dissemination campaign about the process through national and local radio stations. During the last quarter of 2013, the campaign was extended to local television channels. The latter was obtained free of charge. During the third year of implementation, FOPRIDEH/DEE funded workshops and sessions on this issue, under the expectation that laws would be approved that same year, with the support of AHMON and relevant Ministries. This was not the case, even with the advocacy efforts from all sectors. At the moment, the project does not count with financial resources for these types of campaigns, but is able to obtain some openings with radio and television media. 4. Financial Autonomy. USAID/DEE’s model of working through the Mancomunidades is good and should be continued in follow-on programming to promote greater sustainability of project efforts. The project, however, should consider working with Mancomunidades to design a financial sustainability handover plan, such as charging a fee for services, to allow technical officials to continue providing training and technical assistance to municipal officials. In addition, USAID/Honduras should incorporate incentives to municipal authorities to improve their own-source revenue, such as conditioning further assistance on the achievement of particular benchmarks, in future assistance. DEE’s Response This point was also discussed during the May 15th meeting and was not included in the Evaluation report. Sustainability Strategy is based on the ownership of the Municipal Association Finance Unit (UFIM) of all the processes promoted by result 2. In line with this, technical staff from UFIMs has benefited from technical assistance and training, in order for them to serve in the future as local technical assistance. DEE influenced 6 to 8 municipalities to contract permanent UFIM technical staff. Additionally, DEE has been promoting the contracting of a lawyer specialized in fiscal judicial constraint for MANSUCOPA. These actions contradict the conclusion from the evaluators that there is no sustainability strategy related to financial autonomy. 5. Civil Society. Where possible, USAID/DEE should follow up on its trainings and technical assistance programs offered to civil society organizations to ensure that agreements reached and operational plans designed in these sessions are implemented. Moreover, subsequent programming should tailor capacity building efforts to beneficiaries’ particular needs and feature participatory follow-on methods, such as tailored mentoring and coaching approaches, which will be more conducive to sustainability and partner ownership. Furthermore, civil society is weak and frequently subject to the [inclination] of the presiding mayor, frustrating project efforts. FOPRIDEH should go beyond established local civil society organizations, like municipal transparency committees, to identify potential community leaders, such as patronatos, who could be more independent from government authorities. Future support could also create and strengthen the advocacy and capacity of a national-level mayors’ association to build support for decentralization. Since decentralization efforts are wholly dependent upon active citizen participation, future programming in this area should strengthen the capacity of civil society organizations. As civil society in Honduras remains underdeveloped and as local resources remain constrained, public pressure on the central government to legislate change is marginal at best. Legislative and executive buy-in and cooperation will be necessary to pass a law on decentralization and civil society organizations must develop the capacity aggregate and vocalize demands. Public education (as described above) and civil society capacity-building initiatives are, therefore, key inputs for successful decentralization. Comments from DEE: a) On the issue of civil society participation, the evaluation team listed the events organized by DEE , but reports not being able to find much evidence that FOPRIDEH provided a follow up to ensure that organizations put into practice the learned skills. Our opinion does not respond to the characteristics of any particular event. For example, on the issue of decentralization, events focused on the socialization of the Draft Laws of Municipalities and Decentralization as the first stage of the process, to be then continued by a second stage in which training on each draft law would be provided. Unfortunately the process was delayed for political reasons. b) Public policy workshops were designed to last for two or more modules; the first to conceptualize and inform about the design process, and the second to collectively develop with participants a local public policy and an action plan to implement in municipalities where CSOs and municipal governments establish coordination schemes. c) Implementation for the second and third phases for key issues was scheduled for 2014 and 2015. For example, trainings on impact on public policy, conflict resolution, and negotiation processes under Result 2 resulted from budgetary adjustments in the beginning of 2014. d) The advocacy event organized by the special committees from FOPRIDEH and AMHON aimed at promoting the approval of the Decentralization Law, Municipal Law, CAM Law and reforms for environmental laws. This was part of a process in which activities were properly organized and monitored by the coordination staff of Result 1. e) We share the idea that civil society is weak and displays difficulties to position itself as an actor with strong and compelling proposals. Furthermore, we perceive a highly politicized and ideologically driven CS and consider that building a cohesive, coherent and concerted leadership with shared positions a very difficult process. f) It is gratifying that the beneficiaries interviewed have expressed satisfaction with the work done by the Federation. g) Even with the few resources available to FOPRIDEH / DEE, we continue to incorporate our affiliates working on local development, in the socialization of draft laws currently being negotiated and discussed with the national government. It should be noted that the project has been actively participating in discussions organized by the government through the SDHGD, to achieve consensus for a new decentralization national project. 6. Strengthened Coordination. In order to avoid duplicating efforts, leverage limited resources, and promote institutional synergy, FOPRIDEH should with other programs, including other USAID programs. USAID/DEE should collaborate closer with USAID/NEXOS to improve and standardize technical assistance packages provided to technical municipal officials and civil society organizations. Both programs work municipal finance and should learn from one another and identify exchange and best practices. USAID/DEE should also coordinate with USAID/IMPACTOS on the design of social auditing strategies and tools. One particular area where additional coordination is necessary is advocacy. A broader constituency for decentralization is needed to effectively overcome lagging political will. By speaking with one voice, organizations would be better able to exert greater influence on the National Congress and advance decentralization. Specifically, the program would have benefitted from a large-scale and sustained public relations campaign throughout the course of the program to promote the idea of local as a public good, to help mobilize broad-based public opinion and activism around media reform, and to further influence public policy agenda-setting on this issue. By working with other stakeholders, FOPRIDEH would have been able to undertake more strategic and effective advocacy initiatives. DEE’s Response a) From our point of view there is no duplication of efforts since the areas covered by FOPRIDEH/DEE and USAID/NEXOS are completely different. There is not one single municipality or mancomunidad shared by both projects. Overlapping and waste of resources take place when two projects work in the same municipalities and mancomunidades. We do have to acknowledge that we should standardize assistance packages on the issue of municipal finances or at least share what each one is doing and if possible adapt methodologies, but this is to be discussed. b) On the other hand, our project does not work on social auditing topics; therefore we do not understand why we should coordinate with IMPACTOS. c) Advocacy. We disagree with the evaluators on this regard. We have done what we needed to do: advocacy for policy reform, radio and television campaigns, workshops. However, laws have not been approved due to strange and unseen forces which we are unaware of.